Reflections on Reflection — Performance impact for a JSON parser on Android

Lessons on how to evaluate the viability of a software tool

“monkey looking at mirror” by Andre Mouton on Unsplash

As a Java developer, when suggesting a reflection-based library to use in code, there’s a frequent criticism which comes up.

Reflection is Slow

Is it though?

In the early years of Java, it was true that using runtime reflection could often be prohibitively slow. This performance hiccup has been dramatically reduced in recent versions of Android which use a JIT, but it remains valid to a certain extent. However, the performance of reflection itself is not a sufficient factor to judge the performance of a reflection-based tool. Such a tool is very much capable of producing better results than one which doesn’t use reflection, due to other optimizations.

Let’s look at the example of one of the most common use-cases of reflection on Android, JSON parsing and serialization. The standard exchange format with servers generally being JSON, a plethora of JSON parsers and serializers are available in the Java library ecosystem. For the sake of simplicity, the following analysis is limited to parsing, but the same logic can generally be applied to serialization.

Note that this article is not meant an extensive overview of the technical capabilities of each of the JSON libraries presented (nor an assessment of JSON libraries in general). The examples are simple and don’t illustrate all the capabilities of each library.

JSON Libraries

When presenting the usage of JSON libraries, we’ll be using as an example the following JSON object, representing a user.

{  
   "id":1,
   "name":"John Doe",
   "age":21
}

The goal is to present different options relying on popular libraries, used to parse this JSON string to obtain an instance of the following User class which models an entity received from the server.

class User(
  val id: Long,
  val name: String,
  val age: Int
)

JSON.org

JSON.org is the official project which specifies the JSON standard. As such, it has produced a reference Java library to handle the JSON inter-exchange format. It has been included in the Android platform since the initial versions, and is therefore readily available to application developers.

Basic usage of the JSON.org library is fairly straightforward, limited to the JSONObject and JSONArray types. Parsing a JSON string is done by building a JSONObject then fetching its values, typically to build the data structure used by the application.

fun parseUserFromJson(string: String) = 
  JSONObject(string).run {
    User(
      getLong("user"),
      getString("name"),
      getInt("age")
    )
  }

Each class therefore requires its parser to be manually written. The logic is duplicated for the parsing and the serialization. This tool however provides complete flexibility over the parsing mechanism.

JSON.org builds an intermediate model of the JSON data, and programmers are responsible for manually iterating over these objects in order to build their application model from it.

Gson

Gson is a library created by Google to provide a simple way to build instances of custom classes from JSON using runtime reflection. It is a very mature tool, and remains popular in the Android developer community.

Although annotations on properties are not strictly required in order to use Gson, they’re generally preferable because they protect the code from accidental errors introduced by refactoring or obfuscation. The following example defines the user JSON model.

class User(
  @SerializedName("id")
  val id: Long,
  @SerializedName("name")
  val name: String,
  @SerializedName("age")
  val age: Int
)

Parsing a JSON string is then simply done by using a Gson instance to which the expected class is provided.

fun parseUserFromJson(string: String) = 
  Gson().fromJson<User>(string, User::class.java)

Declaring a class with annotations is therefore enough for it to be fully compatible with Gson. Additionally, Gson can be configured to handle some custom logic. It can however become difficult to manage poorly designed JSON schemas, in which case Gson shows its limitations. With the advent of Kotlin, Gson also reveals an important weakness in its handling (or rather non-handling) of nullable types.

Gson parsing doesn’t use an intermediate model. The JSON string is immediately turned into our application model using runtime reflection to map values into fields.

Immutables Type Adapters

Immutables project is not strictly aimed at JSON. Immutables is a compile-time code generation library, which creates immutable classes along with their builder classes, handling all the boilerplate code that this entails. However, the Json integration provided by Immutables adds support for many popular JSON libraries, including Gson, in order to generate custom JSON type adapter at compile-time. This allows JSON strings to be parsed without any runtime reflection.

The classes are declared similarly to when using Gson, except that they’re interfaces for which the implementation is generated.

@Gson.TypeAdapters
@Value.Immutable
interface User(
  @get:SerializedName("id")
  val id: Long,
  @get:SerializedName("name")
  val name: String,
  @get:SerializedName("age")
  val age: Int
)

The Gson instance used for parsing also needs a little bit of additional configuration in order to register the compile-time generated type adapter.

fun parseUserFromJson(string: String) = GsonBuilder()
  .registerTypeAdapterFactory(GsonAdaptersUser())
  .create()
  .fromJson(string, User::class.java)

This tool suffers from the same limitations as Gson in terms of flexibility and configuration when the JSON schema is poorly defined. In some ways, it’s even less flexible because the entire model class is automatically generated, therefore it’s harder to add logic to it. Immutables does, however, fix many of Gson’s issues with null-safety because it manages required attributes.

Like Gson, Immutables also doesn’t use an intermediate model. It uses compile-time reflection to build parsers which work as state machines when creating the application model from JSON.

Benchmarks

Now, let’s look at the performance of these libraries for parsing JSON.

These benchmarks do not pretend to be a perfectly accurate representations of runtime performance. They give us an idea of how the libraries compare to each other.

In order to try to get some perspective on the importance of JSON performance, the benchmarks also include the time needed to write the JSON string to a file.

Feel free to check out the code at https://github.com/MartinDevi/json-benchmarks

Protocol

The performance benchmarks for each JSON library is established using selected sets of data.

1. A flat list of JSON objects with a few attributes with different types. This test is run with a small data set (200 items), labelled as 1.1, and with a large data set (5000 items), labelled as 1.2.
2. A list of JSON objects whose attributes are lists of other JSON objects, with a few string attributes.
3. A list of JSON objects whose attributes are themselves nested JSON objects following the same schema. For the tree structure to terminate, the attributes need to be optional.
4. A list of JSON objects whose attributes are themselves nested lists of JSON objects with the same type. The tree structure terminates once an empty list is reached.
5. A list of complex JSON objects with nested JSON structures involving various types.

From this protocol, we can already expect data type 3 to be the most penalizing to runtime reflection-based tools like Gson, since it requires constant creation of new instances through reflection. The cost of calling constructors through reflection is generally high.

Results

• Motorola Nexus 6 API 23 (Marshmallow)

Nexus 6 API 23

Immutables.org is the fastest parser, as expected. Gson is always slower than JSON.org, but the difference varies depending on the test. In the worst-case scenario for Gson, it’s three times slower. In its best scenario, it’s barely any slower.

• Galaxy S8 API 24 (Nougat)

Galaxy S8 API 24

Immutables.org still yields the best performance for all tests. However, unlike in the previous benchmark, Gson generally has a better performance than JSON.org, the only exception being test 3, which was essentially designed to penalize runtime reflection.

Analysis

After running a few additional tests, I arrived at the conclusion that reflection performance radically changes when running on Android Nougat and above. This is likely due to the introduction of support for Java 8 in this version. I haven’t found any documentation to explain this more clearly. In any case, the first lesson of these tests is that the performance of reflection-based tools actually depends on the environment in which they’re running.

On Android API 24 and higher, Gson therefore generally performs better than JSON.org, despite using runtime reflection. JSON.org’s performance is penalized by the fact that it has to create an intermediate model, leading to more memory allocations, and also because of its internal model which produces a lot of boxing and unboxing operations. The cost of using reflection is compensated by the optimizations that it provides.

Understanding the cost of reflection

Developers shouldn’t judge the performance of a tool by a single criterion

Setting the value of a field through reflection may be slower than setting it directly, but a tool which uses the first solution might not be slower than one which doesn’t. The tool’s performance should be evaluated as a whole, not based on implementation details.

Make sure you understand the difference between runtime and compile-time reflection

Runtime reflection introduces a runtime performance penalty. Compile-time reflection increases the compilation time (which is generally unimportant due to caches) and the binary size (although minification tools can mitigate this considerably), but it is generally extremely fast. Their impact on the resulting application is therefore completely different.

Performance should always be tested and quantified

Don’t trust rumors. Some may be outdated, and some may be more religion than fact. If you have doubts about performance, write your own benchmark. Make sure you choose your data sets carefully, because as we’ve seen the performance differences may vary greatly in each case.

The performance should also be put in perspective with its use-case. In the benchmarks, we can see that the time required to parse JSON seems non-negligible compared with the time required to write it to a file. The performance of a JSON parser may therefore be important when using it for local data storage. When using it for network requests, however, it might not be as important, given that the true bottleneck is generally the HTTP request itself.

Runtime performance isn’t everything

As briefly explained above, the difference between the JSON tools presented doesn’t just lie in their performance. They provide different capabilities, with different possibilities for configuration.

The readability of reflection-based tools is probably their biggest asset. Having code which is easy to understand and maintain is invaluable.