14 KiB
+++ categories = ["kotlin", "android"] date = 2020-12-13 description = "Moshi is a fast and powerful JSON parsing library for the JVM and Android. Today we look into manually parsing JSON to and from Java/Kotlin classes" draft = true slug = "manually-parsing-json-with-moshi" tags = ["moshi", "json parsing", "android", "kotlin"] title = "Manually parsing JSON with Moshi" +++
What is Moshi?
Moshi is a fast and powerful JSON parsing library for the JVM and Android, built by the former creators of Google's Gson to address some of its shortcomings and to have an alternative that was actively maintained.
Unlike Gson, Moshi has excellent Kotlin support and supports both reflection based parsing and a kapt-backed codegen backend that eliminates the runtime performance cost in favor of generating adapters during build time. The kotlin-reflect dependency required for doing reflection-based parsing can add up to 1.8 mB to the final binary, so it's recommended to use the codegen method if possible.
What is an adapter?
An adapter is Moshi-speak for a class that, given an object of type T, contains methods of the signatures @FromJson fun fromJson(reader: JsonReader): T? and @ToJson fun toJson(writer: JsonWriter, value: T?). fromJson is responsible for taking a JSON document and constructing an instance of our type, and toJson is given a writer that can create JSON and value, the object that needs to be converted to JSON.
The actual method names can be anything, the to and from prefix just make it easier to follow.
Why write your own adapters?
Good question. Consider this example class:
@JsonClass(generateAdapter = true)
class TextParts(val heading: String, val body: String? = null)
Pretty straightforward. The JsonClass annotation with generateAdapter = true will attempt to use the codegen backend to write an adapter automatically for this. Let's try converting this to JSON.
val text = TextParts("This is the heading", "And this is the body")
val moshi = Moshi.Builder().build()
// TextPartsJsonAdapter was generated by the codegen backend
println(TextPartsJsonAdapter(moshi).toJson(text))
{"heading":"This is the heading","body":"And this is the body"}
What this means is, given a JSON object that looks like this
{"heading":"This is the heading","body":"And this is the body"}
We can get an instance of TextParts that looks like this
val text = TextParts("This is the heading", "And this is the body")
Cool! Now, let's make things unfortunate. Imagine your backend team is stretched thin, and due to a limitation with how they initially built their database schema, you can only get the above JSON in this form
{"heading":"This is the heading","extras":{"body":"And this is the body"}}
If you try to parse this with the old TextPartsJsonAdapter, your app is going to crash, because the JSON and its Kotlin representation have diverged. The equivalent Kotlin for this new JSON is going to be something like this:
@JsonClass(generateAdapter = true)
class Extras(val body: String? = null)
@JsonClass(generateAdapter = true)
class TextParts(val heading: String, val extras: Extras? = null)
Many things changed here. Your direct access to the body field now needs to go through extras, which just isn't that nice. You're also now incurring the (albeit miniscule) overhead of generating two adapters rather than one. Wouldn't it be great if we could continue to have a flat object like before? Let's try to make that happen.
How to write your own Moshi adapter?
With less effort than one might think! Let's put down the basic building blocks.
class TextPartsJsonAdapter {
@FromJson
fun fromJson(reader: JsonReader): TextParts? {
TODO("Not implemented")
}
@ToJson
fun toJson(writer: JsonWriter: value: TextParts?) {
TODO("Not implemented")
}
}
Now we're ready to start parsing. First, let's implement the toJson part, where we take an instance of the object and then try to write the equivalent JSON for it.
@ToJson
fun toJson(writer: JsonWriter: value: TextParts?) {
// Null values shouldn't arrive to the adapter, this error lets callers know
// what builder options need to be passed to the Moshi.Builder() instance
// to avoid this particular situation.
if (value == null) {
throw NullPointerException("value was null! Wrap in .nullSafe() to write nullable values.")
}
// Use the Kotlin `with` scoping method so we don't need to call
// all methods with the `writer.` prefix.
with(writer) {
// Start the JSON object.
beginObject()
// Since our `extras` field is nullable, and our backend will send
// it as a literal null rather than skip it, we want null values to
// be written into the final JSON.
serializeNulls = true
// Create a JSON field with the name 'heading'
name("heading")
// Set the value of the 'heading' field to the actual heading
value(value.heading)
// Create the 'extras' field
name("extras")
if (value.body != null) {
// If the body text exists, then start a new object and add a
// body field
beginObject()
name("body")
value(value.bodyText)
endObject()
} else {
// Otherwise we put down a literal null
nullValue()
}
// End the top-level object.
endObject()
}
}
Parsing JSON manually is relatively easy to screw up and Moshi will let you know if you get nesting wrong (missed a closing endObject() or endArray()) and other easily detectable problems, but you should definitely have tests for all possible cases. I'll let the readers do that on their own, but if you really need to see an example then scream at me on Twitter and I'll do something about it.
Anyways, that's the object -> JSON part sorted. Now let's try to do the reverse. Here's where we are as of now.
fun fromJson(reader: JsonReader): TextParts? {
TODO("Not implemented")
}
Same as writing JSON, we need to start by making an object.
fun fromJson(reader: JsonReader): TextParts? {
+ // We'll be constructing the object at the end so these
+ // will store the values we read.
+ var heading: String? = null
+ var body: String? = null
+ with(reader) {
+ beginObject()
+ endObject()
+ }
TODO("Not implemented")
}
We have a fixed set of keys that we expect to read, so go ahead and configure a couple instances of JsonReader.Options that we will use to find them in this JSON.
+val topLevelKeys = JsonReader.Options.of("heading", "extras")
+val extrasKeys = JsonReader.Options.of("body")
+
fun fromJson(reader: JsonReader): TextParts? {
// We'll be constructing the object at the end so these
// will store the values we read.
And we're set. You'll see the significance of the Options objects now.
var body: String? = null
with(reader) {
beginObject()
+ while(hasNext()) {
+ when(selectName(topLevelKeys)) {
+ 0 -> heading = readString() ?: throw Util.unexpectedNull(
+ "heading",
+ "text",
+ this
+ )
+ }
+ }
endObject()
}
TODO("Not implemented")
reader.hasNext() is going to continue iterating through the document's tokens until it's completed, which lets us look through the entire document for the parts we need. The selectName(JsonReader.Options) method will return the index of a matched key, so 0 there means that the heading key was found. In response to that, we want to read it as a string and throw if it is null (since it's non-nullable in TextParts). The Util.unexpectedNull method is a little nicety that is part of Moshi's internals and is used by its codegen'd adapters to provide better error messages and we're going to do the same.
"text",
this
)
+ -1 -> {
+ // Skip unknown values
+ reader.skipName()
+ reader.skipValue()
+ }
}
}
endObject()
When I said that selectName returns the index of the matched key, I didn't mention that it returns -1 when it comes across a key that isn't in the Options object. Since we don't care about them, we're going to skip both their name and value and continue right on ahead. Now, we're going to try and parse that inner extras object. A lot is about to happen quickly, but bear with me as I explain things.
"text",
this
)
+ 1 -> {
+ // "extras" is nullable, so we first try to see if it is null.
+ // If it isn't, this will throw and we can then safely assume
+ // a non-null value and proceed.
+ try {
+ reader.nextNull<Any>()
+ } catch (_: JsonDataException) {
+ reader.beginObject()
+ while (reader.hasNext()) {
+ when (reader.selectName(extrasKeys)) {
+ 0 -> body = reader.nextString()
+ -1 -> {
+ // Skip unknown values
+ reader.skipName()
+ reader.skipValue()
+ }
+ }
+ }
+ reader.endObject()
+ }
+ }
-1 -> {
// Skip unknown values
reader.skipName()
reader.skipValue()
Now that you look at it, not really that different from what we did above. The only new thing here is the nextNull method, which simply tries to find a null value and throws the JsonDataException if the value wasn't null.
}
endObject()
}
- TODO("Not implemented")
+ // Satisfy the typechecker and throw in case the JSON body didn't contain the 'heading' field at all
+ require(heading != null) { "heading must not be null" }
+ return TextParts(heading, body)
}
And that's it! The final adapter is going to look like this
class TextPartsJsonAdapter {
val topLevelKeys = JsonReader.Options.of("heading", "extras")
val extrasKeys = JsonReader.Options.of("body")
@FromJson
fun fromJson(reader: JsonReader): TextParts? {
// We'll be constructing the object at the end so these
// will store the values we read.
var heading: String? = null
var body: String? = null
with(reader) {
beginObject()
while(hasNext()) {
when(selectName(topLevelKeys)) {
0 -> heading = readString() ?: throw Util.unexpectedNull(
"heading",
"text",
this
)
1 -> {
// "extras" is nullable, so we first try to see if it is null.
// If it isn't, this will throw and we can then safely assume
// a non-null value and proceed.
try {
reader.nextNull<Any>()
} catch (_: JsonDataException) {
reader.beginObject()
while (reader.hasNext()) {
when (reader.selectName(extrasKeys)) {
0 -> body = reader.nextString()
else -> {
// Skip unknown
reader.skipName()
reader.skipValue()
}
}
}
reader.endObject()
}
}
-1 -> {
reader.skipName()
reader.skipValue()
}
}
}
endObject()
}
// Satisfy the typechecker and throw in case the JSON body didn't contain the 'heading' field at all
require(heading != null) { "heading must not be null" }
return TextParts(heading, body)
}
@ToJson
fun toJson(writer: JsonWriter: value: TextParts?) {
// Null values shouldn't arrive to the adapter, this error lets callers know
// what builder options need to be passed to the Moshi.Builder() instance
// to avoid this particular situation.
if (value == null) {
throw NullPointerException("value was null! Wrap in .nullSafe() to write nullable values.")
}
// Use the Kotlin `with` scoping method so we don't need to call
// all methods with the `writer.` prefix.
with(writer) {
// Start the JSON object.
beginObject()
// Since our `extras` field is nullable, and our backend will send
// it as a literal null rather than skip it, we want null values to
// be written into the final JSON.
serializeNulls = true
// Create a JSON field with the name 'heading'
name("heading")
// Set the value of the 'heading' field to the actual heading
value(value.heading)
// Create the 'extras' field
name("extras")
if (value.body != null) {
// If the body text exists, then start a new object and add a
// body field
beginObject()
name("body")
value(value.bodyText)
endObject()
} else {
// Otherwise we put down a literal null
nullValue()
}
// End the top-level object.
endObject()
}
}
}
This is certainly a lengthy job to do, and this blog post is a result of nearly 8 hours I spent writing JSON adapters by hand. Certainly not recommended if avoidable, but sometimes you just need to. When it comes to it, now you hopefully know how :)