What's Under the Hood? Decoding JSON with Swift

As Swift’s statically-typed characteristics prove to complicate the decoding of serialized objects, there are other characteristics that serve as interesting alternatives, like currying. In this try! Swift NYC talk, Anat Gilboa goes through some of the functional aspects of Swift that make parsing JSON fun and exciting!

---

Introduction (0:00)

I’m going to be talking about JSON parsing. I was initially inspired to give this talk when one of my teammates proposed a solution for migrating our networking layer from Objective-C to Swift. His solution utilized two third-party libraries, Argo and Result, for the JSON decoding and respective error-handling. Since I didn’t have much experience with the JSON decoding world, I decided to check out some of the READMEs and look into the Xcode projects.

The Proposal (0:41)

Argo is a JSON decoding library that extracts models from JSON in a safe way. It uses Swift’s type system to validate the required data fields explicitly and reports them as successful or failed.

I wasn’t sure what currying had to do with the code, so I wanted to check that out. For anyone who’s not familiar with the term, “currying” is the act of giving a function fewer parameters than it takes and returning a function that takes the remaining parameters, so that our user object essentially is constructed like this:

struct User {
	let id: Int
	let name: String
	let age: Int
}

static func create(id: Int)(name: String)(age: Int) -> User {
	return User(id: id, name: name, age: age)
}

Note the parameters are separated with their respective parentheses.

Result is a microframework that allows you to wrap value and error into respective success and failure. In the example here, we’ve got a JSON error that is of ErrorType, it’s an enum, has two cases, and then we have a function that tries to extract a string from the given key in a JSON dictionary.

Result<Value, Error>

typealias JSONObject = [String:AnyObject]

enum JSONError : ErrorType {
	case NoSuchKey(String)
	case TypeMismatch
}

func stringForKey(json: JSONObject, key: String) -> Result<String, JSONError> {
	guard let value = json[key] else {
		return .Failure(.NoSuchKey(key))
	}
	if let value = value as? String {
		return .Success(value)
	}
	else {
		return .Failure(.TypeMismatch)
	}
}

Rather than returning any object, it returns a result that contains a string value for the given key and an error type detailing what went wrong. We can fail on unwrapping the initial with the guard let with no such key, successful unwrap with the value, or fail on a type mismatch. These just happen to be two of the errors that are indicated from the error type.

I had a few questions as to why we would do this instead of throwing, which was introduced in Swift 2.

Parsing JSON in Swift (2:45)

What’s the real issue with parsing JSON in Swift? Since Swift is a statically typed language, we can’t really throw objects into the typed variables and have the compiler trust that the type is what we claimed it to be like we did in Objective-C. Now the compiler is checking to make sure that we don’t cause any runtime errors, which is awesome, but that means that we have more work to do.

So say we’ve got this user model:

struct User {
	let id: Int
	let name: String
	let age: Int
}

Pre-Swift 2, we would have something that might look like this:

let user: User?
if let user = json["user"] as? [String: String] {
	if let id = user["id"] {
		if let name = user["name"] {
			if let age = user["age"] {
				user = User(id, name: name, age: age)
			}
		}
	}
}

If you look at the basic model object, you can see that we have a bunch of nested if lets create our user object. With more properties, this starts to get pretty cumbersome, and just verbose. And not as beautiful.

Also, what happens if we don’t get all the properties? Nothing. Maybe there’s a better way? With Swift 2 they introduced guard, and it looks a little bit better.

let user: User?
guard let user = json["user"] as? [String: String] {
	if let id = user["id"],
			name = user["name"],
			age = user["age"] {
				user = User(id, name: name, age: age)
			}
	} else {
	return nil
}

It’ll only complete if all of those are met, and create that user object. But again, we don’t have too much going on in the error-handling part.

What about do-catch? Well, we know that in a do-catch we can handle errors in that do block of code, and it’s matched in the catch against the clauses to determine if we’ve hit an error. This works for multiple tries in sequence, but you also have to make sure that the function in which you’re doing that do-catch is throwing.

Where does that lead us? I wanted to get an idea of how other people were going about solving the same problem that I was, and I decided to analyze some popular libraries that I found on GitHub. I wanted to take a look inside those projects and find what the underlying implementations were. I didn’t worry about the simplicity of the APIs, because those are just the external-facing things. How are they actually parsing the JSON?

The Subjects (5:02)

These were the subjects. I want to run through a couple of examples.

import SwiftyJSON

struct User {
	let id: Int
	let name: String
	let age: Int
}

let json = JSON(["id":2378, "name":"Jack", "age": 23])
if let name = json["name"].string {
	// Do a thing
} else {
	print(json["name"].error)
}

Here’s an example of decoding our object with SwiftyJSON. We if-let to get the name and extract a string with a .string syntax. It’s really easy. It’s beautiful. Within that block of code we could initialize the name property on the user object, or else we print an error.

public let ErrorUnsupportedType: Int = 999
public let ErrorIndexOutOfBounds: Int = 900
public let ErrorWrongType: Int = 901
public let ErrorNotExist: Int = 500
public let ErrorInvalidJSON: Int = 490

It happens to come with its own errors, defined at the top of the SwiftyJSON.swift file. It’s not really lending itself to more cases, but it really just says, type mismatch, wrong error, non-existing, or invalid JSON, depending on what the actual error is. I thought it was interesting that they didn’t have this as an enum. I’m not sure what the reason for that was, but they’re essentially just error codes.

import Mapper

struct User {
	...
}
extension User : Mappable {
	init(map: Mapper) throws {
		try id = map.from("id")
		try name = map.from("name")
		age = map.optionalFrom("age")
	}
}

Mapper is a JSON deserializing library created by the folks at Lyft. You have an initializer that throws for your object, and you’re set. Under the hood, it’s got an internal JSON from field function in Mapper.swift that essentially checks the value to see if the field is empty and sets it, otherwise throws a mapper error if you have a missing field.

public enum MapperError: ErrorType {
	case ConvertibleError(value: AnyObject?, type: Any.Type)
	case CustomError(field: String?, message: String)
	case InvalidRawValueError(field: String, value: Any, type: Any.Type)
	case MissingFieldError(field: String)
	case TypeMismatchError(field: String, value: AnyObject, type: Any.Type)
}

It’s got an enum, one of the first for now. They declare these in the MapperError.swift file and I thought it was cool because you can throw and extract from the values with respect to each of these types of errors in the enum.

import Freddy

struct User {
	...
}
extension User: JSONDecodable {
	public init(json value: JSON) throws {
		id = try value.int("id")
		name = try value.string("name")
		age = try value.int("age")
	}
}

Freddy is also a really popular library. You make an extension on the object, conform to the JSON Decodable protocol, which creates an instance of the object from a JSON instance. It handles most of the primitive types and has extensions for all of those that implement on init from the JSON. That includes strings, ints, bools, you name it. And the throw error from the internal JSON init is a value, not convertible, which spits out a pretty descriptive message.

public enum Error: ErrorType {
	case IndexOutOfBounds(index: Swift.Int)
	case KeyNotFound(key: Swift.String)
	case UnexpectedSubscript(type: JSONPathType.Type)
	case ValueNotConvertible(value: JSON, to: Any.Type)
}

So again, error types in enum. They suggest doing this in a do-catch on those errors, so not exactly in the init of the model. If you’re not doing it from the init of the model here, as we have on the extension of the init.

Then there’s another library, Decodable.

import Decodable

struct User {
	...
}
extension User: Decodable {
	static func decode(j: Any) throws -> User {
		return try User(
			id: j => "id",
			name: j => "name",
			age: j => "age"
		)
	}
}

You implement a Decodable protocol, decodes, make an extension on the user object again so that we have a protocol that’s defined in Decodable.swift, and has the underlying decode extension on the primitive types, just like we saw before.

public enum DecodingError: ErrorProtocol, Equatable {
	case typeMismatch(expected: Any.Type, actual: Any.Type, Metadata)
	case missingKey(String, Metadata)
	case rawRepresentableInitializationError(rawValue: Any, Metadata)
	case other(ErrorProtocol, Metadata)
}

They have this neat thing for the extractors: what you can expect and what will throw. This was in the README. I thought it was pretty interesting to see how you would implement it with the respective enums that are in the type.

The Findings(9:00)

I want to go through small subset of this list of findings. I hope you noticed the differences and similarities between those libraries.

Generally, we see a protocol that throws. It would be on the initializer of an instance and for a given JSON object, and it would throw some kind of error derived from the JSON value. It could be under the name decodable syllable, JSON decodable, JSON convertible, you name it.

Another interesting thing is the different ways that people map on the properties of an object. How would we actually decode these values? Some would override the custom operator. They would overload an operator. Others would simply create a subscript for the type, just like SwiftyJSON. I found that the libraries that would overload these operators would extract properties over the mapped object. While they might look overwhelming to begin with, I found that in the documentation they were pretty descriptive on what they were outlining and what they were doing for optionals, non-optionals, arrays, and dictionaries. I saw a lot of operators in these libraries; it was crazy. I don’t overload operators that often in my day-to-day job, so it was interesting to see what people came up with.

Subscripting. When I initially learned about JSON I thought it looked so much like nested items in a dictionary that I wanted to index into the object just to extract the value. It seemed so intuitive. But when I started decoding JSON, you can’t really do that, and hence SwiftyJSON makes it seamless in doing that, which I thought was really neat. They support custom subscripting in their underlying implementation.

Error handling. In all cases, you’ll want to handle the results somehow. One way is to implement the result protocol and wrap error types thrown around with success or failure. Other libraries would recommend trying to unwrap in a do and catching on the custom errors thrown by case-to-case. It helps that most of the libraries would have these custom errors for each of the types, like we saw, and would implement their own error-handling with that. But if you want to add some safety, you can wrap that error as defined by error type in that enum with another failure or success. Then you’d see the respective error types in a couple of those libraries that we saw.

Conclusion (12:03)

All in all, I liked a lot of the libraries that I saw, but our team had specific criteria that helped us choose a solution.

One of the main differences has to do with type-checking and respective error handing. No matter what you choose, you’ll want a solution that checks to make sure it has a type-safe solution that the compiler will help you work with receiving the JSON in a way that prevents runtime errors. You might want to also add declarative error-handling on top of that, just like we were doing with the result wrapping the failure cases for success and failure.

You want it to be generic-friendly. I saw a lot of the libraries would, for the primitive types, decode on ints and bools and strings, similar to having a generic function. Others would have their own ints, extensions or inits, which I also thought was interesting.

If you’re looking for a declarative type-safe way to decode JSON, using Argo and Result might be a good choice for you. Here’s an example of using Argo.

import Argo
import Curry
import Result

struct User {
	let id: Int
	let name: String
	let age: Int?
}
extension User: Decodable {
	static func decode(j: JSON) -> Decoded<User> {
		return curry(User.init)
			<^> j <| "id"
			<*> j <| "name"
			<*> j <|? "age"
	}
}

We’ve got a User object conforming to Decodable protocol that decodes. You pass it int a JSON and you curry. Again, it’s not the same currying we saw from earlier; that’s going to be deprecated in Swift 3. Make sure the underlying implementation of this currying from Argo is not that.

You can extract the id, name, and age, and it will fail if you don’t get those properties and it will map to the int, string and int of the respective types.

Thanks to the community members who helped forge this topic. It wasn’t something I was too familiar with, so there were a lot of questions that I had. I want to thank my team members for letting me bounce off questions to them and also for approving my pull requests. And thank you to all of you who have contributed to these JSON decoding libraries.