The Two Sides of Writing Testable Code

There are precisely two things that make functions fully testable: the isolation of effects and the surfacing of ‘co-effects’. We will explore a bit of the formal theory behind these two sides, and show how they lead to code that can be easily tested. We will also show how we do this at Kickstarter by diving into our recently open sourced codebase.

---

Introduction

My name is Brandon Williams, I’m a developer at Kickstarter. Today I’m going to be talking about testing. We’re going to talk about why we want to test and what it takes to write testable code and how to make code more testable.

We recently open sourced all our code at Kickstarter, and the things I’m talking about today are things that we do at the company.

Why Test Code

I’ll say why I test code. It has completely transformed the way I write code, and I only write code if it can be tested. It’s a great boundary to force myself to write code in a functional style if it can be tested. And I’ve slowly started to think of my test code as the actual code that I care about, that the important work I do, not the implementation.

Testing is where I’ve truly documented how I expect the application to work, and where I’ve thought through edge cases and subtleties. I don’t have a good pitch for why you should test, but I think it’s something you have to experience.

A Case Study

/**
* Reads a number from a file on disk, performs a computation,
prints the result to the console, and returns the result.
*/
func compute(file: String) -> Int {

}

We’re going to look at a seemingly simple function and see what makes it difficult to test. First, just the signature and the description of the fuction.

The function is called ‘compute’. It takes a string and it returns an integer. It reads a file from the disk, tries to inrepret the contents of that file as a number, performs some computation on that number, and prints the result to the console. It’s important to note that we are considering the fact that it prints something to the console as a requirement of this function and not just something we’re doing for debugging.

/**
 * Reads a number from a file on disk, performs a computation,
 prints the result to the console, and returns the result.
 */
func compute(file: String) -> Int {
 let value = Bundle.main.path(forResource: file, ofType: nil)
 .flatMap { try? String(contentsOfFile: $0) }
 .flatMap { Int($0) }
 ?? 0
 let result = value * value
 print("Computed: \(result)")
 return result
}

One way of doing so is to use the bundle to find the path to the file, then we initialize the string with the contents of the file and convert that string to an integer. And if any of that fails, we coalesce it to zero, and say we have zero. What we’re doing is we square the integer, value times value. Then we print a message to the console, and return the result.

/**
 * Reads a number from a file on disk, performs a computation,
 prints the result to the console, and returns the result.
 */
func compute(file: String) -> Int {
 let value = Bundle.main.path(forResource: file, ofType: nil) // "/var/.../number.txt"
 .flatMap { try? String(contentsOfFile: $0) } // "123"
 .flatMap { Int($0) } // 123
 ?? 0 // 123
 let result = value * value // 15129
 print("Computed: \(result)") // "Computed: 15129\n"
 return result // 15129
}
compute(file: "number.txt") // 15129

We plugged in number.txt into the function. It finds the path to the file, which is some path on your computer. It reads the string, one, two, three from the file and converts the string to one two three, squares the number, prints the message, and then returns one five one two nine. This function isn’t complicated, and we’ve probably all written code like this before, but maybe you didn’t read from the disk. Maybe you read from a cache or user defaults, or you take it from a singleton or something, but it has some pretty serious short coming when it comes to testing it.

Dig Deeper

The function needs some input in order to do its job. It needs a string that is the name of the file which it’s suppose to read. We’re going to denote this by an arrow going into the function, and to denote the output, we’ll put an arrow leaving the function.

But, in order for this function to do its job, it needs more than just the input number .tx. It needs to have access to a global function, bundle.main.path for resource in order to figure out where the path was on the disk. From that function, you have this other string, called the var folders, and we have no idea how that function works.

Global Function

We have another global function that strings contents of the file, and reads the contents of the file from your disk. Its output depends on the current state of your entire hard drive in order to do its job. So sometimes it can read the file, and sometimes it cannot. We denote these two global things as arrows point down into this function. We do this because global things are like hidden inputs to the function that you didn’t realize the function depended on.

Two Sides of Testing

There are two sides of testing. You have inputs, both explicit and implicit, and you have outputs, both explicit and implicit. It’s crucial to understand these two sides to know how to write testable code.

For output side, it’s easier to understand and it’s more familiar to us. The thing that makes testing output difficult is side effects.

Side Effect

An expression is said to have a side effect if its execution makes an observable change to the outside world. How do you test code with side effects? To do so, you must execute the function, and make an assertion on the state of the world after you execute it.

You can confirm that the side effects you wanted to happen did happen, but you can never be sure that there wasn’t additional side effects that accidentally happen.

func compute(file: String) -> (Int, String) {
 let value = Bundle.main.path(forResource: file, ofType: nil)
 .flatMap { try? String(contentsOfFile: $0) }
 .flatMap { Int($0) }
 ?? 0
 let result = value * value
 return (result, "Computed: \(result)")
}

A better way to handle side effects is to describe effects as much as possible without actually performing the effects. In other words, treat the effects as data, and then you can make assertions on them as you would the output of a function. And this is a very deep topic of managing effects.

The idea is that you push doing the side effects to the boundary of your code. You continue building that data that describe these effects. At the last moment, you have a naive interpreter that can just simply perform the side effects in a very naive way. It’s just told what to do and it does it.

Getting Rid of Side Effect

We can go back to our compute function and get rid of the side effect and print statement. We’re no longer going to log to the console and instead, we change the return value, and now returns a tuple of an int and a string. The string is the message that should be logged to the console.

Now we’re describing what we want done, but we’re not actually doing it. Change in return value means now this function will not compose with any other functions that just simply took an integer as an argument, but this is actually a good thing.

Breaking that composition means we now have to think about how the side effect of logging the console message propagates through a chain of data transformation. If you force yourself to not do side effects, and you did a deep study on how these compositions break and how to fix them, you’d inevitably lead to rediscovering monads.

Testing Input

Testing input means being able to fully control and sculpt the data and the state of the world at the moment you execute a function in order for the function to do its job. What makes testing inputs difficult is something we’re gonna call a co-effect. These are all the globals that the function had to reach out to acces in order to do its job. They’re the things that you have no control over when the function is executed.

We don’t get to specify what globals, the function just did it. There’s a duality here between testing outputs means testing effects, and testing inputs means testing co-effects. We’re going to try to define coeffects like we did for side effects.

Here’s an analogy: If an effect is a change to the outside world after executing an expression, then a co-effect is the state of the world that the expression needs in order to execute. Some call this dependency injection. I could loosely define a co-effect as an expression is said to have a co-effect if it requires a particular state of the world in order to execute.

Code Testing With Co-effects

How do you test code with co-effect because it’s very difficult. You need some way to stub out the entire world, all global values, so you can control what the function sees when it’s executed.

What you can do is you could just embrace all your globals and bundle them up into a big struct and you forbid accessing any globals. We can just call it an environment and we could start adding globals to it.

struct Environment {
 let apiService: ServiceProtocol
 let cookieStorage: HTTPCookieStorageProtocol
}

In production, it would just use a regular date. But in test, you get to control exactly how date behaves. The language of the app of course is a huge co-effect at some global setting on the device. This has been useful for us because it allows us to write test in any language that we support, and we can just loop over English, French, German, Spanish, and just generate screenshots for every single language.

We saw bundle as a co-effect. Reachability is an interesting thing because it’s like global signal that says whether you have Wi-Fi, or cell network, or no cell network, you had probably want that. We use ReactorSwift quite a bit. In ReactorSwift you use schedulers in order to control time, any time you need to do something with time, like delaying a signal, debouncing, things like that. This is a co-effect, and we wanna be able to control that.

struct Environment {
 let apiService: ServiceProtocol
 let cookieStorage: HTTPCookieStorageProtocol
 let currentUser: User?
 let dateType: DateProtocol.Type
 let language: Language
 let mainBundle: BundleProtocol
 let reachability: SignalProducer<Reachability, NoError>
}

In our test, we get to write test that have very subtle ways of checking the way time interleaves with different actions. User defautls is a coeffect that we reach out for everytime.

In our app, we ahve about 24 or maybe more coeffects at this point.

Finding the Path

All we care about is that it either succeeded or failed in finding the path. Those are the two cases you wanna test. To allow testing those cases, we need to put a protocol in front of this function so we can substitute in different things that may fail or succeed. You put a protocol called BundleProtocol. Bundle naturally conforms to this protocol. And then you can have two versions, one that succeeds, one that fails but it doesn’t matter what the path is.

struct Environment {
 let apiService: ServiceProtocol
 let cookieStorage: HTTPCookieStorageProtocol
 let currentUser: User?
 let dateType: DateProtocol.Type
 let language: Language
 let mainBundle: BundleProtocol
 let reachability: SignalProducer<Reachability, NoError>
 let scheduler: DateSchedulerProtocol
 let userDefaults: UserDefaultsProtocol
}

We don’t know how bundle finds paths, so we can just return anything. This is also a co-effect. We don’t really care what it finds, we just care if it succeeded or failed. So, here we could create a protocol, make string conform to it, and then force ourselves to use this any time we wanna load a string from a file.

Bundle.main.path(forResource: file, ofType: nil)

protocol BundleProtocol {
 func path(forResource name: String?, ofType ext: String?) -> String?
}
extension Bundle: BundleProtocol {}

struct SuccessfulPathForResourceBundle: BundleProtocol {
 func path(forResource name: String?, ofType ext: String?) -> String? {
 return "a/path/to/a/file.txt"
 }
}
struct FailedPathForResourceBundle: BundleProtocol {
 func path(forResource name: String?, ofType ext: String?) -> String? {
 return nil
 }
}

String(contentsOfFile: file)

protocol ContentsOfFileProtocol {
 static func from(contentsOfFile file: String) throws -> String
}
extension String: ContentsOfFileProtocol {
 static func from(contentsOfFile file: String) throws -> String {
 return try String(contentsOfFile: file)
 }
}

struct IntContentsOfFile: ContentsOfFileProtocol {
 static func from(contentsOfFile file: String) throws -> String {
 return "123"
 }
}
struct NonIntContentsOfFile: ContentsOfFileProtocol {
 static func from(contentsOfFile file: String) throws -> String {
 return "asdf"
 }
}
struct ThrowingContentsOfFile: ContentsOfFileProtocol {
 static func from(contentsOfFile file: String) throws -> String {
 throw SomeError()
 }
}

func compute(file: String,
  bundle: BundleProtocol = Bundle.main,
  contentsOfFileProtocol: ContentsOfFileProtocol.Type = String.self) -> (Int, String) {
let value = bundle.path(forResource: file, ofType: nil)
  .flatMap { try? contentsOfFileProtocol.from(contentsOfFile: $0) }
  .flatMap { Int($0) }
  ?? 0
let result = value * value
return (result, "Computed: \(result)")
}

Conclusion

The two things that make testing difficult are effects and co-effects. To obtain effects, you think of them as data in their own right, and you just describe the effect and don’t perform it. And then a naive interpreter can perform the effect somewhere else.

To obtain coeffects, put them in a big struct, and don’t ever access a global unless you go through that struct. Thanks.

Q&A

**_Q:Do you like to test drive, or do you more frequently test after?

Brandon: About half of what we do is test drive, and even beyond test driven. For the initial implementation, we do a lot of test driven like bug fixing, where a bug report is simply a test that fails. You should write the failing test, and then fix the code. So we do that far more often than from first principles test driven development, but we still do that quite a bit.

The app environment is very simple. We just use it straight up without thinking in terms of monads. And technically, the app environment is a comonad. So yeah, we just use it straight without really thinking much about it. And we do not do property based testing, although we really like that. But no, we don’t do that right now.