Making Mock Objects More Useful

In Swift, we make mock objects by hand. Their design shapes the way we write unit tests. Can we make mock objects more powerful, so that our tests are more expressive? What can we learn from mocking libraries? Jon Reid will bring his background of writing the Objective-C library OCMockito and apply it to hand-made mocks in Swift.

---

Introduction

My name is Jon Reid and I work as an iOS developer at American Express. Today I want to talk about mock objects, how to make them in Swift, and how to make them more powerful and flexible.

Interaction Test

Why do we need mock objects? Why would you ever want to use a mock object instead of the real object?

Take an analogy in a restaurant where we order a meal from the waiter, and the waiter gives our order to the cook. In a unit test, we don’t want to use the real cook. The real cook takes too long or is not always there. And real cooking uses up resources so it affects the global state.

For unit tests, we want to use a fake cook. Instead of testing the results of real cooking, we test how the waiter communicates with the cook. This is an interaction test; one that define a contract of messages the waiter sends to the cook.

In Swift, we can usually substitute a fake object for a real object by defining a protocol. We use that protocol in our production code. Then we can substitute anything that satisfies that protocol, whether it’s the real object or a fake object.

Let’s talk more about cooking. Let’s say we have a protocol with a method to cook ramen.

enum RamenSoup {
    case shio // salt base
    case shoyu // soy sauce base
    case miso // miso paste base
    case tonkotsu // pork base, creamy
}

protocol CookProtocol {
    func cookRamen(
            bowls: Int,
            soup: RamenSoup,
            extras: [String]) -> Void
}

We can specify the number of bowls we want. We can specify the type of soup from the ramen soup enumeration. We can have extras or toppings. (Since cooking takes time, we would normally have a completion handler, but that gets complicated for our example, so let’s forget about that for today.)

Here’s code for the waiter.

struct Waiter {
    let cook: CookProtocol

    func order() {
        cook.cookRamen(
                bowls: 2,
                soup: .miso,
                extras: ["wakame", // seaweed
                         "tamago",
                         ]) // egg
    }
}

The cook is a constant which we provide during initialization. The waiter code would probably use the builder design pattern to build the order gradually. For our example, let’s just hard code an order method: two bowls of miso ramen with wakame and tamago. How do we test this method?

class MockCook: CookProtocol {
	func cookRamen(
				   bowls: Int,
                   soup: RamenSoup,
                   extras: [String]) -> Void {
  	}
}

class WaiterTests: XCTestCase {
	func testOrder_ShouldCookRamen() {
		  let mockCook = MockCook()
      let waiter = Waiter(cook: mockCook)

		  waiter.order()

	}
}

On the test side, we create a mock cook. Its implementation of the CookRamen method doesn’t do any cooking; instead its job is to capture information about how it was called. This is the important job of a mock object: to record the messages it receives.

In the unit test, we create the mock cook, inject it into the waiter through the initializer, then call the method we want to test, the order method named CookRamen.

Finally, we have to assert something. In an interaction test, we have a relationship between capturing information on method calls and asserting things about that captured information.

Let’s start with the most common pattern I see with Swift mocks. Many people assert that the expected method was called like this:

class WaiterTests: XCTestCase {
	func testOrder_ShouldCookRamen() {
		  let mockCook = MockCook()
      let waiter = Waiter(cook: mockCook)

		  waiter.order()

	  	XCTAssertEqual(mockCook.cookRamenWasCalled)
	}
}

To do that many people add a Boolean property.

class MockCook: CookProtocol {
    var cookRamenWasCalled = false

    func cookRamen(
    			   bowls: Int,
             soup: RamenSoup,
             extras: [String]) -> Void {
    	  cookRamenWasCalled = true
	  }
}

The mocked method sets that property to true, indicating that the method was called. But there’s a problem with this approach: it’s throwing away information. The Boolean only tells us that the method was called when it’s usually a problem if it’s called more than once. Instead, let’s record how many times the method was called.

class MockCook: CookProtocol {
    var cookRamenCallCount = 0

    func cookRamen(
    			   bowls: Int,
                   soup: RamenSoup,
                   extras: [String]) -> Void {
    	cookRamenWasCalled += 1
	}
}

Change the property from a Boolean to an integer and simply increment the count. This changes the test.

class WaiterTests: XCTestCase {
	func testOrder_ShouldCookRamen() {
		  let mockCook = MockCook()
      let waiter = Waiter(cook: mockCook)

		  waiter.order()

	  	XCTAssertEqual(mockCook.cookRamenCallCount, 1)
	}
}

Now we can say the cookRamenCallCount should equal one. Recently one of my tests caught an error by reporting that method was called twice instead of once. I had made an error during refactoring, and the test let me know. This is important not just for checking our first implementation, but for future refactoring.

Now that we have the call count, let’s capture more information about the method parameters.

class MockCook: CookProtocol {
    var cookRamenCallCount = 0
    var cookRamenLastBowls = 0
    var cookRamenLastSoup: RamenSoup?
    var cookRamenLastExtras: [String] = []

    func cookRamen(
                   bowls: Int,
                   soup: RamenSoup,
                   extras: [String]) -> Void {
        cookRamenCallCount += 1
        cookRamenLastBowls = bowls
        cookRamenLastSoup = soup
        cookRamenLastExtras = extras
    }
}

Let’s store each argument in a property. First, the number of bowls. Then, the type of soup, and the array of extras or toppings.

class MockCook: CookProtocol {
    var cookRamenCallCount = 0
    var cookRamenLastBowls = 0
    var cookRamenLastSoup: RamenSoup?
    var cookRamenLastExtras: [String] = []

    func cookRamen(
                   bowls: Int,
                   soup: RamenSoup,
                   extras: [String]) -> Void {
        cookRamenCallCount += 1
        cookRamenLastBowls = bowls
        cookRamenLastSoup = soup
        cookRamenLastExtras = extras
    }
}

We can name the properties to make it clear that we only have the values from the last call. Now we can have a test like this:

class WaiterTests: XCTestCase {
	func testOrder_ShouldCookRamen() {
		  let mockCook = MockCook()
      let waiter = Waiter(cook: mockCook)

		  waiter.order()

      XCTAssertEqual(mockCook.cookRamenCallCount, 1);
      XCTAssertEqual(mockCook.cookRamenLastBowls, 2);
      XCTAssertEqual(mockCook.cookRamenLastSoup, RamenSoup.miso);
      XCTAssertEqual(mockCook.cookRamenLastExtras), ["wakame", "tamago"])
	}
}

We assert that the call count is one, the number of bowls is two, and the type of soup is miso. Also, that we have two extras, wakame and tamago. This test works!

There’s a rule of thumb that a unit test should have one assertion: it should assert one truth. Here, we have four different assertions. It would be nicer to express this as a single assertion, especially when we have multiple tests. This is easy to do by extracting these assertions into a new method in the mock object.

func verifyCookRamen(
        	bowls: Int,
        	soup: RamenSoup,
        	extras: [String]) -> Void {
    	XCTAssertEqual(cookRamenCallCount, 1);
        XCTAssertEqual(cookRamenLastBowls, bowls);
        XCTAssertEqual(cookRamenLastSoup, soup);
        XCTAssertEqual(cookRamenLastExtras, extras)
	}
}

class WaiterTests: XCTestCase {
    func testOrder_ShouldCookRamen() {
       	let mockCook = MockCook()
       	let waiter = Waiter(cook: mockCook)

       	 waiter.order()

       	 mockCook.verifyCookRamen(
            bowls: 2,
            soup: .miso,
            extras: ["wakame", "tamago"])
 	}
}

The test now calls verifyCookRamen. But, what does this do to error messages?

Let’s change the number of expected bowls from two to three just to make the test fail. The error we will get is that two is not equal to three. And it will point to the failed assertion.

XCTAssertEqual(cookRamenLastBowls, bowls);

It’s pointing to the helper method, and not to the test. This is a problem, especially with multiple tests calling the same helper method.

We can fix this by adding two parameters to the helper method.

func verifyCookRamen(
        	bowls: Int,
        	soup: RamenSoup,
        	extras: [String])
          file: StaticString = #file,
          line: UInt = #line) -> Void {
        XCTAssertEqual(1, cookRamenCallCount, "call count", file: file, line: line);
        XCTAssertEqual(bowls, cookRamenLastBowls, "bowls", file: file, line: line);
        XCTAssertEqual(soup, cookRamenLastSoup, "soup", file: file, line: line);
        XCTAssertTrue(extrasMatcher(cookRamenLastExtras), "extras was \(cookRamenLastExtras)", file: file, line: line)
    }

All we have to do is capture the file name and line number as parameters with default values. Then we pass those parameters to the assert statements. Now, when we get a test failure, it points to the verify statement that failed in the test. mockCook.verifyCookRamen(bowls: 2. But, this isn’t a very good error message. It just tells us two is not equal to three.

We specify the what by adding a message to identify each assertion (e.g. “call count”, “bowls”, as in the code snippet above). Now the assertion failure says two is not equal to three bowls. This is a useful error message.

What about the array of extra toppings? What happens if we reverse the toppings (“wakame”, “tamago”)?

We don’t care about the order of the array, instead we only care about its contents, so we want it to still pass. When we run this test, we get an error message.

class WaiterTests: XCTestCase {
    func testOrder_ShouldCookRamen() {
       	let mockCook = MockCook()
       	let waiter = Waiter(cook: mockCook)

       	 waiter.order()

       	 mockCook.verifyCookRamen(
            bowls: 2,
            soup: .miso,
            extras: ["wakame", "tamago"])
 	}
}

XCTAssertEqual failed: (“[“wakame”, “tamago”]”) is not equal to (“[“tamago”, “wakame”]”) - extras

This is an example of a fragile test. A fragile test is over-sensitive - it may pass today, but tomorrow it may break due to a benign change in production code. Ideally, unit tests are our safety net. We want tests to give us confidence so that we can make bold changes yet confirm correctness, but not tests that keep us from changing the code.

Instead of specifying an array of extras to match using equality, let’s specify a predicate. We specify a closure that takes the argument and it evaluates to true or false.

class WaiterTests: XCTestCase {
    func testOrder_ShouldCookRamen() {
       	let mockCook = MockCook()
       	let waiter = Waiter(cook: mockCook)

       	 waiter.order()

         mockCook.verifyCookRamen(
              bowls: 2,
              soup: .miso,
              extrasMatcher: { extras in
                  extras.count == 2 &&
                  extras.contains("tamago") &&
                  extras.contains("wakame") })
      }

  }

Here we can say we want the extras array to have two items, it should contain wakame and that it should also contain tamago. This test passes regardless of the order of the array. It checks what is important but ignores what is unimportant.

Here’s how we change the verify method to support this.

func verifyCookRamen(
            bowls: Int,
            soup: RamenSoup,
            extrasMatcher: (([String]) -> Bool),
            file: StaticString = #file,
            line: UInt = #line) -> Void {
        XCTAssertEqual(1, cookRamenCallCount, "call count", file: file, line: line);
        XCTAssertEqual(bowls, cookRamenLastBowls, "bowls", file: file, line: line);
        XCTAssertEqual(soup, cookRamenLastSoup, "soup", file: file, line: line);
        XCTAssertTrue(extrasMatcher(cookRamenLastExtras),
                "extras was \(cookRamenLastExtras)",
                file: file, line: line)
    }

We change the parameter that used to accept an array of strings. Now it takes a closure that evaluates an array of strings, returning a Boolean. In the assertion, we pass the captured extras to the predicate and assert that the result is true. Finally, where XCTAssertEqual reports both the expected value and the actual value, XCTAssertTrue does not. In the failure message, let’s report the actual value.

Our test with the correct extras in a different order now passes. And, if the test fails (e.g. we get nori instead of wakame), the error message is good but it’s not great.

It’s okay in our IDE when the data is small; if the data is large, it gets harder to visually find the mismatch. If we’re not in Xcode or if it’s in the output continuous integration logs, then it might be an issue. Wouldn’t it be nice to have a predicate with more detailed output?

Hamcrest Matchers

Matchers were originally invented as part of jMock, the very first mock object library. They were then extracted to their own library and even became part of JUnit. Hamcrest is implemented in various languages including Objective-C and Swift.

Hamcrest matchers do a number of things. A matcher is a predicate that evaluates a value; it describes its expectations. If there is a mismatch, a matcher can give precise description of what’s different from the expectation. They can be composed from other matchers in a domain-specific language. Nested matchers, that is matchers made from other matchers, are especially powerful for evaluating collections. Hamcrest is not only a library of pre-defined matchers, it’s also a framework for writing your own custom matchers.

To make it easier to use Swift Hamcrest matchers in mock objects, let’s make a helper.

func applyMatcher<T>(_ matcher: Matcher<T>, label: String, toValue value: T, _ file: StaticString, _ line: UInt) -> Void {
    switch matcher.matches(value) {
    case .match:
        return
    case let .mismatch(mismatchDescription):
        XCTFail(label + " " + describeMismatch(value, matcher.description, mismatchDescription), file: file, line: line)
    }
}

The switch statement calls matches. If the result is a match, there’s nothing to do. If the result is a mismatch, we get the mismatch description as an associated value. Then we call XCTFail adding a label to identify what we’re checking. This is how we call it in our verify statement.

func verifyCookRamenUsingHamcrest(
        bowls: Matcher<Int>,
        soup: Matcher<RamenSoup>,
        extras: Matcher<Array<String>>,
        file: StaticString = #file,
        line: UInt = #line) -> Void {
    applyMatcher(equalTo(1), label: "call count", toValue: cookRamenCallCount, file, line)
    applyMatcher(bowls, label: "bowls", toValue: cookRamenLastBowls, file, line)
    applyMatcher(soup, label: "soup", toValue: cookRamenLastSoup!, file, line)
    applyMatcher(extras, label: "extras", toValue: cookRamenLastExtras, file, line)
}

First you see the argument type is a matcher of an array of strings. Then we call our new helper with the label extras.

This is how our previous code looks like when we add Hamcrest support.

func testOrder_ShouldCookRamen_HamcrestVersion() {
    waiter.order()

    mockCook.verifyCookRamenUsingHamcrest(
        bowls: equalTo(2),
        soup: equalTo(.miso),
        extras: containsInAnyOrder("tamago", "wakame"))
}

ContainsInAnyOrder is a predefined Hamcrest matcher. Let’s change the extras from wakame to chashu (sliced pork) to make the test fail. Here’s what the failure message looks like:

“EXPECTED: a sequence containing in any order [“tamago”, “chashu”]” describes the expectation. “failed: extras GOT [“wakame”, “tamago”] (mismatch: GOT: “wakame”, EXPECTED: “chashu”)” tells you the actual value of the extras parameter. Then it tells you in detail about the mismatch (got wakame expected chashu). Now we have a test that isn’t fragile which also gives a very detailed error message.

What happens if we use matchers for each parameter in our verify method?

func verifyCookRamenUsingHamcrest(
            bowls: Matcher<Int>,
            soup: Matcher<RamenSoup>,
            extras: Matcher<Array<String>>,
            file: StaticString = #file,
            line: UInt = #line) -> Void {
        applyMatcher(equalTo(1), label: "call count", toValue: cookRamenCallCount, file, line)
        applyMatcher(bowls, label: "bowls", toValue: cookRamenLastBowls, file, line)
        applyMatcher(soup, label: "soup", toValue: cookRamenLastSoup!, file, line)
        applyMatcher(extras, label: "extras", toValue: cookRamenLastExtras, file, line)
}

This is very similar to the first time we created this helper method but now we’re no longer calling XCTAssertEqual. Even for the call count which has an implicit expectation of one, I’m using an equalTo matcher.

In the test code, we can still test for equality where we want by using the equalTo matcher. But, the mock object is no longer restricted to equality. For example, for bowls we can say greaterThanOrEqualTo if that would make our test less fragile.

class WaiterTests: XCTestCase {
    func testOrder_ShouldCookRamen() {
        let mockCook = MockCook()
        let waiter = Waiter(cook: mockCook)

        waiter.order()

        mockCook.verifyCookRamen(
            bowls: equalTo(2),
            soup: equalTo(.miso),
            extras: containsInAnyOrder("tamago", "chashu"))
    }
}

Or maybe this test shouldn’t care about the type of soup so we use the anything matcher.

class WaiterTests: XCTestCase {
    func testOrder_ShouldCookRamen() {
        let mockCook = MockCook()
        let waiter = Waiter(cook: mockCook)

        waiter.order()

        mockCook.verifyCookRamen(
            bowls: equalTo(2),
            soup: anything(),
            extras: containsInAnyOrder(equalTo("tamago"),
                                      hasPrefix("chashu"))
    }
}

Look at the last matcher for extras. We don’t have to specify two strings. We can specify two matchers instead. When they’re both equalTo matchers, that’s the same as what we get with two strings. But we can use other types of string matchers. For example, hasPrefix to check for a string prefix.

Mock Recommendations

Here are my recommendations for creating more powerful mock objects in Swift.

• Don’t use a Boolean to record whether a method was called. Instead use an integer to capture the “call count”.
• When you see multiple asserts against a mock, try extracting a helper method.
• Add file name and line number to that helper method and pass those to the asserts.
• Add messages to each assert so we can tell them apart.
• When necessary, don’t assert using equality. Instead use predicates so that the test code can be more flexible and less fragile.
• Consider using SWIFT Hamcrest matchers, which are like predicates on steroids. Remember, our goal isn’t just to have tests. It’s to have tests that enable and encourage refactoring. We want tests that are sensitive to things that are important, but ignore things that are unimportant. We don’t want tests that break like glass. Instead we want tests that are like bamboo, strong but flexible.

If you want to see sample code or get a copy of these slides, please go to my website.