Writing Solid UI Tests on Android

In this session, Valera Zakharov from Slack will share his thoughts on why it’s worth having developers write UI tests for their features, and how UI Testing should be done on Android.

---

Introduction

My name is Valera Zakharov, and I’m an Android engineer at Slack. Prior to Slack, I worked on tooling for Android developers at Google. Part of my team’s mission at Google was to do developer outreach and education - my responsibility was with testing.

Why Test?

The first reason is to make sure that the code works - that is, to have confidence that the code works. When working with legacy code, writing tests also ensures that the code will continue to work in the future for people who will maintain your code.

The second reason to write tests has to deal with empathy. Empathy is one of the core values at Slack. It’s the ability to feel the pain of other people. In our context for example, if you’re developing an API, it’s to feel the pain of the people that will write against your API.

Writing UI tests, specifically, is a great way to empathize with your users. Testing not only helps you have confidence in your code and make sure that it works, but it also helps you feel the pain of whoever will be using your application in the future.

UI Testing is Hard

The reason why UI Testing hard is because of multi-threading. If you do not deal with synchronization, you will get inconsistent passes and fails in your tests, and your results will often look like this:

Espresso

In 2013 I wrote Espresso to help the Google Wallet developers write UI tests, specifically to deal with synchronization in UI testing.

How Espresso works is that for every test interaction, such as clicks or assertions, it takes the interaction and waits until your application is idle, then it will run it on the UI thread. Once the runnable is complete, it checks the result, and propagates it back to you in a friendly fashion.

You can use whichever UI test framework out there, but you need to make sure that it takes care of synchronization, and that it doesn’t do it with just simple sleep statements/retry mechanisms, as that’s not good enough.

Writing Good UI Tests

The way to write good UI tests is to avoid writing them when possible, and if it’s not possible, write them in a targeted and hermetic way.

Avoid writing tests

The reason you should avoid writing tests is that when you run UI tests or any instrumentation tests, not only are you exercising your code and application, you’re also exercising the Android system including the Linux kernel and application framework.

This isn’t efficient, and this is one of the reasons why you commonly see this picture about the testing pyramid:

The testing pyramid says that the majority of your test coverage should come from unit tests, followed by integration and UI tests. How you’re supposed to do this is by separating out your business logic from your UI presentation layer and your data layer.

But in Android, is it possible to make the separation? For example, can you exercise an activity without doing any UI interactions? You can’t, and if you write pure Android without any other layers of abstraction, it doesn’t guide you towards writing a lot of unit tests.

In fact, I believe the testing pyramid for Android looks more like this.

You end up doing a lot of manual testing, UI testing, then unit testing.

Targeted Tests

The good news is that UI tests can still be small and targeted. Suppose you want to test the settings screen in the app, and you first need to log in with credentials before you’re able to do the actual test. The problem here is that you run through a user flow that has nothing to do with what you want to test.

In this case, a targeted test would be to go there directly. You just launch an active intent directly to your activity, define a simple activity test rule with your activity class as a parameter, and the test runner will launch your activity.

Hermetic

her·met·ic /hərˈmedik/ adjective

1. (of a seal or closure) complete and airtight. “a hermetic seal that ensures perfect waterproofing” synonyms: airtight, tight, sealed, zip-locked, vacuum-packed; More

2. relating to an ancient occult tradition encompassing alchemy, astrology, and theosophy.

In the context of testing, hermetic tests should be self-contained, and no state should leak into and from the test.

Hermetic Data

As a rule of thumb, when it comes to hermetic data, you should inject fake data as close to what you’re testing as possible. If your layer of abstraction/data provider layer is sitting directly under the presenter, then that is a great place to mock things out. This will mean the network layer and the persistence layer will not be exercised at all.

The second part of being hermetic is controlling background operations. This is because background operations are non-deterministic. Your application will make network calls, and those calls can take minutes or seconds.

Espresso provides an API called Idling Resource. It gives you the ability as an application developer to tell synchronize with a background task.

Injection

How do you actually make sure that your application running under your test has the mocks that are swapped in for the production modules? We can use dependency injection.

There are dependency injection tools such as Dagger (versions 1 and 2), and Gradle flavors. What’s unique about Gradle flavors is that you can define difference source sets, allowing you to choose one set of code over another. Finally, I’ve seen people roll their own dependency injection - I don’t suggest you do that.

UI Testing Other Applications

Android allows you to use other applications via implicit intents, such as the camera app. This can become a problem in UI testing, as you can’t control the UI outside of your process.

If you want to keep things hermetic, you can use an Espresso extension called Espresso Intents, which would allow us to check the result from an external application.

Limitations of Espresso

Espresso does have some limitations. For example, Espresso is not good with notifications, toasts, and app animations.

A way around this is to use “indirection”. Suppose you want to test toasts. You can write a toaster class, and wrap the code that is calling to make toasts.

In that new module, you capture the toasts that come in, save the state and texts, then later examine it again from your test.