Most engineers start working on apps as prototypes, focusing on functionality until the point the app’s (lack of) architecture becomes an impediment in continuing to build features into the app at a fast pace. This 360AnDev talk will cover experiences with Coursera’s Android app, build up on issues with MVC and go on to illustrate how to start with MVC and get to a solution that fits the apps needs.
Clean architecture is a great way to start thinking about architecture for apps. Using S.O.L.I.D. principles is a philosophy that motivates developers to think of architecture in terms of intents rather than frameworks and build software that is independent of UI, database or libraries. UI changes much faster than databases, and it’s important to decouple former from latter to easily make changes without affecting the latter. We will go through a way of implementing clean architecture using a modified version of VIPER, and into implementation details using a sample application.
Introduction (0:00)
Hi, I am Richa Khandelwal, and I work at Coursera. This talk is about effective Android applications. I always think about Paris when I think of architecture. When I visited the city, I noticed that the 20 arrondissements or administrative districts are organized in exactly the same manner. The scalability is amazing. Paris was originally just 12 arrondissements and they just added one more outer layer and expanded it. That was beautiful.
When we set our intent about architecture, we should think about the same thing. It should be beautiful, easily navigable, maintainable, high quality and performant.
This talk draws upon a lot of my experiences as case studies, working at various companies and on personal projects. I’ll talk about some of the challenges that I identified while working on Android applications, which set the motivation to find the right architectural fit. Once the architecture is envisioned, it’s important to validate it even before starting the first line of code.
I’ll also talk about some validation strategies to implement when you come up with an architecture thought. It’s really important to practice it over a period of time and evolve it. Your architecture should be able to evolve with the changing needs of your application. Then I’ll follow up with some demo and key takeaways.
Challenges (1:49)
First I’ll say something about the application that Coursera has on Play Store. There are about 18 million learners. 40% of these use mobile, desktop, and multiple other platforms. This learner base is growing every month in countries like India, China, Colombia, and Brazil, which means a new set of challenges. Most of these countries have spotty, out-of-G networks and users are very sensitive to bandwidth consumption. Offline access is very important to them.
Working in startups, it’s essential to build products that can be and feel fast. If you are working on an open-source project, collaborating with people geographically separated from you or in a company where the theme is dispersed across multiple teams, it’s really important to have a code base which fosters effective cross-team collaboration.
Finding the Right Architecture Fit (2:59)
I worked on a number of different types of front-end, desktop, back-end, and Android applications. With most of the web and desktop apps, I started with MVC. It worked very well for simple applications, but it needed to be refined a little bit as the application evolved.
When I started with MVC on Android applications, the evolution process had to start right away. The moment the application scaled beyond certain lines of code or a certain number of users, I’d instinctively think about alternatives. One idea is to start looking at other architecture ideas such as MVP, MVDM, and VIPER, but the fact is that the moment you write some sample code, it will all look fine until you map it back to your actual code base. Then you might discover issues that you would never see in a small sample application.
Instead of running around in circles trying one architecture after the other, it’s a good idea to go back to the drawing board and think about the problem itself. When I used to work on web or desktop applications, we typically had a drawing board like the one on slide 12. It will consist of multiple different components in the columns, and each row represents responsibilities of each component. Pretty simple for small applications, but mobile has a lot of other considerations.
Android applications have a lot going on rather than just the application showing certain data to the user from views. There are a number of services running in the background/foreground. There might be broadcast receivers that your application might be listening to, and there might be other background jobs that your application might be running. Apart from that, the device configuration can change dynamically, so your application should be set up to adjust to these.
Typical applications pull data from the network. A lot of others, such as gaming applications, also need to generate a lot of data on the device, store all of this data locally, and then sync them back to their remote server. Many times, when the device is offline, or the user accesses the app from another device, there are additional sets of complications.
Modern devices have many sensors, including motion and environmental sensors. As application developers, we want to use of all of them and build delightful apps for the users. But we must architect our application so that it’s fast for user interaction and we can service the right sensor event to the user at the right time.
If you take all of this complexity and map it back to our drawing board table (slide 15) now each of the components looks a lot more complex than what we initially started with. Especially the controller, which has a lot more to handle now.
And if you started writing it in code, this is what it will look like. The model:
public class CourseModel {
Course getFromDatabase() {
// fetch from db, or local cache
}
Course getFromNetwork() {
// fetch from apis, cache when possible
}
}The view:
public class ExpandableCourseListActivity {
void onSaveInstanceState(Bundle b) {
// save all expanded states
}
}The view will have an additional onSaveInstanceState. I’m sure most of us have seen this, when we need to save the application state on rotation or when the activity goes into the background:
Rather than only dealing with displaying data to the user, the controller needs to deal with system events like if the location gets updated, if the user wants to take a picture from the device, that needs to be relayed back to the system somehow:
void onClick() {}
void onModelUpdated() {}
void onNewNotification() {}
void onLocationUpdated(Location loc) {}
void takePhoto(Activity context) {}
void launchView(Activity context ) {}All of this goes into the controller invariably. And that defeats all of the challenges that we started out to solve.
As we think about what kind of an architecture that we can go to next, starting from MVC, it’s also important to consider how it will adapt to the changing needs of our application. Let’s say there are additional things we want to do in our application, will this architecture be able to support that? Are there components in the application and the architecture itself that can be adapted to handle these additional responsibilities?
We tried our best to capture most of the things that mobile applications need to do in the initial table. We can go back to that and start to think about how can we start from MVC and come up with an architecture that can handle all of these responsibilities and scale up as the application does as well.
The first lesson I learned as a software engineer was whenever you write a problem statement, you should write it in very clear, concise language and then look at the problem statement to extract out the solution from the problem itself.
It’s important to understand the commonalities across the different things that the application is trying to do, and then think about how you can componentize your application into a good architecture that can fit all of these components together.
We can see on slide 21 that the controller is handling a lot of interaction logic. It’s interacting with the model, and also, it’s interacting with the system components and handling system events. If the controller can be oblivious to all of these interactions and can work with the data that the view needs to represent, it can be a lot simpler. And we can add a specialized component that handles all of the interaction. We can call it an Interactor.
New Components (9:13)
As you can see on slide 23, this is how it fits back into our architecture diagram. The controller no longer interacts directly with the model or the system. The interactor handles all of those responsibilities for the application.
Bringing it back, we update our table to accommodate that on slide 24. The view needs to save the view state every time there is an event that warrants that. The controller needs to forward the updates to the view, and in turn also needs to manage updates to the view and save it so that it can relay back to the interactor on things like post request, etc.
We can also extract out these common functionalities into a specialized component called View Model. View Model is not new; it is inspired from MVVM. On slide 26, you can see how it can fit back into the architecture. The view and the controller now communicate through the view model. The view has no idea how the view model is constructed, it just needs to bind to the data in the view model. The controller manages the view model creation and its life cycle. The next slide, 27, shows how the view model fits back into the table that we originally came up with.
Since the controller has minimum control functionality to do, we can call it a “Presenter”. Going back to the table, the “Model” looks very complicated as well. It’s trying to do a lot of things, interacting with the data coming from a lot of sources, and trying to manage consistency between all of these different views of the data. It’s a good idea to extract out all of these complexities into a depository or a data source layer.
This data source layer can manage all the functionalities of handling data management. What it spits out is entities when the interactor makes a request to it. One of the great things about this is also that the data source layer can be tested by itself. It can be maintained by itself, and it can be evolved separately based on the needs of the application.
If you want to add persistence to it, or caching, or change the way the API calls are made, this is all now extracted out of the application. Fitting the “Entities” part back into the application stack, you can see our updated table on slide 31.
If you pay attention to the Presenter, it manages one very big functionality that most applications need to perform, which is navigation. Navigation is navigating between different parts of the applications, and in the Android world, it means creating intents, either explicit or implicit, and fighting off those intents to invoke certain parts of the application.
We can also extract it out into a separate component called FlowController and add it back. The reason for that is, if each presenter is creating intents to invoke the same part of the application, and if that part of the application changes all of the sudden, all of these different presenters need to be changed. Instead of that, all of this can be changed in one place in the FlowController, and the presenters just need to worry about how to generate that intent to have the FlowController make that navigation.
There are multiple different strategies for it. Explicit intents should be avoided mostly. Deep-linking is a great strategy. In fact, if you add deep links into an application from the get-go, if you need to interact with other applications in the future, it would be pretty simple. You wouldn’t have to start from scratch and add deep link center application.
You can see on slide 34 what our final table looks like. Each component handles a distinct responsibility. Entities represent the model objects. The View manages to render the view model. Presenter manages the view events and view model. Interactor takes care of interacting with external entities and spits them out back to the presenter. View model manages the view data and state, and FlowController manages navigation inside the entire app.
One other important thing to do is make sure you apply dependency inversion across all of the different components in your application. The reason why that’s important is you don’t want presenter to be changing. If anything, the view changes. You don’t want the interactor to change if anything in the view or the presenter layers change.
There are multiple mechanisms to add dependency inversion to the application. You can add notifications or callbacks, but the best way to handle this is using Reactive extensions. RxAndroid or RxJava are great libraries to accomplish that.
What that can do is the view can subscribe or unsubscribe to updates from the presenter at any time without the presenter knowing anything about it. The presenter never needs to have a strong reference to the view at all. It can do it’s own things without being aware of anything going on inside the view. The same goes on for all of the different components that we created.
Most importantly, for the data source, you can hook it up to the rest of your application using reactive extensions, and then you can make it very sophisticated using RxJava mechanisms.
We came up with an architecture which is heavily inspired by VIPER architecture, and as we were going through the process of solving this problem at Coursera, we started realizing that this shares a lot of commonality with VIPER that was originally evangelized by Mutual Mobile.
Then the question remains whether or not this will work in practice if we started implementing it in our code. There are various strategies that can be used for that as well.
Validation (15:28)
The first one is finding validation and well-established software principles like S.O.L.I.D. It stands for:
- Single responsibility principle
- Open/closed principle
- Liskov substitution principle
- Interface segregation principle
- Dependency inversion principle
In the architecture that we just talked about, it strictly adheres to three of these principles, not all five. The other two are implementation-dependent. But it definitely guarantees: single responsibility principle, interface segregation, and dependency inversion.
Another way you can validate the architecture is good and maintainable is by adding tests, as these different components are now isolated and manage distinct responsibilities. The view can be tested using Espresso. The presenter can most of the time be a POJO. You can easily test it using JUnit. Interactor ends up being either an Android or Java object, depending on what kind of interactions it’s handling, and then you can use either Android or JUnit tests for it.
You would have to use Android tests for the FlowController in most cases. Android also recently added unit tests which work quite well, so that can also be employed for testing the Android components. Then the data source layer can be tested in isolation and maintained in isolation. The rest of the app has nothing to do with data source layers anymore.
Another way is to try and map it back to clean architecture. Clean architecture fosters a dependency rule. It applies a dependency rule when the dependency only goes in one direction, from the outer layer towards the inner layer and not the other way around. What we came up with, we guarantee that the view can be changed without affecting the entities in the application. And the business logic, it can be changed again without affecting the entities on the data layer. This provides a clear separation of layers just like clean architecture advocates.
Code Generation (17:40)
That was the V1 phase. In this phase, we came up with this architecture and we implemented in practice using lots of different libraries and frameworks. The important is that we did not think about libraries first. We first thought about the intents or the problem that we were trying to solve, and then we started thinking about the libraries that are open-source or advocated by Google that we could use to make all of these problems easier to handle. Once we came up with this architecture, there were a lot of other problems that started surfacing.
One big problem was boilerplate code, and another was a lot of repetitive code, spread across different parts. We started looking at code generation. Once you get it right one time, you will find it’s really easy to replicate from that point on. And whenever you see a repetitive code, you should think about generating it. As Picasso said for artists: “Good developers write code, and great developers generate it.” When the code is generated, you don’t need to write tests. You just need to write tests for the code generator.
In our code base, we started using code generation heavily after this phase. All of the value objects that we pass from the entity layer back up to the presenter layer of the view models, we generate all of them. There are great auto-value extensions that are also available for use that you can use to convert JSON to Java or Jx. Those are worth looking into. We had a lot of event tracking logic all over our application, so those were one of the first candidates for code generation as well.
Later on, as we were building out our data source layer, we figured out that different applications spawned different type of data retention policy. For logged out users it’s fine to show some data, but other data is restricted only to logged in users. The way the apps define these policies to the data source layer was also through annotations, rather than trying to write custom code.
All of those policies were generated in the data source layer. All of our routing layer was also metered in with code generation, so all that needs to be done is a URI, like the one I showed on a previous slide, was passed along to the FlowController, and the FlowController figured out how to translate it into an intent.
That first part tackled all of the issues with developer productivity and also eliminated boilerplate in the code. The latter part was how can we leverage this architecture to also speed up performance in the app?
Since the data source was separated out, it was possible to try out different libraries or mechanisms to speed up data processing. The other thing is that, as the presenters were standardized using the interface segregation principle, all of these presenters had unloaded methods, and some of these presenters went on the critical path of user experience. What I mean by that is if you’re trying to sell something to the user in your application, you would want to make sure that from the first screen that the user sees right to the payment screen. The user experience is snappy. If the user’s experience gets stuck at any point, users start dropping off.
If you care about those sort of optimizations or any other problem that you’re trying to solve in your application, it’s a good idea to think about how you can make that journey faster for the user. What we did was we identified a bunch of presenters on the critical path of user experience, and we called the onLoad method on those presenters right away on an application mode. So as the users navigated from screen to screen, the data was already fetched, persisted and ready. No spinners.
Demos and samples (22:09)
Now I’ll dive into some code examples. There is a demo of how this architecture is implemented in practice, and it is available on GitHub in my CourseraDemoApp repository. It’s a very simple app. It shows how different components in the architecture interact with each other when mapped to code. I’ll go through that a little bit here. In the code example, you will see multiple different classes:
- MainActivity
- FlowController
- CatalogActivity
- CatalogPresenter
- CatalogViewModel
- CatalogInteractor
The launch in activating the application is the main activity:
// MainActivity
void onCreate() {
FlowController.getInstance()
.launchCatalogActivity(this);
}Which just launches and calls the FlowController, to launch another activity. Imagine that you want to execute an A/B test, and you want to figure out which is a more effective activity to show to the user when you’re trying to drive for certain metrics. You can add that A/B test here. It’s really hard to do that when you specify the launcher activity in the manifest.
When you tell the FlowController to launch a specific activity, it figures out how to create that intent:
// FlowController
void launchCatalogActivity(Context c) {
Intent intent = …;
c.startActivity(intent)
}Note that this example is really simple, and it doesn’t cover all of the complexities I talked about before, to make sure that it’s simple to follow and understand. The activity comes up and it creates the presenter:
// CatalogActivity
CatalogPresenter presenter;
void onCreate() {
presenter = new CatalogPresenter(this);
}The presenter subscribes to updates from the presenter:
// CatalogActivity
void onResume() {
Action1<Catalog> action = ;
presenter.subscribeToUpdates(action)
}It’s very important for the activity to unsubscribe to updates from the presenter in onPause for obvious reasons. If the activity goes into the background, it doesn’t want to continue to receive updates from the presenter and mess with the view:
// CatalogActivity
void onPause() {
presenter.unsubscribeToUpdates();
}The presenter creates the view model and the interactor:
// CatalogPresenter
CatalogInteractor interactor;
CatalogPresenter(Context c) {
viewModel = new CatalogViewModel();
interactor = new CatalogInteractor(c);
loadCatalog();
}The interactor, in turn, calls upon the data source to extract the entities out of there:
// CatalogViewModel
// Parcelable
public final Catalog catalog = new Catalog();// CatalogInteractor
Context mContext;
public CatalogInteractor(Context context) {
mContext = context;
}
public Observable loadCatalog() {
return
catalogDataSource.loadCatalog(mContext.getApplication(
));
}And then passes it back to the presenter:
// CatalogPresenter
void loadCatalog() {
interactor.loadCourses().observeOn(AndroidSchedulers.mainThread())
.subscribe(new Action1<>() {
@Override
public void call(List<Course> courses) {
// update the view model
}
});
}The presenter gets a notification from the interactor when data is ready and passes it back to the activity. Encapsulated in view models, and from then on the presenter and the activity, they communicate with each other through view model updates. If the data changes at any time, view model updates propagate them back.
One of the optimizations that can be done here is the moment the user opens the app; you can fetch the data from the persistent storage and pass that right back. Then, go and grab it from the network and update the data. If you use this sort of architecture, you can easily accomplish that.
Challenges (24:35)
All of that seemed easy-breezy, but it really wasn’t. It took a long time. This version of the architecture was finalized one and a half years ago. We were all excited about it, but we are still evolving it to fit the changing application needs. We are still coming up with great ways to ease developer productivity and make sure the application performance and quality don’t degrade over time.
The first challenge we faced was the context. As we know, Android applications have two types of self-context: application and activity contexts. The data source can not mess with activity context, and the view layer cannot mess with the application context. Keeping those contexts separate but still passing them down was really tough through constructive dependency injection. Using Dagger, this could be simplified a lot more, where that part was extracted into the Dagger modules, and the applications no longer needed to deal with the confusion around which context to pass along.
As I mentioned earlier, the moment we started with the architecture noticed a large number of value objects, and then a large number of classes per feature. So we tackle value objects with code generation, and it was really simple to do that with other parcel libraries. With routing and other data source dependencies we were also able to generate code, and after that, the presenter and interactor seemed really easy to deal with. The other challenge was, once we came up with a strategy, there was a lot of discussions and alignment that the team needed to go through, to adopt this architecture, implement it and evolve it as a team.
Also, when onboarding new team members, it was a lot of overhead to make sure that everybody follows the right set of methods to implement this architecture. We created a lot of presentation sample code to educate new hires. Over a period of time, this process has become a lot simpler, and it’s now easy to onboard someone new, or to talk about architecture decisions in the team.
Key Takeaways / Conclusion (26:47)
After all of this, some of the key takeaways from this entire exercise was:
-
Whenever you have the latest idea, you should ship it. In our Android team we have two-week release cycles, so we have a quick turnaround time whenever we come up with new ideas, or we want to enhance an existing feature.
-
What about code refactoring? The moment you have an idea of a new architecture, should you go back and refactor the entire codebase to conform to this new architecture? What we did was, whenever we came up with a new idea, when we came up with the architecture idea inspired by VIPER, we created a new module itself, and we started adding new features within that new module. Every new feature that we added, we implemented it with VIPER, and then assessed the existing features and thought, will we need to redesign it at some point? If that’s true, then we shouldn’t even bother. If that’s not true, then we just reroute those minimal number of modules.
-
After we had done all of that, it was very important to validate that this was working in practice. Code reviews worked smoother. Rarely getting a lot of bugs or feedbacks from our feedback SDK or Google Play store. Was it really confusing for someone to start writing code with this new architecture? And we went through multiple phases and multiple iterations to evolve this entire process so that everything was smoother over a period of time, and reiterated on it.
-
One of the principles that we followed when we were trying to reiterate was the difference between two iterations shouldn’t be huge. You should always feel like you’re converging at some point and making progress. That’s very important to keep in mind.
And with that, I would say that you shouldn’t come up with an architecture idea thinking that that’s the holy grail, you should come up with an architecture that can fit your evolving needs. You should be ready for changes as they come along.
Q & A (29:36)
**Q: How much time did it take to transition to the new architecture in a mature app and what were the pushbacks from the team? **
Richa: We’re still evolving, but we don’t have any more modules left that retain the old architecture. All of our new modules had retained the new architecture. Often we find that there is a pattern that’s superior to what we had before, and we only adopt that in new modules. Historically, every single front-end page or Android application that I’ve written in the past got re-written in six months to a year because we wanted to adopt new designs or new product ideas into them. So that made it really simple.
Regarding pushbacks from the team, the initial phases were really hard. There were a lot of hallway conversations, a lot of meetings talking about does this seem right, these are the other things we can try, and this is what we have. These are the sort of problems that we are facing. We always went back to the drawing board. The set of transitions that I showed you in those tables, they seem really fast, but they happened over a long period of time where we went back to the drawing board often and brainstormed about what are the different things we can try, what makes sense. We read up on VIPER, and all of these different architectures.
Resources
Richa Khandelwal
I am a seasoned Android and back-end developer and a novice front-end developer. On Android, I have worked in the system nitty-gritty as well as on cool and interesting applications. Have been an early adopter of the Android platform as a developer, and proud of the prolific Android community and where Android stands now.
Featured News
Receive news and updates from Realm straight to your inbox