MVC-N: Isolating network calls from View Controllers

Marcus walks you through a design pattern that he has been using on iOS for applications that require and use a large amount of data frequently requested from the internet. The design takes the well known Model View Controller pattern and expands upon it to allow for asynchronous network calls that are isolated from the User Interface controllers.

---

Introduction (0:00)

This post will not introduce you to a new design paradigm. Instead, it will explore something that has existed forever: involving the network layer.

Controller Layer: Anti-Pattern (00:49)

Over my career so far, I see projects and code bases that have a common anti-pattern over and over again. It always starts with a new project.

We start coding: there is already a design, a UI, and a user experience set up. We build a ViewController, based on our UI or UX design, and we’re happy with it. But, it needs data (there is no iOS application on the planet anymore that does not need data from somewhere), generally from the internet. We grab the data via a network request in our ViewController (a very common pattern), and you get the data back from the server. It works fine: our UI displays, and we are happy. First one done.

Flip to the next page on the UI, and now we have to build another ViewController. This one needs different data, maybe even coming from a different server. You do the same thing again, because it worked fine in the first ViewController. We build your interface, maybe with a story board. We go out to the network, get the data, and display it on the screen. Again, success.

We go to the third one. At the third, fourth, or maybe the fifth ViewController, we start realizing that we need data from the first or second ViewController, and they are probably already out of memory (or you are doing something naughty, e.g. grabbing the NavigationController). We are going through the array of ViewControllers looking for ViewController number one, with a comment next to it (“do not move this!”), to get the data out of it. And it works fine.

The UX designer comes back and says: “we have to rearrange a few things in the UI, but it will result in a better user experience”. They suggest moving ViewController four to ViewController two. That does not work well, because we need the data that is in ViewController two in your subsequent ones. What do you do?

You could duplicate the code, but we know that is bad. You start doing “interesting” things. Maybe you start loading the ViewController off screen, or initializing the ViewController but not putting it in the stack (admit it, we have all done it to get that data, because you want that code to execute). Then you start thinking, “I need to do something with this; I need to cache the data”. In reality, you can use any persistence engine you want.

I use Core Data (I ❤️ Core Data). Realm works fine, and a JSON file will work fine, as long as you are caching the data.

So, you cache the data, change the layer, and each one of the ViewControllers will stick it in the cache as they get data. When any other ViewController comes up, they can get the data out of the cache. No. This is bad.

This is what I refer to as the “anti-pattern”. I cannot count the number of times I have seen this, or done it myself. This is the pattern I am attacking. I am openly saying: please stop doing this. We want to write code, we want to see results right away, and we end up doing this. This is a problem.

MVC-N: Network Controller (05:30)

We are doing Model View Controller (MVC), but we need to take MVC a step further.

I am introducing another controller, but not a ViewController. Note: if you do not like the word “controller”, then “manager”, “handler”, or anything else works fine. The point is that it is a controller object, because it is not a view and not a model.

That is what we should be doing at design time, not at the 11th hour when we are shipping tomorrow morning, and saying “I need to refactor all this”. When we have the UI/UX, we understand how the app needs to come together, and that is when we should be looking at it.

“I need data from the network. Where do I put that code to get the data out of the network?” This is where the MVC-N comes in.

MVC-N (06:32)

We have a network controller: it talks to the internet, your local network, or even be a flat file system. It then pushes data into your cache, into your persistence engine. Here comes the exciting part: your entire view layer talks to the cache only. It does not get data from the internet directly. The ViewController’s job is to display or update that data, not to do networking code. This design will serve you forever, and it has so many benefits that you’ll never want to use the anti-pattern ever again.

Network Controller (08:00)

class NetworkController: NSObject {
    let queue = NSOperationQueue()
    var mainContext: NSManagedObjectContext?

    func requestMyData()
    func requestMyData() -> NSFetchedResultsController

    func requestMyData(completion: (Void) -> Bool)
}

There is not much to it: it’s an object. I have it subclassing off an object (even in Swift) because I use KVO in my network controllers. You don’t have to though; if you are using a different caching system, you may not need KVO.

It is just an object, it does not come off of ViewController (or anything else). It is a root level object. We have a couple of properties inside: a variable and a constant. We have a queue and our caching mechanism (Core Data, because I like Core Data).

Exposed to the your ViewControllers are convenience methods and syntactic sugar. There are a few different ways to build these, depending on how much you trust your UI people. If you do trust them, you could use a function that says, “go get my data,” and you can trust them to talk to the cache layer directly, get the data out, and display it to their view controllers. I do not trust them that much, so I tend to give them a second version which gives them back an NSfetchedResultsController instead.

NSfetchedResultsController for Core Data is great glue code that sits between the cache and your ViewControllers. It serves up all the data to the ViewControllers in the structure they want, filtered down to what they want, and sorted in the order that they want. If you have a list of addresses, it will give them to you in alphabetical order, and will give them to you populated.

The other side of the NSfetchedResultsController is that it will notify the ViewController when the data has changed. The ViewController does not need to talk to the cache. All it needs is NSfetchedResultsController, get its data out of there, and then it has delegate callbacks to tell it when the data has changed. This simplifies the ViewController immensely. Your ViewController is waiting to be told to refresh itself.

If your app is following some of the more common UX patterns (e.g. pull-to-refresh), your ViewController may want more information than that the data has updated. It may want to know when the data request is done. If you have a pull to refresh, and the users pulls and pulls and pulls, and you are in a tunnel… they will potentially kick off multiple requests.

You may want to give them some way to say, “The request is still active, and it is not done yet.” That is a promise system. This can be done by having them pass in a closure and saying, “Let me know when you are done. I can update the pull-to-refresh and stop bugging you.”

To the network controller, this is all the external interface is. These are the only things you want public, and maybe not even the queue or even the context, depending on how much you trust your UI people.

What do we do with this network controller? We need to get it into the ViewController. There is a few ways:

1. Dependency injection (12:24)

This is my personal recommendation. When your app starts up, you have an applicationDidFinishLaunching call. You have your root ViewController built, your network controller should get built, and you can hand in the network controller into that root ViewController. Then, when the next scene or ViewController comes in, the root hands it off to the next one, and the next one after that. That is dependency injection.

It’s a simple concept that comes from Objective-C, so no third party framework is needed: it’s just one object handing off and setting the properties into the next object.

2. Singleton (13:27)

The second way is the way I hate: you can turn the network controller into a singleton. It is a fairly safe singleton because one of the negatives of singletons is not being able to tear them down. You cannot reset a singleton, because that defies the point. But, since the network controller itself is a fairly empty skeleton, it is easy to reset the stuff inside of it and you can get away with a singleton.

But, please don’t. Singletons are evil. If that is the way your system and app work and you need to use them, I am not going to judge you (much). Go ahead and use them.

3. AppDelegate property (14:08)

The third way, which is almost as bad, is accessing the AppDelegate directly. Every one of your ViewControllers gets the AppDelegate and calls UIApplication ApplicationDelegate and casts it. I highly discourage you from doing it this way too, because it tightly couples things together.

Overall, use dependency injection. The other two methods are landmines waiting to go off in production.

Network Controler - NSOperation (14:46)

class MyNetworkRequest: NSOperation {
     var context: NSManagedObjectContext?

     init(myParam1: String, myParam2: String)
}

The network controller is a wrapper around network operations. The operations go out and make a request to the network; the data is comes back and gets processed, which you then stick into the cache, and they are done. These are small discrete units of work. That is a beautiful thing, when you are writing code and you can see it all on the screen. You can understand it, and it does one thing: that is good, highly maintainable code. That is what we want to build inside the network controller.

I like to do MyNetworkRequests subclassing off NSOperation. Each NSOperation is designed to do one small discrete unit of work, like fetching a Twitter timeline. When you make the operations small discrete units of work, you can chain them together and start reusing the code.

There are two trains of thought to reusing code:

1. “I will going to publish it online and everybody can use it as a CocoaPod reusable” 👎

2. “I am using the same code in my app in 15 different places. All I have to do is change it in one place, and my entire app updates” 👍

What do they look like?

There are subclasses of NSOperation. Inside them, they have a couple properties we can set. Again, we are dependency injecting: we inject an access to our cache into the operation, and then pass in some parameters (maybe an NSURLRequest, a URL, or something as simple as a search parameter), and let the operation build its whole URL itself.

Warning: There is code below! Don’t get scared and run away from it! 🏃

Network Controller - Code 1 (17:25)

class MyNetworkRequest: NSOperation {
    var context: NSManagedObjectContext?

    private var innerContext: NSManagedObjectContext?
    private var task: NSURLSessionTask?
    private let incomingData = NSMutableData()

    init(myParam1: String, myParam2: String) {
        super.init()
        //set some parameters here
    }

    var internalFinished: Bool = false
    override var finished: Bool {
        get {
            return internalFinished
        }
        set (newAnswer) {
            willChangeValueForKey("isFinished")
            internalFinished = newAnswer
            didChangeValueForKey("isFinished")
        }
    }
}

To begin with, we subclass NSOperation. We have a couple more properties that are private, that are internal to the NSOperation that we are flagging as private, to remind ourselves to not touch these externally.

The first one is an innerContext (to be able to use Core Data on multiple threads, we want to have multiple contexts). If you are going to use a different caching mechanism, follow their design for multi-threading.

I have a love/hate relationship with NSURLSession (leaning more towards love), but it is a superior improvement to NSURLConnection that has deprecated as of iOS 9. The hate part is that it is a slippery slope if you want to use its block implementation: that leads you back into the UIViewControllers, and then all of sudden you are doing off-screen ViewControllers and having memory problems. However, you can also use it in NSOperation design, which I think is the superior way to do it.

We can no longer use the main method inside of NSOperation. Now, no matter how hard you try, NSOperation will not pause while you are doing the network connection. You can hack it (not recommended), you can do some synchronized stuff, and locks (not recommended). As of iOS 6 or 7, they changed NSOperation where they no longer run concurrent. You cannot make an operation run on the thread that had called it anymore. If you set the isConcurrent true or false, it ignores you. It is always guaranteed to run on another thread, or another queue as it is.

We can use the concurrent design of the NSOperation to do our network requests. In Swift, because we cannot access the property directly for finished, we have to do hacking (which I do have a radar for, and I am hoping they will fix this). The idea behind this overrided var is that we need to tell the queue that we are done. To do that, the queue is listening for “isFinished”. In Objective-C, isFinished translates into the variable Finished fine; in Swift, it only listens for “isFinished”, it does not listen for the actual property of Finish, you cannot change it.

That is why our setter is interesting and it is playing with KVO. We are hanging on to our own internal state variables, whether we are finished or not. Whenever we change it, we are firing a KVO for that isFinished accessor.

Network Controller - Code 2 (20:41)

func URLSession(session: NSURLSession, dataTask: NSURLSessionDataTask,
    didReceiveResponse response: NSURLResponse,
    completionHandler: (NSURLSessionResponseDisposition) -> Void) {
    if cancelled {
        finished = true
        sessionTask?.cancel()
        return
    }
    //Check the response code and react appropriately
    completionHandler(.Allow)
}

func URLSession(session: NSURLSession, dataTask: NSURLSessionDataTask,
    didReceiveData data: NSData) {
    if cancelled {
        finished = true
        sessionTask?.cancel()
        return
    }
    incomingData.appendData(data)
}

You can put the next two methods in an abstract parent class. Generally, you get a response back, go out to the network, and say, “go connect to this server and request this data”. The server immediately gives you a response back.

The comment is where you would put that logic and ask the response, “What is the status code? Is the status code 200?” Good. “Is it not?” Crash. However we want to handle that. Once we know that we are in a happy state talking to the server, we pass a parameter into the completion block, the completion analysts hand it to us and say (.Allow). Telling the URLConnection go ahead, continue and start getting data back to our NSURLSession. If you are playing around, that implementation of the method works fine.

The other method gets called zero or more times. The number of calls is not necessarily dependent on the size of the data. There are variables that depend on it, such as cellular data vs. WiFi, or getting dropped packets. The only thing we are going to do is append it to our internal variable, because we do not know how many we are going to get. We are going to keep getting calls, incoming data, and keep appending these bytes into this NSData array, and we are going to wait until we get one more callback, which tells us us we are finished.

Network Controller - Code 3 (23:43)

func URLSession(session: NSURLSession, task: NSURLSessionTask,
    didCompleteWithError error: NSError?) {
    if cancelled {
        finished = true
        sessionTask?.cancel()
        return
    }
    if error != nil {
        self.error = error
        log("Failed to receive response: \(error)")
        finished = true
        return
    }

   //PROCESS DATA INTO CORE DATA

    finished = true
}

The other two above are about as close to boiler plate as I am willing to get, but this is the fun part.

This method is the last one we get, and it is going to say “I succeeded” or “I failed”. Whether you succeed or fail talking to the server, both will call this method. If it failed, we get an error coming in, and we have to handle the error. We will check for cancelled: if we are cancelled, we do not care about the error.

We need to check for errors. You could do an if let, but it does not get you much. I generally hang on to that error. Somebody external to the operation who is listening for the operation to be finished can look at that error and react to it. From here, I do not want to be telling the UILayer that something went wrong, because this guy’s job is to get that data in-house and do something with it. Assuming we are on the happy path (i.e. we have not been cancelled and we do not have an error), we now have data down from the network. We are done talking to the network, and we can move on to our next step. We may decide to process the data right here.

In the same operation, we consume it and stick it into our cache; or, we may decide to chain off into another operation. Both of those are equally valid solutions to the problem.

But I will process the data here, and I stick it into my cache. I do not notify my UI: I am only putting the data into the cache, and it is the cache’s responsibility to notify the UI. We have to set finished = true at the bottom.

So, that is MVC-N, the design implementation that I wish more people would use. First, as I have already discussed, the small discrete units of work make the code much easier to consume. When the next developer comes on board and need to fix a network operation, they do not have to look through long code; they only have to look through discrete code. That is a very big win.

Unit Testing: A Story (27:00)

One of the projects I was brought in to work on was here in San Francisco. The main developer was going on vacation, so they asked me to come in and take care of a couple bugs while he was gone. They had one bug, and they said “if you can fix this one blocker in the next two weeks, we can ship on time”.

The bug was: “on launch, data appears and disappears randomly.” That is all the bug said.

I get the code base, I launch the app, and I don’t touch it. I wait. Some beautifully drawn TableView cells show up, and they disappear. Then different ones show up, and disappear. Then, the data stabilizes with the right answer.

I launched it again and again, with different data in a different order, but it resolved to the same answer. I must have played with it for a half an hou, not even looking at the code yet, only launching it. Then I look at the code:

I found out that they were using Core Data, so I could dive right in. I tried to see if the network code was in the ViewControllers, but they actually had a network controller. They had it all localized in one file; this had to be something simple and cool.

I open the network controller (which they were calling a “data handler”: 12,000 lines of code.

It started innocently. The developer started by writing a method that said “Go get data.” That called a block after the data came back. Inside, he made another block that went and parsed the data into JSON. There, he had another block that stuck it into Core Data. A block within a block within a block. Asynchronous, asynchronous, asynchronous.

When you had one in isolation, it worked. He repeated that pattern and ended up with maybe 15 network calls firing off all simultaneously; 15 asynchronous network calls coming back to 15 asynchronous data processing calls coming back to 15 asynchronous cache calls coming back to15 or more UI updates. Asynchronously. Which data got to the screen first depended on which call got to the server first, which call was back first, and which call had which thread first. It was absolutely a mess.

That is one of the traps of closures and block calls. You do it once, and it works great. You think “it’s an easy, small, discrete unit code: I am going to repeat it.”

We can fix that by doing the operations. We put the operations into a queue. The queue controls how frequent the call is and we have more control over how things fire. With that, we can make unit tests easy, because now we have small discrete units of code. We can initialize that operation; we can fire it manually. We can call OperationStart, and it will fire, and we can listen for that finished method ourselves. We can build unit tests that run on the main thread, wait for them to be finished, and then react to them. We can test each one of our operations in isolation.

We can take that even further. If we inject the URL into the operation, we can even control where that operation goes. It can go to a flat file on disc, and now we have consistent input, and we can test the output.

We can take that even a step further, and we can test it to a known SQL lite file for our cache. Now, we have known input, known original data, and we test the output. We have unit tests that are end to end, testing each one of our network calls.

In block code, this is hard to do; you end up having to mock things out. All we are doing is saying, “you are not talking to the server, talk to this file”, and you are going to use this SQL lite file and now “I am going to test your results and make sure you get your answers back”. Unit tests become simpler.

Network Conditions (32:40)

If you have not heard that Steve Jobs quote, it is one of my favorites. “When my code works right, nobody says a thing. I do not get an ‘attaboy’, I do not get patted on the back. But when my code does not work, all hell breaks loose.”

That is where I live. When the network stuff works great, nobody cares. When my network operations do not work, all of sudden the app is 💩. No matter how many UI engineers we have, no matter how much money we spend on UX, the app is crap because I cannot get data while I am sitting on the subway.

That is the responsibility of the network layer. The persistence layer should say, “I cannot get to the network, here is what I had last”. If you do not understand this concept, quit Facebook on your phone, go into an elevator, wait for the doors to close, and launch Facebook. It does not work. We can fix that by designing our app properly, and having a cache.

Operations can report speeds

On top of that, we can also pay attention to the conditions of our network. When we start an operation, we know when it has started the call NSState. When it finishes, we can call NSState again, and we can subtract one from the other to know how long the operation took. We also know how much data we have back. NSData has a property called length, which tells me how many bytes of data are in the data that we received back from the server. Data / time = bandwidth. If I report the bandwidth to my network controller to aggregate those answers, I have dynamic runtime resolution of my network. I know from second to second how fast my network is.

Control performance via the queue

I can react to that, because NSOperationQueue has a concept called concurrency. I can say, “the network’s great, crank that concurrency up and run as many operations as you possibly can all at once.” Or, I can turn around and say, “the network is crap. Dial it all the way down, do one thing at a time, and set priority in all my NSOperations so my high priority operations will fire first.” Instead of waiting for that cat picture to show up on Twitter, my post actually goes out first before I get that cat picture 😺 coming down.

Network controller can react to conditions

Now my app is more responsive because the timeline request has a higher priority than all that other stuff.

I can take that a step further and say, “the network is bad, I will cancel all low priority operations.” I can pause all requests for images, and I can leave timeline and posting as accessible to the network until the network condition improves.

Central location for canceling of actions

Likewise, we have a central location for cancelling operations. When the app quits, I can kill off those operations. I can say, “the application is ready to exit, cancel all these avatar requests, cancel the timeline request. If you are trying to post, I am going to go ask for more time and let that operation finish”. Even though I have exited the app, your post is still going to go out.

We don’t have to do that. We don’t have to insult our user and force them to wait on our crappy code.

Single decision point for background processing

You know when you try to exit an app, and you hit the home button but nothing happens? Users may be thinking “this phone is 💩. The processor is terrible.” It’s not the phone. It’s someone who wrote code that said “about to enter background”, so the app responds “no, I am going to do this before I return.” The operating system will stop the springboard, waiting for that method to return. It gives them a few seconds before it gives them a kill -9.

They are blocking the main thread of the phone to get their work done, because their work is far more important than you moving on to another application. We do not need to do that. We can be better citizens on the phone, and even though no one will notice, our app will run better, and the operating system will be happier with our app.

Wrap Up (38:33)

• Own your network code.

You notice I did not use AFNetworking, nor did I use ASIHTTPRequest. Write it yourself. I guarantee code you write yourself will be faster than any generic code, that is the law. Whenever you write something that is very specific, it is going to be faster than generics.

• Keep all network code together. Do not put network code in view controllers.

All you are going to do is paint yourself into a corner, and you are begging for a refactor. As soon as you write network code in your ViewController, put a to-do next to it: “I need to refactor this”. Avoid that: do the networking code, put it in one place, and make your life easier.

• All UI should feed from Core Data.

All of your UI should be feeding off of your cache. Your UI should not be talking to the network. In an absolute perfect universe, the UI has absolutely no idea that you are talking to the internet. All it knows is that the cache has the data it wants.

• Disconnect data collection and data display.

Keep the data display and the data collection disconnected from each other: your app becomes more maintainable, easier to mutate, easier to produce, add things to, and reuse.

I am working on a project right now where we have rewritten the app four times, because we do not know what the UI is going to be. We are still developing. However, the backend networking layer is fine no matter what we do. We’ve ripped out the UILayer many times, but we do not have to keep going back and rewriting the network layer, because it is completely abstracted. It has not been touched in three months.

That is my plead to the community. Please stop putting the network code in your ViewController. Treat your network as if it is a first class citizen to your app. When your network code suffers, your entire app suffers. Every time you sit in the Mail app and hit refresh a hundred times, think about that: that is bad networking code. It’s simple to do it better, and once you start doing it, you will not want to do it any other way.

If you have any questions for me, ping me on Twitter, or send me an email. I am happy to answer questions.