Library-Oriented Programming

Creating a simple, encapsulated, maintainable library may seem like a pipe dream, but it’s actually quite achievable with the right philosophy. In his talk at #Pragma Conference 2015, Justin Spahr-Summers imparts knowledge and motivation for designing usable and user-empowering libraries.

---

Introduction (0:00)

I’m Justin Spahr-Summers, or @jspahrsummers everywhere on the Internet. I currently work at Facebook in London, and I have previously worked at GitHub. You probably know me from any of the open source projects I’ve contributed to, if you know them: ReactiveCocoa, Carthage, and Mantle.

I’m here to discuss library-oriented programming, but my real take-home message is: “Build more libraries!” I’ve seen a lot of applications that could have been decomposed into libraries, but either the motivation or the knowledge on how to do so effectively was missing. I’m hoping to provide you with a little bit of both.

More Libraries, Less Coupling (1:13)

The main reason I want to promote library-oriented programming is because libraries are essential for decoupling your code.

What is coupling? If two components or classes are coupled, they are combined in a way that’s hard to separate. For example, UIViewController is highly coupled to UIView. It depends on the implementation of Views, and it doesn’t make sense to talk about ViewControllers without also talking about Views. You can’t test a UIViewController without also implicitly testing UIView, so they’re coupled.

Coupling is bad because it’s a kind of complexity. This complexity means you can’t understand one without understanding the other, and you also can’t test one without testing the other. This makes change harder, because changing one of these coupled classes has ramifications on everything else that it’s coupled to. Say goodbye to safe refactoring.

Simpler code is better code, and decoupling is one way to simplify it.

Unfortunately, it’s really easy to end up coupling things, intentionally or not, when they exist together in the same codebase. It just starts with a hack here or there, with a quick “Oh, I really need to fix this bug, and I’ll depend on this one implementation detail to do it.” Eventually, there are hacks upon hacks upon hacks that prevent you from ever using those things independently again.

Libraries Introduce Boundaries and Good Friction (2:45)

Libraries are helpful because they force an abstraction boundary on you. It’s harder to couple an application to the implementation details of a library than it is to introduce coupling within the application itself. This is partly because the library has more control over what it exposes, and also because the library can be updated separately from your application. You can’t depend as strongly on details, because they may change in a way that is more outside of your control.

Libraries also introduce good friction: they add the necessity of thinking about versioning, proper abstraction, and how to handle backwards compatibility. This is additional work for the library author, but I think it’s a healthy thing. I think libraries help create this mindset of maintainability.

By separating concerns, libraries help simplify your code. And even though there might be more work involved, the end result is less coupling.

What Does Library-Oriented Mean? (3:50)

I think library-oriented means that we should all be thinking about libraries as early as possible. We should create as many as we need to enforce this separation of concerns. In library-oriented programming, it should be a constant consideration, not just an afterthought.

In order to really achieve this mindset, each library should represent exactly one idea, abstraction, or concept. If it starts to represent more, they need to be split out for simplicity. A library with fewer concerns is inherently simpler.

If, on the other hand, you try to solve multiple different problems with the same component, the complexity increases. It doesn’t increase just linearly; the complexity increases exponentially. Solving exactly one problem keeps your library simpler and helps with the goal of decoupling.

One abstraction per library is also helpful because it keeps things more encapsulated. When multiple abstractions or ideas get combined into one, the result is usually leaky: something that tries to cover all these different use cases, but then fails to be generic enough to capture them completely. If you can represent exactly one abstraction in your library, you can encapsulate all of its messy details and expose a nice high-level interface for the consumer. The result is less leaky because the problem you’re solving is more focused.

If you’re creating a simpler library that has exactly one thing in it, it’s also more maintainable. It will be much easier to maintain than monolithic code bases.

Base. Foundation. Utils. If a library has any of these words, it’s wrong. They become dumping grounds, and all that decoupling you did is a lost cause, because then your utils framework gets coupled in too.

Design Pitfalls (5:56)

It’s not enough to split up your code base into libraries and call it a day. Libraries on their own need to be well-designed to be successful, so here are some common pitfalls or problems that I’ve encountered myself.

Library vs. framework (6:11)

First, figure out whether you’re building a library or a framework. These terms seem to be largely interchangeable, and Apple certainly uses them in a weird way that describes your build product more than your design. However, they can mean very different things. The way I like to think of this, which I imported from Andy Matuschak and other, is the following: “you call a library, but a framework calls you.”

Ruby on Rails is a framework, if you’ve ever had the displeasure of using that: it will call into your code and ask you to do things, and then you return control to it. Cocoa is also a framework in the same way. Libraries, though, are just reusable bits of things that you can call into and use effectively.

As a result, libraries are generally simpler and better encapsulated. I would advise to only build a framework if what you want to do can’t be accomplished with a simple library.

Base classes are fragile (7:04)

This is a reference to the fragile base class problem. Basically, if your library vends a base class, consumers get tightly coupled to your library, and that defeats the whole purpose of what we’re trying to achieve.

I actually made this mistake in Mantle, which is a modeling framework, because it has a required Mantle model base class. It was awful trying to change the implementation, because that could silently break all of our users’ subclasses they had created. Sometimes users want to inherit from other things as well, for example from NSManagedObject. They would want to use Core Data with our framework, which should be a reasonable thing to do, but because there are these conflicting base class requirements, they couldn’t. Alternatively, sometimes you need to inherit from UIViewController. If you’ve ever wanted some of the capabilities of your own ViewController and UITableViewController, you’ve probably seen this problem. Inheritance is fragile, and hard to reconcile.

Protocols, final, and conventions (8:05)

Protocols and composition can replace most of the needs you might have for inheritance. Composition is less fragile, allows you to reuse the behavior of multiple different components, and lets users of the library piece together things how they want to in ways you never even thought of. With inheritance, it’s a lot harder for them to do.

If you’re using Swift, remember to use the final keyword anywhere you haven’t explicitly committed to supporting inheritance. This helps prevent the problems above. It conveys this message through the compiler that things can not be inherited or overwritten.

Lastly, remember the conventions of the platform or whatever else you’re integrating with, especially Apple’s. Make your library look like it belongs. Don’t try to just create something different for the heck of it. The exception would be if you’re doing something outside of the norm, like ReactiveCocoa; it can make sense to break the convention, because you want to break expectations. You want to indicate that this is not going to be consistent in any way with things that you’re used to. Most of the time, though, you want to match the conventions.

Dependency Management 😱 (9:24)

You have to think about dependency management, both for your users and for yourself. If your library depends on something else, how do you bring that in? Or how do your users start bringing the library into their projects? There are many different ways to do it.

The original way of doing this would be Xcode workspaces with Git submodules or Git subtrees, which are always available. They’re always going to be installed on your user’s computer, so your user doesn’t have to go out of their way to do anything with these. They are, however, hard to manage. Xcode workspaces and Git submodules work, but they were never really intended for this purpose. Git submodules duplicate checkouts, for example if a depends on b, and c also depends on b, you can have two checkouts of b in your repository, which is annoying. Most importantly, though, Git submodules don’t really help with versioning. You don’t get version resolution you would with a dependency manager. For instance, if you want 1.0 but not 1.5, there’s no way to express that with a Git submodule, and that can be really difficult, especially with transitive dependency.

CocoaPods (10:41)

CocoaPods is a solution that has a lot of traction. It is widely used in the Cocoa community, and it’s very easy to use.

There’s a love-it-or-hate-it thing, though: some people get really turned off by the idea of using CocoaPods, so for them, if a library only offers that as the dependency management, that can be a reason for them to not use that library.

CocoaPods is centrally managed, so it means that you as a maintainer of this library would never have to be responsible for pushing your changes up to the trunk to manage your Podspec.

If you have an Xcode project with your library as well, it doesn’t pick up configuration from that.

Carthage (11:33)

Carthage is another solution. I helped write it, though, so I’m a bit biased. This builds on your standard Xcode project set up and reads information out of that. It can manage Git submodules for you, so you don’t have to deal with that problem. It can also offer prebuilt binaries. If a library puts up binaries on GitHub, you don’t need to build it every time you want to include it in your project; you can just download a Dot framework and be up and running.

Not mutually exclusive (12:05)

The pros and cons are kind of meaningless because these solutions are not mutually exclusive. A CocoaPods Podspec can live alongside an Xcode project in your repository. So if you’re going to be adding dependencies to your library, I’d suggest using the lowest common denominator. Use Git submodules, and use Xcode. That leaves the door open for Carthage and CocoaPods, but it supports users who don’t want to use either as well. People that want to pull in your project don’t need to be beholden to the solution you personally prefer.

Open Source Success (12:41)

Hopefully, if you’re creating libraries, you and your company might be interested in open sourcing them eventually. Here are some other things that might be important to that success.

Documentation (12:55)

Nobody will be able to use your project without proper documentation. Even on open source projects that I consider to have documented pretty well, this is still the number one request. It comes up no matter how much documentation there is. Users can never get enough documentation.

Use it in a real project (13:12)

If you’re going to open source something, use it in a real project as soon as possible. Don’t open source something that you aren’t using yourself. Otherwise, it’s way too easy to get lost in the weeds, or to lose sight of whether your library is actually practical if you’re not applying it to anything yet.

Empower contributors (13:28)

Find some open source contributors, get people excited, and empower them. Give them permission to do things with the library, give them Git permissions, and also give them your sanction. Say, “You can go forth, you can make releases, you can do all this.” They are your most important community because they have a vested interest in the project and they want to help. They can carry on the project even if you can’t spend as much time doing it.

Personal experience: This definitely happened with ReactiveCocoa 3.0. I was in the process of moving internationally and switching jobs, and the contributors did so much to push it to a final release. It was amazing.

Answer questions (14:05)

If it’s open source, someone, somewhere is going to have a question about it. They may not come to you directly with their question: they may go to Stack Overflow, or put it in the GitHub issues on another project for some reason. Having a question answered directly by the maintainers is awesome, so you should be one of those maintainers!

Build outward (not inward) (14:27)

Encourage people to build on your library, don’t build in it. This creates a richer ecosystem, and it reduces your maintenance burden as a library maintainer. As business development people would say, “create a platform!” Make a really small kernel of something that people can build on to develop their own solutions.

Get to 1.0 (14:49)

To many people, an open source project isn’t real until it’s tagged as 1.0. It might be arbitrary, but it matters to some people, and it’s important. Really, get to 1.0! It means you’ll be introducing fewer breaking changes, and your users care about that. Furthermore, planning for a version 1.0 is a great way to focus your efforts and cut out anything that isn’t absolutely necessary to ship.

Don’t lose sight of the goal: more libraries, less coupling.

Q&A (15:50)

Q: How do you develop libraries and the applications that use the libraries in parallel? Especially with CocoaPods, that can be a pain.

Justin: I do think it’s important to develop applications and libraries in parallel, and you’re right, it’s kind of a pain. No tool today makes this workflow super easy. That was one of our goals with Carthage, and I don’t think we accomplished it fully. However, it is important to iterate in conjunction, because otherwise, you have this thing that starts getting further and further disconnected from its real world usage. For now, I would say to power through the pain, and maybe help create better tools that help you solve this problem. It’s true that it’s more work, but it does enforce that boundary, and I think that is the really valuable thing.

Q: Could you go a little bit into the difference between Carthage and CocoaPods?

Justin: The way I like to present it is that CocoaPods is easy and focuses on ease more than anything, whereas Carthage is simple and focuses on doing as little as possible more than anything. There are long explanations in the readme of Carthage that go into the details further, but effectively, Carthage is built on top of the Xcode and Git submodule workflow. In contrast, CocoaPods adds its own flavor via its own Podspecs and stuff, but makes it really easy for anyone to get up and running, and for any library author to create something without necessarily needing to know how Xcode projects work.

Q: You said that it’s not good to use inheritance if you make a library, but what would you say about extensions introduced in Swift?

Justin: I think extensions are a category of their own. It’s not inheritance because you’re not giving someone else the ability to override your extension, hopefully. It’s more like if you provided a free function or a free method, something that wasn’t attached to a class. People would be able to use that in the same way as an extension; it’s just an extension is nicer syntax. It’s easier to use and get started with, and it makes it feel more natural or built in. Unlike categories from Objective-C, the possibility of collisions is much lower, and isn’t as dramatic or bad. I would say use extensions wherever possible, wherever it makes it feel more natural, where it feels like you’re trying to fit in with the platform.

Q: In our setup, we have libraries that use CocoaPods, and we actually find it very easy to manage. When we want to build a new app, we just say, “I want to import this library, this, this, and this,” and we have API ready. Everything seems to be very simple and working. Is there any reason I should switch, or any advantage that I would get if I move from CocoaPods to Carthage?

Justin: They’re not mutually exclusive. If it were to be open source, I think it’s a question that would matter more, because there are some users who just do not want to use CocoaPods. For them, offering an alternative way to get the same effect is really valuable. But internally, use whatever works.

Q: You mentioned the base class problem with Mantle. How would you do that differently or how would you design that to not have the problem?

Justin: Mantle is a modeling framework, for anyone that doesn’t know. The basic gist is that it’s Objective-C specific, you previously inherit it, and then you have these magic behaviors, like isEqual, and copying, etc. However, the main thing that people were using it for was JSON serialization. We realized if we could just factor out the things that matter for JSON serialization into a protocol and allow anyone to implement that protocol on anything, they would get the JSON adapting layer for free. Now, in Mantle 2.0, someone can add this protocol to an NSManagedObject from Core Data, and then they can use their Core Data object with this JSON translation layer. To answer your question, protocols are a great way to do this, and composition is another. In this case, though, composition doesn’t make a whole lot of sense; there’s not an object that you can pull in that does these things for you, but in other cases it’s like if you’re just using inheritance to give users some code for free, just put it into an object that they can add to their class instead, instead of inheriting from it.

Q: I’ve been using Carthage for a small pet project. Imagine that you have framework A and framework B, and framework A uses the version 1.0 of framework C, and framework B uses 2.0 of framework C. How would you resolve this situation in Carthage?

Justin: The short answer is that you don’t. Carthage disallows you from having that scenario, as you’ve probably experienced. The larger ecosystem answer is that Apple doesn’t have good support for solving this problem. If you link in two versions of framework c like 1.0 and 2.0, they’re completely incompatible; you’re just going to have all these symbols colliding at run time and no real predictability to your application. It would be great if we had a way to solve this problem more thoroughly, but I don’t think the tooling from Apple’s side is there. The short answer is you have to be using compatible versions if you have transitive dependencies or nested dependencies.

Q: There are a lot of libraries, and license management is one of the most critical parts of it. Most of the time, people have good libraries but license information is missing. When we incorporate that in our project, we end up being the ones who get blamed for using it, so how can we manage that?

Justin: The short answer is that if something doesn’t provide a license, in most countries that would be copyrighted and you wouldn’t be able to use it, even if they put it on a place like GitHub. The longer answer is that this is a complex problem that people at GitHub are thinking about a lot, like how to surface license information better, and how to automatically give you that information. As of right now, the GitHub API does try to tell you what license a project has, and I think CocoaPods.org might use that information. As to this kind of larger problem of discovery and like aggregation, I’m not aware of any tools that do that today, although it wouldn’t be too hard to write one.

Q: I really liked the empowering users perspective. In one of our libraries, we outright say, “If you submit a single pull request and it gets merged after like maybe one or two comments, we add you to an organization.” Do you have any other tactics that might help increase people feeling like an active part of the system?

Justin: I think it’s interesting because there’s this great divide between people that are willing to put up their code or issue for deep review, versus people that are really willing to take charge and take responsibility on a project. I’ve had trouble bridging this gap myself, but usually I solve it by direct outreach. If I notice someone has been really active, for example, on a repository, I will personally send them an email and say, “Hey, we want to make you a collaborator. There’s no obligation but feel free to do code reviews, and feel free to merge stuff.” One development that helped with that with ReactiveCocoa was creating the Slack channel, because all the contributors hang out in there. People that want to be contributors but don’t necessarily feel confident enough to do so can jump in and ask questions, but then still drive the actions on GitHub themselves. They can ask people in the chat room, “Hey, this pull request is open, and I think it’s good, but what do you all think?” Then they can go back and merge that pull request so it gives them some piece of the action. I guess talking to people and communication is the best answer I’ve found so far.