Ruthlessly Simple Dependency Management with Carthage

Carthage is a new dependency manager for Objective-C and Swift projects, intended to be the simplest way to add frameworks to a Cocoa application. Carthage works by delegating tasks to Xcode and Git, minimizing new concepts as much as possible, so you can continue to use the tools you’re already familiar with.

This talk explains what Carthage is and how to use it, then dives into the philosophy of ruthless simplicity that inspired the project. We compare and contrast Carthage with CocoaPods, the original dependency manager for Cocoa. Finally, we explore how Carthage is architected under-the-hood, and the benefits we’ve seen from writing it completely in Swift.

To see the full code used in demonstrations you can visit Justin’s GitHub.

---

What is Carthage? (0:00)

Carthage is a dependency manager that is intended to be the simplest way to add frameworks to your Cocoa project. It’s also the first dependency manager to officially support Swift, and it’s completely written in Swift itself.

The Problem (1:22)

Why is dependency management a problem worth solving in the first place? Well, one of the first things we learn as programmers is that we should reuse code as much as possible. So, in order to reuse code, we put it in libraries or frameworks, which are intended to encapsulate the components that we might find useful in other projects. Before CocoaPods and Carthage, you could distribute Cocoa libraries via copied source files, zipped binaries, SVN externals, Git submodules, and Git subtrees, but each of them had their own flaws. A dependency manager is really intended to help you pick the versions of the libraries that you want and then set up each library so that you can start using it. Almost every other modern development platform uses a dependency manager, and it’s usually created by the same team that designs the language.

For the longest time GitHub for Mac imported dependencies exclusively through Submodules and Xcode Subprojects. This actually works pretty well until two or more of your dependencies have a dependency that they share themselves. In our case, we have Mantle and ReactiveCocoa and those are used within several of our other resource libaries. There’s a bit of a problem here because the app ultimately must pick exactly one version of each to link and problems can arise if these different projects expect different versions of Mantle and ReactiveCocoa. Git Submodules don’t help you solve this problem.

Why make another tool?

Why not CocoaPods? (3:47)

We also weren’t interested in using CocoaPods. As library authors we were frustrated with the CocoaPods requirement that you add Pod Specs to all your projects. For us, all this information already exists in Xcode and in Git, so why would we want to duplicate it to another place? As users we were frustrated with CocoaPods’ control of the project set up process. We know how to set up Xcode projects and the automated nature of it took away some of our flexibility.

CocoaPods does have an option called no-integrate, which stops it from manipulating your Xcode project, but then you’re still stuck using its generated Pods project if ReactiveCocoa invents its own Xcode project. You have no way to incorporate that instead of the Pods project. You have to use CocoaPods’ mechanism. We also think CocoaPods’ centralized package list makes library authors’ jobs harder because now we as library authors are responsible for deploying to yet another place.

Our Goals (5:38)

We wanted a tool with a completely different philosophy. What we wanted was pretty simple - we wanted a simple coordinator between Git on the left side and Xcode on the right side. This tool would pick compatible versions for all dependencies, check out dependencies with Git, and then build frameworks with Xcode.

How to use it (6:16)

1. Create a Cartfile
   To set up an application project with Mantle, ReactiveCocoa, and ReactiveCocoaLayout, we specify those three GitHub resposities as ones we want to pull in. For Mantle, the tilde indicates a version compatible with the number specified. We want ReactiveCocoa at least 2.4.7 and exactly 0.5.2 for ReactiveCocoa Layout.

   github "Mantle/Mantle" ~> 1.5
github "ReactiveCocoa/ReactiveCocoa" >= 2.4.7
github "ReactiveCocoa/ReactiveCocoaLayout" == 0.5.2

2. Run Carthage We then run carthage update, which downloads and installs all of those dependencies for us. This will also find if those dependencies have other dependencies themselves and automatically download those for you as well.

3. Link Frameworks
   Once you have those frameworks built on-disc by Carthage, you simply drag them into the linked frameworks and libraries build phase.

4. Strip architectures (iOS only)
   For iOS, there’s one more step because of an existing bug in the App Store that disallows universal framework binaries. Luckily Carthage offers a command that’ll take out the simulator bits and allow you to submit to the App Store.

“Ruthlessly Simple” (8:36)

The prevailing design goal which overrides almost all others, is that of ruthless simplicity. We want a tool that is as simple as possible and will try very hard to avoid any features or any other things that can add significant complexity. Simple and easy are not the same thing. If something is easy, that means it’s familiar or approachable; if something is simple, it means there are fewer concepts or concerns to worry about. There’s a great talk called Simple Made Easy on this subject by Rich Hickey, the author of Clojure.

CocoaPods is easy according to this definition. CocoaPods is all about making it as easy as possible to find and use libraries, but it achieves those goals at the costs of complexity. It becomes easier but less simple. Carthage by contrast, we hope is simple. With Carthage, we really wanted to focus on the simplicity because we believe the benefits of simpler things are enormous.

Simpler tools are easier to understand. Simplicity makes it easier to form a mental model of what’s going on behind the scenes, under the hood. If something goes wrong when you’re using Carthage, for example, understanding what it’s doing for you or what it’s trying to do for you may help you resolve the issue. You don’t need to understand all their arcane stuff that lies within the codebase because there’s a really simple mental model for you to follow. Simpler tools are also easier to maintain and easier to contribute to. By keeping things simple we don’t need to handle as many edge cases. By integrating with other tools like Xcode and Git, we delegate responsibility to them, which means we have less to do ourselves. It’s less complexity because there’s less code for doing fewer things. And of course, it’s a lot easier to implement fixes and new features in a simpler code base than a complex one.

Simpler tools are also more flexible and more composable. It’s impossible to predict all the possible ways that someone might want to use our software. If there’s a use case out there that we’re not anticipating for Carthage, it’s a lot easier to bend the simpler tool to your will than try to get a complex tool to do exactly what you want because there are just fewer moving parts. Finally, simpler tools are enhanced when other tools improve. Because Carthage is so small and modular, whenever we hand off responsibility to Git or to Xcode we can automatically benefit from improvements that are made to those with little or no effort on our part. On the other hand, if we were to duplicate that functionality, the more fragile our whole system becomes and the more we have to lose when those tools change.

How it works

1. Parse the Cartfile (12:22)
   Carthage files are written in a subset of what’s known as the ordered graph data language, or OGDL, but it’s really just a dead simple language that’s useful for configuration files like this. Carthage parses the OGDL into a list of dependencies, determines the type of each of those, and parses any semantic version specified. You can also specify a branch or a specific commit.

2. Resolve the dependency graph (13:22)
   Now that we know the dependencies we want, we have to create a dependency graph that represents the versions of each dependency that we want and the relationships between them. We start by picking a graph with just the dependencies and try the latest allowed version for each of them. Most of the time these are the versions that will end up in the final graph, but not always. If requirements confict, we throw out the graph and try a new graph with the next possible version. You might guess this is inefficient in the worst case but it performs surprisingly well in practice, in part because we throw out things the moment they become invalid and because we automatically terminate at the first valid solution. Carthage isn’t a full constraint solver; it’s very specific to the problem of the dependency resolution.

3. Download all dependencies (15:58)
   Now we’ve picked versions for all our dependencies this graph will get flattened down and written into a file that’s called Cartfile.resolved. Now we need to download each dependency. Carthage manages a global cache of dependency repositories which saves you from having to download the same thing ten times across all of your projects. So, the first step of downloading any dependency is making sure that each cache is up to date. After that, we can use Git Checkout inside the Carthage checkouts folder and we’ll just copy all the repository files into that folder to build later.

4. Build each framework (16:41)
   Once every dependency has been downloaded, then we need to start building them. There’s a Carthage build folder that exists at every level in your project and in every dependency folder, and this get symlinked across all of them. Then we list any framework schemes from the root-most Xcode project that we find. We find things that build a dynamic framework and ignore any schemes that build, like static libraries or app projects. Then we build all those schemes by calling Xcode build, and build that scheme for the architecture that the target platform implies. Once we have them we can combine them with lipo. Finally, we copy the built frameworks into the Carthage build folder. Whenever possible, Carthage will download binaries instead of building from scratch, cutting our build time by almost 70%.

Technical Choices & Future Steps (20:10)

• Dynamic frameworks vs. static libraries
  Frameworks come with significant advantages, the most notable being that they’re complete, self-contained, and ready to be used. There are other ancillary benefits, you can include resources and that sort of stuff, but it’s really the self-containedness that’s huge with Carthage. Notably, it’s also required in order to put Swift code into a library. You can’t create a static library with Swift code.

• Swift vs. Objective-C
  Although Swift itself definitely involves its share of obstacles, we saw huge benefits from using it instead of Objective-C. Type Safety really helps, since React Signal in ReactiveCocoa is very much like NSArray in that you don’t know what’s in it at all. With Swift, we can now know whether it is, perhaps, a signal of build phases, or a signal of string output from Xcode. Value types help us reason about state and mutability, and I honestly think it was much faster to write what we were trying to write, because it is so much easier to prototype and shift directions. Swift also has better modularization with regards to visibility and private, internal, or public things.

• ReactiveCocoa
  This is maybe the most contentious choice of the three. ReactiveCocoa is a framework for programming with strings of values over time. We use ReactiveCocoa extensively in Carthage and it turns out to be an especially great application for it because all of the inherently stream-based stuff that we’re doing just makes it so much easier. For example, networking is inherently communicating with this TCP or HTTP stream. We deal with subtasks in Git and Xcode, and they will stream back output to you. So, all of those applications are really great for ReactiveCocoa. We also wanted to test ReactiveCocoa with the Swift API which is part of the upcoming 3.0 release.

1.0 (24:02)

Carthage today is not at 1.0 yet. The first thing that we need is per-project settings. There’s a lot of things users will be able to configure, like if you want to build for only one platform or if you want to check out your dependecies as Git submodules. Right now, all of the flags we have need to be specified on the command line, and we want to make it possible to include these in one of the Carthage configuration files in your working directory so that all uses of your project can follow the same configuration automatically. We also want to carefully review all our public APIs because 1.0 is the first major release, and after that we really want to commit to backwards compatibility.

Q&A (27:00)

Q: Do you plan on supporting third party libraries that aren’t really set up with frameworks yet?
Justin: There’s a couple of solutions right now. You can fork it. You can add your own framework target which should be relatively straightforward. Or you can use the Carthage checkouts folder and include it in your project with Xcode subprojects or a workspace, like how you would do it pre-Carthage. We don’t plan to do much to support that use case in light of trying to keep with the theme of simplicity. Frameworks do a lot of things for us, and adding support for a different product type will enormously complicate everything that Carthage does.

Q: What kind of things do library authors have to do in order to be Carthage getable
Justin: To explain this quickly, you need to share your schemes from Xcode, which shouldn’t be a big deal if you’re using any kind of CI service. You also need to make sure that it builds and that you can add tags to make it easier to pull on projects. There are some more esoteric things, but it’s all covered in the readme.

Q: Did you consider writing Carthage using anything other than Objective-C or Swift?
Justin: Fundamentally we are Cocoa developers, and in order to open it up for as many Cocoa contributors as possible we really wanted to write it in a language that would be familiar to most. I know I don’t have enough experience to write it in say, Ruby.

Q: CocoaPods has open sourced a generic dependency resolver called Molinillo written in Ruby. Did you consider adopting or forking that and implementing it in Swift so the two could share a common dependency resolver?
Justin: Ruby is actually the reason that we can’t use it because it depends on being centralized, and Carthage is not centralized. In order to use this generic dependency resolver, you have to have information about all versions and libraries upfront. In Carthage it’s kind of hard to see from the steps, but it’ll actually do kind of incremental or lazy Git operations as it’s resolving versions. And so, fundamentally it’s just incompatible with the approach we’re taking to dependency resolution.

Q: It seemed like in the Carthage workflow you’re building the frameworks on a developer’s Mac. How does that fit in with the CI build process?
Justin: Basically, it should be equivalent to CI building your app project. You can install Carthage on your CI server, which is what we chose to do. We don’t want to force everyone to use Carthage, so ReactiveCocoa and Mantle for example have Xcode workspaces and projects. They use Carthage to manage their submodules but then the project itself is also Carthage compatible. There’re a lot of crossover advantages with the non-Carthage, non-CocoaPods workflow and honestly, I think that’s been one of the greatest advantages of it. It has motivated a lot of projects to include Xcode projects and frameworks that might not have otherwise.

Q: How do I, as a developer, debug into the source of my dependencies?
Justin: That gets a bit trickier. There, we need to figure out how to incldue debug symbols or if that happens automatically. You can instruct Carthage to build the debug version or debug configuration of your libraries. This is something we haven’t actually extracted into, like, a script that anyone can use, but one of the things we do is we use binaries by default in Carthage for building. At any point you can just run a script and it’ll switch to a submodule checkout which is handy for things like making a fix, pushing it upstream, and then switching back.

Q: As a library author, what’s your suggestion for how I should be working today? Should I maintain a Podspec and test the manual integration in Carthage?
Justin: As a library author I have never maintained CocoaPods integration. The better answer is if you want a Podspec and add it to the CocoaPods repository trunk, you can test Carthage which will verify that anyone can just include your Xcode project. Carthage is really just a superset of what you get with the basic Xcode functionality.

Q: Does Carthage assume that its repository contains only one frame or target?
Justin: It does not contain only one frame or target, but it does try to build from one project. That project can specify any number of schemes that all build their own frameworks and they’ll all get copied over fine. If you were to distribute two frameworks they would need to be in different repositories.

Q: If you had a library that wasn’t open source, would it be an option to make a GitHub project and put the frameworks as releases?
Justin: In order to authenticate with the GitHub API it uses your Git credential cache that you have set up for HTTPS operations. It’ll read that from the Git store and so you can use private repositories. GitHub enterprise is not currently supported, but that’s on the to-do list.

Q: What are benefits or distinctions between working with dependencies as frameworks vs. static libraries?
Justin: One of the big problems that it solves is avoiding duplicate symbols. This is one of the problems that CocoaPods has solved for the longest time too. Frameworks, dynamic linking in general solves this problem by ensuring that you can link in build time, but you’ll need one copy present at runtime. And then of course, frameworks can include resources and you can just distribute them as is.

Q: Do all the developers on a team have to install Carthage before they can build the project?
Justin: Currently, yes. I think there’s been some work to extract that into a shell script that anyone can run because it’s not Carthage-specific at all. That would solve that problem. Another way to solve this would be to not use Carthage binaries again and just check another source. It really depends on what you and your team are comfortable with.

---