Challenges Building a Swift Framework

Apple announced Swift before Realm was first launched. The team quickly realized how big this would be, and fully committed to building a Swift version of the binding. That meant doing a lot of pioneering work. On the way, many new open tools were created, and existing tools were vastly extended and improved. However, there are far more challenges involved in building a framework for Swift. In this talk from MBLTDev 2015, Marius summarizes his team’s experiences, pointing out pitfalls to avoid and giving hints to help you find your sweet spot for development in the fast-evolving Swift ecosystem.

---

Introduction (0:00)

This talk focuses on the challenges of building a Swift framework. I will cover how we overcame these challenges, the experience we got from it, and the lessons learned. I hope it will help you understand how to build open source or third party software on mobile, and how you can do it right.

The topic is particularly interesting for me because I’m working on CocoaPods, a dependency manager. Even Android developers should know about us as of this year, as we had some coverage by Google at this year’s Google I/O event.

For my day-to-day job, I work at Realm.

What Is So Special About Realm? (2:03)

Realm is a fast, embedded database that was built from scratch for mobile. It is used by several Fortune 500 companies, and many Top 100 apps are using it as well.

Realm is special because it’s an object-oriented database that is also fully acid (as every database should be). It’s powered by a multi-version concurrency control algorithm. It has a well-defined threading model through this algorithm, and it has the concept of native links to one or to many relationships in your database. With Realm, they are first class citizens. They’re not tables which are disconnected from each other, so you don’t need expensive join operations. You don’t have object-relational mismatches, or the need to introduce foreign keys. Critically, Realm is also cross-platform. We are on iOS and Android.

We have a core, where we share code that’s written in C++. This is the only real way for a database to share code between both platforms. For Android, a .JAR sits on top of the core. For the .JAR, you only need Java (plus some glue codes for JNI).

What About Swift? (4:35)

The main challenge is how to interface C++ with Swift. Swift 1.0 had very limited interoperability with the Objective-C runtime. Since then, there have been some improvements with Swift 2.0. We realized how important it would be for us and for the developer community to focus on offering Realm Swift.

We had to base Realm Swift on our existing Realm framework, which changed our architecture picture a bit. Essentially, that meant Realm Swift’s framework is built mostly in Swift, but little else changed. How is that practical for us?

When we want to build something on top of something else from a library, there might be some private symbols. This was the case for us. You want to use yourself to build a wrapper on top of that. That is what Realm Swift does here. We didn’t want to expose all the details of the Realm framework just to interoperate in that, and re-expose things to Realm Swift where we are able to use existing private APIs. However, those are not really helpful for end-users of our product, so we had to come up with a way to make that happen.

We came up with a custom Clang Modulemap for our Realm framework:

framework module Realm {
	umbrella header "Realm.h"

	export *
	module * { export * }

	explicit module Private {
		header "RLMArray_Private.h"
		header "RLMObject_Private.h"
		// ...
	}
}

The first part, like the umbrella header and the generic export statements, is just like any other modulemap. The interesting part is the explicit private module. This means there is a specific part of the API that is only available if you import Realm.private, and if you don’t do that, you don’t get those other headers, which are also fixed with _private, so it’s perfectly clear that you shouldn’t use them. They’re also declared in other places as private. However, we can still use just the public portion for Swift. While it is still not transitively made available to the end user, it is not a completely a dedicated module. It’s just a virtual sub present for the front end of the compiler.

A Viable Way to Distribute a Framework (8:04)

Our first idea was to have just one big framework which contained another framework. Then, we would hide away the underlying Objective-C part that you don’t want to deal with if you start out building a new Swift app.

It turns out there were several difficulties with this approach. For one, you would have to mess around with build settings as a user. Even so, this was historically available for a long time on OSX.However, the App Store doesn’t allow this concept at all, with code signing, so it wasn’t usable for us because we want to allow our users to build real apps for the App Store. Essentially, that meant we would have to integrate both frameworks alongside each other, but you will use only the Realm Swift portion of it.

Besides binary distribution, we have dependency managers on our platform. Aside from CocoaPods, there’s another one which came natively in Swift, called “Carthage”. Carthage makes different assumptions: it expects only one framework per repo. We built our stuff all together in one repo, because that makes it easier for us to offer support for our users who have issues. Both parts are just a thin wraparound to the existing Realm framework, so only one repo for our Realm Cocoa team makes sense. With Carthage, though, we had to come up with a workaround.

Essentially, we serve pre-built binaries over GitHub releases. We upload a Carthage framework .zip with both pre-built frameworks in there. This works well because we’re building Realm as a framework over Carthage, so it limits our possibilities of how we are integrate. It is also nice because it removes complexity from the whole thing. Previously, we wanted to build and maintain something in just one way. We have already two ways, with the Xcode project and the CocoaPods pod spec, and this is the next thing.

Pod::Spec.new do |s|
  s.name = 'Realm'
  # ...

  s.module_map = 'Realm/module.modulemap'
end

Pod::Spec.new do |s|
  s.name = 'RealmSwift'
  # ...

  s.dependency 'Realm', "= #{s.version}"
end

We needed to get CocoaPods ready with the framework integration module. We needed to build in modulemap support in CocoaPods. It’s just two independent pod specs. The Realm pod spec declares the Realm modulemap, which is really the most special thing about it, and the Realm Swift pod spec declares the dependency to the first one. It looked like we were almost ready to ship!

Soulful Documentation 🎵 (12:06)

One big thing we needed to care about before shipping was documentation, specifically the API docs. With a new language like Swift, everything is shaken up. We used to have Appledoc for Objective-C. With Swift 1.2, we got another format to document things: restructured text (REST). There was no tooling yet around that, and with Swift 2.0, we got even more changes. We have now Swift-flavored markdown, and for now, at least, that’s the way to go. It’s safe with that, even with Swift 2.1, so we built jazzy.

jazzy is a tool to generate Swift and Objective-C docs. It takes your source code and graphs out your comments by using the parser, and generates nice HTML documentation out of it. You can have your code and your documentation alongside, which makes it easier to maintain if anything changes around, like. If you introduce new parameters or remove existing ones, you have everything all together in one place.

jazzy gets the AST via SourceKitten, a tool built on top of SourceKitService. We built jazzy into CocoaDocs, another CocoaPods project, which automate generates documentation for your pod specs so you don’t have to mess around with that yourself. jazzy is run in the cloud, automatically parses every new version and generates new docs every time you do an update to your library. It then puts them up and makes them available over the CocoaPods website, so as someone who is building a third party library, you don’t need to set them up yourself.

We had other documentation needs as well, like the code examples in our docs. Swift is a rapidly changing programming language, and we often have to go back and update our examples. This was important, because we were getting more and more support requests for users who were making the first attempts using it. I think that’s also important for your readme on GitHub. If your code samples are wrong there, it’s a frustrating experience for your user. So, we built some tooling around it so that we can ensure that the example code we have always builds. This helps us a lot, especially with keeping multiple translations of our docs up to date.

Swift is Evolving Fast: CI Issues (16:17)

The language is changing with every new version, and it turns out that it’s hard to keep up the pace when writing actual apps. At Realm, we have it a bit easy maintaining only two frameworks, but if you write multiple apps, changing from one language to another is a hard change to make. The Apple ecosystem also throws in distractions, like extensions, watch apps, 3D Touch, AppleTV, and all these things you want to deliver to your customers. Swift 1.2, while very different, still remains relevant.

What does that all mean for the ecosystem and further development in such an environment? The elephant in the room is CI; how to support multiple Swift versions at once.

Many developers have come up with a multi-branch model, with a master for the current version, which runs on the Xcode App Store version, while there are other maintained branches such as Swift 1.2 and Swift 2.0.

However, this system has some disadvantages. First, it’s a bit of a hassle to run CI on that with special conditions. You basically just have master as a special branch for CI for releases, and having different branches for releases is a greater hassle. We also don’t want to have to reintegrate new features on different branches, and deal with merge conflicts by trying to automate those things. Our decision was to lump everything together into master:

$ tree -L 2 | 🎩
.
|--- RealmSwift -> RealmSwift-swift2.0
|--- RealmSwift-swift1.2
|    |--- Object.swift
|    |--- …
|    |--- Tests
|
|----RealmSwift-swift2.0
|    |--- Object.swift
|    |--- …
|    |--- Tests
|
|--- RealmSwift-swift2.1 -> RealmSwift-swift2.0
|
|--- …

This has opened up some new problems. We had to start messing around with Xcode. We decided to have multiple directories in our repository with a suffix for the Swift version, according to the Swift files in them and maintain those multiple versions in one stage of the repository in master. The topmost entry just points according to the branch you’re on to alert all the surrounding tooling, like pod specs, to use the right version and select the version with the branch you integrate. For that, we need to put the Realm Swift directory into Xcode as a root Swift directory.

Sadly, it turns out that Xcode doesn’t handle some links that well. Using the built in git functionalities doesn’t work very nicely because it doesn’t find anything. It makes expectations about paths and doesn’t figure out the correct path. If you drag and drop files, it doesn’t work directly and resolves to some links. Finally, we determined to use a Swift version at run time when you’re building the project. It’s not really nice, but it worked for us and helped with some of the problems.

CI Doesn’t Need to Stop at Tests (20:27)

We used CI not only for our source code, but also as our test. We still wanted to be able to run our test on it, so we could run our tests in sync and in different Swift versions. We need to test all integration scenarios as well, which for us meant pre-built binaries, Carthage, and CocoaPods.

For CocoaPods, it’s relatively easy. To do that, there’s a common pod lib lint you can execute, but I won’t dig further into that topic.

CI styleguides are also interesting. With a new language, one of the first problems to tackle is establishing new conventions. For us, we basically adopted the conventions proposed by GitHub. We wanted to make sure that we conformed to those conventions, but we also checked that pull requests coming in from other users maintained the style and didn’t mix up things and format code differently. It’s nice to have some tooling around that as well, because it’s generally nicer to hear feedback about your code from a machine, who is deciding based on rules, than from a human’s opinion. Personal opinion plays a huge role in conventions and style guides.

In order to enforce conventions, we developed Swift Lint, which is our approach to documentation tooling. It uses SourceKitten under the hood to get access to the AST and parsing information, then validates that some code conforms to the conventions we have defined. However, you can configure your own rules, and that’s something we want to set up in our CI as well.

Q&A (23:37)

Q: If we want to use either of the frameworks, is there any chance we might not have to inherit from managed object, or inherit from a super class in the future? And why do we need to do that?

Marius: There are terrific ways around it, but so far you can understand why we would want to do that. Ours has taken the dead weight before us as well, and it makes it really easy for us to define modules in your object module without having separate schema information. That’s the advantage it has for us as of now. One workaround that’s already available is having a schema already encoded in your Realm database, and accessing it only through dynamic accessor objects, which does not rely on having classes. That way, you could directly use them and map your data structures to structs, but there are some problems with that as well.

Q: Earlier this year, Apple announced that Swift would be open source by the end of the year [2015], which might open Swift up to working on other platforms, like Linux, Windows, or Android. Is there any thought around building something like Realm that doesn’t depend on Cocoa and Objective-C, so it could become compatible with native Swift?

Marius: That’s a bit of a difficult question, becoming compatible with native Swift. There are definitely some dependencies in the code base to Cocoa to NSObjects and key value coding, so we support key value observing and some of those nice features. From our current architecture, as you have seen in this talk, it’s far away from being able to use Realm Swift as a standalone, especially on a native Swift stack without having Objective-C involved at all. If you added an open source Objective-C alternative implementation of the standard libraries, maybe, but that’s still not something we’re testing so far. We have to wait for the future, but if it turns out to be something interesting for actual users and developers trying to build stuff, we will definitely investigate whether that would make sense or be a possibility.