Remind's Path to Realm

Today’s blogpost is by Alex Leffelman, an iOS engineer at Remind, & was originally posted on the Remind engineering blog. Thanks Alex!

---

Realm

It’s Hack Week at Remind! This week, everybody at the company gets to work on anything tangentially related to our product that we’re excited about. We’re encouraged to work with others on big product ideas, teacher-delight features, tooling improvements, or anything we think would be awesome but won’t make it into our product road map. Harry and I have taken this opportunity to [start to] migrate our iOS application from Core Data to Realm.

Since the inception of Remind’s iOS application, we’ve used Core Data as our local persistence layer with RestKit as our networking and object mapping solution. We’ve been generally unhappy with the maintenance and upkeep of the RestKit project (not to mention its learning curve for new team members), and any iOS developer will tell you that Core Data is no picnic.

By now you’ve probably heard of Realm. All the cool kids are raving about it. We’ve had our eye on it for quite some time, and in the spirit of Hack Week we’re ready to take the plunge.

Approach

We basically have two general approaches for this migration:

• Make a clean cut and port our entire application over to Realm in a single release
• Progressively transition to Realm over time

For some context, the Remind iOS app currently employs 27 Core Data entities used across almost 600 source classes (.m files). It’s not a massive app by any means, but it’s matured over the last two and a half years into a reasonably complex codebase.

The Clean Break

The obvious appeal of doing a clean one-time migration is that it’s theoretically a one-time cost. You come up with a plan, divide the work, and N days later your app is completely converted, at which point your engineering team can go back to shipping product features. The cost of this approach is a massive investment in testing - both manual and automated. Any code base is a culmination of thousands of hours of problem solving and hundreds of subtle bug fixes that represent the current stability of your application. As with rewriting anything from scratch, there is danger in throwing away that accumulation of knowledge. Even with a solid suite of unit tests and a rigorous manual testing period, it’s unwise to assume you’ve covered all permutations of your users’ data and UI interactions. In the world of iOS development where getting a new version of your app reviewed and released with a major crash fix can easily take a week or more, this kind of migration is a huge risk.

The Slow Burn

Naturally, the opposite is true of a progressive rollout. Maintaining two databases means one can be put entirely behind a server-controlled kill switch; if your first release reveals a crash in an edge case affecting 1% of users (or more!), you can simply shut off the malignant code path and try again next time. On the other hand, nobody thinks the idea of maintaining two parallel databases sounds like much fun. Data synchronization is the main concern: If different views are written to reflect models from different database, the data for both should be consistent. If a change is made in one database, it must be made in the other database at the same time. If we discover an inconsistency in the data, how do we resolve it? This can get hairy fairly quickly. Thankfully, we’re writing a client application that strictly respects our servers as the single source of truth, so this danger is minimal.

After much discussion within our team, we decided to progressively integrate Realm and maintain parallel databases until the migration is complete. This week, the brilliant team behind Realm was kind enough to sit down and discuss our plan with us; their general feeling from working with other teams is that the Clean Break strategy has a better chance of success. However, after explaining our situation and proposal, we eventually got the nod of cautious approval that we needed to get started (thanks, Realm team!). Here’s our plan:

1. Generate a parallel Realm database that mirrors all Core Data transactions
2. Update all of our view controllers and business logic to use Realm entities instead of Core Data entities
3. Replace RestKit with a basic networking-only library, or simply start using NSURLSession
4. Unlink RestKit and Core Data from the application
5. Celebrate

Parallel Databases

As briefly described in our previous post regarding our RestKit setup, we can find a single entry point for our kill-switch-protected database migration in our APICommunicator class. When we get a successful callback from RestKit, we get access to the raw JSON data from the operation and send it off to be mapped into Realm.

- (void)handleSuccessfulRequest:(RDAPIRequest *)request operation:(RKObjectRequestOperation *)operation result:(RKMappingResult *)mappingResult
{
  ...
  NSData *data = operation.HTTPRequestOperation.responseData;
  id json = [NSJSONSerialization JSONObjectWithData:data options:0 error:NULL];

  id<RDAPIResponseMappingProvider> *mapper = [request mappingProvider];
    
    if ([self shouldPerformRealmMapping])
    {
      [self.realm beginWriteTransaction];
    [mapper parseJSONObject:json mapper:mapper realm:self.realm];
    [self.realm commitWriteTransaction];
    }
    ...
}

We added the (currently optional) parseJSONObject:mapper:realm function to our object mapping protocol to handle converting the JSON from our API into new Realm entities. Our shouldPerformRealmMapping function makes sure our kill-switch is off and that our mapper has been updated to implement Realm mapping. With this simple addition, once we implement the new method on all of our mappers, we’ll be inserting matching objects into both databases every time we make a network request.

Interoperability

In order to progressively convert our business logic to use Realm entities instead of Core Data entities, we need a way for the two schemes to cooperate. To illustrate the problem and our solution, let’s look at the following user interaction:

The first screen is displaying a Core Data representation of a group. It then presents the icon chooser, which has been updated to display Realm entities. When an icon is selected, it sends a network request to update the group, and that network request again expects to work with a Core Data entity.

Since our network layer is mapping data into both of our databases simultaneously, all we have to do to make this work is provide some plumbing to fetch the corresponding Realm entity for a Core Data entity, and vice versa.

@interface RLMObject (CoreDataMatching)

+ (NSString *)coreDataEntityName;
- (NSPredicate *)coreDataMatchingPredicate;
- (id)coreDataEntity:(NSManagedObjectContext *)context;

@end

@interface NSManagedObject (RealmMatching)

+ (NSString *)realmEntityName;
- (NSPredicate *)realmMatchingPredicate;
- (id)realmEntity:(RLMRealm *)realm;

@end

The base implementations of these functions cover the vast majority of our entities, doing simple class name conversions (RDUser <-> RDRealmUser) and retrieving them by their ID in the remote database (SELF.remoteId == <id>).

With that simple plumbing in place, our screens can cooperate with other classes whether or not they’ve been migrated to Realm yet.

@implementation RDEditGroupViewController

- (IBAction)pressedAvatar:(id)sender
{
    RDRealmGroup *realmGroup = [self.group realmEntity:self.realm];
    RDSelectAvatarViewController *viewController = [[RDSelectAvatarViewController alloc] initWithRealmGroup:group];
    ...
}

@end

@implementation RDSelectAvatarViewController

- (void)updateGroupWithAvatar:(RDRealmAvatar *)avatar
{
  RDGroup *coreDataGroup = [self.group coreDataEntity:self.managedObjectContext];

  RDUpdateGroupRequest *request = [[RDUpdateGroupRequest alloc] initWithGroup:coreDataGroup];
  request.avatarId = avatar.remoteId;

  [self.apiDispatcher dispatchRequest:request completion:^{
    ...
  }];
}

@end

Safety First

One downside to putting the whole Realm setup behind a kill switch is that we can’t just update the interfaces to our view controllers to use Realm entities; at least in the beginning, the view controllers we migrate will have to be able to operate on either kind of entity. That puts us in the position where we have to either build a common protocol on top of every pair of corresponding entities, dirty our interfaces to accept either kind of entity and use loads of conditional logic, or reimplement Realm versions our view controllers whole hog so we can switch versions at runtime. All of these have their own amount of tedious overhead.

It’s our view that the main risks associated with this migration will be rooted in the mapping layer of our stack. Our Core Data mapping solution is the result of more than 2 years of iteration and expansion, learning how to coax RestKit to do the right thing, handle edge cases, and avoid crashes. The danger in replacing that logic is that we don’t know all the ways some user’s data permutation can confuse or explode our mapping code, and we don’t know how much we’ll have to insulate Realm’s basic property mapping logic.

To mitigate that risk, our initial release of this migration will bluntly do the Realm mapping in a @try/@catch scope and fail silently from the user’s perspective. When we catch the exception we’ll post an event to our servers with the request, mapper, and the raised exception for the mapping we haven’t handled correctly. This will allow us to monitor the stability of our Realm database population and make corrections as we flesh out the rest of the migration.

Since our main concern is the mapping layer, we’ll probably only reimplement a handful of view controllers with the new Realm models while the kill switch is integrated. We’ve already been reimplementing our views using Masonry, so we’ll likely end up with a combination of conditional logic and fully reimplemented classes, depending on the view controller’s complexity. Once the mapping layer is complete and stabilized, it should be safe for us to remove the kill switch and simply update our existing interfaces to use Realm entities.

Fingers Crossed

We had the pleasure of hearing about a few truly horrible gradual migration stories from the team at Realm. Is this the start of another? Only time will tell. We’ll post updates to this blog with any major paint points of the migration, as well as a post-mortem about our experience when it’s complete. Until then, wish us luck!