Gotta Persist 'Em All: Realm as Replacement for SQLite

Realm is a new player on mobile database scene, and acts as a replacement for SQLite on Android and Core Data on iOS by providing a database and an easy-to-use object-relational mapping-like (ORM) interface. In this talk, Siena Aguayo explains how to use it in your own projects and how it compares to working with SQLite in a beginner-friendly sample app with Pokémon.

---

Overview (00:00)

I am Siena, I work at Indiegogo. I am very happy to be presenting to you “Gotta Persist ‘Em All: Realm as Replacement for SQLite”. This is the first conference talk that I ever gave a year ago. If you have never used Realm before, this will be a great introductory talk. If you have, hopefully you will learn something (I do have pictures of Pokemon, maybe that will keep you entertained at the very least).

I came to Indiegogo via Hackbright Academy, a programming boot camp for women. Because I heart Pokemon, I decided last year that one of my New Year’s Resolutions was going to be to catch all the Pokemon (because I am an adult), and I achieved this. I used some Android apps, to help me out with stats, and they were not great. At the time, I had helped Indiegogo with their first Android app, and I thought, “I am a professional, I can make my own Pokedex app”. That was my inspiration, led me to Realm, and I will be using some examples from the app that I started building.

What Realm Is (02:02)

Realm is an embedded mobile database. The core is written in C++, it runs on iOS and Android, and it is the second-most employed mobile database in the world (after SQLite).

Pros (02:26)

• Realm is easy to use. The API is very fluent; you will pick it up right away, it feels natural to start using Realm.
• Similar to an ORM, it handles your object conversion. You do not have to think, “it is stored in the database like this, I will get it out, transform it, and it will be this object class”. All of that is transparent to you.
• Convenient for creating and storing data on the fly. If you need a data store (e.g. notifications), it is great for this real-time interaction.
• Faster than SQLite.
• The team is very responsive (the Realm team is great heart!). They are all friendly, more than willing to help you out on stack overflow, GitHub. If you are having a problem, chances are someone can help you fast.

If SQLite is one, how does Realm compare to that in terms of speed gains? You can see on some of these calls, it is above 20 times.

Cons (04:03)

• No importing. If you (like me) had a big old SQLite database that you want to turn into Realm, you cannot do that easily.
• Still under active development. The reason I have this under “con” it is also a pro because it is a very responsive team. However, when APIs change dramatically, you sometimes do not have time for that in your development cycle (I am sure we have all been there, using this great library, a big update comes along and you are, “my stuff does not work anymore”… and you have to spend time fixing it).
• Have to create your Realm data with Java on an Android device. I could not set some job running that churns out my Realm database on some server somewhere - I have to create it on a device. This may not even apply to you, but I already had this database, and I was looking to transform it into Realm.
• Not much content online (this is becoming increasingly less true). It is a very active developer community, but it has less content than something that has been around for as long as SQL.
• Cannot access objects across threads (it is on the roadmap).

Missing Features (05:36)

I am happy to report that I took three things off this slide from the last time I gave this talk: null support, easy migrations (it is easier now for you to change your schema to do that), and notifications from when data has changed (great job to the team!).

Things that are still missing:

• Auto-incrementing IDs (you have to do that yourself, if that is something you want).
• Stick a map of things in the database (which I think would be cool).
• Compound primary keys. If you need to look up something based on two columns, you cannot do that as a key (of course you could do it as a separate call).
• Cannot test it with Robolectric. This was the real killer for us at Indiegogo when I was, “we should use this in production!”. I am hoping that Realm will pull through in the end!

Compare And Contrast With SQLite (06:36)

Database Connection and Setup (06:41)

Shipping with an existing database (as I was trying to do). You have a database that lives in a folder somewhere, and you copy it to your Android device. There are things that make this easy: SQLite Asset helper library, and Realm. Realm has a sample that is supposed to let you do this: you already have your Realm file, and you copy it over to your device. That is pretty similar.

To access these things in your Java: DBHelper for SQLite dbHelper.getReadableDatabase() or dbHelper.getWritableDatabase(), and Realm is Realm.getDefaultInstance() (not too big of a deal).

Creating Schema (07:22)

SQLite If you work with SQL, you may have done this at some point:

CREATE TABLE 'pokemon' (
  `id` INTEGER NOT NULL,
  `identifier` VARCHAR(79) NOT NULL,
  `species_id` INTEGER,
  `height` INTEGER NUT NULL,
  `weight` INTEGER NOT NULL,
  `base_experience` INTEGER NOT NULL,
  `order` INTEGER NOT NUL,
  `is_default` BOOLEAN NOT NULL,
  PRIMARY KEle(id),
  FOREIGN KEWspecies_id`) REFERENCES pokemon_species ( id )
);

You create your table, you define your columns, what types they are, any other constraints on them (if they are null or not, set your primary key, foreign key, etc.).

Realm One of the cool things about Realm is that to make a new table, you only need to make a new class that extends Realm objects. Then you define all your member variables, and those are the equivalent of your columns (I almost forgot I had Alakazam here… it is like magic!).

Recently, you can implement a Realm model as interface instead, but you do need to annotate it with Realm class. That allows you to stay with the pattern in your app of how you build your model there.

public class Pokemon extends RealmObject {
  @PrimaryKey tJISVULe int id;  
  private String identifier;
  private int speciesId, height, weight, baseExperience, order;
  private boolean isDefault; private RealmList<PokemonType> types;
  private RealmList<Encounter> encounters;

// constructors, getters, setters
}

@RealmClass
public class Pokemon implements RealmModel

This is the takeaway: in Realm, your models ARE your schema. It makes things readable, easy to keep track.

Reading Data (08:38)

Java with SQLite

I have a cursor, raw SQL, I am selecting a Pokemon, I move the cursor to first… and then I think working with cursors is a real pain (certainly there are ways to not work with the cursor). You can use an ORM, there are many cool database libraries, but for the purposes of this example, this is just straight SQL, and the helpers. It is a little long-winded.

public List<Integer> getPokemonTypeData(int id)
  String intString Integer toString(id);

  Cursor cursor = getData(
  "SELECT type_id FROM pokemon_types WHERE pokemon_id = " + intString +
  " ORDER BY slot");
  cursor.moveToFirst();
  ArrayList<Integer> types = new ArrayList<Integer>();
  while (!cursor isAfterLast()) {
    types.add(cursor getInt( )); // this is the real killer
    cursor.moveToNext();
  }
  cursor.close();

  return types;
}

Java with Realm

Realm is much simpler:

The query return result is a Realm result, and you query Realm. It is a very fluent interface (similar to SQL), very intuitive:

Find all

RealmResults<PokemonType> types =
  realm.where(PokemonType.class)
    .equalTo("pokemonId", pokemon getId())
    .findAll();

If you need to find first, it is very similar. If you need to do this asynchronously, you can do that now:

Find First

PokemonType pokemonType =
  realm.where(PokemonType.class)
  .equalTo("pokemonId", pokemon getId())
  .findFirst();

Writing Data (09:45)

These examples are similar (it is hard to get around doing something like this!). You define all your columns, put in your values, insert the Pokemon into your database, and you close out your transaction.

Java with SQLite

public void addBulbasaur() {
  SQLiteDatabase db dbHelper getWriteableDatabase();

  ContentValues values = new ContentVaLues();
  values.put("id", );
  values.put("identifier", 'bulbasaur');
  values.put("height", 7);
  values.put("weight", 69);

  db.insert("pokemon", null, values);
  db.close();
}

Java with Realm

It is very similar with Realm:

public void addBulbasaur() {
  Realm realm = Realm getInstance();
  realm.beginTransaction();

  Pokemon bulbasaur = realm.createObject(Pokemon class);
  bulbasaur.setId(1);
  bulbasaur.setIdentifieW('bulbasaur');
  bulbasaur.setHeight(7);
  bulbasaur.setWeight(69);

  realm.commitTransaction();
  realm.close();
}

You still have to create the object through Realm, but it is not too big of a deal. You set all the individual values, and you commit the transaction. You can do this in a transaction block (pretty straightforward).

public void addBulbasaur() {
  Realm realm = Realm.getInstance();

  realm.executeTransaction(new Realm.Transaction() {
    @Override public void execute(Realm realm) {
      Pokemon bulbasaur = realm.createObject(Pokemon.class);
      bulbasaur.setId(1);
      bulbasaur.setIdentifier('bulbasaur');
      bulbasaur.setHeight(7);
      bulbasaur.setWeight(69);
    }
  });

  realm.close();
}

Adding Relationships and Complex Queries (10:43)

Let’s talk about some more complex examples: how do you define a relationship?

For this example, I will keep going with my Pokemon metaphor. Imagine that we have some classes: Encounter, LocationArea, and Location. Encounter has a LocationArea, LocationArea has a Location.

Here is the app that I was building (see video). That block represents an encounter; within that, you have a location and a LocationArea.

The raw SQL that I had to use to get this from the database that I was using (this is a bit of a contrived example, but I will put it up here anyway): it was horrible (see video). This is similar to a join across five tables. I have some strings that I pulled out later in the translation files. I was a sad panda otter, like Oshawatt. Not a great time.

Realm came in to the rescue. If I want to find my encounters with the ID of Pokemon 322, that is all I have to do, and I have them:

RealmResults<Encounter> encounters = realm.where(Encounter.class)
                                      .equalTo("pokemonId", )
                                      .findAll();

Encounter encounter = realm.where(Encounter.class)
                        .equalTo("pokemonId", 322)
                        .findFirst();

LocationArea locationArea = encounter.getLocationArea();
Location location = locationArea.getLocation();

If I need the LocationArea, I call .getLocationArea on the encounter; if I need the Location, I call .getLocation on the locationArea. Simple, intuitive, beautiful.

We have more Realm objects, and those Realm objects have Realm objects that they have a relationship to. Realm manages all that for you (you can call stuff on other stuff all day long!).

public class Encounter extends RealmObject {
  @PrimaryKey private long id;
  private int versionId, encounterSlotId, minLevel, maxLevel, pokemonId;
  private LocationArea locationArea;
  private EncounterSlot encounterSlot;
  private int encounterConditionId;

  // constructors, getters, setters
}

What Realm Isn’t (12:10)

Realm says that they are not an ORM because your data is not copied into memory.

I came to a Realm meet-up a year ago in preparation for this talk, and one of the first things they said: Realm is not an ORM. This blew my mind, “What? It looks like an ORM!… and it is not” (you can argue with the Realm guys if you disagree with me).

Typical ORM usage, you have your SQLite table, and that maps to a Java class where you have all your columns defined as your member variables.

I came across this while I was debugging (see video), and it helped me understand why Realm is not an ORM. These are break points that I set. I am inspecting my Pokemon, that is an instance of my Pokemon class inherited from the Realm model, and all these member variables are uninitialized. There are data, but they are not showing up. You can see them as part of the two string (Realm knows what is going on), but it actually has not initialized all of those things into the object (crazy black magic, what they do!).

Shout-Out to Other ORMs (13:36)

I did think it would be unfair of me to not mention that there are other ORMs you can use:

• greenDAO
• Active Android
• Sugar ORM
• ORMLite (Java)
• Cupboard

There is a great talk by Michael Pardo about Android data. I am sure there is more by now (it is been a year since I put that slide up!).

Conclusion (13:54)

Realm is fun to play with. It may or may not meet your needs for production, but it is a great emerging technology. The team is awesome, and I do not know how often you find a library that is fun! Definitely check it out.

Some quick acknowledgements: to Chris, for inviting me, and to Christian Melchlor (also from Realm) for feedback on this talk. Pokemon SQLite database is open-sourced.

Q&A (14:30)

Q: Thank you for the talk. I am an iOS developer. Can you explain back from the very beginning when you said there was Robolectric that was not a good idea at Indiegogo…? Siena: For all of our iOS friends, Robolectric is a testing library for Android, mostly for unit tests. It does this crazy thing with Android where you do not have to run your tests on an Android device. It makes some shadow world of Android, and you run your tests there. It is a unit testing library, but Realm is not compatible with it. People use Robolectric for their unit test layer.

Q: You mentioned something about preferences, and I was wondering if you thought that maybe it is suitable for using for preferences Siena: Probably. If you need to store something more complicated than you can store in System Preferences? Yes. I do not see why not.

Q: How about little pieces of data here and there? Siena: Yes, it is a question of library size. Someone else can answer how big Realm is, where you draw that line of how much you are going to use it versus how much it adds to your binary, your mileage may vary.

Q: I think you had tried migrating from SQLite to Realm. Can you expand on how the experience was? Siena: I wanted, “Realm, here is my SQLite database, go”… but that is not a thing. What I would have had to, on an Android device, go through every table and write every row to Realm. This database is big, I was not going to do that. That was how I started. If anyone else has smart suggestions about what to do in that scenario, I would love to hear them.

Q: You could get something that could read to SQLite, output the Java code that would be runable on your device, then put into Realm. Siena: Write something that would generate the Java code for you: pretty cool!

Q: There was a little component that went online recently, for iOS, to import CSV data straight to Realm Siena: Import CSVs to Realm, but it is for iOS: that is cool. If you want to write something, that is also a great idea.