Realm Java 3.0: Collection Notifications, Snapshots and Sorting Across Relationships

We’re releasing version 3.0 of Realm Java today. In this release we have enabled sorting across relationships and we’re giving our live collections RealmResults and RealmList a whole lot more life by adding fine-grained collection notifications, so that your app can respond to elements being added, deleted and modified.

Fine-grained Collection Notifications

Up until now, Realm has provided a single shared RealmChangeListener interface that could be registered on our Realm, RealmObject and RealmResults classes. However that interface came with a limitation. It could not provide information about what had changed, only that something had changed. In many cases that was good enough, but in some cases it was not. The most common example being if you wanted to animate changes in a RecyclerView. Due to Realm’s live-updating objects, the solution up until now have been to use a combination of realm.copyFromRealm() and the DiffUtil class from the Android Support Library. This was hardly ideal.

Today, we are shipping fine-grained collection notifications. It is a new interface that can be registered on RealmResults and will provide information about exactly which objects were added, deleted or changed. This will enable you to just refresh those elements that actually did change, making for a much nicer and responsive UI.

We’ve also updated our Android Adapters library, so the RealmRecyclerViewAdapter will now automatically use the fine-grained notifications for smoother animations. Here’s what it looks like in practice, compared to using RecyclerView without collection notifications:

private final OrderedRealmCollectionChangeListener<RealmResults<Person>> changeListener = new OrderedRealmCollectionChangeListener() {
    @Override
    public void onChange(RealmResults<Person> collection, OrderedCollectionChangeSet changeSet) {
        // `null`  means the async query returns the first time.
        if (changeSet == null) {
            notifyDataSetChanged();
            return;
        }
        // For deletions, the adapter has to be notified in reverse order.
        OrderedCollectionChangeSet.Range[] deletions = changeSet.getDeletionRanges();
        for (int i = deletions.length - 1; i >= 0; i--) {
            OrderedCollectionChangeSet.Range range = deletions[i];
            notifyItemRangeRemoved(range.startIndex, range.length);
        }

        OrderedCollectionChangeSet.Range[] insertions = changeSet.getInsertionRanges();
        for (OrderedCollectionChangeSet.Range range : insertions) {
            notifyItemRangeInserted(range.startIndex, range.length);
        }

        OrderedCollectionChangeSet.Range[] modifications = changeSet.getChangeRanges();
        for (OrderedCollectionChangeSet.Range range : modifications) {
            notifyItemRangeChanged(range.startIndex, range.length);
        }
    }
};

Fine-grained collection notifications work just like all other Realm listeners, so if a query was executed asynchronously (for example, by using findAllAsync()), then the fine-grained changeset will also be calculated on a background thread before delivering the result to the UI thread, without the memory overhead of using DiffUtil to calculate what’s changed.

Collection Snapshots

One of the key design principles for Realm is our concept of “live” objects and collections. It means that if you create a RealmResults from a query, it will stay up-to-date whenever the underlying data changes. This is very effective in any reactive architecture, where you can just register a RealmChangeListener and be notified when the query result is updated.

However in some cases, this “live” nature can lead to unexpected problems when looping over a live collection. Take this code for example:

RealmResults<Guest> uninvitedGuests = realm.where(Guests.class)
  .equalTo("inviteSent", false)
  .findAll(); 

for (i = 0; i < uninvitedGuests.size(); i++) {
  realm.beginTransaction()
  uninvitedGuests.get(i).sendInvite();
  realm.commitTransaction();
}

Here, you make a query for all guests not invited, as you want to send an invite to them. If uninvitedGuests were a normal ArrayList, this would work as you expect and all guests would receive an invite.

However because RealmResults are live updated, and because you execute transactions inside the for-loop, Realm will automatically update the RealmResults after each iteration. This would in turn remove the guest from the uninvitedGuests list causing the entire array to shift, but because it is the for-loop keeping track of the current index, this would now be at the wrong position. The result was that only half the guests would receive an invite. Clearly not ideal.

Two solutions existed:

// 1. Wrap the entire loop in a transaction
realm.beginTransaction()
for (i = 0; i < uninvitedGuests.size(); i++) {
  uninvitedGuests.get(i).sendInvite();
}
realm.commitTransaction();

// 2. Loop the list backwards
for (int i = uninvitedGuests.size() - 1; i >= 0; i--) {
  realm.beginTransaction()
  uninvitedGuests.get(i).sendInvite();
  realm.commitTransaction();
}

Neither of these solutions were obvious though, so in Realm Java 0.89 we made the decision to defer all updates of RealmResults to the next looper event (by using Handler.postAtFrontOfQueue).

This had the advantage that the simple for-loop would work as expected, but it came with the price that the RealmResults could now get a little out of sync, e.g. if we deleted the guest objects instead of setting a boolean on them, they would still be present in the query result, but just as invalid objects.

realm.beginTransaction();
guests.get(0).deleteFromRealm(); // Delete the object from Realm
realm.commitTransaction();
guests.get(0).isValid() == false; // You could now get a reference to a deleted object.

This would rarely happen in practise though, so you could say we chose standard iterator behaviour over staying true to core Realm architectural decisions.

In 3.0 we are revisiting this decision by making RealmResults fully live again, but to make it easier to work with live collections in simple for-loops, we’re adding a createSnapshot() method that can provide the current stable list. Simple for-loops should now use this snapshot if they intend to modify data in a loop (note that in most cases doing transactions in a loop is still an anti-pattern).

RealmResults<Person> uninvitedGuests = realm.where(Person.class).equalTo("inviteSent", false).findAll();
OrderedRealmCollectionSnapshot<Person> uninvitedGuestsSnapshot = uninvitedGuests.createSnapshot();
for (int i = 0; uninvitedGuestsSnapshot.size(); i++) {
    realm.beginTransaction();
    uninvitedGuestsSnapshot.get(i).setInvited(true);
    realm.commitTransaction();
}

Iterators will use this snapshot behind the scenes, so they continue to work in the same way you expect them to today:

realm.beginTransaction();
RealmResults<Person> uninvitedGuests = realm.where(Person.class).equalTo("inviteSent", false).findAll();
for (Person guest : guests) {
    realm.beginTransaction();
    uninvitedGuests.setInvited(true); 
    realm.commitTransaction();
}

We are doing this for a number of reasons. One of them being internal refactorings needed to support fine-grained collection notifications. The other is that it means the semantics of Realm classes are all the same since both RealmResults and RealmList are fully live, and it is easier to both reason about them and document their usage. This had caused some confusion in the past, but we believe that fully supporting and documenting the live nature of Realm classes means we can offer users the most powerful APIs.

We are aware that this change might cause some headaches in existing code bases, but this change will only affect your work if you’re performing write transactions inside simple for-loops. In every other case, your code will continue to work as normal, and we hope that you agree it is better in the long term.

Sorting across relationships

Until now, it was only possible to sort RealmResults using “direct” properties. Starting with Realm Java 3.0, you can now sort query results by properties on the other side of an objects many-to-one relationships

For example, to sort a collection of Person objects based on their dog’s age, you can now do the following:

RealmResults<Person> persons = realm.where(Person.class).findAllSorted(“dog.age”, Sort.ASCENDING);

We’ve also shipped a bunch of bug fixes in the last couple releases. To see the full list of changes, check out the changelog.

---

Thanks for reading. Now go forth and build amazing apps with Realm! As always, we’re around on Stack Overflow, GitHub, or Twitter.