The Realm Object Server synchronizes Realms between devices, provides authentication and access control services for Realms, and offers “serverless” event processing through Realm Functions.

Install Realm Object Server

If you’ve already followed the instructions in Getting Started, you’ve already installed the Realm Object Server, and can skip ahead to Configuration.

There are instructions for upgrading and uninstalling later in this document.

macOS

On macOS, the Realm Object Server is part of the macOS Realm Bundle:

Open the realm-mobile-platform folder. You can start the Realm Object Server by double-clicking start-object-server.command.

Linux

Under Linux, Realm’s package repositories are managed through PackageCloud. For information on installing and running Realm Object Server on Ubuntu, Red Hat Enterprise Linux, or CentOS, visit our Linux installation page.

Cloud Deployment

For information on deploying Realm Object Server on Amazon EC2, Azure, or Digital Ocean, read the documentation on Cloud Deployment.

Running the Server

• RHEL/CentOS 6
• RHEL/CentOS 7 & Ubuntu
• macOS

sudo service realm-object-server status
sudo service realm-object-server start
sudo service realm-object-server stop
sudo service realm-object-server restart

sudo systemctl status realm-object-server
sudo systemctl start realm-object-server
sudo systemctl stop realm-object-server
sudo systemctl restart realm-object-server

The Realm Dashboard

The Dashboard is a browser-based administration application for the Realm Object Server that provides the following:

• Dashboard: System status, including real-time displays of data rate in/out, bytes in/out, open connections, and open Realms
• Realms: Paths, permissions, and owners of Realms synced to this Object Server, with the ability to “drill down” and display the models and contents of individual Realms
• Users: User information and management for this Object Server, including granting and removing administrative privileges
• Functions: Management and entry of Realm Functions
• Logs: System logs for the Object Server, with selectable detail level

To access the Dashboard on the same machine at the default port (9080), open a new browser window and go to http://localhost:9080/. If your server is not your local machine, replace localhost with its hostname or IP address.

The first time you access the Dashboard, you will be prompted to create an admin user by supplying an email address and password. After registering, you can log in using those credentials.

Realm Studio

Realm Studio is our premiere developer tool, built so you can easily manage the Realm Database and Realm Platform. With Realm Studio, you can open and edit local and synced Realms, and administer any Realm Object Server instance. It supports Mac, Windows and Linux.

You can use the Connect to Object Server option in Realm Browser in one of two ways:

1. Specifying the username/email address and password of an administrator account
2. Providing the admin token generated by the Realm Object Server

The admin token is stored in a text file on the server. Under Linux, view the token with:

cat /etc/realm/admin_token.base64

On macOS, the token file is in the realm-object-server folder, inside the Realm Mobile Platform folder. Navigate to that folder and view the token:

cd path-to/realm-mobile-platform
cat realm-object-server/admin_token.base64

Configuring the Server

Realm Object Server includes a configuration file in YAML format, configuration.yml, that comes with sensible defaults for a development environment. The configuration file can be edited to customize the installation and make it production-ready.

The default location of the configuration file depends on your platform:

• Linux: /etc/realm/configuration.yml
• macOS: realm-object-server/object-server inside the Realm Mobile Platform folder

Paths in the configuration file can be either absolute or relative. If relative, they are resolved according to the current working directory. We recommend always using absolute paths to avoid confusion.

Mandatory Settings

We recommend specifying these in configuration.yml, but they can also be given as command line arguments.

The mandatory configuration defines:

1. An existing directory where Realm Object Server will store all its files,
2. Paths to a key pair used for securing access to Realm Object Server to only allow authenticated clients to access it. The keys given must be in PEM format and they must be a matching pair.

Network

In its default configuration Realm Object Server runs internal backend services as well as a proxy module. The backend services are only reachable from the host running the server, and all traffic to them are passed through the proxy module which listen on all interfaces. The following diagram describes the default configuration:

Backend Services

Realm Object Server internally starts two backend services. The sync service is responsible for core synchronization functionality, and listens on port 27800. The http service handles requests for authentication and the dashboard, and listens on port 27080. By default, the services only listen on the host’s IPv4 loopback address (127.0.0.1), and requests are routed through the proxy module.

Should you wish to disable the proxy module and let backend services listen to all IPv4 and IPv6 interfaces, this can be achieved by setting the appropriate listen_address to ::. Similarly, the listening ports can be changed by modifying the appropriate listen_port.

Proxy Module

The included reverse proxy module dispatches all incoming traffic to the appropriate backend service. This proxy is capable of handling both HTTP, WebSocket, HTTPS and Secure WebSocket traffic (but exclusively traffic destined for the Realm Object Server, this is not a general purpose proxy). In the default configuration, the HTTP proxy listens on all IPv4 and IPv6 interfaces on port 9080, but the HTTPS proxy is disabled. Both variants of the proxy can be enabled simultaneously.

If you wish, you can disable the included proxy module and replace it with a standalone proxy installation, or expose the backend services on the network for direct client access.

If you wish to enable the HTTPS Proxy, you must provide certificate and private key files in PEM format by setting the proxy.https.certificate_path and proxy.https.private_key_path options. The certificate cannot be self-signed.

If you would like to be able to connect to Realm Object Server on a port lower than 1024, such as the default HTTPS port 443, you can forward traffic to the port that Realm Object Server is listening on:

sudo iptables -A PREROUTING -t nat -p tcp --dport 443 -j REDIRECT --to-port 9443

Here is an example of a working proxy configuration with HTTPS:

proxy:
  https:
    enable: true
    listen_address: ‘::'
  http:
    enable: false

Logging

Realm Object Server logs messages information about its state and progress. Logs are written to the console or to a file depending on the logging.path setting; the logging detail depends on the logging.level setting.

The value for logging.level can be one of 9 possible values:

The default value for logging.path depends on your platform:

• Linux: var/log/realm-object-server-log
• macOS: stdout on the terminal that launched Realm Object Server

Upgrading

Upgrading macOS

To upgrade the macOS version of Realm Object Server, simply install a new version of the macOS bundle.

Upgrading Linux

If you are upgrading from a version prior to 1.0.0-BETA-4.8, we have renamed the package. Please uninstall the old version, then install the new package.

To upgrade Realm Object Server on Linux, simply upgrade the Realm Object Server package.

• RHEL/CentOS 6
• RHEL/CentOS 7
• Ubuntu 16.04

# Stop the service before upgrading
sudo service realm-object-server stop

# Install a new version if available
sudo yum -y upgrade realm-object-server-developer

# Restart the server to continue operation
sudo service realm-object-server start

# Stop the service before upgrading
sudo systemctl stop realm-object-server

# Install a new version if available
sudo yum -y upgrade realm-object-server-developer

# Restart the server to continue operation
sudo systemctl start realm-object-server

# Update the package definitions
sudo apt-get update

# Stop the service before upgrading
sudo systemctl stop realm-object-server

# Install a new version if available
sudo apt-get upgrade realm-object-server-developer

# Restart the server to continue operation
sudo systemctl start realm-object-server

Access Control

Realm Object Server includes access control mechanisms to restrict which users can sync which Realm files. In order to grant a user access to a Realm, Realm Object Server authenticates the user to verify their identity, then authorizes that the user has the correct permissions to access the Realm.

• Realm Object Server uses configurable authentication providers to validate user credentials and authenticate users. A client connects to the server using some user credentials, and these are given to an appropriate provider for validation. If the credentials are valid, the user is granted access to Realm Object Server. A new user account is created if the credentials are not coupled to an existing account.
• After the client is granted access as a user, it can request read or write access to a specific path identifying a Realm. If the user has the requested access permissions for the given path, the user is authorized to access the Realm, and normal sync operations can begin.

Authentication

Authentication providers allow developers to utilize third-party authentication mechanisms to control access to Realm apps that go beyond Realm’s own username/password mechanism. Realm supports Facebook, Google, and Apple’s iCloud.

These third-party providers currently operate at the single-server level, and will need to be set up on each server that needs to make use of a given authentication mechanism. The setup is done by providing a set of key-value pairs that identify the authentication mechanism and the required keys/secrets.

This information goes into the auth.providers section of the Realm configuration.yml file.

Below are examples that demonstrate how to integrate each of these third-party authentication providers and where applicable links to the developer pages for those services. Complete examples of all authentication providers can be found in the Realm Object Server configuration.yml file.

Username/Password

This standard authentication method is always enabled.

Google

After getting Google OAuth2.0 API Access, uncomment the google provider and add your client ID:

google:
  clientId: '012345678901-abcdefghijklmnopqrstvuvwxyz01234.apps.googleusercontent.com'

Facebook

To enable Facebook Authentication, just uncomment the facebook provider:

facebook: {}

Note that the Realm Object Server must be able to access Facebook APIs to validate received client tokens.

Azure Active Directory

Uncomment the azuread provider and add your Directory ID from the Azure Portal (under “Properties”):

azuread:
  tenant_id: '81560d038272f7ffae5724220b9e9ea75d6e3f18'

iCloud

You will need to create a public key for the Realm Object Server to access CloudKit. The steps are slightly different for Linux and macOS. Note that iCloud client support is only available on Apple platforms: iOS, macOS, tvOS, and watchOS.

• macOS
• Linux (all flavours)

Log in to Apple’s CloudKit Dashboard and select your application. In the left-hand side of the dashboard, select “API Access”, then select “Server-to-Server Keys”. Select “Add Server-to-Server Key”. Give the new key a name and paste in the public key generated above. Click “Save.” After a few seconds, a key will be generated and displayed in the “Key ID” section at the top of the page.

Uncomment the cloudkit provider in the configuration file and enter the Key ID, private key path, container ID and environment information:

cloudkit:
  key_id: '0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef'
  private_key_path: 'cloudkit_eckey.pem'
  container: "iCloud.io.realm.exampleApp.ios"
  # For production deployment on the App Store, you must specify 'production'
  environment: 'development'

Custom Authentication

The Realm Object Server supports the ability to use external authentication providers. This allows users to authenticate users against legacy databases or APIs, or integrate with providers that are not supported out-of-the-box by the Realm Object Server. This section of the manual will walk you through the process of writing and setting up a third-party authentication provider.

The basics of authentication in the Realm Object Server is simple: when a client tries to login, it provides credentials to the server. The server checks those credentials, typically against an external server or API, and grants rights to the client based on the response.

Custom authentication providers can be installed anywhere. By default, the server is configured to look in /usr/local/lib/realm/auth/providers. An authentication provider takes the shape of a Node.js script which uses one of our development APIs.

Writing an authentication provider

Create the directory indicated above if it doesn’t exist yet, and create a file with a .js extension. The filename itself can be anything you want, although we recommend using something significant for the type of authentication provider you are implementing (e.g., github.js). In this example, we will create the FooAuth sample provider, which asks an external server to verify the token given by the client.

// fooauth.js

/**
 * This will be called when the server is started.
 *
 * It should return the constructor for the authentication provider.
 *
 * @param {object} deps - The dependencies passed from the running server
 *                        to this implementation.
 * @param {function} deps.BaseAuthProvider - the base class to use
 * @param {object} deps.problem - a set of exceptions to throw on failure
 * @param {object} deps.models - the models of the admin-Realm
 * @returns {function}
 */
module.exports = function(deps) {
  return class FooAuthProvider extends deps.BaseAuthProvider {

    // return name of this custom authentication provider
    static get name() {
      return 'custom/fooauth';
    }

    // ensure required default options are set (optional function)
    static get defaultOptions() {
      return {
        server: 'https://my.auth.server.example',
      }
    }

    constructor(name, options, requestPromise) {
      super(name, options, requestPromise);

      this.httpMethod = 'GET';
    }

    // perform the authentication verification
    verifyIdentifier(req) {
      // The token submitted by the client
      const token = req.body.data;

      // options for the HTTP request
      const httpOptions = {
        uri: `${this.options.server}/api/auth?token=${token}`,
        method: this.httpMethod,
        json: true,
      };

      // make request to external provider and return result
      return this.request(httpOptions)
        .catch((err) => {
          // Please see src/node/services/problem/http_problem.js
          // for other error cases
          throw new deps.problem.HttpProblem.Unauthorized({
            detail: `Something bad happened: ${err.toString()}`,
          });
        })
        .then((result) => {
          // assume user ID value is in the `userId` JSON key returned
          return result.userId;
        });
    }
  };
}

We begin by defining a new class for our authentication provider, FooAuthProvider, which extends the BaseAuthProvider class. This gives us access to a promise-based HTTP request. Let’s walk through the methods our provider must define.

• The name() function simply returns the name of the authentication provider. It must begin with custom/, to avoid potential name conflicts between first-party and third-party providers.

  You can use this name to define options in the configuration.yml file for the Realm Object Server.

  custom/fooauth:
    server: 'https://newserver.example'

  These options become available to your provider under this.options (e.g., this.options.server).

• The defaultOptions() function is optional. It returns default values for any configuration file variables your code requires, in case they’re not defined.

• The constructor receives the name, options, and the request Promise. It can set instance variables, verify configuration file values, and perform any other initialization tasks.

• The verifyIdentifier() function is the heart of your provider. It’s called with the HTTP request, and it should return a unique user ID that will be used by Realm to either create a new user on the Realm Object Server or log in an existing user, depending on whether the user is already in the system or not.

  In this example, we fill in httpOptions using the server configuration variable and the httpMethod variable defined in the constructor, then make an HTTP request to our provider with that information. On an error, we throw an exception from deps.problem; on success, we return the user ID.

For a real world example, see our AWS Cognito provider on Github:

• cognito.js (Server side)
• AWSCognitoAuthenticationProvider.swift (Part of realm-loginkit)

Configuring the authentication provider

Now that custom authentication provider has been written, the configuration needs to be updated to tell the server to load it. This is done by adding a few lines to the global configuration file:

# configuration.yml
auth:
  providers:
    # The name of a custom authentication provider must start with `custom/` to
    # differentiate it from ROS-vendored authentication providers.
    custom/fooauth:
      # This refers to the `fooauth.js` file.
      implementation: fooauth
      # This is an option, as defined in the custom authentication provider.
      auth_server: 'https://another.server.example'

By simply adding this configuration, the ROS will automatically load the JS file and enable the authentication provider.

Using the custom provider on the client

All the client SDKs support custom authentication out of the box. It is only a matter of sending the correct information using the right login function. For more details, consult the documentation for your language SDKs, and find “Authentication” under the “Sync” heading.

Authorization

Each Realm managed by the Realm Object Server has access permissions that can be set on it; permissions are the fundamental authorization control for the server. Permissions are set on each Realm using three boolean flags:

• The mayRead flag indicates the user can read from the Realm.
• The mayWrite flag indicates the user can write to the Realm.
• The mayManage flag indicates the user can change permissions on the Realm for other users.

Permissions for a Realm can be set on a default basis and a per-user basis. When a user requests access for a Realm, first the Realm Object Server checks to see if there are per-user permissions set for that user on that Realm. If there are no per-user permissions set for that user, the default permissions for the Realm are used. For example, a Realm might have mayRead set true by default, with individual users being granted mayWrite permissions.

By default, a Realm is exclusive to its owner: the owner has all permissions on it, and no other user has any permissions for it. Other users must be explicitly granted access.

Note that admin users are always granted all permissions to all Realms on the Realm Object Server.

The Management Realm

All access management operations are performed by writing special permission change objects to a Management Realm. This is a special synchronized Realm; objects created in it are effectively requests to the server to change default and per-user permissions on other Realms.

Consult the documentation for the binding you’re using to learn how to access the Management Realm. Generally, it’s retrieved by calling a method on a user object (e.g., SyncUser.managementRealm() for Swift).

Permission Change Objects

A user adds PermissionChange objects to the Management Realm to change permissions; both per-user permissions and default Realm permissions are set the same way. The user must have the mayManage permission for the Realm they’re trying to manage to make modifications.

There are five fields in PermissionChange objects. The first two are required; the permission fields are all optional:

• realmUrl (string): the URL of the Realm, or * for all Realms owned by the requesting user
• userId (string): the identify of the user whose permissions are being modified, or * for the Realm default permissions
• mayRead (boolean): read access to the Realm
• mayWrite (boolean): write access to the Realm
• mayManage (boolean): management permission for the Realm

If one of the may* flags is not set in the PermissionChange object, it will remain unchanged. Per-user permissions always take higher priority than default permissions, so changing the default permissions for a Realm will not remove permissions already set on individual users.

The PermissionChange object will be modified by the Realm Object Server after it attempts to apply the permissions. If the permission change succeeds, the statusCode field will be populated and set to 0, and a description may be added in the statusMessage field. If an error occurs, statusCode will be set to a value greater than 0, and the error message will be in statusMessage.

For the specifics of how to create and use PermissionChange objects, please consult the documentation for your client binding.

Permission Offer and Response

The PermissionOffer and PermissionOfferResponse mechanism allows flexible sharing management between users. A PermissionOffer object functions similarly to a PermissionChange object, but creates a token that can be sent to another user to grant them permissions. The steps are, roughly:

1. Create the PermissionOffer object and add it to the Management Realm.
2. Wait for the server to process the object and populate a token property on it.
3. Send the token to another user via an external method: email, chat, your own API, etc.
4. The receiving user creates a PermissionOfferResponse object with the token and adds it to their Management Realm.
5. Wait for the server to process the object and return a successful status.
6. Use the realmUrl property now populated on the PermissionOfferResponse object for syncing with that Realm.

Consult the documentation for your client binding for details.

The Admin Realm

Ultimately, changes made via the Management Realm are made to a single global, authoritative store containing the complete set of permissions for all Realm files and users controlled by a Realm Object Server instance. This Admin Realm is only modifiable by the Object Server itself, and is not accessible to Realm Mobile Database applications.

High Availability

One of the unique aspects of Realm Mobile Platform is that it is designed to be offline-first. Architecturally, clients retain a local copy of data from the server and read or write directly to their copy. The client SDK then sends the changes asynchronously when network connectivity is available and Realm Object Server automatically integrates these changes in a deterministic manner. Such a system is inherently highly available from the perspective of the client. Reads and writes can always occur locally irrelevant of the network conditions or server operations.

Realm is currently working on a clustering solution that supports automatic failover. This functionality will be available in the Enterprise Edition. To learn more and get early access, please contact us.

Backup

The Realm Object Server provides one or two backup systems, depending on your Edition.

• The manual backup system is included in all versions of Realm Mobile Platform via a command line utility. It can be triggered during server operations to create a copy of the data in the Object Server.
• The continuous backup system, available only in the Enterprise Edition, is a backup server running alongside the main process in the Realm Object Server. It continuously watches for changes across all synchronized Realms and sends changes to one or more backup clients.

Both systems create a directory of Realms from which the Realm Object Server can be restarted after a server crash. The backed up data includes the user Realms, all user account information, and all other metadata used by the Realm Object Server. Both manual and continuous backups can be made from a running Realm Object Server without taking it offline.

The following documentation applies to the Manual Backup feature. For more about Continous Backup, read Continuous Backup in the Enterprise Edition documentation.

The manual backup is a console command that backs up a running instance of the Realm Object Server. This command can be executed on a server without shutting it down, any number of times, in order to create an “at rest” backup that can be stored on long-term storage for safekeeping. In order to be as agnostic as possible regarding the way the backup will be persisted, the command simply creates a directory structure containing everything the server needs to get started again in the event of a disk failure.

It is recommended that the resulting directory is compressed and sent to an off-site location, such as Amazon S3 or Online C14.

Because Realms can be modified during the backup process, the backup command uses Realm’s transaction features to take a consistent snapshot of each Realm. However, since the server is running continuously, the backed up Realms do not represent the state of the server at one particular instant in time. A Realm added to the server while a backup is in progress might be completely left out of the backup. Such a Realm will be included in the next backup.

To run realm-backup at the command line, type:

realm-backup SOURCE TARGET

• SOURCE is the data directory of the Realm Object Server (typically configured in /etc/realm/configuration.yml under storage.root_path).
• TARGET is the directory where the backup files will be placed. This directory must be empty or absent when the backup starts for safety reasons.

After the backup command completes, TARGET will be a directory with the same sub directory structure as SOURCE and a backup of all individual Realms.

Server Recovery From A Backup

If the data of a Realm Object Server is lost or otherwise corrupted, a new Realm Object Server can be restarted with the backed up data. This is done simply by stopping the server and copying the latest backup directory (e.g., the TARGET directory in the manual backup procedure, or the storage.root_path directory specified by a continuous backup client) into the Realm Object Server’s data directory (e.g., the SOURCE directory in the manual backup procedure). After the backup has been fully copied into the Realm Object Server’s data directory, the server may be restarted.

Client Recovery From A Backup

Any data added to the Object Server after its most recent backup will be lost when the server is restored from that backup. Since Realm Mobile Database clients communicate with the Realm Object Server continuously, though, it’s possible for them to have been updated with that newer data that’s no longer on the server.

In the case of such an inconsistency, the Realm Object Server will send an error message to the client, and refuse to synchronize data. The error messages relating to synchronization inconsistencies are:

• 207 “Bad server file identifier (INTENT)”
• 208 “Bad client file identifier (IDENT)”
• 209 “Bad server version (IDENT, UPLOAD)”
• 211 “Diverging histories (IDENT)”

If the client receives one of these error messages, the only way to continue synchronization from the server is to start over: erase all the data in the local copy of the Realm that received the error, and re-sync with the server. This way, the client will have all the data (and only the data) on the Realm Object Server and will be in a consistent state.

Note that if this happens, the client will lose any data that only existed on its local, inconsistent copy of the synchronized Realm (i.e., data that was added to it between the time the Realm Object Server crashed and was restored from its backup). The best way to minimize this potential data loss is to use the continuous backup system. If you can’t do that, run the manual backup system frequently.

Troubleshooting

Verify Port Access

Certain factors may impact external (non-localhost) access to the Realm Object Server’s synchronization facility, and the Realm Dashboard. In order to enable access, it may be necessary to open port 9080 on your server(s).

Using the standard Linux tools, these commands will open access to the port:

• RHEL/CentOS 6
• RHEL/CentOS 7
• Ubuntu 16.04

sudo iptables -A INPUT -p tcp -m tcp --dport 9080 -j ACCEPT
sudo service iptables save

sudo firewall-cmd --get-active-zones
sudo firewall-cmd --zone=public --add-port=9080/tcp --permanent
sudo firewall-cmd --reload

sudo ufw allow 9080/tcp
sudo ufw reload

If your environment does not use your distribution’s standard firewall, you will have to modify these instructions to fit your environment.

Configuration Errors

Generally the default settings will work well for most installations. Most settings inside the Realm Object Server configuration file are commented out (the system has pre-programmed default values) and show examples of how they can be to customized. If the configuration file has been customized and there is a problem, error messages will be written to the default log location (in the terminal window under macOS or /var/log/realm-object-server.log on Linux systems).

If the Realm Object Server will not start, an error in the configuration file is usually the cause. To debug the config file, you can use the built-in checker by starting a terminal and using:

realm-object-server --check-configuration /etc/realm/configuration.yml

If running under macOS, replace the configuration file path with the path to your configuration.yml.

When an error occurs, the server’s parser will print a detailed error message stating the problem and possibly highlighting the line the error occurred on. Once all errors are corrected, the server will start; stop the server by pressing ^C and then start it normally by rebooting the system or restarting the Realm Object Server using systemd on Linux or the start-object-server.command on macOS.

Error messages are usually self-explanatory, but here are some potential hints:

• To be able to run Realm Object Server, the following settings are mandatory:
  • Storage root directory: Where Realm Object Server should store all its files. Please set storage.root_path or give the --root CLI argument. The directory must exist before ROS can start.
  • Authentication key pair: For securing access to Realm Object Server to only authenticated clients. Please set auth.private_key_path or give the --private-key CLI argument, and please set auth.public_key_path or give the --public-key CLI argument. The keys given must be in PEM format and they must be a matching pair.

• The configuration for listening ports must not be overlapping, meaning that listen_port values must be unique in the file. Additionally, the ports must not be bound in another process.

• The HTTPS proxy will only start if given paths to a valid certificate and private key in proxy.https.certificate_path and proxy.https.private_key_path. The certificate and private key must be in PEM format and they must be a matching pair. The certificate cannot be self-signed.

Operational Errors

Occasionally it can be useful for debugging purposes to check the Realm Object Server logs - which are in the terminal window on macOS or /var/log/realm-object-server.log on Linux systems. Realm Object Server produces two specific classes of error and warning diagnostics that may be useful to system admins and developers.

Session Specific Errors

• 204 “Illegal Realm path (BIND)” Indicates that the Realm path is not valid for the user.

• 207 “Bad server file identifier (IDENT)” Indicates that the local Realm specifies a link to a server-side Realm that does not exist. This is most likely because the server state has been completely reset.

• 211 “Diverging histories (IDENT)” Indicates that the local Realm specifies a server version that does not exists. This is most likely because the server state has been partially reset (for example because a backup was restored).

Client Level Errors

• 105 “Wrong protocol version (CLIENT)” The client and the server use different versions of the sync protocol due to a mismatch in upgrading.

• 108 “Client file bound in other session (IDENT)” Indicates that multiple sync sessions for the same client-side Realm file overlap in time.

• 203 “Bad user authentication (BIND, REFRESH)” Indicates that the server has produced a bad token, or that the SDK has done something wrong.

• 206 “Permission denied (BIND, REFRESH)” Indicates that the user does not have permission to access the Realm at the given path.

Conflict Resolution

One of the defining features of mobile is the fact that you can never count on being online. Loss of connectivity is a fact of life, and so are slow networks and choppy connections. But people still expect their apps to work!

This means that you may end up having two or more users making changes to the same piece of data independently, creating conflicts. Note that this can happen even with perfect connectivity as the latency of communicating between the phone and the server may be slow enough that they can end up creating conflicting changes at the same time.

What Realm does in this case is that it merges the changes after the application of specific rules that ensure that both sides always end up converging to the same result even, though they may have applied the changes in different order.

This means that you no longer have the kind of perfect consistency that you could have in a traditional database, what you have now is rather what is termed “strong eventual consistency”. The tradeoff is that you have to be aware of the rules to ensure the consistent result you want, but the upside is that by following a few rules you can have devices working entirely offline and still converging on meaningful results when they meet.

At a very high level the rules are as follows:

• Deletes always win. If one side deletes an object it will always stay deleted, even if the other side has made changes to it later on.

• Last update wins. If two sides update the same property, the value will end up as the last updated.

• Inserts in lists are ordered by time. If two items are inserted at the same position, the item that was inserted first will end up before the other item. This means that if both sides append items to the end of a list they will end up in order of insertion time.

Primary Keys

A primary key is a property whose value uniquely identifies an object in a Realm (just like a primary key in a conventional relational database is a field that uniquely identifies a row in a table). Primary keys aren’t required by Realm, but you can enable them on any object type. Add a property to a Realm model class that you’d like to use as the primary key (such as id), and then let Realm know that property is the primary key. The method for doing that is dependent on the language you’re using; in Cocoa, you override the primaryKey() class method, whereas Java and .NET use annotations. (Consult the documentation for your language SDK for more details.)

Once a Realm model class has a primary key, Realm will ensure that no other object can be added to the Realm with the same key value. You can update the existing object; in fact, you can update only a subset of properties on a specified object without fetching a copy of the object from the Realm. Again, consult the documentation for your language SDK for specifics.

For more information, read the Primary Keys Tutorial.

Strings

Strings are special in that you can see them both as scalar values and as lists of characters. This means that you can set the string to a new string (replacing the entire string) or you can edit the string. If multiple users are editing the same string, you want conflicts to be handled at the character level (similar to the experience you would have in something like Google docs).

Counters

Using integers for counting is also a special case. The way that most programming languages would implement an increment operation (like v += 1), is to read the value, increment the result, and then store it back. This will obviously not work if you have multiple parties doing it simultaneously (they may both read 10, increment it to 11, and when it merges you would get the result of 11 rather than the intended 12).

To support this common use case we offer a way to express the intent that you are incrementing (or decrementing) the value, giving enough hints to the merge that it can reach the correct result. Just as with the strings above, it gives you the choice of updating the entire value, or editing it in a way that conveys more meaning, and allow you to get more precise control of the conflict resolution.

Custom Conflict Resolution

The standard way to do custom conflict resolution is to change the value into a list. Then each side can add its updates to the list and apply any conflict resolution rules it wants directly in the data model.

You can use this technique to implement max, min, first write wins, last write wins and pretty much any kind of resolution you can think of.

Installing/Upgrading

• RHEL/CentOS 6
• RHEL/CentOS 7
• Ubuntu 16.04

# Set the token variable:
export PACKAGECLOUD_TOKEN=<the token you received via email>

# Setup Realm's package repository
curl -s https://$PACKAGECLOUD_TOKEN:@packagecloud.io/install/repositories/realm/ros-professional/script.rpm.sh | sudo bash

# Install the Realm Object Server
sudo yum -y install realm-object-server-professional

# Enable and start the service
sudo chkconfig realm-object-server on
sudo service realm-object-server start

# Set the token variable:
export PACKAGECLOUD_TOKEN=<the token you received via email>

# Setup Realm's package repository
curl -s https://$PACKAGECLOUD_TOKEN:@packagecloud.io/install/repositories/realm/ros-professional/script.rpm.sh | sudo bash

# Install the Realm Object Server
sudo yum -y install realm-object-server-professional

# Enable and start the service
sudo systemctl enable realm-object-server
sudo systemctl start realm-object-server

# Set the token variable:
export PACKAGECLOUD_TOKEN=<the token you received via email>

# Setup Realm's package repository
curl -s https://$PACKAGECLOUD_TOKEN:@packagecloud.io/install/repositories/realm/ros-professional/script.deb.sh | sudo bash

# Update repositories
sudo apt-get update

# Install the Realm Object Server
sudo apt-get install realm-object-server-professional

# Enable and start the service
sudo systemctl enable realm-object-server
sudo systemctl start realm-object-server

Upgrading to Enterprise Edition

If you are upgrading to the Enterprise Edition of the Realm Object Server, you must first uninstall the Developer/Professional Edition. Then, using the access token you’ve received from us via email, follow these steps according to your distribution:

• RHEL/CentOS 6
• RHEL/CentOS 7
• Ubuntu 16.04

# Set the token variable:
export PACKAGECLOUD_TOKEN=<the token you received via email>

# Setup Realm's package repository
curl -s https://$PACKAGECLOUD_TOKEN:@packagecloud.io/install/repositories/realm/ros-enterprise/script.rpm.sh | sudo bash

# Install the Realm Object Server
sudo yum -y install realm-object-server-enterprise

# Enable and start the service
sudo chkconfig realm-object-server on
sudo service realm-object-server start

# Set the token variable:
export PACKAGECLOUD_TOKEN=<the token you received via email>

# Setup Realm's package repository
curl -s https://$PACKAGECLOUD_TOKEN:@packagecloud.io/install/repositories/realm/ros-enterprise/script.rpm.sh | sudo bash

# Install the Realm Object Server
sudo yum -y install realm-object-server-enterprise

# Enable and start the service
sudo systemctl enable realm-object-server
sudo systemctl start realm-object-server

# Set the token variable:
export PACKAGECLOUD_TOKEN=<the token you received via email>

# Setup Realm's package repository
curl -s https://$PACKAGECLOUD_TOKEN:@packagecloud.io/install/repositories/realm/ros-enterprise/script.deb.sh | sudo bash

# Update repositories
sudo apt-get update

# Install the Realm Object Server
sudo apt-get install realm-object-server-enterprise

# Enable and start the service
sudo systemctl enable realm-object-server
sudo systemctl start realm-object-server

Load Balancing

The Realm Object Server has extremely high performance; even a single instance can handle tens of thousands of concurrent connections. However, for applications that expect millions of concurrent connections, multiple Object Servers are required. The Enterprise Edition of Realm Mobile Platform offers integrated load balancing, which allows you to horizontally scale to any number of Object Server instances.

Installation

Each node in a load balancing cluster runs a standalone instance of the Realm Sync Worker. This synchronization worker normally runs as part of Realm Object Server, synchronizing Realms between the server and Realm Mobile Database Clients; in its standalone form, it performs load balancing for an Object Server. Each load balancer runs its own instance of Sync Worker, all communicating with the Object Server instance.

The realm-sync-worker package must be installed from the Realm enterprise repository.

• RHEL/CentOS 6
• RHEL/CentOS 7
• Ubuntu 16.04

# Set the token variable:
export PACKAGECLOUD_TOKEN=<the token you received via email>

# Setup Realm's Enterprise package repository. No need to do it if you already installed the ROS Enterprise
curl -s https://$PACKAGECLOUD_TOKEN:@packagecloud.io/install/repositories/realm/ros-enterprise/script.rpm.sh | sudo bash

# Install the Realm Object Server standalone sync worker:
sudo yum -y install realm-sync-worker

# Replace /etc/realm/token-signature.pub with the one from the Realm Object
# Server.

# Enable and start the service
sudo chkconfig realm-sync-worker on
sudo service realm-sync-worker start

# Set the token variable:
export PACKAGECLOUD_TOKEN=<the token you received via email>

# Setup Realm's Enterprise package repository. No need to do it if you already installed the ROS Enterprise
curl -s https://$PACKAGECLOUD_TOKEN:@packagecloud.io/install/repositories/realm/ros-enterprise/script.rpm.sh | sudo bash

# Install the Realm Object Server standalone sync worker:
sudo yum -y install realm-sync-worker

# Replace /etc/realm/token-signature.pub with the one from the Realm Object
# Server.

# Enable and start the service
sudo systemctl enable realm-sync-worker
sudo systemctl start realm-sync-worker

# Set the token variable:
export PACKAGECLOUD_TOKEN=<the token you received via email>

# Setup Realm's Enterprise package repository. No need to do it if you already installed the ROS Enterprise.
curl -s https://$PACKAGECLOUD_TOKEN:@packagecloud.io/install/repositories/realm/ros-enterprise/script.deb.sh | sudo bash

# Update repositories
sudo apt-get update

# Install the Realm Object Server
sudo apt-get install realm-sync-worker

# Replace /etc/realm/token-signature.pub with the one from the Realm Object
# Server.

# Enable and start the service
sudo systemctl enable realm-sync-worker
sudo systemctl start realm-sync-worker

Configuration

Both the Realm Object Server and each individual sync worker must be configured for load balancing operation.

In the Realm Object Server’s configuration.yml file, the sync.servers section describes the cluster of load balancers. Each balancing node must be identified with a unique ID, which can be any string as long as it is a valid filesystem path component.

Example:

sync:
  servers:
  - id: 'alpha'
    address: '::'
    port: 7800
  - id: 'beta'
    address: 'beta.local'
    port: 7800
  - id: 'delta'
    address: 'delta.local'
    port: 7800

If the address field designates an IP address representing all interfaces, i.e. “0.0.0.0” (or “::” if using IPv6), this indicates to the Realm Object Server that a local synchronization worker should be started listening on the IP address of localhost (“127.0.0.1” or “::1” respectively).

The sync workers use a separate configuration file, sync-worker-configuration.yml, which is a subset of the Object Server’s file. The file is created by default on worker startup. Check the network section in the sync-worker-configuration.yml and ensure that the listen_address line is uncommented and correct:

network:
  sync:
    listen_address: '0.0.0.0'

An address of 0.0.0.0 will listen on all interfaces; you may alternatively specify a specific interface by IP address. The server must be listed in the sync.servers section of the Object Server’s configuration file.

The example configuration files provided as part of the Realm Object Server installation provide additional details.

Failover

Realm Object Server supports manual failover, but not automatic failover. You must have Load Balancing configured with sync workers to use failover, and set up redundant sync workers and/or Object Servers as described below in “prerequisites.”

Switching to a backup Sync Worker

Prerequisites

Following the directions in the Continuous Backup backup client section, configure a backup client to point to a machine running sync-worker. The network.server_address setting in the configuration file (object-server-backup-client.yml) should be set to the sync worker’s IP address or domain name.

Install sync-worker on the same machine as this backup client, but do not start the sync-worker.

Set up the sync worker following the configuration instructions under “Load Balancing,” with the following change: Set storage.root_path to point to the same folder the backup client is writing to.

Make sure that the token-signature.pub file is the same for both the backup client and the running sync worker it’s backing up from.

Triggering failover

If a sync-worker machine fails and you want to bring the failover machine online, take the following steps. (“Failover machine” refers to the machine set up in the prerequisite section that will take over for the failed sync worker.)

• Stop the backup client process on the failover machine. (On Ubuntu servers: systemctl stop realm-object-server-backup-client)
• Start the sync worker on the failover machine. (systemctl start realm-sync-worker)
• Open the configuration.yml file of the Realm Object Server and change the IP address (or host name) of the sync-worker that failed to the IP address (or host name) of the failover machine.
• Restart the Realm Object Server. (systemctl restart realm-object-server)

Note: Some monitoring services can automatically update DNS records with the IP address of a failover machine when the primary machine is unreachable, thus eliminating the need for the third step; alternatively, you could update the DNS records manually in place of the third step. In either case, the DNS of the sync worker would need to be given an extremely short TTL (time-to-live) value. Changing the DNS record, whether automatically or manually, rather than editing configuration.yml may make your Object Server unreachable for a period of time (up to the TTL value, and potentially longer if a customer’s name server does not respect that setting).

Switching to a backup Realm Object Server

This requires load balancing with at least one sync-worker configured and running.

Prerequisites

Install the Realm Object Server on a secondary machine (this will be the “failover machine”), but do not start the server. Only one Object Server can be running in a cluster at a time.

Ensure the following files are identical between the running Realm Object Server and the backup server:

• configuration.yml
• token-signature.pub
• token-signature.key
• Any custom authorization providers

Triggering failover

When the primary Realm Object Server goes down, simply start the Object Server on the failover machine. On Ubuntu servers: systemctl start realm-object-server

Upon startup, the failover machine will load synced data from the sync-workers. Client Realm Mobile Databases should be pointing to the load balancer, thus needing no changes to their configurations. On Amazon Web Services, this could be accomplished by setting up an ELB with a health check running on the primary Object Server.

High Availability

One of the unique aspects of Realm Mobile Platform is that it is designed to be offline-first. Architecturally, clients retain a local copy of data from the server and read or write directly to their copy. The client SDK then sends the changes asynchronously when network connectivity is available and Realm Object Server automatically integrates these changes in a deterministic manner. Such a system is inherently highly available from the perspective of the client. Reads and writes can always occur locally irrelevant of the network conditions or server operations.

Realm is currently working on a clustering solution that supports automatic failover. This functionality will be available in the Enterprise Edition. To learn more and get early access, please contact us.

Continuous Backup

The Enterprise Edition of Realm Object Server includes a backup server running alongside the main process. This server monitors the state of a running Object Server, sending changes asynchronously to one or more backup clients. All clients maintain a slightly delayed copy of the server state.

You may continue to use the Manual Backup system described in the main Object Server documentation with the Enterprise Edition. (Manual backup is the only backup system provided for the Professional and Developer Editions.)

Backup Server

The backup server is installed as part of the realm-object-server and the realm-sync-worker. To enable it, run:

• RHEL/CentOS 6
• RHEL/CentOS 7
• Ubuntu 16.04

sudo chkconfig realm-object-server-backup-server on
sudo service realm-object-server-backup-server start

# Starts automatically when starting the realm-object-server
sudo systemctl enable realm-object-server
sudo systemctl start realm-object-server
sudo systemctl status realm-object-server-backup-server

sudo systemctl enable realm-object-server
sudo systemctl start realm-object-server
sudo systemctl status realm-object-server-backup-server

The configuration is done in the Realm Object Server YAML file (/etc/realm/configuration.yml) under the backup section. The keys used by the backup server are:

If backing up a node in a cluster of sync workers, the backup server will read the sync worker configuration file instead, located at /etc/realm/sync-worker-configuration.yml.

Backup Client

The Realm Object Server Backup Client is started on every machine where a backup is needed. The backup client is configured in a separate YAML file.

To install the backup client, you need to setup the enterprise repository as described above.

• RHEL/CentOS 6
• RHEL/CentOS 7
• Ubuntu 16.04

# Set the token variable:
export PACKAGECLOUD_TOKEN=<the token you received via email>

# Setup Realm's Enterprise package repository. No need to do it if you already installed the ROS Enterprise
curl -s https://$PACKAGECLOUD_TOKEN:@packagecloud.io/install/repositories/realm/ros-enterprise/script.rpm.sh | sudo bash

# Install the Realm Object Server Backup Client
sudo yum -y install realm-object-server-backup-client

# Edit the configuration to point to an ROS with backup enabled. The default configuration assumes ROS and the Backup Server live on the same machine.
vim /etc/realm/object-server-backup-client.yml

# Enable and start the service
sudo chkconfig realm-object-server-backup-client on
sudo service realm-object-server-backup-client start

# Set the token variable:
export PACKAGECLOUD_TOKEN=<the token you received via email>

# Setup Realm's Enterprise package repository. No need to do it if you already installed the ROS Enterprise
curl -s https://$PACKAGECLOUD_TOKEN:@packagecloud.io/install/repositories/realm/ros-enterprise/script.rpm.sh | sudo bash

# Install the Realm Object Server
sudo yum -y install realm-object-server-backup-client

# Edit the configuration to point to an ROS with backup enabled. The default configuration assumes ROS and the Backup Server live on the same machine.
vim /etc/realm/object-server-backup-client.yml

# Enable and start the service
sudo systemctl enable realm-object-server-backup-client
sudo systemctl start realm-object-server-backup-client

# Set the token variable:
export PACKAGECLOUD_TOKEN=<the token you received via email>

# Setup Realm's Enterprise package repository. No need to do it if you already installed the ROS Enterprise.
curl -s https://$PACKAGECLOUD_TOKEN:@packagecloud.io/install/repositories/realm/ros-enterprise/script.deb.sh | sudo bash

# Update repositories
sudo apt-get update

# Install the Realm Object Server
sudo apt-get install realm-object-server-backup-client

# Edit the configuration to point to an ROS with backup enabled. The default configuration assumes ROS and the Backup Server live on the same machine.
vim /etc/realm/object-server-backup-client.yml

# Enable and start the service
sudo systemctl enable realm-object-server-backup-client
sudo systemctl start realm-object-server-backup-client

The configuration file is in /etc/realm/object-server-backup-client.yml.

The keys of the backup client configuration are

Synchronous Backup

The Enterprise Edition of Realm Object Server can maintain synchronous backups using the sync-worker system described in Load Balancing. Two sync workers act as a primary and a replica, communicating over the network. The replica receives incremental updates of all Realms after the primary has updated its own copies. If the primary sync worker fails, the Realm Object Server can continue by restarting the replica as the new primary and redirecting the proxy. With synchronous backups, client changesets uploaded to the primary are sent by the primary to the replica. After the replica saves its copy, it sends an acknowledgment back to the primary, which only then acknowledges the changeset’s successful save to the client.

Note: Instead of (or in addition to) synchronous backup, you may continue to use the Manual Backup system described in the main Object Server documentation with the Enterprise Edition. (Manual backup is the only backup system provided for the Professional and Developer Editions.)

One significant advantage of the synchronous backup system is that client resets should never happen, as long as the primary and replica don’t experience simultaneous data loss. A client reset happens when the local copy of a client’s Realm has a newer version than the copy on the Object Server the client is synchronized with. This can happen if the server crashed and was restored, but the local client made unsynced changes after the crash and before the restore. Under this circumstance, the local Realm is entirely replaced with the copy on the server.

The client Realm Mobile Database won’t erase any of its changeset data until after it receives acknowledgment of the successful upload. So, when you’re using the synchronous backup system, data won’t be lost unless both primary and replica fail at the same time: the client won’t get the acknowledgment until both the primary and replica have copies of the data.

In addition to preventing data loss, the synchronous backup system guarantees consistency of internal IDs and version numbers, so a primary can be replaced with a replica without affecting any clients.

Operating Modes

The backup system is built into the Sync Worker instead of being a separate system. A Realm Sync Worker can be started in one of four different modes:

• Primary with no replica: the standard sync worker that does not perform backup.
• Primary with asynchronous replica: A standard sync worker, with the addition that the server delivers its data to a connected replica. A primary with an asynchronous replica will never wait on its replica, and clients are always served immediately with minimal latency—however, there’s no longer a data loss guarantee, as the client receives a data write acknowledgment before the data is replicated.
• Primary with synchronous replica: A sync worker that ensures its data is copied successfully to a replica before it downloads and acknowledges data to a client. This system has slightly higher latency than a primary with an asynchronous replica, but protects against data loss.
• Replica: A sync worker acting as a replica for a primary sync worker. Replicas will not accept connections from clients.

Primaries use the same port for listening for Object Server connections and replica connections. At most one replica is allowed to be connected to a primary sync worker; if a new replica connects while an older replica is connected, the new replica will be given preference, and the older connection will be closed.

Replicas will keep trying to connect to their primaries until they succeed. If the connection is lost, the replica will continue to try to connect indefinitely.

It makes no difference whether a primary or a replica starts first.

Protocol

The Sync Workers have a backup protocol that allows exchange of data in two distinct ways: whole Realm transfer, or a history of incremental changes. Whole Realm transfer is most efficient when the replica starts from an empty slate. Rather than replaying a potentially very long history, the replica receives the whole Realm at once. The Realm backup system uses whole Realm transfer when the replica starts from scratch. Once the replica and primary are synchronized, the workers will switch to using incremental updates.

When a Realm is changed on the primary, the primary sends incremental updates to the replica. If the updates are lost due to a crash or network disconnect, they will never be regenerated. Instead, when the connection is re-established, the primary and replica will check the version numbers of their respective copies of the Realms, and if the primary is ahead of the replica, the whole Realm will be transferred again. (This should be a rare occurrence unless the environment has an excessive amount of network failures.) A sync worker can be reconfigured to be a primary with a replica at any point without migration.

Each Realm is backed up independently of all other Realms, so a downloading Sync client using one Realm will never be blocked by backup activity in other Realms.

Merging and Performance

There are two possible ways to deal with changeset merging on the replica: the replica repeats the merge computation performed by the primary, or the replica receives the result of the merge computation from the primary. The first method uses less network bandwidth, while the second uses less CPU time on the replica.

The Realm backup system lets the replica perform the merge except in cases where the merge computation is very slow. The primary measures the time to perform the merge computation, and in case of a long computation, the whole Realm is transferred to avoid a repeat of the slow merge computation.

Latency is (slightly) higher for a primary with synchronous backup than for the other types of primary. However, due to pipelining, a synchronous primary performs fewer write transactions than the other types of primaries in certain use cases. This could lead to situations where a primary with synchronous backup has higher performance than the two other types. (Of course, the two other types could be rewritten to use pipelining.)

Security

The Sync Workers communicate over unencrypted TCP by default; they can be configured to use TLS over TCP. TLS allows the backup connection to go through unprotected network intermediaries with minimal risk of eavesdropping and tampering.

A replica must present a shared secret to the primary in order to prove that it is allowed to receive backup data. The shared secret is a string and is configured in the configuration files for the primary and replica. A system administrator must generate shared secrets and enter them in the configuration files for the primary and replica.

Failover

When either the primary or replica fails with data loss, the system must be restarted in a new configuration.

The simplest case is when the replica fails. In that case, a new replica must be started and pointed towards the primary. In the case of primary failure, the replica must either be restarted as a primary or its root directory must be transferred to the location of a new primary. A new replica must also be started somewhere.

The proxies must always be configured to point to the primary.

Configuration

As with load balancing, sync workers are configured in a configuration.yml file. To switch configuration, a sync worker must be restarted with a new configuration file.

The new configuration options for backup: are:

• master_slave_shared_secret: the secret shared between the primary and its replica
• master_address and master_port: the connection information a replica uses to connect to its primary
• operating_mode: One of the four operating modes described above. Valid values are:
  • master_with_no_slave
  • master_with_asynchronous_slave
  • master_with_synchronous_slave
  • slave

If operating_mode isn’t specified, the default is master_with_no_slave, creating a normal sync worker with no replication.

If the sync worker is configured ad a replica, the configuration must specify the address/port of its primary (master). The primary address/port specified in the replica’s configuration file is the same as the listening address/port in the primary’s configuration.

The shared secret is sent by the replica to the primary during initial connection. The primary only proceeds with the backup exchange if the shared secrets match.

Here are sample configuration.yml files for a primary and replica, as well as a recovered primary and replica (that is, when a replica is manually promoted to take over as primary).

Primary

storage:
  root_path: root-dir-original-primary

network:
  sync:
    listen_address: '1.2.3.4'
    listen_port: '9001'

backup:
  operating_mode: 'master_with_synchronous_slave'

  master_slave_shared_secret: 'A secret'

Replica

storage:
  root_path: root-dir-original-replica

network:
  sync:
    listen_address: '5.6.7.8'
    listen_port: '9002'

backup:
  operating_mode: 'slave'

  master_address: '1.2.3.4'
  master_port: '9001'

  master_slave_shared_secret: 'A secret'

Recovered Primary (a replica promoted to a primary)

storage:
  root_path: root-dir-original-replica

network:
  sync:
    listen_address: '9.10.11.12'
    listen_port: '9003'

backup:
  operating_mode: 'master_with_synchronous_slave'

  master_slave_shared_secret: 'Possibly another secret'

Replica for Recovered Primary

storage:
  root_path: root-dir-new-replica

network:
  sync:
    listen_address: '13.14.15.16'
    listen_port: '9004'

backup:
  operating_mode: 'slave'

  master_address: '9.10.11.12'
  master_port: '9003'

  master_slave_shared_secret: 'Possibly another secret'

Notes:

• backup.master_address and backup.master_port of the replica must be the same as network.listen_address and network.listen_port of the primary.
• The root_path of the replica becomes the root_path of the next primary. It is also possible to copy the whole directory to another location while the sync worker is stopped.
• A replica still listens for incoming connections, even though it doesn’t accept sync connections. The endpoint /info is active for all sync workers including replicas.
• Proxies will have to be redirected if necessary.
• A communicating primary and replica must have the same secret, but the secret can be changed at any time.

Monitoring

Realm Object Server workers support sending metrics to statsd, which is assumed to be listening at localhost:8125. You can then forward these metrics to graphite or similar systems for monitoring.

All metrics keys start with a prefix of realm.<hostname>:

realm.example.com.connection.online
realm.example.com.connection.failed
realm.example.com.realms.open
realm.example.com.protocol.violated
realm.example.com.protocol.bytes.received
realm.example.com.authentication.failed

Metrics

Enabling Professional and Enterprise APIs

To use the Professional and Enterprise features in server-side applications, you must enable the APIs by including your feature token in the application. (This token is sent to you via email when you sign up for a trial or purchase the Professional or Enterprise editions.) Near the start of your application, include your token and call the respective API:

• Node.js
• .NET Core

const token = "eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9...";

// Unlock Professional Edition APIs
Realm.Sync.setFeatureToken(token);

const string token = "eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9...";

// Unlock Professional Edition APIs
SyncConfiguration.SetFeatureToken(token);

The feature token could also be loaded from a file rather than stored in your code:

• Node.js
• .NET Core

RealmSync.setFeatureToken(fs.readFileSync('./access-token', 'utf-8'));

var token = File.ReadAllText("./feature-token");
SyncConfiguration.SetFeatureToken(token);

Event Handling

An exclusive feature of the Enterprise and Professional Editions of Realm Object Server is event handling capabilities. This functionality is provided in the server-side Node.js SDK via a global event listener API which hooks into the Realm Object Server, allowing you to observe changes across Realms. This could mean listening to every Realm for changes, or Realms that match a specific pattern. For example, if your app architecture separated user settings data into a Realm unique for each user where the virtual path was /~/settings, then a listener could be setup to react to changes to any user’s settings Realm.

Whenever a change is synchronized to the server, it triggers a notification which allows you to run custom server-side logic in response to the change. The notification will inform you about the virtual path of the updated Realm and provide the Realm object and fine-grained information on which objects changed. The change set provides the object indexes broken down by class name for any inserted, deleted, or modified object in the last synchronized transaction.

Creating an Event Handler

To use Realm Event Handling, you’ll need to create a small Node.js application.

Create a directory to place the server files, then create a file named package.json. This JSON file is used by Node.js and npm, its package manager, to describe an application and specify external dependencies.

You can create this file interactively by using npm init. You can also fill in a simple skeleton file yourself using your text editor:

{
    "name": "MyApp",
    "version": "0.0.1",
    "main": "index.js",
    "author": "Your Name",
    "description": "My Cool Realm App",
    "dependencies": {
        "realm": "^1.8.0"
    }
}

We specify the Realm Mobile Platform as version 1.8 or higher. If you have other dependencies for your application, go ahead and specify them in the dependencies section of this file.

After the package.json file is configured properly, type:

npm install

to download, unpack and configure all the modules and their dependencies.

Your event handler will need to access the Object Server with administrative privileges, so you’ll need to get the Object Server’s admin token. Under Linux, view the token with:

cat /etc/realm/admin_token.base64

On macOS, the token is stored in the realm-object-server folder, inside the Realm Mobile Platform folder. Navigate to that folder and view the token:

cd path-to/realm-mobile-platform
cat realm-object-server/admin_token.base64

A sample index.js file might look something like this. This example listens for changes to a user-specific private Realm at the virtual path /~/private. It will look for updated Coupon objects in these Realms, verify their coupon code if it wasn’t verified yet, and write the result of the verification into the isValid property of the Coupon object.

var Realm = require('realm');

var ACCESS_TOKEN = "eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9...";

// Unlock Professional Edition APIs
Realm.Sync.setFeatureToken(ACCESS_TOKEN);

// Insert the Realm admin token here
//   Linux:  cat /etc/realm/admin_token.base64
//   macOS:  cat realm-object-server/admin_token.base64
var ADMIN_TOKEN = 'ADMIN_TOKEN';

// the URL to the Realm Object Server
var SERVER_URL = 'realm://127.0.0.1:9080';

// The regular expression you provide restricts the observed Realm files to only the subset you
// are actually interested in. This is done in a separate step to avoid the cost
// of computing the fine-grained change set if it's not necessary.
var NOTIFIER_PATH = '^/.*/private$';

// The handleChange callback is called for every observed Realm file whenever it
// has changes. It is called with a change event which contains the path, the Realm,
// a version of the Realm from before the change, and indexes indication all objects
// which were added, deleted, or modified in this change
function handleChange(changeEvent) {
  // Extract the user ID from the virtual path, assuming that we're using
  // a filter which only subscribes us to updates of user-scoped Realms.
  var matches = changeEvent.path.match(/^\/([0-9a-f]+)\/private$/);
  var userId = matches[1];

  var realm = changeEvent.realm;
  var coupons = realm.objects('Coupon');
  var couponIndexes = changeEvent.changes.Coupon.insertions;

  for (var couponIndex of couponIndexes) {
    var coupon = coupons[couponIndex];
    if (coupon.isValid !== undefined) {
      var isValid = verifyCouponForUser(coupon, userId);
      // Attention: Writes here will trigger a subsequent notification.
      // Take care that this doesn't cause infinite changes!
      realm.write(function() {
        coupon.isValid = isValid;
      });
    }
  }
}

// create the admin user
var adminUser = Realm.Sync.User.adminUser(adminToken);

// register the event handler callback
Realm.Sync.addListener(SERVER_URL, adminUser, NOTIFIER_PATH, 'change', handleChange);

console.log('Listening for Realm changes');

The heart of the event handler is the handleChange() function, which is passed a changeEvent object. This object has four keys:

• path: The path of the changed Realm (used above with match to extract the user ID)
• realm: the changed Realm itself
• oldRealm: the changed Realm in its old state, before the changes were applied
• changes: an object containing a hash map of the Realm’s changed objects

The changes object itself has a more complicated structure: it’s a series of key/value pairs, the keys of which are the names of objects (i.e., Coupon in the above code), and the values of which are another object with key/value pairs listing insertions, deletions, and modifications to those objects. The values of those keys are index values into the Realm. Here’s the overall structure of the changeEvent object:

{
  path: "realms://server/user/realm",
  realm: <realm object>,
  oldRealm: <realm object>,
  changes: {
    objectType1: {
      insertions: [ a, b, c, ...],
      deletions: [ a, b, c, ...],
      modifications: [ a, b, c, ...]
    },
    objectType2: {
      insertions: [ a, b, c, ...],
      deletions: [ a, b, c, ...],
      modifications: [ a, b, c, ...]
    }
  }
}

In the example above, we get the Coupons and the indexes of the newly inserted coupons with this:

var realm = changeEvent.realm;
var coupons = realm.objects('Coupon');
var couponIndexes = changeEvent.changes.Coupon.insertions;

Then, we use for (var couponIndex of couponIndexes) to loop through the indexes and to get each changed coupon.

Integrating With Another Service

For a complete example of integrating the Event Handling framework with another service (in this case, IBM Watson’s Bluemix), read the tutorial for the Scanner App.

Data Access

The Realm Object Server allow you to access and change any shared Realm server-side using the administration token generated when you start the server. This allows access to all data in that Realm.

To retrieve the admin token on Linux:

cat /etc/realm/admin_token.base64

On macOS the token is stored in the realm-object-server folder within the zip:

cat realm-object-server/admin_token.base64

This can be used to synchronously construct a Realm.Sync.User object which can be passed into the Realm constructor to open a connection to any Realm on the server side.

// Open a Realm using the admin user
var adminToken = '3x4mpl3T0k3n…';
var adminUser = Realm.Sync.User.adminUser(adminToken);
var realm = new Realm({
  sync: {
    user: admin_user,
    url: 'realm://object-server-url:9080/my-realm',
  },
  schema: [{...}
  }]
});

Note that when used with an admin user, the Realm URL does not include the ~ character that resolves into the user ID for non-admin authentication; the admin user does not have a user ID. For non-admin authentication, read the Authentication section in the Object Server Access Control documentation.

Data Connector

The Enterprise Edition of the Realm Object Server offers a Node.js-based adapter API that allows you to access all low-level Object Server operations and data. This can be used to let a synced Realm interact with an existing legacy database such as PostgreSQL: the Realm will also be kept in sync with the external database in real time. Client applications can use the Realm Mobile Database API and get the benefits of working with real-time, native objects.

The Adapter API is set up in a very similar fashion to the Event Handler API described above. Create a small Node.js application by creating a directory to place the server files, then create a package.json for npm dependencies or use npm init to create it interactively.

{
    "name": "MyApp",
    "version": "0.0.1",
    "main": "index.js",
    "author": "Your Name",
    "description": "My Cool Realm App",
    "dependencies": {
        "realm": "^1.8.0"
    }
}

After your other dependencies are specified, use npm install to download, unpack and configure the modules.

As with the event handler API, you’ll need to access the Object Server with administrative privileges, and will need to get the Object Server’s admin token. Under Linux, view the token with:

cat /etc/realm/admin_token.base64

On macOS, the token is stored in the realm-object-server folder, inside the Realm Mobile Platform folder. Navigate to that folder and view the token:

cd path-to/realm-mobile-platform
cat realm-object-server/admin_token.base64

To use the Adapter API, the Node.js application you’re creating will act as a translator, receiving instructions from the Object Server and calling your external database’s API to read and write to it. A sample application might look like this:

var Realm = require('realm');

var ACCESS_TOKEN = "eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9...";

// Unlock Enterprise Edition APIs
Realm.Sync.setFeatureToken(ACCESS_TOKEN);

var adapterConfig = {
  // Insert the Realm admin token here
  //   Linux:  cat /etc/realm/admin_token.base64
  //   macOS:  cat realm-object-server/admin_token.base64
  admin_token: 'ADMIN_TOKEN',

  // the URL to the Realm Object Server
  server_url: 'realms://127.0.0.1:9080',

  // local path for the Adapter API file
  local_path: './adapter',

  // regular expression to limit which Realms will be observed
  realm_path_regex: '/^\/([0-9a-f]+)\/private$/'
};

class CustomAdapter {
  constructor(config) {
    this.adapter = new Realm.Sync.Adapter(
      config.local_path,
      config.server_url,
      Realm.Sync.User.adminUser(config.admin_token),
      config.realm_path_regex,

      // This callback is called any time a new transaction is available for
      // processing for the given path. The argument is the path to the Realm
      // for which changes are available. This will be called for all Realms
      // which match realm_path_regex.
      (realm_path) => {
        var current_instructions = this.adapter.current(realm_path);
        while (current_instructions) {
          // if defined, process the current array of instructions
          this.process_instructions(current_instructions);

          // call advance to progress to the next transaction
          this.adapter.advance(realm_path);
          current_instructions = this.adapter.current(realm_path);
        }
      }
    )
  }

  // This method is passed the list of instructions returned from
  // Adapter.current(path)
  process_instructions(instructions) {
    instructions.forEach((instruction) => {
        // perform an operation for each type of instruction
        switch (instruction.type) {
          case 'INSERT':
            insert_object(instruction.object_type, instruction.identity, instruction.values);
            break;
          case 'DELETE':
            delete_object(instruction.object_type, instruction.identity);
            break;
          // ... add handlers for all other relevant instruction types
          default:
            break;
        }
      })
  }
}

Each instruction object returned by Adapter.current has a type property which is one of the following strings. Two or more other properties containing data for the instruction processing will also be set.

• INSERT: insert a new object
  • object_type: type of the object being inserted
  • identity: primary key value or row index for the object
  • values: map of property names and property values for the object to insert
• SET: set property values for an existing object
  • object_type: type of the object
  • identity: primary key value or row index for the object
  • values: map of property names and property values to update for the object
• DELETE: delete an existing object
  • object_type: type of the object
  • identity: primary key value or row index for the object
• CLEAR: delete all objects of a given type
  • object_type: type of the object
• LIST_SET: set the object at a given list index to an object
  • object_type: type of the object
  • identity: primary key for the object
  • property: property name for the list property to mutate
  • list_index: list index to set
  • object_identity: primary key or row number of the object being set
• LIST_INSERT: insert an object in the list at the given index
  • object_type: type of the object
  • identity: primary key for the object
  • property: property name for the list property to mutate
  • list_index: list index at which to insert
  • object_identity: primary key or row number of the object to insert
• LIST_ERASE: erase an object in the list at the given index: this removes the object
• from the list but the object will still exist in the Realm
  • object_type: type of the object
  • identity: primary key for the object
  • property: property name for the list property to mutate
  • list_index: list index which should be erased
• LIST_CLEAR: clear a list removing all objects: objects are not deleted from the Realm
  • object_type: type of the object
  • identity: primary key for the object
  • property: property name for the list property to clear
• ADD_TYPE: add a new type
  • object_type: name of the type
  • primary_key: name of primary key property for this type
  • properties: Property map as described in Realm.ObjectSchema
• ADD_PROPERTIES: add properties to an existing type
  • object_type: name of the type
  • properties: Property map as described in Realm.ObjectSchema
• CHANGE_IDENTITY: change the row index for an existing object: not called for objects with primary keys
  • object_type: type of the object
  • identity: old row value for the object
  • new_identity: new row value for the object

For full details, including all the instruction types and the data passed to them, consult the API reference for the Realm.Sync.Adapter class.

A PostgreSQL data connector is already implemented, and more are on the way, including MongoDB and Microsoft SQL Server. Any data connector can be customized to your application’s specific needs. If you’re a Realm Enterprise customer, contact your representative for more information.