Migration From SQLite to Realm

Well…

My team is making messenger app. It has many data, and supports offline-Cache. So app should store data.

App data is combined as many types of object. Their relation are so complicated. They are combined as least 15 types object. They relate like 1:N or M:N.

Why we moved to Realm

My app was totally based on SQLite. When data’s relation was simple, it was fast enough. But now, our data structures are so complicated and they have many records.

I used ORMLite as SQLite’ ORM tools. (and still I’m using it.)
(When I chose it, GreenDAO was < 3.0. requery and sqlbrite was not opened. Don’t blame me why I use ORMLite. It’s also a convenient tool.)

Some old devices and users suffer from slow speed. So I tried to be faster.

Thing to tried to be fast on SQLite:

1. To be minimize query.
   Sometimes data could be changed 2 or many columns. The problem is I did the query for each column. So I tried to change simple query.
2. Sometimes EAGER, sometimes LAZY
   LAZY is effective to demand to access data. But for initiating of App’s Meta-data, it’s not efficient.
   For example, Your data structure is 1:N like this : ChatRoom-User. And during initiating, You should access users in each room. If you use LAZY, app should send query while initiating. That time, EAGER could be more efficient.
3. Try to change other library
   I already mentioned about GreenDAO, Requery and SQLBrite, They are also based on SQLite and ORM tool. I checked them. But performance wasn’t been better dramatically that I thought.

Problem is SQLite

Whatever performance is low, I couldn’t be better performance on SQLite. I tried to do everything. But it improved only 10~20 %. I should find other databases. Finally, I chose the Realm.

Realm

Before I chose the Realm, I checked Realm’s performance.

The way to test is this:

1. Same Data Structure. : it combined many datas as 1:1, 1:N, M:N.
2. EAGER Loading. : Realm is based on Zero-copy. so To compare fairly, I chose EAGER Loading.
3. Get average time from 20times Writing/Reading.

On The Bright Side

Realm is soooooooooooo fast. Test result is so amzing.

Write
Realm is 3.5x faster than SQLite (4039ms :1142ms)

Reading
Realm is 2.5x fater than SQLite (6010ms : 2450ms)

(some Realmers reviewed my posting. They said my posting might be wrong. So now I’ve fixed it.)

When I got result, I cannot believe. so I tried 5 times. But result is same.
Realm is so fast to feel satisfied.
But adapting Realm is not easy. I’ve got 4 problems.

On Dark Side

I feel satisfied with Realm’s performance. But When I wrote code in my app, I’ve met some problems.

1.No supporting Collections of Primitive type

class Data extends RealmObject {
  public RealmList<Long> userIds; // Error!!!
}

Upper code will be got error. Realm doesn’t support data-converter like this RealmList<Long>(in memory) <--> String (in db). so I should write some object for primitive type.

class Data extends RealmObject {
  private RealmList<RealmLong> ids;
  private List<Long> userIds;  // for json
  public RealmList<RealmLong> getIds() {
    if (ids == null) {/* init ids from userIds */}
    return ids;
  }
  public List<Long> getUserIds() {
    if (userIds == null) {/* init userIds from ids */}
    return userIds;
  }
}
class RealmLong extends RealmObject {
  public long value;
}

2.To use data from queried, Keep same thread.

Basically, My app’s rule is simple. All logic is in background. Only to set view could be accessed to data on Main-Thread. But Realm forces to keep same thread between accessing and using. They called MANAGED STATUS.

Simple example is this:

Observable.fromCallable(() -> RealmRepository.getSomeData())
  .subscribeOn(Schedulers.io())
  .observerOn(AndroidSchedulers.mainThread())
  .subscribe(data -> data.getXX()) // Error!!!

In background, it is got data. But data.getXX() is called in main-thread.
Data is managed on being accessed thread.

If you want to change status to UNMANAGED,
Use realm.copytFromRealm(Object)

public Data getSomeData() {
  Data data = realm.where(Data.class).findFirst();
  if (data != null) return realm.copyFromRealm(data);
  else return null;
}

This is eager loading & free to keep same thread.
(caution : copyFromRealm could be slower than lazy.)

3.No supporting inheritance

It’s big problem to me. but for performance, I gave it up. I wrote flat code.

// before realm
class Meta {
  public long id;
  public String type;
}
class A {
  public String text;
}
class B {
  public int count;
}
// after realm
class Data {
  public long id;
  public String type;
  public String text;
  public int count'
}

it might be stupid thing. but I have no idea about this. I hope Realm support inheritance.

Realm recommend we use composition. (Reference)

4.MultiProcess Problem

I met this problem after releasing. It’s critical to our team. We’ve got crash report 20x times than before. I was confused and regretted moving realm.

Whatever it did, I should fix it. So I tried to find reason of problem. Fortunately, I found it.

It was occurred multiprocess. Realm doesn’t support multi-process yet. I thought I didn’t access realm at multiprocess.

the problem is this:

public class MyApplication extends Application {
  @Override
  public void onCreate() {
    Realm.init(this); // it might be occurred error!!!
  }
}

I almost forgot multiprocess’s life cycle. MultiProcess also called Application.onCreate()

Realm does lock when accessing and writing record. It’s based on thread-id. But in other process, Realm couldn’t recognize other thread. It could occurred ACCESS_ERROR ,RealmFileException even broken file as causing fatal error.

My tip is this. Realm.init(Context) is to limit access from other process.

If user’s realm file is broken as bad header , Only solution is deleting realm.

If you need to access Realm from multi process, then now only hope is ContentProvider or SQLite. Using ContentProvider is always to access Main Process. I wouldn’t explain how to use ContentProvider or SQLite.

My little tiny tip

If you access realm, you always need Realm.getInstance() and Realm.close(). So I use template code this:

It’s simple way to reduce duplicated code.

This tip is for UNMANAGED status. if you close realm, then it might occur error as MANAGED RealmObject. It means it does also to Reactive Plugin.

Wrap-up

I moved SQLite to Realm. It’s amazing to me for performance. but I met many problems also. almost problems are handled to me. so now my app is stable and fast.

Remember that Realm doesn’t support these :

1. No Collection of Primitive : Wrap Object
2. No Inheritance : Re-struct object, Using composition.
3. No Multi-Thread for using data : Change to UNMANAGED
4. No Multi-Process : Use ContentProvider

You could avoid these problems. After then, you could make fastest application.

And My Last Little tip:
If you need to store data stable, Be combined SQLite and Realm.
Store data to SQLite that never lose one like session-key. to Realm, store data to modify frequently and to be able to initiate easily.

Happy to use Realm.
Thanks to Realmers for reviewing this article. (YongUk Kim, MinWoo Park, Christian Melchior)

Update (2016.12.25)

Bad Realm Header error may be caused by installation process of PlayStore.
It’s not official announcement. but I’ve heard that Realm staffs are analyzing this problem.