Longevity 0.26 - Migrations Framework

Longevity release 0.26 is out, and features the initial version of a schema migrations framework. You might ask, "Why would longevity - a persistence framework for Scala and NoSQL - require its own migrations framework? Couldn't it use any number of existing tools for doing schema migrations?"

Because longevity handles persistence for you, allowing you to write you application in terms of your domain model, the traditional approach to migrating schema with database scripts does not always work out. At present, every back end stores your persistent objects in JSON. Back ends such as SQLite and Cassandra store the JSON in regular text columns. MongoDB offers update commands that allow you to modify your BSON documents dynamically, so a more traditional approach to schema migrations may work for you. It’s also possible that we will be adding column-based back ends for SQLite and Cassandra in the future, which will also be amenable to more traditional approaches.

But even if a database-oriented tool such as FlywayDB works for you, we still want to provide you with an option that raises schema migrations up to the domain model abstraction layer. With longevity migrations, you define your domain model evolution in terms of Scala functions that map from the old version of the domain to the new version. So no database scripts are needed.

I'm excited about this release because the lack of tools to help migrate a schema was, in my mind, the last major showstopper for using longevity in a production environment. Domain models do evolve, and we need to be able to bring existing data along with the changes. It's a pretty bare bones implementation of a migrations framework, but it's entirely functional, and I have a long list of ideas on ways to improve it.

My favorite feature by far on the wish list, I've been calling soft-stop migrations. If our migrations are defined in terms of Scala, that gives us the opportunity to keep both versions of the schema alive in the database simultaneously. While we migrate from one version to another, applications running the old version can use those Scala functions to keep the new version up to date with ongoing changes. This way, the application can continue running throughout the process of a schema migration. This kind of thing is very difficult to do when your migration scripts are written in a database query language. And limiting downtime is highly desirable. It would also be relatively easy to implement, and I'm tempted to do it. But now that I have a basic schema migrations tool in place, I'd like to focus on replacing home-grown reflective code with shapeless for a while.

Check out the chapter on migrations in the user manual to get a feel for how it works.