Two Major Design Flaws

Throughout the time I spent developing longevity, I've made any number of major design flaws. The vision I started out with has changed drastically over time. It's evolved as I experimented with developing the longevity API. While my core guiding principles have stayed in place, the early versions of the library are nearly unrecognizable when compared to the latest version.

Since the beginning, I've had a comprehensive test suite, that has allowed me to correct mistakes and simplify my API with ease and assurance of correctness. And working with the API as it has grown has allowed me to spot problems as I moved forward. But there were two major design flaws that remain in place. Both of them have been nagging me for many months now, until they have become clear in my head. Now I'm ready to admit and address these two errors: the failure to use shapeless, and the lack of column-oriented storage of persistent entities with column-oriented back ends such as Cassandra and SQLite.

Longevity 0.27 - Stepping into Shapeless

After over 8 months since it's previous release, longevity version 0.27 is out! This release is very exciting to me, as it represents longevity's first steps into shapeless. I was able to incorporate shapeless.Generics into the domain model for all of the persistent objects, which will allow me to replace home-grown reflection based code with shapeless going forward.

The externally facing changes are minor. I've replaced a home-grown test data generation tool with ScalaCheck and scalacheck-shapeless. However, this minor change has already reaped benefits, as it replaces a particularly ugly part of the old longevity API - custom generation of test data - with a much more elegant process of supplying the right implicit org.scalacheck.Arbitrary.

This minor feature is really a proof of concept that I would be able to replace my homegrown reflective code with shapeless features. I feel confident that I can migrate the entire architecture over. Although of course, it will take some time. I will write more soon on how I plan to move forward from here. Stay tuned!

Longevity 0.26 - Migrations Framework

Longevity release 0.26 is out, and features the initial version of a schema migrations framework. You might ask, "Why would longevity - a persistence framework for Scala and NoSQL - require its own migrations framework? Couldn't it use any number of existing tools for doing schema migrations?"

Because longevity handles persistence for you, allowing you to write you application in terms of your domain model, the traditional approach to migrating schema with database scripts does not always work out. At present, every back end stores your persistent objects in JSON. Back ends such as SQLite and Cassandra store the JSON in regular text columns. MongoDB offers update commands that allow you to modify your BSON documents dynamically, so a more traditional approach to schema migrations may work for you. It’s also possible that we will be adding column-based back ends for SQLite and Cassandra in the future, which will also be amenable to more traditional approaches.

But even if a database-oriented tool such as FlywayDB works for you, we still want to provide you with an option that raises schema migrations up to the domain model abstraction layer. With longevity migrations, you define your domain model evolution in terms of Scala functions that map from the old version of the domain to the new version. So no database scripts are needed.

I'm excited about this release because the lack of tools to help migrate a schema was, in my mind, the last major showstopper for using longevity in a production environment. Domain models do evolve, and we need to be able to bring existing data along with the changes. It's a pretty bare bones implementation of a migrations framework, but it's entirely functional, and I have a long list of ideas on ways to improve it.

My favorite feature by far on the wish list, I've been calling soft-stop migrations. If our migrations are defined in terms of Scala, that gives us the opportunity to keep both versions of the schema alive in the database simultaneously. While we migrate from one version to another, applications running the old version can use those Scala functions to keep the new version up to date with ongoing changes. This way, the application can continue running throughout the process of a schema migration. This kind of thing is very difficult to do when your migration scripts are written in a database query language. And limiting downtime is highly desirable. It would also be relatively easy to implement, and I'm tempted to do it. But now that I have a basic schema migrations tool in place, I'd like to focus on replacing home-grown reflective code with shapeless for a while.

Check out the chapter on migrations in the user manual to get a feel for how it works.

Longevity Goes Finally Tagless

Longevity - your persistence framework for Scala and NoSQL - has always sported a future-based API. But now in release 0.24, longevity has gone finally tagless, replacing the hardcoded references to scala.concurrent.Future with a generic effect F. We currently support three effects: Scala futures, the cats-effect IO monad, and an old-fashioned blocking API for the purists out there. We plan to support a variety of Task monads as effects in the near future, and you can always write your own effect as well.

For more information on longevity effects, see the user manual.

For examples on how longevity works with different effects, check out this demo code.

This release is exciting for me for many reasons. Perhaps most importantly, longevity now sports a truly functional API with support for IO monads. But the flexibility of the finally tagless approach is truly freeing. We can satisfy the purist functional programmers, and continue to support people who want to do reactive programming with futures, with a minimum of cognitive overhead. It's also exciting to me because it's a major step on my personal path to becoming a better functional programmer.

But the best part for me is just how beautiful the repository API has become. Just look at it. In a sense, this is how this API was always meant to be. Perfectly intuitive, fully typesafe, just the right level of abstraction without a hint of leaks. I couldn't imagine doing persistence in Scala any other way.

Longevity Release 0.23 - Use Type Classes to Improve Type Safety of Persistent API

This longevity release has been a long time in the making. The main improvement here, and the one that has taken by far the most effort, is instituting the use of type classes to provide type safety for the persistence API. I've done a technical writeup on this work in my previous blog post. It's actually pretty interesting because I've never seen type classes quite this way before, so I think it's worth the read. The TLDR is that the repository API is now fully type safe, and the last instance of forcing users to inherit from a longevity trait has been removed.

Here are some of the other major improvements in this release:

An Interesting Use for Type Classes in Scala

This is a technical writeup of the major new feature in the longevity 0.23 release.

There are more than enough writeups on using type classes in Scala already. I normally wouldn't bother to write about it, but I will now, for three reasons. First, the usage of type classes in longevity seems unique to me; I haven't seen type classes used quite this way before. Second, I want to write this up as documentation for any future contributors to the longevity project. Third, I'd really like to get your feedback into what I've done here, and hear your advice about how I might further improve on things.

Longevity now has Better Support for Alternative JDBC Back Ends

Longevity release 0.21 features a new back end called JDBC. It should work in a lot of situations where using SQLite back end with a non-SQLite driver would fail. There is also a much easier development path for creating your own back end, should the generic JDBC back end not be sufficient.

Ten Ways You Can Contribute To An Open Source Project

Okay, so it's really about my open source project. But that's the whole point. I don't want it to be my project. I want it to be your project too! While this article is written to specifically address contributing to longevity, much of the material here will probably apply to other projects as well.

This is a complete rewrite of the contributing page on the longevity web site. You can read it over there if you like. I've also pasted the contents below for your convenience:

New SQLite Back End for Longevity

Longevity release 0.20.0 is out, featuring the following changes:

• Added a SQLite back end.
• Trying to be a bit more formal, changed occurrences of "Mongo" with "MongoDB" in the public API.
• Replaced "partition key" terminology with "primary key". I was undecided about this for a long time leading up to implementing the feature, and I think I made the wrong choice when the time came. "Partition key" is a little over specified for distributed database scenario. It's a little more general to posit that a database table can have multiple keys, but can more powerfully optimize a single one of those keys at your choosing. Now that I have three very different back ends, it's getting harder and harder to overfit longevity to a single database technology!

Keep reading to learn more about the new back end:

More Longevity Awesomeness with Macro Annotations!

I just got longevity release 0.18 out and wow, is it awesome! If you've looked at longevity before, you will be amazed at how easy it has become to start persisting your domain objects. And the best part is that everything persistence related is tucked away in the annotations. Your domain classes are completely free of persistence concerns, expressing your domain model perfectly, and ready for use in all portions of your application. Of course, we also provide a complete persistence layer for you, saving you countless hours of development effort!

See for yourself, here is a simple example demonstrating how easy it is to get started:

More API Simplifications in Longevity 0.17

Longevity is the persistence framework to use with Scala and NoSQL. Release 0.17 simplifies the user API in some very pleasant ways. Empty marker traits Persistent and Embeddable are removed. Also, KeyVal now takes a single type parameter instead of two. In brief, this will make defining the classes you want to persist so much easier and more natural, as you no longer have to extend your persistent classes with empty marker traits.

Longevity Artifacts for Scala 2.11 and 2.12

So I've finally got the longevity build set up to publish artifacts for Scala versions 2.11 and 2.12! I'm a little behind on the game, but, I've been busy ;-). Please use longevity version 0.16.1 for 2.12 artifacts. I'll be publishing artifacts for both Scala versions moving forward.

I'd like to shout out to the wonderful people at ScalaTest - Bill Venners and Chua Chee Seng - for helping me resolve an interoperability issue that made the longevity 2.12 artifacts possible. Thanks again!

Partition Keys in Longevity

I'm so excited to announce the release of longevity version 0.16.0! Longevity now supports partition keys - a key where, given a key value, we can determine the node in a distributed database where the data resides (or would reside, if it existed). This is a critical feature for NoSQL databases in a distributed context. A partition key is implemented with sharding in MongoDB, and with partitioning in Cassandra.

I'll provide a brief example of a partition key here after the jump, but to learn more about it, please take a look at the user manual.

Longevity 0.15.0 - Query Enhancements

Longevity - a persistence framework for Scala and NoSQL - is now up to version 0.15.0! The latest release focuses on query enhancements. We have added orderBy, offset, and limit clauses to our queries and query DSL.

Sample Application for Longevity and Play Framework

I've ported my sample longevity application - Simple Blogging - from Akka HTTP to Play. The Play version is a Lightbend Activator tutorial, which you can find here. Or you can take a look at the source code here on GitHub.

Longevity is a persistence framework for Scala and NoSQL.

Longevity 0.13.0 - Odds and Ends

In the past few weeks longevity has had three minor releases (0.11, 0.12, and 0.13) that clean up a variety of odds and ends that had been building up in the backlog. There was nothing major or exciting in these releases, but they are nonetheless important improvements. I haven't really publicized them much, but I wanted to let you know that longevity development is still proceeding.

Here are some changes that are worth mentioning:

• You no longer have to specify polyType and polyPType in your DerivedTypes and DerivedPTypes, respectively.
• Instead of just creating schema every time, I've added RepoPool.createSchema(), which you can call as you like. Schema creation is still non-destructive. And you can turn on the old behavior with configuration flag longevity.autocreateSchema.
• I added RepoPool.closeSession(), so you can clean up resources held by the underlying database drivers.
• I added some logging output.
• I added JSON marshallers and unmarshallers to the longevity context.

Next on the list is query enhancements! Although I have a couple of things to work on other than longevity proper that may precede it. I need to debug and fix a problem that has come up with longevity tests in ScalaTest version 3.0.0. And I'm thinking I'll port my sample application simbl from Akka HTTP to Play.

Configuration-level Optimistic Locking with Longevity

Longevity - a persistence framework for Scala and NoSQL - has been built from day one to isolate persistence concerns behind a clear, consistent, and easy to use API. The power of this design principle is demonstrated by how easy it is for you to bring optimistic locking to your application. All you have to do is set a configuration flag.

Longevity is growing to full maturity, and most features you would expect from a persistence framework are in place. Our 0.10.0 release brings many incremental improvements - including support for controlled vocabularies. But the central feature of this release is configuration-level optimistic locking.

Longevity Framework Cares About Code Quality

We care a lot about code quality here at longevity framework. You can rest assured that longevity is well-designed, has clean code, and a robust test suite. As of now, code coverage comes in at 95.02% statement, and 93.44% branch, as measured by sbt-scoverage. What software projects do you know of that have that kind of coverage?

Longevity Getting Started Guide Now Available

I'm happy to announce that the Getting Started Guide for longevity is now available. It walks through a sample application called Simple Blogging, which uses longevity on the back end, and Akka HTTP on the front.

This guide will walk you through the basics of using longevity within a real application.

Longevity now has an Activator Tutorial!

Learn how to get started with the Lightbend Activator tutorial here:

http://longevityframework.org/activator.html