Introducing Emblem - A Reflection-based Utility Library for Scala

I've been working on this Scala project for a while now, and I found myself needing to develop some reflection-based tools to do some of the things I wanted to do. These things definitely weren't part of my core code, and I thought other people might find them useful as well, so I split them off into a separate library called emblem. It's open-source and on GitHub. The code is clean and has good API docs, but I haven't gotten around to writing any user documentation yet. I plan to do that here, and I'd like to start by describing my basic goals with emblem, and the design principles I decided to follow.
The first thing I wanted to do was to build some collections whose elements were more tightly bound, type-wise. For instance, suppose I have some Entity classes, and each one has an EntityType companion to contain metadata about the entity type, such as the natural keys for that entity:

trait Entity

trait EntityType[E <: Entity] {
  val naturalKeys: Seq[NaturalKey] = Seq()
}

We provide a default empty set of natural keys, but implementing classes want to override it where appropriate. Our convention is to have the companion object to the entity class implement the EntityType, like so:

case class Customer(
  customerUri: String,
  firstName: String,
  lastName: String
) extends Entity

object Customer extends EntityType[Customer] {
  override val naturalKeys =
    Seq(NaturalKey("Customer.customerUri"))
}

We also have repositories for each of our entity classes, like so:

trait Repository[_ <: Entity]

class CustomerRepo extends Repository[Customer]

Now let's say we want to maintain a map from the entity type to the repository. We could try something like this:

val empty: Map[EntityType[_ <: Entity],
               Repository[_ <: Entity]] = Map()

val singleton = empty + (Customer -> customerRepo)

But the typing of this collection does not reflect the fact that for any given pair in the map, the key and value both have same kind of entity as its type argument. This means we have to cast when retrieving a repo. For instance:

val repositories: Map[EntityType[_ <: Entity],
                      Repository[_ <: Entity]] = initialize()

val customerRepo: Repository[Customer] =
  repositories(Customer).asInstanceOf[Repository[Customer]]

There is no guarantee that the type cast will succeed. (Even worse, the type cast probably will succeed due to type erasure, and you'll get some other ClassCastException down the line.) This is reminiscent of the early days of Java, when you had to cast and trust your luck when pulling a cat out of a list of cats:

Vector catList = initializeCatList();
for (int i = 0; i < catList.size(); i++) {
    Cat cat = (Cat) catList.get(i);
    // ...
}

So I wanted a map that would hide the asInstanceOf I resorted to above, and to hide it safely.

The other thing I wanted was a generic method to traverse over a structure of nested case classes. I want to be able to gracefully handle traversing existing objects, as well as producing the nested structures. For example, the entity classes I described above may need to be read from and written to a database, or some other format such as JSON.

I looked around for existing libraries that handled these kinds of tasks, and I didn't find anything that jumped out at me as a solution. I strongly suspect that there is something in shapeless, or maybe Scalaz, that would do these things for me. And I spent a lot of time investigating shapeless, and trying it out. But the theory behind functional programming is not my strong suit, and I eventually decided that it was too much work to figure out how to use it, and to maintain the resulting code.

One of my design goals for emblem is for it to be easy to understand and use for people coming from a Java background. I don't want people to feel like they need to understand category theory in order to use it. Or to hesitate to use it because the are concerned that other developers on the team might have difficulty understanding it. That's not to say that this is a Java-style library. It is a Scala-style library. One that embraces the OO in the F/OO as much as the F.

A related design decision was to not use macros. I think Scala macros are great tools, and if I needed them, I would probably end up using them after a brief consideration. But I found that I was able to accomplish everything I needed to do using Scala runtime reflection. Macros can be a little mysterious to people who don't fully grok them, and when types pop up that you can't actually find in the source code, some people start to get a little uncomfortable. On the other hand, run-time reflection is an idiom that most Java programmers will at least be familiar with.

So that's the overview. In the next post, we'll look at the core building block of the emblem library: the TypeKey. I'm expecting to write about seven posts on Introducing Emblem in total. I'm aiming to put out one a week, but I may need to skip a week here or there, depending on how things go.