Taming the Cake Pattern with Type Macros

In a previous post we discussed various techniques that could be employed with the component based dependency injection (CBDI; typically referred to as the cake pattern). In particular, we discussed two CBDI features that are not available in DI frameworks such as the Spring framework or Google guice: expressing design constraints between hierarchical components, and encapsulating the details of a composite component. Alongside these advantages, however, the cake pattern has some disadvantages that limit its appeal. The three major problems that I see are:

1. Verbose: Lots of boilerplate.
2. Opaque: It's hard to figure out and understand what is going on by looking at the code because the language structures used do not signify the user's intent. These language structures are somewhat advanced, and may seem a bit foreign to programmers that are relatively new to Scala.
3. Aloof: Compiler error messages are potentially confusing and misleading, because they address the language structures used, and not the user's intent.

(I've heard Daniel Spiewak report that the compiler can also be very slow at handling the cake pattern. I haven't had any personal experience with this problem, so I am not in a position to address it.)

Lately I've been pretty excited about type macros, an experimental Scala language feature, and I've been considering ways in which type macros might relieve some of these problems. In this blog post, I want to describe some hypothetical type macros that might alleviate the first two problems: verbosity and opacity. I haven't had a chance to use Scala macros yet, and while the macros presented here should be possible, I can't say for sure until I've tried it. So think of what is presented here as a spec. I'm going to try to implement it, and I'll let you know how that goes.

If you were hoping for another post on Monads in Scala, I'm sorry, I haven't gotten to that yet. I'll pick up that exercise with my next post.

Type Macros api and impl

The original suggestion that got me started along these lines was from Clint Gilbert in a thread on the scala-user google group. The suggestion was simple: maybe Scala macros could be used to define an interface and a default implementation in a single pass. I think I can accomplish this with two type macros, api and impl. Here's how I want the two type macros to behave:

• api[A] creates a new type that is roughly a copy of A, with the following modifications:
• If B is a declared super-type of A, then api[B] is a declared super-type of api[A]
• Repeats all the vals, vars and defs of A, but making them abstract by cutting out their definitions
• All trait and class definitions C within A are replaced by api[C] by recursively applying this macro
• All references to C within api[A] are replaced by api[C]
• Self-types are copied verbatim
•  impl[A] creates a new type that is roughly a copy of A, with the following modifications:
• Extends api[A]
• If B is a declared super-type of A, then impl[B] is a declared super-type of impl[A]
• Repeats all the vals, vars and defs of A, including their definitions
• All trait and class definitions C within A are replaced by impl[C] by recursively applying this macro
• All references to C within impl[A] are replaced by impl[C]
• Self-types are copied verbatim

Admittedly, all the details are not worked out here, but let's see how it would apply to some of our component based dependency injection examples. Here's our URepositoryComponent:
 

	trait URepositoryComponent {
	protected val uRepository = new URepository
	protected class URepository {
	def getU(uName: String): Option[U] = None // STUB
	}
	}

view raw URepositoryComponent.scala hosted with ❤ by GitHub

I've left the method getU as a stub since its implementation is not relevant here. Now, api[URepositoryComponent] and impl[URepositoryComponent] would expand to something like this:

	trait api[URepositoryComponent] {
	protected val uRepository: api[URepository]
	protected class api[URepository] {
	def getU(uName: String): Option[U]
	}
	}
	trait impl[URepositoryComponent] extends api[URepositoryComponent] {
	override protected val uRepository = new impl[URepository]
	override protected class impl[URepository] extends api[URepository] {
	override def getU(uName: String): Option[U] = None // STUB
	}
	}

view raw apiAndImpl[URepositoryComponent].scala hosted with ❤ by GitHub

The impl type macro might not seem very useful at first, but it does serve one important function: it extends the api. It may be possible, with some trickery, to get URepositoryComponent to extend api[URepositoryComponent], but I sort of like the impl macro because it's clean and it makes the intention clear.

Let's write UServiceComponent, which depends the URepositoryComponent. We'll make the self-type be to api[URepositoryComponent], since the service should not need to know any of the details of the implementation:

	trait UServiceComponent {
	self: api[URepositoryComponent] =>
	protected val uService = new UService
	protected class UService {
	def getU(uName: String): Option[U] = uRepo.getU(uName) // STUB
	}
	}

view raw UServiceComponent.scala hosted with ❤ by GitHub

Again, I've left getU as a stub. We are only interested here in the components and the dependencies between them. Now, api[UServiceComponent] and impl[UServiceComponent] expand as follows:

	trait api[UServiceComponent] {
	self: api[URepositoryComponent] =>
	protected val uService: api[UService]
	protected class api[UService] {
	def getU(uName: String): Option[U]
	}
	}
	trait impl[UServiceComponent] extends api[UServiceComponent] {
	self: api[URepositoryComponent] =>
	override protected val uService = new impl[UService]
	override protected class impl[UService] extends api[UService] {
	override def getU(uName: String): Option[U] = uRepo.getU(uName) // STUB
	}
	}

view raw apiAndImpl[UServiceComponent].scala hosted with ❤ by GitHub

At this point, we are ready to create our component registry, with the our two sub-components. We call the component registry TopComponent, to highlight the fact that the component registry is really no different in kind than the other composite components in the hierarchy. Here is our definition of TopComponent, followed by expansions for api[TopComponent] and impl[TopComponent]:

	trait TopComponent extends
	URepositoryComponent with
	UServiceComponent
	trait api[TopComponent] extends
	api[URepositoryComponent] with
	api[UServiceComponent]
	trait impl[TopComponent] extends
	api[TopComponent] with
	impl[URepositoryComponent] with
	impl[UServiceComponent]

view raw TopComponent.scala hosted with ❤ by GitHub

Now we can define our application with reference only to the APIs, and delay the injection of component implementations until we are ready to create the application instance:

	// Application is abstract because we do not have component implementations yet
	abstract class Application extends api[TopComponent] {
	// application implementation here
	}
	new Application with impl[TopComponent] // inject implementations

view raw Application.scala hosted with ❤ by GitHub

And we are done! It's important to note (or to confirm for yourself!) that while this component model is very simple, this technique also applies to the more complicated nested hierarchical models presented in my earlier blog post.

Let's step back and review all the code that the user had to write:

	trait URepositoryComponent {
	protected val uRepository = new URepository
	protected class URepository {
	def getU(uName: String): Option[U] = None // STUB
	}
	}
	trait UServiceComponent {
	self: api[URepositoryComponent] =>
	protected val uService = new UService
	protected class UService {
	def getU(uName: String): Option[U] = uRepo.getU(uName) // STUB
	}
	}
	trait TopComponent extends URepositoryComponent with UServiceComponent
	abstract class Application extends api[TopComponent] {
	// application implementation here
	}
	new Application with impl[TopComponent] // inject implementations

view raw Application2.scala hosted with ❤ by GitHub

Much of the boilerplate is gone, but not all of it. We still have to wrap the service classes (e.g., UService) in a component, and provide an access point for the injectable (e.g., protected val uService: UService). Let's introduce a component type macro to accomplish this.

Type Macro component

Let's loosely define a new type macro component[A] as follows:

• component[A] is a trait
• For every class or trait B that A extends, component[A] extends component[B] 
• Any self-types of A are lifted up to become self-types of component[A]
• If A is a leaf-level component, and not a composite component that only consists of its parts, then:
• The trait contains a verbatim copy of A, except protected, and with the self-types removed
• The trait contains a protected val of of type A that is defined as a new instance of A. In the simple case, this val should be named after the type name, down-casing the first character of the name. However, to properly support overriding of components, we need to name the val in the same way as the overridden component. So if A extends B, the val should be named b.

Let's also go ahead and define convenience macros componentApi[A] as api[component[A]], and componentImpl[A] as impl[component[A]].  The simple application presented above now becomes this:

	class URepository {
	def getU(uName: String): Option[U] = None // STUB
	}
	class UService {
	self: componentApi[URepository] =>
	def getU(uName: String): Option[U] = uRepo.getU(uName) // STUB
	}
	trait Top extends URepository with UService
	abstract class Application extends componentApi[Top] {
	// application implementation here
	}
	new Application with componentImpl[Top] // inject implementations

view raw Application3.scala hosted with ❤ by GitHub

The Cake Pattern Does Not Clearly Communicate Intent

One of the most confusing things about CBDI is that the language constructs we use do not correlate with our intent. For instance, we see TopComponent extending URepositoryComponent as a way of expressing that the latter is a sub-component of the former. This doesn't really match well with our standard expectations of what inheritance is used for. We also see UServiceComponent self-typing URepositoryComponent to indicate a dependency between the two. These idioms may work well for someone very familiar with Scala, but for those less familiar, it can be challenging. Let's introduce a couple of type macros that might more accurately reflect the intent.

Type Macro hasPart

This may not be the best idea, but I'd like to introduce a type macro hasPart[A] to indicate that an inheritance relationship is actually intended as a component/sub-component relationship. The macro itself can simply return the original type A. I originally wanted to name this hasSubComponent, but that name sort of clashes with the component macro, since hasSubComponent would be applied to underlying service and not the component that is built around it.. So, maybe there is a better name for this, but the basic idea is to replace the definition of Top above with something like this:

	trait Top extends
	hasPart[URepository] with
	hasPart[UService]

view raw Top.scala hosted with ❤ by GitHub

Type Macro hasDependency

It is possible with the cake pattern to use inheritance instead of self-types to declare dependencies. But there is a major difference between the two approaches: If B inherits from C, then A can easily inherit from B without any concern for what C is about. But if B self-types on C, then A cannot inherit from B without caring about C. It either needs to extend C, or include C (or something that extends C) as a self-type. This is what makes design constraints between composite components possible.

While the application of a self-type here is very useful, it can be a little daunting to someone who is used to Java and Spring or Guice. A jumpy developer may mistakenly come to the conclusion that she will have to learn category theory in order to do dependency injection in Scala. It would be nice if we could avoid the use of the self-type altogether, as well as coming up with a construct that communicates the intent: to indicate a dependency.

To accomplish this, let's first create a type macro expandsToSelfType[A], which for all intents and purposes is never exposed to the user. Now, we can define hasDependency[A] as expanding to type componentApi[A] extends expandsToSelfType[A]. Because it extends componentApi[A], code like the following will still compile:

	class UService extends hasDependency[URepository] {
	def getU(uName: String): Option[U] = uRepo.getU(uName) // STUB
	}

view raw UService.scala hosted with ❤ by GitHub

But the inherited expandsToSelfType[A] becomes a marker for the component macro:

• When component[A] sees that A inherits from expandsToSelfType[B], it:
• Removes the inheritance to componentApi[B] and expandsToSelfType[B] from the enclosed version of A that it writes
• self-types on componentApi[B]

Following these new rules, component[UService] expands to the UServiceComponent shown above. I imagine the details of this strategy will change as I try to implement it, but we can see that it should be doable in the least.

Adding a Component Hierarchy

Let's add a component hierarchy similar to the one I described in the earlier paper. Here's how the application looks now:

	class SRepository {
	def getS(sName: String): Option[S] = None // STUB
	}
	class TRepository {
	def getT(tName: String): Option[T] = None // STUB
	}
	class URepository {
	def getU(uName: String): Option[U] = None // STUB
	}
	trait Repository extends
	hasPart[SRepository] with
	hasPart[TRepository] with
	hasPart[URepository]
	class SService extends hasDependency[SRepository] {
	def getS(sName: String): Option[S] = sRepo.getS(sName) // STUB
	}
	class TService extends hasDependency[TRepository] {
	def getT(tName: String): Option[T] = tRepo.getT(tName) // STUB
	}
	class UService extends hasDependency[URepository] {
	def getU(uName: String): Option[U] = uRepo.getU(uName) // STUB
	}
	trait Service extends
	hasDependency[Repository] with
	hasPart[SService] with
	hasPart[TService] with
	hasPart[UService]
	trait Top extends
	hasPart[Repository] with
	hasPart[Service]
	// Application is abstract because we do not have component implementations yet
	abstract class Application extends componentApi[Top] {
	// application implementation here
	}
	new Application with componentImpl[Top] // inject implementations

view raw Application4.scala hosted with ❤ by GitHub

Now, SRepository claiming a dependency on UService will give a type error that points at component[Repository]. SService can only claim a dependency on SRepository because Service claims a dependency on Repository. (Top fills that dependency as it claims Repository as a part.) Notice how much more neatly this language matches with our intent?

Type Macro hasPrivatePart

In the earlier paper, we saw how we could encapsulate the details of a composite component by simply declaring them in the implementation but not in the API. We can create a simple macro hasPrivatePart[A] that expands to hasPart[A] with hiddenInApi[A]. The hiddenInApi[A] tells the api type macro to leave it out. This gives us the following for the ChartViewFactory example presented in the earlier paper:

	trait ChartViewFactory extends
	hasDependency[HistogramViewFactory] with
	hasDependency[ScatterPlotViewFactory] {
	def create(chart: Chart) = chart match {
	case c: Histogram => histogramViewFactory.create(c)
	case c: ScatterPlot => scatterPlotViewFactory.create(c)
	}
	}
	trait ChartView extends
	hasPart[ChartViewFactory] with
	hasPrivatePart[HistogramViewFactory] with
	hasPrivatePart[ScatterPlotViewFactory]
	trait View extends
	hasPart[ChartView] with
	hasPart[TopView]

view raw ChartVIewFactory.scala hosted with ❤ by GitHub

Swapping in an Alternate Implementation

Let's say we want to swap in an alternate version of the URepository. That's around 6 lines of code that are concise and to the point:

	class AltURepository extends URepository {
	override def getU(uName: String): Option[U] = None // STUB
	}
	trait AltRepository extends Repository with hasPart[AltURepository]
	trait AltTop extends Top with hasPart[AltRepository]
	new Application with impl[AltTopComponent]

view raw AltURepository.scala hosted with ❤ by GitHub

Type Macro mock

Part of the allure of a good dependency injection framework is being able to easily inject mock objects produced by a variety of mocking frameworks. In general, these mocking frameworks supply a method that takes the type to be mocked, and returns a mock object of that type. This is the main thing we need to hook in to in order to integrate CDBI with mocking frameworks. Let's describe a type macro that takes a mocking method from one of these frameworks, and produces mock components. We'll informally define type macro mock[A](mockMethod: [B]() => B) as follows:

• extends componentApi[A]
• for every B that A extends, mock[A](mockMethod) extends mock[B](mockMethod)
• if A is a leaf-level component, and not a composite component that only consists of its parts, then:
• mock[A](mockMethod) overrides the protected val of of type api[A] by applying mockMethod to type api[A]

I currently default to ScalaTest and EasyMock for testing and mocking. The mocking method I use with these tools is org.scalatest.mock.EasyMockSugar.mock. So let's define a new type macro easyMock[A] as mock[A](org.scalatest.mock.EasyMockSugar.mock _). It should be trivial to do the same for other mocking frameworks. To get a flavor for how this works, let's apply the easyMock type macro to types URepository, Repository, and Top, and see how they would expand:

	trait easyMock[URepository] extends componentApi[URepository] {
	override protected uRepository = org.scalatest.mock.EasyMockSugar.mock[api[URepository]]
	}
	trait easyMock[Repository] extends
	componentApi[Repository]
	easyMock[SRepository] with
	easyMock[TRepository] with
	easyMock[URepository]
	trait easyMock[Top] extends
	componentApi[Top]
	easyMock[Repository] with
	easyMock[Service]

view raw easyMock.scala hosted with ❤ by GitHub

If you follow through the logic, you will find that easyMock[Top] ends up with mock implementations of all of the leaf level components.

Now, to write a test for the UService component, we mock out the entire component hierarchy with easyMock, and then override the mock of UService with componentImpl[UService], like so:

	class UServiceSpec extends FlatSpec with easyMock[Top] with componentImpl[UService] {
	// UService specifications here
	}

view raw UServiceSpec.scala hosted with ❤ by GitHub

Conclusions

 So I seem to have set myself up for a lot of work for implementing the following macros:

• api
• impl
• component
• hasPart
• hasDependency
• hasPrivatePart
• mock

If it works out along the lines as I imagine in the above examples, this should transform doing component based dependency injection from being opaque and verbose to being clear and concise. If left alone, any type errors produced by this system for misconfigured components are going to be misleading and confusing. Especially for people who are just trying to use these macros as a library, without fully understanding how the cake pattern works. I hope there is room for me to address the error messaging as well, but I simply do not know enough about type macros at this point to say.