by

Hello! This is a series of posts about Aecor — a library for building eventsourced applications in Scala in purely functional way. If this is the first time you see this series, I highly recommend to start with Introduction.

This is the closing post of the series where we’ll have some fun studying Aecor internals. Tagless final approach is what shines the most there and we’ll see a couple of really beautiful examples.

Reading parts 1, 2 and 3 of the series is recommended before starting this one.

Aecor and Tagless Final

As a warm-up, let’s revisit how and where Aecor user faces tagless final. It happens at the most important and interesting place — entity behavior. For example:

Aecor rips some immediate and obvious benefits from such definition format: it can alter the effect type depending on the use case.

The ability to run behaviors in ActionT effect, and do it in MTL style (without premature coupling to specific data structure) is a direct consequence of having tagless final behavior definition.

Next, depending on whether your entity has rejections or not, Aecor can extend the effect to EitherT[ActionT[F, Option[S], E, ?], R, ?]. We also learned a handy wrapper Aecor offers for algebras with effect of such shape, which is EitherK.

When behavior is deployed, Aecor runtime handles the ActionT part of the effect. What’s left is a pretty simple effect: F[Either[R, ?]] for entities with rejectable commands and plain F for behaviors, that accept commands unconditionally. These effects are what client faces, when it sends commands to a deployed entity.

So all those effects are mixed and matched within a single behavior algebra definition. Isn’t it cool? Library user has to write zero additional code to support all these kind of execution semantics — it all works out of the box with the single tagless final behavior.

Aecor can use a behavior you defined in many contexts and for many purposes.
It’s really simple, thanks to tagless final format.

But it’s not the only place where it shines. Yes, we used several different effect types, which means that taking tagless final approach has already paid off. But all of these were familiar monad transformer-ish effects, which everyone is kinda used to.

Aecor runtime has a component, that leverages the same tagless final behavior in a much more unconventional way. On the surface this component might look boring, but thanks to clever design it’s actually quite remarkable.

It’s called WireProtocol.

Wire Protocol

We briefly discussed wire protocol in Part 3, but let’s revisit it’s main purpose.

Entities are deployed into a cluster runtime, and it can easily happen that the node handling the request (node A) isn’t the node that runs the entity instance (node B). In this case:

  1. The original node A has to encode the command and send it to the node B to handle.
  2. Node B has to decode the command on arrival and run it through the entity.
  3. Then node B has to encode command execution result and send it back to node A.
  4. Node A has to decode received result and continue handling original request.

As you can see, there’s a lot of encoding and decoding going on — all of that is a responsibility of a wire protocol. By providing an instance of wire protocol for an algebra we get an ability to execute commands over the wire.

Here’s how the typeclass looks like with all dependencies:

It’s a very dense definition that took me quite a lot to digest. So let’s go step by step.

Invocation

Invocation is just a partially applied call to some specific method on behavior M that returns a value of type A. It “partial” in an unusual way though: it has all the arguments applied, but the instance of the algebra itself will be selected later, when we run the invocation.

For example, let’s take place method on our Booking behavior and create a sample invocation:

If we further draw the analogy to actor-based eventsourcing, invocation is nothing more than a command object. There we create a command message by filling in all the required data, and then send it to some entity actor.

Same thing here — we’re creating an Invocation object, but instead of sending it we use run to execute it on our entity behavior instance. Of course Invocation is defined in a much more generic way, so I’m intentionally overspecializing it here to commands an entities to give a better understanding of the intent.

To summarize and make it simple: Invocation is a specific command, that can be executed on some entity later.

PairE

Not a lot to say here actually. PairE is just a wrapper class to have one value in two contexts at the same time. E here means “existential”: the value type is a type member. Later we’ll see why it’s important.

By the way, a context here is not always an effect — it can be a typeclass instance. A simple example would be PairE[List, Ordering]: for some type A it provides a list of values coupled with an Ordering instance for that same type.

scodec

Encoder and Decoder typeclasses here, as well as BitVector data structure are taken from scodec library, as you can see from the imports. Don’t worry if you’re not familiar with this library — it’s designed in way you’d probably expect it to be. In case you used circe, just switch Json to BitVector in your head, and you get scodec typeclasses roughly.

Client wire protocol

Now as we discussed all the components, let’s fight the boss — wire protocol typeclass itself. First of all, for the sake of better understanding, I would prefer to rename the methods of this typeclass the following way:

So protocol works both at client which sends the command, and server that runs the entity instance, handles the command and sends back the result.

IMO, these names are much closer to how things work. Although overall naming is not a huge concern here — it’s mostly internal API when it comes to real world use.

Let’s look at the client, since command execution starts there. Wire protocol client is just a custom interpreter for our behavior algebra with a very unusual “effect”: (BitVector, Decoder[?]). What the hell is happening here?

To answer this question let’s see how client interpreter works in case of our example booking behavior:

So for each command client interpreter returns:

  1. An encoded version of command in the form of BitVector. It can be sent over the wire to the server.
  2. A decoder for command result. Later, when server responds with the encoded command result, client will be able to decode it.

In our simple case, most of the commands return Unit, so result decoder is kinda useless for them. But when rejections come into play, we have to disambiguate success and rejection, so it has it’s valid use even for commands that result in Unit values.

So here you go — another place where it paid off to have behavior in a form of tagless final algebra. But we’re not done yet, let’s try to understand the server part.

Server wire protocol

On the server side of the wire protocol typeclass we have a single decoder, but not a trivial one.

First, it decodes the command into an Invocation. It makes sense — as we discussed, invocation is a command object, that can be executed on a behavior. Client has all the command data, but not the entity instance, so it encodes all the data into an invocation and sends it over the wire. Server has an entity instance, so it now can run the invocation.

Additionally, server attaches a result encoder to each command invocation: after entity responds, the response has to be encoded before sending it back to the client.

The invocation and result encoder are connected using PairE. And this is where it becomes important PairE is existential in the value type. Let’s imagine it had value type parameter instead. Then what type should we put there for the server?

Let’s remember that we have a single server for the whole behavior algebra. It means, that depending on the command, the type of value is gonna vary: Unit for commands like place or confirm and something else for “reading” command like status.

So hiding the value type of PairE as a type member allows server to use different underlying type for each command, while still have a compile-time proof that invocation result on the left of the pair can be encoded with the encoder on the right.

Wire protocol review

Let’s zoom out and see the whole command handling process in types. Say we’re calling status command from node A (the client) and the corresponding entity instance is running on node B (the server). Here’s what happens:

  1. Both nodes have an instance W: WireProtocol[Booking].
  2. Node A calls val (commandBytes, responseDecoder) = W.client.status.
  3. It sends the command commandBytes: BitVector to the shard region and keeps the decoder for later.
  4. Node B receives the command BitVector and runs W.server.decode(bytes). If decoding is successful it gets a pair: PairE[...] of invocation and result encoder.
  5. Node B obtains proper instance of booking entity and runs the invocation: val res: F[BookingStatus] = pair.first.run(entity).
  6. Then it encodes the result and sends it back to node A (by replying to the shard region): res.map(pair.second.encode).flatMap(sendToNodeA)
  7. Node A receives the result and uses responseDecoder to decode it back into BookingStatus. It is then handed back to original caller.

Having our Booking entity in the form of tagless final algebra helped twice here:

  1. We were able to completely reuse it on the client.
  2. It allowed to define Invocation in such a nice generic way while keeping all the types safe.

Wrap up

I hope this post demonstrated how much Aecor wins from using final tagless. It’s also a great reminder of how much power this pattern gives. If you accidentally hear someone telling that TF is just modern fancy replacement for Java interfaces, show them this series and this post in particular.

This concludes my post series. It was a great journey for me, and I hope you liked it as well. And if it convinced someone to try Aecor on a pet project or even at work — I would be immensely happy to hear that. Please, reach me out on twitter or in comments to share your experience or ask questions.

Thanks for reading!
Peace. Love. Referential Transparency.