Hello! This is a series of posts about Aecor — a library for building eventsourced applications in Scala in purely functional way. If this is the first time you see this series, I highly recommend to start with Introduction.
This is the closing post of the series where we’ll have some fun studying Aecor internals. Tagless final approach is what shines the most there and we’ll see a couple of really beautiful examples.
Aecor and Tagless Final
As a warm-up, let’s revisit how and where Aecor user faces tagless final. It happens at the most important and interesting place — entity behavior. For example:
Aecor rips some immediate and obvious benefits from such definition format: it can alter the effect type depending on the use case.
The ability to run behaviors in
ActionT effect, and do it in MTL style (without premature coupling to specific data structure) is a direct consequence of having tagless final behavior definition.
Next, depending on whether your entity has rejections or not, Aecor can extend the effect to
EitherT[ActionT[F, Option[S], E, ?], R, ?]. We also learned a handy wrapper Aecor offers for algebras with effect of such shape, which is
When behavior is deployed, Aecor runtime handles the
ActionT part of the effect. What’s left is a pretty simple effect:
F[Either[R, ?]] for entities with rejectable commands and plain
F for behaviors, that accept commands unconditionally. These effects are what client faces, when it sends commands to a deployed entity.
So all those effects are mixed and matched within a single behavior algebra definition. Isn’t it cool? Library user has to write zero additional code to support all these kind of execution semantics — it all works out of the box with the single tagless final behavior.
But it’s not the only place where it shines. Yes, we used several different effect types, which means that taking tagless final approach has already paid off. But all of these were familiar monad transformer-ish effects, which everyone is kinda used to.
Aecor runtime has a component, that leverages the same tagless final behavior in a much more unconventional way. On the surface this component might look boring, but thanks to clever design it’s actually quite remarkable.
We briefly discussed wire protocol in Part 3, but let’s revisit it’s main purpose.
Entities are deployed into a cluster runtime, and it can easily happen that the node handling the request (node A) isn’t the node that runs the entity instance (node B). In this case:
- The original node A has to encode the command and send it to the node B to handle.
- Node B has to decode the command on arrival and run it through the entity.
- Then node B has to encode command execution result and send it back to node A.
- Node A has to decode received result and continue handling original request.
As you can see, there’s a lot of encoding and decoding going on — all of that is a responsibility of a wire protocol. By providing an instance of wire protocol for an algebra we get an ability to execute commands over the wire.
Here’s how the typeclass looks like with all dependencies:
It’s a very dense definition that took me quite a lot to digest. So let’s go step by step.
Invocation is just a partially applied call to some specific method on behavior
M that returns a value of type
A. It “partial” in an unusual way though: it has all the arguments applied, but the instance of the algebra itself will be selected later, when we
run the invocation.
For example, let’s take
place method on our
Booking behavior and create a sample invocation:
If we further draw the analogy to actor-based eventsourcing, invocation is nothing more than a command object. There we create a command message by filling in all the required data, and then send it to some entity actor.
Same thing here — we’re creating an
Invocation object, but instead of sending it we use
run to execute it on our entity behavior instance. Of course
Invocation is defined in a much more generic way, so I’m intentionally overspecializing it here to commands an entities to give a better understanding of the intent.
To summarize and make it simple:
Invocation is a specific command, that can be executed on some entity later.
Not a lot to say here actually.
PairE is just a wrapper class to have one value in two contexts at the same time.
E here means “existential”: the value type is a type member. Later we’ll see why it’s important.
By the way, a context here is not always an effect — it can be a typeclass instance. A simple example would be
PairE[List, Ordering]: for some type
A it provides a list of values coupled with an
Ordering instance for that same type.
Decoder typeclasses here, as well as
BitVector data structure are taken from scodec library, as you can see from the imports. Don’t worry if you’re not familiar with this library — it’s designed in way you’d probably expect it to be. In case you used circe, just switch
BitVector in your head, and you get scodec typeclasses roughly.
Client wire protocol
Now as we discussed all the components, let’s fight the boss — wire protocol typeclass itself. First of all, for the sake of better understanding, I would prefer to rename the methods of this typeclass the following way:
So protocol works both at
client which sends the command, and
server that runs the entity instance, handles the command and sends back the result.
IMO, these names are much closer to how things work. Although overall naming is not a huge concern here — it’s mostly internal API when it comes to real world use.
Let’s look at the client, since command execution starts there. Wire protocol client is just a custom interpreter for our behavior algebra with a very unusual “effect”:
(BitVector, Decoder[?]). What the hell is happening here?
To answer this question let’s see how client interpreter works in case of our example booking behavior:
So for each command client interpreter returns:
- An encoded version of command in the form of
BitVector. It can be sent over the wire to the server.
- A decoder for command result. Later, when server responds with the encoded command result, client will be able to decode it.
In our simple case, most of the commands return
Unit, so result decoder is kinda useless for them. But when rejections come into play, we have to disambiguate success and rejection, so it has it’s valid use even for commands that result in
So here you go — another place where it paid off to have behavior in a form of tagless final algebra. But we’re not done yet, let’s try to understand the
Server wire protocol
On the server side of the wire protocol typeclass we have a single decoder, but not a trivial one.
First, it decodes the command into an
Invocation. It makes sense — as we discussed, invocation is a command object, that can be executed on a behavior. Client has all the command data, but not the entity instance, so it encodes all the data into an invocation and sends it over the wire. Server has an entity instance, so it now can
run the invocation.
Additionally, server attaches a result encoder to each command invocation: after entity responds, the response has to be encoded before sending it back to the client.
The invocation and result encoder are connected using
PairE. And this is where it becomes important
PairE is existential in the value type. Let’s imagine it had value type parameter instead. Then what type should we put there for the server?
Let’s remember that we have a single
server for the whole behavior algebra. It means, that depending on the command, the type of value is gonna vary:
Unit for commands like
confirm and something else for “reading” command like
So hiding the value type of
PairE as a type member allows
server to use different underlying type for each command, while still have a compile-time proof that invocation result on the left of the pair can be encoded with the encoder on the right.
Wire protocol review
Let’s zoom out and see the whole command handling process in types. Say we’re calling
status command from node A (the
client) and the corresponding entity instance is running on node B (the
server). Here’s what happens:
- Both nodes have an instance
- Node A calls
val (commandBytes, responseDecoder) = W.client.status.
- It sends the command
commandBytes: BitVectorto the shard region and keeps the decoder for later.
- Node B receives the command
W.server.decode(bytes). If decoding is successful it gets a
pair: PairE[...]of invocation and result encoder.
- Node B obtains proper instance of booking entity and runs the invocation:
val res: F[BookingStatus] = pair.first.run(entity).
- Then it encodes the result and sends it back to node A (by replying to the shard region):
- Node A receives the result and uses
responseDecoderto decode it back into
BookingStatus. It is then handed back to original caller.
Booking entity in the form of tagless final algebra helped twice here:
- We were able to completely reuse it on the
- It allowed to define
Invocationin such a nice generic way while keeping all the types safe.
I hope this post demonstrated how much Aecor wins from using final tagless. It’s also a great reminder of how much power this pattern gives. If you accidentally hear someone telling that TF is just modern fancy replacement for Java interfaces, show them this series and this post in particular.
This concludes my post series. It was a great journey for me, and I hope you liked it as well. And if it convinced someone to try Aecor on a pet project or even at work — I would be immensely happy to hear that. Please, reach me out on twitter or in comments to share your experience or ask questions.
Thanks for reading!
Peace. Love. Referential Transparency.