Skip to content

Custom data validation rules with Shapeless tags

Last updated on 22.11.2020

Incoming data validation is a problem that every API developer faces at some point in time. In this small article I’ll show, how shapeless tags can be used to express custom validation rules for Play JSON deserializator.

Full sbt project for this article can be found here.

Prerequisites

I assume the reader is familiar with basic Play JSON converters and combinators. Though it will help, it’s not a necessary knowledge to get the idea.

Some Play JSON basics can be learned here.

The problem

Let’s say our API accepts credit card payments. We define a simple data model for those cards (expiration date is omitted for brevity):

 

We’re going to receive this as a JSON field of incoming request and have to validate the credentials against some rules:

  • card number is 16 to 19 digits after removing all whitespace chars (that’s why we made it a String, not a Long)
  • cvv is 3 to 5 digits

How can we implement this? An experienced API designer would shout: “Those are not strings!”, and introduce some types. Completely valid point, a model like this:

 

would do the job. Single drawback is you’d have to implement custom serialization/deserialization for CVV and CardNumber to stay with simple JSON strings. By default, an instance of this type would serialize like:

 

Anyway, this is still good design. But what if we want them to be  String’s? For any reason, like we’d have much cleaner code that uses this CreditCard class.
Let’s set this as a requirement and see what we can do.

Simple strings will require defining custom  Reads for all types they belong to. Like if we have card number in some other type T, we’d have to duplicate that rule in Reads[T]. We don’t want that.

Here is where tags come nicely into play.

Tags

As a quick intro, a tag is a marker for an existing type that creates a new type with following properties:

  1. Values of the new type can be used as the values of original untagged type.
  2. Values of the original type can’t be treated as tagged ones. Such code won’t compile.

In this article I will use shapeless tags. A simple usage example:

 

Tags implementation is quite concise, you can look through it in the shapeless repo.

Defining a rule for tagged string

Returning to initial problem, here is how we can use tags to define custom validation rules for those credentials.

First, let’s tag our model fields:

 

Now we can define rules for tagged types:

 

Notice, that as we define Reads for a tagged type, we must return the same tagged type. So here is where we tag values.

Doing so allows us to use default Play macros to define Format for CreditCard (and any other type we’d like to put those “custom” strings in):

 

That’s it. Let’s test:

 

So it works! 🙂
We left our strings almost untouched while not losing Play Reads  granularity.

Thanks for reading!

UPDATE. Note on Play route url binders.

A nice catch from Doug Clinton in comments: this trick won’t work with Play route parameter.
I’ll quote Doug:

The problem is that the generated routes file uses classOf[T]  when creating its invoker. classOf  expects a class type, and won’t compile when the parameter type is @@[String, IdTag] , which does not have a runtime class.

Thank you for addition, Doug!

Acknowledgement

I want to say a big “thank you” to Denis Mikhaylov (aka @notxcain) for introducing this concept to me.

Published inSoftware Development

4 Comments

  1. Doug Clinton Doug Clinton

    Hi Vladimir,

    This is a really useful and interesting article. I’ve been used to using value classes for these kinds of types, which can tend to create a lot of boilerplate. I’ve just gone back to one of my projects and converted one of my Id types to use shapeless tags instead. It works really well for the Json handling, however I have hit a stumbling block with Play’s url binders. If I try to use the tagged type in an Action function the binding does not work.

    The problem is that the generated routes file uses “classOf[T]” when creating its invoker. “classOf” expects a class type, and won’t compile when the parameter type is “@@[String, IdTag]”, which does not have a runtime class.

    I’ll continue to experiment with this and see if I can find some workaround.

    • Hi, Doug!
      Thanks for your reply, you uncovered a problem I haven’t faced. We used tags only for validating json body.
      Please, keep posted, if you dig something in this direction 🙂

      • Doug Clinton Doug Clinton

        I don’t see a way around this, unfortunately. This is down to how Play generates the code for the router, and that it relies on runtime types to do the dispatch. It does highlight an interesting thing about the Scala type system, though. In Java it’s not possible (I believe) to express a type in code that is not, in some way, represented by a class at runtime. Scala, however, can express types that only exist at compile time, and the shapeless tags are an example of this. The type “String with MyTag” is known to the compiler, but no class file is generated for it so there is no runtime representation of it. Hence the routes code can’t call “classOf” on it.

        • Doug, thanks so much for your effort. I’ll add this note to the article.

Comments are closed.