Last updated on 22.11.2020
Incoming data validation is a problem that every API developer faces at some point in time. In this small article I’ll show, how shapeless tags can be used to express custom validation rules for Play JSON deserializator.
Full sbt project for this article can be found here.
Prerequisites
I assume the reader is familiar with basic Play JSON converters and combinators. Though it will help, it’s not a necessary knowledge to get the idea.
Some Play JSON basics can be learned here.
The problem
Let’s say our API accepts credit card payments. We define a simple data model for those cards (expiration date is omitted for brevity):
1 |
case class CreditCard(number: String, holder: String, cvv: String) |
We’re going to receive this as a JSON field of incoming request and have to validate the credentials against some rules:
- card number is 16 to 19 digits after removing all whitespace chars (that’s why we made it a String, not a Long)
- cvv is 3 to 5 digits
How can we implement this? An experienced API designer would shout: “Those are not strings!”, and introduce some types. Completely valid point, a model like this:
1 2 3 |
case class CardNumber(value: String) case class CVV(value: String) case class CreditCard(number: CardNumber, holder: String, cvv: CVV) |
would do the job. Single drawback is you’d have to implement custom serialization/deserialization for CVV and CardNumber to stay with simple JSON strings. By default, an instance of this type would serialize like:
1 2 3 4 5 6 7 8 9 |
{ "number": { "value": "1234567812345678" }, "holder": "Card Holder", "cvv": { "value": "666" } } |
Anyway, this is still good design. But what if we want them to be
String’s? For any reason, like we’d have much cleaner code that uses this
CreditCard class.
Let’s set this as a requirement and see what we can do.
Simple strings will require defining custom Reads for all types they belong to. Like if we have card number in some other type T, we’d have to duplicate that rule in Reads[T]. We don’t want that.
Here is where tags come nicely into play.
Tags
As a quick intro, a tag is a marker for an existing type that creates a new type with following properties:
- Values of the new type can be used as the values of original untagged type.
- Values of the original type can’t be treated as tagged ones. Such code won’t compile.
In this article I will use shapeless tags. A simple usage example:
1 2 3 4 5 6 7 8 9 10 11 |
import shapeless.tag._ trait JustTag // this is tag type // this code is ok, can use tagged String as a plain one def onlyTagged(value: String @@ JustTag): String = s"Tagged string: $value" onlyTagged("plain string") // Compiler error val tagged = tag[JustTag]("tagged") // tag a string value with JustTag onlyTagged(tagged) // OK |
Tags implementation is quite concise, you can look through it in the shapeless repo.
Defining a rule for tagged string
Returning to initial problem, here is how we can use tags to define custom validation rules for those credentials.
First, let’s tag our model fields:
1 2 3 4 5 6 7 8 9 10 |
object tags { sealed trait CardNumber sealed trait CVV } import tags._ case class CreditCard(number: String @@ CardNumber, holder: String, cvv: String @@ CVV) |
Now we can define rules for tagged types:
1 2 3 4 5 6 7 8 9 10 11 12 |
implicit lazy val readsCardNumber: Reads[String @@ CardNumber] = __.read[String] .map(_.replaceAll("\\s", "")) .collect(ValidationError("Invalid card number")) { case num if (16 to 19).contains(num.length) => tag[CardNumber](num) } implicit lazy val readsCVV: Reads[String @@ CVV] = __.read[String]( Reads.pattern("\\d{3,5}".r, "Invalid CVV") ).map(tag[CVV](_)) |
Notice, that as we define Reads for a tagged type, we must return the same tagged type. So here is where we tag values.
Doing so allows us to use default Play macros to define Format for CreditCard (and any other type we’d like to put those “custom” strings in):
1 |
implicit lazy val cardFormat = Json.format[CreditCard] |
That’s it. Let’s test:
1 2 3 4 5 6 7 8 9 10 11 12 13 |
val validCard = Json.obj( "number" -> "1111 2222 3333 4444", "holder" -> "CARD HOLDER", "cvv" -> "666" ) val malformedNumberCard = validCard + ("number" -> JsString("111")) val malformedCVVCard = validCard + ("cvv" -> JsString("6 6 6")) println(Json.fromJson[CreditCard](validCard)) // JsSuccess println(Json.fromJson[CreditCard](malformedNumberCard)) // Invalid card number println(Json.fromJson[CreditCard](malformedCVVCard)) // Invalid CVV |
So it works! 🙂
We left our strings almost untouched while not losing Play
Reads granularity.
Thanks for reading!
UPDATE. Note on Play route url binders.
A nice catch from Doug Clinton in comments: this trick won’t work with Play route parameter.
I’ll quote Doug:
The problem is that the generated routes file uses classOf[T] when creating its invoker. classOf expects a class type, and won’t compile when the parameter type is @@[String, IdTag] , which does not have a runtime class.
Thank you for addition, Doug!
Acknowledgement
I want to say a big “thank you” to Denis Mikhaylov (aka @notxcain) for introducing this concept to me.
Hi Vladimir,
This is a really useful and interesting article. I’ve been used to using value classes for these kinds of types, which can tend to create a lot of boilerplate. I’ve just gone back to one of my projects and converted one of my Id types to use shapeless tags instead. It works really well for the Json handling, however I have hit a stumbling block with Play’s url binders. If I try to use the tagged type in an Action function the binding does not work.
The problem is that the generated routes file uses “classOf[T]” when creating its invoker. “classOf” expects a class type, and won’t compile when the parameter type is “@@[String, IdTag]”, which does not have a runtime class.
I’ll continue to experiment with this and see if I can find some workaround.
Hi, Doug!
Thanks for your reply, you uncovered a problem I haven’t faced. We used tags only for validating json body.
Please, keep posted, if you dig something in this direction 🙂
I don’t see a way around this, unfortunately. This is down to how Play generates the code for the router, and that it relies on runtime types to do the dispatch. It does highlight an interesting thing about the Scala type system, though. In Java it’s not possible (I believe) to express a type in code that is not, in some way, represented by a class at runtime. Scala, however, can express types that only exist at compile time, and the shapeless tags are an example of this. The type “String with MyTag” is known to the compiler, but no class file is generated for it so there is no runtime representation of it. Hence the routes code can’t call “classOf” on it.
Doug, thanks so much for your effort. I’ll add this note to the article.