Scala.js In A Big Web Application Talk

My first talk ever at a big IT conference was about my production experience with Scala.js. It nicely concluded a full year of extensive Scala.js frontend development, that my team was doing at Evolution Gaming starting from May 2016.

I tried to lay out various pros and cons of using Scala.js in a big browser SPA, based on that real experience. I also tried to go into details and provide examples for most of the points.

Talk took place on 16.05.2017 at Riga Dev Days conference in Riga, Latvia.

Slides (english, Speakerdeck)
Video (48m, english):

Cross-platform polymorphic date/time values in Scala with type classes

At Evolution Gaming me and Artsiom work on internal scheduling application, that has a huge ScalaJS frontend. We have to deal with lots of complex date/time logic, both on backend and browser sides.

I quickly realised, that sharing same business logic code across platforms would be a massive advantage. But there was a problem: there were (and still is) no truly cross-platform date/time library for Scala/ScalaJS.

After a small research I settled with a type class-based solution that provides cross-platform java.time.* -like values with full TimeZone/DST support. In this post we will:

  • take a quick look at the current state of date/time support in Scala/ScalaJS;
  • see how to get cross-platform date/time values today with the help of type classes.

Described approach works quite well in our application, so I extracted the core idea into a library, called DTC. If you’re a “Gimme the code!” kind of person, I welcome you to check out the repo.

Prerequisites

I assume, that reader is familiar with ScalaJS and how to set up a cross-platform project. Familiarity with type classes is also required.

The Goal

There’s no solution without a goal. Precise goal will also provide correct context for reasonings in this article. So let me state it.

My primary goal is to be able to write cross-platform code that operates on date/time values with full time zone support.

This also means that I will need implementation(s) that behave consistently across JVM and browser. We’re Scala programmers, so let’s choose JVM behaviour semantics as our second goal.

Current state of date/time in Scala and ScalaJS

So we have our goal, let’s see how we can achieve it.

In this section we’ll go over major date/time libraries and split them into 3 groups: JVM-only, JS-only and cross-platform.

JVM-only

We won’t spend too much time here, everything is quite good on JVM side: we have Joda and Java 8 time package. Both are established tools with rich API and time zone support.

But they can’t be used in ScalaJS code.

JS-only

We’re looking at JS libraries, because we can use them in ScalaJS through facades. When it comes to date/time calculations, there are effectively two options for JavaScript: plain JS Date and MomentJS library.

There’re things that are common for both and make them quite problematic to use for our goal:

  • values are mutable;
  • semantics are different from JVM in many places. For example, you can call date.setMonth(15) , and it will give you a same date in March next year!

There’s also a JS-Joda project, which is not so popular in JS world, but has much more value to JVM developers, because it brings Java8 time semantics to Javascript.

JS Date

JS Date is defined by ECMA Script standard and is available out of the box in any JS runtime. But it has several weaknesses. Major ones are:

  • quite poor API;
  • time zone support is not universal: behaviour depends on environment time zone, and you can’t ask for a datetime value in an arbitrary zone.

Since JS Date is a part of language standard, ScalaJS bindings for it are provided by ScalaJS standard library.

MomentJS

Despite MomentJS values are mutable and still have minor bugs in calculations, it’s quite a good library, especially, if you need full time zone support.

It also has a ScalaJS facade.

JS-joda

JS-joda is implementation of a nice idea: porting java.time.* functionality to Javascript. Though I’ve never used this project, it looks like an established and well-maintained library.

ScalaJS facade is also in place, so you can definitely give it a try in your Scala project.

The only problem with regard to our goal is it still lacks proper DST support. But it’s already in progress, so you can expect it to be fully functional in observable future.

Cross-platform date/time libraries

After a small research, I found three options. Let’s see them in detail.

Scala-js-java-time

This library is the future of cross-platform date/time code. It’s effectively Java 8 time, written from scratch for ScalaJS.

At the time of writing this post, scala-js-java-time already provides LocalTime, LocalDate, Duration , and a handful of other really useful java.time.* classes (full list here).

It means, that you can use these classes in cross-compiled code and you won’t get linking errors: in JVM runtime original java.time.* classes will be used, and JS runtime will be backed by scala-js-java-time implementations.

Problem here, is that we need LocalDateTime and ZonedDateTime in ScalaJS. And they are not there yet.

Spoiler: we’ll be using scala-js-java-time in our final solution for the problem.

Scala Java-Time

Scala Java-Time is a fork of ThreeTen backport project. So it’s main purpose is to provide java.time.* -like functionality on Java 6 & 7.

It is also compiled to ScalaJS, which means we can write cross-platform code with it. And we can even use (to some extent) LocalDateTime!

The only problem is it doesn’t support time zones for ScalaJS yet (providing this support is the main focus of the project now, though).

So this library is close, but still misses our goal by a tiny bit.

Soda time

Soda time is a port of Joda to ScalaJS.

It’s in early development stages and also doesn’t have time zones in ScalaJS, but I still added it to the list, because developers took an interesting approach: they are converting original Joda code with ScalaGen.

So the resulting code really smells Java, but I’m still curious of what this can develop into.

Idea? No, the only option

The reason I’ve given the overview of currently available libraries is simple: it makes clear that there’s only one possible solution to the problem.

There’s no cross-platform library with full time zone support. And for JavaScript runtime there’s only MomentJS, that really fits our requirements. All this leaves us with nothing, except following approach:

  1. We define some type class, that provides rich date/time API. It’s a glue that will allow us to write cross-platform code.
  2. All code, that needs to operate date/time values, becomes polymorphic, like this:
  3. We provide platform-dependent type class instances: java.time.* -based for JVM and MomentJS-based for ScalaJS.
  4. We define common behaviour laws to test the instances against. This will guarantee, that behaviour is consistent across platforms.
  5. MomentJS API is powerful, but it has to be sandboxed and shaped to:
    • provide immutable values;
    • provide JVM-like behaviour;
  6. There’s a limitation, that we can’t overcome without some manual implementation: both JS libraries don’t support nano seconds. So we’ll have to live with milliseconds precision.

We won’t go over all of these points in this article. DTC library does the heavy lifting for all of them. In following sections we’ll just glance over the core implementation and example.

DateTime type class

Let’s just take a small subset of java.time.LocalDateTime API and lift it into a generic type class. We’ll use simulacrum, to avoid common boilerplate:

 

First of all, a total order for DateTime values is defined. So we can extend cats.kernel.Order and get all it’s goodies out of the box.

Second, thanks to scala-js-java-time, we can use LocalTime and LocalDate to represent parts of the value. Also, we can use Duration for addition operations.

For now, let’s just view it as a glue for LocalDateTime. We’ll get to time zone support a bit later.

Cross-compiled business logic

Having our new type class, let’s define some “complex” logic, that needs to be shared across JVM and browser.

 

With syntax extensions in place, the code looks quite nice.

More over, you can notice, that nothing here says, if time should be local or zoned. Code is polymorphic, and we can use different kinds of date/time values, depending on the context.

Now let’s get to the flesh and bones: type class instances.

Type class instances

Let’s start with JVM instance, as it’s going to be quite simple. Follow comments in code for details.

 

With MomentJS it’s going to be much more interesting, because we’ve obliged ourselves to provide values, that are comfortable to work with for a functional programmer.

To enforce immutability, we won’t expose any moment APIs directly. Instead, we’re going to wrap moment values in a simple object, that will be immutable:

 

Several notable things here:

  1. We make both constructor and underlying value private to make sure there’s no way to modify object internals. We’ll provide a custom constructor later.
  2. Notice month value adjustment to provide JVM-like behaviour. You will see much more of such things in DTC, I even had to write a couple of methods from scratch.
  3. To compare two moment values, we use their raw timestamps.

Now it’s trivial to define DateTime instance for our MomentLocalDateTime:

 

Now have everything to run our generic domain logic on both platforms. I’ll leave it as an exercise for my dear reader.

Now let’s discuss some aspects of making this thing work for zoned values as well.

Time Zone support

Not much time is needed to realise, that we need separate type classes for local and zoned values. Reasons are:

  • They obey different laws. For example, you can’t expect a zoned value to have same local time after adding 24h to it.
  • They have different constructor APIs. Zoned value needs time zone parameter to be properly constructed.
  • Zoned values should provide additional APIs for zone conversions.

On the other side, most of querying, addition and modification APIs are common to both kinds of date/time values. And we would like to take advantage of that in cases we don’t really care about a kind of the value and wish to allow using both.

This leads us to following simple hierarchy:

  1. LawlessDateTimeTC (which we initially called DateTime) that contains common methods, specific to all date/time values.
  2. LocalDateTimeTC and ZonedDateTimeTC will extend LawlessDateTimeTC and provide kind-specific methods (constructors, for example).

This -TC suffix is ugly, but name clash in JVM code is worse :).

We will also have to provide a cross-compiled wrapper for time zone identifiers, because java.time.ZoneId is not yet provided by scala-js-java-time, and we don’t really want to pass raw strings around.

Everything else is just an evolution of core idea. Full implementation and more examples are available in the DTC repo.

Note on polymorphism

A side-effect of this solution, is that all your code becomes polymorphic over the specific date/time type. While most of the time you’re going to use single kind of time (zoned or local), there are cases when polymorphism becomes useful.

For example, in an event-sourced system, you may require zoned values for most of the calculations within the domain, as well as commands. But, at the same time, it can be a good idea to store events to journal with UTC values instead.
With type class-based approach, you can use same data structures for both purposes, by just converting between type parameters of different kinds.

Conclusion

Though polymorphic code can look scary for some people, described approach give us following advantages:

  1. Truly cross-platform code, that operates on rich date/time APIs with time zone support.
  2. Polymorphism over specific kind of date/time values.

If you’re working with date/time values in Scala on a daily basis, please, give DTC a try and tell me what you think!

Thanks for reading! 🙂

How to make an idiomatic Javascript library with Scala.js

UPDATE: Article was updated to Scala.js version 0.6.7, which vastly simplifies Promises related section.

Scala.js opens a big world of frontend development to Scala fans. Most of the time Scala.js project ends up being an independent browser or Node.js application. But there are cases, where you would want to make a library for general frontend developers.

There’re some interesting gotchas in writing Scala.js library such way, that it will be natural to use for an average JS developer. In this article we will develop a simple Scala.js library (code) to work with Github API and will focus on the idiomaticity of it’s JS API.

But first, I’m sure you want to ask

Why would I do that?

Reasonable one.
You should consider developing such a library if:

  1. A client application for your Scala API backend already exists, and it’s native Javascript.

    Sad, you will hardly have a chance to write it from scratch with Scala.js, but at least it makes sense to write a communication / interpretation library for those guys.
    It will simplify interaction between you and frontenders in two ways:

    • You can hide some tricky client-side logic there, and expose much simpler API.
    • Your library can work on model classes, defined in backend project (see Cross-Building). You get typesafe isomorphic code almost for free and can forget about client-server protocol synchronization problems.
  2. You develop a public API for developers, like Facebook’s Parse.

    A perfect solution for a Javascript API SDK. See all the advantages of the previous case.

Recently, I’ve faced the first case. Moreover, our REST-like JSON API has two different browser based clients. So developing an isomorphic library was a logical choice.

Let’s start with our library.

Requirements

  1. As Scala developers we want to write all business logic in familiar functional style, being able to use all the handy Scala features.
  2. Library API must be natural for JS developers.

Setting up project

Such a project doesn’t differ from a regular Scala.js app. If you are new to Scala.js, you can read this tutorial first.

Folder structure:

resources/index-fastopt.html — a page that will just load our library and  resources/demo.js file, that will test the API.

API

The purpose of the library is to simplify Github API interaction. For simplicity, we’ll implement only one feature – loading users and their repos by login.

So it’s, basically, a public method and a pair of model classes, that store results (value objects). Model is the place we’ll start writing code.

Model

Let’s define model classes like this:

Everything is easy: User has some repos, a repo is either an origin or a fork. Good old Scala model. How do we export that to JS developers?

For a full reference of exporting features see Export Scala.js APIs to Javascript

Object creation API

Let’s look at, how we should expose such API. It seems an easy solution to expose the constructor:

But this won’t work. You don’t have Option constructor exported, so there’s no way to create homepage  parameter.

Moreover, there are additional limitation for case classes: You can’t export two case constructors that are under inheritance relationship. This code won’t even compile:

So what is the best choice? I found that it’s best to leave constructors alone and just expose JS-friendly factory methods, like this:

Here with the help of js.UndefOr we handle optional parameter JS way: you can pass a String , or don’t pass anything:

Note on caching Scala objects

Making client call  Github() every time is not the best API option. If you don’t need laziness, you can cache it upon startup:

Reading model properties

Seamless types

If we now try to read fork’s name, we’ll get undefined . Fair enough, it’s not exported. Let’s export model properties.

There’re no problem with native types like String , Boolean and Int . They can be exported as is:

A case class field can be exported with @(JSExport@field) annotation. An example for  forks property, that’s not a member of Repo trait:

Option

But as you already can expect, there’s a problem with
homepage: Option[String] . Well, we can export it, but this would be useless – to get the actual string value JS developer would have to call something on an option, and nothing is exported.

On the other side, we’d like to keep Option in place, so that our Scala code, that manipulates value classes, remains powerful and simple.  js.UndefOr[T] API is way less expressive.

A solution here is to export a special JS-friendly getter method:

Let’s try it out, it works:

We retained our beloved Option monad, and exported nice and clean JS API. Great!

List

User.repos is a List , and has the same problems with being exported. Solution here is the same too: we’ll just export it as a plain JS Array :

Now we can even map them 🙂 :

Sum types

There’s still one problem with  Repo trait. As we’re not exporting constructors, given a  Repo instance, JS developer can’t figure out, what kind of  Repo it is.

In Javascript there’s no pattern matching and using inheritance is not so popular, sometimes even questionable. So we have several options here.

  1. Depending on the context, provide methods like isFork: Boolean or  hasForks: Boolean at the base level. This is perfectly fine, but not general enough.
  2. Add  type: String (or whatever name feels suitable to you) property to all sum types.

I choose the second one, because it can be abstracted and used throughout the whole codebase. Here’s how it can be done. Let’s declare a mixin that exports a type property:

We have to use a different name for scala definition, because it’s a reserved word.
That’s it! We can now mix it in:

… and use it:

To make this a little safer, we can store type names constants, that can be compared with instance type property. This can be done typesafe:

Having this helper class we can define these constants in our Github global for example:

Now we can avoid strings in Javascript! An example:

That’s how we dealt with sum types.

What if I can’t change object, that I want to export?

This is a case if you want to (maybe, partially) export your cross-built model classes or other imported library objects. The solution is the same to Option and List with the only difference: you have to implement JS-friendly replacement classes and conversion yourself.

An important rule here is to use JS replacements only for export ( Scala => JS) and instance creation ( JS => Scala ). All business logic must be implemented with pure Scala classes.

Let’s say you have a Commit class, that you can’t change:

Here what you can do to export it:

Then, for example, a Branch  class, that you own, would look like this:

Since in JS environment commits are represented with CommitJS objects, a factory method for Branch  would be:

Of-course, this workaround is not a beautiful thing, but at least it’s type checked. That’s why I think it’s preferable to view your library not only as a value-classes proxy, but as a facade that hides redundant details and simplifies the API. That way you won’t even need to export the underlying model.

That’s all for exporting model. Let’s move on to the more interesting part – loading the content from Github API.

AJAX

Implementation

For the brevity purposes we will use scalajs-dom Ajax extension as a “network” layer. Let’s for some time forget about how we’re going to export things, let’s just implement the API.

For the simplicity, we’ll put everything AJAX-related into API object. It will have two public methods: for loading user and loading repos.

We will also implement a DTO layer, to decouple API from the model. For type-safe error handling we’ll use Xor type from Cats library. The result type of the method call will be Future[String Xor DTO], where DTO is the type of requested data and String will represent error.

I’ve mentioned everything for this listing to be more understandable, here it is:

Deserialization code is hidden, it’s not interesting. The load method returns string error, if response code is not 200, otherwise it converts the response data to JSON and then to DTO’s.

Now we can convert our API results into model classes.

Here we use a monad transformer to combine these “disjunctioned” futures, and then convert DTO’s into model classes.

Well, that is quite idiomatic functional Scala, lots of pleasure. Now let’s think about how we will export  loadUser method to library users.

Share the Future

To follow the article goals we need to answer the question: what is the idiomatic way to handle asynchronous call in Javascript? I already hear experienced frontenders laughing, because there are no such thing. Callbacks, event emitters, promises, fibers, generators, async/await — all of them are somehow valid approaches. So what should we choose?

I think, the closest thing to Scala Future in Javascript are Promises. Promises are very popular and are already native in most modern browsers. So we’ll stick with them.

First, we must let our Scala code know about those promises. Until Scalajs 0.6.7 we would have to use Promise typed facade from scalajs-dom. But with Scalajs 0.6.7 things became much easier, we will just use the “standard” Promises.

All we have to do now is to convert a Future into Promise. Again, since version 0.6.7 this is not more a problem — there’s a toJSPromise converter in JSConverters . We will just need to help it with the left side of our Xor — convert it to a failed Future to get a rejected Promise:

So let’s share the promise with our JS friends! As usual, we put it to Github object, near the original method:

Here in case of failed future we’re rejecting promise with the exception message. That’s all, we can test the whole API now:

Well, we did it! We can use Futures and everything else we are got used to — and still export idiomatic JS API.

For more API usage examples see full demo.js. To play more with the project, just fetch the repo, then build and run it.

Conclusion

Putting it all together, here are some general advice on writing a Javascript library with Scala.js:

  • Cache exported objects on startup.
  • Export seamless types “as is”.
  • Don’t export Options, Lists and other Scala standard. Put a JS-friendly getter nearby, that converts to  js.UndefOr and js.Array. BTW, same with  Map => js.Dictionary.
  • Don’t export constructors. Use a JS-friendly factory method. JS-friendly means it accepts js.* types and converts it to Scala standard types.
  • Mixin a string type property into sum types.
  • Export Future s as js.Promise s
  • Scala first. You are a Scala developer, so don’t limit yourself in any way: use all the power you like. You know now, that you’ll be able to export it.

Links