My Four Year Quest For Perfect Scala.js UI Development
Tags: scala, javascript, react, showdev
Published: 2020-03-19 10:18:29

[Laminar](https://laminar.dev) is my Scala.js UI library that was intended as an alternative to React.js but turned out to be its polar opposite in entirely too many ways as I slowly figured out what perfect UI development is to me, and how to achieve it (spoiler alert: not in that order).

Having just released Laminar v0.8.0 I wanted to reflect for a moment on what this release means for me and hopefully for the wider Scala.js ecosystem, and why I'm doing this in the first place.


## Scala.js

[Scala.js](http://www.scala-js.org/) is an amazing platform to build UI-heavy applications. It is nothing like Typescript or [Flow](https://flow.org/). You're not writing sort-of-typed Javascript, you're writing bona fide Scala, with all the elegant simplicity, safety and expressiveness that it allows and encourages.

That was my realization in 2016 when I first tried Scala.js. What followed soon afterwards was a similarly strong dissatisfaction with the Scala.js UI libraries available at the time. I liked React.js, and technically I _could_ write in React in Scala.js using [ScalaJS-React](https://github.com/japgolly/scalajs-react), but I felt that the simplicity of both Scala and React was lost in that union.

Not fully understanding the reason for that at the time, and unwilling to abandon Scala.js, I tried to look for something simpler than React among popular JS libraries. I found André Staltz's [Cycle.js](https://cycle.js.org/), an _"honestly functional framework for building web interfaces"_ I think it was taglined at the time. Cycle's microverse taught me functional reactive programming (FRP), and successfully wrangling Cycle's API into Scala's type system ([Cycle.scala](https://github.com/raquo/Cycle.scala)) taught me two more things: a) Scala's type system is amazing, and b) I should not be fighting it so hard.

Scala.js is a remarkable achievement in safely and usefully encoding one language's semantics in another (Sébastien's [thesis](https://lampwww.epfl.ch/~doeraene/thesis/) is a great read on this). You can use any Javascript library in a type safe manner, all you need is to describe the API of the library using Scala's type system. Usually this is very easy, much like writing a Typescript definition.

However, even simple-looking Javascript UI libraries tend to exploit the dynamic nature of Javascript quite heavily. For example both React and Cycle.js rely heavily on structural typing which is the norm in Javascript and Typescript, but is absent from Scala.js. As a result, the Scala.js interfaces for such idiomatic Javascript libraries need to be quite thick if they want to be both safe and convenient.

Ultimately I had to concede that **it's impossible to write type safe idiomatic Javascript no matter your language of choice**. You can either write idiomatic Scala that is type safe and interops with Javascript pretty well, or you can write idiomatic Javascript in Typescript which is very compatible but not nearly safe enough for my taste. But I didn't even _want_ to write idiomatic Javascript. I only needed it because Javascript UI libraries are made that way.

With this revelation fresh in my head, the next step was obvious – take the event streaming approach of Cycle.js that I liked, and build my own native Scala.js UI library to avoid the _impedance mismatch_ with Javascript. Idiomatic Scala all the way. Of course my library would use virtual DOM like both Cycle.js and React do because how else could you possibly implement an efficient DOM manipulation library.


## The First Laminar

Right as I figured this out, [Outwatch](https://github.com/OutWatch/outwatch/) was released as if to my exact specifications: a native Scala.js library based on virtual DOM and FRP with event streams. And it even used [Snabbdom](https://github.com/snabbdom/snabbdom), the same virtual DOM library used in Cycle.js. Double win!

I tried Outwatch and... faced the same problem as I had with Cycle.js – even though I understood how the library _worked_, I couldn't figure out how to _use_ it effectively. I didn't understand how to break out from the example pattern and build real applications with it. Something wasn't clicking for me, again.

At this point I was exhausted and couldn't rationally justify diving deeper into this madness. I'm an Economist, I know a [sunk cost](https://www.forbes.com/sites/margiewarrell/2015/09/14/sunk-cost-bias-is-it-time-to-move-on/) when I see it. I have many other interesting things to do. I should have walked away.

And yet, I couldn't admit that I'm only capable of productively working with highly polished mainstream libraries like React. This is no longer a matter of time efficiency. If software is what I do for a living, I need to be _better_, or accept my limit now. And so, cursing all this esoteric stuff I've gotten myself into, I begrudgingly tried one last thing: I essentially reimplemented Outwatch from scratch, except with xstream.js instead of RxJS for the reactive layer.

If you care to see the result, it's [here](https://github.com/raquo/Laminar/tree/e75c9128175615fbf8cd02de838d3845190c1bc8). It did not make me happy. This first prototype of "Laminar" was very hard to use for the same reason I had trouble with Cycle and Outwatch, and having now walked the path myself I finally understood exactly what that reason was: **functional reactive programming (FRP) and virtual DOM don't mix!**

Virtual DOM and FRP solve the exact same problem – efficiently keeping the rendered DOM in sync with application state – but they approach it from entirely opposite directions:

FRP is very targeted and precise – your observables tunnel updated data to exactly where it's needed. When you say `a(href <-- urlStream, "link")`, you directly bind `urlStream` to update this `href` attribute on this element. Or... you would, if not for virtual DOM.

Virtual DOM has a completely different idea of how DOM updates should be propagated. Instead of wiring up explicit data propagation paths with observables, you just need to tell virtual DOM when your data – _any_ data – changes, and it will re-render your _whole_ component (with diffing for efficiency, but that's not important conceptually).

Virtual DOM's approach is the _opposite_ of precision. When a component's props or state change, it doesn't know what exact fields changed, and it doesn't know where that data ends up being rendered without performing a costly re-render. On the contrary, the FRP part has the precision to know these things, but it has to _discard_ that knowledge only for virtual DOM to _re-derive_ that same knowledge from scratch by re-rendering your component.

Alternatively, if you try to hold on to that FRP knowledge to perform targeted updates bypassing the virtual DOM, you will run into another problem – your stream-to-element binding logic now needs to account for virtual DOM's lack of stable references. In virtual DOM the virtual elements are ephemeral, being recreated on every re-render, and real DOM nodes can be discarded and replaced at any time. So any time your component updates you need to patch all your bindings on that component to point to the new element if it changed. That's a lot of work, and we haven't even touched on how to tell virtual DOM that you patched this element outside of virtual DOM so that it knows what to diff the next update against.

Either way you go about it, **FRP + virtual DOM ends up being less than the sum of its parts**.


## The First Okay Laminar

I liked the promise of FRP more than I dreaded throwing out all my virtual DOM work, so that's exactly what I did. A surprisingly short amount of focused work later, Laminar v0.1 shipped without any virtual DOM, using my freshly made [Scala DOM Builder](https://github.com/raquo/scala-dom-builder) which kept track of your application's DOM tree much like Virtual DOM would, except it didn't need to do any diffing, and its "virtual" elements were not ephemeral, their lifetime was tied one-to-one to the corresponding real DOM node.

Having stable references to DOM elements allowed Laminar to bind streams to elements directly, improving its internal efficiency and simplicity, but even more importantly trying to build apps in this new Laminar was immensely liberating, euphoric. For the first time in this long endeavor, I felt relief. Using this version was clunky, but not hopeless, and I could feel that I'm on to something worthwhile.


## Airstream

However, having _finally_ started using FRP to build more than just toy applications, it soon became apparent to me why it isn't more popular on the frontend.

The basic concept of streams is very simple to grasp for a Javascript developer: "lazy Promises that can emit more than once". And yet building an application on that paradigm is not as easy – or as safe – as expected, largely because of event streaming libraries' implementation details. There are many issues with streaming libraries that make them hard to use in practice, for example:

* FRP glitches require parts of your code to be pure of side effects, and you can't easily tell _which_ parts if your application is large enough
* You need to remember to kill the subscriptions you create to avoid memory leaks
* Error handling methods are ridiculously unusable
* Event Streams are great for representing events (duh), but are not a good abstraction for representing state changing over time, and merely adding "current value" to streams does not fix that

After a lot of research, I built [Airstream](https://github.com/raquo/Airstream) to solve these problems. I could write a blog post about each of them, but all my writing budget went into documentation instead because I want to live in a world where libraries are well documented.

Switching Laminar from Xstream.js to Airstream was a massive improvement to the development experience. Thanks to Airstream's Ownership feature, it was now completely impossible to forget to kill subscriptions, even ones you created manually without library helpers (a weakness of all other FRP UI libraries), and the extra boilerplate was more than worth it.


## The Current Laminar

Over time that boilerplate got old, and other hard to fix design flaws surfaced. For example, you couldn't re-mount Laminar elements after unmounting them as their subscriptions were one-time-use, and wouldn't start up again after being killed.

I could have fixed all these issues independently, but for once in Laminar's history I managed to restrain my rush for immediate perfectionism, and let a more natural solution come to me. Laminar wasn't _bad_ anymore, it was already quite good, and for end users the problems were mostly manageable with a couple rules of thumb.

Still, those design flaws never sat well with me, and were never intended to outlive me, so I started biting the bullet last fall and finally chewed through it last weekend, addressing many more problems than originally intended as the proper design crystallized in my mind.

The latest version of Laminar:
* Features a more advanced Airstream Ownership system that fixes the memory management gotchas I mentioned
* At the same time, makes ownership related boilerplate almost non-existent
* While simplifying the API with fewer types and fewer but more consistent patterns

If you haven't used Laminar before, now's a great time to [give it a shot](https://github.com/raquo/Laminar). If you have, the [changelog](https://github.com/raquo/Laminar/blob/master/CHANGELOG.md) should speak for itself.

---

At this point I am so far away from the problems I had in 2016 that I have almost forgotten about all this. And for once in Laminar's history I don't have new massive problems ahead. I'm sure this respite is only temporary, but I would like to savor this moment.

Laminar exists, and is nice to the touch. Scala.js UI development is [Laminar-smooth](https://www.simscale.com/docs/content/simwiki/cfd/what-is-laminar-flow.html) now, as far as I'm concerned. This is all I wanted when I started.

---

_Cover photo is a figure from [this Dragonfly flight research paper](https://jeb.biologists.org/content/207/24/4299)._