High Performance 3D Animation with React + rxjs
Tags: react, functional, threejs, rxjs
Published: 2021-09-10 05:02:06

You know when you [load a website](https://stripe.com/) and it has a bunch of [fancy visualisations](https://www.apple.com/imac-24/) that respond to mouse and scroll position with animation? For most of the web's history creating experiences like these has either been impossible or required masochistic determination.

![Trying to replicate an apple.com product page is an extreme test of sanity](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/bhbin0ndimcn4tyvxo7a.png)
> Trying to replicate an apple.com product page is an <em>extreme</em> test of sanity

It used to be difficult to create pretty much _any_ interface in the browser. Efforts like React, Vue, Svelte, Solid and friends have trivialised the jQuery battles of the past. Now we can express our interfaces declaratively, as a function of `state -> view`.

In fact React has even let us port this idea to _the third dimension_ with [react-three-fiber](https://github.com/pmndrs/react-three-fiber); a wonderful library that uses [three.js](https://github.com/mrdoob/three.js/) as a custom React render target.

```tsx
const ColoredBox = () => {
  const [toggled, setToggled] = useState(false)

  return (
    <mesh onClick={() => setToggled(!toggled)}>
      <boxGeometry args={[1, 1]} />
      <meshStandardMaterial
        color={toggled ? 'blue' : 'red'}
      />
    </mesh>
  )
}
```

{% codesandbox react-three-fiber-demo-bur8n %}
> Try panning, scrolling and clicking

This is, in my opinion, _staggeringly_ little code to implement in-browser 3D. We get a lot for free here courtesy of React's Virtual DOM (VDOM) diffing, suspense and state management. However, there is a sticking point. 

VDOM style renderers are surprisingly fast given what they do, but sadly they are _not quite fast enough_ for state that changes 60 times a second; the standard for "smooth" animation.

# The Future of User Interaction on the Web

I've been wondering, with libraries like `react` and `react-three-fiber` combined with rising support for `webgl`, `wasm` and `wgpu`, are we on the path to far richer interactions in the browser? As a [game developer](https://twopm.studio) I work with a few common game engines and none of them can be considered "declarative". In a typical game the graph of data dependency is far wider and denser than a typical web app and as a result most game engines prioritise performance over clarity. So, how can we get the best of both worlds? Declarative, composable animation logic that reacts to 60hz (minimum!) state updates.

Programmatic animation is a whole sub-specialty within user interface development: tweens, timelines, easing functions, springs, cancellation, the FLIP method... There's a lot of jargon 😵‍💫.

In turn, it's common for us developers to lean on existing libraries to get the job done. [framer-motion](https://github.com/framer/motion), [react-spring](https://react-spring.io/) and [GSAP](https://greensock.com/gsap/) are great libraries, but we can learn a lot more about animation by implementing our own approach. What's more almost all animation libraries require us to work with someone else's conceptual model of user input. They provide extension APIs of course but we tend to implement each of these as closed-box concepts, you can consume them but not compose them freely. 

A contrived but illustrative example: animating positions of 3D objects that are derived from one another and play sound effects based on their progress. This is difficult to implement in many libraries because of built-in assumptions about the _kinds_ of animation we'd like to do and how to _compose_ them.

Recently I came across [samsarajs](http://samsarajs.org/), a library designed for _continuous_ user interfaces. That is, interfaces that may never be "at rest" and are constantly reacting to changes in data. The project is rooted in functional reactive programming or _FRP_. 

Briefly, FRP is focused on one main concept: the data stream. 

> **Stream** _(striːm)_
> A sequence of values distributed over <em>some amount</em> of time

What values? How much time? Those are up to the specific instance. Libraries like [rxjs](https://github.com/ReactiveX/rxjs) provide an algebra for working with streams, letting us mix them together, pluck out select elements and aggregate data over time. Others have explained streams [far better than I can](https://gist.github.com/staltz/868e7e9bc2a7b8c1f754).

In my experience reactions to FRP are mixed. Many people are scared away by its abstract nature, some fear it encourages tightly wound spaghetti code and a dedicated few believe it is the future of programming. I think it's all of the above, FRP is __powerful__ and like any powerful abstraction it is open to abuse. When you have a nuclear-powered ultra-hammer everything looks like an ultra-nail.  

Regardless, `samsarajs`'s fundamental insight is that the layout of an application can be modelled as a stream<sup>[[ref]](http://samsarajs.org/docs/architecture.html)</sup>. Selfishly, I immediately wondered if I could apply this to my problem. 

Animation can also easily be modelled as a stream<sup>[[ref]](https://github.com/leonidas/codeblog/blob/master/2012/2012-01-17-declarative-game-logic-afrp.md)</sup>, it's almost in the definition:

> **Animation** _anɪˈmeɪʃ(ə)n_
> A series of frames shown in succession over time, creating the illusion of movement.

Combining this with input streams from the user we can create a unified model of `user intention` -> `data mutation` -> `animated visualisation`.

![The "dialogue" abstraction, image provided by the cycle.js docs.](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/q62qqgs0sgp21jwfe71y.png)
> The "dialogue" abstraction, image provided by the cycle.js docs.

This model is heavily inspired by [cycle.js](https://cycle.js.org/dialogue.html) which is one of the most mindblowing frameworks around even after 7+ years of development. The **cycle** described by `cycle.js` from `sources` to `sinks` is a conceptual model that I find myself using in every interface, generative artwork or game I create.

So with all that said, is there a way to use FRP and `react-three-fiber` to create performant, declarative animations? Let's find out.

# Implementation

Alright, here's the meaty part. I'm using `React` and `react-three-fiber` for rendering and `rxjs` to provide our streams. My implementation focuses on a three core concepts:

- `useObservable`: values to animate
- `interpolator`: how to transition between values
- `useAnimation`: performant rendering of animations


## useObservable

You might've heard of observables before, the base concept is simple:

> **Observable** _əbˈzəːvəbl_
> A variable which notifies subscribed listeners when the internal value is changed. Listeners can subscribe and unsubscribe from change notifications on demand.

```ts
const scale = useObservable(1)
```

In this case, calling `scale.set(2)` or `scale.swap(x => x + 1)` will change the underlying value and send an update event down the `scale.changes` stream.

```typescript
const scale = useObservable(1)

scale.changes
	 .pipe(filter(x => x > 1))
     .subscribe(x => console.log(`it's ${x}!`));

scale.set(2);
// => it's 2!
scale.set(1);
//
scale.swap(x => x + 1.5);
// => it's 2.5!
```

In [ReactiveX](http://reactivex.io/) terminology, this is a `Subject<T>` wrapped up for easy consumption from React.

## interpolator

```ts
type Interpolator = {
  end: number,
  sample: (t: number) => number
}

const demo: Interpolator = 
  interpolator(0, 1, 'easeOutCubic')
```

An interpolator acts as a translation layer between different numerical ranges. They typically take the form of functions accepting take a value, `t`, from `0...1` and output a value of `t` from `0...1`. This might sound familiar if you've heard of [easing functions](https://easings.net/), which are almost ubiquitous in programmatic animation:

![Comparison of various common easing functions, via Noisecrime on the Unity Forums](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/95lqsu5b513c6bxkglnm.gif)
> Comparison of various common easing functions, via Noisecrime on the Unity Forms

Our interpolators are almost identical except for two important properties: 

### 1. Remapping

```ts
const linear = interpolator(0, 1, 'linear')
console.log(linear(0), linear(0.5), linear(1))
// => 0, 0.5, 1

const warped = mapInterpolator(linear, -2, 4)
console.log(warped(0), warped(0.5), warped(1))
// => -2, 1, 4
```

This is important when we _apply_ an animation. We'll animate values with certain curves between `0...1` but in practice we want to translate that into whatever range is relevant. We might want to animate a box's width between `32px` and `400px` but until the point of actually applying the animation we can preserve our sanity by using the normalised `0...1` range.

### 2. Composition

You can combine interpolators in many useful ways. We might want to add them together, subtract them, multiply them or sequence them one after the other.

Currently I've only written the `sequence` composition, but it demonstrates the principle.

```ts
const bounce = sequence(
  interpolator(0, 1.2, 'easeOutCubic'),
  interpolator(1.2, 1, 'easeOutCubic')
)
console.log(bounce(0), bounce(0.5), bounce(1))
// => 0, 1.2, 1
```

## useAnimation

Finally, the hook that connects it all together. `useAnimation` takes an `observable value`, an `interpolator`, the duration in milliseconds and a function to apply the animated value.

```javascript
useAnimation(scale, bounce, 500, value => {
  mesh.scale.x = mesh.scale.y = value;
})
```

The `value => {}` callback is the application site of our side effects, in FRP terms this is known as a `sink`. Before this function is called _all we are doing_ is changing some numbers in memory over time using an animation curve defined by our `interpolator`, but our `sink` is where we connect to our output.

This may feel a little "bare metal" on first inspection, but I would argue this approach is vital for practical usage. A simple adjustment allows us to use this same animation with `react-three-fiber` or `react-dom`, retargeting only the binding layer.

```typescript
const bounce = sequence(
  interpolator(0, 1.2, 'easeOutCubic'),
  interpolator(1.2, 1, 'easeOutCubic')
)
const scale = useObservable(1);

// react-three-fiber
const mesh = useRef();
useAnimation(scale, bounce, 500, value => {
  mesh.current.scale.x = mesh.current.scale.y = value;
});

// react-dom
const element = useRef();
useAnimation(scale, bounce, 500, value => {
	element.current.style.transform = `scale(${value})`;
});
```
This approach gives us maximum control and flexibility without compromising on performance. You can imagine packaging these `value => {}` callbacks into common pieces `scaleDom`, `rotateDom`, `updateShaderUniform` etc.

```ts
const scaleDom = (el, v) => el.current.style.transform = `scale(${value})`;
const rotateDom = (el, v) => el.current.style.transform = `rotateZ(${value})`;
const setShaderUniform = (shader, uniform, value) => shader.current.uniforms[uniform].value = value;
```

Here's an example sketch I made using this API (try moving your mouse around, panning, zooming etc.):

{% codesandbox use-animation-demo-xtpcd %}
> The source for this entire article including the above sketch is public on [github](https://github.com/bfollington/use-animation).

# How does useAnimation work?

I'm not ready to publish `useAnimation` as a library on `npm` just yet, I'd like to explore the API surface more and put together documentation / examples. That said, you can poke around the sourecode yourself on [github](https://github.com/bfollington/use-animation) and come back if you're confused / curious to know more.

I started with, "what happens when a value we want to animate changes?" Well, we emit a change event on our `.changes` stream. Okay, so then from _that_ change event we need to start an animation from the current value to the new value. As expressed earlier, an animation is a stream of frames... So we need to get one of those.

Thankfully `Subject<T>` from `rxjs` has us covered yet again. If we create a new `Subject`, we can call `.next()` on it to emit a new value whenever we want. So, if we combine a `Subject` with `requestAnimationFrame` we will have a new value published on every renderable frame the browser gives us.

This is a little gnarly in practice, but luckily I found an example from [learnrxjs.com](https://www.learnrxjs.io/learn-rxjs/recipes/gameloop) that worked perfectly. My version is in [frameStream.ts](https://github.com/bfollington/use-animation/blob/main/src/useAnimation/frameStream.ts) and is identical except I don't clamp the framerate to `30`.

The implementation for `react-three-fiber` turned out to be more challenging, I ran into issues asking for multiple `requestAnimationFrame` loops. So, instead, I built on top of `useFrame` to construct a stream held in a React `MutableRef<T>` in a similar way:

```typescript
export const useFrameStream = () => {
  const s = useRef<Subject<number>>(new Subject<number>())
  useFrame(({ clock }) => {
    s.current.next(clock.getDelta())
  })
  return s
}
```

__Okay__, so we've got our framestream. Let's look at `useAnimation` and break it down piece by piece.
We'll start by let's identifying some familiar concepts:
  - `source` is the return value of `useObservable()`
  - `source.changes` is the update stream to the underlying value
  - `frame$` is the stream of `requestAnimationFrame`s

```typescript
export const useAnimation = (
    source: ObservableSource,
    interpolator: Interpolator,
    duration: number,
    sink: (v: Animatable) => void
  ) => {

  // first, store the current animation state seperate to the observed value
  const underlying = React.useRef(source.value())

  React.useEffect(() => {
    // update the render target upon mounting the component
    sink(underlying.current)

    // listen to the update stream from our observable value
    const sub = source.changes
      .pipe(
        // switchMap: the magic operator that enables cancellation
        // our value might change AGAIN mid-animation and
        // we need to cut over to target the updated value
        //
        // switchMap has exactly these semantics, it'll cancel
        // an old stream and replace it with a new one whenever
        // it recieves a value
        switchMap((v) => {
          // capture the time when the animation started
          const baseTime = Date.now()

          return concat(
            // take our frame stream at ~60Hz
            frames$.pipe(
              share(),
              // calculate the % into the total duration we are at
              map((dt) => (Date.now() - baseTime) / duration),
              // only animate while are < 100%
              takeWhile((t) => t < 1),
            ),
            // we append 1 to ensure we send an explicit frame at 100%
            of(1),
            // mapInterpolator warps an interpolator's domain from 0...1
            // to whatever we want
            // here we map [0<->1] to [prev<->current]
          ).pipe(
            map(mapInterpolator(interpolator, underlying.current, v).sample)
          )
        }),
      )
      .subscribe((v) => {
        // finally we store the current value and call
        // the supplied update callback
        underlying.current = v
        sink(v)
      })

    return () => {
      // stop listening for changes when the component unmounts
      sub.unsubscribe()
    }
  }, [duration, source, sink, interpolator])
}
```

# Wrapping Up
As stated above, all the code for this experiment is available on [github](https://github.com/bfollington/use-animation/blob/main/src/useAnimation/useAnimation.ts) with an MIT license. 

If you want to go deeper again then check out the project [README](https://github.com/bfollington/use-animation) and [samsarajs](http://samsarajs.org/). I'd like to try [@most/core](https://github.com/mostjs/core) instead of `rxjs` here since it boasts impressive performance<sup>[[ref]](https://github.com/cujojs/most/tree/master/test/perf)</sup>. To me, this seems like a promising area for further investigation. I've begun to experiment with a similar approach in Unity3d, hopefully more to report soon!

{% twitter 1430922583088799753 %}

This is the first post from my new project [⊙ fundamental.sh](https://fundamental.sh) where I'm attempting to document my favourite abstractions and programming patterns. Please don't hesitate to get in touch with me with feedback, ideas for extension or questions. You can find me on [twitter](https://twitter.com/vivavolt), discord (ben#6177) or [around the web](https://bf.wtf).

If you'd like to be notified of the next time I write about programming subscribe to the [mailing list](https://buttondown.email/fundamental). I only post when I have something worth saying.