Recreating Vue 3 Reactivity API (roughly)
Tags: vue, javascript, typescript
Published: 2020-02-19 16:38:37

{% youtube O0b6A6Wi87Q %}

This article is roughly based off the talk I gave on 20th November 2019 at [Vue.js London #13 meetup](https://www.meetup.com/vuejs-london/events/266072043/). You can find the video of the talk [here](https://www.youtube.com/watch?v=O0b6A6Wi87Q) and the repo [here](https://github.com/ycmjason-talks/2019-11-20-vuejs-london-meetup-13).

Typescript will be used in this article so we can look at the problem in a slightly different perspective. If you hate typescript, you can watch [my talk](https://www.youtube.com/watch?v=O0b6A6Wi87Q) instead which was in Javascript.

## Introduction to Vue 3 Reactivity API

You can read about the [Official Vue 3 Reactivity API](https://vue-composition-api-rfc.netlify.com/#api-introduction). But here is a brief introduction with examples.

There are 4 functions in the reactivity API:

1. `reactive()`
2. `ref()`
3. `computed()`
4. `watch()`

Consider [example 1]:

```ts
import { reactive, watch } from '@vue/runtime-core'

// % in Javascript is remainder operator, e.g. -1 % 5 gives -1.
// The following is an implementation for modulus.
const mod = (x: number, y: number) => ((x % y) + y) % y

const MAX_ROAD_LENGTH = 10

const car = reactive({
  position: 0,
  speed: 2,
})

setInterval(() => {
  car.position = mod(car.position + car.speed, MAX_ROAD_LENGTH)
}, 1000)

watch(() => {
  const road = [...'_'.repeat(MAX_ROAD_LENGTH)]
  road[car.position] = '🚗'

  console.clear()
  console.log(road.reverse().join(''))
})
```

{% replit @ycmjason/recreating-vue-3-reactivity-api-1 %}

This code uses `reactive()` and `watch()` from the reactivity API. `reactive()` create an reactive object, i.e. the retrieval and setting of any properties will be tracked. `watch()` takes in a callback that will be executed immediately; whenever the callback's dependencies are changed, the callback will be evaluated again.



So in this example, `car.position` is updated every `1000ms`. And we will see the car moving from the right to the left.

![car moving to the left](https://thepracticaldev.s3.amazonaws.com/i/kbz0eabcpkmdbixcsqwe.gif)



Consider [example 2]

```ts
import { ref, computed, watch } from '@vue/runtime-core'

const counter = ref(0)

const squared = computed(() => counter.value ** 2)

setInterval(() =>  counter.value += 1, 1000)

watch(() => {
  console.log(`counter: ${counter.value}`, `counter²: ${squared.value}`)
})
```

{% replit @ycmjason/recreating-vue-3-reactivity-api-2 %}

This code uses `ref()`, `computed()` and `watch()`. `ref()` and `computed()` both returns a `Ref`. A `Ref` is simply defined as:

```ts
interface Ref<T> {
  value: T
}
```

From the example, `ref(0)` returns `{ value: 0 }` where the `value` will be reactive. `computed()` takes in a function returns a `Ref` whose value is whatever the function returns.

Hopefully this quick introduction by examples makes sense. If you are in doubt, make sure you read the official description of the [Vue 3 Reactivity API](https://vue-composition-api-rfc.netlify.com/#api-introduction) before reading the rest of the article.

## Quick introduction to ES6 `Proxy`

`Proxy` is an ES6 feature; it is the real magic behind Vue 3's reactivity. You can see the full documentation [here](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Proxy). 

In this introduction, I am just going to include the parts we need from proxy to create reactivity. 

`Proxy` is an object which allow us to programmatically control how it behaves on native operations. 

Consider [example 3]

```ts
const target: Record<any, any> = {}

const p = new Proxy(target, {
  set(setTarget, key: string | number, value) {
    console.log(`=== start p.${key} = ${value} ===`)

    console.log(`setTarget === target -> ${setTarget === target}`)
    console.log({ key, value })

    setTarget[key] = value
    console.log(`=== end p.${key} = ${value} ===`)
    return true
  },

  get(setTarget, key: string | number) {
    console.log(`=== start getting p.${key}} ===`)

    console.log(`getting setTarget[${key}]: ${setTarget[key]}`)

    console.log(`=== end getting p.${key}} ===`)
    return 'nope'
  }
})

p.x = 3 // will trigger set trap
console.log()

target.x = 5 // will not trigger trap

console.log(`p.x: ${p.x}`) // will trigger get trap
console.log()

console.log(`p.y: ${p.y}`) // will trigger get trap
console.log()

console.log(target)
```

{% replit @ycmjason/recreating-vue-3-reactivity-api-3 %}

Here is the output:

```
=== start p.x = 3 ===
setTarget === target -> true
{ key: 'x', value: 3 }
=== end p.x = 3 ===

=== start getting p.x} ===
getting setTarget[x]: 5
=== end getting p.x} ===
p.x: nope

=== start getting p.y} ===
getting setTarget[y]: undefined
=== end getting p.y} ===
p.y: nope

{ x: 5 }
```

*Please note that the reason for `key: string | number` is because Typescript currently cannot handle symbols as keys in objects. This is so stupid and there is a [5-year-old issue] created regarding this. `key` will be typed as `string | number | symbol` otherwise.*

As you can see in the example, we have set up the `set` and `get` trap for the proxy `p`. Whenever `p`'s property is set or retrieved, our traps will be called and we can change how it behaves.

In this example, we always return `'nope'` in the `get` function. This is why we see `'nope'` for both `p.x` and `p.y`.

If you are still unsure about how `Proxy` works, make sure you read more into it in the [mdn documentation](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Proxy). 

## Let's recreate Vue 3's reactivity API

You should be familiar with Vue 3's reactivity API and `Proxy` by now. Let's now try to recreate Vue 3's reactivity API.

### `reactive()` and `watch()`

Let's recall [example 1]:

```ts
import { reactive, watch } from '@vue/runtime-core'

// % in Javascript is remainder operator, e.g. -1 % 5 gives -1.
// The following is an implementation for modulus.
const mod = (x: number, y: number) => ((x % y) + y) % y

const MAX_ROAD_LENGTH = 10

const car = reactive({
  position: 0,
  speed: 2,
})

setInterval(() => {
  car.position = mod(car.position + car.speed, MAX_ROAD_LENGTH)
}, 1000)

watch(() => {
  const road = [...'_'.repeat(MAX_ROAD_LENGTH)]
  road[car.position] = '🚗'

  console.clear()
  console.log(road.reverse().join(''))
})
```

Our aim in this section is to make [example 1] work with our custom`reactive()` and `watch()`.

#### Brute-force "reactivity"

We can quickly make [example 1] work as expected by simply calling the watchers (`watch()` callbacks) whenever a reactive property is set. Let's implement this first and see where we can depart from there.

First, let's keep track of the watchers in `watch()`.

```ts
const watchers: (() => any)[] = []
const watch = (callback: () => any) => {
  callback() // this is what Vue 3 watch() will do
  watchers.push(callback)
}
```

Pretty straightforward. Now we have a list of watchers. Next we have to trigger them whenever a reactive property is changed.

We can achieve this by having `reactive()` to return a proxy whose `set` trap will trigger all `watchers`. 

```ts
const watchers: (() => any)[] = []
const watch = (callback: () => any) => {
  callback() // this is what Vue 3 watch() will do
  watchers.push(callback)
}

const reactive = <T extends object>(t: T): T => {
  return new Proxy(t, {
    set(target, key: keyof T, value) {
      target[key] = value 

      watchers.forEach(watcher => watcher())

      return true
    },
  
    get(target, key: keyof T) {
      return target[key]
    },
  })
}
```

Two things to note about the types:

1. Please note that the reason for `key: keyof T` is because Typescript would require `key` to be a key of `T` before being able to do `target[key] = value`. Without `: keyof T`, `key` will be typed as `stirng | number | symbol` which introduces another problem with the [5-year-old issue] mentioned earlier.
2. Previously `string | number` was sufficient because the target was a   `Record<any, any>`, so typescript knows that the target can be   extended.

An example to illustrate how the type works.

```ts
const r = reactive({ a: 3 }) // r is of type `{ a: number }`
r.a = 5
r.b // this will throw a type error as `b` is not a key of `{ a: number }`
```

Exporting our `watch()` and `reactive()`, we can combine them with [example 1]:

[Example 4]:

{% replit @ycmjason/recreating-vue-3-reactivity-api-4 %}

And the car is moving! ✅

![car moving to the left](https://thepracticaldev.s3.amazonaws.com/i/kbz0eabcpkmdbixcsqwe.gif)

There are couple of problems with this approach:

1. **Watchers will be called N times if we trigger mutate reactive   object N times**

   Watchers should only be fired once after a series of consecutive   mutation. Currently each mutation will trigger the watchers   immediately.

2. **Watchers will be called even when it doesn't need to**

   Watchers should only be reevaluated whenever their dependencies changes. We currently do not care and call the watchers whenever somethings is mutated.

#### Brute-force reactivity (fixing problem 1)

We aim to solve the first problem in the last section. 

To illustrate the problem, I have modified the code to add one more car which will trigger another mutation in the interval. You can see the code in [example 5].

```ts
import { reactive, watch } from './reactivity';

// % in Javascript is remainder operator, e.g. -1 % 5 gives -1.
// The following is an implementation for modulus.
const mod = (x: number, y: number) => ((x % y) + y) % y

const MAX_ROAD_LENGTH = 10

const cars = [
  reactive({
    position: 0,
    speed: 2,
  }),
  reactive({
    position: 2,
    speed: 1,
  }),
]

setInterval(() => {
  for (const car of cars) {
    car.position = mod(car.position + car.speed, MAX_ROAD_LENGTH)
  }
}, 1000)

let callCount = 0;
watch(() => {
  const road = [...'_'.repeat(MAX_ROAD_LENGTH)]

  for (const car of cars) {
    road[car.position] = '🚗'
  }

  console.clear()
  console.log(road.reverse().join(''))
  console.log(`callCount: ${++callCount}`)
})
```

![counter incrementing by 2](https://thepracticaldev.s3.amazonaws.com/i/o1iv5vp111l3mbmcxhv5.gif)

You can see how the `callCount` increments by 2. This is because there are two mutations happening every `1000ms` so the watcher was called twice every `1000ms`.

Our aim is to have the watchers only called once after a series of consecutive mutations.

How do we achieve this? "Firing something only once after a series of invocation"? Does this sound familiar? We actually have probably encountered this already in many places. For example, showing search suggestions only after user has stopped typing for a while; firing scroll listener once only after the user has stopped scrolling for a while?

Debounce! Yes, we can just debounce the watchers. This will allow a series of mutation finish before triggering the watcher. And it will only do it *once*! Perfect for this use case!

I will just use [lodash]'s `debounce` here so we won't need to implement it.

See [example 6]:

```ts
import debounce from 'lodash.debounce'

const watchers: (() => any)[] = []
const watch = (callback: () => any) => {
  callback()
  watchers.push(debounce(callback, 0)) // debouncing callback
}

const reactive = <T extends object>(t: T): T => {
  return new Proxy(t, {
    set(target, key: keyof T, value) {
      target[key] = value 

      watchers.forEach(watcher => watcher())

      return true
    },
  
    get(target, key: keyof T) {
      return target[key]
    },
  })
}
```

{% replit @ycmjason/recreating-vue-3-reactivity-api-6 %}

You can see how the `callCount` only increment by 1 every `1000ms`.

![incrementing 1 at a time](https://thepracticaldev.s3.amazonaws.com/i/uuy062he9m0h5qk9l0sx.gif)

#### Dependency tracking

The second problem: "watchers will be called even when it doesn’t need to", can be solved with dependency tracking. We need to know what a watcher depend on and only invoke the watcher when those dependencies are mutated.

In order to illustrate the problem, I have modified the `index.ts`.

```ts
import { reactive, watch } from './reactivity';

const r1 = reactive({ x: 1 })
const r2 = reactive({ x: 100 })

setInterval(() => {
  r1.x++
}, 1000)

setInterval(() => {
  r2.x--
}, 5000)

watch(() => {
  console.log(`r1.x: ${r1.x}`)
})

watch(() => {
  console.log(`r2.x: ${r2.x}`)
})
```

{% replit @ycmjason/recreating-vue-3-reactivity-api-7 %}

With this example, we can see the problem clearly. We expect `r1.x` to be logged every second and `r2.x` every 5 seconds. But both values are logged every second because all watchers are called.

Here are the steps we can implement dependencies tracking:

1. We can keep track of the dependencies of a watcher in a `Set`, which helps avoid duplications. A dependency is a property in a reactive. We can represent each property in a reactive with a unique identifier.  It could be anything unique but I'll use a `Symbol()` here.
2. Clear the dependencies set before calling the watcher.
3. When a reactive property is retrieved, add the symbol representing that property to the dependencies set. 
4. After the watcher callback finishes, `dependencies` will be populated with symbols that it depends on. Since each watcher now relates to a set of dependencies, we will keep `{ callback, dependencies}` in the `watchers` list.
5. Instead of triggering all watchers as a property is being set, we could trigger only the watchers that depend on that property.

```ts
import debounce from 'lodash.debounce'

const dependencies = new Set<symbol>() // 1. keep track of dependencies

const watchers: ({
  callback: () => any,
  dependencies: Set<symbol>,
})[] = []
const watch = (callback: () => any) => {
  dependencies.clear() // 2. clear dependencies 
  callback()
  // 4. dependencies is populated
  watchers.push({
    callback: debounce(callback, 0),
    dependencies: new Set(dependencies), // make a copy
  })
}

const reactive = <T extends object>(t: T): T => {
  const keyToSymbolMap = new Map<keyof T, symbol>()
  const getSymbolForKey = (key: keyof T): symbol => {
    const symbol = keyToSymbolMap.get(key) || Symbol()
    if (!keyToSymbolMap.has(key)) {
      keyToSymbolMap.set(key, symbol)
    }
    return symbol
  }

  return new Proxy(t, {
    set(target, key: keyof T, value) {
      target[key] = value 

      // 5. only trigger watchers depending on this property
      watchers
        .filter(({ dependencies }) => dependencies.has(getSymbolForKey(key)))
        .forEach(({ callback }) => callback())

      return true
    },
  
    get(target, key: keyof T) {
      dependencies.add(getSymbolForKey(key)) // 3. add symbol to dependencies
      return target[key]
    },
  })
}
```

{% replit @ycmjason/recreating-vue-3-reactivity-api-8 %}

With this we can see the result matches our expectation and this means dependency tracking is working!!!

#### Update dependencies on the fly

A watcher may change its dependencies. Consider the following code:

```ts
const r1 = reactive({ isReady: false })
const r2 = reactive({ x: 1 })

setTimeout(() => {
  r1.isReady = true
}, 1000)

setInterval(() => {
  r2.x++
}, 500)

watch(() => {
  if (!r1.isReady) return
  console.log(`r2.x: ${r2.x}`)
})
```

In this example, we expect the log to happen after 1 second and every 500ms afterwards.

However our previous implementation only logs once:

{% replit @ycmjason/recreating-vue-3-reactivity-api-9 %}

This is because our watcher only access `r1.x` at its first call. So our dependency tracking only keeps track of `r1.x`.

To fix this, we can update the dependencies set every time the watcher is called.

```ts
const watch = (callback: () => any) => {
  const watcher = {
    callback: debounce(() => {
      dependencies.clear()
      callback()
      watcher.dependencies = new Set(dependencies)
    }, 0),
    dependencies: new Set<symbol>(),
  }

  watcher.callback()

  watchers.push(watcher)
}
```

This wraps the dependency tracking into the watcher to ensure the dependencies is always up to date.

With this change, it is now fully working! :tada:

{% replit @ycmjason/recreating-vue-3-reactivity-api-10 %}

### `ref()`, `computed()`

We can build `ref()` and `computed()` pretty easily by composing `reactive()` and `watch()`.

We can introduce the type `Ref` as defined above:

```ts
interface Ref<T> {
  value: T
}
```

Then `ref()` simply returns a `reactive` with just `.value`.

```ts
const ref = <T>(value: T): Ref<T> => {
  return reactive({ value })
}
```

And a `computed()` just return a ref which includes a watcher that updates the value of the ref.

```ts
const computed = <T>(fn: () => T): Ref<T> => {
  const r = ref<T>(undefined as any)

  watch(() => {
    r.value = fn()
  })

  return r
}
```

See the following example:

{% replit @ycmjason/recreating-vue-3-reactivity-api-11 %}



### Conclusion

Thanks for reading this tedious article and hope you have gain some insights about how the magic behind Vue's reactivity works. This article has been worked on across months because I travelled to Japan in the middle of writing this. So please let me know if you spot any mistakes/inconsistency which can improve this article.

The reactivity we have built is just a really rough naive implementation and there are so many more considerations put into the actual Vue 3 reactivity. For example, handling `Array`, `Set`, `Map`; handling immutability etc. So please do not use these code in production.

Lastly, hopefully we will see Vue 3 soon and we can make use of this amazing api to build awesome things! Happy coding!

[example 1]: https://repl.it/@ycmjason/recreating-vue-3-reactivity-api-1
[example 2]: https://repl.it/@ycmjason/recreating-vue-3-reactivity-api-2
[example 3]: https://repl.it/@ycmjason/recreating-vue-3-reactivity-api-3
[example 4]: https://repl.it/@ycmjason/recreating-vue-3-reactivity-api-4
[example 5]: https://repl.it/@ycmjason/recreating-vue-3-reactivity-api-5
[example 6]: https://repl.it/@ycmjason/recreating-vue-3-reactivity-api-6
[example 7]: https://repl.it/@ycmjason/recreating-vue-3-reactivity-api-7
[example 8]: https://repl.it/@ycmjason/recreating-vue-3-reactivity-api-8
[nanoid]: https://www.npmjs.com/package/nanoid
[5-year-old issue]: https://github.com/microsoft/TypeScript/issues/1863
[lodash]: https://lodash.com/
[event loop]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/EventLoop