Speed-up your internationalization calls up to 5-1000 times
Tags: javascript, webdev, performances, internationalization
Published: 2019-02-28 16:00:58

---
title: Speed-up your internationalization calls up to 5-1000 times
published: true
description: A journey to transpilation, optimizations and opcodes on i18n libraries
tags: javascript, webdev, performances, internationalization
cover_image: https://thepracticaldev.s3.amazonaws.com/i/8ctoewn60n3ig5bm0p7y.jpg
---

# Context

It all started two years ago. I was working on a new PWA for a big social network written from scratch that needed a i18n module to handle different languages. The module had to:
- handle **interpolation**.
- handle **PLURAL and SELECT expressions**.
- **be lightweight** (it’s a PWA, must run with limited bandwidth).
- **run fast** (some users had low-end devices).

And that’s where things got creepy, the only possible library was Google Closure MessageFormat. It was not so fast on low-end devices and weighing heavily on our bundle. **So I decided to write my own** with performance in mind.

Fast forward to today, the problem is still the same with i18n libraries, so I opened-source [💋Frenchkiss.js](https://github.com/koala-interactive/frenchkiss.js) **a 1kb i18n library 5 to 1000 times faster** than others.
Stay with me for a journey on performances optimizations.

> 👉 _Time to speed up your webapp for mobile devices!_

---

# 🤷  How are i18n modules working?

Under the hood, **it sucks**, some i18n modules are re-processing the translation on each and every calls, resulting in poor performances.

Here is an example of what can happen inside the translate function _(really simplified/naive version of [Polyglot.js](https://github.com/airbnb/polyglot.js))_.

```js
const applyParams = (text, params = {}) => {
  // Apply plural if exists
  const list = text.split('||||');
  const pluralIndex = getPluralIndex(params.count);
  const output = list[pluralIndex] || list[0];

  // Replace interpolation
  return output.replace(/%\{\s*(\w+)\s*\}/g, ($0, $1) =>  params[$1] || '');
}

applyParams('Hello %{name} !', {
  name: 'John'
});
// => Hello John !
```

In short, on each translations call we split the text, calculate the plural index, create a RegExp and replace all occurrences by the specified given parameter if it exists and returns the result.

**It's not that big of a deal, but are you fine doing it multiple time on each render/filter/directive call ?**

> 👉 It's one of the first things we learn when building app in react, angular, vuejs or any other framework : avoid intensive operations inside render methods, filters and directives, **it will kill your app !**

### Some i18n libraries are doing better !

Some others are optimizing things quite a bit, here comes Angular, VueJs-i18n, Google Closure for example.

How are they doing it ? Actually they parse the string only once and cache a list of opcodes to process them on the next calls.

If you aren’t familiar with opcodes, it’s basically a list of instructions to process, in this case just to build a translation. Here's a possible example of opcodes generated from translations :

```json
[{
  "type": "text",
  "value": "Hello "
}, {
  "type": "variable",
  "value": "name"
}, {
  "type": "text",
  "value": " !"
}]
```

And how we print the result :

```js
const printOpcode = opcodes => opcodes.map(code => (
  (code.type === 'text') ? code.value :
  (code.type === 'variable') ? (params[code.value] || '') :
  (code.type === 'select') ? printOpCode( // recursive
    params.data[params[code.value]] || params.data.other
  ) :
  (code.type === 'plural') ? printOpCode( // recursive
    params.list[getPluralIndex(params[code.value])] || params.list[0]
  ) :
  '' // TODO not supported ?
)).join('');
```

With this type of algorithm, more time is allocated for the first call that generate the opcode but we store it and re-use it for faster performance in the next calls :
- It doesn't split the string.
- It doesn't do intensive regex operation.
- It just read the opcode and merge the result together.

Well, that rocks ! But is it possible to go further ?

---

# 🤔  How can we speed up things ?

[💋Frenchkiss.js](https://github.com/koala-interactive/frenchkiss.js) is going one step further, **it compiles the translation into a native function**, this one is so light and pure that the Javascript can easily [JIT compile it](https://v8.dev/).

### How does it work ?

Quite simple, you can actually build a function from a string doing the following :

```js
const sum = new Function('a', 'b', 'return a + b');

sum(5, 3);
// => 8
```

> For further informations, take a look at [Function Constructor (MDN)](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Function).

The main logic is still to generate an opcode list but instead of using it to generate a translation we use it to generate an optimized function that will returns the translation without further process.

It’s actually possible because of the simple structure of interpolation and SELECT/PLUTAL expressions. It’s basically a returns with some ternary.

```js
const opCodeToFunction = (opcodes) => {
  const output = opcodes.map(code => (
    (code.type === 'text') ? escapeText(code.value) :
    (code.type === 'variable') ? `params[${code.value}]` :
    (code.type === 'select') ? ... :
    (code.type === 'plural') ? ... :
    '' // TODO Something wrong happened (invalid opcode)
  ));

  // Fallback for empty string if no data;
  const result = output.join('+') || "";

  // Generate the function
  return new Function(
    'arg0',
    'arg1',
    `
    var params = arg0 || {};
    return ${result};
  `);
});
```

**⚠️ Note: when building dynamic function, make sure to avoid [XSS injection](https://www.owasp.org/index.php/Cross-site_Scripting_(XSS)) by escaping user input !**

Without further ado, let's see the generated functions _(note: the real generated functions are a little more complex, but you will get the idea)_.

### Interpolation generated function

```js
// "Hello {name} !"
function generated (params = {}) {
  return 'Hello ' + (params.name || '') + ' !';
}
```

> By default, we still fallback to empty string to avoid printing _"undefined"_ as plain text.

### Select expression generated function

```js
// "Check my {pet, select, cat{evil cat} dog{good boy} other{{pet}}} :D"
function generated (params = {}) {
  return 'Check my ' + (
    (params.pet == 'cat') ? 'evil cat' :
    (params.pet == 'dog') ? 'good boy' :
    (params.pet || '')
  ) + ' :D';
}
```

> We don't use strict equality to keep supports for numbers.

### Plural expression generated function

```js
// "Here {N, plural, =0{nothing} few{few} other{some}} things !"
function generated (params = {}, plural) {
  const safePlural = plural ? { N: plural(params.N) } :{};

  return 'Here ' + (
    (params.N == '0') ? 'nothing' :
    (safePlural.N == 'few') ? 'few' :
    'some'
  ) + ' things !';
}
```

> We cache the plural category to avoid re-fetching it in case of multiple checks.

---

# 🚀  Conclusion

With generated functions we were able to execute code **from 5 to 1000 time faster** than others, avoiding doing _RegExp_, _split_, _map_ operations in **rendering critical path** and also avoiding Garbage Collector pauses.

![Benchmark](https://github.com/koala-interactive/frenchkiss.js/blob/master/benchmark/benchmark.jpg?raw=true)

Last best news, it's only **1kB GZIP size !**

> If you're searching for a i18n javascript library to accelerate your PWA, or your SSR, you should probably give [💋Frenchkiss.js](https://github.com/koala-interactive/frenchkiss.js) a try !