Pre-generating multiple index files using Angular Builders and Gulp tasks to serve a multilingual Angular app
Tags: javascript, webdev, tutorial, angular
Published: 2022-08-08 07:26:19

Now that we know how to generate different index files on runtime through **Express template engines**, we are going to build the right index file before serving it, and drop the template engines. The main benefit of this is decoupling from the server, and making all HTML processing go under one hood. This also makes hosting on cloud hosts easier. To pre-generate the index files, we normally use a task runner like Gulp, or Angular builders.

> find the builder code under [StackBlitz ./builder](https://stackblitz.com/edit/angular-runtime-translation?file=builder%2Freadme.md)

Angular builder
---------------

The **CLI builders** provided by Angular allow us to do extra tasks using the `ng run` command. The idea is very basic, the [documentation](https://angular.io/guide/cli-builder) however is very slim. For the purpose of our tutorial, we want to create a simple **local builder** in a subfolder, and run it post build. We do not want to publish over `npm`, nor make our builder reusable.

> I found a good resource: [**Angular CLI under the hood - builders demystified**](https://www.justjeb.com/post/angular-cli-under-the-hood-builders-demystified)

### The building blocks

The main ingredients are:

-   A sub folder with its own `package.json`, that runs its own `tsc`, to generate its own `dist` folder
-   A new target in the main `angular.json` to run the task, in the sub folder

Here are the building blocks

#### builders.json

```json
{
  "builders": {
    "[nameofbuilder]": {
      "implementation": "./dist/[build js file here]",
      "schema": "[path to schema.json]",
      "description": "Custom Builder"
    }
  }
}
```

#### schema.json

```json
// nothing special
{
  "$schema": "http://json-schema.org/schema",
  "type": "object",
  "properties": {
    "[optionname]": {
      "type": "[optiontype]"
    }
  }
}
```

#### root angular.json

```json
"projects": {
    "[projectName]": {
      "architect": {
        // new target
        "[targetName]": {
          // relative path to builder folder
          "builder": "./[path-to-folder]:[nameofbuilder]",
          "options": {
            "[optionname]": "[value]"
          }
        },
			},
		},
}
```

#### tsconfig.json

This probably is the most despicable part of the builder. When we install the latest version of Angular, we want to create our **local builder** using the same settings. Here, we extend the `tsconfig.json` in application root, which, in version 14, uses `"module": "es2020"`, for this to work, we have two options.

-   The first is to override it to use `commonjs`
-   or pass `type="module"` to builder's `package.json`.

I prefer the latter. (Remember, we are not building using Angular CLI, but `tsc` command.)

Another issue is the **default behavior of `tsc`, when building a single folder**. When we have

`builder/singlefolder/index.ts`

The build [computes the shortest path](https://github.com/Microsoft/TypeScript/wiki/FAQ#why-does---outdir-moves-output-after-adding-a-new-file):

`builder/dist/index.js`

**Ignoring the folder structure**. To fix that we always explicitly set `rootDir`

```json
{
	// extend default Angular tsconfig
  "extends": "../tsconfig.json",
  "compilerOptions": {
	  // output into dist folder
	  "outDir": "dist",
		// adding this will force the script to commonjs
		// else it will be fed from root config, which is ES2020 in Angular 14
    // "module": "commonjs",
		// we can also be explicit with
    // "module": "es2020",
		// explicity set rootDir to generate folder structure after tsc build
    "rootDir": "./"
  },
  // include all builders in sub folders
  "include": ["**/*.ts"]
}
```

#### package.json

```json
// builder/package.json
{
  // add builders key, to point to the builders.json file
  "builders": "builders.json",
	// that is the only extra dependency we need
  // install it in sub folder
  "devDependencies": {
    "@angular-devkit/architect": "^0.1401.0"
  },
  // If the tsconfig does not specify "commonjs" and feeds directly
	// from Angular ES2020 setting, then this should be added
  // this is my preferred solution
  "type": "module"
}
```

Remember to exclude `node_modules` inside builder folder in your `.gitignore`

#### Root package.json

The command to run is:

`ng run [projectName]:[targetName]`

So in our root `package.json`, we might want to create a shortcut and name it `post[buildtask]` so that it runs after the main build

```json
// root package
"scripts": {
  // example, if this is the build process
	"build": "ng build --configuration=production",
  // create a new script
  "postbuild": "ng run [projectName]:[targetName]"
},
```

Now running `npm run build` will run both.

We have all the ingredients, let's mix.

### Locale `writeindex` builder

Inside our newly created **builder folder**, I created a single file `locales/index.ts` which has the basic builder structure as specified in [Angular official documentation](https://angular.io/guide/cli-builder). Run `tsc` in scope of this folder. This generates `builder/dist/locale/index.js` .

The `builders.json` file updated:

```json
{
  "builders": {
    "localizeIndex": {
      "implementation": "./dist/locales/index.js",
      "schema": "./locales/schema.json",
      "description": "Generate locales index files"
    }
  }
}
```

And our root `angular.json`

```json
"writeindex": {
  "builder": "./builder:localizeIndex",
  "options": {
     // ... TODO
  }
},
```

The purpose is to generate a new index **placeholder** file that has all language instructions. Then on post build, run the Angular builder. So the `placeholder.html` file should be our target.

> This is safer than `index.html`, we don't want it to be served by accident if we messed up our rewrite rules

The `placeholder.html`

```html
<!doctype html>
<!-- add $lang to be replaced -->
<html lang="$lang">
<head>
		<!-- base href, you have multiple options based on your setup -->
    <!-- if URL based app -->
    <!-- <base href="/$lang/" /> -->

    <!-- if  non url based app -->
    <!-- <base href="/" /> -->

    <!-- for tutorial purposes, produce both options, let angular builder replace it -->
    <base href="$basehref" />

		<!-- here is an update, no need to rewrite language.js when we can serve exact file -->
    <script src="locale/cr-$lang.js" defer></script>

    <link rel="preconnect" href="https://fonts.googleapis.com">
    <link rel="preconnect" href="https://fonts.gstatic.com" crossorigin="">

     <!-- #LTR -->
     <link rel="stylesheet" href="https://fonts.googleapis.com/css2?family=Signika:wght@300..700&display=swap">
     <link rel="stylesheet" href="styles.ltr.css">
     <!-- #ENDLTR -->
     <!-- #RTL -->
     <link rel="stylesheet" href="https://fonts.googleapis.com/css2?family=Tajawal:wght@300;500&display=swap">
     <link rel="stylesheet" href="styles.rtl.css">
     <!-- #ENDRTL -->
</head>
<body>
    <app-root>
          <!-- #LTR -->
          loading
          <!-- #ENDLTR -->
          <!-- #RTL -->
          انتظر
          <!-- #ENDRTL -->
    </app-root>
</body>

</html>
```

Our builder is supposed to build `index.[lang].html` for all supported languages, but for this tutorial's purposes, I am going to make it produce both URL based, and cookie based files. In real life, you usually have one solution.

Our final schema should allow for "source" file, "destination" folder, and supported "languages":

In `locales/schema.json`

```json
{
  "$schema": "http://json-schema.org/schema",
  "type": "object",
  "properties": {
    "source": {
      "type": "string"
    },
    "destination": {
      "type": "string"
    },
  	// let's spice this one up a bit and make it an object
    "languages": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
  				"name": { "type": "string"},
          "isRtl": { "type": "boolean"}
        }
      }
    }
  }
}
```

We update our `angular.json` like this:

```json
// root angular.json architect target, update options
"options": {
  // which file to replicate
  "source": "host/client/placeholder.html",
  // where to place it
  "destination": "host/index",
  // what languages are supported
  "languages": [
		{"name": "ar", "isRtl": true},
		{"name": "en", "isRtl": false},
    {"name": "...", "isRtl": ...}	
	]
}
```

Also update the **build target** to use placeholder file:

`"index": "src/placeholder.html",`

And now the stuffing: the `locales/index.ts`

```typescript
import { BuilderContext, BuilderOutput, createBuilder } from '@angular-devkit/architect';
import { readFileSync, writeFileSync, existsSync, mkdirSync } from 'fs';

// interface the schema options
interface Options  {
    source: string;
    destination: string;
    languages: { name: string; isRtl: boolean }[];
}

// expressions to replace, we can modify and add as many as we want
const reLTR = /<!-- #LTR -->([\s\S]*?)<!-- #ENDLTR -->/gim;
const reRTL = /<!-- #RTL -->([\s\S]*?)<!-- #ENDRTL -->/gim;

// replace ae all lang instances
const reLang = /\$lang/gim;

// this is extra, for this tutorial to produce both options
const reBase = /\$basehref/gim;

export default createBuilder(LocalizeIndex);

function LocalizeIndex(
    options: Options,
    context: BuilderContext,
): BuilderOutput {

    try {
        // create destination folder if not found
        if (!existsSync(options.destination)){
            mkdirSync(options.destination);
        }

        const html = readFileSync(options.source, 'utf8');

        // for every language replace content as we wish
        for (const lang of options.languages) {
            let contents = html.toString();

            if (lang.isRtl) {
                // remove reLTR
                contents = contents.replace(reLTR, '');
            } else {
                contents = contents.replace(reRTL, '');
            }
            // also replace lang
            contents = contents.replace(reLang, lang.name);

						// you should be doing one of the two following for your real life app
            // save file with index.lang.html, base href = /
            writeFileSync(`${options.destination}/index.${lang.name}.html`, contents.replace(reBase, '/'));
            // save file with index.lang.url.html with base href = /lang/
            writeFileSync(`${options.destination}/index.${lang.name}.url.html`, contents.replace(reBase, `/${lang.name}/`));
        }
    } catch (err) {
        context.logger.error('Failed to generate locales.');
        return {
            success: false,
            error: err.message,
        };
    }
    context.reportStatus('Done.');

    return { success: true };
}
```

This uses basic **synchronous file read and write**, and it could make use of more options, like whether it is URL based or not. I don't wish to complicate this tutorial, but you probably can think of more flowery code.

Our command is: `ng run cr:writeindex` (`cr` is our project name). Have a look at StackBlitz under [host/index folder](https://stackblitz.com/edit/angular-runtime-translation?file=host%2Findex%2Findex.ar.html,host%2Findex%2Findex.en.url.html,host%2Findex%2Findex.en.html,host%2Findex%2Findex.ar.url.html) to see the output index files.

This is it on this side. Let's move on to the server.

### Express Routes

In my silly server config file I added a new property: `pre` to try out different routes with prepared HTML. Find those routes with the suffix: `-pre` in the file name under `host/server`.

[StackBlitz prepared server routes](https://stackblitz.com/edit/angular-runtime-translation?file=host%2Fserver%2Froutes-url-pre.js,host%2Fserver%2Froutes-ssr-url-pre.js,host%2Fserver%2Froutes-pre.js,host%2Fserver%2Froutes-ssr-pre.js)

> As we move on from one episode to another, I spot a possible enhancement. With the addition of template engines, we could have dropped the `language.js` rewrite rule, but it kind of slipped. So today, we added the proper script to `placeholder.html`, and no longer have to rewrite it. The rule for `locale/language.js` is dropped.

Almost everything is the same, only the last rewrite that uses template engines, now will serve the right HTML page.

#### Browser only solutions

The final express route is:

```js
// in express routes, browser only, sendFile

// serve the right language index file for all urls
res.sendFile(config.rootPath + `index/index.${res.locals.lang}.html`);
```

#### SSR solutions

The rewrite rule that uses the `ngExpressEngine` now is simpler:

```js
// in express routes, with Angular ssr, render

// serve the right language index
res.render(config.rootPath + `index/index.${res.locals.lang}.html`, {
  req,
  res
});
```

Our express server is becoming simpler as we go on. This opens up more options, like hosting on different cloud hosts, and client-only hosts. Let us first see if we can use gulp, instead of an Angular builder.

Gulp it
-------

We are going to do the same, create gulp in a sub folder, with all what's needed. This makes the main packages simpler, and cleaner, and allows us to later move those tasks to an `npm` shared package. Inside a newly created **gulp folder**, we shall initialize `npm`, and install at least `gulp`. We begin with `gulpfiles.js`, and a single file `locales/index.js`

One way to accomplish the the gulp task is by simply doing NodeJs function calls, repeating the same code above:

```js
// the simplest form of gulp tasks, gulping without using gulp
const { readFileSync, writeFileSync, existsSync, mkdirSync } = require('fs');

const options = {
    source: '../host/client/placeholder.html',
    destination: '../host/index',
    languages: [
        { name: 'ar', isRtl: true },
        { name: 'en', isRtl: false },
        { name: '...', isRtl: ...}]
}
// define same consts, then create the same function in ./builder
exports.LocalizeIndex = function (cb) {
	// instead of context.logger use console.log
	// instead of return function use cb()
   cb();
}
```

In `gulpfile.js` we export the main function.

> You've probably noticed by now that I pick different names for the same thing, reason is, I always want to remember what goes where, and how much dependency these terms have. Obviously, none! `writeindex` has nothing to do with the function name `LocalizeIndex`

```js
const locales = require('./locales');
// export the LocalizeIndex as a "writeindex" gulp task
exports.writeindex = locales.LocalizeIndex;
```

To run it, the command is:

`gulp writeindex`

To run it from our root package, we need to first change directory (windows, sorry think-different people!), then gulp it:

```json
// in root package.json
"postbuild": "cd ./gulp && gulp writeindex",
```

> Note: you always need `gulp cli` to run this command, in a subfolder, or using an `npm` package. Gulp is much more expensive than Angular CLI builders. Some gulp packages have gone stale as well, making it harder to deal with.

Using proper gulp plugins however is more rewarding in the long run. What we need to accomplish is the following:

```js
const _indexEn = function() {
    // read placeholder.html
    return gulp.src('placeholder.html').pipe(
        // transform it with specific language
        transform(function(contents, file) {
            // rewrite content
        }, { encoding: 'utf8' })
				// rename file
        .pipe(rename({ basename: `index.en` }))
				// save to destination
        .pipe(gulp.dest(options.dest))
    );
}
// another one:
const _indexAr = function() {
  // ...
}
// run them in parallel
exports.localizeIndez = gulp.parallel(_indexEn, _indexAr, ...);
```

Besides, `gulp`, we need `gulp-rename` and `gulp-transform`. Install and keep an eye on them, **they are really out of date**. For this tutorial, we are going to produce both index files, with URL based and cookie based apps, but in real life, we'd already know which type we target.

```js
// here is proper gulping

const gulp = require('gulp');
// those plugins are not kept up to date, maybe one Tuesday we shall replace them?
const rename = require('gulp-rename');
const transform = require('gulp-transform');

const options = {
  // relative to gulpfile.js location
  source: '../host/client/placeholder.html',
  destination: '../host/index',
  // allow both types of apps
  isUrlBased: false,
  languages: [
    { name: 'ar', isRtl: true },
    { name: 'en', isRtl: false },
    { name: '...', isRtl: ...},
  ],
};

const reLTR = /<!-- #LTR -->([\s\S]*?)<!-- #ENDLTR -->/gim;
const reRTL = /<!-- #RTL -->([\s\S]*?)<!-- #ENDRTL -->/gim;
const reLang = /\$lang/gim;
const reBase = /\$basehref/gim;

// base function, returns a function to be used as a task
const baseFunction = function (urlBased, lang) {
  return function () {
    // source the placeholder.html
    return gulp.src(options.source).pipe(
      // transform it with specific language
      transform(function (contents, file) {
        // rewrite content
        if (lang.isRtl) {
          contents = contents.replace(reLTR, '');
        } else {
          contents = contents.replace(reRTL, '');
        }

        //  replace lang
        contents = contents.replace(reLang, lang.name);
        // replace base href
        return contents.replace(reBase, urlBased ? `/${lang.name}/` : '/');
      }, { encoding: 'utf8' }))
      // rename file to index.lang.url.html
      .pipe(rename({ basename: `index.${lang.name}${urlBased ? '.url' : ''}` }))
      // save to destination
      .pipe(gulp.dest(options.destination));
  };
};

// for tutorial's purposes, create both url and cookie based files
const allIndexFiles = [];
options.languages.forEach((n) => {
  allIndexFiles.push(baseFunction(true, n));
  allIndexFiles.push(baseFunction(false, n));
});

// in real life, one option:
// const allIndexFiles = options.languages.map(language => baseFunction(options.isUrlBased, language));

// run in parallel
exports.LocalizeIndex = gulp.parallel(...allIndexFiles);
```

This is it. Whether we go with Angular or Gulp, the result is the same. The choice would be swayed either way in the presence of other priorities.

Hosting on Netlify
------------------

There is one more option to force an Angular app to serve different URLs with the same build. To find out, let's try to publish a client-only application on Netlify, where the environment is more strict. Come back next week for the joy. 😴

Thank you for reading this far, are you keeping up with the original task: twisting Angular localization?

### RESOURCES

-   [StackBlitz project](https://stackblitz.com/edit/angular-runtime-translation)
-   [Angular CLI builders](https://angular.io/guide/cli-builder)
-   [Angular CLI under the hood - builders demystified](https://www.justjeb.com/post/angular-cli-under-the-hood-builders-demystified)
-   [Typescript: Why does --outDir moves output after adding a new file?](https://github.com/Microsoft/TypeScript/wiki/FAQ#why-does---outdir-moves-output-after-adding-a-new-file)
-   [Gulp](https://gulpjs.com/)