npm package: @springernature/assets-pipeline
Description: Assets bundler utility to power assets pipeline on projects
Author: springernature
Latest version: 0.2.2

# Assets pipeline
![NPM Version](https://img.shields.io/npm/v/%40springernature%2Fassets-pipeline)

Springer Nature assets pipeline library is a tool to help you design frontend
assets build pipelines for your applications.  
It is rather opinionated toward tools and configurations we favour here at
Springer Nature (e.g. `sass`, `esbuild`, as well as the `eslint`, `stylelint`
linters).

A few benefits to adopt this library:
- Play by the Springer Nature recommended guide lines
- Consolidate the frontend assets pipeline authoring experience across our
applications
- Maintenance and evolution of the library is shared across our discipline. This
effort is indeed centralised at the library level, rather than being duplicated
on every projects. 

## Requirements

This package requires:

- [Node version 20 or greater](https://nodejs.org/en/download/releases).  
Please have a look at our [open source support
page](https://github.com/springernature/frontend-playbook/blob/master/practices/open-source-support.md#node-versions)
for details on which versions of node we support, and why.
- NPM 5+
- Its peer dependencies to be installed in the host project/machine.

## Installation

This library aims to be installed as a development dependency, and can be
install via `npm`:

```sh
npm install -D @springernature/assets-pipeline
```

## Usage

Once the module installed, you then have the `sn-assets-pipeline` command at
your disposal.

With a local installation, the `sn-assets-pipeline` command will not be
available in your system path or you can't use it directly from the command
line. Instead, the local installation of `sn-assets-pipeline` can be run by
calling it from within an npm script (such as `npm run build` or `npm run
serve`) or using `npx sn-assets-pipeline`.

It is meant to be used like:

```sh
sn-assets-pipeline <command>
```

where `<command>` is one of the following:

- `help`: Displays the command's usage
- `build`: Runs the pipeline
- `watch`: Runs the pipeline and re-run upon changes

The command is driven by a configuration: the assets pipeline configuration.  

There are 3 ways to provide this configuration and they follow the below
priority:

1. In a seperate file from which the path is referenced in a
`ASSETS_PIPELINE_CONFIG` environment variable. The path is resolved relatively
from the host project's directory containing the `package.json`.
2. In a seperate `assets-pipeline.config.json` file in the host project's
3. As an `assetsPipeline` property of the host project's `package.json` file
directory

Each projects has it's own structure and therefore there is no such thing as a
default configuration. Please refer to the [Configuration
section](#assets-pipeline-configuration) to setup you very own assets pipeline
configuration.

## Tasks supported by the assets pipeline library

### `clean` task

Taken an array of file/directory paths as a `targets` property, it deletes the
listed files from the file system and empty the listed directories.

Using the `clean` task reveals useful when aiming to keep your distribution
directory current, reflecting the latest state of your assets, and free from
obsolete assets that may lead to potential errors.

[Example of a `clean` task configuration](#clean-task-example)

### `copyAssets` task

Taken an array of file/directories assets described by their `source` and
`destination` properties, it makes a copy of each asset from its `source` to its
`destination`. If the parent directory, and/or any of its ancestors, of the
`destination` does not exist, they will be created.

[Example of a `copyAssets` task configuration](#copyassets-task-example)

### `lintSass` task

Taken an array of [glob file patterns](https://www.npmjs.com/package/glob), it
runs [`stylelint`](https://stylelint.io/) over them to lint any matching
[`Sass`](https://sass-lang.com/) files they match.

Linting relies on host project's stylelint configuration file.

> [!IMPORTANT]
> The host project should have the `stylelint` module installed as a dependency
> to use this task.

> [!WARNING]
> If the host project has a `.stylelintrc`, it must be in JSON format, and it is
> worth noting that it has precedence over any stylelint configuration file
> convention, e.g. `stylelint.config.js`. Thus, if no `.stylelintrc` is
> provided, stylelint searches upwards until it finds a `stylelint.config.js`
> file

[Example of a `lintSass` task configuration](#lintsass-task-example)

### `buildCss` task

Taken a `source`, `destination` among other [`Sass`](https://sass-lang.com/)
compilation options, it compiles Sass files into CSS files.

[Example of a `buildCss` task configuration](#buildcss-task-example)

### `lintJs` task

Taken an array of [glob file patterns](https://www.npmjs.com/package/glob), it
runs [`eslint`](https://eslint.org/) over them to lint any matching JavaScript
files they match.

Linting relies on host project's eslint configuration file.

> [!IMPORTANT]
> The host project should have the `eslint` module installed as a dependency to
> use this task.

[Example of a `lintJs` task configuration](#lintjs-task-example)

### `buildJs` task

Taken an array of `bundles`, it compiles these into browser friendly JavaScript
using [`esbuild`](https://esbuild.github.io/). Additionally it amends an [assets
manifest file](#what-is-the-assets-manifest-file) to include the compiled
javascript files.

**Note**: In case you want to exclude a bundle from the fingerprinting process and
therefore from being stored into the [assets
manifest file](#what-is-the-assets-manifest-file) , you can create a dedicated
[`buildJs` task](#buildjs-task) and ommit the `assetsManifestFile`.

[Example of a `buildJs` task configuration](#buildjs-task-example)

### `watchFiles` task

Taken an array of file/directory patterns to watch (i.e `watchPatterns`), it
does not do anything else than triggering the browser Live Reload.

That reveals useful for reloading after a change happened to "non-assets"
resources (e.g. view context-data, handlebars template...).

**Note**: Depending on how broad the spectrum of your `watchPatterns` is, this
may be cluttering your output. That is a perfect use case for the `quietWatch`
option.

[Example of a `watchFiles` task configuration](#watchFiles-task-example)

## Assets pipeline configuration

### Core properties

#### `tasks` property

An array of tasks objets, it represents the orchestration of
tasks for your pipeline. Therefore the order of tasks in the array dictates the
order of execution.  
Keep this in mind especially if you have a task
that depends on the outcome of another.  
For instance and a task of type `buildJs` may depend on the outcome of a
`copyAssets` task if the copy operation creates files that needs then to be
transpiled in the `buildJs` task.

Tasks types should be picked among [tasks supported by the assets
pipeline library](#tasks-supported-by-the-assets-pipeline-library).

[A full assets pipeline configuration example](#a-full-assets-pipeline-configuration-example)

#### `watchOptions` properties

##### `watchOptions.buildFirst` property

By default, the `watch` command will start by running a `build` and only then
sets itself up to watch for changes. In case you want to decouple the `build`
operation from the `watch` command, you can configure the
`watchOptions.buildFirst` boolean property as in the below example:

```json
"watchOptions": {
  "buildFirst": false
}
```

##### `watchOptions.ignorePatterns` property

An array of [glob file/directory patterns](https://www.npmjs.com/package/glob).
Any change from files and directories matching these patterns will be ignored by
the `watch` command, i.e will not trigger any task and browser live reload.

Below is an example that instructs to not trigger a Live Reload in case of
changes made to testing files:

```json
"watchOptions": {
  "ignorePatterns": [
  "*.test.js",
  "*.spec.js"
  ]
}
```

##### `watchOptions.liveReload` property

If you pass a [`browser-sync`](https://browsersync.io/) configuration here then
live reload will be implemented as part of the watch command.

The `port` property corresponds to the port the `browser-sync` server should
be run from.

Below is an example of a suggested live reload configuration:

```json
"watchOptions": {
  "liveReload": {
      "online": false,
      "open": false,
      "port": 3001,
      "ui": false,
      "proxy": "localhost:8080",
      "logPrefix": "Live reload"
    }
}
```

### Tasks anatomy

The common properties for a tasks are:

- `type` (required): String to be picked among supported tasks:
	- [`clean`](#clean-task)
	- [`copyAssets`](#copyassets-task)
	- [`lintSass`](#lintsass-task)
	- [`buildCss`](#buildcss-task)
	- [`lintJs`](#lintjs-task)
	- [`buildJs`](#buildjs-task)
	- [`watchFiles`](#watchfiles-task)
- `name` (optional): an optional name for the task, which will be displayed in
the console output of your Terminal
- `watchPatterns` (optional): Array of file/directory patterns to match assets
that needs to be monitored for changes in order to re-trigger the execution of
the task and a browser Live Reload. This option is consumed by the `watch`
command. 
- `quietWatch` (optional): Option consumed by the `watch` command. If `true`, it
will not output messages like "Executing task X due to Asset changes in
PATTERN".

In the below examples we will be focusing on the specific properties.

Let's go over the different supported tasks and learn about them by example.

#### `clean` task example

Below is an example of [`clean` task](#clean-task) configuration that instructs
a clean up of the `dist` directory, as well as the deletion of the `foo.bar`,
through its `targets` property.

```json
{
  "name": "Clean dist directory and foo.bar file",
  "type": "clean",
  "targets": [
    "dist",
    "foo.bar"
  ]
},
```

#### `copyAssets` task example

Below is an example of [`copyAssets` task](#copyassets-task) configuration that
instructs the copy of a bunch of assets toward a `dist` directory.

Each `asset` in the `assets` array is described by a `source` and `destination`
property that are hopefully self-explanatory.

The `watchPatterns` option is taken into account in case of running the
`watch` command.  
Below it instructs for a new execution of the `copyAssets` task in case any
change happens under `src/assets/img/logos`, `src/assets/img/icons`,
`src/assets/fonts` or `node_modules/@springernature/elements` directory.

```json
{
  "name": "Copy necessary assets",
  "type": "copyAssets",
  "assets": [
    {
      "source": "src/assets/img/logos",
      "destination": "dist/img/logos"
    },
    {
      "source": "src/assets/img/icons",
      "destination": "dist/img/icons"
    },
    {
      "source": "src/assets/fonts",
      "destination": "dist/fonts"
    },
    {
      "source": "node_modules/@springernature/elements/themes/springernature/img/icons/eds-i-user-single-medium.svg",
      "destination": "dist/img/icons/eds-i-user-single-medium.svg"
    }
  ],
  "watchPatterns": [
    "src/assets/img/{logos,icons}/**/*.*",
    "src/assets/fonts/**/*.*",
    "node_modules/@springernature/elements/**/*.*"
  ]
}
```

#### `lintSass` task example

Below is an example of [`lintSass` task](#lintsass-task) configuration that
instructs, through the `watchPatterns` property, the linting of a bunch of Sass
file from 2 different directories: `src/css` and `src/packages`.

The `watchPatterns` has a double function for that task property as it also
serves as a reference for the `watch` command to know what to monitor for
changes.

The `exitOnError` property lets you decide if the pipeline should break upon
lint issue (`"exitOnError": true`), or if it should continue, printing out
silentely the lint issues (`"exitOnError": false`). It defaults to `true`.

```json
{
  "name": "Lint sass files with stylelint",
  "type": "lintSass",
  "watchPatterns": [
    "src/css/**/*.scss",
    "src/packages/**/*.scss"
  ],
  "exitOnError": true
}
```

#### `buildCss` task example

Below is an example of [`buildCss` task](#buildcss-task) configuration that
instructs the processing of a bunch of Sass files, hosted into source directory
`src/css`, toward a `dist` directory.

The `sassOptions` is a valid Sass options configuration as by [Dart Sass
documentation](https://sass-lang.com/documentation/js-api/interfaces/Options).
The one from the example instructs the creation of source map files for the
generated CSS files.

The `autoprefixer` set to `true`, instructs to autoprefix CSS properties
according to existing `.browserslistrc` file or `browserlist` property in
`package.json`.

The `assetsManifestFile` instructs to create or update the specified [assets
manifest file](#what-is-the-assets-manifest-file) with the generated assets.

The `watchPatterns` option is taken into account in case of running the
`watch` command.  
Below it instructs for a new execution of the `buildCss` task in case any
change happens under `src/css` or `src/packages` directory.


```json
{
  "name": "Compile sass files with dart Sass",
  "type": "buildCss",
  "source": "src/css",
  "destination": "dist",
  "sassOptions": {
    "sourceMap": true
  },
  "autoprefixer": true,
  "assetsManifestFile": "dist/assets.json",
  "watchPatterns": [
    "src/css/**/*",
    "src/packages/**/*"
  ]
}
```

#### `lintJs` task example

Below is an example of [`lintJs` task](#lintjs-task) configuration that
instructs, through the `watchPatterns` property, the linting of a bunch of
JavaScript file from 2 different directories: `src/js` and `src/shared`.

The `watchPatterns` has a double function for that task property as it also
serves as a reference for the `watch` command to know what to monitor for
changes.

The `exitOnError` property lets you decide if the pipeline should break upon
lint issue (`"exitOnError": true`), or if it should continue, printing out
silently the lint issues (`"exitOnError": false`). It defaults to `true`.

```json
{
  "name": "Lint JS files with eslint",
  "type": "lintJs",
  "watchPatterns": [
    "src/js/**/*",
    "src/shared/**/*"
  ],
  "exitOnError": true
},
```

#### `buildJs` task example

Below is an example of [`buildJs` task](#buildjs-task) configuration that
instructs the creating of 2 JavaScript bundles: `dist/main.js` and
`dist/publication.js` meant to be served by the browser.

The `bundles` property is an array of [ESBuild](https://esbuild.github.io)
configurations.

> **Important**: So far we only support the [`outfile` approach to ESBuild
> configuration](https://esbuild.github.io/api/#outfile) which is only applicable
> if there is a single entry point in the `entryPoints` array. We may broaden the
> types of configuration we support in future iterations if the need arises.

The `buildJs` task, is using a bunch of sensible defaults that hopefully
everyone can agree on:

```javascript
//...

// Deduced from the frontend playbook
target: [
	'firefox67',
	'chrome76',
	'safari12',
	'edge79',
	'opera62'
],
format: 'iife',
sourcemap: true,
bundle: true,
// Minifying does not make sense in development environment
minify: process.env.NODE_ENV !== 'development'

// ...
```

These defaults can be overriden in each of your `bundles` entries, but at your
own risk.

The `assetsManifestFile` instructs to create or update the specified [assets
manifest file](#what-is-the-assets-manifest-file) with the generated bundles.

The `watchPatterns` option is taken into account in case of running the
`watch` command.  
Below it instructs for a new execution of the `buildJs` task in case any
change happens under `src/js` or `src/shared` directory.


```json
{
  "name": "Transpile and bundle JavaScript files with esbuild",
  "type": "buildJs",
  "bundles": [
    {
      "entryPoints": [
        "src/js/main.js"
      ],
      "outfile": "dist/main.js"
    },
    {
      "entryPoints": [
        "src/js/publication.js"
      ],
      "outfile": "dist/publication.js"
    }
  ],
  "assetsManifestFile": "dist/assets.json",
  "watchPatterns": [
    "src/js/**/*",
    "src/shared/**/*"
  ]
}
```

#### `watchFiles` task example

Below is an example of [`watchFiles` task](#watchfiles-task) configuration that
instructs a browser Live Reload upon changes under the `handlebars` or the
`context-data` directories.

```json
{
  "type": "watchFiles",
  "name": "Watch for changes in templates and/or context data",
  "quietWatch": true,
  "watchPatterns": [
    "handlebars/**/*.*",
    "context-data/**/*.*"
  ]
},
```
### A full assets pipeline configuration example

```json
{
  "tasks": [
    {
      "name": "Clean dist directory",
      "type": "clean",
      "targets": [
        "dist"
      ]
    },
    {
      "name": "Copy necessary assets",
      "type": "copyAssets",
      "assets": [
        {
          "source": "src/assets/img/logos",
          "destination": "dist/img/logos"
        },
        {
          "source": "src/assets/img/favicons",
          "destination": "dist/img/favicons"
        },
        {
          "source": "src/assets/fonts",
          "destination": "dist/fonts"
        },
        {
          "source": "src/assets/img/icons",
          "destination": "dist/img/icons"
        },
        {
          "source": "node_modules/@springernature/elements/themes/springernature/img/icons/eds-i-user-single-medium.svg",
          "destination": "dist/img/icons/eds-i-user-single-medium.svg"
        },
        {
          "source": "node_modules/@springernature/elements/themes/springernature/img/icons/eds-i-chevron-right-small.svg",
          "destination": "dist/img/icons/eds-i-chevron-right-small.svg"
        }
      ],
      "watchPatterns": [
        "src/assets/img/{logos,favicons,icons,illustrations}/**/*.*",
        "src/assets/fonts/**/*.*"
      ]
    },
    {
      "name": "Lint sass files with stylelint",
      "type": "lintSass",
      "watchPatterns": [
        "src/css/**/*.scss",
        "src/packages/**/*.scss"
      ],
      "exitOnError": true
    },
    {
      "name": "Compile sass files with dart Sass",
      "type": "buildCss",
      "source": "src/css",
      "watchPatterns": [
        "src/css/**/*",
        "src/packages/**/*"
      ],
      "destination": "dist",
      "sassOptions": {
        "sourceMap": true
      },
      "autoprefixer": true,
      "assetsManifestFile": "dist/assets.json"
    },
    {
      "name": "Lint JS files with eslint",
      "type": "lintJs",
      "watchPatterns": [
        "src/js/**/*",
        "src/shared/**/*",
        "tooling/**/*"
      ],
      "exitOnError": false
    },
    {
      "name": "Transpile and bundle Javascript files with esbuild",
      "type": "buildJs",
      "watchPatterns": [
        "src/js/**/*"
      ],
      "bundles": [
        {
          "entryPoints": [
            "src/js/main.js"
          ],
          "outfile": "dist/main.js"
        },
        {
          "entryPoints": [
            "src/js/publication-details.js"
          ],
          "outfile": "dist/publication-details.js"
        }
      ],
      "assetsManifestFile": "dist/assets.json"
    },
    {
      "name": "Watch for changes in the express server",
      "type": "watchFiles",
      "quietWatch": true,
      "watchPatterns": [
        "src/server/**/*",
        "src/view/**/*",
        "src/packages/**/*",
        "tooling/**/*",
        "fixtures/**/*",
        "context-data/**/*"
      ]
    }
  ],
  "watchOptions": {
    "buildFirst": false,
    "ignorePatterns": [
      ".git",
      "node_modules/**/node_modules",
      "*.test.js",
      "*.spec.js"
    ],
    "liveReload": {
      "online": false,
      "open": false,
      "port": 8081,
      "ui": false,
      "proxy": "local-app.springernature.com:8080",
      "logPrefix": "Live reload"
    }
  }
}
```

## What is the assets manifest file?

A recommended pratice is to fingerprint your assets so that you can cache these
files indefinitely. This reduces the number of request the browser needs to
make.  
Fingerprinting is the operation to suffix file names with a hash of the
content of the file (e.g. `main-core-f7f9f66.css` is the fingerprinted file name
for `main-core.css`).

In order to avoid the need for developers to patch any reference of
fingerprinted assets in the code, the idea of an assets manifest file arose. It
contains a JSON object from which fingerprinted filenames can easily be resolved
by your application from their original name.

This manifest file can be created/amended while building the project's assets.
The current assets pipeline library offers, as part of its
[`buildCss`](#buildcss-task) and
[`buildJs`](#buildjs-task) tasks an `assetsManifestFile` property that lets you provide the path
for this file.

The below example file has been generated at build time while building the
following assets: `main-core.css`, `main-enhanced.css`, `main.js` and
`publication.js`.

```json
{
  "main-core": {
    "css": "main-core-f7f9f66.css"
  },
  "main-enhanced": {
    "css": "main-enhanced-1349dc0.css"
  },
  "main": {
    "js": "main-5701042.js"
  },
  "publication": {
    "js": "publication-fbf5769.js"
  }
}
```

Taken you would store the content of this manifest file in an `assets` variable
made available in a handlebars template; you could reference `main-core.css`
this way:


```handlebars
<link rel="stylesheet" href="/{{assets.main-core.css}}">
```

This would then render in the page source HTML as:

```html
<link rel="stylesheet" href="/main-core-f7f9f66.css">
```