npm package: @oliasoft-open-source/node-json-migrator
Description: A library for JSON migrations
Author: Oliasoft AS and contributors
Latest version: 5.0.1

# Node JSON Migrator

## A simple JSON change management tool for Node.js

- Lightweight JSON change management tool for `Node.js` applications
- Written in JavaScript
- Pre-transpiled (no extra build steps)
- Not tied to any framework
- Open source (MIT License)

## Features

- Versioning of JSON payloads
- Write change scripts (migrations) in JavaScript
- Apply changes to JSON payloads using a convenient `migrate()` function
- Works great in a team setting (multiple developers can track and handle changes)
- Supports automated change deployment across multiple environments
- Automatically tracks and applies only new (pending) change scripts
- Automatically detects and warns when previous scripts have been altered
- Optionally track version history in a PostgreSQL database (pg-promise interface)

# How to use

## Installation and setup

The database (if used) must already exist and have permissions granted.

Install the tool directly from Gitlab (published NPM packages may be available in future):

```
npm install --save git+https://gitlab.com/oliasoft-open-source/node-json-migrator.git
```

## Defining migrations (change scripts)

Use the `createMigration()` function to generate a skeleton migration script, for example:

```javascript
import {createMigration} from 'node-json-migrator';

const directory = 'json-migrations';
const description = 'add-panda';
await createMigration(directory, description);
```

This will create and populate skeleton files:

- `add-panda/add-panda.js` is the skeleton change script (alter this to describe the desired payload change)
- `plan.json` is the execution plan (defines the execution order / sequence for migration scripts)

Migration scripts export default functions that receive a payload and return the next payload.
They must use immutable update patterns, and [Immer](https://github.com/immerjs/immer) is recommended for this.
For example, after editing, the scripts might look like this:

**json-migrations/add-panda/add-panda.js** (adds a property to the payload)

```javascript
import { produce } from 'immer';

export default (payload) = produce(state, (draft) => {
   draft.panda = 'Elliot';
});
```

**json-migrations/plan.json** (defines the migration execution plan)

```json
[
  {
    "fileHash": "70fd80fd19828435b8656b8d01f99b840381050ded4aec2dd126e2f0c9864c88",
    "fileName": "add-panda.js",
    "sequence": "1"
  }
]
```

Note: the `fileHash` property is the SHA256 hash of the migration file content. You don't have to manually maintain 
that, the tool will sync it automatically for you.

## Running migrations

In your application source code, import and call the `migrate()` function. The simplest usage (runs all migrations) 
is:

```javascript
import {migrate} from 'node-json-migrator';

const {nextPayload} = await migrate({
  payload: {},
  config: {
    directory: 'json-migrations'
  }
});
```

Normally you want to track payload versions, and only execute pending migrations. To do this, you need to connect to a 
PostgreSQL database and pass a
connected [pg-promise database object](http://vitaly-t.github.io/pg-promise/). Then it looks like this:

```javascript
import pgPromise from 'pg-promise';
import {migrate} from 'node-json-migrator';

//connect to the database somewhere in the database layer
const pgp = pgPromise({});
const pgpHelpers = pgp.helpers;
const database = pgp(/*connection string*/);
database.connect();

//if we know the current payload version, we can run only the newer, pending migration scripts
const currentPayloadVersion = undefined;

const {nextPayload, nextVersion} = await migrate({
  payload: {},
  config: {
    directory: 'json-migrations',
    database, //connected database object
    pgpHelpers, //pg-promise helpers: https://vitaly-t.github.io/pg-promise/helpers.html
    version: currentPayloadVersion
  }
});
```

**Other config options**

```javascript
config = {
  dry: true, //execute without writing to database tables or files
  force: true, //execute bypassing some validation warnings
}
```

## Usage Rules

- files in the plan must exist
- filenames must be unique and follow formatting rules
- sequences must be unique
- not allowed to change sequence numbers after releasing (merging) a migration
- not allowed to remove released files from the plan
- should release only one migration script at a time
- not allowed to alter/modify released migration files (instead, write a new migration to replace it)
- migration scripts should be repeatable (not fail if they run twice on the same payload)

## Backfitting bugfixes

If you release a bad migration script, there is an escape-hatch for backfitting bugfixes:

- rename the faulty script to `.skip.js` (or `.skip2.js` if it happened more than once)
- add a new migration file with the original filename, to replace it (fix bug in new file)
- the replacement script will *not*  re-execute on payloads that already had the original script applied

Or, you can backfit a script earlier in the sequence

- add a new migration file
- set its sequence to for example `1.1` (subsequence number)
- the backfitted script *will* execute one time on all payloads

# How to contribute

Contribution is welcome via [issue tickets](https://gitlab.com/oliasoft-open-source/node-json-migrator/-/issues)
and [merge requests](https://gitlab.com/oliasoft-open-source/node-json-migrator/-/merge_requests).

- coding style (ESLint and prettier) is mandatory (applied via pre-commit hook)
- to test: `npm run test`
    - database is mocked with [pg-mem](https://github.com/oguimbal/pg-mem) (in-memory database)
    - filesystem is mocked [mock-fs](https://github.com/tschaub/mock-fs)
    - coverage reports are in `test/coverage` (not committed)
- to build: `npm run buld` (this transpile a production build to the `dist` directory)
    - the build `dist` directory should be committed