npm package: @incrediblez/i18n-fetcher
Description: A "simple" tool to get your i18n translations out of DB, and put them into i18n JSON file(s).
Author: Incrediblez
Latest version: 1.3.1

# What is it?
A "simple" tool to get your i18n translations out of DB, and put them into i18n JSON file(s).

Or vice versa.

Designed to be used as a part of CI/CD pipeline.

Codebase is availabe at [this Repo](https://bitbucket.org/incrediblez/i18-fetcher/src/main/).

Feel free to drop a feature request, proposal, or bug report.

# How to use?

## Via npx (cli)

1. Create config for the tool

    * see `Settings` chapter below for details

2. Create env file, if needed

    * see `Settings` chapter below for details

3. Run the command

```
npx -p @incrediblez/i18n-fetcher run --env=./somewhere/.env --config=./somewhere/config.json
```

Voila!

## Via npm module

```
npm i --save @incrediblez/i18-fetcher
```

And then just include the `runner.ts` in your project, and use as you wish.

# Settings

To run the package, you will need at least the `config` file.

Names and locations of the `env` file, and `config.json` file can be wherever, as long as Node process, that runs the package can reach these.

Custom locations/names can be supplied via command line arguments.

## Env file

Is optional, can be used to provide "secrets" for static and persisted "configs".

Using common `.env` file format.

You can use it to override settings of Database connection.

You can either specify whole set of DB connection settings in `config.json` file, in section `source` -> `config`, or supply it via `.env` file.

Structure of an env file:

```
  host?: String
  user?: String
  password?: String
  database?: String
  port?: String
```
More detailed info can be found at `common.ts` -> `type Env` definition.

Minimum required info is (regardless of where it is present, in `env` or in `config`):

* host
* user
* password
* database

## Creating config from the TS Type:

The definition of the Config type can be found at:

`common.ts`-> `Config`

Just follow the definition, and you'll be fine :)

## Creating config using schema:

You can also create the config by just creating empty `config.json` file, and add reference to the JSON Schema file, which will then help you out by providing intellisence for filling-in the config.

Just add your config, wherever you need it, and refer our `config.schema.json` schema file from it. F.e. like this:
```
{
  "$schema": "./config.schema.json"
}
```
And then - just fill-in all the required attibutes.

Notes:

* If you would like to use the Schema without installing anything explpicitly on your PC - you can refer tot he schema file via public URL of that file in our repo.

    Something like this:

    ```
    {
      "$schema": "https://bitbucket.org/incrediblez/i18-fetcher/raw/SPECIFIC_COMMIT_SHA/config.schema.json"
    }
    ```

    * To get the actual URL (with SPECIFIC_COMMIT_SHA variable included) - you can navigate to [i18-fetcher/src/main/config.schema.json](https://bitbucket.org/incrediblez/i18-fetcher/src/main/config.schema.json) in our repo, select desired commit on the left, and on the right menu select "Open raw" option. This will generate the link to the actual schema file.
* Most IDEs are supporting json schemas OOB, so - in majority of cases, you can enforce proper schema validation just by adding a refference to the Schema file.
In case your IDE does not automatically highlights the validation errors - just look for a plugin, with JSON support. That should do the trick.

* In this package - schema file is compilled straight from the TS Type, which ensures integrity across whole package, and allows you to use Schema validation, or checking the actual config class from the `common.ts`-> `Config` type. Whichever is easier for you.

## Config caveats:

* While filling-in the `config`, validator will point out that section `config` have to have all the parameters, required to access the DB.

  * But - you can also provide those settings in `.env` file, making sure they are treated as "secrets" (see above chapter about `env`).

  * The runtime of the package will merge `.env` into that section automatically, before performing actual queries on DB.

* In case you are relying on JSON Schema, and your config file is NOT in the root of the package (which is totally OK) - modify `"$schema"`ref to point to the `config.schema.json` location.

# Synthetic language translations files

Package have quite oftenly needed, but not (yet) available anywhere else feature of creating "Synthetic" language translations file.

Basically it introduces "fake" language into your translation files, which, when selected in any system that relies on these translations, will populate all translation placeholders with their correlated translation keys.

In some cases it is very hard to get these keys-per-placeholders correlations out of vendors, or even your own codebase (if it grows too big).

This approach allows to introduce easy remedy for that, which can be used to speed up translations by content specialists, or debug issues with translations easily.

This feature works only on `Export` (creating translation files from Datasource), not vice versa.

To activate it - just provide `synthetic_column_name` field in your configuration.
Value - any language code.

```json
{
  "$schema": "./config.schema.json",
  "source": {
    "table_name": "my_supercool_translations",
    "setup": {
      "translation_key_column": "translation_key",
      "translation_columns": {
        "en": "labels_en",
        "fi": "labels_fi",
        ...
      },
      "synthetic_column_name": "eo" <-- THIS. Generates synthetic language translation file, with translation_keys, and calls it 'eo' (Esperanto).
    },
  }
  ...
}
```

# Import of i18n files into DB

In case you would like to put your current translations into DB - you can do that with this tool too.

It can also update existing table, with changes from your translation files (will respect info that is already in the Table, if there is any).

Configuration (and env file) requirements are the same, and will work same way (accessing the DB configured there, including specified Table, and use LANGUAGE to COLUMNS correlations, marked there.).

The only difference is that - this mode does all that in "reverse" order.

1. Read i18n files into "common" translations object
2. Read DB data
3. Match against DB (find missing, new, or changed translations)
4. Merge those changes into DB, one-by-one.

To activate that mode, just add `instruction=import` flag to your command.

Like this:

```
npx -p @incrediblez/i18-fetcher run --env=./somewhere/.env --config=./somewhere/config.json --instruction=import

```

## Limitations

* This mode works only with "simply"-named files.

    * Setting `destination.name_template` will be ignored in this mode.
    * In case you have multifile setup (one file per language), these files have to be named like `LANG.json`. E.g.: `./somewhere/en.json`, `./somewhere/fi.json`, `./somewhere/etc.json`.


* This mode expects all translations to be provided in "flat" keys.

    **Like this:**

    ```
    {
      "en": { "translation.key.a": "en.a", "translation.key.b": "en.b", "translation.key.c": "en.c", "translation.key.d": "en.d" },
    }
    ```

    **NOT like this:**

    ```
    {
      "en": {
          "translation": { "key": { "a": "en.a", "b": "en.b", "c": "en.c", "d": "en.d" } }
        }
    }
    ```
    _This is a subject of change, when I have time to finish this transformation handler..._

* If one of your translation languages is missing translation in files, but others do not - this mode will clear out translations only for those specific languages in DB. It will not remove whole translation.

* If you would like to remove specific translation key from DB - make sure it is removed on all languages.

# Development of a package

This section is designed for contributors.

## Dev-related Schema updates

To update the config schema, in case you have modified something there, or added some jsdoc comments - just run:

```
node ./node_modules/ts-json-schema-generator/bin/ts-json-schema-generator.js --path './common.ts' --type='Config' > config.schema.json
```

It will update the schema with latest changes. Including comments.

## Run package locally

```
npm run start -- --env=./.env --config=./config.json
```

Or, if you have your `env` at `./.env`, and your `config` at `./config.json` - just:

```
npm run start
```


# Caveats

* Currently tool supports only MySQL2 as a source.

* Currently tool supports only one mode, when translations table contains tralsnations in columns:

  | translation_id | translation_en | translation_etc |
  | -------------- | -------------- | --------------- |
  | homepage.hello | Hello there!   | ?!??!?!?!?!?!?! |

We do have plans (and initial support) for other use-cases, but as currently we are the only ones who are using the tool - we scope development to our accute needs.

Further development of the tool will depend on Community requests, and our needs :)

Please feel free to reach out to us for requests of additional features.