npm package: @mediarithmics/plugins-nodejs-sdk
Description: This is the mediarithmics nodejs to help plugin developers bootstrapping their plugin without having to deal with most of the plugin boilerplate
Latest version: 0.38.0

#### Note about 0.6.0

_Warning:_ We introduced a breaking change about a Typescript Interface definition associated with the `ActivityAnalyzer` support in the `0.7.0.` version of the SDK. This change is fixing a bug that was, in the end, ignoring all the Email hash related processing (User matching, user deduplication, etc.) on mediarithmics platform.

If you are using the Typescript associated types for an `Activity Analyzer`, we recommend you to upgrade to `v0.7.0+` ASAP. The `v0.6.0` was deprecated on NPM repository.

# Plugin SDK

This is the mediarithmics SDK for building plugins in Typescript or raw Node.js easily. As this package includes Typescript interfaces, we recommend that you use it with Typescript to ease your development.

It covers (as of v0.6.0):

- AdRenderer (incl. Templating systems + recommendations)
- Activity Analyzer
- Email Renderer
- Email Router
- Bid Optimizer
- Recommender
- External Audience Feed

## Installation

This module is installed via npm:

```
npm install --save @mediarithmics/plugins-nodejs-sdk
```

## Concepts

### Overall mechanism

The NodeJS plugin SDK consists of a set of abstract class that you have to implement. Those class provides a lot of helpers to make your life easier when having to make calls to the mediarithmics plugin gateway and/or to retrieve data.

This SDK also integrate a lot of very useful Typescript interfaces that we highly recommend you to use.

In order to implement your own logic while building your plugin, you have to override the "main" processing function of the abstract class in your impementation.

This function to override depend on the Plugin type. Those are:

- `onAdContents` for AdRenderer plugins
- `onActivityAnalysis` for Activity Analyzer plugins

If you need a custom Instance Context (see below), you can also override the 'instanceContextBuilder' function of the abstract class.

Once you will have provided your own implementation of the abstract class, you'll have to instantiate it and to give this instance to a 'Runner' that will run it as a web server app.

### Request

A mediarithmics plugin is called by the mediarithmics platform wih a 'Request' that contains all the Data to process / evaluate. Each type of plugin, depending on its functional behavior, is receiving a different request payload.

The plugin SDK contains a typescript interface describing the format of the request for each supported plugin.

A request can be:

- An User activity to process
- An Ad creative to render (e.g. generate HTML/JS)
- An email to render (e.g. generate HTML/raw text)
- A Bid Request to evaluate (e.g. should I bid on it? And at which price?)

Please see the complete documentation for each Plugin Type [here.](https://developer.mediarithmics.com/)

### Instance Context

A plugin instance can have a configuration that will change the way it will process Requests. As a plugin will be called numerous time to process incoming Requests, its configuration must be cached and only refreshed periodically. This SDK is helping you to manage this cache by providing you an "Instance Context" that represents this cache.

A default "Instance Context" is automatically provided by the SDK for each plugin type but you can also provide your own "Instance Context Builder" that will be called periodically to rebuild the cache.

If you need to have a custom _Instance Context_ format because you pre-calculate or charge in memory some values (ex: if you need to compile a Template / load in memory a statistic model / etc.), you can:

1. Override the default _Instance Context Builder_ function of the Plugin class
2. If you are using Typescript, you can extends the Base Interface of the Instance Context of your plugin that is provided so that you can add your custom fields

Note: The plugin instance configuration can be done through the mediarithmics console UI or directly by API.

### Runners

The SDK provides 2 different runners:

- `ProductionPluginRunner(plugin: BasePlugin)`: you have to use this runner in your main JS file. This runner is creating a web server to host your plugin so that it can be called by the mediarithmics platform.
- `TestingPluginRunner(plugin: BasePlugin, transport?: RequestPromiseMockup)`: this is a Runner used to write tests for your plugins. You can provide a mockup for RequestPromise as a parameter to help you writing your tests.

For details on how to use those 2 runners, please refer the examples code snippets provided with the SDK.

#### ProductionPluginRunner

After the instantiation of ProductionPluginRunner, you'll need to call `.start(port?: number, multiProcessEnabled?: boolean)` on the instance to start the web server.

- **port**: Tell on which port to start the webserver. When deployed on mediarithmics platform, the plugin need to listen to `8080` (which is the default value). When setuping local tests, it can be useful to change this port.
- **multiProcessEnabled**: When set to `true`, it tell the SDK to spawn as many processes as there are CPUs on the host in order to ultimately dispatch the load on different cores of the Plugin host. Defaults to `false`.

## Quickstart - Typescript

### SDK import

You have to import the 'core' module of the SDK in your code to access to the Abstract classes and Typescript interfaces. If you need some external integration, as the "Handlebar" templating system for example, you can also import the 'extra' package.

#### Example import

```js
import { core } from '@mediarithmics/plugins-nodejs-sdk';
```

### Abstract class implementation

When implementing a plugin class, you need to give him the main 'processing' function that he will process every time a Request is being received.

#### AdRenderer example

```js
export class MySimpleAdRenderer extends core.AdRendererBasePlugin<
  core.AdRendererBaseInstanceContext
> {
  protected async onAdContents(
    request: core.AdRendererRequest,
    instanceContext: core.AdRendererBaseInstanceContext
  ): Promise<core.AdRendererPluginResponse> {
    .....
  }
}
```

#### Activity Analyzer example

```js
export class MyActivityAnalyzerPlugin extends core.ActivityAnalyzerPlugin {
  protected onActivityAnalysis(
    request: core.ActivityAnalyzerRequest,
    instanceContext: core.ActivityAnalyzerBaseInstanceContext
  ): Promise<core.ActivityAnalyzerPluginResponse> {
    ......
  }
}
```

### Plugin Runner for production

Once you have implemented your own Plugin class, you have to instantiate it and to provide the instance to a Plugin Runner. For Production use, here is how you need to do it:

#### Activity Analyzer example

```js
const plugin = new MyActivityAnalyzerPlugin();
const runner = new core.ProductionPluginRunner(plugin);
runner.start();
```

#### AdRenderer example

```js
const plugin = new MySimpleAdRenderer();
const runner = new core.ProductionPluginRunner(plugin);
runner.start();
```

### Plugin Testing

This SDK provides you a 'TestingPluginRunner' that you can use to mock the transport layer of the plugin (e.g. emulate its call to the platform) and which expose the plugin 'app' on which you can trigger fake calls to test your plugin logic.

The Plugin examples provided with the SDK are all tested and you can read their tests in order to build your own tests.

Testing Plugins is highly recommended.

## Migration from 0.27.x to 0.28.x

The field `stats` has been removed from `BatchUpdatePluginResponse`.
The fields `sent_items_in_error` and `sent_items_in_success` are reintroduced instead.

## Migration from 0.25.x to 0.26.x

The fields `sent_items_in_error` and `sent_items_in_success` from `BatchUpdatePluginResponse` has been removed.
Instead, use the field `stats` which is an Array of `BatchUpdatePluginResponseStat` containing `errors`, `successes` and `operation` fields.

Before:

```ts
const response: core.BatchUpdatePluginResponse = {
  status: 'OK',
  message: 'test_batch_update',
  sent_items_in_error: 0,
  sent_items_in_success: request.batch_content.length,
};
```

After:

```ts
const response: core.BatchUpdatePluginResponse = {
  status: 'OK',
  message: 'test_batch_update',
  stats: [
    {
      successes: request.batch_content.length,
      errors: 0,
      operation: 'UPSERT',
    },
  ],
};
```

## Migration from 0.16.x to 0.17.x

- The `BatchUpdatePluginResponse` return now two new fields, `sent_items_in_error` and `sent_items_in_success`

```ts
export interface BatchUpdatePluginResponse {
  status: BatchUpdatePluginResponseStatus;
  message?: string;
  next_msg_delay_in_ms?: number;
  sent_items_in_success: number;
  sent_items_in_error: number;
}
```

- On `AudienceFeedConnectorBaseInstanceContext`, use `instanceContext.feed.selected_identifying_resources` instead of `instanceContext.selectedIdentifyingResources`

## Migration from 0.15.x to 0.16.x

The HTTP proxy is not used anymore and has been removed from the base class. Remove any usage of `proxyHost`, `proxyPort` and `proxyUrl`: call services directly without proxy.

## Migration from 0.12.x to 0.13.x

For audience feed connectors using `BATCH_DELIVERY` or `FILE_DELIVERY`:

- `grouping_key` is now mandatory for file and batch delivery responses
- `destination_token` is now mandatory for file delivery responses

## Migration from 0.11.x to 0.12.x

- Configuration and rule files has been added to the project.
- Formatting and linting was applied to every files.
- Warning: The tsconfig compilerOptions lib is now based on es2019, meaning that support of nodejs <= 11 is dropped. Check the jenkins environment type used to build your plugin image to check the node version used.

## Migration from 0.10.x to 0.11.x

- new AudienceFeedConnector BatchUpdate route and methode:

- POST: `'/v1/batch_update'` <br/>
- The purpose of this route is to receive a set of batchs of type `T` that were sent one by one in the `onUserSegmentUpdate` response.

```ts
export interface BatchUpdateRequest<T> {
  batch_content: T[];
  ts: number;
  context: AudienceFeedBatchContext;
}

export interface AudienceFeedBatchContext {
  endpoint: string;
  feed_id: string;
  feed_session_id: string;
  segment_id: string;
  datamart_id: string;
}

export interface BatchUpdatePluginResponse {
  status: DeliveredDataPluginResponseStatus;
  message?: string;
  next_msg_delay_in_ms?: number;
}
```

- Breaking changes in UserSegmentUpdatePluginResponse (UPDATE): <br/>

When responding in the `onUserSegmentUpdate` method, we can send data to **FILE_DELIVERY** or **BATCH_DELIVERY**. <br/> **FILE_DELIVERY** accepts a `DeliveryType.content` of type `string`. <br/> **BATCH_DELIVERY** accepts a `DeliveryType.content` of type `T`. <br/>

```ts
export interface UserSegmentUpdatePluginResponse {
  status: UserSegmentUpdatePluginResponseStatus;
  data?: DeliveryType<unknown>[];
  stats?: UserSegmentUpdatePluginResponseStats[];
  message?: string;
  next_msg_delay_in_ms?: number;
}

export type DeliveryType<T> =
  | UserSegmentUpdatePluginFileDeliveryResponseData
  | UserSegmentUpdatePluginBatchDeliveryResponseData<T>;

export interface UserSegmentUpdatePluginFileDeliveryResponseData
  extends UserSegmentUpdatePluginDeliveryContent<string> {
  type: 'FILE_DELIVERY';
  destination_token?: string;
  grouping_key?: string;
}

export interface UserSegmentUpdatePluginBatchDeliveryResponseData<T> extends UserSegmentUpdatePluginDeliveryContent<T> {
  type: 'BATCH_DELIVERY';
}

export interface UserSegmentUpdatePluginDeliveryContent<T> {
  content?: T;
}
```

## Migration from 0.9.x to 0.10.x

- Breaking changes in UserSegmentUpdatePluginResponse:

AudienceFeedConnector's onUserSegmentUpdate method return type has been updated. The optional data element used to be a of type:

```ts
export interface UserSegmentUpdatePluginResponseData {
  destination_token?: string;
  grouping_key?: string;
  content?: string;
  binary_content?: BinaryType;
}
```

it has been updated, depending on the delivery type used, to:

```ts
export type DeliveryType =
  | UserSegmentUpdatePluginFileDeliveryResponseData
  | UserSegmentUpdatePluginBatchDeliveryResponseData;

export interface UserSegmentUpdatePluginResponseData {
  grouping_key?: string;
  content?: string;
  binary_content?: BinaryType;
}

export interface UserSegmentUpdatePluginFileDeliveryResponseData extends UserSegmentUpdatePluginResponseData {
  type: 'FILE_DELIVERY';
  destination_token?: string;
}

export interface UserSegmentUpdatePluginBatchDeliveryResponseData extends UserSegmentUpdatePluginResponseData {
  type: 'BATCH_DELIVERY';
  batch_token?: string;
}
```

Other changes in the interface:

- status can be 'no_eligible_identifier' now (status code 400);
- stats field is changed (UserSegmentUpdatePluginResponseStats);
- in stats, identifier and sync_result become compulsory;
- SyncResult can now have only 3 values (PROCESSED, SUCCESS and REJECTED) in stats;
- tags in stats is now an optional list of tags;

## Migration from 0.8.x to 0.9.x

The init workflow changed, from a `POST /v1/init` call to tokens given in the environment. The **tests** need to be updated:

The previous

```js
request(runner.plugin.app)
  .post('/v1/init')
  .send({...})
  .end((err, res) => { ...
```

is not needed anymore, use the following instead, at the top of your **test** file (replace the values):

```js
process.env.PLUGIN_WORKER_ID = '<previously used worker id>';
process.env.PLUGIN_AUTHENTICATION_TOKEN = '<previously used auth token>';
```

## Migration from 0.6.0 to 0.7.0+

We introduced a non retrocompatible change between 0.6.0 and 0.7.0 SDK version to fix a bug.

The `UserActivity.$email_hash` interface (`EmailHash`) was updated from:

```js
export interface EmailHash {
  hash: string;
  email?: string;
}
```

to

```js
export interface EmailHash {
  $hash: string;
  $email?: string;
}
```

Hence, the fields name were updated; if you were referencing them in your code, you have to refactor it by prepending a `$`.

## Migration from 0.5.x to 0.6.x

- `click_urls` property of `AdRendererRequest` is replaced with `click_urls_info`.

```js
AdRendererBasePlugin.getEncodedClickUrl(redirectUrls: string[])
```

is now

```js
AdRendererBasePlugin.getEncodedClickUrl(clickUrlInfos: ClickUrlInfo[])
```

To push a url in the redirect chain and build an encoded url, for example

```js
if (instanceContext.creative_click_url) {
  adRenderRequest.click_urls.push(instanceContext.creative_click_url);
}

clickUrl = this.getEncodedClickUrl(adRenderRequest.click_urls);
```

should become

```js
if (instanceContext.creative_click_url) {
  adRenderRequest.click_urls_info.push({
    url: instanceContext.creative_click_url,
    redirect_count: 0,
  });
}
clickUrl = this.getEncodedClickUrl(adRenderRequest.click_urls_info);
```

## Migration from 0.4.x to 0.5.x

The 0.5.x release of the Plugin SDK is mainly aiming at simplifying the use of the "Templating" API.

- `engineBuilder` property of `EmailRendererTemplate` and `AdRendererTemplatePlugin` is now declared `abstract` in the SDK and should no longer be instanciated in the Plugin Impl. `constructor` but directly in the class itself.

```js
  constructor(enableThrottling = false) {
    super(enableThrottling);
    this.engineBuilder = new extra.RecommendationsHandlebarsEngine();
  }
```

should become

```js
  engineBuilder = new extra.RecommendationsHandlebarsEngine();

  constructor(enableThrottling = false) {
    super(enableThrottling);
  }
```

- the `EmailRendererTemplate.fetchTemplateContent()` and `AdRendererTemplatePlugin.fetchTemplateContent()` methods have been deleted. They should be replaced by: `BasePlugin.fetchDataFile()` method which is equivalent.

- the `AdRendererTemplatePlugin.fetchTemplateContentProperties()` method have been deleted because it was using the AdLayout mediarithmics API which is being deprecated in favor of DataFile API. If you were using it, you should migrate your template files to a DataFile Plugin property instead of an AdLayout one. `AdRendererBasePlugin.fetchDisplayAdProperties()` will then give you all the details about how to fetch the template content with `BasePlugin.fetchDataFile()`.

- `AdRendererTemplatePlugin.instanceContextBuilder()` method is no longer taking a `template?: string` parameter. The template compilation should now be done in the Plugin Impl. and no longer in the SDK. The returned `AdRendererTemplateInstanceContext` interface have been updated and is no longer containing the `template` and `render_template` properties as well as the SDK is no longer managing this part.

- `instanceContextBuilder()` method of `AdRenderer` & `EmailRenderer` classes now take a `forceReload` parameter. This parameter should be set at `true` for `PREVIEW`/`STAGE` context so that Users can see in real time the output of their changes on the instance properties.

- `AdRendererBasePlugin.fetchDisplayAd`/`AdRendererBasePlugin.fetchDisplayAdProperties` are also taking a `forceReload` boolean parameter. When set to `true`, it will ask to the platform to bypass all caches and give the last known values for the creative / its properties. This should only be used for `PREVIEW`/`STAGE` context (e.g. when `forceReload` parameter passed to the `instanceContextBuilder()` is set at `true`).

## Migration from 0.3.x to 0.4.x

- the type `Value` has been removed and replaced by a serie of specialized types. Following this change, `PluginProperty` has been transformed to a discriminated union (see the eponym section at https://www.typescriptlang.org/docs/handbook/advanced-types.html ).

- in `EmailRendererBaseInstanceContext`, `EmailRouterBaseInstanceContext`, `ActivityAnalyzerBaseInstanceContext`, `BidOptimizerBaseInstanceContext`and `AdRendererBaseInstanceContext` the fields `creativeProperties`, `routerProperties`, `activityAnalyzerProperties`, `bidOptimizerProperties` and `displayAdProperties` have been rename `properties` which is now typed as a `PropertiesWrapper`.

- `PropertiesWrapper` is a class with a constructor that takes as parameter an `Array<PluginProperty>`. The `PropertiesWrapper` normalize the array to give an access to these properties by their `technical_name` in O(1).

- `BidOptimizerPluginResponse` has been replaced by `BidDecision`

- `core.ResponseData` and `core.ResponseListOfData` have been respectively renamed `core.DataResponse` and `core.DataListResponse`

- `core.RecommandationsWrapper` have been renamed to `core.RecommendationsWrapper`

- `core.UserActivityEvent` is now a `type`. If you were using it as a Class (ex: by extending it), you should now use `core.GenericUserActivityEvent` instead.

## Migration from 0.2.x to 0.3.x

The 0.3.0 release of the Plugin SDK introduces some breaking changes in the AdRenderer support.

The following points changed:

### AdRendererRecoTemplatePlugin Vs. AdRendererTemplatePlugin

In v0.2.x, the Plugin SDK only provided `AdRendererRecoTemplatePlugin` for building AdRenderer based on a Templating Engine.

This base class was forcing developers to handle recommendations while for some use case, you only need the 'Templating' without having to handle the recommendation part.

In v0.3.0, there are now 2 classes to build an AdRenderer:

1. AdRendererTemplatePlugin: if you want to do an AdRenderer without any recommendations but with a Templating engine, this is what you want
2. AdRendererRecoTemplatePlugin: if you want to use recommendations in your AdRenderer while also doing templating, this is what you need

### getCreative & getCreativeProperties helper

The AdRenderer base class now only have `getDisplayAd(id)` and `getDisplayAdProperties(id)` helper. Those helpers are replacing the previous `getCreative(id)` and `getCreativeProperties(id)` helpers.

`getDisplayAd` returns a `DisplayAd` interface which is a sub-class of Creative that have some additionnal fields, such as `format`.

### Handlebars context

Previously, the Handlebars extension was providing an `HandleBarRootContext` interface (in `extra`) which was being used for all AdRenderers using Handlebars, whether they were using "Recommendations" or simply doing "basic" templating.

In 0.3.0+, there are 3 Handlebars contexts:

- `URLHandlebarsRootContext`: to be used when replacing macros in URLs
- `HandlebarsRootContext`: to be used when replacing macros in 'simple' templates without recommendations (e.g. when building a Plugin on top of AdRendererTemplatePlugin)
- `RecommendationsHandlebarsRootContext`: to be used when replacing macros in a template used with "Rrcommendations (e.g. when building a Plugin on top of AdRendererRecoTemplatePlugin)

The Handlebars context themselves also changed. This is in order to build a set of standard macros in all AdRenderer Plugin available on mediarithmics platform => hence, you now have to propose values to be replaced in all the standard macros.

### Handlebars macros

All macros are now in UPPER CASE. Some macros (request, creative, etc.) have to be changed before using them with an ad renderer based on the 0.3.0+.

### Handlebar engines

Prior to the v0.3.0, there was only one Handlebars engine provided in the extra package.

With the 0.3.0+, there are now 2 Handlebars engine:

- HandlebarsEngine: to be used when building an AdRenderer without recommendations
- RecommendationsHandlebarsEngine: to be used when building a 'recommendation' Ad Renderer

### StatsClient helper

You can add a StatsClient to your plugins, by importing helpers. Declare a new StatsClient in your plugin's constructor.

```js
protected statsClient: helpers.StatsClient;

 constructor(enableThrottling = false) {
    super(enableThrottling);

    this.statsClient = helpers.StatsClient.init({
      logger: this.logger,
    });

  }
```

Global tags with relevant datas such as artifact_id, build_id or version_id will be added automatically. When initiating the StatsD client, 2 options can be passed:

- optional
  - timerInMs (interval to send stats to datadog in ms (default = 10 minutes))
  - logger

```js
this.statsClient = helpers.StatsClient.init({
  environment: process.env.NODE_ENV,
  logger: this.logger,
});
```

Using StatsD, the StatsClient, can aggregate and send your stats to services such as Datadog. Increment your stats be calling addOrUpdateMetrics method.

```js
this.statsClient.addOrUpdateMetrics({
  metrics: {
    processed_users: { type: MetricsType.GAUGE, value: 4, tags: { datamart_id: '4521' } },
    users_with_mobile_id_count: { type: MetricsType.GAUGE, value: 1, tags: { datamart_id: '4521' } },
  },
});
this.statsClient.addOrUpdateMetrics({
  metrics: {
    processed_users: { type: MetricsType.GAUGE, value: 10, tags: { datamart_id: '4521' } },
  },
});
this.statsClient.addOrUpdateMetrics({
  metrics: {
    apiCallsError: { metricName: 'apiCallsError', type: MetricsType.GAUGE, value: 10, tags: { statusCode: '500' } },
  },
});
```