open-index/open-npm huggingface dataset:
npm package: @ovotech/datadog-metrics-tracker
Description: Track metrics and store them in DataDog
Author: Manikant Sharma
Latest version: 1.3.22

# Datadog Metrics Tracker

Track metrics and store them in Datadog, with secondary logging if Datadog is unavailable or invalid input is provided.

## Using

```bash
yarn add @ovotech/datadog-metrics-tracker
```

```typescript
import { createDataDogConnection, MetricsTracker } from '@ovotech/datadog-metrics-tracker';
import { Logger } from '@ovotech/winston-logger';
import * as winston from 'winston';

// Define a specific tracker
class PerformanceMetricsTracker extends MetricsTracker {
  private static queryTimeMeasurementName = 'query-time';

  async trackQueryTime(timeMs: number, queryName: string) {
    await this.trackPoint(
      PerformanceMetricsTracker.queryTimeMeasurementName,
      { queryName },
      { timeMs: Math.round(timeMs) },
    );
  }
}

// Create Logger and Datadog instances
const winstonLogger = winston.createLogger(...);
const logger = new Logger(winstonLogger, { traceToken: req.headers['X-Trace-Token'] });
const datadog = createDataDogConnection(process.env);

// Create the tracker
const metricsMeta = {
  extraTagName: 'some-value',
};
const tracker = new PerformanceMetricsTracker(datadog, logger, metricsMeta);

// Track a point
await tracker.trackQueryTime(12.34, 'myFirstQuery')
```

As well as a base class that allows you to define custom trackers, there are pre-defined trackers for common operations.
These allow services to use a common interface and not re-implement the same functionality.

- `ExternalRequestMetricsTracker` - track information about calling other services
- `KafkaMetricsTracker` - track actions around the lifecycle of Kafka events
- `ResponseMetricsTracker` - track information about responses from an API

## Beware

Don't add too many unique / varying possibilities, e.g. monetary amounts, to the `tags` - they will set off DataDog Monitor alerts within the business, because too many "custom metrics" escalate DataDog costs.
Multi-variant values, e.g. monetary amounts, should be in the `value(s)`.

## Running the tests

Then you can run the tests with:

```bash
yarn test
```

### Coding style (linting, etc) tests

Style is maintained with prettier and tslint

```
yarn lint
```

## Deployment

Deployment is preferment by lerna automatically on merge / push to master, but you'll need to bump the package version numbers yourself. Only updated packages with newer versions will be pushed to the npm registry.

## Contributing

Have a bug? File an issue with a simple example that reproduces this so we can take a look & confirm.

Want to make a change? Submit a PR, explain why it's useful, and make sure you've updated the docs (this file) and the tests (see [test folder](test)).

## License

This project is licensed under Apache 2 - see the [LICENSE](LICENSE) file for details
npm package: @ovotech/influx-metrics-tracker
Description: Track metrics and store them in an Influx database, with secondary logging if Influx is unavailable
Author: Hayden Field
Latest version: 1.3.7

# Influx Metrics Tracker

Track metrics and store them in an Influx database, with secondary logging if Influx is unavailable or invalid input is provided.

## Using

```bash
yarn add @ovotech/influx-metrics-tracker
```

```typescript
import { createInfluxConnection, MetricsTracker } from '@ovotech/influx-metrics-tracker';
import { Logger } from '@ovotech/winston-logger';
import { InfluxDB, ISingleHostConfig } from 'influx';
import * as winston from 'winston';

// Define a specific tracker
class PerformanceMetricsTracker extends MetricsTracker {
  private static queryTimeMeasurementName = 'query-time';

  async trackQueryTime(timeMs: number, queryName: string) {
    await this.trackPoint(
      PerformanceMetricsTracker.queryTimeMeasurementName,
      { queryName },
      { timeMs: Math.round(timeMs) },
    );
  }
}

// Create Logger and Influx instances
const winstonLogger = winston.createLogger(...);
const logger = new Logger(winstonLogger, { traceToken: req.headers['X-Trace-Token'] });
const influx = createInfluxConnection(process.env);

// Create the tracker
const metricsMeta = {
  extraTagName: 'some-value',
};
const tracker = new PerformanceMetricsTracker(influx, logger, metricsMeta);

// Track a point
await tracker.trackQueryTime(12.34, 'myFirstQuery')
```

As well as a base class that allows you to define custom trackers, there are pre-defined trackers for common operations.
These allow services to use a common interface and not re-implement the same functionality.

- `ExternalRequestMetricsTracker` - track information about calling other services
- `KafkaMetricsTracker` - track actions around the lifecycle of Kafka events
- `ResponseMetricsTracker` - track information about responses from an API

## Running the tests

Then you can run the tests with:

```bash
yarn test
```

### Coding style (linting, etc) tests

Style is maintained with prettier and tslint

```
yarn lint
```

## Deployment

Deployment is preferment by lerna automatically on merge / push to master, but you'll need to bump the package version numbers yourself. Only updated packages with newer versions will be pushed to the npm registry.

## Contributing

Have a bug? File an issue with a simple example that reproduces this so we can take a look & confirm.

Want to make a change? Submit a PR, explain why it's useful, and make sure you've updated the docs (this file) and the tests (see [test folder](test)).

## License

This project is licensed under Apache 2 - see the [LICENSE](LICENSE) file for details
npm package: @ovotech/json-refs
Description: Json Refs Resolver
Author: Ivan Kerin
Latest version: 0.3.2

# JSON Refs

Recursively downloads files referenced in a json-schema, and resolves all the references and provide a map to the referenced contents while its at it.

### Usage

```shell
yarn add @ovotech/json-refs
```

```typescript
import { resolveRefs } from '@ovotech/json-refs';

// Where the contents of https://example.com/assets/test.yml are
// UserResponse:
//   - type: 'object'

const schema = {
  test: { $ref: 'https://example.com/assets/test.yml#/UserResponse' },
};

const resolved = await resolveRefs(schema);

console.log(resolved);
```

Should output

```js
{ schema:
   { test:
      { '$ref': 'https://example.com/assets/test.yaml#/UserResponse' } },
  refs:
   { 'https://example.com/assets/test.yaml': { UserResponse: { type: 'object' } },
     'https://example.com/assets/test.yaml#/UserResponse': { type: 'object' } },
  uris: [ 'https://example.com/assets/test.yaml' ] }
```

The resolved refs include all files with their parsed contents (yaml or json), as well as flat list of links to the individual refs.
There is also a list of all the files downloaded or local file referenced in `uris`.

### Behavior

First of all it would parse the schema, searching for `$refs` if the refs are in the file itself then it would just return those. If there are any external links, it would collect all the URLs present, and would download all of them in parallel. If there are any external links in the downloaded files, it would attempt another batch, until all resources are resolved.

Since it would fill in its internal reference library on each pass, circular references are allowed and would only be resolved once.

### Api

**resolveRefs**

```typescript
function async resolveRefs(schema: JsonPointerObject) => Promise<{
  schema: JsonPointerObject,
  refs: { [key: string]: JsonPointerObject,
  uris: string[]
}>
```

**resolveRefsFile** The same as resolveRefs, but start with downloading and or parsing the initial schema. Can take any url or path to a local file.

```typescript
function async resolveRefsFile(file: string) => Promise<{
  schema: JsonPointerObject,
  refs: { [key: string]: JsonPointerObject,
  uris: string[]
}>
```

## Running the tests

You can run the tests with:

```bash
yarn test
```

### Coding style (linting, etc) tests

Style is maintained with prettier and eslint

```
yarn lint
```

## Deployment

Deployment is preferment by lerna automatically on merge / push to master, but you'll need to bump the package version numbers yourself. Only updated packages with newer versions will be pushed to the npm registry.

## Contributing

Have a bug? File an issue with a simple example that reproduces this so we can take a look & confirm.

Want to make a change? Submit a PR, explain why it's useful, and make sure you've updated the docs (this file) and the tests (see [test folder](test)).

## License

This project is licensed under Apache 2 - see the [LICENSE](LICENSE) file for details
npm package: @ovotech/kafka-consumer
Description: A generic kafka consumer
Author: Ivan Kerin
Latest version: 0.0.1

# Kafka Consumer

A generic kafka consumer.

### Using

```bash
yarn add @ovotech/kafka-consumer
```

```typescript
const onDetectNotification = async (message: AvroMessage) => {
  //handle notification
};

createKafkaConsumer(
  {
    OPTIONS: {},
    KAFKA_SCHEMA_REGISTRY: env.KAFKA_SCHEMA_REGISTRY!,
    KAFKA_TOPICS: ['topic_v1', 'topic_v2'],
  },
  logger,
  onDetectNotification,
);
```

## Running the tests

Then you can run the tests with:

```bash
yarn test
```

### Coding style (linting, etc) tests

Style is maintained with prettier and tslint

```
yarn lint
```

## Deployment

Deployment is preferment by lerna automatically on merge / push to master, but you'll need to bump the package version numbers yourself. Only updated packages with newer versions will be pushed to the npm registry.

## Contributing

Have a bug? File an issue with a simple example that reproduces this so we can take a look & confirm.

Want to make a change? Submit a PR, explain why it's useful, and make sure you've updated the docs (this file) and the tests (see [test folder](test)).

## License

This project is licensed under Apache 2 - see the [LICENSE](LICENSE) file for details
npm package: @ovotech/pg-sql-migrate-cli
Description: run migration scripts incrementally using postgres in the cli
Author: Ivan Kerin
Latest version: 1.0.3
Deprecated: Package no longer supported, use @ovotech/pg-sql-migrate:^2.0.0. It includes a cli

# Postgres migration tool with plain sql

A cli tool to simplify using [@ovotech/pg-sql-migrate](../pg-sql-migrate).

## Using with CLI

```bash
yarn add @ovotech/pg-sql-migrate-sql
yarn pg-migrate create my_migration
```

add a configuration file, which by default is `./pg-sql-migrate.config.json` to configure the connection:

```json
{
  "client": "postgresql://postgres:dev-pass@0.0.0.0:5432/postgres"
}
```

This will create a file `migrations/<timestamp>_my_migration.pgsql` that you can place raw sql into. After that, you can run the migration(s) by calling

```bash
yarn pg-migrate execute
```

## Running the tests

You can run the tests with:

```bash
yarn test
```

### Coding style (linting, etc) tests

Style is maintained with prettier and tslint

```
yarn lint
```

## Deployment

Deployment is preferment by lerna automatically on merge / push to master, but you'll need to bump the package version numbers yourself. Only updated packages with newer versions will be pushed to the npm registry.

## Contributing

Have a bug? File an issue with a simple example that reproduces this so we can take a look & confirm.

Want to make a change? Submit a PR, explain why it's useful, and make sure you've updated the docs (this file) and the tests (see [test folder](test)).

## License

This project is licensed under Apache 2 - see the [LICENSE](LICENSE) file for details
npm package: @ovotech/re-pipeline
Description: A node pipeline implementation with resume on error
Author: Ivan Kerin
Latest version: 1.0.1

# Re Pipeline

A node streams pipeline implementation, that reconnects the pipes on error, once the error has been handled.

### Using

```bash
yarn add @ovotech/re-pipeline
```

Using `stream-mock` to mock out the readable and writable streams. Whenever an error is encountered by any of the streams, it would call the error handler and then reconnect the pipes again.

```typescript
import { rePipeline } from '@ovotech/re-pipeline';
import { ReadableMock, WritableMock } from 'stream-mock';
import { Transform } from 'stream';

const start = new ReadableMock(['one', 'two'], { objectMode: true });
const end = new WritableMock({ objectMode: true });
const upperCase = new Transform({
  objectMode: true,
  transform: (item, encoding, callback) => callback(undefined, String(item).toUpperCase()),
});

const errorHandler = error => console.log(error);

const pipeline = rePipeline(errorHandler, start, upperCase, end);

pipeline.on('finish', () => console.log(end.data));
```

You can use the promise version. When the stream has finished it would resolve the promise, and if any error is encountered, it would reject the promise. This would not reconnect the pipes.

```typescript
import { rePipelinePromise } from '@ovotech/re-pipeline';
import { ReadableMock, WritableMock } from 'stream-mock';
import { Transform } from 'stream';

const start = new ReadableMock(['one', 'two'], { objectMode: true });
const end = new WritableMock({ objectMode: true });
const upperCase = new Transform({
  objectMode: true,
  transform: (item, encoding, callback) => callback(undefined, String(item).toUpperCase()),
});

const main = async () => {
  await pipelinePromise(start, upperCase, end);
  console.log(end.data);
};
```

## Running the tests

Then you can run the tests with:

```bash
yarn test
```

### Coding style (linting, etc) tests

Style is maintained with prettier and tslint

```
yarn lint
```

## Deployment

Deployment is preferment by lerna automatically on merge / push to master, but you'll need to bump the package version numbers yourself. Only updated packages with newer versions will be pushed to the npm registry.

## Contributing

Have a bug? File an issue with a simple example that reproduces this so we can take a look & confirm.

Want to make a change? Submit a PR, explain why it's useful, and make sure you've updated the docs (this file) and the tests (see [test folder](test)).

## License

This project is licensed under Apache 2 - see the [LICENSE](LICENSE) file for details
npm package: @ovotech/ssm-env-secrets
Description: Load secrets from ssm
Author: Ivan Kerin
Latest version: 1.0.1

# SSM ENV Secrets

[![CircleCI](https://circleci.com/gh/ovotech/ssm-env-secrets.svg?style=svg&circle-token=d9be710126b12e0271cb77a09d08a7e2129ba112)](https://circleci.com/gh/ovotech/ssm-env-secrets)
[![npm (scoped)](https://img.shields.io/npm/v/@ovotech/ssm-env-secrets.svg)](https://www.npmjs.com/package/@ovotech/ssm-env-secrets)

This function is loads data from AWS Systems Manager (SSM) Parameter Store. This way you can securely store secrets and retrieve them at runtime in production, but rely on dummy values in dev/testing.

This module ships with TypeScript types.

### Using

```bash
yarn add @ovotech/ssm-env-secrets
```

If you have a parameter in SSM Parameter store called `my-secret-key` with your secret value. And your env vars look like this:

```
NODE_ENV=something
API_ID=my-id
API_KEY=ssm://my-secret-key
```

Then you can load the secret values with:

```ts
import { loadSSMSecrets } from '@ovotech/ssm-env-secrets';
import { SSM } from 'aws-sdk';

const ssm = new SSM({ region: 'eu-west-1' });
const env = loadSSMSecrets(ssm, process.env);

console.log(env.API_KEY); // will hold the decrypted value of "my-secret-key" parameter
```

## Running the tests

The tests require a running ssm mock server, and we're using [localstack](https://github.com/localstack/localstack) for that.
You'll need to start the ssm server:

```bash
SERVICES=ssm localstack start
```

After which you can run all the tests:

```bash
yarn test
```

### Coding style (linting, etc) tests

Style is maintained with prettier and tslint

```
yarn lint
```

## Deployment

To deploy a new version, push to master and then create a new release. CircleCI will automatically build and deploy a the version to the npm registry.

## Contributing

Have a bug? File an issue with a simple example that reproduces this so we can take a look & confirm.

Want to make a change? Submit a PR, explain why it's useful, and make sure you've updated the docs (this file) and the tests (see `test/S3DataSource.spec.ts`). You can run the tests with `SERVICES=ssm localstack start` and `yarn test`.

## Responsible Team

- Boost Internal Tools (BIT)

## License

This project is licensed under Apache 2 - see the [LICENSE](LICENSE) file for details