open-index/open-npm huggingface dataset:
npm package: @mediaurl/cassandra-cache
Description: MediaURL Cassandra cache wrapper
Latest version: 2.3.1

# MediaURL Cassandra caching engine

[Cassandra](https://cassandra.apache.org/) caching engine for MeduaURL addons.

You can find the caching documentation [here](https://github.com/mediaurl/mediaurl-js/blob/main/docs/caching.md).

## Usage with your addon

```bash
# Build your addon
npm run build

# Install the caching engine
npm i @mediaurl/cassandra-cache

# Setup environment
export LOAD_MEDIAURL_CACHE_MODULE=@mediaurl/cassandra-cache
export CASSANDRA_CONFIG=address-of-your-server.com

# Start the addon server
npm start
```

## Testing

```bash
# Start a local cassandra database server
docker run --rm -ti -p 9042:9042 cassandra

# Run the tests
CASSANDRA_CONFIG=localhost npm run test
```
npm package: @mediaurl/i18n
Description: MediaURL tools for translations
Latest version: 2.3.1

# MediaURL i18n module

Helper module to quickly set up translations for addons.

By default the [i18next-node-fs-backend](https://github.com/i18next/i18next-node-fs-backend) backend for `i18next` is used.

If you set the environment variables `LOCIZE_PROJECTID` and `LOCIZE_API_KEY`, the [i18next-node-locize-backend](https://github.com/locize/i18next-node-locize-backend) backend will be used.

## Installation

```shell
npm i --save @mediaurl/i18n
```

## Configuration

Run the `init` function somewhere at the beginning of your script. There are some default options set (see `src/index.ts`) which can be overwritten.

## Usage of the `t` function

Sometimes you may need direct access to the `t` function of i18next. This is how it's done with this module:

```javascript
import { changeLanguage } from "@mediaurl/i18n";

i18n.init({
  ["en", "fr", "tr"],
  { debug: true }
});

const someFunction = async () => {
  const t = await changeLanguage("en");
  console.log(t("some.key", "Some translated text"));
};
```

## Usage for addons

Mainly translations are needed for the addon metadata defined at the createAddon funcitons. There are two recommended ways to translate this data.

### 1. With the `addon` action handler

```javascript
import { createAddon, translateDeep } from "@mediaurl/sdk";
import * as i18n from "@mediaurl/i18n";

i18n.init({
  ["en", "fr", "tr"],
  { debug: true }
});

export const myAddon = createAddon({
  id: "i18n-example",
  name: "i18n:Name of this addon"
});

myAddon.registerActionHandler("addon", async (input, ctx, addon) => {
  const t = await changeLanguage(input.language);
  return translateDeep(addon.getProps(), t);
});
```

### 2. With an exported promise

The addon server supports resolving exported promises. This promises can return an addon or an array of addons.

Also we can benefit from the `TranslatedText` system. The `tAll` function will get all available translations for a key.

```javascript
import { createAddon, translateDeep } from "@mediaurl/sdk";
import { tAll } from "@mediaurl/i18n";

export default (async () => {
  await i18n.init({
    ["en", "fr", "tr"],
    { debug: true }
  });

  const myAddon = createAddon(translateDeep(
    {
      id: "i18n-example",
      name: "i18n:Name of this addon"
    },
    tAll
  ));

  return myAddon;
});
```

## Download translations from locize

If you choose to use locize, you should add your translations to your prjecct root, especially when you are using a serverless environment.

First install the `locize-cli`:

```shell
npm i --save-dev locize-cli
```

Then modify your `package.json`:

```json
{
  "scripts": {
    "update-locize": "node -r dotenv/config node_modules/.bin/locize download --path locales --clean=true -n tmdb"
  }
}
```
npm package: @mediaurl/puppeteer
Description: Puppeteer helper module
Latest version: 2.3.1

# MediaURL puppeteer support

This module gives easy access to puppeteer to help scraping websites.

It has a special router integrated which allows fine grained control of how and if resources are losed.

## Setup

The recommended way to setup puppeteer is with a few [puppeteer-extra](https://github.com/berstend/puppeteer-extra) plugins enabled.

```shell
npm i --save @mediaurl/puppeteer puppeteer-core puppeteer-extra puppeteer-extra-plugin-anonymize-ua puppeteer-extra-plugin-stealth
# To install chromium
npm i --save puppetter
```

Inside your addon, add this code. In this example, two `puppeteer-extra` plugins are used.

```typescript
import { setupPageRules } from "@mediaurl/puppeteer";
import puppeteer from "puppeteer-extra";
import AnonymizeUserAgentPlugin from "puppeteer-extra-plugin-anonymize-ua";
import StealthPlugin from "puppeteer-extra-plugin-stealth";

puppeteer.use(StealthPlugin({}));
puppeteer.use(AnonymizeUserAgentPlugin());
```

## Usage

There are some utility functions which will make the usage of puppeteer a little more easy.

```javascript
addon.registerActionHandler("item", async (input, ctx) => {
  const ruleOptions = {
    ctx,
    rules: [
      { url: [input.url, "example.com/api"], action: "allow" },
      { url: "example.com/js", action: "allow", cache: true },
    ],
    blockPopups: true,
  };

  // Get a browser instance
  const browser = await puppeteer.launch();
  try {
    const page = (await browser.pages())[0];

    // Setup the page rules
    setupPageRules(page, ruleOptions);

    // Open the website and return it's content
    await page.open(input.url);
    return await page.content();
  } finally {
    // Close the browser
    await browser.close();
  }
});
```

### Callbacks inside page rules

To catch one specific URL and return it from an action handler, the following recipe might help you:

```javascript
addon.registerActionHandler("resolve", async (input, ctx) => {
  // outerPromise is a helper to handle this kind of situations.
  // See the documentation of this function for more infos.
  const p = outerPromise(5000);

  const pageRules = [
    { url: [input.url, "example.com/api"], action: "allow" },
    { url: "example.com/js", action: "allow", cache: true },
    {
      resourceType: "media",
      url: "example.com/mediapath/",
      action: async (request) => {
        // This action handler will be called during page load
        const url = await request.url();
        p.resolve(url);
      },
    },
  ];

  // Get a browser instance
  const browser = await puppeteer.launch();
  try {
    const page = (await browser.pages())[0];
    setupPageRules(page, ruleOptions);

    // When calling open, the action function will be triggered
    await page.open(input.url);

    // In case the page was loaded without calling the action
    // function, reject the promise
    p.promise.reject(new Error("Action handler was not called"));
  } finally {
    await browser.close();
  }

  // Wait for the promise
  return await p.promise;
});
```
npm package: @mediaurl/sql-cache
Description: MediaURL SQL cache wrapper
Latest version: 2.3.1

# MediaURL MongoDB caching engine

You can find the caching documentation [here](https://github.com/mediaurl/mediaurl-js/blob/main/docs/caching.md).

## Testing

```bash
# Start postgres and mysql
docker run -d --name sql-cache-test-postgres -p 5432:5432 -e POSTGRES_HOST_AUTH_METHOD=trust postgres:alpine
docker run -d --name sql-cache-test-mysql -p 3306:3306 -e MYSQL_ROOT_PASSWORD=pass mysql:5

# Test
TEST_URL_1=postgres://postgres@localhost/postgres TEST_URL_2=mysql://root:pass@localhost/mysql npm run test

# Cleanup
docker rm -f sql-cache-test-{postgres,mysql}
```