How to mock Elasticsearch with Jest?
Tags: elasticsearch, jest, testing, node
Published: 2022-12-02 20:37:10

Originally published [here](https://medium.com/codex/test-elasticsearch-with-jest-like-a-pro-42386713b899)

## Intro

I guess you are probably using Elasticsearch or planning to since you've opened this article 🙂 Another assumption - you definitely found it a great idea to lock your functionality with some integration tests. And it really is!

The current company I’m working for has 90+% of code coverage with both unit and integration tests! I would recommend everyone cover their code base with tests since as one wise man said:
>  [Without unit tests] You’re not refactoring, you’re just changing shit.
>  — Hamlet D’Arcy

## Setup

Imagine you have a simple RESTful server with some logic that uses Elasticsearch. In the current showcase — CRUD server.
```js
const Hapi = require('@hapi/hapi');
const Qs = require('qs');
const {createHandler} = require("./create/index.js");
const {readAllHandler, readHandler} = require("./read/index.js");
const {updateHandler} = require("./update/index.js");
const {deleteAllHandler, deleteHandler} = require("./delete/index.js");

const init = async () => {

    const server = Hapi.server({
    port: 3000,
    host: 'localhost',
    query: {
        parser: (query) => Qs.parse(query)
    }
    });

    server.route({
    method: 'POST',
    path: '/',
    handler: createHandler
    });

    server.route({
    method: 'GET',
    path: '/{id}',
    handler: readHandler
    });

    server.route({
    method: 'GET',
    path: '/',
    handler: readAllHandler
    });

    server.route({
    method: 'PATCH',
    path: '/{id}',
    handler: updateHandler
    });

    server.route({
    method: 'DELETE',
    path: '/{id}',
    handler: deleteHandler
    });

    server.route({
    method: 'DELETE',
    path: '/',
    handler: deleteAllHandler
    });

    await server.start();

    server.events.on('log', (event, tags) => {
    console.log({event}, {tags})
    if (tags.error) {
        console.log(`Server error: ${event.error ? event.error.message : 'unknown'}`);
    }
    });
    console.log('Server running on %s', server.info.uri);
};

process.on('unhandledRejection', (err) => {

    console.log(err);
    process.exit(1);
});

init();
```
Now you need to cover the logic of each route with some tests to lock functionality and prevent business logic from being broken.

One obvious but not simple solution is to use Docker and spin up Elastic for tests every time.

However, does it worth it? I mean you really want to have a longer spin-up time for the pipeline environment? Maybe there are already-built solutions for that?

This plugin downloads and caches Elasticsearch binary when jest starts, then the plugin automatically starts Elastic on defined ports and tears it down when tests are done.

## How to add tests?

I have an example of with/without jest tests setup in the [pull request](https://github.com/harazdovskiy/elastic-jest-example/pull/2/files) so you can compare all the changes. But now let’s go through it step by step.

### 1. Install additional modules

    yarn add --dev jest @shelf/jest-elasticsearch @types/jest

### 2. Add jest-config.js
```bash
touch jest.config.js
```
```js
module.exports = {
    preset: '@shelf/jest-elasticsearch',
    clearMocks: true,
    collectCoverage: true,
    coverageDirectory: "coverage",
    coverageProvider: "v8"
};
```
Alternatively, you can generate jest config on your own using the [CLI tool](https://jestjs.io/docs/cli#--init).

### 3. Add jest-es-config.js for the plugin

```bash
touch jest-es-config.js
```

```js
const {index} = require('./src/elastic.js');

module.exports = () => {
    return {
    esVersion: '8.4.0',
    clusterName: 'things-cluster',
    nodeName: 'things-node',
    port: 9200,
    indexes: [
        {
        name: index,
        body: {
            settings: {
            number_of_shards: '1',
            number_of_replicas: '1'
            },
            mappings: {
            dynamic: false,
            properties: {
                id: {
                type: 'keyword'
                },
                value: {
                type: 'integer'
                },
                type: {
                type: 'keyword'
                },
                name: {
                type: 'keyword'
                },
            }
            }
        }
        }
    ]
    };
};
```
### 4. Extend package.json script to run tests
```json
{
    "scripts": {
    "test": "jest"
    "serve": "node src/index.js"
    }
}
```
### 5. Tune elastic client
```js
const dotenv = require('dotenv')
dotenv.config()
const {Client} = require('@elastic/elasticsearch');

module.exports.client = new Client({
    node: process.env.NODE_ENV === 'test' ? 'http://localhost:9200' : process.env.ES_URL
})

module.exports.index = 'things'
```
Add a condition that will use NODE_ENV to connect to local spined-up elastic whenever we are running tests.

## Profit!

Now all the things are ready to write and run tests. All routes are fully covered and stored here:
[elastic-jest-example](https://github.com/harazdovskiy/elastic-jest-example)

As an example let’s cover create function business logic.
```js
const {ulid} = require('ulid');
const {client, index} = require("../elastic.js");

module.exports.createHandler = async (request, h) => {
    if (Object.keys(request.payload))
    try {
        const res = await this.create(request.payload)
        return h.response(res).code(200);
    } catch (e) {
        console.log({e})
        return h.response({e}).code(400);
    }
}

// let's cover this function with some tests
module.exports.create = async (entity) => {
    const {
    type,
    value,
    name,
    } = entity;

    const document = {
    id: ulid(),
    type: type.trim().toLowerCase(),
    value: +value.toFixed(0),
    name: name.trim()
    }

    await client.index({
    index,
    document
    });
    return document.id
}
```
Create a test file and add a couple of statements.
```bash
touch  src/create/index.test.js
```

```js
const {create} = require("./index.js");
const {client, index} = require("../elastic");

describe('#create', () => {

// clear elastic every time before running it the statement.
// It's really important since each test would be idempotent.
    beforeEach(async () => {
    await client.deleteByQuery({
        index,
        query: {
        match_all: {}
        }
    })
    await client.indices.refresh({index})
    })

    it('should insert data', async () => {
    expect.assertions(3);
    const res = await create({type: 'some', value: 100, name: 'jacket'})
    await client.indices.refresh();
    const data = await client.search({
        index,
        query: {
        match: {
            "id": res
        }
        }
    })

    expect(res).toEqual(expect.any(String))
    expect(res).toHaveLength(26);
    expect(data.hits.hits[0]._source).toEqual({
        "id": res,
        "name": "jacket",
        "type": "some",
        "value": 100
        }
    );
    })

    it('should insert and process the inserted fields', async () => {
    const res = await create({type: 'UPPERCASE', value: 25.99, name: ' spaces '})
    await client.indices.refresh();
    const data = await client.search({
        index,
        query: {
        match: {
            "id": res
        }
        }
    })
    expect(data.hits.hits[0]._source).toEqual({
        "id": res,
        "name": "spaces",
        "type": "uppercase",
        "value": 26
        }
    );
    })
});
```
A basic testing flow for each business logic function can be simply described like this:

insert data-> run tested function -> check outputs -> clear data -> repeat

Data insertion/deletion can be improved by unifying them into helpers and using additional mooching libs.

The elastic teardown is managed by [@shelf/jest-elasticsearch](https://github.com/shelfio/jest-elasticsearch) lib itself.

One more good convention to follow is to cover each function you are testing with describeblock, so that later you can easily run a specific test with IDE helper without reruning whole suite:

![Running jest tests using Webstorm](https://cdn-images-1.medium.com/max/6344/1*Eek-RJc9sQkOdQIyUxDKvA.png)

## Resources

Now you know how to test your Elasticsearch queries using jest.
[jest-elasticsearch](https://github.com/shelfio/jest-elasticsearch)

Also, you have a small blueprint repo with ready to use setup!
[elastic-jest-example](https://github.com/harazdovskiy/elastic-jest-example)

Hope this article will help you set up and test your elastic project!

## Want to connect?

Follow me on [Twitter](https://twitter.com/harazdovskiy)!

Read more:
[How to update 63 million records in MongoDB 50% faster?](https://medium.com/geekculture/how-to-update-63-million-records-in-mongodb-50-faster-a6dbf23ec9b3)

[Optimizing massive MongoDB inserts, load 50 million records faster by 33%!](https://medium.com/codex/50-million-records-insert-in-mongodb-using-node-js-5c62b7d7af5a)