Elasticsearch to Meilisearch migration guide
Tags: javascript, search, elasticsearch
Published: 2023-04-07 10:06:14

# Elasticsearch migration guide

**Contents**:

1. Introduction
2. Differences
3. Migration script
4. Front-end components 
5. Conclusion

*If you are currently using Elasticsearch and plan to migrate to Meilisearch for your application, follow this guide to help you with the transition.*

## Differences

Before migrating, you may want to understand better what differentiates Elasticsearch from Meilisearch.

Elasticsearch is a distributed search engine that uses clusters and nodes to optimize performance, whereas Meilisearch is a single instance/node with customizable startup options. Meilisearch is still evolving, and many functionalities are not yet supported. You can think of Meilsearch as similar to the Elasticsearch component in the ELK Stack, but without the rest of the components like Kibana, Logstash and Beats.

You may see what new upgrades our developers are working on by visiting the [public roadmap](https://roadmap.meilisearch.com/tabs/4-in-progress).


## Migration Script

This guide will show you how to migrate data from Elasticsearch to Meilisearch using NodeJs and it has no impact on the programming language that will be used with Meilisearch in the future.

### **Initialize project**

Start by creating a directory `elastic-meilisearch-migration` and initialize it as an npm project:

```bash
mkdir elastic-meilisearch-migration
cd elastic-meilisearch-migration
npm init -y
```

Next, create a `script.js` file:

```bash
touch script.js
```

This file will contain our migration code.

### **Install required packages**

To get started, you'll need two npm packages. The first is `@elastic/elasticsearch`, the JavaScript client for the ElasticSearch API, and the second is `meilisearch`, the JavaScript client for the Meilisearch API. To install them, run the below command:

```bash
npm install --save @elastic/elasticsearch meilisearch
```

### ****Create client objects****

We need client instances of both Elasticsearch and Meilisearch to access their respective API. Paste the below code in `script.js`:

```jsx
const { Client: ElasticSearch } = require('@elastic/elasticsearch')
const { MeiliSearch } = require('meilisearch')

const esClient = new ElasticSearch({
  cloud: {
    id: 'ES_CLOUD_ID',
  },
  auth: {
    username: 'ES_USER_NAME',
    password: 'ES_PASSWORD',
  },
})

const meiliClient = new MeiliSearch({
  host: 'MEILI_HOST',
  apiKey: 'MEILI_API_KEY',
})
```

First, we create an Elasticsearch client `esClient` using [elastic cloud](https://www.elastic.co/cloud/) credentials. You can also try other [authentication methods](https://www.elastic.co/guide/en/elasticsearch/client/javascript-api/current/client-connecting.html#client-connecting). Replace the original values for `ES_CLOUD_ID`,`ES_USER_NAME` and `ES_PASSWORD`. 

Then, we create a Meilisearch client `meiliClient` by providing the host url and API key. Also, replace `MEILI_HOST` and `MEILI_API_KEY` with their respective values.

### ****Fetch data from Elasticsearch****

This could be a challenging step because we will iterate the index until the entire index is not searched. In each iteration, we search for the subsequent 10,000 documents, format them, and store them. The iteration will end when we get an empty result. To achieve such a paginated search result, we will use the [Search After](https://www.elastic.co/guide/en/elasticsearch/reference/current/paginate-search-results.html#search-after) method.

This logic has been reduced to the below code.

```jsx
async function getElasticSearchDocuments() {
  let isFetchingComplete = false
  let search_after = null
  let documents = []

  const searchObject = {
    index: 'ES_INDEX',
    size: 10000,
    body: {
      query: {
        match_all: {},
      },
    },
    sort: {
      'id.keyword': 'asc',
    },
  }

  do {
    if (search_after) searchObject.search_after = search_after

    const result = await esClient.search(searchObject)
    const hits = result.hits.hits

    isFetchingComplete = hits.length == 0
    search_after = hits[hits.length - 1]?.sort || null
    documents = documents.concat(hits.map(hit => hit._source))
  } while (isFetchingComplete == false)

  return documents
}
```

Replace `ES_INDEX` with the index name you want to export.

### Upload data to Meilisearch

Add the following code:

```jsx
  const documents = await getElasticSearchDocuments()
  const meiliIndex = meiliClient.index('MEILI_INDEX')
  await meiliIndex.addDocuments(documents)
```

The variable `documents` contain an array of documents ready to be uploaded to Meilisearch. 

After that, we get the index instance and upload all the documents with the `addDocuments` method. Meilisearch will create the index if it doesn't already exist. 

> Note: The `addDocuments` method is [asynchronous](https://docs.meilisearch.com/learn/advanced/asynchronous_operations.html#asynchronous-operations), which means the request is not handled as soon as it is received, but it is put into a queue and processed.
> 

Replace  `MEILI_INDEX` with the index name where you would like documents to be added. 

That's all! When you're ready to run the script, enter the below command:

```bash
node script.js
```

### Finished Script

Here's the complete script code.

```jsx
const { Client: ElasticSearch } = require('@elastic/elasticsearch')
const { MeiliSearch } = require('meilisearch')

const esClient = new ElasticSearch({
  cloud: {
    id: 'ES_CLOUD_ID',
  },
  auth: {
    username: 'ES_USER_NAME',
    password: 'ES_PASSWORD',
  },
})

const meiliClient = new MeiliSearch({
  host: 'MEILI_HOST',
  apiKey: 'MEILI_API_KEY',
})

async function getElasticSearchDocuments() {
  let isFetchingComplete = false
  let search_after = null
  let documents = []

  const searchObject = {
    index: 'ES_INDEX',
    size: 10000,
    body: {
      query: {
        match_all: {},
      },
    },
    sort: {
      'id.keyword': 'asc',
    },
  }

  do {
    if (search_after) searchObject.search_after = search_after

    const result = await esClient.search(searchObject)
    const hits = result.hits.hits

    isFetchingComplete = hits.length == 0
    search_after = hits[hits.length - 1]?.sort || null
    documents = documents.concat(hits.map(hit => hit._source))
  } while (isFetchingComplete == false)

  return documents
}

;(async () => {
  const documents = await getElasticSearchDocuments()
  const meiliIndex = meiliClient.index('MEILI_INDEX')
  await meiliIndex.addDocuments(documents)
})()
```

## Front-end components

[Instant Meilisearch](https://github.com/meilisearch/instant-meilisearch) is a plugin connecting your Meilisearch instance with InstantSearch, giving you access to many (but not all) of the same front-end components as Algolia users. Here is [an up-to-date list of components compatible with Instant Meilisearch](https://github.com/meilisearch/instant-meilisearch/#-api-resources).

## Conclusion

In summary, we have discussed the primary advantages and limitations of both search engines, but our discussion has only touched the tip of the iceberg. The functionalities of Meilisearch and Elasticsearch surpass what has been covered in this article. To obtain a comprehensive comparison between Meilisearch, Elasticsearch, and other search engines, please consult the dedicated section in our documentation.