Full-text search using MongoDB and Elastic Search
Tags: mongodb, elasticsearch, search
Published: 2024-06-18 11:30:09

To build the best search experience for your application you need Full-text search.

Full-text search enables you to find documents that contain a set of words or phrases within their text.

MongoDB has a $text index to provide a limited full-text search in a single collection, more info here [$text](https://www.mongodb.com/docs/manual/reference/operator/query/text/)
If you are using MongoDB Atlas, you can also use [MongoDB Atlas Search](https://www.mongodb.com/docs/atlas/atlas-search/)

If you are using a self-hosted deployment and want to search in more than one collection, elastic search is the best option for you

## MongoDB + ElasticSearch Architecture

![Architecture](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/4q5c0oej78h8zeciig66.png)

We are going to use MongoDB Change Streams (CDC - Change Data Capture) to sync MongoDB data to Elastic Search.

You can read more about it here [The right decisions can simplify and make it easier to scale](https://dev.to/woovi/the-right-decisions-can-simplify-and-make-it-easier-to-scale-1mg9).

This is the simplest implementation to sync a MongoDB collection using change stream to elastic search.

```tsx
import { Client } from '@elastic/elasticsearch';

const esClient = new Client({ node: process.env.ELASTICSEARCH });

const transactionStreamHandler = async (data: ChangeStreamDocument<ITransaction>) => {
  // map mongodb transaction to elasticsearch data
  const esPayload = {
    identifier: data.fullDocument.identifier,
    customer: {
      name: data.fullDocument.customer.name,
      email: data.fullDocument.customer.email,
      taxID: data.fullDocument.taxID?.taxID,
    },
  }

  const document = {
    index: 'woovi-transaction',
    id: data.fullDocument._id,
    document: esPayload,
  };

  await esClient.index(document);
}

const stream = Transaction.watch([], {
    fullDocument: 'updateLookup',
  });

stream.on('change', transactionStreamHandler);
```

`Transaction.watch` will watch for any changes to the Transaction collection.

`transactionStreamHandler` will map the Transaction collection data to the elastic search payload that will be used in the full-text search.
You don't need to save everything on the elastic search, only the fields that will be used on the search.
We are saving Transaction on the `woovi-transaction` elastic search index, we are using `woovi-*` prefix to enable us to search in many collections at once.

# Retrieving data from Elastic Search

To search, we call the elastic search client `search` method with our query.

```ts
const search = 'Sibelius'

const searchQuery = {
    index: 'woovi-*',
    size: 200,  
    query: {
      bool: { 
        must: [
          { query_string: { query: `${search}*` } },
        ],
      }
    }
  }

const search = await esClient.search(searchQuery);
```

In the above query, we search every `woovi-*` index that matches the query `Sibelius`, any Transaction, Charge, User, Customer.
You could search only transaction passing, `woovi-transaction`, or passing a list of indexes to search like `woovi-transaction,woovi-customer`.


## To conclude
Syncing to a database sounds like a hard problem, but MongoDB change streams greatly simplify this task.
This approach is also lightweight.
Try to add full-text search in your application to provide the best search experience for your users.

---
[Woovi](https://www.woovi.com) is an innovative startup revolutionizing the payment landscape. With Woovi, shoppers can enjoy the freedom to pay however they prefer. Our cutting-edge platform provides instant payment solutions, empowering merchants to accept orders and enhance their customer experience seamlessly.

If you're interested in joining our team, we're hiring! Check out our job openings at [Woovi Careers](https://woovi.com/jobs/).

---
Photo by <a href="https://unsplash.com/es/@amayli?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Amélie Mourichon</a> on <a href="https://unsplash.com/s/photos/design-system?orientation=landscape&utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Unsplash</a>