How to Create a Full Autocomplete Search Application with Elasticsearch, Kibana and NestJS - Part 3
Tags: javascript, react, nestjs, elasticsearch
Published: 2022-09-10 20:27:34

Hi guys, welcome to back to the Part 3 of the Elastic, Kibana and NestJS series. In the part 1 of this series, we installed and configured elasticsearch ([check it here if you missed it](https://dev.to/airscholar/how-to-create-a-full-autocomplete-search-application-with-elasticsearch-and-nestjs-part-1-27k4)), in the part 2, we connected elasticsearch with Kibana and ran a few queries ([check here for part 2](https://dev.to/airscholar/how-to-create-a-full-autocomplete-search-application-with-elasticsearch-and-nestjs-part-2-a6b)).

In this article, we will be writing the NodeJS code that will connect and query elasticsearch.

### Loading data into elasticsearch
To enable us write our code effectively, we need data loaded into our `elasticsearch`. We will be using a sample dataset from Kaggle ([Download it here](https://drive.google.com/file/d/1ZtcGb59EkLN0k2q4cpYS3XwYXwQdt4t_/view?usp=sharing).

Follow the FOUR steps below to load it up into elasticsearch
1. Open up Kibana (http://localhost:5601)
![kibana upload](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/p22oohs1zgm21ttsoi7v.png)
2. Under `Get started by adding integrations` click on Upload a file
![upload file](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/xpj9dghft6deblu9d8di.png)
3. Click on import and enter the name of the index you want to put the data in.
![index name](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/fiup1jwqjq6rz6mc72sw.png)
4. Click on import (final page)
![import](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/3ibd2lbquk5dbohtavg5.png)

If you made it to this point, you have successfully imported data into elasticsearch.

### Querying for sample
Goto DevTools (Hamburger on the top left corner of the screen > Scroll down to Management > DevTools)

Run the query below (select it and click on the play button)
```
GET tmdb_movies/_search
```
If you see this, we are good to go!

![querying elasticsearch](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/f816hkc0q1uufuq2dr5d.png)

---
Now, Let's dive into coding, shall we 😊?

## NestJs

> NestJS is a progressive Node. js framework that helps build server-side applications. Nest extends Node. js frameworks like Express or Fastify adding modular organization and a wide range of other libraries to take care of repetitive tasks. It's open-source, uses TypeScript, and is a very versatile Node.

### Creating a NestJs Application
Run the command below to install nestcli and create a new NestJs Application (in the article, the name of the app will be nest-elastic).

```
$ npm i -g @nestjs/cli
$ nest new nest-elastic
```

You will be asked to select a package manager, you can select npm, yarn or pnpm. I will be selecting yarn (you can choose any other one you want 😉). Your project will be setup and we should be ready to code!

![project setup](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/drlzyevtugrac4kuonyf.png)

### Adding elasticsearch to your app
Run the command below to add elasticsearch to your `nest-elastic` and other dependencies.

```bash
yarn add @elastic/elasticsearch @nestjs/elasticsearch @nestjs/config
```

In your root folder add your `.env` file with the following contents
```
ELASTICSEARCH_NODE=https://localhost:9200
ELASTICSEARCH_USERNAME=elastic
ELASTICSEARCH_PASSWORD=elasticPasswordGoesHere
ELASTICSEARCH_MAX_RETRIES=10
ELASTICSEARCH_REQ_TIMEOUT=50000
ELASTICSEARCH_INDEX=tmdb_movies
```

Let's create a separate module that handles only search using elasticsearch. A simple shortcut is to use the command below (you are welcome to do it manually, if you want too)

```
nest g resource search
```

![added search module](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/hmngiiuklyjb6jwfpcfo.png)

Update the `search.module.ts` to have the content below
```
import { Module } from '@nestjs/common';
import { SearchService } from './search.service';
import { SearchController } from './search.controller';
import { ConfigModule, ConfigService } from '@nestjs/config';
import { ElasticsearchModule } from '@nestjs/elasticsearch';

@Module({
  imports: [
    ConfigModule,
    ElasticsearchModule.registerAsync({
      imports: [ConfigModule],
      useFactory: async (configService: ConfigService) => ({
        node: configService.get('ELASTICSEARCH_NODE'),
        auth: {
          username: configService.get('ELASTICSEARCH_USERNAME'),
          password: configService.get('ELASTICSEARCH_PASSWORD'),
        },
        maxRetries: configService.get('ELASTICSEARCH_MAX_RETRIES'),
        requestTimeout: configService.get('ELASTICSEARCH_REQ_TIMEOUT'),
      }),
      inject: [ConfigService],
    }),
  ],
  controllers: [SearchController],
  providers: [SearchService],
  exports: [SearchService],
})
export class SearchModule {}
```

Update `search.service.ts` with the content below:
```
import { ConfigService } from '@nestjs/config';
import { Injectable } from '@nestjs/common';
import { ElasticsearchService } from '@nestjs/elasticsearch';

type dataResponse = {
  UnitPrice: number;
  Description: string;
  Quantity: number;
  Country: string;
  InvoiceNo: string;
  InvoiceDate: Date;
  CustomerID: number;
  StockCode: string;
};

@Injectable()
export class SearchService {
  constructor(
    private readonly esService: ElasticsearchService,
    private readonly configService: ConfigService,
  ) {}

  async search(search: {key: string}) {
    let results = new Set();
    const response = await this.esService.search({
      index: this.configService.get('ELASTICSEARCH_INDEX'),
      body: {
        size: 50,
        query: {
          match_phrase: search
        },
      },
    });
    const hits = response.hits.hits;
    hits.map((item) => {
      results.add(item._source as dataResponse);
    });

    return { results: Array.from(results), total: response.hits.total };
  }
}
```
---
Now, let's add movies modules
```
nest g resource movies
```

Update `movies.controller.ts` with the content below:
```
import { SearchService } from './../search/search.service';
import { Body, Controller, Post } from '@nestjs/common';

@Controller('movies')
export class MoviesController {
  constructor(private readonly searchService: SearchService) {}

  @Post('search')
  async search(@Body() body) {
    return await this.searchService.search(body.data);
  }
}
```

Then `movies.module.ts`
```
import { SearchModule } from './../search/search.module';
import { Module } from '@nestjs/common';
import { MoviesService } from './movies.service';
import { MoviesController } from './movies.controller';

@Module({
  imports: [SearchModule],
  controllers: [MoviesController],
  providers: [MoviesService],
})
export class MoviesModule {}

```

Finally, update `app.module.ts` 
```
import { MoviesModule } from './movies/movies.module';
import { ConfigModule } from '@nestjs/config';
import { Module } from '@nestjs/common';
import { AppController } from './app.controller';
import { AppService } from './app.service';
import { SearchModule } from './search/search.module';

@Module({
  imports: [MoviesModule, ConfigModule.forRoot(), SearchModule],
  controllers: [AppController],
  providers: [AppService],
})
export class AppModule {}
```

Your `package.json` should look like this
```
{
  "name": "nest-elastic",
  "version": "0.0.1",
  "description": "",
  "author": "Yusuf Ganiyu",
  "private": true,
  "license": "UNLICENSED",
  "scripts": {
    "prebuild": "rimraf dist",
    "build": "nest build",
    "format": "prettier --write \"src/**/*.ts\" \"test/**/*.ts\"",
    "start": "nest start",
    "start:dev": "nest start --watch",
    "start:debug": "nest start --debug --watch",
    "start:prod": "node dist/main",
    "lint": "eslint \"{src,apps,libs,test}/**/*.ts\" --fix",
    "test": "jest",
    "test:watch": "jest --watch",
    "test:cov": "jest --coverage",
    "test:debug": "node --inspect-brk -r tsconfig-paths/register -r ts-node/register node_modules/.bin/jest --runInBand",
    "test:e2e": "jest --config ./test/jest-e2e.json"
  },
  "dependencies": {
    "@elastic/elasticsearch": "^8.4.0",
    "@nestjs/common": "^9.0.0",
    "@nestjs/config": "^2.2.0",
    "@nestjs/core": "^9.0.0",
    "@nestjs/elasticsearch": "^9.0.0",
    "@nestjs/mapped-types": "*",
    "@nestjs/platform-express": "^9.0.0",
    "reflect-metadata": "^0.1.13",
    "rimraf": "^3.0.2",
    "rxjs": "^7.2.0"
  },
  "devDependencies": {
    "@nestjs/cli": "^9.0.0",
    "@nestjs/schematics": "^9.0.0",
    "@nestjs/testing": "^9.0.0",
    "@types/express": "^4.17.13",
    "@types/jest": "28.1.4",
    "@types/node": "^16.0.0",
    "@types/supertest": "^2.0.11",
    "@typescript-eslint/eslint-plugin": "^5.0.0",
    "@typescript-eslint/parser": "^5.0.0",
    "eslint": "^8.0.1",
    "eslint-config-prettier": "^8.3.0",
    "eslint-plugin-prettier": "^4.0.0",
    "jest": "28.1.2",
    "prettier": "^2.3.2",
    "source-map-support": "^0.5.20",
    "supertest": "^6.1.3",
    "ts-jest": "28.0.5",
    "ts-loader": "^9.2.3",
    "ts-node": "^10.0.0",
    "tsconfig-paths": "4.0.0",
    "typescript": "^4.3.5"
  },
  "jest": {
    "moduleFileExtensions": [
      "js",
      "json",
      "ts"
    ],
    "rootDir": "src",
    "testRegex": ".*\\.spec\\.ts$",
    "transform": {
      "^.+\\.(t|j)s$": "ts-jest"
    },
    "collectCoverageFrom": [
      "**/*.(t|j)s"
    ],
    "coverageDirectory": "../coverage",
    "testEnvironment": "node"
  }
}
```

### Running the app
You can fire up the app in dev environment using `yarn start:dev`

![running terminal](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/dn43lg536kmx80sjrzzh.png)

### Testing
![postman -1](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/oxc9oqjbnv1a2w6n1d5u.png)

![postman -2](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/56j43kxz1gg52vchioti.png)

You can access the full [source code here](https://github.com/airscholar/nest-elastic-backend)

## Summary
In this article, we are able to import data into elastic using kibana and also connect our NestJs backend app to the leverage the power of elasticsearch.

In the next article, we will be building a simple frontend to query and visualize results from elasticsearch in realtime.

Thanks for reading!