Part 4: Securely connect Node.js server to Elastic Cloud
Tags: node, express, elasticsearch, beginners
Published: 2022-09-07 15:14:33

[Table of Content](https://dev.to/lisahjung/beginners-guide-to-building-a-full-stack-app-nodejs-react-with-elasticsearch-5347) | [Read Next: Part 5: Plan for efficient data storage and search performance in Elasticsearch](https://dev.to/lisahjung/part-5-plan-for-efficient-data-storage-and-search-performance-in-elasticsearch-22mo)

### Resources

*Would you rather watch a video to learn this content? Click on the link below!*
- [Episode 4: Securely connect Node.js server to Elastic Cloud](https://www.youtube.com/watch?v=ezrx6rPrScI)

*The following are the project directories shared in the blog. Click on the link below to access it!*
- [Code for connecting to Elastic Cloud with basic authentication](https://github.com/LisaHJung/beginners-guide-to-creating-a-full-stack-Javascript-app-with-Elasticsearch/tree/2-connect-server-to-Elastic-Cloud-via-basic-authentication)
- [Code for connecting to Elastic Cloud with an API Key](https://github.com/LisaHJung/beginners-guide-to-creating-a-full-stack-Javascript-app-with-Elasticsearch/tree/3-connect-server-to-Elastic-Cloud-via-apiKey)

### Connect Node.js Server to Elastic Cloud 

In [part 3](https://dev.to/lisahjung/part-3-create-an-elastic-cloud-deployment-36bn), we signed up for a free trial of Elastic Cloud and created a deployment. 

This deployment consists of Elasticsearch cluster, Kibana, Fleet, three Elastic Solutions(Enterprise Search, Observability, Security) and more!

Connecting to this deployment will give us access to everything mentioned above, including our Elasticsearch cluster. 

We can connect our server to Elastic Cloud deployment via two different authentication mechanisms:
1. basic authentication
2. API key

*API key* is the best practice for production because it gives users flexibility in creating different API keys for different systems or applications. 

In order to generate an API key, we need to log into Elastic Cloud as a user first(*basic authentication*).  

Let's get started!

**Step 1: Pull up the `earthquake_app` directory with your code editor**

In [part 2](https://dev.to/lisahjung/part-2-build-a-server-using-nodejs-with-express-52a0), we created a project directory called the `earthquake_app`and we built a Node.js server within it. 

Next, we will create a file where we will store Elastic Cloud access credentials.

**Step 2: Set up the `config` directory**

In [part 2](https://dev.to/lisahjung/part-2-build-a-server-using-nodejs-with-express-52a0), we have installed a dependency called `config`. 

The `config` dependency allows you to store node configuration in an external configuration file. It can be customized based on different environments(local/dev/qa/prod/etc). 

In our case, we will be using `config` to store access credentials such as Elastic cloud ID, username, password, and API key.

*Within the `earthquake_app` directory, create a `config` directory. Within the `config` directory, create a `default.json` file as shown below.* 

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/xxmr25yeg9ijtj2frqso.png)

**Step 3: Enter the Elastic Cloud access credentials in the `default.json` file**

*Copy and paste the following code into the `default.json` file.*

```javascript
//in config/default.json
{
    "elastic": {
      "cloudID": "DEPLOYMENT_NAME:CLOUD_ID_DETAILS",
      "username": "elastic",
      "password": "LONGPASSWORD"
    }
}
```
Your `default.json` file should look like the following screenshot. 

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/nrp772jqs1og0wv73wpt.png)
  
We will replace "DEPLOYMENT_NAME:CLOUD_ID_DETAILS"(line 3) and "LONGPASSWORD"(line 5) with our Elastic Cloud deployment access credentials.

In [part 3](https://dev.to/lisahjung/part-3-create-an-elastic-cloud-deployment-36bn), we have downloaded and saved the access credentials.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/uyp63saecvkty8a4dbse.png)

*Find and open the saved credentials.* 

You should see something like this.  

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/rv469l9r7n7d7a79uwa7.png)

*Copy your password and paste it into line 5 "password" as shown below.*

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/cthv6prau0zricrnnv07.png)
 
Next, let's find our "DEPLOYMENT NAME:CLOUD_ID_DETAILS". 

In [part 3](https://dev.to/lisahjung/part-3-create-an-elastic-cloud-deployment-36bn), we learned how to get to the Kibana home page. 

*From the Kibana home page, click on the `menu` icon(red box).* 

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/j3psl019gehcmqrc5onb.png)
  
It will display a drop down menu as shown below. 

*From the drop down menu, click on the `Manage this deployment` option(pink box).*

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/eggampd7cxzbimd60zcy.png)
 
You will see the following on your screen.
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/nnmidpz66sulaohi5lyw.png)
   
Look at the region highlighted with a red box. This is where your deployment name and cloud id information are displayed.

*Click on the copy button(blue box) then paste the info into line 3 of the `default.json` file.* 

Your `default.json` file should look like the following screenshot with your credentials pasted into line 3 and line 5.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/no08ty7jkqmf2yafoomv.png)
 
**Step 4: Connect to Elastic Cloud using basic authentication**

*Within the `server` directory, create a `elasticsearch` directory. Within the `elasticsearch` directory, create a `client.js` file as shown below.* 

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/woutd5yoii5ca0k0u9ar.png)

At this point, you might be wondering how we will get our server to communicate with Elastic Cloud so we can establish a secure connection? 

![giphy](https://media.giphy.com/media/4JVTF9zR9BicshFAb7/giphy.gif)

It's actually quite simple!

[Elasticsearch Client](https://www.elastic.co/guide/en/elasticsearch/client/index.html) is the simplest and easiest way to communicate with your Elasticsearch cluster from your app, in a programming language of your choice. 

`Elasticsearch Client` allows you to easily access every endpoint of Elasticsearch API to perform a variety of tasks.

Since we are building a server using Node.js with Express, we will be using the [Elasticsearch Javascript Client](https://www.elastic.co/guide/en/elasticsearch/client/javascript-api/current/index.html) to connect to our Elasticsearch cluster! 

In `client.js`, we will create an instance of `Elasticsearch Client`. `Elasticsearch Client` will contain our Elastic Cloud access credentials that point to our Elasticsearch cluster. 

Then, we will set up our server to connect to Elastic Cloud and display a message in the terminal of its connection status. 

*In the `client.js` file, copy and paste the following code.*

```javascript
//in server/elasticsearch/client.js
const { Client } = require('@elastic/elasticsearch');
const config = require('config');

const elasticConfig = config.get('elastic');

const client = new Client({
  cloud: {
    id: elasticConfig.cloudID,
  },
  auth: {
    username: elasticConfig.username,
    password: elasticConfig.password
  },
});

client.ping()
  .then(response => console.log("You are connected to Elasticsearch!"))
  .catch(error => console.error("Elasticsearch is not connected."))

module.exports = client;  
```   
Let's go over the code!

**Heads up!**
For reference purposes only, I have included screenshots of code that I will be explaining.

If you need to copy and paste the code, please refer to the code above or the GitHub repo for [connecting to Elastic Cloud with basic authentication](https://github.com/LisaHJung/beginners-guide-to-creating-a-full-stack-Javascript-app-with-Elasticsearch/tree/2-connect-server-to-Elastic-Cloud-via-basic-authentication).
 
*Line 1*
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/gwdmay49qy2p3clw23l8.png)

We require `@elastic/elasticsearch` library we installed in [part 2](https://dev.to/lisahjung/part-2-build-a-server-using-nodejs-with-express-52a0) and set this equal to a constant `Client`.

This library is the official Node.js client for Elasticsearch and it comes with a lot of functionalities. You will see this coming into play in line 6 when we create a new instance of `Elasticsearch Client`. 

*Line 2*
We create a constant called config and require the `config` dependency we installed in [part 2](https://dev.to/lisahjung/part-2-build-a-server-using-nodejs-with-express-52a0). You will see this come into play in line 4 when you use this dependency to get access credentials stored in the `config` directory. 

*Line 4* 
The access credentials were stored under `'elastic'` in `config/default.json` as shown below.
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/n62got7r1ajyqsx1fqdd.png)

We use `config` to `get` credentials stored under `'elastic'` and set it equal to a constant `elasticConfig`. 
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/gwdmay49qy2p3clw23l8.png)

This allows us to securely store and access sensitive data and not store these in the source code. 
  
*Lines 6-14*
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/rfn651n7qboh7spc9o16.png)
We create a new instance of `Elasticsearch Client` with our access credentials(cloudID, username, and password) that points to our Elasticsearch cluster. We set this equal to a constant called `client`.
 
*Lines 16-18*
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/fv3s61d9vpmnwn0v870t.png)

We use the `ping()` method on our `Elasticsearch Client` to check if the Elasticsearch cluster is up and available to process requests.

We have it set up so that if we receive a response(meaning we are connected to Elasticsearch), our terminal will display the message "You are connected to Elasticsearch!".

If an error occurs during this process, our terminal should display the message "Elasticsearch is not connected". 

*Line 20*
We expose our `Elasticsearch Client` via Node.js module exports as this will be used in `server.js`. 

*Go to the `server.js` file within the `server` directory. In line 2, add the following snippet.* 

```javascript
//in server/server.js
const client = require('./elasticsearch/client');
```
Your `server.js` should look like the following screenshot.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/hnkh9kq4inyq6sqac72z.png)
 
In line 2, we require `client.js` from the `elasticsearch` directory. We created an instance of `Elasticsearch Client` in this file. 

We do this as we will need to access our `Elasticsearch Client` from `server.js`.

Let's test if we are able to connect to Elastic Cloud as a user(basic authentication)!

*Start your server by running the following command in the terminal.* 
```javascript
//in terminal earthquake_app directory
npm start
```
You will see that the server is listening on port 3001 and that you are connected to Elasticsearch!

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/c28mzs6jogmf87cfmfrj.png)
 
**Step 5: Generate an API key**
Now that we have logged into Elastic Cloud as a user, we can generate an *API key*.

From this point on, the *API key* will be used to connect to Elastic Cloud.
 
*Within the `server` directory, create a file called `create-api-key.js`.*

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/80sa0fzejs2km374rcr1.png)
 
*In `create-api-key.js`, copy and paste the following code.*

```javascript
//in server/create-api-key.js
const client = require('./elasticsearch/client');

async function generateApiKeys(opts) {
  const body = await client.security.createApiKey({
    body: {
      name: 'earthquake_app',
      role_descriptors: {
        earthquakes_example_writer: {
          cluster: ['monitor'],
          index: [
            {
              names: ['earthquakes'],
              privileges: ['create_index', 'write', 'read', 'manage'],
            },
          ],
        },
      },
    },
  });
  return Buffer.from(`${body.id}:${body.api_key}`).toString('base64');
}

generateApiKeys()
  .then(console.log)
  .catch((err) => {
    console.error(err);
    process.exit(1);
  });

```
Your `create-api-key.js` should look like the following.
 
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/0h4y2ydxl0672254yv9k.png)
 
Let's go over the code!

**Heads up!**
For reference purposes only, I have included screenshots of code that I will be explaining.

If you need to copy and paste the code, please refer to the code above or the GitHub repo for [connecting to Elastic Cloud with API key](https://github.com/LisaHJung/beginners-guide-to-creating-a-full-stack-Javascript-app-with-Elasticsearch/tree/3-connect-server-to-Elastic-Cloud-via-apiKey).

*Line 1*
We require the instance of `Elasticsearch Client` we created in the previous step.  

*Lines 3-21*
We define the `generateApiKeys()` function. This function creates an *API key* that gives access to our Elastic Cloud deployment. 

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/0h4y2ydxl0672254yv9k.png)

*Line 4*
We define a constant `body` and set it equal to `client.security.createAPIKey()` method. 

*Lines 5-18*
Within this method, we define what Elasticsearch cluster privileges are granted to the server connecting with the generated *API key*.

*Line 6* 
We specify the name of the server using this *API key*. 

*Line 7*
Under `role_descriptors`, we define a role(access privileges) granted to this server.

*Line 8* 
We name this role as `earthquakes_example_writer`. You could name this role any way you see fit. 

*Line 9*
We specify that this *API key* comes with cluster `monitor` privilege, which gives read-only access for determining cluster state. 

*Lines 12-13*
This key also grants additional privileges such as `create_index`, `write`, `read`, and manage operations for an index named `earthquakes`. 

*Line 20*
The `security.createApiKey` function(*line 4*) returns an `id` and `api_key` value which are concatenated and encoded in `base64`. 

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/0h4y2ydxl0672254yv9k.png)

*Lines 23-28*
We call the `generateApiKeys()` function here which prints the *API key* in the terminal. If an error occurs, it prints the error instead. 

Let's generate an API key by running the `create-api-key.js` file. 

*In the terminal, create a separate tab. `cd` into the earthquake_app directory and run the following command.*

```javascript
//in terminal within the earthquake_app directory
node server/create-api-key.js
```
You will see the following message in the terminal along with your *API key*.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/kh2vj4fj7l1z93o3yl1s.png)
  
**Step 6: Connect to Elastic Cloud via API key**

We will start by storing the *API key* in the `default.json` file of our `config` directory. 

*Go to `default.json` file and add a comma at the end of *line 5*. In *line 6*, copy and paste the following.*

```javascript
//in config/default.json
"apiKey": "paste your generated API key here"
```
*Copy the API key from your terminal and paste it into *line 6* in `default.json`. 

Your `default.json` file should look like this.
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/uss6bk5ui4j5zzt8b3or.png)
  
*Open the `client.js` file in the `elasticsearch` directory.* 

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/w8n14nygxqbxzeoboegh.png)
 
*Replace `username` and `password`(lines 11 and 12) with the following code.*
```javascript
//in elasticsearch/client.js
apiKey: elasticConfig.apiKey
```
Your `client.js` file should now look like the following screenshot(*lines 11*).

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/vt4bv0qhd0atx1uj4srz.png)
 
*Go back to the terminal tab where you are running the server.* 

You will see that the server is connected to Elasticsearch except that this time we are connected to Elastic Cloud deployment via the API key! 
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/nxudjfndppdhr33zdy4y.png)

Now that our server is connected to our Elasticsearch cluster, it is time to think about the data we want to store in Elasticsearch. 

Move on to [Part 5](https://dev.to/lisahjung/part-5-plan-for-efficient-data-storage-and-search-performance-in-elasticsearch-22mo) to plan for efficient data storage and search performance in Elasticsearch!