How to Easily use GRPC and Protocol Buffers with NodeJS
Tags: beginners, javascript, node, grpc
Published: 2020-03-28 14:35:51

---
title: "How to Easily use GRPC and Protocol Buffers with NodeJS"
description: "How to Easily use GRPC and Protocol Buffers with NodeJS"
published : true
tags : beginner, javascript, nodejs, grpc
cover_image: https://dev-to-uploads.s3.amazonaws.com/i/kbckeo70rre4urj87zq4.jpg
canonical_url: https://adityasridhar.com/posts/how-to-easily-use-grpc-and-protocol-buffers-with-nodejs
series: GRPC Series
---

**This post was originally published in [adityasridhar.com](https://adityasridhar.com/posts/how-to-easily-use-grpc-and-protocol-buffers-with-nodejs)**


This Article will give a basic introduction to GRPC and Protocol Buffers. Following that I will be showing how to use GRPC and Protocol Buffers in a NodeJS Application

# What is GRPC

GRPC is a Open Source High Performance RPC Framework

So what exactly does a RPC do?

Take the following Example 

```javascript
function getEmployee() {
  return "ABCD";
}
function greetEmployee()
{
    let employee = getEmployee();
    console.log("Greet",employee) 
} 
```

Here we have a function **getEmployee** which returns an Employee Name and another function **greetEmployee** which Calls *getEmployee* and gets the employee's Name and then prints a Greeting.

Here **greetEmployee** calling **getEmployee** is a regular function call. 

Now if getEmployee and greetEmployee functions are located in different address spaces, or they are located in 2 different hosts which are separated by the network, then the function call is called a Remote Procedure Call. Here the System which has the **getEmployee** function acts like a Server and the System which has the **greetEmployee** function acts like a Client. 

## What is a Protocol Buffer

Protocol Buffer is the Interface Definition Language which is used by default in GRPC.

* It helps to define the various services offered by a server.
* It helps to define the Structure of the Payloads used in the System
* It helps to serialize the message ( To a Special Binary Format ) and send it across the wire between the server and client.

We will see how to use protocol buffers when we are working on the NodeJS application later in this article.

## What are the Different Types of RPC Supported

### Unary RPC

This is the Simplest RPC available. Here the Client sends a Request Message to the Server. The Server processes the request and then sends a Response message back to the Client. 

**In this Article this is the grpc we will be focussing on.**

### Server Streaming RPC

In this RPC, the client sends a request message to the server, and the server sends a sequence of messages back to the Client in a stream fashion.

### Client Streaming RPC

In this RPC, the client sends a sequence of messages to the server in a stream fashion. The server then processes all these requests and then sends a response message back to the client.

### Bidirectional Streaming RPC

In this RPC, the client sends  a sequence of messages to the server in a stream fashion. The server then processes the request and then sends a sequence of messages back to the client in stream fashion.

# How to use GRPC and Protocol Buffers in NodeJS

Create a folder called as **grpc-nodejs-demo** and Initialize nodejs in it using the following commands

```bash
mkdir grpc-nodejs-demo
cd grpc-nodejs-demo
npm init
```

This will create a **package.json** file. To know more about NodeJS you can read my other article [here](https://adityasridhar.com/posts/how-to-use-nodejs-without-frameworks-and-external-libraries)

## Modify the package.json file

Replace the **package.json** file with the following

```javascript
{
  "name": "grpc-nodejs-demo",
  "version": "1.0.0",
  "description": "",
  "main": "server.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "dependencies": {
    "@grpc/proto-loader": "^0.1.0",
    "grpc": "^1.11.0",
    "lodash": "^4.6.1"
  },
  "author": "Name",
  "license": "MIT"
}
```

Here we are adding 3 dependencies
* **@grpc/proto_loader** and **grpc** dependencies will help us use GRPC and Protocol Buffers in the Application.
* **lodash** is a general utility dependency. It will help simplify some code logic

Once the package.json file has been updated. Run the following command to install the dependencies

```bash
npm install
```

## Defining the Protocol Buffer

In this Example we will build a service which will take employee ID as the input, and give the employee details as the output.

The Service Interface and payloads needed will be specified in a protocol buffer file. Protocol buffer files have an extension of **.proto**

Now let's create the **.proto** file.

Create a folder called as proto within the project. Inside the proto folder, create a file with the name **employee.proto** and add the following code to it

```java
syntax = "proto3";

package employee;

service Employee {

  rpc getDetails (EmployeeRequest) returns (EmployeeResponse) {}
}


message EmployeeRequest {
  int32 id = 1;
}

message EmployeeResponse{
  EmployeeDetails message = 1;
}
message EmployeeDetails {
  int32 id = 1;
  string email = 2;
  string firstName = 3; 
  string lastName = 4;
}
```
So what exactly have we done here?.

`syntax = "proto3";` indicates that we want to use Protocol Buffer version 3. 

`package employee;` indicates that we are creating a package called as employee within which we will define our services

```javascript
service Employee {

  rpc getDetails (EmployeeRequest) returns (EmployeeResponse) {}
}
```

The Above script tells that We are creating a Service called as **Employee**. Within this service we are creating a function ( rpc ) called as **getDetails** which accepts input of type **EmployeeRequest** and provides response in the format **EmployeeResponse**

Next we need to define **EmployeeRequest** and **EmployeeResponse**. This is done in the following script

```java
message EmployeeRequest {
  int32 id = 1;
}

message EmployeeResponse{
  EmployeeDetails message = 1;
}
message EmployeeDetails {
  int32 id = 1;
  string email = 2;
  string firstName = 3; 
  string lastName = 4;
}
```

Here we see that the message **EmployeeRequest** has a single field of type **int32** and name **id**. The number **1** assigned here is a **field number** and it helps during encoding and decoding of the message. Every field defined should have a **unique field number**

We also see that **EmployeeResponse** has a custom field of type **EmployeeDetails** and name **message** having field number of **1**. This means that even **EmployeeDetails** has to be defined which is shown above as well.

**EmployeeDetails** has 4 fields comprising types **int32** and **string**. All of them have **unique field numbers**

*Field numbers between 1 -15 use 1 byte of space during encoding. and field numbers from 2 - 2047 uses 2 bytes for encoding and hence will take up more space. So try to design in such a way that the field numbers are between 1 - 15 as much as possible*

## Creating the GRPC Server

Create a file called as **server.js**

First Let us include all the libraries we need and also define the location where the .proto file is present

```javascript
const PROTO_PATH = __dirname + '/proto/employee.proto';

const grpc = require('grpc');
const protoLoader = require('@grpc/proto-loader');
const _ = require('lodash');
```

Next we need to load the **.proto** file. This is done using the **protoLoader** libary **loadSync** method.

```javascript
let packageDefinition = protoLoader.loadSync(
    PROTO_PATH,
    {keepCase: true,
     longs: String,
     enums: String,
     defaults: true,
     oneofs: true
    });
```

Next from the loaded proto file package Definition we need to get the package we need. This is done using the following script

```javascript
let employee_proto = grpc.loadPackageDefinition(packageDefinition).employee;
```

here we are getting the **employee**  package into **employee_proto** variable.

**employee_proto** will now have all the proto definitions.

Next we need to create some dummy employees data for the server to work with. Create a file called as **data.js** and add the following script into it

```javascript
let employees = [{
    id: 1,
    email: "abcd@abcd.com",
    firstName: "First1",
    lastName: "Last1"   
},
{
    id: 2,
    email: "xyz@xyz.com",
    firstName: "First2",
    lastName: "Last2"   
},
{
    id: 3,
    email: "temp@temp.com",
    firstName: "First3",
    lastName: "Last3"   
},
];

exports.employees = employees;
```

Next we need to import data.js into server.js. Add the following script in **server.js** for this

```javascript
let {employees} = require('./data.js');
```

So **employees** will have the list of employees with their **id, email, firstName and lastName**

The next piece of script creates and starts the GRPC Server.

```javascript
function main() {
  let server = new grpc.Server();
  server.addService(employee_proto.Employee.service, {getDetails: getDetails});
  server.bind('0.0.0.0:4500', grpc.ServerCredentials.createInsecure());
  server.start();
}
```

`let server = new grpc.Server();` is the script which creates a new **GRPC Server**

In the **.proto file** we notice that we have a function called as **getDetails** inside **Employee** Service. 

`server.addService(employee_proto.Employee.service, {getDetails: getDetails});` is the script in which we add the Service implementation. This script says that we are adding the **getDetails** function in **employee_proto.Employee** Service. And Then We are adding this Service to the Server.

`server.bind('0.0.0.0:4500', grpc.ServerCredentials.createInsecure());` is the script which tells that the server will start in port 4500 and have no Authentication

`server.start();` is the script which actually starts the server.

The main thing pending now is to implement **getDetails** function. The below script shows the implementation

```javascript
function getDetails(call, callback) {
  callback(null, 
    {
       message: _.find(employees, { id: call.request.id })
    });
}
```

here **call** has the request parameters and **callback** is where we need to define the implementation.

Inside callback we have `message: _.find(employees, { id: call.request.id })` which says the following 
* Get the employee ID from Input - call.request.id
* search the **employees** list to find the Employee who has that Id
* Return that Employee Details

This completes the Server implementation. Here is the complete script for **server.js**

```javascript
const PROTO_PATH = __dirname + '/proto/employee.proto';

const grpc = require('grpc');
const protoLoader = require('@grpc/proto-loader');
const _ = require('lodash');

let packageDefinition = protoLoader.loadSync(
    PROTO_PATH,
    {keepCase: true,
     longs: String,
     enums: String,
     defaults: true,
     oneofs: true
    });
let employee_proto = grpc.loadPackageDefinition(packageDefinition).employee;

let {employees} = require('./data.js');

function getDetails(call, callback) {
  callback(null, 
    {
       message: _.find(employees, { id: call.request.id })
    });
}

function main() {
  let server = new grpc.Server();
  server.addService(employee_proto.Employee.service, {getDetails: getDetails});
  server.bind('0.0.0.0:4500', grpc.ServerCredentials.createInsecure());
  server.start();
}

main();
```

## Creating the GRPC Client

Create a file called as **client.js**

Copy the following script into **client.js**

```javascript
const PROTO_PATH = __dirname + '/proto/employee.proto';

const grpc = require('grpc');
const protoLoader = require('@grpc/proto-loader');

let packageDefinition = protoLoader.loadSync(
    PROTO_PATH,
    {keepCase: true,
     longs: String,
     enums: String,
     defaults: true,
     oneofs: true
    });
let employee_proto = grpc.loadPackageDefinition(packageDefinition).employee;
```
The Above script loads the **employee** package into **employee_proto** variable in the same way we saw in **server.js**

Next we need a way in which we can call the RPC. In this case we need to be able to call the **getDetails** function which is implemented in the Server.

For this we need to create a **stub** in the client. This is done using below script.

```javascript
let client = new employee_proto.Employee('localhost:4500',
                                       grpc.credentials.createInsecure());
```

This **client Stub** will help us call the **getDetails** function which is defined in **Employee** Service which runs on the Server. The Server in turn runs on port **4500**. The Line of code also indicates that there is no Authentication used

finally we can call the **getDetails** function using the following script

```javascript
let employeeId = 1;
 client.getDetails({id: employeeId}, function(err, response) {
    console.log('Employee Details for Employee Id:',employeeId,'\n' ,response.message);
  });
```

As mentioned earlier the **client stub** helps us call the getDetails function in the Server like a normal function call. To this we pass the **employeeId** as the input. 

Finally the Response comes in the **response** variable. We are then printing the response message.

The complete **client.js** code is given below

```javascript
const PROTO_PATH = __dirname + '/proto/employee.proto';

const grpc = require('grpc');
const protoLoader = require('@grpc/proto-loader');

let packageDefinition = protoLoader.loadSync(
    PROTO_PATH,
    {keepCase: true,
     longs: String,
     enums: String,
     defaults: true,
     oneofs: true
    });
let employee_proto = grpc.loadPackageDefinition(packageDefinition).employee;

function main() {
  let client = new employee_proto.Employee('localhost:4500',
                                       grpc.credentials.createInsecure());
  let employeeId;
  if (process.argv.length >= 3) {
    employeeId = process.argv[2];
  } else {
    employeeId = 1;
  }
  client.getDetails({id: employeeId}, function(err, response) {
    console.log('Employee Details for Employee Id:',employeeId,'\n' ,response.message);
  });
}

main();
```

## Running The Server and Client

### Running the Server

Open a command prompt and run the Server using the following command 

```bash
node server.js
```

This will start the server

Open a new Command prompt and run the Client using the following command

```bash
node client.js
```

When we run the client. It will print the following output

```bash
Employee Details for Employee Id: 1 
 { id: 1,
  email: 'abcd@abcd.com',
  firstName: 'First1',
  lastName: 'Last1' }
```

So here the **client** has called the **getDetails** function in the server. The Client has passed the input for **employeeId** as 1. Then the server went over the data, found the employee with id as **1** and returned that Employee data back to the client.

In This example we have run the client and server in a single machine. But you can also test this out by having the server and client in different hosts.


# Code 

The Code discussed in this article can be found [here](https://github.com/aditya-sridhar/grpc-nodejs-demo)

# References

GRPC official Documentation : [https://grpc.io/](https://grpc.io/)

Protocol Buffers Proto3 documentation : [https://developers.google.com/protocol-buffers/docs/proto3](https://developers.google.com/protocol-buffers/docs/proto3)

# Congrats 😄

You now know what is GRPC And Protocol Buffers. You also know how to use them in a NodeJS Application

Happy Coding 😄

### Feel free to connect with me in [LinkedIn] (https://www.linkedin.com/in/aditya1811/) or follow me in [twitter](https://twitter.com/adityasridhar18).

### If you liked this post, you can checkout my website [https://adityasridhar.com](https://adityasridhar.com) for other similar posts