How to create load balancing servers using Node.js ?

21 Feb, 2022
If your website or application doesn’t receive many requests, you don’t have to use load balancing, but when it becomes very popular and starts to receive a high amount of traffic may your underlying server can not handle it. Because a single NodeJS server is not flexible for handling a very large amount of traffic. 
Adding more machines can solve this issue. But in order to share traffic to your all application server, a load balancer is required.
Load balancer: A load balancer acts as the traffic cop sitting in front of your application servers and routing client requests across all servers capable of fulfilling those requests in a manner that maximizes speed and capacity utilization and ensures that no one server is overworked, which could degrade performance.
How to setup load balancing server?
1. Using Cluster Module: NodeJS has a built-in module called Cluster Module to take the advantage of a multi-core system. Using this module you can launch NodeJS instances to each core of your system. Master process listening on a port to accept client requests and distribute across the worker using some intelligent fashion. So, using this module you can utilize the working ability of your system.
The following example covers the performance difference by using and without using the Cluster Module.
Without Cluster Module: 
Make sure you have installed the express and crypto module using the following command:
npm install express crypto
index.js
Javascript
const { generateKeyPair } = require('crypto');const app = require('express')();  // API endpoint // Send public key as a responseapp.get('/key', (req, res) => {  generateKeyPair('rsa', {    modulusLength: 2048,    publicKeyEncoding: {      type: 'spki',      format: 'pem'    },    privateKeyEncoding: {      type: 'pkcs8',      format: 'pem',      cipher: 'aes-256-cbc',      passphrase: 'top secret'    }  }, (err, publicKey, privateKey) => {      // Handle errors and use the    // generated key pair.    res.send(publicKey);  })})  app.listen(3000, err => {  err ?    console.log("Error in server setup") :    console.log('Server listening on PORT 3000')});
 Run the index.js file using the following command:
node index.js
Output: We will see the following output on the terminal screen:
Server listening on PORT 3000
Now open your browser and go to http://localhost:3000/key, you will see the following output:
—–BEGIN PUBLIC KEY—– MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAwAneYp5HlT93Y3ZlPAHjZAnPFvBskQKKfo4an8jskcgEuG85KnZ7/16kQw2Q8/7Ksdm0sIF7qmAUOu0B773X 1BXQ0liWh+ctHIq/C0e9eM1zOsX6vWwX5Y+WH610cpcb50ltmCeyRmD5Qvf+OE/C BqYrQxVRf4q9+029woF84Lk4tK6OXsdU+Gdqo2FSUzqhwwvYZJJXhW6Gt259m0wD YTZlactvfwhe2EHkHAdN8RdLqiJH9kZV47D6sLS9YG6Ai/HneBIjzTtdXQjqi5vF Y+H+ixZGeShypVHVS119Mi+hnHs7SMzY0GmRleOpna58O1RKPGQg49E7Hr0dz8eh 6QIDAQAB —–END PUBLIC KEY—–
The above code listening on port 3000 and send Public Key as a response. Generating an RSA key is CPU-intensive work. Here only one NodeJS instance working in a single core. To see the performance, we have used autocannon tools to test our server as shown below:
The above image showed that the server can respond to 2000 requests when running 500 concurrent connections for 10 seconds. The average request/second is 190.1 seconds. 
Using Cluster Module:
Javascript
const express = require('express');const cluster = require('cluster');const { generateKeyPair } = require('crypto');  // Check the number of available CPU.const numCPUs = require('os').cpus().length;  const app = express();const PORT = 3000;  // For Master processif (cluster.isMaster) {  console.log(`Master ${process.pid} is running`);    // Fork workers.  for (let i = 0; i < numCPUs; i++) {    cluster.fork();  }    // This event is firs when worker died  cluster.on('exit', (worker, code, signal) => {    console.log(`worker ${worker.process.pid} died`);  });}  // For Workerelse {  // Workers can share any TCP connection  // In this case it is an HTTP server  app.listen(PORT, err => {    err ?      console.log("Error in server setup") :      console.log(`Worker ${process.pid} started`);  });    // API endpoint  // Send public key  app.get('/key', (req, res) => {    generateKeyPair('rsa', {      modulusLength: 2048,      publicKeyEncoding: {        type: 'spki',        format: 'pem'      },      privateKeyEncoding: {        type: 'pkcs8',        format: 'pem',        cipher: 'aes-256-cbc',        passphrase: 'top secret'      }    }, (err, publicKey, privateKey) => {        // Handle errors and use the      // generated key pair.      res.send(publicKey);    })  })}
  
Run the index.js file using the following command: 
node index.js
Output: We will see the following output on terminal screen:
Master 16916 is running
Worker 6504 started
Worker 14824 started
Worker 20868 started
Worker 12312 started
Worker 9968 started
Worker 16544 started
Worker 8676 started
Worker 11064 started
Now open your browser and go to http://localhost:3000/key, you will see the following output:
—–BEGIN PUBLIC KEY—– MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAzxMQp9y9MblP9dXWuQhf sdlEVnrgmCIyP7CAveYEkI6ua5PJFLRStKHTe3O8rxu+h6I2exXn92F/4RE9Yo8EOnrUCSlqy9bl9qY8D7uBMWir0I65xMZu3rM9Yxi+6gP8H4CMDiJhLoIEap+d9Czr OastDPwI+HF+6nmLkHvuq9X5aORvdiOBwMooIoiRpHbgcHovSerJIfQipGs74IiR 107GbpznSUxMIuwV1fgc6mAULuGZl+Daj0SDxfAjk8KiHyXbfHe5stkPNOCWIsbAtCbGN0bCTR8ZJCLdZ4/VGr+eE0NOvOrElXdXLTDVVzO5dKadoEAtzZzzuQId2P/z JwIDAQAB —–END PUBLIC KEY—–
The above NodeJS application is launched on each core of our system. Where master process accepts the request and distributes across all worker. The performed in this case is shown below:
The above image showed that the server can respond to 5000 requests when running 500 concurrent connections for 10 seconds. The average request/second is 162.06 seconds.
So, using the cluster module you can handle more requests. But, sometimes it is not enough, if this is your case then your option is horizontal scaling.
2. Using Nginx: If your system has more than one application server to respond to, and you need to distribute client requests across all servers then you can smartly use Nginx as a proxy server. Nginx sits on the front of your server pool and distributes requests using some intelligent fashion. 
In the following example, we have 4 instances of the same NodeJS application on different ports, also you can use another server.
The file name is index.js
Javascript
const app = require('express')();  // API endpointapp.get('/', (req,res)=>{    res.send("Welcome to GeeksforGeeks !");})  // Launching application on several portsapp.listen(3000);app.listen(3001);app.listen(3002);app.listen(3003);
Now install Nginx on your machine and create a new file in /etc/nginx/conf.d/ called your-domain.com.conf with the following code in it. 
upstream my_http_servers {
    # httpServer1 listens to port 3000
    server 127.0.0.1:3000;

    # httpServer2 listens to port 3001
    server 127.0.0.1:3001;

    # httpServer3 listens to port 3002
    server 127.0.0.1:3002;

    # httpServer4 listens to port 3003
    server 127.0.0.1:3003;
}
server {
    listen 80;
    server_name your-domain.com www.your-domain.com;
    location / {
        proxy_set_header   X-Real-IP $remote_addr;
        proxy_set_header   Host      $http_host;
        proxy_pass         http://my_http_servers;
    }
}
3. Using Express Web Server: There is a lot of advantage to an Express web server. If you are comfortable with NodeJS, you can implement your own Express base load balancer as shown in the following example.
Step 1: Create an empty NodeJS application.
mkdir LoadBalancer
cd LoadBalancer
npm init -y
Step 2: Install required dependencies like ExpressJS, axios, and Concurrently using the following command.
npm i express axios
npm i concurrently -g
Step 3: Create two file config.js for the load balancer server and index.js for the application server.
Here filename is config.js
Javascript
const express = require('express');const path = require('path');const app = express();const axios = require('axios');  // Application serversconst servers = [    "http://localhost:3000",    "http://localhost:3001"]  // Track the current application server to send requestlet current = 0;  // Receive new request// Forward to application serverconst handler = async (req, res) =>{      // Destructure following properties from request object    const { method, url, headers, body } = req;      // Select the current server to forward the request    const server = servers[current];      // Update track to select next server    current === (servers.length-1)? current = 0 : current++      try{        // Requesting to underlying application server        const response = await axios({            url: `${server}${url}`,            method: method,            headers: headers,            data: body        });        // Send back the response data        // from application server to client         res.send(response.data)    }    catch(err){        // Send back the error message         res.status(500).send("Server error!")        }}  // Serve favicon.ico imageapp.get('/favicon.ico', (req, res    ) => res.sendFile('/favicon.ico'));  // When receive new request// Pass it to handler methodapp.use((req,res)=>{handler(req, res)});  // Listen on PORT 8080app.listen(8080, err =>{    err ?    console.log("Failed to listen on PORT 8080"):    console.log("Load Balancer Server "          + "listening on PORT 8080");});
  Here, the filename is index.js
Javascript
const express = require('express');const app1 = express();const app2 = express();  // Handler method const handler = num => (req,res)=>{    const { method, url, headers, body } = req;    res.send('Response from server ' + num);}  // Only handle GET and POST requests// Receive request and pass to handler methodapp1.get('*', handler(1)).post('*', handler(1));app2.get('*', handler(2)).post('*', handler(2));  // Start server on PORT 3000app1.listen(3000, err =>{    err ?    console.log("Failed to listen on PORT 3000"):    console.log("Application Server listening on PORT 3000");});  // Start server on PORT 3001app2.listen(3001, err =>{    err ?    console.log("Failed to listen on PORT 3001"):    console.log("Application Server listening on PORT 3001");});
Explanation: The above code starts with 2 Express apps, one on port 3000 and another on port 3001. The separate load balancer process should alternate between these two, sending one request to port 3000, the next request to port 3001, and the next one back to port 3000.
Step 4: Open a command prompt on your project folder and run two scripts parallel using concurrently.
concurrently "node config.js" "node index.js" 
Output: 
We will see the following output on the console:
Now, open a browser and go to http://localhost:8080/ and make a few requests, we will see the following output:
sweetyty
simranarora5sos
sumitgumber28
Node.js-Methods
NodeJS-Questions
Picked
Node.js
Web Technologies
Writing code in comment?
Please use ide.geeksforgeeks.org,
generate link and share the link here.
JWT Authentication with Node.js
Installation of Node.js on Windows
Difference between dependencies, devDependencies and peerDependencies
Mongoose Populate() Method
Mongoose find() Function
Top 10 Projects For Beginners To Practice HTML and CSS Skills
Difference between var, let and const keywords in JavaScript
How to insert spaces/tabs in text using HTML/CSS?
How to fetch data from an API in ReactJS ?
Differences between Functional Components and Class Components in React