Under Pressure: Benchmarking Node.js on a Single-Core EC2
Tags: node, javascript, api, aws
Published: 2023-12-02 23:48:51

Hi!

In this post, I'm going to _stress_ test a Node.js 21.2.0 **pure** API (no framework!) to see the efficiency of the Event Loop in a limited environment.

I'm using AWS for hosting the servers (EC2) and database (RDS with Postgres).

The main goal is to understand how many requests per second a simple Node API can handle on a single core, then identify the bottleneck and optimize it as much as possible.

Let's dive in!

## Infrastructure
- AWS RDS running Postgres
- EC2 t2.small for the API
- EC2 t3.micro for the load tester

## Database Setup
The database will consist of a single `users` table created with the following SQL query:

```sql
CREATE TABLE IF NOT EXISTS users (
    id SERIAL PRIMARY KEY,
    email VARCHAR(255) NOT NULL,
    password VARCHAR(255) NOT NULL
);
TRUNCATE TABLE users;
```

## API Design
The API will have a single POST endpoint that will be used to save a user to the Postgres database. I know, there are a lot of javascript frameworks out there that I could use to make the development easier, but it's possible to use only Node to handle the requests/responses. 

To connect to the database, I chose the library `pg` as it is the most popular one, we'll start with it.

### Connection Pooling
One thing that is important when connecting to a database is using a connection pool. Without a connection pool, the API needs to open/close a connection to the database at each request, which is extremely inefficient.

A pool allows the API to reuse connections, as we're planning to send a lot of concurrent requests to our API, it's **crucial** to have it.

To check your Postgres database's connection limit, run:
```sql
SHOW max_connections;
```

In my case, I'm using an RDS running on a t3.micro database with these specs:

![AWS RDS configs](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/drfmlonw8tugqluybbev.png)

So this is the outcome of the query:
![Max Connections](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/pu9y3g6xranu8fxppa2q.png)

Cool, having 81 as the maximum number of connections to our database, we know what is the upperbound limit we should not surpass. 

As the API will run on a single-core processor, it's not a good idea to have a high number of connections on the connection pool, as this would cause a lot of headache to the processor (_context switching_).

Let's start with 40.

### Creating the API
We'll start by starting our project with `npm init` and creating our `index.mjs` file. MJS so I can use EcmaScript synthax without doing too much magic/parsing/loading.

The first thing I'll do is add the [pg library](https://www.npmjs.com/package/pg) with `npm add pg`. I'm using `npm` but you can use pnpm, yarn or any other node package manager you want.

Then, let's start by creating our connection pool:
```javascript
import pg from "pg"; // Required because pg lib uses CommonJS 🤢
const { Pool } = pg;

const pool = new Pool({
  host: process.env.POSTGRES_HOST,
  user: process.env.POSTGRES_USER,
  password: process.env.POSTGRES_PASSWORD,
  port: 5432,
  database: process.env.POSTGRES_DATABASE,
  max: 40, // Limit is 81, let's start with 40
  idleTimeoutMillis: 0, // How much time before kicking out an idle client.
  connectionTimeoutMillis: 0, // How much time to disconnect a new client, we don't want to disconnect them for now.
  ssl: false
  /* If you're running on AWS, you'll need to use:
  ssl: {
    rejectUnauthorized: false
  }
  */
});
```
We're using process.env to access the environment variables, so create a `.env` file on the root and fill with your postgres informations:
```bash
POSTGRES_HOST=
POSTGRES_USER=
POSTGRES_PASSWORD=
POSTGRES_DATABASE=
```

Then, let's create a function to persist our user on the database.
```javascript
const createUser = async (email, password) => {
  const queryText =
    "INSERT INTO users(email, password) VALUES($1, $2) RETURNING id";
  const { rows } = await pool.query(queryText, [email, password]);
  return rows[0].id;
};
```

Finally, let's create a node HTTP server by importing the `node:http` package and writing a code to handle new requests, parse from string to JSON, query the database and return 201, 400 or 500 in case of any errors, the final file looks like this.

```javascript
// index.mjs
import http from "node:http";
import pg from "pg";
const { Pool } = pg;

const pool = new Pool({
  host: process.env.POSTGRES_HOST,
  user: process.env.POSTGRES_USER,
  password: process.env.POSTGRES_PASSWORD,
  port: 5432,
  database: process.env.POSTGRES_DATABASE,
  max: 40,
  idleTimeoutMillis: 0,
  connectionTimeoutMillis: 2000,
  ssl: false
  /* If you're running on AWS, you'll need to use:
  ssl: {
    rejectUnauthorized: false
  }
  */
});

const createUser = async (email, password) => {
  const queryText =
    "INSERT INTO users(email, password) VALUES($1, $2) RETURNING id";
  const { rows } = await pool.query(queryText, [email, password]);
  return rows[0].id;
};

const getRequestBody = (req) =>
  new Promise((resolve, reject) => {
    let body = "";
    req.on("data", (chunk) => (body += chunk.toString()));
    req.on("end", () => resolve(body));
    req.on("error", (err) => reject(err));
  });

const sendResponse = (res, statusCode, headers, body) => {
  headers["Content-Length"] = Buffer.byteLength(body).toString();
  res.writeHead(statusCode, headers);
  res.end(body);
};

const server = http.createServer(async (req, res) => {
  const headers = {
    "Content-Type": "application/json",
    Connection: "keep-alive", // Default to keep-alive for persistent connections
    "Cache-Control": "no-store", // No caching for user creation
  };
  if (req.method === "POST" && req.url === "/user") {
    try {
      const body = await getRequestBody(req);
      const { email, password } = JSON.parse(body);
      const userId = await createUser(email, password);

      headers["Location"] = `/user/${userId}`;
      const responseBody = JSON.stringify({ message: "User created" });
      sendResponse(res, 201, headers, responseBody);
    } catch (error) {
      headers["Connection"] = "close";
      const responseBody = JSON.stringify({ error: error.message });
      console.error(error);
      const statusCode = error instanceof SyntaxError ? 400 : 500;
      sendResponse(res, statusCode, headers, responseBody);
    }
  } else {
    headers["Content-Type"] = "text/plain";
    sendResponse(res, 404, headers, "Not Found!");
  }
});

const PORT = process.env.PORT || 3000;

server.listen(PORT, () => {
  console.log(`Server running on http://localhost:${PORT}`);
});
```
Now, after running `npm install`, you can run
```bash
node --env-file=.env index.mjs
```

To start the application, you should see this on your terminal:

![Server is running](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/52y2255wsh7zj4g0xsdi.png)

Congrats, we have built a simple NodeAPI with one endpoint that connects to Postgres through a Connection Pool and inserts a new user to the users table.

## Deploying the API to an EC2
First, create an AWS account and go to EC2 > Instances > Launch an Instance.

Then, create an Ubuntu 64-bit (x86) t2.micro instance, allow SSH traffic and allow HTTP traffic from the Internet.

Your summary should look like this:

![AWS EC2 t2.micro Summary](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/6lqpozvh5xj4b5wbpe9g.png)

You'll need to create a key-value-pair.pem file to be able to SSH into it, I won't cover this in this article, there are already plenty of tutorials teaching how to launch and connect to an EC2 instance on the internet, so find them!


### Allowing TCP connections on port 3000
After creation, we need to allow TCP traffic for port 3000, this is done on the Security Group config (EC2 > Security Groups > Your Security Group)

![Security Group page](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ohd6cmk20fesesyexwa3.png)

At this page, click on "Edit inbound rules", then "Add rule" and fill the form as shown on the image, this will allow us to hit port 3000 of our instance.

![Inbound Rule](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/o1zn64ffemfe2ztisibc.png)

Your final Inbound Rules table should look something like this.

![Inbound Rules](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/x0sxn5zjwn8axfugpbe5.png)

### Connecting to EC2
Download the _.pem_ file in a folder, then access the EC2 instance and copy the public IPV4 IP, then, run this command on the same folder:


![Public IPV4 address](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/lbmnnzrh64mtxxem1gax.png)

```bash
ssh -i <path-to-pen> ubuntu@<public-ipv4-address>
```

If you see this EC2 welcome page, then you're in 🎉

![EC2 Welcome page](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/zktjbpajt9gi8v6u1qi8.png)

#### Installing Node
Let's follow the [Node documentation for Debian/Ubuntu-based Linux distros](https://github.com/nodesource/distributions#debian-and-ubuntu-based-distributions).

Run:
```bash
sudo apt-get update
sudo apt-get install -y ca-certificates curl gnupg
sudo mkdir -p /etc/apt/keyrings
curl -fsSL https://deb.nodesource.com/gpgkey/nodesource-repo.gpg.key | sudo gpg --dearmor -o /etc/apt/keyrings/nodesource.gpg
```

Then:
```bash
NODE_MAJOR=21
echo "deb [signed-by=/etc/apt/keyrings/nodesource.gpg] https://deb.nodesource.com/node_$NODE_MAJOR.x nodistro main" | sudo tee /etc/apt/sources.list.d/nodesource.list
```
**Important:** Double check that the NODE_MAJOR is 21, as we want to use the latest version of Node <3

```bash
sudo apt-get update
sudo apt-get install nodejs -y
node -v
```

And that's what you should see (it may differ the version as this post gets old)

![Node installed](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/3lw847xq9tcjk0zux40j.png)

Nice, now we have a fresh new ubuntu server with node installed, we need to transfer our API code to it and start it.


### Deploying API to EC2
We'll use a tool called _scp_ that uses ssh connection to copy file from local to a target location, in our case, the EC2 instance we just created.

Steps:
- Delete the node_modules folder from the project.
- Go to the parent folder of the root folder of the application.

In my case, the name of the folder is `node-api` (I know, very creative!)

Now, run:
```bash
scp -i <path-to-pem> -r ./node-api ubuntu@<public-ipv4-address>:/home/ubuntu
```

To transfer the folder `node-api` to the /home/ubuntu/node-api folder at our EC2 instance.

You should see something similar to this:
![files transfered](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/qwwoddu8rwhjmqm8pthu.png)

### Running the API on EC2
Head back to the EC2 server using ssh and run
```bash
cd node-api
npm install
NODE_ENV=production node --env-file=.env index.mjs
```

And boom, the API is running on AWS.

Let's double check that it's working by making a POST request passing email and password to the IP of our API, at the port 3000.

You can use curl (on another terminal), to do this:
```bash
curl -X POST -H "Content-Type: application/json" -d {email: user@example.com, password: password} http://<public-ipv4-address>:3000/user
```

The result should look like this:
![User Created](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/pb8wyzd3gi3kz5v53t39.png)

I'm using [Table Plus](https://tableplus.com/) to connect to the RDS Postgres database, you could use any Postgres Client.

To ensure that the API is persisting data to the database, let's run this query:

```sql
 SELECT COUNT(id) FROM users;
```
![Returned](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/m04ak3w0c3nkcklu0tur.png)

It should return 1.

Nice, it's working!

## Stress Test
Now that we have our API working, we need to be able to test how many concurrent requests it can handle with a **single core**.

There are tons of tools to do this, I'll use [Vegeta](https://github.com/tsenart/vegeta)

You can run the following steps from your local machine, but keep in mind that your network may be the bottleneck, as the stress test requires a lot of packages to be sent at the same time.

I'll use another EC2 instance (a more powerful one, t2x.large) running Ubuntu.

### Configuring Vegeta
Follow the docs to install Vegeta on your OS.

Then, create a new folder for load testers on the root folder of the application, it's looking like this:

```bash
node_benchmark/
  node-api/
  load-tester/
    vegeta/
```

Go to the vegeta folder and create a start.sh script with the following content:
```bash
#!/bin/bash
if [[ $# -ne 1 ]]; then
    echo 'Wrong arguments, expecting only one (reqs/s)'
    exit 1
fi

TARGET_FILE="targets.txt"
DURATION="30s"  # Duration of the test, e.g., 60s for 60 seconds
RATE=$1    # Number of requests per second
RESULTS_FILE="results_$RATE.bin"
REPORT_FILE="report_$RATE.txt"
ENDPOINT="http://<ipv4-public-address>:3000/user"

# Check if Vegeta is installed
if ! command -v vegeta &> /dev/null
then
    echo "Vegeta could not be found, please install it."
    exit 1
fi

# Create target file with unique email and password for each request
echo "Generating target file for Vegeta..."
> "$TARGET_FILE"  # Clear the file if it already exists

# Assuming body.json exists and contains the correct JSON structure for the POST request
for i in $(seq 1 $RATE); do 
    echo "POST $ENDPOINT" >> "$TARGET_FILE"
    echo "Content-Type: application/json" >> "$TARGET_FILE"
    echo "@body.json" >> "$TARGET_FILE"
    echo "" >> "$TARGET_FILE"
done

echo "Starting Vegeta attack for $DURATION at $RATE requests per second..."
# Run the attack and save the results to a binary file
vegeta attack -rate=$RATE -duration=$DURATION -targets="$TARGET_FILE" > "$RESULTS_FILE"

echo "Load test finished, generating reports..."
# Generate a textual report from the binary results file
vegeta report -type=text "$RESULTS_FILE" > "$REPORT_FILE"
echo "Textual report generated: $REPORT_FILE"

# Generate a JSON report for further analysis
JSON_REPORT="report.json"
vegeta report -type=json "$RESULTS_FILE" > "$JSON_REPORT"
echo "JSON report generated: $JSON_REPORT"

cat $REPORT_FILE
```

**IMPORTANT**: Replace `<ipv4-public-address>` with the IP of your EC2 Node API Server

Now, create a `body.json` file:
```json
{
  "email": "A1391FDC-2B51-4D96-ADA4-5EEE649A4A75@example.com",
  "password": "password"
}
```

Now you're ready to start load-testing our api.

This script will:
- Run for 30s
- Hit the API with concurrent requests/s defined by the first argument of the script
- Generate a textual and .json file with infos about the test.

Last, but not least, we need to make the `start.sh` file executable, we can do this by running:

```bash
chmod +x start.sh
```

Before running each test, I'll clear the users table on Postgres with the following query.
```sql
TRUNCATE TABLE users;
```

This will help us see how many users were created!

### 1.000 Reqs/s
Alright, let's go to the interesting part, let's see if our single core, 1GB server can handle 1.000 requests per second.

Run 
```bash
./start.sh 1000
```

Wait for the completion, here it generates the following output:

![1.000 reqs/s](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/xkq73oc72n77b956cntp.png)


Let me break it down for you:

At the rate os 1.000 requests per second, the Node API was able to successfully process all of them, returning the expected success status 201.

In average, each request took 4.254 ms to be returned, with 99% of them returning in less than 25.959 ms.

| Metric                | Value      |
|-----------------------|------------|
| Requests per Second   | 1000.04    |
| Success Rate          | 100%       |
| p99 Response Time     | 25.959 ms  |
| Average Response Time | 4.254 ms    |
| Slowest Response Time | 131.889 ms  |
| Fastest Response Time | 2.126 ms   |
| Status Code 201       | 30000      |


Let's check our database:
![Database Count](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/pmhuk66jgsrn8ka9d4sk.png)

Cool, it worked!

Let's try harder and double the number of requests per second.

### 2.000 Requests per second
Run 
```bash
./start.sh 2000
```

Let's check the output

![2.000 reqs/s](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/3tajeemcv0spts1bjpdq.png)


Awesome, it can handle 2.000 requests/second and still keep a 100% success rate.

| Metric                | Value         |
|-----------------------|---------------|
| Requests per Second   | 2000.07       |
| Success Rate          | 100.00%       |
| p99 Response Time     | 2.062 s       |
| Average Response Time | 136.347 ms    |
| Slowest Response Time | 4.067 s       |
| Fastest Response Time | 2.164 ms      |
| Status Code 201       | 60000         |

A couple things to notice here, while the success rate was still 100%, the p99 jumped from 25.959ms to 2.067s (79x slower than the previous test).

The average response time also jumped from 4.254ms to 136.347 (32.1x slower).

So yeah, doubling the number of requests per second is making our server to suffer A LOT.

Let's try harder and see what happens.

### 3.000 Requests per second
```bash
./start.sh 3000
```

![3.000 reqs/s output](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/jrpjhk87ti89omgsmd4l.png)


For 3.000 requests/second our Node.js API started to present problems, being able to process only 52.20%, let's see what happened.

| Metric                | Value      |
|-----------------------|------------|
| Requests per Second   | 2267.72    |
| Success Rate          | 52.20%     |
| p99 Response Time     | 30.001 s   |
| Average Response Time | 6.146 s    |
| Slowest Response Time | 30.156 s   |
| Fastest Response Time | 3.018 ms   |
| Status Code 201       | 36089      |
| Status Code 500       | 21588      |
| Status Code 0         | 11465      |

![Database](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/7u0p04jyrgu2xahsdh9e.png)


For 21,588 requests, our API returned status code 500, let's check the API logs:

![API logs](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/k20ycz69qn4m32m4bii5.png)


We can see that our Postgres connection is hitting timeout, the current connectionTimeoutMillis is configured to be 2000 (2s), let's try increasing this to 30000 and see if that improves our load test.

We can do that by changing the line 13 of index.mjs from 2000 to 30000:
```javascript
connectionTimeoutMillis: 30000
```

Let's run it again:
```bash
./start.sh 3000
```

And the result?

![97.39% success](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/u1le6oi97qx1lzrs20bg.png)


| Metric                | Value      |
|-----------------------|------------|
| Requests per Second   | 2959.90    |
| Success Rate          | 97.39%     |
| p99 Response Time     | 13.375 s   |
| Average Response Time | 6.901 s    |
| Slowest Response Time | 30.001 s   |
| Fastest Response Time | 3.476 ms   |
| Status Code 201       | 86486      |
| Status Code 0         | 2318       |

Nice, by simply increasing the connection timeout for the database we improved the success rate by 45,19%, also, all the 500 errors are now completely gone!

Let's take a look at the remaining errors (status code 0).
![bind address already in use](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/z065105c64nqh55yozdq.png)


Status code 0 usually means that the server reset the connection because it couldn't handle more. 

Let's check if it's CPU, Memory or Network.

At the peak of the test, the CPU is only consuming 13%, so it's not CPU.

![CPU](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/tyzvvakuifqcev40fcc5.png)


By running it again with `htop` I noticed that memory was up only about 70%, so that's also not the problem:

![Memory](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/zq5y9tt88d1mlxicm0qh.png)


Let's try something different.

#### File Descriptors
In unix systems, each new connection (_socket_) is assigned to a File Descriptor. By default, on Ubuntu, the maximum number of open file descriptors is 1024.

You chan check that by running `ulimit -n`.


![limit api](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/rwzqx3xgszt665wovdjw.png)


Let's try increasing that to 2000 and redo the test to see if we can get rid of these 2% timeout errors.

To do so, I'll follow [this tutorial](https://stackoverflow.com/a/11345256) and change to 6000

```bash
sudo vi /etc/security/limits.conf
```

![new limits for nofile](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/fzi1625w5wi632br0ys6.png)

nofile = number of files.
soft = soft limit.
hard = hard limit.

Then reboot the EC2 with `sudo reboot now`.

After logging in, we can see that the limit changed:

![New ulimit is 2000](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/l67nccbt72c1pzeoa9c6.png)


And let's redo the test:

Start the API with
```bash
NODE_ENV=production node --env-file=.env index.mjs
```

And start the load-tester with:
```bash
./start.sh 3000
```

Let's check the results:

![Results with 2000 open files](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/xd0tku5sldik02l5n61i.png)

Surprisingly, the results are worse!

With a maximum of 2.000 open files, the node API successfully answered only 78.43% of the requests.

This is because by having only one core, adding more open sockets make the processor switch between the files more often than the previous version.

Let's try reducing it to 700 to see if it gets better.

(I'll skip the step on how to do it because it's the same).

And let's see the new output with 700 as maximum open files.

![700](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ny59iy4qdahtx51wnej2.png)

With 700 maximum open files, we hit 83.20% success rate. Let's go back to 1024 and try reducing the connection pool to 20 instead of 40.

If that doesn't work, let's assume 3.000 req/s is slightly higher than the limit and we'll try to find the maximum number of requests/s that a single core node API can handle with 100% success.

![93%](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/vvlhvq2k98wt88oafmfd.png)

With 20 connections for the connection pool, the API was able to process 93.06%, proving that we probably don't need 40.

Let's try with 2.600 reqs/s:

### 2.600 reqs/s
```bash
./start.sh 2600
```

And that's the result:
![2600 100% success](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ny6othqgyw49pdzianek.png)


| Metric                | Value      |
|-----------------------|------------|
| Requests per Second   | 2600.04    |
| Success Rate          | 100%       |
| p99 Response Time     | 8.171 s    |
| Average Response Time | 4.573 s    |
| Slowest Response Time | 9.234 s    |
| Fastest Response Time | 5.244 ms   |
| Status Code 201       | 77999      |


That's a wrap!

## Conclusion
This experiment demonstrates the capabilities of a pure Node.js API on a single-core server. 

With a pure Node.js 21.2.0 API, using a single core with 1GB of RAM + connection pool with a maximum 20 connections, we were able to achieve 2.600 requests/s without failures.

By fine-tuning parameters like connection pool size and file descriptor limits, we can significantly impact performance.

What's the highest load your Node.js server has handled? Share your experiences!