Serverless latency avoided with MongoDB connection pool
Tags: mongodb, serverless, latency, aws
Published: 2022-12-02 14:52:35

There are many benefits to building apps using **Serverless** frameworks. Apart from offloading a large amount of the heavy lifting to a cloud-managed service, it is much easier to configure and scale deployments. Logic can be placed closer to the end user and environments can be easily created by aliasing functions. You are only billed for the memory used and the execution time incurred so the cost is usually lower compared to other architectures.

Due to these and many other benefits, Tailwarden is built as a Serverless platform. Made up of hundreds of Lambda functions with an aliased API Gateway in front of them that directs traffic to the functions in our three different environments (Sandbox, Staging, and Production).

On paper that all sounds great but what happens if you have hundreds of lambda functions simultaneously connecting to a MongoDB cluster on a regular basis? Every time a new database connection needs to be created, which we quickly saw was impacting performance as latency increased considerably. We had to avoid having to create a new connection every time that database was queried.

This is what we did to fix it.

## The connection pool

A peculiar but valid way of thinking about a database connection pool is the example of a nightclub. Usually, when you go to a nightclub, you wait in line, pay your entry fee and then you might get a side-eye glance from the bouncer but you are allowed to enter. If you want to briefly leave, the bouncer will either stamp your hand or give you a bracelet which means you can now freely leave and come back without having to get in a queue and do all of the steps you did the first time to get in.

A connection pool is similar in the sense that it’s a pool of available connections that can be used without having to go through all the motions of creating a new connection every time. When a request is complete the connection is returned to the pool and even though it’s not used it’s still available. If you leave the nightclub for some fresh air, you can walk back in as long as you have a bracelet. You can also re-use MongoDB connections, as long as the [driver](https://www.mongodb.com/docs/drivers/) can handle connection pools, and you specify the configuration through your connection method. In our case, we used the [MONGO\_URI](https://www.mongodb.com/docs/v6.0/reference/connection-string/#std-label-mongodb-uri) environment variable to carry the connection string.

By appending the following option ?poolSize=5 (the bracelet) to the value of the MONGO\_URI we maintained a constant pool size of 5 that allowed for a huge increase in performance.

## The Tailwarden stack

First, we had to filter out all Lambda functions that are connecting to our MongoDB cluster by writing a bash script to lookup for functions with a **MONGO\_URI** environment variable.

![Lambda dashboard](https://miro.medium.com/max/700/1*5ADs_B46iNhT7tN9CIz5cQ.png)

The lambda functions connect to the database with a connection string similar to this one.

![mongodb connection URI](https://miro.medium.com/max/583/1*EfOP91ZrXoPyBn-spyxqaQ.png)

In the code snippet below you can see how we pass in the connection credentials with the MONGO\_URI

A few best practices to keep in mind when handling AWS Lambda connections to MongoDB Atlas:

-   The client to the MongoDB server needs to be outside of the AWS Lambda handler function.
-   If you place it inside the function the driver will create a new db connection for each function call which can become expensive and impact performance.
-   If your handler takes a callback as its last argument set the **_callbackWaitsForEmptyEventLoop_** property on the [AWS Lambda Context](http://docs.aws.amazon.com/lambda/latest/dg/nodejs-context.html#nodejs-prog-model-context-properties) object to **false.**
-   This allows a Lambda function to return its result to the caller without requiring that the MongoDB database connection be closed.

![MongoDB connection function](https://miro.medium.com/max/700/1*IOA1ixBzCZVjifb8RfcgzQ.png)

We had to retrieve the functions which have the **MONGO\_URI** environment variable and then append **?PoolSize=5** to the end of them, without deleting the other existing environment variables.

![Lambda function configuration dashboard](https://miro.medium.com/max/700/1*yCOL0SVYswqJ6LLP7VzWBA.png)

## The script

There are multiple strategies we could have gone with but we decided to keep the script as simple as possible. We interface with AWS directly using the [AWS CLI](https://aws.amazon.com/cli/) in combination with [jq](https://stedolan.github.io/jq/) to parse the values or our variables which are in JSON format.

This is the script that got the job done called **updateLambdaVars.sh**:

![updateLambdaVars script](https://miro.medium.com/max/700/1*QUDUGwzw_LWe3gIeuefwUA.png)

## Let’s break it down line by line:

First, we grab the functions that have the **MONGO\_URI** environment variable present using **jq** and store the filteredFunctions as a list.

![Load filteredFunctions variable](https://miro.medium.com/max/700/1*EPfOr-wbTru_btd_YmHirA.png)

We then start a for loop which updates the **MONGO\_URI** for each Lambda function in three steps and ends when it has filtered through the full **filtredFunctions** list.

In the first step of the loop we store the function name in the **functionName** variable, filtering it first with **jq**

![Use jq to filter functionName](https://miro.medium.com/max/621/1*N1ZlS6JFMrAtTorA2X8VPg.png)

Get the current values of the environment variables configured and merge the value **test?poolSize=5** to the end of the **MONGO\_UR**I value and store it in **new\_envvars**

![merge with existed environment variables](https://miro.medium.com/max/700/1*kY_D2AZPquoWdVRwZai7YA.png)

Update the lambda configuration with **update-function-configuration** command by passing the function name variable and the new list of environment variables.

![AWS update-function-configuration command](https://miro.medium.com/max/700/1*maid_JIpu2_HzXAsO7N6Gg.png)

Give the script **permission to execute** and **run it**. The script will go through all Lambda functions in the specified AWS region and include the **poolSize** param to each MongoDB string **URI**:

![Final step](https://miro.medium.com/max/421/1*6ae4RO43yvpHeZZS-wPHUw.png)

## Conclusion

When it comes to Serverless frameworks any issues that affect latency can quickly trigger a chain reaction and potentially snowball to turn your application into something your users don’t want to use. When it comes to some sorts of latency our hands are tied, like [service integration](https://dev.to/aws-builders/the-hidden-serverless-latency-58p4) latency. But that’s not the case with inefficiencies around database connections. Utilize connection pools to ease the load on the database, doing so will increase performance speed and you won’t have to be afraid of the bouncer kicking you out.

> Regardless if you are a Developer, DevOps, or Cloud engineer. Dealing with the cloud can be tough at times, especially on your own. If you are using Tailwarden or Komiser and want to share your thoughts doubts and insights with other cloud practitioners feel free to join our [Tailwarden Discord server](https://discord.tailwarden.com/). Where you will find tips, community calls, and much more.