How to access Neon Postgres from AWS Lambda functions via serverless driver
Tags: cloud, neon, serverless, aws
Published: 2024-02-29 20:48:16


Serverless is a computing paradigm that provides a pay-as-you-use model. It lets you deploy your projects easily and scale to meet large demands without the stress of maintaining or managing servers. 

Wherever there are conversations on serverless, two important concepts often come up:

- Compute — the processing power of a server.
- Storage — the capacity to store data.

Traditionally, accessing a database from a serverless computing service like [AWS Lambda](https://aws.amazon.com/lambda/?utm_source=hackmamba&utm_medium=blog&utm_id=HMBcommunity) is tricky due to the differences in connection protocols. Lambda functions are “ephemeral” — relying on short-lived, HTTP-centric connections, while databases utilize long-lived, TCP-centric connections.

It is possible to send a single-shot HTTP query to retrieve data from your database. However, maintaining a persistent connection (like streaming data to your clients) could lead to timeouts in your lambda runtime or compute costs that balloon unexpectedly.

One way to solve this problem is connecting via a gateway or proxy. This way, your client can establish a long-lived connection (using WebSockets) with the proxy, which then interacts with your backend using standard HTTP requests.

Neon’s [serverless driver](https://neon.tech/docs/serverless/serverless-driver?ref=hm) is a drop-in replacement for `node-postgres` that combines the benefits of WebSockets and HTTP into one simple package, [optimiz](https://neon.tech/blog/quicker-serverless-postgres?ref=hm)[ing](https://neon.tech/blog/quicker-serverless-postgres?ref=hm) [the number of round trips](https://neon.tech/blog/quicker-serverless-postgres?ref=hm) required to interact with a Neon Postgres database.

This article will teach you how to access a Neon Postgres database from an AWS Lambda function via Neon’s serverless driver.

![Data flow between Neon and AWS](https://paper-attachments.dropboxusercontent.com/s_0B288B3ADBD6B4ACFE38D8459D9358ABFF511A58AC71D6877694EE2BA6485001_1701684186287_Untitled.png)

> **Source code**
> Here’s a link to the GitHub [repository](https://github.com/dpkreativ/neon-serverless-lambda?utm_source=hackmamba&utm_medium=blog&utm_id=HMBcommunity) for this article’s demo.


## Prerequisites
- A Neon account. Neon offers a generous free tier that you can start with. Get it [here](https://console.neon.tech/sign_in).
- An AWS account. Explore its free tier [here](https://aws.amazon.com/free).
- [Node.js](https://nodejs.org/) installed on your PC. It comes with `npm`, which you will use to add Neon’s serverless driver to your project.
- A code editor. This article’s demo was built with [Visual Studio Code](https://code.visualstudio.com/).


## Setting up a Neon database

### Create a project
When you [sign up on Neon](https://console.neon.tech/sign_in), you’ll be prompted to create a project and a database like this:

![Create Project page showing inputs for Project name, Database name, and Region](https://paper-attachments.dropboxusercontent.com/s_0B288B3ADBD6B4ACFE38D8459D9358ABFF511A58AC71D6877694EE2BA6485001_1701684328889_Untitled+1.png)


Once created, your project’s dashboard will contain a general overview of your database connection details, branches, compute, and storage data.

![Neon dashboard showing Connection Details, Branches, Usage, and Storage details](https://paper-attachments.dropboxusercontent.com/s_0B288B3ADBD6B4ACFE38D8459D9358ABFF511A58AC71D6877694EE2BA6485001_1701684392578_Untitled+2.png)


Please take note of your database connection string because you will add it as an environment variable in your project.

### Seed data
In your Neon project’s console sidebar, click on ‘**SQL Editor**’ and run the queries below to seed your database with some data:

```sql
    CREATE TABLE cars (
        id SERIAL PRIMARY KEY,
        car TEXT NOT NULL,
        model TEXT NOT NULL,
        created_at TIMESTAMPTZ DEFAULT NOW()
    );
    
    INSERT INTO cars (car, model)
    VALUES
        ('Toyota', 'Camry'),
        ('Honda', 'Civic'),
        ('Ford', 'Mustang'),
        ('Tesla', 'Model S'),
        ('Chevrolet', 'Silverado'),
        ('BMW', '3 Series'),
        ('Mercedes-Benz', 'E-Class'),
        ('Jeep', 'Wrangler'),
        ('Audi', 'Q5'),
        ('Subaru', 'Outback');
```

This SQL script creates a `cars` table with columns for `id`, `car`, `model`, and `created_at`, then inserts the provided car makes and models into the table.

![GIF demonstrating how to run an SQL script in Neon’s SQL editor](https://paper-attachments.dropboxusercontent.com/s_0B288B3ADBD6B4ACFE38D8459D9358ABFF511A58AC71D6877694EE2BA6485001_1701694107965_seed-database.gif)



## Accessing Neon’s database with serverless driver

### Setup project
In your terminal, navigate to your project’s folder and start a new Node.js project by running the command below:


```bash
    mkdir neon-serverless-lambda
    cd neon-serverless-lambda
    npm init -y
```

This will create a Node.js project with default values. Open your project in a code editor and update the `package.json` with the code below:

```json
    {
      "name": "neon-serverless-lambda",
      "version": "1.0.0",
      "description": "",
      "main": "index.js",
      "type": "module",
      "scripts": {
        "test": "echo \\"Error: no test specified\\" && exit 1"
      },
      "keywords": [],
      "author": "",
      "license": "ISC"
    }
```

Here, you added `"type": "module"` to enable EcmaScript’s `import/export` functionality.
In your project’s terminal, run the command below to install Neon’s serverless driver:

```bash
    npm install @neondatabase/serverless
```

### Write function
Open your project in a code editor, then create an `index.js` file and update it with the code below:

```javascript
    // index.js (using `neon` object)
    import { neon } from '@neondatabase/serverless';
    
    export async function handler(event) {
      try {
        const sql = neon(process.env.DATABASE_URL);
        const rows = await sql('SELECT * from cars');
    
        console.log(rows);
        
        return {
          statusCode: 200,
          body: JSON.stringify({
            message: 'Successful',
            data: rows,
          }),
        };
    
      } catch (err) {
        console.log('Error', err);
      }
    }
```

In the code above, you did the following:

- Imported the `neon` object from `@neondatabase/serverless`. This object will allow you to connect to and run SQL queries on a Neon database using a connection string.
- Created a `handler()` function with an `event` parameter.
- In your `handler()` function, you wrote a `try-catch` block that attempts to establish a connection to the database using the `neon` object and the `DATABASE_URL` environment variable. It executes a SQL query (`SELECT * from cars`) to retrieve all rows from the "**cars**" table. Any errors during the database operation are logged using `console.log()`.
- After fetching the rows from the database, your function constructs and returns an HTTP response with a status code of 200 (indicating success) and a JSON body containing a success message ('**Successful**') along with the retrieved rows (`data: rows`).


> **Choosing between the Neon serverless driver’s `neon` or `Pool/Client` objects** 
> If you want to execute multiple queries in a single connection, Neon’s `Pool` object uses WebSockets and can bring latencies down to 4ms once a connection is established. However, it is best to use the `neon` object for single-shot queries. Check out [this](https://neon.tech/blog/http-vs-websockets-for-postgres-queries-at-the-edge?ref=hm) [](https://neon.tech/blog/http-vs-websockets-for-postgres-queries-at-the-edge?ref=hm)[blog post](https://neon.tech/blog/http-vs-websockets-for-postgres-queries-at-the-edge?ref=hm) for more insights.

### Prepare for deployment
In your PC’s file explorer, navigate to your project’s folder and zip all of its contents:

![File explorer showing selected Neon code and .zip file](https://paper-attachments.dropboxusercontent.com/s_0B288B3ADBD6B4ACFE38D8459D9358ABFF511A58AC71D6877694EE2BA6485001_1701686467810_Untitled+3.png)


You will upload this `.zip` file to AWS while creating the Lambda.


## Deploying to AWS

### Create a Lambda function
In your browser, log into your [AWS console](https://console.aws.amazon.com/), search for “**Lambda**,” and select it from the results:

![AWS console with search results for Lambda](https://paper-attachments.dropboxusercontent.com/s_0B288B3ADBD6B4ACFE38D8459D9358ABFF511A58AC71D6877694EE2BA6485001_1701686829591_Untitled+4.png)


Click “**Create function**” and fill out the basic information for your new Lambda function:

![Create function page with inputs for your Lambda’s basic information](https://paper-attachments.dropboxusercontent.com/s_0B288B3ADBD6B4ACFE38D8459D9358ABFF511A58AC71D6877694EE2BA6485001_1701694353994_Untitled+5.png)


After filling out the required basic information, scroll down and click “**Create function**” to finish creating your new Lambda function:

![neon-serverless-lambda dashboard on AWS](https://paper-attachments.dropboxusercontent.com/s_0B288B3ADBD6B4ACFE38D8459D9358ABFF511A58AC71D6877694EE2BA6485001_1701694551082_Untitled+6.png)


### Upload your code as a zip file
In your Lambda function’s dashboard, scroll down to “**Code source**”, click “**Upload from**”, and select “**.zip file**”:

![Code source section in AWS Lambda](https://paper-attachments.dropboxusercontent.com/s_0B288B3ADBD6B4ACFE38D8459D9358ABFF511A58AC71D6877694EE2BA6485001_1701694632659_Untitled+7.png)


Navigate to your project’s folder and select the .zip file you created in the last section, then click “**Save**”:

![AWS Lambda dashboard with .zip file selected](https://paper-attachments.dropboxusercontent.com/s_0B288B3ADBD6B4ACFE38D8459D9358ABFF511A58AC71D6877694EE2BA6485001_1701694673000_Untitled+8.png)


### Set environment variables
In your Lambda function’s dashboard, click “**Configuration**”, then select “**Environment variables**” and click the “**Edit**” button:

![Environment variables screen](https://paper-attachments.dropboxusercontent.com/s_0B288B3ADBD6B4ACFE38D8459D9358ABFF511A58AC71D6877694EE2BA6485001_1701694713697_Untitled+9.png)


In the “**Edit environment variables**” page, click “**Add environment variable**”, then input `DATABASE_URL` as the variable key and your Neon project’s database connection string as the value, then click “**Save**”:

![Edit environment variables page with key and value inputs](https://paper-attachments.dropboxusercontent.com/s_0B288B3ADBD6B4ACFE38D8459D9358ABFF511A58AC71D6877694EE2BA6485001_1701694751377_Untitled+10.png)


### Test your deployment
In your Lambda function, click “**Test**” to create a new test event, then fill out the event name, and click “**Save**”:

![Test event screen with inputs to create a reusable test](https://paper-attachments.dropboxusercontent.com/s_0B288B3ADBD6B4ACFE38D8459D9358ABFF511A58AC71D6877694EE2BA6485001_1701694808443_Untitled+11.png)


After saving your test, click the “**Test**” button to run it. If it is successful, you will see something like this:

![Successful test result on AWS Lambda](https://paper-attachments.dropboxusercontent.com/s_0B288B3ADBD6B4ACFE38D8459D9358ABFF511A58AC71D6877694EE2BA6485001_1701694839570_Untitled+12.png)



## Creating and testing your API

### Create an API with AWS API Gateway
In your Lambda’s “**Function overview**” section, click “**Add trigger**” and select “**API Gateway**”:

![Add trigger page with API Gateway selected](https://paper-attachments.dropboxusercontent.com/s_0B288B3ADBD6B4ACFE38D8459D9358ABFF511A58AC71D6877694EE2BA6485001_1701694906318_Untitled+13.png)


Select “**Create a new API**”, specify “**HTTP API**” as the API type and the security mechanism as “**Open**”, then scroll down and click “**Add**”:

![API Gateway Trigger configuration options](https://paper-attachments.dropboxusercontent.com/s_0B288B3ADBD6B4ACFE38D8459D9358ABFF511A58AC71D6877694EE2BA6485001_1701694962406_Untitled+14.png)


Once your API endpoint has been created, scroll down to “**Triggers**” to see its URL:

![API Gateway trigger for neon-serverless-lambda showing its endpoint URL](https://paper-attachments.dropboxusercontent.com/s_0B288B3ADBD6B4ACFE38D8459D9358ABFF511A58AC71D6877694EE2BA6485001_1701695500731_Untitled+15.png)


### Test your API endpoint
To quickly test your new endpoint, copy its URL and make a “**GET**” request in your preferred API testing platform:

![Thunderclient UI to test API endpoint](https://paper-attachments.dropboxusercontent.com/s_0B288B3ADBD6B4ACFE38D8459D9358ABFF511A58AC71D6877694EE2BA6485001_1701695548832_Untitled+16.png)



## Conclusion

Following this guide, you’ve had hands-on experience bridging the gap between AWS Lambda and Neon’s database using the Neon serverless driver. In this guide, you manually zipped and deployed your code; however, you can explore [The Serverless Framework](https://www.serverless.com/?utm_source=hackmamba&utm_medium=blog&utm_id=HMBcommunity) or [AWS Serverless Application Model](https://docs.aws.amazon.com/serverless-application-model/?utm_source=hackmamba&utm_medium=blog&utm_id=HMBcommunity) [(SAM)](http://?utm_source=hackmamba&utm_medium=blog&utm_id=HMBcommunity) for continuous integration and deployment.

Serverless computing can help you scale to meet enormous demands. When paired with a robust, scalable, serverless database service like Neon, the result is a seamless experience for your development team and end users. Neon also offers innovative features like database branching, autoscaling, and bottomless storage, saving you time and money. [Click here](https://neon.tech/?ref=hm) to explore its free tier today.