Setting Up a Vercel "Log Drain" with AWS
Tags: vercel, nextjs, aws, observability
Published: 2023-01-23 13:46:58


If you're hosting apps on [Vercel](https://vercel.com), you might have noticed that they have a number of quality logging integrations in their Marketplace. But if those don't suit your needs, or if you're already using AWS services (especially Lambda and CloudWatch) and want to keep things lean and simple, this article is for you!

> This might seem a bit involved, but the code is simple and it really only takes about 15 minutes! 😎
Also, searching API logs from Vercel in CloudWatch is a breeze! 🏄🏻 I'll tell you more about that at the end of the article.

## 1. Create a Lambda to process your logs
In your AWS console, go to Services -> Lambda -> Functions, and click "Create Function". Name your function as appropriate, but leave everything else in this first section at the defaults.
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/uvmi205ou4i21q1yiw8q.png "Create Lambda function")

Under "Advanced Settings", you need to select "Enable function URL", choose "NONE" for "Auth type" (we'll be handling authentication ourselves), and check the CORS button.
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/fq9v63okk9xkfm2gndku.png "Create Function Advanced Settings")

Now click "Create Function". Once you're redirected to your new lambda, find "Function URL" in the upper right under "Function Overview", and copy it to your clipboard. You'll need this for the next step.

⛔️ Don't close this tab! You've got more work to do here very soon.

## 2. Create a new Log Drain in Vercel
In a different browser tab, go to your Vercel dashboard, then go to Settings (this is settings for your whole team in Vercel), then click "Log Drains". Here you'll see the "Add a Log Drain" form.

1. Choose your desired sources. (I just wanted to log Next.js API requests, so I only chose "Lambda").
2. For delivery format, we will use "NDJSON" format which is easy for us to process in our logging lambda. (More on that later.)
3. Choose "all projects" or specific projects as you wish. Note that you can easily filter logs by project later from the CloudWatch console. (See end of article for details.)
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/zv4ov07todj7cmu8dlni.png "Log Drain Sources, etc")
4. Under "Endpoint" paste your lambda function URL. (You'll now see a new panel with a "Verify" button. ✋ Don't do this yet!)
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/zeezbiuurvdv089ifnhi.png "Log Drain Endpoint")
5. Click the toggle for "Custom Headers". We will be authenticating requests to our logging lambda via an "authentication" header.
6. Enter "authentication" in the Header Key. Then create a token (using something like the [LastPass generator](https://www.lastpass.com/features/password-generator)), and paste it into "Header Value", prepending "Bearer ". (This is an authentication header convention.) Now click "Add" to add the header to the Log Drain.
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/4s4md526r20d4hls4hp1.png "Custom Headers Form")
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/zysvvv0w3ntq09iang89.png "Custom Header Added")

⛔️ Don't close this tab!

## 3. Verify your logging URL
Vercel requires that you verify your ownership of your logging URL, and to do this, you'll need to modify your lambda to return a header that Vercel specifies.
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/oy79i7lv2om360q65j0q.png "URL Verification Panel")

Go back to your lambda console in AWS, and replace `index.mjs` with this:
```javascript
export const handler = async(event) => {
    const response = {
        statusCode: 200,
        body: JSON.stringify('Hello from Lambda!'),
        headers: {
            'x-vercel-verify': '<vercel-verification-string>'
        },
    };
    return response;
};
```
🟢  Be sure to click "Deploy" to deploy your changes to your live lambda.

Now go back to Vercel and click "Verify". The verification panel should now go away and you'll see a blue check mark next to your URL.

🙌  We're _almost_ ready to "Add Log Drain", but not quite yet!

## 4. Complete your lambda, adding authentication and logging functions

Back in your lambda console tab, you'll need to add the authentication token you generated earlier to your lambda's environment variables. Go to "Configuration -> Environment Variables" then click "Edit". Add your token with the key "AUTH_TOKEN" and click "Save".
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/l5l8ljpyw99jgofm83ez.png "Add Token Env Variable")

Now, replace your function's `index.mjs` with the following:

```javascript
/* global atob */

export const handler = async(event, context) => {
    // Return unauthorized if missing auth header
    if (!event.headers.authentication || event.headers.authentication !== `Bearer ${process.env.AUTH_TOKEN}`) {
        console.log('🔴 Unauthorized request');
        return {
            statusCode: 401,
        };
    }
    /**
     * We convert the request body from Base64,
     * then split the resulting string by new lines.
     * (The NDJSON format sends line-delimited JSON logs.)
     */
    const logs = atob(event.body).split('\n');

    // We create a log entry for each item received.
    logs.forEach((log) => {
        /**
         * The split function will leave an empty item
         * at the end of the array. We don't want to log this.
         */
        if (log !== ''){
          console.log(log);
        }
    });
    
    return {
        statusCode: 200,
    };
};
```
🟢  Again, be sure to click "Deploy" to deploy your changes to your live lambda.

Your lambda is now complete! 🕺🏻

## FINAL STEP: Activate the Log Drain in Vercel

Finally, go back to your Vercel tab and click the "Add Log Drain" button.

Done! 🎉

> (Note that Vercel will show a dialog with information about securing your logging lambda, but we've already handled this, so you can ignore these instructions unless you prefer the authentication method they suggest.)

<hr />

## 🚨 Bonus: Working with API logs in CloudWatch and Logs Insights

To see the logs you've ingested from Vercel, go to your lambda's console, click "Monitor" then click "View CloudWatch logs". To view all logs, click "Search all log streams", then apply filters as needed to find the logs you're looking for.

The way we've set up our logging, with one entry per log sent by Vercel, logs can be filtered as though they are JSON objects using [CloudWatch filter syntax](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/FilterAndPatternSyntax.html#matching-terms-events). For example, to view all `GET` requests, add this to the search/filter input:
`{$.proxy.method = "GET"}`

Or to filter by project, use:
`{$.projectName = "my-nextjs-project"}`

In Log Insights, you can create some valuable visualizations, like how many requests missed the cache per hour:
```
fields @message, @timestamp
|  filter(proxy.vercelCache = "MISS")
|  stats count() by bin(1h)
```
...or stats for requests for specific project grouped by request method:
```
fields @message, @timestamp
|  filter(projectName = "my-nextjs-project")
|  stats count() by proxy.method
```

I hope this helps!