Monitoring AWS Lambda Logs with ClickStack using Rotel

[Edit this page](https://github.com/ClickHouse/clickhouse-docs/blob/main/docs/use-cases/observability/clickstack/ingesting-data/integration-examples/aws-lambda.md)

Community Maintained

TL;DR

This guide shows you how to monitor AWS Lambda functions with ClickStack by using the Rotel Lambda Extension to collect and forward function logs, extension logs, and OpenTelemetry data directly to ClickHouse. You'll learn how to:

- Deploy the Rotel Lambda Extension layer to your Lambda functions
- Configure the extension to export logs and traces to ClickStack
- Optionally disable CloudWatch Logs to reduce costs

This approach can significantly reduce your Lambda observability costs by bypassing CloudWatch Logs entirely.

Time Required: 5-10 minutes

## Integration with existing Lambda functions[​](#existing-lambda "Direct link to Integration with existing Lambda functions")

This section covers configuring your existing AWS Lambda functions to send logs and traces to ClickStack using the Rotel Lambda Extension.

### Prerequisites[​](#prerequisites "Direct link to Prerequisites")

- ClickStack instance running
- AWS Lambda functions to monitor
- AWS CLI configured with appropriate permissions
- Lambda execution role with permissions to add layers

Choose the appropriate Rotel Lambda Extension layer

#### Choose the appropriate Rotel Lambda Extension layer[​](#choose-layer "Direct link to Choose the appropriate Rotel Lambda Extension layer")

Choose the Lambda layer that matches your Lambda runtime architecture. The `{version}` field
is dependent on the AWS region that you're deploying into. Check the [releases](https://github.com/streamfold/rotel-lambda-extension/releases)
page for the latest version numbers that correspond to your region.

|  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- |
| Architecture ARN|  |  |  |  | | --- | --- | --- | --- | | x86-64/amd64 `arn:aws:lambda:{region}:418653438961:layer:rotel-extension-amd64:{version}`| arm64 `arn:aws:lambda:{region}:418653438961:layer:rotel-extension-arm64:{version}` | | | | | |

**Available regions:**

- us-east-2, us-west-2
- eu-central-1, eu-north-1, eu-west-3
- ca-central-1
- ap-southeast-2, ap-northeast-2
- ap-south-1
- sa-east-1

Add the Rotel layer to your Lambda function

#### Add the Rotel layer to your Lambda function[​](#add-layer "Direct link to Add the Rotel layer to your Lambda function")

*In these examples replace `{arch}`, `{region}`, and `{version}` with the appropriate values above.*

##### Option 1: AWS Console[​](#console "Direct link to Option 1: AWS Console")

1. Open the AWS Lambda console
2. Navigate to your Lambda function
3. Scroll to the **Layers** section and click **Add a layer**
4. Select **Specify an ARN**
5. Enter the Rotel layer ARN:

   ```
   arn:aws:lambda:{region}:418653438961:layer:rotel-extension-{arch}:{version}
   ```
6. Click **Add**

##### Option 2: AWS CLI[​](#cli "Direct link to Option 2: AWS CLI")

```
aws lambda update-function-configuration \
  --function-name my-function \
  --layers arn:aws:lambda:{region}:418653438961:layer:rotel-extension-{arch}:{version}
```

##### Option 3: AWS SAM[​](#sam "Direct link to Option 3: AWS SAM")

```
Resources:
  MyFunction:
    Type: AWS::Serverless::Function
    Properties:
      # ... other configuration ...
      Layers:
        - arn:aws:lambda:{region}:418653438961:layer:rotel-extension-{arch}:{version}
```

Configure the extension to export to ClickStack

#### Configure the extension to export to ClickStack[​](#configure-extension "Direct link to Configure the extension to export to ClickStack")

The Rotel Lambda Extension is configured using environment variables. You need to configure the OTLP exporter endpoint to point to your ClickStack instance. The examples assume your AWS Lambda function is able to reach the ClickStack instance.

##### Basic configuration (environment variables)[​](#basic-config "Direct link to Basic configuration (environment variables)")

Add these environment variables to your Lambda function:

```
# Required: ClickStack OTLP endpoint
ROTEL_OTLP_EXPORTER_ENDPOINT=https://clickstack.example.com:4317

# Optional: Authentication headers
ROTEL_OTLP_EXPORTER_CUSTOM_HEADERS="Authorization=<YOUR_INGESTION_API_KEY>"

# Optional: Service name (defaults to Lambda function name)
ROTEL_OTEL_RESOURCE_ATTRIBUTES="service.name=my-lambda-api,service.version=1.0.0"
```

##### Advanced configuration (using .env file)[​](#advanced-config "Direct link to Advanced configuration (using .env file)")

For more complex configurations, create a `rotel.env` file in your Lambda function bundle:

**rotel.env:**

```
ROTEL_OTLP_EXPORTER_ENDPOINT=https://clickstack.example.com:4317
ROTEL_OTLP_EXPORTER_CUSTOM_HEADERS="Authorization=<YOUR_INGESTION_API_KEY>"
ROTEL_OTEL_RESOURCE_ATTRIBUTES="service.name=my-lambda-api,deployment.environment=production"
```

Then set the environment variable to point to this file:

```
ROTEL_ENV_FILE=/var/task/rotel.env
```

##### Using AWS Secrets Manager or Parameter Store[​](#secrets "Direct link to Using AWS Secrets Manager or Parameter Store")

For production deployments, store sensitive values like API keys in AWS Secrets Manager or Parameter Store:

**AWS Secrets Manager Example:**

```
ROTEL_OTLP_EXPORTER_ENDPOINT=https://clickstack.example.com:4317
ROTEL_OTLP_EXPORTER_CUSTOM_HEADERS="Authorization=${arn:aws:secretsmanager:us-east-1:123456789012:secret:clickstack-api-key-abc123}"
```

**AWS Parameter Store Example:**

```
ROTEL_OTLP_EXPORTER_ENDPOINT=https://clickstack.example.com:4317
ROTEL_OTLP_EXPORTER_CUSTOM_HEADERS="Authorization=${arn:aws:ssm:us-east-1:123456789012:parameter/clickstack-api-key}"
```

**Required IAM Permissions:**

Add these permissions to your Lambda execution role:

For Secrets Manager:

```
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "secretsmanager:GetSecretValue",
        "secretsmanager:BatchGetSecretValue"
      ],
      "Resource": "arn:aws:secretsmanager:us-east-1:123456789012:secret:clickstack-api-key-*"
    }
  ]
}
```

For Parameter Store:

```
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "ssm:GetParameters"
      ],
      "Resource": "arn:aws:ssm:us-east-1:123456789012:parameter/clickstack-api-key"
    }
  ]
}
```

Note

AWS API calls for secret retrieval add 100-150ms to cold start latency. Secrets are retrieved in batches (up to 10) and only on initialization, so subsequent invocations aren't impacted.

Test the integration

#### Test the integration[​](#test-integration "Direct link to Test the integration")

Invoke your Lambda function to verify logs are being sent to ClickStack:

```
aws lambda invoke \
  --function-name my-function \
  --payload '{"test": "data"}' \
  response.json
```

Check the Lambda logs for any errors:

```
aws logs tail /aws/lambda/my-function --follow
```

Verify logs in HyperDX

#### Verify logs in HyperDX[​](#verify-logs "Direct link to Verify logs in HyperDX")

Once configured, log into HyperDX (ClickStack's UI) and verify that logs are flowing:

Look for these key attributes in the logs:

- `service.name`: Your Lambda function name
- `faas.name`: AWS Lambda function name
- `faas.invocation_id`: Unique invocation ID
- `cloud.provider`: "aws"
- `cloud.platform`: "aws\_lambda"

## Disabling CloudWatch Logs (cost optimization)[​](#disable-cloudwatch "Direct link to Disabling CloudWatch Logs (cost optimization)")

By default, AWS Lambda sends all logs to CloudWatch Logs, which can be expensive at scale. Once you've verified that logs are flowing to ClickStack, you can disable CloudWatch logging to reduce costs.

Remove CloudWatch permissions from the execution role

#### Remove CloudWatch permissions from the execution role[​](#remove-permissions "Direct link to Remove CloudWatch permissions from the execution role")