Programmatically retrieving secrets from Parameter Store and Secrets Manager
Tags: aws, security, serverless
Published: 2023-07-27 08:26:54

Passwords, API keys, and other secrets are regular participants in our codebase, and as such, we need to make them available for our applications. We have multiple ways to integrate Parameter Store and Secrets Manager items into our Lambda function code.

## 1. Problem statement

Most of the time, we use parameters and secrets in our applications. Apps need connection URLs, and database credentials or have to call 3rd-party endpoints protected with API keys.

We can integrate these parameters and secrets in our code in different ways. Some of them are better, and others are not so good.

Below I'll discuss some ways we can make them available in our <a href="https://aws.amazon.com/lambda/" class="link" target="_blank" rel="noopener noreferrer">Lambda functions</a>. I'll refer to passwords, connection URLs, and API keys as **secrets** in the rest of this writing to be brief.

Although I'll use Lambda functions in the examples, we can transfer the concepts to other compute resources, like <a href="https://aws.amazon.com/ec2/" class="link" target="_blank" rel="noopener noreferrer">EC2</a> instances, and <a href="https://aws.amazon.com/ecs/" class="link" target="_blank" rel="noopener noreferrer">ECS</a> or <a href="https://aws.amazon.com/eks/" class="link" target="_blank" rel="noopener noreferrer">EKS</a> containers.

## 2. Hardcoding

The easiest way is to bake the secrets in the code, which is a strict **no** in most cases for security and <a href="https://en.wikipedia.org/wiki/Don%27t_repeat_yourself" class="link" target="_blank" rel="noopener noreferrer">DRY</a> reasons. Even if the data is not confidential, we might still need it in multiple places. When the data changes, we'll have to change it everywhere, which is not a good practice.

So we are better off adding the secrets to a central place like <a href="https://docs.aws.amazon.com/systems-manager/latest/userguide/systems-manager-parameter-store.html" class="link" target="_blank" rel="noopener noreferrer">Parameter Store</a> or <a href="https://aws.amazon.com/secrets-manager/" class="link" target="_blank" rel="noopener noreferrer">Secrets Manager</a>.

## 3. Using Parameter Store

Parameter Store is great because we can park our secrets there for <a href="https://aws.amazon.com/systems-manager/pricing/" class="link" target="_blank" rel="noopener noreferrer">free</a> in many cases. The task is to retrieve the secret from there and make it available in the Lambda function code.

Although the rest of the blog entry uses Parameter Store as an example, the same concepts will apply to Secrets Manager items.

### 3.1. Deployment time

We can add **dynamic references** to our <a href="https://aws.amazon.com/cloudformation/" class="link" target="_blank" rel="noopener noreferrer">CloudFormation</a> templates. This way, the secret's value will be available at deployment time. We can store it as an **environment variable** in the function code.

An example in a <a href="https://aws.amazon.com/serverless/sam/" class="link" target="_blank" rel="noopener noreferrer">SAM</a> template can look like this:

```yml
Resources:
  DeployTimeFunction:
    Type: AWS::Serverless::Function
    Properties:
      # other properties here
      Environment:
        Variables:
          MY_PLAIN_STRING: '{{resolve:ssm:/test/plain-secret:1}}'
          MY_SM_SECRET: '{{resolve:secretsmanager:test/sm-secret}}'
```

We can do the same with <a href="https://aws.amazon.com/cdk/" class="link" target="_blank" rel="noopener noreferrer">CDK</a>:

```ts
export class LambdaSecretsCdkStack extends cdk.Stack {
  constructor(scope: Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    const fn = new lambda.NodejsFunction(this, 'DeployTimeFunction', {
      // other properties here
      environment: {
        MY_PLAIN_STRING: ssm.StringParameter.fromStringParameterName(
          this,
          'ParameterStoreString',
          '/test/plain-secret'
        ).stringValue,
        MY_SM_SECRET: cdk.SecretValue.secretsManager('test/sm-secret', {
          jsonField: 'name',
        }).toString(),
      },
    });

    // and more code here
  }
}
```

Since SAM and CDK use CloudFormation in the background, the same concepts apply.

The advantage of this method is that we don't need to worry about Parameter Store or Secrets Manager permissions because CloudFormation will handle everything for us. The secret will be readily available, and we can immediately refer to it in the code:

```ts
const { MY_SM_SECRET, MY_PLAIN_STRING } = process.env;

export const handler = async (event) => {
  // use MY_SM_SECRET and MY_PLAIN_STRING in the handler code
}
```

A disadvantage of this method is that if we change the secret's value, we'll need to redeploy the stack and change the version number in the infrastructure code. Also, anyone with access to the Lambda function can see the value in the console.

CloudFormation currently <a href="https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/dynamic-references.html#dynamic-references-ssm-secure-strings" class="link" target="_blank" rel="noopener noreferrer">doesn't support</a> dynamic Parameter Store **secure string** resolution. So if we want to use secure strings in our code, we must choose one of the following methods.

### 3.2. Run time - own code

Another (and more secure) way is to retrieve secrets at run time.

We can do the heavy lifting and write the code ourselves. For example, we can have a helper module to get the secret using the <a href="https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/" class="link" target="_blank" rel="noopener noreferrer">SDK</a>, and cache it like this:

```js
// helper.mjs
import { SSMClient, GetParameterCommand } from '@aws-sdk/client-ssm';

const client = new SSMClient();

let secret;

export const getParameter = async ({ name, withDecryption = false }) => {
  if (secret) {
    console.log('Returning secret from cache');
    return secret;
  }

  const input = {
    Name: name,
    ...(withDecryption && { WithDecryption: withDecryption }),
  };
  const command = new GetParameterCommand(input);

  const parameter = await client.send(command);
  secret = parameter.Parameter.Value;
  console.log('Getting the secret from Parameter Store');
  return secret;
}
```

We can easily use it in the function handler:

```js
// index.mjs
import { getParameter } from './helpers/ssm.mjs';

export const handler = async (event) => {
  try {
    const myParameter = await getParameter({
      name: '/test/my-secret', withDecryption: true,
    });
    // do something with myParameter
  } catch (error) {
    console.error('Error: ', error.message);
    throw error;
  }
}
```

The helper module **caches** the secret and makes it available throughout the execution environment's lifetime. It implies that the code will call Parameter Store only once for each Lambda function lifecycle. The `helper` module is reusable across multiple functions and applications.

The function's **execution role** must have the relevant <a href="https://aws.amazon.com/iam/" class="link" target="_blank" rel="noopener noreferrer">IAM</a> **permissions**. If we allow the `ssm:GetParameters`, `ssm:GetParameter` and `ssm:GetParametersByPath` actions in the role's policy, the function will be able to retrieve various types and numbers of parameters. If we choose to encrypt the secret with a customer-managed <a href="https://aws.amazon.com/kms/" class="link" target="_blank" rel="noopener noreferrer">KMS</a> key (i.e., not the default AWS-managed key), we must add the `kms:Decrypt` permission to the policy, too.

### 3.3. Using a Lambda extension

We can use the <a href="https://aws.amazon.com/about-aws/whats-new/2022/10/aws-parameters-secrets-lambda-extension/" class="link" target="_blank" rel="noopener noreferrer">AWS Parameters and Secrets Lambda Extension</a> to get the secret from Parameter Store.

It's a fun and easy way to retrieve and cache parameters and secrets. I wrote a <a href="https://dev.to/aws-builders/a-different-way-of-retrieving-secrets-in-lambda-functions-4g7h" class="link" target="_blank" rel="noopener noreferrer">post</a> about the extension earlier, so I won't repeat it here.

### 3.4. Using Lambda Powertools

<a href="https://docs.powertools.aws.dev/lambda/typescript/latest/" class="link" target="_blank" rel="noopener noreferrer">Lambda Powertools</a> is a library that provides several utilities to make us write less code and follow best practices. These utilities include logging, metrics, tracing, retrieving parameters, and other helpers.

We can use the **Parameters** utility to get secrets from Parameter Store and Secrets Manager. In addition to obtaining them from the services, we can set a TTL different from the default 5 seconds. We can also create our parameter store provider.

For our use case, a sample code with Lambda Powertools can look like this:

```ts
import { getParameter } from '@aws-lambda-powertools/parameters/ssm';

export const handler = async (event) => {
  try {
    const secret = await getParameter(
      '/test/my-secret', { decrypt: true, maxAge: 300 },
    );
    // do something with the secret from Parameter Store
  } catch (error) {
    console.error('Error: ', error.message);
    throw error;
  }
}
```

The second argument of the `getParameter` function is a config object, where we can set caching with the `maxAge` (in seconds) or decryption with the `decrypt` (boolean) properties. We can also use the `POWERTOOLS_PARAMETERS_MAX_AGE` and `POWERTOOLS_PARAMETERS_SSM_DECRYPT` environment variables to adjust these settings.

We must add the relevant permissions to the function's execution role, as discussed above.

For more information on Lambda Powertools, please see the **Further reading** section.


## 4. Summary

Parameters and secrets are part of our application codes. The best practice is to store them in purpose-built services, from which we can get and share them among multiple applications.

We can retrieve the secrets at deployment or run time. If we choose run-time retrieval, we can write our code or use the prebuilt logic of the AWS Parameters and Secrets Lambda Extension or Lambda Powertools.

## 5. Further reading

<a href="https://docs.aws.amazon.com/systems-manager/latest/userguide/ps-integration-lambda-extensions.html" class="link" target="_blank" rel="noopener noreferrer">Using Parameter Store parameters in AWS Lambda functions</a> - (Almost) everything about the extension

<a href="https://aws.amazon.com/blogs/compute/retrieving-parameters-and-secrets-with-powertools-for-aws-lambda-typescript/" class="link" target="_blank" rel="noopener noreferrer">Retrieving parameters and secrets with Powertools for AWS Lambda (TypeScript)</a> - More information about getting secrets from Parameter Store

<a href="https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_create.html" class="link" target="_blank" rel="noopener noreferrer">Creating IAM policies
</a> - The title of the documentation page says it all