Lambda Powertools: an important criteria when choosing your Lambda language
Tags: engineering, serverless, lambda, cloud
Published: 2023-08-22 12:02:16

**Or why the vast difference between Lambda Powertools per language features matter**

_Written by Samuel Durand_

## Intro

If you work with [AWS Lambdas](https://aws.amazon.com/de/lambda/) regularly, then you should already be using [Lambda Powertools](https://github.com/aws-powertools). This library has a really great set of features and utilities, maintained by AWS, and specifically designed and language-optimized __for AWS Lambda__. This set includes simple features that improve basic Lambda capabilities, such as structured logging and improved tracing, but also more elaborate features normally requiring a lot of boilerplate (i.e., Idempotency - but more on this later).

Due to their popularity, the Lambda Powertools are often mentioned in Lambda related articles, but one feature can quickly become __frustrating__. Some of the best, most often mentioned features, such as feature flags management __only seem to exist__ in one or two of the Lambda languages (and not always in a production ready state). Comparing the Lambda Powertools documentation for [Python](https://docs.powertools.aws.dev/lambda/python/latest/), [Java](https://docs.powertools.aws.dev/lambda/java/), [Typescript](https://docs.powertools.aws.dev/lambda/typescript/latest/), and [.Net](https://docs.powertools.aws.dev/lambda/dotnet/), quickly reveals that this is not a rare occurrence, and that the number of features available is extremely __dependent on which language you work with.__

The goal of this article is to show you how much the features available in the Powertools differ per language, and why it might be judicious to take it into account when __deciding what language you will use__ to develop your Lambdas.

## What's so great about the Powertools?

The Lambda Powertools are designed to make the developer's life easier by providing:

- __less boilerplate__ for both simple and more complex features (such as logging)
- opinionated code __optimized__ for Lambda
- an easier adoption of [AWS best practices](https://docs.aws.amazon.com/lambda/latest/dg/best-practices.html)
- a softer __learning curve__ for newcomers to Lambda and Serverless in general
- for most languages, a set of __Lambda layers__ maintained by AWS

They support a limited number of languages, which currently are [Python](https://docs.powertools.aws.dev/lambda/python/latest/) (2019), [Java](https://docs.powertools.aws.dev/lambda/java/) (2020), [Typescript](https://docs.powertools.aws.dev/lambda/typescript/latest/) (mid-2020), and [.Net](https://docs.powertools.aws.dev/lambda/dotnet/) (2023). The team also plans to add support at some point next year for Go, but there are no official plans for Ruby yet.

To demonstrate what the Powertools can do, let's have a look at one of the most popular features they offer: the [structured logging feature](https://docs.powertools.aws.dev/lambda/typescript/latest/core/logger/).

### Structured logging

The standard logs for a Lambda are usually not very detailed, __not easy to read or to parse__, and __lack contextual information__.

![Logs - plain](//images.ctfassets.net/hqu2g0tau160/5OOFQT21BRgIxeyF0H4OUw/b3304ff21e297aafebe009e1deaf6f43/Logs_-_plain.png)

By simply adding a couple Powertools decorators in your code, you can make Lambda logs considerably more useful. The below example shows what the content of each log would look like if you configured the Powertools to add almost every detail available, including the original payload.

```js
{
    "cold_start": true,
    "function_arn": "arn:aws:lambda:eu-central-1:11111111111:function:PowertoolsExampleStack-S3LambdaPoweredUp6F7EA-BC4pVVUUc5Dd",
    "function_memory_size": 128,
    "function_name": "PowertoolsExampleStack-S3LambdaPoweredUp6F7EA-BC4pVVUUc5Dd",
    "function_request_id": "68e8db49-65f7-430d-9514-dd2af37ce5d3",
    "level": "INFO",
    "message": "Lambda invocation event",
    "service": "s3Uploader",
    "timestamp": "2023-07-25T15:44:34.777Z",
    "xray_trace_id": "1-64bfede2-43e5e22d5f1b1530bea5ef7",
    "event": {
        "resource": "/upload-powered-up",
        "isBase64Encoded": false,
        [...] // Full API Gateway event details
    }
}
```

In this log, some fields can be especially useful:

- __cold_start__: indicates if this function was subject to a cold start
- __function_request_id__ can be used to figure out if this execution is a retry of a previously processed payload
- __xray_trace_id__: can be used to trace a particular request just by searching the logs and without paying for X-Ray
- __service__: this helps identify Lambdas composing a particular Service entity

To configure this, very little change to your code is required. It basically comes down to [adding a few decorators](https://github.com/samudurand/article-powertools-languages-unequal/blob/main/lambda/lambda-powered-up.ts#L66).

This is just an example of how powerful Lambda Powertools can be with very little effort invested. But that's all I will show here, since the topic of this article is not to give a tutorial on how to use the Powertools. If you want to see more on how it can be configured in a full CDK project, including an integration of the Powertools' [Idempotency feature](https://docs.powertools.aws.dev/lambda/typescript/latest/utilities/idempotency/), have a look at my [GitHub repository](https://github.com/samudurand/article-powertools-languages-unequal/tree/main).

## Most common features

Now that you have an idea of the sort of functionality we can achieve with the Lambda Powertools, let's have a look at some of the most common features that most languages support.

__Logging__ ([link](https://docs.powertools.aws.dev/lambda/python/latest/core/logger/))

As demonstrated earlier, this feature provides structured JSON logging with a lot of additional contextual data, including the original event payload, if requested.

__Tracing__ ([link](https://docs.powertools.aws.dev/lambda/python/latest/core/tracer/))

This feature is a thin wrapper for the [AWS X-Ray service](https://aws.amazon.com/xray/), which provides tracing capabilities. It generates traces in X-Rays with additional metadata and time datapoints to help debugging, such as the cold start, exception messages, and response bodies. It also provides helpers to add your own tracing points and will be disabled automatically if not running inside the Lambda environment (great for local development).

__Custom Metrics__ ([link](https://docs.powertools.aws.dev/lambda/python/latest/core/metrics/))

This feature uses the [CloudWatch Embedded Metric Format](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch_Embedded_Metric_Format.html) to allow asynchronous recording of custom metrics in CloudWatch. The metrics are recorded in the logs, then extracted by CloudWatch as metrics at a much lower cost than creating custom metrics via CloudWatch APIs.

__Parameters__ ([link](https://docs.powertools.aws.dev/lambda/python/latest/utilities/parameters/))

Using this feature, you can automatically retrieve parameters stored in the [Parameter Store](https://docs.aws.amazon.com/systems-manager/latest/userguide/systems-manager-parameter-store.html), [Secrets Manager](https://aws.amazon.com/de/secrets-manager/), [AppConfig](https://docs.aws.amazon.com/appconfig/latest/userguide/what-is-appconfig.html), or in [DynamoDB](https://aws.amazon.com/dynamodb/). The parameters will be cached for a configurable amount of time and passed to your handler at runtime. You can even use your own custom parameter store if desired.

__Idempotency__ ([link](https://docs.powertools.aws.dev/lambda/python/latest/utilities/idempotency/))

Event sources rarely provide exactly-once delivery. This is why most event handlers need to be idempotent, which means they have to make sure that processing the same event multiple times will have the same result, no matter how many times it happens. This can be achieved easily with this feature, which relies on DynamoDB for the storage part of the mechanism.

__Feature flags__ ([link](https://docs.powertools.aws.dev/lambda/python/latest/utilities/feature_flags/))

This utility can be used to configure your code to enable or disable features, either statically (e.g., a feature is not yet ready for production) or dynamically (e.g., per user). Each flag value can be retrieved either from [AppConfig](https://docs.aws.amazon.com/appconfig/latest/userguide/what-is-appconfig.html) or from your own custom configurations store.

__Streaming__ ([link](https://docs.powertools.aws.dev/lambda/python/latest/utilities/streaming/))

Lambdas usually have a very limited amount of memory, which can be an issue if you need to process very large S3 objects. You can either increase this value (up to 10Gb) and deal with the associated cost, or you can use the Powertools' streaming utility to process this enormous file a few Mb at a time.

__Batch failure processing__ ([link](https://docs.powertools.aws.dev/lambda/python/latest/utilities/batch/))

When using SQS, Kinesis or DynamoDB streams as an event source with batch processing, if you fail to process one of the events, you have the option to tell those services [which event has been successfully handled](https://docs.aws.amazon.com/lambda/latest/dg/with-sqs.html#services-sqs-batchfailurereporting), and which was not. This process can be extremely simplified by using the Batch Processing feature from the Powertools library.

__Validation__ ([link](https://docs.powertools.aws.dev/lambda/python/latest/utilities/validation/))

This utility provides JSON Schema validation for events and responses, including JMESPath support to unwrap events before validation.

__Large SQS messages handling__ ([link](https://docs.powertools.aws.dev/lambda/java/utilities/sqs_large_message_handling/))

When using SQS, the size of the messages is limited to 256kb. If you need to send a larger event, one of the solutions is to use the AWS SDK to upload the large payload to S3, then to send the reference to this file to SQS. You will then need to download the payload from S3 in the event handler before using it. The Powertools' utility can be used to automatically retrieve the content of that payload from S3 automatically during the event processing.

## Not all languages are equal

All those features are great, but unfortunately, not all features are available for all languages. And even when they are present, they are not always implemented with the same level of support and readiness (prod ready, preview, beta...). This also applies to the features that I previously called "most common." The table below highlights this disparity in language support.

![languages](//images.ctfassets.net/hqu2g0tau160/6n252qCgqrw3NOJwj9GFQa/185fe28a71386ded9579e066436f19a8/Screenshot_2023-08-04_at_16.28.13.png)

It should be pretty obvious that __Python and Java__ are the languages with the most production ready features. This makes sense since they were the first languages added to the Lambda Powertools. You might have also noticed the last row where I mention that __Python has a lot more features__, and by that I meant that the [Python Lambda Powertools](https://docs.powertools.aws.dev/lambda/python/latest/) contain __at least 7 more unique features__ than any other language.

This inequality is probably due to the very high difference of activity level between each language repository, which often happens with open source and community supported projects. If you look at the number of PRs, and at the commit dates and frequency on each library's repository on GitHub, it is clear that Python is by far the most lively repository.

![activity level](//images.ctfassets.net/hqu2g0tau160/2ym2uVKNq4vyU5Fu1txdg9/dc79d9c941a58ab50d1a605c66f9e882/Screenshot_2023-08-02_at_11.55.40.png)

## Choosing the right language

Selecting the language to use for your Lambda usually comes down to the following questions:

- what language is my team most comfortable with?
- what language offers me the features/libraries that are most adapted to my use case?
- how fast does the Lambda startup/execution need to be?
- are cold starts a concern?

Those are the usual questions, but considering how __useful and time saving__ some of the features of the Lambda Powertools are, it can be a good idea to also __include it into the decision process__.

Imagine yourself in the situation where:

- you are building a very simple application processing SQS events referring to large objects stored in S3
- each large object should be downloaded from S3 and split into parts sent to EventBridge as an event
- the logic is simple but you must make sure that events are not processed twice
- you want to stream the S3 object content instead of downloading it all in one go (to avoid using a lot of memory)
- your team is very skilled at Typescript, but has also some experience in Python

![Blank diagram - Page 1 (2)](//images.ctfassets.net/hqu2g0tau160/2RdoCfxPlOGhd86tv09Ssx/6f4b8b3a829313ff86206c77995165c3/Blank_diagram_-_Page_1__2_.png)

In this situation it might __make sense to choose Python__, despite your team having less experience with it. The code will be be relatively simple anyway, but would be even simpler and shorter if you use the __idempotency and S3 streaming__ features from the Python Powertools library.

## Conclusion

Lambda Powertools are a very powerful set of utilities which most Lambdas should be using. But you should never assume that a feature is present in a specific language just because you heard about it, or that this feature is production ready. Furthermore, __considering the value of this toolset__ and depending on your particular use case, it might be worth considering the Powertools __available features per language__ when choosing a programming language for your Lambda. You might save yourself a lot of pain and time by switching to a language you are less familiar with, but for which the capabilities that you need are already implemented, tested and optimized.

Keep in mind though that, just like any external dependency, adding the Lambda Powertools to your project will add some constraints on your project. And it might incite you to use a lot of fancy features you might not really need, and that might impact the overall performances of your application. Furthermore, those features are implemented in an opinionated fashion, which might not fit with your way of working, and offer a more or less limited number of configuration options, depending on the feature and language.

Note that since the AWS Lambda Powertools team recently [became an official AWS Team](https://twitter.com/heitor_lessa/status/1682059954973188105), it should provide more official support and help to fix this features disparity in the medium to long term.

I hope this advice will help you in your future projects, and as usual, if you have questions, or you would like to hear about my next articles, then follow me on [Linkedin](https://www.linkedin.com/in/samuel-durand-8300ba4a/) or contact me by email at samuel.durand@kreuzwerker.de.

---

### Links

- [Lambda Powertools repository](https://github.com/aws-powertools)
- [Powertools Roadmap](https://github.com/aws-powertools/powertools-lambda/projects/1)