Lambda insights to the rescue
Tags: aws, serverless, observability, devops
Published: 2021-10-23 18:57:35

---
title: Lambda insights to the rescue
series: [Lambda Observability]
published: true
description: Enable Lambda Insights
tags: #AWS #Serverless #Observability #DevOps
cover_image: https://www.pawelpiwosz.net/assets/lambdaObservability/sherlock-holmes.jpg
---

In previous episode we enabled tracing and we were able to see more information about our platform, requests, etc. Still, it is not much, but it is better than it was! Let's continue then.

## Lambda Insights

The best description of Lambda insights will be the enhanced monitoring. According to [AWS documentation](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Lambda-Insights.html), Insights is a service which collects, aggregates and summarized the system-level metrics, like CPU usage, memory usage, disk and network, and also collects diagnostic information like Lambda workers shutdown and cold start.

Ok, looks like this helps to understand better what is going on in the application.

## Enable Lambda Insights

Similarly like you did it for X-Ray, navigate to your Lambda, `Configuration` and finally to `Monitoring and operations tools`. 

![Enable Lambda Insights](https://www.pawelpiwosz.net/assets/lambdaObservability/enablelambdainsights.jpg)

AWS tries to update your roles and policies accordingly. When completed, you need to do some requests and **wait some time** before you start using Insights.

Also, what is most important, you should see a new layer added to your Lambda

![Lambda layer for Insights](https://www.pawelpiwosz.net/assets/lambdaObservability/lambdalayer.jpg)

In order to see Insights you can go there through CloudWatch console, or directly from your Lambda (from Monitoring tab).

![View Lambda Insights](https://www.pawelpiwosz.net/assets/lambdaObservability/viewlambdainsights.jpg)

...

...

...

...

![Waiting](https://www.pawelpiwosz.net/assets/lambdaObservability/waiting.jpg)

...

...

...

...

Finally, after many, many minutes (in my case when I prepared this example, it was more than 30 minutes), Insights are generated.

***

## Lambda Insights console

AWS gives the possibility to check mulitple functions at once, but I will go with single function view, as I have only one function enabled at the moment.

Metrics are available and are clear and readable. Invocations, errors, duration,  and all infrastructure related metrics have their own panel.

![Lambda Insights view](https://www.pawelpiwosz.net/assets/lambdaObservability/lambdainsights1.jpg)

Below these panels, AWS placed two tabs. One is for application logs, you saw it already, so I will not focus on it. The second (well, in fact, the first tab) is about Invocations.

![Lambda Insights view](https://www.pawelpiwosz.net/assets/lambdaObservability/lambdainsights2.jpg)

This view gives us a lot of information. Yes, part of it is available in logs, but here we have it without parsing. Duration, consumed memory, CPU time, network usage, error information. 

Additionally, we can see Init duration, which indicates cold start. Nice.

Let's explore it further. On top right corner of this part of the view we have two controls.

![Lambda Insights view](https://www.pawelpiwosz.net/assets/lambdaObservability/lambdainsights3.jpg)

Both will take you to `Logs Insights` with two different queries.

### Performance logs

This link will go to Logs with query shown below

```bash
fields @timestamp, @message, @logStream
| filter function_name = "lambdaDemo-lambdaFunction-qnettGrKoNgh"
| sort @timestamp desc
| limit 20
```

Here we have well formatted metrics about the whole invocation. Not just the Report generated (like in logs), but full information. For example, quite interesting data, like cold start, agent version, timestamp in UNIX format, traceID and many more.

Example of returned record is shown below

![Returned data](https://www.pawelpiwosz.net/assets/lambdaObservability/lambdainsights4.jpg)

### Application logs

Executes another query

```bash
fields @timestamp,@message,@logStream
| sort @timestamp desc
| limit 20
```

This also shows a lot of interesting information. First, visualisation (histogram) is available. Very nice.

Second, we have all logs collected in one place.

![Returned data](https://www.pawelpiwosz.net/assets/lambdaObservability/lambdainsights5.jpg)

***

Of course we can add it to SAM template. Here are the needed changes:

```yaml

AWSTemplateFormatVersion: 2010-09-09
Transform: AWS::Serverless-2016-10-31
Description: Simple Lambda

Resources:
  lambdaFunction:
    Type: AWS::Serverless::Function
    Properties:
      Handler: simplefunction.handler
      CodeUri: lambdafunction/
      Policies:
        - CloudWatchLambdaInsightsExecutionRolePolicy
      Runtime: python3.8
      AutoPublishAlias: live
      Description: Simple demo function
      MemorySize: 128
      Timeout: 10
      Tracing: Active
      Layers:
        - !Sub "arn:aws:lambda:${AWS::Region}:580247275435:layer:LambdaInsightsExtension:14"
      Events:
        simpleApi:
          Type: Api
          TracingEnabled: true
          Properties:
            Path: /
            Method: get
```

I added the policy `CloudWatchLambdaInsightsExecutionRolePolicy` and layer `LambdaInsightsExtension:14` to the Lambda definition. What is important here, I used the syntax `${AWS::Region}` in order to attach the layer from the region in which my Lambda is executed. This is why I used `!Sub` to "translate" this information during the actual execution of the template.

***

Ok, all of this is very nice. Again, we know more about our system. But still, something is missing, something big. In next episode we start to add more detailed tracing / loging / metrics to the Lambda.

***

Cover image by <a href="https://pixabay.com/users/422737-422737/?utm_source=link-attribution&amp;utm_medium=referral&amp;utm_campaign=image&amp;utm_content=462957">Hebi B.</a> from <a href="https://pixabay.com/?utm_source=link-attribution&amp;utm_medium=referral&amp;utm_campaign=image&amp;utm_content=462957">Pixabay</a>
Waiting image by <a href="https://pixabay.com/users/shlomaster-2095460/?utm_source=link-attribution&amp;utm_medium=referral&amp;utm_campaign=image&amp;utm_content=731165">Shlomaster</a> from <a href="https://pixabay.com/?utm_source=link-attribution&amp;utm_medium=referral&amp;utm_campaign=image&amp;utm_content=731165">Pixabay</a>