AWS SnapStart - Part 12 Measuring cold starts and deployment time with Java 21 using different Lambda memory settings
Tags: aws, java, serverless, coldstart
Published: 2024-01-08 17:03:49

## Introduction
In the [part 11](https://dev.to/aws-builders/aws-snapstart-part-11-measuring-cold-starts-with-java-21-using-different-deployment-artifact-sizes-4g29) of our series we measured the cold starts of the Lambda function with Corretto Java 21 runtime without SnapStart enabled, with SnapStart enabled and also applied DynamoDB invocation priming optimization with different  deployment packages/jar sizes between 137KB and 50MB. In this article we will measure the cold starts but will use different memory settings.

## Measuring cold starts with Java 21 with and without SnapStart enabled using different Lambda memory settings.

In our experiment we'll re-use the application introduced in [part 9](https://dev.to/aws-builders/measuring-lambda-cold-starts-with-aws-snapstart-part-9-measuring-with-java-21-5e52) for this. Here is the code for the [sample application](https://github.com/Vadym79/AWSLambdaJavaSnapStart/tree/main/pure-lambda-21). There are basically 2 Lambda functions which both respond to the API Gateway requests and retrieve product by id received from the Api Gateway from DynamoDB. One Lambda function GetProductByIdWithPureJava21Lambda can be used with and without SnapStart and the second one GetProductByIdWithPureJava21LambdaAndPriming uses SnapStart and DynamoDB request invocation priming. We'll measure cold starts using the following memory settings in MBs : 256, 512, 768, 1024, 1536 and 2048.

The results of the experiment below were based on reproducing approximately 100 cold starts for the duration of our experiment which ran for approximately 1 hour. For it (and all experiments from my previous articles) I used the load test tool [hey](https://github.com/rakyll/hey), but you can use whatever tool you want, like [Serverless-artillery](https://www.npmjs.com/package/serverless-artillery) or [Postman](https://www.postman.com/)

**Cold start time without SnapStart in ms:**

| Experiment description                     | p50      | p75      | p90       |p99       | p99.9      | max      
|-----------|----------|-----------|----------|----------|----------|----------|
|256 MB cold start time w/o SnapStart                | 8056.84  | 8298.23   | 8494.11  |9123.72|9256.72|9417.65|
|512 MB cold start time w/o SnapStart  | 4684.92  | 4781.67   | 4944.95  |5014.81|5156.32|5321.34|
|768 MB cold start time w/o SnapStart |3638.06   | 3720.91  |3815.31  |4636.18|4787.51|4895.48|
|1024 MB cold start time w/o SnapStart                | 3157.6  | 3213.85   | 3270.8  |3428.2|3601.12|3725.02|
|1536 MB cold start time w/o SnapStart  | 2665.49  | 2700.56   | 2808.68  |3306.94|3476.05|3613.84|
|2048 MB cold start time w/o SnapStart| 2414.66   | 2463.17    | 2521.7  |2839.59|3012.07|3156.33|

Let's put this data into graph:

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/6r797wqm887yylwqmzqr.png)


**Cold start time with SnapStart without Priming in ms :**

| Experiment description                     | p50      | p75      | p90       |p99       | p99.9      | max      
|-----------|----------|-----------|----------|----------|----------|----------|
|256 MB cold start time with SnapStart w/o Priming                | 5623.35  | 5736.89   | 6526.39  |7020.38|7076.74|7080.47|
|512 MB cold start time with SnapStart w/o Priming  |  2861.27   | 3035.06  |3504.91|3881.08|3927.91|3931.82|
|768 MB cold start time with SnapStart w/o Priming |1976.74   | 2053.27 |2492.62  |2787.87|2799.04|2801.67|
|1024 MB cold start time with SnapStart w/o Priming                | 1626.69  | 1741.10   | 2040.99  |2219.75|2319.54|2321.64|
|1536 MB cold start time with SnapStart w/o Priming  | 1312.14  | 1369.76   | 1703.23 |1753.32|2078.04|2078.96|
|2048 MB cold start time with SnapStart w/o Priming| 1172.00  |1282.32    | 1612.12  |1845.01|2312.60|2313.33|

Let's put this data into graph:

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/7t93xucqfr7qs197acbd.png)

**Cold start time with SnapStart enabled and with DynamoDB invocation Priming in ms :**

| Experiment description                     | p50      | p75      | p90       |p99       | p99.9      | max      
|-----------|----------|-----------|----------|----------|----------|----------|
|256 MB cold start time with SnapStart w/o Priming                | 1135.11  | 1227.14   | 1457.33  |1713.47|1715.19|1716.63|
|512 MB cold start time with SnapStart with Priming  | 862.32  | 935.03  | 1206.47  |1394.62|1398.81|1399.09|
|768 MB cold start time with SnapStart with Priming |839.36   | 918.36  |1107.10  |1248.17|1387.67|1389.05|
|1024 MB cold start time with SnapStart with Priming                | 702.55  | 759.52   | 1038.50  |1169.66|1179.05|1179.36|
|1536 MB cold start time with SnapStart with Priming  | 659.02  | 723.22   | 981.97  |1254.43|1376.62|1377.26|
|2048 MB cold start time with SnapStart with Priming| 687.96|	763.32	|1054.18|1174.35|1184.96|1186.08|

Let's put this data into graph as well:


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/w0wz7ikym4i99xyfbaab.png)

I also measured the deployment time of such sample project with all memory settings mentioned above with SnapStart enabled for all functions and GetProductByIdWithPureJava21LambdaAndPriming function additionally using priming. Here are the results: 

| Experiment description | average deployment time in minutes/seconds|
|-----------|----------|
|256 MB deployment time with SnapStart   |2m 44 s|
|512 MB deployment time with SnapStart   |2m 18 s|
|768 MB deployment time with SnapStart   |2m 17 s|
|1024 MB deployment time with SnapStart  |2m 08 s|
|1536 MB deployment time with SnapStart  |2m 06 s|
|2048 MB deployment time with SnapStart  |2m 06 s|

## Conclusions

In this article we measured the cold start time of the Lambda function without SnapStart with SnapStart and for the latter with additional priming of DynamoDB invocation.

In case of not enabling SnapStart we observed that increasing the memory from 256 MB to 2048 MB constantly brought significant reduction of the cold start time.

In case of enabling of SnapStart but not using priming we observed that increasing the from 256 MB to 1024 MB brought significant reduction of the cold start time but after that the impact of increasing the memory to 1536 MB or even to 2048 MB varied depending on percentile and being significant until p90 and much lower for the percentiles higher than 90.

In case of enabling of SnapStart and with additional priming of DynamoDB invocation we observed that increasing the from 256 MB to 1024 MB brought significant reduction of the cold start time but after that the impact of increasing the memory to 1536 MB or 2048 MB was low for all percentiles. 

So for the SnapStart-enabled Lambda function and considering our use case with Lambda function reading from the DynamoDB table the Lambda memory setting of 1024 MB is a good choice.

Deployment time for the SnapStart enabled on all 3 Lambda functions and additionally applying priming on one of them also became lower by giving the Lambda function more memory.

In the next article I will also provide measurements for the warm execution times for exact this use case, as using different memory settings not only affect the cold start times but also the warm execution times and therefore Lambda costs. Stay tuned!