AWS Lambdaの実装・運用で知っておきたかったこと

# 目的

AWS Labmdaの本格利用を検討して、調査・検証を行ったので、知見をまとめる。

書いてあること

- 運用関連のこと
- コードの実装関連のこと
- cost関連のこと


# 管理

AWS Lambaの運用周りの私見をまとめる。

コードで管理する前から運用してるものがあるので、それらも管理対象としたければ、
aws cli使うか、スクリプト書いて、自前でCICD組むか、lambrollを使っちゃうのが良さそう。


## AWS SAM

公式ではAWSSAMを使うことを推奨している模様。
https://github.com/aws/serverless-application-model

CloudFormationを利用するので、設定ファイルの書き方が独特で、調べにくいので、学習コストが高い。
既に運用済みのLambdaがあった場合、それをコードで管理できるようにする(terraform import的な)ことができなさそう。

テストコードはない

https://docs.aws.amazon.com/ja_jp/serverless-application-model/latest/developerguide/serverless-sam-cli-command-reference.html

ローカルで立ち上げて動作を確認することになりそう。

https://docs.aws.amazon.com/ja_jp/serverless-application-model/latest/developerguide/sam-cli-command-reference-sam-local-invoke.html

定義の間違いは事前に確認できる

https://docs.aws.amazon.com/ja_jp/serverless-application-model/latest/developerguide/sam-cli-command-reference-sam-validate.html

サンプル載せとく

nested application patternの確認
下記のような設定で、1つのconfigでfuncは増やせる。

```yaml
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: >
  sam-test-mukai
 
xxxx omit xxx
 
Resources:
  HelloWorldFunction:
    Type: AWS::Serverless::Function # More info about Function Resource: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction
    Properties:
      CodeUri: hello_world/
      Handler: app.lambda_handler
      Runtime: python3.9
      Architectures:
      - x86_64
      Events:
        HelloWorld:
          Type: Api # More info about API Event Source: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#api
          Properties:
            Path: /hello
            Method: get
  PiyoFunction: ######### ABLE TO ADD second func HERE #########
    Type: AWS::Serverless::Function # More info about Function Resource: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction
    Properties:
      CodeUri: fuga/
      Handler: app.lambda_handler
      Runtime: python3.9
      Architectures:
      - x86_64
      Events:
        HelloWorld:
          Type: Api # More info about API Event Source: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#api
          Properties:
            Path: /fuga
            Method: get
 
xxxx omit xxx
```

削除するときは上記の箇所をけして、build → deploy で関連リソースとともに一緒に消してくれる。

```bash

$ sam build --profile assumed-profile-name
 
xxx omit xxx
 
CloudFormation stack changeset
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Operation                                            LogicalResourceId                                    ResourceType                                         Replacement
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Add                                                ServerlessRestApiDeployment47fc2d5f9d                AWS::ApiGateway::Deployment                          N/A
* Modify                                             ServerlessRestApiProdStage                           AWS::ApiGateway::Stage                               False
* Modify                                             ServerlessRestApi                                    AWS::ApiGateway::RestApi                             False
- Delete                                             PiyoFunctionHelloWorldPermissionProd                AWS::Lambda::Permission                              N/A
- Delete                                             PiyoFunctionRole                                    AWS::IAM::Role                                       N/A
- Delete                                             PiyoFunction                                        AWS::Lambda::Function                                N/A
- Delete                                             ServerlessRestApiDeployment52d2abd56c                AWS::ApiGateway::Deployment                          N/A
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 

Changeset created successfully. arn:aws:***
 
 
2024-02-07 19:56:09 - Waiting for stack create/update to complete

CloudFormation events from stack operations (refresh every 5.0 seconds)
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
ResourceStatus                                       ResourceType                                         LogicalResourceId                                    ResourceStatusReason
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
UPDATE_IN_PROGRESS                                   AWS::CloudFormation::Stack                           sam-test-piyo                                       User Initiated
UPDATE_IN_PROGRESS                                   AWS::ApiGateway::RestApi                             ServerlessRestApi                                    -
UPDATE_COMPLETE                                      AWS::ApiGateway::RestApi                             ServerlessRestApi                                    -
CREATE_IN_PROGRESS                                   AWS::ApiGateway::Deployment                          ServerlessRestApiDeployment47fc2d5f9d                -
CREATE_IN_PROGRESS                                   AWS::ApiGateway::Deployment                          ServerlessRestApiDeployment47fc2d5f9d                Resource creation Initiated
CREATE_COMPLETE                                      AWS::ApiGateway::Deployment                          ServerlessRestApiDeployment47fc2d5f9d                -
UPDATE_IN_PROGRESS                                   AWS::ApiGateway::Stage                               ServerlessRestApiProdStage                           -
UPDATE_COMPLETE                                      AWS::ApiGateway::Stage                               ServerlessRestApiProdStage                           -
UPDATE_COMPLETE_CLEANUP_IN_PROGRESS                  AWS::CloudFormation::Stack                           sam-test-piyo                                       -
DELETE_IN_PROGRESS                                   AWS::Lambda::Permission                              PiyoFunctionHelloWorldPermissionProd                -
DELETE_IN_PROGRESS                                   AWS::ApiGateway::Deployment                          ServerlessRestApiDeployment52d2abd56c                -
DELETE_COMPLETE                                      AWS::Lambda::Permission                              PiyoFunctionHelloWorldPermissionProd                -
DELETE_COMPLETE                                      AWS::ApiGateway::Deployment                          ServerlessRestApiDeployment52d2abd56c                -
DELETE_IN_PROGRESS                                   AWS::Lambda::Function                                PiyoFunction                                        -
DELETE_COMPLETE                                      AWS::Lambda::Function                                PiyoFunction                                        -
DELETE_IN_PROGRESS                                   AWS::IAM::Role                                       PiyoFunctionRole                                    -
DELETE_COMPLETE                                      AWS::IAM::Role                                       PiyoFunctionRole                                    -
UPDATE_COMPLETE                                      AWS::CloudFormation::Stack                           sam-test-piyo                                       -
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 
xxx omit xxx

```

1つのfuncだけ修正したら、１だけ更新される。

```bash
$ sam deploy --guided --profile assumed-profile-name
 
 
xxx omit xxx
 
CloudFormation stack changeset
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Operation                                            LogicalResourceId                                    ResourceType                                         Replacement
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
* Modify                                             PiyoFunction                                        AWS::Lambda::Function                                False
* Modify                                             ServerlessRestApi                                    AWS::ApiGateway::RestApi                             False
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 
 
Changeset created successfully. arn:aws:***
 
 
Previewing CloudFormation changeset before deployment
======================================================
Deploy this changeset? [y/N]: y
 
2024-02-07 20:51:47 - Waiting for stack create/update to complete
 
CloudFormation events from stack operations (refresh every 5.0 seconds)
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
ResourceStatus                                       ResourceType                                         LogicalResourceId                                    ResourceStatusReason
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
UPDATE_IN_PROGRESS                                   AWS::CloudFormation::Stack                           sam-test-piyo                                       User Initiated
UPDATE_IN_PROGRESS                                   AWS::Lambda::Function                                PiyoFunction                                        -
UPDATE_COMPLETE                                      AWS::Lambda::Function                                PiyoFunction                                        -
UPDATE_COMPLETE_CLEANUP_IN_PROGRESS                  AWS::CloudFormation::Stack                           sam-test-piyo                                       -
UPDATE_COMPLETE                                      AWS::CloudFormation::Stack                           sam-test-piyo                                       -
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 
xxx omit xxx
```

## serverless framework

https://github.com/serverless/serverless

serverless の管理まわりの元祖っぽい
SAMの構成と似ていて、SAMはこれを元に作られたんじゃないかという話があった。

メリット・デメリット・使い方もSAMと同じような感じ

terraform import みたいなことができないのがネック。

https://www.serverless.com/framework/docs/tutorial

```bash

❯ sls
(node:46239) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
 
Creating a new serverless project
 
? What do you want to make? (Use arrow keys)
❯ AWS - Node.js - Starter
  AWS - Node.js - HTTP API
  AWS - Node.js - Scheduled Task
  AWS - Node.js - SQS Worker
  AWS - Node.js - Express API
  AWS - Node.js - Express API with DynamoDB
  AWS - Python - Starter
  AWS - Python - HTTP API
  AWS - Python - Scheduled Task
  AWS - Python - SQS Worker
  AWS - Python - Flask API
  AWS - Python - Flask API with DynamoDB
  Other

❯ cat serverless.yml
service: aws-node-project
frameworkVersion: '3'
 
provider:
  name: aws
  runtime: nodejs18.x
  region: ap-northeast-1
  profile: assumed-role-name
 
functions:
  function1:
    handler: index.handler
```

assumed-role 指定できる
https://github.com/serverless/serverless/issues/3833#issuecomment-389739007

```bash
❯ AWS_SDK_LOAD_CONFIG=1 sls deploy --aws-profile=assumed-role-name
(node:41964) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
 
Deploying aws-node-project to stage dev (ap-northeast-1)
 
✔ Service deployed to stack aws-node-project-dev (106s)
 
functions:
  function1: aws-node-project-dev-function1 (1.4 kB)
 
Need a faster logging experience than CloudWatch? Try our Dev Mode in Console: run "serverless dev"
```

### 色々な使い方

```bash

❯ AWS_SDK_LOAD_CONFIG=1 sls invoke --function function1
(node:46864) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
{
    "statusCode": 200,
    "body": "{\n  \"message\": \"Go Serverless v3.0! Your function executed successfully!\",\n  \"input\": {}\n}"
}
 
test/sls-py-tuto/aws-node-project via  v21.1.0 on ☁️  (ap-northeast-1) [10h28m19s] on ☁️   took 4s
❯ AWS_SDK_LOAD_CONFIG=1 sls logs --function function1
(node:46887) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
INIT_START Runtime Version: nodejs:18.v20   Runtime Version ARN: arn:aws:**
START
2024-02-08 12:41:31.521 INFO    hogefuga
END Duration: 3.05 ms (init: 170.44 ms) Memory Used: 66 MB
```

test

https://www.serverless.com/framework/docs/guides/testing

> Then, when you run sls test (your function must already have been deployed) it will make an HTTP request against the hello function and pass if the response has a status code of 200:

monitor

```bash

❯ AWS_SDK_LOAD_CONFIG=1 serverless metrics
(node:51784) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
Service wide metrics
February 7, 2024 2:00 PM - February 8, 2024 2:00 PM
 
invocations: 2
throttles: 0
errors: 0
duration (avg.): 3.5ms
```

## Nx

CI/CDに特化していそうな印象。
ただ、これもSAMに似たような感じになっていたので、ちゃんとは検証してない。

また、aws lambdaのサポートが終わりそう？
AWS SAM のwrapperで、cacheとかを使うっぽい？

https://www.npmjs.com/package/@nx/aws-lamb

## lambroll

https://github.com/fujiwara/lambroll

terraform import みたいなことができるのでこれ一択。

```bash

$ lambroll init --function-name=mukai-test --download --profile=assumed-profile
2024/02/08 22:14:32 [info] lambroll v0.14.7 with function.json
2024/02/08 22:14:34 [info] function mukai-test found
2024/02/08 22:14:34 [info] downloading function.zip
2024/02/08 22:14:34 [info] creating .lambdaignore
Overwrite existing file .lambdaignore? (y/n) [n]: y
2024/02/08 22:14:36 [info] creating function.json
import json
Overwrite existing file function.json? (y/n) [n]: y
2024/02/08 22:14:37 [info] completed
 
cat function.json
{
  "Architectures": [
    "x86_64"
  ],
  "Description": "",
  "EphemeralStorage": {
    "Size": 512
  },
  "FunctionName": "hoge-test",
  "Handler": "lambda_function.lambda_handler",
  "LoggingConfig": {
    "LogFormat": "Text",
    "LogGroup": "/aws/lambda/hoge-test"
  },
  "MemorySize": 128,
  "Role": "arn:aws:**",
  "Runtime": "python3.12"
 
xxx
 
$ ls
function.json   function.zip

$ unzip function.zip
Archive:  function.zip
 extracting: lambda_function.py
 
$ lambroll deploy --function=function.json --src="." --profile=assumed-profile
2024/02/08 22:16:33 [info] lambroll v0.14.7 with function.json
2024/02/08 22:16:33 [info] starting deploy function mukai-test
2024/02/08 22:16:35 [info] creating zip archive from .
2024/02/08 22:16:35 [info] zip archive wrote 307 bytes
2024/02/08 22:16:35 [info] updating function configuration
2024/02/08 22:16:35 [info] State:Active LastUpdateStatus:Successful
2024/02/08 22:16:35 [info] updated function configuration successfully
2024/02/08 22:16:35 [info] updating function code
2024/02/08 22:16:35 [info] State:Active LastUpdateStatus:InProgress
2024/02/08 22:16:35 [info] waiting for LastUpdateStatus Successful
2024/02/08 22:16:36 [info] State:Active LastUpdateStatus:Successful
2024/02/08 22:16:37 [info] update function code request was accepted
2024/02/08 22:16:38 [info] State:Active LastUpdateStatus:InProgress
2024/02/08 22:16:38 [info] waiting for LastUpdateStatus Successful
2024/02/08 22:16:39 [info] State:Active LastUpdateStatus:InProgress
2024/02/08 22:16:39 [info] waiting for LastUpdateStatus Successful
2024/02/08 22:16:41 [info] State:Active LastUpdateStatus:InProgress
2024/02/08 22:16:41 [info] waiting for LastUpdateStatus Successful
2024/02/08 22:16:45 [info] State:Active LastUpdateStatus:Successful
2024/02/08 22:16:45 [info] updated function code successfully
2024/02/08 22:16:45 [info] deployed version 1
2024/02/08 22:16:45 [info] updating alias set current to version 1
2024/02/08 22:16:45 [info] alias current is not found. creating alias
2024/02/08 22:16:45 [info] alias updated
2024/02/08 22:16:45 [info] completed
```

# 実装

AWS Lambda自体を調査し、処理の高速化につながるものや注意点等をまとめる。

https://docs.aws.amazon.com/lambda/latest/dg/foundation-progmodel.html

> The function's class stays in memory, so clients and variables that are declared outside of the handler method in initialization code can be reused. To save processing time on subsequent events, create reusable resources like AWS SDK clients during initialization. Once initialized, each instance of your function can process thousands of requests.

LambdaFunction class はメモリに残り続けるので、初期化時になるべく、使いまわせるものをメモリ上に残す実装にすると、次回の実行時により早くなる。

https://docs.aws.amazon.com/lambda/latest/dg/lambda-runtime-environment.html

> The 10-second timeout doesn't apply to functions that are using provisioned concurrency or SnapStart. For provisioned concurrency and SnapStart functions, your initialization code can run for up to 15 minutes. The time limit is 130 seconds or the configured function timeout (maximum 900 seconds), whichever is higher.

初期化されたコードは最大で15min実行され続ける。

Timeoutの設定で管理すると良い。

> For example, if your account has a concurrency limit of 1,000, you cannot reserve all 1,000 units of concurrency to a single function.

アカウントが作成しているLambdaFunction全体の同時実行数を予約することになるので、それを考慮する必要がある。

> Reserved concurrency – This represents the maximum number of concurrent instances allocated to your function. When a function has reserved concurrency, no other function can use that concurrency. Configuring reserved concurrency for a function incurs no additional charges.

> Provisioned concurrency – This is the number of pre-initialized execution environments allocated to your function. These execution environments are ready to respond immediately to incoming function requests. Configuring provisioned concurrency incurs additional charges to your AWS account.

Reserved concurrencyはただ、台数を割り当てるだけ。

Provisioned concurrencyは事前に初期化された環境を用意するので、リクエストをすぐに捌ける。

今回のような動的リサイズにはProvisioned concurrencyの設定を入れる方が良い。

https://aws.amazon.com/lambda/pricing/

![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/315979/3c2f9f57-1030-99bc-f636-ad7474bb9b3f.png)


リソースの割り当て大きくして、Provisioned concurrencyあげると、料金が高くなる。

https://docs.aws.amazon.com/lambda/latest/dg/provisioned-concurrency.html#managing-provisioned-concurency

> With Application Auto Scaling, you can set your own scaling schedule according to predictable load changes.

オートスケールの設定を入れられる。


https://docs.aws.amazon.com/lambda/latest/dg/best-practices.html#function-code

> Subsequent invocations processed by the same instance of your function can reuse these resources. This saves cost by reducing function run time.

リソースを使い回すことがあるので、初期化の際に適切にオンメモリにしとくと良い

> To maintain your persistent connection, use the keep-alive directive associated with your runtime.

keep-alive を設定すると良い

> This will reduce the amount of time that it takes for your deployment package to be downloaded and unpacked ahead of invocation. 

packageを小さくすると起動が早い

https://docs.aws.amazon.com/lambda/latest/dg/lambda-rolling-deployments.html

AWSCodeDeployやAWS SAMを使わないとRolloing Deploymentができないので、デプロイ時に起動が遅くなる可能性がある。

https://docs.aws.amazon.com/lambda/latest/dg/gettingstarted-limits.html

![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/315979/efb7a39c-a90f-58cd-c1ce-206c3086ba04.png)


https://docs.aws.amazon.com/servicequotas/latest/userguide/request-quota-increase.html

> Larger increase requests will take time to review, process, approve, and deploy. You can track your request case in the AWS Support console. Requests to increase service quotas don't receive priority support. If you have an urgent request, contact AWS Support.

同時実行数の上限の増加申請はAWS Support consoleから


# 最後に

検証させてくれたチームと会社に感謝