AWS CDK - EFS での Python Lambda パッケージ管理

# 内容

[AWS CDK - Python Lambda Layer デプロイ](https://qiita.com/takmot/items/8c6aa3961f93102e2316) で、  
外部パッケージを Layer や Lambda に組み込んだが、  
Lambda に 250MB のサイズ制限があるため、  
pandas 等、重いパッケージを入れるとすぐに制限に引っかかる  
そのため、 EFS に外部パッケージをインストールし、Lambda から使用することでこれを回避する  
EFS への外部パッケージインストールは ECS を使用する  
また、CDKデプロイで自動的に環境を構築したいので、  
Custom Resource で ECS Task を起動する  

# 実装

## VPC

適当な VPC 環境一式と、  
EFS と Lambda に割り当てる Security Group　を作成し、  
インバウンドルールで Lambda から EFS へのアクセスを許可している  

```typescript:CDK実装
    // VPC
    const vpc = new ec2.Vpc(this, `${stackName}-Vpc`, {
      ipAddresses: ec2.IpAddresses.cidr('10.0.0.0/16'),
      maxAzs: 2,
      subnetConfiguration: [
        { name: 'Public', subnetType: ec2.SubnetType.PUBLIC, cidrMask: 24 },
        {
          name: 'Private',
          subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS,
          cidrMask: 24,
        },
        {
          name: 'Isolated',
          subnetType: ec2.SubnetType.PRIVATE_ISOLATED,
          cidrMask: 24,
        },
      ],
    });

    const privateSubnets = vpc.selectSubnets({
      subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS,
    });;

    // Security Group for client
    const clientSg = new ec2.SecurityGroup(this, `${stackName}-ClientSg`, {
      vpc,
      securityGroupName: `${stackName}-ClientSg`,
      allowAllOutbound: true,
    });

    // Security Group for EFS
    const efsSg = new ec2.SecurityGroup(this, `${stackName}-EfsSg`, {
      vpc,
      securityGroupName: `${stackName}-EfsSg`,
      allowAllOutbound: true,
    });
    efsSg.addIngressRule(ec2.Peer.securityGroupId(clientSg.securityGroupId), ec2.Port.tcp(2049));
```

## EFS

```typescript:CDK実装
    // EFS
    const efsFileSystem = new efs.FileSystem(this, `${stackName}-EfsFileSystem`, {
      vpc,
      fileSystemName: `${stackName}-EfsFileSystem`,
      securityGroup: efsSg,
      vpcSubnets: privateSubnets,
    });

    const efsAccessPoint = new efs.AccessPoint(this, `${stackName}-EfsAccessPoint`, {
      fileSystem: efsFileSystem,
      createAcl: {
        ownerUid: '1001',
        ownerGid: '1001',
        permissions: '755',
      },
      path: `/${stackName}`,
      posixUser: {
        uid: '1001',
        gid: '1001',
      },
    });
```

## Lambda

VPC Lambda に必要なポリシー(`service-role/AWSLambdaVPCAccessExecutionRole`)、  
EFS アクセスに必要なポリシー（`AmazonElasticFileSystemClientReadWriteAccess`）をアタッチする  

```typescript:CDK実装
    // Lambda
    const efsMountPath = '/mnt/efs';
    const sampleFunctionName = `${stackName}-sampleFunction`;
    const sampleFunctionRole = new iam.Role(this, `${sampleFunctionName}Role`, {
      roleName: `${sampleFunctionName}Role`,
      assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'),
      managedPolicies: [
        iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'),
        iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaVPCAccessExecutionRole'),
        iam.ManagedPolicy.fromAwsManagedPolicyName('AmazonElasticFileSystemClientReadWriteAccess'),
      ],
    });

    new lambda.Function(this, sampleFunctionName, {
      code: lambda.Code.fromAsset(path.join(__dirname, 'handlers/sample-func')),
      handler: 'index.handler',
      runtime: lambda.Runtime.PYTHON_3_11,
      environment: {
        /* eslint-disable @typescript-eslint/naming-convention */
        EFS_MOUNT_PATH: efsMountPath,
        /* eslint-enable @typescript-eslint/naming-convention */
      },
      filesystem: lambda.FileSystem.fromEfsAccessPoint(efsAccessPoint, efsMountPath),
      functionName: sampleFunctionName,
      logRetention: logs.RetentionDays.ONE_MONTH,
      role: sampleFunctionRole,
      securityGroups: [clientSg],
      timeout: Duration.seconds(30),
      tracing: lambda.Tracing.ACTIVE,
      vpc,
      vpcSubnets: privateSubnets,
    });
```

Lambda 関数では、  
`sys.path.append` で、マウントした EFS を import パスに追加する  

```python:handlers/sample-func/index.py
import os
import sys

efs_mount_path = os.getenv("EFS_MOUNT_PATH")
if type(efs_mount_path) is str:
    sys.path.append(efs_mount_path)

import pandas as pd  # type: ignore


def handler(event, context):
    df = pd.DataFrame(data=["a1", "a2", "a3"])
    print(df)
```

## ECS

ECS に EFS をマウントし、  
EFS 上にPythonパッケージのインストール（`pip install -r requirements.txt -t ${efsMountPath}`）を行うタスクを定義する  

```typescript:CDK実装
    // ECR
    const dockerImageAsset = new ecrAssets.DockerImageAsset(this, `${stackName}-EcrDockerImage`, {
      directory: `${__dirname}/docker`,
    });

    // ECS
    const ecsTaskExecutionRole = new iam.Role(this, `${stackName}-EcsTaskExecutionRole`, {
      assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'),
      managedPolicies: [iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AmazonECSTaskExecutionRolePolicy')],
      roleName: `${stackName}-EcsTaskExecutionRole`,
    });

    const ecsTaskRole = new iam.Role(this, `${stackName}-EcsTaskRole`, {
      assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'),
      managedPolicies: [iam.ManagedPolicy.fromAwsManagedPolicyName('AmazonElasticFileSystemClientReadWriteAccess')],
      roleName: `${stackName}-EcsTaskRole`,
    });

    const cluster = new ecs.Cluster(this, `${stackName}-EcsCluster`, {
      clusterName: `${stackName}-EcsCluster`,
      containerInsights: true,
      vpc,
    });

    const ecsCpu = 256;
    const ecsMemory = 512;
    const taskVolumeName = 'efs-task-volume';
    const taskDefinition = new ecs.FargateTaskDefinition(this, `${stackName}-FargateTaskDefinition`, {
      cpu: ecsCpu,
      executionRole: ecsTaskExecutionRole,
      memoryLimitMiB: ecsMemory,
      taskRole: ecsTaskRole,
    });

    taskDefinition.addVolume({
      name: taskVolumeName,
      efsVolumeConfiguration: {
        fileSystemId: efsFileSystem.fileSystemId,
        authorizationConfig: {
          accessPointId: efsAccessPoint.accessPointId,
        },
        transitEncryption: 'ENABLED',
      },
    });

    const modulePackagingContainer = taskDefinition.addContainer(`${stackName}-ModulePackagingContainer`, {
      image: ecs.ContainerImage.fromEcrRepository(dockerImageAsset.repository, dockerImageAsset.assetHash),
      command: [`pip install -r requirements.txt -t ${efsMountPath}`],
      cpu: ecsCpu,
      logging: ecs.LogDrivers.awsLogs({
        streamPrefix: `${stackName}-ModulePackagingContainer`,
      }),
      memoryLimitMiB: ecsMemory,
    });

    modulePackagingContainer.addMountPoints({
      containerPath: efsMountPath,
      readOnly: false,
      sourceVolume: taskVolumeName,
    });
```

Lambda と同環境のイメージをベースに `requirements.txt` を配置する  

```dockerfile:docker/Dockerfile
FROM public.ecr.aws/lambda/python:3.11

RUN yum update -y

WORKDIR /app
COPY requirements.txt ./

ENTRYPOINT ["/bin/bash", "-l", "-c"]
```

```yaml:docker/docker-compose.yml
version: '3.8'
services:
  python-lambda-packages:
    container_name: python-lambda-packages
    build:
      context: .
      dockerfile: Dockerfile
```

```text:docker/requirements.txt
pandas==2.1.1
```

## Custom Resource

[AwsCustomResource](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.custom_resources.AwsCustomResource.html) を使用する  
`onCreate`, `onUpdate` に指定する action は、  
[@aws-sdk/client-ecs](https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/clients/client-ecs/#aws-sdkclient-ecs) の [Client Commands (Operations List)](https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/clients/client-ecs/#client-commands-operations-list) より選択、  
parameters は、[RunTaskCommand](https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/clients/client-ecs/classes/runtaskcommand.html#input-79) のパラメータに従って設定する  

```typescript:CDK実装
    // Custom Resource
    const ecsTaskRunFunctionName = `${stackName}-EcsTaskRunFunction`;
    const ecsTaskRunFunctionRole = new iam.Role(this, `${ecsTaskRunFunctionName}Role`, {
      assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'),
      inlinePolicies: {
        passrole: new iam.PolicyDocument({
          statements: [
            new iam.PolicyStatement({
              actions: ['iam:GetRole', 'iam:PassRole'],
              resources: [ecsTaskExecutionRole.roleArn, ecsTaskRole.roleArn],
            }),
          ],
        }),
        ecsPolicy: new iam.PolicyDocument({
          statements: [
            new iam.PolicyStatement({
              actions: ['ecs:RunTask'],
              resources: [taskDefinition.taskDefinitionArn],
            }),
          ],
        }),
      },
      managedPolicies: [iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaVPCAccessExecutionRole')],
      roleName: `${ecsTaskRunFunctionName}Role`,
    });

    const ecsTaskRunFunctionContent = {
      action: 'RunTask',
      service: 'ECS',
      parameters: {
        cluster: cluster.clusterArn,
        launchType: 'FARGATE',
        networkConfiguration: {
          awsvpcConfiguration: {
            subnets: privateSubnets.subnetIds,
            securityGroups: [clientSg.securityGroupId],
            assignPublicIp: 'DISABLED',
          },
        },
        taskDefinition: taskDefinition.taskDefinitionArn,
      },
      physicalResourceId: cr.PhysicalResourceId.of(`${stackName}-EcsTaskRunCustomResource`),
    };

    new cr.AwsCustomResource(this, `${stackName}-EcsTaskRunCustomResource`, {
      functionName: ecsTaskRunFunctionName,
      logRetention: logs.RetentionDays.ONE_MONTH,
      onCreate: ecsTaskRunFunctionContent,
      onUpdate: ecsTaskRunFunctionContent,
      role: ecsTaskRunFunctionRole,
      vpc,
      vpcSubnets: privateSubnets,
    });
```

# EC2 で EFS 内の確認

EC2 に ssh で接続する方法は [EC2を使用したRDSに対するssh操作](https://qiita.com/takmot/items/bc1569c05728d43e772c#ec2%E3%82%92%E4%BD%BF%E7%94%A8%E3%81%97%E3%81%9Frds%E3%81%AB%E5%AF%BE%E3%81%99%E3%82%8Bssh%E6%93%8D%E4%BD%9C) にも記載しているため割愛  
EFS をマウントするために amazon-efs-utils ツールをインストールする必要がある  

```bash
sudo yum update
sudo yum install -y amazon-efs-utils
sudo mkdir /mnt/efs
sudo mount -t efs {ファイルシステムID}:/ /mnt/efs
```

# 参考

- [【AWS Lambda】250MB制限を乗り越えてPyTorchを導入する方法](https://valmore.work/aws-labmda-pytorch/)
- [EFSにセットアップしたPythonライブラリをLambdaにimportする方法](https://qiita.com/r_tamura/items/2edf203e1bf8d52f5caa)
- [AWS LambdaでLambda Layersを使う場合、環境変数のPYTHONPATHは使っちゃダメという話](https://dev.classmethod.jp/articles/lambdalayers-should-not-use-pythonpath/)
- [ECS(Fargate)でEFSをマウントさせてみた(2023年版)](https://dev.classmethod.jp/articles/202310-ecs-efs-01/)
- [Python の AWS ベースイメージを使用する](https://docs.aws.amazon.com/ja_jp/lambda/latest/dg/python-image.html#python-image-instructions)
- [ECS: EFS ボリュームをマウントする](https://github.com/aws/aws-cdk/issues/13442)
- [[AWS CDK] APIを呼び出すだけのカスタムリソースならLambda関数は不要な件](https://dev.classmethod.jp/articles/create-custom-resources-with-aws-cdk-without-using-lambda-functions/)