Pose検出するマイクロサービスを作成する

# 概要

Pose検出するマイクロサービスを作成しようとしたところ、意外と苦戦したのでメモ。

Edgeの処理を軽くするため、サーバーサイド（Lambda）でMediaPipeを使用したPose検出を行うマイクロサービスを作成して呼び出そうとした。

# Dockerイメージの作成

Zipの制限に収まらないので、DockerイメージでLambda関数を作成する。

```dockerfile:Dockerfile
# AWS提供ベースイメージ
FROM public.ecr.aws/lambda/python:3.8

COPY app.py ${LAMBDA_TASK_ROOT}

RUN yum -y install mesa-libGL.x86_64 make gcc curl

COPY requirements.txt  .
RUN  pip3 install -r requirements.txt --target "${LAMBDA_TASK_ROOT}"

# モデルをダウンロード
RUN mkdir -p //var/task/mediapipe/modules/pose_landmark/ \
    && curl -SL https://github.com/google/mediapipe/blob/master/mediapipe/modules/pose_landmark/pose_landmark_heavy.tflite?raw=true > /var/task/mediapipe/modules/pose_landmark/pose_landmark_heavy.tflite

# pthread problem workaround
ADD https://raw.githubusercontent.com/mitchellharper12/lambda-pthread-nameshim/master/Makefile .
ADD https://raw.githubusercontent.com/mitchellharper12/lambda-pthread-nameshim/master/pthread_shim.c .
RUN make pthread_shim.so && cp pthread_shim.so /opt

# Set the CMD to your handler (could also be done as a parameter override outside of the Dockerfile)
CMD [ "app.handler" ]

```

```python:app.py
import logging
import sys
import time
import json

import base64
import cv2
import mediapipe as mp
import numpy as np

mp_drawing = mp.solutions.drawing_utils
mp_drawing_styles = mp.solutions.drawing_styles
mp_pose = mp.solutions.pose

def handler(event, context):
  print(event)

  with mp_pose.Pose(
    static_image_mode=True,
    model_complexity=2,
    enable_segmentation=True,
    min_detection_confidence=0.5) as pose:

    img_data = base64.b64decode(event['body'])
    img_np = np.fromstring(img_data, np.uint8)
    image = cv2.imdecode(img_np, cv2.IMREAD_ANYCOLOR)
    image_height, image_width, _ = image.shape
    results = pose.process(cv2.cvtColor(image, cv2.COLOR_BGR2RGB))

    if not results.pose_landmarks:
      return '{"message": "no result"}'

    arr = []
    for index in range(len(mp_pose.PoseLandmark)):
      arr.append({'id':index, 'name':mp_pose.PoseLandmark(index).name, 'x':results.pose_landmarks.landmark[mp_pose.PoseLandmark(index)].x, 'y':results.pose_landmarks.landmark[mp_pose.PoseLandmark(index)].y})
    return json.dumps({'results': arr, 'height': image_height, 'width': image_width})

```

```text:requirements.txt
opencv-python
mediapipe

```

ビルドする。

```bash:
docker build -t test-pose .
```

テストのため実行する。

```bash:
docker run -p=8080:8080 test-pose:latest
```

うまくビルドできたか試してみる

```bash:
curl -X POST "http://localhost:8080/2015-03-31/functions/function/invocations" -d '{"body":""}'
```

画像データが空なのでエラーが出力される。

```text:
{"errorMessage": "OpenCV(4.5.5) /io/opencv/modules/imgcodecs/src/loadsave.cpp:816: error: (-215:Assertion failed) !buf.empty() in function 'imdecode_'\n", "errorType": "error", "stackTrace": ["  File \"/var/task/app.py\", line 26, in handler\n    image = cv2.imdecode(img_np, cv2.IMREAD_ANYCOLOR)\n"]}
```

# Lambda作成の準備

作成したイメージをECRに登録する。最初にタグ付けする。

この例では xxxxxxxxxxxx はアカウントID、リージョンは　ａｐ－ｎｏｒｔｈｅａｓｔ－１　を使用。

```bash:
docker tag test-pose:latest xxxxxxxxxxxx.dkr.ecr.ap-northeast-1.amazonaws.com/test-pose:latest
```

ECRにログイン

```bash:
aws ecr get-login-password | docker login --username AWS --password-stdin xxxxxxxxxxxx.dkr.ecr.ap-northeast-1.amazonaws.com
```

プッシュする

```bash:
aws ecr create-repository --repository-name test-pose --image-scanning-configuration scanOnPush=true --image-tag-mutability MUTABLE

docker push xxxxxxxxxxxx.dkr.ecr.ap-northeast-1.amazonaws.com/test-pose:latest
```

# Lambdaの作成

関数の作成

![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/555397/c395133d-09c8-2bea-1242-1796e63f20a8.png)

作成後にメモリ、タイムアウトを設定

![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/555397/55fbd609-a8cb-1188-b029-db0093cfeac8.png)

新機能、関数URLを作成

![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/555397/b34d6d5f-ad31-ebd6-1b60-87670b284159.png)

![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/555397/f7dfcd98-297e-892b-dd4b-c150fd73aec9.png)

環境変数の設定

![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/555397/c0bfb8c3-6a12-81b5-1fb9-27e2fd1e100f.png)

# テスト

画像をアップロードする

```
curl -X POST -H 'Content-Type: image/png' --data-binary @pose.png https://12345678901234567890123456789012.lambda-url.ap-northeast-1.on.aws/

```

結果

```json:
{
	"results": [
		{
			"id": 0,
			"name": "NOSE",
			"x": 0.5232403874397278,
			"y": 0.12018030881881714
		},
		{
			"id": 1,
			"name": "LEFT_EYE_INNER",
			"x": 0.5452666282653809,
			"y": 0.10341935604810715
		},
		{
			"id": 2,
			"name": "LEFT_EYE",
			"x": 0.5573577880859375,
			"y": 0.1040365993976593
		},
		{
			"id": 3,
			"name": "LEFT_EYE_OUTER",
			"x": 0.5683303475379944,
			"y": 0.10525256395339966
		},
		{
			"id": 4,
			"name": "RIGHT_EYE_INNER",
			"x": 0.504645824432373,
			"y": 0.10398414731025696
		},
		{
			"id": 5,
			"name": "RIGHT_EYE",
			"x": 0.4921768307685852,
			"y": 0.10489142686128616
		},
		{
			"id": 6,
			"name": "RIGHT_EYE_OUTER",
			"x": 0.48190298676490784,
			"y": 0.10615649819374084
		},
		{
			"id": 7,
			"name": "LEFT_EAR",
			"x": 0.5838826894760132,
			"y": 0.11600765585899353
		},
		{
			"id": 8,
			"name": "RIGHT_EAR",
			"x": 0.46655985713005066,
			"y": 0.11607295274734497
		},
		{
			"id": 9,
			"name": "MOUTH_LEFT",
			"x": 0.5485259294509888,
			"y": 0.1413123607635498
		},
		{
			"id": 10,
			"name": "MOUTH_RIGHT",
			"x": 0.50005042552948,
			"y": 0.1397590935230255
		},
		{
			"id": 11,
			"name": "LEFT_SHOULDER",
			"x": 0.6766172051429749,
			"y": 0.2485317587852478
		},
		{
			"id": 12,
			"name": "RIGHT_SHOULDER",
			"x": 0.3809528350830078,
			"y": 0.24369803071022034
		},
		{
			"id": 13,
			"name": "LEFT_ELBOW",
			"x": 0.6916986107826233,
			"y": 0.3813675045967102
		},
		{
			"id": 14,
			"name": "RIGHT_ELBOW",
			"x": 0.34289777278900146,
			"y": 0.37505054473876953
		},
		{
			"id": 15,
			"name": "LEFT_WRIST",
			"x": 0.706161618232727,
			"y": 0.49870720505714417
		},
		{
			"id": 16,
			"name": "RIGHT_WRIST",
			"x": 0.330485463142395,
			"y": 0.4918277859687805
		},
		{
			"id": 17,
			"name": "LEFT_PINKY",
			"x": 0.7168028354644775,
			"y": 0.5329775810241699
		},
		{
			"id": 18,
			"name": "RIGHT_PINKY",
			"x": 0.3190672993659973,
			"y": 0.5271245837211609
		},
		{
			"id": 19,
			"name": "LEFT_INDEX",
			"x": 0.7006966471672058,
			"y": 0.5377188324928284
		},
		{
			"id": 20,
			"name": "RIGHT_INDEX",
			"x": 0.34219032526016235,
			"y": 0.5301030874252319
		},
		{
			"id": 21,
			"name": "LEFT_THUMB",
			"x": 0.6904131174087524,
			"y": 0.5276884436607361
		},
		{
			"id": 22,
			"name": "RIGHT_THUMB",
			"x": 0.35810160636901855,
			"y": 0.5179416537284851
		},
		{
			"id": 23,
			"name": "LEFT_HIP",
			"x": 0.5979184508323669,
			"y": 0.5145007967948914
		},
		{
			"id": 24,
			"name": "RIGHT_HIP",
			"x": 0.43190252780914307,
			"y": 0.511218786239624
		},
		{
			"id": 25,
			"name": "LEFT_KNEE",
			"x": 0.5888926386833191,
			"y": 0.704571545124054
		},
		{
			"id": 26,
			"name": "RIGHT_KNEE",
			"x": 0.4408313035964966,
			"y": 0.702862024307251
		},
		{
			"id": 27,
			"name": "LEFT_ANKLE",
			"x": 0.563112199306488,
			"y": 0.877585768699646
		},
		{
			"id": 28,
			"name": "RIGHT_ANKLE",
			"x": 0.48251473903656006,
			"y": 0.8749133348464966
		},
		{
			"id": 29,
			"name": "LEFT_HEEL",
			"x": 0.5614821910858154,
			"y": 0.8919190168380737
		},
		{
			"id": 30,
			"name": "RIGHT_HEEL",
			"x": 0.49692264199256897,
			"y": 0.8878955841064453
		},
		{
			"id": 31,
			"name": "LEFT_FOOT_INDEX",
			"x": 0.5484899282455444,
			"y": 0.9480408430099487
		},
		{
			"id": 32,
			"name": "RIGHT_FOOT_INDEX",
			"x": 0.4944060742855072,
			"y": 0.9436366558074951
		}
	],
	"height": 858,
	"width": 482
}
```


# 参考

- https://github.com/google/mediapipe
- https://github.com/mitchellharper12/lambda-pthread-nameshim