pytorch-openposeで姿勢推定（MacOS CPU）

##今回やったこと

![スクリーンショット 2021-07-19 21.18.33.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/1754134/82c787cf-1bf5-00ca-4536-e5f5156716f3.png)

![スクリーンショット 2021-07-19 21.36.06.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/1754134/40f78854-66ca-f741-8875-6cf699206789.png)

##*Pytorch-openpose*を利用

当初、次のリポジトリの資源をgit cloneしようとしたが、__*tesorflow=1.4.1*が入れられなかった__ため、利用を諦めた。

- https://github.com/jiajunhua/ildoonet-tf-pose-estimation

変わりに、PyTorch実装の資源を次から入れることにした。

- https://github.com/Hzzone/pytorch-openpose

```bash:Terminal
electron@diynoMacBook-Pro Desktop % git clone https://github.com/Hzzone/pytorch-openpose.git
Cloning into 'pytorch-openpose'...
remote: Enumerating objects: 154, done.
remote: Counting objects: 100% (3/3), done.
remote: Compressing objects: 100% (3/3), done.
remote: Total 154 (delta 0), reused 0 (delta 0), pack-reused 151
Receiving objects: 100% (154/154), 20.18 MiB | 13.12 MiB/s, done.
Resolving deltas: 100% (68/68), done.
electron@diynoMacBook-Pro Desktop % 
```

```bash:Terminal
electron@diynoMacBook-Pro Desktop % cd pytorch-openpose 
electron@diynoMacBook-Pro pytorch-openpose % ls
README.md		demo_camera.py		images			notebooks		src
demo.py			demo_video.py		model			requirements.txt
electron@diynoMacBook-Pro pytorch-openpose % 
```

以下から、body_pose_model.pth	hand_pose_model.pthをダウンロード

- https://www.dropbox.com/sh/7xbup2qsn7vvjxo/AABWFksdlgOMXR_r5v3RwKRYa?dl=0

両ファイルを、./model直下に移動

```bash:Terminal    
electron@diynoMacBook-Pro pytorch-openpose % ls model
electron@diynoMacBook-Pro pytorch-openpose % ls *.pth
body_pose_model.pth	hand_pose_model.pth
electron@diynoMacBook-Pro pytorch-openpose % 
electron@diynoMacBook-Pro pytorch-openpose % mv *.pth ./model/
electron@diynoMacBook-Pro pytorch-openpose % ls *.pth         
zsh: no matches found: *.pth
electron@diynoMacBook-Pro pytorch-openpose %
```

```bash:Terminal    
electron@diynoMacBook-Pro pytorch-openpose % ls model         
body_pose_model.pth	hand_pose_model.pth
electron@diynoMacBook-Pro pytorch-openpose %
```

```bash:Terminal
electron@diynoMacBook-Pro pytorch-openpose % python3 demo_camera.py                                                           
Traceback (most recent call last):
  File "/Users/electron/Desktop/pytorch-openpose/demo_camera.py", line 5, in <module>
    import torch
  File "/usr/local/lib/python3.9/site-packages/torch/__init__.py", line 196, in <module>
    from torch._C import *
RuntimeError: module compiled against API version 0xe but this version of numpy is 0xd
electron@diynoMacBook-Pro pytorch-openpose % 
```

####以下のエラーメッセージが出た。

> RuntimeError: module compiled against API version 0xe but this version of numpy is 0xd

Googleで調べると、*numpy*のバージョンに問題があるらしい。
*requirement.txt*に列挙されている*numpu*のバージョン番号を見てみる。

```bash:Terminal
electron@diynoMacBook-Pro pytorch-openpose % cat requirements.txt 
numpy
matplotlib
opencv-python
scipy
scikit-image
tqdm%   
electron@diynoMacBook-Pro pytorch-openpose %
```

バージョンは特に指定されていない。

以下のサイトから、test.pyを写経。

- https://self-development.info/openpose%E3%81%A7%E5%A7%BF%E5%8B%A2%E6%8E%A8%E5%AE%9A%E3%82%92%E8%A1%8C%E3%81%86%E3%80%90pytorch-openpose%E3%80%91/

```Python3:test.py
import os
import cv2
import copy
from src import util
from src.body import Body
 
INPUT_FILE_NAME = "test.jpg"
 
if __name__ == "__main__":
 
    body_estimation = Body('model/body_pose_model.pth')
 
    target_image_path = 'images/' + INPUT_FILE_NAME
    oriImg = cv2.imread(target_image_path)  # B,G,R order
    candidate, subset = body_estimation(oriImg)
    canvas = copy.deepcopy(oriImg)
    canvas = util.draw_bodypose(canvas, candidate, subset)
 
    basename_name = os.path.splitext(os.path.basename(target_image_path))[0]
 
    result_image_path = "result/pose_" + basename_name + ".jpg"
    cv2.imwrite(result_image_path, canvas)

```

imageファイルをimages直下に格納。
ファイル名は、pose_test.jpg 

![スクリーンショット 2021-07-19 21.16.40.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/1754134/889073b2-9166-2a32-bcf8-dff189828db5.png)

```bash:Terminal    
electron@diynoMacBook-Pro pytorch-openpose % ls images | grep test 
test.jpg
electron@diynoMacBook-Pro pytorch-openpose %
```

####やはり同じエラーが出る。

```bash:Terminal    
electron@diynoMacBook-Pro pytorch-openpose % python3 test.py
Traceback (most recent call last):
  File "/Users/electron/Desktop/pytorch-openpose/test.py", line 5, in <module>
    from src.body import Body
  File "/Users/electron/Desktop/pytorch-openpose/src/body.py", line 8, in <module>
    import torch
  File "/usr/local/lib/python3.9/site-packages/torch/__init__.py", line 196, in <module>
    from torch._C import *
RuntimeError: module compiled against API version 0xe but this version of numpy is 0xd
electron@diynoMacBook-Pro pytorch-openpose % 
```

###Python 3.7.0に切り替える

```bash:Terminal    
electron@diynoMacBook-Pro pytorch-openpose % pyenv local 3.7.0
electron@diynoMacBook-Pro pytorch-openpose % pyenv versions   
  system
  2.7.16
  3.6.0
  3.6.1
  3.6.3
  3.6.3/envs/gpt2_ja
  3.6.6
* 3.7.0 (set by /Users/electron/Desktop/pytorch-openpose/.python-version)
  3.7.4
  3.9.0
  gpt2_ja
electron@diynoMacBook-Pro pytorch-openpose % 
```

改めて、依存ライブラリをinstall

```bash:Terminal    
electron@diynoMacBook-Pro pytorch-openpose % pip3 install -r requirements.txt
Requirement already satisfied: numpy in /usr/local/lib/python3.9/site-packages (from -r requirements.txt (line 1)) (1.19.5)
Requirement already satisfied: matplotlib in /usr/local/lib/python3.9/site-packages (from -r requirements.txt (line 2)) (3.4.2)
Requirement already satisfied: opencv-python in /usr/local/lib/python3.9/site-packages (from -r requirements.txt (line 3)) (4.5.3.56)
Requirement already satisfied: scipy in /usr/local/lib/python3.9/site-packages (from -r requirements.txt (line 4)) (1.7.0)
Requirement already satisfied: scikit-image in /usr/local/lib/python3.9/site-packages (from -r requirements.txt (line 5)) (0.18.2)
Requirement already satisfied: tqdm in /usr/local/lib/python3.9/site-packages (from -r requirements.txt (line 6)) (4.61.2)
Requirement already satisfied: pillow>=6.2.0 in /usr/local/lib/python3.9/site-packages (from matplotlib->-r requirements.txt (line 2)) (8.3.1)
Requirement already satisfied: pyparsing>=2.2.1 in /usr/local/lib/python3.9/site-packages (from matplotlib->-r requirements.txt (line 2)) (2.4.7)
Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.9/site-packages (from matplotlib->-r requirements.txt (line 2)) (0.10.0)
Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.9/site-packages (from matplotlib->-r requirements.txt (line 2)) (1.3.1)
Requirement already satisfied: python-dateutil>=2.7 in /usr/local/lib/python3.9/site-packages (from matplotlib->-r requirements.txt (line 2)) (2.8.1)
Requirement already satisfied: six in /usr/local/lib/python3.9/site-packages (from cycler>=0.10->matplotlib->-r requirements.txt (line 2)) (1.15.0)
Requirement already satisfied: networkx>=2.0 in /usr/local/lib/python3.9/site-packages (from scikit-image->-r requirements.txt (line 5)) (2.5.1)
Requirement already satisfied: imageio>=2.3.0 in /usr/local/lib/python3.9/site-packages (from scikit-image->-r requirements.txt (line 5)) (2.9.0)
Requirement already satisfied: tifffile>=2019.7.26 in /usr/local/lib/python3.9/site-packages (from scikit-image->-r requirements.txt (line 5)) (2021.7.2)
Requirement already satisfied: PyWavelets>=1.1.1 in /usr/local/lib/python3.9/site-packages (from scikit-image->-r requirements.txt (line 5)) (1.1.1)
Requirement already satisfied: decorator<5,>=4.3 in /usr/local/lib/python3.9/site-packages (from networkx>=2.0->scikit-image->-r requirements.txt (line 5)) (4.4.2)
electron@diynoMacBook-Pro pytorch-openpose %  
```

###またダメ

```bash:Terminal    
electron@diynoMacBook-Pro pytorch-openpose % python3 demo_camera.py
Traceback (most recent call last):
  File "/Users/electron/Desktop/pytorch-openpose/demo_camera.py", line 5, in <module>
    import torch
  File "/usr/local/lib/python3.9/site-packages/torch/__init__.py", line 196, in <module>
    from torch._C import *
RuntimeError: module compiled against API version 0xe but this version of numpy is 0xd
electron@diynoMacBook-Pro pytorch-openpose % 
```

####以下の次のアドバイスを採用

- https://github.com/freqtrade/freqtrade/issues/4281

> pip install numpy --upgrade works, something has to be updated in the files, when you figure out what, could you tell me please, thanks

```bash:Terminal    
electron@diynoMacBook-Pro pytorch-openpose % pip3 install numpy --upgrade

（・・・省略・・・）

tensorflow 2.5.0 requires numpy~=1.19.2, but you have numpy 1.21.1 which is incompatible.
Successfully installed numpy-1.21.1
electron@diynoMacBook-Pro pytorch-openpose % 
```

####今度は、cudaがないエラー

```bash:Terminal    
electron@diynoMacBook-Pro pytorch-openpose % python3 demo_camera.py
Traceback (most recent call last):
  File "/Users/electron/Desktop/pytorch-openpose/demo_camera.py", line 15, in <module>
    print(f"Torch device: {torch.cuda.get_device_name()}")
  File "/usr/local/lib/python3.9/site-packages/torch/cuda/__init__.py", line 276, in get_device_name
    return get_device_properties(device).name
  File "/usr/local/lib/python3.9/site-packages/torch/cuda/__init__.py", line 306, in get_device_properties
    _lazy_init()  # will define _get_device_properties
  File "/usr/local/lib/python3.9/site-packages/torch/cuda/__init__.py", line 164, in _lazy_init
    raise AssertionError("Torch not compiled with CUDA enabled")
AssertionError: Torch not compiled with CUDA enabled
electron@diynoMacBook-Pro pytorch-openpose % 
```

以下のサイトの次のアドバイスを採用

- https://qiita.com/val_geass/items/0035b2d0d8dd5ea32db6

> AssertionError: Torch not compiled with CUDA enabled
> はい、cudaね。test_video.pyの110行目と134行目に.cuda()があるので、.cuda()だけを削除し、再び実行。

以下をコメントアウト

> print(f"Torch device: {torch.cuda.get_device_name()}")

```bash:Terminal    
electron@diynoMacBook-Pro pytorch-openpose % cp demo_camera.py demo_camera_cpu.py
```

##今度はちゃんと動いた:sunny:

###Webカメラの画像が動画で表示され、自分の顔に骨格線が重畳された画像が表示された。
体を動かすと、ややタイムラグを伴って、骨格線も追従してついてきた。

CTRL-Zで強制終了

```bash:Terminal    
electron@diynoMacBook-Pro pytorch-openpose % python3 demo_camera_cpu.py
^Z
zsh: suspended  python3 demo_camera_cpu.py
electron@diynoMacBook-Pro pytorch-openpose % 
```

####今度は、test.pyもうまく動いた

```bash:Terminal    
electron@diynoMacBook-Pro pytorch-openpose % python3 test.py   
```

__*result*__ディレクトリ直下に、解析後の画像ファイルが出力されている。

- ファイル名：__*pose*__ + 元のファイル名 * __.jpg*__


```bash:Terminal           
electron@diynoMacBook-Pro pytorch-openpose % ls result
pose_test.jpg

electron@diynoMacBook-Pro pytorch-openpose % 
```

![スクリーンショット 2021-07-19 20.23.14.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/1754134/9cd4d3ad-bc5a-bfea-ba98-4f070fe01697.png)

![スクリーンショット 2021-07-19 20.23.22.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/1754134/b5530a9a-e29e-75e2-beb5-8b7218acd889.png)

####__*demo.py*__もうまく動いた

```bash:Terminal  
electron@diynoMacBook-Pro pytorch-openpose % python3 demo.py
electron@diynoMacBook-Pro pytorch-openpose % 
```

数秒後にウインドウが立ち上がり、次の画像が表示された。

![スクリーンショット 2021-07-19 20.14.23.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/1754134/7d005a12-afea-2c9a-bcdd-efe6d189bf29.png)

###動画は実行に成功しなかった。

```bash:Terminal
electron@diynoMacBook-Pro pytorch-openpose % pip3 install ffmpeg
```

```bash:Terminal
electron@diynoMacBook-Pro pytorch-openpose % python3 demo_video.py dance.mp4                                
ffmpeg version 4.4 Copyright (c) 2000-2021 the FFmpeg developers
  built with Apple clang version 12.0.0 (clang-1200.0.32.29)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/4.4_2 --enable-shared --enable-pthreads --enable-version3 --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libbluray --enable-libdav1d --enable-libmp3lame --enable-libopus --enable-librav1e --enable-librubberband --enable-libsnappy --enable-libsrt --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libspeex --enable-libsoxr --enable-libzmq --enable-libzimg --disable-libjack --disable-indev=jack --enable-avresample --enable-videotoolbox
  libavutil      56. 70.100 / 56. 70.100
  libavcodec     58.134.100 / 58.134.100
  libavformat    58. 76.100 / 58. 76.100
  libavdevice    58. 13.100 / 58. 13.100
  libavfilter     7.110.100 /  7.110.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  9.100 /  5.  9.100
  libswresample   3.  9.100 /  3.  9.100
  libpostproc    55.  9.100 / 55.  9.100
Input #0, rawvideo, from 'pipe:':
  Duration: N/A, start: 0.000000, bitrate: 294796 kb/s
  Stream #0:0: Video: rawvideo (BGR[24] / 0x18524742), bgr24, 853x480, 294796 kb/s, 30 tbr, 30 tbn, 30 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo (native) -> h264 (libx264))
[libx264 @ 0x7f8c14809a00] width not divisible by 2 (853x480)
Error initializing output stream 0:0 -- Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height
Conversion failed!
Traceback (most recent call last):
  File "/Users/electron/Desktop/pytorch-openpose/demo_video.py", line 132, in <module>
    writer(posed_frame)
  File "/Users/electron/Desktop/pytorch-openpose/demo_video.py", line 108, in __call__
    self.ff_proc.stdin.write(frame.tobytes())
BrokenPipeError: [Errno 32] Broken pipe
electron@diynoMacBook-Pro pytorch-openpose % 
```

ウインドウが立ち上がり、次の１フレームで止まった状態で画像が表示された後、落ちた。

![スクリーンショット 2021-07-19 21.10.52.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/1754134/64d811cf-7838-3910-2b6b-c99cc598ea7d.png)

##test.pyを書き換えて、もう２つ、画像ファイルを加工してみる

取り扱う画像ファイルのパスを、コマンドライン引数で受け取るように書き換える。

```Python3:test2.py
import os
import cv2
import copy
from src import util
from src.body import Body
import argparse    

# 動画ファイル名をコマンドライン引数から受け取る
parser = argparse.ArgumentParser(description='')    #
parser.add_argument('file_name')  
args = parser.parse_args()  

file = args.file_name

 
if __name__ == "__main__":
 
    body_estimation = Body('model/body_pose_model.pth')
 
    target_image_path = file
    oriImg = cv2.imread(target_image_path)  # B,G,R order
    candidate, subset = body_estimation(oriImg)
    canvas = copy.deepcopy(oriImg)
    canvas = util.draw_bodypose(canvas, candidate, subset)
 
    basename_name = os.path.splitext(os.path.basename(target_image_path))[0]
 
    result_image_path = "result/pose_" + basename_name + ".jpg"
    cv2.imwrite(result_image_path, canvas)

```

```bash:Terminal
electron@diynoMacBook-Pro pytorch-openpose % ls images/21stEmon.jpg                                            
images/21stEmon.jpg
electron@diynoMacBook-Pro pytorch-openpose % 
```

![スクリーンショット 2021-07-19 21.28.32.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/1754134/fbe6c00b-e209-5ecc-7195-20c196a01502.png)


```bash:Terminal
electron@diynoMacBook-Pro pytorch-openpose %  python3 test2.py ./images/21stEmon.jpg                         
electron@diynoMacBook-Pro pytorch-openpose %  ls result/            
pose_21stEmon.jpg	pose_test.jpg
electron@diynoMacBook-Pro pytorch-openpose % 
```

####骨格は推定できなかった

####この名画面の骨格推定は成功した

![スクリーンショット 2021-07-19 21.27.48.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/1754134/30687126-d5f8-495a-a3d1-435ee4533ad5.png)

```bash:Terminal
electron@diynoMacBook-Pro pytorch-openpose % ls images/moon_walker.jpg             
images/moon_walker.jpg
electron@diynoMacBook-Pro pytorch-openpose % 
```

![moon_walker.jpg](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/1754134/87dd7f04-de6f-e9e8-d971-ebbbf4390b3b.jpeg)


```bash:Terminal
electron@diynoMacBook-Pro pytorch-openpose % python3 test2.py ./images/moon_walker.jpg 
electron@diynoMacBook-Pro pytorch-openpose % ls result/pose_moon_walker.jpg 
result/pose_moon_walker.jpg
electron@diynoMacBook-Pro pytorch-openpose %
```

![スクリーンショット 2021-07-19 21.32.21.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/1754134/b6bec235-4211-4564-f903-444e3be96d5b.png)