Learn "Openpose" from scratch with MobileNetv2 + MS-COCO and deploy it to OpenVINO/TensorflowLite Part.1



**MobileNetV2-PoseEstimation**　[![GitHub stars](https://img.shields.io/github/stars/PINTO0309/MobileNetV2-PoseEstimation.svg?style=social&label=Stars)](https://github.com/PINTO0309/MobileNetV2-PoseEstimation)
<a href="https://github.com/PINTO0309/MobileNetV2-PoseEstimation"><img src="https://github-link-card.s3.ap-northeast-1.amazonaws.com/PINTO0309/MobileNetV2-PoseEstimation.png" width="460px"></a>

# １．Introduction
複数の過去記事の検証により、 IntelのCPUとOpenVINOを組み合わせた場合、半端なGPUや外付けブースタによるパフォーマンスを遥かに凌駕したり、Tensorflow Liteの8ビット量子化を行った場合の驚異的なパフォーマンスを体感してきました。
（国内外でとても有名な日本人エンジニアの方が Tensorflow Lite のGPU対応に尽力されています。 私見ではありますが、Tensorflow Lite は、今後とても可能性を秘めている存在だと考えています。）
ご興味がある奇特な方は下記の過去記事をご覧ください。

**[[24 FPS, 48 FPS] RaspberryPi3 + Neural Compute Stick 2 一本で真の力を引き出し、悟空が真のスーパーサイヤ「人」となった日](https://qiita.com/PINTO/items/cb7ba1dae4bfc74a5966)**

**[「がんばる人のための画像検査機 presented by shinmura0」をOpenVINOで異次元のスピードにパワーアップして異常検出 (CPUのみ 又は Intel HD Graphics 615) その１](https://qiita.com/PINTO/items/7782ed94a59f79ae8a69)**

**[Tensorflow Lite v1.11.0 を自力でカスタマイズしてPython API にMultiThread機能を追加→オフィシャルの２．５倍にパフォーマンスアップ](https://qiita.com/PINTO/items/711828badfb50c576eb1)**

**[「がんばる人のための画像検査機 presented by shinmura0」をラズパイ単体でパワーアップして異常検出 (RaspberryPi3のCPUのみ) その２](https://qiita.com/PINTO/items/0a52062cb6ebe9ef5051)**

**[RaspberryPi3(USB2.0)とLaptopPC(USB3.1)でGoogle Edge TPU Acceleratorを使用してMobileNet-SSD v2の動作スピードを検証してみました（MS-COCO）](https://qiita.com/PINTO/items/dd6ba67643bdd3a0e595)**

今回も無茶なチャレンジを行ってみたいと思います。 検証期間が長くなりそうで、モチベーションを維持するのが大変になってきましたので、学習とモデルの変換→検証という２部構成に記事を分けようと思います。 最終的には自身のGithubリポジトリへ成果物一式をPushしようと思いますので少しお時間をください。

RaspberryPi3などのエッジデバイス上で **`Openpose`** を高速に動作させたかったため、モデルを VGG から MobileNetV2 ベースに置き換えて軽量化します。 この手順を実施すると、 **`OpenVINO`**, **`Tensorflow Lite`** の２種類のモデルが生成されます。
**`Google Edge TPU`** モデルへの変換にもかなりの時間を割きましたが、残念ながら現時点では成功していません。
TensorflowのQuantize処理後のTPUモデルコンパイラによる変換がどうにもうまくいきません。。。
なお、それぞれのフレームワーク特性に応じて作業過程の各種コマンドに差異が有ります。
記事の内容は試行錯誤の結果が露骨に表れていますので、つたない部分が有ります点はご容赦願います。

下図の姿勢推定をエッジデバイスでリアルタイムに実行することを目指します。
下図はこの記事の手順で生成された Tensorflow のプロトコルバッファ形式のモデルファイルを使用して実行した例です。
入力画像のサイズが大きいと、表示される骨がかなり細くなります。
MobileNetV2ベースに置き換えているため、モデル全体のサイズが驚異的な小ささになりました。

**Tensorflow用 .pbファイルのサイズ ＝ 9.3 MB**
**OpenVINO用(FP16) .binファイルのサイズ ＝ 4.4 MB**
**OpenVINO用(FP32) .binファイルのサイズ ＝ 8.8 MB**
**Tensorflow Lite用(UINT8) .tfliteファイルのサイズ ＝ 2.4 MB**

**NVIDIA Tesla K80 x4, vCPU x4, MEM 32GB, 64 batch, 10 epochs, 19021 steps**
![Screenshot 2019-04-21 15:27:17.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/194769/d3ade21c-b0a7-904e-24fb-9844ee23db9e.png)
![Screenshot 2019-04-21 15:46:54.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/194769/eb26bd1c-c23a-858b-0a7c-9e9bcb152b44.png)
![Screenshot 2019-04-21 15:49:58.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/194769/cd8e4f36-b1d6-7425-5824-4d5a36f8bd96.png)
Tensorflow-GPU's Test GTX 1070 FP32 (disabled OpenVINO/Tensorflow Lite)
**Youtube： https://youtu.be/LOYc1lMt_84**
![ezgif.com-optimize (27).gif](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/194769/a3156393-b4a1-4cfb-7c14-14322890981c.gif)

Tensorflow-CPU's Test Core i7 FP32 (disabled OpenVINO/Tensorflow Lite)
**Youtube： https://youtu.be/nEKc7VIm42A**
![ezgif.com-optimize (28).gif](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/194769/7b938741-edee-d4c0-947d-beea987bdbf9.gif)

# ２．Environment

- Training exam environment
  - Ubuntu 16.04 x86_64
  - MEM 16 GB
  - Geforce GTX 1070
  - OpenCV 4.1.0-openvino
  - Tensorflow-GPU v1.12.0
  - CUDA 9.0
  - cuDNN 7.2

- Training production environment
  - GCE
  - Ubuntu 16.04 x86_64
  - MEM 256 GB
  - Tesla K80 x4
  - OpenCV 4.1.0-openvino
  - Tensorflow-GPU v1.12.0
  - CUDA 9.0
  - cuDNN 7.2

![FireShot Capture 011 - Compute Engine - My First Project - Google Cloud Platform_ - console.cloud.google.com.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/194769/a6107aaf-59f0-d045-cb96-aacf63c80104.png)

```console:GPUs
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 410.104      Driver Version: 410.104      CUDA Version: 10.0     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  Tesla K80           Off  | 00000000:00:04.0 Off |                    0 |
| N/A   62C    P8    31W / 149W |     16MiB / 11441MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   1  Tesla K80           Off  | 00000000:00:05.0 Off |                    0 |
| N/A   43C    P8    27W / 149W |      0MiB / 11441MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   2  Tesla K80           Off  | 00000000:00:06.0 Off |                    0 |
| N/A   62C    P8    33W / 149W |      0MiB / 11441MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   3  Tesla K80           Off  | 00000000:00:07.0 Off |                    0 |
| N/A   72C    P8    33W / 149W |      0MiB / 11441MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
|    0      1774      G   /usr/lib/xorg/Xorg                            14MiB |
+-----------------------------------------------------------------------------+
```

# ３．Training procedure

```console:Preparation
$ cd ~
$ git clone https://github.com/ildoonet/tf-pose-estimation.git
$ cd tf-pose-estimation
$ sudo apt-get install -y libcap-dev
$ sudo -H pip3 install python-prctl
$ sudo -H pip3 install setuptools --upgrade
$ sudo -H pip3 install tensorflow-gpu==1.12.0 --upgrade
$ sudo -H pip3 install -r requirements.txt
$ cd tf_pose/pafprocess
$ sudo apt-get install -y swig
$ swig -python -c++ pafprocess.i && python3 setup.py build_ext --inplace

$ cd ../..
$ git clone https://github.com/cocodataset/cocoapi.git
$ cd cocoapi/PythonAPI
$ sudo python3 setup.py build_ext --inplace
$ sudo python3 setup.py build_ext install
$ cd ../..
```

```console:Training
### Downloads MS-COCO Dataset
$ mkdir dataset;cd dataset
$ curl -sc /tmp/cookie "https://drive.google.com/uc?export=download&id=1llUu6071hd0QY2DY5vs_7VPaceI7EVst" > /dev/null
$ CODE="$(awk '/_warning_/ {print $NF}' /tmp/cookie)"
$ curl -Lb /tmp/cookie "https://drive.google.com/uc?export=download&confirm=${CODE}&id=1llUu6071hd0QY2DY5vs_7VPaceI7EVst" -o train2017.zip
$ unzip train2017.zip
$ rm train2017.zip

$ curl -sc /tmp/cookie "https://drive.google.com/uc?export=download&id=1bB8M-WG2LJwmB6YpLMsxUTWwIX3BwYkr" > /dev/null
$ CODE="$(awk '/_warning_/ {print $NF}' /tmp/cookie)"
$ curl -Lb /tmp/cookie "https://drive.google.com/uc?export=download&confirm=${CODE}&id=1bB8M-WG2LJwmB6YpLMsxUTWwIX3BwYkr" -o val2017.zip
$ unzip val2017.zip
$ rm val2017.zip

$ curl -sc /tmp/cookie "https://drive.google.com/uc?export=download&id=1P5pir4LVhev_S1Yvu_K3q_hGERAcFpg6" > /dev/null
$ CODE="$(awk '/_warning_/ {print $NF}' /tmp/cookie)"
$ curl -Lb /tmp/cookie "https://drive.google.com/uc?export=download&confirm=${CODE}&id=1P5pir4LVhev_S1Yvu_K3q_hGERAcFpg6" -o annotations_trainval2017.zip
$ unzip annotations_trainval2017.zip
$ rm annotations_trainval2017.zip

$ cd ..
$ mkdir -p models/train
$ mkdir -p models/pretrained/mobilenet_v2_1.4_224;cd models/pretrained/mobilenet_v2_1.4_224
$ wget https://storage.googleapis.com/mobilenet_v2/checkpoints/mobilenet_v2_1.4_224.tgz
$ tar -xzvf mobilenet_v2_1.4_224.tgz;rm mobilenet_v2_1.4_224.tgz
$ cd ../../..
$ mv tf_pose/pose_dataset.py tf_pose/BK_pose_dataset.py
$ mv tf_pose/train.py tf_pose/BK_train.py
$ nano tf_pose/pose_dataset.py
```

```python:pose_dataset.py
import logging
import math
import multiprocessing
import struct
import sys
import threading

try:
    from StringIO import StringIO
except ImportError:
    from io import StringIO

from contextlib import contextmanager

import os
import random
import requests
import cv2
import numpy as np
import time

import tensorflow as tf

from tensorpack.dataflow import MultiThreadMapData
from tensorpack.dataflow.image import MapDataComponent
from tensorpack.dataflow.common import BatchData, MapData
from tensorpack.dataflow.parallel import PrefetchData
from tensorpack.dataflow.base import RNGDataFlow, DataFlowTerminated

from pycocotools.coco import COCO
from pose_augment import pose_flip, pose_rotation, pose_to_img, pose_crop_random, \
    pose_resize_shortestedge_random, pose_resize_shortestedge_fixed, pose_crop_center, pose_random_scale
from numba import jit

logging.getLogger("requests").setLevel(logging.WARNING)
logger = logging.getLogger('pose_dataset')
logger.setLevel(logging.INFO)
ch = logging.StreamHandler()
ch.setLevel(logging.DEBUG)
formatter = logging.Formatter('[%(asctime)s] [%(name)s] [%(levelname)s] %(message)s')
ch.setFormatter(formatter)
logger.addHandler(ch)

mplset = False


class CocoMetadata:
    # __coco_parts = 57
    __coco_parts = 19
    __coco_vecs = list(zip(
        [2, 9,  10, 2,  12, 13, 2, 3, 4, 3,  2, 6, 7, 6,  2, 1,  1,  15, 16],
        [9, 10, 11, 12, 13, 14, 3, 4, 5, 17, 6, 7, 8, 18, 1, 15, 16, 17, 18]
    ))

    @staticmethod
    def parse_float(four_np):
        assert len(four_np) == 4
        return struct.unpack('<f', bytes(four_np))[0]

    @staticmethod
    def parse_floats(four_nps, adjust=0):
        assert len(four_nps) % 4 == 0
        return [(CocoMetadata.parse_float(four_nps[x*4:x*4+4]) + adjust) for x in range(len(four_nps) // 4)]

    def __init__(self, idx, img_url, img_meta, annotations, sigma):
        self.idx = idx
        self.img_url = img_url
        self.img = None
        self.sigma = sigma

        self.height = int(img_meta['height'])
        self.width = int(img_meta['width'])

        joint_list = []
        for ann in annotations:
            if ann.get('num_keypoints', 0) == 0:
                continue

            kp = np.array(ann['keypoints'])
            xs = kp[0::3]
            ys = kp[1::3]
            vs = kp[2::3]

            joint_list.append([(x, y) if v >= 1 else (-1000, -1000) for x, y, v in zip(xs, ys, vs)])

        self.joint_list = []
        transform = list(zip(
            [1, 6, 7, 9, 11, 6, 8, 10, 13, 15, 17, 12, 14, 16, 3, 2, 5, 4],
            [1, 7, 7, 9, 11, 6, 8, 10, 13, 15, 17, 12, 14, 16, 3, 2, 5, 4]
        ))
        for prev_joint in joint_list:
            new_joint = []
            for idx1, idx2 in transform:
                j1 = prev_joint[idx1-1]
                j2 = prev_joint[idx2-1]

                if j1[0] <= 0 or j1[1] <= 0 or j2[0] <= 0 or j2[1] <= 0:
                    new_joint.append((-1000, -1000))
                else:
                    new_joint.append(((j1[0] + j2[0]) / 2, (j1[1] + j2[1]) / 2))

            new_joint.append((-1000, -1000))
            self.joint_list.append(new_joint)

        # logger.debug('joint size=%d' % len(self.joint_list))

    @jit
    def get_heatmap(self, target_size):
        heatmap = np.zeros((CocoMetadata.__coco_parts, self.height, self.width), dtype=np.float32)

        for joints in self.joint_list:
            for idx, point in enumerate(joints):
                if point[0] < 0 or point[1] < 0:
                    continue
                CocoMetadata.put_heatmap(heatmap, idx, point, self.sigma)

        heatmap = heatmap.transpose((1, 2, 0))

        # background
        heatmap[:, :, -1] = np.clip(1 - np.amax(heatmap, axis=2), 0.0, 1.0)

        if target_size:
            heatmap = cv2.resize(heatmap, target_size, interpolation=cv2.INTER_AREA)

        return heatmap.astype(np.float16)

    @staticmethod
    @jit(nopython=True)
    def put_heatmap(heatmap, plane_idx, center, sigma):
        center_x, center_y = center
        _, height, width = heatmap.shape[:3]

        th = 4.6052
        delta = math.sqrt(th * 2)

        x0 = int(max(0, center_x - delta * sigma))
        y0 = int(max(0, center_y - delta * sigma))

        x1 = int(min(width, center_x + delta * sigma))
        y1 = int(min(height, center_y + delta * sigma))

        for y in range(y0, y1):
            for x in range(x0, x1):
                d = (x - center_x) ** 2 + (y - center_y) ** 2
                exp = d / 2.0 / sigma / sigma
                if exp > th:
                    continue
                heatmap[plane_idx][y][x] = max(heatmap[plane_idx][y][x], math.exp(-exp))
                heatmap[plane_idx][y][x] = min(heatmap[plane_idx][y][x], 1.0)

    @jit
    def get_vectormap(self, target_size):
        vectormap = np.zeros((CocoMetadata.__coco_parts*2, self.height, self.width), dtype=np.float32)
        countmap = np.zeros((CocoMetadata.__coco_parts, self.height, self.width), dtype=np.int16)
        for joints in self.joint_list:
            for plane_idx, (j_idx1, j_idx2) in enumerate(CocoMetadata.__coco_vecs):
                j_idx1 -= 1
                j_idx2 -= 1

                center_from = joints[j_idx1]
                center_to = joints[j_idx2]

                if center_from[0] < -100 or center_from[1] < -100 or center_to[0] < -100 or center_to[1] < -100:
                    continue

                CocoMetadata.put_vectormap(vectormap, countmap, plane_idx, center_from, center_to)

        vectormap = vectormap.transpose((1, 2, 0))
        nonzeros = np.nonzero(countmap)
        for p, y, x in zip(nonzeros[0], nonzeros[1], nonzeros[2]):
            if countmap[p][y][x] <= 0:
                continue
            vectormap[y][x][p*2+0] /= countmap[p][y][x]
            vectormap[y][x][p*2+1] /= countmap[p][y][x]

        if target_size:
            vectormap = cv2.resize(vectormap, target_size, interpolation=cv2.INTER_AREA)

        return vectormap.astype(np.float16)

    @staticmethod
    @jit(nopython=True)
    def put_vectormap(vectormap, countmap, plane_idx, center_from, center_to, threshold=8):
        _, height, width = vectormap.shape[:3]

        vec_x = center_to[0] - center_from[0]
        vec_y = center_to[1] - center_from[1]

        min_x = max(0, int(min(center_from[0], center_to[0]) - threshold))
        min_y = max(0, int(min(center_from[1], center_to[1]) - threshold))

        max_x = min(width, int(max(center_from[0], center_to[0]) + threshold))
        max_y = min(height, int(max(center_from[1], center_to[1]) + threshold))

        norm = math.sqrt(vec_x ** 2 + vec_y ** 2)
        if norm == 0:
            return

        vec_x /= norm
        vec_y /= norm

        for y in range(min_y, max_y):
            for x in range(min_x, max_x):
                bec_x = x - center_from[0]
                bec_y = y - center_from[1]
                dist = abs(bec_x * vec_y - bec_y * vec_x)

                if dist > threshold:
                    continue

                countmap[plane_idx][y][x] += 1

                vectormap[plane_idx*2+0][y][x] = vec_x
                vectormap[plane_idx*2+1][y][x] = vec_y


class CocoPose(RNGDataFlow):
    @staticmethod
    def display_image(inp, heatmap, vectmap, as_numpy=False):
        global mplset
        # if as_numpy and not mplset:
        #     import matplotlib as mpl
        #     mpl.use('Agg')
        mplset = True
        import matplotlib.pyplot as plt

        fig = plt.figure()
        a = fig.add_subplot(2, 2, 1)
        a.set_title('Image')
        plt.imshow(CocoPose.get_bgimg(inp))

        a = fig.add_subplot(2, 2, 2)
        a.set_title('Heatmap')
        plt.imshow(CocoPose.get_bgimg(inp, target_size=(heatmap.shape[1], heatmap.shape[0])), alpha=0.5)
        tmp = np.amax(heatmap, axis=2)
        plt.imshow(tmp, cmap=plt.cm.gray, alpha=0.5)
        plt.colorbar()

        tmp2 = vectmap.transpose((2, 0, 1))
        tmp2_odd = np.amax(np.absolute(tmp2[::2, :, :]), axis=0)
        tmp2_even = np.amax(np.absolute(tmp2[1::2, :, :]), axis=0)

        a = fig.add_subplot(2, 2, 3)
        a.set_title('Vectormap-x')
        plt.imshow(CocoPose.get_bgimg(inp, target_size=(vectmap.shape[1], vectmap.shape[0])), alpha=0.5)
        plt.imshow(tmp2_odd, cmap=plt.cm.gray, alpha=0.5)
        plt.colorbar()

        a = fig.add_subplot(2, 2, 4)
        a.set_title('Vectormap-y')
        plt.imshow(CocoPose.get_bgimg(inp, target_size=(vectmap.shape[1], vectmap.shape[0])), alpha=0.5)
        plt.imshow(tmp2_even, cmap=plt.cm.gray, alpha=0.5)
        plt.colorbar()

        if not as_numpy:
            plt.show()
        else:
            fig.canvas.draw()
            data = np.fromstring(fig.canvas.tostring_rgb(), dtype=np.uint8, sep='')
            data = data.reshape(fig.canvas.get_width_height()[::-1] + (3,))
            fig.clear()
            plt.close()
            return data

    @staticmethod
    def get_bgimg(inp, target_size=None):
        inp = cv2.cvtColor(inp.astype(np.uint8), cv2.COLOR_BGR2RGB)
        if target_size:
            inp = cv2.resize(inp, target_size, interpolation=cv2.INTER_AREA)
        return inp

    def __init__(self, path, img_path=None, is_train=True, decode_img=True, only_idx=-1):
        self.is_train = is_train
        self.decode_img = decode_img
        self.only_idx = only_idx

        if is_train:
            whole_path = os.path.join(path, 'person_keypoints_train2017.json')
        else:
            whole_path = os.path.join(path, 'person_keypoints_val2017.json')
        self.img_path = (img_path if img_path is not None else '') + ('train2017/' if is_train else 'val2017/')
        self.coco = COCO(whole_path)

        logger.info('%s dataset %d' % (path, self.size()))

    def size(self):
        return len(self.coco.imgs)

    def get_data(self):
        idxs = np.arange(self.size())
        if self.is_train:
            self.rng.shuffle(idxs)
        else:
            pass

        keys = list(self.coco.imgs.keys())
        for idx in idxs:
            img_meta = self.coco.imgs[keys[idx]]
            img_idx = img_meta['id']
            ann_idx = self.coco.getAnnIds(imgIds=img_idx)

            if 'http://' in self.img_path:
                img_url = self.img_path + img_meta['file_name']
            else:
                img_url = os.path.join(self.img_path, img_meta['file_name'])

            anns = self.coco.loadAnns(ann_idx)
            meta = CocoMetadata(idx, img_url, img_meta, anns, sigma=8.0)

            total_keypoints = sum([ann.get('num_keypoints', 0) for ann in anns])
            if total_keypoints == 0 and random.uniform(0, 1) > 0.2:
                continue

            yield [meta]


class MPIIPose(RNGDataFlow):
    def __init__(self):
        pass

    def size(self):
        pass

    def get_data(self):
        pass


def read_image_url(metas):
    for meta in metas:
        img_str = None
        if 'http://' in meta.img_url:
            # print(meta.img_url)
            for _ in range(10):
                try:
                    resp = requests.get(meta.img_url)
                    if resp.status_code // 100 != 2:
                        logger.warning('request failed code=%d url=%s' % (resp.status_code, meta.img_url))
                        time.sleep(1.0)
                        continue
                    img_str = resp.content
                    break
                except Exception as e:
                    logger.warning('request failed url=%s, err=%s' % (meta.img_url, str(e)))
        else:
            img_str = open(meta.img_url, 'rb').read()

        if not img_str:
            logger.warning('image not read, path=%s' % meta.img_url)
            raise Exception()

        nparr = np.fromstring(img_str, np.uint8)
        meta.img = cv2.imdecode(nparr, cv2.IMREAD_COLOR)
    return metas


def get_dataflow(path, is_train, img_path=None):
    ds = CocoPose(path, img_path, is_train)       # read data from lmdb
    if is_train:
        ds = MapData(ds, read_image_url)
        ds = MapDataComponent(ds, pose_random_scale)
        ds = MapDataComponent(ds, pose_rotation)
        ds = MapDataComponent(ds, pose_flip)
        ds = MapDataComponent(ds, pose_resize_shortestedge_random)
        ds = MapDataComponent(ds, pose_crop_random)
        ds = MapData(ds, pose_to_img)
        # augs = [
        #     imgaug.RandomApplyAug(imgaug.RandomChooseAug([
        #         imgaug.GaussianBlur(max_size=3)
        #     ]), 0.7)
        # ]
        # ds = AugmentImageComponent(ds, augs)
        ds = PrefetchData(ds, 1000, multiprocessing.cpu_count() * 1)
    else:
        ds = MultiThreadMapData(ds, nr_thread=16, map_func=read_image_url, buffer_size=1000)
        ds = MapDataComponent(ds, pose_resize_shortestedge_fixed)
        ds = MapDataComponent(ds, pose_crop_center)
        ds = MapData(ds, pose_to_img)
        ds = PrefetchData(ds, 100, multiprocessing.cpu_count() // 4)

    return ds


def _get_dataflow_onlyread(path, is_train, img_path=None):
    ds = CocoPose(path, img_path, is_train)  # read data from lmdb
    ds = MapData(ds, read_image_url)
    ds = MapData(ds, pose_to_img)
    # ds = PrefetchData(ds, 1000, multiprocessing.cpu_count() * 4)
    return ds


def get_dataflow_batch(path, is_train, batchsize, img_path=None):
    logger.info('dataflow img_path=%s' % img_path)
    ds = get_dataflow(path, is_train, img_path=img_path)
    ds = BatchData(ds, batchsize)
    # if is_train:
    #     ds = PrefetchData(ds, 10, 2)
    # else:
    #     ds = PrefetchData(ds, 50, 2)

    return ds


class DataFlowToQueue(threading.Thread):
    def __init__(self, ds, placeholders, queue_size=5):
        super().__init__()
        self.daemon = True

        self.ds = ds
        self.placeholders = placeholders
        self.queue = tf.FIFOQueue(queue_size, [ph.dtype for ph in placeholders], shapes=[ph.get_shape() for ph in placeholders])
        self.op = self.queue.enqueue(placeholders)
        self.close_op = self.queue.close(cancel_pending_enqueues=True)

        self._coord = None
        self._sess = None

        self.last_dp = None

    @contextmanager
    def default_sess(self):
        if self._sess:
            with self._sess.as_default():
                yield
        else:
            logger.warning("DataFlowToQueue {} wasn't under a default session!".format(self.name))
            yield

    def size(self):
        return self.queue.size()

    def start(self):
        self._sess = tf.get_default_session()
        super().start()

    def set_coordinator(self, coord):
        self._coord = coord

    def run(self):
        with self.default_sess():
            try:
                while not self._coord.should_stop():
                    try:
                        self.ds.reset_state()
                        while True:
                            for dp in self.ds.get_data():
                                feed = dict(zip(self.placeholders, dp))
                                self.op.run(feed_dict=feed)
                                self.last_dp = dp
                    except (tf.errors.CancelledError, tf.errors.OutOfRangeError, DataFlowTerminated):
                        logger.error('err type1, placeholders={}'.format(self.placeholders))
                        sys.exit(-1)
                    except Exception as e:
                        logger.error('err type2, err={}, placeholders={}'.format(str(e), self.placeholders))
                        if isinstance(e, RuntimeError) and 'closed Session' in str(e):
                            pass
                        else:
                            logger.exception("Exception in {}:{}".format(self.name, str(e)))
                        sys.exit(-1)
            except Exception as e:
                logger.exception("Exception in {}:{}".format(self.name, str(e)))
            finally:
                try:
                    self.close_op.run()
                except Exception:
                    pass
                logger.info("{} Exited.".format(self.name))

    def dequeue(self):
        return self.queue.dequeue()


if __name__ == '__main__':
    os.environ['CUDA_VISIBLE_DEVICES'] = ''

    from pose_augment import set_network_input_wh, set_network_scale
    # set_network_input_wh(368, 368)
    set_network_input_wh(432, 368)
    set_network_scale(8)

    # df = get_dataflow('/data/public/rw/coco/annotations', True, '/data/public/rw/coco/')
    df = _get_dataflow_onlyread('dataset/annotations', True, 'dataset/')
    # df = get_dataflow('/root/coco/annotations', False, img_path='http://gpu-twg.kakaocdn.net/braincloud/COCO/')

    from tensorpack.dataflow.common import TestDataSpeed
    TestDataSpeed(df).start()
    sys.exit(0)

    with tf.Session() as sess:
        df.reset_state()
        t1 = time.time()
        for idx, dp in enumerate(df.get_data()):
            if idx == 0:
                for d in dp:
                    logger.info('%d dp shape={}'.format(d.shape))
            print(time.time() - t1)
            t1 = time.time()
            CocoPose.display_image(dp[0], dp[1].astype(np.float32), dp[2].astype(np.float32))
            print(dp[1].shape, dp[2].shape)
            pass

    logger.info('done')
```

```console:
$ nano tf_pose/train.py
```

```python:train.py
import matplotlib as mpl
mpl.use('Agg')      # training mode, no screen should be open. (It will block training loop)

import argparse
import logging
import os
import time

import cv2
import numpy as np
import tensorflow as tf
from tqdm import tqdm

from pose_dataset import get_dataflow_batch, DataFlowToQueue, CocoPose
from pose_augment import set_network_input_wh, set_network_scale
from common import get_sample_images
from networks import get_network

logger = logging.getLogger('train')
logger.handlers.clear()
logger.setLevel(logging.DEBUG)
ch = logging.StreamHandler()
ch.setLevel(logging.DEBUG)
formatter = logging.Formatter('[%(asctime)s] [%(name)s] [%(levelname)s] %(message)s')
ch.setFormatter(formatter)
logger.addHandler(ch)


if __name__ == '__main__':
    parser = argparse.ArgumentParser(description='Training codes for Openpose using Tensorflow')
    parser.add_argument('--model', default='mobilenet_v2_1.4', help='model name')
    parser.add_argument('--datapath', type=str, default='/data/public/rw/coco/annotations')
    parser.add_argument('--imgpath', type=str, default='/data/public/rw/coco/')
    parser.add_argument('--batchsize', type=int, default=64)
    parser.add_argument('--gpus', type=int, default=4)
    parser.add_argument('--max-epoch', type=int, default=600)
    parser.add_argument('--lr', type=str, default='0.001')
    parser.add_argument('--tag', type=str, default='test')
    parser.add_argument('--checkpoint', type=str, default='')
    parser.add_argument('--input-width', type=int, default=432)
    parser.add_argument('--input-height', type=int, default=368)
    parser.add_argument('--quant-delay', type=int, default=-1)
    args = parser.parse_args()

    modelpath = logpath = './models/train/'

    if args.gpus <= 0:
        raise Exception('gpus <= 0')

    # define input placeholder
    set_network_input_wh(args.input_width, args.input_height)
    scale = 4

    if args.model in ['cmu', 'vgg'] or 'mobilenet' in args.model:
        scale = 8

    set_network_scale(scale)
    output_w, output_h = args.input_width // scale, args.input_height // scale

    logger.info('define model+')
    with tf.device(tf.DeviceSpec(device_type="CPU")):
        input_node = tf.placeholder(tf.float32, shape=(args.batchsize, args.input_height, args.input_width, 3), name='image')
        vectmap_node = tf.placeholder(tf.float32, shape=(args.batchsize, output_h, output_w, 38), name='vectmap')
        heatmap_node = tf.placeholder(tf.float32, shape=(args.batchsize, output_h, output_w, 19), name='heatmap')

        # prepare data
        df = get_dataflow_batch(args.datapath, True, args.batchsize, img_path=args.imgpath)
        enqueuer = DataFlowToQueue(df, [input_node, heatmap_node, vectmap_node], queue_size=100)
        q_inp, q_heat, q_vect = enqueuer.dequeue()

    df_valid = get_dataflow_batch(args.datapath, False, args.batchsize, img_path=args.imgpath)
    df_valid.reset_state()
    validation_cache = []

    val_image = get_sample_images(args.input_width, args.input_height)
    logger.debug('tensorboard val image: %d' % len(val_image))
    logger.debug(q_inp)
    logger.debug(q_heat)
    logger.debug(q_vect)

    # define model for multi-gpu
    q_inp_split, q_heat_split, q_vect_split = tf.split(q_inp, args.gpus), tf.split(q_heat, args.gpus), tf.split(q_vect, args.gpus)

    output_vectmap = []
    output_heatmap = []
    losses = []
    last_losses_l1 = []
    last_losses_l2 = []
    outputs = []
    for gpu_id in range(args.gpus):
        with tf.device(tf.DeviceSpec(device_type="GPU", device_index=gpu_id)):
            with tf.variable_scope(tf.get_variable_scope(), reuse=(gpu_id > 0)):
                net, pretrain_path, last_layer = get_network(args.model, q_inp_split[gpu_id])
                if args.checkpoint:
                    pretrain_path = args.checkpoint
                vect, heat = net.loss_last()
                output_vectmap.append(vect)
                output_heatmap.append(heat)
                outputs.append(net.get_output())

                l1s, l2s = net.loss_l1_l2()
                for idx, (l1, l2) in enumerate(zip(l1s, l2s)):
                    loss_l1 = tf.nn.l2_loss(tf.concat(l1, axis=0) - q_vect_split[gpu_id], name='loss_l1_stage%d_tower%d' % (idx, gpu_id))
                    loss_l2 = tf.nn.l2_loss(tf.concat(l2, axis=0) - q_heat_split[gpu_id], name='loss_l2_stage%d_tower%d' % (idx, gpu_id))
                    losses.append(tf.reduce_mean([loss_l1, loss_l2]))

                last_losses_l1.append(loss_l1)
                last_losses_l2.append(loss_l2)

    outputs = tf.concat(outputs, axis=0)

    with tf.device(tf.DeviceSpec(device_type="GPU")):
        # define loss
        total_loss = tf.reduce_sum(losses) / args.batchsize
        total_loss_ll_paf = tf.reduce_sum(last_losses_l1) / args.batchsize
        total_loss_ll_heat = tf.reduce_sum(last_losses_l2) / args.batchsize
        total_loss_ll = tf.reduce_sum([total_loss_ll_paf, total_loss_ll_heat])

        # define optimizer
        step_per_epoch = 121745 // args.batchsize
        global_step = tf.Variable(0, trainable=False)
        if ',' not in args.lr:
            starter_learning_rate = float(args.lr)
            # learning_rate = tf.train.exponential_decay(starter_learning_rate, global_step,
            #                                            decay_steps=10000, decay_rate=0.33, staircase=True)
            learning_rate = tf.train.cosine_decay(starter_learning_rate, global_step, args.max_epoch * step_per_epoch, alpha=0.0)
        else:
            lrs = [float(x) for x in args.lr.split(',')]
            boundaries = [step_per_epoch * 5 * i for i, _ in range(len(lrs)) if i > 0]
            learning_rate = tf.train.piecewise_constant(global_step, boundaries, lrs)

    if args.quant_delay >= 0:
        logger.info('train using quantized mode, delay=%d' % args.quant_delay)
        g = tf.get_default_graph()
        tf.contrib.quantize.create_training_graph(input_graph=g, quant_delay=args.quant_delay)

    # optimizer = tf.train.RMSPropOptimizer(learning_rate, decay=0.0005, momentum=0.9, epsilon=1e-10)
    optimizer = tf.train.AdamOptimizer(learning_rate, epsilon=1e-8)
    # optimizer = tf.train.MomentumOptimizer(learning_rate, momentum=0.8, use_locking=True, use_nesterov=True)
    update_ops = tf.get_collection(tf.GraphKeys.UPDATE_OPS)
    with tf.control_dependencies(update_ops):
        train_op = optimizer.minimize(total_loss, global_step, colocate_gradients_with_ops=True)
    logger.info('define model-')

    # define summary
    tf.summary.scalar("loss", total_loss)
    tf.summary.scalar("loss_lastlayer", total_loss_ll)
    tf.summary.scalar("loss_lastlayer_paf", total_loss_ll_paf)
    tf.summary.scalar("loss_lastlayer_heat", total_loss_ll_heat)
    tf.summary.scalar("queue_size", enqueuer.size())
    tf.summary.scalar("lr", learning_rate)
    merged_summary_op = tf.summary.merge_all()

    valid_loss = tf.placeholder(tf.float32, shape=[])
    valid_loss_ll = tf.placeholder(tf.float32, shape=[])
    valid_loss_ll_paf = tf.placeholder(tf.float32, shape=[])
    valid_loss_ll_heat = tf.placeholder(tf.float32, shape=[])
    sample_train = tf.placeholder(tf.float32, shape=(4, 640, 640, 3))
    sample_valid = tf.placeholder(tf.float32, shape=(12, 640, 640, 3))
    train_img = tf.summary.image('training sample', sample_train, 4)
    valid_img = tf.summary.image('validation sample', sample_valid, 12)
    valid_loss_t = tf.summary.scalar("loss_valid", valid_loss)
    valid_loss_ll_t = tf.summary.scalar("loss_valid_lastlayer", valid_loss_ll)
    merged_validate_op = tf.summary.merge([train_img, valid_img, valid_loss_t, valid_loss_ll_t])

    saver = tf.train.Saver(max_to_keep=1000)
    config = tf.ConfigProto(allow_soft_placement=True, log_device_placement=False)
    config.gpu_options.allow_growth = True
    with tf.Session(config=config) as sess:
        logger.info('model weights initialization')
        sess.run(tf.global_variables_initializer())

        if args.checkpoint and os.path.isdir(args.checkpoint):
            logger.info('Restore from checkpoint...')
            # loader = tf.train.Saver(net.restorable_variables())
            # loader.restore(sess, tf.train.latest_checkpoint(args.checkpoint))
            saver.restore(sess, tf.train.latest_checkpoint(args.checkpoint))
            logger.info('Restore from checkpoint...Done')
        elif pretrain_path:
            logger.info('Restore pretrained weights... %s' % pretrain_path)
            if '.npy' in pretrain_path:
                net.load(pretrain_path, sess, False)
            else:
                try:
                    loader = tf.train.Saver(net.restorable_variables(only_backbone=False))
                    loader.restore(sess, pretrain_path)
                except:
                    logger.info('Restore only weights in backbone layers.')
                    loader = tf.train.Saver(net.restorable_variables())
                    loader.restore(sess, pretrain_path)
            logger.info('Restore pretrained weights...Done')

        logger.info('prepare file writer')
        file_writer = tf.summary.FileWriter(os.path.join(logpath, args.tag), sess.graph)

        logger.info('prepare coordinator')
        coord = tf.train.Coordinator()
        enqueuer.set_coordinator(coord)
        enqueuer.start()

        logger.info('Training Started.')
        time_started = time.time()
        last_gs_num = last_gs_num2 = 0
        initial_gs_num = sess.run(global_step)

        last_log_epoch1 = last_log_epoch2 = -1
        while True:
            _, gs_num = sess.run([train_op, global_step])
            curr_epoch = float(gs_num) / step_per_epoch

            if gs_num > step_per_epoch * args.max_epoch:
                break

            if gs_num - last_gs_num >= 500:
                train_loss, train_loss_ll, train_loss_ll_paf, train_loss_ll_heat, lr_val, summary = sess.run([total_loss, total_loss_ll, total_loss_ll_paf, total_loss_ll_heat, learning_rate, merged_summary_op])

                # log of training loss / accuracy
                batch_per_sec = (gs_num - initial_gs_num) / (time.time() - time_started)
                logger.info('epoch=%.2f step=%d, %0.4f examples/sec lr=%f, loss=%g, loss_ll=%g, loss_ll_paf=%g, loss_ll_heat=%g' % (gs_num / step_per_epoch, gs_num, batch_per_sec * args.batchsize, lr_val, train_loss, train_loss_ll, train_loss_ll_paf, train_loss_ll_heat))
                last_gs_num = gs_num

                if last_log_epoch1 < curr_epoch:
                    file_writer.add_summary(summary, curr_epoch)
                    last_log_epoch1 = curr_epoch

            if gs_num - last_gs_num2 >= 2000:
                # save weights
                saver.save(sess, os.path.join(modelpath, args.tag, 'model_latest'), global_step=global_step)

                average_loss = average_loss_ll = average_loss_ll_paf = average_loss_ll_heat = 0
                total_cnt = 0

                if len(validation_cache) == 0:
                    for images_test, heatmaps, vectmaps in tqdm(df_valid.get_data()):
                        validation_cache.append((images_test, heatmaps, vectmaps))
                    df_valid.reset_state()
                    del df_valid
                    df_valid = None

                # log of test accuracy
                for images_test, heatmaps, vectmaps in validation_cache:
                    lss, lss_ll, lss_ll_paf, lss_ll_heat, vectmap_sample, heatmap_sample = sess.run(
                        [total_loss, total_loss_ll, total_loss_ll_paf, total_loss_ll_heat, output_vectmap, output_heatmap],
                        feed_dict={q_inp: images_test, q_vect: vectmaps, q_heat: heatmaps}
                    )
                    average_loss += lss * len(images_test)
                    average_loss_ll += lss_ll * len(images_test)
                    average_loss_ll_paf += lss_ll_paf * len(images_test)
                    average_loss_ll_heat += lss_ll_heat * len(images_test)
                    total_cnt += len(images_test)

                logger.info('validation(%d) %s loss=%f, loss_ll=%f, loss_ll_paf=%f, loss_ll_heat=%f' % (total_cnt, args.tag, average_loss / total_cnt, average_loss_ll / total_cnt, average_loss_ll_paf / total_cnt, average_loss_ll_heat / total_cnt))
                last_gs_num2 = gs_num

                #sample_image = [enqueuer.last_dp[0][i] for i in range(4)]
                #outputMat = sess.run(
                #    outputs,
                #    feed_dict={q_inp: np.array((sample_image + val_image) * max(1, (args.batchsize // 16)))}
                #)
                #pafMat, heatMat = outputMat[:, :, :, 19:], outputMat[:, :, :, :19]
                #
                #sample_results = []
                #for i in range(len(sample_image)):
                #    test_result = CocoPose.display_image(sample_image[i], heatMat[i], pafMat[i], as_numpy=True)
                #    test_result = cv2.resize(test_result, (640, 640))
                #    test_result = test_result.reshape([640, 640, 3]).astype(float)
                #    sample_results.append(test_result)
                #
                #test_results = []
                #for i in range(len(val_image)):
                #    test_result = CocoPose.display_image(val_image[i], heatMat[len(sample_image) + i], pafMat[len(sample_image) + i], as_numpy=True)
                #    test_result = cv2.resize(test_result, (640, 640))
                #    test_result = test_result.reshape([640, 640, 3]).astype(float)
                #    test_results.append(test_result)

                # save summary
                #summary = sess.run(merged_validate_op, feed_dict={
                #    valid_loss: average_loss / total_cnt,
                #    valid_loss_ll: average_loss_ll / total_cnt,
                #    valid_loss_ll_paf: average_loss_ll_paf / total_cnt,
                #    valid_loss_ll_heat: average_loss_ll_heat / total_cnt,
                #    sample_valid: test_results,
                #    sample_train: sample_results
                #})
                if last_log_epoch2 < curr_epoch:
                    #file_writer.add_summary(summary, curr_epoch)
                    last_log_epoch2 = curr_epoch

        saver.save(sess, os.path.join(modelpath, args.tag, 'model'), global_step=global_step)
    logger.info('optimization finished. %f' % (time.time() - time_started))
```

```console:Training_exam_with_GTX1070_for_OpenVINO_TFLite_(1)
$ python3 tf_pose/train.py \
--model=mobilenet_v2_1.4 \
--datapath=dataset/annotations/ \
--imgpath=dataset/ \
--batchsize=2 \
--gpus=1 \
--max-epoch=5 \
--lr=0.001
```

```console:Training_production_with_Google_Compute_Engine_4GPUs_for_OpenVINO_TFLite_(2)
$ python3 tf_pose/train.py \
--model=mobilenet_v2_1.4 \
--datapath=dataset/annotations/ \
--imgpath=dataset/ \
--batchsize=64 \
--gpus=4 \
--max-epoch=5 \
--lr=0.001
```

```console:Training_exam_with_GTX1070_for_TPU_(3)
$ python3 tf_pose/train.py \
--model=mobilenet_v2_1.4 \
--datapath=dataset/annotations/ \
--imgpath=dataset/ \
--batchsize=2 \
--gpus=1 \
--max-epoch=5 \
--lr=0.001 \
--quant-delay=180000
```

```console:Training_production_with_Google_Compute_Engine_1GPU_for_TPU_(4)
$ python3 tf_pose/train.py \
--model=mobilenet_v2_1.4 \
--datapath=dataset/annotations/ \
--imgpath=dataset/ \
--batchsize=4 \
--gpus=1 \
--max-epoch=10 \
--lr=0.001 \
--quant-delay=180000
```

```console:
$ nano model_compression.py
```

```model_compression.py
import tensorflow as tf
from tf_pose.networks import get_network

#######################################################################################
### $ python3 model_compression.py
#######################################################################################

def main():

    graph = tf.Graph()
    with graph.as_default():

        input_node = tf.placeholder(tf.float32, shape=(1, 368, 432, 3), name='image')
        net, pretrain_path, last_layer = get_network("mobilenet_v2_1.4", input_node, None, False)

        saver = tf.train.Saver(tf.global_variables())
        sess  = tf.Session()
        sess.run(tf.global_variables_initializer())
        sess.run(tf.local_variables_initializer())

        saver.restore(sess, 'models/train/test/model-19021')
        saver.save(sess, 'models/train/test/model-final-19021')

        graphdef = graph.as_graph_def()
        tf.train.write_graph(graphdef, 'models/train/test', 'model-final.pbtxt', as_text=True)

if __name__ == '__main__':
    main()
```

```console:Execution_of_model_compression.py
$ sed -i s/"(slim.batch_norm,): {'center': True, 'scale': True},"/"(slim.batch_norm,): {'center': True, 'scale': True, 'is_training': False},"/ tf_pose/mobilenet/mobilenet_v2.py
$ python3 model_compression.py
$ sed -i s/"(slim.batch_norm,): {'center': True, 'scale': True, 'is_training': False},"/"(slim.batch_norm,): {'center': True, 'scale': True},"/ tf_pose/mobilenet/mobilenet_v2.py
```

```console:Graph_freeze
$ python3 freeze_graph.py \
--input_graph=models/train/test/model-final.pbtxt \
--input_checkpoint=models/train/test/model-final-19021 \
--output_graph=models/train/test/frozen-model.pb \
--output_node_names=Openpose/concat_stage7 \
--input_binary=False \
--clear_devices=True
```
![frozen-model.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/194769/b1b94907-430e-b8ce-6d03-62b826fd3bfb.png)

```console:graph_optimization
$ cd ..
$ git clone https://github.com/PINTO0309/Bazel_bin.git
$ Bazel_bin/0.19.2/Ubuntu1604_x86_64/install.sh
$ git clone -b v1.12.0 https://github.com/tensorflow/tensorflow.git
$ cd tensorflow
$ git checkout -b v1.12.0
$ sudo bazel build tensorflow/tools/graph_transforms:transform_graph
$ bazel-bin/tensorflow/tools/graph_transforms/transform_graph \
--in_graph=../tf-pose-estimation/models/train/test/frozen-model.pb \
--out_graph=../tf-pose-estimation/models/train/test/frozen-opt-model.pb \
--inputs='image' \
--outputs='Openpose/concat_stage7' \
--transforms='strip_unused_nodes(type=float, shape="1,368,432,3")
              fold_old_batch_norms
              fold_batch_norms
              fold_constants(ignoreError=False)
              remove_nodes(op=Identity, op=CheckNumerics)'
```
![frozen-opt-model.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/194769/d353711e-a985-dd88-73a0-ea2d6d7b3aa5.png)

```console:Quantization_(1)
$ cd ~/tf-pose-estimation
$ mkdir -p models/train/test/tflite
$ tflite_convert \
  --output_file="models/train/test/tflite/output_tflite_graph.tflite" \
  --graph_def_file="models/train/test/frozen-model.pb" \
  --inference_type=QUANTIZED_UINT8 \
  --input_arrays="image" \
  --output_arrays="Openpose/concat_stage7" \
  --mean_values=128 \
  --std_dev_values=128 \
  --default_ranges_min=0 \
  --default_ranges_max=6 \
  --input_shapes=1,368,432,3 \
  --change_concat_input_ranges=false \
  --allow_nudging_weights_to_use_fast_gemm_kernel=true
```

or

```console:Quantization_(2)
$ cd ~/tf-pose-estimation
$ mkdir -p models/train/test/tflite
$ toco \
  --graph_def_file=models/train/test/frozen-model.pb \
  --output_file=models/train/test/tflite/output_tflite_graph.tflite \
  --input_format=TENSORFLOW_GRAPHDEF \
  --output_format=TFLITE \
  --inference_type=QUANTIZED_UINT8 \
  --input_shapes="1,368,432,3" \
  --input_array=image \
  --output_array=Openpose/concat_stage7 \
  --std_dev_values=127.5 \
  --mean_values=127.5 \
  --default_ranges_min=0 \
  --default_ranges_max=6 \
  --post_training_quantize
```

or

```console:Quantization_(3)
$ cd ~/tensorflow
$ bazel-bin/tensorflow/tools/graph_transforms/transform_graph \
  --in_graph=../tf-pose-estimation/models/train/test/frozen-model.pb \
  --out_graph=../tf-pose-estimation/models/train/test/frozen-quant-model.pb \
  --inputs='image' \
  --outputs='Openpose/concat_stage7' \
  --transforms='quantize_weights quantize_nodes'
$ cd ../tf-pose-estimation
```


```console:Convert_to_OpenVINO_FP16
$ mkdir -p models/train/test/openvino/FP16
$ sudo python3 /opt/intel/openvino/deployment_tools/model_optimizer/mo_tf.py \
--input_model models/train/test/frozen-model.pb \
--output_dir models/train/test/openvino/FP16 \
--data_type FP16 \
--input_shape [1,368,432,3]
```

```console:Convert_to_OpenVINO_FP32
$ mkdir -p models/train/test/openvino/FP32
$ sudo python3 /opt/intel/openvino/deployment_tools/model_optimizer/mo_tf.py \
--input_model models/train/test/frozen-model.pb \
--output_dir models/train/test/openvino/FP32 \
--data_type FP32 \
--input_shape [1,368,432,3]
```

# ４．Finally
次回は、OpenVINO と Tensorflow Lite による USB Camera のリアルタイム動作の検証を行います。
モデルのサイズがかなり小さくなりましたので、動作確認をする前からそこそこ早く動作しそうな予感がしています。

# ５．Reference articles
https://coral.withgoogle.com/web-compiler/
https://github.com/tensorflow/tensorflow/tree/r1.13/tensorflow/contrib/quantize#quantization-aware-training
https://github.com/tensorflow/tensorflow/issues/20867
https://github.com/tensorflow/models/blob/master/research/object_detection/export_tflite_ssd_graph.py
https://github.com/tensorflow/models/blob/master/research/object_detection/export_tflite_ssd_graph_lib.py
https://github.com/tensorflow/models/blob/master/research/object_detection/builders/graph_rewriter_builder.py
https://github.com/tensorflow/models/blob/master/research/object_detection/exporter.py
https://github.com/ildoonet/tf-pose-estimation/blob/master/etcs/training.md
https://github.com/ildoonet/tf-pose-estimation/blob/master/etcs/experiments.md
https://github.com/tensorflow/tensorflow/issues/24477
https://github.com/tensorflow/tensorflow/blob/r1.13/tensorflow/contrib/quantize/python/quantize_graph.py
[TensorFlowで学習してモデルファイルを小さくしてコマンドラインアプリを作るシンプルな流れ](https://qiita.com/YusukeSuzuki@github/items/476e599d84eb3d6d184d)
http://tensorflow.classcat.com/2019/01/20/cc-tf-onnx-hub-person-pose-estimation/
https://tech-blog.optim.co.jp/entry/2018/12/03/162348
https://www.tensorflow.org/lite/models/pose_estimation/overview
https://github.com/ildoonet/tf-pose-estimation/issues/132
https://github.com/ildoonet/tf-pose-estimation/issues/174
[Linuxでコマンドラインからマシンスペックを確認する方法](https://qiita.com/DaisukeMiyamoto/items/98ef077ddf44b5727c29)
https://github.com/tensorflow/tensorflow/tree/r1.13/tensorflow/contrib/quantize

https://github.com/tensorflow/models/tree/master/research/slim/nets/mobilenet
https://www.tensorflow.org/api_docs/python/tf/contrib/quantize/create_training_graph