DefaultQuantization, AccuracyAwareQuantization by OpenVINO's post training optimization toolkit, INT8 (Integer Quantization)

# １． Introduction
Organize the procedure for INT8 Quantification of EfficientNet by "post training optimization toolkit" of OpenVINO.

# ２． Environment
- Ubuntu 18.04 x86_64
- Corei7-8750H (No AVX-512 instruction set)
- Tensorflow-gpu v1.15.2
- PyTorch
- Python 3.6
- OpenVINO 2020 R1
- EfficientNet D0 (Pytorch)
- Numpy 1.18.5 ... Only at build time
- Numpy 1.20.1 ... Only at runtime

# ３． Procedure
## ３−１．Install AccuracyChecker and post_training_optimization_toolkit

**https://docs.openvinotoolkit.org/latest/_README.html**

```console:Install_AccuracyChecker_and_post_training_optimization_toolkit
### Install the AccuracyChecker component
$ cd $INTEL_OPENVINO_DIR/deployment_tools/open_model_zoo/tools/accuracy_checker/
$ sudo python3 setup.py install

### Install the toolkit
$ cd $INTEL_OPENVINO_DIR/deployment_tools/tools/post_training_optimization_toolkit/
$ sudo python3 setup.py install
```

## ３−２．Download a trained model of EfficientNet_D0

```console:Download_a_trained_model_of_EfficientNet_D0
$ cd $INTEL_OPENVINO_DIR/deployment_tools/tools/post_training_optimization_toolkit
$ cd libs/open_model_zoo/tools/downloader

$ sudo python3 downloader.py \
  --name efficientnet-b0-pytorch
```
![Screenshot 2020-03-22 14:48:40.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/194769/34121596-9571-7527-0453-697add33b714.png)

## ３−３．Convert Pytorch model to IR model

```console:Convert_Pytorch_model_to_IR_model
$ sudo python3 converter.py \
  --name efficientnet-b0-pytorch \
  --mo $INTEL_OPENVINO_DIR/deployment_tools/model_optimizer/mo.py
```

```console:Log
========= Converting efficientnet-b0-pytorch to ONNX
Conversion to ONNX command: /usr/bin/python3 /opt/intel/openvino_2020.1.023/deployment_tools/open_model_zoo/tools/downloader/pytorch_to_onnx.py --model-path=/opt/intel/openvino_2020.1.023/deployment_tools/open_model_zoo/tools/downloader/public/efficientnet-b0-pytorch --model-name=efficientnet_b0 --import-module=model --weights=/opt/intel/openvino_2020.1.023/deployment_tools/open_model_zoo/tools/downloader/public/efficientnet-b0-pytorch/efficientnet-b0.pth --input-shape=1,3,224,224 --input-names=data --output-names=prob --output-file=/opt/intel/openvino_2020.1.023/deployment_tools/open_model_zoo/tools/downloader/public/efficientnet-b0-pytorch/efficientnet-b0.onnx

ONNX check passed successfully.

========= Converting efficientnet-b0-pytorch to IR (FP16)
Conversion command: /usr/bin/python3 -- /opt/intel/openvino/deployment_tools/model_optimizer/mo.py --framework=onnx --data_type=FP16 --output_dir=/opt/intel/openvino_2020.1.023/deployment_tools/open_model_zoo/tools/downloader/public/efficientnet-b0-pytorch/FP16 --model_name=efficientnet-b0-pytorch --reverse_input_channels '--input_shape=[1,3,224,224]' --input=data '--mean_values=data[123.675,116.28,103.53]' '--scale_values=data[58.395,57.12,57.375]' --output=prob --input_model=/opt/intel/openvino_2020.1.023/deployment_tools/open_model_zoo/tools/downloader/public/efficientnet-b0-pytorch/efficientnet-b0.onnx

Model Optimizer arguments:
Common parameters:
	- Path to the Input Model: 	/opt/intel/openvino_2020.1.023/deployment_tools/open_model_zoo/tools/downloader/public/efficientnet-b0-pytorch/efficientnet-b0.onnx
	- Path for generated IR: 	/opt/intel/openvino_2020.1.023/deployment_tools/open_model_zoo/tools/downloader/public/efficientnet-b0-pytorch/FP16
	- IR output name: 	efficientnet-b0-pytorch
	- Log level: 	ERROR
	- Batch: 	Not specified, inherited from the model
	- Input layers: 	data
	- Output layers: 	prob
	- Input shapes: 	[1,3,224,224]
	- Mean values: 	data[123.675,116.28,103.53]
	- Scale values: 	data[58.395,57.12,57.375]
	- Scale factor: 	Not specified
	- Precision of IR: 	FP16
	- Enable fusing: 	True
	- Enable grouped convolutions fusing: 	True
	- Move mean values to preprocess section: 	False
	- Reverse input channels: 	True
ONNX specific parameters:
Model Optimizer version: 	2020.1.0-61-gd349c3ba4a

[ SUCCESS ] Generated IR version 10 model.
[ SUCCESS ] XML file: /opt/intel/openvino_2020.1.023/deployment_tools/open_model_zoo/tools/downloader/public/efficientnet-b0-pytorch/FP16/efficientnet-b0-pytorch.xml
[ SUCCESS ] BIN file: /opt/intel/openvino_2020.1.023/deployment_tools/open_model_zoo/tools/downloader/public/efficientnet-b0-pytorch/FP16/efficientnet-b0-pytorch.bin
[ SUCCESS ] Total execution time: 14.41 seconds. 
[ SUCCESS ] Memory consumed: 189 MB. 

========= Converting efficientnet-b0-pytorch to IR (FP32)
Conversion command: /usr/bin/python3 -- /opt/intel/openvino/deployment_tools/model_optimizer/mo.py --framework=onnx --data_type=FP32 --output_dir=/opt/intel/openvino_2020.1.023/deployment_tools/open_model_zoo/tools/downloader/public/efficientnet-b0-pytorch/FP32 --model_name=efficientnet-b0-pytorch --reverse_input_channels '--input_shape=[1,3,224,224]' --input=data '--mean_values=data[123.675,116.28,103.53]' '--scale_values=data[58.395,57.12,57.375]' --output=prob --input_model=/opt/intel/openvino_2020.1.023/deployment_tools/open_model_zoo/tools/downloader/public/efficientnet-b0-pytorch/efficientnet-b0.onnx

Model Optimizer arguments:
Common parameters:
	- Path to the Input Model: 	/opt/intel/openvino_2020.1.023/deployment_tools/open_model_zoo/tools/downloader/public/efficientnet-b0-pytorch/efficientnet-b0.onnx
	- Path for generated IR: 	/opt/intel/openvino_2020.1.023/deployment_tools/open_model_zoo/tools/downloader/public/efficientnet-b0-pytorch/FP32
	- IR output name: 	efficientnet-b0-pytorch
	- Log level: 	ERROR
	- Batch: 	Not specified, inherited from the model
	- Input layers: 	data
	- Output layers: 	prob
	- Input shapes: 	[1,3,224,224]
	- Mean values: 	data[123.675,116.28,103.53]
	- Scale values: 	data[58.395,57.12,57.375]
	- Scale factor: 	Not specified
	- Precision of IR: 	FP32
	- Enable fusing: 	True
	- Enable grouped convolutions fusing: 	True
	- Move mean values to preprocess section: 	False
	- Reverse input channels: 	True
ONNX specific parameters:
Model Optimizer version: 	2020.1.0-61-gd349c3ba4a

[ SUCCESS ] Generated IR version 10 model.
[ SUCCESS ] XML file: /opt/intel/openvino_2020.1.023/deployment_tools/open_model_zoo/tools/downloader/public/efficientnet-b0-pytorch/FP32/efficientnet-b0-pytorch.xml
[ SUCCESS ] BIN file: /opt/intel/openvino_2020.1.023/deployment_tools/open_model_zoo/tools/downloader/public/efficientnet-b0-pytorch/FP32/efficientnet-b0-pytorch.bin
[ SUCCESS ] Total execution time: 14.62 seconds. 
[ SUCCESS ] Memory consumed: 190 MB.
```
![Screenshot 2020-03-22 14:51:46.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/194769/7f653af3-00bd-8e62-d4a2-de222e5e572a.png)

## ３−４．Check model definition information

![Screenshot 2020-03-22 14:57:15.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/194769/c001a255-e99a-ca27-d6ce-7e76cb64d333.png)

```yml:$INTEL_OPENVINO_DIR/deployment_tools/open_model_zoo/tools/accuracy_checker/configs/efficientnet-b0-pytorch.yml
models:
  - name: efficientnet-b0-pytorch

    launchers:
      - framework: onnx_runtime
        model: public/efficientnet-b0-pytorch/efficientnet-b0.onnx
        adapter: classification
        inputs:
          - name: data
            type: INPUT
            shape: 1,3,224,224


    datasets:
      - name: imagenet_1000_classes
        reader: pillow_imread

        preprocessing:
          - type: resize
            size: 256
            aspect_ratio_scale: greater
            use_pillow: True
            interpolation: BICUBIC

          - type: crop
            use_pillow: True
            size: 224

          - type: normalization
            mean: (123.675,116.28,103.53)
            std: (58.395,57.12,57.375)

  - name: efficientnet-b0-pytorch

    launchers:
      - framework: dlsdk
        tags:
          - FP32
        model:   public/efficientnet-b0-pytorch/FP32/efficientnet-b0-pytorch.xml
        weights: public/efficientnet-b0-pytorch/FP32/efficientnet-b0-pytorch.bin
        adapter: classification

      - framework: dlsdk
        tags:
          - FP16
        model:   public/efficientnet-b0-pytorch/FP16/efficientnet-b0-pytorch.xml
        weights: public/efficientnet-b0-pytorch/FP16/efficientnet-b0-pytorch.bin
        adapter: classification

    datasets:
      - name: imagenet_1000_classes
        reader: pillow_imread

        preprocessing:
          - type: rgb_to_bgr

          - type: resize
            size: 256
            aspect_ratio_scale: greater
            use_pillow: True
            interpolation: BICUBIC

          - type: crop
            use_pillow: True
            size: 224

global_definitions: ../dataset_definitions.yml
```

## ３−５．Confirmation of definition information of calibration data set (ImageNet)

![Screenshot 2020-03-22 15:07:34.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/194769/185d5a68-d611-e311-9a15-176bf7d45d2a.png)

```yml:$INTEL_OPENVINO_DIR/deployment_tools/open_model_zoo/tools/accuracy_checker/dataset_definitions.yml
  - name: imagenet_1000_classes
    annotation_conversion:
      converter: imagenet
      annotation_file: val.txt
    annotation: imagenet1000.pickle
    data_source: ILSVRC2012_img_val
    metrics:
      - name: accuracy@top1
        type: accuracy
        top_k: 1
      - name: accuracy@top5
        type: accuracy
        top_k: 5
```

## ３−６．Check the contents of the sample definition file of the quantization setting file

![Screenshot 2020-03-22 15:35:22.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/194769/94e11efc-9e85-0598-bf46-6599799c7601.png)
**https://docs.openvinotoolkit.org/latest/_configs_README.html**
**https://docs.openvinotoolkit.org/latest/_compression_algorithms_quantization_README.html**
**https://docs.openvinotoolkit.org/latest/_compression_algorithms_quantization_default_README.html**
**https://docs.openvinotoolkit.org/latest/_compression_algorithms_quantization_accuracy_aware_README.html**

```jsonnet:"target_device"_Specifiable_parameters
        "target_device": "CPU", # ---> [CPU or CPU_VNNI or GPU or VPU]
```

```jsonnet:"algorithms"_Specifiable_parameters
        "algorithms": [
            {
                "name": "DefaultQuantization", # ---> optimization algorithm name [DefaultQuantization, AccuracyAwareQuantization]
                "preset": "performance", # ---> Preset to control quantization mode [performance(Default), accuracy]
                "params": {
                    "stat_subset_size": 300 # ---> The size of the subset for calculating the activation statistics used for quantization. If no parameters are specified, the entire dataset will be used. example）300 images
                }
            }
```

## ３−７．Creating a quantization setting file

```console:
$ cd ~
$ mkdir -p openvino_quantization
$ nano openvino_quantization/efficientnet-b0-pytorch.json
```

```jsonnet:efficientnet-b0-pytorch.json
{
    "model": {
        "model_name": "efficientnet-b0-pytorch",
        "model": "/opt/intel/openvino_2020.1.023/deployment_tools/open_model_zoo/tools/downloader/public/efficientnet-b0-pytorch/FP32/efficientnet-b0-pytorch.xml",
        "weights": "/opt/intel/openvino_2020.1.023/deployment_tools/open_model_zoo/tools/downloader/public/efficientnet-b0-pytorch/FP32/efficientnet-b0-pytorch.bin"
    },
    "engine": {
        "config": "/opt/intel/openvino/deployment_tools/open_model_zoo/tools/accuracy_checker/configs/efficientnet-b0-pytorch.yml"
    },
    "compression": {
        "target_device": "CPU",
        "algorithms": [
            {
                "name": "DefaultQuantization",
                "preset": "performance",
                "params": {
                    "stat_subset_size": 300
                }
            }
        ]
    }
}
```

## ３−８．Preparing the Imagenet dataset
**https://hiro99ma.blogspot.com/2015/10/aicaffe_7.html**
**https://github.com/BVLC/caffe/tree/master/examples/imagenet**
**https://github.com/BVLC/caffe/tree/master/data/ilsvrc12**

```console:
$ cd ~
$ git clone --depth 1 https://github.com/BVLC/caffe.git
$ cd caffe
$ ./data/ilsvrc12/get_ilsvrc_aux.sh
```

```console:
$ ls -1 data/ilsvrc12/
det_synset_words.txt
get_ilsvrc_aux.sh
imagenet.bet.pickle
imagenet_mean.binaryproto
synset_words.txt
synsets.txt
test.txt
train.txt
val.txt
```

```python:view_pickle.py 
import pickle
import base64
import json
import numpy as np

picklepath = 'data/ilsvrc12/imagenet.bet.pickle'
jsonpath = 'data/ilsvrc12/imagenet.bet.json'

class NumpyEncoder(json.JSONEncoder):
    def default(self, obj):
        if isinstance(obj, np.ndarray):
            return obj.tolist()
        return json.JSONEncoder.default(self, obj)

with open(picklepath, 'rb') as file:
    data = pickle.load(file, encoding='latin1')
    print(data)

json_string = json.dumps(data, cls=NumpyEncoder)
with open(jsonpath, 'w') as j:
    json.dump(json_string, j)
```

```console:
$ python3 data/ilsvrc12/view_pickle.py
```

## ３−９．Preparation of annotation data
**https://github.com/opencv/open_model_zoo/blob/master/tools/accuracy_checker/accuracy_checker/annotation_converters/README.md**

```console:
usage: convert_annotation [-o OUTPUT_DIR] [-m META_NAME] [-a ANNOTATION_NAME]
                          [-ss SUBSAMPLE] [--subsample_seed SUBSAMPLE_SEED]
                          [--analyze_dataset]
                          {market1501_reid,mars,voc_segmentation,voc_detection,
                           sample,wider,detection_opencv_storage,lfw,vgg_face,
                           super_resolution,multi_frame_super_resolution,imagenet,
                           icdar_detection,icdar13_recognition,mscoco_detection,
                           mscoco_keypoints,mscoco_segmentation,mscoco_mask_rcnn,
                           mscoco_single_keypoints,cityscapes,movie_lens_converter,
                           brats,brats_numpy,cifar10,mnist_csv,wmt,
                           common_semantic_segmentation,camvid,lpr_txt,
                           image_retrieval,cvat_object_detection,
                           cvat_attributes_recognition,cvat_age_gender,
                           cvat_facial_landmarks,cvat_text_recognition,
                           cvat_multilabel_binary_attributes_recognition,
                           cvat_pose_estimation,cvat_person_detection_action_recognition,
                           squad,xnli,bert_xnli_tf_record,cmu_panoptic_keypoints,
                           clip_action_recognition,continuous_clip_action_recognition}
```

```console:
$ convert_annotation \
  --output_dir ${HOME}/caffe/data/ilsvrc12 \
  --annotation_name imagenet1000.pickle \
  imagenet \
  --annotation_file ${HOME}/caffe/data/ilsvrc12/val.txt \
  --labels_file ${HOME}/caffe/data/ilsvrc12/synset_words.txt \
  --has_background False
```
![Screenshot 2020-03-22 22:40:46.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/194769/b20fc7d4-3624-a795-abe2-61078052cf94.png)

```console:
$ cp ${HOME}/caffe/data/ilsvrc12/imagenet1000.pickle ${HOME}/openvino_quantization && \
  cp ${HOME}/caffe/data/ilsvrc12/val.txt ${HOME}/openvino_quantization
```
![Screenshot 2020-03-22 22:59:55.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/194769/580850e1-2402-332f-c4be-0e316328cda1.png)

## ３−１０．Perform quantization

```console:Commandline_tool：POT=post_training_optimization_toolkit
$ cd ~/openvino_quantization

$ pot \
  --config efficientnet-b0-pytorch.json \
  --save-model

or

$ pot \
  --config efficientnet-b0-pytorch.json \
  --evaluate \
  --save-model
```

```console:Log
/usr/local/lib/python3.6/dist-packages/pot-1.0-py3.6.egg/compression/algorithms/quantization/optimization/algorithm.py:39: UserWarning: Nevergrad package could not be imported. Consider installing itusing pip. Note that nevergrad is compatible only with Python 3.6+
  warnings.warn('Nevergrad package could not be imported. Consider installing it'
Output log dir: ./results/efficientnet-b0-pytorch_DefaultQuantization/2020-03-22_23-09-11
Creating pipeline:
Algorithm: DefaultQuantization
Parameters:
	stat_subset_size           : 300
	target_device              : CPU
===========================================================================
INFO:compression.statistics.collector:Start computing statistics for algorithms : DefaultQuantization
INFO:compression.statistics.collector:Computing statistics finished
INFO:compression.pipeline.pipeline:Start algorithm: DefaultQuantization
INFO:compression.statistics.collector:Start computing statistics for algorithms : ActivationChannelAlignment
INFO:compression.statistics.collector:Computing statistics finished
INFO:compression.statistics.collector:Start computing statistics for algorithms : MinMaxQuantization,BiasCorrection
INFO:compression.statistics.collector:Computing statistics finished
INFO:compression.pipeline.pipeline:Finished: DefaultQuantization
 ===========================================================================
INFO:compression.pipeline.pipeline:Evaluation of generated model
INFO:compression.engines.ac_engine:Start inference on the whole dataset
Total dataset size: 50000
1000 / 50000 processed in 23.321s
2000 / 50000 processed in 23.383s
3000 / 50000 processed in 24.053s
　：
49000 / 50000 processed in 27.800s
50000 / 50000 processed in 26.584s
50000 objects processed in 1258.780 seconds
INFO:compression.engines.ac_engine:Inference finished
accuracy@top1              : 0.76102
accuracy@top5              : 0.9277

```


```console:
$ pot -h
usage: pot [-h] -c CONFIG [-e] [--output-dir OUTPUT_DIR] [-sm] [-d]
           [--log-level {CRITICAL,ERROR,WARNING,INFO,DEBUG}] [--progress-bar]

Post-training Compression Toolkit

optional arguments:
  -h, --help            show this help message and exit
  -c CONFIG, --config CONFIG
                        Path to a config file with task/model-specific
                        parameters
  -e, --evaluate        Whether to evaluate model on whole dataset
  --output-dir OUTPUT_DIR
                        The directory where models are saved. Default:
                        ./results
  -sm, --save-model     Saves fp32 model into output directory
  -d, --direct-dump     Flag to save files without sub folders with algo names
  --log-level {CRITICAL,ERROR,WARNING,INFO,DEBUG}
                        Log level to print: ['CRITICAL', 'ERROR', 'WARNING',
                        'INFO', 'DEBUG']
  --progress-bar        Disable CL logging and enable progress bar
```

## ３−１１．Model benchmark

```console:Build_benchmark_app
$ cd $INTEL_OPENVINO_DIR/deployment_tools/inference_engine/samples/cpp
$ ./build_samples.sh
```

```console:Build_Log
Setting environment variables for building samples...
[setupvars.sh] OpenVINO environment initialized
-- The C compiler identification is GNU 7.4.0
-- The CXX compiler identification is GNU 7.4.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Looking for C++ include unistd.h
-- Looking for C++ include unistd.h - found
-- Looking for C++ include stdint.h
-- Looking for C++ include stdint.h - found
-- Looking for C++ include sys/types.h
-- Looking for C++ include sys/types.h - found
-- Looking for C++ include fnmatch.h
-- Looking for C++ include fnmatch.h - found
-- Looking for strtoll
-- Looking for strtoll - found
-- Found InferenceEngine: /opt/intel/openvino_2020.1.023/deployment_tools/inference_engine/lib/intel64/libinference_engine.so (Required is at least version "2.1") 
-- Configuring done
-- Generating done
-- Build files have been written to: /home/b920405/inference_engine_cpp_samples_build
Scanning dependencies of target gflags_nothreads_static
Scanning dependencies of target format_reader
[ 10%] Building CXX object thirdparty/gflags/CMakeFiles/gflags_nothreads_static.dir/src/gflags_reporting.cc.o
[ 10%] Building CXX object thirdparty/gflags/CMakeFiles/gflags_nothreads_static.dir/src/gflags_completions.cc.o
[ 10%] Building CXX object thirdparty/gflags/CMakeFiles/gflags_nothreads_static.dir/src/gflags.cc.o
[ 16%] Building CXX object common/format_reader/CMakeFiles/format_reader.dir/MnistUbyte.cpp.o
[ 16%] Building CXX object common/format_reader/CMakeFiles/format_reader.dir/bmp.cpp.o
[ 20%] Building CXX object common/format_reader/CMakeFiles/format_reader.dir/format_reader.cpp.o
[ 23%] Building CXX object common/format_reader/CMakeFiles/format_reader.dir/opencv_wraper.cpp.o
[ 26%] Linking CXX shared library ../../intel64/Release/lib/libformat_reader.so
[ 30%] Linking CXX static library ../../intel64/Release/lib/libgflags_nothreads.a
[ 30%] Built target gflags_nothreads_static
[ 30%] Built target format_reader
Scanning dependencies of target hello_classification
Scanning dependencies of target hello_reshape_ssd
Scanning dependencies of target hello_nv12_input_classification
Scanning dependencies of target hello_query_device
Scanning dependencies of target speech_sample
Scanning dependencies of target classification_sample_async
Scanning dependencies of target benchmark_app
Scanning dependencies of target style_transfer_sample
[ 33%] Building CXX object hello_query_device/CMakeFiles/hello_query_device.dir/main.cpp.o
[ 36%] Building CXX object hello_nv12_input_classification/CMakeFiles/hello_nv12_input_classification.dir/main.cpp.o
[ 40%] Building CXX object speech_sample/CMakeFiles/speech_sample.dir/main.cpp.o
[ 43%] Building CXX object classification_sample_async/CMakeFiles/classification_sample_async.dir/main.cpp.o
[ 46%] Building CXX object style_transfer_sample/CMakeFiles/style_transfer_sample.dir/main.cpp.o
[ 50%] Building CXX object benchmark_app/CMakeFiles/benchmark_app.dir/inputs_filling.cpp.o
[ 53%] Building CXX object hello_reshape_ssd/CMakeFiles/hello_reshape_ssd.dir/main.cpp.o
[ 56%] Building CXX object hello_classification/CMakeFiles/hello_classification.dir/main.cpp.o
[ 60%] Building CXX object benchmark_app/CMakeFiles/benchmark_app.dir/main.cpp.o
[ 63%] Linking CXX executable ../intel64/Release/hello_query_device
[ 63%] Built target hello_query_device
[ 66%] Building CXX object benchmark_app/CMakeFiles/benchmark_app.dir/statistics_report.cpp.o
[ 70%] Linking CXX executable ../intel64/Release/classification_sample_async
[ 70%] Built target classification_sample_async
[ 73%] Building CXX object benchmark_app/CMakeFiles/benchmark_app.dir/utils.cpp.o
[ 76%] Linking CXX executable ../intel64/Release/hello_reshape_ssd
[ 80%] Linking CXX executable ../intel64/Release/hello_nv12_input_classification
[ 80%] Built target hello_nv12_input_classification
[ 80%] Built target hello_reshape_ssd
Scanning dependencies of target object_detection_sample_ssd
[ 83%] Linking CXX executable ../intel64/Release/hello_classification
[ 86%] Building CXX object object_detection_sample_ssd/CMakeFiles/object_detection_sample_ssd.dir/main.cpp.o
[ 90%] Linking CXX executable ../intel64/Release/style_transfer_sample
[ 90%] Built target hello_classification
[ 90%] Built target style_transfer_sample
[ 93%] Linking CXX executable ../intel64/Release/speech_sample
[ 93%] Built target speech_sample
[ 96%] Linking CXX executable ../intel64/Release/object_detection_sample_ssd
[ 96%] Built target object_detection_sample_ssd
[100%] Linking CXX executable ../intel64/Release/benchmark_app
[100%] Built target benchmark_app
Scanning dependencies of target ie_samples
[100%] Built target ie_samples

Build completed, you can find binaries for all samples in the ${HOME}/inference_engine_cpp_samples_build/intel64/Release subfolder.
```

```console:Installing_additional_packages
${HDDL_INSTALL_DIR}/install_IVAD_VPU_dependencies.sh
$ sudo reboot
$ cd ${HOME}/inference_engine_cpp_samples_build/intel64/Release
```


```console:Benchmark_FP32_(Before_Integer_Quantization)
$ ./benchmark_app \
  -m ${HOME}/openvino_quantization/results/efficientnet-b0-pytorch_DefaultQuantization/2020-03-28_00-48-49/fp32/efficientnet-b0-pytorch.xml \
  -i ${HOME}/Pictures \
  -d CPU \
  -nthreads 8

Count:      7636 iterations
Duration:   60037.62 ms
Latency:    26.61 ms
Throughput: 127.19 FPS
Peak Virtual Memory (VmPeak) Size, kBytes: 1174064
Peak Resident Memory (VmHWM) Size, kBytes:  276328
```

```console:Benchmark_Integer_Quantization_(Fake_Quantize)
$ ./benchmark_app \
  -m ${HOME}/openvino_quantization/results/efficientnet-b0-pytorch_DefaultQuantization/2020-03-28_00-48-49/optimized/efficientnet-b0-pytorch.xml \
  -i ${HOME}/Pictures \
  -d CPU \
  -nthreads 8

Count:      3284 iterations
Duration:   60132.62 ms
Latency:    63.86 ms
Throughput: 54.61 FPS
Peak Virtual Memory (VmPeak) Size, kBytes: 1718448
Peak Resident Memory (VmHWM) Size, kBytes:  280536
```

# ４．Reference articles
**https://docs.openvinotoolkit.org/latest/_docs_Workbench_DG_Install_from_Docker_Hub.html**

# ５．Model structure
## ５−１． Model structure visualization website
**https://lutzroeder.github.io/netron/**
![Screenshot 2020-03-28 11:35:19.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/194769/fdd04130-63fb-a077-ffa7-c84212cb4afe.png)

## ５−２． Part of the structure of EfficientNet D0

<details><summary>**EfficientNet D0 FP32**</summary><div>

![FireShot Capture 007 - efficientnet-b0-pytorch.xml - lutzroeder.github.io.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/194769/76c784c8-28da-eb84-724e-81d796c88eca.png)

</div></details>

<details><summary>**EfficientNet D0 INT8**</summary><div>

![FireShot Capture 006 - efficientnet-b0-pytorch.xml - lutzroeder.github.io.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/194769/43a9e654-2a45-6491-1fe6-1993b8b94ac8.png)

</div></details>