「アイエエエエ! ニンジャ!? ニンジャナンデ!?」(OpenCV module開発のおともに)

- この記事は[OpenCV Advent Calendar 2021](https://qiita.com/advent-calendar/2021/opencv)の7日目の記事です。
- 他の記事は目次にまとめられています。

■ 「アイエエエエ! ニンジャ!? ニンジャナンデ!?」(OpenCV module開発のおともに)
========================================================================

大規模になりつつあるOpenCVのコンパイル。「遅い」「待たされる」といった印象があるかもしれない。

特に、上手くアプリケーションソフトが動かない場合には、
OpenCVのライブラリ側にもデバッグログや試作コードを仕込んだりもしたくなるはず。

**「何度も何度も、OpenCV Moduleをコンパイルしたいが、そのために待たされるのは嫌！」**

こんな時に手助けになるツールとして、**Ninja** を紹介したい。


■ What is Ninja ?
==================

- https://ninja-build.org/

> Ninja is a small build system with a focus on speed. It differs from other build systems in two major respects: it is designed to have its input files generated by a higher-level build system, and it is designed to run builds as fast as possible.
>
> （和訳）Ninja とは スピードに焦点を当てた、small build systemである。他のbuild systemと比べて、２つの方向性が異なっている。これは、higher-level build systemによって生成された入力ファイルのために設計されており、そして、できる限り高速にbuildすることを目的に設計されている。

:::note info
（makeコマンドを使って並列化オプションを適切に設定できていると、それと比較すると）
動かしてみた感じ、初回のコンパイルから爆速になる！わけではない。
地味に、２回目、３回目のコンパイル時に、ちょっとだけ早くなる、そんな縁の下の力持ち的なツール。
:::

■テスト環境
==============

◯ハードウェア
---------------
- intel 4770K 3.2GHz  4C8T [^1]
- 8GB Memory

[^1]: 今年の冬、ボーナスが万が一間違って出たら買い替えるんだ・・・

◯ソフトウェア
---------------

![Ubuntu-21.10](https://img.shields.io/badge/Ubuntu-21.10-brightgreen)  ![OpenCV-4.5.4](https://img.shields.io/badge/OpenCV-4.5.4-brightgreen)  ![OpenCV_CONTRIB-4.5.4](https://img.shields.io/badge/OpenCV_CONTRIB-4.5.4-brightgreen)   ![GCC-11.2.0](https://img.shields.io/badge/GCC-11.2.0-brightgreen) 

■ OpenCVのbuildにNinjaを使うには？
==================================
いつものbuildだとこんな感じだろうか？

```bash:いつもの平凡なbuild方法
$ cmake -S opencv-4.5.4 -B build_make  -DOPENCV_EXTRA_MODULES_PATH=opencv_contrib-4.5.4/modules/
$ cd build_make
$ make -j 8
```

Ninjaのある生活には、（１）`cmake`のオプションに`-GNinja`を追加する。
そして、（２）`make`の代わりに`ninja`を実行する、たったこれだけでよい！

```bash:Ninjaのある生活
$ apt-get install ninja-build % これは初回だけ
$ $ cmake -S opencv-4.5.4 -B build_ninja -DOPENCV_EXTRA_MODULES_PATH=opencv_contrib-4.5.4/modules/ -GNinja
$ cd build_ninja
$ ninja -j 8
```

:::note info
使い方そのものはmakeとそこまで違わない。これならば、明日、いや今日からNinjaれますね！
:::


■テスト方法
=========
下記条件で何度か試して、real時間を比較した。

- (0) ninja-buildとccacheをあらかじめインストールしておく
- (1) makeコマンドとninjaコマンド、それぞれのコンパイル環境を作る。
- (2) 部分更新のテストに、 freetypeモジュールのソースコードをtouchし、リコンパイル対象にする。
- (2) ccacheのキャッシュをクリアしたり、残したままビルドをして、部分更新に要する時間を比較する。

■実行結果
=========

- 1. ccacheを有効にした状態で、1/6程にまで実行時間の短縮を確認できました。(1.2sec -> 0.20 sec)
- 2. ccacheなしmakeに比べて、ccacheありninjaは実行時間1/10にまで短縮ができました(2.1sec -> 0.20 sec)


|            | make -j8  | make -j8 (ccache) | ninja -j8  | ninja -j8 (ccache) |
|:-:|-:|-:|-:|-:|
| 全build    | - | 1397.793  | - | 1353.996  |
| 部分更新-1  |  2.077 |  1.211  | 1.326 | 0.198 |
| 部分更新-2  |  2.051 |  1.213  | 1.320 | 0.195  |
| 部分更新-3  |  2.082 |  1.206  | 1.320 | 0.195  |

:::note warn
ただし、全体をbuildする時間は多少短くなっている程度。爆速にはならない。
:::

:::note alert
「あれ？コンパイル対象になるモジュールを絞れば、速くなるのでは…」という真理に到達されると、非常に悲しいです。
:::

■まとめ
=========

- **「Ninja」**は、OpenCVのモジュールをデバッグ・開発する人もちょっとだけ幸せにしてくれます！！
- ccacheは有効にしておきましょう。

■ Appendix
==============

<details><summary>cmake結果（クリックすると開きます）</summary><div>

◯cmake結果
------------

```plaintext
-- General configuration for OpenCV 4.5.4 =====================================
--   Version control:               unknown
--
--   Extra modules:
--     Location (extra):            /home/kmtr/work/opencv_contrib-4.5.4/modules
--     Version control (extra):     unknown
--
--   Platform:
--     Timestamp:                   2021-12-01T20:37:02Z
--     Host:                        Linux 5.13.0-22-generic x86_64
--     CMake:                       3.18.4
--     CMake generator:             Ninja
--     CMake build tool:            /usr/bin/ninja
--     Configuration:               Release
--
--   CPU/HW features:
--     Baseline:                    SSE SSE2 SSE3
--       requested:                 SSE3
--     Dispatched code generation:  SSE4_1 SSE4_2 FP16 AVX AVX2 AVX512_SKX
--       requested:                 SSE4_1 SSE4_2 AVX FP16 AVX2 AVX512_SKX
--       SSE4_1 (17 files):         + SSSE3 SSE4_1
--       SSE4_2 (2 files):          + SSSE3 SSE4_1 POPCNT SSE4_2
--       FP16 (1 files):            + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 AVX
--       AVX (5 files):             + SSSE3 SSE4_1 POPCNT SSE4_2 AVX
--       AVX2 (32 files):           + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 FMA3 AVX AVX2
--       AVX512_SKX (8 files):      + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 FMA3 AVX AVX2 AVX_512F AVX512_COMMON AVX512_SKX
--
--   C/C++:
--     Built as dynamic libs?:      YES
--     C++ standard:                11
--     C++ Compiler:                /usr/bin/c++  (ver 11.2.0)
--     C++ flags (Release):         -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Wsuggest-override -Wno-delete-non-virtual-dtor -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections  -msse -msse2 -msse3 -fvisibility=hidden -fvisibility-inlines-hidden -O3 -DNDEBUG  -DNDEBUG
--     C++ flags (Debug):           -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Wsuggest-override -Wno-delete-non-virtual-dtor -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections  -msse -msse2 -msse3 -fvisibility=hidden -fvisibility-inlines-hidden -g  -O0 -DDEBUG -D_DEBUG
--     C Compiler:                  /usr/bin/cc
--     C flags (Release):           -fsigned-char -W -Wall -Werror=return-type -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wuninitialized -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections  -msse -msse2 -msse3 -fvisibility=hidden -O3 -DNDEBUG  -DNDEBUG
--     C flags (Debug):             -fsigned-char -W -Wall -Werror=return-type -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wuninitialized -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections  -msse -msse2 -msse3 -fvisibility=hidden -g  -O0 -DDEBUG -D_DEBUG
--     Linker flags (Release):      -Wl,--exclude-libs,libippicv.a -Wl,--exclude-libs,libippiw.a   -Wl,--gc-sections -Wl,--as-needed
--     Linker flags (Debug):        -Wl,--exclude-libs,libippicv.a -Wl,--exclude-libs,libippiw.a   -Wl,--gc-sections -Wl,--as-needed
--     ccache:                      YES
--     Precompiled headers:         NO
--     Extra dependencies:          dl m pthread rt
--     3rdparty dependencies:
--
--   OpenCV modules:
--     To be built:                 alphamat aruco barcode bgsegm bioinspired calib3d ccalib 
                                    core datasets dnn dnn_objdetect dnn_superres dpm face features2d 
                                    flann freetype fuzzy gapi hdf hfs highgui img_hash imgcodecs 
                                    imgproc intensity_transform line_descriptor mcc ml objdetect optflow 
                                    phase_unwrapping photo plot quality rapid reg rgbd saliency shape 
                                    stereo stitching structured_light superres surface_matching text 
                                    tracking ts video videoio videostab viz wechat_qrcode xfeatures2d 
                                    ximgproc xobjdetect xphoto
--     Disabled:                    world
--     Disabled by dependency:      -
--     Unavailable:                 cudaarithm cudabgsegm cudacodec cudafeatures2d cudafilters 
                                    cudaimgproc cudalegacy cudaobjdetect cudaoptflow cudastereo 
                                    cudawarping cudev cvv java julia matlab ovis python2 python3 sfm
--     Applications:                tests perf_tests apps
--     Applications:                tests perf_tests apps
--     Documentation:               NO
--     Non-free algorithms:         NO
--
--   GUI:                           NONE
--     GTK+:                        NO
--     VTK support:                 YES (ver 9.1.20211127)
--
--   Media I/O:
--     ZLib:                        /usr/lib/x86_64-linux-gnu/libz.so (ver 1.2.11)
--     JPEG:                        /usr/lib/x86_64-linux-gnu/libjpeg.so (ver 80)
--     WEBP:                        /usr/lib/x86_64-linux-gnu/libwebp.so (ver encoder: 0x020e)
--     PNG:                         /usr/lib/x86_64-linux-gnu/libpng.so (ver 1.6.37)
--     TIFF:                        /usr/lib/x86_64-linux-gnu/libtiff.so (ver 42 / 4.3.0)
--     JPEG 2000:                   OpenJPEG (ver 2.3.1)
--     OpenEXR:                     build (ver 2.3.0)
--     HDR:                         YES
--     SUNRASTER:                   YES
--     PXM:                         YES
--     PFM:                         YES
--
--   Video I/O:
--     DC1394:                      NO
--     FFMPEG:                      YES
--       avcodec:                   YES (58.134.100)
--       avformat:                  YES (58.76.100)
--       avutil:                    YES (56.70.100)
--       swscale:                   YES (5.9.100)
--       avresample:                NO
--     GStreamer:                   NO
--     v4l/v4l2:                    YES (linux/videodev2.h)
--
--   Parallel framework:            pthreads
--
--   Trace:                         YES (with Intel ITT)
--
--   Other third-party libraries:
--     Intel IPP:                   2020.0.0 Gold [2020.0.0]
--            at:                   /home/kmtr/work/build_ninja/3rdparty/ippicv/ippicv_lnx/icv
--     Intel IPP IW:                sources (2020.0.0)
--               at:                /home/kmtr/work/build_ninja/3rdparty/ippicv/ippicv_lnx/iw
--     VA:                          NO
--     Lapack:                      NO
--     Eigen:                       YES (ver 3.3.9)
--     Custom HAL:                  NO
--     Protobuf:                    build (3.5.1)
--
--   OpenCL:                        YES (no extra features)
--     Include path:                /home/kmtr/work/opencv-4.5.4/3rdparty/include/opencl/1.2
--     Link libraries:              Dynamic load
--
--   Python (for build):            /usr/bin/python3
--
--   Java:
--     ant:                         NO
--     JNI:                         /usr/lib/jvm/default-java/include /usr/lib/jvm/default-java/include/linux /usr/lib/jvm/default-java/include
--     Java wrappers:               NO
--     Java tests:                  NO
--
--   Install to:                    /usr/local
-- -----------------------------------------------------------------
--
-- Configuring done
-- Generating done
-- Build files have been written to: /home/kmtr/work/build_ninja

```

</div></details>

<details><summary>実行結果詳細（クリックすると開きます）</summary><div>

◯実行結果詳細
---------------

```bash
kmtr@kmtr-virtual-machine:~/work/build_make$ ccache -C
Clearing... 100.0% [===================================================================================================]
kmtr@kmtr-virtual-machine:~/work/build_make$ time make -j 8 > /dev/null

real    23m17.793s
user    167m18.139s
sys     13m39.182s

kmtr@kmtr-virtual-machine:~/work/build_make$ ccache -C && touch ../opencv_contrib-4.5.4/modules/freetype/src/freetype.cpp && time make -j 8 > /dev/null
Clearing... 100.0% [===================================================================================================]

real    0m2.077s
user    0m5.312s
sys     0m2.138s
kmtr@kmtr-virtual-machine:~/work/build_make$ ccache -C && touch ../opencv_contrib-4.5.4/modules/freetype/src/freetype.cpp && time make -j 8 > /dev/null
Clearing... 100.0% [===================================================================================================]

real    0m2.051s
user    0m5.292s
sys     0m2.183s
kmtr@kmtr-virtual-machine:~/work/build_make$ ccache -C && touch ../opencv_contrib-4.5.4/modules/freetype/src/freetype.cpp && time make -j 8 > /dev/null
Clearing... 100.0% [===================================================================================================]

real    0m2.082s
user    0m5.260s
sys     0m2.255s
kmtr@kmtr-virtual-machine:~/work/build_make$ touch ../opencv_contrib-4.5.4/modules/freetype/src/freetype.cpp && time make -j 8 > /dev/null

real    0m1.211s
user    0m3.935s
sys     0m2.036s
kmtr@kmtr-virtual-machine:~/work/build_make$ touch ../opencv_contrib-4.5.4/modules/freetype/src/freetype.cpp && time make -j 8 > /dev/null

real    0m1.213s
user    0m4.092s
sys     0m1.883s
kmtr@kmtr-virtual-machine:~/work/build_make$ touch ../opencv_contrib-4.5.4/modules/freetype/src/freetype.cpp && time make -j 8 > /dev/null

real    0m1.206s
user    0m3.993s
sys     0m2.008s

kmtr@kmtr-virtual-machine:~/work/build_make$ ccache -C
Clearing... 100.0% [===================================================================================================]
kmtr@kmtr-virtual-machine:~/work/build_make$ time make -j 8 > /dev/null

real    0m1.209s
user    0m3.895s
sys     0m1.796s
kmtr@kmtr-virtual-machine:~/work/build_make$ ccache -C
Clearing... 100.0% [===================================================================================================]
kmtr@kmtr-virtual-machine:~/work/build_make$ time make -j 8 > /dev/null

real    0m1.183s
user    0m3.774s
sys     0m1.911s
kmtr@kmtr-virtual-machine:~/work/build_make$ ccache -C
Clearing... 100.0% [===================================================================================================]
kmtr@kmtr-virtual-machine:~/work/build_make$ time make -j 8 > /dev/null

real    0m1.193s
user    0m3.962s
sys     0m1.770s

kmtr@kmtr-virtual-machine:~/work/build_ninja$ time ninja -j8 > /dev/null

real    22m33.996s
user    165m51.780s
sys     12m25.374s

kmtr@kmtr-virtual-machine:~/work/build_ninja$ touch ../opencv_contrib-4.5.4/modules/freetype/src/freetype.cpp && time ninja -j
8 > /dev/null

real    0m0.198s
user    0m0.149s
sys     0m0.049s
kmtr@kmtr-virtual-machine:~/work/build_ninja$ touch ../opencv_contrib-4.5.4/modules/freetype/src/freetype.cpp && time ninja -j 8 > /dev/null

real    0m0.195s
user    0m0.128s
sys     0m0.068s
kmtr@kmtr-virtual-machine:~/work/build_ninja$ touch ../opencv_contrib-4.5.4/modules/freetype/src/freetype.cpp && time ninja -j 8 > /dev/null

real    0m0.198s
user    0m0.124s
sys     0m0.074s
kmtr@kmtr-virtual-machine:~/work/build_ninja$ ccache -C && touch ../opencv_contrib-4.5.4/modules/freetype/src/freetype.cpp && time ninja -j 8 > /dev/null
Clearing... 100.0% [===================================================================================================]

real    0m1.326s
user    0m1.146s
sys     0m0.179s
kmtr@kmtr-virtual-machine:~/work/build_ninja$ ccache -C && touch ../opencv_contrib-4.5.4/modules/freetype/src/freetype.cpp && time ninja -j 8 > /dev/null
Clearing... 100.0% [===================================================================================================]

real    0m1.320s
user    0m1.154s
sys     0m0.165s
kmtr@kmtr-virtual-machine:~/work/build_ninja$ ccache -C && touch ../opencv_contrib-4.5.4/modules/freetype/src/freetype.cpp && time ninja -j 8 > /dev/null
Clearing... 100.0% [===================================================================================================]

real    0m1.320s
user    0m1.135s
sys     0m0.184s

```

</div>