OpenCVのgstreamerでキャプチャすると遅延が少なかったメモ(仮)

## 概要
* [OpenCVのGstreamerバックエンドで高度な動画キャプチャを実現する](https://qiita.com/stnk20/items/242e400853579d511ea3)を読んで、今まで使っていたffmpegではなくgstreamerを試してみた

1. Docker imageで環境構築
2. 比較結果

## 結論
* 海外のカメラだと0.5秒くらい、手元のカメラだと3秒早く、遅延少なく感じた。gstreamerすごい！

## てｓｔ
* historyから必要そうなコマンドを引っ張っているので、間違っているかも...

### 環境構築
* docker使って動かす
* OpenCVは`/data/opencv-4.1.1`に、opencv_contribは`/data/opencv-4.1.1/opencv_contrib-4.1.1`に。

```
xhost +
docker run --runtime=nvidia -it --rm -e DISPLAY=$DISPLAY -e XDG_RUNTIME_DIR=$XDG_RUNTIME_DIR -v /tmp/.X11-unix/:/tmp/.X11-unix -e QT_X11_NO_MITSHM=1 -v /data:/data -w=/data nvidia/cuda:10.0-cudnn7-devel-ubuntu18.04
```

### 関連インストール
* 流用しているので不要なライブラリも含む

```
apt update
apt install cmake curl libeigen3-dev libgtk-3-dev qt5-default freeglut3-dev \
libvtk6-qt-dev libtbb-dev ffmpeg libdc1394-22-dev libavcodec-dev libavformat-dev \
libswscale-dev libjpeg-dev libpng++-dev libtiff5-dev \
libopenexr-dev libwebp-dev libhdf5-dev libpython3.6-dev \
libopenblas-dev liblapacke-dev
```

* python関連

```
curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
python3 get-pip.py --force-reinstall
pip install --no-cache-dir numpy scipy matplotlib opencv-python
```

* gstreamer関連

```
apt install libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev gstreamer1.0-plugins-base gstreamer1.0-plugins-good gstreamer1.0-libav gstreamer1.0-tools gstreamer1.0-rtsp
```

### OpenCVのbuild

```
cmake -G "Unix Makefiles" --build . -D BUILD_CUDA_STUBS=OFF -D BUILD_DOCS=OFF -D BUILD_EXAMPLES=ON -D BUILD_JASPER=OFF -D BUILD_JPEG=OFF -D BUILD_OPENEXR=OFF -D BUILD_PACKAGE=ON -D BUILD_PERF_TESTS=OFF -D BUILD_PNG=OFF -D BUILD_SHARED_LIBS=ON -D BUILD_TBB=OFF -D BUILD_TESTS=OFF -D BUILD_TIFF=OFF -D BUILD_WITH_DEBUG_INFO=ON -D BUILD_ZLIB=OFF -D BUILD_WEBP=OFF -D BUILD_opencv_apps=ON -D BUILD_opencv_calib3d=ON -D BUILD_opencv_core=ON -D BUILD_opencv_cudaarithm=ON -D BUILD_opencv_cudabgsegm=ON -D BUILD_opencv_cudacodec=ON -D BUILD_opencv_cudafeatures2d=ON -D BUILD_opencv_cudafilters=ON -D BUILD_opencv_cudaimgproc=ON -D BUILD_opencv_cudalegacy=ON -D BUILD_opencv_cudaobjdetect=ON -D BUILD_opencv_cudaoptflow=ON -D BUILD_opencv_cudastereo=ON -D BUILD_opencv_cudawarping=ON -D BUILD_opencv_cudev=ON -D BUILD_opencv_features2d=ON -D BUILD_opencv_flann=ON -D BUILD_opencv_highgui=ON -D BUILD_opencv_imgcodecs=ON -D BUILD_opencv_imgproc=ON -D BUILD_opencv_java=OFF -D BUILD_opencv_ml=ON -D BUILD_opencv_objdetect=ON -D BUILD_opencv_photo=ON -D BUILD_opencv_python2=OFF -D BUILD_opencv_python3=ON -D BUILD_opencv_shape=ON -D BUILD_opencv_stitching=ON -D BUILD_opencv_superres=ON -D BUILD_opencv_ts=ON -D BUILD_opencv_video=ON -D BUILD_opencv_videoio=ON -D BUILD_opencv_videostab=ON -D BUILD_opencv_viz=OFF -D BUILD_opencv_world=OFF -D CMAKE_BUILD_TYPE=RELEASE -D WITH_1394=ON -D WITH_CUBLAS=ON -D WITH_CUDA=ON -D WITH_CUFFT=ON -D WITH_EIGEN=ON -D WITH_FFMPEG=OFF -D WITH_GDAL=OFF -D WITH_GPHOTO2=OFF -D WITH_GIGEAPI=OFF -D WITH_GSTREAMER=ON -D WITH_GTK=ON -D WITH_INTELPERC=OFF -D WITH_IPP=ON -D WITH_IPP_A=OFF -D WITH_JASPER=OFF -D WITH_JPEG=ON -D WITH_LIBV4L=ON -D WITH_OPENCL=ON -D WITH_OPENCLAMDBLAS=OFF -D WITH_OPENCLAMDFFT=OFF -D WITH_OPENCL_SVM=OFF -D WITH_OPENEXR=ON -D WITH_OPENGL=ON -D WITH_OPENMP=OFF -D WITH_OPENNI=OFF -D WITH_PNG=ON -D WITH_PTHREADS_PF=OFF -D WITH_PVAPI=OFF -D WITH_QT=ON -D WITH_TBB=ON -D WITH_TIFF=ON -D WITH_UNICAP=OFF -D WITH_V4L=OFF -D WITH_VTK=OFF -D WITH_WEBP=ON -D WITH_XIMEA=OFF -D WITH_XINE=OFF -D WITH_LAPACKE=ON -D WITH_MATLAB=OFF -D OPENCV_GENERATE_PKGCONFIG=ON -D OPENCV_EXTRA_MODULES_PATH=../opencv_contrib-4.1.1/modules/ -D EIGEN_INCLUDE_PATH=/usr/include/eigen3 ..
make -j4
make install
```

### 実行

* `sample.py`をffmpeg版とgstreamer版のOpenCVをインストールしたdocker imageをそれぞれ動かしてテスト。

```python:sample.py
import cv2


cap = cv2.VideoCapture("rtspsrc location=rtsp://b1.dnsdojo.com:1935/live/sys3.stream latency=0 dulation=-1 ! decodebin ! videoconvert ! appsink", cv2.CAP_GSTREAMER)
# cap1 = cv2.VideoCapture("rtsp://b1.dnsdojo.com:1935/live/sys3.stream", cv2.CAP_FFMPEG)

while True:
    ret, img = cap.read()

    if not (ret):
        break
    cv2.imshow("gstreamer_latency0", img)
    key = cv2.waitKey(1)
    if key == 27: #[esc] key
        break

```

* 結果

![Screenshot from 2019-09-17 02-26-09.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/207420/0fb9a4b0-52b5-638c-f104-9bcd13373f99.png)


## まとめ
* gstreamer便利っぽいけど全然情報がないので、だれか使い方教えてほしい…