GTC2021 11月の講演で使ったDeepStream/GStreamer pipeline集

# JetPack4.6でのDeepStreamSDK6.0の動作状況

GTC2021の講演時、JetPack4.6でDeepStreamSDK6.0がTensorRTバージョン不一致で動かない旨、発表してしまいましたが、DeepStreamSDK6.0 Betaの検証結果で、DeepStreamSDK6.0の正式版リリースではJetPack4.6上でも問題なく動作する事を確認いたしました。発表内容を訂正いたします。

https://qiita.com/akira-sasaki/items/8fa3eee3d1b6efdc89a9

# 使用機材

|USBカメラ|URL|
|:--|:--|
|C920|[Web](https://www.logicool.co.jp/ja-jp/products/webcams/hd-pro-webcam-c920n.960-001261.html)|
|OCAM-VRU01BK|[Web](https://www.amazon.co.jp/%E3%82%A8%E3%83%AC%E3%82%B3%E3%83%A0-360%E5%BA%A6%E3%82%AB%E3%83%A1%E3%83%A9-%E3%82%B9%E3%83%9E%E3%83%9B%E7%9B%B4%E6%8C%BF%E3%81%97%E3%82%BF%E3%82%A4%E3%83%97-%E3%82%AA%E3%83%A0%E3%83%8B%E3%82%B7%E3%83%A7%E3%83%83%E3%83%88%E3%83%9F%E3%83%8B-OCAM-VRU01BK/dp/B07BKF66GF?th=1) |


## nvvideoconvert(JetPack4.5.1)

![nvvideoconvert.jpg](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/63450/e66861df-ab04-6790-88af-3d2485f8a7ab.jpeg)


```
gst-launch-1.0 \
v4l2src device="/dev/video0" io-mode=2 ! \
'image/jpeg, width=1280, height=720, framerate=30/1' ! \
jpegdec ! \
tee name=t ! \
queue ! \
nvvideoconvert ! \
"video/x-raw(memory:NVMM),format=RGBA" ! \
nvegltransform ! \
nveglglessink \
window-x=50 window-y=0 \
window-width=355 window-height=200 \
t. ! \
queue ! \
nvvideoconvert src-crop=”20:40:640:360” ! \
"video/x-raw(memory:NVMM),format=RGBA" ! \
nvegltransform ! \
nveglglessink \
window-x=405 window-y=0 \
window-width=355 window-height=200 \
t. ! \
queue ! \
nvvideoconvert dest-crop=”200:200:640:320” ! \
"video/x-raw(memory:NVMM),format=RGBA" ! \
nvegltransform ! \
nveglglessink \
window-x=760 window-y=0 \
window-width=355 window-height=200
```

## nvvidconv(JetPack4.5.1)

![nvvidconv.jpg](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/63450/3cb36f7c-38e0-0c7b-2b13-c32177a157f8.jpeg)


```
gst-launch-1.0 \
v4l2src device="/dev/video0" io-mode=2 ! \
'image/jpeg, width=1280, height=720, framerate=30/1' ! \
jpegdec ! \
tee name=t ! \
queue ! \
nvvidconv ! \
"video/x-raw(memory:NVMM),format=RGBA" ! \
nvegltransform ! \
nveglglessink \
window-x=50 window-y=0 \
window-width=355 window-height=200 \
t. ! \
queue ! \
nvvidconv flip-method=1 ! \
"video/x-raw(memory:NVMM),format=RGBA" ! \
nvegltransform ! \
nveglglessink \
window-x=405 window-y=0 \
window-width=355 window-height=200 \
t. ! \
queue ! \
nvvidconv flip-method=2 ! \
"video/x-raw(memory:NVMM),format=RGBA" ! \
nvegltransform ! \
nveglglessink \
window-x=760 window-y=0 \
window-width=355 window-height=200
```

## 360度カメラ x 2個(JetPack4.5.1)

![normal2cam.jpg](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/63450/dd926a9b-544f-6624-4df4-d74272b6659b.jpeg)


```
gst-launch-1.0 \
nvcompositor name=m \
sink_0::xpos=0 sink_0::ypos=0 \
sink_1::xpos=0 sink_1::ypos=736 ! \
nvegltransform ! \
nveglglessink sync=no \
v4l2src device=/dev/video0 io-mode=2 ! \
'image/jpeg, width=1472, height=736, framerate=30/1' ! \
jpegdec ! \
nvvidconv ! \
"video/x-raw(memory:NVMM),format=RGBA" ! \
m.sink_0 \
v4l2src device=/dev/video1 io-mode=2 ! \
'image/jpeg, width=1472, height=736, framerate=30/1' ! \
jpegdec ! \
nvvidconv ! \
"video/x-raw(memory:NVMM),format=RGBA" ! \
m.sink_1
```

## nvdewrper(JetPack4.5.1)

![nvdewarp.jpg](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/63450/c6bf933f-0597-b7ee-d2b1-9565a85a4cc3.jpeg)


```
gst-launch-1.0 v4l2src device="/dev/video0" ! \
'image/jpeg, width=1472, height=736, framerate=30/1' ! \
jpegdec ! \
nvvideoconvert ! \
'video/x-raw(memory:NVMM)' ! \
nvdewarper config-file=cam0_dewarp_config.txt  ! \
m.sink_0 nvstreammux name=m width=600 height=480 \
batch-size=2 \
num-surfaces-per-frame=2 ! \
nvmultistreamtiler width=600 height=480 ! \
nvegltransform ! \
nveglglessink sync=0 v4l2src device="/dev/video1" ! \
'image/jpeg, width=1472, height=736, framerate=30/1' ! \
jpegdec ! \
nvvideoconvert ! \
'video/x-raw(memory:NVMM)' ! \
nvdewarper config-file=cam1_dewarp_config.txt ! \
m.sink_1
```

dewarp_config.txt

```
gst-launch-1.0 v4l2src device="/dev/video0" ! \
'image/jpeg, width=1472, height=736, framerate=30/1' ! \
jpegdec ! \
nvvideoconvert ! \
'video/x-raw(memory:NVMM)' ! \
nvdewarper config-file=cam0_dewarp_config.txt  ! \
m.sink_0 nvstreammux name=m width=600 height=480 \
batch-size=2 \
num-surfaces-per-frame=2 ! \
nvmultistreamtiler width=600 height=480 ! \
nvegltransform ! \
nveglglessink sync=0 v4l2src device="/dev/video1" ! \
'image/jpeg, width=1472, height=736, framerate=30/1' ! \
jpegdec ! \
nvvideoconvert ! \
'video/x-raw(memory:NVMM)' ! \
nvdewarper config-file=cam1_dewarp_config.txt ! \
m.sink_1
```

## 4つのUSBカメラ(nvv4l2decoder未使用)(JetPack4.5.1)

![4camera.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/63450/c44ea4c6-5073-edbb-9233-eb852dccb53b.png)

```
gst-launch-1.0 \
nvcompositor name=m \
sink_0::xpos=0 sink_0::ypos=0 \
sink_1::xpos=960 sink_1::ypos=0  \
sink_2::xpos=0 sink_2::ypos=720 \
sink_3::xpos=960 sink_3::ypos=720 ! \
nvegltransform ! \
nveglglessink sync=no \
v4l2src device=/dev/video0 io-mode=2 ! \
'image/jpeg, width=960, height=720, framerate=30/1' ! \
jpegdec ! \
nvvidconv ! \
"video/x-raw(memory:NVMM),format=RGBA" ! \
m.sink_0 \
v4l2src device=/dev/video1 io-mode=2 ! \
'image/jpeg, width=960, height=720, framerate=30/1' ! \
jpegdec ! \
nvvidconv ! \
"video/x-raw(memory:NVMM),format=RGBA" ! \
m.sink_1 \
v4l2src device=/dev/video2 io-mode=2 ! \
'image/jpeg, width=960, height=720, framerate=30/1' ! \
jpegdec ! \
nvvidconv ! \
"video/x-raw(memory:NVMM),format=RGBA" ! \
m.sink_2 \
v4l2src device=/dev/video3 io-mode=2 ! \
'image/jpeg, width=960, height=720, framerate=30/1' ! \
jpegdec ! \
nvvidconv ! \
"video/x-raw(memory:NVMM),format=RGBA" ! \
m.sink_3
```

## 4つのUSBカメラ(nvv4l2decoder使用) (JetPack4.6)

![4camera.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/63450/c44ea4c6-5073-edbb-9233-eb852dccb53b.png)

```
gst-launch-1.0 \
nvcompositor name=m \
sink_0::xpos=0 sink_0::ypos=0 \
sink_1::xpos=1280 sink_1::ypos=0  \
sink_2::xpos=0 sink_2::ypos=720 \
sink_3::xpos=1280 sink_3::ypos=720 ! \
nvegltransform ! \
nveglglessink sync=no \
v4l2src device=/dev/video2 io-mode=2 ! \
'image/jpeg, width=1280, height=720, framerate=30/1' ! \
jpegdec ! \
nvvidconv ! \
"video/x-raw(memory:NVMM),format=RGBA" ! \
m.sink_0 \
v4l2src device=/dev/video3 io-mode=2 ! \
'image/jpeg, width=1280, height=720, framerate=30/1' ! \
jpegdec ! \
nvvidconv ! \
"video/x-raw(memory:NVMM),format=RGBA" ! \
m.sink_1 \
v4l2src device=/dev/video4 io-mode=2 ! \
'image/jpeg, width=1280, height=720, framerate=30/1' ! \
jpegdec ! \
nvvidconv ! \
"video/x-raw(memory:NVMM),format=RGBA" ! \
m.sink_2 \
v4l2src device=/dev/video5 io-mode=2 ! \
'image/jpeg, width=1280, height=720, framerate=30/1' ! \
jpegdec ! \
nvvidconv ! \
"video/x-raw(memory:NVMM),format=RGBA" ! \
m.sink_3
```

## 4カメラでObject Detection(JetPack4.5.1)

![4cam_ds.jpg](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/63450/34100806-fd2d-37a0-375e-b4ce240a0006.jpeg)


```
gst-launch-1.0 \
nvstreammux name=m batch-size=4 width=960 height=720 ! \
nvinfer config-file-path=/opt/nvidia/deepstream/deepstream-5.1/samples/configs/deepstream-app/config_infer_primary.txt ! \
nvmultistreamtiler rows=2 columns=2 width=960 height=720 ! \
nvvideoconvert ! \
nvdsosd ! \
nvegltransform ! \
nveglglessink sync=0 \
v4l2src device="/dev/video0" ! \
'image/jpeg, width=960, height=720, framerate=30/1' ! \
jpegdec ! \
nvvideoconvert ! \
'video/x-raw(memory:NVMM)' ! \
m.sink_0 \
v4l2src device="/dev/video1" ! \
'image/jpeg, width=960, height=720, framerate=30/1' ! \
jpegdec ! \
nvvideoconvert ! \
'video/x-raw(memory:NVMM)' ! \
m.sink_1 \
v4l2src device="/dev/video2" ! \
'image/jpeg, width=960, height=720, framerate=30/1' ! \
jpegdec ! \
nvvideoconvert ! \
'video/x-raw(memory:NVMM)' ! \
m.sink_2 \
v4l2src device="/dev/video3" ! \
'image/jpeg, width=960, height=720, framerate=30/1' ! \
jpegdec ! \
nvvideoconvert ! \
'video/x-raw(memory:NVMM)' ! \
m.sink_3
```

## 4カメラでOptical flowとObject Detection(JetPack4.5.1)

![fig001.jpg](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/63450/592fa3fd-d128-4ddc-3abb-6cfba54610ae.jpeg)

```
gst-launch-1.0 \
nvstreammux name=m batch-size=4 width=960 height=720 ! \
nvinfer config-file-path=dsanomaly_pgie_config.txt ! \
tee name=t ! \
nvof ! \
nvofvisual ! \
queue ! \
nvmultistreamtiler rows=2 columns=2 width=480 height=360 ! \
nvvideoconvert ! \
nvegltransform ! \
nveglglessink sync=0  sync=0 t. ! \
queue ! \
nvmultistreamtiler rows=2 columns=2 width=480 height=360 ! \
nvvideoconvert ! \
nvdsosd ! \
nvegltransform ! \
nveglglessink sync=0 \
v4l2src device="/dev/video0" ! \
'image/jpeg, width=960, height=720, framerate=30/1' ! \
jpegdec ! \
nvvideoconvert ! \
'video/x-raw(memory:NVMM)' ! \
m.sink_0 \
v4l2src device="/dev/video1" ! \
'image/jpeg, width=960, height=720, framerate=30/1' ! \
jpegdec ! \
nvvideoconvert ! \
'video/x-raw(memory:NVMM)' ! \
m.sink_1 \
v4l2src device="/dev/video2" ! \
'image/jpeg, width=960, height=720, framerate=30/1' ! \
jpegdec ! \
nvvideoconvert ! \
'video/x-raw(memory:NVMM)' ! \
m.sink_2 \
v4l2src device="/dev/video3" ! \
'image/jpeg, width=960, height=720, framerate=30/1' ! \
jpegdec ! \
nvvideoconvert ! \
'video/x-raw(memory:NVMM)' ! \
m.sink_3
```


```
GST_DEBUG=2 gst-launch-1.0 -v \
videomixer name=m \
sink_0::xpos=0 sink_0::ypos=0  \
sink_1::xpos=960 sink_1::ypos=0  \
sink_2::xpos=0 sink_2::ypos=720  \
sink_3::xpos=960 sink_3::ypos=720 ! \
nvvideoconvert ! \
'video/x-raw(memory:NVMM)' ! \
mux.sink_0 \
nvstreammux name=mux batch-size=1 width=1920 height=1440 ! \
nvinfer config-file-path=dsanomaly_pgie_config.txt ! \
nvmultistreamtiler rows=1 columns=1 width=960 height=720 ! \
nvvideoconvert ! \
nvegltransform ! \
nveglglessink sync=0 \
v4l2src device=/dev/video0 io-mode=2 ! \
'image/jpeg, width=960, height=720, framerate=30/1' ! \
jpegdec ! \
m.sink_0 \
v4l2src device=/dev/video1 io-mode=2 ! \
'image/jpeg, width=960, height=720, framerate=30/1' ! \
jpegdec ! \
m.sink_1 \
v4l2src device=/dev/video2 io-mode=2 ! \
'image/jpeg, width=960, height=720, framerate=30/1' ! \
jpegdec ! \
m.sink_2 \
v4l2src device=/dev/video3 io-mode=2 ! \
'image/jpeg, width=960, height=720, framerate=30/1' ! \
jpegdec ! \
m.sink_3
```


## 転移学習 TAO(JetPack4.5.1)

![tao.jpg](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/63450/c0ea6278-6234-158d-ecc2-77d9a7f1a805.jpeg)


```
gst-launch-1.0 \
nvstreammux name=m batch-size=4 width=960 height=720 ! \
nvinfer config-file-path=/home/jetson/deepstream_tao_apps/configs/ssd_tao/pgie_ssd_tao_config.txt ! \
nvmultistreamtiler rows=2 columns=2 width=1920 height=1440 ! \
nvvideoconvert ! \
nvdsosd ! \
nvegltransform ! \
nveglglessink sync=0 \
v4l2src device="/dev/video0" ! \
'image/jpeg, width=960, height=720, framerate=30/1' ! \
jpegdec ! \
nvvideoconvert ! \
'video/x-raw(memory:NVMM)' ! \
m.sink_0 \
v4l2src device="/dev/video1" ! \
'image/jpeg, width=960, height=720, framerate=30/1' ! \
jpegdec ! \
nvvideoconvert ! \
'video/x-raw(memory:NVMM)' ! \
m.sink_1 \
v4l2src device="/dev/video2" ! \
'image/jpeg, width=960, height=720, framerate=30/1' ! \
jpegdec ! \
nvvideoconvert ! \
'video/x-raw(memory:NVMM)' ! \
m.sink_2 \
v4l2src device="/dev/video3" ! \
'image/jpeg, width=960, height=720, framerate=30/1' ! \
jpegdec ! \
nvvideoconvert ! \
'video/x-raw(memory:NVMM)' ! \
m.sink_3
```