Jetsonで動作したgst-launch-1.0 カメラパイプライン集

## 配線

|デバイス名|ポート|種類|
|:--|:--|:--|
|/dev/video0|cam0|IMX219 CSI Camera |
|/dev/video1|cam1|IMX219 CSI Camera |
|/dev/video2|USB|Logicool 720p|
|/dev/video3|USB|Logicool 720p|
|/dev/video4|USB|Insta360|
|/dev/video5|USB|Insta360|

## Jetapckのバージョン確認

```
cat /etc/nv_tegra_release
```

```
# R32 (release), REVISION: 5.1, GCID: 26202423, BOARD: t186ref, EABI: aarch64, DATE: Fri Feb 19 16:50:29 UTC 2021
```

##  取り込み

|ライブラリ|処理|
|:--|:--|
| nvarguscamerasrc | ARGUS APIを使用したカメラプラグイン |
| nvv4l2camerasrc | V4L2 APIを使用したカメラプラグイン |

## CSIカメラ取り込み(nvarguscamerasrc)

ヘルプの表示

```
nvgstcapture-1.0 --help
```

nvarguscamerasrchは、L4T Multimedia APIの[Libargus Camera API](https://docs.nvidia.com/jetson/l4t-multimedia/group__LibargusAPI.html)がベースになっており、CSIカメラのRawデータをGPUメモリ空間(memory:NVMM)に高速に取り込む事が可能。

cam0を取り込む

```
gst-launch-1.0 \
nvarguscamerasrc sensor-id=0 ! \
nv3dsink
```

cam1を取り込む

```
gst-launch-1.0 \
nvarguscamerasrc sensor-id=1 ! \
nv3dsink
```

サイズ指定で取り込む

```
gst-launch-1.0 \
nvarguscamerasrc sensor-id=0 ! \
'video/x-raw(memory:NVMM), width=1280, height=720, framerate=60/1' ! \
nv3dsink
```

```
gst-launch-1.0 \
nvarguscamerasrc sensor-id=0 ! \
'video/x-raw(memory:NVMM), width=3264, height=2464, framerate=21/1' ! \
nv3dsink
```

```
gst-launch-1.0 \
nvarguscamerasrc sensor-id=0 ! \
nvinfer config-file-path=/opt/nvidia/deepstream/deepstream-6.0/samples/configs/deepstream-app/config_infer_primary.txt ! \
nvmultistreamtiler rows=1 columns=1 width=960 height=720 ! \
nvvideoconvert ! \
nvdsosd ! \
nvegltransform ! \
nveglglessink
```

## USBカメラ

nvv4l2camerasrcを用いて取り込む。nvv4l2camerasrcで取り込んだ映像は、nvv4l2decoderやnvvideoconvert等で、GPUメモリ空間(memory:NVMM)に展開する。

USBカメラのMotionJpegを取り込む

```
gst-launch-1.0 \
v4l2src device="/dev/video2" ! \
'image/jpeg, width=1280, height=720, framerate=30/1' ! \
nvv4l2decoder mjpeg=1  ! \
nvvidconv ! \
nvegltransform ! \
nveglglessink sync=no
```

USBカメラのYUYVを取り込む

```
gst-launch-1.0 \
v4l2src device="/dev/video2" ! \
'video/x-raw, width=640, height=480, framerate=30/1' ! \
nvvideoconvert ! \
nv3dsink
```

USBカメラのH264を取り込む

```
gst-launch-1.0 \
v4l2src device="/dev/video4" ! \
'video/x-h264, width=1920, height=960, framerate=30/1' ! \
nvv4l2decoder !  \
nv3dsink sync=no
```


## 変換系API

|ライブラリ|処理|
|:--|:--|
|nvvidconv|フォーマット変換、スケーリング処理 |
|nvvideoconvert| データのメモリ空間移動(CPUメモリ空間, GPUメモリ空間) NVMMからNVMMへの移動, RAWからNVMMへの移動, NVMMからRAWへの移動, RAWからRAW bufferへの移動 |

## VideoSkin API

|ライブラリ|処理|
|:--|:--|
| nvoverlaysink | (deprecated) [OpenMAX IL](https://wiki.onakasuita.org/pukiwiki/?OpenMAX)を使用したVideoSkin |
| nveglglessink | X11やWaylandで使える[EGL/GLES](https://ja.wikipedia.org/wiki/OpenGL_ES)を使用したVideoSkin |
| nv3dsink | [EGL/GLES](https://ja.wikipedia.org/wiki/OpenGL_ES)を使用したVideoSkin | 
| nvdrmvideosink | DRM（デジタル著作権管理）対応のVideoSkin |

CSI Cameraテスト

```
gst-launch-1.0 \
nvarguscamerasrc sensor-id=0 ! \
nvoverlaysink
```

```
gst-launch-1.0 \
nvarguscamerasrc sensor-id=0 ! \
nvegltransform ! \
nveglglessink
```

```
gst-launch-1.0 \
nvarguscamerasrc sensor-id=0 ! \
nv3dsink
```
USBカメラ(MJPEG)テスト

```
gst-launch-1.0 \
v4l2src device="/dev/video2" ! \
'image/jpeg, width=1280, height=720, framerate=30/1' ! \
nvv4l2decoder mjpeg=1 ! \
nvvidconv ! \
nvegltransform ! \
nveglglessink sync=no
```

USBカメラ(YUYV)テスト

```
gst-launch-1.0 \
v4l2src device="/dev/video2" ! \
'video/x-raw, width=640, height=480, framerate=30/1' ! \
nvvideoconvert ! \
nveglglessink 
```

```
gst-launch-1.0 \
v4l2src device="/dev/video2" ! \
'video/x-raw, width=640, height=480, framerate=30/1' ! \
nvvideoconvert ! \
nv3dsink
```

USBカメラ(H264)テスト

```
gst-launch-1.0 \
v4l2src device="/dev/video4" ! \
'video/x-h264, width=1920, height=960, framerate=30/1' ! \
nvv4l2decoder !  \
nvoverlaysink sync=no
```

```
gst-launch-1.0 \
v4l2src device="/dev/video4" ! \
'video/x-h264, width=1920, height=960, framerate=30/1' ! \
nvv4l2decoder !  \
nvvidconv ! \
nvegltransform ! \
nveglglessink sync=no
```

```
gst-launch-1.0 \
v4l2src device="/dev/video4" ! \
'video/x-h264, width=1920, height=960, framerate=30/1' ! \
nvv4l2decoder !  \
nv3dsink sync=no
```

## Docoder

|ライブラリ|処理|
|:--|:--|
| omxh265dec | OpenMAX IL H.265 Video decoder |
| omxh264dec | OpenMAX IL H.264 Video decoder |
| omxmpeg4videodec | OpenMAX IL MPEG4 Video decoder |
| omxmpeg2videodec | OpenMAX IL MPEG2 Video decoder |
| omxvp8dec | OpenMAX IL VP8 Video decoder |
| omxvp9dec | OpenMAX IL VP9 video decoder |
| nvv4l2decoder | V4L2 H.265, H.264, VP8, VP9, MPEG4, MPEG2 Video decoder|

nvv4l2decoderでほぼ解決できる。

nvv4l2decoderで、MotionJpegを取り込みたい場合は、mjpeg=1オプションをつける

```
nvv4l2decoder mjpeg=1
```

## Encoder

|ライブラリ|処理|
|:--|:--|
| omxh265dec | OpenMAX IL H.265 Video decoder |
| omxh264dec | OpenMAX IL H.264 Video decoder |
| omxmpeg4videodec | OpenMAX IL MPEG4 Video decoder |
| omxmpeg2videodec | OpenMAX IL MPEG2 Video decoder |
| omxvp8dec | OpenMAX IL VP8 Video decoder |
| omxvp9dec | OpenMAX IL VP9 video decoder |
| nvv4l2decoder | V4L2 H.265, H.264, VP8, VP9, MPEG4, MPEG2 Video decoder|
| nvjpegdec | JPEG decoder element |

## nvcompositor

nvcompositorで複数カメラから取り込んだ画像の結合ができる

2個のUSBカメラ(JetPack4.6)

```
gst-launch-1.0 \
nvcompositor name=m \
sink_0::xpos=0 sink_0::ypos=0 \
sink_1::xpos=1280 sink_1::ypos=0 ! \
nvegltransform ! \
nveglglessink sync=no \
v4l2src device=/dev/video2 io-mode=2 ! \
'image/jpeg, width=1280, height=720, framerate=30/1' ! \
nvv4l2decoder mjpeg=1 ! \
queue ! \
nvvidconv ! \
"video/x-raw(memory:NVMM),format=RGBA" ! \
m.sink_0 \
v4l2src device=/dev/video3 io-mode=2 ! \
'image/jpeg, width=1280, height=720, framerate=30/1' ! \
nvv4l2decoder mjpeg=1 ! \
queue ! \
nvvidconv ! \
"video/x-raw(memory:NVMM),format=RGBA" ! \
m.sink_1
```

2個USBカメラ(JetPack4.5.1)

```
gst-launch-1.0 \
nvcompositor name=m \
sink_0::xpos=0 sink_0::ypos=0 \
sink_1::xpos=1280 sink_1::ypos=0 ! \
nvegltransform ! \
nveglglessink sync=no \
v4l2src device=/dev/video2 io-mode=2 ! \
'image/jpeg, width=1280, height=720, framerate=30/1' ! \
jpegdec ! \
nvvidconv ! \
"video/x-raw(memory:NVMM),format=RGBA" ! \
m.sink_0 \
v4l2src device=/dev/video3 io-mode=2 ! \
'image/jpeg, width=1280, height=720, framerate=30/1' ! \
jpegdec ! \
nvvidconv ! \
"video/x-raw(memory:NVMM),format=RGBA" ! \
m.sink_1
```

2個のCSIカメラ

```
gst-launch-1.0 \
nvcompositor name=m \
sink_0::xpos=0 sink_0::ypos=0 \
sink_1::xpos=1280 sink_1::ypos=0 ! \
nvegltransform ! \
nveglglessink sync=no \
nvarguscamerasrc sensor-id=0 ! \
'video/x-raw(memory:NVMM), width=1280, height=720, framerate=30/1' ! \
queue ! \
nvvidconv ! \
m.sink_0 \
nvarguscamerasrc sensor-id=1 ! \
'video/x-raw(memory:NVMM), width=1280, height=720, framerate=30/1' ! \
queue ! \
nvvidconv ! \
m.sink_1
```

1個CSIと1個USBカメラ

```
gst-launch-1.0 \
nvcompositor name=m \
sink_0::xpos=0 sink_0::ypos=0 \
sink_1::xpos=1280 sink_1::ypos=0 ! \
nvegltransform ! \
nveglglessink sync=no \
nvarguscamerasrc sensor-id=0 ! \
'video/x-raw(memory:NVMM), width=1280, height=720, framerate=30/1' ! \
queue ! \
nvvidconv ! \
"video/x-raw(memory:NVMM),format=RGBA" ! \
m.sink_0 \
nvarguscamerasrc sensor-id=1 ! \
'video/x-raw(memory:NVMM), width=1280, height=720, framerate=30/1' ! \
queue ! \
nvvidconv ! \
"video/x-raw(memory:NVMM),format=RGBA" ! \
m.sink_1
```

2個のCSIカメラと,　2個のUSBカメラ(JetPack4.6)

```
gst-launch-1.0 \
nvcompositor name=m \
sink_0::xpos=0 sink_0::ypos=0 \
sink_1::xpos=1280 sink_1::ypos=0 \
sink_2::xpos=0 sink_2::ypos=720 \
sink_3::xpos=1280 sink_3::ypos=720 ! \
nvegltransform ! \
nveglglessink sync=no \
nvarguscamerasrc sensor-id=0 ! \
'video/x-raw(memory:NVMM), width=1280, height=720, framerate=30/1' ! \
queue ! \
nvvidconv ! \
"video/x-raw(memory:NVMM),format=RGBA" ! \
m.sink_0 \
nvarguscamerasrc sensor-id=1 ! \
'video/x-raw(memory:NVMM), width=1280, height=720, framerate=30/1' ! \
queue ! \
nvvidconv ! \
"video/x-raw(memory:NVMM),format=RGBA" ! \
m.sink_1 \
v4l2src device=/dev/video2 io-mode=2 ! \
'image/jpeg, width=1280, height=720, framerate=30/1' ! \
nvv4l2decoder mjpeg=1 ! \
queue ! \
nvvidconv ! \
"video/x-raw(memory:NVMM),format=RGBA" ! \
m.sink_2 \
v4l2src device=/dev/video3 io-mode=2 ! \
'image/jpeg, width=1280, height=720, framerate=30/1' ! \
nvv4l2decoder mjpeg=1 ! \
queue ! \
nvvidconv ! \
"video/x-raw(memory:NVMM),format=RGBA" ! \
m.sink_3
```

2個のCSIカメラと,　2個のUSBカメラ(JetPack4.5.1)

```
gst-launch-1.0 \
nvcompositor name=m \
sink_0::xpos=0 sink_0::ypos=0 \
sink_1::xpos=1280 sink_1::ypos=0 \
sink_2::xpos=0 sink_2::ypos=720 \
sink_3::xpos=1280 sink_3::ypos=720 ! \
nvegltransform ! \
nveglglessink sync=no \
nvarguscamerasrc sensor-id=0 ! \
'video/x-raw(memory:NVMM), width=1280, height=720, framerate=30/1' ! \
queue ! \
nvvidconv ! \
"video/x-raw(memory:NVMM),format=RGBA" ! \
m.sink_0 \
nvarguscamerasrc sensor-id=1 ! \
'video/x-raw(memory:NVMM), width=1280, height=720, framerate=30/1' ! \
queue ! \
nvvidconv ! \
"video/x-raw(memory:NVMM),format=RGBA" ! \
m.sink_1 \
v4l2src device=/dev/video2 io-mode=2 ! \
'image/jpeg, width=1280, height=720, framerate=30/1' ! \
jpegdec ! \
nvvidconv ! \
"video/x-raw(memory:NVMM),format=RGBA" ! \
m.sink_2 \
v4l2src device=/dev/video3 io-mode=2 ! \
'image/jpeg, width=1280, height=720, framerate=30/1' ! \
jpegdec ! \
nvvidconv ! \
"video/x-raw(memory:NVMM),format=RGBA" ! \
m.sink_3
```

2個のCSIカメラと,　2個のUSBカメラ(MJPEG), 2個のUSBカメラ(H264) (JetPack4.6)

```
gst-launch-1.0 \
nvcompositor name=m \
sink_0::xpos=0 sink_0::ypos=0 \
sink_1::xpos=1280 sink_1::ypos=0 \
sink_2::xpos=0 sink_2::ypos=720 \
sink_3::xpos=1280 sink_3::ypos=720 \
sink_4::xpos=0 sink_4::ypos=1440 \
sink_5::xpos=1920 sink_5::ypos=1440 ! \
nvegltransform ! \
nveglglessink sync=no \
nvarguscamerasrc sensor-id=0 ! \
'video/x-raw(memory:NVMM), width=1280, height=720, framerate=30/1' ! \
queue ! \
nvvidconv ! \
"video/x-raw(memory:NVMM),format=RGBA" ! \
m.sink_0 \
nvarguscamerasrc sensor-id=1 ! \
'video/x-raw(memory:NVMM), width=1280, height=720, framerate=30/1' ! \
queue ! \
nvvidconv ! \
"video/x-raw(memory:NVMM),format=RGBA" ! \
m.sink_1 \
v4l2src device=/dev/video2 io-mode=2 ! \
'image/jpeg, width=1280, height=720, framerate=30/1' ! \
nvv4l2decoder mjpeg=1 ! \
queue ! \
nvvidconv ! \
"video/x-raw(memory:NVMM),format=RGBA" ! \
m.sink_2 \
v4l2src device=/dev/video3 io-mode=2 ! \
'image/jpeg, width=1280, height=720, framerate=30/1' ! \
nvv4l2decoder mjpeg=1 ! \
queue ! \
nvvidconv ! \
"video/x-raw(memory:NVMM),format=RGBA" ! \
m.sink_3 \
v4l2src device=/dev/video4 io-mode=2 ! \
'video/x-h264, width=1920, height=960, framerate=30/1' ! \
nvv4l2decoder ! \
queue ! \
nvvidconv ! \
"video/x-raw(memory:NVMM),format=RGBA" ! \
m.sink_4 \
v4l2src device=/dev/video5 io-mode=2 ! \
'video/x-h264, width=1920, height=960, framerate=30/1' ! \
nvv4l2decoder ! \
queue ! \
nvvidconv ! \
"video/x-raw(memory:NVMM),format=RGBA" ! \
m.sink_5
```