Repository: opencv
Issue: opencv_interactive-calibration fails to open gstreamer pipeline for some usb cameras
Labels: priority: low,category: videoio(camera),category: 3rdparty

##### System information (version)
- OpenCV => 4.5.1
- Operating System / Platform => Debian 10, Kernel 4.19.0-13-amd64
- Compiler => gcc

##### Detailed description

I am trying to create camera calibration files for different cameras like:
```bash
opencv_interactive-calibration -t=chessboard -vis=window -w=6 -h=9 -of=camera.xml -pf=defaultConfig.xml
```

This works for a fish eye camera without any problems but fails for the Logitech C920 camera.

Default config for the fish eye camera looks like:
```xml
<?xml version="1.0"?>
<opencv_storage>
    <charuco_dict>0</charuco_dict>
    <charuco_square_length>200</charuco_square_length>
    <charuco_marker_size>100</charuco_marker_size>
    <calibration_step>1</calibration_step>
    <max_frames_num>30</max_frames_num>
    <min_frames_num>10</min_frames_num>
    <solver_eps>1e-7</solver_eps>
    <solver_max_iters>30</solver_max_iters>
    <fast_solver>0</fast_solver>
    <frame_filter_conv_param>0.1</frame_filter_conv_param>
    <camera_resolution>1280 720</camera_resolution>
</opencv_storage>
```

And for the Logitech C920:
```xml
<?xml version="1.0"?>
<opencv_storage>
    <charuco_dict>0</charuco_dict>
    <charuco_square_length>200</charuco_square_length>
    <charuco_marker_size>100</charuco_marker_size>
    <calibration_step>1</calibration_step>
    <max_frames_num>30</max_frames_num>
    <min_frames_num>10</min_frames_num>
    <solver_eps>1e-7</solver_eps>
    <solver_max_iters>30</solver_max_iters>
    <fast_solver>0</fast_solver>
    <frame_filter_conv_param>0.1</frame_filter_conv_param>
    <camera_resolution>1920 1080</camera_resolution>
</opencv_storage>
```

When running the calibration command for the fish eye camera everything works and the calibration file is created as expected.

However using the exact same steps and only changing the camera resolution in the default config file from (1280x720) to (1920x1080) and using the logitech camera the program fails with following output:

```txt
Hot keys:
esc - exit application
s - save current data to .xml file
r - delete last frame
u - enable/disable applying undistortion
d - delete all frames
v - switch visualization
Gtk-Message: 13:05:36.980: Failed to load module "canberra-gtk-module"
[ WARN:0] global /home/joel/CLionProjects/r4yApi/opencv/opencv-master/modules/videoio/src/cap_gstreamer.cpp (961) open OpenCV | GStreamer warning: Cannot query video position: status=0, value=-1, duration=-1
[ WARN:0] global /home/joel/CLionProjects/r4yApi/opencv/opencv-master/modules/videoio/src/cap_gstreamer.cpp (1825) handleMessage OpenCV | GStreamer warning: Embedded video playback halted; module v4l2src0 reported: Internal data stream error.
[ WARN:0] global /home/joel/CLionProjects/r4yApi/opencv/opencv-master/modules/videoio/src/cap_gstreamer.cpp (536) startPipeline OpenCV | GStreamer warning: unable to start pipeline
[ WARN:0] global /home/joel/CLionProjects/r4yApi/opencv/opencv-master/modules/videoio/src/cap_gstreamer.cpp (1085) setProperty OpenCV | GStreamer warning: no pipeline
[ WARN:0] global /home/joel/CLionProjects/r4yApi/opencv/opencv-master/modules/videoio/src/cap_gstreamer.cpp (992) getProperty OpenCV | GStreamer warning: GStreamer: no pipeline
[ WARN:0] global /home/joel/CLionProjects/r4yApi/opencv/opencv-master/modules/videoio/src/cap_gstreamer.cpp (992) getProperty OpenCV | GStreamer warning: GStreamer: no pipeline
[ WARN:0] global /home/joel/CLionProjects/r4yApi/opencv/opencv-master/modules/videoio/src/cap_gstreamer.cpp (1085) setProperty OpenCV | GStreamer warning: no pipeline
[ WARN:0] global /home/joel/CLionProjects/r4yApi/opencv/opencv-master/modules/videoio/src/cap_gstreamer.cpp (1085) setProperty OpenCV | GStreamer warning: no pipeline
[ WARN:0] global /home/joel/CLionProjects/r4yApi/opencv/opencv-master/modules/videoio/src/cap_gstreamer.cpp (1085) setProperty OpenCV | GStreamer warning: no pipeline
[ WARN:0] global /home/joel/CLionProjects/r4yApi/opencv/opencv-master/modules/videoio/src/cap_gstreamer.cpp (992) getProperty OpenCV | GStreamer warning: GStreamer: no pipeline
[ WARN:0] global /home/joel/CLionProjects/r4yApi/opencv/opencv-master/modules/videoio/src/cap_gstreamer.cpp (992) getProperty OpenCV | GStreamer warning: GStreamer: no pipeline
Unable to open video source
[ WARN:0] global /home/joel/CLionProjects/r4yApi/opencv/opencv-master/modules/videoio/src/cap_gstreamer.cpp (501) isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created
```

##### Additional Information

For the fish eye camera:
```bash
gst-device-monitor-1.0
```

```txt
Device found:

	name  : J1455
	class : Video/Source
	caps  : image/jpeg, width=(int)1280, height=(int)720, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30/1;
	        image/jpeg, width=(int)640, height=(int)480, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30/1;
	        image/jpeg, width=(int)320, height=(int)240, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30/1;
	properties:
		udev-probed = true
		device.bus_path = pci-0000:00:14.0-usb-0:2:1.0
		sysfs.path = /sys/devices/pci0000:00/0000:00:14.0/usb1/1-2/1-2:1.0/video4linux/video0
		device.bus = usb
		device.subsystem = video4linux
		device.vendor.id = 0603
		device.vendor.name = "NOVATEK\\x20"
		device.product.id = 8612
		device.product.name = J1455
		device.serial = NOVATEK_J1455
		device.capabilities = :capture:
		device.api = v4l2
		device.path = /dev/video0
		v4l2.device.driver = uvcvideo
		v4l2.device.card = "J1455\ "
		v4l2.device.bus_info = usb-0000:00:14.0-2
		v4l2.device.version = 267168 (0x000413a0)
		v4l2.device.capabilities = 2225078273 (0x84a00001)
		v4l2.device.device_caps = 69206017 (0x04200001)
	gst-launch-1.0 v4l2src ! ...

```

For the Logitech C920:

```bash
gst-device-monitor-1.0
```

```txt
Device found:

	name  : HD Pro Webcam C920
	class : Video/Source
	caps  : video/x-raw, format=(string)YUY2, width=(int)2560, height=(int)1472, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)2/1;
	        video/x-raw, format=(string)YUY2, width=(int)1920, height=(int)1080, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)5/1;
	        video/x-raw, format=(string)YUY2, width=(int)1600, height=(int)896, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 15/2, 5/1 };
	        video/x-raw, format=(string)YUY2, width=(int)1280, height=(int)720, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 10/1, 15/2, 5/1 };
	        video/x-raw, format=(string)YUY2, width=(int)960, height=(int)720, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 15/1, 10/1, 15/2, 5/1 };
	        video/x-raw, format=(string)YUY2, width=(int)1024, height=(int)576, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 15/1, 10/1, 15/2, 5/1 };
	        video/x-raw, format=(string)YUY2, width=(int)800, height=(int)600, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 };
	        video/x-raw, format=(string)YUY2, width=(int)864, height=(int)480, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 };
	        video/x-raw, format=(string)YUY2, width=(int)800, height=(int)448, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 };
	        video/x-raw, format=(string)YUY2, width=(int)640, height=(int)480, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 };
	        video/x-raw, format=(string)YUY2, width=(int)640, height=(int)360, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 };
	        video/x-raw, format=(string)YUY2, width=(int)432, height=(int)240, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 };
	        video/x-raw, format=(string)YUY2, width=(int)352, height=(int)288, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 };
	        video/x-raw, format=(string)YUY2, width=(int)320, height=(int)240, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 };
	        video/x-raw, format=(string)YUY2, width=(int)320, height=(int)180, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 };
	        video/x-raw, format=(string)YUY2, width=(int)176, height=(int)144, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 };
	        video/x-raw, format=(string)YUY2, width=(int)160, height=(int)120, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 };
	        video/x-raw, format=(string)YUY2, width=(int)160, height=(int)90, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 };
	        image/jpeg, width=(int)1920, height=(int)1080, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 };
	        image/jpeg, width=(int)1600, height=(int)896, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 };
	        image/jpeg, width=(int)1280, height=(int)720, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 };
	        image/jpeg, width=(int)960, height=(int)720, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 };
	        image/jpeg, width=(int)1024, height=(int)576, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 };
	        image/jpeg, width=(int)800, height=(int)600, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 };
	        image/jpeg, width=(int)864, height=(int)480, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 };
	        image/jpeg, width=(int)800, height=(int)448, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 };
	        image/jpeg, width=(int)640, height=(int)480, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 };
	        image/jpeg, width=(int)640, height=(int)360, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 };
	        image/jpeg, width=(int)432, height=(int)240, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 };
	        image/jpeg, width=(int)352, height=(int)288, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 };
	        image/jpeg, width=(int)320, height=(int)240, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 };
	        image/jpeg, width=(int)320, height=(int)180, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 };
	        image/jpeg, width=(int)176, height=(int)144, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 };
	        image/jpeg, width=(int)160, height=(int)120, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 };
	        image/jpeg, width=(int)160, height=(int)90, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 };
	properties:
		udev-probed = true
		device.bus_path = pci-0000:00:14.0-usb-0:2:1.0
		sysfs.path = /sys/devices/pci0000:00/0000:00:14.0/usb1/1-2/1-2:1.0/video4linux/video0
		device.bus = usb
		device.subsystem = video4linux
		device.vendor.id = 046d
		device.vendor.name = 046d
		device.product.id = 08e5
		device.product.name = "HD\ Pro\ Webcam\ C920"
		device.serial = 046d_HD_Pro_Webcam_C920_1E2150DF
		device.capabilities = :capture:
		device.api = v4l2
		device.path = /dev/video0
		v4l2.device.driver = uvcvideo
		v4l2.device.card = "HD\ Pro\ Webcam\ C920"
		v4l2.device.bus_info = usb-0000:00:14.0-2
		v4l2.device.version = 267168 (0x000413a0)
		v4l2.device.capabilities = 2225078273 (0x84a00001)
		v4l2.device.device_caps = 69206017 (0x04200001)
	gst-launch-1.0 v4l2src ! ...
```

The calibration module fails to open a stream but I can create it manually in my program and do my processing:
```cpp
video_stream = cv::VideoCapture( "v4l2src device=/dev/video" VIDEO_DEVICE " ! image/jpeg,framerate=30/1,width=1920, height=1080,type=video ! jpegdec ! videoconvert ! video/x-raw ! appsink",
                cv::CAP_GSTREAMER);
```
(`VIDEO_DEVICE` is defined as `0`)

The above works for both cameras (when changing the resolution of course), maybe the calibration module uses `video/-x-raw` directly from the logitech cam and this fails somehow. As my initialisation of the pipeline above always uses `image/jpeg` for both cameras and then coverts it. (At least I think so, I am not that familiar with gstreamer)

##### Steps to reproduce

1. Plug in Logitech C920 Camera into your PC
2. Create the `defaultConfig.xml` as defined above
3. Run `opencv_interactive-calibration -t=chessboard -vis=window -w=6 -h=9 -of=camera.xml -pf=defaultConfig.xml`
4. Error because the pipeline cannot be created

##### Issue submission checklist

 - [x] I report the issue, it's not a question
   <!--
   OpenCV team works with forum.opencv.org, Stack Overflow and other communities
   to discuss problems. Tickets with question without real issue statement will be
   closed.
   -->
 - [x] I checked the problem with documentation, FAQ, open issues,
       forum.opencv.org, Stack Overflow, etc and have not found solution
   <!--
   Places to check:
   * OpenCV documentation: https://docs.opencv.org
   * FAQ page: https://github.com/opencv/opencv/wiki/FAQ
   * OpenCV forum: https://forum.opencv.org
   * OpenCV issue tracker: https://github.com/opencv/opencv/issues?q=is%3Aissue
   * Stack Overflow branch: https://stackoverflow.com/questions/tagged/opencv
   -->
 - [x] I updated to latest OpenCV version and the issue is still there
   <!--
   master branch for OpenCV 4.x and 3.4 branch for OpenCV 3.x releases.
   OpenCV team supports only latest release for each branch.
   The ticket is closed, if the problem is not reproduced with modern version.
   -->
 - [ ] There is reproducer code and related data files: videos, images, onnx, etc
   <!--
   The best reproducer -- test case for OpenCV that we can add to the library.
   Recommendations for media files and binary files:
   * Try to reproduce the issue with images and videos in opencv_extra repository
     to reduce attachment size
   * Use PNG for images, if you report some CV related bug, but not image reader
     issue
   * Attach the image as archive to the ticket, if you report some reader issue.
     Image hosting services compress images and it breaks the repro code.
   * Provide ONNX file for some public model or ONNX file with with random weights,
     if you report ONNX parsing or handling issue. Architecture details diagram
     from netron tool can be very useful too. See https://lutzroeder.github.io/netron/
   -->