OpenCV with GStreamer and QT on Windows | by Imran Bangash | Towards Data Science

OpenCV is an open-source computer vision library aimed mainly at real-time systems. It is based on C++ and offers optimized code for vision processing across different platforms, Windows, Linux, FreeBSD, macOS, etc. Thanks to the active community of developers and researchers, the code is readily available for different steps of the imaging pipelines; starting from an image/video capturing, calibration, pre-processing, feature extraction, to classification. While written in C++, OpenCV has bindings for different languages such as python and Java.
GStreamer is a multimedia framework that supports different media handling components such as audio, video, recording and streaming, etc. It provides high modularity by offering a user to create different pipelines and seamlessly integrate different plugins.While both libraries; OpenCV and GStreamer can be used standalone, however the combination of two offers greater flexibility for handling multimedia streams and data processing.
Building OpenCV with GStreamer on Linux environment could be straight forward as compared to Windows because of the availability of greater online support and installation procedure. However for rapid prototyping, one might feel a need to have OpenCV with GStreamer and QT configuration on the windows environment.
In this article, a step by step guide is provided for building OpenCV from source with GStreamer, QT , OpenCV contrib and CUDA packages. At the end of the tutorial, we will develop a GStreamer pipeline using OpenCV to stream webcam to localhost, receive the data and show the stream using GStreamer-CLI. In the next section, we will download the source files and install the required tools for building OpenCV with GStreamer.
Download source files from OpenCV GitHub repository; it is recommended to download the latest release. This experiment is based on OpenCV 4.3.0
Download source files from OpenCV GitHub repository; it is recommended to download the latest release. This experiment is based on OpenCV 4.3.0
2. Download OpenCV contrib files from the same release tag as the main OpenCV. In this case, it is 4.3.0
3. Download and install CMake. I have downloaded 3.17.2.
4. Download and install GStreamer development package and runtime installer for MSVC 64-bit (VS 2019)
1.16.2 runtime installer , Required for running the program
1.16.2 development installer, REquired for build process
5. Download and install Visual Studio Community edition 2017 or 2019. I have installed 2017 with the MSVC 64-bit compiler.
Microsoft C++ Build tools are included in the Visual Studio installation process but for some reason, I had to download it separately.
6. Download and install QT Creator opensource edition using QT online Installer. I have downloaded QT Creator 4.12.0.
Extract the OpenCV, OpenCV contrib and GSreamter and start configuring the build process.
Configure the setup
Configure the setup
Create a build folder and point to it using CMake as shown in the figure below. Also, give a path to the `modules` folder in OpenCV_EXTRA_MODULES_PATH field.
Check the boxes of OPENCV_DNN_CUDA
Check the box of QT
Check the box of WITH_GSTREMER and give the path of the GStreamer development package in CMake value. Remember to configure this step carefully.
Reference paths are provided below. Please change it according to your settings.
C:/gstreamer/1.0/x86_64/lib/gstapp-1.0.libC:/gstreamer/1.0/x86_64/lib/gstbase-1.0.libC:/gstreamer/1.0/x86_64/include/glib-2.0C:/gstreamer/1.0/x86_64/lib/glib-2.0.libC:/gstreamer/1.0/x86_64/lib/glib-2.0/includeC:/gstreamer/1.0/x86_64/lib/gobject-2.0.libC:/gstreamer/1.0/x86_64/include/gstreamer-1.0C:/gstreamer/1.0/x86_64/lib/gstreamer-1.0.libC:/gstreamer/1.0/x86_64/lib/gstpbutils-1.0.libC:/gstreamer/1.0/x86_64/lib/gstriff-1.0.lib
Configure compiler and select x64 in the Optional platform. We already selected x64 bit in one of the previous steps.
Press the configure option and check the output to see GStreamer and QT has been configured correctly.
QT: YES (ver 5.14.2)GStreamer: YES (1.16.2)
Now press the Generate button
Your files are ready for the Build and Install step.
Build OpenCV in release mode. First change build procedure to x64 mode. Right-click on ALL_BUILD and press build.
Build OpenCV in release mode. First change build procedure to x64 mode. Right-click on ALL_BUILD and press build.
2. Install the files in release mode. Right-click on INSTALL and press build.
3. Repeat the above two steps for debug if you want to build in debug mode also.
4. Your files are ready to be used in QT creator with GStreamer. We need include folder, bin, etc and lib folders from x64 folder for the next step.
5. Create a folder Opencv34 and copy include folder, bin, etc, and lib folders from step4.
We will use the localhost (127.0.0.1) with Port number 50050 to test the sender and receiver GStreamer pipelines on the same computer. Some good references to make your own GStreamer pipelines [link] [link]. GStreamer live camera streaming pipeline is given below as OpenCV code whereas the receiving pipeline is given as GStreamer-CLI.
GStreamer live camera streaming pipeline using OpenCV can be accessed from the repository
GStreamer pipeline for receiving the camera stream is given using Command Line Interface (CLI). Type in the given command in CLI. Challenge; Implement this GStreamer-CLI pipeline in OpenCV and share your GitHub repository as a comment in response.
gst-launch-1.0 udpsrc port=50050 caps=application/x-rtp ! rtpjpegdepay ! jpegparse ! jpegdec ! videoconvert ! autovideosink