How to Implement Barcode Scanner with Webcam Using Qt QCamera
Tags: qt, qr, cpp, webcam
Published: 2021-08-23 05:25:14

A few days ago, I wrote a [blog post](https://www.dynamsoft.com/codepool/qt-desktop-barcode-reader.html) about how to implement a desktop barcode reader application using Qt and C++. The application only supports still image mode. In this article, I am going to add a webcam support to empower the application to scan barcodes in real time.

## CMake Configuration for QCamera on Windows
To add webcam support in Qt application, we need to import the `QCamera` class from `QtMultimedia` library. In my Windows Qt environment, I only installed Qt 6.1.2, which does not include `QtMultimedia` library. According to the [Qt documentation](https://doc.qt.io/qt-5/qcamera.html), we can find the `QCamera` class in Qt 5. 

So, I need to run the Qt maintenance tool to install Qt 5.12.11. 

![Qt maintenance tool](https://www.dynamsoft.com/codepool/img/2021/08/qt-maintenance-tool.png)

Thereafter update the PATH environment variables to downgrade the build tools.

![Qt Windows tool path](https://www.dynamsoft.com/codepool/img/2021/08/qt-windows-build-tool.png)

The QCamera header file is located in `Qt/5.12.11/mingw73_64/include/QtMultimedia/qcamera.h`.

In `CMakeLists.txt` file, I change Qt version to 5 and link the `QtMultimedia` library:

```cmake
cmake_minimum_required(VERSION 3.5)

project(BarcodeReader VERSION 0.1 LANGUAGES CXX)

set(CMAKE_INCLUDE_CURRENT_DIR ON)

set(CMAKE_AUTOUIC ON)
set(CMAKE_AUTOMOC ON)
set(CMAKE_AUTORCC ON)

set(CMAKE_CXX_STANDARD 11)
set(CMAKE_CXX_STANDARD_REQUIRED ON)

if (CMAKE_HOST_WIN32)
    link_directories("${PROJECT_SOURCE_DIR}/platform/windows/lib/") 
elseif(CMAKE_HOST_UNIX)
    link_directories("${PROJECT_SOURCE_DIR}/platform/linux/")
endif()
include_directories("${PROJECT_SOURCE_DIR}/include/")

find_package(Qt5 COMPONENTS Widgets REQUIRED)
find_package(Qt5MultimediaWidgets REQUIRED)

set(PROJECT_SOURCES
        main.cpp
        mainwindow.cpp
        mainwindow.h
        mainwindow.ui
        myvideosurface.h
        myvideosurface.cpp
)

add_executable(${PROJECT_NAME} ${PROJECT_SOURCES})

if (CMAKE_HOST_WIN32)
    target_link_libraries(${PROJECT_NAME} PRIVATE Qt5::Widgets Qt5::MultimediaWidgets "DBRx64")    
elseif(CMAKE_HOST_UNIX)
    target_link_libraries(${PROJECT_NAME} PRIVATE Qt5::Widgets Qt5::MultimediaWidgets "DynamsoftBarcodeReader")
endif()

if(CMAKE_HOST_WIN32)
    add_custom_command(TARGET ${PROJECT_NAME} POST_BUILD 
        COMMAND ${CMAKE_COMMAND} -E copy_directory
        "${PROJECT_SOURCE_DIR}/platform/windows/bin/"      
        $<TARGET_FILE_DIR:${PROJECT_NAME}>)
endif()
```

The `myvideosurface.h` file and `myvideosurface.cpp` file are used to display the webcam preview and recognize barcodes in real time. I will talk about the implementation in the next section.

## Real-time Barcode Scanning  
QCamera contains a `setViewfinder()` function that can be used to display the webcam preview. The supported parameters of `setViewfinder()` include `QVideoWidget`, `QGraphicsVideoItem` and `QAbstractVideoSurface`.

My goal is to get preview frames from the webcam for barcode detection. Therefore, I should subclass the `QAbstractVideoSurface` class and get the frame data in `present()` method:

I create a `myvideosurface.h` file:

```cpp
#ifndef MYVIDEOSURFACE_H
#define MYVIDEOSURFACE_H

#include <QImage>
#include <QPixmap>
#include <QDebug>
#include <QCamera>
#include <QCameraInfo>
#include <QAbstractVideoSurface>
#include <QLabel>
#include <QDateTime>

#include "DynamsoftCommon.h"
#include "DynamsoftBarcodeReader.h"

QT_BEGIN_NAMESPACE
namespace Ui
{
    class MainWindow;
}
QT_END_NAMESPACE

class MyVideoSurface : public QAbstractVideoSurface
{
    Q_OBJECT
private:
    Ui::MainWindow *ui;
    void *reader;
    bool is_detecting;

public:
    MyVideoSurface(QObject *parent, Ui::MainWindow *ui, void *reader);
    ~MyVideoSurface();

    void reset();

    QList<QVideoFrame::PixelFormat>
    supportedPixelFormats(QAbstractVideoBuffer::HandleType type) const;

    bool present(const QVideoFrame &frame);
};
#endif // MYVIDEOSURFACE_H

```

The constructor of `MyVideoSurface` class takes a `QObject` pointer, a `Ui::MainWindow` pointer and a `void` pointer. The `Ui::MainWindow` pointer is used to access all the UI widgets. The `void *reader` pointer is used to detect barcodes.

Next, I create a `MyVideoSurface.cpp` file to add the implementation of `MyVideoSurface` class.

The `supportedPixelFormats()` is to get the supported pixel formats of the webcam. Which format should I use? Honestly, I don't know. I use `QVideoFrame::Format_RGB24` at first, but it seems that `QVideoFrame::Format_RGB24` is not supported by my webcam. According to the error message, I use `QVideoFrame::Format_ARGB32` instead:

```cpp
QList<QVideoFrame::PixelFormat> MyVideoSurface::supportedPixelFormats(QAbstractVideoBuffer::HandleType type) const
{
    return QList<QVideoFrame::PixelFormat>()
               << QVideoFrame::Format_RGB32;
}
```

In `present()` function, I get the frame data from the `QVideoFrame` object and convert it to `QImage` object:

```cpp
bool MyVideoSurface::present(const QVideoFrame &frame)
{
    if (frame.isValid() && is_detecting)
    {
        QVideoFrame cloneFrame(frame);
        cloneFrame.map(QAbstractVideoBuffer::ReadOnly);
        const QImage img(cloneFrame.bits(),
                         cloneFrame.width(),
                         cloneFrame.height(),
                         QVideoFrame::imageFormatFromPixelFormat(cloneFrame.pixelFormat()));

        cloneFrame.unmap();

        return true;
    }
    return false;
}

```

To display the preview frame, I need to make a copy of the frame data:

```cpp
QImage cp = img.copy();
```

In my case, the image I get from the `QVideoFrame` object is vertically inverted. I can flip the image with `QImage::mirrored()` function:

```cpp
cp = cp.mirrored(false, true);
```

Then, I call `DBR_DecodeBuffer()` to detect barcodes in the frame and draw the bounding boxes on the image:

```cpp
int ret = DBR_DecodeBuffer(reader, (unsigned char *)cp.bits(), cp.width(), cp.height(), cp.bytesPerLine(), IPF_ARGB_8888, "");

DBR_GetAllTextResults(reader, &handler);

QPixmap pm = QPixmap::fromImage(cp);
QPainter painter(&pm);
painter.setPen(Qt::red);

QString out = "";
TextResult **results = handler->results;
for (int index = 0; index < handler->resultsCount; index++)
{
    LocalizationResult* localizationResult = results[index]->localizationResult;
    out += "Index: " + QString::number(index) + ", Elapsed time: " + QString::number(start.msecsTo(end)) + "ms\n";
    out += "Barcode format: " + QString(results[index]->barcodeFormatString) + "\n";
    out += "Barcode value: " + QString(results[index]->barcodeText) + "\n";
    out += "Bounding box: (" + QString::number(localizationResult->x1) + ", " + QString::number(localizationResult->y1) + ") "
    + "(" + QString::number(localizationResult->x2) + ", " + QString::number(localizationResult->y2) + ") "
    + "(" + QString::number(localizationResult->x3) + ", " + QString::number(localizationResult->y3) + ") "
    + "(" + QString::number(localizationResult->x4) + ", " + QString::number(localizationResult->y4) + ")\n";
    out += "----------------------------------------------------------------------------------------\n";

    painter.drawLine(localizationResult->x1, localizationResult->y1, localizationResult->x2, localizationResult->y2);
    painter.drawLine(localizationResult->x2, localizationResult->y2, localizationResult->x3, localizationResult->y3);
    painter.drawLine(localizationResult->x3, localizationResult->y3, localizationResult->x4, localizationResult->y4);
    painter.drawLine(localizationResult->x4, localizationResult->y4, localizationResult->x1, localizationResult->y1);
}

DBR_FreeTextResults(&handler);

painter.end();
```

Finally, I display the image on the label and show the barcode recognition results in the text box:

```cpp
ui->label->setPixmap(pm.scaled(ui->label->size(), Qt::KeepAspectRatio, Qt::SmoothTransformation));
ui->textEdit_results->setText(out);
```

So far, the `myvideosurface` part is finished. I can go to the `mainwindow.cpp` to add code for the camera control logic.

Open the `mainwindow.ui` in Qt Creator and add the camera control buttons.

![Qt UI button](https://www.dynamsoft.com/codepool/img/2021/08/qt-ui-camera-button.png)

In the `MainWindow` constructor, I check the camera status and initialize the camera object and surface object:

```cpp
QList<QCameraInfo> cameras = QCameraInfo::availableCameras();
if (cameras.size() > 0) 
{
    for( int i = 0; i < cameras.count(); ++i )
    { 
        QCameraInfo cameraInfo = cameras.at(i);
        qDebug() << cameraInfo.deviceName();
        qDebug() << cameraInfo.description();
        camera = new QCamera(cameraInfo);
        surface = new MyVideoSurface(this, ui, reader);
        camera->setViewfinder(surface);
        break;
    }
}
else {
    ui->pushButton_open->setEnabled(false);
    ui->pushButton_stop->setEnabled(false);
}
```

After that, I bind the button click events to the corresponding functions:

```cpp
connect(ui->pushButton_open, SIGNAL(clicked()), this, SLOT(startCamera()));
connect(ui->pushButton_stop, SIGNAL(clicked()), this, SLOT(stopCamera()));

void MainWindow::startCamera()
{
    surface->reset();
    camera->start();
}

void MainWindow::stopCamera()
{
    camera->stop();
}
```

The barcode scanner is ready to use. Let's build and run the application:

```cpp
mkdir build
cd build
cmake -G "MinGW Makefiles" ..
cmake --build .
BarcodeReader.exe
```

![Qt C++ GUI barcode reader](https://www.dynamsoft.com/codepool/img/2021/08/barcode-scanner-webcam-qcamera.png)

## Source Code
[https://github.com/yushulx/Qt-desktop-barcode-reader](https://github.com/yushulx/Qt-desktop-barcode-reader)