How to Optimize QR Recognition Performance by Image Preprocessing and Parameter Tuning
Tags: opencv, qr, python, barcode
Published: 2021-09-13 01:46:12

When using a Barcode reader or scanner SDK, we primarily focus on the detection speed and recognition accuracy. There are many factors that may affect the performance of the Barcode detection SDK. The factors can be approximately classified into two categories: image quality and algorithm performance. In this article, we take [Dynamsoft Barcode Reader Python edition](https://www.dynamsoft.com/barcode-reader/programming/python/user-guide.html?ver=latest) as the example to discuss how to optimize the performance of the Barcode detection SDK by preprocessing input QR code images and adjusting the algorithm parameters on desktop platforms. 

## Prerequisites
- OpenCV Python
    
    ```bash
    pip install opencv-python
    pip install opencv-contrib-python
    ```
- Dynamsoft Barcode Reader SDK
    
    ```bash
    pip install dbr
    ```
- Download some `QR code` images by [Google Image Search](https://www.google.com.hk/search?q=qr+code&newwindow=1&sxsrf=AOaemvLFnv7npDqwIuSwMy65jrdUC0ZfOw:1631064312663&source=lnms&tbm=isch&sa=X&ved=2ahUKEwjL39iHnO7yAhXdw4sBHXLjCOUQ_AUoAXoECAEQAw&biw=2048&bih=1063#imgrc=jckwUEQ99goOrM)

## Preprocessing QR Images
To evaluate the SDK performance, we use color image, grayscale image, and binary image as the input images.

### Color Image
Let's get started with a color image of a QR code.

![Air-Force-Reserve-San-Diego](https://www.dynamsoft.com/codepool/img/2021/09/Air-Force-Reserve-San-Diego.jpg)

The following code loads and displays the image using OpenCV and reads barcode information using Dynamsoft Barcode Reader.

```python
import cv2
import numpy as np
import dbr
import time

reader = dbr.BarcodeReader()
reader.init_license("LICENSE-KEY") # https://www.dynamsoft.com/customer/license/trialLicense?product=dbr
image = cv2.imread('Air-Force-Reserve-San-Diego.jpg')

def detect(windowName, image, pixel_format):
    try:
        buffer = image.tobytes()
        height = image.shape[0]
        width = image.shape[1]
        stride = image.strides[0]
        start = time.time()
        results = reader.decode_buffer_manually(buffer, width, height, stride, pixel_format, "")
        end = time.time()
        print("Time taken: {:.3f}".format(end - start) + " seconds")
        cv2.putText(image, "Time taken: {:.3f}".format(end - start) + " seconds", (10, 30), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0, 0, 255), 2)

        if results != None:
            for result in results:
                print("Barcode Format : ")
                print(result.barcode_format_string)
                print("Barcode Text : ")
                print(result.barcode_text)

                points = result.localization_result.localization_points
                data = np.array([[points[0][0], points[0][1]], [points[1][0], points[1][1]], [points[2][0], points[2][1]], [points[3][0], points[3][1]]])
                cv2.drawContours(image=image, contours=[data], contourIdx=-1, color=(0, 255, 0), thickness=2, lineType=cv2.LINE_AA)

                x = min(points[0][0], points[1][0], points[2][0], points[3][0])
                y = min(points[0][1], points[1][1], points[2][1], points[3][1])
                cv2.putText(image, result.barcode_text, (x, y), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0, 0, 255), 2)

        cv2.imshow(windowName, image)
    except BarcodeReaderError as bre:
        print(bre)

detect('Color', image, dbr.EnumImagePixelFormat.IPF_RGB_888)
cv2.waitKey(0)
```

![QR code recognition](https://www.dynamsoft.com/codepool/img/2021/09/qr-color-image-recognition.png)

We initialize Dynamsoft Barcode Reader with the [license key](https://www.dynamsoft.com/customer/license/trialLicense?product=dbr).

```python
reader = dbr.BarcodeReader()
reader.init_license("LICENSE-KEY")
```

By default, OpenCV loads images in BGR format. The corresponding enum value of pixel format provided by Dynamsoft Barcode Reader is `dbr.EnumImagePixelFormat.IPF_RGB_888`.

Then we call the `decode_buffer_manually` function to decode the image. 

```python
buffer = image.tobytes()
height = image.shape[0]
width = image.shape[1]
stride = image.strides[0]
results = reader.decode_buffer_manually(buffer, width, height, stride, dbr.EnumImagePixelFormat.IPF_RGB_888, "")
```

The function returns a list of `BarcodeResult` objects. Each `BarcodeResult` object contains the barcode information and the localization result. The `drawContours` function and `putText` function can be used to draw the barcode information and localization result on the image.

```python
points = result.localization_result.localization_points
data = np.array([[points[0][0], points[0][1]], [points[1][0], points[1][1]], [points[2][0], points[2][1]], [points[3][0], points[3][1]]])
cv2.drawContours(image=image, contours=[data], contourIdx=-1, color=(0, 255, 0), thickness=2, lineType=cv2.LINE_AA)

x = min(points[0][0], points[1][0], points[2][0], points[3][0])
y = min(points[0][1], points[1][1], points[2][1], points[3][1])
cv2.putText(image, result.barcode_text, (x, y), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0, 0, 255), 2)
```

### Grayscale Image
Grayscale image only carries intensity information. The barcode recognition API internally converts a color image to a grayscale image. So
theoretically, if the input image is a grayscale image, the recognition speed is faster.

We run the following code to see the time cost:

```python
grayscale_image = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
detect('Grayscale', grayscale_image, dbr.EnumImagePixelFormat.IPF_GRAYSCALED)
```

![QR code recognition for grayscale image](https://www.dynamsoft.com/codepool/img/2021/09/qr-grayscale-image-recognition.png)

The pixel format is changed to `dbr.EnumImagePixelFormat.IPF_GRAYSCALED` accordingly. As expected, decoding the grayscale image converted from the original color image saves a little time.

### Binary Image
Thresholding the grayscale image to binary image can make it easier to analyze. We can also try it out by using the OpenCV function `threshold`:

```bash
# https://docs.opencv.org/4.5.1/d7/d4d/tutorial_py_thresholding.html
blur = cv2.GaussianBlur(grayscale_image,(5,5),0)
ret, thresh = cv2.threshold(blur, 0,255,cv2.THRESH_BINARY + cv2.THRESH_OTSU)
detect('Binary', thresh, dbr.EnumImagePixelFormat.IPF_GRAYSCALED)
```

![QR code recognition for binary image](https://www.dynamsoft.com/codepool/img/2021/09/qr-binary-image-recognition.png)

We use Gaussian blur to reduce noise and smooth the image, and then use the OpenCV function `threshold` to convert the grayscale image to binary image. The Otsu's method is used to automatically determine the optimal global threshold value from the image histogram.


| Image       | Time Cost |  
| ----------- | ----------- |
|Color Image | 0.014 s  | 
| Grayscale Image | 0.010 s        | 
| Binary Image |0.009 s |

### More Tests
Let's take a look at some complicated cases.

#### Perspective Distortion
Here is a perspective distorted QR code image. 

![perspective distorted QR code image](https://www.dynamsoft.com/codepool/img/2021/09/perspective-distortion-qr-code.jpg)

If we use the code above, there is no barcode recognized, no matter color image, grayscale image or binary image. However, it does not mean the barcode image cannot be recognized. I managed to recognize the QR code by manually adjusting and setting the threshold value:

```python
image = cv2.imread('perspective-distortion-qr-code.jpg')
grayscale_image = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
ret, thresh = cv2.threshold(grayscale_image, 125, 255,cv2.THRESH_BINARY) 
detect('Binary', thresh, dbr.EnumImagePixelFormat.IPF_GRAYSCALED)
```

```bash
Time taken: 0.0080 seconds
Barcode Format :
QR_CODE
Barcode Text :
http://goo.by/wb0Ric
```

![recognition for perspective distorted QR](https://www.dynamsoft.com/codepool/img/2021/09/perspective-distortion-qr-code-recognition.jpg)

#### Inverted Color

If we try to read the following QR code image printed with inverted color, it will fail to do the recognition.

![inverted color QR code](https://www.dynamsoft.com/codepool/img/2021/09/inverted-color-qr-code.jpg)

After converting the image to binary image, we can see the black and white are inverted for QR code:

![convert color image to binary image](https://www.dynamsoft.com/codepool/img/2021/09/inverted-color-qr-code-binary.jpg)

To make the QR code recognizable, we use OpenCV function `bitwise_not`:

```python
ret, thresh = cv2.threshold(grayscale_image, 150,255,cv2.THRESH_BINARY )
cv2.bitwise_not(thresh, thresh) 
detect('Inverted', thresh, dbr.EnumImagePixelFormat.IPF_GRAYSCALED)
```

```bash
Time taken: 0.0541 seconds
Barcode Format : 
QR_CODE
Barcode Text :
https://www.jalna.com.au/
```

![recognize inverted binary image](https://www.dynamsoft.com/codepool/img/2021/09/inverted-color-qr-code-binary-recognition.jpg)


## Adjusting the Algorithm Parameters
So far, we have tried some image preprocessing ways to read QR code. Actually, the barcode SDK contains more image processing and computer vision algorithms than the ones we have tried. To cover all cases, we need to know how to tune the parameters of the barcode SDK.

We use the following Python code to print the runtime parameters in console:

```python
print(reader.get_runtime_settings().__dict__)
```

```bash
{'terminate_phase': 32, 'timeout': 10000, 'max_algorithm_thread_count': 4, 'expected_barcodes_count': 0, 'barcode_format_ids': -31457281, 'barcode_format_ids_2': 0, 'pdf_raster_dpi': 300, 'scale_down_threshold': 2300, 'binarization_modes': [2, 0, 0, 0, 0, 0, 0, 0], 'localization_modes': [2, 16, 4, 8, 0, 0, 0, 0], 'colour_clustering_modes': [0, 0, 0, 0, 0, 0, 0, 0], 
'colour_conversion_modes': [1, 0, 0, 0, 0, 0, 0, 0], 'grayscale_transformation_modes': [2, 0, 0, 0, 0, 0, 0, 0], 'region_predetection_modes': [2, 0, 0, 0, 0, 0, 0, 0], 'image_preprocessing_modes': [2, 0, 0, 0, 0, 0, 0, 0], 'texture_detection_modes': [2, 0, 0, 0, 0, 0, 0, 0], 'text_filter_modes': [2, 0, 0, 0, 0, 0, 0, 0], 'dpm_code_reading_modes': [0, 0, 0, 0, 0, 0, 
0, 0], 'deformation_resisting_modes': [0, 0, 0, 0, 0, 0, 0, 0], 'barcode_complement_modes': [0, 0, 0, 0, 0, 0, 0, 0], 'barcode_colour_modes': [1, 0, 0, 0, 0, 0, 0, 0], 'text_result_order_modes': [1, 2, 4, 0, 0, 0, 0, 0], 'text_assisted_correction_mode': 2, 'deblur_level': 9, 'intermediate_result_types': 0, 'intermediate_result_saving_mode': 1, 'result_coordinate_type': 1, 'return_barcode_zone_clarity': 0, 'region_top': 0, 'region_bottom': 0, 'region_left': 0, 'region_right': 0, 'region_measured_by_percentage': 0, 'min_barcode_text_length': 0, 'min_result_confidence': 0, 'scale_up_modes': [1, 0, 0, 0, 0, 0, 0, 0], 'accompanying_text_recognition_modes': [0, 0, 0, 0, 0, 0, 0, 0], 'pdf_reading_mode': 1, 'deblur_modes': [0, 0, 0, 0, 0, 0, 0, 0, 0, 0], 'barcode_zone_min_distance_to_image_borders': 0}
```

There are quite a lot of parameters to tune, but you cannot find a crash course on how to tune these parameters. Fortunately, we can leverage the parameter tuning tool of Dynamsoft Barcode Reader [online demo](https://demo.dynamsoft.com/barcode-reader/) to customize parameters for specific images.

![QR code parameter settings](https://www.dynamsoft.com/codepool/img/2021/09/qr-code-parameter-settings.jpg)

There are five modes available for selection. You can click `Advanced Settings` to expand parameters of a mode. For better studying the parameter differences among different modes, we can scroll down the page and save all parameters to a json file. 

![QR code parameter settings](https://www.dynamsoft.com/codepool/img/2021/09/qr-code-parameter-details.jpg)


There is a tradeoff between the barcode decoding speed and recognition accuracy. The more algorithms are enabled for barcode recognition, the more time it takes.

**Best coverage vs. Best speed**
![QR code parameter settings](https://www.dynamsoft.com/codepool/img/2021/09/algorithm-comparison.jpg)

Now, we upload the perspective distorted QR image to see the difference between the best speed and best coverage.

**Best speed**
It takes 15 milliseconds but fails to recognize the QR code.

![QR perspective speed](https://www.dynamsoft.com/codepool/img/2021/09/qr-perspective-speed.jpg)


**Best coverage**
It successfully recognizes the QR code but takes 828 milliseconds.
![QR perspective coverage](https://www.dynamsoft.com/codepool/img/2021/09/qr-perspective-coverage.jpg)

Both of them are not ideal according to our test in Python code. To guarantee the best performance:
1. Go to `Advanced Settings > Binarization mode > Threshold`. 
2. Change the threshold value to `125`.

Now the QR code is recognized extremely fast. 

![QR perspective threshold](https://www.dynamsoft.com/codepool/img/2021/09/qr-perspective-threshold.jpg)

We can then export the template to a JSON file and load it in our Python code:

```python
error = reader.init_runtime_settings_with_file('perspective-qr.json')
image = cv2.imread('perspective-distortion-qr-code.jpg')
detect('Color', image_copy, dbr.EnumImagePixelFormat.IPF_RGB_888)
```

![QR perspective speed](https://www.dynamsoft.com/codepool/img/2021/09/qr-perspective-template-decoding.jpg)

The result is satisfying.

```bash
Time taken: 0.0080 seconds
Barcode Format :
QR_CODE
Barcode Text :
http://goo.by/wb0Ric
```

As for the inverted QR code, the setting is as follows:

![inverted QR code](https://www.dynamsoft.com/codepool/img/2021/09/inverted-color-qr-code-settings.jpg)

The performance is even better when reading the inverted QR code in Python.

![inverted QR code](https://www.dynamsoft.com/codepool/img/2021/09/inverted-qr-speed.jpg)


```bash
Time taken: 0.0160 seconds
Barcode Format : 
QR_CODE
Barcode Text :
https://www.jalna.com.au/
```

In a nutshell, to get best performance of Dynamsoft Barcode Reader SDK, you should start with the simplest settings and then gradually increase and tune algorithm parameters.

## Source Code
[https://github.com/yushulx/python-qr-decode](https://github.com/yushulx/python-qr-decode)