How to Scan QR Code from Desktop Screen in Python
Tags: qrcode, python, screenshot
Published: 2021-10-11 05:19:13

In our life, we usually take out smartphone to scan QR code. But when we are working on a computer, smartphone may not be the best choice for scanning QR codes that appear on web pages. One reason is the pictures taken from a monitor screen have the Moire' patterns which interfere QR code recognition. The other reason is you may want to use the decoded information directly on PC, such as a URL for opening a website. This article will implement a simple tool with Python to facilitate QR code recognition on desktop screen.

## Installation

The required Python packages include [PIL](https://pillow.readthedocs.io/en/stable/), [OpenCV](https://docs.opencv.org/master/d6/d00/tutorial_py_root.html), [Dynamsoft Barcode Reader](https://www.dynamsoft.com/barcode-reader/programming/python/) and [Qt](https://doc.qt.io/qtforpython/).

```bash
python3 -m pip install pillow opencv-python dbr pyside2
```

## Barcode SDK License
To unlock the capabilities of Dynamsoft Barcode SDK, you'd better apply for a [30-day free trial license](https://www.dynamsoft.com/customer/license/trialLicense?product=dbr).

## Scanning QR Code from Screen
Since I have implemented a [GUI barcode reader](https://www.dynamsoft.com/codepool/gui-barcode-reader-windows-linux-macos.html) using Qt for Python, OpenCV and Dynamsoft Barcode Reader, the remaining stuff is to add screen snipping functionality. Inspired by [https://github.com/harupy/snipping-tool](https://github.com/harupy/snipping-tool), I learned the steps of implementing the screenshot functionality:
1. Create a custom Qt widget and put it on top of the screen.
2. Draw the selected area in `paintEvent()` function while moving the mouse.
3. As the mouse is released, call `PIL.ImageGrab.grab()` to get the image of the selected area.

### Add buttons for snipping events
We open the `design.ui` file in Qt Creator and add two buttons for triggering the snipping events.

![snip button](https://www.dynamsoft.com/codepool/img/2021/10/snip-button.jpg)

Save the file and recompile `design.ui` to `design.py`:

```bash
pyside2-uic design.ui -o design.py
```

In `app_advanced.py`, the two new buttons should be recognizable now. Connect them to slot functions:

```python
self.ui.pushButton_area.clicked.connect(self.snipArea)
self.ui.pushButton_full.clicked.connect(self.snipFull)
```

### Create a custom Qt widget
Create a `SnippingTool.py` file, in which we create a custom Qt widget:

```python
import numpy as np
import cv2
from PIL import ImageGrab
from PySide2 import QtWidgets, QtCore, QtGui
from PySide2.QtCore import Qt

class SnippingWidget(QtWidgets.QWidget):
    is_snipping = False

    def __init__(self, parent=None, app=None):
        super(SnippingWidget, self).__init__()
        self.parent = parent
        self.setWindowFlags(Qt.WindowStaysOnTopHint)

        self.screen = app.primaryScreen()
        self.setGeometry(0, 0, self.screen.size().width(), self.screen.size().height())
        self.begin = QtCore.QPoint()
        self.end = QtCore.QPoint()
        self.onSnippingCompleted = None

    def start(self):
        SnippingWidget.is_snipping = True
        self.setWindowOpacity(0.3)
        QtWidgets.QApplication.setOverrideCursor(QtGui.QCursor(QtCore.Qt.CrossCursor))
        self.show()
```

The size of the widget should be the same as the screen resolution, which can be obtained from the `primaryScreen()` function. 

Next, we handle the mouse events:

```python
def mousePressEvent(self, event):
    self.begin = event.pos()
    self.end = self.begin
    self.update()

def mouseMoveEvent(self, event):
    self.end = event.pos()
    self.update()

def mouseReleaseEvent(self, event):
    SnippingWidget.is_snipping = False
    QtWidgets.QApplication.restoreOverrideCursor()
    x1 = min(self.begin.x(), self.end.x())
    y1 = min(self.begin.y(), self.end.y())
    x2 = max(self.begin.x(), self.end.x())
    y2 = max(self.begin.y(), self.end.y())

    self.repaint()
    QtWidgets.QApplication.processEvents()
    self.close()
```

While the mouse is moving, we draw a rectangle to indicate the selected area in `paintEvent()` function:

```python
def paintEvent(self, event):
    if SnippingWidget.is_snipping:
        brush_color = (128, 128, 255, 100)
        lw = 3
        opacity = 0.3
    else:
        self.begin = QtCore.QPoint()
        self.end = QtCore.QPoint()
        brush_color = (0, 0, 0, 0)
        lw = 0
        opacity = 0

    self.setWindowOpacity(opacity)
    qp = QtGui.QPainter(self)
    qp.setPen(QtGui.QPen(QtGui.QColor('black'), lw))
    qp.setBrush(QtGui.QColor(*brush_color))
    rect = QtCore.QRectF(self.begin, self.end)
    qp.drawRect(rect)
```

### Take screen images
PIL is the Python library for capturing images from the screen. After releasing the mouse, we take the screenshot of the selected area based on the coordinates of the rectangle.

```python
img = ImageGrab.grab(bbox=(x1, y1, x2, y2))

try:
    img = cv2.cvtColor(np.array(img), cv2.COLOR_RGB2BGR)
except:
    img = None
```

Taking full screen image is easy:

```python
img = ImageGrab.grab(bbox=(0, 0, self.screen.size().width(), self.screen.size().height()))
```

### Hide and show the application window
Once the snipping widget is ready, we can call it in button click event. Note: to avoid obstructing the screen, the application window should be minimized before launching the snipping widget and restored after the snipping is completed:

```python
def onSnippingCompleted(self, frame):
    self.setWindowState(Qt.WindowMaximized)
    if frame is None:
        return 

    frame, self._results = self._barcodeManager.decode_frame(frame)
    self.showResults(frame, self._results)

def snipArea(self):
    self.setWindowState(Qt.WindowMinimized)
    self.snippingWidget.start()    

def snipFull(self):
    self.setWindowState(Qt.WindowMinimized)
    self.snippingWidget.fullscreen() 
```


### Test the screen QR reader
1. Run the barcode recognition program:
    
    ```bash
    python3 app_advanced.py
    ```
2. Search Google for [QR code](https://www.google.com/search?q=qr+code&newwindow=1&sxsrf=AOaemvK5OxdrGkIqi3oyDFM3hGYxHuZRKQ:1633746577834&source=lnms&tbm=isch&sa=X&ved=2ahUKEwjkubKjpLzzAhURHjQIHcw3AuoQ_AUoAXoECAEQAw&biw=1463&bih=755&dpr=1.75). 
3. Click the `Select Area` button to scan QR code (one or multiple) returned by search engine.

    ![scan QR code from screen](https://www.dynamsoft.com/codepool/img/2021/10/screen-qr-code-recognition.png)

    You can also make a full screen barcode recognition by one click.

## Source Code
[https://github.com/yushulx/python-gui-barcode-reader](https://github.com/yushulx/python-gui-barcode-reader)