How to Invoke HTTP Web Service of Dynamsoft SDKs with Power Automate for Desktop
Tags: powerautomate, python, lowcode, flask
Published: 2023-05-23 06:19:08

Microsoft Power Automate is a powerful service designed to automate workflows between various apps and services. Its Desktop version provides the added capability of automating desktop-centric operations, with or without user interface interactions. In this article, we will explore how to integrate [Dynamsoft Barcode Reader](https://www.dynamsoft.com/barcode-reader/overview/), [Dynamsoft Document Normalizer](https://www.dynamsoft.com/document-normalizer/docs/introduction/?ver=latest), and [Dynamsoft Label Recognizer](https://www.dynamsoft.com/label-recognition/overview/) into a web service using Flask. By leveraging this web API, we can seamlessly invoke these Dynamsoft tools within Power Automate for Desktop. This enables you to transform the created flow into a convenient desktop application that significantly enhances your daily work efficiency.

<video src="https://user-images.githubusercontent.com/2202306/239825495-01f40f3a-87cd-4f0e-b100-65b363b2832e.mp4" controls="controls" muted="muted" style="max-height:640px; min-height: 200px; max-width: 100%;"></video>

## Installing Python Packages

```bash
pip install dbr mrz-scanner-sdk document-scanner-sdk opencv-python flask
```

- [dbr](https://pypi.org/project/dbr/): Dynamsoft Barcode Reader
- [mrz-scanner-sdk](https://pypi.org/project/mrz-scanner-sdk/): Dynamsoft Label Recognizer
- [document-scanner-sdk](https://pypi.org/project/document-scanner-sdk/): Dynamsoft Document Normalizer
- [opencv-python](https://pypi.org/project/opencv-python/): OpenCV for Python
- [flask](https://pypi.org/project/flask/): Flask web framework

To use Dynamsoft SDKs, you need to apply for a [trial license](https://www.dynamsoft.com/customer/license/trialLicense).

## Does Power Automate Support External Python Libraries?
If you have tried to use Python script in Power Automate for Desktop, you may have noticed that it does not support external libraries installed via pip. This is because Power Automate Desktop uses IronPython, which is a .NET implementation of Python. IronPython is a subset of the Python language, and it does not support external libraries. To address this issue, we can create a web service using Flask, and then invoke the web API using Power Automate HTTP action.


## Setting Up an HTTP Web Service Using Flask
1. Import the dependent packages:
    ```python
    from flask import Flask, request, jsonify
    import dbr
    import base64
    version = dbr.__version__
    import urllib.parse

    import mrzscanner
    import docscanner
    import numpy as np
    import cv2
    import time

    from dbr import *
    ```

2. Set the Dynamsoft license key and initialize the three SDKs. You need to replace the license key with your own. 
    ```python
    license_key = "DLS2eyJoYW5kc2hha2VDb2RlIjoiMjAwMDAxLTE2NDk4Mjk3OTI2MzUiLCJvcmdhbml6YXRpb25JRCI6IjIwMDAwMSIsInNlc3Npb25QYXNzd29yZCI6IndTcGR6Vm05WDJrcEQ5YUoifQ=="
    BarcodeReader.init_license(license_key)
    reader = BarcodeReader()

    mrzscanner.initLicense(license_key)
    mrz_scanner = mrzscanner.createInstance()
    mrz_scanner.loadModel(mrzscanner.load_settings())
        
    docscanner.initLicense(license_key)
    doc_scanner = docscanner.createInstance()
    doc_scanner.setParameters(docscanner.Templates.color)
    ```

3. Create a Flask app and define the web API endpoints:
    ```python
    app = Flask(__name__)

    @app.route('/api/dbr/decode', methods=['POST'])
    def dbr_decode():
        return handle_request(request, 'dbr')

    @app.route('/api/mrz/scan', methods=['POST'])
    def mrz_scan():
        return handle_request(request, 'mrz')

    @app.route('/api/document/rectify', methods=['POST'])
    def document_rectify():
        return handle_request(request, 'document')

    if __name__ == '__main__':
        app.run()
    ```
4. Handle the HTTP request to get the image data from `request.files` or request body. The image data from the request body is encoded in base64 format, so we need to decode it first. 
    ```python
    def handle_request(request, sdk):
        output = []
        
        request_body = request.data.decode('utf-8')
        if request_body != '':
            try:
                
                base64_content = urllib.parse.unquote(request_body)
                file_content = base64.b64decode(base64_content)
            except:
                return 'Invalid base64 string', 400
            
            output = process_file(file_content, sdk)
            
        else:
            if 'file' not in request.files:
                return 'No file uploaded', 400
            
            file = request.files['file']
            
            if file.filename == '':
                return 'Empty file', 400
            
            file_content = file.read()
            
            output = process_file(file_content, sdk)
            
        return jsonify(results=output)
    ```

5. Invoke the corresponding SDK methods to process the image data for barcode detection, document rectification and MRZ recognition respectively. 
    ```python
    def decode_file_stream(file_content):
        output = []
        try:
            results = reader.decode_file_stream(file_content)
            for result in results:
                output.append({'format': result.barcode_format_string, 'text': result.barcode_text})
        except BarcodeReaderError as error:
            output = error
            
        return output

    def mrz_decode_file_stream(file_content):
        output = []
        results = mrz_scanner.decodeMat(file_content)
        for result in results:
            output.append(result.text)
            
        return output


    def document_rectify_file_stream(file_content):
        output = []
        results = doc_scanner.detectMat(file_content)
        for result in results:
            x1 = result.x1
            y1 = result.y1
            x2 = result.x2
            y2 = result.y2
            x3 = result.x3
            y3 = result.y3
            x4 = result.x4
            y4 = result.y4
            
            normalized_image = doc_scanner.normalizeBuffer(file_content, x1, y1, x2, y2, x3, y3, x4, y4)
            image_path = os.path.join(os.getcwd(), str(time.time()) + '.png')
            normalized_image.save(image_path)
            output.append(image_path)
            break
            
        return output

    def process_file(file_content, sdk):
        output = []
        if sdk == 'dbr':
            output = decode_file_stream(file_content)
        elif sdk == 'mrz':
            output = mrz_decode_file_stream(cv2.imdecode(np.frombuffer(file_content, np.uint8), cv2.IMREAD_COLOR))
        elif sdk == 'document':
            output = document_rectify_file_stream(cv2.imdecode(np.frombuffer(file_content, np.uint8), cv2.IMREAD_COLOR))
        return output
    ```

6. Start the Flask server and test the web API with curl commands:
    ```bash
    python app.py

    # barcode
    curl -X POST -F 'file=@./barcode.jpg' 127.0.0.1:5000/api/dbr/decode

    # mrz
    curl -X POST -F 'file=@./mrz.png' http://127.0.0.1:5000/api/mrz/scan

    # document
    curl -X POST -F 'file=@./document.png' http://127.0.0.1:5000/api/document/rectify
    ```

## Invoking Web Service in Power Automate for Desktop
In the following section, we will show you how to create a flow step by step.
1. Launch Power Automate for Desktop and create a new flow.

    ![create flow](https://www.dynamsoft.com/codepool/img/2023/05/new-flow.png)

2. Create a loop. Since we want to keep the flow running as a tool, we can set a big number for the loop count. 

    ![loop](https://www.dynamsoft.com/codepool/img/2023/05/loop.png)

3. In the loop, add a `Display custom form` action. Open `Custom form designer` to add `Choice set input` and `Submit`. The ID of the `Choice set input` is `Options`, which will be used in the next step.

    ![display custom form](https://www.dynamsoft.com/codepool/img/2023/05/display-custom-form.png)

    The custom form looks like this:

    ![custom form](https://www.dynamsoft.com/codepool/img/2023/05/custom-form-display.png)

4. Create a `Switch` action to check the option value with `%CustomFormData2['Options']%`.

    ![switch](https://www.dynamsoft.com/codepool/img/2023/05/switch.png)

5. Create four cases: `Barcode`, `MRZ`, `Document`, and `Quit`. 
    - The `Barcode` case is used to decode 1D/2D barcodes from an image file or a screenshot.
      ![barcode case](https://www.dynamsoft.com/codepool/img/2023/05/barcode-case.png)
    - The `MRZ` case is used to scan MRZ from an image file.
      ![mrz case](https://www.dynamsoft.com/codepool/img/2023/05/mrz-case.png)
    - The `Document` case is used to rectify a document image.  
      ![document case](https://www.dynamsoft.com/codepool/img/2023/05/document-case.png)
    - The default case is used to quit the loop when the selected option is `Quit`.
       ![cases](https://www.dynamsoft.com/codepool/img/2023/05/cases.png)

### Decode Barcodes from an Image File or a Screenshot
**Image**
1. Add a `Display select file dialog` action to let the user select an image file.

    ![select file](https://www.dynamsoft.com/codepool/img/2023/05/select-file.png)

2. Add a `Convert file to Base64` action to convert the image file to base64 format.

    ![convert to base64](https://www.dynamsoft.com/codepool/img/2023/05/convert-to-base64.png)

3. Add a `Invoke web service` action. Set the `URL` to `http://127.0.0.1:5000/api/dbr/decode`, the `Method` to `POST`, the `Accept` to `application/json`, the `Content type` to `application/json` and the `Request body` to `%Base64Text%`.

    ![invoke web service](https://www.dynamsoft.com/codepool/img/2023/05/invoke-web-service.png)

4. Add a `Display message` action to show the decoded results.

    ![display message](https://www.dynamsoft.com/codepool/img/2023/05/display-message.png)

**Screenshot**
1. Add a `Take screenshot` action to take a screenshot. Set the `Capture` to `All screens`, the `Save screenshot to` to `File`, the `Image file` to any path you want, and the `Image format` to `PNG`.

    ![take screenshot](https://www.dynamsoft.com/codepool/img/2023/05/take-screenshot.png)
2. Take the same steps 2 through 4 in the image example we just mentioned.

### Scan MRZ from an Image File
The MRZ case is similar to the barcode case. The only difference is that we change URL to `http://127.0.0.1:5000/api/mrz/scan` instead of `http://127.0.0.1:5000/api/dbr/decode`.

### Rectify a Document Image

Just like the MRZ scenario, we modify the URL to ` http://127.0.0.1:5000/api/document/rectify`. Once the web API is invoked, we acquire the path of the rectified image by using the `Convert JSON to custom object` action. Subsequently, we use the `Write to CMD session` action to display the rectified image.

![rectify document](https://www.dynamsoft.com/codepool/img/2023/05/cmd-session.png)

## Source Code
[https://github.com/yushulx/dynamsoft-sdk-web-service](https://github.com/yushulx/dynamsoft-sdk-web-service)