How to Upload Images in Power Apps for Barcode, MRZ, and Document Detection
Tags: powerapps, powerplatform, powerautomate, lowcode
Published: 2023-12-11 05:10:21

In our [previous article](https://www.dynamsoft.com/codepool/nodejs-rest-endpoint-scan-document-barcode-mrz.html), we explained how to develop API endpoints for reading barcodes, recognizing MRZ, and rectifying document images in Node.js. In this article, we will integrate these REST API endpoints with the low-code platform **Power Apps**.

<video src="https://github.com/yushulx/dynamsoft-sdk-nodejs-REST-API/assets/2202306/6160986e-5ebf-4f39-bd88-8bcebc105ae5" controls="controls" muted="muted" style="max-height:640px; min-height: 200px; max-width: 100%;"></video>


## Loading and Displaying Images within Power Apps
1. Click `Insert` and search for `Add picture` control, then drag it onto the canvas. This control allows you to upload images from your local disk or capture photos using a camera.

    ![Power app add picture](https://www.dynamsoft.com/codepool/img/2023/12/power-app-add-picture.png)

    The control is comprised of an `AddMediaButton` and an image display component.

    ![add picture control](https://www.dynamsoft.com/codepool/img/2023/12/add-picture-control-construction.png)

2. Set the `Image` property of the image control using this formula:

    ```js
    If(IsBlank(AddMediaButton.Media), SampleImage, AddMediaButton.Media)
    ```

Now, we can upload an image and it will be displayed in the image control. You may need to adjust the position and size of both the button and the image for optimal layout.

## Uploading Images from Power Apps to a Self-hosted Node.js Server
Uploading images from Power Apps to a self-hosted Node.js server involves several steps:
1. Convert the image to a base64-encoded string.
2. Establish an endpoint on the Node.js server for receiving the base64-encoded string and saving it as an image file.
3. Develop a Power Automate flow to transmit the base64 string to the server.

### How to Convert an Image to a Base64-Encoded String in Power Apps
In Power Apps, the [JSON](https://learn.microsoft.com/en-us/power-platform/power-fx/reference/function-json) function can be used to convert an image to a base64-encoded string. This is achieved by utilizing the `IncludeBinaryData` parameter. To perform this conversion and store the result in a variable named `ImageJSON`, use the following formula:

```js
Set(ImageJSON, JSON(UploadedImage.Image, JSONFormat.IncludeBinaryData));
```

You can view the generated base64 string by navigating to `Variables` > `ImageJSON`:

![image base64](https://www.dynamsoft.com/codepool/img/2023/12/image-base64.png)

### How to Create an Endpoint for Uploading Images in Node.js
1. Install the [Express](https://www.npmjs.com/package/express) package:

    ```bash
    npm install express
    ```

2. Create a web server using Express. Here's an example of basic server setup code:

    ```js
    const express = require('express');
    const app = express();
    
    const port = process.env.PORT || 3000;

    server.listen(port, '0.0.0.0', () => {
        host = server.address().address;
        console.log(`Server running at http://0.0.0.0:${port}/`);
    });
    ```

3. Create an endpoint to receive a base64-encoded string and save it as an image file on the server. This typically involves parsing the base64 string and writing it to a file. Here's a basic structure of such an endpoint:

    ```js
    app.post('/dynamsoft/dlr/DetectMrz/base64', (req, res) => {
    let jsonObject = req.body;
    let size = Object.keys(jsonObject).length;
    if (size == 0) {
        return res.status(400).send('No file uploaded.');
    }

    Object.keys(jsonObject).forEach(key => {
        let base64Image = jsonObject[key].split(';base64,').pop();
        const imageBuffer = Buffer.from(base64Image, 'base64');

        // Write buffer to a file
        const timestamp = Date.now();
        let filePath = 'uploads/' + timestamp;
        fs.writeFile(filePath, imageBuffer, async (err) => {
                if (err) {
                    console.error('Error writing file:', err);
                } else {
                    console.log('File saved:', filePath);
                }
            });
        });
    });
    ```

    The endpoint is named `/dynamsoft/dlr/DetectMrz/base64`. This can be modified for various image processing tasks. It's designed to accept a `POST` request that includes a JSON object in the request body. This JSON object should contain the base64-encoded string of the image. The provided code will save the image in the `uploads` directory on the server, naming the file with a timestamp.

4. For a quick test, we can use **Ngrok** to expose the local server to the Internet. 

    ```bash
    ngrok http 3000
    ```

    ![ngrok](https://www.dynamsoft.com/codepool/img/2023/12/ngrok.png)


### How to Upload a Base64-Encoded String via Power Automate
1. Open `Power Automate` and create a new flow.
2. Select `Text` as the input type.
    
    ![power automate flow input](https://www.dynamsoft.com/codepool/img/2023/12/power-automate-flow-input.png)

3. Add an `HTTP` action. Set the `Method` to `POST`. For the `URI`, use the endpoint we created previously. The `Body` should contain the input text.

    ![power automate http action](https://www.dynamsoft.com/codepool/img/2023/12/power-automate-flow-http.png)

4. Incorporate a `Response` action to display the server's response.

    ![power automate response action](https://www.dynamsoft.com/codepool/img/2023/12/power-automate-flow-response.png)

    
    Note: The JSON schema for the response needs to be defined according to the server's response format. This will be covered in the following section.

## Reading Barcodes, Recognizing MRZ, and Rectifying Documents in Power Apps

First, let's finalize the server-side code by defining three API endpoints for barcode reading, MRZ recognition, and document rectification:

```js
app.post('/dynamsoft/dbr/DecodeBarcode/base64', (req, res) => {
    // ...
});

app.post('/dynamsoft/dlr/DetectMrz/base64', (req, res) => {
    // ...
});

app.post('/dynamsoft/dlr/DetectMrz/base64', (req, res) => {
    // ...
});
```

For barcode processing: Utilize the `decodeBase64Async` method to decode the base64-encoded string directly:

```js
let base64Image = jsonObject[key].split(';base64,').pop();
try {
    barcode4nodejs.decodeBase64Async(base64Image, barcode4nodejs.barcodeTypes).then((result) => {
        console.log(result);
        res.status(200).send(result);
    }).catch((err) => {
        console.error(err);
        return res.status(500).send('An error occurred while processing the image.');
    });
}
catch (err) {
    console.error(err);
    return res.status(500).send('An error occurred while processing the image.');
}
```

For MRZ and document processing: Since the SDKs for MRZ and documents do not have methods to decode base64 strings, first save the base64 string as a file. Then, pass the file path to the respective SDK methods for further processing.

- **MRZ Recognition**. Recognize the machine-readable zone (MRZ) from an image file and return the parsed results as a JSON string.
    ```js
    try {
        let result = await mrzScanner.decodeFileAsync(filePath);
        let output = "";
        if (result.length == 2) {
            output = mrzScanner.parseTwoLines(result[0].text, result[1].text);
        }
        else if (result.length == 3) {
            output = mrzScanner.parseThreeLines(result[0].text, result[1].text, result[2].text);
        }
        console.log(output);
        res.status(200).send(output);
    }
    catch (err) {
        console.error(err);
        return res.status(500).send('An error occurred while processing the image.');
    }
    ```

- **Document Rectification**. Rectify a document image and save it as a JPEG file. Then, return the file path to display the rectified image in Power Apps.
    ```js
    try {
        let results = await docRectifier.detectFileAsync(filePath);
        let result = results[0];
        result = await docRectifier.normalizeFileAsync(filePath, result['x1'], result['y1'], result['x2'], result['y2'], result['x3'], result['y3'], result['x4'], result['y4']);
        let data = result['data']
        let width = result['width']
        let height = result['height']
        for (let i = 0; i < data.length; i += 4) {
            const red = data[i];
            const blue = data[i + 2];
            data[i] = blue;
            data[i + 2] = red;
        }

        timestamp = Date.now();

        sharp(data, {
            raw: {
                width: width,
                height: height,
                channels: 4
            }
        }).toFile('uploads/' + timestamp + '.jpeg', (err, info) => {
            if (err) {
                console.error('Error:', err);
            } else {
                res.send(JSON.stringify({
                    'image': 'uploads/' + timestamp + '.jpeg'
                }));
            }
        });
    }
    catch (err) {
        console.error(err);
        return res.status(500).send('An error occurred while processing the image.');
    }
    ```

We create three Power Automate Flows for the three endpoints. While these flows are similar in structure, they differ in their response actions.

- The JSON schema of the barcode reading response is:

    ```json
    {
        "type": "array",
        "items": {
            "type": "object",
            "properties": {
                "format": {
                    "type": "string"
                },
                "value": {
                    "type": "string"
                },
                "x1": {
                    "type": "integer"
                },
                "y1": {
                    "type": "integer"
                },
                "x2": {
                    "type": "integer"
                },
                "y2": {
                    "type": "integer"
                },
                "x3": {
                    "type": "integer"
                },
                "y3": {
                    "type": "integer"
                },
                "x4": {
                    "type": "integer"
                },
                "y4": {
                    "type": "integer"
                },
                "page": {
                    "type": "integer"
                },
                "time": {
                    "type": "integer"
                }
            },
            "required": [
                "format",
                "value",
                "x1",
                "y1",
                "x2",
                "y2",
                "x3",
                "y3",
                "x4",
                "y4",
                "page",
                "time"
            ]
        }
    }
    ```

- The JSON schema of the MRZ recognition response is:
    
    ```json
    {
        "type": "object",
        "properties": {
            "type": {
                "type": "string"
            },
            "nationality": {
                "type": "string"
            },
            "surname": {
                "type": "string"
            },
            "givenname": {
                "type": "string"
            },
            "passportnumber": {
                "type": "string"
            },
            "issuecountry": {
                "type": "string"
            },
            "birth": {
                "type": "string"
            },
            "gender": {
                "type": "string"
            },
            "expiry": {
                "type": "string"
            }
        }
    }
    ```

- The JSON schema of the document rectification response is:

    ```json
    {
        "type": "object",
        "properties": {
            "image": {
                "type": "string"
            }
        }
    }
    ```
    
Finally, to complete the app, add more controls to the canvas:
1. Insert a `Drop down` control and set the `Items` property to `["Dynamsoft Barcode Reader", "Dynamsoft Label Recognizer", "Dynamsoft Document Normalizer"]`. 

2. Add a `Button` control and set the `OnSelect` property to the following formula:

    ```js
    Set(ImageJSON, JSON(UploadedImage.Image, JSONFormat.IncludeBinaryData));
    Switch(DropDown.SelectedText.Value, "Dynamsoft Barcode Reader", ClearCollect(Results, BarcodeUpload.Run(ImageJSON)), "Dynamsoft Label Recognizer", ClearCollect(Results, MrzUpload.Run(ImageJSON)), "Dynamsoft Document Normalizer", Set(URL, DocumentUpload.Run(ImageJSON)))
    ```

    This formula will call the corresponding Power Automate flow based on the selection in the dropdown control, utilizing the `Switch` function. The `ClearCollect` function will store the response in the `Results` collection, while the `Set` function will assign the response to the `URL` variable.

3. Add a `Text input` control and set the `Visible` property to the following formula:
    ```js
    If(DropDown.SelectedText.Value = "Dynamsoft Barcode Reader" Or DropDown.SelectedText.Value = "Dynamsoft Label Recognizer", true, false)
    ``` 
    Only the results from barcode and MRZ recognition will be displayed in the `Text input` control. The `Default` property is set to `JSON(Results)`.

4. Add an `Image` control and set the `Visible` property to the following formula:
    ```js
    If(DropDown.SelectedText.Value = "Dynamsoft Document Normalizer", true, false)
    ``` 
    The `Image` property is set to `Concatenate("https://563f-98-96-222-66.ngrok-free.app/", URL.image)`.

## Test the Power App

**Barcode Detection**

![barcode Detection](https://www.dynamsoft.com/codepool/img/2023/12/power-app-barcode-reader.png)

**MRZ Recognition**

![MRZ Recognition](https://www.dynamsoft.com/codepool/img/2023/12/power-app-mrz-recognition.png)

**Document Rectification**

![Document Rectification](https://www.dynamsoft.com/codepool/img/2023/12/power-app-document-rectification.png)

## Source Code
[https://github.com/yushulx/dynamsoft-sdk-nodejs-REST-API.git](https://github.com/yushulx/dynamsoft-sdk-nodejs-REST-API.git)