Building a Web Document Rectification App with Blazor WebAssembly and Dynamsoft Document Normalizer SDK
Tags: webdev, javascript, csharp, dotnet
Published: 2023-08-25 06:57:30

Web-based document management systems are evolving rapidly, and a critical feature often in demand is document rectification. The [Dynamsoft Document Normalizer SDK](https://www.dynamsoft.com/document-normalizer/docs/core/introduction/?ver=latest) offers a suite of user-friendly JavaScript APIs for document detection and rectification in web browsers. This article will guide you through the process of building a web-based document rectification app. 

<video src="https://user-images.githubusercontent.com/2202306/262901513-aa45bd61-552e-48e7-9c3b-4864906fe832.mp4" controls="controls" muted="muted" style="max-height:640px; min-height: 200px; max-width: 100%;"></video>

## Try Online Demo
[https://yushulx.me/dotnet-blazor-document-rectification/](https://yushulx.me/dotnet-blazor-document-rectification/)

## Prerequisites
- [.NET SDK](https://dotnet.microsoft.com/en-us/download)
- Obtain the JavaScript edition of the Dynamsoft Document Normalizer SDK from [NPM](https://www.npmjs.com/package/dynamsoft-label-recognizer) or load it directly from **cdn.jsdelivr.net**:
    ```html
    <script src="https://cdn.jsdelivr.net/npm/dynamsoft-document-normalizer@1.0.12/dist/ddn.js"></script>
    ```

- Request a [trial license](https://www.dynamsoft.com/customer/license/trialLicense?product=ddn&source=codepool&package=js).

## Step 1: Setting Up Your Blazor WebAssembly Project
1. Create a Blazor WebAssembly project with the following command:
    
    ```bash
    dotnet new blazorwasm -o BlazorDocRectifySample
    ```

2. Navigate to the new directory.
    
    ```bash
    cd BlazorDocRectifySample
    ```

3. Open `wwwroot/index.html` and add both the Dynamsoft Camera Enhancer SDK and the Dynamsoft Document Normalizer SDK:

    ```html
    <script src="https://cdn.jsdelivr.net/npm/dynamsoft-camera-enhancer@3.3.5/dist/dce.js"></script>
    <script src="https://cdn.jsdelivr.net/npm/dynamsoft-document-normalizer@1.0.12/dist/ddn.js"></script>
    ```
    **Dynamsoft Camera Enhancer** is a tool used for accessing the webcam video stream. It is available **free of charge**.
    **Dynamsoft Document Normalizer SDK** is utilized for document rectification. To use it, a valid license key is required.

4. Create a `jsInterop.js` file for interoperation between C# and JavaScript, and then add it to the `index.html` file:
    
    ```html
    <script src="jsInterop.js"></script>
    ```

## Step 2: Building the UI
We create two Razor components: one for detecting documents from an image file and the other for detecting documents from a camera stream.

```bash
dotnet new razorcomponent -n ImageFile -o Pages
dotnet new razorcomponent -n CameraStream -o Pages
```

### The `ImageFile` component 

```html
@page "/image-file"
@inject IJSRuntime JSRuntime

<div>
<button class="btn" @onclick="DocDetect">Load file</button>
<button class="btn" @onclick="Rectify">Rectify</button>
<button class="btn" @onclick="Save">Save</button>
</div>

<div>
    <input type="radio" name="format" value="grayscale" @onchange="HandleInputChange">Grayscale
    <input type="radio" name="format" value="color" checked @onchange="HandleInputChange">Color
    <input type="radio" name="format" value="binary" @onchange="HandleInputChange">Binary
</div>

<div class="container">
    <div>
        <div id="imageview">
            <img id="image" />
            <canvas id="overlay"></canvas>
        </div>
    </div>
    <div>
        <canvas id="canvas"></canvas>
    </div>
</div>
```

![Blazor document rectification from image file](https://www.dynamsoft.com/codepool/img/2023/08/blazor-document-rectification-image-file.png)

- The `DocDetect()` method is invoked when the button is clicked. It invokes the `selectFile()` method in JavaScript and pass the C# object reference and canvas IDs to it. When a document is detected, one canvas is used to display the detected quadrilateral and the other to display the rectified document.
    
    ```csharp
    public async Task DocDetect()
    {
        await JSRuntime.InvokeVoidAsync(
        "jsFunctions.selectFile", objRef, "image");
    }
    ```

- The `Rectify()` method triggers document rectification manually.
    
    ```csharp
    public async Task Rectify()
    {
        await JSRuntime.InvokeVoidAsync(
        "jsFunctions.rectify");
    }
    ```

- The `Save()` method saves the rectified document to an image file.
    
    ```csharp
    public async Task Save()
    {
        await JSRuntime.InvokeVoidAsync(
        "jsFunctions.save", objRef, "canvas");
    }
    ```

- The `HandleInputChange()` method handles the radio button change event. It provides three options for the output color format: grayscale, color, and binary.
    
    ```csharp
    public async Task HandleInputChange(ChangeEventArgs e)
    {
        await JSRuntime.InvokeVoidAsync(
        "jsFunctions.setOutputFormat", e.Value.ToString());
    }
    ```

- The `<img>` element is used to display the selected image file. 

### The `CameraStream` component

```html
@page "/camera-stream"
@inject IJSRuntime JSRuntime

<div class="select">
    <label for="videoSource">Video source: </label>
    <select id="videoSource"></select>
</div>

<div>
<button class="btn" @onclick="Rectify">Rectify</button>
<button class="btn" @onclick="Save">Save</button>
</div>

<div class="container">
    <div id="videoview">
        <div class="dce-video-container" id="videoContainer"></div>
        <canvas id="overlay"></canvas>
    </div>
    <div>
        <canvas id="canvas"></canvas>
    </div>
</div>
```

![Blazor document rectification from camera stream](https://www.dynamsoft.com/codepool/img/2023/08/blazor-document-rectification-camera-stream.png)

- The `<select>` element lists all available cameras.
- The buttons and canvases work the same as in the `ImageFile` component.
- Camera stream is displayed in the `<div>` element with the `dce-video-container` class.

## Step 3: Implementing the Document Rectification Logic
The primary logic for document rectification is implemented in JavaScript.  The main interop methods are defined in the `wwwroot/jsInterop.js` file:

```js
window.jsFunctions = {
    initSDK: async function (licenseKey) {
        
    },
    initImageFile: async function (dotnetRef, canvasId) {
        
    },
    initScanner: async function (dotnetRef, videoId, selectId, canvasOverlayId, canvasId) {
        
    },
    selectFile: async function (dotnetRef, canvasOverlayId, imageId) {
        
    },
    rectify: async function () {
        
    },
    updateSetting: async function (color) {
        
    },
    save: async function () {
        
    },
};

```

- The `initSDK()` method initializes the Dynamsoft Document Normalizer SDK with a valid license key. It also sets the runtime settings for document rectification.
    
    ```js
    async function init() {
        normalizer = await Dynamsoft.DDN.DocumentNormalizer.createInstance();
        let settings = await normalizer.getRuntimeSettings();
        settings.ImageParameterArray[0].BinarizationModes[0].ThresholdCompensation = 9;
        settings.NormalizerParameterArray[0].ColourMode = "ICM_COLOUR"; // ICM_BINARY, ICM_GRAYSCALE, ICM_COLOUR
    
        await normalizer.setRuntimeSettings(settings);
    }
    
    initSDK: async function (licenseKey) {
        let result = true;

        if (normalizer != null) {
            return result;
        }
        
        try {
            Dynamsoft.DDN.DocumentNormalizer.license = licenseKey;
        } catch (e) {
            console.log(e);
            result = false;
        }
        
        await init();

        return result;
    }
    ```

- The `initImageFile()` method initializes the canvases for rendering the quadrilateral and the rectified document.
    
    ```js
    initImageFile: async function (dotnetRef, canvasOverlayId, canvasRectifyId) {
        dotnetHelper = dotnetRef;
        initOverlay(document.getElementById(canvasOverlayId));
        canvasRectify = document.getElementById(canvasRectifyId);
        contextRectify = canvasRectify.getContext('2d');
        if (normalizer != null) {
            normalizer.stopScanning();
        }
        await init();

        return true;
    }
    ```

- The `initCameraStream()` method initializes the camera stream and the canvases for rendering the quadrilateral and the rectified document.
    
    ```js
    initCameraStream: async function (dotnetRef, videoId, selectId, canvasOverlayId, canvasId) {
        await init();
        canvasRectify = document.getElementById(canvasId);
        contextRectify = canvasRectify.getContext('2d');
        let canvas = document.getElementById(canvasOverlayId);
        data = {};
        initOverlay(canvas);
        videoContainer = document.getElementById(videoId);
        videoSelect = document.getElementById(selectId);
        videoSelect.onchange = openCamera;
        dotnetHelper = dotnetRef;

        try {
            enhancer = await Dynamsoft.DCE.CameraEnhancer.createInstance();
            await enhancer.setUIElement(document.getElementById(videoId));
            await normalizer.setImageSource(enhancer, { });
            await normalizer.startScanning(true);
            let cameras = await enhancer.getAllCameras();
            listCameras(cameras);
            await openCamera();

            normalizer.onQuadDetected = (quads, sourceImage) => {
                clearOverlay();
                if (quads.length == 0) {
                    return;
                }
                data["file"] = sourceImage;
                let location = quads[0].location;
                data["points"] = quads[0].location;
                drawQuad(location.points);
            };
            enhancer.on("played", playCallBackInfo => {
                updateResolution();
            });

        } catch (e) {
            console.log(e);
            result = false;
        }
        return true;
    }
    
    ```

- The `selectFile()` method detects documents from an image file.
    
    ```js
    selectFile: async function (dotnetRef, imageId) {
        data = {};
        
        if (normalizer) {
            let input = document.createElement("input");
            input.type = "file";
            input.onchange = async function () {
                try {
                    let file = input.files[0];
                    var fr = new FileReader();
                    fr.onload = function () {
                        let image = document.getElementById(imageId);
                        image.src = fr.result;
                        image.style.display = 'block';
                        
                        decodeImage(fr.result);
                    }
                    fr.readAsDataURL(file);

                } catch (ex) {
                    alert(ex.message);
                    throw ex;
                }
            };
            input.click();
        } else {
            alert("The SDK is still initializing.");
        }
    }
    ```

- The `rectify()` method triggers document rectification based on the detected quadrilateral.
    
    ```js
    async function normalize(file, location) {
        if (file == null || location == null) {
            return;
        }
        if (normalizer) {
            normalizedImageResult = await normalizer.normalize(file, {
                quad: location
            });
            if (normalizedImageResult) {
                let image = normalizedImageResult.image;
                canvasRectify.width = image.width;
                canvasRectify.height = image.height;
                let data = new ImageData(new Uint8ClampedArray(image.data), image.width, image.height);
                contextRectify.clearRect(0, 0, canvasRectify.width, canvasRectify.height);
                contextRectify.putImageData(data, 0, 0);
            }
        }
    }

    rectify: async function () {
        await normalize(data["file"], data["points"]);
    },
    ```

- The `updateSetting()` method changes the color format of the rectified document.
    
    ```js
    updateSetting: async function (color) {
        let colorMode = "ICM_GRAYSCALE";
        if (color === 'grayscale') {
            colorMode = "ICM_GRAYSCALE";
        } else if (color === 'color') {
            colorMode = "ICM_COLOUR";
        } else if (color === 'binary') {
            colorMode = "ICM_BINARY";
        }

        if (normalizer && data['file']) {
            let settings = await normalizer.getRuntimeSettings();
            settings.NormalizerParameterArray[0].ColourMode = colorMode;
            await normalizer.setRuntimeSettings(settings);
            normalize(data["file"], data["points"]);
        }
    }
    ```

- The `save()` method saves the rectified document to an image file.
    
    ```js
    save: async function () {
        if (normalizedImageResult) {
            await normalizedImageResult.saveToFile("document-normalization.png", true);
        }
    }
    ```

## Step4: Editing Quadrilateral for Better Rectification
It is inevitable that the detected quadrilateral may not be accurate. To improve the rectification result, the quadrilateral editing feature is required.

We add mouse event handlers to the canvas to update the points of the quadrilateral and redraw it.

```js
function initOverlay(ol) {
    canvasOverlay = ol;
    canvasOverlay.addEventListener("mousedown", updatePoint);
    canvasOverlay.addEventListener("touchstart", updatePoint);
    contextOverlay = canvasOverlay.getContext('2d');
}

function updatePoint(e) {
    let points = data["points"].points;
    let rect = canvasOverlay.getBoundingClientRect();

    let scaleX = canvasOverlay.clientWidth / canvasOverlay.width;
    let scaleY = canvasOverlay.clientHeight / canvasOverlay.height;
    let mouseX = (e.clientX - rect.left) / scaleX;
    let mouseY = (e.clientY - rect.top) / scaleY;

    let delta = 10;
    for (let i = 0; i < points.length; i++) {
        if (Math.abs(points[i].x - mouseX) < delta && Math.abs(points[i].y - mouseY) < delta) {
            canvasOverlay.addEventListener("mousemove", dragPoint);
            canvasOverlay.addEventListener("mouseup", releasePoint);
            canvasOverlay.addEventListener("touchmove", dragPoint);
            canvasOverlay.addEventListener("touchend", releasePoint);
            function dragPoint(e) {
                let rect = canvasOverlay.getBoundingClientRect();
                let mouseX = e.clientX || e.touches[0].clientX;
                let mouseY = e.clientY || e.touches[0].clientY;
                points[i].x = Math.round((mouseX - rect.left) / scaleX);
                points[i].y = Math.round((mouseY - rect.top) / scaleY);
                drawQuad(points);
            }
            function releasePoint() {
                canvasOverlay.removeEventListener("mousemove", dragPoint);
                canvasOverlay.removeEventListener("mouseup", releasePoint);
                canvasOverlay.removeEventListener("touchmove", dragPoint);
                canvasOverlay.removeEventListener("touchend", releasePoint);
            }
            break;
        }
    }
}
```

![edge edit](https://www.dynamsoft.com/codepool/img/2023/08/edge-edit.gif)

## Source Code

[https://github.com/yushulx/dotnet-blazor-document-rectification](https://github.com/yushulx/dotnet-blazor-document-rectification)