How to Build Web Apps to Scan Documents by Edge Detection Using JavaScript and Flutter
Tags: webdev, javascript, programming, flutter
Published: 2022-11-30 06:45:34

[Dynamsoft Document Normalizer SDK](https://www.dynamsoft.com/document-normalizer/docs/introduction/?ver=latest) provides a set of APIs to detect the edges of a document and normalize the document based on the found quadrilaterals. The JavaScript edition of the SDK has been published on [npm](https://www.npmjs.com/package/dynamsoft-document-normalizer). In this article, we will firstly show you how to use the JavaScript APIs in a web application, and then create a Flutter document scan plugin based on the JavaScript SDK. It will be convenient for developers to integrate the document edge detection and normalization features into their Flutter web applications.

## Flutter Document Scan SDK
[https://pub.dev/packages/flutter_document_scan_sdk](https://pub.dev/packages/flutter_document_scan_sdk)

## Using JavaScript API for Document Edge Detection and Perspective Transformation
With a few lines of JavaScript code, you can quickly build a web document scanning application using Dynamsoft Document Normalizer SDK. Here are the steps:

1. Include the JavaScript SDK in your HTML file:

    ```html
    <script src="https://cdn.jsdelivr.net/npm/dynamsoft-document-normalizer@1.0.11/dist/ddn.js"></script>
    ```

2. Apply for a trial license from [Dynamsoft Customer Portal](https://www.dynamsoft.com/customer/license/trialLicense/?product=ddn) and set the license key in your JavaScript code:

    ```javascript
    Dynamsoft.DDN.DocumentNormalizer.license =
            "DLS2eyJoYW5kc2hha2VDb2RlIjoiMjAwMDAxLTE2NDk4Mjk3OTI2MzUiLCJvcmdhbml6YXRpb25JRCI6IjIwMDAwMSIsInNlc3Npb25QYXNzd29yZCI6IndTcGR6Vm05WDJrcEQ5YUoifQ==";
    ```

3. Initialize the document normalizer in an asynchronous function:

    ```javascript

    (async () => {
        normalizer = await Dynamsoft.DDN.DocumentNormalizer.createInstance();
        let settings = await normalizer.getRuntimeSettings();
        settings.ImageParameterArray[0].BinarizationModes[0].ThresholdCompensation = 9;
        settings.ImageParameterArray[0].ScaleDownThreshold = 2300;
        settings.NormalizerParameterArray[0].ColourMode = "ICM_GRAYSCALE"; // ICM_BINARY, ICM_GRAYSCALE, ICM_COLOUR
        await normalizer.setRuntimeSettings(settings);
    })();
    ```

4. Create an input button to load a document image and call the `detectQuad()` method to detect the edges of the document:

    ```javascript
    <input type="file" id="file" accept="image/*" />

    document.getElementById("file").addEventListener("change", function () {
        let file = this.files[0];
        let fr = new FileReader();
        fr.onload = function () {
            let image = document.getElementById('image');
            image.src = fr.result;
            target["file"] = fr.result;

            const img = new Image();
            img.onload = () => {
                if (normalizer) {
                    (async () => {
                        let quads = await normalizer.detectQuad(target["file"]);
                        target["quads"] = quads;
                        points = quads[0].location.points;
                        drawQuad(points);
                    })();
                }
            }
            img.src = fr.result;
        }
        fr.readAsDataURL(file);
    });
    ```

5. Call the `normalize()` method to normalize the document based on the detected points:

    ```javascript
    function normalize(file, quad) {
        (async () => {
            if (normalizer) {
                var tmp = { quad: quad };
                normalizedImageResult = await normalizer.normalize(file, tmp);
            }

        })();
    }
    ```
    
    The data type of the normalized image buffer is `Uint8Array`. In order to render the image on a canvas, you need to convert it to `ImageData`.

    ```javascript
    if (normalizedImageResult) {
        let image = normalizedImageResult.image;
        canvas.width = image.width;
        canvas.height = image.height;
        let data = new ImageData(new Uint8ClampedArray(image.data), image.width, image.height);
        ctx.putImageData(data, 0, 0);
    }
    ```

6. Save the document to a PNG or JPEG file:

    ```javascript
    function save() {
        (async () => {
            if (normalizedImageResult) {
                let data = await normalizedImageResult.saveToFile("document-normalization.png", false);
                console.log(data);
            }
        })();
    }
    ```

After finishing the above steps, you can test the single page HTML file by running `python -m http.server` in the terminal. 

![JavaScript web document edge detection and normalization](https://www.dynamsoft.com/codepool/img/2022/11/javascript-document-scan-edge-detection.png)

## Encapsulating Dynamsoft Document Normalizer JavaScript SDK into a Flutter Plugin
We scaffolded a Flutter web-only plugin project using the following command:

```bash
flutter create --org com.dynamsoft --template=plugin --platforms=web flutter_document_scan_sdk
```

Then add the js package as the dependency in `pubspec.yaml` file. The Flutter JS plugin can make Dart and JavaScript code communicate with each other:

```yaml
dependencies:
  ...
  js: ^0.6.3
```

Go to the `lib` folder and create a new file `web_ddn_manager.dart`, in which we define the classes and methods to interop with the JavaScript SDK.

```dart
@JS('Dynamsoft')
library dynamsoft;

import 'dart:convert';

import 'package:flutter/foundation.dart';
import 'package:flutter_document_scan_sdk/normalized_image.dart';
import 'package:flutter_document_scan_sdk/shims/dart_ui_real.dart';
import 'package:js/js.dart';
import 'document_result.dart';
import 'utils.dart';
import 'dart:html' as html;

/// DocumentNormalizer class.
@JS('DDN.DocumentNormalizer')
class DocumentNormalizer {
  external static set license(String license);
  external static set engineResourcePath(String resourcePath);
  external PromiseJsImpl<List<dynamic>> detectQuad(dynamic file);
  external PromiseJsImpl<dynamic> getRuntimeSettings();
  external static PromiseJsImpl<DocumentNormalizer> createInstance();
  external PromiseJsImpl<void> setRuntimeSettings(dynamic settings);
  external PromiseJsImpl<NormalizedDocument> normalize(
      dynamic file, dynamic params);
}

/// Image class
@JS('Image')
class Image {
  external dynamic get data;
  external int get width;
  external int get height;
}

/// NormalizedDocument class
@JS('NormalizedDocument')
class NormalizedDocument {
  external PromiseJsImpl<dynamic> saveToFile(String filename, bool download);
  external dynamic get image;
}
```

Afterwards, create a `DDNManager` class to add Flutter-specific methods:
- `init()`: Initialize the document normalizer with the resource path and the license key. The resource path is where the js and wasm files are located.

    ```dart
    Future<int> init(String path, String key) async {
      DocumentNormalizer.engineResourcePath = path;
      DocumentNormalizer.license = key;

      _normalizer = await handleThenable(DocumentNormalizer.createInstance());

      return 0;
    }
    ```

- `getParameters()`: Get the runtime parameters of the SDK.

    ```dart
    Future<String> getParameters() async {
      if (_normalizer != null) {
        dynamic settings =
            await handleThenable(_normalizer!.getRuntimeSettings());
        return stringify(settings);
      }

      return '';
    }
    ```

- `setParameters()`: Set the parameters of the SDK.
    ```dart
    Future<int> setParameters(String params) async {
      if (_normalizer != null) {
        await handleThenable(_normalizer!.setRuntimeSettings(params));
        return 0;
      }

      return -1;
    }
    ```

- `detect()`: Detect the edges of the document.
    ```dart
    Future<List<DocumentResult>> detect(String file) async {
      if (_normalizer != null) {
        List<dynamic> results =
            await handleThenable(_normalizer!.detectQuad(file));
        return _resultWrapper(results);
      }

      return [];
    }
    ```

    The result needs to be converted from a JavaScript object to a Dart object.

    ```dart
    List<DocumentResult> _resultWrapper(List<dynamic> results) {
      List<DocumentResult> output = [];

      for (dynamic result in results) {
        Map value = json.decode(stringify(result));
        int confidence = value['confidenceAsDocumentBoundary'];
        List<dynamic> points = value['location']['points'];
        List<Offset> offsets = [];
        for (dynamic point in points) {
          double x = point['x'];
          double y = point['y'];
          offsets.add(Offset(x, y));
        }
        DocumentResult documentResult =
            DocumentResult(confidence, offsets, value['location']);
        output.add(documentResult);
      }

      return output;
    }
    ```

- `normalize()`: Transform the document image based on the four corners.
    ```dart
    Future<NormalizedImage?> normalize(String file, dynamic points) async {
      NormalizedImage? image;
      if (_normalizer != null) {
        _normalizedDocument =
            await handleThenable(_normalizer!.normalize(file, points));

        if (_normalizedDocument != null) {
          Image result = _normalizedDocument!.image;
          dynamic data = result.data;
          Uint8List bytes = Uint8List.fromList(data);
          image = NormalizedImage(bytes, result.width, result.height);
          return image;
        }
      }

      return null;
    }
    ```

- `save()`: Save the normalized document image as JPEG or PNG files to local disk.
    ```dart
    Future<int> save(String filename) async {
      if (_normalizedDocument != null) {
        await handleThenable(_normalizedDocument!.saveToFile(filename, true));
      }
      return 0;
    }
    ```

The detected quadrilaterals and the normalized document image are stored respectively with `DocumentResult` and `NormalizedImage` classes:

```dart
class DocumentResult {
  final int confidence;

  final List<Offset> points;

  final dynamic quad;

  DocumentResult(this.confidence, this.points, this.quad);
}

class NormalizedImage {
  final Uint8List data;

  final int width;

  final int height;

  NormalizedImage(this.data, this.width, this.height);
}

```

In the `flutter_document_scan_sdk_platform_interface.dart` file, we define some interfaces that will be called by Flutter apps.

```dart
Future<int> init(String path, String key) {
  throw UnimplementedError('init() has not been implemented.');
}

Future<NormalizedImage?> normalize(String file, dynamic points) {
  throw UnimplementedError('normalize() has not been implemented.');
}

Future<List<DocumentResult>> detect(String file) {
  throw UnimplementedError('detect() has not been implemented.');
}

Future<int> save(String filename) {
  throw UnimplementedError('save() has not been implemented.');
}

Future<int> setParameters(String params) {
  throw UnimplementedError('setParameters() has not been implemented.');
}

Future<String> getParameters() {
  throw UnimplementedError('getParameters() has not been implemented.');
}
```


The corresponding methods for the web platform are implemented in the `flutter_document_scan_sdk_web.dart` file:

```dart
@override
Future<int> init(String path, String key) async {
  return _ddnManager.init(path, key);
}

@override
Future<NormalizedImage?> normalize(String file, dynamic points) async {
  return _ddnManager.normalize(file, points);
}

@override
Future<List<DocumentResult>> detect(String file) async {
  return _ddnManager.detect(file);
}

@override
Future<int> save(String filename) async {
  return _ddnManager.save(filename);
}

@override
Future<int> setParameters(String params) async {
  return _ddnManager.setParameters(params);
}

@override
Future<String> getParameters() async {
  return _ddnManager.getParameters();
}
```

So far, the Flutter document scan plugin is done. In the next section, we will create a Flutter app to test the plugin.

## Steps to Build a Flutter Web Application to Scan and Save Documents
Before going through the following steps, you need to get a license key from [here](https://www.dynamsoft.com/customer/license/trialLicense/?product=ddn).

### Step 1: Install Dynamsoft Document Normalizer JavaScript SDK and the Flutter Document Scan Plugin
Install the Flutter Document Scan Plugin. 

```bash
flutter pub add flutter_document_scan_sdk
```

Since the Flutter plugin for web does not contain the JavaScript SDK, you must include the `ddn.js` file in the `index.html` file. The JavaScript SDK can be acquired via the  `npm i dynamsoft-document-normalizer` command or use the CDN link `https://cdn.jsdelivr.net/npm/dynamsoft-document-normalizer@latest/dist/ddn.js`.

```html
<script src="https://cdn.jsdelivr.net/npm/dynamsoft-document-normalizer@1.0.11/dist/ddn.js"></script>
```

### Step 2: Initialize the Flutter Document Scan Plugin
When initializing the Flutter Document Scan Plugin, you need to specify the path of the `ddn.js` file and the license key. 

```dart
int ret = await _flutterDocumentScanSdkPlugin.init(
        "https://cdn.jsdelivr.net/npm/dynamsoft-document-normalizer@1.0.11/dist/",
        "DLS2eyJoYW5kc2hha2VDb2RlIjoiMjAwMDAxLTE2NDk4Mjk3OTI2MzUiLCJvcmdhbml6YXRpb25JRCI6IjIwMDAwMSIsInNlc3Npb25QYXNzd29yZCI6IndTcGR6Vm05WDJrcEQ5YUoifQ==");
```

The built-in templates allow you to change the color mode of the normalized document image. The supported color modes are `grayscale`, `binary` and `color`. 

```dart
ret = await _flutterDocumentScanSdkPlugin.setParameters(Template.grayscale);
```

### Step 3: Load Image Files and Normalize Documents in Flutter
We are going to load a document image for edge detection and normalization. There are two existing plugins available for loading image files in Flutter web applications: [image_picker](https://pub.dev/packages/image_picker) and [file_selector](https://pub.dev/packages/file_selector). 


Here we use the `file_selector` plugin to load image files via a button click event.

```dart
MaterialButton(
  textColor: Colors.white,
  color: Colors.blue,
  onPressed: () async {
    const XTypeGroup typeGroup = XTypeGroup(
      label: 'images',
      extensions: <String>['jpg', 'png'],
    );
    final XFile? pickedFile = await openFile(
        acceptedTypeGroups: <XTypeGroup>[typeGroup]);
    if (pickedFile != null) {
      image = await loadImage(pickedFile);
      file = pickedFile.path;
    }
  },
  child: const Text('Load Document')),
```

As the image is loaded, we can call `detect()` and `normalize()` methods to detect the document edges and normalize the document image.

```dart
detectionResults =
    await _flutterDocumentScanSdkPlugin
        .detect(file);

if (detectionResults.isEmpty) {
  print("No document detected");
} else {
  print("Document detected");
  await normalizeFile(
      file, detectionResults[0].points);
}
```

### Step 4: Draw Custom Shapes and Images with Flutter CustomPaint

To verify the detection and normalization results, we'd better show them by UI elements.

The type of the image we have opened is `XFile`. The `XFile` can be decoded into an `Image` object by the `decodeImageFromList()` method:

```dart
Future<ui.Image> loadImage(XFile file) async {
  final data = await file.readAsBytes();
  return await decodeImageFromList(data);
}
```

Although `Image` widget does not support drawing custom shapes, we can draw it with the `CustomPaint` widget. The `CustomPaint` widget allows us to draw custom shapes and images. The following code shows how to draw detected edges on the original image and how to draw the normalized document image.

```dart
class ImagePainter extends CustomPainter {
  ImagePainter(this.image, this.results);
  final ui.Image image;
  final List<DocumentResult> results;

  @override
  void paint(Canvas canvas, Size size) {
    final paint = Paint()
      ..color = Colors.red
      ..strokeWidth = 5
      ..style = PaintingStyle.stroke;

    canvas.drawImage(image, Offset.zero, paint);
    for (var result in results) {
      canvas.drawLine(result.points[0], result.points[1], paint);
      canvas.drawLine(result.points[1], result.points[2], paint);
      canvas.drawLine(result.points[2], result.points[3], paint);
      canvas.drawLine(result.points[3], result.points[0], paint);
    }
  }

Widget createCustomImage(ui.Image image, List<DocumentResult> results) {
    return SizedBox(
      width: image.width.toDouble(),
      height: image.height.toDouble(),
      child: CustomPaint(
        painter: ImagePainter(image, results),
      ),
    );
  }

Center(
  child: Row(
      mainAxisAlignment: MainAxisAlignment.spaceEvenly,
      children: [
    SingleChildScrollView(
      child: Column(
        children: [
          image == null
              ? Image.asset('images/default.png')
              : createCustomImage(image!, detectionResults),
        ],
      ),
    ),
    SingleChildScrollView(
      child: Column(
        children: [
          normalizedUiImage == null
              ? Image.asset('images/default.png')
              : createCustomImage(normalizedUiImage!, []),
        ],
      ),
    ),
  ])),
```

### Step 5: Run the Flutter Web Document Scanning Application

```bash
flutter run -d chrome
```

![Flutter web document edge detection and normalization](https://www.dynamsoft.com/codepool/img/2022/11/flutter-document-edge-detection-normalization.png)


## Source Code
[https://github.com/yushulx/flutter_document_scan_sdk](https://github.com/yushulx/flutter_document_scan_sdk)