How to Build Flutter Document Scanning Plugin for Windows and Linux
Tags: flutter, dart, cpp, programming
Published: 2022-12-07 08:50:32

Last week, we finished a Flutter document scanning plugin for web, which is just a start. This week, we continue to empower the plugin for desktop development using [Dynamsoft Document Normalizer C++ SDK](https://www.dynamsoft.com/document-normalizer/docs/introduction/). The plugin will support Windows and Linux. In this article, you will see the steps to build the Flutter desktop document scanning plugin and the differences of Flutter C++ code for Windows and Linux.

## Flutter Document Scan SDK
[https://pub.dev/packages/flutter_document_scan_sdk](https://pub.dev/packages/flutter_document_scan_sdk)

## Development Environment
- Flutter 3.3.9
- Ubuntu 22.04
- Windows 10

## Add Support for Windows and Linux in Flutter Plugin
To support Windows and Linux, run the following command in the project root directory:

```bash
flutter create --org com.dynamsoft --template=plugin --platforms=windows,linux .
```

After generating the platform-specific code, we update the `pubspec.yaml` file:

```yaml
plugin:
    platforms:
      linux:
        pluginClass: FlutterDocumentScanSdkPlugin
      windows:
        pluginClass: FlutterDocumentScanSdkPluginCApi
      web:
        pluginClass: FlutterDocumentScanSdkWeb
        fileName: flutter_document_scan_sdk_web.dart
```


You should have noticed that the Flutter plugin project contains `flutter_document_scan_sdk_web.dart` and `flutter_document_scan_sdk_method_channel.dart` files. The former is used for web. We had added implementations last week. The latter is used for mobile and desktop. It uses method channel to interact with the native platform:

```dart
class MethodChannelFlutterDocumentScanSdk
    extends FlutterDocumentScanSdkPlatform {
  @visibleForTesting
  final methodChannel = const MethodChannel('flutter_document_scan_sdk');

  @override
  Future<String?> getPlatformVersion() async {
    final version =
        await methodChannel.invokeMethod<String>('getPlatformVersion');
    return version;
  }

  @override
  Future<int?> init(String path, String key) async {
    return await methodChannel
        .invokeMethod<int>('init', {'path': path, 'key': key});
  }

  @override
  Future<int?> setParameters(String params) async {
    return await methodChannel
        .invokeMethod<int>('setParameters', {'params': params});
  }

  @override
  Future<String?> getParameters() async {
    return await methodChannel.invokeMethod<String>('getParameters');
  }

  @override
  Future<List<DocumentResult>> detect(String file) async {
    List? results = await methodChannel.invokeListMethod<dynamic>(
      'detect',
      {'file': file},
    );

    return _resultWrapper(results);
  }

  List<DocumentResult> _resultWrapper(List<dynamic>? results) {
    List<DocumentResult> output = [];

    if (results != null) {
      for (var result in results) {
        int confidence = result['confidence'];
        List<Offset> offsets = [];
        int x1 = result['x1'];
        int y1 = result['y1'];
        int x2 = result['x2'];
        int y2 = result['y2'];
        int x3 = result['x3'];
        int y3 = result['y3'];
        int x4 = result['x4'];
        int y4 = result['y4'];
        offsets.add(Offset(x1.toDouble(), y1.toDouble()));
        offsets.add(Offset(x2.toDouble(), y2.toDouble()));
        offsets.add(Offset(x3.toDouble(), y3.toDouble()));
        offsets.add(Offset(x4.toDouble(), y4.toDouble()));
        DocumentResult documentResult = DocumentResult(confidence, offsets, []);
        output.add(documentResult);
      }
    }

    return output;
  }

  @override
  Future<NormalizedImage?> normalize(String file, dynamic points) async {
    Offset offset = points[0];
    int x1 = offset.dx.toInt();
    int y1 = offset.dy.toInt();

    offset = points[1];
    int x2 = offset.dx.toInt();
    int y2 = offset.dy.toInt();

    offset = points[2];
    int x3 = offset.dx.toInt();
    int y3 = offset.dy.toInt();

    offset = points[3];
    int x4 = offset.dx.toInt();
    int y4 = offset.dy.toInt();
    Map? result = await methodChannel.invokeMapMethod<String, dynamic>(
      'normalize',
      {
        'file': file,
        'x1': x1,
        'y1': y1,
        'x2': x2,
        'y2': y2,
        'x3': x3,
        'y3': y3,
        'x4': x4,
        'y4': y4
      },
    );

    if (result != null) {
      return NormalizedImage(
        result['data'],
        result['width'],
        result['height'],
      );
    }

    return null;
  }

  @override
  Future<int?> save(String filename) async {
    return await methodChannel
        .invokeMethod<int>('save', {'filename': filename});
  }
}
```

## Linking Libraries and Writing C/C++ Code

### Linking Shared Libraries in Flutter Plugin
Flutter uses CMake to build the Windows and Linux plugins. So the first step is to setting up the `CMakeLists.txt` file. We need to configure `link_directories`, `target_link_libraries`, and `flutter_document_scan_sdk_bundled_libraries`. 

**Windows**
```cmake
link_directories("${PROJECT_SOURCE_DIR}/lib/") 

target_link_libraries(${PLUGIN_NAME} PRIVATE flutter flutter_wrapper_plugin "DynamsoftCorex64" "DynamsoftDocumentNormalizerx64")

set(flutter_document_scan_sdk_bundled_libraries
  "${PROJECT_SOURCE_DIR}/bin/"
  PARENT_SCOPE
)
```

**Linux**

```cmake
link_directories("${PROJECT_SOURCE_DIR}/lib/") 

target_link_libraries(${PLUGIN_NAME} PRIVATE flutter "DynamsoftCore" "DynamsoftDocumentNormalizer")

set(flutter_document_scan_sdk_bundled_libraries
  
  PARENT_SCOPE
)
```

The linking library files of Dynamsoft Document Normalizer SDK are `DynamsoftCore` and `DynamsoftDocumentNormalizer`. On Windows, they are named `DynamsoftCorex64.lib` and `DynamsoftDocumentNormalizerx64.lib`, whereas on Linux, they are named `libDynamsoftCore.so` and `libDynamsoftDocumentNormalizer.so`. To make Windows plugin work, take one more step to bundle the corresponding dll files.

### Implementing Flutter C/C++ Code Logic for Windows and Linux
The native entry points of the plugin are `FlutterDocumentScanSdkPlugin::HandleMethodCall` and `flutter_document_scan_sdk_plugin_handle_method_call` respectively for Windows and Linux. Because the Flutter header files and data types are different between the two platforms, you have to write code accordingly.

**Windows**
```cpp
#include "flutter_document_scan_sdk_plugin.h"

#include <windows.h>

#include <VersionHelpers.h>

#include <flutter/method_channel.h>
#include <flutter/plugin_registrar_windows.h>
#include <flutter/standard_method_codec.h>

#include <memory>
#include <sstream>

void FlutterDocumentScanSdkPlugin::HandleMethodCall(
      const flutter::MethodCall<flutter::EncodableValue> &method_call,
      std::unique_ptr<flutter::MethodResult<flutter::EncodableValue>> result){}
```

**Linux**
```cpp
#include "include/flutter_document_scan_sdk/flutter_document_scan_sdk_plugin.h"

#include <flutter_linux/flutter_linux.h>
#include <gtk/gtk.h>
#include <sys/utsname.h>

#include <cstring>

static void flutter_document_scan_sdk_plugin_handle_method_call(
    FlutterDocumentScanSdkPlugin *self,
    FlMethodCall *method_call){}
```

To handle the method call, we need to parse the method name and arguments firstly. 

**Windows**

```cpp
const auto *arguments = std::get_if<EncodableMap>(method_call.arguments());

if (method_call.method_name().compare("init") == 0)
{
  std::string license;
  int ret = 0;

  if (arguments)
  {
    auto license_it = arguments->find(EncodableValue("key"));
    if (license_it != arguments->end())
    {
      license = std::get<std::string>(license_it->second);
    }
    ret = DocumentManager::SetLicense(license.c_str());
  }

  result->Success(EncodableValue(ret));
}
...
```

**Linux**

```cpp
if (strcmp(method, "init") == 0)
{
  if (fl_value_get_type(args) != FL_VALUE_TYPE_MAP)
  {
    return;
  }

  FlValue *value = fl_value_lookup_string(args, "key");
  if (value == nullptr)
  {
    return;
  }
  const char *license = fl_value_get_string(value);

  int ret = DocumentManager::SetLicense(license);
  g_autoptr(FlValue) result = fl_value_new_int(ret);
  response = FL_METHOD_RESPONSE(fl_method_success_response_new(result));
}
...
```

As you can see, not only the APIs of parsing the method name are different, but also the data type of the arguments. For example, the `license` argument is `std::string` on Windows, whereas it is `const char*` on Linux.

Now, we create a `document_manager.h` file to implement the methods that calls Dynamsoft Document Normalizer API. Since Dynamsoft Document Normalizer allows you to write code in the same way on Windows and Linux, most of the code can be shared except for wrapping the return values as Flutter data type.

- Import Dynamsoft Document Normalizer SDK header files:

  ```cpp
  #include "DynamsoftCore.h"
  #include "DynamsoftDocumentNormalizer.h"

  using namespace std;
  using namespace dynamsoft::ddn;
  using namespace dynamsoft::core;
  ```

- Initialize and destroy Dynamsoft Document Normalizer:

  ```cpp
  class DocumentManager
  {
  public:
      ~DocumentManager()
      {
          if (normalizer != NULL)
          {
              DDN_DestroyInstance(normalizer);
              normalizer = NULL;
          }

          FreeImage();
      };

      const char *GetVersion()
      {
          return DDN_GetVersion();
    }

    void Init()
    {
        normalizer = DDN_CreateInstance();
        imageResult = NULL;
    }
    private:
      void *normalizer;
      NormalizedImageResult *imageResult;

      void FreeImage()
      {
          if (imageResult != NULL)
          {
              DDN_FreeNormalizedImageResult(&imageResult);
              imageResult = NULL;
          }
      }
  };
  ```

- Set the [license key](https://www.dynamsoft.com/customer/license/trialLicense/?product=ddn):

  ```cpp
  static int SetLicense(const char *license)
  {
      char errorMsgBuffer[512];
      // Click https://www.dynamsoft.com/customer/license/trialLicense/?product=ddn to get a trial license.
      int ret = DC_InitLicense(license, errorMsgBuffer, 512);
      if (ret != DM_OK)
      {
          cout << errorMsgBuffer << endl;
      }
      return ret;
  }
  ```

- Detect document edges by calling `DDN_DetectQuadFromFile()`. On Windows, the detected information is stored in `EncodableMap`, whereas on Linux, it is stored in `FlValue` created by `fl_value_new_map()`. 

  **Windows**
  ```cpp
  EncodableList Detect(const char *filename)
  {
      EncodableList out;
      if (normalizer == NULL)
          return out;

      DetectedQuadResultArray *pResults = NULL;

      int ret = DDN_DetectQuadFromFile(normalizer, filename, "", &pResults);
      if (ret)
      {
          printf("Detection error: %s\n", DC_GetErrorString(ret));
      }

      if (pResults)
      {
          int count = pResults->resultsCount;

          for (int i = 0; i < count; i++)
          {
              EncodableMap map;

              DetectedQuadResult *quadResult = pResults->detectedQuadResults[i];
              int confidence = quadResult->confidenceAsDocumentBoundary;
              DM_Point *points = quadResult->location->points;
              int x1 = points[0].coordinate[0];
              int y1 = points[0].coordinate[1];
              int x2 = points[1].coordinate[0];
              int y2 = points[1].coordinate[1];
              int x3 = points[2].coordinate[0];
              int y3 = points[2].coordinate[1];
              int x4 = points[3].coordinate[0];
              int y4 = points[3].coordinate[1];

              map[EncodableValue("confidence")] = EncodableValue(confidence);
              map[EncodableValue("x1")] = EncodableValue(x1);
              map[EncodableValue("y1")] = EncodableValue(y1);
              map[EncodableValue("x2")] = EncodableValue(x2);
              map[EncodableValue("y2")] = EncodableValue(y2);
              map[EncodableValue("x3")] = EncodableValue(x3);
              map[EncodableValue("y3")] = EncodableValue(y3);
              map[EncodableValue("x4")] = EncodableValue(x4);
              map[EncodableValue("y4")] = EncodableValue(y4);
              out.push_back(map);
          }
      }

      if (pResults != NULL)
          DDN_FreeDetectedQuadResultArray(&pResults);

      return out;
  }
  ```

  **Linux**
  ```cpp
  FlValue* Detect(const char *filename)
  {
      FlValue* out = fl_value_new_list();
      if (normalizer == NULL)
          return out;

      DetectedQuadResultArray *pResults = NULL;

      int ret = DDN_DetectQuadFromFile(normalizer, filename, "", &pResults);
      if (ret)
      {
          printf("Detection error: %s\n", DC_GetErrorString(ret));
      }

      if (pResults)
      {
          int count = pResults->resultsCount;

          for (int i = 0; i < count; i++)
          {
              FlValue* result = fl_value_new_map ();

              DetectedQuadResult *quadResult = pResults->detectedQuadResults[i];
              int confidence = quadResult->confidenceAsDocumentBoundary;
              DM_Point *points = quadResult->location->points;
              int x1 = points[0].coordinate[0];
              int y1 = points[0].coordinate[1];
              int x2 = points[1].coordinate[0];
              int y2 = points[1].coordinate[1];
              int x3 = points[2].coordinate[0];
              int y3 = points[2].coordinate[1];
              int x4 = points[3].coordinate[0];
              int y4 = points[3].coordinate[1];

              fl_value_set_string_take (result, "confidence", fl_value_new_int(confidence));
              fl_value_set_string_take (result, "x1", fl_value_new_int(x1));
              fl_value_set_string_take (result, "y1", fl_value_new_int(y1));
              fl_value_set_string_take (result, "x2", fl_value_new_int(x2));
              fl_value_set_string_take (result, "y2", fl_value_new_int(y2));
              fl_value_set_string_take (result, "x3", fl_value_new_int(x3));
              fl_value_set_string_take (result, "y3", fl_value_new_int(y3));
              fl_value_set_string_take (result, "x4", fl_value_new_int(x4));
              fl_value_set_string_take (result, "y4", fl_value_new_int(y4));

              fl_value_append_take (out, result);
          }
      }

      if (pResults != NULL)
          DDN_FreeDetectedQuadResultArray(&pResults);

      return out;
  }
  ```

- Call `DDN_NormalizeFile()` to normalize the document based on detected corner coordinates:
  ```cpp
  Quadrilateral quad;
  quad.points[0].coordinate[0] = x1;
  quad.points[0].coordinate[1] = y1;
  quad.points[1].coordinate[0] = x2;
  quad.points[1].coordinate[1] = y2;
  quad.points[2].coordinate[0] = x3;
  quad.points[2].coordinate[1] = y3;
  quad.points[3].coordinate[0] = x4;
  quad.points[3].coordinate[1] = y4;

  int errorCode = DDN_NormalizeFile(normalizer, filename, "", &quad, &imageResult);
  if (errorCode != DM_OK)
      printf("%s\r\n", DC_GetErrorString(errorCode));

  if (imageResult)
  {
      ImageData *imageData = imageResult->image;
      int width = imageData->width;
      int height = imageData->height;
      int stride = imageData->stride;
      int format = (int)imageData->format;
      unsigned char* data = imageData->bytes;
      int orientation = imageData->orientation;
      int length = imageData->bytesLength;
  }
  ```

- Create a Flutter C++ map object to pass the normalized image data to Flutter:
  
  **Windows**:
  ```cpp
  EncodableMap map;
  map[EncodableValue("width")] = EncodableValue(width);
  map[EncodableValue("height")] = EncodableValue(height);
  map[EncodableValue("stride")] = EncodableValue(stride);
  map[EncodableValue("format")] = EncodableValue(format);
  map[EncodableValue("orientation")] = EncodableValue(orientation);
  map[EncodableValue("length")] = EncodableValue(length);
  std::vector<uint8_t> rawBytes(rgba, rgba + width * height * 4);
  map[EncodableValue("data")] = rawBytes;
  ```

  **Linux**:
  ```cpp
  FlValue* result = fl_value_new_map ();
  fl_value_set_string_take (result, "width", fl_value_new_int(width));
  fl_value_set_string_take (result, "height", fl_value_new_int(height));
  fl_value_set_string_take (result, "stride", fl_value_new_int(stride));
  fl_value_set_string_take (result, "format", fl_value_new_int(format));
  fl_value_set_string_take (result, "orientation", fl_value_new_int(orientation));
  fl_value_set_string_take (result, "length", fl_value_new_int(length));
  fl_value_set_string_take (result, "data", fl_value_new_uint8_list(rgba, width * height * 4));
  ```

  Note: According to the image color format, we need to do the following conversion before wrapping the data:
  - If the output image is black and white, each byte represents 8 pixels. Each bit of a byte needs to be converted to a byte as 0 or 255. The final length of the image bytes should be the same as the grayscale image.

    ```cpp
    unsigned char *grayscale = new unsigned char[width * height];
    binary2grayscale(data, grayscale, width, height, stride, length);

    void binary2grayscale(unsigned char* data, unsigned char* output, int width, int height, int stride, int length) 
    {
        int index = 0;

        int skip = stride * 8 - width;
        int shift = 0;
        int n = 1;

        for (int i = 0; i < length; ++i)
        {
            unsigned char b = data[i];
            int byteCount = 7;
            while (byteCount >= 0)
            {
                int tmp = (b & (1 << byteCount)) >> byteCount;

                if (shift < stride * 8 * n - skip) {
                    if (tmp == 1)
                        output[index] = 255;
                    else
                        output[index] = 0;
                    index += 1;
                }

                byteCount -= 1;
                shift += 1;
            }

            if (shift == stride * 8 * n) {
                n += 1;
            }
        }
    }
    ```
  - The format of pixel data given to the Flutter `decodeImageFromPixels()` method only supports `rgba8888` and `bgra8888`. So, we need to construct the pixel data in the format of `rgba8888` or `bgra8888` based on the output image color format.

    ```cpp
    unsigned char *rgba = new unsigned char[width * height * 4];
    memset(rgba, 0, width * height * 4);
    if (format == IPF_RGB_888)
    {
        int dataIndex = 0;
        for (int i = 0; i < height; i++)
        {
            for (int j = 0; j < width; j++)
            {
                int index = i * width + j;

                rgba[index * 4] = data[dataIndex + 2];     // red
                rgba[index * 4 + 1] = data[dataIndex + 1]; // green
                rgba[index * 4 + 2] = data[dataIndex];     // blue
                rgba[index * 4 + 3] = 255;                 // alpha
                dataIndex += 3;
            }
        }
    }
    else if (format == IPF_GRAYSCALED)
    {
        int dataIndex = 0;
        for (int i = 0; i < height; i++)
        {
            for (int j = 0; j < width; j++)
            {
                int index = i * width + j;
                rgba[index * 4] = data[dataIndex];
                rgba[index * 4 + 1] = data[dataIndex];
                rgba[index * 4 + 2] = data[dataIndex];
                rgba[index * 4 + 3] = 255;
                dataIndex += 1;
            }
        }
    }
    else if (format == IPF_BINARY)
    {
        unsigned char *grayscale = new unsigned char[width * height];
        binary2grayscale(data, grayscale, width, height, stride, length);

        int dataIndex = 0;
        for (int i = 0; i < height; i++)
        {
            for (int j = 0; j < width; j++)
            {
                int index = i * width + j;
                rgba[index * 4] = grayscale[dataIndex];
                rgba[index * 4 + 1] = grayscale[dataIndex];
                rgba[index * 4 + 2] = grayscale[dataIndex];
                rgba[index * 4 + 3] = 255;
                dataIndex += 1;
            }
        }

        free(grayscale);
    }
    ```

So far, we have completed the implementation of the Flutter Document Scan SDK plugin for Windows and Linux. The next step is to test the plugin on desktop platforms.

## Test Flutter Document Scan SDK Plugin on Windows and Linux
Since the example code we implemented for web is shared with Windows and Linux, there is no code change required. We can directly run the example code on Windows and Linux.

**Windows**
```bash
flutter run -d windows
```

![Flutter windows document edge detection and normalization](https://www.dynamsoft.com/codepool/img/2022/12/flutter-windows-desktop-document-scanner.png)


**Linux**
```bash
flutter run -d linux
```

![Flutter Linux document edge detection and normalization](https://www.dynamsoft.com/codepool/img/2022/12/flutter-linux-desktop-document-scanner.png)

## Source Code
[https://github.com/yushulx/flutter_document_scan_sdk](https://github.com/yushulx/flutter_document_scan_sdk)