How to Create a Flutter plugin of Passport MRZ Recognition for Windows, Linux, Android, iOS and Web
Tags: flutter, mrz, ocr, android
Published: 2021-08-09 07:03:57

Since [Dynamsoft Label Recognizer](https://www.dynamsoft.com/label-recognition/overview/) supports **Windows**, **Linux**, **Android**, **iOS** and **Web**, we can create a **5 in 1** Flutter plugin for it to cover **desktop**, **mobile** and **web** platform development. In this article, you will see how to write an MRZ Flutter plugin using different platform-specific code, such as **Java**, **Swift**, **Dart**, **C++** and **JavaScript**.

## Download flutter_ocr_sdk from Pub.dev
[https://pub.dev/packages/flutter_ocr_sdk](https://pub.dev/packages/flutter_ocr_sdk)

## Initialize Flutter MRZ Detection Plugin for Multiple Platforms
We scaffold a Flutter plugin project for multiple platforms by running the following command:

```bash
flutter create --org com.dynamsoft --template=plugin --platforms=android,ios,windows,linux,web -a java flutter_ocr_sdk
```

### MRZ Model 
To implement the package, the first step is to download the Dynamsoft Label Recognizer SDK and add it to the project. From [https://www.dynamsoft.com/label-recognition/downloads](https://www.dynamsoft.com/label-recognition/downloads), you can get shared libraries and model files. 

The MRZ model consists of four files: `MRZ.caffemodel`, `MRZ.json`, `MRZ.prototxt`, and `MRZ.txt`. We put them in the `lib/model` folder of the Flutter project and add the asset path to `pubspec.yaml`:

```yaml
assets:
    - lib/model/
```

### Linking MRZ OCR Libraries
The ways of linking third-party libraries in Flutter plugin project are different for multiple platforms.

#### Windows and Linux
Both Windows and Linux use **CMake** to build the project. We copy header files and shared libraries to the target platform folder and then configure the `CMakeLists.txt` file. 

**Windows**

```cmake

cmake_minimum_required(VERSION 3.14)

set(PROJECT_NAME "flutter_ocr_sdk")
project(${PROJECT_NAME} LANGUAGES CXX)

set(PLUGIN_NAME "flutter_ocr_sdk_plugin")

link_directories("${PROJECT_SOURCE_DIR}/lib/") 

list(APPEND PLUGIN_SOURCES
  "flutter_ocr_sdk_plugin.cpp"
  "flutter_ocr_sdk_plugin.h"
)

add_library(${PLUGIN_NAME} SHARED
  "include/flutter_ocr_sdk/flutter_ocr_sdk_plugin_c_api.h"
  "flutter_ocr_sdk_plugin_c_api.cpp"
  ${PLUGIN_SOURCES}
)

apply_standard_settings(${PLUGIN_NAME})

set_target_properties(${PLUGIN_NAME} PROPERTIES
  CXX_VISIBILITY_PRESET hidden)
target_compile_definitions(${PLUGIN_NAME} PRIVATE FLUTTER_PLUGIN_IMPL)

target_include_directories(${PLUGIN_NAME} INTERFACE
  "${CMAKE_CURRENT_SOURCE_DIR}/include")
target_link_libraries(${PLUGIN_NAME} PRIVATE flutter flutter_wrapper_plugin "DynamsoftLabelRecognizerx64")

set(flutter_ocr_sdk_bundled_libraries
  "${PROJECT_SOURCE_DIR}/bin/"
  PARENT_SCOPE
)
```


**Linux**

```cmake
cmake_minimum_required(VERSION 3.10)

set(PROJECT_NAME "flutter_ocr_sdk")
project(${PROJECT_NAME} LANGUAGES CXX)

set(PLUGIN_NAME "flutter_ocr_sdk_plugin")

link_directories("${PROJECT_SOURCE_DIR}/lib/") 

add_library(${PLUGIN_NAME} SHARED
  "flutter_ocr_sdk_plugin.cc"
)

apply_standard_settings(${PLUGIN_NAME})

set_target_properties(${PLUGIN_NAME} PROPERTIES
  CXX_VISIBILITY_PRESET hidden)
target_compile_definitions(${PLUGIN_NAME} PRIVATE FLUTTER_PLUGIN_IMPL)

target_include_directories(${PLUGIN_NAME} INTERFACE
  "${CMAKE_CURRENT_SOURCE_DIR}/include")
target_link_libraries(${PLUGIN_NAME} PRIVATE flutter "DynamsoftLabelRecognizer")
target_link_libraries(${PLUGIN_NAME} PRIVATE PkgConfig::GTK)

set(flutter_ocr_sdk_bundled_libraries
  ""
  PARENT_SCOPE
)

```

#### Android and iOS
**Gradle** and **CocoaPods** are used to build Android and iOS projects. The minimum supported version of Android is **21** and the minimum supported version of iOS is **9**.


**Configure build.gradle for Android**
```gradle
rootProject.allprojects {
    repositories {
        maven {
            url "https://download2.dynamsoft.com/maven/aar"
        }
        google()
        mavenCentral()
    }
}

apply plugin: 'com.android.library'

android {
    compileSdkVersion 31

    compileOptions {
        sourceCompatibility JavaVersion.VERSION_1_8
        targetCompatibility JavaVersion.VERSION_1_8
    }

    defaultConfig {
        minSdkVersion 21
    }
}

dependencies {
    implementation "com.dynamsoft:dynamsoftlabelrecognizer:2.2.20"
}
```

**Configure flutter_ocr_sdk.podspec for iOS**

```ruby
Pod::Spec.new do |s|
  s.name             = 'flutter_ocr_sdk'
  s.version          = '1.0.0'
  s.summary          = 'A wrapper for Dynamsoft OCR SDK, detecting MRZ in passports, travel documents, and ID cards.'
  s.description      = <<-DESC
A wrapper for Dynamsoft OCR SDK, detecting MRZ in passports, travel documents, and ID cards.
                       DESC
  s.homepage         = 'https://github.com/yushulx/flutter_ocr_sdk'
  s.license          = { :file => '../LICENSE' }
  s.author           = { 'yushulx' => 'lingxiao1002@gmail.com' }
  s.source           = { :path => '.' }
  s.source_files = 'Classes/**/*'
  s.dependency 'Flutter'
  s.platform = :ios, '9.0'
  s.dependency 'DynamsoftLabelRecognizer', '2.2.20'
  # Flutter.framework does not contain a i386 slice.
  s.pod_target_xcconfig = { 'DEFINES_MODULE' => 'YES', 'EXCLUDED_ARCHS[sdk=iphonesimulator*]' => 'i386' }
  s.swift_version = '5.0'
end
```

#### Web
When using the plugin for web, developers need to include the JavaScript library of Dynamsoft Label Recognizer in their `index.html` file:

```html
<script src="https://cdn.jsdelivr.net/npm/dynamsoft-label-recognizer@2.2.11/dist/dlr.js"></script>
```

In the next section, we will write the shared Dart code for all platforms.


## Dart Code for MRZ Recognition
The `flutter_ocr_sdk_platform_interface.dart` defines the methods to be implemented by each platform. 

```dart
abstract class FlutterOcrSdkPlatform extends PlatformInterface {
  FlutterOcrSdkPlatform() : super(token: _token);

  static final Object _token = Object();

  static FlutterOcrSdkPlatform _instance = MethodChannelFlutterOcrSdk();

  static FlutterOcrSdkPlatform get instance => _instance;

  static set instance(FlutterOcrSdkPlatform instance) {
    PlatformInterface.verifyToken(instance, _token);
    _instance = instance;
  }

  Future<int?> init(String path, String key) {
    throw UnimplementedError('init() has not been implemented.');
  }

  Future<List<List<MrzLine>>?> recognizeByBuffer(
      Uint8List bytes, int width, int height, int stride, int format) {
    throw UnimplementedError('recognizeByBuffer() has not been implemented.');
  }

  Future<List<List<MrzLine>>?> recognizeByFile(String filename) {
    throw UnimplementedError('recognizeByFile() has not been implemented.');
  }

  Future<int?> loadModel() async {
    throw UnimplementedError('loadModel() has not been implemented.');
  }
}

```

- `init()`: initialize the OCR SDK with the SDK path (Web Only) and the [license key](https://www.dynamsoft.com/customer/license/trialLicense/?product=dlr) of Dynamsoft Label Recognizer.
- `loadModel()`: load the MRZ recognition model.
- `recognizeByBuffer()`: recognize the MRZ from the image buffer.
- `recognizeByFile()`: recognize the MRZ from the image file.


The `flutter_ocr_sdk_web.dart` file implements the methods for web, and the `flutter_ocr_sdk_method_channel.dart` file implements the methods for Android, iOS, Windows and Linux.

### MRZ Detection API for Web
1. In the `web_dlr_manager.dart` file, declare the JavaScript properties and functions to be called:

    ```dart
    @JS('Dynamsoft')
    library dynamsoft;

    import 'dart:convert';
    import 'dart:typed_data';

    import 'package:js/js.dart';

    import 'mrz_line.dart';
    import 'utils.dart';

    /// DocumentNormalizer class.
    @JS('DLR.LabelRecognizer')
    class LabelRecognizer {
      external static set license(String license);
      external static set engineResourcePath(String resourcePath);
      external static PromiseJsImpl<LabelRecognizer> createInstance();
      external PromiseJsImpl<void> updateRuntimeSettingsFromString(String settings);
      external PromiseJsImpl<List<dynamic>> recognize(dynamic source);
      external PromiseJsImpl<List<dynamic>> recognizeBuffer(
          Uint8List buffer, int width, int height, int stride, int format);
    }
    ```

2. Create a `DLRManager` class to interoperate with the JavaScript library:

    ```dart
    class DLRManager {
      LabelRecognizer? _recognizer;

      Future<int> init(String path, String key) async {
        LabelRecognizer.engineResourcePath = path;
        LabelRecognizer.license = key;

        _recognizer = await handleThenable(LabelRecognizer.createInstance());

        return 0;
      }

      Future<List<List<MrzLine>>?> recognizeByFile(String file) async {
        if (_recognizer != null) {
          List<dynamic> results =
              await handleThenable(_recognizer!.recognize(file));
          return _resultWrapper(results);
        }

        return [];
      }

      Future<List<List<MrzLine>>?> recognizeByBuffer(
          Uint8List bytes, int width, int height, int stride, int format) async {
        if (_recognizer != null) {
          List<dynamic> results = await handleThenable(
              _recognizer!.recognizeBuffer(bytes, width, height, stride, format));
          return _resultWrapper(results);
        }

        return [];
      }

      Future<int?> loadModel() async {
        if (_recognizer != null) {
          await handleThenable(_recognizer!.updateRuntimeSettingsFromString("MRZ"));
        }
        return 0;
      }

      List<List<MrzLine>> _resultWrapper(List<dynamic> results) {
        List<List<MrzLine>> output = [];

        for (dynamic result in results) {
          Map value = json.decode(stringify(result));

          List<dynamic> area = value['lineResults'];
          List<MrzLine> lines = [];
          if (area.length == 2 || area.length == 3) {
            for (int i = 0; i < area.length; i++) {
              MrzLine line = MrzLine();
              line.text = area[i]['text'];
              line.x1 = area[i]['location']['points'][0]['x'];
              line.y1 = area[i]['location']['points'][0]['y'];
              line.x2 = area[i]['location']['points'][1]['x'];
              line.y2 = area[i]['location']['points'][1]['y'];
              line.x3 = area[i]['location']['points'][2]['x'];
              line.y3 = area[i]['location']['points'][2]['y'];
              line.x4 = area[i]['location']['points'][3]['x'];
              line.y4 = area[i]['location']['points'][3]['y'];
              lines.add(line);
            }
          }
          output.add(lines);
        }

        return output;
      }
    }
    ```

### MRZ Detection API for Mobile and Desktop
Comparing to the web, the implementation of loading MRZ model for mobile and desktop is a little bit complicated, because the native APIs are different between mobile and desktop. Thus, the platform check is required in the `flutter_ocr_sdk_method_channel.dart` file.

```dart
Future<int?> loadModel() async {
  final directory = await getApplicationDocumentsDirectory();

  String modelPath = 'packages/flutter_ocr_sdk/lib/model/';

  bool isDesktop = false;
  if (Platform.isWindows || Platform.isLinux) {
    isDesktop = true;
  }

  int? ret = 0;
  var fileNames = ["MRZ"];
  for (var i = 0; i < fileNames.length; i++) {
    var fileName = fileNames[i];

    var prototxtName = '$fileName.prototxt';
    var prototxtBufferPath = join(modelPath, prototxtName);
    ByteData prototxtBuffer = await loadAssetBytes(prototxtBufferPath);

    var txtBufferName = '$fileName.txt';
    var txtBufferPath = join(modelPath, txtBufferName);
    ByteData txtBuffer = await loadAssetBytes(txtBufferPath);

    var characterModelName = '$fileName.caffemodel';
    var characterModelBufferPath = join(modelPath, characterModelName);
    ByteData characterModelBuffer =
        await loadAssetBytes(characterModelBufferPath);

    if (isDesktop) {
      List<int> bytes = prototxtBuffer.buffer.asUint8List();
      await File(join(directory.path, prototxtName)).writeAsBytes(bytes);

      bytes = txtBuffer.buffer.asUint8List();
      await File(join(directory.path, txtBufferName)).writeAsBytes(bytes);

      bytes = characterModelBuffer.buffer.asUint8List();
      await File(join(directory.path, characterModelName))
          .writeAsBytes(bytes);
    } else {
      loadModelFiles(
          fileName,
          prototxtBuffer.buffer.asUint8List(),
          txtBuffer.buffer.asUint8List(),
          characterModelBuffer.buffer.asUint8List());
    }
  }

  var templateName = 'MRZ.json';
  var templatePath = join(modelPath, templateName);
  String template = await loadAssetString(templatePath);
  if (isDesktop) {
    var templateMap = json.decode(template);
    templateMap['CharacterModelArray'][0]['DirectoryPath'] = directory.path;

    ByteData templateBuffer = await loadAssetBytes(templatePath);
    List<int> bytes = templateBuffer.buffer.asUint8List();
    await File(join(directory.path, templateName)).writeAsBytes(bytes);
    await methodChannel.invokeMethod('loadModel',
        {'path': directory.path, 'template': json.encode(templateMap)});
  } else {
    ret = await loadTemplate(template);
  }

  return ret;
}
```

On mobile, the model files are loaded directly from the assets folder. Whereas on desktop, the model files are copied to the application documents directory and then loaded from the directory.

The other three methods are similar to the web implementation.

```dart
Future<int?> init(String path, String key) async {
  return await methodChannel
      .invokeMethod<int>('init', {'path': path, 'key': key});
}

Future<List<List<MrzLine>>?> recognizeByBuffer(
    Uint8List bytes, int width, int height, int stride, int format) async {
  List<dynamic>? results =
      await methodChannel.invokeMethod('recognizeByBuffer', {
    'bytes': bytes,
    'width': width,
    'height': height,
    'stride': stride,
    'format': format,
  });

  if (results == null || results.isEmpty) return [];

  return _resultWrapper(results);
}

Future<List<List<MrzLine>>?> recognizeByFile(String filename) async {
  List<dynamic>? results =
      await methodChannel.invokeMethod('recognizeByFile', {
    'filename': filename,
  });

  if (results == null || results.isEmpty) return [];

  return _resultWrapper(results);
}

List<List<MrzLine>> _resultWrapper(List<dynamic> data) {
  List<List<MrzLine>> results = [];

  for (List<dynamic> area in data) {
    List<MrzLine> lines = [];
    if (area.length == 2 || area.length == 3) {
      for (int i = 0; i < area.length; i++) {
        MrzLine line = MrzLine();
        Map<dynamic, dynamic> map = area[i];
        line.confidence = map['confidence'];
        line.text = map['text'];
        line.x1 = map['x1'];
        line.y1 = map['y1'];
        line.x2 = map['x2'];
        line.y2 = map['y2'];
        line.x3 = map['x3'];
        line.y3 = map['y3'];
        line.x4 = map['x4'];
        line.y4 = map['y4'];
        lines.add(line);
      }
    }

    results.add(lines);
  }

  return results;
}
```


### MRZ Parser
MRZ parser is used to parse the MRZ text and get the information of the passport. The MRZ parser is implemented in the `mrz_parser.dart` file.

```dart
class MRZ {
  /// Parse two lines of MRZ string.
  static MrzResult parseTwoLines(String line1, String line2) {
    MrzResult mrzInfo = MrzResult();
    String type = line1.substring(0, 1);
    RegExp exp = RegExp(r'[I|P|V]');
    RegExpMatch? match = exp.firstMatch(type);
    if (match == null) {
      return mrzInfo;
    }

    if (type == 'P') {
      mrzInfo.type = 'PASSPORT (TD-3)';
    } else if (type == 'V') {
      if (line1.length == 44) {
        mrzInfo.type = 'VISA (MRV-A)';
      } else if (line1.length == 36) {
        mrzInfo.type = 'VISA (MRV-B)';
      }
    } else if (type == 'I') {
      mrzInfo.type = 'ID CARD (TD-2)';
    }

    // Get issuing State information
    String nation = line1.substring(2, 5);
    exp = RegExp(r'[0-9]');
    match = exp.firstMatch(nation);
    if (match != null) return mrzInfo;
    if (nation[nation.length - 1] == '<') {
      nation = nation.substring(0, 2);
    }
    mrzInfo.nationality = nation;
    // Get surname information
    line1 = line1.substring(5);
    int pos = line1.indexOf('<<');
    String surName = line1.substring(0, pos);
    exp = RegExp(r'[0-9]');
    match = exp.firstMatch(surName);
    if (match != null) return mrzInfo;
    surName = surName.replaceAll('<', ' ');
    mrzInfo.surname = surName;
    // Get givenname information
    String givenName = line1.substring(surName.length + 2);
    exp = RegExp(r'[0-9]');
    match = exp.firstMatch(givenName);
    if (match != null) return mrzInfo;
    givenName = givenName.replaceAll('<', ' ');
    givenName = givenName.trim();
    mrzInfo.givenName = givenName;
    // Get passport number information
    String passportNumber = '';
    passportNumber = line2.substring(0, 9);
    passportNumber = passportNumber.replaceAll('<', ' ');
    mrzInfo.passportNumber = passportNumber;
    // Get Nationality information
    String issueCountry = line2.substring(10, 13);
    exp = RegExp(r'[0-9]');
    match = exp.firstMatch(issueCountry);
    if (match != null) return mrzInfo;
    if (issueCountry[issueCountry.length - 1] == '<') {
      issueCountry = issueCountry.substring(0, 2);
    }
    mrzInfo.issuingCountry = issueCountry;
    // Get date of birth information
    String birth = line2.substring(13, 19);
    DateTime now = DateTime.now();
    int currentYear = now.year;
    if (int.parse(birth.substring(0, 2)) > (currentYear % 100)) {
      birth = '19$birth';
    } else {
      birth = '20$birth';
    }
    birth =
        '${birth.substring(0, 4)}/${birth.substring(4, 6)}/${birth.substring(6, 8)}';
    mrzInfo.birthDate = birth;

    // Get gender information
    String gender = line2[20];
    exp = RegExp(r'[M|F|x|<]');
    match = exp.firstMatch(gender);
    if (match == null) return mrzInfo;
    mrzInfo.gender = gender;
    // Get date of expiry information
    String expiry = line2.substring(21, 27);
    exp = RegExp(r'[A-Za-z]');
    match = exp.firstMatch(expiry);
    if (match != null) return mrzInfo;
    if (int.parse(expiry.substring(0, 2)) >= 60) {
      expiry = '19$expiry';
    } else {
      expiry = '20$expiry';
    }
    expiry =
        '${expiry.substring(0, 4)}/${expiry.substring(4, 6)}/${expiry.substring(6)}';
    mrzInfo.expiration = expiry;

    return mrzInfo;
  }

  /// Parse three lines of MRZ string.
  static MrzResult parseThreeLines(String line1, String line2, String line3) {
    MrzResult mrzInfo = MrzResult();
    String type = line1.substring(0, 1);
    RegExp exp = RegExp(r'[I|P|V]');
    RegExpMatch? match = exp.firstMatch(type);
    if (match == null) {
      return mrzInfo;
    }

    mrzInfo.type = 'ID CARD (TD-1)';
    // Get nationality information
    String nation = line2.substring(15, 18);
    exp = RegExp(r'[0-9]');
    match = exp.firstMatch(nation);
    if (match != null) return mrzInfo;
    nation = nation.replaceAll('<', '');
    mrzInfo.nationality = nation;
    // Get surname information
    int pos = line3.indexOf('<<');
    String surName = line3.substring(0, pos);
    exp = RegExp(r'[0-9]');
    match = exp.firstMatch(surName);
    if (match != null) return mrzInfo;
    surName = surName.replaceAll('<', ' ');
    surName.trim();
    mrzInfo.surname = surName;
    // Get givenname information
    String givenName = line3.substring(surName.length + 2);
    exp = RegExp(r'[0-9]');
    match = exp.firstMatch(givenName);
    if (match != null) return mrzInfo;
    givenName = givenName.replaceAll('<', ' ');
    givenName = givenName.trim();
    mrzInfo.givenName = givenName;
    // Get passport number information
    String passportNumber = '';
    passportNumber = line1.substring(5, 14);
    passportNumber = passportNumber.replaceAll('<', ' ');
    mrzInfo.passportNumber = passportNumber;
    // Get issuing country or organization information
    String issueCountry = line1.substring(2, 5);
    exp = RegExp(r'[0-9]');
    match = exp.firstMatch(issueCountry);
    if (match != null) return mrzInfo;
    issueCountry = issueCountry.replaceAll('<', '');
    mrzInfo.issuingCountry = issueCountry;
    // Get date of birth information
    String birth = line2.substring(0, 6);
    exp = RegExp(r'[A-Za-z]');
    match = exp.firstMatch(birth);
    if (match != null) return mrzInfo;

    DateTime now = DateTime.now();
    int currentYear = now.year;
    if (int.parse(birth.substring(0, 2)) > (currentYear % 100)) {
      birth = '19$birth';
    } else {
      birth = '20$birth';
    }
    birth =
        '${birth.substring(0, 4)}/${birth.substring(4, 6)}/${birth.substring(6, 8)}';
    mrzInfo.birthDate = birth;

    // Get gender information
    String gender = line2[7];
    exp = RegExp(r'[M|F|x|<]');
    match = exp.firstMatch(gender);
    if (match == null) return mrzInfo;
    gender = gender.replaceAll('<', 'X');
    mrzInfo.gender = gender;
    // Get date of expiry information
    String expiry = '20$line2.substring(8, 14)';
    exp = RegExp(r'[A-Za-z]');
    match = exp.firstMatch(expiry);
    if (match != null) return mrzInfo;
    expiry =
        '${expiry.substring(0, 4)}/${expiry.substring(4, 6)}/${expiry.substring(6)}';
    mrzInfo.expiration = expiry;

    return mrzInfo;
  }
}
```

## Platform-specific Code of MRZ Recognition for Android, iOS, Windows and Linux
In the above section, we have implemented the MRZ recognition logic in Dart. For web, the Dart code can be directly used. Now, we will write Java, Swift and C++ code for Android, iOS, Windows, and Linux. 


### Get Function Names and Arguments via Method Channel

**Android**

```java
public void onMethodCall(@NonNull MethodCall call, @NonNull Result result) {
  switch (call.method) {
    case "init": {
      final String license = call.argument("key");
      break;
    }
    case "recognizeByFile": {
      final String filename = call.argument("filename");
      final Result r = result;
    }
    break;
    case "recognizeByBuffer": {
      final byte[] bytes = call.argument("bytes");
      final int width = call.argument("width");
      final int height = call.argument("height");
      final int stride = call.argument("stride");
      final int format = call.argument("format");
      final Result r = result;
    }
    break;
    case "loadModelFiles": {
      final String name = call.argument("name");
      final byte[] prototxtBuffer = call.argument("prototxtBuffer");
      final byte[] txtBuffer = call.argument("txtBuffer");
      final byte[] characterModelBuffer = call.argument("characterModelBuffer");
    }
    break;
    case "loadTemplate": {
      final String template = call.argument("template");
    }
    break;
    default:
      result.notImplemented();
  }
}
```

**iOS**

```swift
public func handle(_ call: FlutterMethodCall, result: @escaping FlutterResult) {
  let arguments: NSDictionary = call.arguments as! NSDictionary
  switch call.method {
      case "init":
          completionHandlers.append(result)
          let license: String = arguments.value(forKey: "key") as! String
      case "loadModelFiles":
          let name: String = arguments.value(forKey: "name") as! String
          let prototxtBuffer: FlutterStandardTypedData = arguments.value(forKey: "prototxtBuffer") as! FlutterStandardTypedData
          let txtBuffer: FlutterStandardTypedData = arguments.value(forKey: "txtBuffer") as! FlutterStandardTypedData
          let characterModelBuffer: FlutterStandardTypedData = arguments.value(forKey: "characterModelBuffer") as! FlutterStandardTypedData
      case "loadTemplate":
          if self.recognizer == nil {
              result(.none)
              return
          }
          let params: String = arguments.value(forKey: "template") as! String
      case "recognizeByFile":
          if recognizer == nil {
              result(.none)
              return
          }

          DispatchQueue.global().async {
              let filename: String = arguments.value(forKey: "filename") as! String
          }
      case "recognizeByBuffer":
          if self.recognizer == nil {
              result(.none)
              return
          }

          DispatchQueue.global().async {
              let buffer: FlutterStandardTypedData = arguments.value(forKey: "bytes") as! FlutterStandardTypedData
              let width: Int = arguments.value(forKey: "width") as! Int
              let height: Int = arguments.value(forKey: "height") as! Int
              let stride: Int = arguments.value(forKey: "stride") as! Int
              let format: Int = arguments.value(forKey: "format") as! Int
              let enumImagePixelFormat = EnumImagePixelFormat(rawValue: format)
          }
      default:
          result(.none)
      }
}
```

**Windows**

```cpp
void FlutterOcrSdkPlugin::HandleMethodCall(
      const flutter::MethodCall<flutter::EncodableValue> &method_call,
      std::unique_ptr<flutter::MethodResult<flutter::EncodableValue>> result)
{

  const auto *arguments = std::get_if<EncodableMap>(method_call.arguments());

  if (method_call.method_name().compare("init") == 0)
  {
    std::string license;
    int ret = 0;

    if (arguments)
    {
      auto license_it = arguments->find(EncodableValue("key"));
      if (license_it != arguments->end())
      {
        license = std::get<std::string>(license_it->second);
      }
    }
  }
  else if (method_call.method_name().compare("loadModel") == 0)
  {
    std::string path, params;
    int ret = 0;

    if (arguments)
    {
      auto path_it = arguments->find(EncodableValue("path"));
      if (path_it != arguments->end())
      {
        path = std::get<std::string>(path_it->second);
      }

      auto params_it = arguments->find(EncodableValue("template"));
      if (params_it != arguments->end())
      {
        params = std::get<std::string>(params_it->second);
      }
    }
  }
  else if (method_call.method_name().compare("recognizeByFile") == 0)
  {
    std::string filename;
    EncodableList results;

    if (arguments)
    {
      auto filename_it = arguments->find(EncodableValue("filename"));
      if (filename_it != arguments->end())
      {
        filename = std::get<std::string>(filename_it->second);
      }
    }
  }
  else if (method_call.method_name().compare("recognizeByBuffer") == 0)
  {
    EncodableList results;

    std::vector<unsigned char> bytes;
    int width = 0, height = 0, stride = 0, format = 0;

    if (arguments)
    {
      auto bytes_it = arguments->find(EncodableValue("bytes"));
      if (bytes_it != arguments->end())
      {
        bytes = std::get<vector<unsigned char>>(bytes_it->second);
      }

      auto width_it = arguments->find(EncodableValue("width"));
      if (width_it != arguments->end())
      {
        width = std::get<int>(width_it->second);
      }

      auto height_it = arguments->find(EncodableValue("height"));
      if (height_it != arguments->end())
      {
        height = std::get<int>(height_it->second);
      }

      auto stride_it = arguments->find(EncodableValue("stride"));
      if (stride_it != arguments->end())
      {
        stride = std::get<int>(stride_it->second);
      }

      auto format_it = arguments->find(EncodableValue("format"));
      if (format_it != arguments->end())
      {
        format = std::get<int>(format_it->second);
      }
    }
  }
  else
  {
    result->NotImplemented();
  }
}
```

**Linux**
```cpp
static void flutter_ocr_sdk_plugin_handle_method_call(
    FlutterOcrSdkPlugin* self,
    FlMethodCall* method_call) {
  g_autoptr(FlMethodResponse) response = nullptr;

  const gchar* method = fl_method_call_get_name(method_call);
  FlValue* args = fl_method_call_get_args(method_call);

  if (strcmp(method, "init") == 0)
  {
    if (fl_value_get_type(args) != FL_VALUE_TYPE_MAP)
    {
      return;
    }

    FlValue *value = fl_value_lookup_string(args, "key");
    if (value == nullptr)
    {
      return;
    }
    const char *license = fl_value_get_string(value);
  }
  else if (strcmp(method, "loadModel") == 0)
  {
    if (fl_value_get_type(args) != FL_VALUE_TYPE_MAP)
    {
      return;
    }

    FlValue *value = fl_value_lookup_string(args, "path");
    if (value == nullptr)
    {
      return;
    }
    const char *path = fl_value_get_string(value);

    value = fl_value_lookup_string(args, "template");
    if (value == nullptr)
    {
      return;
    }
    const char * params = fl_value_get_string(value);

    int ret = self->manager->LoadModel(path, params);
  }
  else if (strcmp(method, "recognizeByFile") == 0)
  {
    if (fl_value_get_type(args) != FL_VALUE_TYPE_MAP)
    {
      return;
    }

    FlValue *value = fl_value_lookup_string(args, "filename");
    if (value == nullptr)
    {
      return;
    }
    const char *filename = fl_value_get_string(value);

  }
  else if (strcmp(method, "recognizeByBuffer") == 0)
  {
    if (fl_value_get_type(args) != FL_VALUE_TYPE_MAP) {
      return;
    }

    FlValue* value = fl_value_lookup_string(args, "bytes");
    if (value == nullptr) {
      return;
    }
    unsigned char* bytes = (unsigned char*)fl_value_get_uint8_list(value);

    value = fl_value_lookup_string(args, "width");
    if (value == nullptr) {
      return;
    }
    int width = fl_value_get_int(value);

    value = fl_value_lookup_string(args, "height");
    if (value == nullptr) {
      return;
    }
    int height = fl_value_get_int(value);

    value = fl_value_lookup_string(args, "stride");
    if (value == nullptr) {
      return;
    }
    int stride = fl_value_get_int(value);

    value = fl_value_lookup_string(args, "format");
    if (value == nullptr) {
      return;
    }
    int format = fl_value_get_int(value);

  }
  else {
    response = FL_METHOD_RESPONSE(fl_method_not_implemented_response_new());
  }

  fl_method_call_respond(method_call, response, nullptr);
}
```

### The Init() Method

**Android**

```java
LicenseManager.initLicense(
  license, context,
      new LicenseVerificationListener() {
          @Override
          public void licenseVerificationCallback(boolean isSuccessful, CoreException e) {
              if (isSuccessful)
              {
                  result.success(0);
              }
              else {
                  result.success(-1);
              }
          }
      });
```

**iOS**

```swift
DynamsoftLicenseManager.initLicense(license, verificationDelegate: self)

public func licenseVerificationCallback(_ isSuccess: Bool, error: Error?) {
  if isSuccess {
      completionHandlers.first?(0)
  } else{
      completionHandlers.first?(-1)
  }
}
```

**Windows**

```cpp
char errorMsgBuffer[512];
int ret = DLR_InitLicense(license, errorMsgBuffer, 512);
recognizer = DLR_CreateInstance();
```

**Linux**

```cpp
char errorMsgBuffer[512];
int ret = DLR_InitLicense(license, errorMsgBuffer, 512);
recognizer = DLR_CreateInstance();
```

### The LoadModel() Method

**Android**

```java
try {
    mLabelRecognizer.appendCharacterModelBuffer(name, prototxtBuffer, txtBuffer, characterModelBuffer);
    mLabelRecognizer.initRuntimeSettings(content);
} catch (Exception e) {
    e.printStackTrace();
}
```

**iOS**

```swift
DynamsoftLabelRecognizer.appendCharacterModel(name, prototxtBuffer: prototxtBuffer.data, txtBuffer: txtBuffer.data, characterModelBuffer: characterModelBuffer.data)

try? self.recognizer!.initRuntimeSettings(params)
```

**Windows**
```cpp
char errorMessage[256];
int ret = DLR_AppendSettingsFromString(recognizer, params, errorMessage, 256);
```

**Linux**

```cpp
char errorMessage[256];
int ret = DLR_AppendSettingsFromString(recognizer, params, errorMessage, 256);
```

### The RecognizeByFile() Method

**Android**
```java
DLRResult[] results = null;
try {
    results = mLabelRecognizer.recognizeFile(fileName);
} catch (Exception e) {
    Log.e(TAG, e.toString());
}
```

**iOS**

```swift
let res = try? self.recognizer!.recognizeFile(filename)
```

**Windows**

```cpp
int ret = DLR_RecognizeByFile(recognizer, filename, "locr");
```

**Linux**

```cpp
int ret = DLR_RecognizeByFile(recognizer, filename, "locr");
```

### The RecognizeByBuffer() Method

**Android**
```java
DLRResult[] results = null;
ImageData data = new ImageData();
data.bytes = bytes;
data.width = width;
data.height = height;
data.stride = stride;
data.format = format;
try {
    results = mLabelRecognizer.recognizeBuffer(data);
} catch (Exception e) {
    Log.e(TAG, e.toString());
}
```
**iOS**

```swift
let imageData = iImageData.init()
imageData.bytes = buffer.data
imageData.width = width
imageData.height = height
imageData.stride = stride
imageData.format = enumImagePixelFormat!
let res = try? self.recognizer!.recognizeBuffer(imageData)
```

**Windows**

```cpp
ImageData data;
data.bytes = buffer;
data.width = width;
data.height = height;
data.stride = stride;
data.format = pixelFormat;
data.bytesLength = length;

int ret = DLR_RecognizeByBuffer(recognizer, &data, "locr");
```

**Linux**

```cpp
ImageData data;
data.bytes = buffer;
data.width = width;
data.height = height;
data.stride = stride;
data.format = pixelFormat;
data.bytesLength = length;

int ret = DLR_RecognizeByBuffer(recognizer, &data, "locr");
```

## Building a Flutter App to Recognize Passport MRZ
1. Add `image_picker`, `file_selector` and `flutter_ocr_sdk` to `pubspec.yaml`. Both `image_picker` and `file_selector` are used to select an image file from the local file system, but they are implemented for different platforms.
    
    ```yaml
    dependencies:
      ...

      flutter_ocr_sdk:
      image_picker:
      file_selector:
    ```
2. Create a stateful widget:
    
    ```dart
    import 'dart:typed_data';

    import 'package:file_selector/file_selector.dart';
    import 'package:flutter/material.dart';
    import 'dart:async';

    import 'dart:io';

    import 'package:flutter_ocr_sdk/flutter_ocr_sdk.dart';
    import 'package:flutter_ocr_sdk/flutter_ocr_sdk_platform_interface.dart';
    import 'package:flutter_ocr_sdk/mrz_line.dart';
    import 'package:flutter_ocr_sdk/mrz_parser.dart';
    import 'package:image_picker/image_picker.dart';

    import 'package:flutter/foundation.dart' show kIsWeb;

    import 'dart:ui' as ui;

    Future<void> main() async {
      runApp(
        MaterialApp(
          title: 'MRZ OCR',
          home: Scaffold(
            appBar: AppBar(
              title: const Text("MRZ OCR"),
            ),
            body: MRZApp(),
          ),
        ),
      );
    }

    class MRZApp extends StatefulWidget {
      @override
      MobileState createState() => MobileState();
    }
    ```
3. Initialize the MRZ detection SDK:
    
    ```dart
    class MobileState extends State<MRZApp> {
      late FlutterOcrSdk _mrzDetector;
      final picker = ImagePicker();

      @override
      void initState() {
        super.initState();

        initSDK();
      }

      Future<void> initSDK() async {
        _mrzDetector = FlutterOcrSdk();
        int? ret = await _mrzDetector.init(
            "https://cdn.jsdelivr.net/npm/dynamsoft-label-recognizer@2.2.11/dist/",
            "DLS2eyJoYW5kc2hha2VDb2RlIjoiMjAwMDAxLTE2NDk4Mjk3OTI2MzUiLCJvcmdhbml6YXRpb25JRCI6IjIwMDAwMSIsInNlc3Npb25QYXNzd29yZCI6IndTcGR6Vm05WDJrcEQ5YUoifQ==");
        await _mrzDetector.loadModel();
      }
    }
    ```

4. Build the UI:
    
    ```dart
    Widget build(BuildContext context) {
      double width = MediaQuery.of(context).size.width;
      double height = MediaQuery.of(context).size.height;
      double left = 5;
      double mrzHeight = 50;
      double mrzWidth = width - left * 2;
      return Scaffold(
        body: Stack(children: [
          Center(
            child: Column(
                mainAxisAlignment: MainAxisAlignment.spaceEvenly,
                children: [
                  MaterialButton(
                    textColor: Colors.white,
                    color: Colors.blue,
                    onPressed: () async {
                      showDialog(
                          context: context,
                          builder: (BuildContext context) {
                            return const Center(
                              child: CircularProgressIndicator(),
                            );
                          });
                      pictureScan('gallery');
                    },
                    child: const Text('Pick gallery image'),
                  ),
                  MaterialButton(
                    textColor: Colors.white,
                    color: Colors.blue,
                    onPressed: () async {
                      showDialog(
                          context: context,
                          builder: (BuildContext context) {
                            return const Center(
                              child: CircularProgressIndicator(),
                            );
                          });
                      pictureScan('camera');
                    },
                    child: const Text('Pick camera image'),
                  ),
                ]),
          )
        ]),
      );
    }
    ```

5. Press the button to load an image file from the local file system and then recognize MRZ from the image:
    
    ```dart
    void pictureScan(String source) async {
      XFile? photo;
      if (kIsWeb || Platform.isAndroid || Platform.isIOS) {
        if (source == 'camera') {
          photo = await picker.pickImage(source: ImageSource.camera);
        } else {
          photo = await picker.pickImage(source: ImageSource.gallery);
        }
      } else if (Platform.isWindows || Platform.isLinux) {
        const XTypeGroup typeGroup = XTypeGroup(
          label: 'images',
          extensions: <String>['jpg', 'png', 'bmp', 'tiff', 'pdf', 'gif'],
        );
        photo = await openFile(acceptedTypeGroups: <XTypeGroup>[typeGroup]);
      }

      if (photo == null) {
        if (!mounted) return;
        Navigator.pop(context);
        return;
      }

      String information = 'No results';

      List<List<MrzLine>>? results =
          await _mrzDetector.recognizeByFile(photo.path);
      if (results != null && results.isNotEmpty) {
        for (List<MrzLine> area in results) {
          if (area.length == 2) {
            information =
                MRZ.parseTwoLines(area[0].text, area[1].text).toString();
          } else if (area.length == 3) {
            information = MRZ
                .parseThreeLines(area[0].text, area[1].text, area[2].text)
                .toString();
          }
        }
      }

      if (!mounted) return;
      Navigator.pop(context);
      Navigator.push(
        context,
        MaterialPageRoute(
          builder: (context) => DisplayPictureScreen(
              imagePath: photo!.path, mrzInformation: information),
        ),
      );
    }
    ```
6. Display the MRZ information on a stateless widget:

    ```dart
    Image getImage(String imagePath) {
      if (kIsWeb) {
        return Image.network(imagePath);
      } else {
        return Image.file(
          File(imagePath),
          fit: BoxFit.contain,
          height: double.infinity,
          width: double.infinity,
          alignment: Alignment.center,
        );
      }
    }
    class DisplayPictureScreen extends StatelessWidget {
      final String imagePath;
      final String mrzInformation;

      const DisplayPictureScreen(
          {Key? key, required this.imagePath, required this.mrzInformation})
          : super(key: key);

      @override
      Widget build(BuildContext context) {
        return Scaffold(
          appBar: AppBar(title: const Text('MRZ OCR')),
          body: Stack(
            alignment: const Alignment(0.0, 0.0),
            children: [
              getImage(imagePath),
              Container(
                decoration: const BoxDecoration(
                  color: Colors.black45,
                ),
                child: Text(
                  mrzInformation,
                  style: const TextStyle(
                    fontSize: 14,
                    color: Colors.white,
                  ),
                ),
              ),
            ],
          ),
        );
      }
    }
    ```

7. Run the app on web, Windows, Linux, Android and iOS platforms.

    **Web**

    ![Flutter MRZ recognition in web](https://www.dynamsoft.com/codepool/img/2023/01/flutter_mrz_ocr_web.png)

    **Windows and Linux**

    ![Flutter MRZ OCR in Windows](https://www.dynamsoft.com/codepool/img/2023/01/flutter-mrz-ocr-windows.png)

    **Android and iOS**

    ![Flutter Passport MRZ recognition](https://www.dynamsoft.com/codepool/img/2021/07/flutter-passport-mrz-recognition.jpg)

## Source Code
[https://github.com/yushulx/flutter_ocr_sdk](https://github.com/yushulx/flutter_ocr_sdk)