Developing .NET MAUI iOS Apps to Scan Barcode, Document and MRZ
Tags: dotnet, maui, ios, csharp
Published: 2024-03-07 09:05:34

In the previous article, we demonstrated how to incorporate iOS MRZ detection frameworks into the [MrzScannerSDK](https://www.nuget.org/packages/MrzScannerSDK) NuGet package. Having mastered the integration of iOS frameworks for a standalone NuGet package, we now turn our attention to integrating iOS SDKs for barcode, MRZ, and document detection into the comprehensive .NET MAUI library: [Capture.Vision.Maui](https://www.nuget.org/packages/Capture.Vision.Maui). Our ultimate goal is to enable the development of cross-platform .NET MAUI applications for **Windows**, **Android** and **iOS** using a single codebase.


## Integrating iOS Frameworks into Document Detection NuGet Package 
Based on our experience with integrating the MRZ detection SDK, we can apply similar steps to incorporate the iOS Document Detection SDK into the [DocumentScannerSDK](https://www.nuget.org/packages/DocumentScannerSDK) NuGet package.

### Step 1: Add iOS Frameworks to the NuGet Package
The Dynamsoft Document Normalizer SDK offers a suite of frameworks for iOS:
- `DynamsoftCore`
- `DynamsoftDocumentNormalizer`
- `DynamsoftImageProcessing`
- `DynamsoftIntermediateResult`

First, we utilize **Objective Sharpie** to create the C# bindings for these frameworks. Following this, we add all iOS frameworks and generated files to an iOS binding project.

![iOS binding project structure](https://www.dynamsoft.com/codepool/img/2024/03/ios-binding-project-structure.png)

The `csproj` file for the binding project is structured as follows:

```xml
<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFramework>net7.0-ios</TargetFramework>
    <Nullable>enable</Nullable>
    <ImplicitUsings>true</ImplicitUsings>
    <AssemblyName>DocumentScannerSDK</AssemblyName>
    <IsBindingProject>true</IsBindingProject>
    <GeneratePackageOnBuild>true</GeneratePackageOnBuild>
  </PropertyGroup>

  <ItemGroup>
    <ObjcBindingApiDefinition Include="DynamsoftCore/ApiDefinitions.cs" />
    <ObjcBindingCoreSource Include="DynamsoftCore/StructsAndEnums.cs" />
    <NativeReference Include="DynamsoftCore.framework">
      <Kind>Framework</Kind>
      <Frameworks></Frameworks>
    </NativeReference>
  </ItemGroup>

  <ItemGroup>
    <ObjcBindingApiDefinition Include="DynamsoftDocumentNormalizer/ApiDefinitions.cs" />
    <ObjcBindingCoreSource Include="DynamsoftDocumentNormalizer/StructsAndEnums.cs" />
    <NativeReference Include="DynamsoftDocumentNormalizer.framework">
      <Kind>Framework</Kind>
      <Frameworks></Frameworks>
    </NativeReference>
  </ItemGroup>

  <ItemGroup>
    <ObjcBindingApiDefinition Include="DynamsoftImageProcessing/ApiDefinitions.cs" />
    <NativeReference Include="DynamsoftImageProcessing.framework">
      <Kind>Framework</Kind>
      <Frameworks></Frameworks>
    </NativeReference>
  </ItemGroup>

  <ItemGroup>
    <ObjcBindingApiDefinition Include="DynamsoftIntermediateResult/ApiDefinitions.cs" />
    <NativeReference Include="DynamsoftIntermediateResult.framework">
      <Kind>Framework</Kind>
      <Frameworks></Frameworks>
    </NativeReference>
  </ItemGroup>

</Project>
```

### Step 2: Align the C# API with Android's Implementation
Copy the `DocumentScanner.cs` file from the Android binding project into the iOS binding project. Modify the code as necessary to ensure it functions correctly on iOS.

- `InitLicense`:

    ```csharp
    public class LicenseVerification : LicenseVerificationListener
    {
        public override void LicenseVerificationCallback(bool isSuccess, NSError error)
        {
            if (!isSuccess)
            {
                System.Console.WriteLine(error.UserInfo);
            }
        }
    }

    public static void InitLicense(string license, object? context = null)
    {
        DynamsoftLicenseManager.InitLicense(license, new LicenseVerification());
    }
    ```

- `SetParameters`：

    ```csharp
    public void SetParameters(string parameters)
    {
        try
        {
            NSError error;
            normalizer.InitRuntimeSettingsFromString(parameters, out error);
        }
        catch (Exception ex)
        {
            Console.WriteLine(ex.ToString());
        }
    }
    ```

- `DetectFile` and `DetectBuffer`:

    ```csharp
    public Result[]? DetectFile(string filename)
    {
        NSError error;
        iDetectedQuadResult[]? results = normalizer.DetectQuadFromFile(filename, out error);
        return GetResults(results);
    }
    
    private Result[]? GetResults(iDetectedQuadResult[]? results)
    {
        if (results == null) return null;

        var result = new Result[results.Length];
        for (int i = 0; i < results.Length; i++)
        {
            iDetectedQuadResult tmp = results[i];
            iQuadrilateral quad = tmp.Location;
            result[i] = new Result()
            {
                Confidence = (int)tmp.ConfidenceAsDocumentBoundary,
                Points = new int[8]
                {
                    (int)((NSValue)quad.Points[0]).CGPointValue.X, (int)((NSValue)quad.Points[0]).CGPointValue.Y,
                    (int)((NSValue)quad.Points[1]).CGPointValue.X, (int)((NSValue)quad.Points[1]).CGPointValue.Y,
                    (int)((NSValue)quad.Points[2]).CGPointValue.X, (int)((NSValue)quad.Points[2]).CGPointValue.Y,
                    (int)((NSValue)quad.Points[3]).CGPointValue.X, (int)((NSValue)quad.Points[3]).CGPointValue.Y,
                }
            };
        }

        return result;
    }

    public Result[]? DetectBuffer(byte[] buffer, int width, int height, int stride, ImagePixelFormat format)
    {
        NSData converted = NSData.FromArray(buffer);

        iImageData imageData = new iImageData()
        {
            Bytes = converted,
            Width = width,
            Height = height,
            Stride = stride,
            Format = (EnumImagePixelFormat)format,
        };
        NSError error;
        iDetectedQuadResult[]? results = normalizer.DetectQuadFromBuffer(imageData, out error);
        return GetResults(results);
    }
    ```

- `NormalizeFile` and `NormalizeBuffer`:
    
    ```csharp
    public NormalizedImage NormalizeFile(string filename, int[] points)
    {
        iQuadrilateral quad = new iQuadrilateral();
        quad.Points = new NSObject[4];
        quad.Points[0] = NSValue.FromCGPoint(new CGPoint(points[0], points[1]));
        quad.Points[1] = NSValue.FromCGPoint(new CGPoint(points[2], points[3]));
        quad.Points[2] = NSValue.FromCGPoint(new CGPoint(points[4], points[5]));
        quad.Points[3] = NSValue.FromCGPoint(new CGPoint(points[6], points[7]));

        NSError error;
        iNormalizedImageResult? result = normalizer.NormalizeFile(filename, quad, out error);
        return GetNormalizedImage(result);
    }

    public NormalizedImage NormalizeBuffer(byte[] buffer, int width, int height, int stride, ImagePixelFormat format, int[] points)
    {
        iImageData imageData = new iImageData()
        {
            Bytes = NSData.FromArray(buffer),
            Width = width,
            Height = height,
            Stride = stride,
            Format = (EnumImagePixelFormat)format,
        };

        iQuadrilateral quad = new iQuadrilateral();
        quad.Points = new NSObject[4];
        quad.Points[0] = NSValue.FromCGPoint(new CGPoint(points[0], points[1]));
        quad.Points[1] = NSValue.FromCGPoint(new CGPoint(points[2], points[3]));
        quad.Points[2] = NSValue.FromCGPoint(new CGPoint(points[4], points[5]));
        quad.Points[3] = NSValue.FromCGPoint(new CGPoint(points[6], points[7]));
        NSError error;
        iNormalizedImageResult? result = normalizer.NormalizeBuffer(imageData, quad, out error);
        return GetNormalizedImage(result);
    }

    private NormalizedImage GetNormalizedImage(iNormalizedImageResult? result)
    {
        NormalizedImage normalizedImage = new NormalizedImage();
        if (result != null)
        {
            iImageData imageData = result.Image;
            normalizedImage.Width = (int)imageData.Width;
            normalizedImage.Height = (int)imageData.Height;
            normalizedImage.Stride = (int)imageData.Stride;
            normalizedImage.Format = (ImagePixelFormat)imageData.Format;
            normalizedImage.Data = imageData.Bytes.ToArray();
        }
        return normalizedImage;
    }
    ```

### Step 3: Resolve Build Errors
The C# binding code generated may lead to build errors. These errors must be addressed and corrected manually.

![iOS binding project build errors](https://www.dynamsoft.com/codepool/img/2024/02/ios-binding-library-build-error.png)

### Step 4: Compile the NuGet Package

```bash
# build dll for desktop
cd desktop
dotnet build --configuration Release

# build dll for android
cd android
dotnet build --configuration Release

# build dll for iOS
cd ios
dotnet build --configuration Release

# build nuget package
nuget pack .\DocumentScannerSDK.nuspec
```

## Merging DocumentScannerSDK and MrzScannerSDK into a Single NuGet Package
The `DynamsoftCore` framework serves as a crucial dependency for both the `MrzScannerSDK` and `DocumentScannerSDK` NuGet packages. Utilizing both packages within a .NET MAUI project leads to a build conflict due to this shared dependency. To overcome this challenge, we introduced a new NuGet package named [CaptureVision](https://www.nuget.org/packages/CaptureVision), which consolidates all frameworks and C# files from both `MrzScannerSDK` and `DocumentScannerSDK` for compilation.

![capture vision iOS binding project](https://www.dynamsoft.com/codepool/img/2024/03/capture-vision-ios-binding-project.png)

## Resolving Build Conflicts between BarcodeQRCodeSDK and CaptureVision
The [BarcodeQRCodeSDK](https://www.nuget.org/packages/BarcodeQRCodeSDK) NuGet package and the `CaptureVision` NuGet package are compatible within the same .NET MAUI project for **Windows** and **Android**. However, a build conflict arises when attempting to compile our demo application for iOS. This conflict is due to certain data structures, such as `iImageData` and `iQuadrilateral`, being defined in both the `DynamsoftCore` and `BarcodeQRCodeSDK`. To address this issue, a viable solution involves removing the redundant data structures from the `BarcodeQRCodeSDK` NuGet package. 


## Completing the Capture.Vision.Maui NuGet Package with iOS Support
Utilizing `CaptureVision 2.0.1` and `BarcodeQRCodeSDK 2.3.7`, we are now able to finalize the `Capture.Vision.Maui` NuGet package by incorporating iOS-specific code within `Platforms/iOS/NativeCameraView.cs`. The code snippet below illustrates the process of detecting documents and MRZ from camera images on iOS.

```csharp
if (cameraView.EnableDocumentDetect)
{
    DocumentScanner.Result[] results = documentScanner.DetectBuffer(bytearray,
                            (int)width,
                            (int)height,
                            (int)bpr,
                            DocumentScanner.ImagePixelFormat.IPF_ARGB_8888);
    DocumentResult documentResults = new DocumentResult();
    if (results != null && results.Length > 0)
    {
        documentResults = new DocumentResult
        {
            Confidence = results[0].Confidence,
            Points = results[0].Points
        };

        if (cameraView.EnableDocumentRectify)
        {
            NormalizedImage normalizedImage = documentScanner.NormalizeBuffer(bytearray,
                            (int)width,
                            (int)height,
                            (int)bpr,
                            DocumentScanner.ImagePixelFormat.IPF_ARGB_8888, documentResults.Points);
            documentResults.Width = normalizedImage.Width;
            documentResults.Height = normalizedImage.Height;
            documentResults.Stride = normalizedImage.Stride;
            documentResults.Format = normalizedImage.Format;
            documentResults.Data = normalizedImage.Data;
        }
    }

    cameraView.NotifyResultReady(documentResults, (int)width, (int)height);

}

if (cameraView.EnableMrz)
{
    MrzResult mrzResults = new MrzResult();
    try
    {
        MrzScanner.Result[] results = mrzScanner.DetectBuffer(bytearray,
                            (int)width,
                            (int)height,
                            (int)bpr,
                            MrzScanner.ImagePixelFormat.IPF_ARGB_8888);


        if (results != null && results.Length > 0)
        {
            Line[] rawData = new Line[results.Length];
            string[] lines = new string[results.Length];

            for (int i = 0; i < results.Length; i++)
            {
                rawData[i] = new Line()
                {
                    Confidence = results[i].Confidence,
                    Text = results[i].Text,
                    Points = results[i].Points,
                };
                lines[i] = results[i].Text;
            }


            Dynamsoft.MrzResult info = MrzParser.Parse(lines);
            mrzResults = new MrzResult()
            {
                RawData = rawData,
                Type = info.Type,
                Nationality = info.Nationality,
                Surname = info.Surname,
                GivenName = info.GivenName,
                PassportNumber = info.PassportNumber,
                IssuingCountry = info.IssuingCountry,
                BirthDate = info.BirthDate,
                Gender = info.Gender,
                Expiration = info.Expiration,
                Lines = info.Lines
            };

        }
    }

    catch (Exception ex)
    {
        System.Diagnostics.Debug.WriteLine(ex.Message);
    }

    cameraView.NotifyResultReady(mrzResults, (int)width, (int)height);
}
```

## Developing a .NET MAUI iOS App for Barcode, Document, and MRZ Detection
To adapt the existing demo app for iOS, simply replicate the license initialization code from the Android section and incorporate it into `Platforms/iOS/Program.cs`:

```csharp
using ObjCRuntime;
using UIKit;
using Dynamsoft;

namespace Capture.Vision.Maui.Example
{
    public class Program
    {
        static void Main(string[] args)
        {
            DocumentScanner.InitLicense("LICENSE-KEY");
            BarcodeQRCodeReader.InitLicense("LICENSE-KEY");
            MrzScanner.InitLicense("LICENSE-KEY");

            UIApplication.Main(args, null, typeof(AppDelegate));
        }
    }
}
```

![.NET MAUI iOS: scan barcode, document and mrz](https://www.dynamsoft.com/codepool/img/2024/03/dotnet-maui-ios-barcode-document-mrz.png)

## Source Code
[https://github.com/yushulx/Capture-Vision-Maui](https://github.com/yushulx/Capture-Vision-Maui)