How to Build Mobile Document Scanning App with Xamarin.Forms for Android and iOS
Tags: dotnet, xamarin, xamarinforms, mobile
Published: 2022-11-11 06:00:37

Mobile document scanning apps bring convenience to users by allowing them to scan travel documents, ID cards, passports and other types of documents on the go. This article aims to help C# developers, who do not have the background in computer vision, to build a mobile document scanning app for both Android and iOS using Xamarin.Forms and [Dynamsoft Document Normalizer SDK](https://www.dynamsoft.com/document-normalizer/docs/introduction/?ver=latest). 


## Dynamsoft.DocumentNormalizer.Xamarin.Forms
Xamarin.Forms enables developers to build cross-platform mobile apps with C# and XAML from a single codebase. [Dynamsoft.DocumentNormalizer.Xamarin.Forms](https://www.nuget.org/packages/Dynamsoft.DocumentNormalizer.Xamarin.Forms) is a Xamarin.Forms wrapper for Dynamsoft Document Normalizer SDK. The primary features of the SDK include:
- Camera view for real-time scanning
- Document edge detection
- Perspective correction 
- Parameter settings for image normalization
- File saving

## Online Documentation
[https://www.dynamsoft.com/document-normalizer/docs/programming/xamarin/user-guide.html](https://www.dynamsoft.com/document-normalizer/docs/programming/xamarin/user-guide.html?ver=latest)


## Steps to Build a Mobile Document Scanning App with Xamarin.Forms
In the following paragraphs, we will use Visual Studio 2022 for Windows to build the document scanning app. 

### Create and Configure Xamarin.Forms Project for Android and iOS
Create an empty Xamarin.Forms project in Visual Studio 2022 and pair the project to a remote macOS. 

![Xamarin Forms project](https://www.dynamsoft.com/codepool/img/2022/11/xamarin-forms-create-configure.png)

Try to build the project for Android and iOS respectively. If you suffered from the iOS build issue `Error MT4109: Failed to compile the generated registrar code` caused by **Xcode 14**, you can visit [https://github.com/xamarin/xamarin-macios/issues/15954](https://github.com/xamarin/xamarin-macios/issues/15954) to find the solution. The issue can be fixed by installing [xamarin.ios-16.0.0.72.pkg](https://bosstoragemirror.blob.core.windows.net/wrench/6.0.4xx-xcode14/6756a11462d8aa6e73d4a320712b276caba159c1/6718459/package/notarized/xamarin.ios-16.0.0.72.pkg). To run the app on `iOS 16`, turn on `Settings > Privacy & Security > Developer Mode`.

The `Xamarin.Forms version` used by Dynamsoft.DocumentNormalizer.Xamarin.Forms is `5.0.0.2515`, whereas the default version of Xamarin.Forms in Visual Studio 2022 is `5.0.0.2196`. To install the SDK, you need to update the version of Xamarin.Forms by right-clicking the project and selecting `Manage NuGet Packages`.

![Update Xamarin.Forms version](https://www.dynamsoft.com/codepool/img/2022/11/xamarin-forms-version-update.png)

Then install the SDK by searching `Dynamsoft.DocumentNormalizer.Xamarin.Forms` in the NuGet package manager.

![Xamarin.Forms document scanning SDK](https://www.dynamsoft.com/codepool/img/2022/11/xamarin-forms-document-scan-sdk.png)

The final step is to add the camera permission for Android and iOS.

**AndroidManifest.xml**
```xml
<uses-permission android:name="android.permission.CAMERA" />
```

**Info.plist**

```xml
<key>NSCameraUsageDescription</key>
<string>This app is using the camera</string>
```

### Initialize the Document Normalizer SDK
Before getting started to write the shared code, you need to initialize the SDK in the platform-specific code. It is a little bit different between Android and iOS.

```csharp
// Android MainActivity.cs
using DDNXamarin.Droid;

protected override void OnCreate(Bundle savedInstanceState)
{
    LoadApplication(new App(new DCVCameraEnhancer(this), new DCVDocumentNormalizer(), new DCVLicenseManager(this)));
}

// iOS AppDelegate.cs
using DDNXamarin.iOS;
public override bool FinishedLaunching(UIApplication app, NSDictionary options)
{
    LoadApplication(new App(new DCVCameraEnhancer(), new DCVDocumentNormalizer(), new DCVLicenseManager()));
}
```

In the shared code `App.xaml.cs`, we register the license key and create static variables to store the `DCVCameraEnhancer` and `DCVDocumentNormalizer` instances for later use. The license key can be obtained from [https://www.dynamsoft.com/customer/license/trialLicense](https://www.dynamsoft.com/customer/license/trialLicense?product=dbr).

```csharp
using DDNXamarin;

public partial class App : Application, ILicenseVerificationListener
{
    public static ICameraEnhancer dce;
    public static IDocumentNormalizer ddn;
    public static ILicenseManager licenseManager;

    public App(ICameraEnhancer enhancer, IDocumentNormalizer normalizer, ILicenseManager manager)
    {
        ...

        dce = enhancer;
        ddn = normalizer;

        licenseManager = manager;
        licenseManager.InitLicense("DLS2eyJoYW5kc2hha2VDb2RlIjoiMjAwMDAxLTE2NDk4Mjk3OTI2MzUiLCJvcmdhbml6YXRpb25JRCI6IjIwMDAwMSIsInNlc3Npb25QYXNzd29yZCI6IndTcGR6Vm05WDJrcEQ5YUoifQ==", this);
        ...
    }

    public void LicenseVerificationCallback(bool isSuccess, string msg)
    {

    }
}

```

### Content Page for Camera Preview and Real-time Scanning
Create a new content page `CustomRendererPage.xaml` and add a `DCVCameraView` control to it. 

```csharp
<?xml version="1.0" encoding="utf-8" ?>
<ContentPage xmlns="http://xamarin.com/schemas/2014/forms"
             xmlns:x="http://schemas.microsoft.com/winfx/2009/xaml"
             xmlns:dynamsoft = "clr-namespace:DDNXamarin;assembly=DDN-Xamarin"
             x:Class="DocumentScanner.CustomRendererPage">
    <ContentPage.Content>
        <AbsoluteLayout>
            <dynamsoft:DCVCameraView AbsoluteLayout.LayoutBounds="0,0,1,1" AbsoluteLayout.LayoutFlags="All" x:Name="preview">
            </dynamsoft:DCVCameraView>
            <Button x:Name="capture" Text="Capture"
                    AbsoluteLayout.LayoutBounds="0.5,0.5,120,50" AbsoluteLayout.LayoutFlags="PositionProportional"
                    Clicked="OnButtonClicked">
            </Button>
        </AbsoluteLayout>
    </ContentPage.Content>
</ContentPage>
```

The `DCVCameraView` control is a custom renderer for Xamarin.Forms. It does not only wrap the native camera view, but also provides the real-time document edge detection feature. 

```csharp
public partial class CustomRendererPage : ContentPage, IDetectResultListener
{
    public static ICameraEnhancer dce;
    public static IDocumentNormalizer ddn;


    public CustomRendererPage()
    {
        InitializeComponent();
        App.ddn.SetCameraEnhancer(App.dce);
        App.ddn.AddResultListener(this);
    }

    public void DetectResultCallback(int id, ImageData imageData, DetectedQuadResult[] quadResults)
    {

        if (imageData != null && quadResults != null)

        {

            Device.BeginInvokeOnMainThread(async () => {
                await Navigation.PushAsync(new QuadEditorPage(imageData, quadResults));
            });
        }
    }

    protected override void OnAppearing()
    {

        base.OnAppearing();
        App.dce.Open();
        App.ddn.StartDetecting();
    }

    protected override void OnDisappearing()
    {
        base.OnDisappearing();
        App.dce.Close();
        App.ddn.StopDetecting();

    }

    void OnButtonClicked(object sender, EventArgs e)
    {
        App.ddn.EnableReturnImageOnNextCallback();
    }
}
```

**Android**

![Xamarin.Forms document edge detection for Android](https://www.dynamsoft.com/codepool/img/2022/11/xamarin-android-document-edge-detection.jpg)

**iOS**

![Xamarin.Forms document edge detection for iOS](https://www.dynamsoft.com/codepool/img/2022/11/xamarin-ios-document-edge-detection.jpg)

Once we get the detected document edges via the callback function, we can launch a new page to edit the document edges.

### Content Page for Editing Document Edges
Create a new content page `QuadEditorPage.xaml` and add a `DCVImageEditorView` control to it. 

```csharp
<?xml version="1.0" encoding="utf-8" ?>
<ContentPage xmlns="http://xamarin.com/schemas/2014/forms"
             xmlns:x="http://schemas.microsoft.com/winfx/2009/xaml"
             xmlns:dynamsoft = "clr-namespace:DDNXamarin;assembly=DDN-Xamarin"
             x:Class="DocumentScanner.QuadEditorPage">
    <ContentPage.Content>
        <AbsoluteLayout>
            <dynamsoft:DCVImageEditorView AbsoluteLayout.LayoutBounds="0,0,1,1" AbsoluteLayout.LayoutFlags="All"
                x:Name="imageEditor">
            </dynamsoft:DCVImageEditorView>

            <Button
                x:Name="normalize"
                Clicked="OnNormalizeClicked"
                Text="Save"
                AbsoluteLayout.LayoutBounds="0.5,0.5,120,50"
                AbsoluteLayout.LayoutFlags="PositionProportional">
            </Button>

        </AbsoluteLayout>

    </ContentPage.Content>
</ContentPage>
```

The `DCVImageEditorView` control allows you to edit the document edges and save the points to a `Quadrilateral` object. 

```csharp
 public partial class QuadEditorPage : ContentPage
{
    ImageData data;
    DetectedQuadResult[] results;
    public QuadEditorPage(ImageData imageData, DetectedQuadResult[] results)
    {
        InitializeComponent();
        data = imageData;
        this.results = results;
    }

    protected override void OnAppearing()
    {
        base.OnAppearing();
        if (data != null)
        {
            imageEditor.OriginalImage = data;
        }
        if (results != null)
        {
            imageEditor.DetectedQuadResults = results;
        }
    }

    void OnNormalizeClicked(object sender, EventArgs e)
    {
        try
        {
            var quad = imageEditor.getSelectedQuadResult();
            if (quad != null)
            {
                Navigation.PushAsync(new NormalizedPage(data, quad));
            }
        }
        catch (Exception exception)
        {
            Device.BeginInvokeOnMainThread(async () => {
                await DisplayAlert("Error", exception.ToString(), "OK");
            });
        }

    }
}
```

**Android**

![Xamarin.Forms document edge editor for Android](https://www.dynamsoft.com/codepool/img/2022/11/xamarin-android-document-edge-edit.jpg)

**iOS**

![Xamarin.Forms document edge editor for iOS](https://www.dynamsoft.com/codepool/img/2022/11/xamarin-ios-document-edge-edit.jpg)

After checking and editing the document edges, we can launch a new page to normalize the document based on the selected quadrilateral.

### Content Page for Document Cropping and Perspective Correction
Create a new content page `NormalizedPage.xaml` to show the normalized document. 

```csharp
<?xml version="1.0" encoding="utf-8" ?>
<ContentPage xmlns="http://xamarin.com/schemas/2014/forms"
             xmlns:x="http://schemas.microsoft.com/winfx/2009/xaml"
             xmlns:local="clr-namespace:DocumentScanner"
             x:Class="DocumentScanner.NormalizedPage">
    <ContentPage.Content>
        <AbsoluteLayout>
            <local:GestureView x:Name="gestureView" AbsoluteLayout.LayoutBounds="0.5,0.5,1024,1024"
                AbsoluteLayout.LayoutFlags="PositionProportional">
            </local:GestureView>

            <Button
                x:Name="shareButton"
                Clicked="OnShareClicked"
                Text="Share"
                AbsoluteLayout.LayoutBounds="0.5,1,120,50"
                AbsoluteLayout.LayoutFlags="PositionProportional">
            </Button>

            <StackLayout RadioButtonGroup.GroupName="colors" Orientation="Horizontal">
                <RadioButton Content="Binary" IsChecked="True" CheckedChanged="RadioButton_CheckedChanged"/>
                <RadioButton Content="Color" CheckedChanged="RadioButton_CheckedChanged"/>
                <RadioButton Content="Grayscale" CheckedChanged="RadioButton_CheckedChanged"/>
            </StackLayout>

        </AbsoluteLayout>
    </ContentPage.Content>
</ContentPage>
```

We put the image in a `GestureView.xaml` control to allow the user to zoom and pan the image. 

```csharp
<?xml version="1.0" encoding="UTF-8"?>
<ContentView xmlns="http://xamarin.com/schemas/2014/forms" 
             xmlns:x="http://schemas.microsoft.com/winfx/2009/xaml"
             x:Class="DocumentScanner.GestureView">
  <ContentView.Content>
        <StackLayout  HorizontalOptions="CenterAndExpand"
                 VerticalOptions="CenterAndExpand"
                     >
            <Image 
                    x:Name="image">
            </Image>

        </StackLayout>
    </ContentView.Content>
</ContentView>
```

The gesture detection code is from Microsoft's [Xamarin.Forms Cookbook](https://learn.microsoft.com/en-us/xamarin/xamarin-forms/app-fundamentals/gestures/pinch). 

```csharp
public partial class GestureView : ContentView
{
    double currentScale = 1;
    double startScale = 1;
    double x = 0;
    double y = 0;

    public Image getImage()
    {
        return image;
    }
    void OnPinchUpdated(object sender, PinchGestureUpdatedEventArgs e)
    {
        ...
    }

    void OnPanUpdated(object sender, PanUpdatedEventArgs e)
    {
        switch (e.StatusType)
        {
            case GestureStatus.Running:
                Content.TranslationX = Math.Max(Math.Min(0, x + e.TotalX), -Math.Abs(Content.Width * Content.Scale - App.ScreenWidth));
                ...
                break;

            ...
        }
    }

    public GestureView()
    {
        InitializeComponent();

        var pinchGesture = new PinchGestureRecognizer();
        pinchGesture.PinchUpdated += OnPinchUpdated;
        GestureRecognizers.Add(pinchGesture);

        var panGesture = new PanGestureRecognizer();
        panGesture.PanUpdated += OnPanUpdated;
        GestureRecognizers.Add(panGesture);
    }
}
```

Code change is required for pan translation calculation.

```diff
- Content.TranslationX = Math.Max (Math.Min (0, x + e.TotalX), -Math.Abs (Content.Width - App.ScreenWidth));
- Content.TranslationY = Math.Max (Math.Min (0, y + e.TotalY), -Math.Abs (Content.Height - App.ScreenHeight));
+ Content.TranslationX = Math.Max(Math.Min(0, x + e.TotalX), -Math.Abs(Content.Width * Content.Scale - App.ScreenWidth));
+ Content.TranslationY = Math.Max(Math.Min(0, y + e.TotalY), -Math.Abs(Content.Height * Content.Scale - App.ScreenHeight));
```

Back to the `NormalizedPage`, we specify the color template by selecting a radio button. 

```csharp
private void UpdateNormalizedImage(string template)
{
    App.ddn.InitRuntimeSettings(template);
    normalizedImage = App.ddn.Normalize(imageData, quadrilateral);

    Image image = gestureView.getImage();

    image.Source = normalizedImage.image.ToImageSource();
    if (Device.RuntimePlatform == Device.iOS)
    {
        image.RotateTo(normalizedImage.image.orientation);
    }
}

private void RadioButton_CheckedChanged(object sender, CheckedChangedEventArgs e)
{
    RadioButton button = sender as RadioButton;
    if (button != null)
    {
        if (button.Content.Equals("Binary") && button.IsChecked)
        {
            UpdateNormalizedImage(Templates.binary);
        }
        if (button.Content.Equals("Color") && button.IsChecked)
        {
            UpdateNormalizedImage(Templates.color);
        }
        if (button.Content.Equals("Grayscale") && button.IsChecked)
        {
            UpdateNormalizedImage(Templates.grayscale);
        }
    }
}
```

The templates are defined in the `Templates` class. 

```csharp
public class Templates
{
    public static string binary = @"{
    ""GlobalParameter"":{
        ""Name"":""GP""
    },
    ""ImageParameterArray"":[
        {
            ""Name"":""IP-1"",
            ""NormalizerParameterName"":""NP-1""
        }
    ],
    ""NormalizerParameterArray"":[
        {
            ""Name"":""NP-1"",
            ""ColourMode"": ""ICM_BINARY"" 
        }
    ]
}";

    public static string color = @"{
    ""GlobalParameter"":{
        ""Name"":""GP""
    },
    ""ImageParameterArray"":[
        {
            ""Name"":""IP-1"",
            ""NormalizerParameterName"":""NP-1""
        }
    ],
    ""NormalizerParameterArray"":[
        {
            ""Name"":""NP-1"",
            ""ColourMode"": ""ICM_COLOUR"" 
        }
    ]
}";

    public static string grayscale = @"{
    ""GlobalParameter"":{
        ""Name"":""GP""
    },
    ""ImageParameterArray"":[
        {
            ""Name"":""IP-1"",
            ""NormalizerParameterName"":""NP-1""
        }
    ],
    ""NormalizerParameterArray"":[
        {
            ""Name"":""NP-1"",
            ""ColourMode"": ""ICM_GRAYSCALE"" 
        }
    ]
}";
}

```

Finally, we can save and share the normalized document image as JPEG, PNG or PDF.

```csharp
string file = Path.Combine(FileSystem.CacheDirectory, "normalized.png");
App.ddn.SaveToFile(normalizedImage, file);
Device.BeginInvokeOnMainThread(async () => {

    await Share.RequestAsync(new ShareFileRequest
    {
        Title = Title,
        File = new ShareFile(file),
        PresentationSourceBounds = DeviceInfo.Platform == DevicePlatform.iOS && DeviceInfo.Idiom == DeviceIdiom.Tablet
        ? new System.Drawing.Rectangle(0, 20, 0, 0)
        : System.Drawing.Rectangle.Empty
    });
});
```

**Android**

![Xamarin.Forms document normalization for Android](https://www.dynamsoft.com/codepool/img/2022/11/xamarin-android-document-perspective-correction.jpg)

**iOS**

![Xamarin.Forms document normalization for iOS](https://www.dynamsoft.com/codepool/img/2022/11/xamarin-ios-document-perspective-correction.jpg)


## Source Code
[https://github.com/yushulx/Xamarin-forms-document-scanner](https://github.com/yushulx/Xamarin-forms-document-scanner)