How to Build Mobile Check Capture App with Xamarin.Forms and Dynamsoft Document SDK
Tags: xamarinforms, xamarin, mobile, programming
Published: 2023-05-16 07:29:28

A Mobile Check Capture app, also known as a Check Deposit app or Mobile Check Deposit app, is a mobile application that allows users to deposit checks using their smartphones or tablets. It provides a convenient way for individuals and businesses to deposit checks without visiting a bank branch or ATM. Typically, a Mobile Check Capture app utilizes the camera on a mobile device to capture images of the front and back of a check. This article will show how to build a mobile check capture app for both Andorid and iOS platforms using Xamarin.Forms and [Dynamsoft Document Normalizer](https://www.dynamsoft.com/document-normalizer/docs/introduction/?ver=latest). 

## Video Demo
<video src="https://user-images.githubusercontent.com/2202306/238512142-7330eaea-7207-484b-a955-df110a4a88e6.mp4" controls="controls" muted="muted" style="max-height:640px;max-width:100%;"></video>

## NuGet Packages
- [Dynamsoft.DocumentNormalizer.Xamarin.Forms](https://www.nuget.org/packages/Dynamsoft.DocumentNormalizer.Xamarin.Forms)

## Steps to Build a Mobile Check Capture App
In the following sections, we will guide you through the steps to build a mobile check capture app. The functionalities of the app include: 
- Capture images of the front and back of a check from camera stream.
- Detect the check edges.
- Edit the quadrilateral area of the check.

### Step 1: Create a Xamarin.Forms Project
1. In Visual Studio 2022, create a new project using the **Mobile App (Xamarin.Forms)** template.
2. Open **NuGet Package Manager** to install **Dynamsoft.DocumentNormalizer.Xamarin.Forms**:
    
    ![Dynamsoft.DocumentNormalizer.Xamarin.Forms](https://www.dynamsoft.com/codepool/img/2023/05/document-normalizer-xamarin-forms.png)

3. Configure `Info.plist` for iOS and `AndroidManifest.xml` for Android. 

    **iOS Info.plist**
    ```xml
    <key>NSCameraUsageDescription</key>
	<string>This app is using the camera</string>
    ```

    **Android AndroidManifest.xml**
    ```xml
    <?xml version="1.0" encoding="utf-8"?>
    <manifest xmlns:android="http://schemas.android.com/apk/res/android" android:versionCode="1" android:versionName="1.0" package="com.companyname.documentscanner">
        <uses-sdk android:minSdkVersion="21" android:targetSdkVersion="31" />
        <application android:label="DocumentScanner.Android" android:theme="@style/MainTheme"></application>
        <uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />
    </manifest>

    ```

### Step 2: Initialize Dynamsoft Document Normalizer
1. Apply for a [trial license](https://www.dynamsoft.com/customer/license/trialLicense?product=ddn) of Dynamsoft Document Normalizer.
2. Write platform-specific code to initialize Dynamsoft Document Normalizer. 

    **iOS AppDelegate.cs**
    ```csharp
    public override bool FinishedLaunching(UIApplication app, NSDictionary options)
    {
        global::Xamarin.Forms.Forms.Init();

        App.ScreenWidth = UIScreen.MainScreen.Bounds.Width;
        App.ScreenHeight = UIScreen.MainScreen.Bounds.Height;

        LoadApplication(new App(new DCVCameraEnhancer(), new DCVDocumentNormalizer(), new DCVLicenseManager()));

        return base.FinishedLaunching(app, options);
    }
    ```

    **Android MainActivity.cs**
    ```csharp
    protected override void OnCreate(Bundle savedInstanceState)
    {
        base.OnCreate(savedInstanceState);

        Xamarin.Essentials.Platform.Init(this, savedInstanceState);
        global::Xamarin.Forms.Forms.Init(this, savedInstanceState);

        var width = Resources.DisplayMetrics.WidthPixels;
        var height = Resources.DisplayMetrics.HeightPixels;
        var density = Resources.DisplayMetrics.Density;

        App.ScreenWidth = (width - 0.5f) / density;
        App.ScreenHeight = (height - 0.5f) / density;

        LoadApplication(new App(new DCVCameraEnhancer(this), new DCVDocumentNormalizer(), new DCVLicenseManager(this)));
    }
    ```

3. Pass the instancess of `ICameraEnhancer`, `IDocumentNormalizer` and `ILicenseManager` to the constructor of `App` in `App.xaml.cs`.

    ```csharp
    public static ICameraEnhancer dce;
    public static IDocumentNormalizer ddn;
    
    public static double ScreenWidth;
    public static double ScreenHeight;

    public App(ICameraEnhancer enhancer, IDocumentNormalizer normalizer, ILicenseManager manager)
    {
        InitializeComponent();

        dce = enhancer;
        ddn = normalizer;

        MainPage = new NavigationPage(new MainPage(manager));
    }
    ```

4. Set and verify the license in `MainPage.xaml.cs`.

    ```csharp
    public partial class MainPage : ContentPage, ILicenseVerificationListener
    {
        private ILicenseManager licenseManager;
        private bool isLicenseValid = true;

        public MainPage(ILicenseManager licenseManager)
        {
            InitializeComponent();
            this.licenseManager = licenseManager;
            licenseManager.InitLicense("DLS2eyJoYW5kc2hha2VDb2RlIjoiMjAwMDAxLTE2NDk4Mjk3OTI2MzUiLCJvcmdhbml6YXRpb25JRCI6IjIwMDAwMSIsInNlc3Npb25QYXNzd29yZCI6IndTcGR6Vm05WDJrcEQ5YUoifQ==", this);
        }

        public void LicenseVerificationCallback(bool isSuccess, string msg)
        {
            if (!isSuccess)
            {
                Device.BeginInvokeOnMainThread(async () => {
                    isLicenseValid = false;
                    await DisplayAlert("Error", msg, "OK");
                });
            }
        }
    }
    ```

    The `LicenseVerificationCallback()` is invoked in a background thread. To update the UI, we need to use `Device.BeginInvokeOnMainThread()`.
 
### Step 3: Design the UI of the Main Page
There are three Xamarin.Forms widgets used on the main page: 
- Image: Display a static PNG image built as embedded resource.
- Label: Display some description text.
- Button: Navigate to the check information page.

![app main page](https://www.dynamsoft.com/codepool/img/2023/05/check-capture.jpg)

How to load an embedded resource image in Xamarin.Forms? 
1. Add an image file to the project and set its **Build Action** to **Embedded resource**.
    
    ![Xamarin.Forms embedded resource](https://www.dynamsoft.com/codepool/img/2023/05/xamarin-forms-image-resource.png)

2. Create an `ImageResourceExtension` class to load embedded resource images. 

    ```csharp
    // You exclude the 'Extension' suffix when using in Xaml markup
    [Preserve(AllMembers = true)]
    [ContentProperty(nameof(Source))]
    public class ImageResourceExtension : IMarkupExtension
    {
        public string Source { get; set; }

        public object ProvideValue(IServiceProvider serviceProvider)
        {
            if (Source == null)
                return null;

            var imageSource = ImageSource.FromResource(Source, typeof(ImageResourceExtension).GetTypeInfo().Assembly);

            return imageSource;
        }
    }
    ```

3. In a XAML file, import the namespace of `local` and load a PNG file as follows:

    ```xml
    <?xml version="1.0" encoding="utf-8" ?>
    <ContentPage xmlns="http://xamarin.com/schemas/2014/forms"
                xmlns:x="http://schemas.microsoft.com/winfx/2009/xaml"
                xmlns:local="clr-namespace:DocumentScanner;assembly=DocumentScanner" 
                x:Class="DocumentScanner.MainPage"
                BackgroundColor="Transparent"
                Title="CheckCapture">

        <StackLayout Margin="20,20,20,20" >
            <Image Source="{local:ImageResource DocumentScanner.icon-cover.png}"/>
        </StackLayout>

    </ContentPage>
    ```

    The `local:ImageResource` can be used to set an image source for a button as well:
    ```cs
    <Button x:Name="customRenderer" Text="Deposit Check" TextColor="White" HorizontalOptions="Center"  Clicked="OnCustomRendererButtonClicked" BackgroundColor="Orange" ImageSource="{local:ImageResource DocumentScanner.icon-capture.png}" HeightRequest="50" WidthRequest="200"/>
    ```

### Step 4: Design the UI of the Check Information Page
The check information page contains the following widgets:
- TitleView: A custom title view that contains a title label and an upload button.

    ```xml
    <NavigationPage.TitleView>
        <StackLayout Orientation="Horizontal" Margin="0,0,10,0">
            <Label Text="Check" TextColor="white" FontAttributes="Bold" VerticalOptions="Center" HorizontalOptions="StartAndExpand"/>
            <Button
                x:Name="upload"
                Clicked="OnUploadClicked"
                BackgroundColor="transparent"  HeightRequest="50" WidthRequest="50" ImageSource="{local:ImageResource DocumentScanner.icon-upload.png}" VerticalOptions="Center">
            </Button>
        </StackLayout>
    </NavigationPage.TitleView>
    ```
- Editor: Input the check amount.
    ```xml
    <StackLayout Orientation="Horizontal" Margin="20,0,20,20">
        <Label Text="Amount: $" FontAttributes="Bold"
    VerticalOptions="Center" />

        <Editor x:Name="amount" HorizontalOptions="FillAndExpand" Placeholder="" VerticalOptions="Center"/>
    </StackLayout>
    ```
- Frame: Create a layout with rounded corners.
    ```xml
    <Frame CornerRadius="10" Margin="20,20,20,20">
        <StackLayout HeightRequest="250" WidthRequest="400" BackgroundColor="White" >
        </StackLayout>
    </Frame>
    ```
- Image: Display the captured check image. The image is clickable for launching the editor page.
    ```xml
    <Image x:Name="front_image" Aspect="AspectFit">
        <Image.GestureRecognizers>
            <TapGestureRecognizer Tapped="FrontImageTapped" />
        </Image.GestureRecognizers>
    </Image>
    ```
- Label: Display some description text.
- Button: Launch the check capture page.

![check page](https://www.dynamsoft.com/codepool/img/2023/05/check-page.jpg)


To update the image source after capturing or editing image quadrilaterals from another content page, you can use `MessagingCenter` to handle the communication between the pages. 

In the check info page, subscribe to the `ImageData` message. The `CustomRendererPage` is the check capture page.

```cs
MessagingCenter.Subscribe<CustomRendererPage, InfoData>(this, "ImageData", (sender, arg) =>
{
    App.ddn.InitRuntimeSettings(Templates.color);

    NormalizedImageResult normalizedImage = App.ddn.Normalize(arg.imageData, arg.quad);

    if (isFront)
    {
        _frontData = arg;
        front_image.Source = normalizedImage.image.ToImageSource();
        
        if (Device.RuntimePlatform == Device.iOS)
        {
            front_image.RotateTo(normalizedImage.image.orientation + 180);
        }
        else
        {
            front_image.RotateTo(270);
        }
    }
    else
    {
        _backData = arg;
        back_image.Source = normalizedImage.image.ToImageSource();
        
        if (Device.RuntimePlatform == Device.iOS)
        {
            back_image.RotateTo(normalizedImage.image.orientation + 180);
        }
        else
        {
            back_image.RotateTo(270);
        }
    }
});
```

In the `CustomRendererPage`, send the image data to the check info page:

```cs
public void DetectResultCallback(int id, ImageData imageData, DetectedQuadResult[] quadResults)
{
    if (imageData != null && quadResults != null)
    {
        Device.BeginInvokeOnMainThread(async () => {
            ImageData data = new ImageData();
            data.imageSource = imageData.imageSource;
            data.bytes = new List<byte>(imageData.bytes);
            data.width = imageData.width;
            data.height = imageData.height;
            data.stride = imageData.stride;
            data.format = imageData.format;
            data.orientation = imageData.orientation;

            InfoData info = new InfoData();
            info.imageData = data;
            info.quad = quadResults[0].Location;

            MessagingCenter.Send(this, "ImageData", info);

            await Navigation.PopAsync();

        });
    }
}
```


### Step 5: Capture Images of the Front and Back of a Check
The `CustomRendererPage` is used to recognize document edges from camera stream in real-time. To display the camera stream, add the `DCVCameraView` widget. Users should hold the screen in landscape mode to get better capture experience. Therefore, label and button are both rotated 90 degree.

```xml
<?xml version="1.0" encoding="utf-8" ?>
<ContentPage xmlns="http://xamarin.com/schemas/2014/forms"
             xmlns:x="http://schemas.microsoft.com/winfx/2009/xaml"
             xmlns:dynamsoft = "clr-namespace:DDNXamarin;assembly=DDN-Xamarin"
             xmlns:local="clr-namespace:DocumentScanner;assembly=DocumentScanner" 
             x:Class="DocumentScanner.CustomRendererPage"
             Title="Scan Check">
    <ContentPage.Content>
        <AbsoluteLayout>
            <dynamsoft:DCVCameraView AbsoluteLayout.LayoutBounds="0,0,1,1" AbsoluteLayout.LayoutFlags="All" x:Name="preview">
            </dynamsoft:DCVCameraView>
            <Label Text="Please use landscape mode" Rotation="90" TextColor="White" AbsoluteLayout.LayoutBounds="0.7,0.5,300,300" AbsoluteLayout.LayoutFlags="PositionProportional"/>
        
            <Button x:Name="capture" 
                    AbsoluteLayout.LayoutBounds="0.5,0.95,80,80" AbsoluteLayout.LayoutFlags="PositionProportional"
                    Clicked="OnButtonClicked" ImageSource="{local:ImageResource DocumentScanner.icon-capture.png}" Rotation="90" BackgroundColor="Orange" BorderRadius="40" 
        >
            </Button>
        </AbsoluteLayout>
    </ContentPage.Content>
</ContentPage>
```

The button triggers `DetectResultCallback` when the document edges are found.

```cs
void OnButtonClicked(object sender, EventArgs e)
{
    App.ddn.EnableReturnImageOnNextCallback();
}
```

![check edge detection](https://www.dynamsoft.com/codepool/img/2023/05/check-detection.jpg)

### Step 6: Edit the Quadrilateral Area of the Check
The editor page contains a `DCVImageEditorView`, which supports changing and saving the four corners of the document.

```xml
<?xml version="1.0" encoding="utf-8" ?>
<ContentPage xmlns="http://xamarin.com/schemas/2014/forms"
             xmlns:x="http://schemas.microsoft.com/winfx/2009/xaml"
             xmlns:dynamsoft = "clr-namespace:DDNXamarin;assembly=DDN-Xamarin"
              xmlns:local="clr-namespace:DocumentScanner;assembly=DocumentScanner" 
             x:Class="DocumentScanner.QuadEditorPage">
    <NavigationPage.TitleView>
        <StackLayout Orientation="Horizontal" Margin="0,0,10,0">
            <Label Text="Edit and Save Quad" TextColor="white" FontAttributes="Bold" VerticalOptions="Center" HorizontalOptions="StartAndExpand"/>
            <Button
                x:Name="normalize"
                Clicked="OnNormalizeClicked"
                BackgroundColor="transparent"  HeightRequest="50" WidthRequest="50" ImageSource="{local:ImageResource DocumentScanner.icon-save.png}" VerticalOptions="Center">
            </Button>
        </StackLayout>
    </NavigationPage.TitleView>
    
    <ContentPage.Content>
        <AbsoluteLayout>
            <dynamsoft:DCVImageEditorView AbsoluteLayout.LayoutBounds="0,0,1,1" AbsoluteLayout.LayoutFlags="All"
                x:Name="imageEditor">
            </dynamsoft:DCVImageEditorView>

        </AbsoluteLayout>

    </ContentPage.Content>
</ContentPage>
``` 

In the check info page, send the image and the detected quadrilateral to the editor page:

```cs
private void BackImageTapped(object sender, EventArgs e)
{
    
    if (_backData != null) {
        DetectedQuadResult result = new DetectedQuadResult();
        result.Location = _backData.quad;
        Navigation.PushAsync(new QuadEditorPage(_backData.imageData,  new DetectedQuadResult[] {result}));
    }
    
}
```

In the editor page, get the updated quadrilateral and pass it back to the info page via `MessagingCenter`:

```cs
async void OnNormalizeClicked(object sender, EventArgs e)
{
    try
    {
        var quad = imageEditor.getSelectedQuadResult();
        if (quad != null)
        {
            InfoData data = new InfoData();
            data.imageData = this.data;
            data.quad = quad;
            MessagingCenter.Send(this, "ImageData", data);
            await Navigation.PopAsync();
        }
    }
    catch (Exception exception)
    {
        Device.BeginInvokeOnMainThread(async () => {
            await DisplayAlert("Error", exception.ToString(), "OK");
        });
    }
}
```

![check quadrilateral editing](https://www.dynamsoft.com/codepool/img/2023/05/check-edit.jpg)

## Source Code
[https://github.com/yushulx/Xamarin-forms-document-scanner/tree/checkcapture](https://github.com/yushulx/Xamarin-forms-document-scanner/tree/checkcapture)