How to Build a Windows Desktop App for Document, Barcode, and MRZ Detection with C# and .NET WinForms
Tags: dotnet, windows, programming, csharp
Published: 2023-07-31 08:11:22

Dynamsoft's Capture Vision SDKs consist of the [Dynamsoft Document Normalizer](https://www.dynamsoft.com/document-normalizer/docs/introduction/?ver=latest), [Dynamsoft Barcode Reader](https://www.dynamsoft.com/barcode-reader/overview/), and [Dynamsoft Label Recognizer](https://www.dynamsoft.com/label-recognition/overview/). All of these SDKs are cross-platform, supporting **Windows**, **Linux**, **Android**, **iOS** and **web**. Each vision SDK can be used independently or in combination with others. In this article, we'll show you how to build a Windows desktop app that integrates Dynamsoft's vision APIs for document rectification, barcode scanning, and MRZ detection using C# and .NET WinForms.

<video src="https://user-images.githubusercontent.com/2202306/256767567-1ce1b1f9-df32-4db8-af74-0ce808768572.mp4" controls="controls" muted="muted" style="max-height:640px; min-height: 200px; max-width: 100%;"></video>

## Development Environment
- Visual Studio 2022
- Visual Studio Code
- .NET 6.0 SDK or later

## NuGet Packages
**OpenCV**
- [OpenCvSharp4](https://www.nuget.org/packages/OpenCvSharp4/)
- [OpenCvSharp4.Extensions](https://www.nuget.org/packages/OpenCvSharp4.Extensions/)
- [OpenCvSharp4.runtime.win](https://www.nuget.org/packages/OpenCvSharp4.runtime.win/)

We use OpenCV to access the camera and display the video stream.

**Dynamsoft Vision SDKs**
- [BarcodeQRCodeSDK](https://www.nuget.org/packages/BarcodeQRCodeSDK/): Used to detect 1D and 2D barcodes.
- [DocumentScannerSDK](https://www.nuget.org/packages/DocumentScannerSDK/): Used to detect document edges and rectify documents.
- [MrzScannerSDK](https://www.nuget.org/packages/MrzScannerSDK/): Used to detect MRZ (**Machine Readable Zone**) on **passport**, **Visa**, **ID card** and other **travel documents**.

To utilize the Dynamsoft Vision SDKs, you need a valid license. A free trial license is available from the [Dynamsoft Customer Portal](https://www.dynamsoft.com/customer/license/trialLicense?product=dbr&source=codepool). You have the option to request individual licenses for each of the three SDKs, or a unified license that covers all SDKs.

## Create a Windows Forms Project
In Visual Studio, create a new `Windows Forms` project for `.NET`, not for the `.NET Framework`.

![Create a WinForms project](https://www.dynamsoft.com/codepool/img/2023/07/windows-form-template.png)

In the Solution Explorer, right-click the project and select **Manage NuGet Packages**. Search for the above NuGet packages and install them.

```xml
<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <OutputType>WinExe</OutputType>
    <TargetFramework>net6.0-windows</TargetFramework>
    <Nullable>enable</Nullable>
    <UseWindowsForms>true</UseWindowsForms>
    <ImplicitUsings>enable</ImplicitUsings>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="BarcodeQRCodeSDK " Version="2.3.4" />
    <PackageReference Include="DocumentScannerSDK " Version="1.1.0" />
    <PackageReference Include="MrzScannerSDK" Version="1.2.0" />
    <PackageReference Include="OpenCvSharp4" Version="4.6.0.20220608" />
  </ItemGroup>
  <ItemGroup>
    <PackageReference Include="OpenCvSharp4.Extensions" Version="4.5.5.20211231" />
    <PackageReference Include="OpenCvSharp4.runtime.win" Version="4.6.0.20220608" />
  </ItemGroup>
</Project>
```

## Windows Form UI Design
The UI contains some basic controls, such as `ToolStripStatusLabel`, `ToolStripMenuItem`, `PictureBox`, `Button`, `CheckBox`, and `RichTextBox`.

![Windows capture vision UI design](https://www.dynamsoft.com/codepool/img/2023/07/windows-capture-vision-ui-design.png)

- There are two `PictureBox` controls. One is used to display the input image or video stream. The other is used to display the captured image with the detected results.
- The `ToolStripMenuItem` allows you to enter the license key and SDK-relevant templates.
- The `CheckBox` is used to enable or disable the corresponding SDK. You will see the corresponding vision effect in the `PictureBox` control.
- The `Button` provides the operations of loading an image, toggling the camera, as well as saving the captured image.
- The `ToolStripStatusLabel` makes you know the status of the SDKs.
- The `RichTextBox` displays the detected results.

## How to Initialize Dynamsoft Vision SDKs
Press `F7` to view the code behind the form. Within the constructor, initialize the SDKs and set the license keys.

```csharp
using Dynamsoft;
using static Dynamsoft.MrzScanner;
using static Dynamsoft.DocumentScanner;
using static Dynamsoft.BarcodeQRCodeReader;

using MrzResult = Dynamsoft.MrzScanner.Result;
using DocResult = Dynamsoft.DocumentScanner.Result;
using BarcodeResult = Dynamsoft.BarcodeQRCodeReader.Result;

public Form1()
{
    InitializeComponent();
    FormClosing += new FormClosingEventHandler(Form1_Closing);
    string license = "DLS2eyJoYW5kc2hha2VDb2RlIjoiMjAwMDAxLTE2NDk4Mjk3OTI2MzUiLCJvcmdhbml6YXRpb25JRCI6IjIwMDAwMSIsInNlc3Npb25QYXNzd29yZCI6IndTcGR6Vm05WDJrcEQ5YUoifQ==";

    ActivateLicense(license);

    // Initialize camera
    capture = new VideoCapture(0);
    isCapturing = false;

    // Initialize MRZ scanner
    mrzScanner = MrzScanner.Create();
    mrzScanner.LoadModel();

    // Initialize document scanner
    documentScanner = DocumentScanner.Create();
    documentScanner.SetParameters(DocumentScanner.Templates.color);

    // Initialize barcode scanner
    barcodeScanner = BarcodeQRCodeReader.Create();
}

private void ActivateLicense(string license)
{
    int ret = MrzScanner.InitLicense(license);
    ret = DocumentScanner.InitLicense(license);
    BarcodeQRCodeReader.InitLicense(license);
    if (ret != 0)
    {
        toolStripStatusLabel1.Text = "License is invalid.";
    }
    else
    {
        toolStripStatusLabel1.Text = "License is activated successfully.";
    }
}
```

The MRZ model can be found in the NuGet package. 

![MRZ model](https://www.dynamsoft.com/codepool/img/2023/07/mrz-model.png)

While developing the .NET application, the model will be automatically sourced from the NuGet package path. However, for the distribution of the application, it is necessary to copy the model to the directory where the executable file resides and explicitly specify the path to the model.

```csharp
string? assemblyPath = System.IO.Path.GetDirectoryName(
        System.Reflection.Assembly.GetExecutingAssembly().Location
    );

string modelPath = assemblyPath == null ? "" : Path.Join(assemblyPath, "model");

mrzScanner.LoadModel(modelPath);
```

## Menu Items and Input Dialogs
When you click on the menu item, an input box will appear, allowing you to enter the license key or template string. This box can be created by customizing a form:

```csharp
public static string InputBox(string title, string promptText, string value)
{
    Form form = new Form();
    TextBox textBox = new TextBox();
    Button buttonOk = new Button();
    Button buttonCancel = new Button();

    form.Text = title;
    textBox.Text = value;

    buttonOk.Text = "OK";
    buttonCancel.Text = "Cancel";
    buttonOk.DialogResult = DialogResult.OK;
    buttonCancel.DialogResult = DialogResult.Cancel;

    textBox.SetBounds(12, 36, 372, 20);
    buttonOk.SetBounds(60, 72, 80, 30);
    buttonCancel.SetBounds(260, 72, 80, 30);

    form.ClientSize = new System.Drawing.Size(400, 120);
    form.Controls.AddRange(new Control[] { textBox, buttonOk, buttonCancel });
    form.FormBorderStyle = FormBorderStyle.FixedDialog;
    form.StartPosition = FormStartPosition.CenterScreen;
    form.MinimizeBox = false;
    form.MaximizeBox = false;
    form.AcceptButton = buttonOk;
    form.CancelButton = buttonCancel;

    DialogResult dialogResult = form.ShowDialog();
    return textBox.Text;
}
```

![Input box](https://www.dynamsoft.com/codepool/img/2023/07/input-box.png)

The click event handler of the menu item is as follows:

```csharp
enterLicenseKeyToolStripMenuItem.Click += enterLicenseKeyToolStripMenuItem_Click;
dDNToolStripMenuItem.Click += dDNToolStripMenuItem_Click;
dBRToolStripMenuItem.Click += dBRToolStripMenuItem_Click;

private void enterLicenseKeyToolStripMenuItem_Click(object sender, EventArgs e)
{
    string license = InputBox("Enter License Key", "", "");
    if (license != null && license != "")
    {
        ActivateLicense(license);
    }
}

private void dDNToolStripMenuItem_Click(object sender, EventArgs e)
{
    string template = InputBox("Set DDN Template", "", "");
    if (template != null && template != "")
    {
        documentScanner.SetParameters(template);
    }
}

private void dBRToolStripMenuItem_Click(object sender, EventArgs e)
{
    string template = InputBox("Set DBR Template", "", "");
    if (template != null && template != "")
    {
        barcodeScanner.SetParameters(template);
    }
}
```

## Load Image Files from the Local Disk
To load an image file from the local disk, you can use the `OpenFileDialog` class. The `ListBox` control can be used to store the history of loaded images.

```csharp
private void buttonFile_Click(object sender, EventArgs e)
{
    StopScan();
    using (OpenFileDialog dlg = new OpenFileDialog())
    {
        dlg.Title = "Open Image";
        dlg.Filter = "Image files (*.bmp, *.jpg, *.png) | *.bmp; *.jpg; *.png";

        if (dlg.ShowDialog() == DialogResult.OK)
        {
            listBox1.Items.Add(dlg.FileName);
        }
    }
}
```

## Show Camera Video Stream
The camera video stream is implemented using OpenCV's `VideoCapture` class. A worker thread is created, running an infinite loop, to constantly capture the video stream and display frames in the `PictureBox` control.

```csharp
private void StartScan()
{
    buttonCamera.Text = "Stop";
    isCapturing = true;
    thread = new Thread(new ThreadStart(FrameCallback));
    thread.Start();
}

private void StopScan()
{
    buttonCamera.Text = "Camera Scan";
    isCapturing = false;
    if (thread != null) thread.Join();
}

private void FrameCallback()
{
    while (isCapturing)
    {
        capture.Read(_mat);
        Mat copy = new Mat(_mat.Rows, _mat.Cols, MatType.CV_8UC3);
        _mat.CopyTo(copy);

        ...

        pictureBoxSrc.Image = BitmapConverter.ToBitmap(copy);
    }
    ...
}
```

## Detect and Rectify Documents
When a frame is captured as a `Mat` object, we initially create a duplicate of it. We use one frame buffer as an input for the `DetectBuffer()` method, while the other is employed to display the detection results using the `DrawContours()` method.

```csharp
private Mat DetectDocument(Mat mat, Mat canvas)
{
    int length = mat.Cols * mat.Rows * mat.ElemSize();
    byte[] bytes = new byte[length];
    Marshal.Copy(mat.Data, bytes, 0, length);

    _docResults = documentScanner.DetectBuffer(bytes, mat.Cols, mat.Rows, (int)mat.Step(), DocumentScanner.ImagePixelFormat.IPF_RGB_888);
    if (_docResults != null)
    {
        DocResult result = _docResults[0];
        if (result.Points != null)
        {
            Point[] points = new Point[4];
            for (int i = 0; i < 4; i++)
            {
                points[i] = new Point(result.Points[i * 2], result.Points[i * 2 + 1]);
            }
            Cv2.DrawContours(canvas, new Point[][] { points }, 0, Scalar.Blue, 2);
        }
    }
    return canvas;
}
```

Once the document is detected,  the `NormalizeBuffer()` method can be employed to crop and rectify the document. This method returns a `NormalizedImage` object. To display the `NormalizedImage` object in the `PictureBox` control, you need to convert it to a `Mat` object first. Then use `BitmapConverter` class to convert the `Mat` object to a `Bitmap` object:

```csharp
private void PreviewNormalizedImage()
{
    if (_docResults != null)
    {
        DocResult result = _docResults[0];
        int length = _mat.Cols * _mat.Rows * _mat.ElemSize();
        byte[] bytes = new byte[length];
        Marshal.Copy(_mat.Data, bytes, 0, length);

        NormalizedImage image = documentScanner.NormalizeBuffer(bytes, _mat.Cols, _mat.Rows, (int)_mat.Step(), DocumentScanner.ImagePixelFormat.IPF_RGB_888, result.Points);
        if (image != null && image.Data != null)
        {
            Mat newMat;
            if (image.Stride < image.Width)
            {
                // binary
                byte[] data = image.Binary2Grayscale();
                newMat = new Mat(image.Height, image.Width, MatType.CV_8UC1, data);
            }
            else if (image.Stride >= image.Width * 3)
            {
                // color
                newMat = new Mat(image.Height, image.Width, MatType.CV_8UC3, image.Data);
            }
            else
            {
                // grayscale
                newMat = new Mat(image.Height, image.Stride, MatType.CV_8UC1, image.Data);
            }

            Mat copy = new Mat(_mat.Rows, _mat.Cols, MatType.CV_8UC3);
            newMat.CopyTo(copy);

            pictureBoxDest.Image = BitmapConverter.ToBitmap(copy);
        }
    }
}
```

## Read Barcode and QR Code
Similar to the document detection process, we use the `DecodeBuffer()` method to detect barcodes and QR codes:

```csharp
private Mat DetectBarcode(Mat mat, Mat canvas)
{
    int length = mat.Cols * mat.Rows * mat.ElemSize();
    byte[] bytes = new byte[length];
    Marshal.Copy(mat.Data, bytes, 0, length);

    BarcodeResult[]? results = barcodeScanner.DecodeBuffer(bytes, mat.Cols, mat.Rows, (int)mat.Step(), BarcodeQRCodeReader.ImagePixelFormat.IPF_RGB_888);
    if (results != null)
    {
        foreach (BarcodeResult result in results)
        {
            string output = "Text: " + result.Text + Environment.NewLine + "Format: " + result.Format1 + Environment.NewLine;
            this.BeginInvoke((MethodInvoker)delegate {
                richTextBoxInfo.AppendText(output);
                richTextBoxInfo.AppendText(Environment.NewLine);
            });
                
            int[]? points = result.Points;
            if (points != null)
            {
                OpenCvSharp.Point[] all = new OpenCvSharp.Point[4];
                int xMin = points[0], yMax = points[1];
                all[0] = new OpenCvSharp.Point(xMin, yMax);
                for (int i = 2; i < 7; i += 2)
                {
                    int x = points[i];
                    int y = points[i + 1];
                    OpenCvSharp.Point p = new OpenCvSharp.Point(x, y);
                    xMin = x < xMin ? x : xMin;
                    yMax = y > yMax ? y : yMax;
                    all[i / 2] = p;
                }
                OpenCvSharp.Point[][] contours = new OpenCvSharp.Point[][] { all };
                Cv2.DrawContours(canvas, contours, 0, new Scalar(0, 255, 0), 2);
                if (result.Text != null) Cv2.PutText(canvas, result.Text, new OpenCvSharp.Point(xMin, yMax), HersheyFonts.HersheySimplex, 1, new Scalar(0, 0, 255), 2);
            }
        }
    }

    return canvas;
}
```

The `BeginInvoke()` method is used to update the UI from a non-UI thread.

## Enhance MRZ Detection with Document Rectification
MRZ detection relies heavily on the alignment of characters. It is better to make the MRZ area horizontal for more accurate detection. Although the MRZ SDK does not feature quadrilateral distortion correction, it can employ the document scanner SDK to achieve the preprocessing. By combining these two SDKs, MRZ detection from the document image can be significantly optimized.

```csharp
private Mat DetectMrz(Mat mat, Mat canvas)
{
    int length = mat.Cols * mat.Rows * mat.ElemSize();
    byte[] bytes = new byte[length];
    Marshal.Copy(mat.Data, bytes, 0, length);
    _mrzResults = mrzScanner.DetectBuffer(bytes, mat.Cols, mat.Rows, (int)mat.Step(), MrzScanner.ImagePixelFormat.IPF_RGB_888);
    if (_mrzResults != null)
    {
        string[] lines = new string[_mrzResults.Length];
        var index = 0;
        foreach (MrzResult result in _mrzResults)
        {
            lines[index++] = result.Text;
            this.BeginInvoke((MethodInvoker)delegate {
                richTextBoxInfo.Text += result.Text + Environment.NewLine;
            });
            
            if (result.Points != null)
            {
                Point[] points = new Point[4];
                for (int i = 0; i < 4; i++)
                {
                    points[i] = new Point(result.Points[i * 2], result.Points[i * 2 + 1]);
                }
                Cv2.DrawContours(canvas, new Point[][] { points }, 0, Scalar.Red, 2);
            }
        }

        JsonNode? info = Parse(lines);
        if (info != null)
        {
            this.BeginInvoke((MethodInvoker)delegate {
                richTextBoxInfo.Text = info.ToString();
            });
            
        }
    }

    return canvas;
}

private void PreviewNormalizedImage()
{
    if (_docResults != null)
    {
        ...
        if (checkBoxMrz.Checked)
        {
            copy = DetectMrz(newMat, copy);
        }
        pictureBoxDest.Image = BitmapConverter.ToBitmap(copy);
    }
}
```

**MRZ SDK only**

![only MRZ detection](https://www.dynamsoft.com/codepool/img/2023/07/only-mrz-detection.png)

**Combine document detection and MRZ detection**

![combine document detection and MRZ detection](https://www.dynamsoft.com/codepool/img/2023/07/document-rectification-mrz.png)

## Source Code
[https://github.com/yushulx/dotnet-barcode-qr-code-sdk/tree/main/example/barcode_document_mrz](https://github.com/yushulx/dotnet-barcode-qr-code-sdk/tree/main/example/barcode_document_mrz)