Day 5— Azure Open AI Challenge: Document Intelligence
Tags: programming, ai, beginners, tutorial
Published: 2024-03-30 10:11:12



Designed for developers familiar with Azure and generative AI, the guide walks you through the process of harnessing the power of the prebuilt model for document intelligence.

## Introduction

The Microsoft team has recently launched an Open AI challenge, in which a developer can learn how to build Azure AI solutions and apps.

### Prerequisite

* Experience working with Azure and Azure portals.

* An understanding of generative AI.

* Experience in one high-level programming language like C# or Python

## Getting Started

The article demonstrates the use of a Prebuilt Model — Invoice in the document intelligence service.

### What is a Prebuilt Model — Invoice?

Use a predefined model to identify and collect standard elements from particular types of documents. In this instance, we employ the built-in invoice model to process and extract information from an invoice.

The guide will cover the following features

* The steps to configure the document intelligence service on the Azure Portal.

* A console application in C# to process a sample document.

**Reference:** [Sample document](https://github.com/ssukhpinder/AzureOpenAI/blob/main/samples/Azure.OpenAI.DocumentIntelligence/sample-document/sample-invoice.pdf)

### Step 1: Navigate to the [Azure Portal](https://portal.azure.com/)

Search for Azure Document Intelligence and fill out the following details

![](https://cdn-images-1.medium.com/max/2000/1*Dfg8L2moEkM53jkR_U8HHQ.png)

### Step 2: Choose Network

![](https://cdn-images-1.medium.com/max/2000/1*hAnPz_ygPVK_4jIj6b1XCg.png)

### Step 3: Assign Identity

Enable system-assigned identity to grant the resource access to other existing resources. For this demonstration, we don't require identity.

![](https://cdn-images-1.medium.com/max/2000/1*2k6l79yQpl949CdcIj-mdw.png)

### Step 4: Add Tags

For this exercise tag names are not required. But in a production environment, it should be added as it’s a best practice.

![](https://cdn-images-1.medium.com/max/2000/1*n0Z6Fs1kOW1hWsNlcUsVjg.png)

### Step 5: Review & Create

Post validation checks by Azure Cloud, and proceed with creating the resource. Make sure you review the details entered in the previous steps.

![](https://cdn-images-1.medium.com/max/2000/1*61d8McTTvvhT_22YWc4q_g.png)

## Let’s switch to code

Please find below the sample document highlighting the fields that will be processed as part of this exercise using C# code.

The expected output should be as follows along with the confidence score
```
    Vendor Name: 'CONTOSO LTD.', with confidence 0.93.
    Customer Name: 'MICROSOFT CORPORATION', with confidence 0.915.
    Invoice Total: '$110', with confidence 0.97.
```
![](https://cdn-images-1.medium.com/max/2000/1*-m6CLCrFr_aI2C23aYjiFA.png)

### Step 1: Create a console application

To test image generation, create a console application in Visual Studio or Visual Studio Code.
```csharp
    dotnet new console
```
### Step 2: Read the configuration

Read the configuration from appsettings.json file
```csharp
    IConfiguration config = new ConfigurationBuilder()
        .AddJsonFile("appsettings.json")
        .Build();
    string? endpoint = config["AzureEndpoint"];
    string? apiKey = config["AzureKey"];
    string? documentUrl = config["DocumentUrl"];
```
### Step 3: Create a client
```csharp
    var cred = new AzureKeyCredential(apiKey);
    var client = new DocumentAnalysisClient(new Uri(endpoint), cred);
```
### Step 4: Analyze the Invoice

Install the NuGet package using the following command 
```csharp
    dotnet add package Azure.AI.FormRecognizer
```
and add the following code
```csharp
    AnalyzeDocumentOperation operation = await client.AnalyzeDocumentFromUriAsync(WaitUntil.Completed, "prebuilt-invoice", fileUri);
    await operation.WaitForCompletionAsync();
    
    AnalyzeResult result = operation.Value;
```
### Step 5: Display Invoice Information

The final step is to display the invoice information to the user using the following code snippet.

To keep things straightforward, we’re not displaying every key-value pair returned by the service here. For a comprehensive list of all fields supported and their respective types, please refer to the [Invoice concept page](https://learn.microsoft.com/en-us/azure/ai-services/document-intelligence/concept-invoice?view=doc-intel-3.1.0#field-extraction).

```csharp
    foreach (AnalyzedDocument invoice in result.Documents)
    {
        if (invoice.Fields.TryGetValue("VendorName", out DocumentField? vendorNameField))
        {
            if (vendorNameField.FieldType == DocumentFieldType.String)
            {
                string vendorName = vendorNameField.Value.AsString();
                Console.WriteLine($"Vendor Name: '{vendorName}', with confidence {vendorNameField.Confidence}.");
            }
        }
    
    
    
        if (invoice.Fields.TryGetValue("CustomerName", out DocumentField? customerNameField))
        {
            if (customerNameField.FieldType == DocumentFieldType.String)
            {
                string customerName = customerNameField.Value.AsString();
                Console.WriteLine($"Customer Name: '{customerName}', with confidence {customerNameField.Confidence}.");
            }
        }
    
        if (invoice.Fields.TryGetValue("InvoiceTotal", out DocumentField? invoiceTotalField))
        {
            if (invoiceTotalField.FieldType == DocumentFieldType.Currency)
            {
                CurrencyValue invoiceTotal = invoiceTotalField.Value.AsCurrency();
                Console.WriteLine($"Invoice Total: '{invoiceTotal.Symbol}{invoiceTotal.Amount}', with confidence {invoiceTotalField.Confidence}.");
            }
        }
    }
    Console.WriteLine("\nAnalysis complete.\n");
```

### Step 6: Run the code

Make sure the document URL is added to the appsettings.json file.

![](https://cdn-images-1.medium.com/max/2206/1*xAf0Td2i7meAf4_gDNerbQ.png)

The URL for the example document is provided below.

[https://github.com/MicrosoftLearning/mslearn-ai-document-intelligence/blob/main/Labfiles/01-prebuild-models/sample-invoice/sample-invoice.pdf?raw=true](https://github.com/MicrosoftLearning/mslearn-ai-document-intelligence/blob/main/Labfiles/01-prebuild-models/sample-invoice/sample-invoice.pdf?raw=true)

### Complete Program.cs file
```csharp
    // See https://aka.ms/new-console-template for more information
    
    
    // Build a config object and retrieve user settings.
    using Azure;
    using Azure.AI.FormRecognizer.DocumentAnalysis;
    using Microsoft.Extensions.Configuration;
    
    IConfiguration config = new ConfigurationBuilder()
        .AddJsonFile("appsettings.json")
        .Build();
    string? endpoint = config["AzureEndpoint"];
    string? apiKey = config["AzureKey"];
    string? documentUrl = config["DocumentUrl"];
    
    Uri fileUri = new Uri(uriString: documentUrl);
    
    Console.WriteLine("\nConnecting to Forms Recognizer at: {0}", endpoint);
    Console.WriteLine("Analyzing invoice at: {0}\n", fileUri.ToString());
    
    // Create the client
    
    var cred = new AzureKeyCredential(apiKey);
    var client = new DocumentAnalysisClient(new Uri(endpoint), cred);
    
    // Analyze the invoice
    AnalyzeDocumentOperation operation = await client.AnalyzeDocumentFromUriAsync(WaitUntil.Completed, "prebuilt-invoice", fileUri);
    await operation.WaitForCompletionAsync();
    
    AnalyzeResult result = operation.Value;
    
    // Display invoice information to the user
    foreach (AnalyzedDocument invoice in result.Documents)
    {
        if (invoice.Fields.TryGetValue("VendorName", out DocumentField? vendorNameField))
        {
            if (vendorNameField.FieldType == DocumentFieldType.String)
            {
                string vendorName = vendorNameField.Value.AsString();
                Console.WriteLine($"Vendor Name: '{vendorName}', with confidence {vendorNameField.Confidence}.");
            }
        }
    
    
    
        if (invoice.Fields.TryGetValue("CustomerName", out DocumentField? customerNameField))
        {
            if (customerNameField.FieldType == DocumentFieldType.String)
            {
                string customerName = customerNameField.Value.AsString();
                Console.WriteLine($"Customer Name: '{customerName}', with confidence {customerNameField.Confidence}.");
            }
        }
    
        if (invoice.Fields.TryGetValue("InvoiceTotal", out DocumentField? invoiceTotalField))
        {
            if (invoiceTotalField.FieldType == DocumentFieldType.Currency)
            {
                CurrencyValue invoiceTotal = invoiceTotalField.Value.AsCurrency();
                Console.WriteLine($"Invoice Total: '{invoiceTotal.Symbol}{invoiceTotal.Amount}', with confidence {invoiceTotalField.Confidence}.");
            }
        }
    }
    Console.WriteLine("\nAnalysis complete.\n");
```
### Complete code on GitHub

Make sure to give it a star on GitHub and provide feedback on how to improve the tool further..!!

To access the document intelligence sample, move to the directory samples > Azure.OpenAI.DocumentIntelligence

[GitHub - ssukhpinder/AzureOpenAI: Sample for testing Azure Open AI GPT3 Turbo Model](https://github.com/ssukhpinder/AzureOpenAI)

### C# Programming🚀

Thank you for being a part of the C# community! Before you leave:

#### If you’ve made it this far, please show your appreciation with a clap and follow the author! 👏️️

Follow us: [X](https://twitter.com/sukhsukhpinder) | [LinkedIn](https://www.linkedin.com/in/sukhpinder-singh/) | [Dev.to](https://dev.to/ssukhpinder) | [Hashnode](https://dotnet.hashnode.dev/) | [Newsletter](https://www.linkedin.com/newsletters/net-programming-7031098498754195456/) | [Tumblr](https://www.tumblr.com/settings/blog/codewithsukh)

Visit our other platforms: [GitHub](https://github.com/ssukhpinder) | [Instagram](https://www.instagram.com/codewithsukh/) | [Tiktok](https://www.tiktok.com/@codewithsukh) | [Quora](https://www.quora.com/profile/Sukhpinder-Singh-4) | [Daily.dev](https://app.daily.dev/devcard)

More content at [C# Programming](https://medium.com/c-sharp-progarmming)