Package: azure-ai-documentintelligence
Azure AI Document Intelligence client library for PythonAzure AI Document Intelligence (previously known as Form Recognizer) is a cloud service that uses machine learning to analyze text and structured data from your documents. It includes the following main features:Layout - Extract content and structure (ex. words, selection marks, tables) from documents.Document - Analyze key-value pairs in addition to general layout from documents.Read - Read page information from documents.Prebuilt - Extract common field values from select document types (ex. receipts, invoices, business cards, ID documents, U.S. W-2 tax documents, among others) using prebuilt models.Custom - Build custom models from your own data to extract tailored field values in addition to general layout from documents.Classifiers - Build custom classification models that combine layout and language features to accurately detect and identify documents you process within your application.Add-on capabilities - Extract barcodes/QR codes, formulas, font/style, etc. or enable high resolution mode for large documents with optional parameters.Getting startedInstallating the packagepython-mpipinstallazure-ai-documentintelligencePrequisitesPython 3.7 or later is required to use this package.You need anAzure subscriptionto use this package.An existing Azure AI Document Intelligence instance.Create a Cognitive Services or Document Intelligence resourceDocument Intelligence supports bothmulti-service and single-service access. Create a Cognitive Services resource if you plan to access multiple cognitive services under a single endpoint/key. For Document Intelligence access only, create a Document Intelligence resource. Please note that you will need a single-service resource if you intend to useAzure Active Directory authentication.You can create either resource using:Option 1:Azure Portal.Option 2:Azure CLI.Below is an example of how you can create a Document Intelligence resource using the CLI:# Create a new resource group to hold the Document Intelligence resource# if using an existing resource group, skip this stepazgroupcreate--name<your-resource-name>--location<location># Create the Document Intelligence resourceazcognitiveservicesaccountcreate\--name<your-resource-name>\--resource-group<your-resource-group-name>\--kindFormRecognizer\--sku<sku>\--location<location>\--yesFor more information about creating the resource or how to get the location and sku information seehere.Authenticate the clientIn order to interact with the Document Intelligence service, you will need to create an instance of a client.
Anendpointandcredentialare necessary to instantiate the client object.Get the endpointYou can find the endpoint for your Document Intelligence resource using theAzure PortalorAzure CLI:# Get the endpoint for the Document Intelligence resourceazcognitiveservicesaccountshow--name"resource-name"--resource-group"resource-group-name"--query"properties.endpoint"Either a regional endpoint or a custom subdomain can be used for authentication. They are formatted as follows:Regional endpoint: https://<region>.api.cognitive.microsoft.com/
Custom subdomain: https://<resource-name>.cognitiveservices.azure.com/A regional endpoint is the same for every resource in a region. A complete list of supported regional endpoints can be consultedhere. Please note that regional endpoints do not support AAD authentication.A custom subdomain, on the other hand, is a name that is unique to the Document Intelligence resource. They can only be used bysingle-service resources.Get the API keyThe API key can be found in theAzure Portalor by running the following Azure CLI command:azcognitiveservicesaccountkeyslist--name"<resource-name>"--resource-group"<resource-group-name>"Create the client with AzureKeyCredentialTo use anAPI keyas thecredentialparameter,
pass the key as a string into an instance ofAzureKeyCredential.fromazure.core.credentialsimportAzureKeyCredentialfromazure.ai.documentintelligenceimportDocumentIntelligenceClientendpoint="https://<my-custom-subdomain>.cognitiveservices.azure.com/"credential=AzureKeyCredential("<api_key>")document_analysis_client=DocumentIntelligenceClient(endpoint,credential)Create the client with an Azure Active Directory credentialAzureKeyCredentialauthentication is used in the examples in this getting started guide, but you can also
authenticate with Azure Active Directory using theazure-identitylibrary.
Note that regional endpoints do not support AAD authentication. Create acustom subdomainname for your resource in order to use this type of authentication.To use theDefaultAzureCredentialtype shown below, or other credential types provided
with the Azure SDK, please install theazure-identitypackage:pip install azure-identityYou will also need toregister a new AAD application and grant accessto Document Intelligence by assigning the"Cognitive Services User"role to your service principal.Once completed, set the values of the client ID, tenant ID, and client secret of the AAD application as environment variables:AZURE_CLIENT_ID,AZURE_TENANT_ID,AZURE_CLIENT_SECRET."""DefaultAzureCredential will use the values from these environmentvariables: AZURE_CLIENT_ID, AZURE_TENANT_ID, AZURE_CLIENT_SECRET"""fromazure.ai.documentintelligenceimportDocumentIntelligenceClientfromazure.identityimportDefaultAzureCredentialendpoint=os.environ["DOCUMENTINTELLIGENCE_ENDPOINT"]credential=DefaultAzureCredential()document_analysis_client=DocumentIntelligenceClient(endpoint,credential)Key conceptsDocumentIntelligenceClientDocumentIntelligenceClientprovides operations for analyzing input documents using prebuilt and custom models through thebegin_analyze_documentAPI.
Use themodel_idparameter to select the type of model for analysis. See a full list of supported modelshere.
TheDocumentIntelligenceClientalso provides operations for classifying documents through thebegin_classify_documentAPI.
Custom classification models can classify each page in an input file to identify the document(s) within and can also identify multiple documents or multiple instances of a single document within an input file.Sample code snippets are provided to illustrate using a DocumentIntelligenceClienthere.
More information about analyzing documents, including supported features, locales, and document types can be found in theservice documentation.DocumentIntelligenceAdministrationClientDocumentIntelligenceAdministrationClientprovides operations for:Building custom models to analyze specific fields you specify by labeling your custom documents. ADocumentModelDetailsis returned indicating the document type(s) the model can analyze, as well as the estimated confidence for each field. See theservice documentationfor a more detailed explanation.Creating a composed model from a collection of existing models.Managing models created in your account.Listing operations or getting a specific model operation created within the last 24 hours.Copying a custom model from one Document Intelligence resource to another.Build and manage a custom classification model to classify the documents you process within your application.Please note that models can also be built using a graphical user interface such asDocument Intelligence Studio.Sample code snippets are provided to illustrate using a DocumentIntelligenceAdministrationClienthere.Long-running operationsLong-running operations are operations which consist of an initial request sent to the service to start an operation,
followed by polling the service at intervals to determine whether the operation has completed or failed, and if it has
succeeded, to get the result.Methods that analyze documents, build models, or copy/compose models are modeled as long-running operations.
The client exposes abegin_<method-name>method that returns anLROPollerorAsyncLROPoller. Callers should wait
for the operation to complete by callingresult()on the poller object returned from thebegin_<method-name>method.
Sample code snippets are provided to illustrate using long-running operationsbelow.ExamplesThe following section provides several code snippets covering some of the most common Document Intelligence tasks, including:Extract LayoutUsing the General Document ModelUsing Prebuilt ModelsBuild a Custom ModelAnalyze Documents Using a Custom ModelManage Your ModelsAdd-on capabilitiesExtract LayoutExtract text, selection marks, text styles, and table structures, along with their bounding region coordinates, from documents.fromazure.core.credentialsimportAzureKeyCredentialfromazure.ai.documentintelligenceimportDocumentIntelligenceClientendpoint=os.environ["DOCUMENTINTELLIGENCE_ENDPOINT"]key=os.environ["DOCUMENTINTELLIGENCE_API_KEY"]document_intelligence_client=DocumentIntelligenceClient(endpoint=endpoint,credential=AzureKeyCredential(key))withopen(path_to_sample_documents,"rb")asf:poller=document_intelligence_client.begin_analyze_document("prebuilt-layout",analyze_request=f,content_type="application/octet-stream")result=poller.result()foridx,styleinenumerate(result.styles):print("Document contains{}content".format("handwritten"ifstyle.is_handwrittenelse"no handwritten"))forpageinresult.pages:print("----Analyzing layout from page #{}----".format(page.page_number))print("Page has width:{}and height:{}, measured with unit:{}".format(page.width,page.height,page.unit))forline_idx,lineinenumerate(page.lines):words=line.get_words()print("...Line #{}has word count{}and text '{}' within bounding polygon '{}'".format(line_idx,len(words),line.content,line.polygon,))forwordinwords:print("......Word '{}' has a confidence of{}".format(word.content,word.confidence))forselection_markinpage.selection_marks:print("...Selection mark is '{}' within bounding polygon '{}' and has a confidence of{}".format(selection_mark.state,selection_mark.polygon,selection_mark.confidence,))fortable_idx,tableinenumerate(result.tables):print("Table #{}has{}rows and{}columns".format(table_idx,table.row_count,table.column_count))forregionintable.bounding_regions:print("Table #{}location on page:{}is{}".format(table_idx,region.page_number,region.polygon,))forcellintable.cells:print("...Cell[{}][{}] has content '{}'".format(cell.row_index,cell.column_index,cell.content,))forregionincell.bounding_regions:print("...content on page{}is within bounding polygon '{}'".format(region.page_number,region.polygon,))print("----------------------------------------")Using Prebuilt ModelsExtract fields from select document types such as receipts, invoices, business cards, identity documents, and U.S. W-2 tax documents using prebuilt models provided by the Document Intelligence service.For example, to analyze fields from a sales receipt, use the prebuilt receipt model provided by passingmodel_id="prebuilt-receipt"into thebegin_analyze_documentmethod:fromazure.core.credentialsimportAzureKeyCredentialfromazure.ai.documentintelligenceimportDocumentIntelligenceClientendpoint=os.environ["DOCUMENTINTELLIGENCE_ENDPOINT"]key=os.environ["DOCUMENTINTELLIGENCE_API_KEY"]document_analysis_client=DocumentIntelligenceClient(endpoint=endpoint,credential=AzureKeyCredential(key))withopen(path_to_sample_documents,"rb")asf:poller=document_analysis_client.begin_analyze_document("prebuilt-receipt",analyze_request=f,locale="en-US",content_type="application/octet-stream")receipts=poller.result()foridx,receiptinenumerate(receipts.documents):print(f"--------Analysis of receipt #{idx+1}--------")print(f"Receipt type:{receipt.doc_typeifreceipt.doc_typeelse'N/A'}")merchant_name=receipt.fields.get("MerchantName")ifmerchant_name:print(f"Merchant Name:{merchant_name.get('valueString')}has confidence: "f"{merchant_name.confidence}")transaction_date=receipt.fields.get("TransactionDate")iftransaction_date:print(f"Transaction Date:{transaction_date.get('valueDate')}has confidence: "f"{transaction_date.confidence}")ifreceipt.fields.get("Items"):print("Receipt items:")foridx,iteminenumerate(receipt.fields.get("Items").get("valueArray")):print(f"...Item #{idx+1}")item_description=item.get("valueObject").get("Description")ifitem_description:print(f"......Item Description:{item_description.get('valueString')}has confidence: "f"{item_description.confidence}")item_quantity=item.get("valueObject").get("Quantity")ifitem_quantity:print(f"......Item Quantity:{item_quantity.get('valueString')}has confidence: "f"{item_quantity.confidence}")item_total_price=item.get("valueObject").get("TotalPrice")ifitem_total_price:print(f"......Total Item Price:{format_price(item_total_price.get('valueCurrency'))}has confidence: "f"{item_total_price.confidence}")subtotal=receipt.fields.get("Subtotal")ifsubtotal:print(f"Subtotal:{format_price(subtotal.get('valueCurrency'))}has confidence:{subtotal.confidence}")tax=receipt.fields.get("TotalTax")iftax:print(f"Total tax:{format_price(tax.get('valueCurrency'))}has confidence:{tax.confidence}")tip=receipt.fields.get("Tip")iftip:print(f"Tip:{format_price(tip.get('valueCurrency'))}has confidence:{tip.confidence}")total=receipt.fields.get("Total")iftotal:print(f"Total:{format_price(total.get('valueCurrency'))}has confidence:{total.confidence}")print("--------------------------------------")You are not limited to receipts! There are a few prebuilt models to choose from, each of which has its own set of supported fields. See other supported prebuilt modelshere.Build a Custom ModelBuild a custom model on your own document type. The resulting model can be used to analyze values from the types of documents it was trained on.
Provide a container SAS URL to your Azure Storage Blob container where you're storing the training documents.More details on setting up a container and required file structure can be found in theservice documentation.fromazure.ai.formrecognizerimport(DocumentIntelligenceAdministrationClient,ModelBuildMode,)fromazure.core.credentialsimportAzureKeyCredentialendpoint=os.environ["DOCUMENTINTELLIGENCE_ENDPOINT"]key=os.environ["DOCUMENTINTELLIGENCE_API_KEY"]container_sas_url=os.environ["CONTAINER_SAS_URL"]document_model_admin_client=DocumentIntelligenceAdministrationClient(endpoint,AzureKeyCredential(key))poller=document_model_admin_client.begin_build_document_model(ModelBuildMode.TEMPLATE,blob_container_url=container_sas_url,description="my model description",)model=poller.result()print(f"Model ID:{model.model_id}")print(f"Description:{model.description}")print(f"Model created on:{model.created_on}")print(f"Model expires on:{model.expires_on}")print("Doc types the model can recognize:")forname,doc_typeinmodel.doc_types.items():print(f"Doc Type: '{name}' built with '{doc_type.build_mode}' mode which has the following fields:")forfield_name,fieldindoc_type.field_schema.items():print(f"Field: '{field_name}' has type '{field['type']}' and confidence score "f"{doc_type.field_confidence[field_name]}")Analyze Documents Using a Custom ModelAnalyze document fields, tables, selection marks, and more. These models are trained with your own data, so they're tailored to your documents.
For best results, you should only analyze documents of the same document type that the custom model was built with.fromazure.core.credentialsimportAzureKeyCredentialfromazure.ai.documentintelligenceimportDocumentIntelligenceClientendpoint=os.environ["DOCUMENTINTELLIGENCE_ENDPOINT"]key=os.environ["DOCUMENTINTELLIGENCE_API_KEY"]model_id=os.getenv("CUSTOM_BUILT_MODEL_ID",custom_model_id)document_analysis_client=DocumentIntelligenceClient(endpoint=endpoint,credential=AzureKeyCredential(key))# Make sure your document's type is included in the list of document types the custom model can analyzewithopen(path_to_sample_documents,"rb")asf:poller=document_analysis_client.begin_analyze_document(model_id=model_id,analyze_request=f,content_type="application/octet-stream")result=poller.result()foridx,documentinenumerate(result.documents):print(f"--------Analyzing document #{idx+1}--------")print(f"Document has type{document.doc_type}")print(f"Document has document type confidence{document.confidence}")print(f"Document was analyzed with model with ID{result.model_id}")forname,fieldindocument.fields.items():field_value=field.get("valueString")iffield.get("valueString")elsefield.contentprint(f"......found field of type '{field.type}' with value '{field_value}' and with confidence{field.confidence}")# iterate over tables, lines, and selection marks on each pageforpageinresult.pages:print(f"\nLines found on page{page.page_number}")forlineinpage.lines:print(f"...Line '{line.content}'")forwordinpage.words:print(f"...Word '{word.content}' has a confidence of{word.confidence}")ifpage.selection_marks:print(f"\nSelection marks found on page{page.page_number}")forselection_markinpage.selection_marks:print(f"...Selection mark is '{selection_mark.state}' and has a confidence of{selection_mark.confidence}")fori,tableinenumerate(result.tables):print(f"\nTable{i+1}can be found on page:")forregionintable.bounding_regions:print(f"...{region.page_number}")forcellintable.cells:print(f"...Cell[{cell.row_index}][{cell.column_index}] has text '{cell.content}'")print("-----------------------------------")Additionally, a document URL can also be used to analyze documents using thebegin_analyze_documentmethod.fromazure.core.credentialsimportAzureKeyCredentialfromazure.ai.documentintelligenceimportDocumentIntelligenceClientfromazure.ai.documentintelligence.modelsimportAnalyzeDocumentRequestendpoint=os.environ["DOCUMENTINTELLIGENCE_ENDPOINT"]key=os.environ["DOCUMENTINTELLIGENCE_API_KEY"]document_analysis_client=DocumentIntelligenceClient(endpoint=endpoint,credential=AzureKeyCredential(key))url="https://raw.githubusercontent.com/Azure/azure-sdk-for-python/main/sdk/documentintelligence/azure-ai-documentintelligence/tests/sample_forms/receipt/contoso-receipt.png"poller=document_analysis_client.begin_analyze_document("prebuilt-receipt",AnalyzeDocumentRequest(url_source=url))receipts=poller.result()Manage Your ModelsManage the custom models attached to your account.fromazure.ai.documentintelligenceimportDocumentIntelligenceAdministrationClientfromazure.core.credentialsimportAzureKeyCredentialfromazure.core.exceptionsimportResourceNotFoundErrorendpoint="https://<my-custom-subdomain>.cognitiveservices.azure.com/"credential=AzureKeyCredential("<api_key>")document_model_admin_client=DocumentIntelligenceAdministrationClient(endpoint,credential)account_details=document_model_admin_client.get_resource_info()print("Our account has{}custom models, and we can have at most{}custom models".format(account_details.custom_document_models.count,account_details.custom_document_models.limit))# Here we get a paged list of all of our modelsmodels=document_model_admin_client.list_models()print("We have models with the following ids:{}".format(", ".join([m.model_idforminmodels])))# Replace with the custom model ID from the "Build a model" samplemodel_id="<model_id from the Build a Model sample>"custom_model=document_model_admin_client.get_model(model_id=model_id)print("Model ID:{}".format(custom_model.model_id))print("Description:{}".format(custom_model.description))print("Model created on:{}\n".format(custom_model.created_on))# Finally, we will delete this model by IDdocument_model_admin_client.delete_model(model_id=custom_model.model_id)try:document_model_admin_client.get_model(model_id=custom_model.model_id)exceptResourceNotFoundError:print("Successfully deleted model with id{}".format(custom_model.model_id))Add-on CapabilitiesDocument Intelligence supports more sophisticated analysis capabilities. These optional features can be enabled and disabled depending on the scenario of the document extraction.The following add-on capabilities are available for 2023-07-31 (GA) and later releases:barcode/QR codeformulafont/stylehigh resolution modelanguageNote that some add-on capabilities will incur additional charges. See pricing:https://azure.microsoft.com/pricing/details/ai-document-intelligence/.TroubleshootingGeneralDocument Intelligence client library will raise exceptions defined inAzure Core.
Error codes and messages raised by the Document Intelligence service can be found in theservice documentation.LoggingThis library uses the standardlogginglibrary for logging.Basic information about HTTP sessions (URLs, headers, etc.) is logged atINFOlevel.DetailedDEBUGlevel logging, including request/response bodies andunredactedheaders, can be enabled on the client or per-operation with thelogging_enablekeyword argument.See full SDK logging documentation with exampleshere.Optional ConfigurationOptional keyword arguments can be passed in at the client and per-operation level.
The azure-corereference documentationdescribes available configurations for retries, logging, transport protocols, and more.Next stepsMore sample codeSee theSample READMEfor several code snippets illustrating common patterns used in the Document Intelligence Python API.Additional documentationFor more extensive documentation on Azure AI Document Intelligence, see theDocument Intelligence documentationon docs.microsoft.com.ContributingThis project welcomes contributions and suggestions. Most contributions require
you to agree to a Contributor License Agreement (CLA) declaring that you have
the right to, and actually do, grant us the rights to use your contribution.
For details, visithttps://cla.microsoft.com.When you submit a pull request, a CLA-bot will automatically determine whether
you need to provide a CLA and decorate the PR appropriately (e.g., label,
comment). Simply follow the instructions provided by the bot. You will only
need to do this once across all repos using our CLA.This project has adopted theMicrosoft Open Source Code of Conduct. For more information,
see the Code of Conduct FAQ or contactopencode@microsoft.comwith any
additional questions or comments.