Parsing international passports with Mindee's API
Tags: passport, ocr, mindee, machinelearning
Published: 2021-02-25 21:48:44

---
title: Parsing international passports with Mindee's API
published: true
description: 
tags: passport, ocr, mindee, machinelearning
//cover_image: https://direct_url_to_image.jpg
canonical_url: https://mindee.com/documentation/apis/tutorial-parsing-passports-with-mindees-api
---

<h1>Parsing international passports with Mindee&#39;s API</h1>

<p>&nbsp;</p>

<p>Many onboarding processes in mobile or web apps require to extract some data from ID documents. In this tutorial, you will learn how to automatically extract data from passports to offer to your users the best onboarding experience.</p>

<p>&nbsp;</p>

<p>We will walk through the steps to use Mindee&rsquo;s Internation Passport&nbsp;Parsing API.&nbsp; Let&rsquo;s get started!&nbsp;</p>

<p>&nbsp;</p>

<h2>API Prerequisites</h2>

<ol>
	<li>You&rsquo;ll need a free Mindee account. <a href="https://platform.mindee.net/signup" target="_blank">Sign up</a>&nbsp;and confirm your email to login.</li>
	<li>A picture&nbsp;of a passport.&nbsp; You can use yours&nbsp;safely as data protection is one of our priority and we won&#39;t send your images to any third party application. You can also download a fake one <a href="https://mindee-public-website.s3.amazonaws.com/blog/2020/09/03/fake_passport.jpg">here</a>.</li>
</ol>

<p>&nbsp;</p>

<p>&nbsp;</p>

<p><img alt="" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2020/08/07/passport_pic.jpg" /></p>

<p>&nbsp;</p>

<p>&nbsp;</p>

<h2>Setting up the API</h2>

<p>&nbsp;</p>

<p>Log into your Mindee account and access your passport API environment by clicking the <strong>International</strong>&nbsp;<strong>Passport&nbsp;</strong>card:</p>

<p>&nbsp;</p>

<p>&nbsp;</p>

<p><img alt="passport api mindee" src="https://mindee-public-website-dev.s3.amazonaws.com/blog/2020/08/07/passport_article.jpg" /></p>

<p>&nbsp;</p>

<p>To activate the API, click the &ldquo;Try for Free&rdquo; button. This will give you access the API for free 50 times a month.&nbsp; There are four sections on the API landing page, noted in the left navigation. You are currently on the dashboard, and there&nbsp;are additional links to &ldquo;Documentation&rdquo;, &ldquo;Credentials&rdquo; and &ldquo;Live Interface&rdquo;.&nbsp; The docs tab has all of the technical details you&rsquo;ll need to build for the passports API endpoint, and the Live Interface is a cool interactive demo.</p>

<p>We&#39;ll start by clicking &quot;credentials&quot; and creating a new API token, in this case, named Tutorial:</p>

<p>&nbsp;</p>

<p><img src="https://mindee-public-website.s3.amazonaws.com/blog/2020/09/01/pasted-image-0-1.png" style="height:198px; width:400px" /></p>

<p>&nbsp;</p>

<p>Click &ldquo;Add New Key&rdquo; and you&rsquo;ll be able to see your API token,.</p>

<p>&nbsp;</p>

<p>If you move back to the documentation tab, you can pick your API token, and the language, and create an API call.&nbsp; Here is the API call in cURL:</p>

<pre>
<code class="language-bash">curl -X POST \ https://api.mindee.net/products/passport/v1/predict \ 
-H 'X-Inferuser-Token: {apiToken}’ \
-H 'content-type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW' \
-F file=@/path/to/your/file.png</code></pre>

<p>&nbsp;</p>

<p>Simply replace <em>{apiToken}</em>&nbsp;with your new API token and <em>/path/to/your/file/png</em> with the path to your receipt.&nbsp;</p>

<p><br />
That&#39;s it! We&#39;re all set to run this command in the terminal, and see the data that Mindee can extract.</p>

<p>In this tutirla, I&#39;ll use a sample [Cyprus passport](https://commons.wikimedia.org/wiki/File:Cyprus_passport_data_page.jpg) from Wikimedia.</p>

<p><img alt="Cyprus Passport used in this example" src="https://mindee-public-website.s3.amazonaws.com/blog/2020/09/04/1024px-cyprus_passport_data_page.jpg" style="height:334px; width:514px" /></p>

<h2>&nbsp;</h2>

<h2>Extracted fields</h2>

<p>&nbsp;</p>

<p>The JSON response can be a bit verbose, so we will break it into sections to describe what we are seeing.&nbsp; The initial sections describe the API and the API response:</p>

<pre>
<code class="language-json">"call": {
		"endpoint": {
			"name": "passport",
			"version": "1.0"
		},
		"finished_at": "2020-09-04T20:57:51+00:00",
		"id": "7d81f5fe-fce1-4b63-b3de-cc5948c39b55",
		"n_documents": 1,
		"n_inputs": 1,
		"processing_time": 1.988,
		"started_at": "2020-09-04T20:57:49+00:00"
	},
	"documents": [
		{
			"id": "57aded44-a216-45be-859c-f949cc5b011d",
			"name": "1024px-Cyprus_passport_data_page.jpg"
		}
	],</code></pre>

<p>Here, we can see that passport API was called, and it processed one document in about 2 seconds.&nbsp; What we are really interested in seeing is the results, and these can be found in the predictions section of the JSON file:</p>

<h2>&nbsp;</h2>

<h2>Predictions</h2>

<p>&nbsp;</p>

<p>The API response is in alphabetical order of items extracted from the document:</p>

<p>&nbsp;</p>

<h3><strong>Birth Date</strong></h3>

<pre>
<code class="language-json">"birth_date": {
				"probability": 1.0,
				"segmentation": {
					"bounding_box": [
						[.053,	0.912],
						[0.955,0.912],
						[0.955,0.969],
						[0.053,0.969]
					]
				},
				"value": "1970-01-01"
			},</code></pre>

<p>The API matched the brithdate in the MRX fields (more on that in a minute), so is 100% confident that the birthdate is 1 January, 1970. The bounding box gives four (x,y) points in the image where the date of birth can be found.</p>

<p>&nbsp;</p>

<h3><strong>Country of issuance</strong></h3>

<p>The API then attempts to parse the country that issued the passport.&nbsp;</p>

<pre>
<code class="language-json">"country": {
				"probability": 0.99,
				"segmentation": {
					"bounding_box": [
						[0.479,0.199],
						[0.522,0.199],
						[0.522,0.234],
						[0.479,0.234]
					]
				},
				"value": "CYP"
			},</code></pre>

<p>&nbsp;</p>

<p>The API is 99% certain that Cyprus is the issuing country. This is really incredible, as the API team tells me that there are no Cyprus passports in the data training set. Again, 4 (x,y) coordinates in the image point out where CYP is used.</p>

<h3>&nbsp;</h3>

<h3><strong>Expiration Date</strong></h3>

<pre>
<code class="language-json">"expiry_date": {
				"probability": 1.0,
				"segmentation": {
					"bounding_box": [
						[0.053,0.912],
						[0.955,0.912],
						[0.955,0.969],
						[0.053,0.969]
					]
				},
				"value": "2020-12-01"
			},</code></pre>

<p>Many countries do not allow entry with under six months of validity left on the passport.&nbsp; This woman should probably start the renewal process,&nbsp; as the API has identified that this passport will expire on 1 December, 2020.</p>

<h3>&nbsp;</h3>

<h3><strong>Gender</strong></h3>

<pre>
<code class="language-json">			"gender": {
				"probability": 0.1,
				"segmentation": {
					"bounding_box": [
						[0.053,0.912],
						[0.955,0.912],
						[0.955,0.969],
						[0.053,0.969]
					]
				},
				"value": "F"
			},</code></pre>

<p>The API&#39;s confidence is not high on this one, just 10%, but it does correctly identify the gender as female.</p>

<h3>&nbsp;</h3>

<h3><strong>Given Name</strong></h3>

<pre>
<code class="language-json">			"given_names": [
				{
					"probability": 0.99,
					"segmentation": {
						"bounding_box": [
							[0.048,0.844],
							[0.952,0.844],
							[0.952,0.895],
							[0.048,0.895]
						]
					},
					"value": "AFRODITI"
				}
			],</code></pre>

<p>The API is really certain (99% confidence) that the given name on the passport is Afroditi, meaning that our passport holder was named after the Greek goddess of beauty and love.</p>

<h3>&nbsp;</h3>

<h3><strong>ID Number</strong></h3>

<pre>
<code class="language-json">			"id_number": {
				"probability": 1.0,
				"segmentation": {
					"bounding_box": [
						[0.673,0.197],
						[0.774,0.197],
						[0.774,0.233],
						[0.673,0.233]
					]
				},
				"value": "K00000413"
			},</code></pre>

<p>While we are not numbers, in many ways, we are identified by numbers. In this case, the passport ID number, which must be used when booking flights, checking into hotels, etc.&nbsp; The API has found the location (marked by the 4 (x,y) points in the bounding box, for the Passport number, which is K00000413.</p>

<p>&nbsp;</p>

<h3><strong>Issuance Date</strong></h3>

<p>Contining the alphabetical trip through the passport fields, we next come to the date the passport was issued:</p>

<p>&nbsp;</p>

<pre>
<code class="language-json">			"issuance_date": {
				"probability": 0.18,
				"segmentation": {
					"bounding_box": [
						[0.353,0.586],
						[0.426,0.586],
						[0.426,0.62],
						[0.353,0.62]
					]
				},
				"value": "2014-01-21"
			},</code></pre>

<p>In this case, the API missed the issuance date.&nbsp; However, while our API does miss in it&#39;s predictions, in this case, the issue is due to the fact that the passport is a sample (we&#39;ll have a blog post on this soon).&nbsp; The issuance date is missing from the MRZ zone of the sample&nbsp;passport, so the API is unable to accurately measure the issuance date.</p>

<p>&nbsp;</p>

<h3><strong>Surname</strong></h3>

<pre>
<code class="language-json">"surname": {
				"probability": 0.98,
				"segmentation": {
					"bounding_box": [
						[0.33,0.272],
						[0.444,0.272],
						[0.444,0.309],
						[0.33,0.309]
					]
				},
				"value": "ANONYMOU"
			}</code></pre>

<p>Our fake passport holder has a surname of Anonymou, which is correctly identified by the API, along with bounding boxes.</p>

<p>&nbsp;</p>

<h3><strong>MRZ1 and MRZ2</strong></h3>

<p>The last two regions identified by the API (which I pulled *slightly* out of alphabetical order for readability) are the 2 Machine Readable Zone (MRZ) lines:</p>

<pre>
<code class="language-json">"mrz1": {
				"probability": 0.99,
				"segmentation": {
					"bounding_box": [
						[0.048,0.844],
						[0.952,0.844],
						[0.952,0.895],
						[0.048,0.895]
					]
				},
				"value": "P&lt;CYPANONYMOU&lt;&lt;AFRODITI&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;"
			},
"mrz2": {
				"probability": 0.1,
				"segmentation": {
					"bounding_box": [
						[0.053,0.912],
						[0.955,0.912],
						[0.955,0.969],
						[0.053,0.969]
					]
				},
				"value": "K000004134CYP7001017F2012010&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;4"
			},</code></pre>

<p>The MRZ zones (if you cannot tell from the JSON above) are the two lines of machine readable text at the bottom of every passport page.&nbsp; MRZ1 is line one, and MRZ2 is line 2.&nbsp; Most of the identifying information can be pulled from this area.&nbsp; The confidence interval for line one is high, as the fake passport has this area filled out correctly.&nbsp; However, the confidence level is low, as the algorithm expected more characters other than &quot;&lt;&quot; to supply the issuance date.</p>

<p>&nbsp;</p>

<h2>Conclusion</h2>

<p>&nbsp;</p>

<p>And there you have it! Mindee&#39;s passport API parsed a passport in under 2 seconds, matching nearly all of the sections of the passport quickly and accurately. Give it&nbsp;try with our free tier, and let us know in the chat how it works for you!</p>