Title: Text Detection and OCR with Amazon Rekognition API - PyImageSearch

URL Source: https://pyimagesearch.com/2022/03/21/text-detection-and-ocr-with-amazon-rekognition-api/

Published Time: 2022-03-21T13:00:00+00:00

Markdown Content:
* * *

**Table of Contents**
---------------------

*   **[Text Detection and OCR with Amazon Rekognition API](#blog_title)**

*   [Configuring Your Development Environment](#Configuring)
*   [Amazon Rekognition API for OCR](#Amazon_Rekognition)
*   [Obtaining Your AWS Rekognition Keys](#Obtaining)
*   [Installing Amazon’s Python Package](#Installing)
*   [Having Problems Configuring Your Development Environment?](#Problems)
*   [Project Structure](#Project)
*   [Creating Our Configuration File](#Creating)
*   [Implementing the Amazon Rekognition OCR Script](#Implementing)
*   [Amazon Rekognition OCR Results](#Results)

*   **[Summary](#Summary)**

*   [Citation Information](#Citation_Info)

* * *

In this tutorial, you will:

*   Learn about the Amazon Rekognition API
*   Discover how the Amazon Rekognition API can be used for OCR
*   Obtain your Amazon Web Services (AWS) Rekognition Keys
*   Install Amazon’s `boto3` package to interface with the OCR API
*   Implement a Python script that interfaces with Amazon Rekognition API to OCR an image

![Image 1](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%201024%20575'%3E%3C/svg%3E)

This lesson is part 1 in a 3-part series on ****Text Detection and OCR****:

1.  **_Text Detection and OCR with Amazon Rekognition API_ (today’s tutorial)**
2.  _Text Detection and OCR with Microsoft Cognitive Services_
3.  _Text Detection and OCR with Google Cloud Vision API_

**To learn about text detection and OCR,** **_just keep reading._**

![Image 2](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20128%20128'%3E%3C/svg%3E)

#### Looking for the source code to this post?

[Jump Right To The Downloads Section](#download-the-code)

* * *

**[Text Detection and OCR with Amazon Rekognition API](#TOC)**
--------------------------------------------------------------

So far, we’ve primarily focused on using the Tesseract OCR engine. However, other optical character recognition (OCR) engines are available, some of which are _far more accurate_ than Tesseract and capable of accurately OCR’ing text, even in complex, unconstrained conditions.

Typically, these OCR engines live in the cloud. Many are proprietary-based. To keep these models and associated datasets proprietary, the companies do not distribute the models themselves and instead put them behind a REST API.

While these models do tend to be more accurate than Tesseract, there are some downsides, including:

*   An internet connection is required to OCR images — that’s less of an issue for most laptops/desktops, but if you’re working on the edge, an internet connection may not be possible
*   Additionally, if you are working with edge devices, then you may not want to spend the power draw on a network connection
*   There will be latency introduced by the network connection
*   OCR results will take longer because the image needs to be packaged into an API request and uploaded to the OCR API. The API will need to chew on the image and OCR it, and then finally return the results to the client
*   Due to the latency and amount of time it will take to OCR each image, it’s doubtful that these OCR APIs will be able to run in real-time
*   They cost money (but typically offer a free trial or are free up to a number of monthly API requests)

Looking at the previous list, you may wonder why on earth would we cover these APIs at all — what is the benefit?

As you’ll see, the primary benefit here is _accuracy._ So first, consider the amount of data that Google and Microsoft have from running their respective search engines. Then consider the amount of data Amazon generates _daily_ from simply printing shipping labels.

**These companies have an** **_incredible_** **amount of image data — and when they train their novel, state-of-the-art OCR models on their data, the result is an** **_incredibly_** **robust and accurate OCR model.**

In this tutorial, you’ll learn how to use the Amazon Rekognition API to OCR images. In upcoming tutorials, we will cover Microsoft Azure Cognitive Services and Google Cloud Vision API.

* * *

### **[Configuring Your Development Environment](#TOC)**

To follow this guide, you need to have the OpenCV library installed on your system.

Luckily, OpenCV is pip-installable:

$ pip install boto3

**If you need help configuring your development environment for OpenCV, we _highly recommend_ that you read our** [**_pip install OpenCV_ guide**](https://www.pyimagesearch.com/2018/09/19/pip-install-opencv/) — it will have you up and running in a matter of minutes.

* * *

### **[Amazon Rekognition API for OCR](#TOC)**

The first part of this tutorial will focus on obtaining your AWS Rekognition Keys. These keys will include a public access key and a secret key, similar to SSH, SFTP, etc.

We’ll then show you how to install `boto3`, the Amazon Web Services (AWS) software development kit (SDK) for Python. Finally, we’ll use the `boto3` package to interface with Amazon Rekognition OCR API.

Next, we’ll implement our Python configuration file (which will store our access key, secret key, and AWS region) and then create our driver script used to:

1.  Load an input image from disk
2.  Package it into an API request
3.  Send the API request to AWS Rekognition for OCR
4.  Retrieve the results from the API call
5.  Display our OCR results

We’ll wrap up this tutorial with a discussion of our results.

* * *

### **[Obtaining Your AWS Rekognition Keys](#TOC)**

You will need additional information from our companion site for instructions on obtaining your AWS Rekognition keys. You can find the instructions [here](http://pyimg.co/vxd51).

* * *

### **[Installing Amazon’s Python Package](#TOC)**

To interface with the Amazon Rekognition API, we need to use the `boto3` package: the AWS SDK. Luckily, `boto3` is incredibly simple to install, requiring only a single `pip`\-install command:

$ pip install boto3

If you are using a Python virtual environment or an Anaconda environment, be sure to use the appropriate command to access your Python environment _before_ running the above command (otherwise, `boto3` will be installed in the system install of Python).

* * *

### **[Having Problems Configuring Your Development Environment?](#TOC)**

[![Image 3](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20500%20334'%3E%3C/svg%3E)](https://www.pyimagesearch.com/pyimagesearch-university/)

**Figure 1:** Having trouble configuring your dev environment? Want access to pre-configured Jupyter Notebooks running on Google Colab? Be sure to join [PyImageSearch University](https://www.pyimagesearch.com/pyimagesearch-university/) — you’ll be up and running with this tutorial in a matter of minutes.

All that said, are you:

*   Short on time?
*   Learning on your employer’s administratively locked system?
*   Wanting to skip the hassle of fighting with the command line, package managers, and virtual environments?
*   **Ready to run the code** **_right now_** **on your Windows, macOS, or Linux system?**

Then join [PyImageSearch University](https://www.pyimagesearch.com/pyimagesearch-university/) today!

**Gain access to Jupyter Notebooks for this tutorial and other PyImageSearch guides that are** **_pre-configured_** **to run on Google Colab’s ecosystem right in your web browser!** No installation required.

And best of all, these Jupyter Notebooks will run on Windows, macOS, and Linux!

* * *

### **[Project Structure](#TOC)**

Start by accessing the **_“Downloads”_** section of this tutorial to retrieve the source code and example images.

Before we can perform text detection and OCR with Amazon Rekognition API, we first need to review our project directory structure.

|-- config
|   |-- \_\_init\_\_.py
|   |-- aws\_config.py
|-- images
|   |-- aircraft.png
|   |-- challenging.png
|   |-- park.png
|   |-- street\_signs.png
|-- amazon\_ocr.py

The `aws_config.py` file contains our AWS access key, secret key, and region. You learned how to obtain these values in the **[Obtaining Your AWS Rekognition Keys](#AWS_Keys)** section.

Our `amazon_ocr.py` script will take this `aws_config`, connect to the Amazon Rekognition API, and then perform text detection and OCR to each image in our `images` directory.

* * *

### **[Creating Our Configuration File](#TOC)**

To connect to the Amazon Rekognition API, we first need to supply our access key, secret key, and region. If you haven’t yet obtained these keys, go to the **[Obtaining Your AWS Rekognition Keys](#AWS_Keys)** section and be sure to follow the steps and note the values.

Afterward, you can come back here, open `aws_config.py`, and update the code:

\# define our AWS Access Key, Secret Key, and Region
ACCESS\_KEY = "YOUR\_ACCESS\_KEY"
SECRET\_KEY = "YOUR\_SECRET\_KEY"
REGION = "YOUR\_AWS\_REGION"

Sharing my API keys would be a security breach, so I’ve left placeholder values here. **Be sure to update them with your API keys; otherwise, you will be unable to connect to the Amazon Rekognition API.**

* * *

### **[Implementing the Amazon Rekognition OCR Script](#TOC)**

With our `aws_config` implemented, let’s move on to the `amazon_ocr.py` script, which is responsible for:

1.  Connecting to the Amazon Rekognition API
2.  Loading an input image from disk
3.  Packaging the image in an API request
4.  Sending the package to Amazon Rekognition API for OCR
5.  Obtaining the OCR results from Amazon Rekognition API
6.  Displaying our output text detection and OCR results

Let’s get started with our implementation:

\# import the necessary packages
from config import aws\_config as config
import argparse
import boto3
import cv2

**Lines 2-5** import our required Python packages. Notably, we need our `aws_config`, along with `boto3`, which is Amazon’s Python package, to interface with their API.

Let’s now define `draw_ocr_results`, a simple Python utility used to draw the output OCR results from Amazon Rekognition API:

def draw\_ocr\_results(image, text, poly, color=(0, 255, 0)):
   # unpack the bounding box, taking care to scale the coordinates
   # relative to the input image size
   (h, w) = image.shape\[:2\]
   tlX = int(poly\[0\]\["X"\] \* w)
   tlY = int(poly\[0\]\["Y"\] \* h)
   trX = int(poly\[1\]\["X"\] \* w)
   trY = int(poly\[1\]\["Y"\] \* h)
   brX = int(poly\[2\]\["X"\] \* w)
   brY = int(poly\[2\]\["Y"\] \* h)
   blX = int(poly\[3\]\["X"\] \* w)
   blY = int(poly\[3\]\["Y"\] \* h)

The `draw_ocr_results` function accepts four parameters:

1.  `image`: The input image that we’re drawing the OCR’d text on
2.  `text`: The OCR’d text itself
3.  `poly`: The polygon object/coordinates of the text bounding box returned by Amazon Rekognition API
4.  `color`: The color of the bounding box

**Line 10** grabs the width and height of the `image` we’re drawing on. **Lines 11-18** then grab the bounding box coordinates of the text ROI, taking care to scale the coordinates by the width and height.

Why do we perform this scaling process?

Well, as we’ll find out later in this tutorial, Amazon Rekognition API returns bounding boxes in the range `[0, 1]`_._ Multiplying the bounding boxes by the original image width and height brings the bounding boxes back to the original image scale.

From there, we can now annotate the `image`:

   # build a list of points and use it to construct each vertex
   # of the bounding box
   pts = ((tlX, tlY), (trX, trY), (brX, brY), (blX, blY))
   topLeft = pts\[0\]
   topRight = pts\[1\]
   bottomRight = pts\[2\]
   bottomLeft = pts\[3\]

   # draw the bounding box of the detected text
   cv2.line(image, topLeft, topRight, color, 2)
   cv2.line(image, topRight, bottomRight, color, 2)
   cv2.line(image, bottomRight, bottomLeft, color, 2)
   cv2.line(image, bottomLeft, topLeft, color, 2)

   # draw the text itself
   cv2.putText(image, text, (topLeft\[0\], topLeft\[1\] - 10),
      cv2.FONT\_HERSHEY\_SIMPLEX, 0.8, color, 2)

   # return the output image
   return image

**Lines 22-26** build a list of points corresponding to each vertex of the bounding box. Given the vertices, **Lines 29-32** draw the bounding box of the rotated text. **Lines 35 and 36** draw the OCR’d text itself.

We then return the annotated output `image` to the calling function.

With our helper utility defined, let’s move on to command line arguments:

\# construct the argument parser and parse the arguments
ap = argparse.ArgumentParser()
ap.add\_argument("-i", "--image", required=True,
   help="path to input image that we'll submit to AWS Rekognition")
ap.add\_argument("-t", "--type", type=str, default="line",
   choices=\["line", "word"\],
   help="output text type (either 'line' or 'word')")
args = vars(ap.parse\_args())

The `--image` command line argument corresponds to the path to the input image we want to submit to the Amazon Rekognition OCR API.

The `--type` argument can be either `line` or `word`, indicating whether or not we want the Amazon Rekognition API to return OCR’d results grouped into _lines_ or as individual _words._

Next, let’s connect to Amazon Web Services:

\# connect to AWS so we can use the Amazon Rekognition OCR API
client = boto3.client(
   "rekognition",
   aws\_access\_key\_id=config.ACCESS\_KEY,
   aws\_secret\_access\_key=config.SECRET\_KEY,
   region\_name=config.REGION)

# load the input image as a raw binary file and make a request to
# the Amazon Rekognition OCR API
print("\[INFO\] making request to AWS Rekognition API...")
image = open(args\["image"\], "rb").read()
response = client.detect\_text(Image={"Bytes": image})

# grab the text detection results from the API and load the input
# image again, this time in OpenCV format
detections = response\["TextDetections"\]
image = cv2.imread(args\["image"\])

# make a copy of the input image for final output
final = image.copy()

**Lines 51-55** connect to AWS. Here we supply our access key, secret key, and region.

Once connected, we load our input image from disk as a binary object (**Line 60**) and then submit it to AWS by calling the `detect_text` function and supplying our `image`.

Calling `detect_text` results in a `response` from the Amazon Rekognition API. We then grab the `TextDetections` results (**Line 65**).

**Line 66** loads the input `--image` from disk in OpenCV format, while **Line 69** clones the image to draw on it.

We can now loop over the text detection bounding boxes from the Amazon Rekognition API:

\# loop over the text detection bounding boxes
for detection in detections:
   # extract the OCR'd text, text type, and bounding box coordinates
   text = detection\["DetectedText"\]
   textType = detection\["Type"\]
   poly = detection\["Geometry"\]\["Polygon"\]

   # only draw show the output of the OCR process if we are looking
   # at the correct text type
   if args\["type"\] == textType.lower():
      # draw the output OCR line-by-line
      output = image.copy()
      output = draw\_ocr\_results(output, text, poly)
      final = draw\_ocr\_results(final, text, poly)

      # show the output OCR'd line
      print(text)
      cv2.imshow("Output", output)
      cv2.waitKey(0)

# show the final output image
cv2.imshow("Final Output", final)
cv2.waitKey(0)

**Line 72** loops over all `detections` returned by Amazon’s OCR API. We then extract the OCR’d `text`, `textType` (either _“word”_ or _“line”_), along with the bounding box coordinates of the OCR’d text (**Lines 74 and 75**).

**Line 80** makes a check to verify whether we are looking at either `word` or `line` OCR’d text. If the current `textType` matches our `--type` command line argument, we call our `draw_ocr_results` function on both the `output` image and our `final` cloned image (**Lines 82-84**).

**Lines 87-89** display the current OCR’d line or word on our terminal and screen. That way, we can easily visualize each line or word without the output image becoming too cluttered.

Finally, **Lines 92 and 93** show the result of drawing _all_ text on our screen at once (for visualization purposes).

* * *

### **[Amazon Rekognition OCR Results](#TOC)**

Congrats on implementing a Python script to interface with Amazon Rekognition’s OCR API!

Let’s see our results in action, first by OCR’ing the entire image, line-by-line:

$ python amazon\_ocr.py --image images/aircraft.png
\[INFO\] making request to AWS Rekognition API...
WARNING!
LOW FLYING AND DEPARTING AIRCRAFT
BLAST CAN CAUSE PHYSICAL INJURY

**Figure 2** shows that we have OCR’d our input `aircraft.png` image line-by-line successfully, thereby demonstrating that the Amazon Rekognition API was able to:

1.  Locate each block of text in the input image
2.  OCR each text ROI
3.  Group the blocks of text into lines

![Image 4](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20559%20500'%3E%3C/svg%3E)

**Figure 2:** OCR’ing images line-by-line using the Amazon Rekognition API.

But what if we wanted to obtain our OCR results at the _word_ level instead of the _line_ level?

That’s as simple as supplying the `--type` command line argument:

$ python amazon\_ocr.py --image images/aircraft.png --type word
\[INFO\] making request to AWS Rekognition API...

WARNING!
LOW
FLYING
AND
DEPARTING
AIRCRAFT
BLAST
CAN
CAUSE
PHYSICAL
INJURY

As our output and **Figure 3** show, we’re now OCR’ing text at the _word_ level.

![Image 5](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20700%20469'%3E%3C/svg%3E)

**Figure 3:** OCR’ing an image word-by-word using the Amazon Rekognition OCR API.

I’m a big fan of Amazon Rekognition’s OCR API. While AWS, EC2, etc., do have a bit of a learning curve, the benefit is that once you know it, you then understand Amazon’s entire web services ecosystem, which makes it _far easier_ to start integrating different services.

I also appreciate that it’s super easy to get started with Amazon Rekognition API. Other services (e.g., Google Cloud Vision API) make it a bit harder to get that “first easy win.” If this is your first foray into using cloud service APIs, definitely consider using Amazon Rekognition API first before moving on to Microsoft Cognitive Services or the Google Cloud Vision API.

* * *

### What's next? We recommend [PyImageSearch University](https://pyimagesearch.com/pyimagesearch-university/?utm_source=blogPost&utm_medium=bottomBanner&utm_campaign=What%27s%20next%3F%20I%20recommend).

**Course information:**  
84 total classes • 114+ hours of on-demand code walkthrough videos • Last updated: February 2024  
★★★★★ 4.84 (128 Ratings) • 16,000+ Students Enrolled

**I strongly believe that if you had the right teacher you could _master_ computer vision and deep learning.**

Do you think learning computer vision and deep learning has to be time-consuming, overwhelming, and complicated? Or has to involve complex mathematics and equations? Or requires a degree in computer science?

That’s _not_ the case.

All you need to master computer vision and deep learning is for someone to explain things to you in _simple, intuitive_ terms. _And that’s exactly what I do_. My mission is to change education and how complex Artificial Intelligence topics are taught.

If you're serious about learning computer vision, your next stop should be PyImageSearch University, the most comprehensive computer vision, deep learning, and OpenCV course online today. Here you’ll learn how to _successfully_ and _confidently_ apply computer vision to your work, research, and projects. Join me in computer vision mastery.

**Inside PyImageSearch University you'll find:**

*   ✓ **84 courses** on essential computer vision, deep learning, and OpenCV topics
*   ✓ **84 Certificates** of Completion
*   ✓ **114+ hours** of on-demand video
*   ✓ **Brand new courses released _regularly_**, ensuring you can keep up with state-of-the-art techniques
*   ✓ **Pre-configured Jupyter Notebooks in Google Colab**
*   ✓ Run all code examples in your web browser — works on Windows, macOS, and Linux (no dev environment configuration required!)
*   ✓ Access to **centralized code repos for _all_ 536+ tutorials** on PyImageSearch
*   ✓ **Easy one-click downloads** for code, datasets, pre-trained models, etc.
*   ✓ **Access** on mobile, laptop, desktop, etc.

[Click here to join PyImageSearch University](https://pyimagesearch.com/pyimagesearch-university/?utm_source=blogPost&utm_medium=bottomBanner&utm_campaign=What%27s%20next%3F%20I%20recommend)

* * *

**[Summary](#TOC)**
-------------------

In this tutorial, you learned how to create your Amazon Rekognition keys, install `boto3`, a Python package used to interface with AWS, and implement a Python script to make calls to the Amazon Rekognition API.

The Python script was simple, requiring less than 100 lines to implement (including comments).

Not only were our Amazon Rekognition OCR API results correct, but we could also parse the results at both the _line_ and _word_ level, giving us finer granularity than what the EAST text detection model and Tesseract OCR engine give us (at least without fine-tuning several options).

In the next lesson, we’ll look at the Microsoft Cognitive Services API for OCR.

* * *

### **[Citation Information](#TOC)**

****Rosebrock, A.**** “Text Detection and OCR with Amazon Rekognition API,” _PyImageSearch_, D. Chakraborty, P. Chugh, A. R. Gosthipaty, S. Huot, K. Kidriavsteva, R. Raha, and A. Thanki, eds., 2022, [https://pyimg.co/po6tf](https://pyimg.co/po6tf)

@incollection{Rosebrock\_2022\_OCR\_Amazon\_Rekognition\_API,
  author = {Adrian Rosebrock},
  title = {Text Detection and OCR with Amazon Rekognition API},
  booktitle = {PyImageSearch},
  editor = {Devjyoti Chakraborty and Puneet Chugh and Aritra Roy Gosthipaty and Susan Huot and Kseniia Kidriavsteva and Ritwik Raha and Abhishek Thanki},
  year = {2022},
  note = {https://pyimg.co/po6tf},
}

* * *

![Image 6: Featured Image](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20500%20276'%3E%3C/svg%3E)

### Unleash the potential of computer vision with Roboflow - Free!

*   Step into the realm of the future by [signing up or logging into your Roboflow account](https://roboflow.com/?ref=pyimagesearch). Unlock a wealth of innovative dataset libraries and revolutionize your computer vision operations.
*   Jumpstart your journey by choosing from our broad array of datasets, or benefit from PyimageSearch’s comprehensive library, crafted to cater to a wide range of requirements.
*   Transfer your data to Roboflow in any of the 40+ compatible formats. Leverage cutting-edge model architectures for training, and deploy seamlessly across diverse platforms, including API, NVIDIA, browser, iOS, and beyond. Integrate our platform effortlessly with your applications or your favorite third-party tools.
*   Equip yourself with the ability to train a potent computer vision model in a mere afternoon. With a few images, you can import data from any source via API, annotate images using our superior cloud-hosted tool, kickstart model training with a single click, and deploy the model via a hosted API endpoint. Tailor your process by opting for a code-centric approach, leveraging our intuitive, cloud-based UI, or combining both to fit your unique needs.
*   Embark on your journey today with absolutely no credit card required. Step into the future with Roboflow.

[Join Roboflow Now](https://roboflow.com/?ref=pyimagesearch)

**To download the source code to this post (and be notified when future tutorials are published here on PyImageSearch), _simply enter your email address in the form below!_**

![Image 7](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20410%20567'%3E%3C/svg%3E)

#### Download the Source Code and FREE 17-page Resource Guide

Enter your email address below to get a .zip of the code and a **FREE 17-page Resource Guide on Computer Vision, OpenCV, and Deep Learning.** Inside you'll find my hand-picked tutorials, books, courses, and libraries to help you master CV and DL!