Title: Tesseract OCR: Text localization and detection - PyImageSearch

URL Source: https://pyimagesearch.com/2020/05/25/tesseract-ocr-text-localization-and-detection/

Published Time: 2020-05-25T14:00:00+00:00

Markdown Content:
In this tutorial, you will learn how to utilize Tesseract to detect, localize, and OCR text, all within a single, efficient function call.

![Image 1](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20700%20387'%3E%3C/svg%3E)

Back in September, I showed you [how to use OpenCV to detect and OCR text.](https://pyimagesearch.com/2018/09/17/opencv-ocr-and-text-recognition-with-tesseract/) This method was a three stage process:

1.  Use [OpenCV’s EAST text detection model](https://pyimagesearch.com/2018/08/20/opencv-text-detection-east-text-detector/) to **detect the presence of text** in an image
2.  **Extract the text Region of Interest (ROI) from the image** using basic image cropping/NumPy array slicing
3.  Take the text ROI, and then **pass it into Tesseract to actually OCR the text**

Our method worked quite well but was a bit complicated and less efficient due to the multistage process.

PyImageSearch reader Bryan wonders if there is a better, more streamlined way:

> _Hi Adrian,_
> 
> _I noticed that OpenCV’s uses the EAST text detection model. I assume text detection also exists inside Tesseract?_
> 
> _If so, is there anyway we can utilize Tesseract to both detect the text and OCR it without having to call additional OpenCV functions?_

You’re in luck, Bryan. **Tesseract _does_ have the ability to perform text detection and OCR in a single function call** — and as you’ll find out, it’s quite easy to do!

**To learn how to detect, localize, and OCR text with Tesseract, _just keep reading._**

![Image 2](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20128%20128'%3E%3C/svg%3E)

#### Looking for the source code to this post?

[Jump Right To The Downloads Section](#download-the-code)

In the first part of this tutorial, we’ll discuss the concept of text detection and localization.

From there, I will show you how to install Tesseract on your system.

We’ll then implement text localization, detection, and OCR using Tesseract and Python.

Finally, we’ll review our results.

### What is text localization and detection?

**Text detection is the process of localizing _where_ an image text is.**

You can think of text detection as a specialized form of **object detection.**

In object detection, our goal is to (1) detect and compute the _bounding box_ of all objects in an image and (2) determine the _class label_ for each bounding box, similar to the image below:

![Image 3](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20600%20600'%3E%3C/svg%3E)

**Figure 1:** Tesseract can be used for both text localization and text detection. Text localization can be thought of as a specialized form of object detection

In text detection, our goal is to automatically compute the bounding boxes for every region of text in an image:

![Image 4](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%200%200'%3E%3C/svg%3E)

**Figure 2:** Once text has been localized/detected in an image, we can decode it using OCR software. Tesseract can be used for text localization/detection as well as OCR.

Once we have those regions, we can then OCR them.

### How to install pytesseract for Tesseract OCR

![Image 5](https://b2633864.smushcdn.com/2633864/wp-content/uploads/2020/05/tesseract_text_localization_tesseract_logo.png?lossy=2&strip=1&webp=1)

**Figure 3:** Installing Tesseract and pytesseract allows you to use Python code to perform text detection and OCR.

I have provided instructions for installing the [Tesseract OCR engine](https://github.com/tesseract-ocr/tesseract) as well as [pytesseract](https://github.com/madmaze/pytesseract) (the Python bindings used to interface with Tesseract) in my blog post [_OpenCV OCR and text recognition with Tesseract._](https://pyimagesearch.com/2018/09/17/opencv-ocr-and-text-recognition-with-tesseract/)

Follow the instructions in the _“How to install Tesseract 4”_ section of that tutorial, confirm your Tesseract install, and then come back here to learn how to detect and localize text with Tesseract.

### Project structure

Go ahead and grab today’s .zip from the **_“Downloads”_** section of this blog post. Once you extract the files, you’ll be presented with an especially simple project layout:

% tree
.
├── apple\_support.png
└── localize\_text\_tesseract.py

0 directories, 2 files

As you can see, we have only one Python script to review today — the `localize_text_tesseract.py` file.

Secondly, we have a single image to test our OCR script with. Feel free to grab other photos and graphics to test today’s code with as well!

Implementing text localization, text detection, and OCR with Tesseract
----------------------------------------------------------------------

We are now ready to implement text detection and localization with Tesseract.

Open up a new file, name it `localize_text_tesseract.py`, and let’s get to work:

\# import the necessary packages
from pytesseract import Output
import pytesseract
import argparse
import cv2

# construct the argument parser and parse the arguments
ap = argparse.ArgumentParser()
ap.add\_argument("-i", "--image", required=True,
	help="path to input image to be OCR'd")
ap.add\_argument("-c", "--min-conf", type=int, default=0,
	help="mininum confidence value to filter weak text detection")
args = vars(ap.parse\_args())

We begin by importing packages, namely pytesseract and OpenCV. Be sure to refer to the _“How to install pytesseract for Tesseract OCR”_ section above for installation links.

Next, we parse two [command line arguments](https://pyimagesearch.com/2018/03/12/python-argparse-command-line-arguments/):

*   `--image`: The path to the input image upon which we will perform OCR
*   `--min-conf`: In order to filter weak text detections, a minimum confidence threshold can be provided. By default, we’ve set the threshold to `0` so that all detections are returned

Let’s go ahead and run our input `--image` through `pytesseract` next:

\# load the input image, convert it from BGR to RGB channel ordering,
# and use Tesseract to localize each area of text in the input image
image = cv2.imread(args\["image"\])
rgb = cv2.cvtColor(image, cv2.COLOR\_BGR2RGB)
results = pytesseract.image\_to\_data(rgb, output\_type=Output.DICT)

**Lines 17 and 18** load the input `--image` and swap color channel ordering from BGR (OpenCV’s default) to RGB (compatible with Tesseract and pytesseract).

Then we detect and localize text using Tesseract and the `image_to_data` function (**Line 19**). This function returns `results`, which we’ll now post-process:

\# loop over each of the individual text localizations
for i in range(0, len(results\["text"\])):
	# extract the bounding box coordinates of the text region from
	# the current result
	x = results\["left"\]\[i\]
	y = results\["top"\]\[i\]
	w = results\["width"\]\[i\]
	h = results\["height"\]\[i\]

	# extract the OCR text itself along with the confidence of the
	# text localization
	text = results\["text"\]\[i\]
	conf = int(results\["conf"\]\[i\])

Looping over the text localizations (**Line 22**), we begin by extracting the bounding box coordinates (**Lines 25-28**).

To grab the OCR’d `text` itself, we extract the information contained within the `results` dictionary using the `"text"` key and index (**Line 32**). This is the _recognized_ text string.

Similarly, **Line 33** extracts the confidence of the text localization (the confidence of the _detected_ text).

From here, we’ll filter out weak detections and annotate our `image`:

\# filter out weak confidence text localizations
	if conf > args\["min\_conf"\]:
		# display the confidence and text to our terminal
		print("Confidence: {}".format(conf))
		print("Text: {}".format(text))
		print("")

		# strip out non-ASCII text so we can draw the text on the image
		# using OpenCV, then draw a bounding box around the text along
		# with the text itself
		text = "".join(\[c if ord(c) < 128 else "" for c in text\]).strip()
		cv2.rectangle(image, (x, y), (x + w, y + h), (0, 255, 0), 2)
		cv2.putText(image, text, (x, y - 10), cv2.FONT\_HERSHEY\_SIMPLEX,
			1.2, (0, 0, 255), 3)

# show the output image
cv2.imshow("Image", image)
cv2.waitKey(0)

Comparing confidence versus our `--min-conf` command line argument ensures that the confidence is sufficiently high (**Line 36**).

In our terminal, we `print` information for debugging/informational purposes, including both the confidence and text itself (**Lines 38-40**).

OpenCV’s `cv2.putText` function doesn’t support non-ASCII characters, so we need to strip any non-ASCII characters out. This is handled by **Line 45,** where we work with character ordinals (`ord(c)`). Be sure to refer to this [ASCII chart in Wikipedia](https://simple.wikipedia.org/wiki/ASCII#/media/File:ASCII-Table-wide.svg) as needed.

With the special characters eliminated from our `text`, now we’ll annotate the output image. **Line 46** draws a bounding box around the detected text, and **Lines 47 and 48** draw the `text` itself just above the bounding box region.

Finally, using OpenCV’s `imshow` function, we display the result on our screen (**Line 51**). In order to keep the GUI window on the screen longer than a few milliseconds, the `cv2.waitKey(0)` call locks the GUI window as visible until any key is pressed.

Great job performing OCR with Tesseract and pytesseract.

### Tesseract text localization, text detection, and OCR results

We are now ready to perform text detection and localization with Tesseract!

Make sure you use the **_“Downloads”_** section of this tutorial to download the source code and example image.

From there, open up a terminal, and execute the following command:

$ python localize\_text\_tesseract.py --image apple\_support.png
Confidence: 26
Text: a

Confidence: 96
Text: Apple

Confidence: 96
Text: Support

Confidence: 96

![Image 6](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20700%20387'%3E%3C/svg%3E)

**Figure 4:** Using Tesseract to perform text detection and OCR with Python. Without a confidence threshold set, there is room for misidentified text regions, as is evident in the _top-left_ of this graphic.

Here, you can see that Tesseract has detected all regions of text and OCR’d each text region. The results look good, but what is up with Tesseract thinking the leaf in the Apple logo is an `a`?

If you look at our terminal output, you’ll see that particular text region has _low confidence._

We can improve our Tesseract text detection results simply by supplying a `--min-conf` value:

$ python localize\_text\_tesseract.py --image apple\_support.png --min-conf 50
Confidence: 96
Text: Apple

Confidence: 96
Text: Support

Confidence: 96
Text: 1-800-275-2273

![Image 7](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20700%20387'%3E%3C/svg%3E)

**Figure 5:** Using Tesseract to perform text detection and OCR with Python. By setting a confidence threshold, we are able to eliminate the false detection, as in **Figure 4**.

Here, we are filtering out any text detections and OCR results that have a confidence `<= 50`, and as our results show, the low quality text region has been filtered out.

When developing your own text detection and OCR applications with Tesseract, consider using the `image_to_data` function — it’s super easy to use and makes text localization a breeze.

### What's next? We recommend [PyImageSearch University](https://pyimagesearch.com/pyimagesearch-university/?utm_source=blogPost&utm_medium=bottomBanner&utm_campaign=What%27s%20next%3F%20I%20recommend).

**Course information:**  
84 total classes • 114+ hours of on-demand code walkthrough videos • Last updated: February 2024  
★★★★★ 4.84 (128 Ratings) • 16,000+ Students Enrolled

**I strongly believe that if you had the right teacher you could _master_ computer vision and deep learning.**

Do you think learning computer vision and deep learning has to be time-consuming, overwhelming, and complicated? Or has to involve complex mathematics and equations? Or requires a degree in computer science?

That’s _not_ the case.

All you need to master computer vision and deep learning is for someone to explain things to you in _simple, intuitive_ terms. _And that’s exactly what I do_. My mission is to change education and how complex Artificial Intelligence topics are taught.

If you're serious about learning computer vision, your next stop should be PyImageSearch University, the most comprehensive computer vision, deep learning, and OpenCV course online today. Here you’ll learn how to _successfully_ and _confidently_ apply computer vision to your work, research, and projects. Join me in computer vision mastery.

**Inside PyImageSearch University you'll find:**

*   ✓ **84 courses** on essential computer vision, deep learning, and OpenCV topics
*   ✓ **84 Certificates** of Completion
*   ✓ **114+ hours** of on-demand video
*   ✓ **Brand new courses released _regularly_**, ensuring you can keep up with state-of-the-art techniques
*   ✓ **Pre-configured Jupyter Notebooks in Google Colab**
*   ✓ Run all code examples in your web browser — works on Windows, macOS, and Linux (no dev environment configuration required!)
*   ✓ Access to **centralized code repos for _all_ 536+ tutorials** on PyImageSearch
*   ✓ **Easy one-click downloads** for code, datasets, pre-trained models, etc.
*   ✓ **Access** on mobile, laptop, desktop, etc.

[Click here to join PyImageSearch University](https://pyimagesearch.com/pyimagesearch-university/?utm_source=blogPost&utm_medium=bottomBanner&utm_campaign=What%27s%20next%3F%20I%20recommend)

Summary
-------

In this tutorial, you learned how to use Tesseract to detect text, localize it, and then OCR it.

The benefit of using Tesseract to perform text detection and OCR is that we can do so _in just a single function call_, making it easier than the [multistage OpenCV OCR process](https://pyimagesearch.com/2018/09/17/opencv-ocr-and-text-recognition-with-tesseract/).

That said, OCR is still an area of computer vision that is far from solved.

Whenever confronted with an OCR project, be sure to apply both methods and see which method gives you the best results — let your empirical results guide you.

**To download the source code to this post (and be notified when future tutorials are published here on PyImageSearch), _simply enter your email address in the form below!_**

![Image 8](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20410%20567'%3E%3C/svg%3E)

#### Download the Source Code and FREE 17-page Resource Guide

Enter your email address below to get a .zip of the code and a **FREE 17-page Resource Guide on Computer Vision, OpenCV, and Deep Learning.** Inside you'll find my hand-picked tutorials, books, courses, and libraries to help you master CV and DL!