Introduction to text detection using OpenCV and pytesseract
Tags: python, opencv, tutorial, imageprocessing
Published: 2021-03-02 10:38:56

## Introduction

Detecting and reading text from images is a complex task with a multitude of factors that is needed to take into account, it can be something like detecting handwritten text on a piece of paper to detect subtitles in a movie. 

In this short tutorial we are going to utilise OpenCVs image manipulation tools and the Python wrapper for  Google’s Tesseract-OCR Engine Python-tesseract or pytesseract to detect a specific number of predefined words in a video sequence.

## The fun part

You need to have OpenCV for Python and pytesseract on your system, so start with installing them via `pip:` 
```python
pip install opencv-python
pip install pytesseract
```
The idea here is to detect when (on what frame) the pre/post credits in a video shows up. For the simplicity of this tutorial just print the word and the frame number to the console. 

We start by importing OpenCV and pytesseract and define our `list` with words.

```python
from cv2 import cv2
import pytesseract

LIST = ['director', 'directed', 'written', 'writer',
        'producer', 'produced', 'production', 'lead'] 
```

The input will be a path to a local video file or a url, the url can also be a HLS manifest. Then we extract the first frame and define our frame counter. 

```python
path = input('Enter path to file or hls manifest: ')

vidcap = cv2.VideoCapture(path)
success,frame = vidcap.read()
framenbr = 0
```

What we want to do now is to read the next frame and process it as long as there is content left in our `vidcap` variable. If no content is present we are done and can break out of the reading loop.

```python
while success:
    success,frame = vidcap.read()
    if not success:
        break
```

We are now going to process the frame using OpenCV to make it easier for pytesseract to extract the text by convert the image to grayscale. 

```python
gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
```

We can also add layers of processing to the image at this stage but I will skip this for the simplicity of this tutorial. 

Now on to text extraction using pytesseract:

```python
data = pytesseract.image_to_string(gray, lang='eng', config='--psm 6').lower() 
```

Here `gray` is the grayscale image, `lang='eng'` is of course the language that we expect the text which we are trying to detect to be in, and `config='--psm 6'` defines what setting pytesseract will use when extracting the text. Page segmentation mode 6 will make pytesseract assume a single uniform block of text on extraction. 

The available page segmentation modes are:

* 0    Orientation and script detection (OSD) only.
* 1    Automatic page segmentation with OSD.
* 2    Automatic page segmentation, but no OSD, or OCR. (not implemented)
* 3    Fully automatic page segmentation, but no OSD. (Default)
* 4    Assume a single column of text of variable sizes.
* 5    Assume a single uniform block of vertically aligned text.
* 6    Assume a single uniform block of text.
* 7    Treat the image as a single text line.
* 8    Treat the image as a single word.
* 9    Treat the image as a single word in a circle.
* 10    Treat the image as a single character.
* 11    Sparse text. Find as much text as possible in no particular order.
* 12    Sparse text with OSD.
* 13    Raw line. Treat the image as a single text line,
    bypassing hacks that are Tesseract-specific.

The only thing left is to compare what we have in our `data` variable with our predefined list and continue to the next frame.

```python
if any(word in data for word in LIST):
    print(data + '\nFrame number: ' + str(framenbr))
framenbr += 1
```

Here is the complete example code:

```python
from cv2 import cv2
import pytesseract

LIST = ['director', 'directed', 'written', 'writer',
        'producer', 'produced', 'production', 'lead']

path = input('Enter path to file or hls manifest: ')

vidcap = cv2.VideoCapture(path)
success,frame = vidcap.read()
framenbr = 0

while success:
    success,frame = vidcap.read()
    if not success:
        break
    gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
    gray = cv2.resize(gray, None, fx=2, fy=2, interpolation=cv2.INTER_LINEAR)
    data = pytesseract.image_to_string(gray, lang='eng', config='--psm 6').lower()
    if any(word in data for word in LIST):
        print(data + '\nFrame number: ' + str(framenbr))
    framenbr += 1
```

## Conclusion

If we would take this to the next step it could be used for  detecting when an intro occurs in a tv-serie to make it easier for the viewer to skip it or to continue to the next episode.

If you need assistance in the development and implementation of this, our team of video developers are happy to help you out. If you have any questions or comments just drop a line in the comments section to this post.