‘Image text extraction with OCR using Open CV and PyTesseract’
By: Data Science Learners
Tags: Data Science, Tesseract, Ocr, Computer Vision, Opencv

Dr. Deepali Kulkarni

Vijay Chakole

This blog is for the beginners in data science who want to explore the most talked and interesting area of AI, Computer Vision.

Though all of us want to explore the facets of computer vision like Face Recognition, Object Recognition, Optical Character recognition; lengthy and complex GitHub codes make it difficult to interpret and understand .

Our strategy is to start with simple codes and explore it further.

There are simple codes in Open CV , Tesseract which are worth trying to start with.

What is OCR and text extraction ?

Optical Character Recognition. In other words, OCR systems transform a two-dimensional image of text, that could contain machine printed or handwritten text from its image representation into machine-readable text. OCR as a process generally consists of several sub-processes to perform as accurately as possible. These are as follows

OCR has been very popular term for its simple usage to convert hard copied data into a machine readable data into a desired format. Once it is converted into a formats like CSV, XML , JASON ; it adds more agility to data analysis and interpretation .

Do we have to use it along with Machine Learning or deep learning algorithms all the time?? Answer is NOT NECESSARILY.

Why Open CV?

OpenCV is the huge open-source library for the computer vision, machine learning, and image processing . The library has more than 2500 optimized algorithms which can be used to detect and recognize faces, identify objects, classify human actions in videos, track camera movements, track moving objects, extract 3D models and many more tasks.

What is Tesseract ?

Tesseract is an OCR engine with support for Unicode and the ability to recognize more than 100 languages out of the box. It is offered by Google and hence the most used library for the OCR projects .

What are some real time used cases of OCR ?

OCR is now a common word before its inception around 70s. Character recognition (Marshall, 2020) is widely used in many organizations which has helped to solve many business problems .License plate scanning , Cambridge Exam paper assessment , Google Translator etc. are many successful OCR implementation which add further scope .

Since then OCR has picked up fast and is being used by millions of people all over the world. Data Science aspirants are now working on the same to solve some day to day tasks and challenges. This blog starts with a simple code which has Open CV and Pytesseract libraries used to detect and extract text from a simple document without any GUI or web based tool. It does not involve any machine learning/deep learning algorithms as it a simple code to start with.

Let’s get started…..

Coding language used is Python .IDE used is Spyder as beginners may feel notebook non-responsive while running a specific command of Open CV.

Workflow at a glance before starting ..

1. Upload image

Import all the necessary packages .

Coders can refer the respective documentations for installing these packages including Tesseract . I will save time on that and directly dive into the simple coding

2. Pre-processing

Computer can read image in form of an array of pixels ranging from 0 to 255. Colourful images read by OpenCV are generally is BGR format (Blue, Green, Red) and a 3D array. The reason early developers at OpenCV chose BGR colour format is that back then BGR colour format was popular among camera manufacturers and software providers.

Cv2.imread can read the image on accepting image path.

NumPy Array 3D

I am taking an example of Marksheet to read and extract the text. This 3D array is Rows, Columns and 3 dimensions of the color scheme. In Pillow, the order of colors is assumed to be RGB (red, green, blue) and that is the reason we convert the image into reverse color scheme i.e. (RGB).

You can now view the image .

In my case image seems to be big in size which affecting the complete view of the image. We can resize the image to fit to the window.

Resizing takes width and height as an input but does not preserves the aspect ratio of the original image.

Input Image of a Marksheet

Image pre-processing involves lot of aspects and OpenCV library has wide options. All the steps can be written into a separate blog . I am limiting to a few as this blog focuses more on ‘SelectROIs’ function.

3. Selecting ROIs(Regions of Interest)

This function of OpenCV is useful in Object Tracking or simply putting boxes/frames around the targeted object.

By selectROI function of cv2 we can define the box around the text we want to extract in the image. cv2.selectROI(frame, False)

SelectROI takes arguments like input array that is image , and dragging the rectangle option from center is set false . If true center of selection will match initial mouse position. In opposite case a corner of selection rectangle will correspond to the initial mouse position. It also take first argument of setting a name for the window.

SelectROIs allows users to select multiple ROIs on the given image.

Running this function pops up image window and we can easily drag boxes around the data points. Draw a box and press ‘enter’ to select another and so on..

Window will be destroyed after pressing ‘Esc’.

Blue box around Candidate’s Name

I have selected 3 regions ‘Candidate’s Name’, ‘Reg.NO’ and ‘Enrollment no’.

As an output this will create an Numpy Object Array of the coordinates for the selected boxes. These coordinates are the ‘x’ and ‘Y’ coordinates of upper left corner and the bottom right corner of the selected boxes.

(31, 65, 125, 120)

^ ^ ^ ^

| | | |

x1 y1 | y2 = 120 + 65

x2 = 125 + 31

Output of 3 selected ROIs

Compiling and cropping all the selected boxes

These cropped images are seen by using same function cv2.imshow()

Here is the result!!!

Successfully cropped the boxes. Now these would be passed to Pytesseract for text detection.

4. Text detection and extraction

Pytesseract has different options to extract the text in form of string or a data frame.

In this case string seems to be suitable as the information we need is a string. In case of a table or other items , data frame is a suitable option. Custom config involves psm( page segmentation mode ) which is set 6 which assumes a single uniform block of text.

Out Text in form of a string

Tesseract supports multiple languages till its fourth version. This simple code can further be modified to extract text of different image documents in different formats. Simple code is not only easy to understand but also has a tremendous scope of converting it into a real time used case . So to construct a tool /app we don’t always need a foot long complex code but a strong logic and smart use of available resources.

Happy coding !! Happy Exploring !!!!!…………….

drdeepalikulkarni29@gmail.com

vijaychakole23@gmail.com

References

Babu, S. C. (n.d.). Retrieved from www.nanonets.com: https://nanonets.com/blog/receipt-ocr/

Docs, T. (n.d.). Retrieved from https://tesseract-ocr.github.io/

Marshall, C. (2020, Feb). Retrieved from https://medium.com/mysuperai/how-is-ocr-used-in-the-real-world-e82d1354f07b

team, O. C. (2020, December). www.opencv.org. Retrieved from https://opencv.org/about/