PP-OCR — New SOTA in Character Recognition
By: Mikhail Raevskiy
Tags: Artificial Intelligence, Deep Learning, Data Science, Machie Learning, AI

1. Text detection

The goal of text detection is to locate the text area in the image. Generally, it is hoped that the detection result is divided into fields. PP-OCR uses a segmentation-based DB text detection algorithm. The starting point of using this method is the text detection method based on segmentation, which can get the tight bounding box of the text. In addition, the post-processing of DB is relatively simple and convenient for practical applications. In order to further improve the effect and efficiency of the text detector, PP-OCR adopts the following 6 strategies, ultra-lightweight backbone network selection, head weight reduction, SE module abandonment, cosine learning rate, warm-up learning rate, and FPGM model cropping Finally, the size of the text detection model in PP-OCR is only 1.4M.

2. Detection frame calibration

In order to improve the text recognition effect in the detection frame and maintain the consistency of the text, it is generally desired that the text frame to be recognized is in a positive horizontal direction. Since the DB text detection result is represented by a polygon with 4 points, it is easy to transform the detection result in a horizontal direction after an affine transformation. If the transformed image is in the vertical orientation, it will become horizontal after being rotated 90 degrees. But after changing to the horizontal orientation, the text may be upside down. So a text direction classifier is needed to determine whether the text is upside down. If the text is upside down, it can be recognized after correcting it. Training a text direction classifier is a basic image classification task. In order to further improve the effect and efficiency of the text direction classifier, PP-OCR adopts the following 4 strategies, ultra-lightweight backbone network selection, data enhancement, increasing input resolution and PACT int8 quantization, and finally, the text classifier in PP-OCR The size of the model is only 500KB.

3. Text recognition

The goal of text recognition is to convert text line images into text. PP-OCR uses CRNN, a commonly used method for text recognition. Although the CRNN text recognition method was proposed in 2016 and has some history, it is the most commonly used and effective text recognition method for Chinese recognition. CRNN mainly combines convolutional features and sequence features, and uses the CTC loss function to solve the problem of inconsistency between predicted labels and true value labels. In order to further improve the effect and efficiency of text recognition, PP-OCR adopts the following 9 strategies, ultra-lightweight backbone network selection, data enhancement, cosine learning rate, increasing feature map resolution, regularization parameters, warm-up learning rate, head Lightweight, pre-trained large model and PACT int8 quantization, in the end, the size of the Chinese and English text recognition model in PP-OCR is only 1.5M. The size of the pure English digital text recognition model is only 900KB.

Training data

The researchers collected data for both Chinese and English. There are three small datasets in total, which are combined into one big one:

97,000 images for detecting text in the image;

for detecting text in the image; Dataset for classifying the direction of text from 600,000 images ;

; 17.9 million images for OCR task

Demonstration