General OCR Theory: Towards OCR-2.0 via a Unified End-to-end Model
Authors: name: Haoran Wei; name: Chenglong Liu; name: Jinyue Chen; name: Jia Wang; name: Lingyu Kong; name: Yanming Xu; name: Zheng Ge; name: Liang Zhao; name: Jianjian Sun; name: Yuang Peng; name: Chunrui Han; name: Xiangyu Zhang
Published: 2024-09-03 08:41:31

Abstract: Traditional OCR systems (OCR-1.0) are increasingly unable to meet people's
usage due to the growing demand for intelligent processing of man-made optical
characters. In this paper, we collectively refer to all artificial optical
signals (e.g., plain texts, math/molecular formulas, tables, charts, sheet
music, and even geometric shapes) as "characters" and propose the General OCR
Theory along with an excellent model, namely GOT, to promote the arrival of
OCR-2.0. The GOT, with 580M parameters, is a unified, elegant, and end-to-end
model, consisting of a high-compression encoder and a long-contexts decoder. As
an OCR-2.0 model, GOT can handle all the above "characters" under various OCR
tasks. On the input side, the model supports commonly used scene- and
document-style images in slice and whole-page styles. On the output side, GOT
can generate plain or formatted results (markdown/tikz/smiles/kern) via an easy
prompt. Besides, the model enjoys interactive OCR features, i.e., region-level
recognition guided by coordinates or colors. Furthermore, we also adapt dynamic
resolution and multi-page OCR technologies to GOT for better practicality. In
experiments, we provide sufficient results to prove the superiority of our
model.