Platypus: A Generalized Specialist Model for Reading Text in Various
  Forms
Authors: name: Peng Wang; name: Zhaohai Li; name: Jun Tang; name: Humen Zhong; name: Fei Huang; name: Zhibo Yang; name: Cong Yao
Published: 2024-08-27 06:24:51

Abstract: Reading text from images (either natural scenes or documents) has been a
long-standing research topic for decades, due to the high technical challenge
and wide application range. Previously, individual specialist models are
developed to tackle the sub-tasks of text reading (e.g., scene text
recognition, handwritten text recognition and mathematical expression
recognition). However, such specialist models usually cannot effectively
generalize across different sub-tasks. Recently, generalist models (such as
GPT-4V), trained on tremendous data in a unified way, have shown enormous
potential in reading text in various scenarios, but with the drawbacks of
limited accuracy and low efficiency. In this work, we propose Platypus, a
generalized specialist model for text reading. Specifically, Platypus combines
the best of both worlds: being able to recognize text of various forms with a
single unified architecture, while achieving excellent accuracy and high
efficiency. To better exploit the advantage of Platypus, we also construct a
text reading dataset (called Worms), the images of which are curated from
previous datasets and partially re-labeled. Experiments on standard benchmarks
demonstrate the effectiveness and superiority of the proposed Platypus model.
Model and data will be made publicly available at
https://github.com/AlibabaResearch/AdvancedLiterateMachinery/tree/main/OCR/Platypus.