EATEN: EntityAware Attention for Single Shot Visual Text Extraction Extracting Text of Interest (ToI) from images is a crucial part of many OCR applications, such as entity recognition of cards, invoices, and receipts. Most of the existing works employ complicated engineering pipeline, which contains OCR and structure information extraction, to fulfill this task. This paper proposes an Entityaware Attention Text Extraction Network called EATEN, which is an endtoend trainable system to extract the ToIs without any postprocessing. In the proposed framework, each entity is parsed by its corresponding entityaware decoder, respectively. Moreover, we innovatively introduce a state transition mechanism which further improves the robustness of visual ToI extraction. In consideration of the absence of public benchmarks, we construct a dataset of almost 0.6 million images in three realworld scenarios (train ticket, passport and business card), which is publicly available at https:github.combeacandlerEATEN. To the best of our knowledge, EATEN is the first single shot method to extract entities from images. Extensive experiments on these benchmarks demonstrate the stateoftheart performance of EATEN.

OCRVQA: Visual Question Answering by Reading Text in Images The problem of answering questions about an image is popularly known as visual question answering (or VQA in short). It is a wellestablished problem in computer vision. However, none of the VQA methods currently utilize the text often present in the image. These texts in images provide additional useful cues and facilitate better understanding of the visual content. In this paper, we introduce a novel task of visual question answering by reading text in images, i.e., by optical character recognition or OCR. We refer to this problem as OCRVQA. To facilitate a systematic way of studying this new problem, we introduce a largescale dataset, namely OCRVQA200K. This dataset comprises of 207,572 images of book covers and contains more than 1 million questionanswer pairs about these images. We judiciously combine wellestablished techniques from OCR and VQA domains to present a novel baseline for OCRVQA200K. The experimental results and rigorous analysis demonstrate various challenges present in this dataset leaving ample scope for the future research. We are optimistic that this new task along with compiled dataset will openup many exciting research avenues both for the document image analysis and the VQA communities.