Attend, Copy, Parse Endtoend Information Extraction from Documents Document information extraction tasks performed by humans create data consisting of a PDF or document image input, and extracted string outputs. This endtoend data is naturally consumed and produced when performing the task because it is valuable in and of itself. It is naturally available, at no additional cost. Unfortunately, stateoftheart word classification methods for information extraction cannot use this data, instead requiring wordlevel labels which are expensive to create and consequently not available for many real life tasks. In this paper we propose the Attend, Copy, Parse architecture, a deep neural network model that can be trained directly on endtoend data, bypassing the need for wordlevel labels. We evaluate the proposed architecture on a large diverse set of invoices, and outperform a stateoftheart production system based on word classification. We believe our proposed architecture can be used on many real life information extraction tasks where word classification cannot be used due to a lack of the required wordlevel labels.

EATEN: EntityAware Attention for Single Shot Visual Text Extraction Extracting Text of Interest (ToI) from images is a crucial part of many OCR applications, such as entity recognition of cards, invoices, and receipts. Most of the existing works employ complicated engineering pipeline, which contains OCR and structure information extraction, to fulfill this task. This paper proposes an Entityaware Attention Text Extraction Network called EATEN, which is an endtoend trainable system to extract the ToIs without any postprocessing. In the proposed framework, each entity is parsed by its corresponding entityaware decoder, respectively. Moreover, we innovatively introduce a state transition mechanism which further improves the robustness of visual ToI extraction. In consideration of the absence of public benchmarks, we construct a dataset of almost 0.6 million images in three realworld scenarios (train ticket, passport and business card), which is publicly available at https:github.combeacandlerEATEN. To the best of our knowledge, EATEN is the first single shot method to extract entities from images. Extensive experiments on these benchmarks demonstrate the stateoftheart performance of EATEN.