Information Extraction (IE) for  document images is often approached as a BIO tagging problem , where the model sequentially goes through and classifies each recognized input token into one of the information categories. However, such problem setup has two inherent limitations that (1) it can only extract a flat list of information and (2) it assumes that the input data is serialized, often by a simple rule-based script. Nevertheless, real-world documents often contain hierarchical informationin the form of two-dimensional language data in which the serialization can be highly non-trivial . To tackle these issues, we propose SPADE\spadesuit (SPatial DEpendency parser) , an end-to-end spatial dependency parser that is serializer-free and capable of modeling an arbitrary number of information layers , making it suitable for parsing structure-rich documents such as receipts and multimodal documents such as name cards. We show that SPADE\spadesuit outperforms the previous BIO tagging-based approach on name card parsing task and achieves comparable performance on receipt parsing task. Especially, when the receipt images have non-flat manifold representing physical distortion of receipt paper in real-world, SPADE\spadesuit outperforms the tagging-based method by a large margin of 25.8\% highlighting the strong performance of SPADE\spadesuit over spatially complex document .
Information Extraction (IE) for semi-structured document images is often approached as a sequence tagging problem by classifying each recognized input token into one of the IOB (Inside, Outside, and Beginning) categories. However, such problem setup has two inherent limitations that (1) it cannot easily handle complex spatial relationships and (2) it is not suitable for highly structured information, which are nevertheless frequently observed in real-world document images . To tackle these issues, we first formulate the IE task as spatial dependency parsing problem that focuses on the relationship among text segment nodes in the documents. Under this setup, we then propose SPADE (SPAtial DEpendency parser) that models highly complex spatial relationships and an arbitrary number of information layers in the documents in an end-to-end manner. We evaluate it on various kinds of documents such as receipts , name cards, forms, and invoices, and show that it achieves a similar or better performance compared to strong baselines including BERT-based IOB taggger, with up to 37.7\% improvement .
A
semi-structured
R
BIO tagging problem , where the model sequentially goes through and classifies
sequence tagging problem by classifying
R
information
IOB (Inside, Outside, and Beginning)
R
can only extract a flat list of information
cannot easily handle complex spatial relationships
R
assumes that the input data is serialized, often by a simple rule-based script. Nevertheless,
is not suitable for highly structured information, which are nevertheless frequently observed in
R
documents often contain hierarchical informationin the form of two-dimensional language data in which the serialization can be highly non-trivial
document images
R
propose SPADE\spadesuit (SPatial
first formulate the IE task as spatial dependency parsing problem that focuses on the relationship among text segment nodes in the documents. Under this setup, we then propose SPADE (SPAtial
R
, an end-to-end spatial dependency parser that is serializer-free and capable of modeling
that models highly complex spatial relationships and
R
, making it suitable for parsing structure-rich
in the documents in an end-to-end manner. We evaluate it on various kinds of
R
and multimodal documents such as name cards. We show that SPADE\spadesuit outperforms the previous BIO tagging-based approach on name card parsing task and achieves comparable performance on receipt parsing task. Especially, when the receipt images have non-flat manifold representing physical distortion of receipt paper in real-world, SPADE\spadesuit outperforms the tagging-based method by a large margin of 25.8\% highlighting the strong performance of SPADE\spadesuit over spatially complex document
, name cards, forms, and invoices, and show that it achieves a similar or better performance compared to strong baselines including BERT-based IOB taggger, with up to 37.7\% improvement