Learning Compositional Neural Information Fusion for Human Parsing. This work proposes to combine neural networks with the compositional hierarchy of human bodies for efficient and complete human parsing. We formulate the approach as a neural information fusion framework. Our model assembles the information from three inference processes over the hierarchy: direct inference (directly predicting each part of a human body using image information), bottomup inference (assembling knowledge from constituent parts), and topdown inference (leveraging context from parent nodes). The bottomup and topdown inferences explicitly model the compositional and decompositional relations in human bodies, respectively. In addition, the fusion of multisource information is conditioned on the inputs, i.e., by estimating and considering the confidence of the sources. The whole model is endtoend differentiable, explicitly modeling information flows and structures. Our approach is extensively evaluated on four popular datasets, outperforming the stateofthearts in all cases, with a fast processing speed of 23fps. Our code and results have been released to help ease future research in this direction.

Attend, Copy, Parse Endtoend Information Extraction from Documents Document information extraction tasks performed by humans create data consisting of a PDF or document image input, and extracted string outputs. This endtoend data is naturally consumed and produced when performing the task because it is valuable in and of itself. It is naturally available, at no additional cost. Unfortunately, stateoftheart word classification methods for information extraction cannot use this data, instead requiring wordlevel labels which are expensive to create and consequently not available for many real life tasks. In this paper we propose the Attend, Copy, Parse architecture, a deep neural network model that can be trained directly on endtoend data, bypassing the need for wordlevel labels. We evaluate the proposed architecture on a large diverse set of invoices, and outperform a stateoftheart production system based on word classification. We believe our proposed architecture can be used on many real life information extraction tasks where word classification cannot be used due to a lack of the required wordlevel labels.