  A real-world information extraction (IE) system for semi-structured document
images often involves a long pipeline of multiple modules, whose complexity
dramatically increases its development and maintenance cost. One can instead
consider an end-to-end model that directly maps the input to the target output
and simplify the entire process. However, such generation approach is known to
lead to unstable performance if not designed carefully. Here we present our
recent effort on transitioning from our existing pipeline-based IE system to an
end-to-end system focusing on practical challenges that are associated with
replacing and deploying the system in real, large-scale production. By
carefully formulating document IE as a sequence generation task, we show that a
single end-to-end IE system can be built and still achieve competent
performance.
