Before revision: In this paper we introduce a novel approach to the problem of understanding documents where  the local semantics is influenced by non-trivial layout. Namely , we modify the Transformer  architecture in a way that allows it to use the graphical features defined by the layout , without the need to re-learn the language semantics from scratch , thanks to starting the training process from a model pretrained on classical language modeling tasks .
After revision: In this paper we introduce a novel approach to the problem of understanding documents where the local semantics . To this end , we modify the Transformer architecture in a way that allows it to use the graphical features defined by the layout , without the need to re-learn the language semantics from scratch , thanks to starting the training process from a model pretrained on classical language modeling tasks .