Before revision:  Despite the widespread  of pre-training models for NLP applications, they almost focused on text-level manipulation, while neglecting the layout and style information that is vital for document image understanding.
After revision:  Despite the widespread of pre-training models for NLP applications, they almost focused on text-level manipulation, while neglecting layout and style information that is vital for document image understanding.