Before revision:  In this paper, we present a new approach to improve the capability of language model pre-training on VRDs. Firstly, we introduce a new  IE model that is query-based and employs the span extraction formulation instead of the commonly used sequence labelling approach.
After revision:  In this paper, we present a new approach to improve the capability of language model pre-training on VRDs. Firstly, we introduce a new query-based IE model that is query-based and employs the span extraction formulation instead of the commonly used sequence labelling approach.