Before revision:  In this paper we propose a new model architecture DeBERTa(Decoding-enhanced BERT with disentangled attention) that improves the BERT and RoBERTa models using two novel techniques.
After revision:  In this paper we propose a new model architecture DeBERTa(Decoding-enhanced BERT with dis-entangled attention) that improves the BERT and RoBERTa models using two novel techniques.