Before revision:  In this paper we propose a new model architecture DeBERTa(Decoding-enhanced BERT with disentangled attention) that improves the BERT and RoBERTa models using two novel techniques.
After revision:  In this paper wepropose a new model architecture DeBERTa(Decoding-enhanced BERT with disentangled attention) that improves the BERT and RoBERTa models using two novel techniques.