We present BERTweet, the first public large-scale pre-trained language model for English Tweets. Our BERTweet is trained using the RoBERTa pre-training procedure (Liu et al., 2019), with the same model configuration as BERT-base (Devlin et al., 2019). Experiments show that BERTweet outperforms strong baselines RoBERTa-base and XLM-R-base (Conneau et al., 2020), producing better performance results than the previous state-of-the-art models on three Tweet NLP tasks: Part-of-speech tagging, Named-entity recognition and text classification. We release BERTweet  to facilitate future research and downstream applications on Tweet data. Our BERTweet is available at : URL
We present BERTweet, the first public large-scale pre-trained language model for English Tweets. Our BERTweet , having the same architecture as BERT-base (Devlin et al., 2019), is trained using the RoBERTa pre-training procedure (Liu et al., 2019). Experiments show that BERTweet outperforms strong baselines RoBERTa-base and XLM-R-base (Conneau et al., 2020), producing better performance results than the previous state-of-the-art models on three Tweet NLP tasks: Part-of-speech tagging, Named-entity recognition and text classification. We release BERTweet under the MIT License to facilitate future research and applications on Tweet data. Our BERTweet is available at URL
R
is trained using the RoBERTa pre-training procedure (Liu
, having the same architecture as BERT-base (Devlin
R
with the same model configuration as BERT-base (Devlin
is trained using the RoBERTa pre-training procedure (Liu
A
under the MIT License
D
downstream
D
: