  In this paper, we describe our system for the AAAI 2021 shared task of
COVID-19 Fake News Detection in English, where we achieved the 3rd position
with the weighted F1 score of 0.9859 on the test set. Specifically, we proposed
an ensemble method of different pre-trained language models such as BERT,
Roberta, Ernie, etc. with various training strategies including
warm-up,learning rate schedule and k-fold cross-validation. We also conduct an
extensive analysis of the samples that are not correctly classified. The code
is available
at:https://github.com/archersama/3rd-solution-COVID19-Fake-News-Detection-in-English.
