In this task, you will be presented with a context from an academic paper and a question separated with a
. You have to answer the question based on the context.

To test the applicability of the proposed MWA attention, we choose three publicly available Chinese pre-trained models as the basic encoder: BERT, ERNIE, and BERT-wwm. In order to make a fair comparison, we keep the same hyper-parameters (such maximum length, warm-up steps, initial learning rate, etc) as suggested in BERT-wwm BIBREF13 for both baselines and our method on each dataset.
Question: What pre-trained models did they compare to?
BERT, ERNIE, and BERT-wwm