  Eye movement data during reading is a useful source of information for
understanding language comprehension processes. In this paper, we describe our
submission to the CMCL 2021 shared task on predicting human reading patterns.
Our model uses RoBERTa with a regression layer to predict 5 eye-tracking
features. We train the model in two stages: we first fine-tune on the Provo
corpus (another eye-tracking dataset), then fine-tune on the task data. We
compare different Transformer models and apply ensembling methods to improve
the performance. Our final submission achieves a MAE score of 3.929, ranking
3rd place out of 13 teams that participated in this shared task.
