Before revision:  Given thata key challenge with this task is the limitedsize of annotated data, our model relies onpretrained contextual representations from dif-ferent state-of-the-art transformer-based lan-guage models (i.e., BERT and RoBERTa), andon a variety of training methods for further en-hancing model generalization and robustness:multi-step fine-tuning and multi-task learning, and adversarial training.
After revision:  Given thata key challenge with this task is the limitedsize of annotated data, our model relies onpretrained contextual representations from dif-ferent state-of-the-art transformer-based lan-guage models (i.e., BERT and RoBERTa), and on a variety of training methods for further en-hancing model generalization and robustness:multi-step fine-tuning and multi-task learning, and adversarial training.