Before revision:  In our evaluation on fact verification systems , we show that fine-tuning with EWC Pareto dominates standard fine-tuning, yielding models  lower levels of URLetting on the original task for equivalent gains in accuracy on the fine-tuned task. Additionally, we show that systems trained on NLI can be fine-tuned to improve their accuracy on stress test challenge tasks with minimal loss in accuracy on the MultiNLI datasetdespite greater domain shift .
After revision:  In our evaluation on fact verification systems , we show that fine-tuning with EWC Pareto dominates standard fine-tuning, yielding models lower levels of URLetting on the original task for equivalent gains in accuracy on the fine-tuning (unbiased) dataset .