Before revision:  Moreover, we obtain consistent and promising results for the GLUE and SNLI benchmarks .
After revision:  Moreover, we obtain consistent and promising results for NLU benchmarks. In particular, pre-training on the argument schemes raises zero-shot accuracy on the GLUE diagnostics by up to 15 percentage points .