Before revision:  Comprehensive pre-training and fine-tuning experiments on GLUE and SQuAD downstream tasks  show that EarlyBERT easily achieves comparable performance to standard BERT  with 35~45\% less training time. 
After revision:  Comprehensive pre-training and fine-tuning experiments on GLUE and SQuAD downstream tasks show that EarlyBERT easily achieves comparable performance to standard BERT with 35~45\% less training time. Code is available at URL