Before revision: As the excessive pre-training cost arouses the need to improve efficiency , considerable efforts have been made to train BERT progressively--start from an inferior but low-cost model and gradually  increase the computational complexity.
After revision: Due to the excessive cost of large-scale language model pre-training , considerable efforts have been made to train BERT progressively--start from an inferior but low-cost model and gradually increase the computational complexity.