Before revision:  This fact remains true across a wide variety of BERT-based models  . Based on this observation, we propose a quite simple method to boost the performance of BERT.
After revision:  This fact remains true across a wide variety of BERT-based models , even when some layers are duplicated . Based on this observation, we propose a quite simple method to boost the performance of BERT.