Before revision:  In this way, our approach brings a deeper level of interaction between the original and compact models , and smooths the training process . Compared to the previous knowledge distillation approaches for BERT compression, our approach leverages only one loss functionand one hyper-parameter, liberating human effort from hyper-parameter tuning .
After revision:  In this way, our approach brings a deeper level of interaction between the original and compact models . Compared to the previous knowledge distillation approaches for BERT compression, our approach leverages only one loss functionand one hyper-parameter, liberating human effort from hyper-parameter tuning .