Before revision:  Furthermore, TUPE unties the CLS \texttt{[ symbol from other positions to provide it with a more specific role to capture the global representation of the sentence . Extensive experiments and ablation studies on GLUE benchmark demonstrate the effectiveness and efficiency of the proposed method : TUPE outperforms several baselines on almost all tasks by a large margin. In particular, it can achieve a higher score than baselines while only using 30\% pre-training computational costs. We release our code at URL
After revision:  Furthermore, TUPE unties the CLS \texttt{[ symbol from other positions to provide it with a more specific role to capture the global representation of the sentence . Extensive experiments and ablation studies on GLUE benchmark demonstrate the effectiveness and efficiency of the proposed method . Codes and models are released at URL