Before revision:  In this paper, we try to solve this issue, and propose two effective approaches , namely 1) mutual information  based estimation and 2) reconstruction loss based estimation , to measure the hardness of learning the representation for a word and determine its computational depth. Results on the classic text classification task ( 24 datasets in various sizes and domains ) show that our approaches achieve superior performance while preserving higher efficiency in computation over the vanilla Transformer and previous depth-adaptive models.
After revision:  In this paper, we try to solve this issue, and propose two effective approaches , namely 1) mutual information based estimation and 2) reconstruction loss based estimation . We conduct experiments on the text classification task ( 24 datasets in various sizes and domains ) show that our approaches achieve superior performance while preserving higher efficiency in computation over the vanilla Transformer and previous depth-adaptive models.