Before revision:  In this paper, a novel data-efficient approach to transfer the knowledge from a teacher model to a student model is presented. Here, the teacher model uses self-regulation to select appropriate samplesfor training and identifies their significance in the process.
After revision:  In this paper, a novel data-efficient approach to transfer the knowledge from a teacher model to perform distillation using only a subset of the training data, making it more data-efficient. For this purpose, the training of the teacher model is modified to include self-regulation to select appropriate samplesfor training and identifies their significance in the process.