Task: Summarization
Input (English): Deep neural networks with millions of parameters may suffer from poor generalizations due to overfitting. To mitigate the issue, we propose a new regularization method that penalizes the predictive distribution between similar samples. In particular, we distill the predictive distribution between different samples of the same label and augmented samples of the same source during training. In other words, we regularize the dark knowledge (i.e., the knowledge on wrong predictions) of a single network, i.e., a self-knowledge distillation technique, to force it output more meaningful predictions.   We demonstrate the effectiveness of the proposed method  via  experiments  on  various  image  classification  tasks:  it  improves  not only the generalization ability, but also the calibration accuracy of modern neural networks.
Output (English): We propose a new regularization technique based on the knowledge distillation.