On mobile devices, distillation is a straightforward technique to improve the performance of deep learning models. We train a big and complicated network or ensemble model in this process, which may extract essential elements from the given data and, as a result, make superior predictions.