Before revision:  While large-scale and complete data can train out fine DNNs, collecting it is always difficult, expensive, and time-consuming. Also, the significance of both accuracy and efficiency cannot be over-emphasized due to the requirementof real-time recognition. To alleviate the conflicts between weak data and high computational consumption of DNNs , we propose a new training framework named Spirit Distillation(SD) . It extends the ideas of fine-tuning-based transfer learning(FTT) and feature-based knowledge distillation.
After revision:  While large-scale training data requirement. Inspired by the ideas of Fine-tuning-based Transfer Learning (FTT) and feature-based knowledge distillation , we propose a new training framework named Spirit Distillation(SD) . It extends the ideas of fine-tuning-based transfer learning(FTT) and feature-based knowledge distillation.