Before revision:  To alleviate the conflicts between weak data and high computational consumption of DNNs , we propose a new training framework named Spirit Distillation(SD) . It extends the ideas of fine-tuning-based transfer learning(FTT) and feature-based knowledge distillation.
After revision:  To alleviate the conflicts between weak data and high computational consumption of DNNs , we propose a new knowledge distillation method for cross-domain knowledge transference and efficient data-insufficient network training, named Spirit Distillation(SD) . It extends the ideas of fine-tuning-based transfer learning(FTT) and feature-based knowledge distillation.