Before revision:  To alleviate the conflicts between weak data and high computational consumption of DNNs , we propose a new training framework named Spirit Distillation(SD) . It extends the ideas of fine-tuning-based transfer learning(FTT) and feature-based knowledge distillation. By allowing the student to mimic its teacher in feature extraction, the gap of general features between the teacher-student networks is bridged. The Image Party distillation enhancement method(IP) is also proposed, which shuffling images from various domains, and randomly selecting a few as mini-batch.
After revision:  To alleviate the conflicts between weak data and high computational consumption of DNNs , we propose a new training framework named Spirit Distillation(SD) , which allow the student network to mimic the teacher network to extract general features, so that a compact and accurate student network can be trained for real-time semantic segmentation of road scenes. Then, in order to further alleviate the trouble of insufficient data and improve the robustness of the student, an Enhanced Spirit Distillation (ESD) method is proposed, which shuffling images from various domains, and randomly selecting a few as mini-batch.