# Is One Teacher Model Enough to Transfer Knowledge to a Student Model?
## 5 Experiments
### Hypothesis Validation  
In this first group of experiments summarized in Tables 1 and 2, as described in Section 3.1, we eliminated the translator and changed the shape of the penultimate layer of the student to adapt it to the size of the features' layer of the teacher. To try to understand how this solution works, we used different neural network models on the Cifa10, Cifar100, and Cub200 datasets. Furthermore, we compared the results with the original approaches proposed in [PERSON] et al. [2] and KD [1].  
For the experiment on Cifar10 and Cifar100, the size of the input images was set to \(32\times 32\) and we also set a learning rate to 0.0001, the batch size was set at 128, and we performed a total of 200 training epochs. We can see in Table 1 the comparison with the KD and [PERSON] et al. approaches. We can see how the KD approach (which does not use the translator for features but uses the original values) works well compared to the [PERSON] et al. approach which uses a translator. Our solution, on the other hand, without the translator, is able to outperform any other proposed method. The effectiveness of the data augmentation solution (see Section 3.5) on Cifar10, which has a limited number of classes, is particularly noteworthy. Instead, this method is less effective on Cifar100, which has a much higher number of classes. Table 1 validates our hypothesis at the translator level: the model proposed by [PERSON] et al. without the translator level generally works better.  
In this last experiment of this first group, we used the same dimensions of the features as for the previous experiments and the aim was to validate the goodness of our model.  
Figure 5: A sample image for each of the 200 classes of the Cub200-2011 dataset.  
Table 2 reports all numerical results. We trained a MobileNetV2 [18] on the CUB200 dataset using a pre-trained model NTS-Net [8; 19] as a teacher. The MobileNetV2 student feature layer \(f_{l_{1}}\) was adapted to the size of the teacher that is 10240. We used a batch size equal to 10. In these experiments, we used \(448\times 448\) size input images obtained by scaling the original image to \(600\times 600\) and extracting a centered crop of size \(448\times 448\). In terms of the best results, the teacher-less experiments had a learning rate of 0.0001 which did not change over the epochs. Instead, for the experiment with the teacher, we have a learning rate of 0.001 and after 100 epochs, it is multiplied by 0.1. In Table 2, we can see the accuracy of the student when trained without a teacher compared to the version trained using a teacher. We highlight the capability to increase the learning process using the same baseline but without the teacher model.  
The NTS-Net [8] is a CNN successfully applied to solve fine-grained classification problems. This concatenates the features extracted from the whole image and four crops using attention mechanisms. We experimentally proved the effectiveness of our proposal over the CUB200 dataset and report the results in Table 2, where the student network can improve their accuracy thanks to teacher-student learning strategy.