# Is One Teacher Model Enough to Transfer Knowledge to a Student Model?
## 5 Experiments
### Analysis of the Proposed Method with a Single Teacher and Student  
As we saw in the previous group of experts, the problem is not the translator itself, but that the feature translation phase does not know that the features need to be translated for a classification problem that the student has to solve. From this observation, we introduced our proposal: the student topology is not modified and we introduced the translator again but this time, adding another cross-entropy loss function integrated with the transfer learning process. In this way, the student can learn from the teacher's features while also learning to classify.  
In this group of experiments, we used the Lenet5 and Mobilenetv3 models as students and Resnet50 and InceptionV3 as teachers on Cifar10 and Cifar100. The solution with only one teacher and one student proposed by us is compared with the accuracy reported by the model proposed by [PERSON] et al. and by the KD model. In this group of experiments, the student model has the feature layer not adapted to the teacher size, because we have the translator. As in the other experiments on Cifar10 and Cifar100, the size of the input images was set to 32 \(\times\) 32 and we also set a learning rate to 0.0001, the batch size was set at 128, and we performed a total of 200 training epochs. The results are reported in Table 3.  
From the comparative results, we can see that the KD model, as we explained in Section 2 and as the authors demonstrated in their article [1], does not always show improvements in accuracy compared to the same model without transfer learning (in fact, their best result  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Name** & **Without Teacher** & **With Teacher** \\ \hline MobileNetV2 & 42.5 & **49.2** \\ \hline \hline \end{tabular}
\end{table}
Table 2: In this table, we report the test accuracy obtained by a MobileNetV2 used as a student with an NTS-Net as a teacher over CUB200 dataset.  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Dataset** & **Student** & **Baseline** & **[PERSON]** & **KD** & **T1** & **T2** \\ \hline Cifar10 & Lenet5 & 64.12 & 60.64 & 61.01 & 64.77 & **64.89** \\ Cifar10 & Mobilenetv3 & 60.98 & 62.12 & 60.59 & **64.42** & 64.29 \\ \hline Cifar100 & Lenet5 & 32.60 & 32.37 & 33.57 & **34.25** & 34.09 \\ Cifar100 & Mobilenetv3 & 30.26 & 30.29 & 25.22 & 31.78 & **33.17** \\ \hline \hline \end{tabular}
\end{table}
Table 3: Accuracy in percentage after 200 epochs over Cifar10 and Cifar100 using InceptionV3 (T1) and T1 plus Resnet50 (T2) as teachers. The baseline here is the model that does not have the modified layer because we make use of the translator.  
is obtained when they do the transfer learning between an ensemble and a single network, unlike what we do).  
The approach proposed by [PERSON] et al. gives good results in transferring learning between a complex network and a simplified version of the same network, but in a more generic case such as those shown in Table 3, we can see that its results are not exciting.  
In conclusion, this group of experiments with Cifar10 and Cifar100, comparing our results with those of the other models, we can see that our proposal always improves accuracy using a single teacher and is therefore more reliable than the other solutions used.