## 4 Experiments
### Ablation Studies  
**Large-Scale Teacher Model Comparisons. ** This section illustrates a comparative study of various advanced architectures for the selection of an optimal teacher model in our knowledge distillation process for pig face recognition. The analysis focused on ResNet-152, Vision Transformer (ViT-base), and Res2 Net-101, each differing in design principles and performance metrics. Figure 12 conveys the training dynamics of each model, with the ViT-base model exhibiting a steep initial loss reduction, stabilizing at a low value, indicating its capacity for rapid learning. Its corresponding accuracy graph shows a trajectory of swift improvement, where, despite starting from a lower value due to the lack of pre-trained weights, it ascends rapidly, surpassing the other models and nearing the teacher model's top performance. Table 3 confirms the superior capabilities of the ViT-base, with the highest Accuracy and Recall scores, suggesting its potential as a teacher model. Although ViT-base demands more computational resources, as indicated by its FLOPs and parameters, its performance in our experiments justifies its selection as the teacher model in the distillation process. The trade-off between the computational cost and the performance gain is deemed acceptable, particularly because the distilled student model, informed by ViT-base, effectively bridges the gap, delivering high accuracy with reduced complexity that are suitable for embedded systems in smart farming applications. **Lightweight Student Model Selection. ** In the quest to optimize a lightweight student model for pig face recognition, our exploration led us to evaluate several architectures renowned for their efficiency. The graph accompanying this text delineates the training loss and accuracy over epochs for these contenders, including ShuffleNet-V1, ShuffleNet-V2, and MobileNet-V2. The left side of Figure 13 showcases the three models' training loss. The training loss and Accuracy graphs depict the learning efficiency of various models. Initially, ShuffleNet-V2 shows a higher loss compared to MobileNet-V2 and ShuffleNet-V1 but rapidly converges to a lower loss, indicating effective learning. In terms of accuracy, ShuffleNet-V2 demonstrates a remarkable improvement over epochs, surpassing the other models and approaching the performance of more complex architectures. This is further corroborated by Table 4, which shows ShuffleNet-V2 achieving the highest F1 Score, denoting a superior balance between Precision and Recall, ultimately making it the preferred student model for knowledge distillation in pig face recognition tasks. Figure 12: Comparison of teacher models’ loss and Accuracy curves. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Model** & **Accuracy** & **Precision** & **Recall** & **F1 Score** & **FLOPs** & **Parameters** \\ \hline ResNet & 0.921 & 0.919 & 0.916 & 0.921 & 11.6 & 58.2 \\ ViT-base & 0.972 & 0.978 & 0.970 & 0.974 & 55.5 & 86.1 \\ Res2 Net & 0.944 & 0.946 & 0.950 & 0.946 & 8.1 & 43.2 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Comparison of state-of-art teacher model for pig face recognition task. **State-of-the-art Knowledge Distillation Method Comparison for Pig Face Recognition. ** A critical aspect of our research was to evaluate the effectiveness of different knowledge distillation (KD) techniques applied to the student model in the context of pig face recognition. To this end, we compared several state-of-the-art distillation methods, with the aim of identifying which method best leverages the teacher model's knowledge to enhance the student model's performance. The evaluated methods included Relational Knowledge Distillation (RKD), Weighted Soft Label Distillation (WSDD), and Traditional Knowledge Distillation (KD). Table 5 presents the comparative results of these methods. In distillation experiments, we selected ViT-base and ShuffleNet-V2 as teacher and student models, respectively. RKD emerged as the leading technique, achieving the highest Accuracy of 0.968 and an F1 Score of 0.972.