## 3 Materials and Methods
### Proposed Lightweight Pig Face Recognition Method Based on Knowledge Distillation
The main advantage of RKD lies in its ability to capture and transfer the relational aspects of the teacher model's feature representations, rather than merely mimicking the direct outputs of the teacher model. This is particularly crucial for knowledge transfer between models with significant structural differences. We adopted a two-stage distillation strategy (offline distillation). Initially, the ViT teacher model was pre-trained on a large-scale pig face recognition dataset to capture rich feature representations. Subsequently, we transferred the 'knowledge' of these representations to the ShuffleNet V2 student model via the RKD method. Specifically, we focused on minimizing two principal loss functions: the Distance-Wise Loss and the Angle-Wise Loss. The Distance-Wise Loss encourages the student model to learn the distance relations between features in the teacher model, while the Angle-Wise Loss focuses on the distribution of angles between feature vectors. In our study, we implemented the Relational Knowledge Distillation (RKD) method to transfer complex relational knowledge from a Vision Transformer (ViT) teacher model to a ShuffleNet V2 student model, focusing on pig face recognition. The RKD process commences with both models undergoing forward propagation; the ViT processes input images and generates high-dimensional feature representations from its Transformer encoder block, while the ShuffleNet V2 produces its feature representations from the last pooling layer. The essence of RKD lies in quantifying and minimizing the relational discrepancies between these feature representations, achieved through the RKD loss that encompasses both distance-wise and angle-wise relational comparisons. During backward propagation, this loss guides the student model in adjusting its parameters to align more closely with the teacher's feature representations. The student model's ability to mimic the teacher is refined through iterative updates aimed at reducing the RKD loss, alongside minimizing the classification loss for the primary task of pig face recognition. This training methodology enables the student model to not only acquire a nuanced understanding of pig face features similar to the ViT but to do so within a more compact and computationally efficient framework. The result is a distilled model that retains essential relational knowledge critical for effective recognition, making it well-suited for deployment in environments where computational resources are limited. In the distillation training process of the lightweight pig face recognition model, the model minimizes the multiple-task losses, including distillation loss \(\mathcal{L}_{RKD}\) and classification loss \(\mathcal{L}_{Cls}\), so that the lightweight student model can better learn the feature expression of the teacher model, as shown in Equation (3):  
\[\mathcal{L}_{Total}=\mathcal{L}_{RKD}+\mathcal{L}_{Cls}, \tag{3}\]  
The distillation loss from relational distillation loss consists of two kinds of loss: \(\mathcal{L}_{RKD-D}\), \(\mathcal{L}_{RKD-A}\), as presented in Equation (4):  
\[\mathcal{L}_{RKD}=\sum_{(x_{1},\dots,x_{n})\in\mathcal{X}^{\text{N}}}\iota( \psi(t_{1},\dots,t_{n}),\psi(s_{1},\dots,s_{n})), \tag{4}\]  
RKD loss calculates a relational potential \(\psi\) for each n-tuple of data examples and transfers information through the potential from the teacher to the student. Here, \((x_{1},x_{2},\dots,x_{n})\) is an n-tuple drawn from \(X^{\text{N}}\), \(\psi\) is a relational potential function that measures relational energy of the given n-tuple, and \(\iota\) is a loss that penalizes difference between the teacher and the student. Classification loss is a logarithmic loss to classify over 56 pig classes and is computed based on the output score.