## 3 Materials and Methods
### Proposed Lightweight Pig Face Recognition Method Based on Knowledge Distillation  
To trade off accuracy and computational cost, the advanced offline knowledge distillation technique [44] was employed to enhance the accuracy of the lightweight pig face recognition model. In the offline knowledge distillation technique, the student model learns by adopting insights from a teacher model that has already been trained, whereas the teacher model obtains more precise results but is computationally expensive. The lightweight student model can perform better by transferring high-order feature information from the teacher model. To implement this method, initially, the more comprehensive teacher model is first trained on our pig face recognition dataset to establish a robust knowledge base before distillation. Secondly, this teacher model guides and imparts knowledge onto a smaller, more agile student model during the distillation process. **Large-Scale Teacher Model Selection**. In the proposed method, choosing an advanced high-precision classification model as the teacher network is a critical step in offline knowledge distillation for enhancing the performance of the student network. Vision Transformer (ViT) was opted as our core teacher model due to its exemplary performance. ViT applies the self-attention mechanism to sequences of image patches, enabling the model to capture global dependencies across the entire image. The model streamlines pig face recognition using a Vision Transformer (ViT) as shown in Figure 5. It starts with an input image of dimension \(\mathbb{R}^{H\times W\times C}\), which is normalized and resized to \(\mathbb{R}^{384\times 384\times 3}\). The image is then divided into patches, flattened, and linearly embedded to \(\mathbb{R}^{N\times(p^{2}\cdot C)}\), with a class token and positional embeddings incorporated, resulting in \(\mathbb{R}^{(N+1)\times d}\), where \(N\) is the patch count (in this case, 576) and \(d\) is the embedding size (768). The Transformer encoder processes these to output \(\mathbb{R}^{(N+1)\times d}\), capturing inter-patch dependencies. An MLP head then interprets this feature-rich representation, outputting a classification score that reflects the model's certainty in the pig face identification. **Lightweight Student Model Selection**. For the student model, we opted for the acclaimed lightweight architecture, ShuffleNet-V2, leveraging its capabilities to distill the high-order pig face features inherited from our teacher model, ResNet18. ShuffleNet V2, renowned for its balance between performance and computational efficiency, capitalizes on channel shuffle and pointwise group convolution strategies. This design reduces computational costs while ensuring accuracy. As depicted in Figure 6, the input to ShuffleNet V2 is an image of dimensions 224 \(\times\) 224 across 3 RGB channels. Initial layers process this input through convolution, batch normalization, and a Rectified Linear Unit (ReLU) activation. Subsequent operations further refine these features, reducing spatial dimensions. The core of ShuffleNet V2 lies in its distinct units: Unit 1 and Unit 2. These units integrate operations such as channel splitting, depthwise and pointwise convolutions, and the signature channel  
Figure 4: The MS-SSIM filters similar images in our pig face dataset. shuffling for enhanced cross-channel information flow. The architecture concludes with a series of convolutional and pooling layers, culminating in a fully connected layer. This final layer produces an output corresponding to the 6 distinct pig categories. Figure 5: The architecture of ViT-base as teacher model. Figure 6: The architecture of Shufflenet-V2 as student model. **Relational Knowledge Distillation Technique**. In this study, we explore the efficacy of using Relational Knowledge Distillation (RKD) to transfer knowledge from a large Vision Transformer (ViT) teacher model to a compact ShuffleNet V2 student model.