# Few-Shot Image Classification via Mutual Distillation
## 4 Experiments
### Ablation Study  
In this section, a series of experiments are conducted to evaluate the effectiveness of our algorithm, which is based on the concept of mutual distillation. Consequently, we will compare the performance of metric-based meta-learning models before and after the application of mutual distillation. As delineated in Section 3.2, there exist two distinct forms of similarity metrics. The typical model of one form is a prototypical network (PN), andthe typical model of the other form is a relation network (RN). To ensure a fair comparison, we re-implement the results of PN and RN with the same setting and investigate the effectiveness of mutual distillation in the following two respects. **Individual networks with different similarity metrics. ** We examine the effects of mutual distillation between PN and RN, which employ distinct similarity metrics. In this configuration, the individual networks involved in mutual distillation are denoted as MDMM-PN and MDMM-RN. The results are presented in Table 1, where the following observations can be made:  
(1) Except for the 1-shot task on MiniImageNet, PN outperforms RN under all other conditions. This discrepancy suggests that the non-linear metric module with learnable parameters in PN may make it more susceptible to overfitting. (2) After the mutual distillation of PN and RN, the accuracy of RN noticeably improves. For instance, on the CUB dataset, RN's accuracy increases from 72.69% to 74.19% for the 1-shot task and from 86.98% to 91.03% for the 5-shot task. Similar trends are observed on MiniImageNet, where RN's accuracy rises from 54.68% to 54.93% for the 1-shot task and from 71.61% to 73.58% for the 5-shot task. This analysis highlights that PN can effectively serve as a regularizer for RN, and the process of mutual distillation substantially enhances the performance of metric-based meta-learning models. **Individual networks with same similarity metrics. ** In this section, we delve into the impact of mutual distillation between individual networks that share the same form of similarity metrics. Specifically, we explore mutual distillation between two prototypical networks and two relation networks, denoted as MDMM-PN&PN and MDMM-RN&RN, respectively. The outcomes of the mutual distillation process are summarized in Table 2, and the following observations are made:  
(1) On MiniImageNet, MDMM-PN&PN achieves 56.93% accuracy for the 1-shot task and 77.33% for the 5-shot task. These results signify improvements of 1.54% and 2.06%, respectively, compared to using the prototypical network (PN) in isolation. Similarly, on the CUB dataset, MDMM-PN&PN demonstrates an accuracy of 76.50% for the 1-shot task and 91.29% for the 5-shot task, showcasing enhancements of 0.09% and 1.89%, respectively, compared to the performance of PN individually. This analysis highlights that individual prototypical networks in mutual distillation effectively supply each other with a regularization term. (2) MDMM-RN&RN achieves 58.09% accuracy for 1-shot tasks and 72.99% for 5-shot tasks on MiniImageNet, showing a respective improvement of 1.54% and 1.38% compared to RN. On CUB, MDMM-RN&RN performs at 74.64% for 1-shot tasks and 87.49% for 5-shot tasks, demonstrating improvements of 1.95% and 0.51% for the respective task types. This analysis highlights the effectiveness of mutual distillation between individual RN models, as they provide each other with valuable regularization terms. Ablation results further