# Few-Shot Image Classification via Mutual Distillation
###### Abstract  
Due to their compelling performance and appealing simplicity, metric-based meta-learning approaches are gaining increasing attention for addressing the challenges of few-shot image classification. However, many similar methods employ intricate network architectures, which can potentially lead to overfitting when trained with limited samples. To tackle this concern, we propose using mutual distillation to enhance metric-based meta-learning, effectively bolstering model generalization. Specifically, our approach involves two individual metric-based networks, such as prototypical networks and relational networks, mutually supplying each other with a regularization term. This method seamlessly integrates with any metric-based meta-learning approach. We undertake comprehensive experiments on two prevalent few-shot classification benchmarks, namely miniImageNet and Caltech-UCSD Birds-200-2011 (CUB), to demonstrate the effectiveness of our proposed algorithm. The results demonstrate that our method efficiently enhances each metric-based model through mutual distillation. metric-based meta-learning; knowledge distillation; few-shot image classification +
Footnote †: journal: Applied  
0000-0001-9993-3300-3300-3300-3300-3300-3300-3300-3300-3300-3300-3300-3300-3300-3300-3300-33000-3300-33000-33000-33000-33000-33000-330  
types of meta-learning methods have collectively propelled the field of few-shot image classification forward. In recent years, metric-based meta-learning techniques have demonstrated increasing effectiveness in tackling the challenges associated with few-shot image classification. These approaches provide an alluring combination of straightforwardness and exceptional efficiency [8]. The first method used in this field was Siamese networks [9], which use paired networks to represent support and query images individually, using a weighted L1 distance metric to measure their similarity. Following that, [PERSON] et al. [10] presented matching networks, which integrate memory and attention mechanisms to extract unique characteristics for support and query samples. They additionally implemented a comprehensive differentiable KNN classifier to generate predictions about the categories of query samples. Expanding upon this groundwork, [PERSON] et al. [11] introduced prototypical networks, which are an expansion of the matching networks framework. The indication was that samples belonging to the same class tended to form a cluster around a singular prototype representation. Following that, the query samples were classified according to their closeness to the closest prototype. In contrast to the previously mentioned approaches, [PERSON] et al. [12] presented relational networks, which simultaneously acquired feature embeddings and non-linear metrics. The feature embedding and similarity metric were both highlighted by these prominent models in metric-based meta-learning. It is worth mentioning that the recent progress in this domain has resulted in substantial enhancements in these two essential elements [13; 14; 15; 16]. Few-shot image classification has been greatly improved by the above approaches. However, the key issue with few-shot learning is that the data of each class is not enough to express a concept adequately. The training of end-to-end networks in these meta-learning approaches involves utilizing sampled mini-batches referred to as episodes. Within each episode, the training is conducted with a limited number of labeled examples per class. Initially, these approaches utilize backbone networks with low complexity, such as four-layer convolutional networks, to mitigate the risk of overfitting. To improve classification performance, more recent work has relied on a higher complexity network as the base learner to enhance feature representation. Neural networks such as Resnet12 and WRN28 are increasingly being employed as backbone networks in metric-based meta-learning techniques.