# Few-Shot Image Classification via Mutual Distillation
## 2 Related Work  
The task of few-shot image classification focuses on recognizing novel categories with a limited number of instances available for each class. Recent research has mainly addressed this challenge through meta-learning, which has strong generalization capabilities. In essence, meta-learning acquires meta-knowledge from analogous tasks to aid in handling previously unseen tasks. These methods can be broadly categorized into three groups depending on the nature of their meta-knowledge: (1) model-based techniques; (2) optimization-based techniques; and (3) metric-based techniques. Our study matches the third category. Metric-based meta-learning methods have a primary objective: to learn an embedding space where samples from the same class exhibit proximity, while samples belonging to distinct classes are positioned distantly, facilitating effective few-shot learning. Siamese networks [9] play a crucial role in this process by comparing the likeness between a query image and support images within the acquired embedding space. These networks undergo training using a triplet loss, which emphasizes the relative distances between the anchor (query image), positive, and negative samples. Matching networks [10] employ attention LSTM and bidirectional LSTM to extract features from both support and query sets. The classification task is then facilitated by comparing the extracted features using cosine similarity. Prototypical networks [11] improved upon matching networks by computing the dissimilarities between a query image and the prototype of each class with Euclidean distance. Different from the above works that used a pre-specified distance metric, relational networks [12] were proposed to learn a deep non-linear metric for comparing the connection between the query image and support images in the embedding space. From the above well-known models, we can see that metric-based methods are characterized by two fundamental elements: a feature extractor and a metric module. A great  
Figure 1: Illustration of our motivation under 5-way, 5-shot few-shot task. Class boundary 1 and class boundary 2 are, respectively, produced by the metric-based models in mutual distillation. As shown, our method can push class boundary 1 and boundary 2 to the optimal class boundary. deal of work is being carried out around these two key aspects. On one hand, the ongoing research focus is directed toward enhancing the discriminative representations generated by the feature extractor. [PERSON] et al. [20] introduced a deformable feature extractor to address the sensitivity of CNN-based networks to spatial location relationships between semantic objects in comparative images. This innovative approach is complemented by a dual correlation attention mechanism designed to enhance local connectivity in the extracted features, contributing to improved discriminative power. [PERSON] et al. [21] introduced a cross-attention module that produces cross-attention maps for each pair of class features and query sample features. This approach accentuates target object areas, thereby augmenting the discriminative capabilities of the extracted features. [PERSON] et al. [13] introduced a Category Traversal Module, an advanced mechanism aimed at discerning features relevant to the task by considering both intra-class commonality and inter-class distinctiveness within the feature space. [PERSON] et al. [22] introduced the deep subspace network (DSN), showcasing its effectiveness in generating expressive representations across a broad spectrum of supervised few-shot image classification tasks. [PERSON] et al. [8] introduced an adaptive margin principle for learning more discriminating embedding spaces with better generalization ability. [PERSON] et al. [23] introduced a novel embedding structure that encodes relative spatial relationships between features, achieved through the application of a capsule network. Conversely, the choice of metric holds a crucial role in metric-based methods. As an instance, [PERSON] et al.