# Squeezing Backbone Feature Distributions to the Max for Efficient Few-Shot Learning
## 1 Introduction  
Thanks to their outstanding performance, deep learning methods have been widely considered for vision tasks such as image classification and object detection. In order to reach top performance, these systems are typically trained using very large labeled datasets that are representative enough of the inputs to be processed afterward.  
However, in many applications, it is costly to acquire or annotate data, resulting in the impossibility of creating such large labeled datasets. Under this condition, it is challenging to optimize deep learning architectures considering the fact they typically are made of way more parameters than the dataset can efficiently tune. This is the reason why in the past few years, few-shot learning (i.e., the problem of learning with few labeled examples) has become a trending research subject in the field. In more detail, there are two settings that authors often consider: (a) \"inductive few-shot\", where only a few labeled samples are available during training, and prediction is performed on each test input independently, and (b) \"transductive few-shot\", where prediction is performed on a batch of (non-labeled) test inputs, allowing to take into account their joint distribution.  
Few-shot learning is critical to many applications. To name a few, it has been considered for vision [1; 2; 3], audio [4; 5; 6], language [7; 8; 9], and medical imaging [10; 11; 12]. More generally, few-shot learning can be used to provide proofs-of-concept while limiting the costs of data labeling or to help in pseudo-annotation of datasets. This importance of the problem of few-shot learning explains the abundant literature across the recent years.  
Many works in the domain are built based on a \"learning to learn\" guidance, where the pipeline is to train an optimizer [13; 14; 15] with different tasks of limited data so that the model is able to learn generic experience for novel tasks. Namely, the model learns a set of initialization parameters that are in an advantageous position for the model to adapt to anew (small) dataset. Recently, the trend evolved towards using well-thought-out feature extractors, called backbones [1; 2; 16; 17; 18; 19], that are trained one time on a large generic dataset in order to produce easily classified feature vectors.  
A main problem of the existing methods is that they typically require priors about the data balance between considered classes to perform at their best [1; 20]. These methods could be patched to work efficiently under other regimes but would still require the knowledge of data distribution between classes. In this work, we introduce a new methodology with a double aim: 1--providing state-of-the-art performance, as reported using standardized benchmarks in the field of few-shot learning, and 2--not requiring any priors about data distribution among classes.  
To achieve this goal, we introduce a novel methodology, summarized in Figure 1, that combines feature preprocessing, self-distillation and an optimal transport-based framework. The utility of these ingredients is demonstrated using ablation tests.  
The outline of the paper is as follows. In Section 2, we introduce related work and discuss the novelty of the proposed approach. In Section 3, we introduce the proposed methodology. Section 4 contains several experiments and benchmark results, along with corresponding discussions. Finally, Section 5 presents the conclusion.