# Squeezing Backbone Feature Distributions to the Max for Efficient Few-Shot Learning
## 3 Materials and Methods
### Feature Preprocessing  
As mentioned in Section 2, many works hypothesize, explicitly or not, that the features from the same class are aligned with a specific distribution (often Gaussian-like). However, this aspect is rarely experimentally verified. In fact, it is very likely that features obtained using the backbone architecture are not Gaussian. Indeed, usually, the features are obtained after applying a ReLU function [39] and exhibit a positive and yet skewed distribution mostly concentrated around 0 (more details can be found in the next section). Multiple works in the domain [20; 35] discuss the different statistical methods (e.g., batch normalization) to better fit the features into a model. Although these methods may have provable assets for some distributions, they could worsen the process if applied to an unexpected input distribution. This is why we propose to preprocess the obtained raw feature vectors so that they better align with typical distribution assumptions in the field. Denote \(f_{\varphi}(\mathbf{x})=[f_{\varphi}^{h}(\mathbf{x}),\ldots,f_{\varphi}^{h}( \mathbf{x}),\ldots,f_{\varphi}^{d}(\mathbf{x})]\in(\mathbb{R}^{+})^{d}, \mathbf{x}\in\mathbf{D}_{novel}\) as the obtained features on \(\mathbf{D}_{novel}\), and let \(f_{\varphi}^{h}(\mathbf{x}),1\leq h\leq d\) denote its value in the hth position. The preprocessing methods applied in our proposed algorithms are as follows:  
(E) Euclidean normalization. Also known as L2-normalization, which is widely used in many related works [19; 35; 37], this step scales the features to the same area so that large variance feature vectors do not predominate the others. Euclidean normalization can be given by:  
\[f_{\varphi}(\mathbf{x})\leftarrow\frac{f_{\varphi}(\mathbf{x})}{\|f_{\varphi} (\mathbf{x})\|_{2}} \tag{1}\]  
(P) Power transform. The power transform method [1; 40] simply consists of taking the power of each feature vector coordinate. The formula is given by:  
\[f_{\varphi}^{h}(\mathbf{x})\leftarrow(f_{\varphi}^{h}(\mathbf{x})+\epsilon)^{ \beta},\ \ \beta\
eq 0 \tag{2}\]  
where \(\epsilon=1\times 10^{-6}\) is used to make sure that \(f_{\varphi}(\mathbf{x})+\epsilon\) is strictly positive in every position, and \(\beta\) is a hyper-parameter. The rationale of the preprocessing above is that power transform, often used in combination with euclidean normalization, has the functionality of reducing the skew of the distribution and mapping it to a close-to-Gaussian distribution, adjusted by \(\beta\). After experiments, we found that \(\beta=0.5\) gives the most consistent results for our considered experiments, which corresponds to a square-root function that has a wide range of usage on features [41]. We will analyze this ability and the effect of power transform in more detail in Section 4. Note that power transform can only be applied if considered feature vectors contain nonnegative entries, which will always be the case in the remainder of this work. (M) Mean subtraction. With mean subtraction, each sample is translated using \(\mathbf{m}\in\left(\mathbb{R}^{+}\right)^{d}\), the projection center. This is often used in combination with euclidean normalization in order to reduce the task bias and better align the feature distributions [20]. The formula is given by:  
\[f_{\varphi}(\mathbf{x})\gets f_{\varphi}(\mathbf{x})-\mathbf{m} \tag{3}\]  
The projection center is often computed as the mean values of feature vectors related to the problem [20; 35]. In this paper, we compute it either as the mean feature vector of the base dataset (denoted as \(\mathrm{M_{b}}\)) or the mean vector of the novel dataset (denoted as \(\mathrm{M_{n}}\)), depending on the few-shot settings. Of course, in both of these cases, the rationale is to consider a proxy to what would be the exact mean value of feature vectors on the considered task. In our proposed method, we deploy these preprocessing steps in the following order: Power transform (P) on the raw features, followed by a Euclidean normalization (E).