# Squeezing Backbone Feature Distributions to the Max for Efficient Few-Shot Learning
## 4 Results and Discussions
### Ablation Studies
To better show the effect of this proposed technique on feature distributions, we depict in Figure 2 the distributions of an arbitrarily selected feature for three randomly selected novel classes of miniImageNet when using WRN, before and after applying the power transform. In addition, we also added to the figure the feature distributions after applying batch normalization for comparison purposes. We observe quite clearly that (1) raw features exhibit a positive distribution mostly concentrated around 0, a similar behavior is also observed for batch norm, and (2) power transform is able to reshape the feature distributions to close-to-Gaussian distributions. We observe similar behaviors with other datasets as well. Moreover, in order to visualize the impact of this technique with respect to the position of feature points, in Figure 3, we plot the feature vectors of three randomly selected classes from \(\mathbf{D}_{novel}\). Note that all feature vectors in this experiment are reduced to 3-dimensional ones corresponding to their largest eigenvalues. From Figure 3, we can observe that the power transform, often followed by an L2-normalization, can help shape the class distributions to become more gathered and Gaussian-like [57]. Influence of the number of unlabeled samples: in order to better understand the gain in accuracy due to having access to more unlabeled samples, we depict in Figure 4 the evolution of accuracy as a function of \(q\), when the number of classes \(n=5\) is fixed. Interestingly, the accuracy quickly reaches a close-to-asymptotical plateau, emphasizing the ability of the method to quickly exploit available information in the task. \begin{table}
\begin{tabular}{c c c c c} \hline \hline  & \multicolumn{2}{c}{**Inductive (NCM)**} & \multicolumn{2}{c}{**Transductive (BMS)**} \\
**Preprocessing** & **1-Shot** & **5-Shot** & **1-Shot** & **5-Shot** \\ \hline None & \(55.30\pm 0.21\%\) & \(78.34\pm 0.15\%\) & \(77.62\pm 0.26\%\) & \(87.96\pm 0.13\%\) \\ Batch Norm [60] & \(66.81\pm 0.20\%\) & \(83.57\pm 0.13\%\) & \(73.74\pm 0.21\%\) & \(88.07\pm 0.13\%\) \\ L2N [35] & \(65.37\pm 0.20\%\) & \(83.46\pm 0.13\%\) & \(73.84\pm 0.21\%\) & \(88.15\pm 0.13\%\) \\ CL2N [35] & \(63.88\pm 0.20\%\) & \(80.85\pm 0.14\%\) & \(73.12\pm 0.28\%\) & \(86.47\pm 0.15\%\) \\ EM\({}_{\text{b}}\)E & \(68.05\pm 0.20\%\) & \(83.76\pm 0.13\%\) & \(80.28\pm 0.26\%\) & \(88.36\pm 0.13\%\) \\ PEM\({}_{\text{b}}\)E & \(\mathbf{68.43\pm 0.20\%}\) & \(\mathbf{84.67\pm 0.13\%}\) & \(82.01\pm 0.26\%\) & \(89.50\pm 0.13\%\) \\ EM\({}_{\text{n}}\)E & \(\backslash\) & \(\backslash\) & \(80.14\pm 0.27\%\) & \(88.39\pm 0.13\%\) \\ PEM\({}_{\text{n}}\)E & \(\backslash\) & \(\backslash\) & \(\mathbf{82.07\pm 0.25\%}\) & \(\mathbf{89.51\pm 0.13\%}\) \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Comparison of 1-shot and 5-shot accuracy on miniImageNet (backbone: WRN) when using various preprocessing steps on the extracted features. Best results are in bold. Figure 2: Distributions of an arbitrarily chosen feature for 3 novel classes with different preprocessing techniques: raw (**left**), batch norm (**middle**) and power transform (**right**). Influence of hyperparameters: in order to test how much impact the hyperparameters could have on our proposed method in terms of prediction accuracy, here we select two important hyperparameters that are used in BMS and observe their impact. Namely, the number of training epochs \(e\) in logistic regression and the regulation parameter \(\lambda\) used for computing the prediction matrix \(\mathbf{P}\). In Figure 5, we show the accuracy of our proposed method as a function of \(e\) (top) and \(\lambda\) (bottom). Results are reported for BMS* in 1-shot settings, and for BMS in 5-shot settings. From the figure, we can see a slight uptick of accuracy as \(e\) or \(\lambda\) increase, followed by a downhill when they become larger, implying an overfitting of the classifier. We chose our optimal parameters from these experiments.