# Squeezing Backbone Feature Distributions to the Max for Efficient Few-Shot Learning
## 3 Materials and Methods
### Boosted Min-Size Sinkhorn  
In the case of transductive few-shot, we introduce a method that consists of iteratively refining estimates for the probability each unlabeled sample belongs to any of the considered classes. This method is largely based on the one we introduced in [1], except it does not require priors about sample distributions in each of the considered classes. Denoting \(i\in[1,\dots,l+u]\) as the sample index in \(\mathbf{D}_{novel}\) and \(j\in[1,\dots,n]\) as the class index, the goal is to maximize the following log post-posterior function:  
\[L(\theta)=\sum_{i}\log P(l(\mathbf{x}_{i})=j|\mathbf{x}_{i};\theta)\] \[= \sum_{i}\log\frac{P(\mathbf{x}_{i},l(\mathbf{x}_{i})=j;\theta)}{ P(\mathbf{x}_{i};\theta)} \tag{4}\] \[\propto \sum_{i}\log\frac{P(\mathbf{x}_{i}|l(\mathbf{x}_{i})=j;\theta)}{ P(\mathbf{x}_{i};\theta)},\]  
Here, \(l(\mathbf{x}_{i})\) denotes the class label for sample \(\mathbf{x}_{i}\in\mathbf{Q}\cup\mathbf{S}\), \(P(\mathbf{x}_{i};\theta)\) denotes the marginal probability, and \(\theta\) represents the model parameters to estimate. Assuming a Gaussian distribution on the input features for each class, here we define \(\theta=\mathbf{w}_{j},\forall j\) where \(\mathbf{w}_{j}\in\mathbb{R}^{d}\) stand for the weight parameters for class \(j\). We observe that Equation (4) can be related to the cost function utilized in optimal transport [42], which is often considered to solve classification problems, with constraints on the sample distribution over classes. To that end, a well-known Sinkhorn [43] mapping method is proposed. The algorithm aims at computing a class allocation matrix among novel class data for a minimum Wasserstein distance. Namely, an allocation matrix \(\mathbf{P}\in\mathbb{R}_{+}^{(l+u)\times n}\) is defined where \(\mathbf{P}[i,j]\) denotes the assigned portion for sample \(i\) to class \(j\), and it is computed as follows:  
\[\mathbf{P} =\text{Sinkhorn}(\mathbf{C},\mathbf{p},\mathbf{q},\lambda)\] \[=\underset{\mathbf{p}_{\text{c}}\in\mathbb{U}(\mathbf{p},\mathbf{ q})}{\text{argmin}}\sum_{ij}\mathbf{P}[i,j]\mathbf{C}[i,j]+\lambda H( \mathbf{P}), \tag{5}\]  
where \(\mathbb{U}(\mathbf{p},\mathbf{q})\in\mathbb{R}_{+}^{(l+u)\times n}\) is a set of positive matrices for which the rows sum to \(\mathbf{p}\) and the columns sum to  
the right of (5), it stands for the entropy of \(\mathbf{\tilde{P}}\): \(H(\mathbf{\tilde{P}})=-\sum_{ij}\mathbf{\tilde{P}}[i,j]\log\mathbf{\tilde{P}}[i,j]\), regularized by a hyper-parameter \(\lambda\). Increasing \(\lambda\) would force the entropy to become smaller, so that the mapping is less diluted. This term also makes the objective function strictly convex [43; 44] and thus a practical and effective computation. From lemma 2 in [43], the result of the Sinkhorn allocation has the typical form \(\mathbf{P}=\mathrm{diag}(\mathbf{u})\cdot\exp(-\mathbf{C}/\lambda)\cdot \mathrm{diag}(\mathbf{v})\). It is worth noting that here we assume a soft class allocation, meaning that each sample can be \"sliced\" into different classes. We will present our proposed method in detail in the following paragraphs. Given all that is presented above, in this paper, we propose an Expectation-Maximization (_EM_) [45] based method, which alternates between updating the allocation matrix \(\mathbf{P}\) and estimating the parameter \(\theta\) of the designed model, in order to minimize Equation (5) and maximize Equation (4). For a starter, we define a weight matrix \(\mathbf{W}\) with \(n\) columns (i.e., one per class) and \(d\) rows (i.e., one per dimension of feature vectors), and for column \(j\) in \(\mathbf{W}\), we denote it as the weight parameters \(\mathbf{w}_{j}\in\mathbb{R}^{d}\) for class \(j\) in correspondence with Equation (4).