# Squeezing Backbone Feature Distributions to the Max for Efficient Few-Shot Learning
## 3 Materials and Methods
### Boosted Min-Size Sinkhorn
```
Inputs:\(\mathbf{C},\mathbf{p}=\mathbf{1}_{l+\mathbf{u}},\mathbf{q}=k\mathbf{1}_{n},\lambda\) Initializations:\(\mathbf{P}=Softmax(-\lambda\mathbf{C})\) for\(iter=1\) to 50 do \(\mathbf{P}[i,:]\leftarrow\mathbf{p}[i]\cdot\frac{\mathbf{P}[i,:]}{\mathbf{P}[i, :]},\forall i\) \(\mathbf{P}[:,j]\leftarrow\mathbf{q}[j]\cdot\frac{\mathbf{P}[:,j]}{\sum_{i} \mathbf{P}[i,j]}\) if \(\sum_{i}\mathbf{P}[i,j]<\mathbf{q}[j],\forall j\) endfor return\(\mathbf{P}\)
```  
**Algorithm 1** Min-size Sinkhorn  
Updating weights  
This step is considered as the \(M\)-step of the proposed algorithm, in which we use a variant of the logistic regression algorithm in order to find the model parameter \(\theta\) in the form of weight parameters \(\mathbf{w}_{j}\) for each class. Note that \(\mathbf{w}_{j}\), if normalized, is equivalent to the prototype for class \(j\) in this case. Given the fact that in Equation (4), we also take into account the marginal probability, it can be further broken down as:  
\[P(\mathbf{x}_{i};\theta)=\sum_{j}P(\mathbf{x}_{i}|l(\mathbf{x}_{i})=j;\theta)P (l(\mathbf{x}_{i})=j), \tag{9}\]  
We observe that Equation (4) corresponds to applying a softmax function on the negative logits computed through an L2-distance function between samples and class prototypes (normalized). This fits the formulation of a linear hypothesis between \(f_{\varphi}(\mathbf{x}_{i})\) and \(\mathbf{w}_{j}\) for logit calculations, hence the rationale for utilizing logistic regression in our proposed method. Note that contrary to classical logistic regression, we implement here a form of self-distillation. Indeed, we use soft labels contained in \(\mathbf{P}\) instead of one-hot class indicator targets, and these targets are refined iteratively. The procedure of this step is as follows: now that we have a polished allocation matrix \(\mathbf{P}\), we firstly initialize the weights \(\mathbf{w}_{j}\) as follows:  
\[\mathbf{w}_{j}\leftarrow\mathbf{u}_{j}/\|\mathbf{u}_{j}\|_{2}, \tag{10}\]  
where  
\[\mathbf{u}_{j}\leftarrow\sum_{i}\mathbf{P}[i,j]f_{\varphi}(\mathbf{x}_{i})/ \sum_{i}\mathbf{P}[i,j]. \tag{11}\]  
We can see that elements in \(\mathbf{P}\) are used as coefficients for feature vectors to linearly adjust the class prototypes [1]. Similar to Equation (6), here \(\mathbf{w}_{j}\) is the normalized newly-computed class prototype that is a vector of length 1. Next, we further adjust weights by applying a logistic regression, and the optimization is performed by minimizing the following loss:  
\[\frac{1}{l+u}\cdot\sum_{i}\sum_{j}\ -\ \log(\frac{\exp\left(\mathbf{S}[i,j] \right)}{\sum_{i=1}^{n}\exp\left(\mathbf{S}[i,\gamma]\right)})\cdot\mathbf{P} [i,j], \tag{12}\]where \(\mathbf{S}\in\mathbb{R}^{(l+u)\times n}\) contains the logits, and each element is computed as:  
\[\mathbf{S}[i,j]=\kappa\cdot\frac{\mathbf{w}_{j}^{\top}f_{\varphi}(\mathbf{x}_{i} )}{\|\mathbf{w}_{j}\|_{2}}. \tag{13}\]  
Note that \(\kappa\) is a scaling parameter, it can also be seen as a temperature parameter that adjusts the confidence metric to be associated with each sample. It is learnt jointly with \(\mathbf{W}\). The deployed logistic regression comes with hyperparameters on its own. In our experiments, we use an SGD optimizer with a gradient step of 0.1 and 0.8 as the momentum parameter, and we train over \(e\) epochs. Here, we point out that \(e\geq 0\) is considered an influential hyperparameter in our proposed algorithm, \(e=0\) indicates a simple update of \(\mathbf{W}\) as the normalized adjusted class prototypes (Equation (10)) computed from \(\mathbf{P}\) in Equation (11), without further adjustment of logistic regression. In addition, note that when \(e>0\), we project columns of \(\mathbf{W}\) to the unit hypersphere at the end of each epoch. * Estimating the class minimum size  
We can now refine our estimate for the min-size \(k\) for the next iteration.