# A Large-Class Few-Shot Learning Method Based on High-Dimensional Features
## 3 Methodology
### The Basic Idea of Model Building  
Ref. [15] proved that as long as samples from the same class are compactly distributed in the high-dimensional space and the heterogeneous samples are sufficiently dispersed, the large-class FSL learning model can achieve better generalization. Drawing on this theory, we propose a large-class FSL method based on high-dimensional features. As shown in Figure 1, we first use a deep convolutional neural network to extract high-dimensional features, we then unitize the obtained high-dimensional features, project the sample onto the hypersphere, and then use the global orthogonal regularization strategy to make different categories of samples as orthogonal as possible on the hypersphere, so as to achieve the purpose of compactness within the class and the dispersion between classes. Ref. [34] proves that two independent and uniformly sampled points on the unit sphere are orthogonal with high probability. Let \(x_{i}\) and \(\hat{x}_{i}\) be two points from different classes sampled from the unit sphere in \(d\)-dimensional space. The mean and the second moment of \(\phi(x_{i})^{T}\phi(\hat{x}_{i})\) are, respectively,  
\[\mathbb{E}(\phi(x_{i})^{T}\phi(\hat{x}_{i}))=0\text{ and }\mathbb{E}((\phi(x_{i})^{T}\phi( \hat{x}_{i}))^{2})=\frac{1}{d}. \tag{1}\]  
This paper draws on this idea and characterizes the orthogonality of two samples on the sphere through Equation (1). We define a global orthogonal regularization term as follows:  
\[ \begin{split}\mathcal{L}_{GOR}=&(\frac{1}{n}\sum_{ i=1}^{n}\phi(x_{i})^{T}\phi(\hat{x}_{i}))^{2}\\ &+\max\{0,\frac{1}{n}\sum_{i=1}^{n}(\phi(x_{i})^{T}\phi(\hat{x}_ {i}))^{2}-\frac{1}{d}\}.\end{split}  \tag{2}\]It can be seen that the regular term tries to make the mean value of \(\phi(x_{i})^{T}\phi(\bar{x}_{i})\) equal to zero, and the second moment equal to the reciprocal of the dimension number of the feature space, that is, the feature embedding of different classes are orthogonal to each other. In this way, the samples from different classes are fully dispersed, and those from same class are compactly distributed on the feature sphere. Since the normalized feature embeddings are distributed on a hypersphere, in order to describe the relationships between the samples distributed on the sphere, we adopt AM-Softmax Loss [33], which is based on the angular distance metric. The AM-Softmax Loss function can be formalized as follows:  
\[\mathcal{L}_{AM}=-\frac{1}{n}\sum_{i=1}^{n}\log\frac{e^{\bar{s}\cdot(cos\theta _{y_{i}j}-m_{ft})}}{e^{\bar{s}\cdot(cos\theta_{y_{i}j}-m_{ft})}+\Sigma_{k=1,k \
eq y_{i}}e^{\bar{s}\cdot cos\theta_{k,i}}} \tag{3}\]  
where \(cos\theta_{y_{i}j}\) represents the angular distance between the feature vector of the \(i\)-th sample in the support set, \(S_{spt}\), and the weight vector corresponding to the ground truth class of the \(i\)-th sample in the classifier weight matrix. The value \(m_{ft}\) represents the hyperparameter of the angular interval, and we set the scaling factor, \(s\), to a learnable parameter. Combining Equations (2) and (3), the loss function of HF-FSL is  
\[ \begin{split}\mathcal{L}=&\mathcal{L}_{AM}+\lambda _{ft}\cdot\mathcal{L}_{GOR}\\ =&-\frac{1}{n}\sum_{i=1}^{n}\log\frac{e^{\bar{s} \cdot(cos\theta_{y_{i}j}-m_{ft})}}{e^{\bar{s}\cdot(cos\theta_{y_{i}j}-m_{ft})} +\Sigma_{k=1,k\
eq y_{i}}e^{\bar{s}\cdot cos\theta_{k,i}}}\\ &+\lambda_{ft}\cdot((\frac{1}{n}\sum_{i=1}^{n}\phi(x_{i})^{T} \phi(\bar{x}_{i}))^{2}\\ &+\max\{0,\frac{1}{n}\sum_{i=1}^{n}\big{(}\phi(x_{i})^{T}\phi( \bar{x}_{i})\big{)}^{2}-\frac{1}{d}\}),\end{split}  \tag{4}\]  
where the hyperparameter, \(\lambda_{ft}\), is the weight coefficient of the regular term. Figure 1: HF-FSL architecture. HF-FSL uses an task-independent auxiliary dataset, \(D_{base}\), for pretraining. In the imprinting stage HF-FSL only uses support set, \(S_{spt}\).