# Hierarchical Prototypes Polynomial Softmax Loss Function for Visual Classification
## 3 The Proposed Method
### Prototype Learning  
For classification tasks of deep convolutional neural networks, the Softmax loss is usually used as its loss function. Given a classification task has \(C\) classes, the feature vector of each input sample is \(x_{i}\), and the class label of the sample is \(y_{i}\in[1,C]\), then, we can easily calculate cross-entropy loss of this sample:  
\[\ell_{Softmax}(x_{i})=-\log P_{i,y_{i}}=-\log\frac{e^{f_{i}y_{i}}}{\sum_{k=1}^{ C}e^{f_{i},k}} \tag{1}\]  
whereby \(f_{i,j}\) denotes the distance of feature vector \(x_{i}\) while class prototypes \(W_{j}\). \(P_{i,y_{i}}\) denote the probability of \(x_{i}\) being predicted to be its ground-truth class \(y_{i}\). Currently, Euclidean distance \(f_{i,j}=-\alpha\big{\|}x_{i}-W_{j}\big{\|}_{2}^{2}\) and inner product \(f_{i,j}=W_{j}^{T}x_{i}\) are widely used to measure the similarity of feature vector and class prototypes. Essentially, the class prototype is the representative feature vector of the class to which the sample belongs, and usually a good class prototype vector is generally located in the feature center of the class. Although Euclidean distance and inner product distance have good geometric interpretability, visual classification tasks require class prototype vectors and feature vectors to have good separability. However, in actual tasks, due to sample complexity and diversity, especially fine-grained image classification tasks, class prototype feature vectors are poorly separable in the low-dimensional space. To solve this problem, proponents of Support Vector Machines (SVM) have proved that the vector in the low-dimensional space is mapped to the high-dimensional space, so that the feature vector in the low-dimensional space is converted into a separable high-dimensional space. \[f_{i,j}=\big{\langle}\varphi(x_{i}),\varphi\big{(}W_{j}\big{)}\big{\rangle} \tag{2}\]  
where \(\varphi(\bullet):\mathbb{R}^{n}\rightarrow\mathbb{R}^{d}(d\gg n)\) is a non-linear mapping function. In a higher-dimensional space, this method allows the calculation of class prototypes distance \(f_{i,j}\). The computational complexity of this method is much larger than the Euclidean distance or inner product distance in the low-dimensional space, and this shortcoming can be compensated by the kernel method. \[f_{i,j}=\big{\langle}\varphi(x_{i}),\varphi\big{(}W_{j}\big{)}\big{\rangle}= \sum_{g}c_{j}\Big{(}x_{i}^{T}W_{j}\Big{)}^{j}=k\big{(}x_{i},W_{j}\big{)} \tag{3}\]  
where \(k(\cdot,\cdot):\mathbb{R}^{n}\times\mathbb{R}^{n}\rightarrow\mathbb{R}\) is a kernel function. Its computational complexity is the same as the complexity of the inner product of the vector. The coefficient \(c_{j}\) is determined by \(\varphi(\bullet)\) or \(k(\cdot,\cdot)\). \[K_{i,j}=K_{PKF}(x_{i},W_{j})=(\mathrm{x}^{T}\mathrm{w}+c_{p})^{d_{p}}=\sum_{0 }^{d_{p}}c_{p}^{d_{p}-j}(\mathrm{x}^{T}\mathrm{w})^{j} \tag{4}\]  
Figure 2: (**a**) Standard convolution filters panel, (**b**) depth-wise convolution filters, (**c**) point-wise convolution filters. In the figure, \(\mathrm{K}\times\mathrm{K}\) represents the kernel size; \(\mathrm{C}\) represents the number of input channels; \(\mathrm{H}\times\mathrm{W}\) represents the size of spatial information; and \(\mathrm{N}\) represents the number of output channels; \({}^{\star}\) is a convolution operation [26]. where \(K_{PKF}(\cdot,\cdot)\) is a polynomial kernel function. The kernel function we consider in this paper is the polynomial kernel of degree \(d_{p}\). \(c_{p}\) is a constant. \[\ell_{\text{Softmax}}(x_{i})_{\text{PKF-Softmax}}=-\log P_{i,y_{i}}=-\log\frac{e^ {s^{*}(x^{\text{T}}\text{w}+c_{p})^{d_{p}}}}{\sum_{1}^{C}e^{s^{*}(x^{\text{T}} \text{w}+c_{p})^{d_{p}}}} \tag{5}\]  
where the hyperparameter \(s\) is a scale parameter in order to enlarge the range of PKF-scores. In SVM, the polynomial kernel has been successfully used for dealing with natural language processing (NLP) tasks when \(d_{p}=2\)[27]. Regarding the image recognition task, we use \(d_{p}=2\).