# Hierarchical Prototypes Polynomial Softmax Loss Function for Visual Classification
## 4 Experimental Results and Discussion
### Ablation Study  
The part of the ablation experiment in which the loss function we proposed was added to different stages of the network is shown in Table 3. In the experiment, we adopted three kinds of lightweight models, and added the proposed loss function at stage 5, stage 4, stage 3, stage 2 and the FC layer, respectively. Compared to the original cross-entropy loss performance, the hierarchical polynomial loss function, when the proposed loss function was added to the three layers of stage 3, stage 2, and the FC at the same time, the trained network performance was better. Of course, it is not advisable to add loss functions to all layers. In the shallow part of the network, it is necessary to extract more low-level visual features, while in the higher layers of the network, more attention should be paid to semantic features. \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline \multirow{2}{*}{**Model**} & \multicolumn{2}{c}{**ImageNet-100**} & \multicolumn{2}{c}{**Stanford Dogs**} & \multicolumn{2}{c}{**Stanford Cars**} & \multicolumn{2}{c}{**CUB-200-2011**} \\ \cline{2-9}  & **Orig** & **Ours** & **Orig** & **Ours** & **Orig** & **Ours** & **Orig** & **Ours** \\ \hline Mobilenetv2 (\(\times\)0.5) & 74.17 & 74.84 & 44.56 & 52.63 & 83.73 & 85.25 & 52.36 & 54.46 \\ Mobilenetv2 (\(\times\)0.75) & 76.87 & 78.71 & 45.81 & 56.33 & 85.81 & 86.62 & 59.47 & 61.60 \\ Mobilenetv2 (\(\times\)1.0) & 78.97 & 81.05 & 48.43 & 60.42 & 87.07 & 87.97 & 61.28 & 66.08 \\ Mobilenetv3-small (\(\times\)0.5) & 65.72 & 65.98 & 33.42 & 40.30 & 72.92 & 75.04 & 42.85 & 47.54 \\ Mobilenetv3-small (\(\times\)0.75) & 67.87 & 68.40 & 38.59 & 44.15 & 74.99 & 77.94 & 52.15 & 55.63 \\ Mobilenetv3-small (\(\times\)1.0) & 70.00 & 72.15 & 46.51 & 47.45 & 79.10 & 80.69 & 52.39 & 56.65 \\ Mobilenetv3-large (\(\times\)0.5) & 71.58 & 74.37 & 41.83 & 51.42 & 80.45 & 84.80 & 49.59 & 58.76 \\ Mobilenetv3-large (\(\times\)0.75) & 75.81 & 77.30 & 45.27 & 55.53 & 80.61 & 85.23 & 52.63 & 60.12 \\ Mobilenetv3-large (\(\times\)1.0) & 76.91 & 78.56 & 47.01 & 57.31 & 82.48 & 86.01 & 55.73 & 62.68 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Comparisons of hierarchical polynomial loss function and cross-entropy loss function, Orig represents the standard cross-entropy used by the network while ours represents the loss proposed in this article. The result is the top 1 of the test set, the unit is %. \begin{table}
\begin{tabular}{c c c c} \hline \hline \multirow{2}{*}{**Model**} & \multicolumn{2}{c}{**ImageNet100**} \\ \cline{2-5}  & **Original Softmax Loss** & **RBF-Softmax** & **Qurs** \\ \hline Mobilenetv2 (\(\times\)1.0) & 78.97 & 79.51 & 81.05 \\ Mobilenetv3-small (\(\times\)1.0) & 70.00 & 70.68 & 72.15 \\ Mobilenetv3-large (\(\times\)1.0) & 76.91 & 77.31 & 78.56 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Comparison with other loss functions. RBF denotes radial basis loss function. The result is the top 1 of the test set, the unit is %. According to Equation (6), the high power and constant term of the polynomial have different effects on the loss calculation. To explore its impact on the network performance, we used two datasets, Mobilenetv2 and Mobilenetv3-small on ImageNet100 and Stanford-dogs for experimentation. The alpha in Table 4 represents different high powers, while gamma represents different constant terms. It is shown that when the high power is set to 2 and the constant term is set to 1, the reasoning performance of the network is the best. As mentioned earlier, the hierarchical loss function we propose can be used to calculate the corresponding loss at different stages of the network. However, the output of each stage of the network is not a logit; therefore, loss function cannot be directly calculated. It is thus necessary to reduce the dimension of the feature map at each stage to obtain its feature vector, and then find the distance from the class prototype vector to obtain its logit and label calculation loss. Maximum pooling and average pooling methods can be used to reduce the dimensionality of the feature map.