# Hierarchical Prototypes Polynomial Softmax Loss Function for Visual Classification
## 4 Experimental Results and Discussion
### Experimental Setup  
To verify the performance of the proposed loss function on image classification tasks, we used the well-known Pytorch deep learning architecture [33]. The experiment was performed on the UltraLAB graphics workstation that was equipped with 192 GB memory and 8 NVIDIA GTX-2080 graphics processors. Each graphics card had 8 GB memory. The workstation used the Windows server operating system.  
To verify the performance of the proposed loss function in training lightweight networks, we selected various well-known lightweight convolutional neural networks. Due to the use of depth-with separable convolutions, lightweight neural networks greatly reduce the number of parameters and calculations. Their main characteristics are introduced below.  
We adopted the cosine learning rate schedule for training of all models. The initial learning rate was 0.05, and was scheduled to reach zero within a single cosine period for 300 epochs. The models used SGD with a momentum of 0.9 and a weight decay of \(10^{-4}\) for optimization. To prevent the model from overfitting during the training process, we used the same data enhancement method for all datasets. First, the enhancement method adjusts the size of the picture to 256 \(\times\) 256, then performs random cropping, and a horizontal flipping operation. The batch size of ImageNet-100 and Stanford dogs was set to 96, while the batch size of Stanford cars and CUB-200-2011 was 16.  
First, three lightweight nets were used for comparative experiments on four datasets, as shown in Table 1. Each lightweight network uses different network widths, \(\times\)1.0 represents the original model, \(\times\)0.75 shows that the number of channels of the convolutional layer of the model is 0.75 times that of the original model, and \(\times\)0.5 is 0.5 times that of the original model. Orig means that the network uses standard cross-entropy for testing, and ours means that the network was trained using the hierarchical polynomial loss function we proposed. Table 1 shows that the network trained by our proposed hierarchical polynomial  
Figure 4: Sample instance images of the datasets.  
loss function does not increase the amount of calculation and parameter in the inference stage of the lightweight network. The accuracy of the three different lightweight networks is in different data. The accuracy of the three lightweight networks has been greatly improved, especially on the Stanford dogs and the CUB-200 dataset. The result is a good verification that our proposed loss function can enable the network to learn better features.  
In addition to the standard cross-entropy loss function, we compared the proposed loss function with other kinds of loss functions such as the central loss function and the contrast loss function, etc. on the ImageNet100 dataset. To fairly compare the effects of different loss functions on the network performance, we set the same hyperparameters to train the network. Table 2 shows that the network trained by our proposed hierarchical polynomial loss function has a great improvement compared to other loss functions. This is because the proposed loss function can effectively enhance the gradient return in the back propagation stage, and solve the problem of gradient disappearance caused by very small gradients.