# Learning a Hierarchical Global Attention for Image Classification
## 5 Experiments  
### Results  
To demonstrate the capacity of HGA, we evaluate the image classification performance on ImageNet [11] 2012 dataset. The ImageNet dataset contains 1.28 million images for training and  
Figure 2: Applications of HGA on different network designs. (**a**) The vanilla ResBlock [15]. (**b**) The application of HGA on ResBlock. (**c**) The vanilla Inception Block [14]. (**d**) The application of HGA on Inception Block. around 50,000 images for validation, which cover 1000 different classes. Top-1 and top-5 error are considered as the indicators of classification capacity. The images are augmented by randomly cropping, flipping, and rotation. Training data are resized as \(224\times 224\) to suit the origin settings of baseline network. The parameters are updated by SGD optimizer with learning rate as \(lr=0.6\) and shrunk 10 times for every 30 epochs. We totally update the model for 100 epochs. For a better representation of the HGA embedding method, we provide the details on two modern backbones, which is shown in Table 1. As there are a large number of filters in the deeper layers, a linear transformation is applied for dimension reduction. In the table, \(HGA(a,b,c)\) denotes a three-step operation. First, a full connection (FC) layer is conducted to shrink the dimension from \(a\) to \(b\), then an HGA attention with \(b\) channels is utilized. After the attention mechanism, the dimension will be restored from \(b\) to \(c\) with a FC layer. Herein, we consider the error rate as the indicator for different networks. In fact, statistically significant is a less concerned indicator for deep learning-based image classification. On one hand, Top-1 and Top-5 error (%) can well describe the performance for practical applications. On the other hand, CNN has a good generalization ability and performs well on different classification datasets. Usually when a network works well on one class, it will also work well on another one. In this situation, Analysis of Variance (ANOVA) may be not useful for evaluating the network performance. From this point of view, we only compare the numeric values but not perform the statistically significant test. We compare the single-crop error rates on the ImageNet validation set and adopt better values between the reported results from their origin papers and our reproduction version. The results are shown in Table 2. We apply the HGA to five modern backbones: ResNet-50, ResNet-101, ResNet-152, ResNeXt-50, and ResNeXt-101, which have proved to be remarkable design patterns. From the results, HGA provides a significant performance improvement with restricted increase on FLOPs and parameters. Notice that the accuracy rates from vanilla models are reported from the origin paper, and the HGA versions are trained by ourselves. From the numerical comparison with vanilla models, HGA achieves average 7.94% improvement on Top 1 error and 16.63% on Top 5 error. The FLOPs of HGA version only increases 6.23% on average. Furthermore, we also compare our HGA with recent elaborate attention mechanism SENet [7] and SKNet [35]. The results are shown in Table 3. From the results, HGA achieves better classification accuracy than other attention mechanisms. To demonstrate  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Output Size** & **ResNet-50**[15] & **HGA-ResNet-50** & **HGA-ResNeXt-50** \\ \hline \(112\times 112\) & \(conv,7\times 7,64,stride=2\) & \(conv,7\times 7,64,stride=2\) & \(conv,7\times 7,64,stride=2\) \\ \(56\times 56\) & \(maxpool,3\times 3,stride=2\) & \(maxpool,3\times 3,stride=2\) \\ \(\left[ \begin{array}{c}conv,1\times 1,64\\ conv,3\times 3,64\\ conv,1\times 1,256\\ \end{array} \right]\times 3\) & \(\left[ \begin{array}{c}conv,1\times 1,64\\ conv,3\times 3,64\\ conv,1\times 1,256\\ HGA(256,32,256)\\ \end{array} \right]\times 3\) & \(\left[ \begin{array}{c}conv,1\times 1,128\\ conv,3\times 3,128\\ conv,1\times 1,256\\ HGA(256,32,256)\\ \end{array} \right]\times 3\) \\ \(28\times 28\) & \(\left[