# Efficient Attention Mechanism for Dynamic Convolution in Lightweight Neural Network
## 4 Experimental Results
### Numerous Results
#### 4.2.2 Ablation Studies Results  
A series of ablation experiments were conducted on MobilenetV1 \(\times\) 1.0. In these experiments, we set the default number of attention cores \(k\) to 4, and the temperature \(\tau\) of softmax to 30. The number of channels: The choice of channel size and number of channels determine the degree to which the model reduces its redundancy. It defaults the pooling layer to adaptive average pooling, and the output size of the pooling is \(5\times 5\) spatial information. When EAM-DyNet uses the RCR method with different sizes of channel on different datasets, namely two channels, four channels, and eight channels. The experimental results are shown in Table 3. After experimental verification, it was found that when the selected channel was the same size as the attention core, and its accuracy rate reached the highest. Compared with the standard static and standard dynamic networks, the EAM-DyNet network increased by 1.1 and 0.43 percentage points. The methods of channel reduction: For the selection method of different channels, EAM-DyNet uses 1/4C as the default size, and chooses \(0\sim 1/4C\), \(1/4C\sim 1/2C\), \(1/2C\sim 3/4C\), \(3/4C\)-\(C\), then takes every four channels to compare their results with the RCR and CGR methods, and the results are shown in Table 4. In addition, it calculates the parameters and FLOPs when choosing the channel of 1/4C size and choosing the channel with the RCR and CGR method, the result of calculation is shown in Table 5. After comparing the FLOPs, the parameters, and the accuracy rate of different methods, we found that when using the RCR and CGR methods, the parameter amount and FLOPs were the lowest, but the accuracy rate was high. The reason why the calculation amount is too high when 1/4C is selected is because as the network becomes deeper, the number of channels will increase, and the calculation amount will increase. However, the channels selected by RCR and CGR were always four channels. Through the results in the two tables, we found that the parameters and FLOPs of the RCR and CGR methods were reduced by 4.5 M and 257.75 M, but the accuracy remained flat or even greater than the other method. \begin{table}
\begin{tabular}{c c c c} \hline \hline \multirow{2}{*}{**The Methods of Channels Selection**} & **CIFAR10** & **CIFAR100** & **ImageNet100** \\ \cline{2-4}  & **Top-1 (\%)** & **Top-1 (\%)** & **Top-1 (\%)** \\ \hline DyNet MobilenetV1 \(\times\) 1.0 & 93.99 & 74.76 & 75.94 \\ \hline EAM-DyNet MobilenetV1 (take every 4 channels) & 93.72 & 74.84 & 76.58 \\ EAM-DyNet MobilenetV1 \(\times\) 1.0 (0\(\sim\)1/4C) & 93.79 & 74.38 & 75.92 \\ EAM-DyNet MobilenetV1 \(\times\) 1.0 (1/4C–1/2C) & 93.90 & 75.18 & 75.20 \\ EAM-DyNet MobilenetV1 \(\times\) 1.0 (1/2C–3/4C) & 93.62 & 75.03 & 75.80 \\ EAM-DyNet MobilenetV1 \(\times\) 1.0 (3/4C–C) & 94.03 & 74.99 & 76.23 \\ \hline EAM-DyNet MobilenetV1 \(\times\) 1.0(RCR) & 93.61 & 74.83 & **76.37** \\ EAM-DyNet MobilenetV1 \(\times\) 1.0(CGR) & **93.93** & **75.21** & 76.11 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: The accuracy of standard static network, standard dynamic network, and EAM-DyNet in different channel selection methods on MobilenetV1. \begin{table}
\begin{tabular}{c c c c} \hline \hline \multirow{2}{*}{**The Number of Channels (RCR)**} & **CIFAR10** & **CIFAR100** & **ImageNet100** \\ \cline{2-4}  & **Top-1 (\%)** & **Top-1 (\%)** & **Top-1 (\%)** \\ \hline DyNet MobilenetV1 \(\times\) 1.0 & 93.99 & 74.76 & 75.94 \\ \hline EAM-DyNet MobilenetV1 \(\times\) 1.0 (2 channels) & 93.95 & 74.50 & 75.78 \\ EAM-DyNet MobilenetV1 \(\times\) 1.0 (4 channels) & 93.61 & **74.83** & **76.37** \\ EAM-DyNet MobilenetV1 \(\times\) 1.0 (8 channels) & 94.07 & 74.70 & 75.45 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: The accuracy of standard static network, standard dynamic network, and EAM-DyNet when using the Random Channel Reduction (RCR) method to take out channels of different sizes on MobilenetV1.