# Multi-Dimensional Fusion Attention Mechanism with Vim-like Structure for Mobile Network Design
## 4 Experiment
### Image Classification  
#### 4.3.1 CIFAR-100 Classification Experiment  
To rigorously evaluate the new module, we conducted classification experiments on the CIFAR-100 dataset. The new findings reveal that the overhead of the Mambaclass multidimensional attention module is minimal in terms of both parameters and computation. This efficiency prompted us to integrate the proposed module, MDVA (multi-dimensional Vim-like attention), into the lightweight MobileNet network and ShuffleNet network. Table 3 summarizes the experimental results based on the MobileNet architecture and ShuffleNet architecture.  
Using MobileNetV2 [46] as the baseline, we compared the multidimensional attention module with other lightweight attention modules for mobile networks, including the widely used SE attention [6], CBAM [8], and CA [21], as detailed in Table 3.  
As indicated in Table 3, incorporating these attention mechanisms improved classification performance by approximately 1%. Notably, the spatial attention module of CBAM did not significantly enhance mobile network performance. However, the proposed multidimensional Vim-like attention yielded the best results.  
\begin{table}
\begin{tabular}{c c c c} \hline
**Setting** & **\#.Param. (M)** & **FLOPs (G)** & **Top-1 (\%)** \\ \hline Baseline & 2.35 & 0.33 & 76.0 \\ +SE & 4.62 & 0.33 & 77.3\({}_{+1.3}\) \\ +C Attention & 2.37 & 0.35 & 76.4\({}_{+0.4}\) \\ +H Attention & 2.37 & 0.35 & 75.6\({}_{-0.4}\) \\ +W Attention & 2.37 & 0.35 & 75.7\({}_{-0.3}\) \\ +HW Attention & 2.37 & 0.36 & 76.7\({}_{+0.7}\) \\ +CH Attention & 2.37 & 0.37 & 76.4\({}_{+0.4}\) \\ +MDVA Attention & 2.37 & 0.35 & 76.9\({}_{+0.9}\) \\ \hline \end{tabular} The results from various experimental settings were compared, highlighting the advantages of the MDVA method. The education rates and baseline results are based on the MobileNetV2 model. It was observed that models utilizing either channel (C) attention, vertical (W) attention, horizontal (H) attention individually or the combination of channel and vertical (CW) attention, channel and horizontal (CH) attention, or vertical and horizontal (WH) attention could achieve performance comparable to models with more comprehensive attention mechanisms. However, the new approach, which integrates channel, horizontal, and vertical attention (MDVA), consistently produced the best results.  
\end{table}
Table 1: Ablation studies for MDVA on MobileNetV2.  
\begin{table}
\begin{tabular}{c c c c} \hline
**Setting** & **\#.Param. (M)** & **FLOPs (G)** & **Top-1 (\%)** \\ \hline ShuffleNetV2-0.5 & 0.44 & 0.04 & 71.0 \\ +CBAM & 0.45 & 0.04 & 71.5\({}_{+0.5}\) \\ +MDVA & 0.45 & 0.05 & 71.6\({}_{+0.6}\) \\ \hline ShuffleNetV2-1.0 & 1.36 & 0.15 & 75.1 \\ +CBAM & 1.36 & 0.15 & 76.4\({}_{+0.4}\) \\ +MDVA & 1.36 & 0.15 & 75.3\({}_{+0.2}\) \\ \hline ShuffleNetV2-1.5 & 2.58 & 0.31 & 76.3 \\ +CBAM & 2.60 & 0.31 & 76.5\({}_{+0.4}\) \\ +MDVA & 2.58 & 0.31 & 76.7\({}_{+0.7}\) \\ \hline \end{tabular}
\end{table}
Table 2: Comparisons of different attention methods under different multipliers when taking ShuffleNetV2 [47] as the baseline.