# Multi-Dimensional Fusion Attention Mechanism with Vim-like Structure for Mobile Network Design
## 3 Materials and Methods
### MDVA Method
#### 3.2.2 Multi-Dimensional Attention  
[PERSON] (MambaOut) [18] empirically demonstrated that the self-supervised module (SSM) is not essential for image classification tasks due to the absence of long sequences or autoregressive properties. However, for detection and segmentation tasks, where long sequence properties are present, SSM may offer potential benefits. Based on these findings and [PERSON]'s recommendation, we have eliminated the SSM module from Mamba, like MambaOut, and redefined multidimensional attention as a \"1 + 3\"-branch module that takes an input tensor and outputs a tensor of equivalent shape. In the \(x_{\text{left}}\) branch, we apply the input tensor \(\chi\in\mathbb{R}^{C\times H\times W}\) through a sigmoid activation layer (\(\sigma\)) to retain the original channel information. Meanwhile, in the \(x_{\text{right-1}}\) branch, we input the tensor to each component of the proposed multidimensional attention module's right branch. Here, we establish interaction between the height dimension and channel dimension by rotating \(\chi\) 90 degrees counterclockwise along the height axis to obtain \(\varphi(\chi_{1})\) with dimensions \(W\times H\times C\). Subsequently, reducing \(\varphi(\chi_{1})\) via DR-Pooling yields a \(2\times H\times C\) tensor, which then undergoes convolutional processing using a \(7\times 7\) kernel size followed by batch normalization, resulting in an intermediate output of shape \(1\times H\times C\). The attention weight is generated through a sigmoid activation layer (\(\sigma\)), applied to \(\varphi(\chi_{1})\) before being rotated back 90 degrees clockwise along the height axis to restore the original input shape. In the \(x_{\text{right-2}}\) branch, rotation of \(\chi\) by 90 degrees counterclockwise along the width axis produces a rotated tensor \(\chi_{2}\) with dimensions \(H\times C\times W\). This tensor is reduced via DR-Pooling, yielding a \(2\times C\times W\) tensor, which, after convolutional processing and batch normalization, results in an output tensor with dimensions \(1\times C\times W\). The attention weight is generated through a sigmoid activation layer (\(\sigma\)) before being rotated back 90 degrees clockwise along the width axis, returning it to the original input shape. Finally, the channels of the input tensor \(\chi_{3}\) are reduced using a process called dimensional reduction pooling (DR-Pooling). This transformation results in a new tensor with dimensions \(2\times H\times W\). This tensor is then passed through a convolutional layer, followed by a batch normalization layer, resulting in a final output with dimensions \(1\times H\times W\). The attention weights generated from each branch are independently applied to the corresponding tensors from the left-hand side. Finally, the outputs from each branch are averaged together to produce the result. The MDVA attention flow is shown in Figure 3.  
By incorporating the multi-dimensional reduction module, the system can effectively integrate spatial and channel-wise information, enhancing the overall performance of the attention mechanism without increasing computational complexity. By redefining the multidimensional attention mechanism in this way, we can efficiently capture cross-dimensional dependencies and enhance the representational capacity of the model without increasing the number of parameters. This approach is particularly beneficial for tasks requiring detailed spatial and channel interdependencies.