## 3 Materials and Methods
### RMT Spatial Prior Block  
The RMT Spatial Prior Block is an innovative vision module that combines the retentive mechanism of Retentive Networks (RetNets) [35] with the attention mechanism of Vision Transformer (ViT), greatly improving the efficiency of global information modeling while integrating explicit spatial priors. The primary framework of the RMT Spatial Prior Block includes the Manhattan Self-Attention (MaSA) mechanism [36] and an efficient decomposition strategy, enabling outstanding performance across various vision tasks. Figure 4 provides a detailed representation of the module's structure.  
The RMT Spatial Prior Block incorporates a two-dimensional spatial decay matrix based on Manhattan distance, embedding explicit spatial priors into the self-attention mechanism. By attenuating weights based on the distances between tokens, the target token can more effectively allocate attention resources when perceiving global information. The MaSA mechanism extends the one-dimensional temporal decay of RetNets into two-dimensional space, designing a decay matrix based on Manhattan distance. The at  
Figure 4: Structural Diagram of the RMT Spatial Prior Block.  
tention scores in this matrix progressively diminish as the distance from the target token increases, thereby explicitly embedding spatial priors. The attenuation matrix is formulated as follows:  
\[D_{nm}^{2d}=\gamma^{|x_{n}-x_{m}|+|y_{n}-y_{m}|} \tag{3}\]  
where \((x_{n},y_{n})\) and \((x_{m},y_{m})\) represent the two-dimensional coordinates of two tokens, and \(\gamma\) denotes the attenuation factor.  
The spatial decay matrix is utilized to adjust attention allocation, replacing the traditional Softmax operation in standard self-attention. The improved self-attention formula is as follows:  
\[\text{MaSA}(X)=\left(\text{Softmax}(QK^{\intercal})\odot D^{2d}\right)V \tag{4}\]  
where \(\odot\) represents element-wise multiplication.  
In low-resolution data scenarios, due to the limited spatial resolution of the data, capturing local information is particularly important. MaSA's attention mechanism based on Manhattan distance effectively enhances the interaction ability of neighboring regions by explicitly modeling spatial attenuation, thus having a significant advantage in capturing fine-grained features. In addition, its two-dimensional receptive field preservation feature enables the model to comprehensively consider a larger spatial range even at low resolution to avoid information loss. This stability and adaptability of performance makes MaSA's performance on low-resolution data an important manifestation of its comprehensive capabilities.  
When processing high-resolution feature maps, the computational cost of global attention is often prohibitive. To address this, MaSA introduces a decomposition strategy with linear complexity, breaking down the attention and spatial decay matrix into operations computed separately along horizontal and vertical directions. This method preserves the full two-dimensional receptive field while achieving linear computational complexity. Specifically, the two-dimensional attention is decomposed into horizontal and vertical one-dimensional operations:  
\[\begin{split} Attn_{H}&=\text{Softmax}\big{(}Q_{H}K _{H}^{\intercal}\big{)}\odot D^{H},\\ Attn_{W}&=\text{Softmax}\big{(}Q_{W}K_{W}^{ \intercal}\big{)}\odot D^{W}\end{split} \tag{5}\]  
where \(D^{H}\) and \(D^{W}\) represent the attenuation matrices in the horizontal and vertical directions, respectively. Finally, they are combined as follows:  
\[\text{MaSA}(X)=Attn_{H}(Attn_{W}V)^{\intercal} \tag{6}\]  
The decomposed Manhattan Self-Attention not only retains the complete two-dimensional receptive field but also significantly reduces computational complexity, making it particularly suitable for handling early-stage high-resolution feature maps. The RMT Spatial Prior Block introduces a novel paradigm for constructing general-purpose vision backbone networks through its efficient design and flexible spatial priors.