## 3 Proposed Method
### Cross-Modality Feature Fusion Module (CMFFM)  
In the previous subsection, we separately extracted the features from the PAN and MS images and inputted them into the CMFFM. This section presents the structure of the CMFFM as illustrated in Figure 4.  
First, we feed the modality-specific features (\(H\times W\times C\)) of PAN and MS into CMFFM as inputs, respectively. Then, the PAN feature and MS feature are concatenated to obtain a fused feature (\(H\times W\times 2C\)) along the channel dimension. The number of channels is then adjusted to obtain the final fusion feature of size (\(H\times W\times C\)). At the same time, the PAN feature and MS feature are separately fed into SSTM for further feature extraction. At this point, the size of the new PAN and MS features remains (\(H\times W\times C\)). Finally, the new PAN and MS features are concatenated with the previous fusion features along the channel dimension to obtain the cross-modality feature as the output of CMFFM. The main aim of the CMFFM is to utilize the SSTMs to further extract cross-modality features from both PAN and MS images. We leverage fusion features to enhance cross-modality details, thereby improving integration. The direct application of standard transformers, particularly in the computer vision domain, presents several challenges, primarily  
Figure 4: Cross-Modality Feature Fusion Module (CMFFM). Figure 3: Self-Attention Block (SAB). due to two factors: scale differences and the high resolution of images. The first challenge pertains to scale. In a street scene image, for instance, various objects such as cars and pedestrians differ significantly in size, a phenomenon not present in natural language processing. The second challenge is related to image resolution. When using pixels as the basic unit, the resulting sequence length can become impractically large. To address these issues, researchers have explored various approaches, including partitioning the image into smaller windows and performing self-attention calculations on these windows. To address the aforementioned challenges, the Swin Transformer network has been introduced [23]. This network employs a sliding window mechanism for feature extraction, which not only enhances computational efficiency but also significantly reduces the sequence length by performing self-attention calculations within the confines of each window. Simultaneously, the shifting operation enables interaction between adjacent windows, fostering cross-window connections across different layers. This design effectively facilitates a form of global modeling while maintaining computational efficiency. To capture the relevant information from PAN and MS images, the approach prioritizes the integration of key features. We incorporate the Swin Transformer and employ the shifting mechanism to facilitate interaction between the information. This leads to the proposal of the structure. As a key component of the CMFFM, we illustrate the detailed structure of the SSTM in Figure 5. Each SSTM is composed of two consecutive Cross-Attention Blocks (CABs), with the structure of each CAB depicted in Figure 6. The PAN (MS) feature and the fused feature are fed into the CAB as the query (Q) and key-value (K, V) pairs, respectively. This strategy fully capitalizes on the distinct advantages of both MS and PAN, facilitating a more effective utilization of the correlative dependence and complementary information between spatial (spectral) features and their fused counterparts. The attention function CA (\(F_{a}\), \(F_{b}\)) can be expressed as  
\[CA(F_{a},F_{b})=Attention(Q(F_{b}),K(F_{a}),V(F_{a})) \tag{2}\]  
Given two features, \(F_{a}\) and \(F_{b}\), the attention mechanism can be employed to model their relationship. CA indicates the relationships between \(F_{a}\) and \(F_{b}\). As defined in Equation (1), the attention function is employed in our application to evaluate CA (\(\cdot\)). Figure 5: Successive Swin Transformer Module (SSTM).