## 1 Introduction
This limitation becomes particularly pronounced in tasks that necessitate an understanding of the global context, such as sequence modeling or establishing global associations within images. Consequently, this shortcoming may hinder their performance in scenarios where the extraction of global features is critical. Conversely, Transformers have gained significant traction as a robust architecture for modeling sequential data [21; 22]. Originally developed for natural language processing tasks, their versatility has extended into a wide range of domains, including image processing [23] and time-series analysis [24]. Transformers have demonstrated remarkable performance in various tasks such as image classification [25], segmentation [26], object detection [27], and image captioning [28]. Unlike Convolutional Neural Networks (CNNs), which rely on local receptive fields, Transformers employ a self-attention mechanism that captures global dependencies across the entire input sequence. This ability to model long-range relationships enables them to effectively handle tasks that require contextual understanding and interactions between distant elements in the data. Consequently, Transformers are particularly advantageous for applications involving complex spatial and temporal dynamics, making them well suited for change detection in remote sensing and other fields requiring global feature representation. In remote sensing images, objects frequently display varying scales and shapes, which complicates the process of single-scale feature extraction in capturing their complex characteristics [29]. For example, the same object, such as an airplane, may appear at considerably different scales due to factors such as the type of imaging sensor, variations in flight altitudes, or changes in viewing angles, leading to substantial size discrepancies across images. Single-scale feature extraction techniques, which operate at a fixed resolution or scale, are often inadequate in capturing the full spectrum of information across these varying scales. Therefore, the use of multi-scale feature extraction methods is advantageous, as these approaches can effectively integrate the spatial structures and semantic information of objects across different scales [30]. This multi-scale strategy notably improves the accuracy and robustness of tasks such as object classification, target detection, and image segmentation. Furthermore, the attention mechanism plays a pivotal role in the processing of remote sensing images [31; 32]. These images typically contain substantial background information and noise, which may hinder the model's ability to discern and analyze critical objects or regions. By integrating attention mechanisms, models can autonomously learn to prioritize significant areas or objects within the image [33]. This capability not only improves the model's performance in recognizing and analyzing essential objects but also facilitates more effective processing of remote sensing data [34]. In the domain of change detection, [PERSON] et al. were pioneers in applying the Siamese Convolutional Network to bi-temporal remote sensing image analysis, achieving remarkable results. The fundamental concept of Siamese networks involves passing two input samples through the same neural network to extract features, subsequently enabling a comparison of similarity or dissimilarity in the feature space. Building upon this foundation, [PERSON] et al. designed several effective change detection architectures based on Fully Convolutional Neural Networks (FCNNs), including Fully Convolutional Early Fusion (FC-EF) and Fully Convolutional Siamese Concatenation (FC-Siam-conc). The primary distinction between these networks lies in their methods of fusing bi-temporal information [35]. [PERSON] et al. further advanced this field by developing a dual-task constrained Siamese Convolutional Neural Network that improves focus loss to effectively tackle sample imbalance, thereby enhancing change detection accuracy [36]. Similarly, [PERSON] et al.