###### Abstract
Fully Convolutional Networks (FCNs) [10], an encoder-decoder model for semantic segmentation, replaced fully connected layers with convolutional layers and used upsampling in the decoder to counteract size reduction from convolution and pooling. DeepLab [11] series expanded the receptive field with dilated convolutions to avoid resolution loss from pooling and enhanced multi-scale context capture through spatial pyramid pooling. DeepLabv3+ [12] further refined an encoder-decoder structure, progressively restoring spatial resolution and significantly improving boundary segmentation accuracy. The introduction of U-Net [13] further improved road extraction accuracy in remote sensing images. U-Net's skip connections enable multi-scale feature fusion, retaining high-level semantic information alongside low-level details. Due to its versatility and efficiency, U-Net has become one of the most popular models in image segmentation. Variants like U-Net++ [14] with dense connections, ResUNet [15] with residual connections, and U-SegNet [16] with SegNet's [17] pooling index transfer mechanism improve gradient propagation and computational efficiency and are adapted for complex segmentation tasks. CNNs have been extensively applied in road extraction with notable success. However, their reliance on convolution operations for local feature extraction inherently limits the receptive field, leading to insufficient contextual information and capping performance. Expanding the receptive field to capture global context requires deeper networks, which can result in issues like gradient vanishing or explosion. NL-LinkNet [18] was pioneering in incorporating non-local operations into LinkNet [19], using an attention-like mechanism to directly calculate the similarity between any two positions in the feature map. This capturesglobal context while preserving local details, making it especially suitable for long-span targets like roads. The approach enhanced global consistency and sparked interest in using attention mechanisms, including Transformers, for road extraction. In recent years, the successful application of Vision Transformer (ViT) [20] across various computer vision tasks has drawn significant attention to Transformers in the field [21; 22; 23]. Transformer models replace CNN's convolutional operations with self-attention mechanisms, enabling better global context capture from images without stacking layers to expand the receptive field. SETR [24] first applied Transformers to semantic segmentation by treating it as a sequence-to-sequence prediction task. It introduced and evaluated three decoder architectures, ultimately developing a comprehensive model that overcomes FCN limitations and achieving top performance on multiple datasets. Pyramid Vision Transformer (PVT) [25] employed a progressively shrinking pyramid structure with spatially reduced attention layers to generate multi-scale features, achieving strong results in object detection and segmentation. Twins [26] enhanced PVT by replacing fixed positional encoding with conditional encoding, minimizing the impact of image resolution on model performance and computational load, thus allowing flexible handling of multi-scale features. CCNet [27] introduced a criss-cross attention mechanism, improving computational efficiency and reducing complexity without sacrificing segmentation quality. Given roads' extensive geometric characteristics, capturing global features is essential in remote sensing images, prompting more researchers to integrate Transformers into road extraction tasks. RNGDet [28] combined Transformers with graph-based depth-first search algorithms to iteratively generate road networks from aerial images, improving the model's ability to capture road segments near complex intersections. [PERSON] et al. [29] addressed road shape features by using an efficient strip Transformer module to model long-range road dependencies, effectively capturing the elongated nature of roads.