# DCS-TransUperNet: Road Segmentation Network Based on CSwin Transformer with Dual Resolution
## 3 Method
### Multi-Scale Feature Representations and Fusion  
Although MSA effectively establishes long-range dependencies between patches, the pixel-level features in the patch are disregarded. This detailed information is particularly important for the prediction of dense small roads. In addition, ViT achieves better performance through the fine-grained patch size. In order to enhance the performance and robustness of road extraction, a multi-scale CSwin Transformer is used for feature extraction. It is complementary to the information from different patches in feature extraction. Coarse-grained features are easier to be captured by large patches, while the small scale has a faculty for obtaining fine-grained characteristics. Although multi-layer perceptron introduces the local information between batches, the pixel-level information inside the patch is still missing. In DCS-TransUperNet, these problems are alleviated to some extent. Inspired by this, a multi-scale dual CSwin Transformer encoder is proposed. Specifically, two branch structures are adopted to obtain the features at different spatial levels. One is the main branch (patch size is 4), and the other is the auxiliary (patch size is 8). As a result, the resolutions of the output features obtained from the large-scale branch are \((H/4)\times(W/4),\ (H/8)\times(W/8),\ (H/16)\times(W/16)\) and \((H/32)\times(W/32)\) while those of the other branch are \((H/8)\times(W/8),\ (H/16)\times(W/16),\ (H/32)\times(W/32)\) and \((H/64)\times(W/64)\). Feature representations containing abstract information are obtained from dual branch encoders. The following work is how to realize the information interaction between them effectively. A simple way is to straightforwardly concatenate the features from different branches followed by convolution layers. However, this direct way cannot obtain long-distance dependence and global connections from different scale features. Hence, a new FFM is introduced, which relies on the advantages of MSA to make the interaction more effective between multi-scale features. In particular, the standard Transformer is chosen rather than CSwin Transformer in FFM. The main reason is that the standard Transformer works on the feature map based on a rectangle block. A token is generated to be size specific based on one branch's features map. The next step is to calculate the attention weight of the tokens sequence reshaped by the other branch. In addition, the number of the Transformer block in FFM is only two, which will not cause computational complexity. A more intuitive explanation is shown in Figure 3.  
FFM can aggregate information from different scale features. Let us take the output features from the first CSwin Transformer stage as an example and explain them in detail, and the same measure is suitable for the large-scale branch. In particular, the two branches' output features from the same stage \(j\) (\(j\): 1, 2, 3, 4) are represented by  
\[H^{j} = [h_{1}^{j},h_{2}^{j},\dots,h_{h\times w/16}^{j}]\in\mathbb{R}^{ \mathbb{C}\times(\frac{h}{4}\times\frac{w}{4})} \tag{3}\] \[F^{j} = [f_{1}^{j},f_{2}^{j},\dots,f_{h\times w/64}^{j}]\in\mathbb{R}^{ \mathbb{C}\times(\frac{h}{8}\times\frac{w}{8})} \tag{4}\]  
Figure 3: Illustration of FFM plays an important role as a core component in features interaction from different scales. Then the transformation output of \(H^{j}\) is obtained by the global average pooling layer. \[\widetilde{h}^{j}=Flatten(Gavgpool(H^{j})) \tag{5}\]  
where \(\widetilde{h}^{j}\in\mathbb{R}^{C\times 1}\), Gavgpool indicates global average pooling, and then the flatten operation is used. For further description, the global abstract information is expressed by token \(\widetilde{h}^{j}\), making the information fusion with \(F^{j}\) at the pixel level. In the meantime, \(\widetilde{h}^{j}\) is concatenated with \(F^{j}\) to form a sequence of \(1+C\times((H/8)\times(W/8))\) tokens, followed by a standard Transformer to calculate the global self-attention:  
\[