# Segmenting Urban Scene Imagery in Real Time Using an Efficient UNet-like Transformer
## 1 Introduction
[19] designed a Transformer- and CNN-based hybrid network to segment the high-resolution urban imagery, and a spatial pyramid pooling module and an auxiliary boundary detection strategy were applied to obtain better segmentation performance. However, although the self-attention mechanism based on the Softmax function is effective, the quadratic computation complexity, O (\(\eta^{2}\)), directly related to the length of the input sequence \(n\), leads to great computational costs, which limits the efficiency and real-time application capability of the model. Some studies have solved this problem by limiting the global receptive field to a smaller range, such as using sparse global attention [20; 21] or a smaller attention window [22; 23]. Although effective, these methods also pose the following two problems: (1) useful information in other regions (outside the window) may be overlooked, and (2) the model's potential to model global context is sacrificed to some extent. On the other hand, the linear attention mechanism (LNA) [24] provides a new way to address the trade-off between complexity and accuracy performance in Transformer-based models. Specifically, LNA uses a separated kernel function to replace the Softmax function in the self-attention mechanism [25]. As a result, LNA need not compute the pair-wise similarity first as Softmax attention (QK\({}^{\text{T}}\)). In other words, LNA can adjust the computation order by computing the KV\({}^{\text{T}}\) first based on the associative law of matrix multiplication (Figure A1), as a result, decreasing the computational complexity from O (N\({}^{2}\)d) to O (Nd\({}^{2}\)). The channel dimension, d, is usually much smaller than the token number N in modern classical Vision Transformers. Nevertheless, it is still a complex problem to design a module that is as efficient as Softmax self-attention when applying the LNA. Efficient Vit [26] used depth-wise separable convolution to improve the local information modeling capability of the LNA module to achieve a higher classification accuracy. [PERSON] [27] proposed the idea of a linear angular kernel to model the spectral similarity between each Q and K. Hydra attention [28] used the cosine similarity to simulate the Softmax function to reduce the computational complexity to O (Nd). [PERSON] [29] and [PERSON] [30]approximated the Softmax self-attention matrix by matrix decomposition to achieve high efficiency while [PERSON] the computational complexity. There have been many efforts by scholars to outperform the Softmax attention mechanism, yet current LNA-based models face a dilemma between model complexity and performance. Firstly, a simple simulation of the Softmax self-attention function [26] would seriously affect the model performance capability; secondly, elaborately designed and complex kernel function [31] or matrix transformation methods [29; 30] would incur additional computation costs and influence the model's real-time application capability. Therefore, there are still significant challenges in utilizing LNA-based models efficiently and effectively. Currently, the model's segmentation accuracy and efficiency are rarely considered simultaneously in many existing methods, failing to solve the trade-off problem in real-time application situations. In this study, we aim to design a network that can accurately segment urban scene images while ensuring the model's efficiency. Inspired by the advanced work in the Flatten Transformer [25], which can obtain better performance than Softmax attention while incurring lower computation complexity, we designed a new global-local information modeling module (GLIM) based on the proposed WMFSA and constructed a UNet-like network (Flauformer) based on the CNN encoder and Transformer decoder. Furthermore, the window shift operation in the Swin Transformer is effective but incurs additional computation overhead.