## 3 Proposed Method
### GL-Deep Fusion  
As noted in Section 3.1, the integration of features from both branches is of paramount importance for the performance of GLSNet. To tackle this issue, the global-local deep fusion method (GL-Deep Fusion) was designed. The core idea is to leverage an enhanced transformer architecture that employs a dual-encoder and single-decoder mechanism to amalgamate global and local features effectively. Unlike traditional transformer frameworks that necessitate extensive computations for deriving the matrices of queries, keys, and values, the proposed encoders are adept at directly aligning with the features provided by the dual-branch network, thereby significantly reducing computational overhead and memory consumption. This innovative approach facilitates a memory-efficient fusion process that capitalizes on the distinct advantages of both the global and local branches, culminating in a more robust and accurate model for feature integration. Figure 2: Overview of GLSNet. The global shallow branch and local deep branch use shallow neural networks and deep neural networks to capture global contextual information and local detail. Then, the GL-Deep Fusion completes the fusion of global–local information, ultimately completing the segmentation task of ultra-high resolution images. **Transformer attention function**[39]: The attention function is defined for matrices of queries \(Q\), keys \(K\), and values \(V\) where the dimensions are \(d_{k}\times n\) and \(d_{v}\times n\), respectively. Here, \(n\) denotes the number of elements within the set, while \(d_{k}\) and \(d_{v}\) are the dimensionalities associated with the keys and values. This function calculates the dot products between the queries and keys, applies a scaling factor of \(\sqrt{d_{k}}\) to stabilize the softmax operation, and subsequently uses the softmax function to generate a weighted distribution across the values. \[Attention(Q,K,V)=Softmax(\frac{QK^{T}}{\sqrt{d_{k}}})\cdot V, \tag{1}\]  
where \(QK^{T}\) corresponds to the dot product of the query matrix with the transpose of the key matrix. The division by \(\sqrt{d_{k}}\) serves as a form of normalization to maintain the balance of the softmax output. **Branch-Attention (B2A)**: B2A (Figure 3 right) facilitates the interaction between two distinct branches, allowing one to query information from the other. This is particularly useful for establishing relationships between different data representations within the model. \[B2A_{branch1,branch2}=Attention(F^{(q)}_{branch1},F^{(k)}_{branch2},F^{(v)}_{ branch2}), \tag{2}\]  
where \(F^{(q)}_{branch1}\) represents the query matrices from the branch1, and \(F^{(k)}_{branch2}\), \(F^{(v)}_{branch2}\) are the key and value matrices derived from branch2. As shown in Figure 3, GL-Deep Fusion is based on the branch-attention and is designed with a dual-cross-encode and single-decode structure. Information sequences from both the global branch and the local branch are taken as input sequences directly, denoted as \(F_{global}\) and \(F_{local}\). The design allows for a significant decrease in redundant computations, which enhances the effectiveness of the dual-branch structure's benefits. The first encoder generates a global-local sequence that encompasses the local relevance information. It can be represented as  
\[B2A_{global-local}=Attention(F^{(q)}_{global},F^{(k)}_{local},F^{(v)}_{ local}) \tag{3}\]  
In parallel and symmetrically, the second encoder takes queries from \(F_{local}\) and key-value pairs from \(F_{global}\), generating a local-global sequence with globally relevant information. Its representation is  
\[B2A_{local-global}=Attention(F^{(q)}_{local},F^{(k)}_{global},F^{(v)}_{ global}) \tag{4}\]  
Figure 3: Overview of the GL-Deep Fusion module structure. As shown in the **left** figure, the core structure of GL-Deep Fusion includes a dual-cross encoder and a single decoder.