# CCFNet: Collaborative Cross-Fusion Network for Medical Image Segmentation
## 3 Method
### Parallel Fusion Layer  
The parallel fusion layer has two branches, namely, the Transformer branch and the CNN branch, which process information in distinct ways. In the CNN branch, local features are collected hierarchically through a convolution operation, and local clues are also saved as feature maps. The parallel fusion layer fuses the feature representation of the CNN in a parallel manner through cascaded attention modules, which maximizes the preservation of local features and global representations. The parallel fusion layer is composed of six CCFMs superposed. An image has two completely different representations: global features and local features. The former focuses on model object-level relationships between remote parts, while the latter aims at fine-grained details and is beneficial for pixel-level localization and tiny object detection. As shown in Figure 2(b), a CCFM is used to efficiently combine these encoded features of the Transformer and CNN, which can interactively fuse convolution-based local features and Transformer-based global representations. The CCFM has two inputs, \(C_{i-1}\) and \(T_{i-1}\), where \(C_{i-1}\) is the input on the CNN branch with the same resolution as \(T_{i-1}\), \(T_{i-1}\) is the input on the Transformer branch, and \(C_{i-1}^{{}^{\prime}}\) is the feature map formed after extracting features on the CNN branch with the same resolution and number of channels as \(T_{i-1}\), which can be expressed as  
\[C_{i-1}^{{}^{\prime}}=\text{GELU}(\text{Normal}(\text{Conv2d}(C_{i-1}))). \tag{6}\]  
The Transformer aggregates information between global tokens, but CNN only aggregates information in the limited local field of view of the convolution kernel, which leads to certain feature semantic differences between the Transformer and CNN. Therefore, by superimposing the feature maps of the CNN and Transformer, the CSF block adaptively fuses the self-attention weights with common information between them to calculate the mutual relationship between local tokens and global tokens. As shown in Figure 2(b), the CSF block consists of ConvAttention and CMLP. Like the traditional attention mechanism, the basic module of ConvAttention is multi-head self-attention (MHSA). As shown in Figure 3(a), the difference is that ConvAttention has two inputs, adding \(T_{i-1}\) and \(C_{i-1}^{{}^{\prime}}\) to obtain feature maps \(F_{i}\) and \(T_{i-1}\) as its input. In addition, ConvAttention uses convolutional mapping. The specific operation is that \(T_{i-1}\) generates \(V_{i}\) through \(3\times 3\) convolutional mapping, and \(F_{i}\) generates \(Q_{i}\) and \(K_{i}\) through \(3\times 3\) convolutional mapping. Subsequently, we use the flatten operation to project the patches into the \(d\)-dimensional embedding space as the input of the underlying module MHSA in the ConvAttention block. \[F_{i}=C_{i-1}^{{}^{\prime}}+T_{i-1}, \tag{7}\]  
\[Q_{i}/K_{i}=\text{Flatten}(\text{GELU}(\text{Normal}(\text{Conv2d}(F_{i})))), \tag{8}\]  
\[V_{i}=\text{Falatten}(\text{GELU}(\text{Normal}(\text{Conv2d}(T_{i-1})))). \tag{9}\]  
The MHSA is performed on the obtained \(Q_{i}\), \(K_{i}\), and \(V_{i}\), an MHSA comprises \(h\) parallel self-attention heads. The calculation process is as follows:  
\[\text{MHSA}(Q_{i},K_{i},V_{i})=\text{Concat}(head_{1},...,head_{h})W_{mtsa}, \tag{10}\]  
where \(W_{mtsa}\in R^{d\times d}\) represents the multi-headed trainable parameter weights. The self-attention of each head in MHSA is calculated as  
\[\text{Attention}(Q,K,V)=\text{SoftMax}(\frac{QK^{T}}{\sqrt{d}}+B)V, \tag{11}\]  
where \(Q\), \(K\), and \(V\in R^{N_{T}\times d}\) are the query, key, and value matrices, which are obtained by convolution projection, \(N_{T}=W_{h}\times W_{w}\) denotes the number of patch tokens. \(\{W_{h},W_{w}\}\) stand for the size of the feature \(F_{i}/T_{i}\), and \(d\) is the query/key dimension. We follow [28,44] by including a relative position bias \(B\in R^{N_{T}\times N_{T}}\).