# A Multi-Path Semantic Segmentation Network Based on Convolutional Attention Guidance
## 3 Method
### Convolutional Attention  
We employ a convolutional attention network as the main path. For the construction of encoder blocks, a structure similar to ViT [6] is adopted. However, instead of using the self-attention mechanism, a new multi-scale convolutional attention (MSCA) module is introduced. As depicted in Figure 2b, MSCA comprises four components: depthwise convolution for aggregating local information, multi-path convolution for capturing multi-scale contexts, spatial and channel reconfiguration modules (SRU and CRU), and \(1\times 1\) convolution for modeling relationships between different channels. GELU represents the activation function, BN denotes batch normalization and Add denotes an addition operation. Here, k \(\times\) k indicates the use of depthwise separable convolution with a kernel size of k \(\times\) k. The outputs of the \(1\times 1\) convolution are directly used as attention weights to rebalance the input of MSCA. Mathematically, MSCA can be expressed as shown in Equations (1)-(3):  
\[Att=\sum_{i=0}^{3}Conv_{i}(Conv_{5\times 5}(F)) \tag{1}\]  
\[Att\_R=Conv_{1\times 1}(Att+C(S(Att)))\otimes F \tag{2}\]  
\[Out=Att\_R\otimes F \tag{3}\]  
where \(Conv_{i}(i=0,1,2,3)\) represents the convolutional layers in the diagram, with kernel sizes of \(1\times 1\), \(7\times 7\), \(11\times 11\) and \(21\times 21\), respectively. \(C\) and \(S\) represent CRU and SRU, while F represents the input features. \(Att\_R\) and \(Out\) denote the attention map and output, respectively. In Equation (3), the symbol \(\otimes\) denotes element-wise matrix multiplication. Stacking a series of building blocks results in the proposed convolutional encoder yields MSCAN (shows in Figure 2a). MSCAN adopts a hierarchical structure, where the MP consists of four stages. Each stage consists of L MSCANs, and the spatial resolution decreases as \(\frac{H}{4}\times\frac{W}{4}\), \(\frac{H}{8}\times\frac{W}{8}\), \(\frac{H}{16}\times\frac{W}{16}\), \(\frac{H}{32}\times\frac{W}{32}\), where \(H\) and \(W\) are the height and width of the input image, respectively. Each stage includes a downsampling block with a batch normalization layer [24], followed by a convolutional layer with a stride of 2 and a kernel size of \(3\times 3\). The third stage contains a stack of L = 12 MSCAN modules, while the remaining stages are stacked three times each. Additionally, spatial and channel recurrent units (SRUs and CRUs) are introduced in our MSCA module. For the SRUs, our aim is to leverage spatial redundancy in the features using a separate-and-reconstruct operation. The purpose of the separate operation is to extract information-rich feature maps from those with less information. A scaling factor from the group normalization layer is used to assess the information content of different feature maps. Specifically, given an intermediate feature map \(X\) with dimensions \(N\times C\times H\times W\), where \(N\) is the batch axis, \(C\) is the channel axis and \(H\) and \(W\) are the spatial height and width axes, we firstly normalize the input as shown in Equation (4):  
\[X_{out}=GN(X)=\gamma\frac{X-\mu}{\sqrt{\sigma^{2}+\epsilon}}+\beta \tag{4}\]  
\[W_{\gamma}=\{w_{i}\}=\frac{\gamma_{i}}{\sum_{j=1}^{C}\gamma_{j}}(i,j=1,2, \ldots,C) \tag{5}\]  
\[W=Gate(Sigmoid(W_{\gamma}(GN(X)))) \tag{6}\]  
where \(\mu\) and \(\sigma\) are the mean and standard deviation of \(X\), \(\epsilon\) is a small positive constant added for stability and \(\gamma\) and \(\beta\) are trainable affine transformations. The normalization related weights are obtained using the trainable parameters \(\gamma\) in the group normalization layer \(GN\), as shown in Equation (5), representing the importance of different feature maps. The weighted feature map is then mapped to the \((0,1)\) range through a \(Sigmoid\) function, and a threshold gate \(Gate\) is applied to set weights above the threshold to 1 for informative weights \(W_{1}\) and set those below the threshold to 0 for non-informative weights \(W_{2}\). The entire process is represented by Equation (6).