# JANNet: A U-Shape Network with Jump Attention for Semantic Segmentation of Road Scenes
## 2 Materials and Methods
### Jump Attention Module (JAM)  
In deep learning application scenarios, we often need to deal with exceptionally large amounts of data, but only a small fraction of most of the data is important, and attention mechanisms are used for extracting the highlights from large amounts of data in deep learning tasks. In image processing, the attention mechanism can also play a good role, mainly by further feature extraction of the feature map, calculating the attention weights on the image channel or space, and re-acting on the original feature map to selectively extract image information. This has been previously used by networks such as SE-Net [18] and CBAM [19]. In this case, SE-Net first performs a Squeeze operation on the feature map, which uses global pooling to synthesize the input features of size 2C \(\times\)\(H\)\(\times\)\(W\) into a C \(\times\) 1 \(\times\) 1 feature description. After the above Squeeze operation, the network only gets a global description, which cannot be used as the weight of the channel. Therefore, the authors proposed the Excitation operation, which contains two fully connected layers, and the Sigmoid activation function, to obtain the channel attention weights for processing the original feature map. Attention mechanisms are also often used in the field of weather image segmentation [25; 26; 27]. We design the jump attention module to address the problem that the commonly used U-shaped network structure does not further process the low-dimensional features during the jump connection process, considering that the high-dimensional features have rich semantic information, which can help guide the selection of low-dimensional information, so as to achieve the selection of more accurate resolution information, and, finally, combine the processed feature maps to the high-level features for upsampling operations. Its structure is shown in Figure 4.  
As shown in the figure, the attention structure in this paper utilizes both high-dimensional features and low-dimensional features as a way to compensate for the UNet network's lack of any processing of low-level jump connections afterwards. The feature map size of the upsampling process is 2C \(\times\)\(H\)\(\times\)\(W\), which is converted to C \(\times\) 1 \(\times\) 1 channel attention weights after a series of pooling and feature extraction, and feature fusion processing with low-dimensional features of size \(C\)\(\times\) 2\(H\)\(\times\) 2\(W\). The method is to first convolve the low-dimensional features by 3 \(\times\) 3, followed by matrix multiplication with the channel attention weights extracted from the high-dimensional features to obtain the attention-processed feature map for subsequent upsampling with the high-dimensional feature evaluation for the next step. The formula for the jump attention module is given in Equation (2). \[M_{c}(L)=M_{c}(H)\times Conv_{3\times 3}(L) \tag{2}\]  
Figure 3: ConvLSTM. In the formula, \(M_{c}(H)\) represents the attention weight calculated from the high - dimensional feature map. \(Conv_{3\times 3}\) represents the 3 \(\times\) 3 convolutions, \(L\) represents the low - dimensional feature map, and \(M_{c}(L)\) represents the feature map after JAM processing. Adding the attention mechanism on image channels was proposed in 2018 by [PERSON] et al. 's SE-Net model [18]. This paper adds a parallel maximum pooling layer compared to the channel attention mechanism in SE-Net. A three-layer perceptron is also used, instead of the original 1 \(\times\) 1 convolution, to extract the channel attention information, and, finally, the channel attention weights with the same number of channels and low-dimensional features are output. The structure of the channel attention mechanism designed in this paper is shown in Figure 5.  
The channel attention mechanism computes the maximum pooling and average pooling of the input features and feeds them to the \(MLP\) unit, a two-layer neural network for computing the attention weights on the picture channels.