# Residual-Attention UNet++: A Nested Residual-Attention U-Net for Medical Image Segmentation
## 2 Related Work
H-Dense UNet transforms 3D volume blocks into 2D adjacent slices by using the transform processing function F; then these 2D slices are sent to 2D DenseUNet to extract on-chip features. The original input of 3D and the prediction result after 2D DenseUNet conversion are concatenated and sent to the 3D network to extract inter-slice features. After the fusion training, the results obtained from the training and the results obtained from the previous 2D training are added to increase the feature fusion. Finally, Finally, the final result is predicted by HFF. Attention U-Net [24] introduces the attention mechanism into U-Net, intensively extract significant features useful for specific tasks (such as relevant tissues or organs) from medical images, and suppresses the irrelevant inputs [25]. Before splicing the features on each resolution of the encoder with the corresponding features in the decoder, an attention module is used to readjust the output characteristics of the encoder. The module generates a gating signal to control the importance of features at different spatial locations. [PERSON] et al. [26] proposed a new medical image segmentation model called \"UNet++\", in which the encoder and decoder sub-networks were connected through a series of nested, dense skip pathways. The improvement of UNet++ from U-Net is that UNet++ adopts redesigned skip pathways to connect the encoder network and the decoder network. In U-Net, the feature map in the encoder network is directly transmitted to the decoder network; however, through the redesigned skip pathways in UNet++, the feature map of the encoder network is mapped to the decoder network through dense convolution blocks. In fact, the feature graph semantic level in the encoder is close to the feature graph semantic level in the decoder through dense convolution blocks. Relying on the redesigned skip pathways, UNet++ improves the semantic gap between the feature maps of encoder and decoder subnetworks; however, dense convolution blocks also affect the gradient propagation, which will make the training process more complicated. According to the residual neural network proposed by [PERSON] et al. [27], it can facilitate training and address the degradation problem. The residual network consists of sequentially stacked residual units, which can be illustrated as the following form:  
\[ \begin{cases}y_{l}=x_{l}+\mathcal{F}(x_{l},\mathcal{W}_{l})\\ x_{l+1}=L(y_{l})\end{cases}  \tag{1}\]  
where \(x_{l}\) and \(x_{l+1}\) are the input and output of the \(l^{th}\) residual unit respectively, \(\mathcal{F}(\cdot)\) is the residual function, and \(\mathcal{W}_{l}\) are parameters of the block. \(L(y_{l})\) is the activation function. Figure 1(a) presents the plain and residual unit. As shown in Figure 1(b), the residual unit contains two combinations of batch normalization (BN), ReLU, and convolutional layer. Although the residual unit improves the gradient propagation, it also increases the network parameters, which can easily lead to overfitting. For this, we introduced another module called dropout [28] in the residual unit. Dropout is a regularization technique, which drops a unit (along with connections) at training phase with a certain probability p (a common value is \(p=0.5\)). It was mostly applied on top layers that had a large number of parameters to prevent feature coadaptation and overfitting [29]. As shown in Figure 1(c), the dropout layer was added behind the first convolution layer in the residual unit. Attention mechanism is a core technology that has been widely used in natural language processing (NLP), statistical learning, image detection, speech recognition, and other fields. It was the proposal of Non-local [30] that applied the attention mechanism to computer vision for the first time. In order to focus on the target organs related to the segmentation task, we refer to the method proposed by Attention U-Net, which adds an Attention Gate to the network architecture.