# Residual-Attention UNet++: A Nested Residual-Attention U-Net for Medical Image Segmentation
## 2 Related Work
SegNet [18] is a symmetrical segmentation network of an encoder and decoder, which achieves end-to-end pixel-level image segmentation [15]. The encoder network is basically the same as VGG16 in structure, which is stacked by several convolution layers and pooling layers. SegNet discards the full connection layer to retain the high-resolution feature map at the encoder output position, and greatly reduces the number of parameters in the encoder. The decoder corresponds to the encoder. Its function is to restore the low-resolution codingfeature map to the complete input resolution, and the output will finally be passed to the multi category softmax classifier, which can be used in the pixel-level classification task to generate the independent probability of pixels in different categories. Based on FCN, [PERSON] et al. [19] proposed a deep model named \"U-Net\" in 2015, which is dedicated to biomedical image segmentation. Convolutional encoding and decoding units are the two main parts of this structure, as shown in Figure 1. In these two parts of the network, the ReLU activation follows the basic convolution operation while 2 \(\times\) 2 max pooling follows the activation operation in the encoding unit. On the other hand, the up-convolution operation is implemented in the decoding unit, which can up-sample the feature maps. Crop and copy are used to perform feature fusion between encoding unit and decoding unit, which are pivotal due to the loss of border pixels in every convolution. U-Net model ensures the accuracy of positioning and the acquisition of context information at the same time. In addition, this model can achieve higher segmentation results with less training samples, which is very important for the small number of medical image samples. Different variants of U-Net have been proposed in the past few years. For instance, [PERSON] et al. [20] proposed a network called \"3 DU-Net\" for volumetric segmentation that learns from sparsely annotated volumetric images, which extends the previous U-Net architecture by replacing all 2D operations with their 3D counterparts. [PERSON] et al. [21] proposed a volume-based, fully convolutional, end-to-end deep learning model for 3D medical images, which is called \"V-Net\". It consists of upsampling and downsampling. The main function of the former is to extract image features, including five convolution units, each of which is composed of convolution layer, pooling layer, and activation function. Convolution adopts 5 \(\times\) 5 \(\times\) 5, with a filling size of 2 and a stride of 1. The pooling layer can reduce the parameters in the training process, reduce the memory occupation rate, and improve the training speed. The activation function is the PReLU [22] function. The residual learning structure is also added to the convolution block to effectively alleviate the problem of gradient disappearance. There is a cascade between downsampling and upsampling. The Cascade fuses the feature map in the downsampling process with the feature map after the deconvolution of the upsampling, and then the convolution operation is carried out, which can effectively prevent the loss of features in the downsampling process, and retain the target location information in the downsampling part and the edge features of the image. [PERSON] et al. [8] proposed a network called \"R2U-Net\" for medical image  
Figure 1: The structure of the U-Net. segmentation, which replaces the plain convolution block in the encoder-decoder network with the recurrent residual convolution units (RRCU). The advantage of R2U-Net model is that it performs element-by-element feature summation outside the U-Net model, and integrates high-dimensional abstract feature information and low-dimensional detailed feature information. Besides, the feature accumulation inside the RRCU structure can ensure a stronger feature representation and effectively increase the depth of the network structure. H-Dense UNet [23] was proposed for liver and tumor segmentation in 2017, which is a novel, hybrid, densely connected U-Net.