## 2 DeepLab v3+ and Proposed Framework  
### DeepLab v3+  
DeepLab v3+ [17] is the latest work in the DeepLab series neural networks of semantic segmentation. Its predecessors include DeepLab v1, v2, and v3. The DeepLab series of neural networks use deep convolutional neural networks (DCNNs), which have excellent translation invariance so that they have good image-level classification capabilities. However, it is difficult for DCNNs to deal with the pixel-level classification problem, which is also an urgent problem for semantic segmentation. Therefore, in light of the following main problems, the author [17] has proposed solutions in different versions of DeepLab.  
Continuous convolution and pooling in DCNNs inevitably lead to a decrease in the resolution of feature maps, which affects the final segmentation accuracy. Inspired by the paper [18], the author of DeepLab used the atrous convolution in the v2 version. The mathematical reasoning process of the hole convolution is described in detail [18] elsewhere; we discuss it briefly. Figure 1 represents the comparison of convolution principles in two different ways.  
Let \(F:Z^{2}\to R\) be a discrete function, and \(\Omega_{r}=[-r,r]^{2}\cap Z^{2}\)\((r>0)\). Let the convolution kernel be \(k:\Omega_{r}\to R\), then the size of the convolution kernel is \((2r+1)^{2}\). The definition of discrete convolution \(*\) can be obtained from the above conditions:  
\[(F*k)(p)=\sum_{\phi+t=p}F(s)k(t) \tag{1}\]  
when considering the atrous convolution, let \(l\) be the hole coefficient. The atrous convolution \(*_{l}\) with the atrous coefficient \(l\) can be defined as:  
\[(F*_{l}k)(p)=\sum_{s+lt=p}F(s)k(t) \tag{2}\]  
Secondly, another major problem with DCNNs is that due to the fixed parameters of the fully connected layer at the end of the network, the input images need to have a fixed size. The usual solution is to crop or scale the input image. However, in the process of cropping and scaling, the target object in the picture is deformed, resulting in a decrease in recognition accuracy. To address this problem, atrous spatial pyramid pooling (ASPP) is added to the DeepLab series neural network combining the design ideas of atrous convolution and spatial pyramid pooling (SPP) [19], which are represented in Figure 2.  
Figure 1: Illustrations of (**a**) convolution and (**b**) atrous convolution.  
The spatial pyramid pooling structure is generally placed before the last fully connected layer of the network. When the feature map of the size \((w,h)\) output by the previous layer is passed into the SPP, it is divided multiple times. The same image is divided into \(m\times n\) image blocks of size \((w/m,h/n)\) by the division scale \((m,n)\). Each image block in each divided image is put into the pooling layer separately and finally linked to a fully connected layer after joining. The biggest advantage of the structure is that it solves the size problem of the input picture. Due to the flexibility of the input size, the network can extract the convergence features under different pooling window sizes to improve the classification accuracy. Similarly, each feature map is separately convolved by convolution kernels with different dilation rates and then concatenated together in an atrous spatial pyramid pooling structure.  
In the v3+ version, in order to integrate multiscale information, the v3 network is designed as an Encoder, and then the spatial resolution is restored in the Decoder structure. The Encoder-Decoder model is commonly used in semantic segmentation. During the test, DeepLab v3+ showed good segmentation performance. In the PASAL VOC 2012 dataset, it produced an mIOU index of 89.0, which is the best performance so far. Therefore, we choose DeepLab v3+ as the main network for feature extraction in our research. Figure 2(a) shows the network structure of DeepLab v3+.  
Figure 3: \(Cont\).  
Figure 2: Illustration of (**a**) spatial pyramid pooling (SPP) structure and (**b**) atrous spatial pyramid pooling (ASPP) structure in DeepLab v3+.