# Portrait Semantic Segmentation Method Based on Dual Modal Information Complementarity
## 2 Related Works  
### Semantic Segmentation Based on CNN  
Convolutional neural network (CNN) can automatically learn different mixtures of representational features from training images to perform semantic segmentation of images [5]. On this basis, academics have designed a large number of new efficient semantic segmentation algorithms, of which the semantic segmentation model represented by U-net [6] has now become a popular choice for semantic segmentation as it can perform well on smaller datasets. The structure of U-net is shown in Figure 1, which is mainly divided into three parts: the encoder, the skip connection, and the decoder. In the U-net network, the encoder stageis responsible for the feature extraction of the image, where successive down-sampling operations allow the model to obtain the low-resolution information of the image. The decoder stage is responsible for the reduction of the image feature map, which regains the deep feature information of the image. The hopping connection stage connects the encoder output features with the decoder stage features, which helps the model to effectively fuse the deep and shallow features of the image. The establishment of skip connection can help the model to achieve semantic segmentation at the pixel level, which can provide more fine-grained features for segmentation.  
However, when U-net is applied to the field of portrait semantic segmentation, the fixed receptive field of the convolutional kernel in the encoder makes the model unable to adaptively adjust the receptive field, which leads to the model being unable to capture the pixel information of the different sizes of portrait targets, resulting in the loss of some of the scale information of the image [7]. To address such problems, [PERSON] et al. [8] proposed the deeplad v3+ model, which better solves the problem of the model failing to capture the information of different scales in the image, and also proposed the Atrous Spatial Pyramid Pooling (ASPP) module. Among them, the ASPP module parallelizes the void convolution of multiple branches with different expansion rates, to extract multi-scale features of different sizes of sensory fields, which helps the model to better focus on the pixel information of different sizes of target objects in the image [9]. However, deeplad v3+ is not sufficient for the multi-scale connection of feature map [10], and does not pay attention to the semantic gap between the encoder and decoder features, due to the direct connection between the encoder and the decoder, which may lead to poor target edge boundary segmentation accuracy. To address such problems, [PERSON] et al. [11] proposed Attention U-net, which adds attention coefficients to the down-sampling results and prunes the feature map to reduce the semantic gap by introducing an attention gate in the jump connection stage. While a single attention mechanism lacks the interaction between local feature information and global information, various combinations of attention mechanisms can help the model pay more attention to pixel location information [12]. In this regard, by linking multiple attention mechanisms in the hopping stage, we can obtain the attention weights of feature vectors in multiple dimensions, such as channel or space, so as to give more deep information to the down-sampling results, and thus reduce the semantic gap between the up-sampling results and the down-sampling results [13; 14].