# Portrait Semantic Segmentation Method Based on Dual Modal Information Complementarity
## 1 Introduction  
Semantic segmentation is one of the main research topics in the field of computer vision, and it is a necessary process for computers to recognize the information features of images from the whole to the pixel level. The core of semantic segmentation is to divide images into non-overlapping sub-blocks with different semantic labels based on the information features of image itself, such as gray distribution between image pixels and image texture [1]. Portrait semantic segmentation is an important part of semantic segmentation. The purpose of portrait semantic segmentation is to separate the portrait semantic region from the complex background information semantic region in the picture- containing portrait, so as to obtain a good portrait boundary [2]. In real life, people transform scenes into images by cameras or other devices, and then segment and label the semantics of objective portrait objects in the images by using algorithms, so as to carry out applications and facilitate people's lives. A well-trained high-precision portrait segmentation model can also be used as a preprocessing module of the whole portrait intelligent system, avoiding the feature extraction of irrelevant information, such as background, in the subsequent operation, and thus improving the accuracy of the whole system [3]. However, the real portrait image has the characteristics of different target sizes and various background information, which makes the semantic segmentation of portrait quite challenging. First of all, compared with conventional semantic segmentation, the portrait semantic segmentation model needs to face more images with larger target sizes, which tests the generalization of the model and makes it more difficult to guarantee the segmentation details of the model [4]. Meanwhile, the existing model is prone to the phenomena of target portrait lossand background misjudgment in low-light scenes, such as nighttime, due to the fact that the background pixels and target pixels in the portrait image are relatively close to each other, and the segmentation accuracy of the model needs to be improved.  
In this paper, in order to solve the above problems, we propose a portrait semantic segmentation model based on dual-modal information complementarity that is in turn based on U-net network. The main contributions of this paper are summarized as follows:  
1. In the encoder stage, we extract image feature based on RGB mode and gray mode: RGB image and grayscale image are simultaneously used as inputs to model encoder, corresponding feature representation is obtained, and feature fusion is carried out. The gray image can be directly converted by RGB image, which avoids the overhead of collecting other modal images in the data set, and realizes the information complementarity and feature extraction in the dual mode. The feature extraction of gray-scale image improves the ability of the model to extract the target figure under low light.
2. We propose the SK-ASSP (Selective Kernel--Atrous Spatial Pyramid Pooling) module as a new feature extraction module in the encoder stage: SK-ASSP dynamically adjusts the weight of the void convolution of different receptive fields according to the size of the target in the image through soft attention, thus providing feature extraction capabilities for portrait targets of different sizes. At the same time, SK-ASSP uses edge optimization branch to improve the edge segmentation accuracy of target portrait according to edge feature map.
3. We propose the mixed-parallel attention mechanism to improve the encoder-decoder connection process: through parallel channel attention module and position attention module, the channel information and location information of input features are captured respectively.