# Portrait Semantic Segmentation Method Based on Dual Modal Information Complementarity
## 3 Methods
### Loss Function  
In order to obtain the context information of the target portrait in the image, we use void convolution in the model encoder phase of this paper. Compared with standard convolution, although it has a larger receptive field, its edge segmentation accuracy will be reduced. [PERSON] et al. [27] proposed a multi-loss method to enhance the final semantic segmentation results, and used different data functions to constrain the data results in different aspects, achieving a good effect on the output of the constrained model. Inspired by this, the model in this paper optimizes the semantic segmentation results by mixing losses, which means that the output of the model is constrained by two loss functions. The composition of the loss function used in this paper is as follows:  
\[loss=\alpha loss_{pb}+\beta loss_{ssim} \tag{3}\]  
where \(loss_{pb}\) stands for pixel-based \(loss\), \(lossd_{ssim}\) stands for Structural Similarity. \(\alpha\) and \(\beta\) are bias parameters (where \(\alpha+\beta=1\)), which is to gradually extract information and perform preliminary evaluation and constraint on the segmented edge information and total precision information.  
Pixel-based loss: It is mainly used to compare the pixel-level difference between two images. It can be divided into two parts: the first part is the total amount of information carried by pixels, which is represented by binary cross entropy. The second part, based on the pixel value gap measurement, compares the difference between the segmentation results and the artificially-generated labels in pairs. It is defined as:  
\[\begin{split}& pbl=1-pri\\ &=1-\frac{1}{\left(\begin{array}{c}N\\ 2\end{array}\right)}\sum\limits_{i,i,i,j}\left[I\left(I_{i}^{S}=I_{j}^{S_{K}} \right)p_{ij}+I\left(I_{i}^{S}\
eq I_{j}^{S_{K}}\right)p_{ij}\right]\end{split} \tag{4}\]  
where \(N\) represents the number of pixels. \(S_{k}\) represents the set of split labels, \(S\) represents the set of groundtruth, and \(P_{ij}\) represents probabilities. \(P_{ij}\) is calculated based on the average of all divided pixel pairs. The \(p_{ii}\) is between 0 and 1, where 0 means that the segmentation result is opposite to the segmentation label, and 1 means that the segmentation result is exactly correct and each pixel is accurately classified.  
_ssim_: It is mainly used to measure the degree of similarity between the predicted sample and the real sample. Because it takes into account the local neighborhood of each pixel, a higher weight can be assigned to the boundary [28]. It is defined as:  
\[ssim=1-\frac{\left(2\mu_{x}\mu_{y}+C_{1}\right)\left(2\sigma_{xy}+C_{2}\right) }{\left(\mu_{x}^{2}\mu_{y}^{2}+C_{1}\right)\left(\sigma_{x}^{2}+\sigma_{y}^{ 2}+C_{2}\right)} \tag{5}\]  
where \(x\) and \(y\) are denots as pixels of two pictures respectively, \(\mu_{x}\) and \(\mu_{y}\) are the mean values of \(x\) and \(y\), and \(\sigma_{x}\) and \(\sigma_{y}\) are the standard deviations of \(x\) and \(y\). \(C_{1}\) is set to 0.022 and \(C_{2}\) is set to 0.042 to prevent the denominator from being 0 [29].  
When combined with these two loss functions, the details of the target portrait boundary can be optimized effectively in the segmentation process, thus improving the overall segmentation accuracy of the model.