# Image Semantic Segmentation Fusion of Edge Detection and AFF Attention Mechanism
## 3 Proposed Model
### Loss Function  
We design the loss function according to the positions of the AFF attention mechanism. The loss function of the model is the sum of the loss weights of the improved HED network and the improved PSP-Net network. That is, \(L_{\text{sfe}}=L_{\text{sf}}+L_{\text{se}}\). \(L_{\text{sfe}}\) is the loss function of the improved HED network and the improved PSP-Net network, \(L_{\text{sf}}\) is the loss function of the improved HED network, \(L_{\text{se}}\) is the loss function of the improved PSP-Net network. \[L_{\text{sf}}=\text{argmin}(L_{s}(W,w)+L_{f}(W,w,h)) \tag{3}\]  
\[L_{\text{se}}=(1-\gamma)(L_{s}+L_{f}) \tag{4}\]  
Figure 16: Four different joining positions. (**a**) corresponds to position I in Figure 15, adding AFF to the improved HED network; (**b**) corresponds to position II in Figure 15, adding AFF to the encoder MobileNetV3 network; (**c**) corresponds to position III in Figure 15, adding AFF to the PSP-Net decoder; (**d**) corresponds to position IV in Figure 15, adding AFF to the fusion of semantic features and edge features. The loss function of the I AFF is:  
\[L_{\text{sf-}A}=\theta_{1c}[\text{argmin}(L_{s}(W,w)+L_{f}(W,w,h))]+\theta_{2c}L_{ A}(m,m^{*}) \tag{5}\]  
In Equation (5), \(L_{\text{sf-}A}\) is the loss function when the AFF is introduced into the improved HED network, \(L_{s}\) is the loss function of the five side output networks in the improved HED network. \(L_{f}\) is the loss function of edge information fusion. \(W\) is the set of parameters in the network, \(w\) is the set of parameters of the output layer, and h is expressed as the weight of the ensemble. \(\theta_{1c}\) and \(\theta_{2c}\) are the parameters of the equilibrium loss function \(L_{\text{sf-}A}\), which are both set to 0.5 in this paper. The loss function of the II AFF is:  
\[ \begin{split}& L_{E1-A}=-\theta 1a\big{\{}\sum\limits_{j=1}^{l} {(1-y^{j})\ln[(1-p_{8}^{j})(1-p_{16}^{j})(1-p_{32}^{j})]}\\ &+\sum\limits_{j=1}^{l}{y^{j}\ln(p_{8}^{j}p_{16}^{j}p_{32}^{j})+ \beta\sum}||\omega||_{2}\big{\}}-\theta_{2a}L_{A}(m,m^{*})\end{split}  \tag{6}\]  
In Equation (6), \(L_{E1-A}\) is the loss function when the AFF is added to the encoder side. It is the sum of the loss function of encoder and the loss function of AFF. The encoder side uses the MobileNetV3 network to extract multi-level feature information. The feature maps of 1/8, 1/16 and 1/32 sizes of the coding layer are selected for training, so the probability feature maps of class j of the 1/8, 1/16 and 1/32 layers are defined as \(p_{8}^{j}\), \(p_{16}^{j}\) and \(p_{32}^{j}\), respectively, which are the expected output of class j. \(L_{A}(m,m^{*})\) is the loss function of the AFF attention mechanism, a per-pixel cross-entropy loss function based on the mask. \(m\), \(m^{*}\) are the masks generated by the attention mechanism and their corresponding mask labels. \(\theta_{1b}\) and \(\theta_{2b}\) are balanced loss functions with the value 1. The loss function of the III AFF is  
\[L_{E2-A}=\theta_{1b}\{-\sum\limits_{j=1}^{l}{[y^{j}\ln\overset{\cap}{k^{j}}+ (1-y^{j})\ln(1-k^{j})+\beta\sum}||\omega||^{2}\}+\theta_{2b}L_{A}(m,m^{*}) \tag{7}\]  
In Equation (7), \(L_{E2-A}\) is the loss function when AFF is added to the PSP-Net decoder side. It is the sum of the loss function on the decoder side and the loss function of AFF. \(k^{j}\) is the jth probabilistic feature layer of the network output layer. \(\theta_{1b}\) and \(\theta_{2b}\) are the parameters of the equilibrium loss function \(L_{E2-A}\). In this paper, both are set to 1. The loss function of the IV AFF is  
\[L_{\text{sfc-}A}=\theta_{1d}[\gamma(L_{E1}+L_{E2}+L_{Dice})+(1-\gamma)(L_{s}+ L_{f})]+\theta_{2d}L_{A}(m,m^{*}) \tag{8}\]  
In Equation (8), \(L_{\text{sfc-}A}\) is the loss function after adding AFF to the semantic and edge feature fusion part. It is the sum of the loss function of the edge detection network, the loss function of the semantic segmentation network, and the loss function of AFF.