# Efficient Semantic Segmentation Using Multi-Path Decoder
## 3 Method
By adding upsampled \(LC_{4}\) and reduced \(R_{3}\), new feature maps are generated and are denoted as \(LC_{3}\). In the same way, we gradually enlarge feature maps from bottom to top and use lateral connections to fuse features encoded by shallower layers of ResNet. These feature maps are denoted as \(LC_{1},LC_{2},LC_{3},LC_{4}\), respectively. They have different sizes, gradually decreasing from \(LC_{1}\) to \(LC_{4}\) by a ratio of 2. To fuse them, we enlarge them up to the size of \(LC_{1}\) by bilinear interpolation and concatenate these resized feature maps. And then a convolutional layer is followed to reduce the channel dimension of these concatenated feature maps to the number of object categories. Then we concatenate feature maps output by four decoders and another convolutional layer reduce the channel dimension of these concatenated feature maps to the number of object categories. In order to supervise each path of the decoder, we calculate loss for feature maps output by each path of the decoder. Four classifiers are applied after four paths of the main decoder. The above process is summarized in Algorithm 1. Where \(LC_{ij}\) represents the \(j\)-th level feature maps in the \(i\)-th branch of the main decoder. \(C_{i}\) represents the concatenation of four level feature maps in the \(i\)-th branch. \(S_{i}\) represents the prediction of the \(i\)-th branch. \(S\) represents the final prediction of our model. ```
1:The image need to be segmented, \(Image\);
2:The segmentation result, \(S\), the main loss, \(L_{m}\), and the loss for each decoder, \(L_{d}\). 3:\(R_{j}=Encoder(Image)\), where \(j\in[1,4]\);
4:for each \(i\in[1,4]\)do
5:\(LC_{i,4}=ASPP(R_{4})\);
6:for each \(j\in[1,3]\)do
7:\(LC_{ij}=RN(R_{j})+Upsample(LC_{i,j+1})\);
8:endfor
9:\(C_{i}=Concat(LC_{i,j})\), where \(j\in[1,4]\)
10:\(S_{i}=Conv(C_{i})\)
11:\(L_{d,i}=\)NLLLoss\((S_{i},Label)\)
12:endfor
13:\(S=Conv(Concat(S_{i}))\),where \(i\in[1,4]\)
14:\(L_{m}=\)NLLLoss\((S,Label)\)
15:\(L_{d}=\sum L_{d,i}\)
```  
**Algorithm 1** Main Decoder in the Multi-Path Decoder Network. Considering the importance of edge information in semantic segmentation, we use another decoder to learn edge information in addition to the main decoder. Like deep supervision proposed in PSPNet for their model, what the edge decoder learns is a supervision for our model. In the edge decoder, we also use ASPP module and gradually recover edge information by combining information encoded by the shallow layers. The steps of generating the predictions of edges in the edge decoder are the same as main decoder. The entire process for edge decoder is summarized in Algorithm 2. ```
0: Feature map output by the encoder, \(R_{j}\), where \(j\in[1,4]\);
0: The edge loss, \(Le\). 1:\(LC_{4}=ASPP(R_{4})\);
2:for each \(j\in[1,3]\)do
3:\(LC_{j}=RN(R_{j})+Upsample(LC_{j+1})\);
4:endfor
5:\(C=Concat(LC_{j})\), where \(j\in[1,4]\)
6:\(S=Conv(C)\)
7:\(Le=\)NLLLoss(\(S,EdgeLabel\))
```  
**Algorithm 2** Edge Decoder in the Multi-Path Decoder Network. In our proposed model, we need to calculate three kind of losses in total, the main loss \(L_{m}\), the supervision loss for each decoder \(L_{d}\) and the edge supervision loss \(L_{e}\). Thus, the integrated loss function \(L_{total}\) is finally formulated as below:  
\[L_{total}=L_{m}+\alpha L_{d}+\beta L_{e}, \tag{1}\]  
where \(\alpha\) and \(\beta\) are weights for balancing the losses.