# Efficient Semantic Segmentation Using Multi-Path Decoder
## 3 Method  
The framework of our model is demonstrated in Figure 1, termed as MPDNet. All semantic segmentation methods face a contradiction between accuracy and speed. The backbone used in our framework is ResNet. There are four stages, Res1, Res2, Res3, Res4 in ResNet. The spatial resolution of feature maps put out by each stage of ResNet is 1/4, 1/8, 1/16, 1/32 of the original image. These feature maps are denoted as \(R_{1},R_{2},R_{3},R_{4}\), respectively. The feature maps from four different stages have different sizes. Many models use feature maps with different receptive field sizes to represent different context information and then connect these feature maps to yield new feature maps which encode multi-scale context information. The DeepLab series is one of the best. Their proposed ASPP module contains one \(1\times 1\) convolutional layer and three \(3\times 3\) convolutional layers with dilation rates of 6, 12, 18 respectively. The feature maps output by the encoder of DeepLab is 1/16 of the original image during training and 1/8 during inference. To save training and inference time, we do not follow their method. This makes that the height and width of feature maps output by the encoder of our model is 1/32 of the original image. Thus, we use feature maps in the shallower layers with high resolution to compensate the low resolution of high-level features and yield accurate segmentation results. To encode multi scale context information, we concatenate feature maps output by each block of the backbone. At the same time, we found that under such circumstances, ASPP does not need to be as large as the original version in DeepLab to achieve good results. All the information is contained in the bottom feature maps in DeepLab (except DeepLabv3+). We do not need to be like this. Before inputting the  
Figure 1: An overview of the MPDNet. Feature maps of different sizes are extracted by the encoder. The main decoder has four identical paths to extract different types of features. Feature maps output by four paths in the main decoder are fused to generate segmentation results. The edge decoder learns edge information and generates boundary segmentation. bottom feature map into each atrous convolution branch, we input the bottom feature maps into a \(1\times 1\) convolutional layer to reduce the number of channels to 512. This makes the number of FLOPs used in the modified ASPP in our model at least 77% lower than that of ASPP in DeepLab during inference. And when we use four ASPPs with 64 output channels instead of one with 256 output channels, the segmentation result is better and the model is smaller. In semantic segmentation task aiming at real-time inference, a powerful and efficient upsampling datapath is very important. The most efficient upsampling datapath is to use a \(1\times 1\) convolutional layer as lateral connection and fuse enlarged feature maps with feature maps delivered by lateral connection. Then send the largest feature maps to a classifier. Efficient but powerless. Many previous \"U-shape\" models use several \(3\times 3\) convolutions as the lateral connection. Then upsample these feature maps to a quarter of the original image, concatenate them and reduce the channel dimension to the number of object categories. We find that the concatenation operation improve the segmentation results a lot. But the concatenation is operated on the largest feature maps, which requires many FLOPs. Therefore, we use a new lateral connection method. In order to extract different type of features, we use four identical ASPP modules and four identical decoders for four ASPPs. In each decoder, we denote the feature maps output by the ASPP module as \(LC_{4}\). \(LC_{4}\) encodes the highest level of semantic information. Then the number of channels of \(R_{3}\) is reduced to 64 by a lateral connection module which contains a \(64\times C\times 1\times 1\) convolutional layer, one \(64\times 64\times 3\times 3\) convolutional layer with 64 groups and one \(19\times 64\times 1\times 1\) convolutional layer.