# Efficient Semantic Segmentation Using Multi-Path Decoder
###### Abstract  
Benefiting from the booming of deep learning, the state-of-the-art models achieved great progress. But they are huge in terms of parameters and floating point operations, which makes it hard to apply them to real-time applications. In this paper, we propose a novel deep neural network architecture, named MPDNet, for fast and efficient semantic segmentation under resource constraints. First, we use a light-weight classification model pretrained on ImageNet as the encoder. Second, we use a cost-effective upsampling datapath to restore prediction resolution and convert features for classification into features for segmentation. Finally, we propose to use a multi-path decoder to extract different types of features, which are not ideal to process inside only one convolutional neural network. The experimental results of our model outperform other models aiming at real-time semantic segmentation on Cityscapes. Based on our proposed MPDNet, we achieve 76.7% mean IoU on Cityscapes test set with only 118.84 GFLOPs and achieves 37.6 Hz on 768 \(\times\) 1536 images on a standard GPU.  
deep learning; semantic image segmentation; convolutional neural network +
Footnote †: journal:  
inside only one convolutional neural network. Because they are very different kinds of information, this approach may not be optimal. Therefore, we propose a multi-path decoder. Different types of features are extracted by different branches in the decoder. Then, feature maps output by different branches are fused to generate segmentation results.  
In recent years, some studies have used edge detection as auxiliary information to help intensify the prediction of pixels on the edge of things and stuff. In [10], [PERSON] et al. proposed to process shape information in a separate branch, resulting in a two-stream convolutional neural network (CNN) architecture for semantic image segmentation. [11] treated the edges as another semantic category to enable the network to be aware of the boundary layout. [12] extended the CNN-based agnostic edge detector proposed in [13] and allows each edge pixel to be associated with multiple categories. Although the shape branch helps to improve the accuracy of the semantic segmentation model, we think it is unnecessary to incorporate an edge detection branch to the semantic segmentation model during inference. Among them, we propose a dual-stream CNN structure for semantic segmentation. We explicitly links shape information to a single processing branch, namely the shape stream, but the edge detection task and the semantic segmentation task in our model are independent. We do not use the edge branch during inference, which reduces computational complexity of our model. As an additional semantic class, edge learning enables the network to understand the layout of boundaries.  
In this paper, we present an effective lightweight architecture for semantic segmentation on datasets of few annotated images, based on ResNet features and multi-path decoder. The results of our proposed approach cannot match the state-of-the-art models, but smaller and faster than them. To our best knowledge, our model surpasses any other models aiming at real-time inference. By experimenting and evaluating on two public datasets, Cityscapes and ADE20K, we prove the effectiveness of our approach. Our MPDNet achieves 76.7% mean IoU (Intersection over Union) on the Cityscapes test set and 43.14% on ADE20K validation set with the single scale input.  
The paper is structured as follows. In Section 2, we introduce two kinds of semantic segmentation models. In Section 3, we demonstrate our approach in detail. We give an overview of the whole architecture, elaborate the core module in our model and describe the computation process of the framework. In Section 4, the ablation experimental results and the comparison with other semantic segmentation methods are shown to prove the effectiveness of our approach. Finally, our conclusions are shown in Section 5.