# DTS-Net: Depth-to-Space Networks for Fast and Accurate Semantic Object Segmentation
## 2 Related Work  
Recent semantic segmentation studies have shown that encoder-decoder architecture can efficiently perform segmentation tasks. The first encoder-decoder architecture was the fully convolutional network (FCN) [5], in which the same architecture was used for image classification but the final dense layers were replaced with 1 \(\times\) 1 convolutional layers having the same weights. The decoder stage was a simple up-sampling layer. The FCN obtained relatively good results, motivating further research on encoder-decoder architectures. Later, SegNet [6] was proposed as a deep encoder-decoder architecture. This architecture features pooling indices shared between the max-pooling layers in the encoder stage and the corresponding max-unpooling layers in the decoder stage. SegNet exhibited impressive semantic segmentation results on outdoor and indoor segmentation datasets. U-Net [7] is another impressive encoder-decoder architecture, which was proposed for microscopy cell segmentation in medical images. U-net suggests residual connections between corresponding layers in the encoder and decoder stages and, hence, achieves considerable segmentation accuracy. The four versions of DeepLabVx also constitute considerable contributions to the semantic segmentation task. DeepLabV1 [8] tackled the problem of inefficient down-sampling through a wide field of view convolution using the Atrous convolution [8], which increases the spatial field of the convolutional window using the same weights as the normal convolution. In that work, the conditional random field (CRF) was also proposed, which uses an energy function derived from the summation of a unary potential term calculated from the probability distribution of the output label of each pixel, along with a binary potential term calculated from the correlation between pixel labels. In general, the CRF allows the model to learn small image details. With DeepLabV2 [9], Atrous Spatial Pyramid Pooling (ASPP) was proposed to enhance the model learning at multiple scales of the feature maps. In addition, the VGG16 [10] used in DeeplabV1 was replaced with ResNet [11], which yielded better performance. The ASPP was further improved for DeepLabV3 [12], with the use of different sampling rates in the ASPP in a cascaded manner. Finally, for DeepLabV3+ [13], the encoder architecture was replaced with a depth-wise separable convolution-based architecture, and Aligned Xception was adopted. The latter is a modified version of Xception [14] that replaces the max-pooling layers in the original  
Figure 1: Sample predicted and filtered semantic segmentation results of proposed method: (top to bottom) samples from PASCAL VOC2012 [2], NYUV2 [3], and CITYSCAPES [4] benchmarks, respectively. architecture with strided convolutional layers with more Xception blocks; this facilitates higher accuracy and speed during processing. [PERSON] et al. proposed the pyramid scene parsing network (PSPNet) [15], which adopts a pyramid parsing module that learns the global context of the image through region-based aggregation. Pyramid pooling is employed to learn the context of the image from the final small feature maps. Later, [PERSON] et al. presented the image cascade network (ICNet) [16], which performs high-speed semantic segmentation on high-resolution images using cascade feature fusion. The latter approach mixes the features obtained from the input image on different scales in a method called cascade label guidance. Up-sampling layers are then used to resize the image to the input image size. As another approach, the bilateral segmentation network (BiSeNet) [17] performs real-time semantic segmentation using an architecture with two paths. The first is a spatial path for spatial information preservation and the other is a context path for general context learning through down-sampling. The features obtained by the two paths are then combined via a feature fusion technique. ResNeSt, developed by [PERSON] et al.