# DTS-Net: Depth-to-Space Networks for Fast and Accurate Semantic Object Segmentation
## 1 Introduction  
Semantic segmentation is an important task in computer vision, as it constitutes pixel-wise classification of an image to mask each object in the scene. Hundreds of applications, such as autonomous driving, robotics, medical diagnostics, image editing, and augmented reality applications, incorporate semantic segmentation. Recent studies on semantic segmentation have achieved promising results using convolutional neural networks (CNNs), particularly encoder-decoder CNN architectures. In such architectures, the semantic segmentation task is modeled in two stages: the encoding stage, in which the image is down-sampled to obtain its deep semantic features, and the decoding stage, in which the semantic features are up-sampled to obtain a semantic segmentation mask of the same size as the input image. Encoder-decoder architecture can achieve highly accurate segmentation results; however, the decoder stage adds considerable computational complexity to the overall model and the input image size of such models is usually small. Furthermore, the decoding-stage reconstruction is inefficient because de-convolution or up-sampling layers are used, which eliminate small details and sometimes propagate noise. The semantic segmentation problem can be seen as an image-to-image translation problem, as the aim of this task is to construct segmentation masks equivalent to each object in the input image. Further, the objects in the segmentation mask have the sameboundaries as the input image; thus, the segmentation mask can be considered as an alternative to the image and, hence, the segmentation task can be performed using image reconstruction techniques. The efficient sub-pixel CNN [1] has shown promising results in image and video super-resolution tasks because of its depth-to-space (DTS) layer, from which a high-resolution image can be reconstructed from many low-resolution images. This layer performs image reconstruction through pixel reordering of the low-resolution feature maps obtained from the CNN network, in order to form super-pixels of the high-resolution image with very accurate borders and object details. Thus, the DTS layer efficiently obtains accurate semantic segmentation masks with clear borders, at far higher speeds than the traditional encoder-decoder architectures usually used for segmentation tasks. The DTS layer has many advantages, including its lower computation count and considerably higher accuracy compared with the decoder stage of the encoder-decoder architecture. Therefore, modeling can be accelerated, while high segmentation accuracy is maintained. Vision transformers, which apply a transformer network to images in a manner similar to natural language processing, are a recent achievement in computer vision. Vision transformers allow parallelization in the processing of a sequence dataset or a sequence of patches of the target image. This is achieved through positional encoding, which allows the network to learn the position of the patch in the input \"big image\". Vision transformer-based methods achieve better accuracy than CNN-based methods for classification, segmentation, and object detection tasks without increased computational cost. In this study, we propose the DTS-Net deep network for semantic segmentation using a sub-pixel CNN, to address the semantic segmentation complexity problem that arises for encoder-decoder architectures with or without attention methods while retaining high segmentation accuracy. We also present DTS-Net-Lite, a lightweight version of this network.