# DTS-Net: Depth-to-Space Networks for Fast and Accurate Semantic Object Segmentation
## 5 Results
### DTS-Net for Joint Depth Estimation and Semantic Segmentation  
We trained DTS-Net on the simultaneous joint task of depth estimation and semantic segmentation, as discussed in Section 4.5. The proposed method yielded promising results even though the network performed both tasks simultaneously. The output semantic segmentation and depth estimation had fine details. DTS-Net extracted the important features for each task while using the same feature extractor but with different branches of 1 \(\times\) 1 \(\times\) 1024 convolutional layers, which were followed by the DTS layer. NLF filtration was then applied to the segmentation output. The network showed promising error values and accuracy for the depth estimation task while attaining a slightly inferior semantic segmentation performance. To evaluate the depth estimation, we evaluated the following depth estimation metrics:  
* the absolute relative error (REL): \(\frac{1}{n}\sum_{i}^{n}\frac{|y-\hat{y}|}{y}\);
* the relative difference squared (Sq_REL): \(\frac{1}{n}\sum_{i}^{n}\frac{||y-\hat{y}||^{2}}{y}\);
* the root mean squared error (RMSE): \(\sqrt{\frac{1}{n}\sum_{i}^{n}(y-\hat{y})^{2}}\);
* the disparity error (px): \(\frac{1}{n}\sum_{i}^{n}|y-\hat{y}|\);
* the threshold accuracy \(\delta_{i}\) of \(y_{p}\): \(max(\frac{y}{y},\frac{\hat{y}}{y})=\delta<thr\) for the commonly used threshold values \(thr=1.25,1.25^{2},1.25^{3}\). Figure 9: Quality comparison between the semantic segmentation results obtained from the proposed method (DTS-Net+NLF), DeepLab, DeepLabv3, PSPNet, and DeepLabv3+ on PASCAL VOC2012 validation benchmark, and between DTS-Net+NLF, PSPNet, and DeepLabv3 on CITYSCAPES validation benchmarks. where \(n\) is the number of pixels in the depth map, \(i\) indicates an iterator over the pixels, \(y\) and \(\ddot{y}\) are the ground truth and predicted pixel values, respectively. The joint trained network yielded high-quality results, as shown in Figure 10. DTS-Net+NLF attained a REL of 0.0987 and \(\delta_{1}\) of 90.5% on the CITYSCAPES benchmark. Regarding the semantic segmentation in the joint task, the network attained mIOU values of 72.1% with NLF filtration which is much lower than this for the single task. Further, DTS-Net+NLF attained a REL value of 0.102 and \(\delta_{1}\) of 94.59% while attaining an mIOU of 49.1% on the NYUV2 benchmark. The proposed method outperformed the SOTA methods for joint semantic segmentation and depth estimation on the depth estimation by some margin, however, the semantic segmentation was not compared because the segmentation configuration for the models in the comparison is not similar to ours. Table 6 presents the comparison for the depth estimation results with the SOTA methods in terms of REL, Sq Rel, RMSE, the disparity error (px), and the delta accuracies. The proposed method had low depth errors and high delta accuracy for depth estimation surpassing all other joint semantic segmentation and depth estimation accuracies and error values while maintaining a high mIOU for semantic segmentation. \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**Method** & **Dataset** & **REL** & **Sq Rel** & **RMSE** & \(\delta_{1}\) & \(\delta_{2}\) & \(\delta_{3}\) & **px** \\ \hline SemDepth+CRF [32] & NYUV2 & 0.158 & 0.121 & 0.641 & 0.769 & 0.950 & 0.988 & - \\ TRL-ResNet50 [33] & NYUV2 & 0.144 & - & 0.501 & 0.815 & 0.962 & 0.992 & - \\ HybridNet A2 [34] & CS & 0.240 & 4.27 & 12.09 & 0.597 & 0.822 & 0.929 & - \\  & NYUV2 & 0.202 & 0.186 & 0.682 & 0.613 & 0.892 & 0.974 & - \\ ESOSD-Net [35] & NYUV2 & 0.145 & - & 0.514 & 0.805 & 0.962 & 0.992 & - \\  & CS & - & - & - & - & - & - & 2.41 \\ \hline DTS-Net+NLF & NYUV2 & 0.102 & 0.503 & 0.310 & 0.946 & 0.997 & 0.999 & - \\  & CS & 0.098 & 1.093 & 7.627 & 0.905 & 0.956 & 0.971 & 1.62 \\ \hline \hline \end{tabular}
\end{table}