# Attention-Based Background/Foreground Monocular Depth Prediction Model Using Image Segmentation
###### Abstract  
While many monocular depth estimation methods have been proposed, determining depth variations in outdoor scenes remains challenging. Accordingly, this paper proposes an image segmentation-based monocular depth estimation model with attention mechanisms that can address outdoor scene variations. The segmentation model segments images into foreground and background regions and individually predicts depth maps. Moreover, attention mechanisms are also adopted to extract meaningful features from complex scenes to improve foreground and background depth map prediction via a multi-scale decoding scheme. From our experimental results, we observed that our proposed model outperformed previous methods by 27.5% on the KITTI dataset. deep learning; depth information; image segmentation; Laplacian pyramid; monocular depth estimation +
Footnote †: journal: Medical Image Segmentation  
background depth by fusing feature maps of different scales. The authors of [3] proposed a simple and fast multi-scale architecture with three prediction tasks. In addition, some researchers have adopted conditional random fields (CRFs) [4; 8; 9] to refine depth estimation techniques. The authors of [4] employed two CNNs to extract absolute and relative features that could reveal the absolute depth and relative distance between neighboring regions. Then, a CRF was applied to fine-tune the outputs of the two CNNs to estimate the depth of whole images. Supervision-based methods require vast quantities of corresponding ground truth depth data for training. In contrast, unsupervised monocular depth estimation methods [15; 16; 17; 18; 19; 20; 21; 22; 23; 24] can train models without any ground truth data. These methods [15; 16; 17; 18; 19; 20; 21; 22; 23; 24] use the geometric constraints between frames as supervisory signals during the training phase. The authors of [16] used an image reconstruction loss between calibrated stereo pairs of color images to improve depth estimation accuracy. Some studies [20; 22; 23; 24] have used ego-motion to predict depth; however, this may suffer from unidentified moving objects, resulting in poor prediction performance in certain areas. The authors of [15] proposed a geometry consistency loss for scale-consistent prediction that could avoid interference from moving objects and occlusion. Some studies [25; 26; 27; 28; 29; 30; 31; 32; 33] have implicitly modeled the relationship between semantic segmentation and depth to improve performance. The authors of [27; 28] proposed the joint training of a shared network with two different tasks: semantic segmentation and depth estimation. This was found to be helpful for both techniques. The authors of [29] proposed two types of propagation to learn transformations between semantic segmentation and depth feature spaces: cross-task and task-specific propagation. As more attention mechanisms have been developed, some researchers have attempted to add attention mechanisms to monocular depth estimation methods to improve prediction accuracy. Based on multi-scale CRFs [34], the authors of [35] used a structured attention method to automatically extract multi-scale features to automatically regulate flows from different scales. In addition, the authors of [36] proposed a patch-wise attention method to enhance local area prediction accuracy. Patch-wise attention can extract features between neighboring pixels in the channel and spatial dimensions in local areas. As shown in Figure 1, significant differences in field depth affect depth estimation such that models cannot adequately predict foreground and background depth maps at the same time, especially in outdoor scenes. Previous studies [3; 6; 7; 12; 13; 14; 35] have proposed multi-scale-based methods to solve this problem. However, these methods do not take advantage of image segmentation, which can partition images into several image segments, i.e., image regions or objects, that can provide hints to improve depth estimation.