# A Sparsity-Invariant Model via Unifying Depth Prediction and Completion
## 2 Related Work  
### Monocular Depth Estimation  
The monocular depth estimation task infers the corresponding depth map from a single color image. Due to the absence of spatial information and depth scale from sparse depth points, most monocular depth estimation methods focus on exploring the geometric relationships in the scene. Some works [27; 28; 29] utilize the geometric constraint that \"the surface composed of any three points in the plane should be perpendicular to the surface normal vector of this plane\" to design corresponding network architectures and loss functions. Structdepth [28] introduces Manhattan normal constraints and coplanarity constraints. The Manhattan normal constraint aligns major surfaces, such as floors, ceilings, and walls, with dominant directions. The coplanarity constraint enforces coplanarity among a set of three-dimensional points within the same planar region. [PERSON] et al. [29] propose an adaptive surface normal constraint method. They first randomly sample a set of three-dimensional points on a specified plane to determine the local plane and then derive the constraint relationship between the plane and the surface normals. [PERSON] et al. [31; 32] introduce global geometric constraints based on \"virtual normal\" to address the generalization problem of monocular depth estimation. Virtual normal refers to the normal determined by a pseudo-plane formed by randomly sampling three points from three-dimensional space. Geometric constraint losses between predicted and ground truth values are utilized to enhance the accuracy and robustness of monocular depth estimation. In addition to adjusting the loss functions, some models directly incorporate plane constraints into their network. Among them, Geonet [25] and Geonet++ [26] simultaneously predict the depth and surface normal from a single image. They employ one network to convert depth into surface normal and another network to convert surface normal into depth, thereby enforcing geometric consistency and accuracy between the two outputs. Such a design constrains the network to achieve high consistency and accuracy between the two outputs. In the studies by [PERSON] et al. (2017) and [PERSON] et al. (2018), depth is not directly predicted. Instead, based on the conversion relationship between the depth and surface normal vector, plane coefficients are first predicted and then converted into a depth map. By constraining regions with consistent plane coefficients, the consistency of local planes is achieved. We know that monocular depth estimation suffers from the issue of scale ambiguity; thus, it usually did not directly take l1 or l2 as loss to supervise the network. [PERSON] et al. (2017) proposed an algorithm to combine multiple loss terms adaptively for training a monocular depth estimator. The most commonly used loss in this task for direct depth supervision is scale-invariant loss ([PERSON] et al., 2014) due to its scale invariance. This loss emphasizes the relative relationships between pixels in the depth map, considering the relationships between pixel pairs as the criterion, comparing the consistency between the depth differences of pixel pairs in the predicted result and those in the corresponding pixel pairs in the ground truth depth map. By optimizing this loss function, the influence of absolute global scale on the results is mitigated. [PERSON] et al. (2018) discretized continuous depth into a series of intervals and transformed the depth estimation into a classification problem. Then, they designed a strategy with gradually increasing intervals to discretize the depth values. AdaBins (2018) divides the depth range into different intervals based on the characteristics of each image, enabling the network to adaptively focus on regions with varying depths. Its main contribution lies in a Mini-ViT module which uses global information to compute the width of each depth interval as well as the probability values of the depth intervals. In this way, the method is able to generate smoother depth maps.