# CAPDepth: 360 Monocular Depth Estimation by Content-Aware Projection
###### Abstract  
Solving the depth estimation problem in a 360\({}^{\circ}\) image space, which has holistic scene perception, has become a trend in recent years. However, depth estimation in common 360\({}^{\circ}\) images is prone to geometric distortion. Therefore, this study proposes a new method, CAPDepth, to address the geometric-distortion problem of 360\({}^{\circ}\) monocular depth estimation. We reduce the tangential projections by an optimized content-aware projection (CAP) and a geometric embedding module to capture more features for global depth consistency. Additionally, we adopt an index map and a de-blocking scheme to improve the inference efficiency and quality of our CAPDepth model. Our experiments show that CAPDepth greatly alleviates the distortion problem, producing smoother, more accurate predicted depth results, and improves performance in panoramic depth estimation.  
d 202515769 12202515769 and then merges them into the ERP format. OmniFusion [16] aggregates global features by embedding geometric information, improving consistency and depth estimation accuracy. Similarly, HRDFuse [17] introduces a spatial-feature alignment (SFA) module to obtain spatially aligned index maps, further integrating geometric information and improving depth-prediction consistency. However, the tangential projections used by these methods still exhibit large depth distortions in local areas. These distorted depths are learned as input to the model and accumulated over multiple iterations, reducing the depth estimation accuracy.  
In this study, we propose CAPDepth, an indoor 360\({}^{\circ}\) monocular depth estimation method that focuses on projection optimization. It captures global features through an attention mechanism to avoid incorrect depth estimation caused by confusion of local features. It is suitable for closed environments such as rooms and corridors. We introduce two key components to address the distortion issues of tangential projections and seamlessly merge depth results into the ERP format. First, we introduce a new content-aware projection module in our network, which segments ERP images using optimized CAP projection. Specifically, the module adjusts the projection parameters to perceive changes in content features, such as line curvature and object stretching, by using an improved feature extractor and computation of geometric distortion. The module will then calculate the best distortion balance point and obtain the tangential image with minimum distortion. This approach ensures the largest possible FoV while reducing overall geometric distortion. Second, we improve the geometry-aware fusion module to extract global geometry information and compensate for feature differences between patches. We perform image registration by de-blocking the filtering algorithm on the set overlapping area. Moreover, we build an index graph to record the mapping relationship between patches and ERP during the CAP projection process, which is used by merging patches into the ERP format. This approach enables a faster inference speed and smoother, more accurate depth estimation for our network.  
In summary, the contributions of this study are as follows:  
* This study proposes a network called CAPDepth, which improves performance by reducing the distortion caused by tangent projection and combining ERP global features;
* We first introduce the Adaptive Content-Aware Projection (CAP) module and improve the feature extractor to effectively reduce the geometric distortion caused by projection;
* We optimize the Geometric Fusion (GAF) module by merging patches into the ERP format more smoothly and accurately through a new feature fusion method, which also improves the model efficiency.