# CAPDepth: 360 Monocular Depth Estimation by Content-Aware Projection
## 3 Method
### Content-Aware Projection  
CAPDepth relies on further reducing the distortion of tangential images to resolve severe irregular geometric distortions in 360\({}^{\circ}\) images. It uses optimized CAP projection in the content-aware projection module to generate a set of low-distortion tangential images. The following section introduces the CAP projection and some optimizations to it. In traditional projection methods, the rectilinear projection causes excessive stretching and distortion at the edge of the image, as illustrated by the ceiling shape in the green and red boxes in Figure 3a. Meanwhile, stereoscopic projection introduces severe bending of the straight line segments, as shown by the edge segments of the table in the green and red boxes in Figure 3b. In the Pannini projection [25] based on cylindrical projection, as introduced in the previous section, two projection parameters, \(d\) and \(\upsilon c\), are introduced to manually seek the best compromise between horizontal and radial line bending and object stretching. Therefore, the CAP projection senses image content by extracting straight-line bending features and object-stretching features to calculate the distortio  
Figure 3: (**a**–**c**) are patches obtained through stereographic projection, rectilinear projection, and CAP projection (\(d\) = 0.4, \(\upsilon c\) = 0.3), respectively. It can be seen that the CAP projection balances the defects of the first two projections and reduces the overall distortion. The red box shows the geometric distortion that occurs at the edge of the image compared to the green box at the center. (**a**) Stereographic projection. (**b**) Rectilinear projection. (**c**) CAP (\(d\) = 0.4, \(\upsilon c\) = 0.3). image. It adaptively changes the projection parameters according to the distortion level to find the tangential projection with the least distortion, as shown in Figure 3c. To further improve the accuracy and efficiency of projection, we optimize the CAP projection in three ways. First, in terms of extracting straight-line bending features, we change the line-segment detector EDLines [27] to ELSED [28]. The key to ELSED is the local-segment growing algorithm, which connects gradient-aligned pixels in the presence of small discontinuities. This approach greatly improves the accuracy and time efficiency of line-segment detection. We adapt it based on its advantages and change the subsequent line-segment merging calculation algorithm. As shown in Figure 4, the line segment extraction effect has been improved, more short lines are merged into long lines, and more very close straight lines are discarded. This makes the calculation of line curvature (LC) and line inclination (LI) more accurate, thereby improving the calculation accuracy of straight line curvature features. The measurement method of LI and LC is shown in Figure 5. Second, in extracting object stretching features, we chose a combination of semantic segmentation [29] and [PERSON]'s indicatrices [30] (The measurement method is shown in Figure 5). Third, in terms of feature calculation, this research refers to the classification error method [31] used in [32] to evaluate the effectiveness of various pooling functions such as Sum, Average, Max, Min, and multiple distortion measurement methods [26,33]. Finally, the pooling function \(P\) is used to calculate the straight line bending feature Line distortions combination (LMC) [26], as shown in Formula (7), and the pooling function \(PF\) is used to calculate the object bending feature OAD [26], as shown in Formula (8). In the experiment, they all demonstrate improvements in both time efficiency and overall performance. \[P=\text{Sum}\big{(}LMC_{p}\big{)},LMC=LC+LI \tag{7}\]  
where \(LMC\) is a vector containing the distortion metric features of all line segments detected in the viewport; \(LMC_{p}\) means taking the \(p\%\) maximum distortion feature in \(LMC\).