# CasOmniMVS: Cascade Omnidirectional Depth Estimation with Dynamic Spherical Sweeping
## 3 Method
### Spherical Cost Volume  
Given the intrinsic and extrinsic matrices of a camera, we can project a point from the camera coordinate system to the coordinate system located at the camera rig center. Thus, \begin{table}
\begin{tabular}{c c c c c c} \hline
**Stage \(t\)** & \(\mathbf{\alpha_{t}}\) & \(\mathbf{L_{t}}\) & \(\mathbf{s_{t}}\) & \(\mathbf{\lambda_{t}}\) & \(\mathbf{w_{t}}\) \\ \hline
1 & 0.25 & 48 & - & 4.0 & 0.5 \\
2 & 0.50 & 32 & 3 & [1.0, 4.0] & 1.0 \\
3 & 1.00 & 8 & 1 & [1.0, 2.0] & 2.0 \\ \hline \end{tabular}
\end{table} 
Table 1: Hyperparameter Settings. The width and height of the regularized spherical cost volume from each stage is \(\mathbf{\alpha_{t}}H_{output}\times\mathbf{\alpha_{t}}W_{output}\). Figure 3: Network submodules. Each stage has its own independent uncertainty net and upsampling layer. (**a**) Upsampling layer; (**b**) uncertainty net. we can project the feature \(f_{i}\) from the \(i\)th camera onto the central ERP view, which can be written as  
\[\mathbf{x}_{c}=[x_{c},y_{c},z_{c}]^{\mathbf{T}}, \tag{3}\]  
\[\mathbf{x}=M_{Ei}\mathbf{x}_{c}, \tag{4}\]  
\[F_{i}=\mathbf{\Pi}_{i}\Big{(}M_{Ii},f_{i},\mathbf{x}=[x,y,z]^{\mathbf{T}}\Big{)}, \tag{5}\]  
where \(\mathbf{\Pi}_{i}\) is the projection function, \(M_{Ii}\) and \(M_{Ei}\) are the camera intrinsic and extrinsic (cam2 rig) matrices, respectively, and \(\mathbf{x}_{c}\) is the coordinate of point \(\mathbf{P}\) in the camera coordinate system. \(\mathbf{x}\) contains the 3D point coordinates in the ERP view, and \(F_{i}\) is the ERP feature. As shown in Figure 4a, we can easily transform 3D coordinates into polar coordinates. Let \(\theta\) be the horizontal angle in the range \([-\pi,\pi]\), \(\phi\) be the pitch angle in the \([-\frac{\pi}{2},\frac{\pi}{2}]\), and \(d\) be the radial distance from the point to the center. Equation (5) can be rewritten as  
\[\theta =\arcsin\frac{x}{\sqrt{x^{2}+z^{2}}} \tag{6}\] \[\phi =\arctan\frac{y}{\sqrt{x^{2}+z^{2}}}\] \[d =\sqrt{x^{2}+y^{2}+z^{2}}\] \[F_{i} =\mathbf{\Pi}_{i}(M_{Ii},f_{i},(\theta,\phi,d))\]  
The values of \(\theta\) and \(\phi\) can be readily derived from ERP images. Consequently, a backward warp can be performed using \(\theta\), \(\phi\), and \(d\) to obtain the ERP feature. Varying \(d\) yields different projection outcomes, enabling computation of the spherical cost volume. We divide the inverse of the depth into a finite number of hypothetical spheres, which is determined as  
\[idx=\frac{N-1}{d^{{}^{\prime}}_{max}-d^{{}^{\prime}}_{min}}\Big{(}d^{{}^{ \prime}}-d^{{}^{\prime}}_{min}\Big{)}, \tag{7}\]  
where \(N\) is the total number of hypothetical spheres, \(d^{{}^{\prime}}\) is the inverse depth \(\frac{1}{d}\), \(d^{{}^{\prime}}_{max},d^{{}^{\prime}}_{min}\) are the maximum and minimum values, respectively, and \(idx\) represents the index of the hypothetical sphere \(S_{idx+1}\). As shown in Figure 4b, given a series of hypothetical spheres \(\{S_{1},S_{2},\dots,S_{j},\dots,S_{L}\}\), we can obtain the ERP features at each index based on Equations (4), (6) and (7). By  
Figure 3:0 (**a**) Illustration of the coordinate system. The color arrows indicate the three axes of the 3D Cartesian coordinate system. The point \(\mathbf{P}\) can be projected into the ERP view with the camera rig center as the origin. \(\mathbf{P}^{\prime}\) is the projection point of \(\mathbf{P}\) on the **XOZ** plane. \(\theta\) is the angle between \(\mathbf{\hat{O}}\mathbf{P}^{\prime}\) and \(\mathbf{\hat{O}}\mathbf{Z}\). \(\phi\) is the angle between \(\mathbf{\hat{O}}\mathbf{P}\) and \(\mathbf{XOZ}\). (**b**) Illustration of multi-view spherical sweeping. We can assume that \(P_{j}\) is in different hypothetical spheres \(S_{j}\) to construct a spherical cost volume. concatenating these features together, we form the spherical feature volume for the \(i^{th}\) camera, which is denoted as  
Figure 3:1  
where \(SC_{i}\) is the spherical feature volume, and \(F_{i,j}\) is the feature of the hypothetical sphere \(S_{j}\).