# A 3D Estimation Method Using an Omnidirectional Camera and a Spherical Mirror
## 4 3D Position Estimation of Spherical Mirror
### 3D Position Estimation Based on Elliptical Shape  
The regions segmented using deep learning in Section 4.1 contain errors. In the case of a spherical mirror, the 3D circle is imaged as a circle, whereas for a spherical cap-shaped mirror, the 3D circle of the cut surface is imaged as an ellipse. By applying ellipse fitting, the  
Figure 3: Process of segmenting mirror image in omnidirectional image: The procedure begins with the segmenting of the omnidirectional image into perspective projection images via cube mapping. Subsequently, a deep learning model is applied to the perspective projection image containing the mirror image for further segmentation. shape of the ellipse is ascertained. In conjunction with our prior knowledge of the mirror, such as the radius \(R\) of the mirror sphere, the support plane that contains the 3D circle is identified. Consequently, this method makes it possible to determine the 3D position \(\mathbf{C_{s}}\) of the mirror. A diagram of the relationship between the camera and the spherical mirror is shown in Figure 4. In the XYZ coordinate system where the camera position is the origin \(\mathbf{O}\), the imaging plane is \(\mathbf{Z}=f\), the horizontal axis on the imaging plane is the x-axis, and the vertical axis is the y-axis. The ellipse on the image plane can be represented as a matrix \(\mathbf{Q}\), and the plane supporting the 3D circle in the scene can be estimated by considering the rotation around the camera to the ellipse [17,18]. Using the eigenvalues \(\lambda_{1}\), \(\lambda_{2}\), \(\lambda_{3}(\lambda_{2}\geq\lambda_{1}>0>\lambda_{3})\) of the matrix \(\mathbf{Q}\) with the determinant as -1 and corresponding unit eigenvectors \(\mathbf{u_{1}}\), \(\mathbf{u_{2}}\), \(\mathbf{u_{3}}\), the normal unit vector \(\mathbf{n}\) of the support plane is calculated by Equation (1). \[\mathbf{n}=\mathcal{N}\Big{[}\sqrt{\lambda_{2}-\lambda_{1}}\mathbf{u_{1}}+\sqrt{ \lambda_{1}-\lambda_{3}}\mathbf{u_{3}}\Big{]} \tag{1}\]  
where \(\mathcal{N}[\cdot]\) means normalization. If the radius \(r\) of the 3D circle to be imaged is known, the distance \(h_{p}\) from the camera to the support plane \(h_{p}\) is calculated by:  
\[h_{p}=\sqrt{\lambda_{1}^{3}r} \tag{2}\]  
When the support plane of the 3D circle is perpendicular to the optical axis of the camera, the relationship between the normal \(\mathbf{n}\) of the plane and the center of the 3D circle on the image \(\mathbf{m}=(x_{C},\ y_{C},\ f)^{T}\) is shown in Equation (3). Therefore, the vector \(\mathbf{p_{c}}\) from the camera to the center of the 3D circle is calculated by Equation (4), where \(\mathbf{K}\) is the camera intrinsic matrix. \[\mathbf{m}=\mathbf{Q}^{-1}\mathbf{n} \tag{3}\] \[\mathbf{p_{c}}=\mathbf{K}\mathbf{m}=\mathbf{K}\mathbf{Q}^{-1}\mathbf{n} \tag{4}\]  
Figure 4: Relationship between the Camera and the Spherical Mirror: The estimation of the 3D position of the spherical mirror involves identifying the supporting plane (normal \(\mathbf{n}\), distance \(h_{p}\) from the camera) using the ellipse shape of the mirror image (matrix \(\mathbf{Q}\)). This process employs prior knowledge of the spherical mirror’s shape, including the radius \(R\) of the mirror sphere and the radius r of the 3D circle. For the support plane of the 3D circle (normal: \(\mathbf{n}\), distance: \(h_{p}\)), let \(d_{c}\) be the length from the camera to the center of the 3D circle, and let \(\mathbf{p_{c}}\) be the bearing vector; then, \(d_{c}\) is calculated as:  
\[d_{c} = \frac{h_{p}}{\mathbf{n}^{T}\mathbf{p_{c}}} \tag{5}\]  
The 3D position of the spherical mirror \(\mathbf{C_{s}}\) is illustrated in Figure 4 and is calculated by Equation (6)). Substituting \(d_{c}\) and \(\mathbf{p_{c}}\) with Equations (4) and (5), we have Equation (7).