# Calibration of Double-Plane-Mirror Catadioptric Camera Based on Coaxial Parallel Circles

Authors: Yue Zhao; Yuyang Chen; Liping Yang
Journal: Journal of Sensors
 (2022)

---

## Abstract

A catadioptric camera with a double-mirror system, composed of a pinhole camera and two planar mirrors, can capture multiple catadioptric views of an object. The catadioptric point sets (CPSs) formed by the contour points on the object lie on circles, all of which are coaxial parallel. Based on the property of the polar line of the infinity point with respect to a circle, the infinity points in orthogonal directions can be obtained using any two CPSs, and a pole and polar pair with respect to the image of the absolute conic (IAC) can be obtained through inference of the Laguerre theorem. Thus, the camera intrinsic parameters can be solved. Furthermore, as the five points needed to fit the image of a circle are not easy to obtain accurately, only sets in which five points can be located can be obtained, whereas the points on the line of intersection between the two plane mirrors and the ground plane can easily be obtained accurately. An optimization method based on the analysis of neighboring point sets to compare the intersection points with an image of the center of multiple circle images fitted using the point sets is proposed. Bundle adjustment is then applied to further optimize the camera intrinsic parameters. The feasibility and validity of the proposed calibration methods and their optimization were confirmed through simulation and experiments. Two primary innovations were obtained from the results of this study: (1) by applying coaxial parallel circles to the double-plane-mirror catadioptric camera model, a variety of calibration methods were derived, and (2) we found that the overall model could be optimized by analyzing the features of the neighboring point set and bundle adjustment.


---

## Body

## 1. Introduction

Vision technology, a modern research hotspot, has been applied in various engineering fields [1–3]. The acquisition of camera intrinsic parameter data and the solving of these parameters through the process of “camera calibration” has become one of the most important issues in the field of vision technology. Camera calibration is an essential step in the process of recovering 3D spatial information from 2D image plane information [4]. To expand the small field-of-view (FOV) of traditional cameras, Baker and Nayar [5] proposed the catadioptric camera system, which can expand the FOV by adding a mirror in front of the camera lens. The catadioptric camera is extensively used in several fields, such as 3D modeling [6–8], virtual monitoring [9, 10], and robot navigation [11].A plane catadioptric system [12, 13] consists of one or more plane mirrors with a pinhole camera. Camera calibration and 3D reconstruction can be performed using a catadioptric system with a single plane mirror and a pinhole camera [14]; however, the system can only provide two views of an object, limiting the acquisition of image information. Therefore, double-plane mirrors are used to obtain multiple catadioptric views of an object. Several methods have been proposed for determining the internal parameters of a double-mirror camera system. Gluckman and Nayar [15] only solved the focal length of the camera by determining the relationship between the corresponding points between views. As it is not easy to extract the corresponding points experimentally, Forbes et al. [16] extracted the silhouettes of objects from images with multiple views and used the epipolar tangents of these silhouettes to estimate the internal camera parameters, avoiding the extraction of the corresponding points between views; however, the physical coordinates of the silhouettes need to be precisely located. Zhang et al. [17] presented the geometric relationship between the double-plane-mirror catadioptric system and circular motion. Hernández et al. [18] further improved on the theoretical system developed by Zhang et al. [17] by proposing the use of circular motion to determine the camera’s internal parameters, but this method requires the direction of the rotation axis or observation. In addition, Hu et al. [19] used the relationship between the corresponding points in a catadioptric camera with a double-mirror system and the silhouettes of the views to solve the camera intrinsic parameters. Ying et al. [20] presented the pole-polar relationship of the silhouettes between five views and solved the camera intrinsic parameters by first determining the coordinates of the vanishing point corresponding to the three coordinate axes and then using the approaches developed in the studies described above. Circles are common in daily life, and the usage of circles in camera calibration has been extensively researched [21–24]. Zhao and Li [25] used the imaged circular points (ICPs) and the orthogonal vanishing points to solve the camera intrinsic parameters. Although some suggestions have been made in the recent literature, the problem of solving for the vanishing point in the direction of the double mirrors remains to be solved.Referring to the above literature, this study proposes camera calibration methods based on coaxial parallel circles. In a double-plane-mirror catadioptric camera system, the catadioptric point set (CPS) formed by any point on an object is located on the same circle [21], and these circles are coaxial parallel. The camera intrinsic parameters can be solved according to the corresponding linear constraints. This paper proposes an approach that improves upon the methods described in the literature review above by incorporating the use of coaxial parallel circles into the double-plane-mirror catadioptric camera model; furthermore, a variety of calibration methods are applied to obtain more accurate camera intrinsic parameters through the use of nonlinear optimization.


## 2. Imaging Model

This section briefly reviews the imaging model of a double-plane-mirror catadioptric camera [20], the camera projection process, and the property of the polar of the infinity point with respect to a circle.


### 2.1. Imaging Model of a Double-Plane-Mirror Catadioptric Camera

The imaging model comprises a real pinhole cameraC0 and two real plane mirrors M1 and M2, as shown in Figure 1 [20]. When the included angle of the double mirrors ranges from 60° to 90°, single internal reflection occurs. Here, two virtual planar mirrors are donated as M3 and M4, the image of M2 in M1 is virtual mirror M3, and the image of M1 in M2 is virtual mirror M4. Cii=1~4 represents reflection of C0 in the real and virtual mirrors. The images of C0 in M1 and M2 are virtual cameras C1 and C2, respectively. The image of C1 in M3 is virtual camera C3, and the image of C2 in M4 is virtual camera C4. As an object will form four catadioptric virtual images, an image is captured with camera C0 to obtain five views Tii=0~4 of the object. View T0 is the image of the real object with C0, views T1 and T2 are the images reflected by the object in M1 and M2, respectively, and views T3 and T4 are the images of T1 and T2 in M3 and M4, respectively. According to the principle of planar mirror imaging, the four groups of views (T0,T1), (T0,T2), (T1,T3), and (T2,T4) are symmetrical to M1, M2, M3, and M4, respectively. Similarly, cameras Cii=0~4 and views Ti have the same symmetry. Let the intersection lines between M1, M2, M3, and M4 and the ground plane be L1, L2, L3, and L4, respectively.Figure 1
Views of an object in the world coordinate system.


### 2.2. Camera Projection Process

In Figure1, the world coordinate system from Section 2.1, Ow−xwywzw, and Cii=0~4 is on the same plane and takes the intersection point between this plane and the line of intersection between M1 and M2 as the origin, Ow. The direction of the line connecting C0 and Ow is the zw-axis, the yw-axis is perpendicular to the plane of Cii=0~4, and the xw-axis is determined by the right-hand coordinate system. Defining X=xyz1T in Ow−xwywzw as the homogeneous coordinates of a space point and x=uv1T as the homogeneous coordinates of the image point on the corresponding 2D image, the projection process can be expressed as
(1)λx=PX,where λ is the scale factor, and P is a 3×4 projection matrix, which can be expressed as
(2)P=KRt.In a double-mirror catadioptric camera, the real cameraC0 also has four virtual reflections. It realizes a stereo vision system with fixed internal parameters and varying external parameters. Therefore,
(3)Pi=KRiti,i=0,1,2,3,4,where Pi is the projection matrix of Ci, Ri is the 3D rotation matrix, and ti is the translation vector. Ri and ti are the external parameters of camera Ci; K is the camera intrinsic parameter matrix of camera Cii=0–4, which is a 3×3 upper triangular matrix:
(4)K=fusu00fvv0001,where u0v01T is the homogeneous coordinate of the principal point, fu and fv are the focal lengths in the u and v directions, respectively, and s is the skew factor between the u and v axes of the image plane.


### 2.3. Property of the Polar Line of the Infinity Point with respect to a Circle

Theorem 1.
[26]. If L′ and L are a group of conjugate diameters of a circle, infinity point P∞ in the direction of L and infinity point P′∞ in the direction of L′ are a pair of orthogonal infinity points.Proposition 2.
The direction of an infinity point is orthogonal to the direction of the polar line of the infinity point with respect to a circle.Proof.
In Figure2, the infinity point is denoted as P∞, and the polar line of P∞ with respect to circle C is the diameter L′. The infinity point in the direction of diameter L′ is denoted as P′∞, and the polar line of P′∞ with respect to circle C is denoted as diameter L. From polar transformation in the projective space [26], it is known that P∞ is on diameter L; L and L′ are a group of conjugate diameters of circle C, which intersect at the center O. According to Theorem 1, points P∞ and P′∞ are a pair of infinity points in orthogonal directions.Figure 2
Property of the polar line of the infinity point with respect to a circle.


## 2.1. Imaging Model of a Double-Plane-Mirror Catadioptric Camera

The imaging model comprises a real pinhole cameraC0 and two real plane mirrors M1 and M2, as shown in Figure 1 [20]. When the included angle of the double mirrors ranges from 60° to 90°, single internal reflection occurs. Here, two virtual planar mirrors are donated as M3 and M4, the image of M2 in M1 is virtual mirror M3, and the image of M1 in M2 is virtual mirror M4. Cii=1~4 represents reflection of C0 in the real and virtual mirrors. The images of C0 in M1 and M2 are virtual cameras C1 and C2, respectively. The image of C1 in M3 is virtual camera C3, and the image of C2 in M4 is virtual camera C4. As an object will form four catadioptric virtual images, an image is captured with camera C0 to obtain five views Tii=0~4 of the object. View T0 is the image of the real object with C0, views T1 and T2 are the images reflected by the object in M1 and M2, respectively, and views T3 and T4 are the images of T1 and T2 in M3 and M4, respectively. According to the principle of planar mirror imaging, the four groups of views (T0,T1), (T0,T2), (T1,T3), and (T2,T4) are symmetrical to M1, M2, M3, and M4, respectively. Similarly, cameras Cii=0~4 and views Ti have the same symmetry. Let the intersection lines between M1, M2, M3, and M4 and the ground plane be L1, L2, L3, and L4, respectively.Figure 1
Views of an object in the world coordinate system.


## 2.2. Camera Projection Process

In Figure1, the world coordinate system from Section 2.1, Ow−xwywzw, and Cii=0~4 is on the same plane and takes the intersection point between this plane and the line of intersection between M1 and M2 as the origin, Ow. The direction of the line connecting C0 and Ow is the zw-axis, the yw-axis is perpendicular to the plane of Cii=0~4, and the xw-axis is determined by the right-hand coordinate system. Defining X=xyz1T in Ow−xwywzw as the homogeneous coordinates of a space point and x=uv1T as the homogeneous coordinates of the image point on the corresponding 2D image, the projection process can be expressed as
(1)λx=PX,where λ is the scale factor, and P is a 3×4 projection matrix, which can be expressed as
(2)P=KRt.In a double-mirror catadioptric camera, the real cameraC0 also has four virtual reflections. It realizes a stereo vision system with fixed internal parameters and varying external parameters. Therefore,
(3)Pi=KRiti,i=0,1,2,3,4,where Pi is the projection matrix of Ci, Ri is the 3D rotation matrix, and ti is the translation vector. Ri and ti are the external parameters of camera Ci; K is the camera intrinsic parameter matrix of camera Cii=0–4, which is a 3×3 upper triangular matrix:
(4)K=fusu00fvv0001,where u0v01T is the homogeneous coordinate of the principal point, fu and fv are the focal lengths in the u and v directions, respectively, and s is the skew factor between the u and v axes of the image plane.


## 2.3. Property of the Polar Line of the Infinity Point with respect to a Circle

Theorem 1.
[26]. If L′ and L are a group of conjugate diameters of a circle, infinity point P∞ in the direction of L and infinity point P′∞ in the direction of L′ are a pair of orthogonal infinity points.Proposition 2.
The direction of an infinity point is orthogonal to the direction of the polar line of the infinity point with respect to a circle.Proof.
In Figure2, the infinity point is denoted as P∞, and the polar line of P∞ with respect to circle C is the diameter L′. The infinity point in the direction of diameter L′ is denoted as P′∞, and the polar line of P′∞ with respect to circle C is denoted as diameter L. From polar transformation in the projective space [26], it is known that P∞ is on diameter L; L and L′ are a group of conjugate diameters of circle C, which intersect at the center O. According to Theorem 1, points P∞ and P′∞ are a pair of infinity points in orthogonal directions.Figure 2
Property of the polar line of the infinity point with respect to a circle.


## 3. Calibration of the Double-Mirror Catadioptric Camera

This section details the calibration methods proposed in this study. We discuss the relationship between the image of the CPS (ICPS) and the orthogonal vanishing point, the ICPs, and the pole and polar of the image of the absolute conic (IAC), respectively.As shown in Figure3, in the double-mirror catadioptric camera system, let A0 be any real point in space and the images of point A0 in mirrors Mii=1~4 be points Aii=0~4, respectively, where A1 and A2 are the reflected points of A0; A3 and A4 are the catadioptric points of A0.Definition 3.
IfA0 is any real point in space and Aii=0~4 is the corresponding catadioptric points in mirrors Mi, respectively, A0,A1,A2,A3,A4 is called a CPS.Figure 3
Two pairs of infinity points,Pj∞ and P′j∞j=1,2, in orthogonal directions.In Figure3, CA and CB are the circles fitted by Ai and Bii=0~4, respectively. Based on the symmetry of plane-mirror imaging, CA and CB are coaxial parallel circles [21]. It is known that the line passing through A0 and A1 is parallel to the line passing through B0 and B1 and intersects at infinity point P1∞. Similarly, the line passing through A0 and A2 is parallel to the line passing through B0 and B2 and intersects at infinity point P2∞. The infinity line through points P1∞ and P2∞ is L∞. The polar lines of points P1∞ and P2∞ with respect to circle CA intersect infinity line L∞ at infinity points P′1∞ and P′2∞, respectively. According to Proposition 2 mentioned in Section 2.3, Pj∞ and P′j∞j=1,2 are two pairs of infinity points in orthogonal directions.The description above and in Figure3 represents the spatial configuration of an image with corresponding spatial constraints. See the Appendix for the corresponding definitions, propositions, and proofs.


## 4. Nonlinear Optimization

This section includes two parts: the first involves the optimization of the fitting ofca and cb and application to the calibration algorithm. The next involves the consideration of the camera intrinsic parameters obtained through the optimization algorithm as the initial value and iteration for optimizing the camera intrinsic parameter calculation.


### 4.1. Optimization of the Conics Fitted in the Image

Analyzing the particularity of biplane-mirror imaging, only five fitting points can be selected from the image of a circle connecting the object point and its views; therefore, it is crucial to extract these five points accurately.Definition 4.
On the image plane, the set of pixels that represent a space point in a view is called the effective pixels of the point.In Figure1, the four intersecting lines L1, L2, L3, and L4 under camera C0 can be extracted using the least-squares fitting method [27] to calculate the coordinates of the pairwise combinatorial intersection points, and their average value can be obtained as od. Let D0 and E0 be any two points on an object in contact with the ground plane. The image of D0 and E0 in M1 and M2 is D1 and D2 and E1 and E2, respectively. The image of D1 and E1 in M3 is D3 and E3, respectively; the image of D2 and E2 in M4 is D4 and E4, respectively.Selectqi, hii=0~4, as the effective pixels of images di, eii=0~4 of space points Di, Eii=0~4 on view Ti, and fit q, h conics, respectively (i.e., the two families of conics). Among them, randomly select two conics ci and cji=1~q,j=1~h and calculate their generalized eigenvectors, three of which represent three points in the image [28]. The points in the two conics are denoted as oiji=1~q,j=1~h. The optimization objective is as follows:
(5)fopt1=mindistoij,od,where dist represents the Euclidean distance between two points in {}, and min represents taking the minimum value in {}. oij corresponding to fopt1, and ci, cj corresponding to oij, i.e., the optimal conic, can be obtained from the above two families of conics. Then, the pixel points required for fitting ci and cj can be obtained.Similarly, using the two conicsci and cj and applying the same method, the edge space points along the outskirts of the object can be optimized, including certain points in contact without the ground, to fit circle images of their projections.The vanishing line and the circular-point images are recovered using the images of the two concentric circles, and the images of the dual circular points denoted asImJm are calculated. Then, in view Ti, according to the effective pixels corresponding to the space points, multiple conics ck can be fitted, i.e., a family of conics in which the value of k is related to the number of selected effective pixels. The intersection points among a family of conics and the vanishing lines are solved, respectively, to obtain the dual of their intersection points, denoted as IkJk. Then, the optimization objective is as follows:
(6)fopt2=minImJm−IkJk2.determine IkJk corresponding to fopt2 and ck corresponding to IkJk, which is the optimal conic selected from the family of conics. Thus, the five pixels required for fitting ck can be obtained.


### 4.2. Optimization of the Camera Internal Parameters

Through bundle adjustment [29], the camera intrinsic parameters can be optimized using the reprojection points on a circle. In general, the coordinates of the center of a circle and its radius r can determine a circle.Therefore,N images and M fitting conics are selected for each image. According to the settings of the world coordinate system depicted in Figure 1, the coordinates of the center of each circle can be represented as 0,yst,0,1, with s=1~N,t=1~M. The coordinates of points Fstis=1~N,t=1~M,i=1~5 on the circle corresponding to views Tii=0~4 can be set as the angle parameter ωstis=1~N,t=1~M,i=1~5; therefore, points Fsti can be expressed as follows:
(7)Fsti=rstcosωsti,yst,rstsinωsti,1T.The above optimization problem can be changed as(8)gopt1=min∑s=1N∑t=1M∑i=15distfsti,μKRtFsti,where μ is the scale factor; fsti is the projection point of Fsti, which can be obtained as described in Section 4.1, and R and t can be obtained [20]. The next step is to solve the minimum value of the cost function to optimize the internal parameter matrix.The LM method is used to solve this optimization problem by selecting the initial value of the function (8) and its expected optimization target. Using the optimization method described in Section 4.1, some more accurate pixel points can be obtained to fit the circle images. As the calculated initial values of the camera intrinsic parameters do not deviate considerably from the actual values, the obtained camera intrinsic parameters can be used as the initial values.


## 4.1. Optimization of the Conics Fitted in the Image

Analyzing the particularity of biplane-mirror imaging, only five fitting points can be selected from the image of a circle connecting the object point and its views; therefore, it is crucial to extract these five points accurately.Definition 4.
On the image plane, the set of pixels that represent a space point in a view is called the effective pixels of the point.In Figure1, the four intersecting lines L1, L2, L3, and L4 under camera C0 can be extracted using the least-squares fitting method [27] to calculate the coordinates of the pairwise combinatorial intersection points, and their average value can be obtained as od. Let D0 and E0 be any two points on an object in contact with the ground plane. The image of D0 and E0 in M1 and M2 is D1 and D2 and E1 and E2, respectively. The image of D1 and E1 in M3 is D3 and E3, respectively; the image of D2 and E2 in M4 is D4 and E4, respectively.Selectqi, hii=0~4, as the effective pixels of images di, eii=0~4 of space points Di, Eii=0~4 on view Ti, and fit q, h conics, respectively (i.e., the two families of conics). Among them, randomly select two conics ci and cji=1~q,j=1~h and calculate their generalized eigenvectors, three of which represent three points in the image [28]. The points in the two conics are denoted as oiji=1~q,j=1~h. The optimization objective is as follows:
(5)fopt1=mindistoij,od,where dist represents the Euclidean distance between two points in {}, and min represents taking the minimum value in {}. oij corresponding to fopt1, and ci, cj corresponding to oij, i.e., the optimal conic, can be obtained from the above two families of conics. Then, the pixel points required for fitting ci and cj can be obtained.Similarly, using the two conicsci and cj and applying the same method, the edge space points along the outskirts of the object can be optimized, including certain points in contact without the ground, to fit circle images of their projections.The vanishing line and the circular-point images are recovered using the images of the two concentric circles, and the images of the dual circular points denoted asImJm are calculated. Then, in view Ti, according to the effective pixels corresponding to the space points, multiple conics ck can be fitted, i.e., a family of conics in which the value of k is related to the number of selected effective pixels. The intersection points among a family of conics and the vanishing lines are solved, respectively, to obtain the dual of their intersection points, denoted as IkJk. Then, the optimization objective is as follows:
(6)fopt2=minImJm−IkJk2.determine IkJk corresponding to fopt2 and ck corresponding to IkJk, which is the optimal conic selected from the family of conics. Thus, the five pixels required for fitting ck can be obtained.


## 4.2. Optimization of the Camera Internal Parameters

Through bundle adjustment [29], the camera intrinsic parameters can be optimized using the reprojection points on a circle. In general, the coordinates of the center of a circle and its radius r can determine a circle.Therefore,N images and M fitting conics are selected for each image. According to the settings of the world coordinate system depicted in Figure 1, the coordinates of the center of each circle can be represented as 0,yst,0,1, with s=1~N,t=1~M. The coordinates of points Fstis=1~N,t=1~M,i=1~5 on the circle corresponding to views Tii=0~4 can be set as the angle parameter ωstis=1~N,t=1~M,i=1~5; therefore, points Fsti can be expressed as follows:
(7)Fsti=rstcosωsti,yst,rstsinωsti,1T.The above optimization problem can be changed as(8)gopt1=min∑s=1N∑t=1M∑i=15distfsti,μKRtFsti,where μ is the scale factor; fsti is the projection point of Fsti, which can be obtained as described in Section 4.1, and R and t can be obtained [20]. The next step is to solve the minimum value of the cost function to optimize the internal parameter matrix.The LM method is used to solve this optimization problem by selecting the initial value of the function (8) and its expected optimization target. Using the optimization method described in Section 4.1, some more accurate pixel points can be obtained to fit the circle images. As the calculated initial values of the camera intrinsic parameters do not deviate considerably from the actual values, the obtained camera intrinsic parameters can be used as the initial values.


## 5. Proposed Algorithm for Recovering the Euclidean Structure

In accordance with the above discussion, the main steps of the calibration algorithm are as follows:Input: adjust the camera position to obtainnn≥3 imagesOutput: matrix of the intrinsic parameters of the camera,KStep 1.
Extract the effective pixel coordinates of the two ICPSsani and bnii=0~4 and fit the two conics cna and cnb, according to the least squares method [30] and the optimization method presented in Section 4.1.Step 2.
Use pointsani and bnii=0,1,2 on cna and cnb to solve the vanishing line ln, according to (A.1), (A.2), and (A.3).Step 3.
Solve the orthogonal vanishing pointsvnj and v′njj=1,2 using (A.4) and (A.5).Step 4.
Solve ICPsmnI and mnJ using (A.1) and (A.2); obtain vanishing points v′n3 and vanishing line lns using (17–21).Step 5.
Ifvnj and v′njj=1,2 are obtained, solve ω using (A.14) (method 1); if mnI and mnJ are obtained, solve ω using (A.16) (method 2); if v′n3 and lns are obtained, solve ω using (A.13) (method 3).Step 6.
Obtain the camera intrinsic parametersK through the Cholesky factorization of ω and the solution of the inverse matrix.Step 7.
Further, optimize the internal parametersK as described in Section 4.2.


## 6. Experimental

To verify the feasibility and effectiveness of the proposed algorithm, we performed simulation and experiments. Ying et al. [21] proposed a self-calibration theory based on the analysis of the structural characteristics obtained by imaging using a double-plane-mirror camera; Zhao and Li [25] subsequently modified this approach to meet the requirements of specific applications and described some newer methods developed by other researchers. We therefore compared results obtained using Ying et al.’s [21] and Zhao and Li’s [25] methods.


### 6.1. Simulated Data

In the simulation, the camera intrinsic parameters were set as follows:(9)K=5000.54000450320001.The angle between the two plane mirrors was set within60°~90°. Two points in the included corner were selected randomly, and their reflections were generated and projected on the image plane. Gaussian noise with zero mean and standard deviation σ (noise level) was added to each image point, and σ was varied from 0to3 pixels. For each σ, 100 independent experiments were performed, and the absolute error of the camera intrinsic parameters was calculated. The obtained results are shown in Figures 4(a)–4(e). It can be observed that the absolute errors of fu,fv,u0,v0,s increase with the increase in the σ. At the same σ, the absolute errors of our algorithms are similar to those of Ying et al.’s [21] and Zhao and Li’s [25] algorithms.Figure 4
Variations in the absolute errors of (a)fu, (b) fv, (c) u0, (d) v0, and (e) s in the results obtained using method 1, method 2, method 3, Ying’s method [21], and Zhao’s method [25], respectively, for various levels of σ.
(a)(b)(c)(d)(e)Figure5 depicts the absolute error of the camera intrinsic parameters under different σ, where optimization method 1, method 2, and method 3 are denoted as Method1 OP, method 2 OP, and method 3 OP, respectively. Under the same σ, as the absolute errors fu,fv are nearly equal, Figure 5 shows only one, similar to u0,v0. After optimization, it can be seen that the absolute error of the calculated camera parameters is lower.Figure 5
Variations in the absolute errors of (a)fu, (b) u0, and (c) s in the results obtained using method 1, method 2, method 3, method 1 OP, method 2 OP, and method 3 OP, respectively, for various levels of σ.
(a)(b)(c)


### 6.2. Actual Experiment

We first took pictures of the checkerboard grid from three different poses, as shown in Figures6(a)–6(c), using rectangular mirrors, which could be stood up more easily. Camera parameters calculated using reference [31] were used as the ground truth. To carry out calibration, a doll placed was placed between the two plane mirrors with the intrinsic parameters left unchanged. A double mirror with an included angle of 74° (within 60°~90°) was placed on a horizontal desktop such that there was no occlusion between the five views. Multiple pictures were captured from different positions, and three images with a better effect were selected to solve the camera intrinsic parameters. The three selected images are displayed in Figures 7(a)–7(c) with an image resolution of 1440×1080 (pixels). The results of Canny edge detection [32] are shown in Figures 7(d)–7(f). The silhouette and the silhouette coordinates of the five views in each image are shown in Figures 7(g)–7(i) and 7(j)–7(l), respectively.Figure 6
(a)–(c) Checkerboard images taken from different positions.
(a)(b)(c)Figure 7
(a)–(c) Images showing five views of a doll between double planar mirrors. (d)–(f) Canny edge detection results for images (a)–(c). (g)–(i) Silhouettes of the five views corresponding to the images in (d)–(f), respectively. (j)–(l) Coordinates of the silhouettes of the five views corresponding to the images in (g)–(i), respectively.
(a)(b)(c)(d)(e)(f)(g)(h)(i)(j)(k)(l)The silhouette outlines and epipolar tangency lines of the five views in Figures7(a) and 7(b) are shown in Figures 8(a) and 8(b), respectively. The image coordinates of the silhouette outlines and the intersections of the epipolar tangency lines in the image are shown in Figures 8(c) and 8(d). Therefore, the coordinates of the ICPSs can be obtained in each image [25].Figure 8
(a, b) Silhouette outlines and epipolar tangency lines of the five views in Figures7(a) and 7(b), respectively. (c, d) Image coordinates of the silhouettes and intersection points of the epipolar tangency lines of the five views in the respective images.
(a)(b)(c)(d)According to the abovementioned procedure, the three algorithms proposed in this study as well as the calibration algorithms in references [21, 25] were used to perform 100 independent experiments. In addition, the method described in Chapter 4 was used to optimize and average the three algorithms proposed in this study. The average values of the results are listed in Table 1. It is seen that the results are similar, with the results of method 1 OP, method 2 OP, and method 3 OP closer to the ground truth, proving the effectiveness of the respective calibration algorithms.Table 1
Calibration results obtained using real data (unit: pixel).
MethodCamera intrinsic parametersfufvu0v0sGround truth1580.30661578.9804650.0029477.87220.0747Method 11576.43171573.7439650.8347476.54360.0813Method 21593.64351588.7013645.9737478.89740.0756Method 31582.14581578.7357653.5698469.53770.07904Ying et al. [21]1594.08241590.3454658.7831487.15370.0579Zhao and Li [25]1571.00741567.3497654.7834465.78670.0654Method 1 OP1578.36121577.8926650.4637478.44360.0764Method 2 OP1588.47241585.6731647.7634478.97240.0733Method 3 OP1579.11361576.3324652.2278475.56230.07593According to the characteristics of plane-mirror imaging and the reconstruction method in reference [16], we used the visible silhouettes of the five views to obtain the epipolar geometric relationship between the views, determine the relationship between the rotation and translation between views, and reconstruct the approximate 3D structure of the object. In this study, we selected the images in Figures 8(a) and 8(b) to reconstruct the 3D model of the doll. Because the calculation results obtained using the different methods were similar and those obtained experimentally using method 1 were better, only the camera intrinsic parameters data obtained using method 1 and method 1 OP (Table 1) were for used for the 3D reconstruction. The reconstruction results are shown in Figures 9. Table 2 lists the norm of deviation values between the real and reconstructed dolls for several corresponding feature points in the same reference frame. From Figures 9 and Table 2, it is easy to see that the reconstruction results are very similar to the doll depicted in Figures 7(a) and 7(b). The right and back views of the reconstruction results obtained using method 1 (Figures 9(a) and 9(b), respectively) are highly similar to those obtained using method 1 OP (Figures 9(c) and 9(d), respectively).Figure 9
Approximate reconstruction results obtained using method 1: (a) right view of reconstruction result obtained using method 1. (b) Back view of reconstruction result obtained using method 1, (c) Right view of reconstruction result obtained using method 1 OP. (d) Back view of reconstruction result obtained using method 1 OP.
(a)(b)(c)(d)Table 2
Comparison between method 1 and method 1 OP results and ground truth in reconstruction of 3D feature pointsfpii=1~5 (unit: mm).
MethodNorm of deviation values between method ground truthfp1fp2fp3fp4fp5Method 10.27771.11470.33160.47220.6817Method 1 OP0.16620.96110.32130.44230.6751


## 6.1. Simulated Data

In the simulation, the camera intrinsic parameters were set as follows:(9)K=5000.54000450320001.The angle between the two plane mirrors was set within60°~90°. Two points in the included corner were selected randomly, and their reflections were generated and projected on the image plane. Gaussian noise with zero mean and standard deviation σ (noise level) was added to each image point, and σ was varied from 0to3 pixels. For each σ, 100 independent experiments were performed, and the absolute error of the camera intrinsic parameters was calculated. The obtained results are shown in Figures 4(a)–4(e). It can be observed that the absolute errors of fu,fv,u0,v0,s increase with the increase in the σ. At the same σ, the absolute errors of our algorithms are similar to those of Ying et al.’s [21] and Zhao and Li’s [25] algorithms.Figure 4
Variations in the absolute errors of (a)fu, (b) fv, (c) u0, (d) v0, and (e) s in the results obtained using method 1, method 2, method 3, Ying’s method [21], and Zhao’s method [25], respectively, for various levels of σ.
(a)(b)(c)(d)(e)Figure5 depicts the absolute error of the camera intrinsic parameters under different σ, where optimization method 1, method 2, and method 3 are denoted as Method1 OP, method 2 OP, and method 3 OP, respectively. Under the same σ, as the absolute errors fu,fv are nearly equal, Figure 5 shows only one, similar to u0,v0. After optimization, it can be seen that the absolute error of the calculated camera parameters is lower.Figure 5
Variations in the absolute errors of (a)fu, (b) u0, and (c) s in the results obtained using method 1, method 2, method 3, method 1 OP, method 2 OP, and method 3 OP, respectively, for various levels of σ.
(a)(b)(c)


## 6.2. Actual Experiment

We first took pictures of the checkerboard grid from three different poses, as shown in Figures6(a)–6(c), using rectangular mirrors, which could be stood up more easily. Camera parameters calculated using reference [31] were used as the ground truth. To carry out calibration, a doll placed was placed between the two plane mirrors with the intrinsic parameters left unchanged. A double mirror with an included angle of 74° (within 60°~90°) was placed on a horizontal desktop such that there was no occlusion between the five views. Multiple pictures were captured from different positions, and three images with a better effect were selected to solve the camera intrinsic parameters. The three selected images are displayed in Figures 7(a)–7(c) with an image resolution of 1440×1080 (pixels). The results of Canny edge detection [32] are shown in Figures 7(d)–7(f). The silhouette and the silhouette coordinates of the five views in each image are shown in Figures 7(g)–7(i) and 7(j)–7(l), respectively.Figure 6
(a)–(c) Checkerboard images taken from different positions.
(a)(b)(c)Figure 7
(a)–(c) Images showing five views of a doll between double planar mirrors. (d)–(f) Canny edge detection results for images (a)–(c). (g)–(i) Silhouettes of the five views corresponding to the images in (d)–(f), respectively. (j)–(l) Coordinates of the silhouettes of the five views corresponding to the images in (g)–(i), respectively.
(a)(b)(c)(d)(e)(f)(g)(h)(i)(j)(k)(l)The silhouette outlines and epipolar tangency lines of the five views in Figures7(a) and 7(b) are shown in Figures 8(a) and 8(b), respectively. The image coordinates of the silhouette outlines and the intersections of the epipolar tangency lines in the image are shown in Figures 8(c) and 8(d). Therefore, the coordinates of the ICPSs can be obtained in each image [25].Figure 8
(a, b) Silhouette outlines and epipolar tangency lines of the five views in Figures7(a) and 7(b), respectively. (c, d) Image coordinates of the silhouettes and intersection points of the epipolar tangency lines of the five views in the respective images.
(a)(b)(c)(d)According to the abovementioned procedure, the three algorithms proposed in this study as well as the calibration algorithms in references [21, 25] were used to perform 100 independent experiments. In addition, the method described in Chapter 4 was used to optimize and average the three algorithms proposed in this study. The average values of the results are listed in Table 1. It is seen that the results are similar, with the results of method 1 OP, method 2 OP, and method 3 OP closer to the ground truth, proving the effectiveness of the respective calibration algorithms.Table 1
Calibration results obtained using real data (unit: pixel).
MethodCamera intrinsic parametersfufvu0v0sGround truth1580.30661578.9804650.0029477.87220.0747Method 11576.43171573.7439650.8347476.54360.0813Method 21593.64351588.7013645.9737478.89740.0756Method 31582.14581578.7357653.5698469.53770.07904Ying et al. [21]1594.08241590.3454658.7831487.15370.0579Zhao and Li [25]1571.00741567.3497654.7834465.78670.0654Method 1 OP1578.36121577.8926650.4637478.44360.0764Method 2 OP1588.47241585.6731647.7634478.97240.0733Method 3 OP1579.11361576.3324652.2278475.56230.07593According to the characteristics of plane-mirror imaging and the reconstruction method in reference [16], we used the visible silhouettes of the five views to obtain the epipolar geometric relationship between the views, determine the relationship between the rotation and translation between views, and reconstruct the approximate 3D structure of the object. In this study, we selected the images in Figures 8(a) and 8(b) to reconstruct the 3D model of the doll. Because the calculation results obtained using the different methods were similar and those obtained experimentally using method 1 were better, only the camera intrinsic parameters data obtained using method 1 and method 1 OP (Table 1) were for used for the 3D reconstruction. The reconstruction results are shown in Figures 9. Table 2 lists the norm of deviation values between the real and reconstructed dolls for several corresponding feature points in the same reference frame. From Figures 9 and Table 2, it is easy to see that the reconstruction results are very similar to the doll depicted in Figures 7(a) and 7(b). The right and back views of the reconstruction results obtained using method 1 (Figures 9(a) and 9(b), respectively) are highly similar to those obtained using method 1 OP (Figures 9(c) and 9(d), respectively).Figure 9
Approximate reconstruction results obtained using method 1: (a) right view of reconstruction result obtained using method 1. (b) Back view of reconstruction result obtained using method 1, (c) Right view of reconstruction result obtained using method 1 OP. (d) Back view of reconstruction result obtained using method 1 OP.
(a)(b)(c)(d)Table 2
Comparison between method 1 and method 1 OP results and ground truth in reconstruction of 3D feature pointsfpii=1~5 (unit: mm).
MethodNorm of deviation values between method ground truthfp1fp2fp3fp4fp5Method 10.27771.11470.33160.47220.6817Method 1 OP0.16620.96110.32130.44230.6751


## 7. Conclusions

In this study, we proposed methods for obtaining the camera intrinsic parameters of a double-mirror catadioptric camera system. When the included angle of the two mirrors is between60°and90°, five views of an object can be obtained in an image. The camera intrinsic parameters can be obtained by capturing three images from different positions. According to the symmetry of plane-mirror imaging, the circles fitted by some of the CPSs are coaxial parallel. Using two CPSs, two infinity points can be obtained, whose connection provides an infinity line. Using the property that the direction of the infinity point is orthogonal to the direction of its polar line with respect to a circle, two pairs of infinity points in the orthogonal direction can be obtained. Moreover, the infinity line and circles fitted by the CPSs intersect at the circular points. The orthogonal vanishing points or the image of the circular points can be obtained on the image plane. In addition, based on the pole-polar relationship and inference of the Laguerre theorem, the image of the common axis of the coaxial parallel circles can be obtained; therefore, the vanishing point can be determined in the axis direction, which is in the direction of the normal vectors of the projection plane including the parallel circles. Both of them have a pole-polar relationship with respect to the IAC. Therefore, if two pairs of ICPSs are extracted on the image plane, the three calibration methods can be applied to solve the camera intrinsic parameters. Analysis of the features of the neighboring point set indicate that the centers of the circles comprising the corresponding catadioptric five-point sets are on a straight line; hence, the image of the circle fitted on the image can be optimized. The determined camera parameters are considered as the initial values and further optimized through binding adjustment. In simulation, the absolute errors of our methods are slightly lower than those obtained using Ying et al.’s [21] and Zhao and Li’s [25] methods, and the optimized absolute errors converge rapidly, resulting in estimated values that are stable around exact values with a small degree of jitter. Comparison of our results with the ground truth value obtained using real data indicated that the methods have low absolute error. In performing 3D reconstruction, the two methods for example produced similar norms of deviation from ground truth.Our experimental approach led to several limitations in terms of results. First, to ensure that five points within each catadioptric mirror would be on the same plane, our experiments were limited to the use of rectangular mirrors. As a result, we were unable to examine cases in which the five points were not on the same plane, as would occur with the use of nonrectangular mirrors. Second, we did not consider cases in which the shape features of an imaged object were partially occluded. Finally, no experiments were carried out in overexposed or dark settings.



---

*Source: 7145400-2022-08-25.xml*