## 4 Coordinate System Conversion Algorithm-based Depth Estimation
### Removing Camera Distortion  
Due to the structural characteristics of cameras, distortion occurs when capturing images. Distortion types include radial and tangential distortions related to the lens and the image sensor, respectively. Radial distortion is a phenomenon in which light rays that pass farther from the center of a lens are considerably bent, appearing mainly in the form of barrel distortion. Tangential distortion occurs during the camera manufacturing process and is caused by the lens and image sensor planes being not perfectly in parallel, but with an inclination. Other types of distortion may also occur, but as the size is relatively large for radial and tangential distortions, distortion correction is mainly aimed at these. To obtain accurate back projection from image coordinates to 3D coordinates, this distortion correction must be preceded. Correction of both radial and tangential distortions can be performed using coordinates obtained by projecting image coordinates onto a normal coordinate system.  
\[\begin{split} x_{distort}&=x_{undistor}(1+k_{1}r^{2 }+k_{2}4)\\ y_{distort}&=y_{undistor}(1+k_{1}r^{2}+k_{2}4) \end{split} \tag{1}\]  
Equation (1) is a model that is converted from coordinates (\(x_{undistor}\), \(y_{undistor}\)) with no distortion on the normal coordinate system to coordinates (\(x_{distort}\), \(y_{distort}\)) with radial distortion applied according to the lens model. \(k_{1}\), \(k_{2}\) are radial distortion coefficients obtained in the camera calibration step, and r is the radius from the optical center of  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Internal Factors** & **Value** & **Distortion Factors** & **Value** \\ \hline \(f_{x}\) & 1185.29 & \(k_{1}\) & \(-0.331729\) \\ \(f_{y}\) & 1185.16 & \(k_{2}\) & \(0.098764\) \\ \(c_{x}\) & 946.25 & \(p_{1}\) & \(-0.000110\) \\ \(c_{y}\) & 475.07 & \(p_{2}\) & \(0.000696\) \\ \(f_{x}\) & 1185.29 & \(k_{1}\) & \(-0.331729\) \\ \hline \hline \end{tabular}
\end{table}
Table 3: SNO-6084R CCTV camera internal factors and distortion coefficients.  
Figure 9: Image for internal calibration of SNO-6084R CCTV.  
the normal coordinate system. If the distortion is severe, a term \(k_{3}\) may be added to the expression in the form of a Taylor series.  
\[\begin{array}{ll}x_{distort}=x_{undistor}+&(2p_{1}x_{undistor}y_{undistor }\\ y_{distort}=y_{undistor}+&\left(p_{2}\left(r^{2}+2y^{2}\right)\right.\\ &+\left.2p_{2}x_{undistor}y_{undistor}\right)\end{array} \tag{2}\]  
Equation (2) refers to the tangential distortion model and tangential distortion coefficient.  
Distortion is usually proportional to the size of the field of view of the camera. The angle of view of the SNO-6084R CCTV is approximately \(105^{\circ}\), which is smaller than that of a wide-angle camera that shoots a wide angle of view of \(120^{\circ}\) or more, and the degree of distortion is greater than that of a web camera with a field of view of approximately \(80^{\circ}\). The picture on the right of Figure 10 is a distortion-corrected image using Equations (1) and (2). Even with the naked eye, the degree of distortion is not sufficiently small to distinguish the difference before and after correction. As a difference of only a few pixels can considerably change the coordinate conversion calculation result, the distortion correction procedure must be performed.