# SMILE: Segmentation-Based Centroid Matching for Image Rectification via Aligning Epipolar Lines
## 5 Conclusions  
In this paper, we propose a new image rectification method using 2D rotation that does not require a calibration pattern, as opposed to image rectification using parameters obtained through camera calibration for stereo images consisting of left and right pairs. The proposed method selects object pairs as reference points in the left and right image pairs, performs segmentation on these object pairs, and then calculates the center coordinates of each object. Subsequently, rotation and adjustment are performed so that the centers of the object pairs have the same y-coordinate values in both images. This aligns the y-values of the object pairs in both images, thereby inducing parallel epipolar lines. The effectiveness of the proposed method is demonstrated through validation using metrics such as the \begin{table}
\begin{tabular}{l c c c c c} \hline  & **Original** & \multicolumn{3}{c}{**Rectified**} \\ \hline  & F-1 & & & F-2 & \\
0.0000 & 0.0002 & \(-\)0.1414 & \(-\)0.0000 & \(-\)0.0000 & 0.0025 \\ \(-\)0.0002 & 0.0000 & 0.0004 & 0.0000 & 0.0000 & \(-\)0.0160 \\
0.1346 & \(-\)0.0046 & 1.0000 & \(-\)0.0028 & 0.0150 & 1.0000 \\ \hline  & G-1 & & & G-2 & \\
0.0000 & 0.0000 & \(-\)0.0187 & \(-\)0.0000 & \(-\)0.0000 & 0.0071 \\ \(-\)0.0000 & \(-\)0.0000 & 0.0766 & 0.0000 & 0.0000 & \(-\)0.0313 \\
0.0166 & \(-\)0.0757 & 1.0000 & \(-\)0.0057 & 0.0305 & 1.0000 \\ \hline \end{tabular}
\end{table} 
Table 2: _Cont._ \begin{table}
\begin{tabular}{l c c} \hline
**Metric** & **Original** & **Proposed** \\ \hline Y-disparity mean & 34.61 & 24.79 \\ Y-disparity Std & 46.19 & 56.32 \\ X-disparity entropy & 1.31 & 1.31 \\ SSIM & 0.4210 & 0.4412 \\ Feature matching accuracy & 0.2650 & 0.2599 \\ Area-based correlation & 0.3616 & 0.3847 \\ \hline \end{tabular}
\end{table} 
Table 3: Analysis of results in Figure 3 through metrics. fundamental matrix, Y-disparity, X-disparity entropy, SSIM, feature matching accuracy, and area-based correlation, as well as the visualization of feature point matching and disparity maps. In addition, the fast average processing time of 0.07 s makes it suitable for real-time use even in high-resolution image processing environments, and its low computational complexity indicates strong potential for practical applications. Since this method does not require a calibration pattern, it does not achieve perfect rectification like methods that use patterns. However, when comparing the results before and after processing, it is confirmed that the method effectively improves the outcome. The method proposed in this paper is an image rectification approach that uses objects easily obtainable in nature as reference points, without using reference points such as calibration patterns required for camera calibration. However, this method has limitations in that it may be difficult to apply if there are no objects in the stereo images or if the objects present in the images belong to classes not included in the datasets used to train the object detection models used for object segmentation. Additionally, while strict conditions were set to select the same objects in the left and right images, there may be issues with poor results if different objects are selected or if only objects with large rotation angles are present. In the future, we plan to continue research to perform more accurate image rectification without the need for calibration patterns. Conceptualization, J.C. and D.L. ; methodology, J.C.; software, J.C.; validation, J.C.; formal analysis, J.C.; investigation, J.C.; resources, D.L. ; data curation, J.C.; writing--original draft preparation, J.C.; writing--review and editing, J.C. and D.L. ; visualization, J.C.; supervision, D.L. ; project administration, D.L. ; funding acquisition, D.L. All authors have read and agreed to the published version of this manuscript. This research was supported by the Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (No. 2022R11 IA3069352). Not applicable.