## 3 Results and Discussion  
### Evaluation of Automated Calibration Algorithm  
In order to assess the achieved improvement of the automated calibration algorithm, we process the laser calibration images of five different series obtained from different cameras and laser projection units with 15 images each. By the use of binary classifiers, the detection results on the checkerboard pattern are categorized for each frame according to the following Table 1.  
The sensitivity \(TPR\)  
\[TPR=\frac{TP}{TP+FN} \tag{3}\]  
is a measure for the success of the detection, while the success of the post-selection is quantified by the specificity \(TNR\)  
\[TNR=\frac{TN}{TN+FP}. \tag{4}\]  
Figure 11: Flowchart of the laser point tracking. White corners indicate new processing steps and red corners indicate a necessary or optional user interaction.  
In Figure 12, the results of the present algorithm (green) are compared to the previous calibration procedure by [20] (blue). These measures are calculated for each frame separately and then averaged over all images in each series. The vertical error bars indicate the corresponding standard deviations. The sensitivity (top) and the specificity (bottom) are significantly improved by the new pre- and post-processing steps. Please note that sensitivity and specificity have to be considered in combination with each other. The specificity of 100% in Series 3 suggests a successful calibration process, but arises from the fact that the previous algorithm barely detected any corners at all as indicated by 0.6% sensitivity in the same series.  
The mean values in the upper right corners of Figure 12 are derived from averaging all frames of all series. The improvement in sensitivity is 40.1% and in specificity 22.3%. Statistical analysis using IBM SPSS Statistics v21 (i.e., pairwise comparison of calibration frames over all series, applying Wilcoxon Rank Test - data was not normally distributed) showed a statistically highly significant  
\begin{table}
\begin{tabular}{c c c} \hline \hline  & **Inner Corners** & **Outer Corners** \\  & **of Checkerboard** & **of Checkerboard** \\ \hline Detected by algorithm & True positive (TP) & False positive (FP) \\ Not detected by algorithm & False negative (FN) & True negative (TN) \\ \hline \hline \end{tabular}
\end{table}
Table 1: Confusion matrix for evaluation of automated calibration algorithm.  
Figure 12: Evaluation of the automated calibration algorithm compared to its previous version by [20] including the sensitivity (**top**) and the specificity (**bottom**). The depicted bars represent the average value for each series while the vertical error bars indicate the standard deviation within each series. The horizontal lines represent the average value over all frames of all series and the error bars indicate the corresponding standard deviation.  
improvement (\(p=0.000\)) between the methods of [PERSON] [20] and [PERSON] [22] for both measures. In addition to that, the standard deviation is considerably decreased from 38.2% to 15.1% for the sensitivity, which is owed to the extended pre-processing. The remarkable decrease from 19.6% to 0.7% in the standard deviation of the specificity arises from the newly introduced post-selection steps. This indicates a reduced dependence on the quality of the calibration images. In conclusion, these results demonstrate that the new calibration algorithm is effective and robust.