## 4 Evaluations
### Precise Segmentation and Trained OCR of Early Printed Books
#### 4.3.3 Evaluating the Iterative Training Approach  
The manual correction effort not only scaled with the number of lines that have to be corrected, but also with their recognition quality. To be able to evaluate the effects and benefits of the iterative training thoroughly, many different values and results were recorded. Since their evaluation and interpretation is a quite complex task, we first introduce them and describe them in detail in Figure 9 before we list the results of selected works in Table 6.  
\begin{table}
\begin{tabular}{c c c c c c c c c c c c c c} \hline \hline  & & & & \multicolumn{3}{c}{**New Data**} & \multicolumn{3}{c}{**Eval**} & **All Data** & \multicolumn{3}{c}{**Correction**} \\ \cline{3-13}
**Book Short** & **User Exp.** & **It.** & \#P & **\#L** & \(\mathbf{\ell}_{\text{Corr}}\) & \(\mathbf{\ell}_{\text{Corr}}\) & \(\mathbf{\ell}_{\text{Corr}}\) & **CER** & **CER** & **\#P** & **\#L** & \(\mathbf{\ell}_{\text{min}}\) & **MMM** & **SU** \\ \hline C1541 & 2B & 1 & 3 & 88 & 18 & 12 & 4.80 & 5.51 & 3 & 88 & & & \\  & & 2 & 5 & 146 & 23 & 9.5 & 2.52 & 3.09 & 8 & 234 & & & \\  & & 3 & 20 & 613 & 41 & 4.0 & 0.90 & 1.29 & 28 & 847 & & & \\  & & 4 & - & - & - & - & 0.92 & - & - & 82 & 169 & 2.1 \\ \hline P1484 & 2B & 1 & 5 & 110 & 14 & 7.6 & 3.53 & 3.95 & 5 & 110 & & & \\  & & 2 & 6 & 116 & 8 & 4.1 & 0.89 & 1.48 & 11 & 226 & & & \\  & & 3 & - & - & - & - & - & 0.34 & - & - & 22 & 29 & 1.3 \\ \hline N1499 & 2A & 1 & 2 & 105 & 65 & 37 & 25.22 & 23.59 & 2 & 105 & & & \\  & & 2 & 3 & 138 & 20 & 8.7 & 0.54 & 2.23 & 5 & 243 & & & \\  & & 3 & 5 & 389 & 25 & 3.9 & 1.24 & 1.63 & 10 & 632 & & & \\  & & 4 & - & - & - & - & 0.20 & - & - & 110 & 474 & 3.6 \\ \hline \hline \end{tabular}
\end{table}
Table 6: Evaluation of the ITA. For each book processed by a user, we provide all values and results necessary to reconstruct and evaluate the progress of the GT production and training. In the new data column, the number of the newly added Pages (#P) and the corresponding number of Lines (#L) is listed, as well as the time required to produce the transcription. Furthermore, the CER is given, which is calculated from the OCR result achieved by the model from the previous iteration and the newly created GT. For comparison, the CER achieved on the separate and constant Evaluation set (Eval) is recorded. All data shows the number of available GT pages and lines at this point which then serve as training data for the new model, which is used for the next iteration. In the correction columns, we compare the actual required correction time when applying the ITA (ITA) with the projected time when only using the output of the Mixed Model (MM) to get to the point where the final (and in case of MM, the first) model is trained. The Speedup factor (SU) is calculated for each book for the two user groups separately and for both groups combined.  
Results  
There were several interesting things to be taken away from the results summarized in Table 6. First of all, it is shown that the ITA yielded a significant speedup regarding the correction time. On average, the manual effort was almost cut in half (average speedup factor 1.9, last column), with the experienced users benefiting considerably more compared to the inexperienced ones (factors of 2.3 and 1.3).  
Another eye-catching abnormality was the discrepancies between the performances of the same models on the new and the evaldata. While some deviations had to be expected and can be considered negligible, others seem to be too substantial to be disregarded as variance. For example, when processing, C1557 achieved a good CER of 2% on the new data, but at the same time struggled severely with the eval data (10% CER). An explanation is given in the next section.