## 6 Experimental Evaluation
### Setup  
The following datasets are used for experimental evaluation (see Section 3):  
* The **training set** consists of 30,000 synthetically generated pages. * The **alignment set** includes 899 manuscript pages together with their transcription. * The independent **test set** contains the same 47 manuscript pages used by [PERSON] et al. [3], which were taken from ten different manuscripts. Please note that because the proposed method is annotation-free, we did not have access to a human-annotated validation set for fine-tuning hyper-parameters of the system. Instead, reasonable defaults have been chosen and have been visually validated on several pages of the alignment set. The test set has only been used to measure the final system performance. All 899 pages of the alignment set are used for alignment and all 47 pages of the test set are used for evaluating character detection, both before and after self-training. We distinguish two states of the character detection system, accordingly:  
* **Synthetic character detection. ** The YOLO-based character detection system is called _synthetic_ before self-training, because it is trained on synthetic page images. * **Adapted character detection. ** After self-training on the aligned pages it is called _adapted_, because it has been adapted to real manuscript pages. The system is retrained after aligning all pages of the alignment set. For page synthetization, we consider a total of 30,000 pages with variable scaling of the main text area, number of columns and rows, and image distortions (see Section 5.1). We have also conducted preliminary experiments with 10,000 and 60,000 pages, respectively. While the results for the former were clearly worse, only slight improvements were observed for the latter. Four variants of synthetic pages are considered with respect to the characters used:  
* **Printed (1 class). ** All 26,969 characters of the Nom font are used for synthetization with uniform distribution. All bounding boxes are labeled with the same class, i.e., character detection is trained for bounding box regression only. * **Printed (4855 classes)**. Only the 4855 characters actually appearing in the 899 transcribed pages are used for synthetization. Furthermore, the bounding boxes are labeled with the actual character class, i.e., character detection is trained conjointly for bounding box regression and character classification. * **Handwritten (CASIA). ** Images of traditional Chinese characters from the CASIA-AHCDB [9] are used for synthetization, more specifically all 832,939 images from the training set of the basic category of the first style, including a total of 2365 character classes. The bounding boxes are labeled with one class. * **Handwritten (Nom). ** Images of Nom characters from the alignment result are used. The bounding boxes are labeled with one class. This scenario is also a form of self-training. However, the synthetic pages that are generated do not include the manuscript background. Please note that only the characters are different for the four variants. Otherwise, the 30,000 synthetic training pages are generated in the same way. For YOLO-based object detection, we use a standard implementation from Ultralytics ([[https://github.com/ultralytics/yolov5](https://github.com/ultralytics/yolov5)]([https://github.com/ultralytics/yolov5](https://github.com/ultralytics/yolov5)); accessed on 25 May 2021) and distinguish the following types of models:  
* **YOLOv5s, YOLOv5m, YOLOv5l, YOLOv5x. ** Different model sizes ranging from 7.3M weights (YOLOv5s) to 87.7M weights (YOLOv5x), pre-trained on the COCO dataset [36] and using hyper-parameters suggested for fine-tuning, (\"hyp.finetune.yaml\",version Oct 11, 2020, commit cc03c1d5727e178438e9f0 ce0450 fa6 bdbbe1 ea7.) including an initial learning rate of 0.0032. * **YOLOv5s-scr, YOLOv5m-scr, YOLOv51-scr, YOLOv5x-scr.