Detecting Erroneously Recognized Handwritten Byzantine Text (2023)

Abstract: Handwritten text recognition (HTR) yields textual output that comprises errors, which are considerably more compared to that of recognised printed (OCRed) text. Post-correcting methods can eliminate such errors but may also introduce errors. In this study, we investigate the issues arising from this reality in Byzantine Greek. We investigate the properties of the texts that lead post-correction systems to this adversarial behaviour and we experiment with text classification systems that learn to detect incorrect recognition output. A large masked language model, pre-trained in modern and fine-tuned in Byzantine Greek, achieves an Average Precision score of 95%. The score improves to 97% when using a model that is pre-trained in modern and then in ancient Greek, the two language forms Byzantine Greek combines elements from. A century-based analysis shows that the advantage of the classifier that is further-pre-trained in ancient Greek concerns texts of older centuries. The application of this classifier before a neural post-corrector on HTRed text reduced significantly the post-correction mistakes.

## introduction handwritten text recognition (htr) concerns the automated transcription of a handwritten text, extracting its computerised form from an image. applying htr on an old manuscript can lead to a high character error rate (cer), often due to the lack of training data (pavlopoulos et al., 2023). in the real world, this erroneous htr output is post-corrected by experts, who get involved, typically, in a tedious and time-consuming task. delaying the delivery of manually curated data, hinders both: the preservation of digitised manuscripts that have not yet been transcribed; and the progress of htr for historical manuscripts, which struggles with low resources. this is why researchers from digital humanities have started encouraging the development of natural language processing (nlp) solutions, in an attempt to assist the experts toward this goal, since improving the accuracy of recognition will be exciting for the disciplines struggling with vast corpora and little man power (mcgillivray et al., 2020). ## related work our work focuses on the text classification of recognised handwritten text. in the absence of published studies for handwritten text, however, we summarise work focused on the recognition of printed text. printed and handwritten materials differ, because the latter often yields scrambled recognition output ( ## experimental analysis we opted for machine and deep learning benchmarks, with different transfer learning settings. @xcite ## discussion in this section, first, we assess the possibly adversarial nature of post-correction. then, we describe a synthetic error detection application, followed by a century-based error analysis we performed. ## conclusion this work showed that detecting erroneous recognised lines improves the accuracy of handwritten byzantine text recognition, aiming to assist the disciplines struggling with vast corpora and little man power. by contrast to printed text, errors in handwritten text vary greatly, some of which are easier and others difficult to catch. our experiments showed that two bert models, one pre-trained in modern and the other pre-trained in modern then in ancient greek perform well for the task of erroneous handwritten recognised text classification. the latter performs better, especially for manuscripts from older centuries and for lines with higher recognition error rate. by using a neural textto-text transformer for error correction along with our best performing classifier, we found that adversarial behaviour is not uncommon in the neural corrector's output which could have been significantly limited if a bert classifier, pre-trained and fine-tuned, had classified the input first. directions of future work comprise the development of a larger dataset, which, although not easily developed in our domain of focus, it could potentially allow us to draw more conclusions and lead to further findings. furthermore, we plan to extend our dataset with calibrated evaluation subsets, in order to gain more perspective on the performance of systems for specific error types, including ones that occur naturally and are not meant to be corrected.