A Question of Confidence: Using OCR Technology for Script analysis (2023)

Abstract: The following article proposes a method employing the Tesseract OCR engine to aid palaeographic analysis and scribal identification. Repurposing the so-called confidence score provided by the OCR engine, different methods of visualization are used to surface differences between font families, script types and manuscript hands.

## introduction this paper introduces a simple method for conducting technology-assisted analysis of script and handwriting styles in printed books and manuscripts. the approach described uses a side product of a tried and tested technology: optical character recognition (ocr). ocr software is traditionally employed for the automatic transcription of text from a digital image to machine-readable output. the method used here largely ignores the transcription but focuses on the so-called confidence scores. confidence scores are usually employed in the process of ocr recognition to assess the probability that the output is correct. normally, low-scoring results are undesirable as they signal a lower probability of accuracy. in this case, however, low scores will be used to identify pages, words or characters that could be of interest for a palaeographic analysis. digital methods, including artificial intelligence (ai), have been plied before to the field of palaeography, for example for the purpose of automatic transcription, the classification of writing styles or scribal identification (see e.g. @xcite @xcite . a contest organized in 2017 by the fifteenth international conference on frontiers in handwriting recognition in 2017, for example, solicited ai-based solutions for the classification of medieval script types, providing a set of labelled training material ("icdar2017 competition on the classification of medieval handwritings in latin script" 2017). a report submitted by @xcite discusses the efficacy of several submissions and remarks that the premise of the task itself builds on a simplified reality. medieval script types do not always have firm boundaries, that is, definitive sets of features that reliably set one type apart from another; some hybrid forms are not easily described with one single label. the categorization of medieval script types often moves in grey zones and a technology trained on human-labelled script types is therefore not automatically free from human bias. conversely, unsupervised learning, which does not rely on labelled data, does not necessarily sort material in ways that are meaningful to scholars and can also be hard to interpret @xcite . thus, whilst ai shows potential for palaeographic analysis, some caution is warranted: worst case scenario, human subjectivity is replaced by ai's accountability gap. the approach introduced here is not meaning to replace human expertise with an automated solution. rather, it attempts to re-purpose a preexisting technology as a heuristic tool that can accelerate the palaeographer's, book historian's, bibliophile's quest for areas of interest in a book or on a page, with the help of ocr confidence scores. the paper mainly showcases different modes of visualizing confidence metrics to aid the discovery of palaeographic phenomena. the following argument will briefly introduce the metrics of word and character confidence and how they can be employed in script or scribal analysis. proposed approach employs confidence scores to identify divergences in script style or abnormalities in letter shapes on book-or page level. the elements identified are then inspected with the help of statistics to establish whether they are of significance. throughout, the open-source ocr a question of confidence: using ocr technology for script analysis antonia karaisl, waseda university engine tesseract (lstm version) is used with different ocr models. foot_0 2 visualizing confidence scores on printed and handwritten material ## conclusion in summary, this article presented an approach to re-purposing ocr technology for identifying peculiarities in historic scripts or differences in scribal hands. the argument aims to show that even though many standard ocr models, in this case tesseract, are overtly trained to focus on the recognition of specific languages in print, the sensitivity of ocr models to differences in fonts can be exploited to highlight differences in script or scribal hand. this is done with the help of the socalled confidence score, which signals the certainty with which the ocr engine assumes the output to be correct. the argument above is outlining several methods of visualizing the confidence score and how this can aid palaeographic analysis. the method is emphatically not intended to classify scripts or to authenticate hands. the experiments merely test confidence scores for their heuristic potential in identifying differences between script types and hands. there are some downsides to this approach; firstly, the confidence score is a blurry metric that can be influenced by many factors, not all of which are relevant to a palaeographic discussion (for example ink bleed-through, speckles or skewed pages). which factor is chiefly to blame for a low confidence score is not necessarily clear. gathering scores from every single exponent, by default from a variety of contexts, however, can help to gain a balanced perspective in that regard. expectations should also be tempered with a view to comprehensiveness -ocr confidence scores cannot be expected to highlight all differences in a font or hand. in that sense, the method as sketched presents a means to break the ice. from a practical perspective, creating ground truth to train bespoke models is time-intensive and low-volume models such as the one used here are not necessarily reliable. within the small scope of this investigation, such a model might have been sufficient for proving a concept, but a better model trained on more ground truth or endorsed by more advanced technology might be needed for a more thorough analysis. with the steady advance of ocr technology and more and more sophisticated attempts to create models for low-volume scripts, it stands to hope that there will be new solutions to the latter issue before too long. lastly, one might argue that these experiments merely help to surface phenomena that are visible to the naked eye anyway. without providing analytical value in itself, the method leaves the ultimate interpretation of these discoveries to the palaeographer's expertise. it should not be forgotten, however, that palaeographic analysis is a painstaking process, and the identification of such differences is extremely time-consuming when done by hand and from scratch. the real value of this method, therefore, is to direct said naked eye to the phenomena in the first place, that is, to speed up the discovery process -and at the best of times help palaeographers discover elements they did not know they were looking for.