C u R e D : Deep Learning Optical Character Recognition for Cuneiform Text Editions and Legacy Materials (2024)

Abstract: Cuneiform documents, the earliest known form of writing, are prolific textual sources of the ancient past. Experts publish editions of these texts in transliteration using specialized typesetting, but most remain inaccessible for computational analysis in traditional printed books or legacy materials. Off-the-shelf OCR systems are insufficient for digitization without adaptation. We present CuReD (Cuneiform Recognition-Documents), a deep learning-based human-in-the-loop OCR pipeline for digitizing scanned transliterations of cuneiform texts. CuReD has a character error rate of 9% on clean data and 11% on representative scans. We digitized a challenging sample of transliterated cuneiform documents, as well as lexical index cards from the University of Pennsylvania Museum, demonstrating the feasibility of our platform for enabling computational analysis and bolstering machine-readable cuneiform text datasets. Our result provide the first human-in-the-loop pipeline and interface for digitizing transliterated cuneiform sources and legacy materials, enabling the enrichment of digital sources of these low-resource languages.

## introduction the cuneiform writing system was used to write around a dozen different ancient languages over a period of more than three millennia. many of these complex writing systems were logo-syllabic and of different language families, from the agglutinative sumerian in southern mesopotamia, to the family of hurrian and urartian in northern mesopotamia and armenia, to indo-european hittite and luwian in anatolia. while the records of many of these languages are in the hundreds or thousands, it is semitic akkadian with its main babylonian and assyrian dialects that is attested on hundreds of thousands of ancient texts @xcite . what all of them share is a similar critical apparatus: a standard latin transcription and notation system, developed by experts in scholarly publications, from the mid-19 th century to the early 20 th century (see appendix a), and is still used to this day. legacy materials such as personal notebooks of curators or researchers, or card catalogues in universities and museums use this notation system extensively (fig. 1 ). many publications and legacy materials have been scanned or photographed, but are largely unavailable as machine-readable text. the ability to automatically digitize them using optical character recognition (ocr) would make their contents readily available to experts and the general public. they can be further used in computational research into the languages, cultures, and history of these societies, as well as a wider use of natural language processing (nlp) techniques, such as part-of-speech tagging, named entity recognition, sentiment analysis, machine translation, and more. this in turn can further enhance cross-lingual research and the creation of linked open data entities as well as knowledge graphs @xcite @xcite @xcite . existing ocr models trained on texts in other languages such as english are not suitable for this task. they do not recognize the diacritics, typographical oddities like mixed upper-and lower-case or sub-and super-script, as well as special symbols required for digitizing cuneiform transliterations. furthermore, many off-the-shelf models are biased by their prior training on character sequences in the @xcite . on the right an example of cuneiform transliteration from the neo-babylonian text corpus published in the dissertation of r. b. dillard @xcite . in order to overcome these challenges, we trained a custom deep-learning based ocr model on transliterations of akkadian, bootstrapping with artificially-generated data, and then fine-tuned with a small set of manually-labelled examples. the data for training and testing the model were taken from open richly annotated cuneiform corpus (oracc) and their equivalent print publications in pdf format. however, those texts were from one period (neo-assyrian) and followed the same editorial conventions. to estimate the real-world usability of the model, we performed two additional digitization experiments on transliterated text produced with a typewriter during the 1970s and 80s: (1) 81 previously undigitized neo-babylonian administrative and archival daily documents published in the 1975 dissertation of raymond b. dillard @xcite ; both were particularly difficult to ocr, and were not a part of the model's training. in the case of the dillard texts, we show that with fine-tuning on only 10 texts, the models' results rose from 53% to 85% accuracy. similarly, in the case of the sumerian lexical cards, after only 60 text lines, the model improved from 87% to 94% accuracy. thus, the model requires a minimal number of examples in order to be a significant assistant in the digitization process of ancient documents. the model is published on the digital pasts lab github repository and is freely available as an online tool in the babylonian engine website, which is undergoing a transformation into a standalone browserbased application. the tool and model will facilitate the digitization of hundreds of thousands of published cuneiform text lines in transliteration, which were previously unavailable for further computational or quantitative study. ## ocr results on training and testing data results in the new cured model has a character error rate (cer) of 9% on clean data and 11% on representative scans. we observe that manually fine-tuning the model with real dataset images greatly improves our accuracy, even though the fine-tuning dataset was extremely small. the baseline model, only trained on artificial data, overfit to this type of data and did not generalize well to real scans. visually observing the baseline model output showed that it regularly had trouble distinguishing certain characters (e.g. "a" vs. "u"), and we hypothesize that this is because of the different appearance of these characters in the artificial training data fonts and the fonts used in the test data. fine-tuning likely helps the model to quickly adapt to these differences. ## related literature to the best of our knowledge, this is the first custom-trained ocr model for transliterated cuneiform documents, trained initially on transliterations. see, however, the tesseract-based model used for ocr'ing secondary literature in assyriology, which includes text editions @xcite . human-in-the-loop pipelines for transcribing historical and epigraphical documents from other periods, however, are revolutionizing how those are being recorded and studied in the humanities and the galleries, libraries, archives, and museums (glam) sector. some of the most impactful tools in this regard are transkribus and escriptorium, each of which has produced hundreds of studies based on their ocr/htr engine, and several more are on the rise @xcite @xcite . it is important to separate, however, the ocr/htr efforts from latin transliterations and ocr/htr of cuneiform signs themselves on clay tablets, stone inscriptions, etc. identifying cuneiform signs requires other designated models, and the several advances in recent years are summarized in @xcite ; see also the newly published contribution by @xcite . the high-performance of our model on minimal ground-truth data was possible due to the rel-ative simplicity of generating representative artificial training data. improvements in recent years in the generation of data that is similar enough to ground truth is proving more and more vital in aiding the digitization of low-resource languages, such as cuneiform @xcite and aramaic inscriptions @xcite , to name a few. these methods are probably to be vital in the upcoming years to push forward the digitization of ancient languages @xcite . although our work only covers ocr digitization of transliterations as printed in published sources, there has also been work on automatic conversion of such transliterations to phonological transcriptions representing how texts were pronounced in the akkadian language. see @xcite for an example of a such deep-learning based model. ## conclusion to aid the community of cuneiform experts in digitizing published records of cuneiform texts, we developed an ocr system for recognizing akkadian latin transliterations written using standard scholarly conventions. because of a lack of natural labelled training data, we bootstrapped an ocr model using the kraken open-source framework by generating artificial training data, rendering text from the saa corpus using various fonts and text styles. after fine-tuning the resulting model on a small set of manually-labelled scans, we achieved 89% accuracy on a representative set of scans. we integrated this model in a human-in-the-loop tool called cured (cuneiform recognition documents), to allow scholars and students to ocr various scanned or photographed materials, and help continuously improve their model. we further showed this tool in practice, by performing two real-world experiments ocr'ing text editions and legacy lexical materials in machine typeface, both of which included handwritten notation. the finetuning of the two experiments was integrated into our model, which is also on the cured online tool, making it already highly effective for ocr'ing machine typed transliterations. minimal fine-tuning is needed to improve its results on unseen texts, and the same should hold true for transliterations of other languages using the cuneiform script. we provide this model as an open-source contribution to researchers of the ancient near east and the general public, in hopes that it will make cuneiform inscriptions more accessible in machine-readable form. limitations our current ocr system has been tested only on latin transliterations of akkadian and sumerian cuneiform texts, but not on the other languages of the ancient near east using the cuneiform script. while we assume this transfer learning would be easy for the model given the similarities in the transliteration practices (see appendix a), that remains to be seen. additionally, both experiments show how the model can be effectively fine-tuned with few examples to drastically improve performance. however, the results are never perfect. a common challenge in both experiments is the presence of many handwritten notes, such as accents, subscripts, diacritics, special characters, square brackets, or simply marginalia scribbled around the text. these factors lead to inaccuracies in the ocr results, particularly creating errors in the line segmentation. the cured model, with its human-friendly interface, permits users to quickly correct the remaining errors. the fine-tuning process makes the correction phase extremely efficient. it may not completely make typing of editions a thing of the past, but it reduces the time by at least 90%. in addition, further manual improvements can be considered, such as validating the ocr'ed results against known cuneiform sign readings, or combining cured with handwritten text recognition (htr) @xcite to identify marginalia etc. furthermore, the continual fine-tuning of the model makes it familiar with additional typefaces and editorial conventions. the significant uptick in accuracies before fine-tuning between the experiments (from 53% on the dillard texts to 87% on the psd card catalogue) shows this quality, as both experiments share a similar typeface. initial results on unseen texts will thus continue to improve as more corpora are added for training, and fewer and fewer examples will be required for fine-tuning. not anticipate any major ethical concerns stemming from this work.