S um T ablets: A Transliteration Dataset of S umerian Tablets (2024)

Abstract: Sumerian transliteration is a conventional system for representing a scholar's interpretation of a tablet in the Latin script. Thanks to visionary digital Assyriology projects such as ETCSL, CDLI, and Oracc, a large number of Sumerian transliterations have been published online, and these data are well-structured for a variety of search and analysis tasks. However, the absence of a comprehensive, accessible dataset pairing transliterations with a digital representation of the tablet's cuneiform glyphs has prevented the application of modern Natural Language Processing (NLP) methods to the task of Sumerian transliteration.

To address this gap, we present SumTablets, a dataset pairing Unicode representations of 91,606 Sumerian cuneiform tablets (totaling 6,970,407 glyphs) with the associated transliterations published by Oracc. We construct SumTablets by first preprocessing and standardizing the Oracc transliterations before mapping each reading back to the Unicode representation of the source glyph. Further, we retain parallel structural information (e.g., surfaces, newlines, broken segments) through the use of special tokens. We release SumTablets as a Hugging Face Dataset (CC BY 4.0) and open source data preparation code via GitHub.

Additionally, we leverage SumTablets to implement and evaluate two transliteration baselines: (1) weighted sampling from a glyph's possible readings, and (2) fine-tuning an autoregressive language model. Our fine-tuned language model achieves an average transliteration character-level F-score (chrF) of 97.55, demonstrating the immediate potential of transformer-based transliteration models in allowing experts to rapidly verify generated transliterations rather than manually transliterating tablets one-by-one.

## introduction sumerian is the world's earliest attested written language, marking the transition from prehistory into history as well as reflecting a rich written tradition spanning three thousand years. these texts are an invaluable resource in the study of ancient near eastern culture, politics, economics, and more. during the latter half of the fourth millennium bce, a sophisticated record-keeping system emerged in southern mesopotamia, now known as proto-cuneiform (selz, 2020). over time this system evolved foot_0 to handle natural language. by about 2900 bce this writing system, known as cuneiform, is concretely recognizable as encoding sumerian. mesopotamian scribes originally devised the cuneiform script to write sumerian. this script was later adapted to encode other languages throughout the near east, such as akkadian. to form glyphs, scribes would typically compose stylus impressions on a wet clay tablet 2 . because cuneiform writing was impressed or inscribed on durable materials, texts have survived to the present in tremendous quantity @xcite . uncovered during archaeological excavations of ancient cities beginning in the nineteenth century ce, these tablets had to be subsequently deciphered. deciphering sumerian, a language isolate, proved particularly challenging, and some periods and genres are still not completely understood. sumerian cuneiform glyphs are frequently poly-valent; that is, they have many possible readings (of no necessary semantic or phonetic relation) depending on the context. for instance, 𒅗 can be read as ka "mouth," dug 4 "to speak," kiri 3 "nose," zuh "to steal," the syllable ka, and more. when reading a tablet, an assyriologist must often consider various possibilities for each glyph to achieve a set of consistent readings. they represent their interpretation through the process of transliteration. transliteration is a modern, conventional system for representing sumerian in the latin alphabet. conventions were established at various points in the modern, 150-year history of sumerian decipherment and do not necessarily reflect the current understanding of sumerian phonology or morphology. in transliterations, homophones are distinguished via subscripts; for instance, e and e 2 are homophonic-but semantically unrelatedreadings of different glyphs. additionally, hyphens are used to join nominal/verbal roots with affixes @xcite . in 1996, the electronic text corpus of sumerian literature (etcsl) @xcite project began publishing transliterations online. this project became archival in 2006, soon followed by other projects such as the cuneiform digital library initiative (cdli) (cdli contributors, 2024) and the open richly annotated cuneiform corpus (oracc) @xcite . thanks to we follow assyriological convention by referring to texts generically as tablets. because assyriologists are reading from either the physical text or an image, no digital representation of the original text's glyphs is typically recorded. today, most cuneiform glyphs have been added to unicode foot_2 . however, easily accessible foot_3 , standardized datasets of paired sumerian unicode glyphs and transliterations remain limited, barring the development of transliteration models. in this paper, we present the first large-scale, easily accessible dataset of 91,606 sumerian tablets as glyph-transliteration pairs, containing a total of 6,970,407 glyphs. we additionally include ids foot_4 , period, and genre metadata for each tablet to be used for results analysis. our dataset, sumtablets, is derived from a collection of publicly available sumerian language resources, primarily the electronic pennsylvania sumerian dictionary (epsd2) @xcite and the oracc sign list (osl) @xcite . these projects aggregate and index transliteration data from across oracc, which shares data with cdli and includes data from other current and former projects 6 . because of how they are formatted and because they do not include parallel unicode glyph tablet representations, however, the data on oracc are not immediately suited for glyph-to-transliteration tasks. we preprocess these data to clean and standardize them, converting structure-related anno-tations into special tokens. then, since a given reading maps back to only one glyph, we utilize unicode-reading dictionaries provided by epsd2 and osl to convert each reading back into its source glyph. we upload our dataset to hugging face (hug-gingface inc., 2024), the largest and most widely utilized library for sharing datasets for machine learning tasks. we intend to use hugging face's git-based version control to provide experiment reproducibility over time, with versions containing snapshots of the continuously updated oracc data. our dataset, sumtablets, builds on previous open-source projects by: using our dataset, we develop and compare two baseline transliteration approaches. the first is a weighted dictionary mapping; for each glyph we sample one of the glyph's possible readings according to its frequency. the second is a language model that we fine-tune for the glyph-totransliteration task. as far as we are aware, we are the first to develop an automatic sumerian transliteration model. evaluated on a held-out test set, the dictionary-lookup approach obtains a character-level f-score (chrf) @xcite of 61.22, while the fine-tuned model achieves a chrf score of 97.54. our goals in releasing this dataset are to facilitate the development of transliteration models and to demonstrate the potential of adapting large pretrained multilingual models for the task. we envision web-based tooling built on top of neural transliteration models helping assyriologists to generate transliterations more quicklyallowing them to rapidly validate model outputs rather transliterating each tablet from scratchand target review of potential errors in existing transliterations. additionally, transliteration models serve as an essential step in eventually developing a complete sumerian translation pipeline. finally, as a language isolate, sumerian poses a unique syntactic challenge for cross-lingual models, and opens new avenues of research into the transfer of language understanding. ## related work to the best of our knowledge, our work represents the first to formulate sumerian transliteration as an nlp task and to develop a transliteration model. however, prior works have utilized nlp techniques for other tasks in parsing and analyzing sumerian cuneiform. the machine translation and automated analysis of cuneiform languages (mtaac) project @xcite sought to develop a pipeline for sumerian annotation, translation, and information extraction, working primarily with ur iii transliterations. @xcite this data to include the electronic text corpus of sumerian royal inscriptions (etcsri) @xcite . @xcite then used mtaac data in conjunction with cdli and etcsl data to train models for partof-speech (pos) tagging, named entity recognition (ner), and translation, aiming primarily to build generalizable cross-lingual methods for performing these tasks on low-resource languages. the compass @xcite also explores using cuneiform data for research tasks, such as reconstructing social graphs. perhaps most similar to our work, gordin et al. develop a neural network to automatically transliterate akkadian from unicode cuneiform glyphs. others have built datasets also representing tablets' glyphs in unicode. @xcite oracc dataset to build a dataset of 13,662 tablets for the task of language and dialect identification. more recently, chen et al. used cdli data to create cuneiml, a dataset of 38,947 tablets with photos, unicode glyphs, transliterations, and metadata, also designed primarily for classification tasks. both of these datasets include both sumerian and akkadian texts, whereas our dataset only includes monolingual sumerian texts. furthermore, our dataset is larger, designed specifically for the transliteration task, and is easily accessible through hugging face. outside of nlp, an exciting area of research is using computer vision methods to identify cuneiform signs from images @xcite . efforts in visual classification and transcription of cuneiform are enabled by projects that have open-sourced high-quality 2d and 3d images of tablets @xcite . and beyond cuneiform, assael et al. used deep learning methods to restore fragmented ancient texts in ancient greek. as sumerian is a low-resource language, it is infeasible to train a transformer-based language model on sumerian from scratch rather than adapting cross-lingual representations in existing models. fortunately, the recent success of large crosslingual nlp models such as mbert @xcite , xlm-r @xcite , m-t5 @xcite , and bloom (bigscience @xcite have steadily raised the bar for zero-and few-shot cross-lingual performance on benchmarks such as xtreme @xcite and mega @xcite . recently, benchmarks to measure a model's ability to perform nlp tasks in extremely low-resource and orthographically-diverse languages have emerged, such as indicxnli @xcite for lowresource indian languages, and sukhareva et al. who develop a pos tagging benchmark for hittite, another cuneiform language. sumtablets marks the first benchmark for sumerian neural machine transliteration. ## creating sumtablets sumtablets is built upon the metadata and transliterations provided by epsd2 via json files foot_6 . these transliterations were created or manually typed by scholars working in different projects around the world over decades of evolving knowledge of sumerian vocabulary and grammar; they also contain extensive (but not useful for our purposes) embedded ascii annotation. we begin by preprocessing the transliterations to normalize conventions, remove annotations, and convert formatting information into special tokens. then, we use dictionaries built from epsd2 and osl resources to map each reading back to a unicode representation of its source glyph. the result is a set of unicode glyph-transliteration pairs with parallel formatting, allowing language models to most effectively learn the relationships between the two representations. ## evaluating transliteration performance the scale and standardization of sumtablets enables new methods to be applied to the task of sumerian transliteration. in this section, we leverage our dataset to develop and compare two transliteration approaches: a straight-forward 'dictionary baseline' and a 'neural baseline'. first, we define the transliteration task. ## limitations we note that our work has some limitations, both in terms of the sumtablets dataset and the transliteration model. ## conclusion we introduce sumtablets, the first collection of paired glyph-transliterations extracted from 91,606 sumerian tablets. our dataset provides a resource for experts and non-experts alike to contribute to the development of transliteration models. we define the transliteration task, evaluation method, and establish a baseline performance so that future results may be compared. we also demonstrate that-despite sumerian's status as a low-resource language and language isolatelarge pretrained multilingual language models can be adapted to perform the sequence-to-sequence task of transliterating a sequence of unicode cuneiform glyphs with remarkable accuracy. with such an abundance of extant texts and so few specialists capable of reading them, we believe transliteration models will enable assyriologists to spent less time on tedious, from-scratch transliteration and more time on research and translation.