# CREMMA Medii Aevi: Literary manuscript text recognition in Latin

Thibault Clérice, Malamatenia Vlachou-Efstathiou, Alix Chagué

<figure><img src="image_1.png" /></figure>

## To cite this version:

Thibault Clérice, Malamatenia Vlachou-Efstathiou, Alix Chagué. CREMMA Medii Aevi: Literary manuscript text recognition in Latin. Journal of Open Humanities Data, 2023, 9, pp.4. 10.5334/johd.97. hal-03828353v5

HAL Id: hal-03828353

https://enc.hal.science/hal-03828353v5

Submitted on 16 Apr 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>
<figure><img src="image_3.png" /></figure>

# CREMMA Medii Aevi: Literary Manuscript Text Recognition in Latin

RESEARCH PAPER

|u[ ubiquity press

THIBAULT CLÉRICE

<figure><img src="image_4.png" /></figure>

MALAMATENIA VLACHOU-EFSTATHIOU

<figure><img src="image_5.png" /></figure>

ALIX CHAGUÉ

<figure><img src="image_6.png" /></figure>

## ABSTRACT

This paper presents a novel segmentation and handwritten text recognition dataset for Medieval Latin from the 11th to the 16th century. It connects with Medieval French datasets, as well as earlier Latin datasets, by enforcing common guidelines, bringing 263,000 new characters and now totaling over a million characters for medieval manuscripts in both languages. We provide our own addition to Ariane Pinche's Old French guidelines to deal with specific Latin cases. We also offer an overview of how we addressed this dataset compilation through the use of pre-existing resources. With a higher abbreviation ratio and a better representation of abbreviating marks, we offer new models that outperform the Old French base model on Latin datasets, improving accuracy by 5% on unknown Latin manuscripts.

## CORRESPONDING AUTHOR:

Thibault Clérice

Centre Jean Mabillon, École nationale des Chartes, PSL University, Paris, France

thibault.clerice@chartes.psl.eu

## KEYWORDS:

Handwritten Text Recognition; Latin; manuscripts; Middle Ages; Layout Segmentation

## TO CITE THIS ARTICLE:

Clérice, T., Vlachou-Efstathiou, M., & Chagué, A. (2023). CREMMA Medii Aevi: Literary Manuscript Text Recognition in Latin. *Journal of Open Humanities Data, 9: 4*, pp. 1-19. DOI: https://doi.org/10.5334/johd.97

<footnote>*Author affiliations can be found in the back matter of this article</footnote>
# 1 CONTEXT AND MOTIVATION

**Institutional and academic contexts** Handwritten text recognition (HTR) and its upstream task layout segmentation (LS) have become two important topics in the context of Digital Humanities and digital approaches to cultural heritage collections in the GLAM<fnref n="1" /> domain. Its growth over the past three to four years in digital projects can easily be linked to the emergence of user interfaces (UI) allowing for the annotation of ground truths (GTs, data that will be used for training), training new models (for the transcription and, lately, for the segmentation) and for the automatic transcription of the users' own data. At first, only Transkribus (Kahle, Colutto, Hackl, & Mühlberger, 2017) provided such a service through the READ project without fees or infrastructure requirements.<fnref n="2" /> At the end of the 2020 European Union funding, Transkribus became a paid service accelerating the interest growth of at least one alternative, namely eScriptorium (Kiessling, Tissot, Stokes, & Ezra, 2019) at the EPHE-Scripta-PSL. Unlike the former, the latter is completely open source, at the cost of not offering a centralized server.

In this context, the *Consortium pour la Reconnaissance d'Écritures Manuscrites des Matériaux Anciens* (CREMMA) project was created to fund a regional server. Its aims are to support students' training and to provide local researchers with a free solution. The CREMMA funding consisted of a grant for the initial cost of the infrastructure as well as an evaluation grant for providing base models for the community of CREMMA's users. The latter was divided into two main languages: French and Latin, from the 9th to the 21st century. A postdoctoral position, CREMMAlab provided the infrastructure with complementary time for building a dataset (CREMMA Medieval) and expertise around transcribing medieval manuscripts.

As the CREMMA project was being drafted, Chagué and Clérice (2020) provided a solution for facilitating the FAIR principles in an HTR context and providing machine-actionable metadata for datasets. *HTR-United*, both a catalog of open source HTR ground truths and a toolkit to strengthen the control of documentation and validity of HTR data, records, as of late October 2022, 56 datasets composed of 41.5 million characters, 725,862 lines in over 13 languages and 6 scripts. *HTR-United*'s catalog provides a useful overview to build new datasets which can complement previous ones.

**HTR for Latin and Old French** Handwriting in the Middle Ages can, in a simplistic way, be divided into two big writing systems: cursive and calligraphy (Bischoff, 1985, pp. 58 sqq.). They reflect two complementary practices, namely cursive hands (*écritures d'usage*), which are more common to everyday and administrative documents such as accounting books and letters, and book hands.<fnref n="3" /> While cursive represents a harder challenge due to the variability of handwriting styles, both families have the potential to be highly abbreviated, depending on the expected audience of the document: literary classics, such as Cicero or Vergilius, might be less abbreviated than pharmaceutical recipes, scholastic works, or accounting books. This situation resulted in mainly two different kinds of strategies for creating *HTR* ground truth datasets: (1) datasets that would resolve abbreviations directly in the transcription (a practice found mostly used by historians, and quite common for cursive, specifically in France) and (2) datasets that would keep a diplomatic approach to transcription.

Our dataset builds on the experience of Ariane Pinche, specifically her work on the CREMMA Medieval dataset, which treats different variations of Old French from the 13th to the 15th century, with a heavy focus on the first section of the period. As the first recipient of the CREMMALab post-doctoral funding, Pinche co-organized a research seminar around the formalization of transcription guidelines for graphemic transcription of Old French (Pinche, 2022c). Based on her recommendations, a few datasets emerged around the École nationale des chartes and the CREMMA project. Notably, the Gallic(orpor)a corpora (Gabay, Pinche, Leroy, & Christensen, 2022) and the course project DecameronFR (Biay, Boby, Konstantinova, & Cappe, 2022) provided two additions for Old French and Middle French data, centered around the end of the middle ages. On the opposite, the Caroline Minuscule project (Hawk, Karaisl, & White, 2018) was realigned in ALTO XML and adapted to the guidelines as it provided some foundations for recognizing the

<footnote n="1">1 Gallery Library Archives Museums.</footnote>

<footnote n="2">2 https://readcoop.eu/our-story/.</footnote>

<footnote n="3">³ The distinctive functions gradually ceased to exist/converged as they were used interchangeably depending on the context.</footnote>
Caroline script specific to the first centuries of the early middle age. Vlachou-Efstathiou (2022a,b) provided a complementary dataset based on the transcriptions of two Latin manuscripts from the 9th century. When the work for *CREMMA Medii Aevi* began, we identified a lack of data for the second half of the middle age (1100–1500, see Table 1).

Clérice et al.
Journal of Open
Humanities Data
DOI: 10.5334/johd.97

<table><caption>**Table 1** Datasets following the Pinche Guidelines or adapted through Choco-Mufin. Characters' counts are rounded to the closest thousands.</caption><thead><tr><th>AUTHORS</th><th>DATASET</th><th>PROJECT</th><th>CHARACTERS</th><th>PERIOD</th><th>LANGUAGE</th></tr></thead><tbody><tr><td>White, Karaisl, and Clérice (2022)</td><td>Caroline Minuscule</td><td>Rescribe</td><td>17,000</td><td>800–1200</td><td>Latin</td></tr><tr><td>Vlachou-Efstathiou (2022a)</td><td>Eutyches</td><td>-</td><td>87,000</td><td>850–900</td><td>Latin</td></tr><tr><td>Pinche (2022a)</td><td>CREMMA Medieval</td><td>CREMMAlab</td><td><strong>593,000</strong></td><td>1100–1499</td><td>French</td></tr><tr><td>-</td><td><strong>CREMMA Medii Aevi</strong></td><td>CREMMA</td><td>263,000</td><td>1100–1600</td><td>Latin</td></tr><tr><td>Biay et al. (2022)</td><td>DecameronFR</td><td>-</td><td>20,000</td><td>1430–1455</td><td>French</td></tr><tr><td>Gabay et al. (2022)</td><td>Manuscrits du 15e siècle</td><td>GalliCorpora</td><td>169,000</td><td>1400–1500</td><td>French</td></tr><tr><td><strong>Total</strong></td><td></td><td></td><td>1,149,000</td><td></td><td></td></tr></tbody></table>

## 2 DATASET DESCRIPTION

**Object name:** CREMMA-Medieval-LAT-0.1.1.zip

**Format names and versions:** XML (ALTO), JPEG

**Creation dates:** 2022-01-01 / 2022-09-22

**Dataset creators:** Thibault Clérice (Organization, Curation, Transcription, Design), Malamatenia Vlachou- Efstathiou (Curation, Transcription, Design), Alix Chagué (Organization)

**Language:** Latin

**License:** CC0

**Repository name:** Zenodo (http://dx.doi.org/10.5281/zenodo.7013436)

**Publication date:** 2022-10-20

## 3 METHOD

### 3.1 GENERAL ASPECTS OF THE CORPUS

**Corpus construction theory** Borrowing the terminology from the linguistic domain (Bauer & Aarts, 2000) where data construction methods have long been examined, evaluated, and reconsidered, we shall examine the following methodological aspects. Contrary to the notion of “sampling” which is, by definition, a random selection procedure, “corpus construction” implies a systematic selection of materials that obey a specific rationale, where its efficiency depends on the research question. “Representative sampling” is where these two approaches converge. Sampling secures efficiency in research by providing a rationale for studying only parts of a population without losing information. Its key feature is “representativeness” of the system in question. Sampling criteria and focal variables correlate. In HTR for medieval manuscripts, “representativeness” was approached in terms of the medieval handwritten Latin language’s characteristics as a system comprised of abbreviations, ligatures, and punctuation signs alongside graphemes. Different genres, scripts, and their degrees of formality served as instances of this system.

**Document sampling strategy** From the three registers making up the construction of a qualitative corpus according to Bauer and Aarts (2000), namely channel, domain, and function, only the first parameter is constant in our case: the sample represents exclusively the written Latin language while giving room to texts of multiple functions addressed to different audiences belonging to various genres (while not aiming at exhaustiveness at this stage). The corpus construction can be regarded as a cyclical process: it has not been entirely determined a priori but rather evolved, bearing in mind the logic of complementarity regarding the already existing
datasets. Estimated abbreviation rate and use of specific characters, known genres and scripts were implemented to compensate for what was thought to be missing from the network of the corpus and the corpus itself in order to make it as "representative" as possible. HTR engines are language agnostic, but the same cannot be said for the resulting models, which means that it depends on the representativeness of the sample to determine whether a model will work on "similar" or "out-of-domain" documents.

Three distinctive selection processes have been applied in our case:

1. The first set of documents was selected purely on their linguistic feature, their readability, and their availability as both digitized manuscripts and editions which could be found either online or in local libraries. It led to the inclusion of classical texts such as Seneca's *Medea*. Script was not taken into account.

2. In a logic of complementarity, the second part of the corpus was dictated inversely by content. More specifically, given the relative absence of ligatures and abbreviations in classical texts, we chose documents that would display a higher degree of abbreviations. This both induced or led to a genre selection process, specifically for medical and scholastic data. At the same time, script diversity was added to the consideration and came naturally as a sort of by-product.

3. Finally, as we wanted to test Kraken models, we sought a transcription project that would provide us with data that would help us evaluate our own. This led to the alignment of the Eichenberger and Suwelack (2021) dataset, produced in the context of a transcribathon in Berlin and containing genres new to our corpus (Book of Hours, Psalms, etc.).

**Quantitative aspects of the corpus** Corpus size depends largely on the subjective criteria and resources of each project and little can be said as a general rule: one needs to consider the limitations that stem from the effort put into producing the corpus, the budget available, the number of representations one wants to characterize, and some minimal and maximal requirements (in our case the quota for the production of an efficient HTR model). Building a turn-key HTR model applicable to as large a range of unseen manuscripts as possible is undoubtedly the end goal. With the production of ground truth being expensive but with increasingly more open-access models available to the public, the challenge is finding the right combination of GTs (either to create a model from scratch or to fine-tune an existing one) that yield the best results. This is where considerations of size and variety enter the discussion and affect directly the quantitative corpus construction strategy.

More specifically, while conducting an experiment on Caroline Minuscule OCR models, Hawk et al. (2018) conclude that “relative preponderance”<fnref n="4" /> in small training pools was a considerably more important factor than that of size, which inversely impacts the accuracy of the models resulting from larger training pools. A careful conclusion would be that a specific combination of manuscripts can yield exceptional results, even though the reasons behind such results or the criteria for the respective manuscripts to be combined are not entirely clear yet. This means that quantity-wise we sought to find a balance between the diversity and size of the GT, always making sure that the ground truth yields an efficient model for individual manuscripts on the training set. Training and fine-tuning experiments conducted by Pinche showed that a specialized model per script isn’t always necessary, but the variety of the training set increases its robustness. Therefore, the size of each GT belonging to the training set was limited to 5 pages per script variation (depending on the density of the layout),<fnref n="5" /> examining whether this balance can contribute to the production of generic models.<fnref n="6" />

<footnote n="4">4 The proportionally higher or lower representation of a manuscript or subgroup of manuscripts in the training pool and the subsequent effect on the accuracy of the respective test manuscript or subgroup.</footnote>

<footnote n="5">5 Aside from 3 documents coming from the *Faithful transcriptions dataset*, that were utilized rather as evaluating tools at the end of the project.</footnote>

<footnote n="6">6 On GT size for OCR experiments see Ströbel, Clematide, and Volk (2020).</footnote>
**Segmentation vocabulary: SegmOnto** With the emergence of efficient layout analyzers and easy-to- use interfaces, the need for efficient segmentation models increases (as does the need for large amounts of data) based on the aggregation of heterogeneous documents. Alongside text recognition, eScriptorium allows for layout annotation using ontologies and controlled vocabularies. For this, researchers need to agree on a limited common vocabulary and share common practices to facilitate the interoperability of their ground truth.

In order to identify the different areas of the document and the type of lines present on the page as well as to characterize them from a codicological point of view, we decided to implement the controlled vocabulary SegmOnto (Gabay, Camps, Pinche, & Jahan, 2021). SegmOnto was born out of the need for a small/restricted common ontology based on existing standards for the description and analysis of document layout, ranging from content categorization to text recognition, mainly addressing the case of manuscripts and early printed books.

SegmOnto has already been implemented in several projects led by Pinche and connected to the CREMMALab project such as Gabay et al. (2022), resulting in segmentation models mainly for late medieval manuscripts and early prints.<fnref n="7" /> As per the CREMMA Medii Aevi dataset, the documents present two kinds of layout: multi-columns and singular columns, for which lines are most often long, except for the Psalms and Book of Hours. SegmOnto offers multiple levels of description, of which only the first is completely standardized, as the second is intended for custom refinement and the third for local and document-based differentiation. For the purposes of the project, only the first level of SegmOnto has been utilized, such as MainZone for columns and MargintextZone for marginalia.

**Pinche's Transcription Guidelines** Pinche (2022c) stressed that HTR was an answer to the need for scientific projects to acquire textual data either to undertake editions or to constitute large corpora. Her guidelines address the need to establish principles common to projects dealing with the transcription of manuscripts in order to:

* build shareable, reusable, and durable ground truth data sets;

* produce robust generic models, reusable in “out-of-domain” manuscripts;

* minimize the collective cost, including that of training people;

* build GT that seeks to optimize the learning space of HTR models.

Pinche has privileged a graphemic transcription, which reproduces graphemes, i.e. a canonical form for each character, instead of a graphetic one, which tries to reproduce each variation of a letter (such as ʃ and s).<fnref n="8" /> Pushing the imitation too far through a graphetic approach induces a risk of making the transcription harder to complete (as it requires technical skills to recognize differentiated shapes of characters), harder to make uniform (specifically as more annotators are to participate in a dataset) and potentially unusable for HTR (as it might introduce more characters and ultimately noise for HTR engine to learn). Therefore, in cases where functional signs have more than one graphetic manifestation but essentially the same function, they could be represented by the same sign: for example, for every manifestation of the paragraph sign, we opt for the pilcrow sign “¶” (U+00B6) on every occasion, instead of several variations such as “∫” (U+F1E1).<fnref n="9" /> In the context of the guidelines, we set up a list of allowed characters and a list of common and rare cases (such as Table 2 and 3).

On the topic of abbreviations, resolving them produces specific difficulties for HTR engines, as it leads them to learn more about the language than they originally intended.<fnref n="10" /> Abbreviations

<footnote n="7">⁷ More information and case studies can be found here: https://segmonto.github.io/.</footnote>

<footnote n="8">⁸ See Pinche, Duval, and Camps (2022). On this particular topic, Gueville and Wrisley (2022) and Pinche took different paths. However, while graphemic transcription cannot be turned and reused for graphetic model training, graphetic transcriptions can be turned easily into graphemic GT, at the cost of establishing a “translation” table for each character.</footnote>

<footnote n="9">⁹ While most of the special characters exist as such in MUFI a conscious effort has been made to avoid as much as possible the private domain of MUFI.</footnote>

<footnote n="10">¹⁰ Most HTR engines learn directly from transcriptions and do not include a separate mechanism for abbreviation resolution or spotting. Transcriptions produced by these models are thus not showing where an abbreviation was resolved, making it difficult to distinguish HTR errors from abbreviation resolution errors. The stakes do not specifically concern the scores, which seem to be close to each other (Camps, Vidal-Gorène, & Vernet, 2021), but the long-term use of ground truth data and silver data in a sustainable way.</footnote>
are not resolved in our dataset, as this constitutes rather an interpretative act linked to the specificity of each document. It is not the same as a textual prediction and it could prove to be detrimental to the extension of an HTR model in the long term. Pinche's graphemic approach without abbreviation resolution simplifies the interpretation step of the text, and in turn, the reduction of characters diversity ultimately smooths both the human transcriber and the HTR engine's learning curves.

In order to ensure the rigorous application of these guidelines and the homogeneity of the data produced, we introduced quality control to the production and publication workflow. Each manuscript transcription was passed through ChocoMufin (Clérice & Pinche, 2021), using project-provided character translation and control tables.

This software, alongside these tables, allows for each dataset to be both controlled at the character level and adapted to guideline specifications and modifications. It also allows for project-specific transcription guidelines to be translated to a more common one such as CREMMALab's (Pinche & Camps, 2022).<fnref n="11" /> This process has been largely used in the first months of the CREMMA Medieval project, as the guidelines were still being drafted. It allowed Pinche to produce or align datasets first and harmonize later, as long as the harmonization was from an upper level of details (closer to graphetic) to a lower level (closer to graphemic).

## 3.2 TRANSCRIPTION GUIDELINES FOR THE CREMMA MEDII AEVI

The section that follows aims to guide the reader through the transcription norms followed for the *Medii Aevi* dataset, illustrating the process and the more common and complex cases, especially where new characters have been introduced compared to the *CREMMA Medieval* dataset.

The project adheres to the general principles laid out by Pinche (Pinche, 2022c, Tables pp. 4–15) concerning the base cases (punctuation, word separation, functional signs, superscript letters, abbreviations, ligatures, and roman numerals). Using the project-provided character conversion table, ChocoMufin controls the transcription and corrects any anticipated error by transforming the character automatically so it conforms to the pre-defined guidelines (data should be used in their post-ChocoMufin converted state as it sometimes corrected mistranscription). However, where the guidelines were not directly addressing the situation (new characters, new types of abbreviations), we positioned ourselves and interpreted the guidelines in light of the situation. Each decision was discussed with the original guidelines’ author.<fnref n="12" />

In general, the main differences that we isolated between the CREMMA Medieval and Medii Aevi datasets, stemming from the language as well as the genre's own characteristics, are:

1. the dataset bears no accentuated vowels like in the Old French texts (a rare event though for the corpus);<fnref n="13" />

2. no normalization or distinction of u and v was provided, nor of i and j;

3. two variations of con are found, namely the antisigma and the 9-shaped form;

4. a higher diversity of abbreviating character usage and signification;

5. Arabic numerals alongside roman, mostly in scholastic and medical treatises.

**Reference marks, functional signs, and punctuation** In general, complex medieval punctuation has been simplified as much as possible, with single sign punctuation being reduced to “.” and commas will be rendered as “,”. Double sign punctuation (mainly *punctus elevatus* and *punctus interrogativus*) are consistently reduced to “:”. The hyphenation for words that continue to the next line has been marked with a unique “-” (U+002D) sign, following 3.1. Table 2 gives a representative example.

<footnote n="11">11 In order to read the translation table, MUFI-compatible fonts are recommended, such as Junicode.</footnote>

<footnote n="12">12 This includes discussion with other projects, such as Gervers, Manton, Boutreux, and Elema (2018), which led to the inclusion of the stricken-through D (see Table 3).</footnote>

<footnote n="13">13 This is different from i pointing, which is not taken into account by either corpus.</footnote>
all variations of it, cf. below) is transcribed by ʾ. Etiam can also be found abbreviated by a combination of the Tironian et and the macron symbol (see Table 4).

<table><thead><tr><th>CHARACTER(S)</th><th>UNICODE</th><th>RESOLUTION</th><th>EXAMPLES</th></tr></thead><tbody><tr><td>ʾ</td><td>U+204A</td><td>Et</td><td></td></tr><tr><td>ʾ+ɔ</td><td>U+204A + U+0303</td><td>Etiam</td><td></td></tr><tr><td>פ</td><td>U+A76D</td><td>-is</td><td></td></tr><tr><td>ד</td><td>U+0111</td><td>d + any desinence<br>truncation</td><td></td></tr><tr><td>9</td><td>U+A76F</td><td>con</td><td></td></tr><tr><td>≈</td><td>U+2248</td><td>esse</td><td></td></tr><tr><td>÷</td><td>U+00F7</td><td>est/id est</td><td></td></tr><tr><td>;</td><td>U+F1AC</td><td>-que/-bus/-m/-et</td><td></td></tr><tr><td>ṭ</td><td>U+A775</td><td>-rum</td><td></td></tr></tbody></table>

Clérice et al.
Journal of Open
Humanities Data
DOI: 10.5334/johd.97

<caption>**Table 3** Freestanding, letter-combining abbreviations and their corresponding transcription signs. *d* cannot be found in our dataset and is mentioned here as it might be a common case in other datasets.</caption>

**Ligatures**, ie. combinations of more than two letters in one form with the reduction of proclitic and enclitic letters or abbreviating symbols placed above or joined with letters are reduced to their original alphabetical components. Ligatures between letters in cursive scripts such as the ſt (U+FB05) ligature or the two ſf (U+FB00) ligature are resolved as -st- and -ff-. For the very frequent *quia*, the transcription *qr* has been privileged, avoiding the MUFI sign ḳ that belongs to the private domain. More examples are provided in Table 4.<fnref n="15" />

**Superscripts letters and interlinear additions** A standard way of contracting a word is by adding a superscript letter which gives information about the abbreviated sequence. Frequent ones are open a, u, o, or the ending of a word altogether. These were all rendered with the aid of superscript characters (Pinche, 2022c, p. 11). Ergo and *igitur* are two of the most frequent examples of abbreviations with superscript letters. Letters without any baseline letter are simply represented with the same combining superscript character and space as the supporting baseline character (e.g. “a¹”: space + combining a + space + combining t cf. Figure 1).

Superscript letters, alongside abbreviating functions, were sometimes used to render interlinear additions. Missing content or annotations are added in the interlinear space, especially in manuscripts of scholastic and medical content. This was something that was at first a challenge for the transcription process due to segmentation constraints. It can be, at times, impossible to completely differentiate the segmentation masks of two vertically adjacent letters (like the interlinear additions). Therefore, provided that the corresponding combining letter exists and both words can be formulated, no new lines were carved for the interlinear additions. Where this was deemed too complex, interlinear additions were omitted (see Figure 2).

<footnote n="15">15 Other transcription guidelines privilege “q2” as a reference to the “r rotunda-shaped” abbreviation sign that lays next to q the choice of *qr* from our part being the reduction to the r rotunda-shaped abbreviation sign to the simpler r. The original insular abbreviation has a simple vertical tilde next to the letter “q”.</footnote>
<table><thead><tr><th>TYPE</th><th>TRANSCRIPTION</th><th>UNICODE</th><th>DESCRIPTION OR RESOLUTION</th><th>EXAMPLES</th></tr></thead><tbody><tr><td>Ligature</td><td>st</td><td>-</td><td>Normally transcribed ligature</td><td></td></tr><tr><td>Ligature</td><td>.n.</td><td>-</td><td>enim</td><td></td></tr><tr><td>Ligature</td><td>qr</td><td>-</td><td>quia</td><td></td></tr><tr><td>Monogrammatic Ligature</td><td>qd</td><td>-</td><td>quod</td><td></td></tr><tr><td>Monogrammatic Ligature</td><td>Et</td><td>-</td><td>Et</td><td></td></tr><tr><td>Contraction</td><td>aŭl</td><td>-</td><td>Long vertical tilde transcribed by two tildes</td><td></td></tr><tr><td>Contraction</td><td>ěě</td><td>-</td><td>Long vertical tilde transcribed by two tildes;</td><td></td></tr><tr><td>Contraction</td><td>tþa</td><td>-</td><td>Two stacked tildes</td><td></td></tr></tbody></table>

<figure><img src="image_7.png" /></figure>

<figure><img src="image_8.png" /></figure>

<figure><img src="image_9.png" /></figure>

<figure><img src="image_10.png" /></figure>

<figure><img src="image_11.png" /></figure>

<figure><img src="image_12.png" /></figure>

<figure><img src="image_13.png" /></figure>

<figure><img src="image_14.png" /></figure>

<figure><img src="image_15.png" /></figure>

<figure><img src="image_16.png" /></figure>

<figure><img src="image_17.png" /></figure>

<figure><img src="image_18.png" /><figcaption>**Table 4** Ligatures and special contraction cases.</figcaption></figure>

**Figure 1** Examples of contraction use of superscript letters. Manuscripts in the following order: BIS 193, CML 13027, Montpelier H-318, Montpelier H-318, Vat. Pal. lat.373, BIS 193.

<caption>**Figure 2** All examples come from the CML 13027 manuscript.</caption>

**Rare characters and Numerals** Referring to corpus construction practices for balanced corpora, Maniaci (1993) stresses that “sporadically attested variables will therefore be preferred to those that appear in all – or almost all – the individuals that are part of the corpus.” Rare characters, a subset of freestanding abbreviation signs, specifically occurring in the *Medii Aevi* dataset are therefore given special attention (cf. Table 5). In two of the manuscripts, both of medical content, some occurrences of graphemes for the denotation of the metric values ounce and semuncia were encountered. For their transcription, 3 (U+2125) and Σ (U+10192) were used. “Barred O” is represented by Ø (U+2205) and is widely used to transcribe the word instans instead of ϵ (U+A74B) that, according to MUFI documentation stands for the abbreviation of obi(i)t (Coulson & Babcock, 2020, p. 10).

Last but not least, in addition to roman numerals, often preceded and followed by dots such as “.ii.”, Arabic numerals are also comprised in the dataset, mainly due to the medical treatises (see Figures 3 and 4).
<table><thead><tr><th>TYPE</th><th>TRANSCRIPTION</th><th>UNICODE</th><th>DESCRIPTION OR RESOLUTION</th><th>EXAMPLES</th></tr></thead><tbody><tr><td>Symbols</td><td>§</td><td>U+2125</td><td>Ounce</td><td></td></tr><tr><td>Symbols</td><td>£</td><td>U+10192</td><td>*Semi-Ounce</td><td></td></tr><tr><td>Abbreviations</td><td>Ø</td><td>U+2205</td><td>instans</td><td></td></tr></tbody></table>

<figure><img src="image_19.png" /></figure>

<figure><img src="image_20.png" /></figure>

<figure><img src="image_21.png" /></figure>

<figure><img src="image_22.png" /><figcaption>**Table 5** Rare characters found in Montpellier H318, Phil., Col. of Phys. 10a 135 and BIS 193.</figcaption></figure>

<figure><img src="image_23.png" /><figcaption>**Figure 3** Manuscripts in the following order: Latin 16195, Phi. 10 a. 135 (x3), BIS 193, CML13027, Egerton 821, Latin 6395.</figcaption></figure>

<figure><img src="image_24.png" /><figcaption>**Figure 4** Snippet of Arabic numerals from BnF, lat.15461, fol.13r for comparison purposes.</figcaption></figure>

**Production pipeline** The data was built using eScriptorium and Kraken for both segmentation of zones and lines (specifically the BLLA model). Manuscripts were annotated successively. First, the manuscript is automatically segmented, then its segmentation is manually corrected, and finally the text is transcribed. Once each sample is entirely annotated, its use of characters is controlled via the ChocoMufin software, while its conformity to the segmentation classification vocabulary is controlled by HTRVX. Finally, data are released on Github.<fnref n="16" /> All the combining and abbreviation signs suggested for use by the present adaptation of Pinche's guidelines can be also found on a custom-made eScriptorium keyboard configuration, in order to facilitate reuse and compatibility with the guidelines.<fnref n="17" />

## 4 RESULTS AND DISCUSSION

**Properties of the resulting dataset** The resulting version of the dataset (see Table 6) is built on 18 + 3 manuscripts. All alignments are original alignments, but some draw their original transcription from online projects (cf. Acknowledgements).

The current version of the dataset shows a wide variety of genres, and thus a wide vocabulary. From medical and grammatical content to literary and scholastic, a certain level of arbitrariness is introduced in the sequence of characters as they are not as repetitive and predictable from the machine as in a homogeneous genre or topic-driven dataset. The collection was built not to be representative of one specific use of the Latin language and is not thematically unified, while the CREMMA Medieval dataset focuses more on literary texts, specifically hagiographic and chanson de geste texts. Medical and scholastic genres, furthermore, induce the use of a range of rare characters and often underrepresented letters (such as “z”, “y” and “k”).

<footnote n="16">16 https://github.com/htr-united/cremma-medieval-lat.</footnote>

<footnote n="17">17 Available here: https://github.com/HTR-United/CREMMA-Medieval-LAT/blob/main/keyboard.json.</footnote>
Other features, such as layout and type of digitization (microfilm or original), provide different representations of texts, with more or less noise in the mask of each line given the space between them, with more or less contrast between information. Colored text yields less “information” in digitized manuscripts as they tend to be a duller form of grey that black ink, while clearly departing from the manuscript “background” in color.

A timespan of 5 centuries separates the earliest and the oldest manuscripts, with a clear focus on the period starting in the 1200s and finishing in 1500. This leads to a good representation of a variety of Gothic scripts,<fnref n="18" /> including personal hands alongside formal categories such as the one described by Rossi (2022), with different levels of execution (cursivity and formality).

<table><thead><tr><th>SHELFMARK ID</th><th>PAGES</th><th>TYPE</th><th>DATE</th><th>STATUS</th><th>SCRIPT</th><th>FOLIO SAMPLING</th><th>DEGREE OF ABBREVIATIONS</th></tr></thead><tbody><tr><td>Egerton 821</td><td>4</td><td>Medic.</td><td>1100-1199</td><td>Color</td><td>Praegothica</td><td>Sequential</td><td>medium</td></tr><tr><td>Montpellier H318</td><td>5</td><td>Medic.</td><td>1100-1299</td><td>Color</td><td>Semitextualis Libraria</td><td>Sequential</td><td>high</td></tr><tr><td>CCCC MSS 236</td><td>5</td><td>Lit.</td><td>1200-1225</td><td>Color</td><td>Textualis Libraria</td><td>Sequential</td><td>medium</td></tr><tr><td>CLM 13027</td><td>5</td><td>Medic.</td><td>1250-1299</td><td>Color</td><td>Southern Textualis Libraria</td><td>Sequential</td><td>high</td></tr><tr><td>Latin 16195</td><td>4</td><td>Medic.</td><td>1250-1299</td><td>Microfilm</td><td>Semitextualis Currens</td><td>Sequential</td><td>high</td></tr><tr><td>† MsWettF 15</td><td>5</td><td>Schol.</td><td>1270-1280</td><td>Color</td><td>Textualis Libraria</td><td>Sequential</td><td>high</td></tr><tr><td>Laur. Plut. 33.31</td><td>5</td><td>Lit.</td><td>1300-1310</td><td>Color</td><td>Textualis Meridionalis</td><td>Sequential</td><td>low</td></tr><tr><td>Arras 861</td><td>5</td><td>Lit.</td><td>1300-1399</td><td>Color</td><td>Textualis Formata</td><td>Sequential</td><td>medium</td></tr><tr><td>† BIS 193</td><td>5</td><td>Schol.</td><td>1300-1399</td><td>Color</td><td>Textualis currens</td><td>Sequential</td><td>high</td></tr><tr><td>Phil., Col. of Phys. 10a 135</td><td>5</td><td>Medic.</td><td>1300-1399</td><td>Color</td><td>Cursiva recentior</td><td>Sequential</td><td>medium</td></tr><tr><td>† Mazarine Ms. 915</td><td>4</td><td>Schol.</td><td>1300-1399</td><td>Color</td><td>Textualis Meridionalis</td><td>Sequential</td><td>high</td></tr><tr><td>‡ UBL, Ms 758</td><td>15</td><td>Eccl.</td><td>1320-1340</td><td>Color</td><td>Textualis Libraria</td><td>Semi-Sequential</td><td>low</td></tr><tr><td>Latin 6395</td><td>6</td><td>Lit.</td><td>1325-1399</td><td>Microfilm</td><td>Semitextualis Libraria</td><td>Sequential</td><td>low</td></tr><tr><td>Laur. Plut. 39.34</td><td>5</td><td>Lit.</td><td>1400-1499</td><td>Color</td><td>Humanistica Cursiva</td><td>Sequential</td><td>low</td></tr><tr><td>† Vat. Pal. Lat. 373</td><td>4</td><td>Schol.</td><td>1400-1499</td><td>Microfilm</td><td>Hybrida Currens</td><td>Sequential</td><td>low</td></tr><tr><td>Laur. Plut. 53.08</td><td>4</td><td>Gramm.</td><td>1459</td><td>Color</td><td>Personal Humanistica</td><td>Sequential</td><td>medium</td></tr><tr><td>Laur. Plut. 53.09</td><td>4</td><td>Gramm.</td><td>1400-1499</td><td>Color</td><td>Humanistica Rotunda</td><td>Sequential</td><td>low</td></tr><tr><td>‡ Berlin, Hdschr. 25</td><td>17</td><td>Eccl.</td><td>1400-1499</td><td>Color</td><td>Textualis Formata</td><td>Semi-Sequential</td><td>low</td></tr><tr><td>‡ Berlin, Germ. Oct. 511</td><td>6</td><td>Eccl.</td><td>1400-1499</td><td>Color</td><td>Hybrida formata</td><td>Semi-Sequential</td><td>low</td></tr><tr><td>Latin 8236</td><td>5</td><td>Lit.</td><td>1471-1499</td><td>Microfilm</td><td>Humanistica Cursiva</td><td>Random</td><td>low</td></tr><tr><td>† CCCC MSS 165</td><td>5</td><td>Schol.</td><td>1500-1599</td><td>Color</td><td>Personal Cursive</td><td>Sequential</td><td>medium</td></tr></tbody></table>

<caption>**Table 6** Basic features and length of the dataset in chronological order. Medic. stands for medical, Lit. for literature, Schol. for scholastic commentaries, Gramm. for grammatical commentaries, Eccl. for church literature (book of hours, psalms, etc.). Texts preceded by a ‡ are aligned and corrected using the Berlin Transcribathon dataset, by a † using the SCTA TEI editions. The complete metadata table can be found in the more detailed data-registry.csv of the dataset.</caption>

<footnote n="18">18 Characterisation of scripts was made by the transcriber where the information was not available on the notice of the manuscript. The criteria followed for the Gothic scripts are those of Derolez (2003).</footnote>
**Character frequencies in the CREMMA Medieval and the Medii Aevi datasets** We set up this corpus to both complement the CREMMA Medieval dataset and grow the available set of data for Latin through the Middle Ages, noting that at least two datasets for Medieval Latin existed already (Caroline Minuscule and Eutyches) in abbreviated form for pre-10th century documents.

Unlike *CREMMA Medieval*, our approach has been feature-driven to compensate for rare characters in the dataset network. In this regard, we succeeded, as we have a higher frequency of special characters in our dataset than in Pinche's dataset, despite being smaller overall (see Table 7 and Figure 5). Only three characters are more represented in *CREMMA Medieval*: the Tironian Et, the superscript combining R (common on words such as “grand”), and “&”. The character 9 is equally present in both datasets: resolved as con- or com-, it is often used in words such as 9mence (*commence*). Some very frequent diacritics, such as the horizontal lines and vertical lines transcribed as tildes, are more frequent in our dataset, by a factor of 2.51 for horizontal ones and of 3.93 for vertical ones. This will allow better recognition of these two frequent marks, as it now totals around 19,000 occurrences in both datasets for the horizontal tilde and 4,500 for the vertical one, making them the first and the third most represented abbreviating characters.

<table><thead><tr><th>LANG</th><th>TYPE</th><th>WORDS</th><th>WORDS %</th><th>UNIQUE WORDS</th><th>UNIQUE WORDS %</th><th>FREQ. OF UNIQUE WORDS > 1</th></tr></thead><tbody><tr><td>Latin</td><td>abbr.</td><td>6,855</td><td>11.94%</td><td>1,460</td><td>6.24%</td><td>279</td></tr><tr><td>Latin</td><td>others</td><td>50,557</td><td>88.06%</td><td>21,935</td><td>93.76%</td><td>5,025</td></tr><tr><td>Old French</td><td>abbr.</td><td>5,755</td><td>4.15%</td><td>1,457</td><td>4.89%</td><td>286</td></tr><tr><td>Old French</td><td>others</td><td>132,828</td><td>95.85%</td><td>28,315</td><td>95.11%</td><td>8,726</td></tr></tbody></table>

<table><thead><tr><th>CHARACTER</th><th>UNICODE</th><th>LATIN</th><th>OLD FRENCH</th><th>% IN LATIN</th><th>RATIO</th></tr></thead><tbody><tr><td>J</td><td>U+204A</td><td>2228.0</td><td>4400.0</td><td>33.61</td><td>0.51</td></tr><tr><td>r</td><td>U+036C</td><td>148.0</td><td>219.0</td><td>40.33</td><td>0.68</td></tr><tr><td>&amp;</td><td>U+0026</td><td>83.0</td><td>116.0</td><td>41.71</td><td>0.72</td></tr><tr><td>9</td><td>U+A76F</td><td>850.0</td><td>779.0</td><td>52.18</td><td>1.09</td></tr><tr><td>p</td><td>U+A751</td><td>1500.0</td><td>919.0</td><td>62.01</td><td>1.63</td></tr><tr><td>ı</td><td>U+0365</td><td>1486.0</td><td>820.0</td><td>64.44</td><td>1.81</td></tr><tr><td>-</td><td>U+0303</td><td>14445.0</td><td>5759.0</td><td>71.50</td><td>2.51</td></tr><tr><td>a</td><td>U+0363</td><td>2024.0</td><td>732.0</td><td>73.44</td><td>2.77</td></tr><tr><td>9</td><td>U+A770</td><td>1763.0</td><td>523.0</td><td>77.12</td><td>3.37</td></tr><tr><td>ı</td><td>U+033E</td><td>3827.0</td><td>973.0</td><td>79.73</td><td>3.93</td></tr><tr><td>é</td><td>U+0364</td><td>518.0</td><td>120.0</td><td>81.19</td><td>4.32</td></tr><tr><td>p</td><td>U+A753</td><td>462.0</td><td>80.0</td><td>85.24</td><td>5.78</td></tr><tr><td>ü</td><td>U+1DD1</td><td>1018.0</td><td>137.0</td><td>88.14</td><td>7.43</td></tr><tr><td>å</td><td>U+1DE4</td><td>978.0</td><td>55.0</td><td>94.68</td><td>17.78</td></tr><tr><td>°</td><td>U+0366</td><td>870.0</td><td>61.0</td><td>93.45</td><td>14.26</td></tr></tbody></table>

Some manuscripts have nearly no abbreviation (cf. Table 9). Laur. Plut. 39.34 notably so, as it only contains 3 abbreviated words which is a single character abbreviation (j, et). A little less than half of our manuscripts are less abbreviated than the most abbreviated text in the *CREMMA Medieval* dataset, while the other half can exceed it by up to ten points. However, both languages show similar maximum frequencies in terms of non-single letter abbreviations (abbreviations made up of a single Unicode codepoint such as j, &, p).<fnref n="19" />

<caption>**Table 7** Comparative statistics table on abbreviations: for each dataset, we look at words that are abbreviated (abbr.) or non-abbreviated (others). It reads the following way: “11.94% of words in the Latin corpus are abbreviated.”</caption>

<caption>**Table 8** Abbreviating signs, present more than 50 times in both the Latin and the Old French CREMMA datasets. The CREMMA Medieval (Old French) dataset is comprised of 693,052 characters in total, which makes it more than twice the size of CREMMA Medii Aevi. Despite this difference, most abbreviated characters are more represented in the Latin dataset.</caption>

<footnote n="19">19 This definition, while useful to quantify some phenomenon, is debatable and should not be used to make a quantitative conclusion on these languages, they merely inform us about our dataset. For example, *etiam* (j + tilde) is technically a single letter with a diacritic, but will be counted as two characters in our case.</footnote>
Finally, despite showing a similar number of *pages*, we see a large variation in terms of word density with a limited variation in terms of unique words (cf. Table 8). This shows how pages as a metric are not enough to characterize a corpus for HTR and Layout segmentation purposes: the number of columns, lines, and potentiality of words or characters supplements the first. To showcase this argument, the Berlin, Hdschr. 25 manuscript has the highest number of pages (17) but the third lowest amount of words (961).

<table><caption>**Table 9** Statistics per manuscript. “Un.” stands for Unique, “Abbr.” for Abbreviated or Abbreviation, “NSCA” for Non-Single Character Abbreviation. The lowest and highest values are in bold typeface. The separation between Laur. Plut. 53.08 and UBLMs. 758 represents the highest abbreviation ratio in the CREMMA Medieval dataset.</caption><thead><tr><th>MANUSCRIPT</th><th>WORDS</th><th>UN. WORDS</th><th>ABBR. WORDS</th><th>ABBR. RATIO</th><th>NSCA</th><th>NSCA RATIO</th><th>UN. ABBR.</th><th>UN. ABBR. RATIO</th></tr></thead><tbody><tr><td>Laur. Plut. 39.34</td><td>783</td><td>571</td><td>3</td><td>0.38%</td><td>0</td><td>0.00%</td><td>1</td><td>0.18%</td></tr><tr><td>Berlin, Germ. Oct. 511</td><td><strong>171</strong></td><td>134</td><td>1</td><td>0.58%</td><td>0</td><td>0.00%</td><td>1</td><td>0.75%</td></tr><tr><td>Berlin, Hdschr. 25</td><td>961</td><td>654</td><td>12</td><td>1.25%</td><td>3</td><td>0.31%</td><td>6</td><td>0.92%</td></tr><tr><td>Latin 8236</td><td>1475</td><td>1057</td><td>33</td><td>2.24%</td><td>5</td><td>0.34%</td><td>6</td><td>0.57%</td></tr><tr><td>Laur. Plut. 33.31</td><td>1278</td><td>858</td><td>36</td><td>2.82%</td><td>17</td><td>1.33%</td><td>21</td><td>2.45%</td></tr><tr><td>Laur. Plut. 53.09</td><td>1300</td><td>798</td><td>38</td><td>2.92%</td><td>10</td><td>0.77%</td><td>9</td><td>1.13%</td></tr><tr><td>CCCC MSS 165</td><td>1521</td><td>713</td><td>49</td><td>3.22%</td><td>28</td><td>1.84%</td><td>23</td><td>3.23%</td></tr><tr><td>CCCC MSS 236</td><td>1239</td><td>874</td><td>68</td><td>5.49%</td><td>44</td><td>3.55%</td><td>24</td><td>2.75%</td></tr><tr><td>Latin 6395</td><td>3304</td><td>2418</td><td>190</td><td>5.75%</td><td>85</td><td>2.57%</td><td>72</td><td>2.98%</td></tr><tr><td>Laur. Plut. 53.08</td><td>2985</td><td>1870</td><td>195</td><td>6.53%</td><td>94</td><td>3.15%</td><td>67</td><td>3.58%</td></tr><tr><td>UBL, Ms. 758</td><td>4468</td><td>2393</td><td>297</td><td>6.65%</td><td>72</td><td>1.61%</td><td>64</td><td>2.67%</td></tr><tr><td>Arras 861</td><td>2416</td><td>1601</td><td>164</td><td>6.79%</td><td>101</td><td>4.18%</td><td>80</td><td>5.00%</td></tr><tr><td>Egerton 821</td><td>981</td><td>677</td><td>71</td><td>7.24%</td><td>28</td><td>2.85%</td><td>31</td><td>4.58%</td></tr><tr><td>Phil., Col. of Phys. 10a 135</td><td>1487</td><td>1057</td><td>151</td><td>10.15%</td><td>52</td><td>3.50%</td><td>44</td><td>4.16%</td></tr><tr><td>Montpellier H318</td><td>4456</td><td>2316</td><td>458</td><td>10.28%</td><td>131</td><td>2.94%</td><td>109</td><td>4.71%</td></tr><tr><td>Vat. Pal. Lat. 373</td><td>2258</td><td>1203</td><td>234</td><td>10.36%</td><td>69</td><td>3.06%</td><td>67</td><td>5.57%</td></tr><tr><td>Latin 16195</td><td>4135</td><td>1676</td><td>569</td><td>13.76%</td><td>168</td><td>4.06%</td><td>107</td><td>6.38%</td></tr><tr><td>MsWettF 15</td><td>3574</td><td>1452</td><td>501</td><td>14.02%</td><td>172</td><td>4.81%</td><td>107</td><td>7.37%</td></tr><tr><td>CLM 13027</td><td>6499</td><td>3612</td><td>970</td><td>14.93%</td><td>340</td><td>5.23%</td><td>257</td><td>7.12%</td></tr><tr><td>BIS 193</td><td><strong>7370</strong></td><td>2731</td><td>1161</td><td>15.75%</td><td>413</td><td>5.60%</td><td>244</td><td>8.93%</td></tr><tr><td>Mazarine Ms. 915</td><td>4751</td><td>1873</td><td>824</td><td>17.34%</td><td>350</td><td>7.37%</td><td>195</td><td>10.41%</td></tr></tbody></table>

<figure><img src="image_25.png" /><figcaption>**Figure 5** Frequencies of character classes across manuscripts.</figcaption></figure>
## 5 IMPLICATIONS/APPLICATIONS

With this addition to the overall amount of datasets available, we now have 1.149 million characters for medieval manuscripts with book scripts, ranging from the 9th to the 15th century. These data offer more than characters: we can imagine using them in the context of linguistic studies (evolution of dialects, abbreviation usage, etc.) thanks to the shared transcription norm, or in codicology studies (evolution of layouts, relation between layouts) using the common segmentation vocabulary, both using the original data or automatically annotated one.

HTR data and models have a fairly high level of reuse potential. First and foremost, while it is still relatively a rare reuse, these data, visualised correctly, can easily serve as teaching materials: e-teaching of paleography has been gaining some traction,<fnref n="20" /> but simply moving away from printed to digital and interactive hand-outs using open data and transcription is a first step that undoubtedly some have already taken.<fnref n="21" /> Then reuse can move to the analysis of transcription themselves: Stutzmann (2018) and Stutzmann, Mariotti, and Ceresato (2020) have shown that analysis of graphematic data can yield information about scribal practices. Finally, such data can be used for model training. Project like Possamaï, Gaiffre, Souvaye, Duval, and Ducos (2022) and Foehr-Janssens, Ventura, Carnaille, and Meylan (2021) have used automatic transcription models to speed-up the transcription process of large collection of manuscripts, using base models which were then fine-tuned on sample of data to yield better results, such as described by Pinche (2022b, 4.4). Finally, models can be used for data-mining and operating research at scale on non-manually transcribed manuscripts: Camps, Clérice, and Pinche (2021) proved the hypothesis of a 19th-century scholar by analysing a full manuscript with automatic transcription, Franzini et al. (2018) proposed also a stylometrical analysis of data obtained through automatic transcription.

As a direct output, we trained a model which would allow for transcribing or starting the transcription of Latin medieval manuscripts. In order to evaluate the gain from our data, we trained three models:<fnref n="22" />

1. a model containing all data from Table 1, to help transcribe Latin and Medieval French manuscripts, which is the end goal of this paper;

2. a model containing every dataset but our own, to evaluate the impact regarding the quantity of data we add for Latin (i.e., to find out if the original Carolingian datasets were enough to break the language model of the Old French datasets);

3. a model containing only Old French data, from incunabula of the 15th century to the main dataset CREMMA Medieval.

From Medii Aevi, as stated earlier, all aligned data from the *Faithful Transcription Data Set* are kept for testing, as an out-of-domain set. Each model uses at least 10% of the pages of each dataset for the development set. CREMMA Medieval and Medii Aevi are split furthermore with another 10% subset for evaluation, proposing “in Domain” evaluation.

<table><thead><tr><th>MODEL</th><th>MEDIEVAL OLD FRENCH (IN DOMAIN)</th><th>MEDIEVAL LATIN (IN DOMAIN)</th><th>UBL</th><th>BGO</th><th>BH25</th></tr></thead><tbody><tr><td>All</td><td>94.30</td><td>90.15</td><td>71.69</td><td>79.12</td><td>85.10</td></tr><tr><td>No CREMMA Medii Aevi</td><td>94.04</td><td>80.68</td><td>67.68</td><td>78.02</td><td>81.89</td></tr><tr><td>Only Old French</td><td>94.01</td><td>78.10</td><td>67.49</td><td>76.81</td><td>80.74</td></tr></tbody></table>

**Table 10** General accuracy results of the models. Model All contains all data presented in Table 1, model No CREMMA Medii Aevi contains everything but the present dataset, model Only Old French contains all datasets but Latin one (Eutyches, Caroline, CREMMA Medii Aevi). Two types of test sets are present: the “In Domain” dataset are pages from the same manuscripts as the models, all others (UBL 758, BGO 511, and B.H. 25) are manuscripts from the *Faithful Transcriptions Data Set* aligned in CREMMA Medii Aevi but not used for training purposes.

<footnote n="20">20 See Brookes, Stokes, Watson, and De Matos (2015), Burghart (2011) and the interactive facsimiles of http://theleme.enc.sorbonne.fr/dossiers/index.php. Ad fontes Hodel and Nadig (2019) has implemented training material in paleography also via interactive facsimiles: https://www.adfontes.uzh.ch/en/3001/training/einleitung.</footnote>

<footnote n="21">21 We know at least of Olivier Canteaut at the Ecole nationale des chartes who has been using it for this purpose.</footnote>

<footnote n="22">22 All models are trained with Kraken 4.1.2. Parameters are the base one of this version, as well as the following: NFD Unicode normalization (-u NFD, augmentation of data through albumentations (--augment), batch size 16 (-B 16), fixed splits (--fixed-splits), learning rate 0.0001 (--Irate 0.0001), model architecture [1,120,0,1 Cr3,13,32 Do0.1,2 Mp2,2 Cr3,13,32 Do0.1,2 Mp2,2 Cr3,9,64 Do0.1,2 Mp2,2 Cr3,9,64 Do0.1,2 S1(1x0)1,3 Lbx200 Do0.1,2 Lbx200 Do0.1,2 Lbx200 Do] (--spec "[...]")</footnote>
<table><caption>**Table 11** Details on errors from the test presented in Table 10. Space % shows the portion of error points due to bad spacing, e.g. All Model has a 94.30% accuracy on CREMMA Medieval test set, which means a 5.7% Character Error Rate (CER); not recognized SPACES represent 1.7 points of CER, more than a quarter of the CER. Other numbers are absolute values of missed characters (deletion or substitutions) to make comparisons between models possible; insertions are not accounted for.</caption><thead><tr><th>MODEL</th><th>TEST</th><th>[SPACE] %</th><th>[SPACE]</th><th>TILDE</th><th>VERT. TILDE</th><th>7</th><th>9</th><th>17</th><th>15</th><th>0</th><th>0</th><th>4</th><th>0</th><th>0</th></tr></thead><tbody><tr><td>All</td><td>CREMMA Medieval</td><td>1.7</td><td>803</td><td>77</td><td>46</td><td>0</td><td>10</td><td>17</td><td>15</td><td>0</td><td>0</td><td>0</td><td>4</td><td>0</td><td>0</td></tr><tr><td>No CREMMA Medii Aevi</td><td>CREMMA Medieval</td><td>1.7</td><td>726</td><td>89</td><td>55</td><td>0</td><td>15</td><td>12</td><td>15</td><td>0</td><td>0</td><td>0</td><td>3</td><td>0</td><td>0</td></tr><tr><td>Only Old French</td><td>CREMMA Medieval</td><td>1.7</td><td>733</td><td>86</td><td>50</td><td>0</td><td>12</td><td>15</td><td>20</td><td>0</td><td>0</td><td>0</td><td>2</td><td>0</td><td>0</td></tr><tr><td>All</td><td>CREMMA Medii Aevi</td><td>1.7</td><td>74</td><td>27</td><td>31</td><td>0</td><td>3</td><td>2</td><td>3</td><td>0</td><td>8</td><td>2</td><td>2</td><td>0</td><td>1</td></tr><tr><td>No CREMMA Medii Aevi</td><td>CREMMA Medii Aevi</td><td>2.8</td><td>138</td><td>78</td><td>92</td><td>0</td><td>17</td><td>8</td><td>11</td><td>1</td><td>17</td><td>15</td><td>2</td><td>15</td><td>32</td></tr><tr><td>Only Old French</td><td>CREMMA Medii Aevi</td><td>3.1</td><td>149</td><td>91</td><td>87</td><td>0</td><td>16</td><td>10</td><td>9</td><td>1</td><td>17</td><td>20</td><td>2</td><td>15</td><td>33</td></tr><tr><td>All</td><td>BGO</td><td>2.9</td><td>22</td><td>1</td><td>0</td><td>1</td><td>1</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>1</td><td>2</td></tr><tr><td>No CREMMA Medii Aevi</td><td>BGO</td><td>2.3</td><td>13</td><td>3</td><td>0</td><td>1</td><td>1</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>1</td><td>2</td></tr><tr><td>Only Old French</td><td>BGO</td><td>2.3</td><td>13</td><td>1</td><td>0</td><td>1</td><td>1</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>1</td><td>2</td></tr><tr><td>All</td><td>BH25</td><td>1.9</td><td>63</td><td>44</td><td>18</td><td>0</td><td>2</td><td>0</td><td>8</td><td>0</td><td>5</td><td>1</td><td>2</td><td>3</td><td>10</td></tr><tr><td>No CREMMA Medii Aevi</td><td>BH25</td><td>1.8</td><td>73</td><td>68</td><td>21</td><td>0</td><td>4</td><td>0</td><td>9</td><td>0</td><td>5</td><td>1</td><td>2</td><td>3</td><td>12</td></tr><tr><td>Only Old French</td><td>BH25</td><td>2.1</td><td>100</td><td>71</td><td>21</td><td>0</td><td>5</td><td>0</td><td>5</td><td>0</td><td>5</td><td>1</td><td>2</td><td>3</td><td>12</td></tr><tr><td>All</td><td>UBL</td><td>4.4</td><td>274</td><td>67</td><td>48</td><td>0</td><td>12</td><td>0</td><td>54</td><td>10</td><td>30</td><td>2</td><td>14</td><td>30</td><td>43</td></tr><tr><td>No CREMMA Medii Aevi</td><td>UBL</td><td>6.0</td><td>482</td><td>256</td><td>76</td><td>0</td><td>38</td><td>0</td><td>69</td><td>11</td><td>37</td><td>7</td><td>16</td><td>71</td><td>71</td></tr><tr><td>Only Old French</td><td>UBL</td><td>5.7</td><td>484</td><td>239</td><td>77</td><td>0</td><td>28</td><td>0</td><td>59</td><td>11</td><td>37</td><td>7</td><td>15</td><td>71</td><td>71</td></tr></tbody></table>
The results show a massive improvement for the in-domain Latin dataset (see Table 10) and an insignificant one for Old French. The addition of *Medii Aevi* provides overall better results on out-of-domain datasets: UBL Mss. 758 and Berlin, Hdschr. 25 transcriptions improved by 4.2 points at least while Berlin, Germ. Oct. 511 (BGO), the smallest transcription set of the dataset, only improved by 2.4. This improvement derives equally from the simple addition of Latin into the model, as shown by the clear gap between the mixed model with Carolingian data: not only the model might benefit from Latin in general (as potentially shown by the simple addition of the Carolingian data), but it also gains in performance out of the amount of data from the same period as *CREMMA Medieval*. We actually see in Table 11 that there are much fewer errors on characters that saw their frequencies reach new highs. The *All* model does only a fourth of the error of the *Only Old French* model on tildes or two-thirds on vertical tildes for the UBL manuscript. The -rum abbreviation (r) or the -et/-ed/-ibus one (:) are quite new to Medieval datasets in general, which explains the clear difference in results. Overall, this dataset helped create a model allowing for readable outputs (see Appendix Table 12 for a side-by-side comparison) on Medieval manuscripts, or at least transcriptions that can help produce new data.

## ADDITIONAL FILE

The additional file for this article can be found as follows:

* **Appendix Table 12.** Ground-truth (left) and prediction (right) of the new model on UBL Mss. 758, 24r. Yellow highlighting shows the differences between transcriptions. DOI: https://doi.org/10.5334/johd.97.s1

## ACKNOWLEDGEMENTS

Several transcriptions are the product of alignment and adaptation of existing projects that have worked on the manuscripts in question. In the case of existing digitized transcriptions, an alignment and correction were performed. In the case of printed editions, they served as a guide for obscure passages and *dubia*.

* For the manuscripts: MsWettF 15, BIS 193, Latin 6395, Vat. Pal. Lat. 373 and CCCC MSS 165, the transcriptions of Sentences Commentary Text Archive (SCTA) Project by Witt (2016).<fnref n="23" /> In the case of *dubia*, additional corrections have been made for the faithful reproduction of the abbreviations;

* for Berlin, Hdschr. 25, *Faithful Transcriptions Data Set* (Eichenberger & Suwelack, 2021);

* for the Donatus manuscripts – Laurentianus Pluteus 53.08 and 53.09 – the edition of HyperDonat by Bruno Bureau & Christian Nicolas has been consulted (Bureau, Nicolas, & Ingarao, 2008) and (Pinche, Bureau, & Nicolas, 2016), preserving, nevertheless, the manuscript *lectiones/errors*;

* In the same vein, for Latin 16195, the critical edition of *Questiones de coitu* (Cartelle, 2017), for Montpelier H 318 and CLM 1302, the critical edition of *Liber minor de coitu* (Cartelle, 1987) and for Philadelphia, College of Physicians, 10a 135, the critical edition of the *Tractatus de sterilitate* (Cartelle, 1993) by Enrique Montero Cartelle were consulted respectively as reference.

Tools used for verification of any *dubia* in original transcriptions:

* The online version of the Capelli: https://www.adfontes.uzh.ch/fr/ressourcen/abkuerzungen/cappelli-online

* During the deliberation regarding the use of special characters, the MUFI recommendations for Latin (last version) were respected (Wills, 2016) available here: https://folk.uib.no/hnooh/mufi/specs/MUFI-CodeChart-3-0.pdf.

<footnote n="23">23 The GitHub repository of the project can be found here: https://github.com/scta-texts and their reading room here: https://scta.lombardpress.org/</footnote>
## FUNDING STATEMENT

The project CREMMA was funded by the DIM MAP (now DIM PAMIR) under the supervision of the Conseil Régional d'Île de France. Part of the alignment of data for the *Faithful Transcription Data* Set and the complete writing time for this paper was done under the funding of the second phase of CREMMALab post-doc (Thibault Clérice). The article publication fees are provided by the Centre Jean Mabillon.

## COMPETING INTERESTS

The authors have no competing interests to declare.

## AUTHOR CONTRIBUTIONS

Thibault Clérice and Alix Chagué were responsible for the project administration. TC super- vised and curated the data of the project. Malamatenia Vlachou-Efstathiou and TC produced the data, MV-E contributing more than half of the transcription work. All contributed to the writing of this paper.

## AUTHOR AFFILIATIONS

Thibault Clérice ID orcid.org/0000-0003-1852-9204

Centre Jean Mabillon, École nationale des Chartes, PSL University, Paris, France

Malamatenia Vlachou-Efstathiou ID orcid.org/0000-0002-9397-356X

École nationale des Chartes, PSL University, Paris, France

Alix Chagué ID orcid.org/0000-0002-0136-4434

INRIA, CNRS, Paris, France

## REFERENCES

* Bauer, M. W., & Aarts, B. (2000). Corpus construction: A principle for qualitative data collection.

Qualitative researching with text, image and sound: A practical handbook, 19-37. DOI: https://doi.org/10.4135/9781849209731.n2

* Biay, S., Boby, V., Konstantinova, K., & Cappe, Z. (2022). Tnah-2021-decameronfr. Retrieved from https://github.com/PSL-Chartes-HTR-Students/TNAH-2021-DecameronFR. DOI: https://doi.org/10.5281/zenodo.6126376

* Bischoff, B. (1985). Paléographie de l'antiquité romaine et du moyen âge. Paris: Ed. Picard.

* Brookes, S., Stokes, P. A., Watson, M., & De Matos, D. M. (2015). The digipal project for european scripts and decorations. *Essays and Studies*, 68, 25-59. DOI: https://doi.org/10.1017/9781782046073.003

* Bureau, B., Nicolas, C., & Ingarao, M. (2008). Hyperdonat, commentaire attribué à aelius donat aux comédies de térence.

* Burghart, M. (Ed.). (2011). Album interactif de paléographie médiévale/interactive album of mediaeval palaeography. UMR 5648 CIHAM. Retrieved from https://paleographie.huma-num.fr/

* Camps, J.-B., Clérice, T., & Pinche, A. (2021, 11). Noisy medieval data, from digitized manuscript to stylometric analysis: Evaluating Paul Meyer's hagiographic hypothesis. *Digital Scholarship in the Humanities*, 36(Supplement 2), ii49-ii71. DOI: https://doi.org/10.1093/llc/fqab033

* Camps, J.-B., Vidal-Gorène, C., & Vernet, M. (2021). Handling heavily abbreviated manuscripts: Htr engines vs text normalisation approaches. In *International conference on document analysis and recognition* (pp. 306-316). DOI: https://doi.org/10.1007/978-3-030-86159-9_21

* Cappelli, A. (1899). Dizionario di abbreviature latine ed italiane: usate nelle carte e codici specialmente nel medio-evo ripprodotte con oltre 13000 segni incisi. Hoepli.

* Cartelle, E. M. (1987). *Liber minor de coitu: Tratado menor de andrologia. anónimo salernitano*. Universidad, Valladolid.

* Cartelle, E. M. (1993). *Tractatus de sterilitate: Anónimo de montpellier (s. xiv)*. Universidad, Valladolid.

* Cartelle, E. M. (2017). Questiones de coitu. *Cuadernos de Filología Clásica. Estudios Latinos*, 37(1), 51. DOI: https://doi.org/10.5209/CFCL.56186

* Chagué, A., & Clérice, T. (2020). Htr-united: Ground truth resources for the htr and ocr of patrimonial documents [dataset].

* Clérice, T., & Pinche, A. (2021, 9). Choco-mufin, a tool for controlling characters used in ocr and htr projects. Retrieved from https://github.com/PonteIneptique/choco-mufin. DOI: https://doi.org/10.5281/zen-odo.5356154
Coulson, F., & Babcock, R. (2020). *The oxford handbook of latin palaeography*. Oxford University Press, USA. DOI: https://doi.org/10.1093/oxfordhb/9780195336948.001.0001

Derolez, A. (2003). *The palaeography of gothic manuscript books: From the twelfth to the early sixteenth century*. Cambridge University Press.

Eichenberger, N., & Suwelack, H. (2021, October). *Faithful transcriptions data set: Tei/xml-encoded transcriptions of medieval theological manuscripts* (Tech. Rep.). DOI: https://doi.org/10.5281/zenodo.5582483

Foehr-Janssens, Y., Ventura, S., Carnaille, C., & Meylan, A. (2021). « canoniser les sept sages » (c7s): livre, langues, écriture sérielle (xiiiᵉ-xve s.). FNRS. Retrieved from https://data.snf.ch/grants/grant/197853

Franzini, G., Kestemont, M., Rotari, G., Jander, M., Ochab, J. K., Franzini, E., ... Rybicki, J. (2018). Attributing authorship in the noisy digitized correspondence of jacob and wilhelm grimm. *Frontiers in Digital Humanities*, 5, 4. DOI: https://doi.org/10.3389/fdigh.2018.00004

Gabay, S., Camps, J.-B., Pinche, A., & Jahan, C. (2021). Segmonto: common vocabulary and practices for analysing the layout of manuscripts (and more). In *16th international conference on document analysis and recognition* (icdar 2021).

Gabay, S., Pinche, A., Leroy, N., & Christensen, K. (2022). *Données htr manuscrits du 15ᵉ siècle*. HTR United. Retrieved from https://github.com/Gallicorpora/HTR-MSS-15e-Siecle

Gervers, M., Manton, A., Boutreux, A., & Elema, A. (2018). Text as image, image as text [Project]. Retrieved from https://www.utsc.utoronto.ca/research/prj/text-image-image-text-charter-integrity-and-topic-modelling (Funded by the Social Sciences and Humanities Research Council of Canada (SSHRC))

Gueville, E., & Wrisley, D. J. (2022, July). *Transcribing Medieval Manuscripts for Machine Learning*. Retrieved from https://halshs.archives-ouvertes.fr/halshs-03725166 (working paper or preprint)

Hawk, B., Karaisl, A., & White, N. (2018). Modelling medieval hands: practical ocr for caroline minuscule. *Digital Humanities Quaterly*, 13.

Hodel, T., & Nadig, M. (2019). Grundlagen der mediävistik digital vermitteln: Ad fontes, aber wie? *Das Mittelalter*, 24(1), 142–156. DOI: https://doi.org/10.1515/mial-2019-0010

Kahle, P., Colutto, S., Hackl, G., & Mühlberger, G. (2017). Transkribus-a service platform for transcription, recognition and retrieval of historical documents. In *2017 14th iapr international conference on document analysis and recognition (icdar)* (Vol. 4, pp. 19–24). DOI: https://doi.org/10.1109/ICDAR.2017.307

Kiessling, B., Tissot, R., Stokes, P., & Ezra, D. S. B. (2019). escriptorium: an open source platform for historical document analysis. In *2019 international conference on document analysis and recognition workshops* (icdarw) (Vol. 2, pp. 19–19). DOI: https://doi.org/10.1109/ICDARW.2019.10032

Maniaci, M. (1993). Che fare del proprio corpus? *Gazette du livre médiéval*, 22(1), 27–37. DOI: https://doi.org/10.3406/galim.1993.1230

Pinche, A. (2022a, 6). *Cremma medieval*. Retrieved from https://github.com/HTR-United/cremma-medieval. DOI: https://doi.org/10.5281/zenodo.5235185

Pinche, A. (2022b, November). *Generic HTR Models for Medieval Manuscripts The CREMMALab Project*. Retrieved from https://hal.archives-ouvertes.fr/hal-03837519 (working paper or preprint)

Pinche, A. (2022c, June). *Guide de transcription pour les manuscrits du Xe au XVe siècle*. Retrieved from https://hal.archives-ouvertes.fr/hal-03697382

Pinche, A., Bureau, B., & Nicolas, C. (2016). Hyperdonat, digital edition project. In *Tei conference and members' meeting 2016*.

Pinche, A., & Camps, J.-B. (2022). *Cremmalab project: Transcription guidelines and htr models for french medieval manuscripts. In Colloque" documents anciens et reconnaissance automatique des écritures manuscrites"*.

Pinche, A., Duval, F., & Camps, J.-B. (2022, March). *Création de modèle(s) HTR pour les documents médiévaux en ancien français et moyen français, Xe-XIVe siècles*. Retrieved from https://hal.archives-ouvertes.fr/hal-03615557 (working paper or preprint)

Pluta, O. (2020, 12). 9Abbreviations. In *The Oxford Handbook of Latin Palaeography*. Oxford University Press. DOI: https://doi.org/10.1093/oxfordhb/9780195336948.013.109

Possamaï, M., Gaiffre, B., Souvaye, G., Duval, F., & Ducos, J. (2022). *Liber, les décades de bersuire*. ANR. Retrieved from https://anr.fr/Projet-ANR-21-CE27-0008

Rossi, M. C. (2022). *Scritture di dotti nell'università del duecento. In Xxiie colloque de paléographie latine*. Retrieved from https://cipl.hypotheses.org/maria-cristina-rossi-univ-pisa

Ströbel, P. B., Clematide, S., & Volk, M. (2020). How much data do you need? about the creation of a ground truth for black letter and the effectiveness of neural ocr.

Stutzmann, D. (2018, March). Variability as a Key Factor For Understanding Medieval Scripts: the ORIFLAMMS project (ANR-12-CORP-0010). In S. Brookes, M. Rehbein, & P. Stokes (Eds.), *Digital Palaeography*. Routledge. Retrieved from https://shs.hal.science/halshs-01778620

Stutzmann, D., Mariotti, V., & Ceresato, F. (2020, February). Les abréviations dans les manuscrits français du XIIIᵉ siècle : analyses statistiques. In *L'emersione delle scritture volgari – L'émergence des écrits en langue vulgaire – The rise of vernacular writing La prospettiva paleografica – Le point*
de vue paléographique – The palaeographical perspective. XXI Convegno del Comité international de paléographie latine. Firenze, Italy. Retrieved from https://shs.hal.science/halshs-03560918

Vlachou-Efstathiou, M. (2022a). Voss.lat.o.41 - eutyches "de uerbo" glossed. HTR United.

Vlachou-Efstathiou, M. (2022b, 6). Voss.Lat.O.41 - Eutyches "de uerbo" glossed. Retrieved from https://github.com/malamatenia/Eutyches

White, N., Karaisl, A., & Clérice, T. (2022). *Caroline minuscule by rescribe*. HTR United. Retrieved from https://github.com/rescribe/carolineminuscule-groundtruth

Wills, T. (2016). The medieval unicode font initiative. *Medieval Unicode Font Initiative*.

Witt, J. C. (2016). Digital scholarly editions as api consuming applications: lessons and examples from the sentences commentary text archive and lombardpress. *Digital Scholarly Editions as Interfaces*, 24.

Clérice et al.
Journal of Open
Humanities Data
DOI: 10.5334/johd.97

TO CITE THIS ARTICLE:

Clérice, T., Vlachou-Efstathiou, M., & Chagué, A. (2023).
CREMMA Medii Aevi: Literary Manuscript Text Recognition in Latin. *Journal of Open Humanities Data*, 9: 4,
pp. 1–19. DOI: https://doi.org/10.5334/johd.97

Published: 12 April 2023

COPYRIGHT:

© 2023 The Author(s). This is an
open-access article distributed
under the terms of the Creative
Commons Attribution 4.0
International License (CC-BY
4.0), which permits unrestricted
use, distribution, and
reproduction in any medium,
provided the original author
and source are credited. See
http://creativecommons.org/
licenses/by/4.0/.

*Journal of Open Humanities Data* is a peer-reviewed open access journal published by Ubiquity Press.