# CATMuS-Medieval: Consistent Approaches to Transcribing ManuScripts

Ariane Pinche, Thibault Clérice, Alix Chagué, Jean-Baptiste Camps,

Malamatenia Vlachou-Efstathiou, Matthias Gille Levenson, Olivier

Brisville-Fertin, Federico Boschetti, Franz Fischer, Michael Gervers, et al.

► To cite this version:

Ariane Pinche, Thibault Clérice, Alix Chagué, Jean-Baptiste Camps, Malamatenia Vlachou-Efstathiou, et al.. CATMuS-Medieval: Consistent Approaches to Transcribing ManuScripts. Digital Humanities - DH2024, ADHO, Aug 2024, Washington DC, United States. hal-04346939

HAL Id: hal-04346939

https://inria.hal.science/hal-04346939v1

Submitted on 15 Dec 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
¹CIHAM (UMR 5648), CNRS, ²ALMAnACH, INRIA

# CATMuS-Medieval

## *Consistent Approaches to Transcribing ManuScripts*

Authors: A. Pinche, T. Clérice, A. Chagué, J-B. Camps, M. Vlachou-Efstathiou, M. Gille

Levenson, O. Brisville-Fertin, F. Boschetti, F. Fischer, M. Gervers, A. Boutreux, A.

Manton, S. Gabay, P. O'Connor, W. Haverals, M. Kestemont, C. Vandyck

Presenting speakers: Ariane Pinche¹, Thibault Clérice²
ariane.pinche@cnrs.fr, thibault.clérice@inria.fr

DH 2024: Reinvention & Responsibility, August 8, 2024
<figure><img src="image_2.png" /></figure>

① Introduction

② History of the CATMuS Project

2.1 First Initiatives

2.2 Gathering projects: CATMuS Chronology

③ Harmonizing Data: CATMuS-Medieval Transcription Guidelines

3.1 Why establish transcription guidelines?

3.2 Birth of the CATMuS-Medieval Guidelines

3.3 CATMuS-Medieval guidelines: principles

④ CATMuS-Medieval datasets and models

4.1 CATMuS-Medieval datasets

4.2 CATMuS-Medieval models

⑤ Conclusion
<figure><img src="image_3.png" /></figure>

* 2016, the OCRopy tool (Breuel 2014)- designed as an open-source, easily trainable OCR engine-piqued the interest of digital humanists

* 2016–2019, the landscape of handwritten text recognition (HTR) underwent a profound transformation

* Model training became feasible with the advent of general public GPUs

* Programming libraries like Tensorflow and PyTorch became standard

* Kraken gained prominence (Kiessling 2022)

* Transkribus emerged as a comprehensive solution, enabling manual annotation, model training, and post-correction, (Kahle, Colutto, Hackl, and Mühlberger 2017)
<figure><img src="image_4.png" /></figure>

* 2020 - ..., ATR became more and more common in research projects in humanities

* Expansion of the use of intuitive platforms: eScriptorium, (Stokes et al. 2021) and Transkribus

* Need to standardize methods and data sets to enable data gathering and production of a generic model

<figure><img src="image_5.png" /><figcaption>Figure: BnF, Latin, 8001, 13th century</figcaption></figure>

<figure><img src="image_6.png" /><figcaption>Figure: BnF, French, 777, 15th century</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>Figure: BnF, Espagnol, 533, 15th century</figcaption></figure>
<figure><img src="image_8.png" /></figure>

* **CATMuS** stands for **Consistent Approaches to Transcribing ManuScripts.**

* International initiative involving 17 collaborators: A. Pinche, T. Clérice, A. Chagué, J-B. Camps, M. Vlachou-Efstathiou, M. Gille Levenson, O. Brisville-Fertin, F. Boschetti, F. Fischer, M. Gervers, A. Boutreux, A. Manton, S. Gabay, P. O'Connor, W. Haverals, M. Kestemont, C. Vandyck

* From Europe and North America

* Provides guidelines, datasets, and models for HTR

<figure><img src="image_9.png" /></figure>
<figure><img src="image_10.png" /></figure>

① Introduction

② History of the CATMuS Project

2.1 First Initiatives

2.2 Gathering projects: CATMuS Chronology

③ Harmonizing Data: CATMuS-Medieval Transcription Guidelines

3.1 Why establish transcription guidelines?

3.2 Birth of the CATMuS-Medieval Guidelines

3.3 CATMuS-Medieval guidelines: principles

④ CATMuS-Medieval datasets and models

4.1 CATMuS-Medieval datasets

4.2 CATMuS-Medieval models

⑤ Conclusion
<figure><img src="image_11.png" /></figure>

Everything began at DH 2019, with: Jean-Baptiste Camps, Thibault Clérice, and Ariane Pinche. “Noisy medieval data, from digitized manuscript to stylometric analysis: Evaluating Paul Meyer’s hagiographic hypothesis”. In: *Digital Scholarship in the Humanities* 36.Supplement_2 (Oct. 2021), pp. ii49–ii71. ISSN: 2055-7671. URL: https://doi.org/10.1093/llc/fqab033 (visited on 12/03/2021)

<figure><img src="image_12.png" /></figure>
<figure><img src="image_13.png" /></figure>

Collective initiatives on HTR, in partnership with École nationale des chartes (ENC), DIM MAP and INRIA:

* CREMMA (Consortium Reconnaissance d'Écriture Manuscrite des Matériaux Anciens) to create HTR datasets and help set up an eScriptorium instance

* CREMMALab

* Methodological reflection on corpus transcription protocols to optimizing HTR models for medieval French Manuscripts

* Producing a dataset to train generic model for medieval French manuscripts, (Pinche 2022)

* CREMMA Medii Aevi: dataset for medieval manuscripts in Latin, (Clérice, Chagué, and Vlachou-Efstathiou 2023)
<figure><img src="image_14.png" /></figure>

① CREMMA & CREMMALab (DIM MAP, région île de France)

② Gallic(orpor)a

③ FoNDUE

④ DEEDS

⑤ HTRomance (training data production for French and Latin manuscripts, Bibliothèque nationale de France)

⑥ HTRogène (multilingual training data production, Biblissima+, ongoing)

Total expenses: €100,000 for infrastructure, postdoctoral, training data producing, not counting “hours” of team members.

<figure><img src="image_15.png" /></figure>
<figure><img src="image_16.png" /></figure>

① Introduction

② History of the CATMuS Project

2.1 First Initiatives

2.2 Gathering projects: CATMuS Chronology

③ Harmonizing Data: CATMuS-Medieval Transcription Guidelines

3.1 Why establish transcription guidelines?

3.2 Birth of the CATMuS-Medieval Guidelines

3.3 CATMuS-Medieval guidelines: principles

④ CATMuS-Medieval datasets and models

4.1 CATMuS-Medieval datasets

4.2 CATMuS-Medieval models

⑤ Conclusion
<figure><img src="image_17.png" /></figure>

# Why establishing transcription guidelines ?

> "Well-prepared material is key to producing general recognition models. It is unthinkable that single scholars and small project teams could provide enough training material to train a general model independently." (Hodel, Schoch, Schneider, and Purcell 2021)

Transcription guidelines are needed to:

* Create **consistent data**

* Build **reusable ground truth data** sets to minimize the collective cost

* Produce **generic models**

<figure><img src="image_18.png" /></figure>

**Figure:** Prediction using Transkribus Coloso Español model. Capture kindly provided by J. M. Fradejas Rueda.
<figure><img src="image_19.png" /></figure>

* The CATMuS guidelines follow on from those of the CREMMALab project, available here: https://hal.archives-ouvertes.fr/hal-03697382

* This first draft is the results of a reflection carried during the years 2021 and 2022 at the ENC.

* The aim was:

* To ensure compatibility of transcription data

* To find a medium path to ensure feasibility and consistency of the data

* To produce data reusable by historians, linguists, paleographists, philologists

* Not to constitute a definitive transcription or a final edition

<figure><img src="image_20.png" /><figcaption>Figure: Example of a conceptual text acquisition pipeline</figcaption></figure>
<figure><img src="image_21.png" /></figure>

How to transcribe manuscripts to ensure consistency in the HTR model?

* Define transcription methods suitable for machine learning.

* Determine the desired level of detail in transcription.

<figure><img src="image_22.png" /><figcaption>Figure: <de la uirtut> or <dela (12tut>.</figcaption></figure>

* Ensure compatibility of data

* Use a predefined characters set

* Document your choices

<table><tr><td>LATIN SMALL LETTER P WITH STROKE (p harre drit) p</td><td>p</td><td>U+A751</td></tr><tr><td>LATIN SMALL LETTER P WITH FLOURISH (p harre courbe) p</td><td>p</td><td>U+A753</td></tr><tr><td>LATIN SMALL LETTER Q WITH DIAGONAL STROKE (q harre) q</td><td>q</td><td>U+A759</td></tr><tr><td>TIRONIAN SIGN ET (abréviation strionomie de « et »)</td><td>7</td><td>U+204A</td></tr><tr><td>DIVISION SIGN (abréviation de « est »)</td><td>+</td><td>U+00F7</td></tr></table>
<figure><img src="image_23.png" /></figure>

Guidelines are based on basic principles:

* Reducing each letter form to a standardized representation

* Distinctions like <u>/<v> and <i>/<j> are reduced to <u> and <i>

* Keeping abbreviations (and thus, reducing the part of the language-specific traits the model has to learn)

* Restricting the characters used for specific purposes according to the MUFI

* Reducing the complexity of medieval punctuation: single sign = “.” and double sign = “:.”

Caroline
k.13b

tunc est autem quod premissa est eumque euticus est euticus qui inerpretat fortunat?

potut hêr i bona dritate; q. tibi videbß potut hêr i bona dritate; q. tibi videbß

Caroline
k.15b

Præphætus
12b.13b

tunc est autem quod premissa est eumque euticus est euticus qui inerpretat fortunat?

potut hêr i bona dritate; q. tibi videbß potut hêr i bona dritate; q. tibi videbß

Hybrida
15b.16b

Gothica Textus
13b.16b

tunc est autem quod premissa est eumque euticus est euticus qui inerpretat fortunat?

potut hêr i bona dritate; q. tibi videbß potut hêr i bona dritate; q. tibi videbß

Humanistic
16b

Semiticales
13b.16b

tunc est autem quod premissa est eumque euticus est euticus qui inerpretat fortunat?

potut hêr i bona dritate; q. tibi videbß potut hêr i bona dritate; q. tibi videbß

Incunabulum
15b
<figure><img src="image_24.png" /></figure>

* New languages and new challenges:

* Latin involves new abbreviation to represent, such as <it> for <rum>;

* Middle English involves new signs for new phonetic realization, like <ð> (eth) representing the voiced and voiceless dental fricative.

* The establishment of transcription rules remains an ongoing and evolving effort for ensuring data quality and homogeneity.

* The guidelines will be available at the following link: https://catmus-guidelines.github.io - Work in progress.

<figure><img src="image_25.png" /></figure>
<figure><img src="image_26.png" /></figure>

① Introduction

② History of the CATMuS Project

2.1 First Initiatives

2.2 Gathering projects: CATMuS Chronology

③ Harmonizing Data: CATMuS-Medieval Transcription Guidelines

3.1 Why establish transcription guidelines?

3.2 Birth of the CATMuS-Medieval Guidelines

3.3 CATMuS-Medieval guidelines: principles

④ CATMuS-Medieval datasets and models

4.1 CATMuS-Medieval datasets

4.2 CATMuS-Medieval models

⑤ Conclusion
<figure><img src="image_27.png" /></figure>

<figure><img src="image_28.png" /></figure>

<figure><img src="image_29.png" /></figure>

CATMuS
Medieval

<figure><img src="image_30.png" /></figure>

* CATMuS dataset is published and documented on hugging face: https://huggingface.co/datasets/CATMuS/medieval

* built upon 17 different repositories

* c. 175.000 lines and 6.2M characters

* contains 245 different documents

* 9 different langages

* Its most represented centuries are the 15th century, the 14th, and the 13th.

* Over-representation of some genre and language: French-Narratives (29.2% of the dataset), Treatises-Latin (24.2%) and Treatises-Castillan (23%), linked to the history of the data gathering
<figure><img src="image_31.png" /></figure>

<figure><img src="image_32.png" /><figcaption>Figure: Absolute accuracy</figcaption></figure>

<caption>Figure: Improvement over
CREMMA Generic</caption>

Figure: Test micro-accuracy per language and scripta. Bars represents the total number of characters per categorical feature.
<figure><img src="image_33.png" /></figure>

* A general model for medieval manuscripts: Ariane Pinche et al. “CATMuS Medieval”. lat. In: (Nov. 2023). Publisher: Zenodo. URL: https://zenodo.org/records/10066219 (visited on 01/08/2024)

* A general model for gothic prints: Sonia Solfrini and Simon Gabay. “CATMuS Gothic Print”. frm. In: (Jan. 2024). Publisher: Zenodo. URL: https://zenodo.org/records/10599911 (visited on 03/27/2024)

* A general model for prints: Simon Gabay and Thibault Clérice. “CATMuS-Print [Large]”. fra. In: (Jan. 2024). Publisher: Zenodo. URL: https://zenodo.org/records/10592716 (visited on 03/27/2024)
<figure><img src="image_34.png" /></figure>

① Introduction

② History of the CATMuS Project

2.1 First Initiatives

2.2 Gathering projects: CATMuS Chronology

③ Harmonizing Data: CATMuS-Medieval Transcription Guidelines

3.1 Why establish transcription guidelines?

3.2 Birth of the CATMuS-Medieval Guidelines

3.3 CATMuS-Medieval guidelines: principles

④ CATMuS-Medieval datasets and models

4.1 CATMuS-Medieval datasets

4.2 CATMuS-Medieval models

⑤ Conclusion
<figure><img src="image_35.png" /></figure>

* Thanks to the guidance provided by the guidelines and the support of our sponsors, we have significantly **increased the number of available, interoperable datasets** for training HTR models in recent years.

* This increase has enabled us to develop **new generic models for medieval manuscripts**.

* Consistent transcription standards, collaboration and data sharing across languages have **improved the accuracy of HTR models**

* *Future developments:*

* Continuing the extension of the project to Old English and Middle Dutch

* Extending the temporal scope from the Middle Ages to the contemporary era
<figure><img src="image_36.png" /></figure>

[1] Thomas M. Breuel. *Ocropy: Python-based tools for document analysis and OCR*. 2014. URL: https://github.com/tmbdev/ocropy..

[2] Jean-Baptiste Camps, Thibault Clérice, and Ariane Pinche. "Noisy medieval data, from digitized manuscript to stylometric analysis: Evaluating Paul Meyer's hagiographic hypothesis". In: *Digital Scholarship in the Humanities* 36.Supplement_2 (Oct. 2021), pp. ii49–ii71. ISSN: 2055-7671. URL: https://doi.org/10.1093/llc/fqab033 (visited on 12/03/2021).

[3] Thibault Clérice, Alix Chagué, and Malamatenia Vlachou-Efstathiou. *CREMMA Medii Aevi*. 2023. URL: https://github.com/HTR-United/CREMMA-Medieval-LAT.

[4] Simon Gabay and Thibault Clérice. "CATMuS-Print [Large]". fra. In: (Jan. 2024). Publisher: Zenodo. URL: https://zenodo.org/records/10592716 (visited on 03/27/2024).

[5] Tobias Mathias Hodel, David Selim Schoch, Christa Schneider, and Jake Purcell. "General Models for Handwritten Text Recognition: Feasibility and State-of-the Art. German Current as an Example". In: *Journal of open humanities data* 7.13 (2021), pp. 1–10.

[6] Philip Kahle, Sebastian Colutto, Günter Hackl, and Günter Mühlberger. "Transkribus - A Service Platform for Transcription, Recognition and Retrieval of Historical Documents". In: *2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR)*. Vol. 04. ISSN: 2379-2140. Nov. 2017, pp. 19–24. DOI: 10.1109/ICDAR.2017.307
<figure><img src="image_37.png" /></figure>

[7] Benjamin Kiessling. *The Kraken OCR system*. Version 4.1.2. Apr. 2022. URL: https://kraken.re.

[8] Ariane Pinche. *Cremma Medieval*. June 2022. URL: https://github.com/HTR-United/cremma-medieval.

[9] Ariane Pinche et al. "CATMuS Medieval", lat. In: (Nov. 2023). Publisher: Zenodo. URL: https://zenodo.org/records/10066219 (visited on 01/08/2024).

[10] Sonia Solfrini and Simon Gabay. "CATMuS Gothic Print", frm. In: (Jan. 2024). Publisher: Zenodo. URL: https://zenodo.org/records/10599911 (visited on 03/27/2024).

[11] Peter A. Stokes, Benjamin Kiessling, Daniel Stökl Ben Ezra, Robin Tissot, and El Hassane Gargem. *The eScriptorium VRE for Manuscript Cultures – Classics@Journal*. en-US. 2021. URL: https://classics-at.chs.harvard.edu/classics18-stokes-kiessling-stokl-ben-ezra-tissot-gargem/ (visited on 12/06/2023).