<figure><img src="image_1.png" /></figure>

HAL
open science

# Building a cross-lingual dataset from medieval manuscript text recognition Challenges and outcomes of CATMuS

Thibault Clérice

<figure><img src="image_2.png" /></figure>

## To cite this version:

Thibault Clérice. Building a cross-lingual dataset from medieval manuscript text recognition Challenges and outcomes of CATMuS. École thématique. Computational Humanities Research Group Spring Seminar, Londres, United Kingdom. 2025. hal-05150349

HAL Id: hal-05150349

https://hal.science/hal-05150349v1

Submitted on 8 Jul 2025

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_3.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_4.png" /></figure>

# Building a cross-lingual dataset from medieval manuscript text recognition

## Challenges and outcomes of CATMuS

T. Clérice¹

¹ALMAnaCH, Inria, Paris, France

20.01.2025

<figure><img src="image_5.png" /></figure>

<figure><img src="image_6.png" /></figure>

<figure><img src="image_7.png" /></figure>

<figure><img src="image_8.png" /></figure>

<figure><img src="image_9.png" /></figure>

<figure><img src="image_10.png" /></figure>
<figure><img src="image_11.png" /></figure>

Introduction

Issues that require solutions

Transcription Guidelines ?

So what if I go the editorial way ?

CATMuS Medieval

<figure><img src="image_12.png" /></figure>
<figure><img src="image_13.png" /></figure>

### 1. A Picture

<figure><img src="image_14.png" /></figure>

<figure><img src="image_15.png" /></figure>
<figure><img src="image_16.png" /></figure>

1. A Picture

2. (Opt.) Post-processed

<figure><img src="image_17.png" /></figure>

<figure><img src="image_18.png" /></figure>

<figure><img src="image_19.png" /></figure>

<figure><img src="image_20.png" /></figure>

<figure><img src="image_21.png" /></figure>

<figure><img src="image_22.png" /></figure>

<figure><img src="image_23.png" /></figure>

<figure><img src="image_24.png" /></figure>
<figure><img src="image_25.png" /></figure>

1. A Picture

2. (Opt.) Post-processed

3. Layout Analysis (Zones & Lines)

<figure><img src="image_26.png" /></figure>

<figure><img src="image_27.png" /></figure>

<figure><img src="image_28.png" /></figure>

<figure><img src="image_29.png" /></figure>

<figure><img src="image_30.png" /></figure>

<figure><img src="image_31.png" /></figure>

<figure><img src="image_32.png" /></figure>

<figure><img src="image_33.png" /></figure>

<figure><img src="image_34.png" /></figure>

<figure><img src="image_35.png" /></figure>

<figure><img src="image_36.png" /></figure>

<figure><img src="image_37.png" /></figure>

<figure><img src="image_38.png" /></figure>

<figure><img src="image_39.png" /></figure>

<figure><img src="image_40.png" /></figure>
<figure><img src="image_41.png" /></figure>

1. A Picture

2. (Opt.) Post-processed

3. Layout Analysis (Zones & Lines)

4. Text Prediction (← HTR)

del uéssetor fas, empero facha ta racion sobre aquel las cauzas del uitge de claradas, e general men en lacauza de la premieira ede la segonda apellacion: lo pencut sia condemppatz, en lac despensas de totacz las cauzas al uenssedor. Empero enlas petitas cauzas que non sobre pucion són aualensa de, L. f. nor sia Legut ad alcun dapellar. Q son deuteire ara acopozi

o deuteire local sos crezetors acopozi cion menat tion aura amenatz, empero cardis que non pot pagar o fenhier de nò poder pagar : op sos deutes fo ra laulila. oamaizon de religion sen sera fugitz, facha donation de sos bens onon. Siapres la composizion en trels facha, epueie al deutor sora trobat ren de pagar non contrastan aital composition: otrans acció o cun coument, oprenontiation del prezen establimen o p cascuna autia razon asos crezedors en totas ma nieirac; sia còstreg de pagar, q fug aglieiza oamatzó..

Ni p eniuria op fortag al cus hom aglicizadreligio o oamaizon de religion sen fugira ode monpes lier se ab sentada q atrobatz en alcun luoc ede la corr citatz oamo nestatz non uenra de fra .x. dias onò a trobatz, lacrida publicamen crida que ala cort uen ga de fra .xxx. dias se alacort nos prezentara re la publica crida auut lialm en picitat p conuecut o confes delcrim endreg sia auut . esi p sentientia Pe

<figure><img src="image_42.png" /></figure>

<figure><img src="image_43.png" /></figure>

<figure><img src="image_44.png" /></figure>

<figure><img src="image_45.png" /></figure>

<figure><img src="image_46.png" /></figure>

<figure><img src="image_47.png" /></figure>

<figure><img src="image_48.png" /></figure>

<figure><img src="image_49.png" /></figure>

<figure><img src="image_50.png" /></figure>

<figure><img src="image_51.png" /></figure>

<figure><img src="image_52.png" /></figure>

<figure><img src="image_53.png" /></figure>

<figure><img src="image_54.png" /></figure>

<figure><img src="image_55.png" /></figure>

<figure><img src="image_56.png" /></figure>

<figure><img src="image_57.png" /></figure>

<figure><img src="image_58.png" /></figure>
<figure><img src="image_59.png" /></figure>

1. A Picture

2. (Opt.) Post-processed

3. Layout Analysis (Zones & Lines)

4. Text Prediction (← HTR)

5. (Opt.) Manual Correction

del uêssedor fag, empero facha ta ?acion sobre aquel las cauzas del futge de claradas. e general men en la cauza de la premieira e de la segonda apellacion; lo uencut sia condemnatz, en las despenssas de totas las cauzas al uêssedor. Empero en las petites cauzas que non sobre pueion sôma o ualensa de l4 non sia. Legut ad alcun dapellar. Q son deuteire al uêssora acopzo!

o. deuteire local sos crezedors acompozion menat ion aura amenatz. empero cardis que non pot pagar o fenhier de nô poder pagar; op sos deutes fo ra la uila. o a maizon de religion sen sera fugitz, facha donation de sos bens o non. Si apres la composition en trels facha. e puis al deutor sera trobat ren de pagar non contrastan aital composition; o transacció o al cun couinent. o p renonciacion del prezen establimen o p cascuna autra razon a sos crezedors en totas ma nieiras sia cöstreg de pagar, q fug a gliciza o a maizon.

Si p eniuria o p fortag alcus hom a glicizad religio o a maizon de religion sen fugira o de monpes' lier se ab sentada q atrobatz en alcun luoc e de la cort citatz o a monestätz non uenra de fra x. dias o nô a trobatz, la crida publicamen crida que a la cort uen ga de fra. xxx. dias se a la cort nos prezentara de la publica crida auut lialmen pcitat p conuencut o confes del crim endreg sia auut. e si p sententia pe

<figure><img src="image_60.png" /></figure>
<figure><img src="image_61.png" /></figure>

In the context of ATR, AI =

* Algorithms (model architectures)

* Software (Implementation of architectures)

* Models (Weights learned by the model)

* Training data

But also very often:

* Infrastructure or computing hardware (GPU for training, CPU for inference)

* Infrastructure for data creation

* Research!

<figure><img src="image_62.png" /></figure>

<figure><img src="image_63.png" /></figure>

<figure><img src="image_64.png" /></figure>

<figure><img src="image_65.png" /></figure>

<figure><img src="image_66.png" /></figure>

<figure><img src="image_67.png" /></figure>

<figure><img src="image_68.png" /></figure>

<figure><img src="image_69.png" /></figure>

<figure><img src="image_70.png" /></figure>

<figure><img src="image_71.png" /></figure>

<figure><img src="image_72.png" /></figure>

<figure><img src="image_73.png" /></figure>

<figure><img src="image_74.png" /></figure>
<figure><img src="image_75.png" /></figure>

Two HTR paradigms:

* HTR Engine based on baseline (through segmentation) that uncurves baselines and computes polygons around the line. That's what Kraken does. This allows for the handling of curved and round lines. Allows for more exotic situations.

* HTR Engine based on BBox or complex polygon: a rectangle is drawn around the text, leaving potential noise of the previous and next lines inside. Unable to deal with curved lines.

<figure><img src="image_76.png" /></figure>

<figure><img src="image_77.png" /></figure>

<figure><img src="image_78.png" /></figure>

<figure><img src="image_79.png" /></figure>

<figure><img src="image_80.png" /></figure>

<figure><img src="image_81.png" /></figure>

<figure><img src="image_82.png" /></figure>

<figure><img src="image_83.png" /></figure>

<figure><img src="image_84.png" /></figure>

<figure><img src="image_85.png" /></figure>

<figure><img src="image_86.png" /></figure>

<figure><img src="image_87.png" /></figure>

<figure><img src="image_88.png" /></figure>

<figure><img src="image_89.png" /></figure>

<figure><img src="image_90.png" /></figure>
<figure><img src="image_91.png" /></figure>

<figure><img src="image_92.png" /></figure>

<figure><img src="image_93.png" /></figure>

<figure><img src="image_94.png" /></figure>

<figure><img src="image_95.png" /></figure>

<figure><img src="image_96.png" /></figure>

<figure><img src="image_97.png" /></figure>

<figure><img src="image_98.png" /></figure>

<figure><img src="image_99.png" /></figure>

<figure><img src="image_100.png" /></figure>

<figure><img src="image_101.png" /></figure>

<figure><img src="image_102.png" /></figure>

<figure><img src="image_103.png" /></figure>

<figure><img src="image_104.png" /></figure>

<figure><img src="image_105.png" /></figure>

More and more transformer-based
architecture meant to do
everything at once: segmentation,
text recognition and formatting.
For example, the Nougat model
from Metaª.

<figure><img src="image_106.png" /></figure>

<figure><img src="image_107.png" /></figure>

<figure><img src="image_108.png" /></figure>

<figure><img src="image_109.png" /></figure>

<figure><img src="image_110.png" /></figure>

<figure><img src="image_111.png" /></figure>

<figure><img src="image_112.png" /></figure>

<figure><img src="image_113.png" /></figure>

<footnote>ªLukas Blecher et al. (2023). Nougat: Neural Optical Understanding for Academic Documents. arXiv: 2308.13418 [cs.LG]. URL: https://arxiv.org/abs/2308.13418.</footnote>
<figure><img src="image_114.png" /></figure>

► Generic Models are models thought to be compatible with a wide variety of documents, in a specified space: generic can be ultra-generic (any kind of script, any period) or generic within a smaller space (any Latin mss 10 to 12th cent)

<figure><img src="image_115.png" /></figure>
<figure><img src="image_116.png" /></figure>

* Generic Models are models thought to be compatible with a wide variety of documents, in a specified space: generic can be ultra-generic (any kind of script, any period) or generic within a smaller space (any Latin mss 10 to 12th cent)

* Generic Models aim at balancing overall accuracy on all documents rather than learning more about a single one.

<figure><img src="image_117.png" /></figure>

<figure><img src="image_118.png" /></figure>

<figure><img src="image_119.png" /></figure>

<figure><img src="image_120.png" /></figure>

<figure><img src="image_121.png" /></figure>

<figure><img src="image_122.png" /></figure>

<figure><img src="image_123.png" /></figure>

<figure><img src="image_124.png" /></figure>

<figure><img src="image_125.png" /></figure>

<figure><img src="image_126.png" /></figure>

<figure><img src="image_127.png" /></figure>

<figure><img src="image_128.png" /></figure>

<figure><img src="image_129.png" /></figure>

<figure><img src="image_130.png" /></figure>

<figure><img src="image_131.png" /></figure>

<figure><img src="image_132.png" /></figure>

<figure><img src="image_133.png" /></figure>

<figure><img src="image_134.png" /></figure>

<figure><img src="image_135.png" /></figure>
<figure><img src="image_136.png" /></figure>

* Generic Models are models thought to be compatible with a wide variety of documents, in a specified space: generic can be ultra-generic (any kind of script, any period) or generic within a smaller space (any Latin mss 10 to 12th cent)

* Generic Models aim at balancing overall accuracy on all documents rather than learning more about a single one.

* Specific Models are generally either created from scratch (trained from data of a specific manuscript or subset of), or derived from a generic model: we take the weight of a generic model and further teaches it to learn more about a new subset of documents.

<figure><img src="image_137.png" /></figure>
<figure><img src="image_138.png" /></figure>

* Generic Models are models thought to be compatible with a wide variety of documents, in a specified space: generic can be ultra-generic (any kind of script, any period) or generic within a smaller space (any Latin mss 10 to 12th cent)

* Generic Models aim at balancing overall accuracy on all documents rather than learning more about a single one.

* Specific Models are generally either created from scratch (trained from data of a specific manuscript or subset of), or derived from a generic model: we take the weight of a generic model and further teaches it to learn more about a new subset of documents.

* This Generic → Specific Models process is called fine-tuning, and it allows for faster training time (less energy, less money spent) but also in the case of small dataset better result: the model does not start from kindergarten again, and can keep some of its previous knowledge as well.

<figure><img src="image_139.png" /></figure>
<figure><img src="image_140.png" /></figure>

Introduction

Issues that require solutions

Transcription Guidelines ?

So what if I go the editorial way ?

CATMuS Medieval

<figure><img src="image_141.png" /></figure>
<figure><img src="image_142.png" /></figure>

continuous

<figure><img src="image_143.png" /></figure>

<figure><img src="image_144.png" /></figure>

“aerated”

<figure><img src="image_145.png" /></figure>

<figure><img src="image_146.png" /></figure>

separated

<figure><img src="image_147.png" /><figcaption>Figure: Spacing</figcaption></figure>

<figure><img src="image_148.png" /></figure>
<figure><img src="image_149.png" /></figure>

continuous

<figure><img src="image_150.png" /></figure>

<figure><img src="image_151.png" /></figure>

“aerated”

<figure><img src="image_152.png" /></figure>

<figure><img src="image_153.png" /></figure>

separated

<figure><img src="image_154.png" /><figcaption>Figure: Spacing</figcaption></figure>

<figure><img src="image_155.png" /><figcaption>Figure: Allographs</figcaption></figure>

<figure><img src="image_156.png" /></figure>
<figure><img src="image_157.png" /></figure>

continuous

<figure><img src="image_158.png" /></figure>

<figure><img src="image_159.png" /></figure>

“aerated”

<figure><img src="image_160.png" /></figure>

<figure><img src="image_161.png" /></figure>

separated

<figure><img src="image_162.png" /></figure>

Figure: Spacing

<figure><img src="image_163.png" /></figure>

<figure><img src="image_164.png" /></figure>

<figure><img src="image_165.png" /></figure>

Figure: Allographs

<figure><img src="image_166.png" /></figure>

nulla, - natura, - nota, - nomina

Figure: Abbreviations

<figure><img src="image_167.png" /></figure>
<figure><img src="image_168.png" /></figure>

<table><thead><tr><th></th><th></th><th></th><th></th></tr></thead><tbody><tr><td>Caroline<br>8<sup>th</sup>-13<sup>th</sup></td><td>eutex tenebat qui interpretae forunat</td><td>pultit heri in hominum quatuorque sibi habes</td><td>Cursiva<br>14<sup>th</sup>-15<sup>th</sup>g</td></tr><tr><td>Praegothica<br>12<sup>th</sup>-13<sup>th</sup></td><td>euticus et euticus qui inpretat fortunat</td><td>potut heri in bona quatuoritate et tibi videbſr</td><td>Hybrida<br>15<sup>th</sup>-16<sup>th</sup></td></tr><tr><td>Gothica Textualis<br>13<sup>th</sup>-16<sup>th</sup></td><td>ora qub[que] ipso[rum] ymago d[ur]que ipse picta</td><td>domus s[ac]ui milia homi quos ipse inandria feci aut inopia & cognatorum neglige</td><td>Humanistic<br>16<sup>th</sup></td></tr><tr><td>Semitextualis<br>13<sup>th</sup>-16<sup>th</sup></td><td>e q[ui]que pauonē q[ui]que auſtere gallina refugi</td><td>inandria sic aut inopia & cognatorum neglige</td><td>Incunabulum<br>15<sup>th</sup></td></tr><tr><td></td><td>C omo que creo no fuellen</td><td>De Sperchio fluio.xxiii.oceāi filio</td><td></td></tr><tr><td></td><td>C omo que creo no fuessen</td><td>De Sperchio fluio.xxiii.oceāi filio</td><td></td></tr></tbody></table>

A (small) sample of the types of scripts that can be found...

<figure><img src="image_169.png" /></figure>
<figure><img src="image_170.png" /></figure>

Transcribing is:

* Making choices;

* Interpreting;

* Translating;

* Describing.;

Transcribing is therefore betraying.

<figure><img src="image_171.png" /></figure>

<figure><img src="image_172.png" /></figure>

<figure><img src="image_173.png" /></figure>

<figure><img src="image_174.png" /></figure>

<figure><img src="image_175.png" /></figure>

<figure><img src="image_176.png" /></figure>

<figure><img src="image_177.png" /></figure>

<figure><img src="image_178.png" /></figure>
<figure><img src="image_179.png" /></figure>

1. What are the different transcription questions?

<figure><img src="image_180.png" /></figure>

<figure><img src="image_181.png" /></figure>

<figure><img src="image_182.png" /></figure>

<figure><img src="image_183.png" /></figure>

<figure><img src="image_184.png" /></figure>

<figure><img src="image_185.png" /></figure>

<figure><img src="image_186.png" /></figure>

<figure><img src="image_187.png" /></figure>

<figure><img src="image_188.png" /></figure>

<figure><img src="image_189.png" /></figure>

<figure><img src="image_190.png" /></figure>

<figure><img src="image_191.png" /></figure>

<figure><img src="image_192.png" /></figure>

<figure><img src="image_193.png" /></figure>

<figure><img src="image_194.png" /></figure>

<figure><img src="image_195.png" /></figure>
<figure><img src="image_196.png" /></figure>

1. What are the different transcription questions?

2. Textual content
(Abbreviation/Normalization):
should we write Charles (as in a book), Carles (without homogenization), Carl.
(Unresolved abbreviation), carl.
(Preserving the forms)

<figure><img src="image_197.png" /></figure>

<figure><img src="image_198.png" /></figure>

<figure><img src="image_199.png" /></figure>

<figure><img src="image_200.png" /></figure>

<figure><img src="image_201.png" /></figure>

<figure><img src="image_202.png" /></figure>

<figure><img src="image_203.png" /></figure>

<figure><img src="image_204.png" /></figure>

<figure><img src="image_205.png" /></figure>

<figure><img src="image_206.png" /></figure>

<figure><img src="image_207.png" /></figure>

<figure><img src="image_208.png" /></figure>

<figure><img src="image_209.png" /></figure>

<figure><img src="image_210.png" /></figure>

<figure><img src="image_211.png" /></figure>

<figure><img src="image_212.png" /></figure>

<figure><img src="image_213.png" /></figure>
<figure><img src="image_214.png" /></figure>

1. What are the different transcription questions?

2. Textual content
(Abbreviation/Normalization)

3. Punctuation
(Modernization/Reproduction):
", " (modernized), "" (
("semantic"), " " (faithful to the manuscript?)

<figure><img src="image_215.png" /></figure>

<figure><img src="image_216.png" /></figure>

<figure><img src="image_217.png" /></figure>

<figure><img src="image_218.png" /></figure>

<figure><img src="image_219.png" /></figure>

<figure><img src="image_220.png" /></figure>

<figure><img src="image_221.png" /></figure>

<figure><img src="image_222.png" /></figure>

<figure><img src="image_223.png" /></figure>

<figure><img src="image_224.png" /></figure>

<figure><img src="image_225.png" /></figure>

<figure><img src="image_226.png" /></figure>

<figure><img src="image_227.png" /></figure>

<figure><img src="image_228.png" /></figure>

<figure><img src="image_229.png" /></figure>
<figure><img src="image_230.png" /></figure>

1. What are the different transcription questions?

2. Textual content
(Abbreviation/Normalization)

3. Punctuation
(Modernization/Reproduction)

4. Graphical variation of letters:
round s or long s?

<figure><img src="image_231.png" /></figure>

<figure><img src="image_232.png" /></figure>

<figure><img src="image_233.png" /></figure>

<figure><img src="image_234.png" /></figure>

<figure><img src="image_235.png" /></figure>

<figure><img src="image_236.png" /></figure>

<figure><img src="image_237.png" /></figure>

<figure><img src="image_238.png" /></figure>

<figure><img src="image_239.png" /></figure>

<figure><img src="image_240.png" /></figure>

<figure><img src="image_241.png" /></figure>

<figure><img src="image_242.png" /></figure>

<figure><img src="image_243.png" /></figure>

<figure><img src="image_244.png" /></figure>

<figure><img src="image_245.png" /></figure>

<figure><img src="image_246.png" /></figure>
<figure><img src="image_247.png" /></figure>

1. What are the different transcription questions?

2. Textual content (Abbreviation/Normalization)

3. Punctuation (Modernization/Reproduction)

4. Graphical variation of letters

5. Spacing issues: “ma tere” (modern)? “matere” (presence/absence)? “m a tere” (proportional)?

<figure><img src="image_248.png" /></figure>

<figure><img src="image_249.png" /></figure>

<figure><img src="image_250.png" /></figure>

<figure><img src="image_251.png" /></figure>

<figure><img src="image_252.png" /></figure>

<figure><img src="image_253.png" /></figure>

<figure><img src="image_254.png" /></figure>

<figure><img src="image_255.png" /></figure>

<figure><img src="image_256.png" /></figure>

<figure><img src="image_257.png" /></figure>

<figure><img src="image_258.png" /></figure>

<figure><img src="image_259.png" /></figure>

<figure><img src="image_260.png" /></figure>
<figure><img src="image_261.png" /></figure>

Introduction

Issues that require solutions

Transcription Guidelines ?

So what if I go the editorial way ?

CATMuS Medieval

<figure><img src="image_262.png" /></figure>
<figure><img src="image_263.png" /></figure>

There are three options:

1. Allograph transcription

► One handwritten sign = one
digital sign.

► Unit: allograph.

► Reduced language model.

► Large number of classes.

► Rare classes = no recognition.

► Risk of missing Unicode
symbols!

► The more classes, the higher the
error rate.

<figure><img src="image_264.png" /></figure>

<figure><img src="image_265.png" /></figure>
<figure><img src="image_266.png" /></figure>

There are three options:

1. Allograph transcription

2. “Edition-style” transcription

► Unit: word

► Resolution of abbreviations,

► Punctuation normalization
according to modern practices,

► More easily exploitable data,

► "Cheaper" in the short term;

► **BUT** the machine sees only one
line at a time,

► Context is sometimes
insufficient!

► Depends on the language,
document, and genre!

<figure><img src="image_267.png" /></figure>
<figure><img src="image_268.png" /></figure>

There are three options:

1. Allograph transcription

2. "Edition-style" transcription

3. Graphematic transcription

► One handwritten sign = one
digital sign.

► Unit: character

► Preservation of abbreviations

► Compromise between simplifying
ML tasks and usability

► More independent from
linguistic variations

► **But** requires post-processing

I. —, ⌜ = m, n

II. 9, 2 = con, com, cum, cun

III. 9 2 = us, os, is, s

IV. ≈ ≈ .. = r, re, rb, ar

V. 2s ≈ v ⌜ = ur, tur, er

VI. 27, 28, 29 = rum

VII. 7, & = et, e.

Cappelli, "The Elements of
Abbreviation", p. 13

<figure><img src="image_269.png" /></figure>

<figure><img src="image_270.png" /></figure>

<figure><img src="image_271.png" /></figure>

<figure><img src="image_272.png" /></figure>

<figure><img src="image_273.png" /></figure>

<figure><img src="image_274.png" /></figure>

<figure><img src="image_275.png" /></figure>

<figure><img src="image_276.png" /></figure>
<figure><img src="image_277.png" /></figure>

There are three options:

1. Allograph transcription

2. "Edition-style" transcription

3. Graphematic transcription

Only the first option can automatically convert into the third.

► $v \to u, j \to u$ and similar are easy to do.

<figure><img src="image_278.png" /></figure>
<figure><img src="image_279.png" /></figure>

Tend to prefer "close to the manuscript" transcriptions

Paleographs

Linguists

Tend to prefer "normalized" texts

Historian

Medieval Literary Studies

Content is more important than language

Tend to care about the language

<figure><img src="image_280.png" /></figure>
<figure><img src="image_281.png" /></figure>

Tend to prefer "close to the manuscript" transcriptions

Computer Vision

Paleographs

HTR ?

Linguists

Tend to prefer "normalized" texts

Historian

Medieval Literary Studies

NLP

Content is more important than language

Tend to care about the language

<figure><img src="image_282.png" /></figure>
<figure><img src="image_283.png" /></figure>

Introduction

Issues that require solutions

Transcription Guidelines ?

So what if I go the editorial way ?

CATMuS Medieval

<figure><img src="image_284.png" /></figure>
<figure><img src="image_285.png" /></figure>

* A model is produced on data. **It learns what it sees.**

* A (Kraken) model **sees line by line**, it's the only context it has.

* A (Kraken) model does **learn some languages features by "accident"** when being trained. And the **more** a dataset is **multilingual** or include linguistic differences (lexical, spelling, syntax to a certain degree), the **more** it **resists** guessing based on linguistic features.

* As a consequence, a (Kraken) model **trained on one language** only will always **perform poorly on another language**, or to be precise, more poorly than if it had been trained on the second one: **models are lazy, they take the easy path.**

* You can teach a model to normalize text. But then, what am I teaching and asking from the model? Am I giving the model as much knowledge as the editor has when they read (paratextual knowledge specifically)?

<figure><img src="image_286.png" /></figure>
<figure><img src="image_287.png" /></figure>

* A model is produced on data. **It learns what it sees.**

* A (Kraken) model **sees line by line**, it's the only context it has.

* A (Kraken) model does **learn some languages features by "accident"** when being trained. And the **more** a dataset is **multilingual** or include linguistic differences (lexical, spelling, syntax to a certain degree), the **more** it **resists** guessing based on linguistic features.

* As a consequence, a (Kraken) model **trained on one language** only will always **perform poorly on another language**, or to be precise, more poorly than if it had been trained on the second one: **models are lazy, they take the easy path.**

* You can teach a model to normalize text. But then, what am I teaching and asking from the model? Am I giving the model as much knowledge as the editor has when they read (paratextual knowledge specifically)?

* **And if I do, because it's more convenient, what do I risk?**

<figure><img src="image_288.png" /></figure>
<figure><img src="image_289.png" /></figure>

Sergio Torres Aguilar and Vincent Jolivet (Dec. 2023).
"Handwritten Text Recognition for Documentary Medieval
Manuscripts". In: *Journal of Data Mining & Digital
Humanities* Historical Documents and automatic text
recognition. DOI: 10.46298/jdmdh.10484. URL:
https://jdmdh.episciences.org/12732

<figure><img src="image_290.png" /></figure>

<figure><img src="image_291.png" /></figure>

<figure><img src="image_292.png" /></figure>

<figure><img src="image_293.png" /></figure>

<figure><img src="image_294.png" /></figure>

<figure><img src="image_295.png" /></figure>

<figure><img src="image_296.png" /></figure>

<figure><img src="image_297.png" /></figure>

<figure><img src="image_298.png" /></figure>

<figure><img src="image_299.png" /></figure>

<figure><img src="image_300.png" /></figure>

<figure><img src="image_301.png" /></figure>

<figure><img src="image_302.png" /></figure>

<figure><img src="image_303.png" /></figure>

<figure><img src="image_304.png" /></figure>
<figure><img src="image_305.png" /></figure>

"Upon closer examination, these errors often coincide with misrecognition of typical phenomena in medieval handwriting. These include indistinctness in characters composed of successive minims (single strokes), such as n, m, i, u (e.g., indiuidue, mandauimus); misrecognized ligatures, such as st and ct; undeveloped or incompletely developed abbreviations by suspension, which are easy to fit (e.g., no[-bis], franc[-orum]); by contraction, which are harder to fit (e.g., m[a]g[is]t[er], d[o]m[i]n[us]); abbreviations whose expansion depends on the declension, (v.g. 'Par.' which can be resolved as Par[is], Par[isiensis] or Par[isiense] depending on the context)." Aguilar and Jolivet 2023

<figure><img src="image_306.png" /></figure>

<figure><img src="image_307.png" /></figure>

<figure><img src="image_308.png" /></figure>

<figure><img src="image_309.png" /></figure>

<figure><img src="image_310.png" /></figure>

<figure><img src="image_311.png" /></figure>

<figure><img src="image_312.png" /></figure>

<figure><img src="image_313.png" /></figure>

<figure><img src="image_314.png" /></figure>

<figure><img src="image_315.png" /></figure>

<figure><img src="image_316.png" /></figure>
<figure><img src="image_317.png" /></figure>

Example: https://endp chartes.psl.eu/facsimile/LL119/369

<figure><img src="image_318.png" /></figure>

<figure><img src="image_319.png" /></figure>

<figure><img src="image_320.png" /></figure>

<figure><img src="image_321.png" /></figure>

<figure><img src="image_322.png" /></figure>

<figure><img src="image_323.png" /></figure>

<figure><img src="image_324.png" /></figure>

<figure><img src="image_325.png" /></figure>

<figure><img src="image_326.png" /></figure>

<figure><img src="image_327.png" /></figure>

<figure><img src="image_328.png" /></figure>

<figure><img src="image_329.png" /></figure>
<figure><img src="image_330.png" /></figure>

Sergio Torres Aguilar (2024). *Handwritten Text Recognition for Historical Documents using Visual Language Models and GANs*. URL: https://hal.science/hal-04716654/

<figure><img src="image_331.png" /></figure>

<figure><img src="image_332.png" /></figure>
<figure><img src="image_333.png" /></figure>

> "The efficacy of using explicit language models to improve the accuracy of HTR has been demonstrated in previous studies, yielding promising results[...]. These models capture statistical correlations among tokens in natural language, providing context awareness and proposing more contextually appropriate interpretations. This is particularly crucial for addressing challenges such as complex layouts, ancient or noisy writing, or out-of-domain issues, where a purely graphical interpretation may fall short in providing an accurate transcription, as it lacks the linguistic nuances and contextual understanding that language models bring to the table. In contrast to classical CRNN models, which implicitly learn language features and focus on identifying patterns directly from the data[...], the integration of a language model involves a statistical analysis of how words interrelate in a language and the likelihood of certain sequences co-occurring. This approach results in more precise and contextually informed predictions." Aguilar 2024

<figure><img src="image_334.png" /></figure>
<figure><img src="image_335.png" /></figure>

> "Most errors from Transformer models are linked to mis-predictions at the word or sub-word level, paradoxically increasing the CER more than the WER. [...] This strategy is highly effective, as evidenced by the significant difference in WER between Transformer and CRNN models. CRNN models often propose seemingly verbatim transcriptions [...], that] "imitate" the input, especially when faced with unfamiliar words or phrases, rather than understanding the underlying linguistic patterns. Consequently, many of their outputs ignore the lexical relationships within the text, resulting in transcriptions that may be technically accurate at the character level but correspond to non-existent or nonsensical words." Aguilar 2024

<figure><img src="image_336.png" /></figure>

<figure><img src="image_337.png" /></figure>

<figure><img src="image_338.png" /></figure>

<figure><img src="image_339.png" /></figure>

<figure><img src="image_340.png" /></figure>

<figure><img src="image_341.png" /></figure>

<figure><img src="image_342.png" /></figure>

<figure><img src="image_343.png" /></figure>

<figure><img src="image_344.png" /></figure>

<figure><img src="image_345.png" /></figure>

<figure><img src="image_346.png" /></figure>

<figure><img src="image_347.png" /></figure>
<figure><img src="image_348.png" /></figure>

> "Most errors from Transformer models are linked to mis-predictions at the word or sub-word level, paradoxically increasing the CER more than the WER. [...] This strategy is highly effective, as evidenced by the significant difference in WER between Transformer and CRNN models. CRNN models often propose seemingly verbatim transcriptions [...], that] "imitate" the input, especially when faced with unfamiliar words or phrases, rather than understanding the underlying linguistic patterns. Consequently, many of their outputs ignore the lexical relationships within the text, resulting in transcriptions that may be technically accurate at the character level but correspond to non-existent or nonsensical words." Aguilar 2024

<figure><img src="image_349.png" /></figure>

<figure><img src="image_350.png" /></figure>

<figure><img src="image_351.png" /></figure>

<figure><img src="image_352.png" /></figure>

<figure><img src="image_353.png" /></figure>

<figure><img src="image_354.png" /></figure>

<figure><img src="image_355.png" /></figure>

<figure><img src="image_356.png" /></figure>

<figure><img src="image_357.png" /></figure>

<figure><img src="image_358.png" /></figure>

<figure><img src="image_359.png" /></figure>

<figure><img src="image_360.png" /></figure>

<figure><img src="image_361.png" /></figure>
<figure><img src="image_362.png" /></figure>

1

GT H[enrions] de Flasigni a vendu a H[enri] conte de Lucel[burch] le fié et l'rierfié ke messires Giles d'Yrei

TR y Hugues de Flasigni a vendu a H. conte de Lucembour le fié et l'rier fié ke mesigneur Giles Dyrei

TR β Hugues de Flasigni a vendu a H. conte de Luce le fié et larier fié ke messire Giles d'Yrei.

CRNN β H. de Flasigni a Vendu a H. ote de Lutre le fié et larier fié kee Mesigre il Dyrei

2

GT inceps qui in locum defuncti paren- | tis qui ex eodem natis sunt succes

TR y inceps quin locum defuncti patri | tis, qui ex eodem natisunt succes

TR β inceptis quin locum defuncti Petri | tis qui ex eodem natis sunt succes

CRNN β inceps quim locum defunctiparei | tisqui ex fodern natisunt succer

<figure><img src="image_363.png" /></figure>
<figure><img src="image_364.png" /></figure>

<table><tr><td>3</td><td>omni peccatorum quia ex quo dyabolus</td><td>omni cœt ifusui. stultæ cœt</td></tr><tr><td>GT</td><td>omnium peccatorum quia ex quo dyaconus</td><td>quando esset infusum, stultus esset</td></tr><tr><td>TR y</td><td>omni peccatorum quod ex quo dyaus</td><td>quando esset in futurum. stultus esset</td></tr><tr><td>TR β</td><td>onium pocorum quia ex quo dyas</td><td>que esset in fusum scultus esset</td></tr><tr><td>CRNN β</td><td></td><td>que essent infusus. stultus essent</td></tr><tr><td>4</td><td>omni populo et cetera. Quod figuratum est hester. vüj. iudeis nova lux oriri</td><td></td></tr><tr><td>GT</td><td>omni populo et cetera. Quod sigillatum est hester VIIIº iudeis nova lux oriri</td><td></td></tr><tr><td>TR y</td><td>omni populo et cetera. Quod sigillatum est hester : VIIIº, indeis nova lux oriri</td><td></td></tr><tr><td>TR β</td><td>omni populo et cetera. Quod signatum est hester : VIIIº, indeis nova lux oriri</td><td></td></tr><tr><td>CRNN β</td><td>omni propulo et tr Rd figruatum est hester VIII iudeis nova lux oriri</td><td></td></tr></table>

T. expanded to Tours instead of Trèves, because Trèves was not in the data: models are situated, constrained. Abbreviation expansion needs context.

<figure><img src="image_365.png" /></figure>
<figure><img src="image_366.png" /></figure>

Natalia Bottaioli et al. (2024). “Normalized vs Diplomatic Annotation: A Case Study of Automatic Information Extraction from Handwritten Uruguayan Birth Certificates”.

In: *Document Analysis and Recognition – ICDAR 2024 Workshops*. Ed. by

Harold Mouchère and Anna Zhu. Cham: Springer Nature Switzerland, pp. 40–54.

ISBN: 978-3-031-70645-5

<figure><img src="image_367.png" /></figure>

<figure><img src="image_368.png" /></figure>

<figure><img src="image_369.png" /></figure>

<figure><img src="image_370.png" /></figure>

<figure><img src="image_371.png" /></figure>

<figure><img src="image_372.png" /></figure>

<figure><img src="image_373.png" /></figure>

<figure><img src="image_374.png" /></figure>

<figure><img src="image_375.png" /></figure>

<figure><img src="image_376.png" /></figure>

<figure><img src="image_377.png" /></figure>

<figure><img src="image_378.png" /></figure>

<figure><img src="image_379.png" /></figure>

<figure><img src="image_380.png" /></figure>

<figure><img src="image_381.png" /></figure>

<figure><img src="image_382.png" /></figure>

<figure><img src="image_383.png" /></figure>

<figure><img src="image_384.png" /></figure>
<figure><img src="image_385.png" /></figure>

<figure><img src="image_386.png" /></figure>

<figure><img src="image_387.png" /></figure>
<figure><img src="image_388.png" /></figure>

<table><thead><tr><td>Field</td><td>Normalized DAN</td><td>Diplomatic DAN</td></tr></thead><tbody><tr><td>document number</td><td>232</td><td>232</td></tr><tr><td>enrollee's full name</td><td>Noble Alonso Rocio</td><td>NOBLE ALONSO ROCÍO</td></tr><tr><td>year of enrollment</td><td>doce</td><td>doce</td></tr><tr><td>jurisdiction</td><td>Primera</td><td>de la 1a</td></tr><tr><td>department</td><td>Tacuarembó</td><td>Tacuarembó</td></tr><tr><td>1st parent's name</td><td>José E. Noble</td><td>José Enrique Noble</td></tr><tr><td>date field 1</td><td>veintinueve</td><td>veintinueve</td></tr><tr><td>date field 2</td><td>febrero de 2012</td><td>febrero de 2012</td></tr><tr><td>2nd parent's name</td><td>Rocio M. Alonso</td><td>Rocio Marimelda Alonso</td></tr></tbody></table>

<figure><img src="image_389.png" /></figure>
<figure><img src="image_390.png" /></figure>

<table><thead><tr><th rowspan="3">Field</th><th colspan="4">Normalized DAN</th><th colspan="4">Diplomatic DAN</th></tr><tr><th colspan="2">CER (%)</th><th colspan="2">WER (%)</th><th colspan="2">CER (%)</th><th colspan="2">WER (%)</th></tr><tr><th>reg.</th><th>norm.</th><th>reg.</th><th>norm.</th><th>reg.</th><th>norm.</th><th>reg.</th><th>norm.</th></tr></thead><tbody><tr><td>document number</td><td>3.51</td><td>-</td><td>10.00</td><td>-</td><td>3.51</td><td>-</td><td>10.00</td><td>-</td></tr><tr><td>year of enrollment</td><td>0.00</td><td><strong>0.00</strong></td><td>0.00</td><td><strong>0.00</strong></td><td>3.31</td><td>3.31</td><td>5.00</td><td>5.00</td></tr><tr><td>jurisdiction</td><td>0.00</td><td><strong>0.00</strong></td><td>0.00</td><td><strong>0.00</strong></td><td>0.92</td><td>0.92</td><td>3.33</td><td>3.33</td></tr><tr><td>department</td><td>0.00</td><td><strong>0.00</strong></td><td>0.00</td><td><strong>0.00</strong></td><td>4.50</td><td>0.00</td><td>5.00</td><td>0.00</td></tr><tr><td>date field 1</td><td>0.00</td><td><strong>0.00</strong></td><td>0.00</td><td><strong>0.00</strong></td><td>0.61</td><td>0.61</td><td>3.85</td><td>3.85</td></tr><tr><td>date field 2</td><td>0.00</td><td><strong>0.00</strong></td><td>0.00</td><td><strong>0.00</strong></td><td>0.35</td><td>0.00</td><td>1.67</td><td>0.00</td></tr><tr><td>enrollee's full name</td><td>2.25</td><td>2.25</td><td>9.64</td><td>9.64</td><td>2.94</td><td><strong>1.73</strong></td><td>14.46</td><td><strong>6.02</strong></td></tr><tr><td>1st parent's name</td><td>10.72</td><td>10.43</td><td>36.36</td><td>36.36</td><td>2.83</td><td><strong>2.02</strong></td><td>12.86</td><td><strong>10.00</strong></td></tr><tr><td>2nd parent's name</td><td>15.81</td><td>15.81</td><td>35.19</td><td>35.19</td><td>7.17</td><td><strong>5.70</strong></td><td>28.36</td><td><strong>20.90</strong></td></tr><tr><td>global</td><td>4.36</td><td>4.32</td><td>13.92</td><td>13.92</td><td>3.05</td><td><strong>2.04</strong></td><td>11.27</td><td><strong>7.51</strong></td></tr></tbody></table>

<figure><img src="image_391.png" /></figure>
<figure><img src="image_392.png" /></figure>

"One of the possible reasons why Diplomatic DAN works better than Normalized DAN on these three fields was already explained in the description of the dataset: annotations provided by the civil registry included some abbreviation criteria on both parents' names fields where, very frequently, the second name was abbreviated to its initial (e.g., "Alberto Carlos Bustos" was transcribed as "Alberto C. Bustos"). Somehow, this was learned by the model but with mixed results. In cases where the enrollee has a second name and the model was able to understand which letter to use, Normalized DAN produced a correct output (i.e. correctly abbreviated the middle name). In other cases, it made a mistake when abbreviating the middle name. **In cases where no middle name was handwritten in the document, the model just invented one**, presumably to agree with the format of the data it was trained on." Bottaioli et al. 2024

<figure><img src="image_393.png" /></figure>

<figure><img src="image_394.png" /></figure>

<figure><img src="image_395.png" /></figure>

<figure><img src="image_396.png" /></figure>

<figure><img src="image_397.png" /></figure>

<figure><img src="image_398.png" /></figure>

<figure><img src="image_399.png" /></figure>

<figure><img src="image_400.png" /></figure>

<figure><img src="image_401.png" /></figure>

<figure><img src="image_402.png" /></figure>

<figure><img src="image_403.png" /></figure>

<figure><img src="image_404.png" /></figure>

<figure><img src="image_405.png" /></figure>
<figure><img src="image_406.png" /></figure>

* There are very few combinations of fields (history, literature, philosophy, etc.), countries (France, Italy, Spain, Germany) or even subcategories (anglo-normand specialists vs. picard specialist) that would produce the same transcription depending on the tradition of their field.

* Any error is lost to the HTR: the decision to resolve is not visible in these models. The only reason these errors were caught is because papers' authors looked at the result.

* Punctuation, capitalization, normalization of abbreviation are domain-specific.

<figure><img src="image_407.png" /></figure>

<figure><img src="image_408.png" /></figure>

<figure><img src="image_409.png" /></figure>

<figure><img src="image_410.png" /></figure>

<figure><img src="image_411.png" /></figure>

<figure><img src="image_412.png" /></figure>
<figure><img src="image_413.png" /></figure>

* There are very few combinations of fields (history, literature, philology, etc.), countries (France, Italy, Spain, Germany) or even subcategories (anglo-normand specialists vs. picard specialist) that would produce the same transcription depending on the tradition of their field.

* Any error is lost to the HTR: the decision to resolve is not visible in these models. The only reason these errors were caught is because papers' authors looked at the result.

* Punctuation, capitalization, normalization of abbreviation are domain-specific.

* **This means that no datasets would be compatible !**

* A solution is then to keep an approach that stay simple and leave some work for the researchers. And we can see how to address, domain per domain, what we can do to semi-automatize the translation.

<figure><img src="image_414.png" /></figure>

<figure><img src="image_415.png" /></figure>

<figure><img src="image_416.png" /></figure>

<figure><img src="image_417.png" /></figure>

<figure><img src="image_418.png" /></figure>

<figure><img src="image_419.png" /></figure>

<figure><img src="image_420.png" /></figure>

<figure><img src="image_421.png" /></figure>

<figure><img src="image_422.png" /></figure>

<figure><img src="image_423.png" /></figure>

<figure><img src="image_424.png" /></figure>

<figure><img src="image_425.png" /></figure>

<figure><img src="image_426.png" /></figure>

<figure><img src="image_427.png" /></figure>

<figure><img src="image_428.png" /></figure>

<figure><img src="image_429.png" /></figure>

<figure><img src="image_430.png" /></figure>

<figure><img src="image_431.png" /></figure>

<figure><img src="image_432.png" /></figure>

<figure><img src="image_433.png" /></figure>

<figure><img src="image_434.png" /></figure>
<figure><img src="image_435.png" /></figure>

Introduction

Issues that require solutions

Transcription Guidelines ?

So what if I go the editorial way ?

CATMuS Medieval

<figure><img src="image_436.png" /></figure>
<figure><img src="image_437.png" /></figure>

Thibault Clérice et al. (2024). “CATMuS Medieval: A multilingual large-scale cross-century dataset in Latin script for handwritten text recognition and beyond”. In: *International Conference on Document Analysis and Recognition*. Springer, pp. 174–194. URL: https://inria.hal.science/hal-04453952

Also https://catmus-guidelines.github.io and

https://huggingface.co/datasets/CATMuS/medieval

<figure><img src="image_438.png" /></figure>

<figure><img src="image_439.png" /></figure>

<figure><img src="image_440.png" /></figure>

<figure><img src="image_441.png" /></figure>

<figure><img src="image_442.png" /></figure>

<figure><img src="image_443.png" /></figure>

<figure><img src="image_444.png" /></figure>

<figure><img src="image_445.png" /></figure>

<figure><img src="image_446.png" /></figure>

<figure><img src="image_447.png" /></figure>

<figure><img src="image_448.png" /></figure>

<figure><img src="image_449.png" /></figure>
<figure><img src="image_450.png" /></figure>

* Graphic distinctions for the same character like 〈s〉 (“round 〈s〉”) and 〈ı〉 (“long 〈s〉”), or 〈2〉 (“round” 〈r〉) and 〈ı〉 (“short” 〈r〉), are disregarded to mitigate potential cascading effects on other letters like a, d, e, etc.

* Ligatures between letters, such as 〈st〉 or 〈ct〉, are treated as variations in form, leading to the independent transcription of their constituent characters.

* Any emphasis on the letter, regardless of its form, is transcribed as capital letters, including if the emphasis is only conveyed by the size of the letter (and does not result in a different shape).

<figure><img src="image_451.png" /></figure>
<figure><img src="image_452.png" /></figure>

The various signs fall into two main categories:

1. A large number are additions to base letters with diacritical marks, such as macrons (e.g. 〈ē〉 for est, -em, etc.), and superscript letters (〈qı〉 for qui), and their meaning is often context-dependent. Diacritics are thus treated as separate characters using Unicode decomposed form (NFD).

2. Few are separate semantically distinct signs, such as 〈&〉 for et/e), or letters with strike-through marks, such as 〈p〉. For these, we resorted to the Medieval Unicode Font Initiative (MUFI) Haugen 2009.

<figure><img src="image_453.png" /></figure>

<figure><img src="image_454.png" /></figure>

<figure><img src="image_455.png" /></figure>

<figure><img src="image_456.png" /></figure>

<figure><img src="image_457.png" /></figure>

<figure><img src="image_458.png" /></figure>

<figure><img src="image_459.png" /></figure>

<figure><img src="image_460.png" /></figure>

<figure><img src="image_461.png" /></figure>

<figure><img src="image_462.png" /></figure>

<figure><img src="image_463.png" /></figure>
<figure><img src="image_464.png" /></figure>

* Single full stops are transcribed as (.);

* Double signs are represented by (;);

* Commas are transcribed as (,) ,

* Question marks are transcribed as (?), as soon as they appear in the Latin inventory in late middle ages.

<figure><img src="image_465.png" /></figure>

<figure><img src="image_466.png" /></figure>

<figure><img src="image_467.png" /></figure>

<figure><img src="image_468.png" /></figure>

<figure><img src="image_469.png" /></figure>

<figure><img src="image_470.png" /></figure>
<figure><img src="image_471.png" /></figure>

<table><thead><tr><th>Dataset</th><th>Type</th><th>Century</th><th>Language</th><th>Script type</th><th>Task</th><th>Abbr. Res.</th><th>Char. Var.</th><th>Modern.</th><th>Quantity</th></tr></thead><tbody><tr><td>IAMHistDB/St. Gall</td><td>Mss</td><td>9</td><td>Latin</td><td>Bookscript</td><td>HTR</td><td></td><td></td><td></td><td>1,410 l.</td></tr><tr><td>IAMHistDB/Perzival</td><td>Mss</td><td>13</td><td>German</td><td>Bookscript</td><td>HTR</td><td>✓</td><td>✓</td><td></td><td>4,477 l.</td></tr><tr><td>ICFHR 2016-READ</td><td>Mss</td><td>15-19</td><td>Ger.</td><td>Modern</td><td>HTR</td><td></td><td></td><td></td><td>10,550 l.</td></tr><tr><td>ORIFLAMMS</td><td>Mss</td><td>12-14</td><td>French, Lat.</td><td>Book., Docum.</td><td>HTR</td><td>✓</td><td>✓</td><td>✓</td><td>120,111 l.</td></tr><tr><td>HIMANIS</td><td>Mss</td><td>14-15</td><td>Fr., Lat.</td><td>Documentary</td><td>HTR</td><td>✓</td><td>✓</td><td>✓</td><td>23,112 l.</td></tr><tr><td>HOME - Alcar</td><td>Mss</td><td>12-14</td><td>Fr., Lat.</td><td>Bookscripts</td><td>HTR, NER</td><td>✓</td><td></td><td>✓</td><td>74806 l.</td></tr><tr><td>e-NDP</td><td>Mss</td><td>14-16</td><td>Fr., Lat.</td><td>Documentary</td><td>HTR</td><td>✓</td><td></td><td>✓</td><td>33,735 l.</td></tr><tr><td>RODRIGO</td><td>Mss</td><td>16</td><td>Spanish</td><td>Bookscript</td><td>HTR</td><td></td><td></td><td></td><td>20,357 l.</td></tr><tr><td>Esposalles/INDEX</td><td>Mss</td><td>15</td><td>Catalan</td><td>Modern</td><td>HTR</td><td></td><td></td><td></td><td>1,563 l.</td></tr><tr><td>LAM</td><td>Mss</td><td>17-18</td><td>Italian</td><td>Modern</td><td>HTR, DC</td><td></td><td></td><td></td><td>25,823 l.</td></tr><tr><td>MPS</td><td>Mss</td><td>13-16</td><td>Dutch</td><td>Documentary</td><td>DC</td><td></td><td></td><td></td><td>1,706 d.</td></tr><tr><td>CLAMM</td><td>Mss</td><td>5-15</td><td>Lat.</td><td>Bookscripts</td><td>SC, DC</td><td></td><td></td><td></td><td>10,800 i.</td></tr><tr><td>Multiple Font Groups</td><td>EP</td><td>15-17</td><td>Multi.</td><td>Fonts</td><td>FC</td><td></td><td></td><td></td><td>35,623 i.</td></tr></tbody></table>

Table: Overview of parallel datasets. Abbr. Res. = resolved abbreviations. Char. Var. = allographetic (s)/(fi). Modern. = edition-like modernization. Mss stands for Manuscripts, EP for Early Prints, SC/FC/DC for Script, Font, and Date Classification.

<figure><img src="image_472.png" /></figure>
<figure><img src="image_473.png" /></figure>

<table><thead><tr><th>Dataset</th><th>Type</th><th>Century</th><th>Language</th><th>Script type</th><th>Task</th><th>Abbr. Res.</th><th>Char. Var.</th><th>Modern.</th><th>Quantity</th></tr></thead><tbody><tr><td>IAMHistDB/St. Gall</td><td>Mss</td><td>9</td><td>Latin</td><td>Bookscript</td><td>HTR</td><td></td><td></td><td></td><td>1,410 l.</td></tr><tr><td>IAMHistDB/Perzival</td><td>Mss</td><td>13</td><td>German</td><td>Bookscript</td><td>HTR</td><td>✓</td><td>✓</td><td></td><td>4,477 l.</td></tr><tr><td>ICFHR 2016-READ</td><td>Mss</td><td>15-19</td><td>Ger.</td><td>Modern</td><td>HTR</td><td></td><td></td><td></td><td>10,550 l.</td></tr><tr><td>ORIFLAMMS</td><td>Mss</td><td>12-14</td><td>French, Lat.</td><td>Book., Docum.</td><td>HTR</td><td>✓</td><td>✓</td><td>✓</td><td>120,111 l.</td></tr><tr><td>HIMANIS</td><td>Mss</td><td>14-15</td><td>Fr., Lat.</td><td>Documentary</td><td>HTR</td><td>✓</td><td>✓</td><td>✓</td><td>23,112 l.</td></tr><tr><td>HOME - Alcar</td><td>Mss</td><td>12-14</td><td>Fr., Lat.</td><td>Bookscripts</td><td>HTR, NER</td><td>✓</td><td></td><td>✓</td><td>74806 l.</td></tr><tr><td>e-NDP</td><td>Mss</td><td>14-16</td><td>Fr., Lat.</td><td>Documentary</td><td>HTR</td><td>✓</td><td></td><td>✓</td><td>33,735 l.</td></tr><tr><td>RODRIGO</td><td>Mss</td><td>16</td><td>Spanish</td><td>Bookscript</td><td>HTR</td><td></td><td></td><td></td><td>20,357 l.</td></tr><tr><td>Esposalles/INDEX</td><td>Mss</td><td>15</td><td>Catalan</td><td>Modern</td><td>HTR</td><td></td><td></td><td></td><td>1,563 l.</td></tr><tr><td>LAM</td><td>Mss</td><td>17-18</td><td>Italian</td><td>Modern</td><td>HTR, DC</td><td></td><td></td><td></td><td>25,823 l.</td></tr><tr><td>MPS</td><td>Mss</td><td>13-16</td><td>Dutch</td><td>Documentary</td><td>DC</td><td></td><td></td><td></td><td>1,706 d.</td></tr><tr><td>CLAMM</td><td>Mss</td><td>5-15</td><td>Lat.</td><td>Bookscripts</td><td>SC, DC</td><td></td><td></td><td></td><td>10,800 i.</td></tr><tr><td>Multiple Font Groups</td><td>EP</td><td>15-17</td><td>Multi.</td><td>Fonts</td><td>FC</td><td></td><td></td><td></td><td>35,623 i.</td></tr><tr><td>CATMuS (Ours)</td><td>Mss, EP</td><td>8-16</td><td>Multi.</td><td>Bookscripts<br>Documentary<br>Fonts</td><td>HTR, SC, DC</td><td></td><td></td><td></td><td>165,347 l.</td></tr></tbody></table>

Table: Overview of parallel datasets. Abbr. Res. = resolved abbreviations. Char. Var. = allographetic (s)/(f). Modern. = edition-like modernization. Mss stands for Manuscripts, EP for Early Prints, SC/FC/DC for Script, Font, and Date Classification.

<figure><img src="image_474.png" /></figure>
<figure><img src="image_475.png" /></figure>

<table><caption>Figure: Representative examples of the main script types lines, with CATMuS annotations.</caption><thead><tr><th></th><th></th><th></th></tr></thead><tbody><tr><td>Caroline<br>8<sup>th</sup>-13<sup>th</sup></td><td>eusex teuricuf qui unerpretæ for tunæ.<br>eutex 1 euticus qui inerpretat fortunat?</td><td>pwtir hri i bona qutare s, q nbi Videbi<br>potut heri i bona qtitate k; q tibi videbfr</td></tr><tr><td>Praegothica<br>12<sup>th</sup>-13<sup>th</sup></td><td>ora qly ifofr ymago di ape sic pietæ<br>iora qb; ifort ymago dr æpe sic picta.</td><td>wmae. &. vn milia ho? quæ : ipr<br>domos .s. iii milia ho<sup>m</sup> quos 7 ipe</td></tr><tr><td>Gothica Textualis<br>13<sup>th</sup>-16<sup>th</sup></td><td>e qre pauonē qre anterē gallina refugi<br>e qre pauonē qre anserē gallina refugi</td><td>mandria sic aut inopia & cognatorum neglige<br>inandria sic aut inopia & cognatorum neglige</td></tr><tr><td>Semitextualis<br>13<sup>th</sup>-16<sup>th</sup></td><td>C omo que creo no fuenen<br>C omo que creo no fuessen</td><td>De Sperchio fluio.xxiii.oceāi filio<br>De Sperchio fluio.xxiii.oceāi filio</td></tr></tbody></table>

<figure><img src="image_476.png" /></figure>
<figure><img src="image_477.png" /></figure>

<figure><img src="image_478.png" /><figcaption>**Figure:** Distribution of the dataset across languages, centuries, and scripts, in % of the total character count.</figcaption></figure>

<figure><img src="image_479.png" /></figure>

<figure><img src="image_480.png" /></figure>

<figure><img src="image_481.png" /></figure>

<figure><img src="image_482.png" /></figure>

<figure><img src="image_483.png" /></figure>

<figure><img src="image_484.png" /></figure>

<figure><img src="image_485.png" /></figure>

<figure><img src="image_486.png" /></figure>

<figure><img src="image_487.png" /></figure>

<figure><img src="image_488.png" /></figure>

<figure><img src="image_489.png" /></figure>

<figure><img src="image_490.png" /></figure>

<figure><img src="image_491.png" /></figure>
<figure><img src="image_492.png" /></figure>

► 20 manuscripts

► 10 epochs (each line seen ten times at most)

► Doable on a modern laptop in ≤ 10 minutes (↑ accessibility, ↓ power consumption)

► 50 % of their lines for training the model, 50 % to test

<figure><img src="image_493.png" /></figure>

<figure><img src="image_494.png" /></figure>

<figure><img src="image_495.png" /></figure>

<figure><img src="image_496.png" /></figure>

<figure><img src="image_497.png" /></figure>

<figure><img src="image_498.png" /></figure>

<figure><img src="image_499.png" /></figure>

<figure><img src="image_500.png" /></figure>

<figure><img src="image_501.png" /></figure>

<figure><img src="image_502.png" /></figure>

<figure><img src="image_503.png" /></figure>

<figure><img src="image_504.png" /></figure>

<figure><img src="image_505.png" /></figure>
<figure><img src="image_506.png" /></figure>

* What we can see in CATMuS is that AI hallucination are limited, and we have more misreading in models than hallucinations.

* Yet, some historians (mostly) and researcher in medieval literature still prefer to train a specific (!= generic) model for a text, a genre or a manuscript where abbreviation are resolved.

* The models have the limitation we have seen and won't be reusable until "technology fix it".

* Hallucination are hidden, as they are not marked as abbreviation resolution: we can't know when an abbreviation is badly resolved, an HTR error was made or an hallucination occured. And unless we see the manuscript...

* By doing so, their data is partially incompatible with other datasets : **short term win, long term cost**.

* On the other hand, CATMuS, by design, limits hallucinations and keeps the manuscript "text" quite close.

* On the short-term, it's more costly for project to run CATMuS approaches, as they will need to resolve abbreviation manually.

* Abbreviation resolution is on the other hand a task very close to other task in NLP such as lemmatization. The technology is here, what we miss now is the data to support this technology. **Long term win, short term cost.**

<figure><img src="image_507.png" /></figure>
<figure><img src="image_508.png" /></figure>

https://huggingface.co/datasets/CATMuS/medieval

CATMuS is a team project, run mainly by Ariane Pinche, Alix Chagué, Simon Gabay and Thibault Clérice. CATMuS medieval owes a lot to multiple people, but our main data curator regarding metadata is Malamatenia Vlachou-Efstatiou, who we thank again.

<figure><img src="image_509.png" /></figure>

<figure><img src="image_510.png" /></figure>

<figure><img src="image_511.png" /></figure>

<figure><img src="image_512.png" /></figure>

<figure><img src="image_513.png" /></figure>

<figure><img src="image_514.png" /></figure>