<figure><img src="image_1.png" /></figure>

HAL
open science

# OCR17: Ground Truth and Models for 17th c. French Prints (and hopefully more)

Simon Gabay, Thibault Clérice, Christian Reul

► To cite this version:

Simon Gabay, Thibault Clérice, Christian Reul. OCR17: Ground Truth and Models for 17th c. French Prints (and hopefully more). Journal of Data Mining and Digital Humanities, 2023, 2023, 10.46298/jdmdh.6492. hal-02577236v2

HAL Id: hal-02577236

https://hal.science/hal-02577236v2

Submitted on 16 May 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution - ShareAlike 4.0 International License
# OCR17: Ground Truth and Models for 17th century French Prints (and hopefully more)

Simon Gabay<fnref n="1" />, Thibault Clérice<fnref n="2" />, Christian Reul³

1Université de Neuchâtel, Université de Genève (Switzerland)

2École nationale des chartes, Centre Jean Mabillon (EA 3624), PSL University (France)

3Universität Würzburg (Germany)

Corresponding author: Simon Gabay, simon.gabay@unige.ch

## Abstract

Machine learning begins with machine teaching: in the following paper, we present the data that we have prepared to kick-start the training of reliable OCR models for 17th century prints written in French. The construction of a representative corpus is a major challenge: we need to gather documents from different decades and of different genres to cover as many sizes, weights and styles as possible. Historical prints containing glyphs and typefaces that have now disappeared, transcription is a complex act, for which we present guidelines. Finally, we provide preliminary results based on these training data and experiments to improve them.

## Keywords

OCR, 17th century French, training data, transcription guidelines, corpus building

## I INTRODUCTION

OCR engines such as Abbyy<fnref n="1" /> or Tesseract [Smith, 2007] today come with models that work perfectly for the most recent documents written in French (i.e. the 19th and 20th centuries)<fnref n="2" />, but older periods are still less well handled by the machines. Our objective is therefore to solve this problem, and to propose the tools, as much as the data and the method necessary for the processing of historical documents following the recommendations on open science [Chagué et al., 2020], and especially prints in French of the 17th century. In doing so, we hope to prepare the digitisation of the entire period of the Ancien Régime, the prints of the 18th century and the second half of the 16th century written in French being relatively similar to those of the 17th century.

The reliability of OCR models depends on both the quantity and the quality of training data. On the one hand, quantity needs to be produced and made freely available to other scholars, which is sadly not always the case. On the other hand, quality needs to be properly defined, since philological traditions vary from one place to another [Duval, 2018], but also from one period to another [Gabay, 2014, Duval, 2015]. Both problems need therefore to be addressed to propose a reliable solution for the OCRisation of historical prints written in French.

Following the example of *GT4HistOCR* [Springmann et al., 2018], which mainly focuses on German (and marginally Latin [Springmann et al., 2016]), we have designed a corpus of Ground

<footnote n="1">¹https://pdf.abbyy.com</footnote>

<footnote n="2">²Several models are available at this address: https://github.com/tesseract-ocr/tessdata_best.</footnote>
Truth (GT) made of c. 30,000 lines taken from 37 prints in French of the 17th century (tab. 9). These documents have been carefully chosen so that they contain different kinds of typefaces (style, weight, size), and thus cover a maximum of the fonts possibly used for this type of document. Because graphetic variants that have now disappeared may have existed (such as the long *s*: *f*), transcription is a particularly technical act for historical documents, and we describe our transcription guidelines. Based on these data, we offer robust state-of-the art models for two open source OCR engines, both available to users *via* simple interfaces: *Kraken* [Kiessling, 2019]/*eScriptorium* [Kiessling et al., 2019] and *Calamari* [Wick et al., 2020]/*OCR4all* [Reul et al., 2019].

## II CORPUS BUILDING

Several corpora exist today for 17th century documents. We have already mentioned *GT4HistOCR* [Springmann et al., 2018], but others are available such as the *IMPACT Dataset* [Papadopoulos et al., 2013], of which 80% of the documents date however from the 19th and 20th centuries. The first corpus is mainly focusing on German<fnref n="3" />, as we previously mentioned, such as the RIDGES dataset [Springmann and Lüdeling, 2017], and the second corpus contains only about 15% of documents in French. In both cases we do not control (or know precisely) the transcription guidelines, which is an important philological problem. We have therefore decided to create our own corpus.

Producing training data in order to kick-start the creation of a generic model for 17th c. documents written in French implies the gathering of various sources, which can be selected in many ways, from piling up data from different projects to the scrupulous association of complementary sources. For our project, due to the paucity of available data, we chose to follow the second option, which is not a simple task. We therefore had to define a method to select the documents that should be included in our corpus.

<table><caption>Table 1: Distribution of the prints in the training corpus per decade</caption><thead><tr><td>Decade</td><td>Total items</td><td>Total lines</td></tr></thead><tbody><tr><td>00's</td><td>1</td><td>617</td></tr><tr><td>10's</td><td>1</td><td>198</td></tr><tr><td>20's</td><td>3</td><td>2,689</td></tr><tr><td>30's</td><td>5</td><td>3,159</td></tr><tr><td>40's</td><td>5</td><td>3,527</td></tr><tr><td>50's</td><td>3</td><td>2,008</td></tr><tr><td>60's</td><td>5</td><td>5,089</td></tr><tr><td>70's</td><td>4</td><td>3,836</td></tr><tr><td>80's</td><td>5</td><td>3,336</td></tr><tr><td>90's</td><td>5</td><td>3,709</td></tr></tbody></table>

<table><caption>Table 2: Distribution of the prints in the training corpus per genre</caption><thead><tr><td>Genre</td><td>Total</td></tr></thead><tbody><tr><td>Drama</td><td>17</td></tr><tr><td>Poetry</td><td>4</td></tr><tr><td>Novel</td><td>3</td></tr><tr><td>Letter</td><td>2</td></tr><tr><td>Philosophy</td><td>2</td></tr><tr><td>Physics</td><td>2</td></tr><tr><td>Sermon</td><td>1</td></tr><tr><td>Theology</td><td>1</td></tr><tr><td>Travel</td><td>1</td></tr><tr><td>Maxims</td><td>1</td></tr><tr><td>Medicine</td><td>1</td></tr><tr><td>Memoirs</td><td>1</td></tr><tr><td>Mechanics</td><td>1</td></tr></tbody></table>

<footnote n="3">³It is important to note that a large part of the German-written corpora are not only printed in *antiqua* but in *fraktur*, which considerably minimises their interest for the OCRisation of documents in French.</footnote>
Since the advent of corpus pragmatics, linguists have been working on how to associate data to obtain representativeness, i.e. “the extent to which a sample includes the full range of variability in a population” [Biber, 1993], but such a notion is now more and more debated [Raineri and Debras, 2019]. Following the example of corpus linguists using extralinguistic criteria (socio-logical, demographic...) [Crowdy, 1993], we have decided to select samples mostly according to bibliographical metadata (printing date and place, literary genre, author...), which serve as a proxy for paleographical information – a good diachronic distribution should for instance ensure a correct representation of the very diverse typographical material (e.g. fig. 1). We also took into account digital information (size and resolution of the images), in addition, of course, to a careful philological analysis of the documents.

<figure><img src="image_3.png" /><figcaption>(a) Bossuet, *Oraison*, 1683</figcaption></figure>

<caption>(b) Racine, *Oeuvres*, 1676</caption>

<caption>Figure 1: Mixing styles, heights and casing in 17th c. French prints</caption>

Prints production dates are distributed over the century, with a special attention for books printed between 1620 and 1700 (tab. 1) because it covers one of the most important periods in the literary history of France, that of classical French. Regarding genre, the result can be seen as a two-tier corpus (tab. 2), with a primary one consisting of literary texts (drama, poetry, novels...) and a secondary one made of scientific works (medicine, mechanics, physics...). If the vast majority has been printed in Paris, we have also included books coming from Belgium (Brussels) and Holland (Leiden), which were major production centres at the time [van Eeghen, 1960–1978, Eisenstein, 1992]<fnref n="4" />.

<figure><img src="image_4.png" /><figcaption>Figure 2: Tristan L'Hermite, *Panthée*, 1639</figcaption></figure>

<table><caption>Table 3: Percentage of uppercase letters in dramatic texts vs. non-dramatic texts in our dataset.</caption><thead><tr><th></th><th>Lower</th><th>Upper</th><th>Total</th></tr></thead><tbody><tr><td>Dramatic texts</td><td>396,984<br>90.17%</td><td>43,295<br>9.83%</td><td>440,279</td></tr><tr><td>Non-dramatic Texts</td><td>297,527<br>95.96%</td><td>12,544<br>4.04%</td><td>310,071</td></tr></tbody></table>

As we can see, the corpus is not balanced, since not only literary texts, but also plays are clearly

<footnote n="4">⁴A detailed list of the contents of the corpus can be found in the appendix (cf. tab. 9).</footnote>
over-represented. Such a choice has been made for two reasons. On the one hand, we need GT in italics, and since versified texts use this type of style a lot [Speyer, 2019], we increase the amount of data in italics by selecting plays in verse. On the other hand, we must also add capital letters in the GT, and therefore find examples of this type of character: plays, once again, are an abundant source with the names of the speakers written in capitals (fig. 2 and tab. 3). Such a strategy should help us deal with highly complex layouts (fig. 1).

Regarding the resolution, images used can be divided into three classes: 72 (20 prints), 400 (14 prints) and 600 dpi (1 print) (fig. 3). Indeed, many scans available online are in low resolutions (usually 72 dpi, an older computer standard introduced by Apple in the 1980s), which introduces significant changes in the shape of letters (fig. 4) that our model needs to handle properly.

pas la leule faueur qui met

(a) Balzac, *Lettres*, 1624, 72 dpi

ures auec l'eau du petit tuyau qui

(b) Pascal, *Equilibre*, 1663, 400 dpi

de verre ou autre matiere, dans

(c) Papin, *Manière*, 1682, 600 dpi

<figure><img src="image_5.png" /><figcaption>Figure 4: Impact of the resolution on the letter *e*, which can be confused with *c* because of the disappearance of the eye.</figcaption></figure>

### III TRANSCRIPTION GUIDELINES

**Theoretical background.** Transcription is a very delicate matter: more than copying, transcribing has to be understood as an act of translation [Robinson and Solopova, 1993] and, as the saying goes, traduttore, traditore. Following Robinson and Solopova, there are four different levels of transcription, rearranged into two categories by D. Stutzmann [Stutzmann, 2011]: those that describe the image (called “graphic” and “allographetic” transcriptions) and those that describe the text (called “graphemic” and “regularised”). For our project, we exclude the two most extreme types of the spectrum: graphic transcription (which retains all the visual richness of the original) would be far too time-consuming, and regularised transcription (which fully aligns the spelling to a standard) would be linguistically too poor. Only the allographetic and the graphemic transcriptions will therefore interest us.

The allographetic transcription aims at keeping all the graphetic richness: it reduces all the graphic variants to extended types, and thus gives access to various forms of each letter or sign. For instance, the distinction between <ſ> and 
 is kept because they are two graphetic variants of 
, but not the distinction between <ſ> with or without a leftward swash, because they both would be graphic variations of the <ſ>. Long impossible for material reasons, such transcriptions are now accessible to all researchers thanks to projects such as the Medieval Unicode Font Initiative (MUFI, Haugen [2015]), that have played an important role by designing and pushing new
Unicode code points to the Unicode standard [Consortium, 2019]<fnref n="5" />.

The graphemic transcription further limits variation by reducing the different possible types to their meaning in the alphabetic system. Unlike the allographetic approach, graphemic transcription is better known to philologists because it resembles the traditional semi-diplomatic transcription. Scholars have clarified its execution framework, and have been using it in their editions for a long time. In practice, words like *eltoit* are transposed as *estoit* in the edited version, but not as *était* which would be a regularised transcription. A few points of detail remain however debated, in particular concerning the need to expand the abbreviations when adopting a graphemic approach<fnref n="6" />.

**Project framework.** As our intent for our data is to produce OCR models and to transcribe automatically print to do more research on the French language used in these prints, our transcription guidelines lies on the side of the graphemic transcription, without regularisation. However, we introduced few graphetic concerns, listed below, because graphetic variations can be a linguistic evidence, such as the long *s*, which inform us of the content of the type boxes used by printers<fnref n="7" />. The result is a hypbrid graphemic-allographetic transcription with very punctual regularisations.

<figure><img src="image_6.png" /></figure>

fon interprete & Protecole en fes Eſ-
cripts; (s'il n'eſt là meſme celuy de So-
crates, fon plus diuin Precepteur) leur

<caption>Figure 5: Excerpt and transcription example of Marie de Gournay, *Egalité*, 1622</caption>

**Normalisation of spelling.** Our choice leads us to keep the original spelling of the source (e.g. fig. 5). We include in spelling the absence of normalisation for letters such as <u>/<v> and <i>/<j>, whose usage was different from the current one (no consonant/vowel distinction): e.g., we transcribe *diuin* where one would normalise it as *divin*. We respect accent absence (e.g. *interprete* and not *interprète*), but we transcribe dotless i (<i>) as <i>, as it is in many cases a printing problem. To avoid confusion for the machine, commas used as cedillas (*FRANC*, *OIS* and not *FRANÇOIS*) and apostrophes used as accents (*ARME'ES* and not *ARMÉES*) are kept as they are, and not regularised as accents or cedillas. Historical spellings (e.g. *Élscripts*, normalised *Écrits*) and calligraphic letters (e.g. *celuy*, normalised *celui*) are kept.

**Variation of letters and ligatures.** As mentioned, we keep one allographetic variation: the long *s* (e.g. *mefme* and not *mesme*). Other variations are ignored. Aesthetic ligatures that still exist in French (e.g. <œ> vs <oe>) have been encoded, but not those that have disappeared despite their possible existence in Unicode (e.g. <ৎ>)<fnref n="8" />. Examples are provided in Table 4⁹.

<footnote n="5">⁵In certain cases, characters which were not accepted (yet) by the Unicode governing bodies might be stored in the private zone of Unicode, being only supporter with MUFI-related font as a consequence.</footnote>

<footnote n="6">⁶The *Conseils pour l'édition des textes médiévaux* (which are a reference for rigorous philological editing of texts, medieval or not) suggest the expansion of abbreviations [Bourgain and Vieillard, 2001, p. 61] and the absence of expansion in graphemic transcriptions is presented as a “hybrid” practice by D. Stutzmann [Stutzmann, 2011, p. 251]. However, HTR data production and edition should be seen as two different tasks, and moreover, transcription and abbreviation resolution should be seen as two different computational tasks.</footnote>

<footnote n="7">⁷The use of ligatures, often involving a long *s*, has slowed down the use of accents [Biedermann-Pasques, 1992, p. 92].</footnote>

<footnote n="8">⁸Not all ligatures are present in the unicode standard or in MUFI: the task would therefore have been too complicated for a very limited interest.</footnote>

<footnote n="9">⁹In the dataset, some folders are named with *mufi*: they include a richer use of unicode character. These folders weren't use for training purposes but were used to evaluate the weight of a wider transcription of allographs.</footnote>
**About spacing.** Spacing is a problem because the compositor can “pack” the words so that they all fit into the line space. It is therefore typographical information that must be treated with care, but in most cases we follow the graphemic approach, which tends to distinguish units grammatically rather than graphically, while retaining period peculiarities.

IV EXPERIMENTS

**General set-up.** In order to train and evaluate models, we use a regular 80% of the produced dataset for training, 10% for development purposes and 10% for evaluation. The split is produced at the level of each print, resulting *de facto* as a *in-domain* test.

<table><caption>Table 4: Main transcription choices</caption><thead><tr><th>Category</th><th>Description</th><th>Status</th><th>Transcription</th><th>Example</th></tr></thead><tbody><tr><td>Ligature</td><td>Ligature O+E &lang;œ&rang;</td><td>Graphetic</td><td>U+0153/U+0152</td><td>cœur</td></tr><tr><td>Ligature</td><td>Ligature A+E &lang;æ&rang;</td><td>Graphetic</td><td>U+00E6/U+00C6</td><td>Cæ- Ægytte</td></tr><tr><td>Ligature</td><td>Ligature long S+T &lang;ft&rang;</td><td>Graphemic</td><td>No ligature</td><td>Chrestienté.</td></tr><tr><td>Ligature</td><td>Ligature L+L &lang;ll&rang;</td><td>Graphemic</td><td>No ligature</td><td>eternelle</td></tr><tr><td>Ligature</td><td>Ligature C+T &lang;ct&rang;</td><td>Graphemic</td><td>No ligature</td><td>Edict?</td></tr><tr><td>Ligature</td><td>Ligature S+P &lang;sp&rang;</td><td>Graphemic</td><td>No ligature</td><td>l'esprit</td></tr><tr><td>Ligature</td><td>Ligature long S+L &lang;fl&rang;</td><td>Graphemic</td><td>No ligature</td><td>meulent</td></tr><tr><td>Ligature</td><td>Ligature U+S &lang;us&rang;</td><td>Graphemic</td><td>No ligature</td><td>induidus</td></tr><tr><td>Ligature</td><td>Ligature S+I &lang;fi&rang;</td><td>Graphemic</td><td>No ligature</td><td>quafi</td></tr><tr><td>Ligature</td><td>Ligature long S+long S &lang;ff&rang;</td><td>Graphemic</td><td>No ligature</td><td>jeuneffé</td></tr><tr><td>Ligature</td><td>Ligature F+F+I &lang;ffi&rang;</td><td>Graphemic</td><td>No ligature</td><td>luffifance</td></tr><tr><td>Ligature</td><td>Ligature I+S &lang;is&rang;</td><td>Graphemic</td><td>No ligature</td><td>puis</td></tr><tr><td>Allograph</td><td>Capital E</td><td>Graphemic</td><td>U+0045</td><td>Et Et</td></tr><tr><td>Allograph</td><td>Capital A</td><td>Graphemic</td><td>U+0041</td><td>A Au</td></tr><tr><td>Allograph</td><td>Capital M</td><td>Graphemic</td><td>U+004D</td><td>Moy Mais Mais</td></tr><tr><td>Allograph</td><td>Small E with long finial</td><td>Graphemic</td><td>U+0065</td><td>espece</td></tr><tr><td>Allograph</td><td>Tittle as tilde or dot</td><td>Regularised</td><td></td><td>je jugé</td></tr><tr><td>Allograph</td><td>Small long and short S</td><td>Graphetic</td><td>U+017F</td><td>fans</td></tr><tr><td>Abbreviation</td><td>Combining tilde &lang;&Oslash;&rang;</td><td>Graphetic</td><td>U+0303</td><td>d'autât deuös</td></tr><tr><td>Abbreviation</td><td>Combining Macron &lang;&Oslash;&rang;</td><td>Graphemic</td><td>U+0303</td><td>vâterie</td></tr><tr><td>Abbreviation</td><td>Ampersand &lang;&&;&rang;</td><td>Graphetic</td><td>U+0026</td><td>&amp;</td></tr><tr><td>Diacritics</td><td>Combining vertical line &lang;&Oslash;&Oslash;&rang;</td><td>Regularised</td><td></td><td>a</td></tr><tr><td>Diacritics</td><td>Apostrophe</td><td>Graphetic</td><td>U+0027</td><td>ARMEÉS</td></tr><tr><td>Diacritics</td><td>Comma</td><td>Graphetic</td><td>U+002C</td><td>FRANC, OIS</td></tr><tr><td>Hyphenation</td><td>Hyphen</td><td>Codified &lang;&rarr;&rang;</td><td>U+00AC</td><td>foigncu- imbecile, di-</td></tr></tbody></table>
We additionally produced 4 others small samples for out-of-domain testing based on centuries, from the 16th c. to the 19th (cf. tab. 10,11, 12 for details, tab. 5 otherwise). We specifically designed these out-of-domain samples to exclude gothic and other special fonts such as *civilités* ones (cf. fig. 6), as our training corpora only include roman or italic typefaces.

<table><caption>Table 5: Description of test sets, character counts are in NFC.</caption><thead><tr><td>Dataset</td><td>Caracters</td></tr></thead><tbody><tr><td>17th c.</td><td>91,104</td></tr><tr><td>16th c.</td><td>18,542</td></tr><tr><td>18th c.</td><td>16,691</td></tr><tr><td>19th c.</td><td>13,103</td></tr></tbody></table>

<figure><img src="image_7.png" /><figcaption>Figure 6: Non-selected typefaces</figcaption></figure>

Two separate open-source OCR engines are used for training OCR models, namely *Kraken* [Kiessling, 2019] and *Calamari* [Wick et al., 2020]. Both tools were used in order to leverage their various differences in order to produce the best model possible. Default engine model architecture as well as hyperparameters were used for the baseline model.

**Kraken Experiment: artificial lines vs. synthetic data.** Texts were normalised using unicode's decomposition normalisation (NFD). This results in splitting characters such as «é» into two characters «e» + «¢» (combining acute accent, U+0301). This has become in the French DH community of Kraken the *de facto* choice for French language OCR.

To improve the efficiency of the engine, two additional experiments have made. On the one hand, we tested a larger model architecture than the base one<fnref n="10" />, doubling the filter size of each convolutional layer, respectively from 32 to 64 and from 64 to 128, to handle the heterogeneity of the training data. On the other hand, we used a synthetic training set on top of the manually compiled one with 27 different fonts<fnref n="11" />.

**Calamari Experiment: Multiple voters and data augmentation.** Regarding *Calamari*, we have tested another type of unicode normalisation (NFC) making sure that diacritics are combined<fnref n="12" />. We replicated here the successful protocol from Reul et al. [2018] by combining model fine-tuning (FT) – *i.e.* building from existing models (historical non-French *Antiqua*) instead of starting the training from scratch –, voting (VT) – *i.e.* training five models instead of one and combining their outputs during predictions –, and data augmentation (DA), – *i.e.* generating modified images of the input lines by blurring them, stretching them, etc.

<footnote n="10">¹⁰Base VGSL architecture of Kraken recognition model: [1, 48, 0, 1 Cr3, 3, 32 Do0.1, 2 Mp2, 2 Cr3, 3, 64 Do0.1, 2 Mp2, 2 S1(1x0)1, 3 Lbx100 Do].</footnote>

<footnote n="11">¹¹Namely: IM FELL English SC, IM FELL English, IM FELL Great Primer, IM FELL Double Pica, IM FELL Double Pica SC, IM FELL DW Pica, 1592 GLC Garamond, 1689GLCGaramondW00SC-Norm, Garamond, EB Garamond, EB Garamond 12 All SC, 1689 Almanach, Fournier MT Std, Bodoni 72 Oldstyle, Didot, Chapbook, DTLElzevirS, DTL Elzevir, P22 Operina Romano, Hultog, JSL Ancient, Old Claude LP Std, Chapbook, 1756DutchW01-Normal, 1726RealEspanolaW01-Rg, 1776_Independence, Palatino.</footnote>

<footnote n="12">¹²As a result, score between both engines are not comparable, as they do not use the same unicode normalisation which results in a different number of evaluated characters.</footnote>
# V ANALYSIS

<table><caption>Table 6: Accuracy (1-CER) for the experiment with *Kraken*.</caption><thead><tr><th>Augm. Architect.</th><th>Artif. Data</th><th>17th c.</th><th>16th c.</th><th>18th c.</th><th>19th c.</th></tr></thead><tbody><tr><td>-</td><td>-</td><td>97.47%</td><td>97.74%</td><td><strong>97.78%</strong></td><td>94.50%</td></tr><tr><td>Yes</td><td>-</td><td><strong>97.92%</strong></td><td><strong>98.06%</strong></td><td><strong>97.78%</strong></td><td>94.23%</td></tr><tr><td>-</td><td>Yes</td><td>96.65%</td><td>97.26%</td><td>97.74%</td><td><strong>95.50%</strong></td></tr><tr><td>Yes</td><td>Yes</td><td>97.26%</td><td>97.68%</td><td>97.84%</td><td>94.84%</td></tr></tbody></table>

<table><caption>Table 7: Accuracy (1-CER) for the experiment with *Calamari*.</caption><thead><tr><th>Pretrain.</th><th>Voters</th><th>Data Augm.</th><th>17th c.</th><th>16th c.</th><th>18th c.</th><th>19th c.</th></tr></thead><tbody><tr><td>-</td><td>1</td><td>-</td><td>98.47%</td><td>98.14%</td><td>98.27%</td><td>93.11%</td></tr><tr><td>Yes</td><td>1</td><td>Yes</td><td>98.76%</td><td>98.49%</td><td>96.47%</td><td><strong>97.05%</strong></td></tr><tr><td>Yes</td><td>5</td><td>Yes</td><td><strong>99.05%</strong></td><td><strong>98.68%</strong></td><td><strong>98.78%</strong></td><td><strong>97.05%</strong></td></tr></tbody></table>

Considering the (deliberately) extreme heterogeneity of our data, such scores are promising (cf. tab. 6 &7). However, it is clear that, regarding Kraken, synthetic data did not improve results at all, except for 19th c., and might actually in some cases lowered the score (specifically for the in-domain test). Kraken however benefited from a larger model, and this change impacted also out of-domain results except from later one (18th and 19th c.). *Calamari* shows again that the protocol from Reul et al. [2018] is beneficial to the results and incremental (multiple voters enhance the results of the already better ones from data augmentation and pretraining).

Despite being focused on the 17th c., the dataset is able to produce model resistant to changes in neighbouring centuries. We see that in both case, the accuracy drops by less than one percentage point. This is definitely due to the filtering of gothic fonts and special typefaces of the 16th c. prints, but also to the limited changes in common typefaces between these centuries. As for 19th century, the score dropped more for Kraken (-2 to -3 percentage points) than for Calamari (-1.7 points) for their best performing models on other centuries. Only the use of artificial data allowed for performance gains on 19th c. for Kraken, most probably due to the regularity provided by them.

Confusion table from *Calamari* (cf. tab. 14, 13, 15, 16, 17) shows an important issue with spacing recognition, and as such, word segmentation. This could be linked to both the density of the composition in early modern prints or because of the instability of the graphic segmentation, some words being sometimes welded (*puisque*) and sometimes not (*puis que*, cf. tab. 8).

<table><caption>Table 8: Possible sources of word segmentation errors. The token SPACE indicates the problematic zone.</caption><thead><tr><th>Spacing error</th><th>Example</th><th>Transcription</th></tr></thead><tbody><tr><td>Composition</td><td>
lemēt
, qu'il oſe pretendre,</td><td>lemēnt, qu'il oſe [NO SPACE] pretendre</td></tr><tr><td>Graphic segmentation</td><td>
le trióphe
; mais puis que ce</td><td>le trióphe; mais puis [SPACE] que ce</td></tr></tbody></table>

Another important source of error is linked to the <f>, which, once again, can be linked to pagegraphic problems (confusion <f>/
). The confusion <f>/
 might be related to the language model overtaking the OCR, or more simply, input errors on the side of the GT<fnref n="13" />.

Both issues can be treated with post-processing steps. Segmentation or <f>/
 confusion can be approached as a character classification from a pure natural language processing point of view, as shown per Clérice [2020]. In this paper, content were encoded at the character level with a per-character binary classification (word boundary vs. in-word content) which resulted in very high

<footnote n="13">¹³As a reviewer kindly said, “muscle memory” is sometimes quite strong.</footnote>
accuracy. The same process could be applied to both type of confusions. However, regarding the <f>/
 confusion, another option would be to drop the differentiation of the allographs at training and testing time, until enough GT has been produced to avoid this kind of issues.

# VI FUTURE WORK

The diachronic efficiency of the model can be improved by adding data for more recent prints: c. 20,000 additional lines will be added, to carry further tests on the creation of a model for French prints in general, and not only modern prints. Out-of-domain tests sets composed of non-francophone prints should also be created to test the efficiency of the model on similar prints in other languages.

While creating the GT, we have corrected the layout of each image. Alto and PageXML will be used to train a segmenter, the importance of which must not be underestimated since it is on its result that the OCR is performed.

# AUTHOR CONTRIBUTIONS

S.G. designed the research project, built the corpus and prepared the data for training. T.C. helped all along the process providing advice, scripts and feedback. C. R. performed the experiments with *Calamari*. All authors discussed the results and contributed to the final manuscript.

# DATA

Training data is available online (10.5281/zenodo.3826894): it contains all the GT used to train models, and it is distributed with a CC-BY licence. Ongoing research on OCR, with additional data and scripts, is available on Github (https://github.com/e-ditiones/OCR17).

# ACKNOWLEDGMENTS

This paper would not have been possible without the help of J.-B. Camps (PSL-ENC).

# ADDENDUM

This article has been originally written and submitted in 2020, and it was corrected in 2023. While we received reviews early, we were not able to complete the correction proposed in due time. We still think that this paper paints an interesting state of OCR and HTR at the time.

However, since then, tools have evolved, and the dataset has evolved has well. It became the OCR17+ dataset<fnref n="14" />, using ALTO-XML representation instead of line based segmentation. It has been largely completed since then through project such as GalliCorpora<fnref n="15" />. Kraken has since then adopted augmentation of images, and uses larger line input rather than higher convolution filters to reach better results.

# References

* Douglas Biber. Representativeness in Corpus Design. *Literary and Linguistic Computing*, 8(4):243–257, January 1993. URL https://academic.oup.com/dsh/article/8/4/243/928942.

* Liselotte Biedermann-Pasques. *Les grands courants orthographiques au XVIIe siècle et la formation de l'orthographe moderne. Impacts matériels, interférences phoniques, théories et pratiques* (1606–1736). De Gruyter, Berlin, Boston, 1992. URL https://doi.org/10.1515/9783110938593.

<footnote n="14">¹⁴https://github.com/Heresta/OCR17plus.</footnote>

<footnote n="15">¹⁵https://github.com/Gallicorpora.</footnote>
Pascale Bourgain and Françoise Vieillard. *Conseils pour l'édition des textes médiévaux*, volume 3, Textes littéraires. Ecole nationale des chartes - Comité des travaux historiques et scientifiques, Paris, 2001.

Alix Chagué, Thibault Clérice, and Floriane Chifoleau. HTR united, a centralization effort of htr and ocr ground-truth repositories for french languages, 2020. URL https://github.com/HTR-United/htr-united.

Thibault Clérice. Evaluating Deep Learning Methods for Word Segmentation of Scripta Continua Texts in Old French and Latin. *Journal of Data Mining and Digital Humanities*, 2020. URL https://hal.archives-ouvertes.fr/hal-02154122.

Unicode Consortium. Unicode 12.0.0, 2019. URL http://www.unicode.org/versions/Unicode12.0.0.

Steve Crowdy. Spoken Corpus Design. *Literary and Linguistic Computing*, 8(4):259–265, January 1993. URL https://academic.oup.com/dsh/article/8/4/259/928943.

Frédéric Duval. Les éditions de textes du XVIIe siècle. In *Manuel de la philologie de l'édition*, pages 369–394. De Gruyter, Berlin, Boston, 2015. URL https://www.degruyter.com/view/books/9783110302608/9783110302608-017/9783110302608-017.xml.

Frédéric Duval, editor. *Pratiques philologiques en Europe*. Études et rencontres. Publications de l'École nationale des chartes, Paris, September 2018. URL http://books.openedition.org/enc/692.

Elizabeth L. Eisenstein. *Grub Street Abroad: Aspects of the French Cosmopolitan Press from the Age of Louis XIV to the French Revolution*. Clarendon Press, Oxford, 1992.

Simon Gabay. Pourquoi moderniser l'orthographe? Principes d'ecdotique et littérature du XVIIe siècle. *Vox Romanica*, 73(1):27–42, 2014. URL https://elibrary.narr.digital/article/99.125005/vox201410027.

Odd Einar Haugen. *MUFI character recommendation v. 4.0*. Medieval Unicode Font Initiative, December 2015. URL https://bora.uib.no/handle/1956/10699.

Benjamin Kiessling. Kraken - an Universal Text Recognizer for the Humanities. In *Digital Humanities Conference 2019 - DH2019*, Utrecht, The Netherlands, July 2019. ADHO. URL https://dev.clariah.nl/files/dh2019/boa/0673.html.

Benjamin Kiessling, Robin Tissot, Peter Stokes, and Daniel Stökl Ben Ezra. eScriptorium: An Open Source Platform for Historical Document Analysis. In *2019 International Conference on Document Analysis and Recognition Workshops (ICDARW)*, volume 2, pages 19–19, September 2019. URL https://doi.org/10.1109/ICDARW.2019.10032.

Christos Papadopoulos, Stefan Pletschacher, Christian Clausner, and Apostolos Antonacopoulos. The impact dataset of historical document images. In *Proceedings of the 2nd International Workshop on Historical Document Imaging and Processing*, HIP '13, page 123–130, New York, NY, USA, 2013. Association for Computing Machinery. URL https://doi.org/10.1145/2501115.2501130.

Sophie Raineri and Camille Debras. Corpora and Representativeness: Where to go from now? *CogniTextes. Revue de l'Association française de linguistique cognitive*, 19, 2019. URL http://journals.openedition.org/cognitextes/1311.

Christian Reul, Uwe Springmann, Christoph Wick, and Frank Puppe. Improving OCR Accuracy on Early Printed Books by combining Pretraining, Voting, and Active Learning. *Journal for Language Technology and Computational Linguistics*, 33(1: Special Issue on Automatic Text and Layout Recognition), 2018. URL https://doi.org/10.21248/jlcl.33.2018.216.

Christian Reul, Dennis Christ, Alexander Hartelt, Nico Balbach, Maximilian Wehner, Uwe Springmann, Christoph Wick, Christine Grundig, Andreas Büttner, and Frank Puppe. OCR4all—An Open-Source Tool Providing a (Semi-)Automatic OCR Workflow for Historical Printings. *Applied Sciences*, 9(22), January 2019. URL https://www.mdpi.com/2076-3417/9/22/4853.

Peter Robinson and Elizabeth Solopova. Guidelines for Transcription of the Manuscripts of the Wife of Bath's Prologue. In *The Canterbury Tales Project Occasional Papers 1*, 1993.

Ray Smith. An overview of the tesseract ocr engine. In *Ninth International Conference on Document Analysis and Recognition (ICDAR 2007)*, volume 2, pages 629–633, Curitiba, Brazil, 2007. URL https://doi.org/10.1109/ICDAR.2007.4376991.

Miriam Speyer. Les dieux écrivent-ils en italiques ? Typographie et mise en livre de pièces en vers et en prose. *L'Habillage du livre et du texte aux XVIIe et XVIIIe siècles*, 9, 2019. URL https://hal-normandie-univ.archives-ouvertes.fr/hal-02184237.

Uwe Springmann and Anke Lüdeling. OCR of historical printings with an application to building diachronic corpora: A case study using the RIDGES herbal corpus. *Digital Humanities Quarterly*, 11-2, 2017. URL http://www.digitalhumanities.org/dhq/vol/11/2/000288/000288.html.

Uwe Springmann, Florian Fink, and Klaus U. Schulz. Automatic quality evaluation and (semi-) automatic improvement of OCR models for historical printings. *arXiv:1606.05157 [cs]*, June 2016. URL http://arxiv.org/abs/1606.05157.

Uwe Springmann, Christian Reul, Stefanie Dipper, and Johannes Baiter. Ground Truth for training OCR engines
on historical documents in German Fraktur and Early Modern Latin. *Journal for Language Technology and Computational Linguistics*, 33:97–114, 2018. URL https://arxiv.org/abs/1809.05501.

Dominique Stutzmann. Paléographie statistique pour décrire, identifier, dater... Normaliser pour coopérer et aller plus loin ? *Kodikologie und Paläo-graphie im digitalen Zeitalter = Codicology and Palaeography in the Digital Age*, 2:247–277, 2011. URL https://halshs.archives-ouvertes.fr/halshs-00596970.

Isabella H. van Eeghen. *De Amsterdamse Boekhandel 1680-1725*. Publicaties van de Gemeentelijke Archiefdienst. Scheltema & Holkema, Amsterdam, 1960–1978.

Christoph Wick, Christian Reul, and Frank Puppe. *Calamari - A High-Performance Tensorflow-based Deep Learning Package for Optical Character Recognition*. *Digital Humanities Quarterly*, 14(1), 2020. URL http://www.digitalhumanities.org/dhq/vol/14/2/000451/000451.html.
<table><caption>Table 10: Testing data, 16th c.</caption><thead><tr><th>Author</th><th>Title</th><th>Date</th><th>Place</th><th>Publisher</th><th>Lines</th><th>LigaturesDPI</th><th>Size</th><th>Library ID</th></tr></thead><tbody><tr><td>Bartas</td><td>La Sepmaine</td><td>1578</td><td>Paris</td><td>M. Gadoulleau</td><td>62</td><td>No</td><td>415</td><td>2840×3880</td><td>BNF</td><td>cb303572930</td></tr><tr><td>Beroalde</td><td>Avantures de Floride</td><td>1594</td><td>Tours</td><td>J. Mettayer</td><td>63</td><td>No</td><td>400</td><td>1174×2186</td><td>BNF</td><td>cb30092726b</td></tr><tr><td>Calvin</td><td>Institution de la religion</td><td>1562</td><td>Geneve</td><td>G. Bourgeois</td><td>54</td><td>No</td><td>400</td><td>2633×4078</td><td>BNF</td><td>cb365761545</td></tr><tr><td>Du Bellay</td><td>La Défence et illustration</td><td>1549</td><td>Paris</td><td>A. l'Angelier</td><td>58</td><td>No</td><td>400</td><td>1589×2445</td><td>BNF</td><td>cb11968311h</td></tr><tr><td>Du Fail</td><td>Discours d'Eutrapel</td><td>1585</td><td>Rennes</td><td>N. Glamet</td><td>65</td><td>No</td><td>416</td><td>1596×2576</td><td>BNF</td><td>cb30367435k</td></tr><tr><td>Rabelais</td><td>Tiers Livre</td><td>1546</td><td>Paris</td><td>Ch. Wechsel</td><td>46</td><td>No</td><td>400</td><td>1573×2647</td><td>BNF</td><td>cb31167405f</td></tr><tr><td>Ronsard</td><td>Les Amours</td><td>1552</td><td>Paris</td><td>Vve de la Porte</td><td>59</td><td>No</td><td>400</td><td>1678×2711</td><td>BNF</td><td>cb432409623</td></tr></tbody></table>

<table><caption>Table 11: Testing data, 18th c.</caption><thead><tr><th>Author</th><th>Title</th><th>Date</th><th>Place</th><th>Publisher</th><th>Lines</th><th>LigaturesDPI</th><th>Size</th><th>Library ID</th></tr></thead><tbody><tr><td>Buffon</td><td>Histoire naturelle</td><td>1750</td><td>La Haye</td><td>P. De Hondt</td><td>112</td><td>No</td><td>400</td><td>3756×4582</td><td>BNF</td><td>cb301741874</td></tr><tr><td>Laclos</td><td>De la Monarchie</td><td>1791</td><td>Paris</td><td>Impr. nationale</td><td>79</td><td>No</td><td>400</td><td>2274×3430</td><td>BNF</td><td>cb302389989</td></tr><tr><td>Diderot</td><td>Essais sur la peinture</td><td>1785</td><td>Paris</td><td>Fr. Buisson</td><td>56</td><td>No</td><td>400</td><td>1881×2903</td><td>BNF</td><td>cb44312299p</td></tr><tr><td>Marivaux</td><td>Le jeu de l'amour et du hazard</td><td>1730</td><td>Paris</td><td>Briasson</td><td>59</td><td>No</td><td>400</td><td>1440×2609</td><td>BNF</td><td>cb30886471g</td></tr><tr><td>Montesquieu</td><td>Lettres persanes</td><td>1721</td><td>Amsterdam</td><td></td><td>48</td><td>No</td><td>400</td><td>1452×2588</td><td>BNF</td><td>cb119437548</td></tr><tr><td>Rousseau</td><td>Les Pensées</td><td>1764</td><td>Amsterdam</td><td></td><td>60</td><td>No</td><td>400</td><td>1664×2904</td><td>BNF</td><td>cb31257216h</td></tr><tr><td>Voltaire</td><td>Zadig</td><td>1748</td><td>Paris/Nancy</td><td>L.-Fr. Prault/A. Leseure</td><td>44</td><td>No</td><td>600</td><td>2022×3676</td><td>BNF</td><td>cb316044160</td></tr></tbody></table>

<table><caption>Table 12: Testing data, 19th c.</caption><thead><tr><th>Author</th><th>Title</th><th>Date</th><th>Place</th><th>Publisher</th><th>Lines</th><th>LigaturesDPI</th><th>Size</th><th>Library ID</th></tr></thead><tbody><tr><td>Chateaubriand</td><td>Atala</td><td>1801</td><td>Paris</td><td>Mignereit/Dupont</td><td>41</td><td>No</td><td>600</td><td>1914×3280</td><td>BNF</td><td>cb30227639h</td></tr><tr><td>Constant</td><td>Adolphe</td><td>1816</td><td>Paris/Londres</td><td>Treutel et Würtz/H. Colburn</td><td>40</td><td>No</td><td>400</td><td>1560×2653</td><td>BNF</td><td>cb319643212</td></tr><tr><td>Flaubert</td><td>Salammbô</td><td>1863</td><td>Geneve</td><td>M. Lévy frères</td><td>61</td><td>No</td><td>400</td><td>2264×3348</td><td>BNF</td><td>cb304403988</td></tr><tr><td>Gautier</td><td>Le Roman de la momie</td><td>1858</td><td>Paris</td><td>L. Hachette</td><td>57</td><td>No</td><td>400</td><td>1605×2739</td><td>BNF</td><td>cb30490246s</td></tr><tr><td>Hugo</td><td>Odes</td><td>1823</td><td>Paris</td><td>Persan/Pélicier</td><td>42</td><td>No</td><td>600</td><td>2208×3656</td><td>BNF</td><td>cb32263200h</td></tr><tr><td>Musset</td><td>A quoi rêvent les jeunes filles</td><td>1833</td><td>Paris</td><td>E. Renduel</td><td>59</td><td>No</td><td>400</td><td>1858×3193</td><td>BNF</td><td>cb30999539c</td></tr><tr><td>Nerval</td><td>Sènes de la vie orientale</td><td>1848</td><td>Paris</td><td>F. Sartorius</td><td>53</td><td>No</td><td>400</td><td>2300×3643</td><td>BNF</td><td>cb32482331k</td></tr></tbody></table>
<table><caption>Table 13: Confusion table for the best Calamari models, in-domain test, 17th c. prints</caption><thead><tr><th>GT</th><th>PRED</th><th>COUNT</th><th>PERCENT</th></tr></thead><tbody><tr><td>{}</td><td>{}</td><td>39</td><td>4.36%</td></tr><tr><td>{é}</td><td>{e}</td><td>22</td><td>2.46%</td></tr><tr><td>{,}</td><td>{,}</td><td>22</td><td>2.46%</td></tr><tr><td>{1}</td><td>{}</td><td>21</td><td>2.35%</td></tr><tr><td>{f}</td><td>{f}</td><td>20</td><td>2.23%</td></tr><tr><td>{}</td><td>{}</td><td>19</td><td>2.12%</td></tr><tr><td>{i}</td><td>{}</td><td>14</td><td>1.56%</td></tr><tr><td>{t}</td><td>{}</td><td>13</td><td>1.45%</td></tr><tr><td>{'}</td><td>{}</td><td>12</td><td>1.34%</td></tr><tr><td>{c}</td><td>{e}</td><td>10</td><td>1.12%</td></tr></tbody></table>

<table><caption>Table 16: Confusion table for the best Calamari models, out-of-domain test, 19th c. prints</caption><thead><tr><th>GT</th><th>PRED</th><th>COUNT</th><th>PERCENT</th></tr></thead><tbody><tr><td>{}</td><td>{}</td><td>51</td><td>13.18%</td></tr><tr><td>{t}</td><td>{1}</td><td>32</td><td>8.27%</td></tr><tr><td>{f}</td><td>{f}</td><td>21</td><td>5.43%</td></tr><tr><td>{»}</td><td>{n}</td><td>19</td><td>4.91%</td></tr><tr><td>{è}</td><td>{é}</td><td>18</td><td>4.65%</td></tr><tr><td>{è}</td><td>{e}</td><td>13</td><td>3.36%</td></tr><tr><td>{e}</td><td>{o}</td><td>12</td><td>3.10%</td></tr><tr><td>{—}</td><td>{}</td><td>11</td><td>5.68%</td></tr><tr><td>{c}</td><td>{e}</td><td>8</td><td>2.07%</td></tr><tr><td>{»}</td><td>{}</td><td>7</td><td>3.62%</td></tr></tbody></table>

<table><caption>Table 14: Confusion table for the best Calamari models, out-of-domain test, 16th c. prints</caption><thead><tr><th>GT</th><th>PRED</th><th>COUNT</th><th>PERCENT</th></tr></thead><tbody><tr><td>{}</td><td>{}</td><td>33</td><td>12.69%</td></tr><tr><td>{'}</td><td>{}</td><td>14</td><td>5.38%</td></tr><tr><td>{f}</td><td>{f}</td><td>11</td><td>4.23%</td></tr><tr><td>{}</td><td>{}</td><td>11</td><td>4.23%</td></tr><tr><td>{é}</td><td>{e}</td><td>5</td><td>1.92%</td></tr><tr><td>{s}</td><td>{f}</td><td>5</td><td>1.92%</td></tr><tr><td>{,}</td><td>{}</td><td>5</td><td>1.92%</td></tr><tr><td>{e}</td><td>{é}</td><td>5</td><td>1.92%</td></tr><tr><td>{.}</td><td>{}</td><td>4</td><td>1.54%</td></tr><tr><td>{a}</td><td>{à}</td><td>3</td><td>1.15%</td></tr></tbody></table>

<table><caption>Table 15: Confusion table for the best Calamari models, out-of-domain test, 18th c. prints</caption><thead><tr><th>GT</th><th>PRED</th><th>COUNT</th><th>PERCENT</th></tr></thead><tbody><tr><td>{}</td><td>{}</td><td>60</td><td>25.75%</td></tr><tr><td>{s}</td><td>{f}</td><td>14</td><td>6.01%</td></tr><tr><td>{è}</td><td>{e}</td><td>9</td><td>3.86%</td></tr><tr><td>{è}</td><td>{é}</td><td>7</td><td>3.00%</td></tr><tr><td>{u}</td><td>{n}</td><td>4</td><td>1.72%</td></tr><tr><td>{c}</td><td>{e}</td><td>4</td><td>1.72%</td></tr><tr><td>{è}</td><td>{ê}</td><td>4</td><td>1.72%</td></tr><tr><td>{f}</td><td>{f}</td><td>4</td><td>1.72%</td></tr><tr><td>{-}</td><td>{-}</td><td>3</td><td>1.29%</td></tr><tr><td>{à}</td><td>{a}</td><td>3</td><td>1.29%</td></tr></tbody></table>

<table><caption>Table 17: Confusion table for the best Calamari models, out-of-domain test, 16th c., 18th c., 19th c. prints</caption><thead><tr><th>GT</th><th>PRED</th><th>COUNT</th><th>PERCENT</th></tr></thead><tbody><tr><td>{}</td><td>{}</td><td>144</td><td>16.36%</td></tr><tr><td>{f}</td><td>{f}</td><td>36</td><td>4.09%</td></tr><tr><td>{t}</td><td>{1}</td><td>33</td><td>3.75%</td></tr><tr><td>{è}</td><td>{é}</td><td>25</td><td>2.84%</td></tr><tr><td>{è}</td><td>{e}</td><td>23</td><td>2.61%</td></tr><tr><td>{»}</td><td>{n}</td><td>20</td><td>2.27%</td></tr><tr><td>{s}</td><td>{f}</td><td>19</td><td>2.16%</td></tr><tr><td>{}</td><td>{}</td><td>18</td><td>2.05%</td></tr><tr><td>{'}</td><td>{}</td><td>14</td><td>1.59%</td></tr><tr><td>{e}</td><td>{o}</td><td>12</td><td>1.36%</td></tr></tbody></table>