# Initiation to Handwritten Text Recognition

with eScriptorium

Alix Chagué - alix.chague@inria.fr
(Inria; UdeM; EPHE)

Naples - April 16, 2024

Horizons of digital philology
# Creating data and training a model

<figure><img src="image_1.png" /></figure>
# Steps to apply HTR

## 1. Retrieve images

<figure><img src="image_2.png" /></figure>
# Steps to apply HTR

1. Retrieve images

2. Process images (optional)

<figure><img src="image_3.png" /></figure>
# Steps to apply HTR

1. Retrieve images

2. Process images (optional)

3. Detect the layout of the image

a. can be project specific

b. can be generic or very detailed

<figure><img src="image_4.png" /></figure>
# Steps to apply HTR

1. Retrieve images

2. Process images (optional)

3. Detect the layout of the image

a. can be project specific

b. can be generic or very detailed

4. Detect text lines ("segment") and line order

<figure><img src="image_5.png" /></figure>
# Steps to apply HTR

1. Retrieve images

2. Process images (optional)

3. Detect the layout of the image

a. can be project specific

b. can be generic or very detailed

4. Detect text lines ("segment") and line order

5. Predict the text in the lines

<figure><img src="image_6.png" /></figure>
# Steps to apply HTR

1. Retrieve images

2. Process images (optional)

3. Detect the layout of the image

a. can be project specific

b. can be generic or very detailed

4. Detect text lines ("segment") and line order

5. Predict the text in the lines

6. Export the data (XML ALTO, XML PAGE, TXT...)

<Layout>

<Page WIDTH="4284" HEIGHT="2772" PHYSICALImgNR="9" ID="eSc_dummypage_">

<PrintSpace HPOS="0" VPOS="0" WIDTH="4284" HEIGHT="2772">

<TextBlock HPOS="2414" VPOS="1133" WIDTH="1702" HEIGHT="1364" ID="eSc_textblock_074cf1c7" TAGREFS="BT11982">

<Shape>

<Polygon POINTS="2414 1244 2414 2497 4112 2497 4116 1244 ... 1244" />

</Shape>

<TextLine ID="eSc_line_7588489e" TAGREFS="LT407" BASELINE="2754 1346 3177 1342" HPOS="2749" VPOS="1182" WIDTH="423" HEIGHT="183">

<Shape>

<Polygon POINTS="2752 1328 2749 1221 2788 1235 2837 1235 ... 1358" />

</Shape>

<String CONTENT="Août" HPOS="2749" VPOS="1182" WIDTH="423" HEIGHT="183" />

</TextLine>
# The data you create can be used by others!

## Gold Data

Even if you don't train a model yourself, you can create training data for HTR!

1. define a portion of your corpus (even just 5-10 pages)

2. pay special attention to the correctness of their annotation (text and layout)

3. document the transcription rules you followed

4. publish it with appropriate licencing and credits

5. reference it in HTR-United

It doesn't have to take more than one day of work and it can be very useful to others!
<figure><img src="image_7.png" /></figure>

# Software environment

<figure><img src="image_8.png" /></figure>
# Kraken

- Python-based software to train and apply

- transcription models

- layout detection models

- (more since Kraken 5.0.0?)

- Developed by Ben Kiessling

- Available in command line, as a Python module, or
  through eScriptorium

- Documentation: https://kraken.re

- Source code: https://github.com/mittagessen/kraken

<figure><img src="image_9.png" /></figure>
eScriptorium

- Web application available as a free software in open source

- Manage transcription campaign (train/apply Kraken models, load documents, manually annotate documents, team work, etc)

- Developed by the SCRIPTA PSL project

- Requires a server (locally emulated with Docker or a public server)

- Documentation: https://escriptorium.readthedocs.io/

- Source code: https://gitlab.com/scripta/escriptorium

<figure><img src="image_10.png" /></figure>
# What do you need to use eScriptorium?

- install a local instance or obtain an access (account) to an existing eScriptorium server

- have digitizations of the documents you wish to transcribe

- local files (`.png`, `.jpeg`, `.pdf`)

.tif are sometimes too heavy

- files stored by an institution on a IIIF server

- then you need a link to a IIIF manifest:

https://gallica.bnf.fr/iiif/ark:/12148/btv1b84259980/manifest.json
# Demo of eScriptorium

- https://escriptorium.inria.fr

- login:

- password:
<figure><img src="image_11.png" /></figure>
# Transkribus

- Software as a service relying on some open source contents (like PyLaia)

- Desktop application (not sustained anymore) and web application

- Other services in the ecosystem (publication, querying, etc)

- Several changes in the business plan over the past few years

- https://www.transkribus.org/plans

- Developed and maintained by READ-COOP

- Allow to train/apply transcription models, apply segmentation models

- Does not allow exporting models