# Scaling up Automatic Structuring of Manuscript Sales Catalogues

Lucie Rondeau Du Noyer, Simon Gabay, Mohamed Khemakhem, Laurent Romary

► To cite this version:

Lucie Rondeau Du Noyer, Simon Gabay, Mohamed Khemakhem, Laurent Romary. Scaling up Automatic Structuring of Manuscript Sales Catalogues. TEI 2019: What is text, really? TEI and beyond, Sep 2019, Graz, Austria. hal-02272962

HAL Id: hal-02272962

https://inria.hal.science/hal-02272962

Submitted on 28 Aug 2019

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Scaling up Automatic Structuring of
Manuscript Sales Catalogues

**Lucie Rondeau du Noyer**

{surname.name@chartes.psl.eu}

Ecole des Chartes, Paris

**Simon Gabay**

{surname.name@unine.ch}

Université de Neuchâtel

**Mohamed Khemakhem**

{name.surname@inria.fr}

Inria, team ALMAnaCH, Paris

Centre Marc Bloch, Berlin

Université Paris Diderot, Paris

**Laurent Romary**

{name.surname@inria.fr}

Inria, team ALMAnaCH

Keywords: Machine learning, manuscript sales catalogues, 19th c. France

Manuscript Sales Catalogues (MSC) are highly important for authenticating documents and studying the reception of authors. Their regular publication throughout Europe since the beginning of the 19th c. has consequently raised the interest around scaling up the means for automatically structuring their contents.

Following successful first encoding tests with *GROBID-Dictionaries* [1,2] on a single MSC collection [3], we aim in this paper to present the results of more advanced tests of the system's capacity to handle a larger corpus with MSC of different dealers, and therefore multiple layouts.

## Corpus

Four different types of catalogues published between the middle of the 19th c. and the beginning of the 20th c. have been tested.

<figure><img src="image_2.png" /><figcaption>**Figure 1 - Type 1: Revue des autographes**, Gabriel Charavay. (Première série N°42, Decembre 1874)</figcaption></figure>
<figure><img src="image_3.png" /><figcaption>**Figure 2 - Type 2:** *Revue des autographes, des curiosités de l'histoire et de la biographie, Gabrielle Charavay (Seconde série N°56, 1934)*</figcaption></figure>

<figure><img src="image_4.png" /><figcaption>**Figure 3 - Type 3:** *Catalogue de lettres autographes et manuscrits, Auguste Laverdet (N°1, April 1856.)*</figcaption></figure>

<figure><img src="image_5.png" /><figcaption>**Figure 4 - Type 4:** *Catalogue d'une intéressante collection de lettres autographes..., Etienne Charavay (December, 14th 1908)*</figcaption></figure>

## Experiment

To parse the presented MSC corpus we followed the same encoding presented in earlier experiments [3]. We tried then to focus our experiments on two aspects: feature engineering and cumulative samples training.

For the former we tested tuning the GROBID models at three levels of segmentation following two variations: unigram and bigram features. The difference between the two categories is that a label predicted by a trained model is based only on the features of the input token - case of unigram - where a bigram feature template takes also the label of the previous token into consideration.
For the second experimenting aspect we tested the performance of the system on models trained separately for each layout and a general model with all the data.

To that end we used 10 annotated pages for training and 5 others for evaluation, chosen to be representative of each series of catalogues. [4]

<table><caption>Table 1: All fields F1-score of **Unigram Feature Templates**</caption><thead><tr><th rowspan="2">MSC</th><th colspan="3">GROBID Models</th></tr><tr><th>Lexical Entry</th><th>Form</th><th>Sense</th></tr></thead><tbody><tr><td>Type 1</td><td>72.49</td><td>93.72</td><td>73.62</td></tr><tr><td>Type 2</td><td>57.07</td><td>84.42</td><td>60.5</td></tr><tr><td>Type 3</td><td>60.07</td><td>74.71</td><td>48.07</td></tr><tr><td>Type 4</td><td>60.58</td><td>92.03</td><td>40.91</td></tr><tr><td>All types mixed</td><td>63.99</td><td>86.42</td><td>54.81</td></tr></tbody></table>

<table><caption>Table 2: All fields F1-score of **Bigram Feature Templates**</caption><thead><tr><th rowspan="2">MSC</th><th colspan="3">GROBID Models</th></tr><tr><th>Lexical Entry</th><th>Form</th><th>Sense</th></tr></thead><tbody><tr><td>Type 1</td><td>98.05</td><td>100</td><td>98.71</td></tr><tr><td>Type 2</td><td>99.16</td><td>95.02</td><td>90.78</td></tr><tr><td>Type 3</td><td>96.01</td><td>92.89</td><td>88.1</td></tr><tr><td>Type 4</td><td>92.78</td><td>96.83</td><td>86.64</td></tr><tr><td>All types mixed</td><td>95.43</td><td>97.44</td><td>92.77</td></tr></tbody></table>

## Conclusion

Two important conclusions can be drawn from this test. First, bigram feature templates are more efficient than unigram templates. Second, a general model potentially increases scores for certain levels (*form* and *sense*) but not all of them (*lexical entry*), which raises the question of the pertinence of a hybrid model, choosing the best solution for each level.

## References
1. Mohamed Khemakhem, Laurent Romary, Simon Gabay, Hervé Bohbot, Francesca Frontini, et al. Automatically Encoding Encyclopedic-like Resources in TEI. *The annual TEI Conference and Members Meeting*, Sep 2018, Tokyo, Japan.

2. Mohamed Khemakhem, Luca Foppiano, Laurent Romary. Automatic Extraction of TEI Structures in Digitized Lexical Resources using Conditional Random Fields. electronic lexicography, eLex 2017, Sep 2017, Leiden, Netherlands.

3. Mohamed Khemakhem, Axel Herold, Laurent Romary. Enhancing Usability for Automatically Structuring Digitised Dictionaries. GLOBALEX workshop at LREC 2018, May 2018, Miyazaki, Japan. 2018.

4. Lucie Rondeau du Noyer, Simon Gabay, Mohamed Khemakhem, Laurent Romary. *Training and evaluation data for encoding Manuscript Sales Catalogues with GROBID dictionaries*, Paris: École nationale des chartes (PSL)/Neuchâtel: université de Neuchâtel, 2019, https://github.com/lairaines/grobid-TEI-2019.

## Biographies

*Lucie Rondeau du Noyer* is a " History and New Technologies" masters' student at the Ecole Nationale des Chartes and a graduate student at the Ecole Normale supérieure (Paris).

*Simon Gabay* is post-doc at the University of Neuchâtel (Switzerland), where he teaches DH and carries research on 17th c. French literature and modern manuscripts. He is currently working on a database of sold manuscripts in 19th c. France.

*Mohamed Khemakhem* is a PhD candidate at Inria, team ALMAnaCH (Paris), Paris 7 University and Centre Marc Bloch (Berlin). His research is focused on parsing lexical and encyclopedic legacy resources using standard-based machine learning models.

*Laurent Romary* is senior researcher at Inria, team ALMAnaCH and works on data modelling and standards in humanities computing.