SIMARA
Full name: SIMARA: a database for key-value information extraction from full-page handwritten documents

# Description

We propose a new database for information extraction from historical handwritten documents. The corpus includes 5,393 finding aids from six different series, dating from the 18th-20th centuries. Finding aids are handwritten documents that contain metadata describing older archives. They are stored in the National Archives of France and are used by archivists to identify and find archival documents. 

Each document is annotated at page-level, and contains seven fields to retrieve. The localization of each field is not available in such a way that this dataset encourages research on segmentation-free systems for information extraction.

The dataset is available at https://zenodo.org/record/7868059

## Details for each series and entity type

| Series              | Train | Validation | Test | Total (%) |
| ------------------- | ----- | ---------- | ---- | --------: |
| E series            | 322   | 64         | 79   |       8.6 |
| L series            | 38    | 8          | 4    |       0.9 |
| M series            | 128   | 21         | 27   |       3.3 |
| X1a series          | 2209  | 491        | 469  |      58.8 |
| Y series            | 940   | 205        | 196  |      24.9 |
| Douët s'Arcq series | 141   | 22         | 29   |       3.5 |
| Total               | 3778  | 811        | 804  |       100 |

| Entities       | Train | Validation | Test  | Total (%) |
| -------------- | ----- | ---------- | ----- | --------: |
| date           | 8406  | 1814       | 1799  |      10.4 |
| title          | 35531 | 7495       | 8173  |      44.5 |
| serie          | 3168  | 664        | 676   |       3.9 |
| analysis       | 25988 | 5130       | 5602  |      31.9 |
| volume_number  | 3913  | 808        | 813   |       4.8 |
| article_number | 3181  | 665        | 678   |       3.9 |
| arrangement    | 644   | 122        | 153   |       0.8 |
| Total          | 80831 | 16698      | 17894 |       100 |

## Data encoding

Transcriptions with entities are encoded in the `labels.json` JSON file. Special tokens are used to represent named entities. Please not that there are only opening NER tokens: each entity spans all words until the next entity starts. 

| Entities       | Special token | Symbol unicode |
| -------------- | ------------- | -------- |
| date           | ⓓ            | `\u24d3` |
| title          | ⓘ            | `\u24d8` |
| serie          | ⓢ            | `\u24e2` |
| analysis       | ⓒ            | `\u24d2` |
| volume_number  | ⓟ            | `\u24df` |
| article_number | ⓐ            | `\u24d0` |
| arrangement    | ⓥ            | `\u24e5` |
 
# Cite us!
The dataset is presented in details in the following article:

```bib
@article{simara2023,
    author = {Solène Tarride and Mélodie Boillet and Jean-François Moufflet and Christopher Kermorvant},
    title = {SIMARA: a database for key-value information extraction from full-page handwritten documents},
    year = {2023},
    journal={Proceedings of the 17th International Conference on Document Analysis and Recognition},
}
```

Introducing paper: SIMARA: a database for key-value information extraction from full pages