# Experimenting With Generic Recognition Systems for Kuzushiji Documents: Furigana Extraction as a Use-Case

Hugo Scheithauer, Laurent Romary

<figure><img src="image_1.png" /></figure>

## To cite this version:

Hugo Scheithauer, Laurent Romary. Experimenting With Generic Recognition Systems for Kuzushiji Documents: Furigana Extraction as a Use-Case. JADH2024 - 13th Conference of Japanese Association for Digital Humanities "Leveraging AI and Digital Humanities for Sustainable Infrastructure", JADH, Sep 2024, Tokyo, Japan. hal-04738212

HAL Id: hal-04738212

https://inria.hal.science/hal-04738212v1

Submitted on 15 Oct 2024

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_3.png" /></figure>

RÉPUBLIQUE
FRANÇAISE

Liberté
Égalité
Fraternité

Inria

<figure><img src="image_4.png" /></figure>

<figure><img src="image_5.png" /></figure>

École Pratique
des Hautes Études

PSL

<figure><img src="image_6.png" /></figure>

# Experimenting With Generic Recognition Systems for Kuzushiji Documents: Furigana Extraction as a Use-Case

<figure><img src="image_7.png" /></figure>

Hugo Scheithauer, PhD Candidate, ALMAnaCH, Inria, Paris / École Pratique des Hautes Études, Paris (EPHE)

Laurent Romary, Directorate for Scientific Information and Culture, Inria, Paris

<figure><img src="image_8.png" /></figure>

Excerpt from 太閤真顕記, Edo period, Bibliothèque nationale de France, https://gallica.bnf.fr/ark:/12148/btv1b8451517r
# Kuzushiji Automatic Recognition: a (brief and recent) Research Timeline (1/2)

<figure><img src="image_9.png" /></figure>

<figure><img src="image_10.png" /></figure>

ほうらい山, 1661-1681, Bibliothèque nationale de France, https://gallica.bnf.fr/ark:/12148/btv1b8304433t

**Kuzushiji (くずし字)**: style of cursive Japanese script used in pre-modern (before the early 20th century) written and printed texts.
## Kuzushiji Automatic Recognition: a (brief and recent) Research Timeline (2/2)

When Kuronet was introduced, Clanuwat et al. (2019) highlighted the intentional exclusion of **furigana** (phonetic annotations placed next to kanji, either in Katakana or Hiragana) due to the absence of labels for annotation in the JCB dataset.

The authors also stressed the need to **differentiate between the main text and annotations** in automatic processing of Kuzushiji documents.

Clanuwat et al., 2019, releases **Kuronet** for Kuzushiji recognition, a highly efficient deep learning architecture trained on the JCB dataset.

2019

<figure><img src="image_11.png" /></figure>

Iroe shokunin burui, 大田南畝, 1784, Edo period, Bibliothèque nationale de France,
https://gallica.bnf.fr/ark:/12148/btv1b531420548
<figure><img src="image_12.png" /></figure>

# Furigana Detection: Another Step Toward a Comprehensive Kuzushiji Transcription

→ To our knowledge, fine-tuning a system like Kuronet on new annotated data to accommodate furigana extraction is not available to the public.

→ Other custom deep learning architectures created for the Kuzushiji Kaggle competition in 2019 necessitate machine learning engineering expertise for customisation.

→ Furigana processing presents a **dual opportunity**:

* Its intrinsic value as another step toward **comprehensive Kuzushiji transcriptions**,

* And **testing generic systems for Kuzushiji recognition** (layout analysis, and automatic text recognition), as well as creating new shareable datasets for this task.

* YOLO architecture for layout analysis and object detection.

* Automatic text recognition engine Kraken.
<figure><img src="image_13.png" /></figure>

# The Ancient Japanese Collections of the French National Library

→ The French National Library holds a large **collection of Japanese manuscripts and various documents spanning throughout the Edo period**. A lot of documents already digitized are available on its digital library, **Gallica**.

- 293 manuscripts,

- 1096 prints and photographs,

- 72 documents related to performing arts,

- 148 maps.

... And much more documents waiting to be digitized.

The only downside is that a lot of the metadata are often incomplete.

<figure><img src="image_14.png" /></figure>

Japonais 615 (3), Edo period, Bibliothèque nationale de France,
https://gallica.bnf.fr/ark:/12148/btv1b10510296h
<figure><img src="image_15.png" /></figure>

# The Furigana-Kanji dataset: presentation (1/2)

→ **15 manuscripts** were sampled from the French National Library to constitute a first version of the Furigana-Kanji dataset.

<table><thead><tr><th>Archival Resource Key (ark)</th><th>Title</th><th>Romanized title</th><th>Alternative title</th><th>Author</th><th>Date</th><th>Annotated pages</th></tr></thead><tbody><tr><td>ark:/12148/btv1b8304433t</td><td>ほうらい山</td><td>Hôrai san</td><td>N/A</td><td>N/A</td><td>1661-1681</td><td>12</td></tr><tr><td>ark:/12148/btv1b8451517r</td><td>太閤真顕記</td><td>Taikō shinkenki</td><td>N/A</td><td>N/A</td><td>Edo period</td><td>10</td></tr><tr><td>ark:/12148/btv1b10506231j</td><td>N/A</td><td>N/A</td><td>Japonais 5610</td><td>N/A</td><td>Edo period</td><td>20</td></tr><tr><td>ark:/12148/btv1b10510296h</td><td>N/A</td><td>N/A</td><td>Japonais 615 (3)</td><td>N/A</td><td>Edo period</td><td>13</td></tr><tr><td>ark:/12148/btv1b10511050r</td><td>隅田川兩岸一覧三巻 / 蓮飾北斉画</td><td>Sumidagawa ryōgan ichiran 3 t. / Ill. Katsushika Hokusai</td><td>N/A</td><td>N/A</td><td>1800-1868</td><td>2</td></tr><tr><td>ark:/12148/btv1b10511093r</td><td>N/A</td><td>N/A</td><td>Japonais 632 (2)</td><td>N/A</td><td>Edo period</td><td>19</td></tr><tr><td>ark:/12148/btv1b10512765d</td><td>N/A</td><td>Arabia monogatari: kaikan kyōki</td><td>N/A</td><td>永峰秀樹 訳</td><td>1875</td><td>20</td></tr><tr><td>ark:/12148/btv1b10512768r</td><td>N/A</td><td>Fukushoku zukai</td><td>N/A</td><td>N/A</td><td>Edo period</td><td>29</td></tr><tr><td>ark:/12148/btv1b10525770x</td><td>N/A</td><td>Hisago gundan gojūyon-jō</td><td>N/A</td><td>歌川 芳斃</td><td>1864-1865</td><td>19</td></tr><tr><td>ark:/12148/btv1b10527650d</td><td>N/A</td><td>Kiyomizu tsuya monogatari</td><td>N/A</td><td>N/A</td><td>Edo period</td><td>14</td></tr><tr><td>ark:/12148/btv1b60002827</td><td>北条時頼記図会</td><td>Hójó jiraiki zue</td><td>N/A</td><td>池田東蘆</td><td>1788-1857</td><td>19</td></tr><tr><td>ark:/12148/btv1b60002879</td><td>N/A</td><td>Tôto shōkei ichiran</td><td>N/A</td><td>葛飾 北斉</td><td>1760-1849</td><td>10</td></tr><tr><td>ark:/12148/btv1b531420548</td><td>彩画職人部類</td><td>Iroe shokunin burui</td><td>N/A</td><td>大田南畝</td><td>1784</td><td>14</td></tr><tr><td>ark:/12148/btv1b6000280d</td><td>N/A</td><td>N/A</td><td>Hokusai manga / Katsushika</td><td>葛飾 北斉</td><td>1760-1849</td><td>25</td></tr><tr><td>ark:/12148/btv1b10509001k</td><td>傅神開手 北斉画式</td><td>Denshin kaishu Hokusai gashiki</td><td>N/A</td><td>葛飾 北斉</td><td>1825</td><td>9</td></tr><tr><td>ark:/12148/btv1b10523261c</td><td>花鳥写真図彙 / 北尾紅翠斉</td><td>Shashin kachō zue / Kitao Kōsuisai</td><td>N/A</td><td>北尾 重政</td><td>1805</td><td>8</td></tr><tr><td>ark:/12148/btv1b10523259j</td><td>花鳥写真図彙 / 北尾紅翠斉</td><td>Shashin kachō zue / Kitao Kōsuisai</td><td>N/A</td><td>北尾 重政</td><td>1805</td><td>9</td></tr></tbody></table>

Total: 252
# The Furigana-Kanji dataset: presentation (2/2)

<figure><img src="image_16.png" /></figure>

Iroe shokunin burui. 大田南欧, 1784. Edo period, Bibliothèque nationale de France, https://gallica.bnf.fr/ark:/12148/btv1b531420548

Hisago gundan gojūyon-jō, 歌川芳斎. 1864-1865.
Bibliothèque nationale de France, https://gallica.bnf.fr/ark:/12148/btv1b10525770x

<figure><img src="image_17.png" /></figure>

ほうらい山, 1661-1681. Bibliothèque nationale de France, https://gallica.bnf.fr/ark:/12148/btv1b8304433t

<figure><img src="image_18.png" /></figure>

北条時記館図会. 池田東篠, 1788-1857. Bibliothèque nationale de France, https://gallica.bnf.fr/ark:/12148/btv1b60002827

<figure><img src="image_19.png" /></figure>
<figure><img src="image_20.png" /></figure>

# Training YOLO for Furigana Detection in Object Detection Models

Based on the growing popularity and efficiency of the **YOLO architecture** in DH projects (Clérice, 2023 & Clérice et al., 2024), we decided to opt for an **object detection approach** for furigana detection.

YOLO segments an image into zones based solely on **visual features**, assigning a label to each zone.

<figure><img src="image_21.png" /><figcaption>Snapshot of the Roboflow annotation online interface.</figcaption></figure>
<figure><img src="image_22.png" /></figure>

Effective Annotation Strategies for Furigana detection (1/3)

Sometimes, furigana can easily be matched with their kanji at a glance.

<figure><img src="image_23.png" /></figure>

花鳥写真図彙/北尾紅翠斎, 北尾重政, 1805, Bibliothèque nationale de France, https://gallica.bnf.fr/ark:/12148/btv1b10523261c
Effective Annotation Strategies for Furigana
detection (2/3)

.... But often they are not.

<figure><img src="image_24.png" /><figcaption>Hokusai manga / Katsushika, 葛飾 北斉, 1760-1849, Bibliothèque
nationale de France, https://gallica.bnf.fr/ark:/12148/btv1b6000280d</figcaption></figure>

<figure><img src="image_25.png" /><figcaption>北条時賴記図会 池田東籚、1788-1857, Bibliothèque nationale de France,
https://gallica.bnf.fr/ark:/12148/btv1b60002827</figcaption></figure>
<figure><img src="image_26.png" /></figure>

# Effective Annotation Strategies for Furigana detection (3/3)

To evaluate the performance of the YOLO object detection model for furigana detection, we employed **two layout annotation schemes**:

1. **Annotating the main text and furigana independently**, allowing for differentiation between them. Schema inspired from Bjerregaard et al., 2022.

2. **Nesting the furigana annotations within the kanji annotations**, enabling both the distinction between the main text and furigana and the association of furigana with its corresponding kanji. The objective is to **detect only kanji annotated with furigana**.

<figure><img src="image_27.png" /></figure>

Japonais 632 (2), Edo period,
Bibliothèque nationale de
France,
https://gallica.bnf.fr/ark:/12148/bt
v1b10511093r
# Layout Annotation Using the SegmOnto Controlled Vocabulary

S

https://segmonto.github.io/

MainZone:Character

MainZone:Ruby

→ The **Kanji-Furigana** dataset was annotated following the **SegmOnto controlled vocabulary**, designed for **describing the content of books and manuscripts**.

→ Using a controlled vocabulary ensures that the dataset follows the **FAIR principles**.

<figure><img src="image_28.png" /><figcaption>Annotation schema 1</figcaption></figure>

北条時穂記図会, 池田東饒, 1788-1857, Bibliothèque nationale de France,
https://gallica.bnf.fr/ark:/12148/btv1b60002827
# Model Performance for Detecting Unlinked Text and Furigana (1): Quantitative Analysis

<figure><img src="image_29.png" /></figure>

Very good scores can be achieved with a relatively small and diverse dataset.

The model can be visualized and tested online at:

https://app.roboflow.com/jadh2024/furigana-detection-unlinked/visualize/3

To be uploaded on GitHub at:

https://github.com/HugoSchtr/Kanji-Furigana-Dataset

<figure><img src="image_30.png" /></figure>
# Model Performance for Detecting Unlinked Text and Furigana (1): Qualitative Analysis (1/2)

太閤真顕記, Edo Period, Bibliothèque nationale de France, https://gallica.bnf.fr/ark:/12148/btv1b8451517r

<figure><img src="image_31.png" /><figcaption>Ground truth</figcaption></figure>

<figure><img src="image_32.png" /><figcaption>Prediction</figcaption></figure>
## Model Performance for Detecting Unlinked Text and Furigana (1): Qualitative Analysis (2/2)

ほうらい山, 1661-1681, Bibliothèque nationale de France,
https://gallica.bnf.fr/ark:/12148/btv1b8304433t

<figure><img src="image_33.png" /></figure>

Ground truth

Missing
furigana
instance

<figure><img src="image_34.png" /></figure>

Prediction

Split error (but
that's not very
important as
long as we can
differentiate
between the
main text and
the furigana)
# Model Performance for Detecting Linked Kanji and Furigana (2): Quantitative Analysis

<figure><img src="image_35.png" /></figure>

<figure><img src="image_36.png" /></figure>

<figure><img src="image_37.png" /></figure>

Again, **very good scores** (even better than the previous experiment) can be achieved with a **relatively small and diverse dataset**.

The model can be visualized and tested online at:
https://app.roboflow.com/jadh2024/annotation-detection/visualize/15
Uploaded on GitHub:
https://github.com/HugoSchtr/Kanji-Furigana-Dataset

<figure><img src="image_38.png" /></figure>
## Model Performance for Detecting Linked Kanji and Furigana (2): Qualitative Analysis (1/2)

<figure><img src="image_39.png" /><figcaption>Ground truth</figcaption></figure>

Hallucinated kanji annotated with furigana

<figure><img src="image_40.png" /><figcaption>Prediction</figcaption></figure>

Missed furigana annotations
Model Performance for Detecting Linked Kanji and Furigana (2): Qualitative Analysis (2/2)

<figure><img src="image_41.png" /></figure>

Ground truth

<figure><img src="image_42.png" /></figure>

Prediction

Missed furigana annotation, but near perfect detection.
<figure><img src="image_43.png" /></figure>

# Flaws in Object Detection for Linked Kanji and Furigana

→ The dataset includes manuscripts and illustrated manuscripts but **lacks highly complex layouts** (Clanuwat et al., 2019).

→ The second annotation schema assumes that each kanji works separately, which is not always the case: it does not take into account complex linguistic phenomena such as **Jukujikun (熟字訓)** as object detection models are based on visual features, not textual features.

→ The second annotation schema works well for documents where kanji and furigana are distinctly separated. However, when **furigana is written cursively across multiple kanji, it becomes difficult to determine whether the model splits the furigana based on subtle annotated visual writing features or arbitrarily.**

<figure><img src="image_44.png" /><figcaption>Figures taken from Clanuwat et al., 2019. The layout does not lay out the text in a clear sequential order and regular columns.</figcaption></figure>
<figure><img src="image_45.png" /></figure>

Expanding the Furigana-Kanji dataset

- Sample more documents from the French National Library to have a more diverse dataset.

- Try to reach out to curators to complete the documents' metadata.

- Explore and sample documents from other French heritage institutions

→ Expanding the dataset will help determine **whether a generic Furigana detection model can be developed or if domain-specific factors play a critical role in models' performance.**

<figure><img src="image_46.png" /><figcaption>当家秘歌集, 1762, Bibliothèque universitaire des langues et civilisations,
https://bina.bulac.fr/s/bina/ark:/73193/b3r2dm</figcaption></figure>
<figure><img src="image_47.png" /></figure>

# Kraken for Kuzushiji Recognition: Evaluating a Generic Text Recognition System (1/2)

- Kraken is a generic, language-agnostic and all-in-one text recognition engine.

- It features trainable **layout analysis** (for text regions and lines) and **character recognition** models that support all Unicode scripts.

- Kraken has demonstrated satisfactory results on historical Chinese documents (18th-20th century) (Brisson et al., 2023) with challenges similar to Kuzushiji recognition, such as unbalanced class distributions, which encouraged us to experiment with it.
# Kraken for Kuzushiji Recognition: Evaluating a Generic Text Recognition System (2/2)

## What Kuzushiji datasets do we have to experiment with Kraken?

→ The **KMNIST dataset** (Clanuwat et al., 2018).

It comprises three subsets:

- **Kuzushiji-MNIST**: 10 classes of hiragana characters.

- **Kuzushiji-49**: 49 different hiragana characters.

- **Kuzushiji-Kanji**: 3832 unbalanced classes of kanji characters.

<table><thead><tr><th>Hiragana</th><th>Unicode</th><th>Samples</th></tr></thead><tbody><tr><td>お (o)</td><td>U+304A</td><td>7000</td></tr><tr><td>き (ki)</td><td>U+304D</td><td>7000</td></tr><tr><td>す (su)</td><td>U+3059</td><td>7000</td></tr><tr><td>つ (tsu)</td><td>U+3064</td><td>7000</td></tr><tr><td>な (na)</td><td>U+306A</td><td>7000</td></tr></tbody></table>

<figure><img src="image_48.png" /></figure>

<table><thead><tr><th>Hiragana</th><th>Unicode</th><th>Samples</th></tr></thead><tbody><tr><td>は (ha)</td><td>U+306F</td><td>7000</td></tr><tr><td>ま (ma)</td><td>U+307E</td><td>7000</td></tr><tr><td>や (ya)</td><td>U+3084</td><td>7000</td></tr><tr><td>れ (re)</td><td>U+308C</td><td>7000</td></tr><tr><td>を (wo)</td><td>U+3092</td><td>7000</td></tr></tbody></table>

<figure><img src="image_49.png" /></figure>

Taken from Clanuwat et al., 2018. The 10 classes of the Kuzushiji-MNIST

<figure><img src="image_50.png" /></figure>

Taken from Clanuwat et al., 2018. Examples of some of the 3832 classes in the Kuzushiji-Kanji subset.
<figure><img src="image_51.png" /></figure>

## Evaluating Kraken on the KMNIST Dataset

<table><thead><tr><td>Train Set</td><td>Test Set</td><td>Kraken Accuracy (%)</td></tr></thead><tbody><tr><td>Kuzushiji-MNIST (train)</td><td>Kuzushiji-MNIST (test)</td><td>90.8</td></tr><tr><td>Kuzushiji-49 (train)</td><td>Kuzushiji-49 (test)</td><td>86.5</td></tr><tr><td>Kuzushiji-Kanji (train, 80%)</td><td>Kuzushiji-Kanji (test, 20%)</td><td>48.9</td></tr></tbody></table>

→ Kraken was trained on the **official split for both Kuzushiji-MNIST and Kuzushiji-49** subsets. For the Kuzushiji-Kanji subset, **we implemented a balanced split to ensure the presence of all classes** in the train and test sets.

→ **Kraken is optimized for line-based training data**, which can explain the relatively low scores achieved on the Kuzushiji-49 subset, and the mediocre accuracy on the Kuzushiji-Kanji subset.

→ We hope to provide a **baseline for future evaluations of Kraken on Kuzushiji recognition**.
# eScriptorium (PSL - SCRIPTA): a GUI for Automatic Text Recognition Projects

<figure><img src="image_52.png" /></figure>

* GUI for **transcribing** textual documents, **creating training sets**, training **segmentation and transcription models**, exporting dataset in standardized formats (XML ALTO), etc.

* Project management tool

* eScriptorium uses the language-agnostic transcription engine **Kraken** (Benjamin Kiessling, PSL)

<figure><img src="image_53.png" /></figure>

An annotated page of 歌学提要 (doi:10.20730/200008003) (国文研書誌ID / Kokubunken Bibliography ID: 200008003, from the JCB dataset.
## eScriptorium as an Automatic Text Recognition Infrastructure

→ **CREMMA** (Consortium for the handwritten text recognition of ancient materials) is an **infrastructure** led by Inria and the École nationale des chartes (Jean-Mabillon Center). It is funded by the DIM "Ancient and Heritage Materials."

→ Inria provides an access to an instance of eScriptorium hosted on a **powerful server** (GPUs & storage).

→ In exchange, users engage themselves to make their training data open and to publish their metadata on the **HTR-United** catalog.

<figure><img src="image_54.png" /></figure>
# Incorporating Furigana Detection in Kuzushiji Documents Digital Editions

<figure><img src="image_55.png" /></figure>

北条時賴記図会、池田東篠、1788-1857, Bibliothèque nationale de France,
https://gallica.bnf.fr/ark:/12148/btv1b60002827

<figure><img src="image_56.png" /></figure>

ALTO XML ensures the creation
of **standardized interoperable
datasets** by being easily
shareable, and acts as a pivot
format to convert layout analysis
and recognition results into TEI

Semi-automatic
transformation into TEI
based on text zones
coordinates and
transcribed text
# Training Kraken with the JCB dataset (1/2)

→ The **JCB dataset** is also a significant resource to further our experiment with Kraken.

→ Some work is required to convert the JCB dataset into a format usable for training Kraken (ALTO XML), as it is currently redistributed as CSV files. But doing so would allow to have an **ALTO XML line-level annotated dataset** for training line detection and Kuzushiji recognition Kraken models.

<figure><img src="image_57.png" /></figure>

An annotated page of 歌学提要 (doi:10.20730/200008003) (国文研書誌 ID / Kokubunken Bibliography ID: 200008003, from the JCB dataset).
# Training Kraken with the JCB dataset (2/2)

However, Clanuwat et al. (2019) already highlighted that line-based recognition models are not ideal for Kuzushiji documents, as these texts often lack a consistent layout, which complicates line ordering and recognition.

Experiments with Kraken's new line ordering model need to be conducted.

<figure><img src="image_58.png" /></figure>

An annotated page of 歌学提要 (doi:10.20730/200008003) (国文研書誌ID / Kokubunken Bibliography ID: 200008003, from the JCB dataset.)
<figure><img src="image_59.png" /></figure>

Conclusion

- Detecting furigana is **easily achievable** with today's object detection technology, even with relatively little amounts of annotated diverse data.

- The questions are: what is the most efficient layout annotation schema for furigana processing in the context of archiving and editing Kuzushiji documents? How can we incorporate furigana processing in edition pipelines?

- Furigana detection experiments also pave the way to broader experiments on **Kuzushiji documents layout analysis**.

- Work still need to be done on Kraken for Kuzushiji recognition, but Kraken and eScriptorium offers an opportunity to create an infrastructure for creating and sharing new Kuzushiji FAIR datasets to the wider research community.
Thank you! All questions and feedback are welcomed!

<figure><img src="image_60.png" /><figcaption>花鳥寫真図彙 / 北尾紅翠溶, 北尾 重政, 1805, Bibliothèque nationale de France, https://gallica.bnf.fr/ark:/12148/btv1b10523261c</figcaption></figure>
References

- T. Clanuwat, A. Lamb, A. Kitamoto, KuroNet: Pre-Modern Japanese Kuzushiji Character Recognition with Deep Learning, 2019. URL: http://arxiv.org/abs/1910.09433. doi:10.48550/arXiv.1910.09433, arXiv:1910.09433 [cs].

- Y. Hashimoto, Y. likura, Y. Hisada, S. Kang, T. Arisawa, A. Okajima, T. Yada, R. Goyama, D. K.-B, The Kuzushiji Project: Developing a Mobile Learning Application for Reading Early Modern Japanese Books, 2016. URL: https://www semanticscholar org/paper/The-Kuzushiji-Project%3A-Developing-a-Mobile-Learning-Hashimoto-likura/394041efdd271fa54588516d93b46e48ff15ec61.

- K. Ueki, T. Kojima, Survey on Deep Learning-based Kuzushiji Recognition, 2020. URL: http://arxiv.org/abs/2007.09637. doi:10.48550/arXiv.2007.09637, arXiv:2007.09637 [cs].

- G. Jocher, A. Chaurasia, J. Qiu, Ultralytics YOLO, 2023. URL: https://github.com/ultralytics/ultralytics.

- B. Kiessling, Kraken - a Universal Text Recognizer for the Humanities, 2019. URL: https://dataverse.nl/dataset.xhtml?persistentId=doi:10.34894/Z9G2EX. doi:10.34894/Z9G2EX.

- T. Clanuwat, M. Bober-Irizar, A. Kitamoto, A. Lamb, K. Yamamoto, D. Ha, Deep Learning for Classical Japanese Literature, 2018. URL: http://arxiv.org/abs/1812.01718. doi:10.20676/00000341, arXiv:1812.01718 [cs, stat].

- T. N. Hung, T. N. Cuong, K. Asanobu, N. Masaki, Segmenting Text in Japanese Historical Document Images Using Convolutional Neural Networks, 2019 (2019) 253-260. URL: https://cir.nii.ac.jp/crid/1050855522047807616.

- A. Lamb, T. Clanuwat, S. Han, M. Bober-Irizar, A. Kitamoto, Predicting the ordering of characters in Japanese historical documents, 2021. arXiv:2106.06786.

- N. K. Bjerregaard, V. Cheplygina, S. Heinrich, Detection of Furigana Text in Images, 2022. URL: http://arxiv.org/abs/2207.03960. doi:10.48550/arXiv.2207.03960, arXiv:2207.03960 [cs].

- N.-M. Sven, R. Matteo, Page Layout Analysis of Text-heavy Historical Documents: a Comparison of Textual and Visual Approaches, 2022. URL: http://arxiv.org/abs/2212.13924. doi:10.48550/arXiv.2212.13924, arXiv:2212.13924 [cs].

- T. Clérice, J. Janes, H. Scheithauer, S. Bénière, L. Romary, B. Sagot, "Layout Analysis Dataset with SegmOnto". In: DH2024 - Annual conference of the Alliance of Digital Humanities

- Organizations. ADHO. Washington DC, United States, Aug. 2024. URL: https://inria.hal.science/hal-04513725.

- T. Clérice, You Actually Look Twice At it (YALTAi): using an object detection approach instead of region segmentation within the Kraken engine, Journal of Data Mining & Digital Humanities Historical Documents and... (2023) 9806. URL: http://arxiv.org/abs/2207.11230. doi:10.46298/jdmdh.9806, arXiv:2207.11230 [cs].

- S. Gabay, J.-B. Camps, A. Pinche, C. Jahan, SegmOnto: common vocabulary and practices for analysing the layout of manuscripts (and more), in: 1st International Workshop on Computational Paleography (IWCP@ICDAR 2021), Lausanne, Switzerland, 2021. URL: https://hal.science/hal-03336528.

- A. I. Solis, J. Zarkovacki, J. Ly, A. Atyabi, Recognition of Handwritten Japanese Characters Using Ensemble of Convolutional Neural Networks, 2023. URL: http://arxiv.org/abs/2306.03954. doi:10.48550/arXiv.2306.03954, arXiv:2306.03954 [cs].

- P. A. Stokes, B. Kiessling, D. S. B. Ezra, R. Tissot, E. H. Gargem, The eScriptorium VRE for Manuscript Cultures, Classics@ Journal (2021). URL: https://classics-at.chs.harvard.edu/classics18-stokes-kiessling-stokl-ben-ezratissot-gargem/.

- C. Brisson, F. Constant, M. Bui, Chinese Historical documents Automatic Transcription (CHAT) models, 2023. URL: https://doi.org/10.5281/zenodo.8383732. doi:10.5281/zenodo.8383732.

- A. Chagué, T. Clérice, Sharing HTR datasets with standardized metadata: the HTR-United initiative, 2022. URL: https://inria.hal.science/hal-03703989.

- Z. Shen, K. Zhang, M. Dell, A Large Dataset of Historical Japanese Documents with Complex Layouts, 2020. URL: http://arxiv.org/abs/2004.08686. doi:10.48550/arXiv.2004.08686, arXiv:2004.08686 [cs].
# Bonus: check the quality of your data when training models!

<figure><img src="image_61.png" /><figcaption>## Furigana object detection model - 2nd annotation schema</figcaption></figure>

“Dirty”, not proofread, version of the dataset

<figure><img src="image_62.png" /></figure>

Same dataset, but clean

<figure><img src="image_63.png" /></figure>