March 14, 2024
Council on East Asian Libraries

New Developments in Digital Services at the
National Diet Library:
OCR Text and Machine Learning
Toru AOIKE
Research and Development for Next-Generation
Systems Office, National Diet LibraryTable of contents
1. Optical Character Recognition (OCR) and its technical challenges➢ Seven initiatives at the NDL➢ OCR for Japanese documents and R&D elements➢ Introduction of our R&D office➢ OCR-related projects and recent developments

2. NDL Lab’s Experimental Services➢ NDL Ngram Viewer➢ NDLkotenOCR (NDL古典籍OCR)➢ Next Digital Library21．OCR and its technical challenges

3

Seven initiatives for connecting information resources
and intellectual activities at the NDLInitiative 5: Accelerating digitization ofmaterials includes generating full-text data

• Benefits of full-text data
- Keyword searches- Datasets for machine learning- For persons with print disabilities

5

OCR for Japanese documents• Generating full-text data for massive volumes ofdigitized images (cost, time)• Dealing with typographical conventions from theMeiji era, including obsolete character forms orfonts (quality)

Development of an AI-OCR model optimized for
digitized materials at the NDL
https://doi.org/10.11501/881298

6

Generating high-quality text data with OCR
Generating accurate text data from books and periodicals by material type andpublication date↓
Investigating the quality of existing OCR software and services↓
Defining criteria and establishing specifications for OCR quality↓
Developing and improving OCR↓
Inspecting to confirm that OCR quality exceeds NDL specifications
This OCR project entailed significant research and development of new technologythat was outsourced to vendors in accordance with specifications set by and subject
to final inspection by the NDL.
7

Which office is responsible for R&D at the NDL?
Research and Development for Next-Generation Systems Office(Launched in 2011)

Department responsible for research and demonstration of newlibrary services based on advanced information technology
Me！
Office staff
1 office head, 1 assistant section head, 2 staff members,
2 part-time staff members, 3 part-time researchers, and 1 associate member

8Research and Development for Next-Generation Systems Office⚫ Action policy➢ Improvement of services and operations in response to the digital shift
Research and development of technology for utilizing digitized materials to expand searchfunctionality and streamline the creation of bibliographic data
This contains the OCR-related projects➢ Promoting utilization of digital information resources
Release developed programs and datasets to the public➢ Providing access to diverse cultural resources and an infrastructure for their utilization
Development and operation of the JAPAN SEARCH website➢ Long-term preservation of digital materials
Migration and emulation technology survey for electronic publications packaged in media (USBmemory, floppy disks, MO, etc.)
9OCR-Related Projects : Overview in FY 2021
➢ (1) Mass conversion of digitized images to text data during FY 2021
- Target: 2,470,000 items (223,000,000 images)= almost all materials that had been digitized as of 2020

Detailed information: https://lab.ndl.go.jp/data_set/ocr_en/r3_text/➢ (2) Development of NDLOCR during FY 2021
- Development of AI-OCR software for Japanese materials- To be used for the text conversion of materials digitized after 2021 by the NDL- To be made freely available as open-source software (CC BY 4.0)

Detailed information: https://lab.ndl.go.jp/data_set/ocr_en/r3_software/
10

Overview: Digitized and OCRed Materials at the NDL
To be OCRed from Jan. 2024
by NDLOCR(750,000 items)
1868

1800

Pre-modern materials (80,000 items)

1900

1969

Books
(970,000 items)

1987

2000
1995

(600,000items)

OCRed in FY2022
experimental project(mentioned later)
OCRed in FY2021

Periodicals(1.33 million items): Digitized: Prepared for digitization from FY2023

11

NDL Digital Collections: Full-Text Search• Text data produced in FY 2021 for
2.47 million items is now fullysearchable.
• Queried keywords and theirsurrounding text are displayed assnippets in the search results.
• Starting from Jan. 2024, additional
750,000 items are being OCRed (byNDLOCR) and are gradually madeavailable for full text search in NDL Digital
Collections (the project to be completedby March 2025)
https://www.ndl.go.jp/jp/news/fy2023/240118_03.html
12Improvement of NDLOCR during FY 2022-2023• NDLOCR was developed for use in generating text data from materials digitized by theNDL from FY2021 onward• NDLOCR ver. 1 was released in FY2021, ver. 2.0 was released in FY2022,
and ver. 2.1 with additional improvements was released in FY2023.
• Ver. 2.1 averaged a character recognition rate of 95.24% for books and periodicals
published after 1868 (Meiji era to the present).
Detailed information: https://lab.ndl.go.jp/data_set/r4ocr/r4_software/• All versions are released under CC BY license from the official NDL Lab’s GitHub andcan be freely used: https://github.com/ndl-lab/ndlocr_cli• Machine learning datasets for OCR derived from materials whose copyrightprotection has expired are released under the public domain mark.
13

Example of layout recognition of NDLOCR

Yellow areas are annotations (headnotes in the example)
The results of recognition are below:「公請-恒例臨時の法席は必ず請召の僧に與ふ之を公請僧といふ……」

Red areas indicate text
The results of recognition are below:「治承元年五月五日の日、天台座主明雲大僧正、公請を停止せらる〻上、藏人を御使にて……」

永井一孝 [校] 『平家物語』 有朋堂書店 1927
https://dl.ndl.go.jp/pid/1223268/1/51
14

Use of NDLOCR at the NDL• The system that automates every step of the process from OCR conversion
to indexing and cataloging in the NDL Digital Collections
Reserve OCR processing on
the web browser

16 OCR processes running in parallel

Indexing to NDL Digital Collections

OCR Process
OCR Process・・・
OCR Process

Night or day, seven days a week, between 60 and 80 books per hour
are added and appear in the results of full-text searches.

15

Technical overview 3.Message Broker
2. Task
Scheduler
From Celery's send_task
API Register a task foreach target book1. OCR reservation Form

Receives tasks from Celery and load them into two
different queues:
1. Normal Queue
Load tasks in FIFOs and distribute them to workers
2. Priority Queue (interrupt tasks)
Interrupt the queue and distribute them to workers

The OCR system
combines open source
technologies and was
built in-house.
5.MySQL Backend
Metadata on the results of theOCR process is stored.

4.Celery Worker
OCR processing
Worker0(GPU:0)Worker1(GPU:1)・・・
Worker15(GPU:15)• Saving OCR text data
• Indexing to NDL Digital Collections

NDLOCR Tutorial
Google Colaboratory's version of NDLOCR, which can be used easily from a web browser, and tutorial
materials on how to use it are provided.
NDLOCRおよびNDL古典籍OCRのver.2を用いたノートブックを作成しました。 (zenn.dev)
Google Colabを用いたNDLOCRアプリの使い方の動画を作成しました。 (zenn.dev)(Prepared by Dr. Satoru Nakamura, Assistant Professor at Historiographical Institute, the University ofTokyo)

An instructional video on how to use it is also available:
https://youtu.be/46p7ZZSul0o

We hope you will make use of this service to convert
your own library's digitized materials into text!

17

2． NDL Lab’s Experimental Services(1) NDL Ngram Viewer(2) NDLkotenOCR (NDL古典籍OCR)(3) Next Digital Library18

(1) NDL Ngram Viewer
https://lab.ndl.go.jp/ngramviewer/• Experimental analytic tool for the corpus ofdigitized books and periodicals (2.3 millionitems)• Displays a graph showing frequency at which
the queried keywords appear over a given
timeframe• Useful features:- Supports regular expression search- Target corpus can be filtered to include only
books, periodicals, or PD books.

Source code (CC BY) and dataset (frequency
statistics generated from the text data in PD)of the service are available on the NDL Lab'sGitHub.
19(1) NDL Ngram Viewer
Example:「我思うゆえに我あり」（Cogito ergo sum.）• A dictum coined by the French philosopher René Descartes

https://commons.wikimedia.org/wiki/File:Frans_Hals_-_Portret_van_Ren%C3%A9_Descartes.jpg• This phrase has been translated into Japanese in various ways.
「我思う故に我あり」「我思う故に我在り」「われ思うゆえにわれあり」「我思うゆえに我あり」「われ思う故にわれあり」「われ思うゆえにわれ在り」「われ思う故にわれ在り」「我思ふ故に我在り」「我思う故に我有り」「われ思う故にわれ有り」「吾思ふ故に吾在り」……

All of these phrases mean “Cogito ergo sum”.

(1) NDL Ngram Viewer
Example:「我思うゆえに我あり」（“Cogito ergo sum.”）• It is difficult to search for all the keywords you need when there aremany possibilities to represent.
• The use of regular expression search makes this problem a whole loteasier.
「(われ|我|吾)(思|おも).*(故|ゆえ)に.*り」

(2) NDLkotenOCR : OCR Experiment for Pre-Modern Materials• In FY2022, the R&D Office completed
in-house development ofNDLkotenOCR, an AI-OCR software forgenerating full-text data of pre-modernmaterials (mostly before 1868).

• Source code of NDLkotenOCR isavailable to the public. (CC BY)
https://github.com/ndllab/ndlkotenocr_cli
https://doi.org/10.11501/2558997

English version of the information page! https://lab.ndl.go.jp/data_set/r4_kotenocr_en/22(2) NDLkotenOCR
Full text search using NDLkotenOCR
A full-text search function is provided in the Next Digital Library (see below) using text data for about
80,000 pre-modern materials created by NDLkotenOCR (This text data is not yet included in the NDL
Digital Collections.)
Although there is still room for improvement in recognition performance and some materials cannot beread well, it is useful for getting an approximate idea of the content (median value of 0.92).

Search results of「おきな（old man）」
竹取物語 - 次世代デジタルライブラリー (ndl.go.jp)

Search results of 「月蝕（lunar eclipse）」[師守記] - 次世代デジタルライブラリー (ndl.go.jp)23

(3) Next Digital LibraryService URL: https://lab.ndl.go.jp/dl/⚫ Main Functionality• Full-text search of OCR-generated text
• Automatic extraction and listing of illustrations in documents• Image search function (search for similar illustrations)and so on…

⚫ Search target
280,000 books and 80,000 pre-modern materials in the publicdomain available at NDL Digital Collections (http://dl.ndl.go.jp/)➢ It experiments with next-generation library technologies for digitized materials in the public domain.
➢ Technologies that prove to be effective will be included in NDL Digital Collections and elsewhere.
24

(3) Next Digital Library
Also used to demonstrate how services are presented.

OCR text data can be overlaid on theimage by pressing the "T" button.

Text data and image data can be
downloaded from the download
button in the lower right corner.

The hit points of the full-text search are displayed with pins on the image.

25

(3) Next Digital Library“Image-to-Image” Search
Image search function by image

https://lab.ndl.go.jp/dl/illust/search?image=2553657_75_2

https://lab.ndl.go.jp/dl/illust/search?image=1208153_217_0
26

(3) Next Digital Library“Text-to-Image” Search

Example 1• Image search function by free text
• Supports multilingual queries by using
machine translation
Example 1:“可愛い犬” (“cute dog(s)” in Japanese)
Example 2:“un homme jouant du violon”(“a man playing the violin” in French)

Example 2

Example 3:“พระอาทิตย์ตก” (“Sunset” in Thai)
Example 327Ecosystem of New Library Services
R&D activities

Next Digital LibraryFeedback

NDL Ngram Viewer

Field trial on experimentalservices

No
Not yetGo?

Yes!

NDL Digital Collections

Japan Search

Implementation into
regular library services
28