Developing new library services
using AI (machine learning) :an introduction tothe Next Digital Library
Tahee ONUMA, Chief, R&D Office
National Diet Library, Japan
18 September 2021

The R&D Office Founded in October 2011 Conducts research into cutting-edge technologies incollaboration with researchers from outside the NDL

 Releases experimental or newly-developed services on the NDLLab website(⇒ next slide), such as:

 The Next Digital Library *today’s main topic
 NDC Predictor

 Japan Search Bibliographic Information Retrieval and Visualization System…

The NDL Lab
 Website for conducting

field trials of new libraryservices First launched in May 2013,renovated in March 2020

 Recent activities focus onmachine learning

https://lab.ndl.go.jp/

Why machine learning?
Machine learning helps improvesearchability based on content

Keyword full-text search and illustration searchreadability of digitized images

Bleaching, automatic image processing
Functionality like this is implemented and tested on theNext Digital Library.

The Next Digital Library Launched in 2019 by theR&D Office A testing ground forexperimental functionality
developed by the NDL Lab

 Can be used to search theNDL Digital Collection for
documents in the publicdomain Two search modes:
Keyword full-text searchIllustration searchhttps://lab.ndl.go.jp/dl/

Keyword full-text search 30,000 items on industrial andcommercial subject matter(Category 6 of the Nippon
Decimal Classification)
 Searches full texts generated byOCR

 Identify and access individual

pages that include keywords

Illustration search (1)Search for similar images
23,000,000 public-domain images from 336,000 digitized books, rarebooks, and historical materials in the NDL Digital Collection

Increased potential of information exploration that goes beyond

keyword searchIllustration search (2)Image extraction: deep learning method “semantic segmentation“→ Will change in the near future
Feature extraction: trained using ImageNet dataset

Search engine: vdaas/vald (https://github.com/vdaas/vald)Blue regions
indicate areas
extracted toillustrations.

Background bleaching Whitens pages discolored due to aging

 Improves readability
 Based on Neural Network model “pix2pix:GAN”

Automatic image splitting
 Automated splitting of pages and removal of margins

 Optimizes browsing on smartphones and tablets
Detection of the gutter position

Detection of the printed page

Further application to other services
Research for the Next Digital Library
Detection of splitting position

Image search

A tool to assist quality inspections ofdigitized materials.

Search thumbnail images inJapan Search(https://jpsearch.go.jp/)

OKNG

Centered

Forced out

Library services from the National Diet LibraryJapan Search A national, integrated searchplatform for digital archives inJapan Launched August 2020
 Search 22.8 million metadatafrom 131 databases (August2021)

 Similar Image Search function

using technology tested on
the Next Digital Libraryhttps://jpsearch.go.jp/

GitHub Provides access to programs (CC BY) and datasets (PD)created by the R&D Office A hub for the exchange of expertise with engineers from allover the world

https://github.com/ndl-lab

Ongoing projects
 Development of OCR software Preparing OCR-generated texts of digitized materials

on the NDL Digital Collections

 Providing access to high-quality datasets