京 都 大 学 防 災 研 究 所 年 報第47号 C 平成16年3月

Annuals of Disas. Prev. Res. Inst., Kyoto Univ., No. 47 C, 2004

Digitization of disaster prevention Printed Matters and Video information,andconstruction of the search engine which can search theseon the Internet website at high speed

Hideki MATSUURA, Kenichi TATSUMI, Hideo TAGAWA,
Yoshinori YOSHIDA, Tsutomu MIURA, Tetsuro TAKAYAMA,
Hiroo WADA and Norio HIRANO

Synopsis
At Disaster Prevention Research Institute (DPRI), a large amount of disasterprevention research base Printed Matters has stored since 1951. The aim of assignment
research subject of Division of Technical Affairs is that these base materials are widely
exhibited to society, and can be used. In this paper, we show digitization of the DPRI
Annuals that is published every year, and construction of the search engine which cansearch the Printed Matters on the Internet website at high speed in fiscal year 2003.
Much workforce and time was needed for the digitization work. We reduced adocument and picture size of the Annuals to the minimum, which were comprehensible.
In addition, we devised so that the work efficiency was enhanced. As a result, about20,000 pages of the DPRI Annuals (12 years) were exhibited to the public on Internetwebsite in a short period. The three search engine services, which users can be selected,
has been available on the Internet website in May 2003. Anyone enabled the above
ingenuity to browse the Annual documents easily, and the search engine that can
respond to a large user layer was realized.
Keywords: DPRI Annuals; OCR; Digital image; PDF; search engine; database

1. Introduction
Disaster Prevention Research Institute (DPRI)
was established in 1951. The field of research inDPRI now covers a wide range of disaster-related
topics. DPRI plays a key role in the research on"Investigation of disaster theory and construction
related to disaster prevention study”.
Recently, telecommunication and multimedia
technologies have developed rapidly. As a result,much information is exchanged all over the world.

Therefore, to retrieve only the most necessary
information and effectively use the information isimportant.
To date, DPRI has accumulated a large amountof disaster prevention materials. To further promotedisaster prevention research, dissemination of Printed
Matters in these materials is necessary. If thesePrinted Matters were available on the DPRI Internet
website, that would be informative to the public.
We paid particular attention to the DPRI
Annuals that are an important component of Printed

Matters. After the first publication in 1958, theAnnuals continued to be published up to now.
However, obtaining the old Annuals is difficult
because the reprint of the old Annuals is difficult.
The data of disaster prevention in the Annuals is notreadily available, when the public require the data.
Therefore, we wrote this paper to the methods toexhibit the Annual documents on Internet website,
and to enable the users to acquire the documents at
any time. Specifically, the Annual paper documents
were converted to the Portable Document Format(PDF) files, and were exhibited on the DPRI Internetwebsite. With that three search engine wereconstructed, which is usable for the public to acquirenecessary data in the DPRI Annuals.
Note that a particular bit of software (AcrobatReader) is required to open the PDF file. Acrobat
Reader can be downloaded for free on the website of
Adobe Systems Incorporated Company. Because
PDF format is a de facto standard for electronic
documents in Japan, we chose the PDF format.
2. Digitizing the DPRI Annual documents
We started to digitize the Annuals from No.45.

outlined below:[1] Each page of the papers is converted into digital
image (bit map image format) files on an imagescanner and is saved.
[2] The blots of digital image files are eliminated on
the Picture edit software.
[3] Digital image files made in process [1] and [2] areconverted to PDF files on WinReader8.0 (a type ofWindows OCR application software).
[4] The character strings of the Annual documents areextracted from digital image files on WinReader8.0.
The strings are saved in the text file. In addition, weinput strings data into a database. Process [4] wasdone simultaneously with process [3].
Most work was relatively simple, however muchworkforce was needed. Therefore, we trained parttime staffs, who supported us the work from January
to March, 2003. If the work of these processes isdone in order as they are, it is should be emphasizedthat great time will be spent on the work.
Hence the processes were divided into as
smallest possible tasks, for the increase in efficiencyof time shortening and the work sake. Consequently,
there were not great differences between individuals
in the outcome of the work. The operational
flowchart is shown in Figure 1.

2.1 Processes of digitizing the Annual papers
The sequence of digitizing the Annual papers isFig. 1

Digitizing DPRI Annual documents

In addition, technical officials did the more
complex part of the processes that was required
professional skills. The three specific works are as
follows:･ Supervision of part-time staffs’ work andmanagement of the saved data
･ Standardization of the format of digital image datain the process [2]･ Making the website for exhibiting the Annuals andconstructing the DPRI Annuals search engine
2.2 PDF documents of the DPRI Annuals
In the process [3], the character strings of theAnnual documents with digital images were able tobe converted to PDF file. However, for “Loading
PDF file error”, there may be one problem that PDF
files, which contain Japanese fonts, could not bebrowsed in foreign countries. The following two
methods were employed to solve this problem:[A] The Japanese fonts are built into a PDF file.
[B] Only digital images are converted to PDF file.
Method [A] requires time and cost very much,
hence we chose method [B] to make PDF files
composed only of digital images. The work describedherein was conducted from January to March 2003.
As a result, about 20,000 pages of the DPRI Annuals(12 years) are exhibited to the public on Internetwebsite. The search engine service that offers data ofthe DPRI Annuals on the Internet website began inMay 2003.

We constructed the three search engines tosearch data of the Annuals to be efficient andconvenient for users. Users’ knowledge related toDPRI Annuals varies depending on their backgrounds.
Thus, we enabled the users to select the mostappropriate search engine service for users.
3.1 The index search engine
The index search engine is directed to the user
who has accurate knowledge about the Annuals suchas publication year and other information. The list ofvolumes (Fig. 2) and indexes of each volume (Fig. 3)
were written in HTML files. In the HTML file ofindexes, the link to a PDF file of each Annual
document (Fig. 4) is specified by the document title.
After the document is found in indexes, users canclick the title that users wish to access and browse a
PDF file of the Annuals.

3. Construction of the search engines for browsing
the DPRI Annuals
The search engine for browsing the DPRI
Annuals that we constructed this time consists ofthree components:
・ The Index search: Users choose the volume
number of the DPRI Annuals and search theAnnual document from the displayed indexes.
・The Category search: Users search the Annual
document using the information of the limited
categories which users know (title, author,
publication year and other information).
・ The Full-text search: Users search the Annual
document using arbitrary words.

Fig. 2 DPRI Annuals list

Fig. 3 DPRI Annuals indexes

Fig. 4 A PDF file of DPRI Annuals

3.2 The category search engine
If users don’t know in which volume the Annual
document that users wish to browse, users may beunable to search the document with the index search.
We constructed the category search engine sothat users can search the Annual document withinformation of categories such as titles, authors,synopsis, etc.
The system structure is “Apache+PHP+MySQL”, the soft are the open source software.
MySQL is easy to use compared with expensive“business” databases like Oracle or SQLServer. The
server script language “PHP” which can directly
connected to the database operates as a module ofApache. The structure has high affinity with database,and has smaller burdens than the CGI, which runs
with a different process. The system structure ofdatabase”Apache+PHP+MySQL”
which
iscombined both of them, is supported throughout theworld.

We chose “Apache+PHP+MySQL” structure
because this structure has an excellent cost
performance and has special features (high speed andstability). Note that PDF files of the DPRI Annuals
contain only digital images. Thus, we extracted thestrings data of Annual document related to categories
and compiled the data into Microsoft Access database.
The Microsoft Access database is specified interface
parameters with MySQL ODBC Driver to beexported to the MySQL database.
To use the category search engine, users have towrite arbitrary words in each text box that users cantype the category keyword into, and pushthe ”search” button on the search form of category
search website (Fig. 5). Then users select the Annual
document that users wish to browse among the search
results and push “pdfview” in title part (Fig. 6).
Consequently, a pdf file of the Annuals PDF
document appears.

Fig. 5 The category Search engine

Fig. 6 Result of a search

The workflow of the category search engine isshown in Fig.7. A part of “query” receives thekeyword parameter that users typed from the searchform of category search engine website, and searches
the keyword in the database table “Annual”. The
database table employs “ID” (Table 1).
To extract only the data that fits specificconditions, we use “SELECT” command; besides, formore obscure conditions search, we use “LIKE”
command. In such search, fields of titles, authors, keywords, the synopsis, publication years and volume
numbers are searched. In addition, records thatinclude some keywords carried from the search form
are extracted.

Searching form
Push the button

Input
Database

Table.1 Field of ID table
Field Name

Data typeID

Auto NumberYearInt(11)TitlejMediumtext

AuthorjMediumtext

Keywordj

Mediumtext

Synopsisj

Mediumtext

Title

Mediumtext

AuthorMediumtext

KeywordMediumtext

Synopsis

Mediumtext

Numberj

Varchar(50)NumberVarchar(50)Page

Varchar(50)LinkVarchar(50)

Link2

Varchar(50)「Search」“Annual”
QueryID table

Fig. 7 The workflow of category search engine

3.3 The full-text search engine
If users have little knowledge of the category
keywords that related to the Annual document, users
take time to find one in the index search engine or inthe category search engine.

Thus, we constructed the full-text search enginethat enables them to retrieve with a few free words.
Structure of the search engine is “Namazu”, the only
open source software in the Japanese full-text searchsoftwares. “Namazu” is a free and easy-to-use
Japanese full-text search engine. If “Namazu” runs asa CGI, the search engine can be constructed. Notethat several softwares such as “Apache”, “Perl”,“File-Mmagic”, “nkf”, and “KAKASI” are required
for the full-text search on the Internet website. Thesoft wares are open source soft wares and have good
cost performance.
Note that the PDF files of the Annual documents
contain no text data. Accordingly we made LinkHTML files (reference file that refers to acorresponding PDF file of a Annual document) that
added full-text of the Annuals document. Accessing
the Link-HTML, users can browse the PDF files ofthe Annual document after a few seconds. The flowof the search engine is shown in Fig. 8.
To use the full-text search engine, users have towrite free-word in the blank space (“Input text box”),
and push the “search!” button (Fig. 9). Then users

select the Annual document that users wish to browse,
and push the document title or "Link is here" in thesearch result list (Fig. 10). Consequently, a PDF fileof the Annuals PDF document appears.

“Input text box”

Fig. 9 Input Keywords

Fig. 8 The full-text search engine

Fig. 10 The full-text search result list

4. Conclusions
By the end of fiscal year 2003, we completed
digitization the DPRI Annuals from No.34 to No.45.
The service of the three search engines that enablehigh-speed search of the Annuals has run since May
2003.
In fiscal year 2004, we are going to continue thework of digitization of the remaining Annuals
documents and construct upgraded search engine thatoffer the data of many other disaster prevention
Printed Matters in DPRI (Video, Printed matterexceeding A4 size).
Acknowledgments
This research was supported by the Japanese
Ministry of Education, Culture, Sports, Science andTechnology (MEXT) 21st Century COE Program for
DPRI, Kyoto University (Program Leader: Prof.

Yoshiaki KAWATA, Assignment research Leader:Prof. Haruo HAYASHI) as well as the Grant-in-Aidfor Scientific Research (Principal Investigator: NorioHIRANO, Kyoto University).
The authors wish to thank the colleagues andprofessors that helped us on this project and the 21stCentury COE Program committee for allowing us toresearch on this project.
References
Satoshi TATEOKA, (2003): A guide to a formal WeB
Database system by MySQL+PHP -Point of Web
application development Advanced server side
programming -, Gijutsu-Hyohron co.
Hajime BABA, (2001): Construction and practicaluse of a system - Japanese full-text searchthoroughness guide-, SOFTBANK co.
Haruo HAYAMI, (2002): Database IT Text, Ohmshaco.

要 旨
防災研究所が設立されて以来、研究成果として膨大な印刷物が蓄積されてきた。本稿では、印刷物の中でも毎年発刊される年報の電子ファイル化と、インターネット上で年報文献を高速検索できるシステム構築について、平成15年度分を報告する。文献の電子ファイル化作業は、多大な労力と時間を必要とするが、作業効率をあげることで短期間に2万ページの文献を電子ファイル化した。また、ユーザが用途に応じて検索方法を選択できる、より広い利用者層に対応した年報検索サービスを実現させた。
キーワード: 防災研究所年報; 光学式読取装置; デジタル画像; PDF;検索システム; データベース