Proceedings of the Third NTCIR Workshop

A Experiment Report about a Web Information Retrieval Systemfor 3rd NTCIR Web Task

Iwao NAGASHIRO
Department of Information and Network, Tokai University
2-3-23, Takanawa, Minato-ku, Tokyo 108-8619, Japannagasiro@keyaki.cc.u-tokai.ac.jp
Dafeng CAO
Beijing Center for Japanese Studies
2, Xi Shan Huan Bei Lu, Beijing, China
cdfeng@163.net

Abstractand verify our new method for retrieval precision. By
the system we joined 3rd NTCIR web task fromWe joined 3rd NTCIR web task from October 2001.

October 2001, and this paper is a brief report of ourFor this task, we constructed a small web

work, it includes basic idea for web information

information retrieval system. By this system, weretrieval,completed “dry run” and “formal run” retrieval

description and some retrieval experiment results.

webinformation

retrievalsystemtopics of the task. In this report we will give a brief
description about our basic method for web

2

Basic Method for Web Retrieval

information retrieval, our web information retrieval
system and some retrieval experiment results.

Here we will show some basic method for our web

Keywords: Information Retrieval, Search Engine,retrieval system. The diagram of our system is given

Site Rank.

in Figure 1. All of the computers are x86-based withWindows operating system.

1

Introduction
2.1 Procedure of our web search system
Internet web sites have increased very rapidly andspread over all of world in resent year. It is possible

We used the test collection resources provided byfor every one to access internet from his office orNII (National Informatics Institute) as original web

home. But because web sites are developed in great

documents.

disorder, it becomes difficult to find necessary

system, we processed these documents by following

information precisely from year to year. However

procedures.

For implementing a web retrieval

information retrieval technology gives some solution(1) Extract important parts (words, sentences orfor this problem. We have constructed a web

short paragraphs) with some discriminating

information retrieval system to get experiment results

tags for making index data file.

© 2003 National Institute of Informatics

The Third NTCIR Workshop, Sep.2001 - Oct. 2002

included in the unit of web article. So we make index
data file by the unit of web article. From these pages
we extract following sentences and save these toUser PC

User PC

User PC

index data file.
(1)

Sentence between <title> and </title>
tag.

Ethernet(2)

Sentence of first level heading between<h1> and </h1> tags.

HUB

(3)

Sentenceofsecondlevelheading

between <h2> and </h2> tags.
(4)

Sentence of third level heading between<h3> and </h3> tags.

Web Server

Library Storage

Library Manager(5)

between <a href=”……”> and </a> tags.
(6)
(7)
(8)
(9)(10)(11)

Sentence in big font between <fontsize=”+2”> and </font> tags.

each keyword from hit original documents.
(12)

Sentence in big font between <big> and</big> tags.

short passage, relevancy, site rank to request
user.

Sentence in big font between <fontsize=”+3”> and </font> tags.

at first.

(5) Return all search results with URL address,

Sentence in big font between <fontsize=”+4”> and </font> tags.

(3) When received a search request, web server(4) If hit, then find the short passage including

Sentence in big font between <fontsize=”5”> and </font> tags.

for each site. Also sort and save these data towill search the keywords from index data file

Sentence in big font between <fontsize=”6”> and </font> tags.

(2) Extract all hyper linked web site from originalsite rank data file.

Sentence in big font between <fontsize=”7”> and </font> tags.

Figure 1. System diagram

documents, and calculate a site rank parameter

Sentence of description of hyperlink(13)

Sentence in emphatic font between<strong> and </strong> tags.

2.2 Making Index Data File
Because a web site is consisted by a lot of(14)Sentence in list after <li> tag.

2.3 Ranking of Web Site

homepage, and every homepage includes many
hyperlink to other homepage generally, we think that

In order to provide a parameter for the quality of

the information of a target homepage should be is

web site, we used a site rank parameter which can begathered not only from the target page itself but also

calculated by count the appearance times of it isfrom all of these pages linked from the target page.

linked from other pages with different URL address.

As the link page from a target page can be divided

For the real Internet it is difficult to show the site

into two type, the first type of these link pages haverank. By using NII test collection, because it is asame URL address with the target page, and second

collection of offline web documents, we try to findtype of these link pages have different URL with thethe site rank of all web site appeared in the collection

target page. We think the target page and all of link

documents. The flow chart for calculating site rank is

pages with same URL address should be considered

shown in Figure 2.

as one web article. Specific information properly isProceedings of the Third NTCIR Workshop

begin

begin

Extract URL address of hyperlink
from all documents

Look for each keyword from index
data file

Save URL address list to data file

Hit != 0

Calculate site rank for each URL
address

Yes
Calculate site rank for each URL
address

No

Return message of"no hit"

Logic operation of search results
fo each keywordSort site rank in decrease order

Find sentances include keyword
from original documents
Save site rank to data fileCalculate relevance parameter
end

Find site rank for hit site
Figure 2. Flow chart for calculating site rank

2.4

Return URL, passage sentences
relevance and site rank to user

Search Information by Keywords

For the use of information retrieval by internet
browser, a web server with search engine must beconstructed. Our search engine provides a threeend

keyword input fields, and also provides a compoundsearch condition by a logical expression of three
keywords. Generally, the search engine looks for

Figure 3. Flow chart of search engine

each keyword from index data file, and then provides
results of logical operation specified by user’s

3System Description

request. For the retrieval precision, we search thekeywords from original documents when theThe diagram of our web information retrieval test

keywords is found in index data file, and also wesystem has been shown in Figure 1. The libraryshow the relevance parameter by search results fromstorage is a hard disk for original web documents,original documents. The flow chart of our searchindex data file and site rank data file. The libraryengine is shown in Figure 3.

manager is a computer for making index data file,

The Third NTCIR Workshop, Sep.2001 - Oct. 2002

and calculating site rank. And the web server isThe PC screen of user’s request input is shown incomputer that provides web services, especially aFigure 4. Three keywords can be specified in oursearch engine service. Some detail description issystem, and user’s retrieval request can be given by agiven as following.

logical expression of the three keywords.

3.1 Server machine
The server machine is an x86-based personal
computer, its CPU clock is 1000MHz, and its
physical memory is 2096KB, and also a 500GB raid
is included. The operating system of this machine isMicrosoft Windows 2000 Server.

3.2 WEB Server
For realizing a web search engine, we used a free
web server of Apache for Win32. Also we configured
it to execute CGI program. The programming

Figure 4. User’s request input field

language we used for CGI is Perl. And the Perlinterpreter we used for our system is ActivePerl

4.2 Search Results

5.6.1.
The PC screen of messages for each keyword is

3.3 CGI Program

shown in Figure 5. The system can show some detailmessages if the keyword is hit in a web document,In order to construct a web information retrieval
system described in section 2, we prepared three CGI

which includes a document number, passages, andname of HTML tag.

program.
(1) makeindex.cgi: This program makes a index
data file from original web site documents.
(2) siterank.cgi: This program gives a rank
parameter of a web site by count theappearance times of it is linked from otherpages.
(3) searchft.cgi: This program accepts user’s
requestandprovidessearching

results

including some evaluation parameters.

4Retrieval Experiment
Here we will give a short description of our web

Figure 5. Messages for each keyword

information retrieval experiment by PC screen of
search engine program.

The PC screen of total search results for user’s
request is shown in Figure 6. This table is listed in

4.1 Input Field

document ID, title of the page, relevance to user’s
request, URL address and rank of the site. The tableProceedings of the Third NTCIR Workshop

is sorted in decrease order of relevance.

References[1] The Clever Project. Hypersearching the Web.
Scientific American, June, 1999.
[2]

Website:SEARCHENGINE

http://www.searchenginewatch.com

Figure 6. Total results for user’s request

5Conclusions
By the information retrieval experiments of 3rd

NTCIR web task, we have found some problems thatshould be improved.
(1) Because a web site includes many pages forrelated contents, we should search the user’s
request from all page of the site, so that weshould give a method to determine that which
hyperlink is belong to the site and which one isnot.
(2) Our system needs to read the index file fromhard disk every time, so that if the web server
can preload the index file, the speed of thissystem will be improved obviously.
(3) Generally a Perl program needs more run time,
we should improve the system performance by
updating a high speed Perl interpreter (forexample, PerlEx of Active State)
We will continue our work to give a more precise
method, and improve the web retrieval system.

Acknowledgment
The authors would like to thank web task co-chairof 3rd NTCIR for their support and suggestions in thisexperiment. The authors also would like to thank NII(National Institute of Informatics) for the permissionof the Open Laboratory.

WATCH,