Overview of Japanese and English Information Retrieval Tasks (JEIR)
at the Second NTCIR Workshop
Noriko Kando, Kazuko Kuriyama, Masaharu Yoshioka{kando, kuriyama, yoshioka}@nii.ac.jp
Abstract

Task: Topics
J-J:
E-E:
J-E:

1. Introduction

J

Documents
J

E
J

Results
J

E

E

E

E

E-J:
1

J-JE:

E

J

J

J

J

JorE

E-JE:
E

where J: Japanese, E: English
MonoLIR

CLIR

1;

# Groups** *922**** ** **CanadaJapan
KoreaTaiwanUK
USATotal************

2. Task Descriptions for JEIR Tasks
2.1 The Procedures
::

MLIR

:

CLIR

J- E- sum J-E E-J J- E- sum tot
J E
JE JE
al#Groups
22 11 22
Enrld#Groups
Submitted
17 7 17#Submitted
Runs
93 18 111*
*11314111614 11 111731121014256440 30 14 1195 206:

3.1 Documents
Training Set
E Collection(NTCIR-1)

J Collection
NACSIS Test Collection 1

Conference Paper Database

NACSIS

JCollection
Collection

Collection

AcademicE
E Collection
J Collection

J CollectionE

3. The Test Collection
Test Set

TOPICSDOCUMENTS

NTCIR-1Topics0001-0083

NTCIR-1

NTCIR-2
Topics0101-0049

training

English-Japanesealignments areknown

NTCIR-2
New Documents
test

English-Japanesealignments are
unknown

J Collections

E Collections

J Collection

E Collection

Segmented Japanese Texts<REC><ACCN>kaken-j-0924516300</ACCN><YEAR>1992</YEAR><SBJ1 TYPE="kanji">802@:</SBJ1>@ @ Q@ @<PJNM TYPE="kanji">
Q [[ QQ@ @ ­Q </PJNM><ABST TYPE="kanji"><ABST.P> Q @ @A@
Q@@ @@ @@ @ûe@@Q[[ @ @@A@@ @@ @Q
@ @@@ @Â\@ @@@ @ @E@ ­@@ @@@ @@B@3QN@ @@ºL@ @@ @@@@­@@@B</ABST.P><ABST.P>1@.@Q [[ @ @Qv@ @Q Í@ @@A@SGML@Q `@ @@@E@ ¦Qv@¦I@ @L @@@ @ LQ ®QDQL
QÝ @ @(@Document Query Language@)@ @@B@SQL@ @@ @A@
Q@ @µ@@ L@@ Â \ @@@ B@ @Q@ @}</ABST.P><ABST.P>2@.@
I@ @ ¦@ @A@v @ @@w @@Q@E@ ¦Qw¦@ @[@ @@ @Ý @ @A@ [[Q @ @[[( @ GUI @ ) @@ p@­ @@ B
Q@ @ Q ¬Qv</ABST.P><ABST.P>3@.@
Q @ @ñ Q@ @Q [[ Q Q
Q @ @@ @A@DQL@ @L @Qv@ @ @@ [@ @@ÄpQQ Z@Q @ @ ­@@B</ABST.P><ABST.P>4@.@ [@ @ [@ @ÚQ ®@ @ ­@ @A@LAN@@ISDN@
î@ ®@@® @ @ F@@B</ABST.P><ABST.P>5@.@
I@ @ Q«\@A@g
QS @A@Q¸x@@ @@ ¿@@A@Àp@ @@ @Û @ @ð Q ®
@ ¢@@B</ABST.P><ABST.P>@ @@@ [@ @«\@ @ü @ @ v«@
F@A@ ¢@ @ Ê@A@ñ Q@ @Q
æ@ @[ QID@ @@@ @L @@@ @@@A@@@ @À@@@ @Û @ @@B@@A@ [@ @@ @w @@Qv@ @@ ¦@@ v@ @@@ @@ @@A@ Ä@@@ @ ®@@À @ @@ ¢@@@ @Û @@B</ABST.P><ABST.P>È @ @ Ê@A@ Q@ @Ý Q O@A@@À Q ®@ @«@ @ F@
@A@Û @ @ @@ @ j@¦ @@ @@@@A@Àp»@Â\«@ @¦
@B</ABST.P></ABST><KYWD TYPE="kanji">
Q [[ /Q/Q/ SGML / GUI / UQ </KYWD></REC>HAPPINESS/BASE3.5<REC><ACCN>kaken-j-0924516300</ACCN><YEAR>1992</YEAR><SBJ1 TYPE="kanji">802:</SBJ1><PJNM TYPE="kanji">[[­ </PJNM><ABST TYPE="kanji"><ABST.P>
 A
ûe
[[A
Â\E
­
B3 N
ºL

­
B</ABST.P><ABST.P>1.
[[v
Í
A
SGML
`
E ¦v
¦I L
L ® DQL(Document Query Language)
ÝBSQL
AµL
Â\B</ABST.P><ABST.P>2.
} I
¦ Avw
E ¦w¦
[ÝA [[[[ (GUI) p­
B</ABST.P><ABST.P>3.
¬v
ñ[[ADQL Lv
[ÄpZ@­
B[Ú®
­</ABST.P><ABST.P>4. [ALANISDN î®
®
FB</ABST.P><ABST.P>5.
I«\Ag S A¸¿AÀpÛðx
®
¢B</ABST.P><ABST.P>[«\ üv« FA ¢ÊAñQ æ[ IDLAÀÛ
B
A [wv
¦v
A Ä®
À¢Û
B
</ABST.P><ABST.P>È
ÊAÝ
OA
À®
«
F
AÛ
j¦
AÀp» Â\« ¦B</ABST.P></ABST><KYWD TYPE="kanji">[[ /// SGML / GUI / U </KYWD></REC>

A sample of the document recordminmax

average stdev

NTCIR-1 Conference Paper Abstracts
J Collection
E Collection

01

8,003
4,363

521.7 224.0
531.9 254.9

NTCIR-2 Grant-in-aid Report Sub-file
J Collection
E Collection

515

17,111
20,396

1,366.8 291.4
1,614.6 492.7

Newspaper articles (Japanese only)
IREX

20

15,770895.9 977.8

J Collection (NTCIR-1)
180000160000140000120000100000
80000
60000
40000

7501-7750:

6001-6250

6751-7000:

5251-5500

4501-4750

3751-4000

3001-3250

2251-2500

1501-1750

0751-1000

20000
1-250

<REC>
<ACCN>kaken-e-2469487463</ACCN><YEAR>1992</YEAR><SBE1 TYPE="alpha">802: *</SBE1><PJNE TYPE="alpha">Development of a Full Text Retrieval
System based on Logical Structure of Documents</PJNE><ABSE TYPE="alpha"><ABSE.P>The investigators haveconducted research and development of a system with which users
can retrieve documents from a full text database containing fulldocuments such as academic papers. Results obtained were as
follows: </ABSE.P><ABSE.P>1. Based on detailed analysis ofretrieval requests for full text databases, they made detailed designof a formal notation DQL (Document Query Language) for
describing retrieval and display requests efficiently according to
the document type definition of SGML. </ABSE.P><ABSE.P>2.
They designed a user system software which displays document
structures and let users select elements with pointing devices andspecify retrieval conditions and display instructions. The system
was developed on work-stations using graphical user interface
systems. </ABSE.P><ABSE.P>3. They developed a server system
software on a main frame computer which stores documents in a
relational database management system by expressing the structure
as binomial relations among elements and processes retrievalrequests described in DQL. </ABSE.P><ABSE.P>4. Theydeveloped a communication method between sever and user
systems and confirmed the functionality by experiments using
LAN and ISDN. </ABSE.P><ABSE.P>5. They evaluated theoverall performance, usability, retrieval precision etc andconsidered the problems and their solutions toward practicaluse.</ABSE.P><ABSE.P>Through evaluation, the necessity ofperformance improvement of the server system was revealed.
Further investigation has made it clear that including referenced
record identifiers within the binomial relation records is effective.
A future issue is to implement this method. The necessity for theuser system to represent specified retrieval requests more
understandably was also revealed. Several methods alreadyproposedshouldbestudiedbyexperimenters.</ABSE.P><ABSE.P>As the result of the research,
the feasibility of the design concept and the implementation of thissystem was confirmed, the approach to existing problems waspresented, and the reality of the full-fledged system wasshown.</ABSE.P></ABSE><KYWE TYPE="alpha">Full Text Database / Information
Retrieval / Document Structure / SGML / GUI / DistributedProcessing</KYWE></RECTopic Preparation
NTCIR-2 Grand-in-Aid

180000160000140000120000100000
80000
60000
40000
20000

IREX-IR NewsPaperArticles

3.2 Topics

7501-7750:

6751-7000:

6001-6250

5251-5500

4501-4750

3751-4000

3001-3250

2251-2500

1501-1750

1-250

751-100004. Evaluations
Discussion Points

andExamples

ofNon-interpolated average precision over all
relevant documents,
Recall-level Precision
Document-level Precision
R-precision

3.3 Relevance Judgments (Right Answers)(i) Appropriate Algorithms and Parameters forJapanese Text Retrieval

(iii) Retrieval of Mixed-Language Texts andEnglish Terms in Japanese Texts(ii) Relationship between Text Segmentation andRetrieval Algorithms(iv) Application for Interactive Systems

5.1 Monolingual Runs

List of Monolingual IR Tasks Active Participants

5. Retrieval Results5.2 CLIR Runs

List of CLIR Tasks Active Participants(2) J-JE Runs and E-JE Runs

(1) J-E and E-J RunsACKNOWLEDGMENTS

6. Summary and Future Directions

REFERENCESiFiF

iiFF