Proceedings of the Third NTCIR Workshop

An English Translation of the Topics Used for the Evaluation of
the Web Retrieval Task at the Third NTCIR Workshop
NTCIR-3 Web Retrieval Task Organizers
ntcadm-web@nii.ac.jp<TOPIC><NUM>0008</NUM><TITLE CASE="b">Salsa, learn, methods</TITLE><DESC>I want to find out about methods for learning how to dance the salsa</DESC><NARR><BACK>I would like to find out in detail how best to learn how to dance the salsa, which iscurrently very popular. For example, if I should go to dance classes, I need detailed information such aswhere I should go and what the class would be like. Documents simply saying that it is popular without
giving any detailed information are irrelevant.
</RELE></NARR><CONC>Salsa, learn, methods, place, curriculum</CONC><RDOC>NW011992774, NW011992731, NW011992734</RDOC><USER>1st year Master's student, female, 2.5 years search experience </USER></TOPIC><TOPIC><NUM>0010</NUM><TITLE CASE="b">Aurora, conditions, observation</TITLE><DESC>For observation purposes, I want to know the conditions that give rise to an aurora</DESC><NARR><BACK>I want to observe an aurora so I want to know the conditions necessary for its
occurrence and the mechanis m behind it.</BACK><RELE>Aurora observation records, etc. list the place
and time so only documents that provide additional information such as the weather and temperature at
the time of occurrence are relevant. </RELE></NARR><CONC>Aurora, occurrence, conditions, observation, mechanism</CONC><RDOC>NW003201843, NW001129327, NW002699585</RDOC><USER>1st year Master's student, female, 2.5 years search experience</USER></TOPIC>© 2003 National Institute of Informatics

The Third NTCIR Workshop, Sep.2001 - Oct. 2002

<TOPIC><NUM>0011</NUM><TITLE CASE="b">Japanese envoy to Tang Dynasty China, customs, culture</TITLE><DESC>I want to learn about customs that have become part of Japanese culture that were brought backfrom China by Japanese envoys sent to Tang and Sui Dynasty China during the Nara and Heianperiods.</DESC><NARR><BACK>Interest in China has grown in recent years and I'm interested in cultural links between
China and Japan.</BACK><RELE>Relevant documents will explain what culture Japanese envoys sent
to Tang and Sui Dynasty China during the Nara and Heian periods brought back with them, and what
remains a part of Japanese customs today. Detailed explanations of what was brought back are notnecessary but documents must mention that it was first introduced into Japan by a Japanese envoy toeither Tang Dynasty China or Sui Dynasty China.</RELE><NARR><CONC>Japanese envoy to Sui Dynasty China, Japanese envoy to Tang Dynasty China, culture, customs,
Japan, China</CONC><RDOC>NW009980333, NW005148950, NW009946105</RDOC><USER>1st year Master's student, male, 5 years search experience</USER></TOPIC><TOPIC><NUM>0012</NUM><TITLE CASE="b">New Years, ozoni soup, locality</TITLE><DESC>I wish to find out about differences in the ingredients and miso stock used to make ozoni soup at
New Years in each region.</DESC><NARR><TERM>For this purpose, ozoni refers solely to the soup with rice cakes eaten at New
Years.</TERM><RELE>Relevant documents will describe what region the ozoni soup is from, and whatkind of ingredients and miso is used. </RELE><NARR><CONC>New Years, ozoni soup, locality, region, miso, ingredients, mochi rice cakes</CONC><RDOC>NW008964739, NW008964750, NW014479093</RDOC><USER>1st year Master's student, male, 5 years search experience</USER></TOPIC><TOPIC><NUM>0013</NUM><TITLE CASE="c" RELAT="2-3">Kyoto, temple, shrine</TITLE><DESC>I want to find more in-depth information on temples and shrines in Kyoto, such as information
pertaining to their historical background and presence in the region</DESC>

Proceedings of the Third NTCIR Workshop<NARR><BACK>I am very familiar with Kyoto but even if I know where something is I know very
little more than that. I would like to be able to provide friends visiting from outside of Kyoto with a little
more knowledgeable information when showing them around.</BACK><RELE>Relevant documents
include all documents providing more information on temples and shrines in Kyoto than just where theyare. Documents that are not particularly objective such as travel literature are relevant if they provide
information other than just where something is located; for example, that it is famous for its big
trees.</RELE><NARR><CONC>Kyoto, temple, shrine, sightseeing, history, background</CONC><RDOC>NW009150865, NW003143867, NW001625667</RDOC><USER>1st year Master's student, male, 5 years search experience</USER></TOPIC><TOPIC><NUM>0014</NUM><TITLE CASE="b">Dreams, future, effort</TITLE><DESC>I wonder what kind of things people embrace as their dreams for the future, and what they do toachieve them.</DESC><NARR><RELE>Relevant documents describe peoples' dreams, and what they do and the effort they
make to achieve them. I am not interested in dreams that are too fantastic or impossible to achieve. Given
that the foundation for the direction of efforts made to realize such fantastic dreams can be ambiguous, let
us use the strength of the link between a dream and the efforts made to achieve it as our criteria forjudgment, and documents describing dreams for which there is only a weak link between the dream andefforts made to achieve it are irrelevant.</RELE><NARR><CONC>Dreams, future, effort, target, plan, oneself, I</CONC><RDOC>NW002695723, NW006763670, NW012273176</RDOC><USER>1st year Master's student, male, 5 years search experience</USER></TOPIC><TOPIC><NUM>0015</NUM><TITLE CASE="c" RELAT="1-2">Ozone, ozone hole, human body</TITLE><DESC>I want to learn about the effects destruction of the ozone layer and expansion of the ozone hole
have on the human body</DESC><NARR><RELE>When was it that destruction of the ozone layer was given mass coverage by the
media? When was it that various media reports on destruction of the ozone layer first appeared? Surely
Freon gas is better controlled than before and levels have fallen accordingly. Or is it that this is having noThe Third NTCIR Workshop, Sep.2001 - Oct. 2002

effect, destruction of the ozone layer continuing and the ozone hole is expanding steadily. The mostinteresting aspect of this debate is what effects this is having on the human body. I have heard that care isrequired when going outdoors in Australia, but surely this will be the case in Japan in the future. At the
moment, I am particularly interested in the effects this is having on the human body, and am looking forrelevant information.</BACK><RELE>Relevant documents will indicate the effects destruction of theozone layer and expansion of the ozone hole have on the human body. Documents that only describe themechanisms behind destruction of the ozone layer and expansion of the ozone hole are irrelevant. Rather
than superficial information, relevant documents will provide detailed explanations of what reactions
actually occur within the body.</RELE><NARR><CONC>Ozone layer, destruction, ozone hole, expansion, human body, effects, ultraviolet rays</CONC><RDOC>NW003634645, NW006375578, NW013077125</RDOC><USER>1st year Master's student, male, 5 years search experience</USER></TOPIC><TOPIC><NUM>0016</NUM><TITLE CASE="b">Genome, drug design, trend</TITLE><DESC>Find documents describing recent trends in genomic drug design</DESC><NARR><BACK>Use of genetic engineering in the development of drugs with the aim of creating ordermade drugs is gaining attention. I would like to survey recent trends in this kind of "genomic drug
design".</BACK><RELE>Relevant documents will provide information on trends in "genomic drug
design </RELE></NARR><CONC>Genome, gene, genetic engineering, drug design, order made treatment, side effect</CONC><RDOC>NW008064950, NW005588746, NW002348955</RDOC><USER>1st year Master's student, male, 3 years search experience</USER></TOPIC><TOPIC><NUM>0017</NUM><TITLE CASE="b">Yakyu, American baseball, comparison</TITLE><DESC>Find documents that compare Japanese and American baseball</DESC><NARR><TERM>The term "yakyu" refers to Japanese baseball, whereas "baseball" is used to describe
American baseball.</TERM><BACK>Not only do the rules for Japanese and American baseball differ,
but various other differences exist, including their aesthetics and philosophy. For example, a big
difference exists in Japanese and American feelings about intentionally walking a batter. I'm interested inreading documents describing such differences.</BACK><RELE>Relevant documents will provide aProceedings of the Third NTCIR Workshop

comparison of Japanese and American baseball</RELE></NARR><CONC>Japanese baseball, American baseball, Japan, America, comparison, differences</CONC><RDOC>NW004164416, NW000013074, NW000013069</RDOC><USER>1st year Master's student, male, 3 years search experience</USER></TOPIC><TOPIC><NUM>0018</NUM><TITLE CASE="a">Rope work, knots</TITLE><DESC>Find documents describing the various knots used in rope work</DESC><NARR><BACK>Rope work is used in various circumstances, such as mountain climbing, yachting andemergency situations. I am interested in the kinds of knots they use for rope work in such circumstances.
</BACK><RELE>Relevant documents will describe knots used in rope work.</RELE></NARR><CONC>Rope work, knots, rope, outdoors</CONC><RDOC>NW003792815, NW002139129, NW014441356</RDOC><USER>1st year Master's student, male, 3 years search experience</USER></TOPIC><TOPIC><NUM>0019</NUM><TITLE CASE="b">Plum tree, place of interest, Tokyo</TITLE><DESC>Find documents describing what kind of places of interest in Tokyo are famous for their plumtrees</DESC><NARR><BACK>There are many places of interest in Tokyo that are famous for their plum trees. I
would like to know where these places of interest are.</BACK><RELE>Relevant documents will note
places of interest in Tokyo that are famous for their plum trees, and descriptions of theseplaces.</RELE></NARR><CONC>Tokyo, Tokyo Metropolitan Area, plum, place of interest, plum festival</CONC><RDOC>NW006419384, NW008668568, NW006752574</RDOC><USER>1st year Master's student, male, 3 years search experience</USER></TOPIC><TOPIC><NUM>0020</NUM><TITLE CASE="b">Speed reading method, effects</TITLE><DESC>Find documents describing the effects of various speed reading methods</DESC>

The Third NTCIR Workshop, Sep.2001 - Oct. 2002

<NARR><BACK>Various speed reading methods are gaining attention, and books on many different
speed reading methods are available from bookshops and the like. I am interested in learning how
effective different speed reading methods are.</BACK><RELE>Relevant documents will describe theeffects of speed reading methods.</RELE></NARR><CONC>Speed reading method, speed reading skills, effects, consequences</CONC><RDOC>NW012685828, NW012685804, NW012685820</RDOC><USER>1st year Master's student, male, 3 years search experience</USER></TOPIC><TOPIC><NUM>0022</NUM><TITLE CASE="b">Stock investment, internet, introduction</TITLE><DESC>Find documents that provide an introduction to stock investment via the internet</DESC><NARR><BACK>Online stock brokerage companies have emerged and we are now able to engage instock investment over the internet. I'm interested in documents that describe stock investment over theinternet in layman terms.</BACK><RELE>Relevant documents will provide and introductory outline ofstock investment over the internet.</RELE></NARR><CONC>Stock investment, stock, internet, online, introduction</CONC><RDOC>NW014785566, NW001862410, NW012845761</RDOC><USER>1st year Master's student, male, 3 years search experience</USER></TOPIC><TOPIC><NUM>0023</NUM><TITLE CASE="b">Extinction, mammals, crisis</TITLE><DESC>Find documents describing mammals on the verge of extinction</DESC><NARR><BACK>I have heard that more than a few mammals are on the verge of extinction due todegradation of the global environment. I am interested in finding out what kind of mammals faceextinction.</BACK><RELE>Relevant documents will provide information on mammals on the verge ofextinction.</RELE></NARR><CONC>Extinction, mammals, crisis, animals, environmental destruction</CONC><RDOC>NW012576928, NW006103434, NW000647550</RDOC><USER>1st year Master's student, male, 3 years search experience</USER></TOPIC><TOPIC>Proceedings of the Third NTCIR Workshop<NUM>0024</NUM><TITLE CASE="b">Taping, method</TITLE><DESC>Find documents describing various methods of taping</DESC><NARR><BACK>Taping techniques are indispensable in sports. I'm looking for documents that describe
methods of taping in an exhaustive manner.</BACK><RELE>Relevant documents will mention methodsof taping. </RELE></NARR><CONC>Taping, taping tape, method, technology, diagram</CONC><RDOC>NW002787478, NW010566071, NW000546948</RDOC><USER>1st year Master's student, male, 3 years search experience</USER></TOPIC><TOPIC><NUM>0027</NUM><TITLE CASE="c" RELAT="2-3">Miyabe Miyuki, book review, review</TITLE><DESC>Find book reviews of novels written by Miyabe Miyuki</DESC><NARR><BACK>The rising female mystery writer Miyabe Miyuki is drawing acclaim. I would like toread reviews by readers in the general public to learn how her novels are being received and as a guide for
future purchases.</BACK><RELE>Documents containing even a single book review of any novel by
Miyabe Miyuki are relevant. However, reviews written for commercial purposes such as online
bookstores

andreviews

dominatedbycatchphrases

are

of

no

value,

andtherefore

irrelevant.</RELE></NARR><CONC>Miyabe Miyuki, novelist, author, book review, review, impression</CONC><RDOC>NW001439430, NW001219389, NW000411423</RDOC><USER>2nd year Master's student, male, 5 years search experience</USER></TOPIC><TOPIC><NUM>0028</NUM><TITLE CASE="b">Copyright, digital content, network</TITLE><DESC>I would like to know how copyright of digital contents are currently being dealt with in the
network-oriented society of recent years</DESC><NARR><BACK>The concept of copyright, how we handle it, and ways of protecting it have undergone
great changes due to the explosive diffusion of the Internet and digitalization of various information.
Moreover, unless users have a correct understanding of copyright, there is the chance that they will
infringe upon it unknowingly. I want to gather materials describing the current status of
copyright.</BACK><RELE>Relevant documents will mention copyright for the content and distribution

The Third NTCIR Workshop, Sep.2001 - Oct. 2002of digital information on networks. As my aim is to gather information, documents need not be complete
or exhaustive. Documents that don't mention copyright directly such as copyright protection technologyoracompanyororganization

announcement

pertaining

toacopyright

arepartially

relevant.</RELE></NARR><CONC>Copyright, information theory, protection of intellectual property, digital content, digital media,
network, internet</CONC><RDOC>NW001662100, NW002120650, NW013092030</RDOC><USER>1st year Master's student, male, 5 years search experience</USER></TOPIC><TOPIC><NUM>0029</NUM><TITLE CASE="c" RELAT="2-3">Speaker, evaluation, comparison</TITLE><DESC>Find documents describing the results of evaluations and comparisons of speaker quality andperformance.</DESC><NARR><BACK>Speakers comprise an important component of AV equipment as it is speakers that
actually produce the sound. As a result, although many speakers are very expensive, various differences
exist in terms of sound quality depending on the manufacturer and the type of equipment. I want this kindof information for buying speakers.</BACK><RELE>Relevant documents will provide information on
the speakers' performance and special characteristics, and comparisons of different speakers' soundquality. As much information is subjective, relevance is not determined by authenticity. Documents notconsidered to contain meaningful information, either because they are extremely brief or vague, such as"the sound quality of such and such is good" are relevant because it is possible they are of value
depending on whose o pinion it is. Only documents comparing the sound quality of speakers are relevant.
Comparisons of the speakers with similar ones for which the amplifier has been changed, etc. areirrelevant.</RELE></NARR><CONC>Speaker, audio system, sound quality, performance, special characteristic</CONC><RDOC>NW002068765, NW006836735, NW013930134</RDOC><USER>1st year Master's student, male, 5 years search experience</USER></TOPIC><TOPIC><NUM>0030</NUM><TITLE CASE="b">Academy Award, recipient, successive generation</TITLE><DESC>I want to know who have been the recipients of successive generations of Academy
Awards</DESC>

Proceedings of the Third NTCIR Workshop<NARR><BACK>Not only are the Academy Awards the greatest film festival in the world for film fans,they also comprise an important index for evaluation of the fame, status, income ranking and various
other aspects of people in the film industry. I am interested in who has basked in this glory. I want to
know

who

have

been

the

recipientsofsuccessive

generationsofAcademyAwards.</BACK><RELE>Relevant documents will provide information on the recipients of successive
generations of Academy Awards (lists of successive recipients, etc.) Documents describing the Academy
Awards but not award recipients are irrelevant. Documents describing the receipt of an Academy Award
when profiling a person in the film industry are also irrelevant.</RELE></NARR><CONC>Academy Award, film, recipient, people in the film industry, director, actor</CONC><RDOC>NW000015594, NW000797878, NW014532075</RDOC><USER>1st year Master's student, male, 5 years search experience</USER></TOPIC><TOPIC><NUM>0031</NUM><TITLE CASE="c" RELAT="2-3">Qualifying exam, information processing, IT</TITLE><DESC>I want to know what kind of qualifying exams exist in the fields of information processing and
IT</DESC><NARR><BACK>Demand for information processing engineers and technicians has increased in recent
years in response to digitalization of information and the IT revolution. As a consequence, qualificationsin the fields of information processing and IT are attracting widespread attention. I would like to acquire
qualifications in these fields but I don't know what qualifications there are and what kind of qualifying
exams

exist,

so

I

would

like

to

find

out

what

these

exams

and

qualifications

entail.</BACK><RELE>Relevant documents will provide information on qualifying exams in the fields
of information processing and IT. Documents introducing vocational colleges for qualifying exams areirrelevant unless they also provide information pertaining to these exams.</RELE></NARR><CONC>Qualifying exam, qualification, engineer, information processing, IT, information technology,
information-oriented society</CONC><RDOC>NW012117536, NW002076911, NW007707153</RDOC><USER>1st year Master's student, male, 5 years search experience</USER></TOPIC><TOPIC><NUM>0032</NUM><TITLE CASE="c" RELAT="2-3">Article 9 of the Constitution, interpret, opinion</TITLE><DESC>I would like to learn various persons' opinions on how Japan's renunciation of war as stated in

The Third NTCIR Workshop, Sep.2001 - Oct. 2002

Article 9 of the Constitution should be interpreted</DESC><NARR><BACK>Issues such as the right to group self-defense, PKO, and PKF emerge from theexistence of the Self-Defense Forces and have always been problematic ni terms of interpreting theConstitution. Renunciation of war is stated in Article 9 of the Constitution but various interpretations of
this exist. It is dangerous to discuss this issue from a fixed viewpoint, and I think acquisition a correctunderstanding of the issue must start with learning various peoples' interpretations. I am looking for
documents

describing

variouspeoples'

interpretations

andopinionsof

Article

9

of

the

Constitution.</BACK><RELE>Relevant documents will note opinions pertaining to Article 9 of theConstitution. Documents introducing publications mentioning Article 9 of the Constitution are irrelevant.
</RELE></NARR><CONC>Article 9 of the Constitution, the Constitution of Japan, Constitution of Peace, interpretation of
the Constitution, renouncement of war, armaments, Self-Defense Forces</CONC><RDOC>NW007974506, NW008332819, NW007981630</RDOC><USER>1st year Master's student, male, 5 years search experience</USER></TOPIC><TOPIC><NUM>0033</NUM><TITLE CASE="b">Ishikawa Prefecture, local product, souvenir</TITLE><DESC>I am looking for local products of Ishikawa Prefecture that will make good souvenirs</DESC><NARR><BACK>I will soon take a trip to Ishikawa Prefecture. As I will buy souvenirs for friends, I
want to find out what kinds of products are available. I am looking for documents describing local
products

andsouvenirs,

and

what

kind

of

local

products

are

good

for

souvenirs.</BACK><RELE>Relevant documents will provide information on local products of Ishikawa
Prefecture. However, documents providing information on products unsuitable for giving as souvenirs areirrelevant.</RELE></NARR><CONC>Ishikawa Prefecture, tourist site, travel, local product, specialty, souvenir, gift</CONC><RDOC>NW006681067, NW014201259, NW003018838</RDOC><USER>1st year Master's student, male, 5 years search experience</USER></TOPIC><TOPIC><NUM>0034</NUM><TITLE CASE="b">Kubrick, film, impression</TITLE><DESC>I want to hear peoples' impressions of films directed by Kubrick</DESC><NARR><BACK>The name Stanley Kubrick invokes images of a distinguished film director leading theProceedings of the Third NTCIR Workshop

times with vanguard imagery in science fiction films such as 2001: A Space Odessy. However, many
people express difficulty in understanding his work because of its abstractness. I am interested in findingout how people view his work. I want to read documents noting their impressions of films directed by
Kubrick.</BACK><RELE>Relevant documents will describe impressions and evaluations of Stanley
Kubrick's films. Documents mentioning Kubrick or his works are partially relevant. However, documents
failing to mention impressions or evaluations are irrelevant.</RELE><TERM>Relevant Kubrick films
include Killer's Kiss, The Killing, Paths of Glory, Spartacus, Lolita, Dr. Strangeglove, 2001:A Space
Odessy, A Clockwork Orange, Barry Lyndon, The Shining, Full Metal Jacket, Eyes Wide Shut, Day of theFight, Flying Padre, and Fear and Desire.</TERM></NARR><CONC>Stanley Kubrick, film director, film, work, impression, appraise</CONC><RDOC>NW009544012, NW013286418, NW001495565</RDOC><USER>1st year Master's student, male, 5 years search experience</USER></TOPIC><TOPIC><NUM>0035</NUM><TITLE CASE="b">Sanguozhi, game, theme</TITLE><DESC>I want to know what kind of games themed around Sanguozhi exist</DESC><NARR><BACK>The Chinese legend Sanguozhi Engi (Sanguozhi) comprises the theme of many games
in Japan. I am interested in finding out what kind of games like this exist.</BACK><RELE>Relevant
documents will introduce, explain, or at the very least list, games themed around Sanguozhi, irrespectiveof the genre and platform of the game.</RELE></NARR><CONC>Sanguozhi, Sanguozhi Engi, game, theme, subject, title, list</CONC><RDOC>NW002293735, NW007006551, NW003137277</RDOC><USER>1st year Master's student, male, 4 years search experience</USER></TOPIC><TOPIC><NUM>0036</NUM><TITLE CASE="b">Pipe organ, concert hall, address</TITLE><DESC>I want to know which concert halls in Japan have pipe organs</DESC><NARR><RELE>Only concert halls in Japan are relevant. Relevant documents will clarify the existenceof a pipe organ, and the address and location of the concert hall.</RELE></NARR><CONC>Pipe organ, concert hall, hall, address, venue, location</CONC><RDOC>NW007798520, NW008842171, NW009965758</RDOC><USER>1st year Master's student, male, 4 years search experience</USER>The Third NTCIR Workshop, Sep.2001 - Oct. 2002</TOPIC><TOPIC><NUM>0037</NUM><TITLE CASE="b">Motorbike, touring, report</TITLE><DESC>I want to read documents containing touring reports on motorbike trips in Japan</DESC><NARR><BACK>I would like to try touring by motorbike so I am interested in documents containing
touring reports.</BACK><RELE>Relevant documents will include places visited, routes taken, andimpressions thereof. Documents only providing impressions are partially relevant. Documents containing
touring reports for countries overseas and introducing motorbikes are irrelevant.</NARR><CONC>Motorbike, touring, report, touring report, domestic, travel</CONC><RDOC>NW011284950, NW006709796, NW006709798</RDOC><USER>1st year Master's student, male, 4 years search experience</USER></TOPIC><TOPIC><NUM>0038</NUM><TITLE CASE="c" RELAT="2-3">Accelerator, medical treatment, treatment</TITLE><DESC>I want to know what kind of treatments accelerators designed for medical applications are usedfor</DESC><NARR><BACK>Treatment with proton rays emitted by an accelerator is thought to be more effectivein treating deep tumors than traditional x-ray treatment. I am interested in learning what kind oftechnology is actually used and what kind of treatment it is used for.</BACK><RELE>Relevant
documents will describe both treatment methods that use accelerators designed for medical purposes andtechnical explanations. Documents providing either only technical explanations or descriptions oftreatment methods are partially relevant. Documents that only introduce institutions that use accelerators
or companies that make accelerators are irrelevant.</NARR><CONC>Accelerator, medical treatment, treatment, apply, cancer, x-ray, cyclotron, synchrotron</CONC><RDOC>NW005902433, NW002457339, NW012795852</RDOC><USER>1st year Master's student, male, 4 years search experience</USER></TOPIC><TOPIC><NUM>0039</NUM><TITLE CASE="b">Miyazaki Hayao, animation, film</TITLE><DESC>I want to know about the works of Miyazaki Hayao, the director of animated films</DESC>

Proceedings of the Third NTCIR Workshop<NARR><BACK>I want to learn what kind of works have been produced by the director Miyazaki
Hayao, father of Miyazaki animation, which is so highly regarded both in Japan and overseas. I also wantto learn what the content of these works.</BACK><RELE>Relevant documents will describe the namesof films Miyazaki Hayao has worked on and their contents. Documents that only describe the names andcontent of works produced at Jubilee Studio by people other than the Miyazaki Hayao are partiallyrelevant. Documents that only introduce products and the impressions of people producing information
are irrelevant.</RELE></NARR><CONC>Miyazaki Hayao, animation, film, Miyazaki animation, Ghibli, work</CONC><RDOC>NW007974394, NW002369854, NW013080912</RDOC><USER>1st year Master's student, male, 4 years search experience</USER></TOPIC><TOPIC><NUM>0040</NUM><TITLE CASE="b">Honjo Manami, works starred in, actress</TITLE><DESC>I want to find out what works the actress Honjo Manami starred in</DESC><NARR><BACK>I want to know what films and dramas the actress Honjo Manami, who is active inboth TV and commercials, has starred in.</BACK><RELE>Relevant documents will provide informationon the works Honjo Manami has played a leading role in. Documents providing information on works she
has appeared in are partially relevant. Documents that don't mention what works she has appeared or
starred in are irrelevant.</NARR><CONC>Honjo Manami, actress, leading role, work, appear, drama</CONC><RDOC>NW011474831, NW006379088, NW001706374</RDOC><USER>1st year Master's student, male, 4 years search experience</USER></TOPIC><TOPIC><NUM>0041</NUM><TITLE CASE="c" RELAT="1-2">Impressionist, Monet, art museum</TITLE><DESC>I want to find out where pictures by impressionist painters can be viewed</DESC><NARR><BACK>The term impressionism derives from the name of a work of art by Monet"Impressionism: Sunrise".</BACK><TERM>People who advocate impressionism in art are referred to as
impressionists.</TERM><RELE>Relevant documents will provide information on impressionist painters,
their works (names only is fine), and which art museum these works can be viewed
in.</RELE></NARR><CONC>Impressionist, Monet, Manet, Sisley, Pissarro, picture, art museum, Louis=Leroy</CONC>The Third NTCIR Workshop, Sep.2001 - Oct. 2002

<RDOC>NW010642903, NW003795687, NW003948114</RDOC><USER>2nd year graduate student, female, 4 years search exp erience</USER></TOPIC><TOPIC><NUM>0042</NUM><TITLE CASE="c" RELAT="1-2">Easter, Christ</TITLE><DESC>Find documents describing Easter, a celebration of the resurrection of Christ</DESC><NARR><BACK>Easter, an event that commemorates the resurrection of Christ, falls on the Sunday
following the first full moon after the spring equinox.</BACK><RELE>Relevant documents will
introduce how Easer is celebrated. </RELE></NARR><CONC>Easter, Christ, church, Easter bunny, Easter egg, spring</CONC><RDOC>NW002542912, NW002008347, NW000837198</RDOC><USER>2nd year Master's student, female, 4 years search experience</USER></TOPIC><TOPIC><NUM>0043</NUM><TITLE CASE="c" RELAT="1-3">Chiffon cake, directions, cake/TITLE><DESC>Find documents describing how to make chiffon cake</DESC><NARR><RELE>Relevant documents will describe how to make chiffon cake (ingredients andquantities), and introduce different variations of chiffon cake.</RELE></NARR><CONC>Chiffon cake, cake, directions, confectionery, variation</CONC><RDOC>NW013569355, NW011761975, NW009137107</RDOC><USER>2nd year Master's student, female, 4 years search experience</USER></TOPIC><TOPIC><NUM>0044</NUM><TITLE CASE="c" RELAT="2-3">Aromatherapy, aroma oil, aroma candle</TITLE><DESC>I want to learn about the kinds of fragrances used in oils and candles for aromatherapy, and theireffects</DESC><NARR><TERM>Aromatherapy involves warming scented oils and burning scented candles to induce a
relaxing effect on the mind and body.

The effect achieved depends on the fragrance

used.</TERM><RELE>Relevant documents will explain the effects different oils and candles
have.</RELE></NARR>

Proceedings of the Third NTCIR Workshop<CONC>Aromatherapy, oil, candle, incense, relax, fragrance</CONC><RDOC>NW002550340, NW011762026, NW010053209</RDOC><USER>2nd year graduate student, female, 4 years search experience</USER></TOPIC><TOPIC><NUM>0046</NUM><TITLE CASE="b">Natural yeast bread, shop, location</TITLE><DESC>I want to find shops that make bread with natural yeast</DESC><NARR><BACK>Recently, you can find bread that has been made with natural yeast rather than instantyeast but most bread shops don't use natural yeast, perhaps because it is a little more expensive. So I wantto know where I can buy bread made with natural yeast.</BACK><RELE>Relevant documents will
provide the address and telephone number of shops selling bread made with natural yeast. Shops that only
sell bread via the Internet are relevant.</RELE></NARR><CONC>Bread, bread made with natural yeast, shop, location, telephone number</CONC><RDOC>NW014515983, NW014460912, NW011582340</RDOC><USER>2nd year Master's student, female, 4 years search experience</USER></TOPIC><TOPIC><NUM>0047</NUM><TITLE CASE="c" RELAT="1-2">Capsaicin, capsicum, effect</TITLE><DESC>I want to find out about the actions and effects of the capsaicin found in capsicums</DESC><NARR><TERM>Capsaicin is a constituent of capsicums, and assists dieting by stimulating the body's
metabolism through absorption. </TERM><RELE>Relevant documents will explain the actions andeffectsofcapsaicin.

Documents

describing

one

or

other

of

these

aspectsarepartially

relevant.</RELE></NARR><CONC>Capsaicin, capsicum, effect, metabolism, diet</CONC><RDOC>NW008446279, NW001007286, NW013789117</RDOC><USER>2nd year Master's student, female, 4 years search experience</USER></TOPIC><TOPIC><NUM>0048</NUM><TITLE CASE="c" RELAT="1-2">Anthocyanin, blueberry, eyesight</TITLE><DESC>I want to find out in detail about the effects of anthocyanin, a constituent of blueberries,

The Third NTCIR Workshop, Sep.2001 - Oct. 2002

etc.</DESC><NARR><BACK>Blueberries are said to be good for your eyes but this is due to anthocyanin, aconstituent of blueberries. I want to find out about how its effects vary with the quantity you
eat.</BACK><RELE>Relevant documents will explain in detail the quantity of blueberries you need to
eat to have an effect on your eyesight. Documents that do not describe the effects anthocyanin has on thehuman body are irrelevant.</RELE></NARR><CONC>Anthocyanin, blueberry, effect, eyesight, eye</CONC><RDOC>NW002846429, NW013182930, NW012890983</RDOC><USER>2nd year Master's student, female, 4 years search experience</USER></TOPIC><TOPIC><NUM>0049</NUM><TITLE CASE="b">Polyphenol, type, effect</TITLE><DESC>I want to learn about the different kinds of polyphenols and their respective effects</DESC><NARR><BACK>Polyphenol is a constituent of food such as tea and fruit, and I want to learn in detailwhat food it is contained in and what effects it has.</BACK><RELE>Relevant documents will explain
the different kinds of polyphenols and their effects. Documents that only contain information on one
aspect or the other are partially relevant.</RELE></NARR><CONC>Polyphenol, type, effect, tea, chocolate</CONC><RDOC>NW007878611, NW002618366, NW011603751</RDOC><USER>2nd year Master's student, female, 4 years search experience</USER></TOPIC><TOPIC><NUM>0052</NUM><TITLE CASE="c" RELAT="2-3">Lake, water quality, clarity</TITLE><DESC>In the midst of environmental destruction, I want to find out about the quality and clarity ofwater in Japanese lakes</DESC><NARR><RELE>Provision of survey results with detailed, numerical figures for the clarity of water inJapanese lakes is a condition for relevance. Inclusion of comparisons with other lakes or periodic
comparisons is not a condition for relevance. Information such as the name of the surveying institution,materials cited, and fiscal year will be referred to in terms of determining the reliability of scientificallyobjective explanations of the survey results. Likewise, this information will also be used to judge thescientificobjectivityofsurveyed.</RELE><NARR>

estimatedvalues

for

old

lakes

which

can

no

longerbeProceedings of the Third NTCIR Workshop<CONC>Japan, lake, water quality, clarity, survey, environment</CONC><RDOC>NW000630783, NW007643872</RDOC><USER>1st year Master's student, male, 5 years search experience</USER></TOPIC><TOPIC><NUM>0053</NUM><TITLE CASE="b">Automobile, future image, Japan</TITLE><DESC>I want to know what the future holds in store for Japanese automobiles</DESC><NARR><BACK>Japan has been in recession for a long time. Asian countries such as Korea, Taiwan,
and China enjoy cost competitiveness and comprise emerging economies. Whilst many manufacturingindustries struggle under these conditions, I am interested to see what the future holds for Japanese cars,
and would like to hear various opinions on the subject.</BACK><RELE>Relevant documents will
indicate the strong points and weaknesses of Japanese cars in terms of what the future holds. It is thecontent of the document rather than whether or not it is a personal or professional opinion, or whether it
concerns a future image of the industry as a whole or a future image of a single industry, that determines
the relevance of the document.</RELE><NARR><CONC>Japan, automobile, future image, cost, technology, factory</CONC><RDOC>NW008614586, NW014395145</RDOC><USER>1st year Master's student, male, 5 years search experience</USER></TOPIC><TOPIC><NUM>0056</NUM><TITLE CASE="b">Calculus of variations, introduction</TITLE><DESC>Find documents providing an introduction to calculus of variations</DESC><NARR><BACK>I want to studycalculus of variations but it is quite difficult so I am looking for
documents that explain it in layman's terms.</BACK><RELE>Relevant documents will explain thebasics of calculus of variations in layman's terms</RELE></NARR><CONC>calculus of variations, introduction, explain</CONC><RDOC>NW009557906, NW006088235</RDOC><USER>1st year Master's student, male, 3 years search experience</USER></TOPIC><TOPIC><NUM>0057</NUM>

The Third NTCIR Workshop, Sep.2001 - Oct. 2002

<TITLE CASE="b">Turtle, lifespan</TITLE><DESC>Find documents describing the lifespan of a turtle</DESC><NARR><BACK>Cranes are said to live a thousand years and turtles to live ten thousand but I wonder
how many years do turtles really live? I know they can't possibly live ten thousand years but I want toknow just how long they do live.</BACK><RELE>Relevant documents will describe the lifespan of aturtle</RELE></NARR><CONC>Turtle, lifespan, how many years, longevity</CONC><RDOC>NW012227814, NW012944900</RDOC><USER>1st year Master's student, male, 3 years search experience</USER></TOPIC><TOPIC><NUM>0058</NUM><TITLE CASE="b">Shinetsu main line, Usui Pass, method</TITLE><DESC>I want to learn the history of how trains made it over the Usui Pass on the Shinetsu mainline</DESC><NARR><BACK>Usui Pass, which lies on the Shinetsu main line between Yokogawa and Karuizawa, is
known as the steepest and most difficult place in Japan. As a result, the Abt system which uses toothed
wheels, assisted driving using a helper engine in conjunction with the main train, and various othermethods have been adopted to get trains over the Usui Pass. I want to learn the history of which methods
were used at which times to get trains over the pass.</BACK><RELE>Relevant documents will describe
how trains got over the Usui Pass from the time the Shinetsu main line opened until the section between
Yokogawa and Karuizawa was closed. Documents describing either the Abt method or assisted driving,
but not both, are only partially relevant.</RELE></NARR><CONC>Shinetsu main line, Usui pass, Yokokaru (Yokogawa-Karuizawa), steep slope, Abt system,
assisted driving </CONC><RDOC>NW001892672, NW010532427</RDOC><USER>1st year Master's student, male, 4 years search experience</USER></TOPIC><TOPIC><NUM>0059</NUM><TITLE CASE="b">N gauge, HO gauge, meaning</TITLE><DESC>I want to know the meaning of N and HO as in N gauge and HO gauge</DESC><NARR><BACK>N gauge and HO gauge are specifications used to describe types of railway models(rail widths). I want to find out what N and HO mean.</BACK><RELE>Relevant documents willProceedings of the Third NTCIR Workshop

describe the meaning or the origin of the N in N gauge and the HO in HO gauge. Documents that only
provide the beginning of standards are irrelevant.</RELE></NARR><CONC>N gauge, HO gauge, railway model, meaning, source, standard</CONC><RDOC>NW002286492, NW007402924</RDOC><USER>1st year Master's student, male, 4 years search experience</USER></TOPIC><TOPIC><NUM>0060</NUM><TITLE CASE="b">Yggdrasil the world tree, Norse mythology, name</TITLE><DESC>I want to know the name of the Yggdrasil, the world tree that appears in Norse
mythology</DESC><NARR><BACK>A massive tree called "Yggdrasil" that supports the world appears in Norse mythology.
I would like to learn the name of this tree.</BACK><RELE>Relevant documents will clarify the name of
the Yggdrasil the world tree that appears in Norse mythology.</RELE></NARR><CONC>Yggdrasil the world tree, Norse mythology, mythology, legend, name, title</CONC><RDOC>NW006604746, NW008528972</RDOC><USER>1st year Master's student, male, 4 years search experience</USER></TOPIC><TOPIC><NUM>0061</NUM><TITLE CASE="a">Tommy February, Kawase Tomoko, The Brilliant Green</TITLE><DESC>I want to find out about Kawase Tomoko (aka Tommy February), the vocalist in The Brilliant
Green</DESC><NARR>I want to find out about Kawase Tomoko (aka Tommy February), who reached No. 1 on thealbum hit chart and is gaining attention in fashion magazines.<TERM>Tommy February is the name thevocalist from The Brilliant Green used for her solo activities, so Tommy February and Kawase Tomoko
are the same person.</TERM><RELE>Relevant documents will provide information on Tommy
February and Kawase Tomoko. Documents providing information on the band The Brilliant Green
without providing information on individual members are only partially relevant. Documents providing
no more than an introduction to their album are irrelevant.</RELE></NARR><CONC>Tommy February, Kawase Tomoko, The Brilliant Green, solo activity, vocalist</CONC><RDOC>NW007067557, NW002650572</RDOC><USER>1st year Master's student, male, 4 years search experience</USER></TOPIC>The Third NTCIR Workshop, Sep.2001 - Oct. 2002

<TOPIC><NUM>0062</NUM><TITLE CASE="c" RELAT="1-2">Shiba inu, Japanese dog, characteristic</TITLE><DESC>Find documents explaining the attractions and characteristics of Shiba inu</DESC><NARR><BACK>I want to learn about the temperament, species and characteristics of Shiba inu. If
possible, I would like to learn their ideal physique.</BACK><TERM>"Shiba inu" is a kind of Japanese
dog native to Nagano, Gifu and Tottori prefectures. Traditionally, they were used for hunting small
animals but they are also pets and watchdogs.</TERM><RELE>Relevant documents will introduce thetemperament, physical characteristics and species of Shiba inu.</NARR><CONC>Dog, Shiba inu, Japanese dog, characteristic, species, koshiba, pet</CONC><RDOC>NW008724745, NW003988803</RDOC><USER>2nd year Master's student, female, 4 years search experience</USER></TOPIC><TOPIC><NUM>0063</NUM><TITLE CASE="b">Great Barrier Reef, Australia, travel</TITLE><DESC>I want information on traveling to the Great Barrier Reef</DESC><NARR><TERM>The coastal area of north eastern Australia is known as the Great Barrier Reef, and its
warm and beautiful sea makes it ideal for sightseeing tours.</TERM><RELE>Documents describing thehighlights and specialties of the Great Barrier Reef are relevant.</RELE></NARR><CONC>Sea, Great Barrier Reef, Australia, highlight, sightseeing, travel</CONC><RDOC>NW005472644, NW003945104</RDOC><USER>2nd year Master's student, female, 4 years search experience</USER></TOPIC>