N TNII
Testbeds and C
Community for IInformation access R
ResearchThe 14th NTCIR (NTCIR-14)
Testbeds and Community for Information access Research

Call for Task Participation
Visit NTCIR-14 website for registration: http://research.nii.ac.jp/ntcir/ntcir-14/

Let's participate in a collaborative activity for enhancing Information Access technologies!
Development of Information Access technologies based on techniques of Information Retrieval, Natural Language Processing,and Database Management becomes increasingly more important for many applications, (e.g., providing eﬀective access toWeb resources and text archives, and analyzing big data obtained from various kinds of sensors). It is indispensable fordeveloping such technologies to experimentally evaluate them by using test collections constructed under collaborations ofmany researchers. Over the past 20 years, NTCIR has been formulating the infrastructure for the evaluation, and contributing
to development of the Information Access technologies. A total of about 75 "evaluation tasks" have been organized, attracting over 950 participant research groups worldwide so far. Furthermore, over 4,000 research groups have signed up to use
the NTCIR test collections in their research. Consequently, NTCIR has been a major forum for researchers to intensively
discuss the evaluation methodology of emerging information access technologies.
NTCIR-14 now calls for task participation of anyone interested in research on information access technologies and theirevaluation, such as retrieval from a large amount of document collections, question answering and natural language processing. We welcome students, young researchers, professors who supervise students, researchers working for a company, andanyone who is interested in informatics.

NTCIR-14 Tasks
Lifelog-3

Advance the state-of-the-art research in lifelogging as an application of information
retrieval. http://ntcir-lifelog.computing.dcu.ie/

OpenLiveQ-2

Question retrieval task in which participants can evaluate their systems in a production
environment of Yahoo Japan Corporationʼs cQA service. http://www.openliveq.net/

QALab-PoliInfo

Question answering task for fact checking using Japanese regional assembly minutes.
https://poliinfo.github.io/

STC-3

Emotional Conversation Generation, Dialogue Quality, and Nugget Detection subtasks
using Chinese and English dialogue data. http://sakailab.com/ntcir14stc3/

WWW-2

Ad hoc web search. http://www.thuir.cn/ntcirwww2

CENTRECan we replicate/reproduce best practices from CLEF and/or TREC?
http://www.centre-eval.org/ntcir14/FinNum

Fine-grained numeral understanding in ﬁnancial social media data

Registration due

Lifelog-3 : Dec 31, 2018; OpenLiveQ-2 : Aug 1, 2018; QALab-PoliInfo : Nov 19, 2018;
STC-3 : Aug 31, 2018; WWW-2 : Aug 31, 2018; CENTRE : Sep 30, 2018of each task

NTCIR-14 Important Dates

Jun 2018:Jun-Jul 2018:Aug-Dec, 2018:

Dataset release*
Dry run*Formal run*

Events with * vary across tasks. For the latest and detailed schedule,please visit each task website.

Feb 1, 2019:
Mar 15, 2019:
May 1, 2019:
Jun 10-13, 2019:

Evaluation results returnParticipant paper due
Camera-ready paper due
NTCIR-14 Conf. & EVIA 2019

Post-conference Proceedings from LNCS

Selected participant papers will be published in the post-conference
proceedings from Lecture Notes on Computer Science (LNCS).

NTCIR-14 Program Committee Co-Chairs: Makoto P. Kato, Yiqun Liu
NTCIR-14 General Co-Chairs:
Charles L. A. Clarke, Noriko Kando

Contact (secretariat) : ntc-secretariat@nii.ac.jp

N T
NII
Testbeds and C
Community for InformationI
access Research

Aims of NTCIR-14
NTCIR-14 mainly focuses on
three topics on IA technology:

Search for questions

OpenLiveQ
Search for web pages

1. Heterogeneous information
accessSearch

WWW
Search for lifelog data

Heterogeneous data

Lifelog

2. Dialogue generation andanalysis

A!

Summarizing dialog data

QALab

3. Meta research on information
access communities

Generating dialogues

Summarize
Generate

STC
Reproduce the best practices

CENTRE

B?

Dialog data
Reproduce

The 13th NTCIR Conference & EVIA 2017
http://research.nii.ac.jp/ntcir/ntcir-13/•9Tasks• 44 Task organizers• 65 Research groups•8Countries/Regions• 207 ParticipantsContact (secretariat) : ntc-secretariat@nii.ac.jp

C…