NTCIR-13 Kick-off Event

Program• Overview of NTCIR• Overview of NTCIR-13• Task Introduction (7 min x 9 tasks)• Why participate?

• How to participate• Important Dates• Q&A2

Overview of NTCIROverview of NTCIR-13

Organization at NTCIR-13

GeneralCo-chairs

Participants

Task
Organizers

ProgramCommittee

5General Co-chairs

Charles L. A. Clarke

Noriko Kando

Tetsuya Sakai

6

Program Committee (PC)
•••
•
•
••
•
••

Tat-Seng Chua
Nicola FerroKalervo Jarvelin
Gareth Jones
Chin-Yew LinMaarten de Rijke
Mark Sanderson
Ian Soboroff
Yiqun Liu
Makoto P. Kato

7

NTCIR-13 General Schedule

Note that each task can have its own schedule
Aug 2016

Kick-off event in Tokyo

Dec 2016

Task Registration Due

Jan 2017

Dataset Release

Jan-May 2017

Dry RunMar-July 2017

Formal Run

Aug-Sep, 2017

Evaluation Result Release

Aug-Sep, 2017

Early draft Task Overview Release

Sep-Oct, 2017

Participant paper (draft) submission due

Nov 1, 2017

All camera-ready copy for the Proceedings due

Dec 5-8, 2017

NTCIR-13 Conference8

Focuses of NTCIR-13

1. Answering complex questions andqueries
2. Mining knowledge from a large
amount of human generated data
3. Application of 2 to 19By understanding texts
Answering complex questions

QALab
Replying to tweets

By understanding intents

Responding toComplex Questions

Search for questions

OpenLiveQ
Search for web pages

WWW

STC
Answer

Questions
Mining lifelog data
for search

Lifelog?

?

?

Users

Mining user actions

Datafor searchStoredAKG

From text data

From sensor data

Patients from microblogs/blogs

Relevance from brain signals

NAILS

Mining HumanGenerated Data

MedWeb
Emotions and causes from news

ECA

NTCIR-13 Tasks• CORE– Lifelog-2: Personal Lifelog Organisation & Retrieval Task
– MedWeb: Medical Natural Language Processing for WebDocument– OpenLiveQ: Open Live Test for Question Retrieval– QALab-3: QA Lab for Entrance Exam
– STC-2: Short Text Conversation• PILOT– AKG: Actionable Knowledge Graph– ECA: Emotion Cause Analysis– NAILS: Neurally Augmented Image Labelling Strategies– WWW: We Want Web11

How to participate

How to participate

1. Read the task
description andCFP carefully
2. Contact a TO if you
have questions
3. Decide a task toparticipate
4. Register as aparticipant at
NTCIR website

5. Fill in User
Agreement Forms
6. Keep an eye on atask’s ML, website,etc. to follow theactivity
Don’t hesitate tosend a feedback toTO
13

What Participants Must Do

1. Submit your runs

2. Submit a paper to NTCIR-13 Conference
3. Attend the NTCIR-13 conference
and give a presentation

14

Why participate? (Students)• Easy start-up• Publications• Increase your visibility• Diverse tasks

15

Why participate? (Industry)• Establish your brand• Faster development• Fair benchmarking• Early access toresultant resources16

EVIA 2017
A satellite workshop of the NTCIR conference focusing on
information access evaluation••

Theme– Crossing evaluation campaign boundaries:
are tasks/tracks/labs across campaigns
beneficial and feasible?

Topics•– Test collection formation, evaluation metrics,
and evaluation environments– Statistical issues in retrieval evaluation– User studies and the evaluation of humancomputer interaction in information retrieval •(HCIR)– Evaluation methods for multilingual,multimedia, or mobile information access– Novel information access tasks and their•
evaluation– Evaluation and assessment using implicit
user feedback, crowdsourcing, living labs, orinferential methods– Evaluation issues in industrial and enterprise
retrieval systems

PC Chairs– Nicola Ferro, University of Padua, Italy•

CLEF Steering Committee Chair•

TREC– Ian Soboroff, National Institute of Standards
and Technology (NIST), USAPapers– Full papers: 8-10 pages– Short papers: 2-4 pages– ACM SIGIR Format

Tentative Schedule––––Submission deadline: mid September 2017
Notification: mid October 2017
Camera ready: late October 2017
Workshop: early December 2017

Wrap-up• The thirteenth cycle of NTCIR has started• Nine exciting tasks are running• Lots of opportunities for innovative work

What we need is your participation!
18

http://research.nii.ac.jp/ntcir/ntcir-13/

Thank you for your attention!
For further enquiries, contact the NTCIR officentc-secretariat nii.ac.jp

Q & A?