NTCIR-16 Kick-off Event
March 29, 2021
15:00 ‒ 16:30
http://research.nii.ac.jp/ntcir/ntcir-16/

Program• Overview of NTCIR• Overview of NTCIR-16• Task Introduction (10 tasks)• Why participate?
• How to participate• Important Dates• Q&A2

Overview of NTCIROverview of NTCIR-16

5Organization at NTCIR-16

GeneralCo-chairs

Participants

TaskOrganizers

ProgramCommittee

General Co-chairs

Charles L. A. Clarke
University of Waterloo, Canada

Noriko Kando
National Institute of Informatics, Japan

Makoto P. Kato
University of Tsukuba, Japan

Yiqun Liu
Tsinghua University, China6

NTCIR-16 Program Committee (PC)• Takehiro Yamamoto (Co-chair, University of Hyogo, Japan)• Zhicheng Dou (Co-chair, Renmin University of China, China)• Ben Carterette, University of Delaware• Hsin-Hsi Chen, National Taiwan University• Nicola Ferro, University of Padova• Gareth Jones, Dublin City University• Noriko Kando, NII• Makoto P. Kato, University of Tsukuba• Yiqun Liu, Tsinghua University• Jian-Yun Nie, University of Montreal• Douglas Oard, University of Maryland• Tetsuya Sakai, Waseda University• Mark Sanderson, RMIT University• Ian Soboroff, NIST7

8NTCIR-16 General ScheduleNote that each task can have its own schedule
DateEventMar 29, 2021

NTCIR-16 Kickoff

Apr-Jul, 2021

Task Registration DueJul, 2021

Dataset Release

Jul-Sep, 2021

Dry RunSep-Dec, 2021

Formal Run

may vary
across tasks

Feb 1, 2022

Evaluation Result Release to the participants

Feb 1, 2022

Task overview paper release (draft)

Mar 1, 2022

Submission due of participant papers

May 1, 2022

Camera-ready participant paper due

Jun, 2022

NTCIR-16 ConferenceHighlight of NTCIR-16

9

We have 10 exciting tasks at this round‒ Probably the largest NTCIRin terms of the number of running tasks?

1. Modern IR tasks from data to human
2. Deep NLP tasks in specialized domains

NTCIR-16 Tasks
CORE TASKS
1. WWW-4: We Want Web 4 with CENTRE
2. DialEval-2: Dialogue Evaluation 2
3. FinNum-3: Investorʼs and Managerʼs Fine-grained Claim
Detection
4. Lifelog-4: Lifelog Access and Retrieval
5. QA Lab-PoliInfo-3: Question Answering Lab for Political
Information
6. Data Search 2: Data Search 2
PILOT TASKS
7. RCIR: Reading Comprehension for Information Retrieval
8. Real-MedNLP: Real document-based Medical Natural
Language Processing
9. SS: Session Search
10. ULTRE: Unbiased Learning to Ranking Evaluation Task

10

Modern IR tasks from data to humanDocument

Data
Data Search-2

Real-MedNLP

MedicalHuman

WWW-4

Lifelog-4

SSRCIRULTRE

DialEval-2

FinNum-3

QA LabPoliInfo-3

Finance

Politics

Deep NLP tasks in Specialized DomainsTask Introduction
from Task Organizers

NTCIR-16 Tasks
CORE TASKS
1. WWW-4: We Want Web 4 with CENTRE
2. DialEval-2: Dialogue Evaluation 2
3. FinNum-3: Investorʼs and Managerʼs Fine-grained ClaimDetection
4. Lifelog-4: Lifelog Access and Retrieval
5. QA Lab-PoliInfo-3: Question Answering Lab for PoliticalInformation
6. Data Search 2: Data Search 2
PILOT TASKS
7. RCIR: Reading Comprehension for Information Retrieval
8. Real-MedNLP: Real document-based Medical Natural
Language Processing
9. SS: Session Search
10. ULTRE: Unbiased Learning to Ranking Evaluation Task

13

How to participate

15

How to participate
1. Read the task
description and CFP
carefully
2. Contact a TO if youhave questions

5. Fill in User Agreement
Forms
6. Keep an eye on a taskʼsML, website, etc. tofollow the activity

3. Decide a task toparticipate
4. Register as aparticipant at NTCIRwebsite

Donʼt hesitate to senda feedback to TO

What Participants Must Do

1. Submit your runs
2. Submit a paper to NTCIR-16 Conference‒ Demonstrate your systems and methods

3. Attend the NTCIR-16 conference
and give a presentation‒ Every group will give a poster presentation‒ Some will give an oral presentation, too16

Why participate? (Students)• Easy start-up‒ Much of experimental
setup is provided‒ Performance measuresare (often) defined• Increase yourvisibility‒ Demonstration of youridea and achievement
for academics andindustries• Publications‒ Comparison with otherparticipants canproduce stronger
arguments‒ Inspired by theinternational community
for future work

• Diverse tasks‒ A variety of Information
access tasks to tackle

17Why participate? (Industry)• Establish your brand‒ To your end-users andcompetitors‒ Recruit smart people• Fair benchmarking‒ Comparison with yourown products can beless reproducible‒ Critical selfassessments• Faster development‒ Brushing up yourproduct or eliminatingbugs in a limited periodof time

• Early access toresultant resources‒ Secondary resources
developed by the task
are yours, too18

Wrap-up• The 16th cycle of NTCIR has started‒ Will conclude in June 2022

• 10 exciting tasks are running‒ Organized by leading researchers worldwide• Lots of opportunities for innovative work
‒ Exchange great ideas with the community

What we need is your participation!
19http://research.nii.ac.jp/ntcir/ntcir-16/

Thank you for your attention!
For further enquiries, contact the NTCIR officentc-secretariat nii.ac.jp

Q & A?