Introduction to NTCIR-11 RITE-VAL Task
(Recognizing Inference in Text and Validation)

SuguruMatsuyoshi1

YotaroWatanabe2

YusukeMiyao3

Tomohide
Shibata41University2Tohoku

3National Institute4Kyotoof Yamanashi

Universityof Informatics

University

TerukoMitamura5

Chuan-Jie
Lin6

Cheng-Wei
Shih7

5Carnegie Mellon

6National Taiwan

7Academia

University

Ocean University

SinicaNTCIR-11 Kick-Off Event September 2nd, 2013

Overview of RITE-VAL• RITE is a benchmark task for automatically detecting
the following semantic relations between twosentences:– entailment, paraphrase and contradiction.

• Given a text t1, can a computer infer that a hypothesis
t2 is most likely true (i.e., t1 entails t2) ?
– t1: Yasunari Kawabata won the Nobel Prize in Literaturefor his novel “Snow Country.”– t2: Yasunari Kawabata is the writer of “Snow Country.”• Target languages:– Japanese, Simplified Chinese, Traditional Chinese, andEnglish.

RITE-VAL Subtasks
Other
NTCIR tasks
QA Lab

Applications
QAIR

Fact
Validation
Multiplesentencelevel
inference
Pyramid ofentailment
recognition
technology

Sentence-level
inference

sentence
sentenceentailment ?

BCparaphrase?
entailment ?
contradiction?

MCnegation…

modification

coordination

quantificationcase alternation

Unit
Test

entailment ?
contradiction?

documents

phrase rel.

Linguistic
phenomenalevel inferenceWorldknowledge

Search≡≡≡ ≡≡lexical rel.

SystemValidation

sentence

Application oriented

sent.

sent. sent. sent. sent. sent.

sent.

sent.

sent.

sent. sent. sent. sent. sent.

sent.

sent.

RITE1 RITE2 RITE-VAL

Foundation oriented

Main two tasks of RITE-VAL
Fact
Validation

t2: The Kamakura Shogunate began

in Japan in the 12th century.

sentence

Search≡≡≡ ≡≡documents

SystemValidation

Search for evidence orcounter-evidence for t2.
Docs contradict t2.

t1: In the Meiji Constitution, legal clear distinction

Category:
modification

sent. sent. sent. sent. sent.

sent.

sent. sent. sent. sent. sent.

t2: In the Meiji Constitution, distinction between the…

sent.

between the Imperial Family and Japan had beenallowed.
Imperial Family and Japan had been allowed.

negation

modification

coordination

quantificationcase alternationphrase rel.

Unit
Test

lexical rel.

Worldknowledge

entailment ?
contradiction?

Docs entail t2.

t1: In the Meiji Constitution, distinction between theImperial Family and Japan had been allowed.
sent.

sent.

Category:sent. sent.
meronymy

t2: In the Meiji Constitution, distinction between theEmperor and Japan had been allowed.

Why you should participate• Products from RITE-VAL can benefit various research fields
in NLP (not limited to recognizing textual entailment!).
– Core technologies: Semantic processing, Lexical knowledge
acquisition, Machine learning, etc.
– Applications: Information retrieval, Question answering,Document summarization, etc.

• We welcome a wide variety of participants from all over theworld.
– From undergraduate students to industry researchers.
– Resource pool, which we have built since NTCIR-10 RITE-2, will
help you make a prototype system quickly.
– Adding a new resource or tool to the pool is welcomed.

• Website: https://sites.google.com/site/ntcir11riteval/