NTCIR-13 Pilot Task:
Emotion Cause Analysis(ECA)
Organizers:
Ruifeng Xu, Harbin Institute of Technology, China
Yulan He, Aston University, United Kingdom
Qin Lu, The Hong Kong Polytechnic University, Hong Kong, China
Kam-Fai Wong, The Chinese University of Hong Kong, Hong Kong, China

Motivation
Definitions

CONTENTS

ExamplesTask Design
DatasetSchedule

Motivation• There has been a large body of research work on
sentiment/emotion analysis from text
• Several bake-offs/shared tasks were organized to evaluate
the sentiment/emotion analysis techniques Multilingual Opinion Analysis Task (NTCIR 6, 7, 8 2007-2009)http://blueberry.slis.tsukuba.ac.jp/moat/index.php/MOAT Emotion Analysis in Chinese Weibo (NLPCC 2013,2014)
http://tcci.ccf.org.cn/conference/2014/pages/page04_eva.html
 Sentiment Analysis in Twitter (Semeval 2013,2015)
http://alt.qcri.org/semeval2015/task10/

Motivation• Most Existing Task: Emotion Analysis
 Classify the text to pre-defined emotion categories
 Extract Holder/Target of emotion
 To know the emotion expressed in the text

• Challenge Task: Emotion Cause Analysis (ECA) Extract the expression of stimuli or cause of emotion from text
 To know what cause the emotion
 Essential to emotion understanding No existing public-available dataset with emotion cause
annotation

Definitions• Emotion：An affective state of consciousness
experienced, as distinguished from cognitive and volitional
states of consciousness• Emotion Categories: Following Ekaman Six categories
 Happiness, Sadness, Anger, Disgust, Fear, SurpriseDefinitions• Emotion Cause The text expression corresponding to the things or events which
stimulates the generation or transformation of human emotions• Direct Cause The cause expression in the text which directly stimulates thehuman emotions• Elaborated Cause The elaborated text expression of emotion causeExample
ISIS blow up pagan temple in Palmyra that has stood for
2,000 years. The world community is infuriated by thisdestruction.
Emotion category
Anger

Direct cause“this destruction”

Elaborated cause“ISIS blow uppagan temple
in Palmyra thathas stood for
2,000 years”

Example(Known the things happened in the last 10 minutes of hislife, a lot of people left the sad tears)
Emotion category

Direct causeSad（Known the things
happened in the last
10 minutes of hislife）

Task Design• Subtask 1: Emotion Cause Detection at the Clause level To evaluate the techniques for detecting the clause whichcontains emotion cause A clause-level binary text classification problem
 Evaluation Metrics: classification-based

Task Design• Subtask 2: Emotion cause extraction
 To evaluate the techniques for detecting the boundary of theemotion cause An information extraction task
 Evaluation Metrics: overlapping-based

Task Design• Lenient Evaluation
 Subtask 1: the clauses containing either direct cause orelaborated cause are regarded as cause clauses
 Subtask 2: both the boundaries of direct cause andelaborated cause are valid• Strict Evaluation
 Subtask 1: only the clauses containing elaborated cause areregarded as cause clauses
 Subtask 2: only the boundaries of elaborated cause are valid

Dataset• The annotation follows the W3C Emotion Markup Language(EML)• Example:
ISIS blow up pagan temple in Palmyra that has stood for 2,000 years.
The world community is infuriated by this destruction.
Annotation result:<emotion type=“Anger” keyword=“infuriated”><elaborated_cause POS=“n”
clause_dis=“-1”> ISIS blow up pagan temple in Palmyra that has stood for
2,000 years. </elaborated cause> The world community is infuriated by<direct_cause POS=“n” clause_dis=“0”> this destruction. </directcause></emotion>

Dataset• Two languages
 English/Simplified Chinese• Training Dataset
 2,500 paragraphs with emotion and emotion cause annotation• Testing Dataset
 5,00 paragraphs with emotion annotations• Developing Dataset
 10,000 paragraphs with emotions

ScheduleData
2016-08-15
2017-12-01
2017-01-04
2017-07-03
2017-07-07
2017-08-01
2017-12

EventFirst call for participation
Registration due
Sample data release
Formal run threads release
Formal run result due
Delivery of evaluation results
NTCIR-13 Conference & EVIA 2017 in NII, Tokyo, Japan

Welcome to Participate• Pilot Task Website
http://hlt.hitsz.edu.cn/ECA.html• Reference
Lin Gui, Ruifeng Xu, Dongyin Wu, Qin Lu and Yu Zhou,
Event-Driven Emotion Cause Extraction with Corpus
Construction, Proceedings of EMNLP 2016

NTCIR-13THANKS