言語処理学会 第24回年次大会 発表論文集 (2018年3月)

Towards Exploiting Argumentative Context for
Argumentative Relation Identification
Tatsuki Kuribayashi††Paul Reisert‡

Naoya Inoue†

Kentaro Inui† ‡‡Tohoku University
RIKEN Center for Advanced Intelligence Project (AIP){kuribayashi,naoya-i,inui}@ecei.tohoku.ac.jp
paul.reisert@riken.jp

1 Introduction
Argument mining is the task of identifying argument structures in argumentative texts. This task
is useful for many applications such as document
summarization, opinion mining, and automated essay scoring [10, 13]. In the literature, several subtasks for argument mining have been extensively studied, such as argument component type classification,
stance classification, and argumentative relation identification [7, 11, 4, 8, 9, 5]. This paper addresses thetask of argumentative relation identification due to its
recent popularity in argument mining.
Consider the argumentative text1 in Figure 1, whereargument components (ACs), basic units of arguments,
are already identified. Argument component type classification aims at classifying ACs into a premise or
claim (e.g. classifying AC1 into a claim and AC2 intoa premise). Stance classification aims at classifying the
stance of ACs towards a claim as either proponent or
opponent (e.g. classifying AC1 into a proponent stanceand AC2 into a opponent stance). Argumentative relation identification aims to identify an argumentative
link between two ACs, and if it exists, classify it into
two classes: attack or support (e.g. identifying the attack relation from AC2 to AC1 ).
Conventional approaches have focused on creating
features using the local input ACs rather than using
macro-level information such as the overall structureof an argument [7, 11, 1]. However, argumentative relations are closely related to each other and they form
argument diagrams [6]. Thus, we speculate that the information of surrounding argumentative context (e.g.
other argumentative relations) can be useful for predicting a relation.
For example, in Figure 1, AC2 attacks AC1 and AC3
attacks AC2 . If we were to predict the attack relationfrom AC3 to AC2 , knowing whether AC2 is attacking another AC would be useful information, because awriter frequently uses such macro structure as a tactic
for strengthening their argument. For example, in Figure 1, the writer gives a possible counter-argument totheir claim (AC2 attacks AC1 ) and then attacks it immediately (AC3 attacks AC2 ), which makes it diﬃcult
for others to attack.
1 Slightly modified version of the text (micro b006) taken from
Peldszus and Stede (2015) [7]

Topic : Should Germany introduce the death penalty?
AC1 : The death penalty is a legal means that as suchis not practicable in Germany.
AC2 : Even if many people think that a murderer has
already decided on the life or death of another person,
AC3 : this is precisely the crime that we should not
repay with the same.
attackattackAC1[Claim]AC2[Premise]

AC3[Premise]stance:Proponent

stance:Opponent

stance:Proponent

Figure 1: An example of argument structure with threeACs
A corpus-based analysis in previous research [3] revealed that some macro-level structures are frequently
observed across diﬀerent corpora. In this paper, we extend the previous work by investigating the eﬀectiveness of argumentative context (i.e. macro-level information such as other surrounding argumentative relations) as a clue of argumentative relation identification.
Our experiments demonstrate that such macro-level information is helpful for predicting argumentative relations.

2 Related work
Previous work has focused on two types of approaches for solving the task of argumentative relation identification. The first approach is to formalize
argumentative relation identification as a structured
prediction problem (i.e. predicting a graph consisting of argumentative relations from an argumentativetext) [7, 9, 11, 8]. To predict a graph, Peldszus et al. [7]use the Maximum Spanning Tree (MST) algorithm andStab et al. [11] use Integer Linear Programming (ILP).
However, these works do not exploit information fromother predicted argumentative relations when predicting a relation. Niculae et al. [5] use factor graphs forstructured prediction. They report that higher-orderfeatures (e.g. combination features of argumentative
links) increase the precision of AC type classificationand link identification (i.e. whether argumentative relation exists or not between two ACs). Potash et al. [9]use Pointer Networks [12], which considers the previous
prediction in the decoding steps, for AC type classifica-― 284 ―

Copyright(C) 2018 The Association for Natural Language Processing.
All Rights Reserved.

Table 1: Macro-level structures found in the MT and PE corpus (excerpt from [3]). The highest percentage
among three relation types is in bold and the lowest one is underlined. In PE, 9,000 neither relation pairs were
sampled (about 10% of all neither pairs in the corpus).
MT
relation
attacksupportneither

attackedsourcesupported

62/174(36%)
23/290(8%)
582/2000(29%)

16/174(9%)
51/290(18%)
686/2000(34%)

attackedsourcesupported

attackedby another27/174(16%)
139/290(48%)
354/2000(18%)

targetsupported
attackingby another
90/17464/174(52%)(37%)
202/290
21/290(70%)(7%)
373/2000
638/2000
(19%)
(32%)

attackedby another
68/219(31%)
175/3613(5%)
195/9000(2%)

targetsupported
attackingby another106/219
38/219(48%)(17%)
2978/3613 43/3613
(82%)
(1%)
1993/9000
243/9000
(22%)
(3%)

supporting
23/174(13%)
57/290(20%)
1197/2000(60%)

PE
relation
attacksupportneither

36/219(16%)
18/3613(0%)
224/9000(2%)

32/219(15%)
499/3613(14%)
2368/9000(26%)

attackattackedsupported

Sourcenot supported

supportattack / support / neitherXX

not attacked

TargetXX

XX not support
not attack

Figure 2: Macro-level structures examined in Kuribayashi et al. [3], shown as dotted lines. Red, oval arrow
depicts an attack relation and blue, open arrow depicts
a support relation.
tion and link identification. However, it still remains an
open issue as to whether such higher-order features areuseful for argumentative relation identification (classifying a relation into attack, support and neither ).
The second approach for solving the task of argumentative relation identification is to formalize the task asa pairwise multi-class classification problem [4, 1]. Cocarascu et al. [1] use a Siamese Neural Network-basedclassifier with a Long Short-Term Memory [2], where
the input feature vector is constructed from the information from input ACs only. Nguyen et al. [4] exploit
discourse structure features for argumentative relation
identification. Their work is closest to our work in thesense of using macro-level information. However, we
focus on macro-level information constructed using argumentative relations in a document.
Kuribayashi et al. [3] show the potential eﬀectiveness of argumentative context in argumentative relation identification. However, their experiments are
preliminary because they used gold-information for extracting macro-level features. In this paper, we show
the eﬀectiveness of macro-level features without using
gold-information and examine other types of argumentative context (i.e. macro-level features such as argumentative flow).

supporting
19/219(9%)
624/3613(17%)
5616/9000(62%)

texts (one paragraph each, 5.1 ACs on average). Eachtext consists of an argumentative topic (e.g. Should
Germany introduce the death penalty? ) and a monologue text discussing this topic (e.g. Figure 1). Eachtext is composed of segmented ACs, which have AC
type (claim or premise), stance (proponent or opponent), and argumentative relations between ACs (support, attack, rebut, undercut, normal, example, or add ).
Following [7], we reduced rebut and undercut to attack,and normal and example to support. For add relations
from ACi to ACj , we first create a link from ACi to
ACk , the grandparent of ACi , with the same relationfrom ACj to ACk . In our experiment, we use 174 attack relations, 290 support relations, and 2,000 neither
relations (neither attack nor support relation) obtainedby this conversion process.

4 Macro-level argumentative structure
analysis

Kuribayashi et al. [3] analyzed typical macro-levelstructures on several corpora. In addition to the MT
corpus, they analyzed the persuasive essay (henceforth,PE ) corpus [11], which consists of 402 essays (5 paragraphs, 15 ACs on average) posted in online forums.
They extracted all support, attack, and neither relations from each corpora. For notational convenience,
we call the starting point of each relation a source AC(i.e. an AC which supports/attacks something) andthe end point of each relation a target AC (i.e. an
AC which is supported/attacked by the source AC). In
addition, Kuribayashi et al. [3] defined a macro-levelstructure as the combination of a relation type and thestate of the source AC and target AC, as illustrated in
Figure 2. For the state of an AC, they considered thefollowing properties:• whether the source AC is attacked/supported byanother AC• whether the target AC is attacked/supported by an3 Data
other ACThis study uses the arg-microtexts (henceforth, MT ) • whether the target AC attacks/supports anotherAC
corpus [7], which contains 112 argumentative short― 285 ―

Copyright(C) 2018 The Association for Natural Language Processing.
All Rights Reserved.

{

sfi,j = p(cci = premise), p(fui = attack),
The results on MT and PE corpora are shown in Table 1. The skewed distribution indicates typical macrop(fui = support), p(roi = opp) × p(roj = pro),level argumentative structures frequently used in eachp(roi = pro) × p(roj = opp), p(roi = opp) × p(roj = opp),}
corpus. Kuribayashi et al. [3] also discovered that bothp(roi = pro) × p(roj = pro), p(ati,j = yes)
corpora have similar tendencies on macro-level structures and provided a detailed discussion about the reThis model uses sfi,j as features for the main classisults.
fier.

5 Identifying argumentative relations 5.2 SSM + macro model
Given our observation in Section 4, information
with macro-level information
To evaluate the eﬀectiveness of macro-level featuresin argumentative relation identification, we add macrolevel features to a baseline model and compare the performance of the models in argumentative relation identification. To consider macro-level information, we use
output (meta-features) of the subtask-level classifiers(introduced in Sec. 5.1) for other ACs. The main classifier considers the meta-features (e.g. an AC seems to
be attacking something) of ACs over macro-range topredict local argumentative relations.
One merit of this approach for considering other argumentative relations is that the model does not become complex because it does not need dynamic features such as history of prediction of other argumentative relations.
5.1 Baseline models
Simple model (SM): As a baseline model, we use a
simple logistic regression classifier. This classifier takes
a pair of ACs as input and outputs the relation type
between the pair. We represent the pair of ACs (input) as a binary-valued feature vector, following Peldszus and Stede [7]. We extract surface features such aslemma, part-of-speech tags, and segment length fromthe source, target, and their adjacent ACs. See theoriginal paper [7] for further details.

about incoming and outgoing relations on a source
and target AC will be helpful for predicting a relation. To consider incoming relations, we pre-train two
additional classifiers. One classifier predicts whether asegment is attacked or not (attacked classifier), andthe other predicts whether a segment is supported or
not (supported classifier). Then, we use the outputof the two classifiers for a source and target segment
as a part of the macro-features. To consider outgoing relations of a target AC, we use the output of afunction classifier on the target segment as additional
macro-features. To summarize, we use the following as
macro-level features:{mfi,j =p(fuj = attack), p(fuj = support),p(attackedi = yes), p(supportedi = yes), p(attackedj = yes),}p(supportedi = yes)This model uses sfi,j ∪ mfi,j as features for the mainclassifier.
5.3 SSM + macro + flow model
To consider additional argumentative context, we define the following features (probabilities) as argumentative flow:• whether the adjacent ACs of source and target AC
are attacked/supported by another AC• whether the adjacent ACs of source and target AC
attacks/supports another AC• the argument component type (claim or premise) of
the following; target AC, source-adjacent ACs, andtarget-adjacent ACs• the stances of the following: source AC, target AC,
source-adjacent ACs, and target-adjacent ACs

Subtask-stacked model (SSM): Following Peldszus
and Stede [7], we use a subtask-stacked model (slightly
modified version of their work) as a baseline model.
There are four subtasks which are closely related to argumentative relation identification: central claim identification (cc), role identification (ro), function classification (fu), and attachment classification (at). At
first, they pre-train the following subtask-level classi{
fiers:
f fi,j = p(cck = claim), p(ccj = claim),• cc classifier: predicts whether an AC is claim or ap(fuk = attack), p(fuk = support), p(attackedk = yes),premise.
p(supportedk = yes), p(rok = opp), p(rok = pro), p(roi = opp),}• fu classifier: predicts whether an AC attacks somep(roi = pro), p(roj = opp), p(roj = pro) | k = i, j ± 1
thing, supports something, or no function (if an AC
is claim, it has no outgoing edge).
This model use sfi,j ∪ mfi,j ∪ f fi,j as features for the• ro classifier: predicts whether an AC’s stance is pro- main classifier.
ponent or opponent towards a central claim.
• at classifier: predicts whether two ACs have an ar- 6 Evaluation
gumentative relation or not.
6.1 Setting
We compare the performance of each model in the
Using the predicted probabilities p(·) of each subtask
as meta-features, we train a logistic regression classi- following two tasks:
fier (main classifier), which predicts an argumentative (i) classifying whether a given segment pair has a support relation or not (support detection).
relation (attack, support, or neither ). We define themeta-feature set for wi,j (relation from ACi to ACj ) (ii) classifying whether a given segment pair has a attack
relation or not (attack detection).
as the following:― 286 ―

Copyright(C) 2018 The Association for Natural Language Processing.
All Rights Reserved.

Table 2: Performance of all models in each task.
model
Baseline
Baseline SS
SS + macro
SS + macro + flowmacro
.601 ± .130.604 ± .030
.638 ± .029
.627 ± .080

supportACi

support vs non-support
support = yes
f1
precision.355 ± .064
.312 ± .062
.369 ± .063
.259 ± .045
.419 ± .042 .301 ± .037
.400 ± .053
.301 ± .048

recall.457 ± .164
.643 ± .113
.698 ± .075
.615 ± .108

macro
.620 ± .031
.516 ± .118.520 ± .119
.536 ± .099

attack vs non-attack
attack = yes
f1
precision.297 ± .059 .290 ± .059
.247 ± .049
.154 ± .034
.253 ± .047
.157 ± .033
.266 ± .040
.166 ± .030

recall.315 ± .083
.670 ± .129
.682 ± .123
.697 ± .108

2. An AC following an attacking AC has for stance.
(i.e. even when a writer attacks with against stance,
they switch their stance into for stance immediately.) (see Figure 4)

supportACi+1

7 Conclusions and future work

In this paper, we examined the usefulness of typiFigure 3: An AC (e.g. ACi+1 ) which follows a supcal
macro-level argumentative structures observed in
porting AC (e.g. ACi ) tends to support something.
argumentative texts and created a model for capturingattackmacro-level information such as argumentative context.
Our results showed that with the addition of macroACi
ACi+1
level features such as argumentative context, we canreasonably predict argumentative relations. In our fustance:
Proponent
ture work, we will create a more sophisticated model
capable of jointly learning the subtask-level classifiers
Figure 4: An AC (e.g. ACi+1 ) following an attacking
and main classifier. Furthermore, we will exploit moreAC (e.g. ACi ) with a proponent stance.
corpora, such as the PE corpus, for evaluating our models.
2
We extract all AC pairs in MT corpus and predict therelation type. All models are evaluated on 10 iterations Acknowledgementof 5 × 3-fold nested cross validation. The reported reThis work was supported by JST CREST Grant
sults are average and standard deviation over the 10 Number JPMJCR1513 and JSPS KAKENHI Grant
folds. We tuned all hyperparameters using the inner Number 16H06614 and 15H01702.
3-fold CV from the training data. We use macro F1and F1 for each class as an evaluation metric. The Referencessubtask-level classifiers are also trained in the above [1] Oana Cocarascu and Francesca Toni. Identifying attack andsupport argumentative relations using deep learning. ACL,
regime.
pages 1385–1390, 2017.
6.2 Results
The results are shown in Table 2. For task (i), we
found that our SSM+macro model outperforms theother models. This results indicate that macro-level
information is useful for predicting support relations.
For task (ii), all the models predict an attack relation
with low f1 score. We attribute this to the fact that
there are few attack relations in the MT corpus. On
the other hand, we found that our model predicts an
attack relation with high recall.
We speculate that the eﬀectiveness of macro-levelmeta-features subsequently depend on the performance of the subtask-classifiers. Therefore, we speculate that our performance is hindered by mistakes
in our sub-task classifiers. We examined the feature weights learned by our SSM+macro model andSSM+macro+flow model, we then found that they indicate a similar tendency to that of Kuribayashi etal. [3]’s analysis. In addition, we found flow-level tendencies such as the following:
1. An AC following a supporting AC tends to support.
(see Figure 3)[2] Sepp Hochreiter and Jürgen Schmidhuber. Long short-term
memory. Neural Computation, 9(8):1735–1780, 1997.
[3] Tatsuki Kuribayashi, Paul Reisert, Naoya Inoue, and Kentaro
Inui. Examining macro-level argumentative structure features
for argumentative relation identification. pages 1–6, 2017.
[4] Huy V Nguyen and Diane J Litman. Context-aware Argumentative Relation Mining. ACL, pages 1127–1137, 2016.
[5] Vlad Niculae, Joonsuk Park, and Claire Cardie. Argument Mining with Structured SVMs and RNNs. ACL, pages 985–995,
2017.
[6] Andreas Peldszus and Manfred Stede. From Argument Diagrams to Argumentation Mining in Texts: a survey. International Journal of Cognitive Informatics and Natural Intelligence, 7(1):1–31, 2013.
[7] Andreas Peldszus and Manfred Stede. Joint prediction in MSTstyle discourse parsing for argumentation mining. EMNLP,
pages 938–948, 2015.
[8] Isaac Persing and Vincent Ng. End-to-End Argumentation Mining in Student Essays. NAACL, pages 1384–1394, 2016.
[9] Peter Potash, Alexey Romanov, and Anna Rumshisky. Here’s
My Point: Joint Pointer Architecture for Argument Mining.
EMNLP, pages 1375–1384, 2017.
[10] Yi Song, Michael Heilman, Beata Beigman Klebanov, and Paul
Deane. Applying Argumentation Schemes for Essay Scoring.
Proceedings of the First Workshop on Argumentation Mining,(2011):69–78, 2014.
[11] Christian Stab and Iryna Gurevych. Parsing argumentation
structures in persuasive essays. Computational Linguistics,
43(3):619–659, 2017.
[12] Oriol Vinyals, Meire Fortunato, and Navdeep Jaitly. Pointernetworks. In Advances in Neural Information Processing Systems, pages 2692–2700. 2015.
[13] Henning Wachsmuth, Khalid Al-Khatib, and Benno Stein. Using argument mining to assess the argumentation quality of essays. COLING, pages 1680–1691, 2016.

2 In future work, we will also perform the experiments using
the PE corpus.

― 287 ―

Copyright(C) 2018 The Association for Natural Language Processing.
All Rights Reserved.