Abstract: When assessing relations between argumentative units (e.g., support or attack), computational systems often exploit disclosing indicators or markers that are not part of elementary argumentative units (EAUs) themselves, but are gained from their context (position in paragraph, preceding tokens, etc.). We show that this dependency is much stronger than previously assumed. In fact, we show that by completely masking the EAU text spans and only feeding information from their context, a competitive system may function even better. We argue that an argument analysis system that relies more on discourse context than the argument's content is unsafe, since it can easily be tricked. To alleviate this issue, we separate argumentative units from their context such that the system is forced to model and rely on an EAU's content. We show that the resulting classification system is more robust, and argue that such models are better suited for predicting argumentative relations across documents.

Introduction: In recent years we have witnessed a great surge in activity in the area of computational argument analysis (e.g. BIBREF0 , BIBREF1 , BIBREF2 , BIBREF3 ), and the emergence of dedicated venues such as the ACL Argument Mining workshop series starting in 2014 BIBREF4 . Argumentative relation classification is a sub-task of argument analysis that aims to determine relations between argumentative units A and B, for example, A supports B; A attacks B. Consider the following argumentative units (1) and (2), given the topic (0) “Marijuana should be legalized”: This example is modeled in Figure FIGREF3 . It is clear that (1) has a negative stance towards the topic and (2) has a positive stance towards the topic. Moreover, we can say that (2) attacks (1). In discourse, such a relation is often made explicit through discourse markers: (1). However, (2); On the one hand (1), on the other (2); (1), although (2); Admittedly, (2); etc. In the absence of such markers we must determine this relation by assessing the semantics of the individual argumentative units, including (often implicit) world knowledge about how they are related to each other. In this work, we show that argumentative relation classifiers – when provided with textual context surrounding an argumentative unit's span – are very prone to neglect the actual textual content of the EAU span. Instead they heavily rely on contextual markers, such as conjunctions or adverbials, as a basis for prediction. We argue that a system's capacity of predicting the correct relation based on the argumentative units' content is important in many circumstances, e.g., when an argumentative debate crosses document boundaries. For example, the prohibition of marijuana debate extends across populations and countries – argumentative units for this debate can be recovered from thousands of documents scattered across the world wide web. As a consequence, argumentative relation classification systems should not be (immensely) dependent on contextual clues – in the discussed cross-document setting these clues may even be misleading for such a system, since source and target arguments can be embedded in different textual contexts (e.g., when (1) and (2) stem from different documents it is easy to imagine a textual context where (2) is not introduced by however but instead by an `inverse' form such as e.g. moreover).

Question: How are the EAU text spans annotated?
Answer: Answer with content missing: (Data and pre-processing section) The data is suited for our experiments because the annotators were explicitly asked to provide annotations on a clausal level.