Discourse-based Argument Segmentation and Annotation (2021)

Abstract: The paper presents a discourse-based approach to the analysis of argumentative texts departing from the assumption that the coherence of a text should capture argumentation structure as well and, therefore, existing discourse analysis tools can be successfully applied for argument segmentation and annotation tasks. We tested the widely used Penn Discourse Tree Bank full parser (Lin et al., 2010) and the state-of-the-art neural network NeuralEDUSeg (Wang et al., 2018) and XLNet (Yang et al., 2019) models on the two-stage discourse segmentation and discourse relation recognition. The two-stage approach outperformed the PDTB parser by broad margin, i.e. the best achieved F1 scores of 21.2 % for PDTB parser vs 66.37% for NeuralEDUSeg and XLNet models. Neural network models were fine-tuned and evaluated on the argumentative corpus showing a promising accuracy of 60.22%. The complete argument structures were reconstructed for further argumentation mining tasks. The reference Dagstuhl argumentative corpus containing 2,222 elementary discourse unit pairs annotated with the top-level and fine-grained PDTB relations will be released to the research community.

## introduction enormous and ever growing digital content provides information where opinions, sentiment and arguments can be identified and analysed. for example, news and social media content is searched to filter or weight the validity of statements @xcite , to identify the presence of fake news and false claims @xcite , to analyse opinions in public discussions @xcite , to detect opinion manipulation @xcite , to predict consumers sentiment @xcite , to study citizen engagement @xcite , and to recognize stance in political online debates @xcite . arguments from legal @xcite , financial @xcite or medical @xcite documents are extracted to support professional decision-making. natural argumentation is the focus of numerous educational scenarios assessing student's essays quality @xcite and training argumentation and debate skills @xcite . automatic extraction and analysis of arguments from heterogeneous data is one of the important tasks of argumentation mining which aims to provide structured data for computational models of argument and reasoning engines @xcite . while for some applications, an argument can be considered as an atomic entity without internal structure, for others defining its structure becomes crucial. for example, to recognize the speaker 'stance' foot_0 in online debates, the whole post can be acknowledged as an argument in 'favour' or 'against' a certain motion. an argument is, therefore, analysed given the other supporting or attacking arguments @xcite . other argumentation mining tasks require structured argumentation models, e.g. tasks that aim at understanding and emulation of human inference, investigating patterns of reasoning, and tasks that focus on extraction and validity assessment of arguments. identification and classification of argument components are rather challenging tasks @xcite . the argument definition, the description of elementary units and building blocks of an argument, relations between and inside these units, the argument structures and argumentation schemes are still under debate. a simple argument structure is often considered as consisting of a claim that is supported by evidence @xcite . a claim is an assertion that the argument aims to prove, i.e. a claim is a conclusion whose merit must be established. evidence presents (a set of) proposition(-s) which provide grounds for drawing the conclusion. automatic recognition of relevant semantic units involves two tasks: (1) segmentation of a text into meaningful units; and (2) annotation of these units capturing (part of) their meaning. many argumentation mining studies assume that the boundaries of the argument components have been previously detected by other means, thus they focus on the classification task @xcite . other consider segmentation as a sub-task and perform both segmentation and classification @xcite . we define argument structure recognition to involve: (1) segmentation of a text into elementary argumentative units assuming that they correspond to elementary discourse units; (2) discourse relation detection between them; (3) classification of the identified relations; (4) classification of the identified argumentative units based on the classified discourse relations; and (5) argument completionreconstruction of implicit units to achieve a complete argument structure, see also @xcite . in this study we evaluate state-of-theart discourse parsers and machine learning models on automatic segmentation and discourse relation classification tasks, and then apply them to extract arguments from argumentative texts. the rest of the paper is structured as follows. in section 2, we discuss related work concerning argument structure recognition. section 3 presents established discourse theories as theoretical and empirical framework for argument analysis and argumentation modelling. the connection to the existing iso 24617-8 standard for discourse relation annotation is made. section 4 discusses the performed experiments elaborating on the datasets, tools and outcomes. section 5 summarizes the results and outlines the future research. ## discourse analysis discourse theory aims at explaining the coherence of a text. its central notion is coherence, also called rhetorical or discourse relation -a semantic or pragmatic relation between two adjacent text spans. even though text coherence and argumentation structure are not identical, discourse structure can reveal new unexplored properties of argumentation. bridging from discourse to argumentation, @xcite chooses the rst framework where all parts of a text are involved into a discourse structure and organized as a tree, with elementary discourse units (edus) as leaves. an edu is a minimal building block of a discourse tree which typically corresponds to a clause @xcite ). 2 rst specifies how edus and larger units are connected, where some text spans are more important than the others, i.e. nucleus or multiple nuclea are the central part of a relation in the text supported by a satellite. the corresponding rst tagset contains 78 discourse relations which can be grouped into 16 classes sharing one type of rhetorical meaning. another influential discourse analysis frame-work is defined within penn discourse tree bank (pdtb, prasad et al. (2005)). pdtb does not make strong assumptions about the overall structure of a text and does not suggest what kinds of high-level structures may be created from the annotated lowlevel relations and arguments. the pdtb analysis is focused on the discourse relation between two text segments called arg1 and arg2 which can be treated as edus. pdtb accounts for the lexical items that can signal discourse relations -discourse connectives. in the case of explicit connectives, arg2 is the argument to which the connective is syntactically bound, and arg1 is the other argument. in the case of relations between adjacent sentences, arg1 and arg2 reflect the linear order of the arguments, with arg1 before arg2. pdtb does not constrain an edu to be a single clause or single sentence, however, the framework follows a minimality principle requiring an argument to contain the minimal amount of information needed to interpret the relation successfully. the pdtb annotation scheme forms the basis of the iso dr-core (iso 24617-8) discourse relations annotation standard @xcite . even though rst and pdtb annotation frameworks make different assumptions about the discourse structure and define different sets of relations, @xcite suggest an automatic alignment of their relations and evaluates the mapping discrepancies. figure 2 discourse analysis within both annotation frameworks includes (1) segmentation of the text into edus; and (2) the recognition of discourse relations between these units. discourse parsers typically perform both tasks. for example, @xcite designed a full parser to perform the pdtb annotations. the system first identifies discourse connectives, label the corresponding arg1 and arg2 spans and assign an explicit relation. if no connective was identified, the system classifies the statement pair as having one of the other relation types, i.e. implicit, entrel, altlex, norel. wang and lan (2015) extended the parser with extractors for arg1, arg2 and non-entrel relations. @xcite improved recognition of the implicit relations. recent works explore deep learning techniques which use architectures for multi-task learning @xcite @xcite or adversarial neural networks @xcite . while many studies focus exclusively on the discourse relation recognition assuming that the text is already pre-segmented, others also consider discourse segmentation task. early generation segmenters were rule-based systems @xcite , whereas more recent approaches view this task as sequence labeling problem and use deep learning @xcite @xcite . multilingual discourse segmentation is addressed in @xcite @xcite . the presented study concerns both segmentation and classification tasks assessing the performance of the state-of-the-art tools on the argumentative corpus. design and results are reported in the next section. ## experimental design we conducted the following experiments: (1) evaluating the quality of the existing full discourse parsers on edus segmentation and relation classification tasks; (2) two-stage discourse segmentation and relation annotation; ## conclusions and future work the presented study reviewed discourse-based approaches to argumentative discourse analysis. we evaluated three widely used tools on argument segmentation and annotation tasks, namely, a rulebased pdtb full parser @xcite , a bilstm-crf model for discourse units segmentation @xcite and an xlnet based discourse relations classifier @xcite . our experiments demonstrated that the pdtb parser achieved an f1 score of 22.61% on the span identification and 21.20% on the joint span identification and relation recognition tasks. this performance has been considered unsatisfactory for further use. deep learning models, in contrast, showed significantly better performance: f1 scores ranging from 47.94% to 79.83% were achieved on the segmentation task, and accuracy of 60.22% and 50.48% for top-level and fine-grained discourse relation classification, respectively. we successfully applied the best performing models to segment and annotate the argumentative corpus dagstuhl15512 argquality and conducted the detailed error analysis. the obtained argumentative discourse units were manually corrected and annotated with the fine-grained pdtb discourse relations. this corpus contains 2,222 annotated unit pairs and presents a valuable resource for further argumentation mining studies and will be released to the community. the obtained results opened up many interesting prospects for future research. for example, various argumentation schemes can be reconstructed based on the proposed approach, and evaluated within numerous contexts and domains. argument and argumentation quality can be assessed and robust reasoning engines designed.