Discourse Structure-Aware Prefix for Generation-Based End-to-End Argumentation Mining (2024)

Abstract: End-to-end argumentation mining (AM) aims to extract the argumentation structure including argumentation components and their argumentation relations from text. Recent developments in end-to-end AM models have demonstrated significant progress by redefining the AM task as a sequence generation task, exhibiting simplicity and competitive performance. Nevertheless, these models overlook the integration of supplementary discourse structure information, a crucial factor for comprehending argumentation structures, resulting in suboptimal outcomes. In this study, we propose the DENIM framework, which generates discourse structure-aware prefixes for each layer of the generation model. These prefixes imbue the generation-based AM model with discourse structures, thereby augmenting the overall generation process. Moreover, we introduce a multi-task prompt coupled with a three-step decoding strategy, aiming to optimize the efficiency and effectiveness of argumentation structure decoding. Extensive experiments and analyses on two benchmark datasets show that DENIM achieves state-of-the-art performances on two AM benchmarks.

## introduction argumentation mining (am) has recently received much research attention @xcite @xcite , which aims to analyze and understand argumentation texts to obtain argumentation structure. generally, am generally comprises three subtasks following @xcite : 1) argumentation component segmentation (acs) detects the boundaries of token-level argumentative segments, which are known as acs; 2) argumentation component type classification (actc) classifies the * equal contribution. segment-level acs into the categories (i.e., claim and premise); 3) argumentation relation classification (arc) further classifies the ar types (i.e., no-relation, support and attack) between ac pair. the end-to-end am task is highly challenging due to the complexity of simultaneously addressing these three subtasks within a unified framework. early studies focus on only a subset of the three subtasks @xcite in am. recently, some studies have formulated the end-to-end am as a dependency parsing task @xcite . however, dependency parsing requires a complex pre-processing and post-processing process to match the argumentation structure with the elaborately designed dependency graph. @xcite tackled am by combining sequence labeling and multi-class classification tasks within a multi-task learning framework. inspired by the success of the generative methods, a generation-based end-to-end am model @xcite is proposed to transform the am as a unified generation task. compared to traditional classification-based methods @xcite , recently proposed generative am models @xcite have demonstrated simplicity in task decoding and competitive performance. however, existing generative approaches primarily focus on problem reformulation and, unlike traditional classification-based methods, do not incorporate additional discourse structure information, which has been proven effective for argument mining in prior works @xcite . as shown in figure 1 , the discourse structure graph derived from the input text summarizes the semantic structure of the text, where the elementary discourse units (edus) are nodes and the discourse relations (e.g., contrast) between edus form edges. in this graph, many nodes and edges share strong similarities with the argumentation structure. for example, the ac3 " they introduce new products" can be mapped to the edu5, and its relative ac can be found using edge "contrast". hence, we argue that the discourse structure graph could provide important clues for generative-based am models to figure out the argumentation structure, potentially enhancing overall performance and improving generalizability. to address the aforementioned issue, we propose denim (discourse structure-aware prefix generation-based argumentation mining), a generation-based end-to-end am model that incorporates discourse structure information into prefix @xcite to guide the generation-based am models. specifically, we employ an additional discourse structure encoder to convert the input discourse structure graph into dense vectors. then, these vectors will be disassembled and allocated to each transformer layer within generation-based am models as prefixes. these generated prefixes are converted into supplementary key and value matrices, exerting influence on the attention calculation process and guiding the generation mechanism. furthermore, we design a multi-task prompt using a fixed template and a three-step decoding strategy tailored for am. denim initially produces spans of all acs for acs, subsequently classifies their types for actc, and ultimately identifies the relations between ac pairs for arc. the computation process in each step adheres to a fixed format and is parallelized on gpus. compared to the previous generation-based am model @xcite using an autoregressive decoding strategy, our method demonstrates enhanced efficiency through task-specific decoding and improved controllability with a fixed prompt template. our contributions can be summarized as follows. (1) we propose a discourse structure-aware prefix to encode discourse structure information for the generation-based end-to-end am model. (2) we design a multi-task prompt with a three-step decoding strategy for am. this approach is more effective and efficient than the traditional autoregressive decoding strategy. (3) we conduct extensive experiments on two am benchmark datasets. the experimental results present that our method significantly outperforms the strong baselines. subsequently, these dense vectors will be disassembled and distributed to each layer of the bart so the generation is guided by the discourse structure information (section 2.3). lastly, we introduce a multi-task prompt with a three-step decoding strategy to alleviate the time-consuming and instability of sequence decoding for effectively and efficiently generating the argumentation structure (section 2.4). ## experimental setup datasets to evaluate the effectiveness of our denim model, we conduct extensive experiments on two widely used am datasets, named: aaec @xcite and ab-strct @xcite . evaluation metrics we employ the same evaluation metrics as the previous works @xcite @xcite , including f 1 score and macro averaged score (denoted as macro). we calculate f 1 scores for determining the ac span for acs. we adopt the f 1 and macro averaged score for actc and for determining the relation type (not including no-relevant) for arc. we also introduce the link score @xcite , used to measure f 1 scores for identifying the existence of relations regardless of their types. baselines we compare denim with the following strong baseline models. for the aaec dataset, we compare our model with five strong baselines, including bipam (ye and teufel, 2021), bipamsyn @xcite , bart-b @xcite , gmam @xcite and st @xcite . for the abstrct dataset, we compare our model with fhree strong baselines, which are bart-b @xcite , gmam @xcite and st @xcite . implementation details denim is implemented in pytorch on an nvidia tesla a100-pcie-40gb and employs the bart base foot_3 . our model is optimized using adaw @xcite with the learning rates of 3e-5 and weight decay of 1e-5 on both abstrct and aaec datasets. for both datasets, we set the batch size to 4 and adopt dropout @xcite with a dropout rate of 0.1 to avoid overfitting. we set the layer number l of rgcn to 1 because of its best performance. all experiments are performed five times with different random seeds, and the evaluation scores are averaged. our code is available at https://github.com/syiswell/denim . ## related work argumentation mining argumentation mining aims to extract the argumentation structure from the argumentation text, which involves three subtasks @xcite @xcite @xcite . early work @xcite performed joint inference in an integer linear programming (ilp) framework. with the success of deep learning, @xcite investigated neural techniques for am and formalized the end-to-end am task into multiple other tasks. @xcite extended the approach of @xcite using biaffine dependency parsing and achieved promising performance. however, it requires tedious pre-and postprocessing. @xcite proposed a generative framework with a constrained pointer mechanism and a reconstructed positional encoding for end-to-end am. @xcite proposed a longformer (beltagy et al., 2020) based model with biaffine functions for am. in addition, some works explore incorporating discourse information for am. @xcite introduced a non-deep learning approach that leverages rhetorical structure theory to analyze argument structures within biomedical corpora. @xcite introduced two theories, including rhetorical structure theory and segmented discourse representation theory, to annotate and analyze argumentative texts. @xcite proposed an lstm-based transfer learning approach that uses contextual representations learned from discourse parsing tasks as input for argument mining models. @xcite employed the annotated discourse units and relations as auxiliary information and proposed two transfer learning approaches (i.e., multi-task learning and sequential learning) with bilstm and crf models for am. @xcite proposed a deep dependency parsing model to explore the relationship between rhetorical and argument structures. different from them, we explore how to incorporate this additional information into a generation-based end-to-end am and enhance it with prefix and prompt learning. prefix tuning and prompt tuning prompt tuning @xcite has achieved desirable performance in the field of natural language processing (nlp) @xcite @xcite @xcite . prefix tuning @xcite , a member of the prompt-based tuning family, can trigger the desired generation of plms by only optimizing small continuous prefix vectors. to the best of our knowledge, we are the first to explore the potential of prefix tuning with discourse information for am and propose an effective and efficient multi-task prompt with a three-step decoding strategy. ## conclusion in this paper, we designed discourse structureaware prefixes, which introduce discourse structure information to the generation-based am model, thereby enhancing the generation. additionally, we introduce a multi-task prompt complemented by a three-step decoding strategy, optimizing the efficiency and effectiveness of argumentation structure decoding. extensive experiments and analyses on two benchmarks show that our method outperformed strong baselines significantly.