A Generative Model for End-to-End Argument Mining with Reconstructed Positional Encoding and Constrained Pointer Mechanism (2022)

Abstract: Argument mining (AM) is a challenging task as it requires recognizing the complex argumentation structures involving multiple subtasks.To handle all subtasks of AM in an end-to-end fashion, previous works generally transform AM into a dependency parsing task.However, such methods largely require complex pre- and post-processing to realize the task transformation.In this paper, we investigate the end-to-end AM task from a novel perspective by proposing a generative framework, in which the expected outputs of AM are framed as a simple target sequence. Then, we employ a pre-trained sequence-to-sequence language model with a constrained pointer mechanism (CPM) to model the clues for all the subtasks of AM in the light of the target sequence. Furthermore, we devise a reconstructed positional encoding (RPE) to alleviate the order biases induced by the autoregressive generation paradigm.Experimental results show that our proposed framework achieves new state-of-the-art performance on two AM benchmarks.

## introduction as a fundamental task of computational argumentation, argument mining (am) has drawn much research attention recently @xcite @xcite . the ultimate goal of am is to analyze and understand argumentative text, so as to obtain structured argumentation knowledge that can support a diverse range of downstream tasks, such as argument persuasiveness prediction @xcite , automated essay scoring @xcite @xcite , argument generation @xcite figure 1: a simplified example of am task. two argument components are marked in green and blue, respectively, where the former is a claim and the latter is a premise. in addition, there is a support relation from the premise to the claim. our proposed target sequence corresponding to this example is shown at the bottom. given a piece of argumentative text as input, an end-to-end am system needs to identify both the argument components (acs) and the argumentative relations (ars) between them. an example is shown in figure 1 . specifically, am generally comprises four fine-grained subtasks @xcite : 1) component segmentation detects the boundaries of fine-grained argumentative segments, which are known as acs; 2) component classification classifies the acs into the categories defined by argumentation schemes; 3) relation detection determines whether there is an ar between two acs; 4) relation classification further classifies the types of the ars. following @xcite and @xcite , we refer the first two subtasks as argument component identification (aci), and the last two subtasks as argumentative relation identification (ari). the end-to-end am task is highly challenging as it is difficult to solve all the am subtasks synchronously in a unified framework. most previous work focuses on only a subset of the four fine-grained subtasks @xcite @xcite @xcite @xcite . however, only a limited number of studies are devoted to the end-to-end am scenario @xcite @xcite . recent research efforts formulate the end-to-end am as a dependency parsing task and apply existing dependency parsers to solve it @xcite . such methods, however, require not only a tedious preprocessing process to transform the argumentation structure into an elaborately-designed dependency graph, but also a complex post-processing process to ensure that each dependency of the predicted output is completely consistent with that in the desired dependency graph of the am task. thus, developing an elegant, simple, and effective framework for the end-to-end am task is still an important challenge of great significance. inspired by the recent success of the generative methods for information extraction @xcite , we propose to address the end-to-end am task via a unified generative framework. we first devise a target sequence to express the outputs of all the subtasks of am. an example is shown in figure 1 . subsequently, the pre-trained bart @xcite model is adopted to model the dependencies between the target sequence and the input argumentative text through the pointer mechanism. further, we introduce a reconstructed positional encoding (rpe) scheme in the bart decoder to alleviate the order biases induced by the autoregressive generation paradigm. in addition, considering the long length of the acs and the pattern of the target sequence, we present a constrained pointer mechanism (cpm), which is manifested as an auxiliary task at the training time and as a constrained decoding method at the inference time. this constrained pointer mechanism can help the model to generate more accurate ac boundaries and fewer invalid target sequences. compared to the previous dependency parsing-based method, it is more straightforward and easier to formalize the end-to-end am task into a generation task. also, in our proposed method, the predicted target sequence can be easily converted to the expected outputs of am without complex post-processing. we conduct extensive experiments on two am benchmarks of different structures to show the superiority of our method. experimental results show that our proposed method achieves substantial improvements over several strong baselines, yielding the state-of-the-art performance on both benchmark datasets. in addition, we carry out further analysis to show that the proposed rpe and cpm can significantly reduce the errors in the generated target sequence, thus leading to performance improvements. ## task formulation formally, for the end-to-end am, the input is a piece of argumentative text @xmath0, w 2 , . . . , w nx ] with n x tokens. the first goal is to extract a set of acs ## method inspired by @xcite , we utilize a bartbased generative framework as our basic model, which takes x as input and generates the target sequence y with the vanilla pointer mechanism. since acs are much longer and have more ambiguous boundaries than named entities @xcite or aspect term @xcite , it is more challenging to solve the end-to-end am task by a generative framework. hence, we introduce a constrained pointer mechanism (cpm) to help the model generate more accurate ac boundaries and fewer invalid predictions. further, we propose a reconstructed positional encoding (rpe) to alleviate the order biases introduced by the autoregressive paradigm in the basic model. ## conclusion in this paper, we transform the end-to-end am task into a generation task and apply the pre-trained bart model with a pointer mechanism to solve it. to better adapt this generative model to the endto-end am task, we replace the original positional encoding of the bart decoder with our proposed reconstructed positional encoding. on the other hand, we present a constrained pointer mechanism to further improve our model, which is achieved by multi-task learning during training and constrained decoding during inference. the extensive experimental results and detailed analysis demonstrate the superiority of our proposed method.