A Unified Representation and a Decoupled Deep Learning Architecture for Argumentation Mining of Students’ Persuasive Essays (2022)

Abstract: We develop a novel unified representation for the argumentation mining task facilitating the extracting from text and the labelling of the non-argumentative units and argumentation components—premises, claims, and major claims—and the argumentative relations—premise to claim or premise in a support or attack relation, and claim to major-claim in a for or against relation—in an end-to-end machine learning pipeline. This tightly integrated representation combines the component and relation identification sub-problems and enables a unitary solution for detecting argumentation structures. This new representation together with a new deep learning architecture composed of a mixed embedding method, a multi-head attention layer, two biLSTM layers, and a final linear layer obtain state-of-the-art accuracy on the Persuasive Essays dataset. Also, we have introduced a decoupled solution to identify the entities and relations first, and on top of that, a second model is used to detect distance between the detected related components. An augmentation of the corpus (paragraph version) by including copies of major claims has further increased the performance.

## introduction arguments are composed of statements, called claims, that take a position on a controversial subject and other statements, referred to as premises, that support or rebut the claims. when arguments are presented in text form, these argument components are realized as contiguous text spans. the writing also contains non-argumentative text spans. the argument and non-argumentative text spans are collectively referred to as argumentative discourse units (adus). argumentation mining is usually viewed as the identification of argumentative structures: separating the argumentative adus from the non-argumentative adus, classifying the argumentative adus as premises and claims, and finding the relationships among the argumentative adus. since we are using the persuasive essay (pe) dataset @xcite these subtasks can be made more precise: 1) segment the argument components from the nonargumentative text, 2) label each argument component as a major-claim, claim, or premise, 3) determine which premises are in a relationship with claims or premises using a text distance measure, and 4) classify the stance of the relations between argument components. since we are using the persuasive essay (pe) dataset @xcite we will use the description of these tasks as given by @xcite : 1) segmenting the adus: separate the argumentative text spans from the non-argumentative text, 2) labeling each argument component as a major-claim, claim, or premise, 3) determining which premises are in a relationship with claims or premises and representing this relation as the text distance (the number of sentences before or after) between a premise and its related argument component (in the pe corpus, which major-claim is related to a claim is not annotated using the text distance method), and 4) classifying the stance of the relations between argument components ('for' and 'against' for the relationship between claims and major-claims; @xcite . previous research has approached the development of a computational argumentation mining method from two distinct viewpoints. input for the first approach is plain text and this approach solves all four of the subtasks mentioned above. @xcite provide the pe dataset, which we use in the development of our method. @xcite produce the state-of-the-art method to which we compare our new method. recently, @xcite , using the pe dataset, have developed an unsupervised machine learning method that provides all but the stance information for the relations. the second view of argumentation mining assumes the first subtask has been done @xcite @xcite @xcite @xcite . the method proposed here takes the first approach, solving all four subtasks. as there are subtasks, previous argumentation mining works have decoupled various subtasks, solved them separately, and then combined the solutions. the end-to-end learning method proposed here differentiates itself from these previous works by approaching the problem with a unified representation. our research contributions are summarized as follows: ## related work computational argumentation mining deals with finding argumentation structures in text. @xcite established that argument mining would need to detect claims and premises and their relationships. @xcite provided the pe dataset, a corpus annotated with a scheme that includes claims, premises, and also attack or support relations. @xcite addressed the argumentation problem by training independent models for each of the subtasks and then combining them with an integer linear programming model for the end-to-end task. @xcite achieved state-of-the-art performance on the pe corpus by addressing the problem as a sequence tagging problem. they have the best accuracy of 61.67% by using a modified version of the lstm-er model, introduced by @xcite , which uses a stacked architecture of sequence and tree lstms. @xcite presented the first findings on end-to-end argument mining in student essays using a pipeline approach by performing joint inference using an integer linear programming (ilp) framework. @xcite introduced an unsupervised approach, topic modeling, to detect claims and premises. @xcite have also developed an unsupervised machine learning method that provides all but the stance information for the relations. a number of works have investigated approaches for subtasks 2, 3, and 4. early work is epitomized by @xcite and @xcite where they develop a novel methodology for predicting argument structure by dividing it into different sub-tasks (relation, central claim, role, and function classification). @xcite presented the first neural network-based approach to argumentation mining, focusing on extracting links between argument components and classifying types of argument components as a secondary goal. niculae et al. (2017) their analysis of other works regarding the span representation led them to the development of a simple task-dependent addition for the asp. @xcite avoid previous inefficient enumeration operations for detecting relational attributes. for that, they introduce a transition-based methodology that follows an incremental procedure for building graphs based on argumentation. we note from @xcite how additional handcrafted features can boost the accuracy on certain sequence tagging tasks. @xcite and @xcite also noted the importance of discourse connectives in the argumentation mining task. ## research methodology here we present the method that we have developed to generate the argumentation structure for the pe data set. first, the data set is described. then, we introduce the multi-label representation that allows us to consider argumentation mining as a single unified problem. lastly, instead of presenting the final model with an ablation study, we present our method in a bottom-up style, starting with a base architecture to which we add, providing in ## error analysis we have done some error analysis and comparison between various neural architectures to see how differently all of the models perform on the argumentation task. also, we have measured the distance prediction accuracy of the unified-am model and compare it with that of @xcite . we observe a higher accuracy of predicting longer distance in the paragraphs. one of the key strategies that we have followed for all of these experimental setups: we ensure the models share all of their learned parameters while solving any particular subtask (component detection and labelling, relation classification, or accurate distance prediction) of the main argumentation mining problem. this denser representation of the whole argumentation task enables our neural models to share all of the parameters while making predictions for each of the subtasks which has led to a high performance. @xcite showed that lstm-er model's probability of correctness given true distance is below 40% and it becomes below 20% when the distances are larger than 3. but in our case, our analysis shows above 50% accuracy for distances 1, 2, and 3 (for unified-am). our final model (see figure 1 ) has higher accuracy regarding smaller distances but its prediction accuracy declines as we observe larger distance values in the pe corpus. for major-claim, premise, and claim, there are two different tags in the pe corpus, b: beginning of a component and i: continuation of a component. non-argumentative tokens are tagged as 'o' in the bio scheme. we compare the component segmentation task (subtask 1) results with other works that have been mentioned in @xcite . ## conclusions and future work in this work, we show that rather than using a complex stacked architecture for a problem which has different subtasks (where all the subtasks are related to each other), we can have a compact and unified representation of all the sub-problems and can tackle it as a single problem with less complicated architectures. we obtain an improved perfor-mance over @xcite in recognizing the argument components and relations. we further improve this result by introducing the flair stacked embedding @xcite to represent the text input. we introduce a multi-head attention layer to the neural architecture which leads us to the highest accuracy on the pe corpus. observing that the imbalanced corpus may be creating problems for this model to learn certain underrepresented features of the corpus, we have used the standard technique of data augmentation to achieve further gains in performance. we have created one augmented version of the pe training corpus by using different combinations of the n-grams that occur immediately before approximately two-thirds of the major claim components (see section 4.2) in the paragraph version of the corpus. by using the augmentation methodology, we further improve the unified-am model's performance on the test set. we have obtained the highest token level accuracy, c-f1, r-f1, and the global f1 score (which is the combination of both c-f1 and r-f1 scores) on the paragraph version of the pe corpus by applying the augmentation technique. we have obtained better results on the original essay version of the corpus. shared parameter values across different subtasks enhanced the accuracy score and also the model's capability for accurate detection of components, relations and distance. our work has shown a robust method which jointly solves the component and relation identification tasks on the essay and paragraph levels of the persuasive essays corpus. future work includes a modified, yet unified, representation for other corpora and using contextual embeddings to enhance the representations of the argumentative texts.