Computational Modelling of Undercuts in Real-world Arguments (2024)

Abstract: Argument Mining (AM) is the task of automatically analysing arguments, such that the unstructured information contained in them is converted into structured representations. Undercut is a unique structure in arguments, as it challenges the relationship between a premise and a claim, unlike direct attacks which challenge the claim or the premise itself. Undercut is also an important counterargument device as it often reflects the value of arguers. However, undercuts have not received the attention in the filed of AM they should have — there is neither much corpus data about undercuts, nor an existing AM model that can automatically recognise them. In this paper, we present a real-world dataset of arguments with explicitly annotated undercuts, and the first computational model that is able to recognise them. The dataset consists of 400 arguments, containing 326 undercuts. On this dataset, our approach beats a strong baseline in undercut recognition, with F_1 = 38.8\% , which is comparable to the performance on recognising direct attacks. We also conduct experiments on a benchmark dataset containing no undercuts, and prove that our approach is as good as the state of the art in terms of recognising the overall structure of arguments. Our work pioneers the systematic analysis and computational modelling of undercuts in real-world arguments, setting a foundation for future research in the role of undercuts in the dynamics of argumentation.

## introduction social media allows people to express divergent opinions on the same subject and to reach many more people than was possible in earlier times. however, the ubiquity of the internet and social media also has some negative consequences. one of these is the growing polarisation between individuals holding different beliefs and opinions. it is thus increasingly important to promote productive communication and understanding among people with opposing perspectives. this is where argument mining (am) comes into play. am aims to automatically identify and extract arguments from natural language texts @xcite . it can convert unstructured textual information into structured argument data, which not only identifies the argumentative text segments in the text but also the relations between them @xcite . a critical aspect of am is recognising and understanding various argumentative structures, including undercuts. an undercut challenges the relationship between a premise and a claim @xcite , unlike direct attacks that challenge the claim or the premise itself. due to its complex structure, it is difficult to annotate undercuts or to computationally model them. there exist some am datasets with annotation of undercuts @xcite ), but they are often limited in the size, the quality of source text, or the annotation scheme. to our best knowledge, there is no existing am models that can automatically recognise undercuts. to addresses this gap, in this paper we present a novel dataset of real-world arguments from quora foot_0 , a popular question-answering platform. our dataset consists of 400 arguments, including 326 explicitly annotated undercuts, making it the largest am dataset with such annotations to date. we also develop the first computational approach capable of recognising undercuts, proposing an innovative undercut-inclusive dependency representation and a gnn-based neural dependency parser. our work contributes to the field of am in several ways. firstly, we provide a comprehensive dataset with detailed annotations of undercuts, offering a valuable resource for future research. secondly, our undercut-inclusive representation allows existing neural dependency parsers to process undercuts effectively, preserving their unique status within argument structures. lastly, our experimental results demonstrate that our gnn-based parser outperforms existing biaffine parsers in recognising undercuts and maintains state-of-the-art performance in the general am task. ## related work our work is closely related to existing approaches to am, and the studies on undercuts in the field of am. ## the quoraam dataset we collected 400 arguments from quora, and named this dataset as the quoraam dataset. compared to user-generated content on other online discussion platforms such as changemyview, kialo, idebate.org, and twitter, arguments on quora are more in line with our research interest. on quora, users can present detailed and well-reasoned points of view in their answers to a question. as a result, each answer on quora can be seen as containing a stand-alone cogently structured complete argument, often supplemented with explanations and supporting evidence. platforms including changemyview, kialo and idebate.org are specifically designed for interactive debates and discussions, with a focus on the process of changing one's mind through constructive conversations. each post on such platforms may only contain an incomplete argument, and intertextual referencing is frequent among posts, making the analysis of such posts more difficult. twitter, unlike those previous platforms, is a more open and informal platform, where arguments can take on a more combative tone. as a result, arguments there may not always be rational and often lack the depth and nuance seen on the other platforms. additionally, due to twitter's character limit, arguments in tweets tend to be very short, with structures that may be too simplistic to warrant a detailed analysis. therefore, we chose quora over other online discussion platforms. we first manually pre-segmented the quoraam dataset, and then trained two annotators to annotate this dataset using our annotation scheme. this dataset will be used for our experiments in this paper. to the best of our knowledge, this is the biggest argument mining dataset that contains explicit annotation of undercuts. ## computational modelling of undercuts through a redesign of an existing dependency representation of arguments by @xcite , we are able to directly use existing neural dependency parsers to computationally model undercuts. we also apply a modified gnn-based dependency parser for improved performance. ## experiments we conducted two experiments in order to test: results from these experiments can also be used to test the effectiveness of our gnn-based parser on am in general. ## results and discussion in experiment 1, for undercut recognition, both the results from two experiments suggest that the gnn-based parser is more efficient than the biaffine parser. in experiment 1, gnn_inc significantly outperforms biaff_inc by 3.8% for component identification (gn n _inc = 66.2%, biaf f _inc = 62.4%, @xmath0.01) and 4.1% for relation identification (gn n _inc = 45.8%, biaf f _inc = 41.7%, @xmath1.01); in experiment 2, gnn_inc significantly outperforms biaff_inc by 1.0% for component identification (gn n _inc = 73.8%, biaf f _inc = 72.8%, @xmath2.01) and 3.5% for relation identification (gn n _inc = 49.4%, biaf f _inc = 45.9%, @xmath3.01). regarding the comparison between the undercutexclusive and the undercut-inclusive representations in experiment 2, the results reveals no significant difference in performance. both representations yield similar f 1 scores for component and relation identification, which implies that our undercut-inclusive representation can also be used for arguments containing no undercuts, without performance compromise. this result suggests the flexibility of our undercut-inclusive representation. ## conclusion in this study, we addressed a critical gap in am by focusing on the computational recognition of undercuts, a complex yet essential structure in arguments. existing am research has largely overlooked undercuts, primarily due to the lack of annotated datasets and effective computational models. to tackle this, we introduced a novel dataset sourced from quora. this dataset, is the largest that contains undercuts, providing a valuable resource for future am research. we also developed the first computational approach capable of recognising undercuts, featuring an undercut-inclusive dependency representation and a gnn-based neural dependency parser. our experiments indicated that our undercutinclusive representation can be effectively used for undercut recognition, and it does not compromise performance on datasets without undercuts, showcasing its flexibility and robustness. the results also demonstrated that the gnn-based parser is effective in general am as well as in recognising undercuts. the gnn-based parser's ability to capture higher-order dependencies was evident, showing a notable advantage in accurately identifying the intricate structures of undercuts.