Exploiting Dialogue Acts and Context to Identify Argumentative Relations in Online Debates (2024)

Abstract: Argumentative Relation Classification is the task of determining the relationship between two contributions in the context of an argumentative dialogue. Existing models in the literature rely on a combination of lexical features and pre-trained language models to tackle this task; while this approach is somewhat effective, it fails to take into account the importance of pragmatic features such as the illocutionary force of the argument or the structure of previous utterances in the discussion; relying solely on lexical features also produces models that over-fit their initial training set and do not scale to unseen domains. In this work, we introduce ArguNet, a new model for Argumentative Relation Classification which relies on a combination of Dialogue Acts and Dialogue Context to improve the representation of argument structures in opinionated dialogues. We show that our model achieves state-of-the-art results on the Kialo benchmark test set, and provide evidence of its robustness in an open-domain scenario.

## introduction argumentative dialogues are discussions between two or more parties involving an opinionated topic, i.e. any topic which may divide the interlocutors into a number of conflicting opinions. these discussions are usually different from ordinary conversations, in that the speakers' goal is usually to convince their interlocutors of their own point of view by defending their own stance and attacking their opponents' arguments. figure 1 shows an example of a debate from the kialo online debate platform. a key aspect in the study of argumentative dialogues is identifying the relationship between an argument step in the discussion and preceding argument steps introduced by other speakers; this task is commonly referred to as argumentative relation classification @xcite , or sometimes argument polarity prediction @xcite when it only involves a binary classification between two possible relations. in this work, we will use the term argumentative relation classification, to avoid any confusion with similar tasks such as sentiment analysis or stance classification. figure 1 : an example of a debate from the kialo online debate platform. green nodes agree with the original thesis (in blue), while red nodes disagree with it. nodes are annotated with the argumentative move that they perform on their parent node in the graph (i.e. support or attack). users annotate their own stance towards the thesis, as well as their argumentative move towards the node they are responding to. existing works in the literature that aim to solve this task usually rely on either hand-crafted syntactic and lexical features @xcite , pre-trained language models @xcite or both @xcite . while these models are becoming increasingly accurate, there are some shortcomings in their approach. first, they often ignore any non-lexical aspect of the dialogue, which hinders their capability to correctly understand the conversation. second, they have limited understanding of the surrounding context of the argument contributions, and struggle to take long-term dependencies into account. finally, they are often tested in a domain-specific scenario in which the system learns to predict relations between argument contributions that belong in the same dataset it was trained on; this makes it hard to correctly assess their capability to adapt to unseen conversations, which is crucial for practical applications such as the development of automated dialogue agents. in this work, we explore the hypothesis that contextual information and pragmatic features (such as dialogue act tags) can be highly beneficial in increasing the accuracy of argumentative relation classification models. we also aim at analysing how much existing models can generalise to entirely unseen topics of discussion, and how these features can help a model become less dependent on its training domain. there is evidence in the literature that dialogue act tags may be used as a feature to improve a model's understanding of the argumentative structure of a debate @xcite . there is also evidence that contextual information is highly beneficial for argument mining tasks and, more specifically, to increase the accuracy of argumentative relation classification models @xcite . we build on this existing evidence and introduce argunet, a novel approach to argumentative relation classification that relies on on a combination of dialogue acts and a specialised encoding of the previous nodes in the debate. argunet uses iso 24617-2 dialogue acts (das) annotated with the dashnet architecture @xcite to enrich the input utterances with additional syntactic and pragmatic information. bert @xcite is used to encode the enriched input utterances into dense sentence embeddings, with the addition of utterance manipulation strategies from @xcite to further increase the effectiveness of the contextual embeddings from bert. our approach is trained and tested on data from the kialo online debate platform, a high-quality, publicly-available source of conversations annotated with argumentative relations. we use the same kialo scrape introduced by @xcite ; however, instead of shuffling the argument contributions and dividing them in a training and test split, we split at the debate level, so that contributions from the same debate will not appear in different splits. this is done to test the hypothesis that existing models identify lexical information in the training debates and are able to use this information when tested on argument contributions from the same debates. we also sampled an additional, smaller collection of kialo debates called kialoabortion that involve discussions on reproductive rights, which we use to further test our hypothesis that argumentative relation classification is highly sensitive to the topic of the classified debates. in our experimental section, we provide evidence that the argunet architecture achieves state-of-theart results on the kialo dataset; we also provide evidence that our model outperforms existing models in the literature when tested on debates from the kialoabortion test set, which shows how ar-gunet can generalise to unseen domains better than existing architectures. ## related work the formal study of argumentative discussions is known in the literature as argumentation theory @xcite . @xcite divides argumentative study into four separate tasks: identification, which involves identifying argumentative dialogue units (adus) in a dialogue and inserting them into a pre-determined argumentation scheme; analysis, which deals with identifying premises and conclusion of each argument; evaluation, which involves assessing an argument's quality and persuasive power; and invention, which involves the creation of novel arguments for the debate. in this work we will focus on the task of identification of pre-constructed adus in an argumentation scheme. the identification of a logical structure for reasoning goes back to the seminal works by @xcite and @xcite , which introduced defeasible logic, a formalism in which conclusions are supported by premises that may no longer be justified when additional premises are introduced. dung (1995) introduced an abstract theory of acceptability of arguments in which arguments are seen as a set of logical statements, and each argument can be accepted or defeated depending on whether it clashes with other arguments. @xcite elaborated on this theory and presented a framework for structured arguments in which arguments can be supported with premises that justify their validity, and other arguments can attack the speaker's viewpoint by either attacking the argument directly, or one of its premises. @xcite combine textual entailment and argumentation graph into a unified framework that aims at automatically detecting accepted and defeated arguments based on the entailment between them. @xcite adopted this scheme in their study on argumentative relation classification on the kialo corpus, and defined default inference and default conflict relations between arguments that support and attack each other respectively. the scheme was adopted by @xcite , who use natural language inference models to directly compute argumentative relations. this approach, however, does not distinguish between the semantic problem of determining logical relations between argument steps and the pragmatic problem of determining dialogue moves in a sequence of contributions. @xcite introduced a graph-like scheme for argumentative moves in a debate called the bipolar argumentation graph (bag), in which claims are represented as nodes in a weighted graph, and can be supported by other claims or premises that can either support or attack each other. as the kialo dataset uses a graph-like structure that resembles a bag, we will sometimes use their terminology in this work, particularly when referring to the argumentative moves between argument nodes. various models have been proposed in the literature for the annotation of argumentation schemes. one of the earliest examples of a formal approach to argumentative relation classification is @xcite , which proposes an approach based on textual entailment. @xcite uses a combination of skip-thought vectors and cosine similarity to predict argumentative relations in parliamentary debates; their work is one of the earliest that takes advantage of pre-trained word embeddings for this task. @xcite propose a neural architecture based on lstm cells to annotate a multi-topic corpus which included debates on movies, technology and politics; they formulate the problem as a three-way classification problem between the classes attack, support and neither. @xcite proposed a set of strong baselines for argumentative relation prediction in a dataset-independent setting, which included an attention-based model and an autoencoder. their emphasis on dataset-independent classification is highly relevant to our work; however, they do not analyse the difference between in-domain and out-of-domain accuracy for their model and they do not provide details on how they split their data when separating training and test sets. recently, @xcite proposed graphnli, a graph-based neural architecture that uses graph walking techniques to obtain contextual information, which is then encoded with roberta embeddings @xcite . their model was a source of inspiration for our work, as it shares our reliance on context encoding for argumentative relations classification; however, their approach does not use pragmatic features like dialogue acts, and it also uses weighted averaging for embeddings rather than relying on a structured approach for context encoding, which we argue is less effective when trying to capture contextual information. the idea of adopting dialogue acts (das) as input features for argument mining systems has been investigated before in the literature. @xcite proposed a unified framework for argumentative analysis and inference which used das as part of the argumentation scheme, and used it to annotate a discussion from @xcite . @xcite introduced inference anchoring theory (iat), a framework designed to model arguments via a combination of argumentative moves and the das associated with them. both works utilised da schemes that are difficult to adopt due to the scarcity of annotated data. @xcite use iso 24617-2 das as part of a model designed to understand the ar-dimension communicative function task propquestion, setquestion, choicequestion, inform, agree, disagree, answer, directive, commissive social greeting, goodbye, thanking, acceptthanking, apology, ac-ceptapology feedback allofeedback ## experiments and results in this section, we illustrate the results of our experimental study. we ran three sets of experiments for this study: the first one was aimed at assessing argunet's accuracy when trying to determine the argumentative relation between two argument contributions, and compare it to existing methods in the literature, the second one was aimed at measuring the impact of each feature of the model via an ablation study, while the third one aimed at measuring how much our model and existing models rely on domain-specific lexical information in order to produce their prediction. we replicated the following models from the literature: for the ablation study, we implemented the following variations of the argunet model: the ums strategies are still maintained in the network. all of the variants implemented for the ablation study maintain a context window of @xmath0. ## conclusions in this work, we introduced argunet, a neural model for the classification of argumentative relations between argument contributions in online debates. we showed how it achieves state-of-theart results when tested on the kialo dataset of online debates, and provided evidence that its defining features, namely the use of dialogue acts and well-structured encoding of the context of the conversation, are highly beneficial for the task at hand. finally, we showed how its architecture is more robust to out-of-domain classification when compared to existing approaches in the literature, and provided a comparison between in-domain and outof-domain performance for all of our baselines.