Can Unsupervised Knowledge Transfer from Social Discussions Help Argument Mining? (2022)

Abstract: Identifying argument components from unstructured texts and predicting the relationships expressed among them are two primary steps of argument mining. The intrinsic complexity of these tasks demands powerful learning models. While pretrained Transformer-based Language Models (LM) have been shown to provide state-of-the-art results over different NLP tasks, the scarcity of manually annotated data and the highly domain-dependent nature of argumentation restrict the capabilities of such models. In this work, we propose a novel transfer learning strategy to overcome these challenges. We utilize argumentation-rich social discussions from the ChangeMyView subreddit as a source of unsupervised, argumentative discourse-aware knowledge by finetuning pretrained LMs on a selectively masked language modeling task. Furthermore, we introduce a novel prompt-based strategy for inter-component relation prediction that compliments our proposed finetuning method while leveraging on the discourse context. Exhaustive experiments show the generalization capability of our method on these two tasks over within-domain as well as out-of-domain datasets, outperforming several existing and employed strong baselines.

## introduction computational argument mining from texts is the fine-grained process of understanding opinion dynamics. in the most fundamental sense, argument understanding requires the identification of the opinions posed and justifications provided to support or falsify them. generally, automated argument mining is a multi-stage pipeline identified with three general steps @xcite -separating argumentative spans from non-argumentative ones, classifying argument components, and inducing a structure among them @xcite . while different argumentation models define different taxonomies for argument components, popular approaches broadly categorize them as 'claims' and 'premises' @xcite @xcite . as these components are not necessarily aligned to sentence-level segments and can be reflected within clausal levels, the task of argument component identification requires a token-level boundary detection of components and component type classification. context of argumentation in online discussions. online discussions originating from backand-forth posts from users reflect a rich interaction of opinion dynamics on large scale. in figure 1 , we show a sample argument component annotation of consecutive posts from two users. the token-level granularity of components ensures that a single sentence may contain multiple components of the same (in 1st post) or different kinds (in 2nd and 4th posts). moreover, two adjacent spans of texts, even with the same argumentative role, can be defined as two separate components (see the 4th post for example). it is trivial to say that the meaning of any post (as well as its argumentative role) is de-pendent on the context. to be specific, the third post can be identified as argumentative (a premise in this case) only when its predecessor post and its components are taken as the context. similarly, a certain span of the first post is quoted in the second one signaling a concrete manifestation of dialogic continuity. one may even observe the user-specific argumentation styles: 1st user (author of the first and third posts) usually keeps claims and premises in separate sentences, while the 2nd user prefers to use multi-component, complex sentences. existing studies on argumentation formalism recognize such continuity and define inter-post component relations @xcite . however, the previous approaches for automated extraction, classification and relating argumentative components work on individual posts only and define the inter-post discourse in the later stages of relation prediction. this is trivially counter-intuitive for two major reasons: (i) if we consider two text spans from separate comments to be linked by some argumentative relation, then there exists a continuity of discourse between these spans and a model is likely to benefit if it decides the boundaries and types of these two components conditioned on that continuous information; (ii) users carry their style of argumentation (simple consecutive sentences vs. long complex ones, usage of particular markers like 'i think that' etc.), and if the model is informed about these while observing the complete conversation with back-and-forth posts, it is more likely to extract correct components easily. scarcity of labeled data. irrespective of the domain, argument annotation is a resource-intensive process. a few previous studies @xcite attempted to exploit a large amount of unlabeled data in a semi-supervised fashion. however, such methods require the components to be defined at sentencelevel (and thereby adding redundant spans into the predictions) as they perform some sentence similarity matching to generate pseudo-labels. pretrained language models like bert @xcite provide a workaround to handle the scarcity of task-specific annotated data. a parameter-intensive model is initially trained in a self-supervised manner on a large bulk of text; this pretraining enables the model to learn general language representation, which is then finetuned on task-specific labeled data. however, the amount of the latter still deter-mines the expressive power of such models @xcite . present work. considering these challenges, we formulate a novel transfer learning method using transformer-based language models. we use large amount of unlabelled discussion threads from reddit's r/changemyview (cmv) community as the source of argumentative knowledge. pretrained, transformer-based language models are finetuned on this dataset using a masked language modelling task. instead of randomly masking tokens to predict, we select several markers in the text that are shown to signal argumentative discourse in previous works @xcite . the language models are then made to predict these markers in the mlm task, thereby learning to relate different components of text according to their role in the argumentation presented. we call this novel finetuning method selective masked language modeling (smlm). furthermore, to explore the role of context in argument mining, we use smlm to finetune a post-level language model based on bert @xcite and roberta @xcite and a thread-level language model based on longformer @xcite . we present efficient incorporation of several reddit-specific structural cues into the longformer architecture. these finetuned language models are then used for two fundamental components of argument mining: tokenlevel argument component identification (aci) and inter-component relation type prediction (rtp). to further utilize the smlm-based training of the language models, we propose a novel prompt-based approach to predict relations among argument components. we perform exhaustive experiments to explore the efficacy of our proposed methods for argument mining in both in-domain and out-ofdomain benchmark datasets: manually annotated reddit discussions and scientific papers. our experiments show clear improvements achieved by our methods (0.59 and 0.69 f1 for aci and rtp, respectively) over several state-of-the-art baselines. foot_0 2 related work a general overview of argument mining can be found in the survey articles by @xcite and @xcite . in the current scope, we look into three major areas of research in argument mining. ## selective mlm finetuning of pretrained language models though pretrained language models are developed to overcome the problem of small annotated data on different language processing tasks, they still require task-specific finetuning for better results @xcite . in the specific domain of argument mining, annotated data is scarce, and attempting to finetune a massive language model with very small training data comes with the risk of overfitting. moreover, different datasets follow different strategies for annotation. we seek to devise a novel transfer learning strategy where a given transformer-based pretrained language model is directed to focus on argumentative discourse using large-scale, unlabelled data. we choose the changemyview (cmv) community as the source of this transfer for two specific reasons: (i) it provides us with a large, readily available resource of interactions strictly focused on debates around versatile topics, and (ii) discussions in cmv contain a mixture of dialogic continuity over successive turns along with elaborate argumentation presented in a single turn. we hypothesize that such a versatile combination of discourse can make the language model more generalizable over dialogic as well as monologic argument mining tasks. ## evaluation we evaluate the models based on precision, recall, and f1 scores for predicting claims and premises. for a more rigorous setting, we use exact match of the whole span between gold and predicted labels, i.e., if the gold label is [o, b-claim, i-claim, i-claim, i-claim, o] then only the predictions [o, b-claim, i-claim, i-claim, i-claim, o], or [o, iclaim, i-claim, i-claim, i-claim, o] can be considered as true positives. we use the popular seqeval (nakayama, 2018) framework. ## conclusion we presented the results for two important tasks in the argument mining pipeline, viz., aci and rtp. the experiments clearly elucidated the importance of alignment between the downstream and pre-trainig tasks, and the effect of various ways of modelling the tasks. the importance of entire thread's context in discussion forums, as well