Multi-Task Learning Improves Performance in Deep Argument Mining Models (2024)

Abstract: The successful analysis of argumentative techniques in user-generated text is central to many downstream tasks such as political and market analysis. Recent argument mining tools use state-of-the-art deep learning methods to extract and annotate argumentative techniques from various online text corpora, but each task is treated as separate and different bespoke models are fine-tuned for each dataset. We show that different argument mining tasks share common semantic and logical structure by implementing a multi-task approach to argument mining that meets or exceeds performance from existing methods for the same problems. Our model builds a shared representation of the input and exploits similarities between tasks in order to further boost performance via parameter-sharing. Our results are important for argument mining as they show that different tasks share substantial similarities and suggest a holistic approach to the extraction of argumentative techniques from text.

## introduction text content generated by online users is a fundamental source of information for understanding the ideas, feelings, and behavior of large populations of interest for social scientists. within these texts, it is important to be able to recognize ideas and worldviews expressed by individuals on a large scale. to this end, argument mining (am) has emerged in recent years as a sub-field of natural language processing (nlp) focusing on creating language models capable of detecting and classifying argumentative strategies in online texts. within am, several different sub-tasks have been proposed. for example, @xcite focus on identifying agreement and disagreement in online texts, @xcite propose a method to distinguish factual from emotional argumentation techniques, @xcite detect the presence of certain rhetorical figures in arguments, and @xcite produce measures of argument quality. these are only some examples of the many distinct classification tasks that have been identified in am, not to mention a wide range of work on span identification (e.g. @xcite . in this paper, we suggest that all these am sub-tasks share substantial similarity and use this idea to formulate a model that achieves high accuracy in several of these problems. more specifically, existing work in am treats many of the classification tasks within the field as separate problems and focuses on fine-tuning bespoke models for each task (e.g. @xcite @xcite . while this approach has been demonstrated to work in many settings, it fails to take advantage of the substantial similarities between am tasks. in this paper we propose to take advantage of the similarities across am tasks by constructing a multi-task model @xcite @xcite , that is trained on all tasks at once and builds a shared latent representation of the inputs for each task, and uses this representation to make more accurate predictions for each individual task. our models also provide evidence that am sub-tasks do indeed share substantial conceptual overlap @xcite ; the latent representations of different tasks output by our model depicted in figure 1 clearly depict clusters of individual tasks as well as substantial overlap between these clusters in representation space, indicating that the same latent features are informative for multiple tasks. the model we propose achieves performance similar to or greater than existing models on all tasks for which we had information on previous metrics, and it also surpasses individual-task models fine-tuned on similar architectures for these tasks. in addition, our models allow for substantial computational gains over individual-task models as they permit training inference for many outputs at once, instead of training and evaluating an individual model for each desired task. overall, our results have the important implication for am as a field that further research and model-building should not only focus on taking advantage of the structure of the specific task of interest (e.g. @xcite , but also on incorporating information from similar tasks into the model for better performance. ## related work we build on an active research agenda in argument mining (am)-the automated extraction of argumentative structure, reasoning, and features from text @xcite . @xcite identify two stages in am: identifying arguments within documents and classifying those arguments on their characteristics, such as supporting, attacking, or background information. our work is situated in the second stage, involving the identification of features or typologies of arguments. much computational work in am has investigated argumentation in online interactions @xcite @xcite , due in part to the vast amounts of available data and the ease of collecting it. but some scholars have used news articles to construct corpora of propaganda and fact-checking (e.g., @xcite . still others have leveraged monologues like persuasive essays or legal decisions @xcite . we incorporate all three types of data into our models to further show that tasks with different data-generating processes and textual characteristics nevertheless exhibit common semantic structure. there is evidence that many natural language tasks share a common core @xcite , and models trained on one task tend to also perform well on others. @xcite demonstrate that multi-task approaches benefit model performance in several natural language tasks such as topic detection and sentiment analysis. multi-task approaches have been more rare within am, but two existing works suggest the framework may offer benefits to these unique tasks. @xcite apply multi-task learning to token-level tagging for am tasks. they consider six datasets each with different token annotations and train a recurrent model to learn all of them at once, providing initial evidence that gains in model performance can be attained via multi-task learning in am. similar results are echoed in schiller et al., who focus on stance detection instead. @xcite present an end-to-end multi-task architecture for identifying argument components in unstructured text. our task differs from theirs in two ways: first, morio et al.'s model is especially focused on span identification and relation classification, wherein the model links, for example, premise to claim. our primary objective in this paper would more accurately be described as component classification. second, the component classification part of their model is focused on more traditional am tasks like classifying texts as claims for or against. we focus instead on argument characteristics that are more complex or subjective and, in some cases, can be understood as being nested within overarching concepts. @xcite propose a model trained simultaneously on two tasks: argument identification within texts and argument-rebuttal pair matching across texts. again, our approach is focused on argument classification, not span identification, and we aim to classify argument types. moreover, our use of multi-task learning differs slightly. whereas cheng et al. train a model to perform two complementary but quite distinct tasks, we show that a single model can perform multiple argument classification tasks simultaneously. accordingly, our proposed architecture differs from both @xcite a prevalent model architecture for multi-task learning within computer vision involves segre-gating the network into shared and task-specific components. this conventional structure, termed a "shared trunk" @xcite , typically comprises a universal feature extractor, constructed of convolutional layers that are employed by all tasks, and a distinct output branch for each task @xcite @xcite . further enhancements on this shared trunk template have been made by @xcite and @xcite , who incorporated taskspecific modules into the original framework. this template is not confined to computer vision but is also prevalent in multi-task learning models in nlp. traditional feed-forward architectures, using the shared trunk template in combination with task-specific modules, have been utilized for multitask nlp by a variety of researchers @xcite @xcite . these architectures bear a structural resemblance to their counterparts in computer vision, featuring a shared, global feature extractor followed by task-specific output branches. however, in the context of nlp, the features in question are text representations. ## data we draw on three benchmark corpora to create a dataset with a diverse number of argument characteristics. we take eight tasks from the internet argument corpus (iac), a collection of posts extracted from several online debate and discussion fora @xcite . each post is annotated on a variety of characteristics, such as whether the post expresses disagreement or uses an emotion-or fact-based argument, with a value in [-5, 5] on each characteristic. some researchers have dichotomized these data by removing observations around the midpoint @xcite . this practice is not appropriate in the multi-task setting, however, as it would remove too much information that the model could use to build shared representations across tasks. instead, we dichotomize the data by simply cutting on the scale midpoint. a wide array of studies have used the iac to construct unique tasks @xcite @xcite and train single-task models @xcite @xcite . three tasks have received notable attention: the classification of disagreement @xcite ; @xcite , emotional or factual arguments @xcite , and nasty or nice tone @xcite . the second benchmark corpus we draw on is ibm-rank-30k, a corpus of crowd-sourced arguments @xcite . two quality scoring functions then translated binary annotations into a continuous value of argument quality in [0, 1]. we use scores produced by the authors' weighted average scoring function because it accounts for coder reliability, leading to less noisy annotations. as with the iac labels, we dichotomize the data by cutting on the scale midpoint. the final corpus is introduced by @xcite , who collect articles from both propagandistic and non-propagandistic news sources and annotate sentences within each article that contain one or more of eighteen different propaganda techniques, such as loaded language or causal oversimplification. we extract all sentences from each article, including those that are annotated as containing no propaganda techniques. data from all three corpora are combined to create our final dataset. we use 80% of the data for training and set aside 10% each for validation and test sets. finally, to help guard against overfitting, we conduct four types of data augmentation on the training set @xcite . in back-translation, we translate the text into a different language, then translate it back to the original language. we choose german as the target language for its high lexical similarity to english. in contextual word embedding, we randomly choose thirty percent of tokens, feed the surrounding words to bert @xcite , and substitute the predicted word in for the original. in synonym augmentation, we randomly choose thirty percent of tokens and substitute the most similar word from the wordnet lexical database @xcite . finally, in random cropping, we randomly delete thirty percent of tokens. ## empirical results we evaluate our multi-task model's performance in terms of prediction metrics, computational efficiency, and comparison against existing metrics on the target labels. we also offer evidence that the tasks we combine do indeed share important similarities by presenting text embeddings and intermediate layer representations, in figure 1 and figure 3 . we show that our model performs favorably in comparison to previously published models ( ## conclusion natural language tasks share substantial semantic and structural similarities, and deep learning models have been shown to be able to take advantage of these similarities in order to achieve better performance @xcite . in this paper, we further extend this result to the field of argument mining. we show that am tasks do indeed share a substantial amount of features, and that these shared features can be used to boost model performance across previously unrelated tasks. we combine three data sources and propose models that outperform existing models on several of these tasks. our models are also more computationally efficient and have better overall predictive accuracy than single-task models with comparable architectures. aside from the practical usefulness of our models, our results are important for argument mining as a field, as they suggest that further research and model building should focus on exploiting commonalities between different tasks to boost performance. in future work, we propose to extend our analysis to several other am tasks that share commonalities with those studied here (e.g. @xcite , as well as other language tasks such as topic modeling. we also propose devising improved model architectures for our multi-task setting. in particular, we propose to take advantage of frameworks such as contrastive learning (e.g. @xcite to encode known similarities between tasks within the representations learned by the model.