Transferring Confluent Knowledge to Argument Mining (2022)

Abstract: Relevant to all application domains where it is important to get at the reasons underlying sentiments and decisions, argument mining seeks to obtain structured arguments from unstructured text and has been addressed by approaches typically involving some feature and/or neural architecture engineering. By adopting a transfer learning methodology, and by means of a systematic study with a wide range of knowledge sources promisingly suitable to leverage argument mining, the aim of this paper is to empirically assess the potential of transferring such knowledge learned with confluent tasks. By adopting a lean approach that dispenses with heavier feature and model engineering, this study permitted both to gain novel empirically based insights into the argument mining task and to establish new state of the art levels of performance for its three main sub-tasks, viz. identification of argument components, classification of the components, and determination of the relation among them.

## introduction argument mining is a natural language processing (nlp) task consisting in taking unstructured text as input and returning it annotated such that each portion occurring in it that is an argument is properly delimited and analysed @xcite @xcite @xcite @xcite . argument mining relates to the high-level human capacity of reasoning @xcite , it is at the core of social interaction concerned with persuasion @xcite , and it is of utmost importance to enhance applications across different domains that aim at enhancing their services beyond mere sentiment analysis, on the basis of the reasons uncovered for the associated sentiments and decisions @xcite . argument mining has been decomposed into a number of sub-tasks. while the number and profiling of these tasks depends on the theoretical approach adopted to analyse arguments @xcite , they typically involve some sort of delimitation of the text segments conveying argument components, the classification of the roles of these components (e.g. premises, conclusions, etc.), and the classification of the type of relation among those components (e.g. support, attack, etc.) @xcite . these sub-tasks and their eventual pipeline in argument mining have been addressed by means of supervised deep learning approaches that involve some degree of neural architecture engineering @xcite @xcite a.o. recently, first attempts to approach argument mining with transformers have been reported in the literature @xcite a.o., tough at an exploratory level that leaves much of its strength still untapped. this has been combined with experimentation with transfer learning @xcite . given its complexity, and the associated difficulty in producing gold labelled data, argument mining is a task with a scarcity of data sets that are needed to support supervised learning approaches. enhancing the argument mining task by transferring knowledge elicited when solving other natural language processing tasks is thus a promising approach to alleviate such scarceness. this has been tried in the literature @xcite @xcite @xcite a.o., though at a haphazard level that leaves still much of its potential to be studied. for humans, argumentation is a high level cognitive task that goes together with a number of other capacities relating to linguistic syntactic and semantic processing, to entailment and paraphrasing, to question answering and language comprehension, to reasoning, to common sense, etc. @xcite . interestingly, there is now available in the literature a wide range of data sets and respective nlp tasks that permit to address a wide range of these different dimensions and use them as auxiliary sources of knowledge in transfer learning approaches to argument mining @xcite a.o. in this context, our goal is to empirically assess the potential of transfer learning to support argument mining by means of a systematic study with a wide range of possible sources of related tasks and knowledge possibly suitable to be transferred. in this paper we report on the findings of exploring a vast experimental space that results from: performing sequential single-step transfer learning from over 40 auxiliary tasks to each one of three main sub-tasks of argument mining @xcite during the fine-tuning phase (section 4); further explore the source tasks that supported the best single-step transfer learning by experimenting with ways of possibly combining them in multi-step transfer learning processes, and further explore these tasks in a multi-task transfer learning setting (section 5). this is preceded by an overview of related work (section 2) and by the presentation of the experimental setup adopted (section 3). by undertaking this study, not only new state-ofthe-art results were achieved for argument mining, as also new empirically based insights were gained on how this task can be enhanced, showing the effectiveness of transfer learning to leverage argument mining and to alleviate its data scarcity when combined with a lean approach that dispenses with heavier feature and model engineering. ## related work transfer learning is a technique in machine learning that leverages knowledge from other, so called source tasks to improve the learning of a target task @xcite , being a methodology to alleviate the lack of labelled data for the latter @xcite . ## experimental space and settings for the tasks that are the source of knowledge to be transferred to argument mining models, we resorted to a vast array of annotated data sets listed in ## single-step transfer a first batch of experiments was concerned with single-step sequential transfer learning where the source tasks were those listed in given the large number of data points in this experimental space, concessions were made considering the compute footprint, and we limited the hyper-parameter search by using the recommended values @xcite ). 4 4 inspired by the stilt approach @xcite we adopted the jiant toolkit @xcite , an open source toolkit for transfer learning experiments. for the fine-tuning of the target tasks, we performed a hyperparameter search with three learning rates and three seeds on the target task development set, creating a total of 396 models. the aaec development set was extracted from 10% of the original training data, thus the training data consists of the remaining 90%. based on the top-performing result obtained from the development set, hyper-parameters were determined for the test set. further descriptions of hyperparameterization together with all materials to reproduce the experiments are available at https://github.com/ nlx-group/transfer-am . ## multi-step and multi-task transfer a second batch of experiments was concerned with multi-step and multi-task transfer learning. the 5 all scores were obtained with roberta-base. finally, besides sequential transfer, also multitask transfer learning was experimented with, involving the three argument mining sub-tasks altogether, and also pairs including two of them. motivated by these pairings of the sub-tasks, we returned to one-step methodology, and for the sake of completeness, we experimented also with every combination of two such sub-tasks. ## further analysis no correlation was found between the task scores and the size of their training data. using the coefficient of determination, .101/.002 and .001 r 2 is obtained for identifying argument components, clausal and relational properties, respectively. we performed a manual analysis of the output on top-performing tasks in the single-step transfer (cb, rte, quail). we notice that shorter arguments tend to be incorrectly tagged as o (outside) while more extensive arguments tend to be incorrectly divided into two arguments; also, some discourse markers introducing arguments, as "there is clear evidence that" or "thus it is apparent that", tend to be wrongly labelled as the beginning and inside of an argument segment. transfer learning experiments on clausal properties follow the same error pattern as the baseline, with most errors emerging from labelling major claims as claims, claims as premises and premises as claims. for relation identification, linked arguments were identified with higher precision and recall than the baseline. transferring knowledge from argument mining sources was examined also by extending the language modelling phase. despite above-baseline scores, no statistical significance was found. 7 7 conclusions and future work the results in this paper were obtained from a large experimental space that permitted a systematic empirical study aimed at assessing the viability of transfer learning to leverage neural argument mining with confluent knowledge. major findings and results are: on the side of empirically motivated improvements based on more thoughtful approaches, it is possible to explore carefully articulated chains of transfer with curriculum and meta-learning, and also hybrid deep learning and symbolic approaches aimed to solve transfer learning catastrophic forgetting among other issues.