End-to-end Argument Mining with Cross-corpora Multi-task Learning (2022)

Abstract: Mining an argument structure from text is an important step for tasks such as argument search and summarization. While studies on argument(ation) mining have proposed promising neural network models, they usually suffer from a shortage of training data. To address this issue, we expand the training data with various auxiliary argument mining corpora and propose an end-to-end cross-corpus training method called Multi-Task Argument Mining (MT-AM). To evaluate our approach, we conducted experiments for the main argument mining tasks on several well-established argument mining corpora. The results demonstrate that MT-AM generally outperformed the models trained on a single corpus. Also, the smaller the target corpus was, the better the MT-AM performed. Our extensive analyses suggest that the improvement of MT-AM depends on several factors of transferability among auxiliary and target corpora.

## introduction argument(ation) mining (am), the task of identifying an argument structure from text, has been gaining attention in recent years @xcite . also known as argument(ation) structure parsing @xcite , am typically identifies argumentative component spans, classifies the type of the components, and classifies the relations between the components. in span identification, we discriminate argumentative text units (i.e., spans) from non-argumentative ones in a given text. in component classification, we classify the spans into argumentative labels such as claim and premise. relation classification detects argumentative links between the components and classifies each link into a relation label such as support and attack. researchers have been utilizing various datasets (am corpora) to develop am models. these corpora are designed on the basis of different theoretical frameworks and conceptualizations @xcite . for example, @xcite developed a corpus called the microtext corpus (mtc) on the basis of freeman's theory of the macro-structure of arguments @xcite to capture hypothetical dialectical exchanges. @xcite created the argument-annotated essays corpus (aaec), in which a connected tree is used to represent the structure of each argument in a paragraph. a notable feature of these am corpora is that, due to the lack of a clear underlying framework on which to base the annotation, new annotated corpora are likely to diverge in terms of the span, type of components, and type of relations. in addition, developing an am corpus is expensive because of the annotation cost @xcite . for example, @xcite conducted multiple calibration phases when training annotators to improve the low inter-annotator agreement, which drove up costs. this suggests that it would be infeasible to create a sufficiently large am corpora for a new domain, and the resultant lack of data inevitably degrades the performance of argument structure parsing. given the lack of a large annotated am corpus, multi-task learning on am corpora labeled with various annotation schemes may be a promising approach for improving parsing performance. few studies have investigated this except @xcite and @xcite . @xcite showed the effectiveness of multi-task learning for component classifications under low-resource settings, while @xcite showed how argumentative link prediction can be improved by multi-corpora training. however, to the best of our knowledge, no prior studies have conducted extensive analysis for all am tasks (i.e., span, component, and relation tasks) with an end-to-end model because it is challenging to take various differently annotated corpora into account for the modeling. through our observations, we hypothesize that parsing performance by multi-task learning can be improved by the potential transferability between am corpora. figure 1 (top) shows two different am corpora: cornell erulemaking corpus (cdcp; @xcite ) and aaec. the relations of cdcp (i.e., reason) and aaec (i.e., support) have different label names but share the concept of support connectivity, where both argumentative component pairs have a similar implicit relation. in figure 1 (bottom), cdcp and mtc use a similar type of span. although it is not a common definition, we refer to the span type as segment-based span, where the text is segmented into consecutive elementary discourse units or elementary units in which most of the units are clauses or sentences @xcite , and the text does not contain or rarely contains non-argumentative component parts. the potential ability of these transferable properties is referred to as transferability in this study. in this paper, we propose multi-task argument mining (mt-am), an end-to-end cross-corpus training model for am, to address the shortage of training corpora. we first describe single-corpus learning, namely, a single-task (st) model, and then detail the extension of the model (mt-am) for multi-task learning on various corpora. for the st model, we provide a simple but flexible architecture called the span-biaffine architecture that can handle different concepts of spans, types of component labels, and graph structures. mt-am involves two training stages: multi-task pre-training to capture transferable properties between auxiliary corpora and a target corpus, and target corpus fine-tuning to adjust the parameters for each target corpus. experiments using five am corpora showed that mt-am performed generally better than the st model. more importantly, the smaller the target corpus was, the better mt-am performed, suggesting the effectiveness of mt-am for remedying the lack of training data, which is likely in line with the previous work @xcite . to investigate transferability among the corpora, we also examined all the corpus pairs of an auxiliary corpus and target corpus and found that the choice of auxiliary corpus affects the parsing performance (figure 5 and finally, we further discuss transferability under low-resource settings and advocate three types of hypothetical transferability in am on the basis of the following observations. (i) data/training sufficiency: with a small number of data samples or training steps for the target corpus, mt-am detected larger number of components or relations than an st model (figure 7 ). (ii) annotation compatibility: partial compatibility of the annotation design between the auxiliary corpora and target corpus helped mt-am improve the parsing performance (figures 8 and 9 , and ## overview of am corpora we focus on five differently annotated corpora: aaec, a medical abstract corpus named abs-trct @xcite ), foot_0 cdcp, mtc, and the argument-annotated scidtb (aasd; accuosto and saggion, 2020). these corpora are useful for discussing differences and similarities in argument structure. aaec @xcite ) contains student essays. there are two types of data: essay-level and paragraph-level @xcite . a stance for a controversial theme is expressed by a majorclaim component as well as claim components, and premise components justify or refute the claims. attack and support labels are defined as relations. the span covers a statement, which can stand in isolation as a complete sentence, according to the aaec annotation guidelines @xcite . all components are annotated with minimum boundaries of a clause or sentence excluding so-called ''shell'' language such as on the other hand and hence it is always said that, as seen in figure 1 (bottom). thus, this corpus discriminates argumentative text units from non-argumentative ones, producing many non-argumentative component parts. this is different from the segment-based spans that do not contain or barely contain non-argumentative component parts. mtc @xcite is based on freeman's theory of the macro-structure of arguments @xcite and incorporates the ideas of toulmin @xcite into diagramming techniques. mtc introduces dialectical exchange between pro (proponent) and opp (opponent) components. relations include add, exa (example), reb (rebut), sup (support), and und (undercut). according to precedent studies, we pre-processed the add relations similarly to @xcite . mtc introduces a segment-based span where a segment usually contains a clause or sentence. cdcp @xcite consists of comments in which five types of components (fact, testimony, reference, value, and policy) and two types of supporting relations (reason and evidence) are annotated on the basis of the study by @xcite . the spans are segmented into elementary units with a proposition consisting of a sentence or a clause. cdcp also discriminates argumentative text units from non-argumentative ones, but we classify the span type of cdcp as segment-based because very few non-argumentative units exist in the corpus. we pre-processed this corpus using a similar approach to @xcite , where continuous spans are merged and transitive closures are processed. abstrct @xcite includes abstracts of randomized controlled trials (rcts) from the medline database. 3 majorclaim, claim, and evidence components, along with support, attack, and partial-attack relations, are annotated for various diseases (e.g., neoplasm, glaucoma, hepatitis, diabetes, and hypertension). similar to aaec, this corpus discriminates argumentative text units from non-argumentative ones, producing non-argumentative component parts. we used 350 training, 50 development, and 100 test neoplasm texts. aasd @xcite was created to address the lack of a scientific am corpus. the authors enriched annotations for a subset of scidtb @xcite by providing component types for proposal, assertion, result, observation, means, and description. support and attack relations are provided on the basis of the study by @xcite . aasd also includes detail, additional, and sequence relations. the spans are segmented into elementary discourse units, similar to mtc. ## experiments we compared the mt-am and st models (section 5.2), investigated the transferability (section 5.3), and conducted an in-depth analyses under low-resource settings (section 5.4). ## conclusion in this paper, we focused on argument mining (am) for handling a scarcity of resources and proposed an end-to-end multi-task am (mt-am) model for various corpora. experiments showed that the proposed mt-am generally outperformed single-task models and further improved the parsing performance under low-resource settings. our extensive analyses suggest that the advantage of mt-am is due to its three types of transferability: data/training sufficiency, annotation compatibility, and semantic compatibility. in future work, we will develop more accurate mt-am parsers on the basis of these transferability hypotheses. also, it has been suggested by a reviewer that using a trained parser to generate a silver corpus could be a means of extending the training data. future work should thus address the development of a silver corpus. we are also interested in multi-lingual training using the proposed system.