Target Inference in Argument Conclusion Generation (2020)

Abstract: In argumentation, people state premises to reason towards a conclusion. The conclusion conveys a stance towards some target, such as a concept or statement. Often, the conclusion remains implicit, though, since it is self-evident in a discussion or left out for rhetorical reasons. However, the conclusion is key to understanding an argument and, hence, to any application that processes argumentation. We thus study the question to what extent an argument’s conclusion can be reconstructed from its premises. In particular, we argue here that a decisive step is to infer a conclusion’s target, and we hypothesize that this target is related to the premises’ targets. We develop two complementary target inference approaches: one ranks premise targets and selects the top-ranked target as the conclusion target, the other finds a new conclusion target in a learned embedding space using a triplet neural network. Our evaluation on corpora from two domains indicates that a hybrid of both approaches is best, outperforming several strong baselines. According to human annotators, we infer a reasonably adequate conclusion target in 89% of the cases.

## introduction the conclusion (or claim) of a natural language argument conveys a pro or con stance towards some target, such as a controversial concept or statement @xcite . it is inferred from a set of premises. conclusions are key to understanding arguments, and hence, critical for any downstream application that processes argumentation. the task of identifying conclusions has been studied intensively in the context of argument mining @xcite and automatic essay assessment @xcite . in genres other than essays, however, conclusions often remain implicit, since they are clear from the context of a discussion @xcite or hidden on pur- raising the school leaving age promotes equal opportunities. parents who left school at a young age are also more likely to have children who leave school early. making sure that everyone gets the same amount of time at school promotes equality. forcing all children to stay in school longer will help break this cycle of disadvantage. parents who left school at a young age making sure [...] same amount of time at school. forcing all children to stay in school longer con pro pro premise target identification (1) conclusion target inference (3) conclusion text generation raising the school leaving age pro stance classification (2) stance inference tackled in this paper pose for rhetorical reasons, as is often the case in news editorials @xcite . this alters the task entirely to become a synthesis task: given an argument's premises, generate its conclusion. as detailed in section 2, research on argumentation synthesis is still limited. existing approaches focus on generating single claims @xcite , new arguments @xcite , counterarguments @xcite , or argumentative texts @xcite . closer to conclusion generation, @xcite summarized the main points of online debates, and @xcite worked on identifying the main claim of an argument through abstractive summarization. to our knowledge, however, no approach so far reconstructs an argument's conclusion from its premises. in general, we consider the synthesis task outlined above. conceptually, we decompose this task into three steps, as depicted in figure 1: (1) inferring the conclusion's target from the premises, (2) inferring the conclusion's stance, and (3) generating the conclusion's text with the inferred stance and the inferred target. in this paper, we focus on the first step by proposing two computational approaches for conclusion target inference. as sketched in figure 1 , we hypothesize that the conclusion target is related to the targets of the argument's premises. to obtain premise targets, we train a state-of-the-art sequence labeling model @xcite on target-annotated claims @xcite . since the exact relation of premise and conclusion targets is unknown, we develop two complementary inference approaches: one approach ranks premise targets based on their likelihood of being a conclusion target. the other one employs a triplet neural network @xcite that generates a conclusion target embedding from the premise targets in a learned target embedding space. a unique facet of the latter is the integration of the network with a knowledge base of targets (built from any training set), namely, the approach returns the known target whose embedding is closest to the generated embedding. we compare the approaches against several baselines, including an existing sequence-to-sequence model for argument summarization (with and without encoded premise targets). for evaluation purposes, we study argument corpora from two genres where the correct conclusions are given: student essays @xcite and debate portals @xcite . on these corpora, we empirically test how often an inferred target matches the target found in the ground-truth conclusion. moreover, we let human annotators manually check the adequacy of the inferred targets. in our experiments, both approaches consistently outperform sequence-to-sequence generation, justifying the explicit modeling of the relation between premise and conclusion targets. according to manual evaluation, a combined version of the two approaches infers an at least somewhat adequate target in 89%, and a fully adequate target in 55% of the cases, indicating the practical applicability of our target inference in conclusion generation. in summary, the contributions of this paper are: foot_0 ## related work arguments have been modeled in different ways, focusing on the roles of their components @xcite , their inference scheme @xcite , or the interplay between their pro and con components @xcite . on an abstract level, the models all share that they consider an argument as a conclusion (in terms of a claim) and a set of premises (reasons to support or object the claim). we restrict our view to this abstract model here. even though this paper is about inferring conclusion targets, our ultimate goal is to reconstruct the whole conclusion of an argument. computational approaches to identify conclusions in a text have been pioneered research on student essay assessment @xcite . @xcite show the importance of essay conclusions in applications, whereas @xcite specifically target an essay's overall conclusion, i.e., its thesis (also known as major, main, or central claim). given the importance of theses, we dedicate one experiment particularly targeting them below. the classification of argument components (as theses, conclusions, premises, etc.) is a core task in argument mining @xcite and has been approached for different genres @xcite . as @xcite observe, though, real-world arguments often leave the conclusion implicit, particularly where it is clear in the context of a discussion. in genres such as news editorials, conclusions may even be left out on purpose, in order to persuade readers in a "hidden" manner @xcite . if an implicit conclusion is needed, it hence needs to be synthesized. argumentation synthesis research is on the rise. early argument generation approaches relied on rule-based discourse planning techniques @xcite . later, @xcite generalized target-stance relations from claims and used them to automatically create new arguments. the relations were curated manually, though. an approach that finds the best conclusion for generation among a set of candidate claims was presented by @xcite . @xcite built upon this approach to phrase texts with multiple arguments. others recycled targets and predicates of claims in new claims @xcite , generated arguments with specific inference schemes for user-defined content @xcite , modeled rhetorical aspects in synthesis @xcite , and composed arguments that follow a strategy @xcite . all these methods synthesize new argumentative content. in contrast, we aim for the missing components of given arguments. as such, our task resembles enthymeme reconstruction. an enthymeme is an implicit premise, usually the warrant (or major premise) that clarifies how a conclusion is inferred from the given premises @xcite . motivated by the importance of finding the thesis, @xcite study how to identify such enthymemes given the other components. similarly, @xcite present the task of identifying the correct warrant from two options, and @xcite aim to generate the premise connecting an aspect-related opinion to an overall opinion. instead of missing premises, we aim to synthesize (parts of) an argument's conclusion. for any text generation task, a candidate technique is sequence-to-sequence models @xcite . relevant in the given context, @xcite used such models to generate counterarguments, and @xcite extended this approach by planning and retrieval mechanisms. with a comparable intention, @xcite modified the bias of news headlines from right-toleft or vice versa. closest to our work is the approach of @xcite whose sequenceto-sequence model generates summaries for opinionated and argumentative text. like us, the authors face the problem of varying numbers of input components, and tackle this using an importancebased sampling method. for their evaluation, they crawled arguments from idebate.org. we use this dataset in our experiments. unfortunately, their manual evaluation considers opinionated text only, leaving the semantic adequacy of the generated argument summaries unclear. the exact connection to summarization is unclear, which is why we include an approximation of the model of @xcite as a baseline in our experiments. general research on summarization is manifold and beyond the scope of this work. for a survey, we refer the reader to @xcite . in recent work, we summarize the core of an argument to be used as a snippet in the context of argument search by a two-sentence extract @xcite and @xcite create abstractive summaries of the main points in a debate. we hypothesize a dependency between the target and stance of a conclusion and those of the premises. at a high level, this resembles the work of @xcite where aspects and sentiments are modeled for the extractive summarization of opinions. we focus on the inference of conclusion targets in this work. our approach builds upon ideas of @xcite , who classify the stance of premises to a conclusion. to do so, they identify and relate targets in these components, and model stance with sentiment. we do not explicitly tackle stance inference here, because our focus is a conclusion's target. to identify premise targets, we first train a state-of-the art sequence tagger using contextualized word embeddings @xcite on the corpus of @xcite . from these premise targets, we then infer the conclusion target, as explained below. ## data before discussing our target inference approach in section 4, this section briefly introduces the datasets that we use in our analyses and experiments. to allow for evaluating the given task, the conclusion is always given in these datasets. ## approach we now present our approach to infer the target of an argument's conclusion from its premises. based on a premise target identifier, it employs two complementary sub-approaches: one ranks premise targets by their potential representativeness for the (later unknown) conclusion, and then picks the topranked premise target. the other predicts candidate embeddings for the conclusion target from the topranked premise targets, and then picks the conclusion target from a knowledge base of targets whose embedding is most similar to those embeddings. ## automatic evaluation in this section, we report on empirical experiments, along with their results, performed to evaluate our approaches to target inference. ## manual evaluation to assess the actual quality of the inferred conclusion targets, we manually evaluated our approaches (optimistic and pessimistic scenario) and the baseline b2 (seq2seq (w/ premise targets)) in comparison to the ground-truth targets using amazon mechanical turk. for this, we sampled 100 random instances from the idebate test set. in a single task, an argument's premises were given along with the conclusion target of either approach. annotators had to judge the adequacy of the target for the given premises as fully, somewhat, or not adequate. each instance was judged by five annotators. no one judged multiple targets for the same argument. 8 ## conclusion an argument's conclusion comprises its stance towards the target it discusses. still, the conclusion is often left implicit in real life, because it is clear for humans or hidden for rhetorical reasons. we have conceptualized the task of reconstructing the conclusion from the argument's premises as (1) inferring the conclusion's target, (2) inferring its stance, and (3) phrasing its actual text. then we have focused on the first step in which we infer the conclusion target given a set of premises. hypothesizing that the conclusion target depends on the premise targets, we have developed two new and complementary target inference approaches: premise targets (ranking) returns the premise target that is most likely adequate for the conclusion, while target embedding (learning) generates a conclusion target embedding from the premises and matches it against a target knowledge base. on three datasets from two domains (debate portals and student essays), our approaches outperform several baselines, including a state-of-the-art neural sequence-to-sequence summarizer. the latter also benefits from modeling premise targets, additionally supporting our hypothesis. in terms of bleu, meteor, and accuracy, target embedding (learning) and a hybrid of both approaches turned out particularly strong, whereas premise targets (ranking) was best in a manual evaluation. overall, we manage to infer an at least somewhat adequate conclusion target in 89% of all cases, indicating the practical applicability of our approaches. combining target inference with stance classification in future work, we can already generate basic conclusions, say, "raising the school leaving age is good". a more elaborate phrasing approach may take over context information from the premises.