Effect Graph: Effect Relation Extraction for Explanation Generation (2023)

Abstract: Argumentation is an important means of communication. For describing especially arguments about consequences, the notion of effect relations has been introduced recently. We propose a method to extract effect relations from large text resources and apply it on encyclopedic and argumentative texts. By connecting the extracted relations, we generate a knowledge graph which we call effect graph. For evaluating the effect graph, we perform crowd and expert annotations and create a novel dataset. We demonstrate a possible use case of the effect graph by proposing a method for explaining arguments from consequences.

## introduction argumentation is a challenging task because its goal is to convince an audience. one broadly used type of arguments is the argument from consequences, which has been specifically addressed in recent literature @xcite @xcite . the premise of an argument from consequences states that if a is brought about, good or bad consequences will plausibly occur, which leads to the conclusion that a should or should not be brought about @xcite . the following statement is such an argument in favor of legal abortions: legal abortions protect women. ## related work our method to extract effect relations is most similar to the one proposed by @xcite . they extract effect relations in order to classify stances of arguments from consequences. just as ours, their extraction method is purely heuristic and relies on dependency parsing. the main differences we introduced are due to the following reasons: first, the method of @xcite relies on sentence-topic pairs to identify the effect relation's subject, instead of sentences only. second, it requires the effect relation's object to have a sentiment in order to calculate the stance which is not necessary for our task. because of this and the first reason, the subjects and objects which are derived by detecting patterns in the dependency parse are no longer controlled for by either linking to the topic or a sentiment lexicon, so we pose other restrictions on both of them. third, it is designed to extract an effect relation whenever possible, thus emphasizing recall, in order to enable the stance detection. in contrast, we want to rather focus on precision. alkhatib et al. (2020) 2020 ) divide the relation extraction task into several subtasks for which they train specific classifiers, with one exception: for identifying the effect relation's subject and object, they use the supervised openie model of @xcite . openie (open information extraction) is the task to extract relationships between entities from text. in contrast to conventional information extraction, in openie, the relationships are not predefined @xcite . however, openie can also be applied for relation extraction with domain specific relations by performing relation mapping @xcite . @xcite propose a supervised approach, in our case, we consider it sufficient to filter and map the relations using an effect lexicon. similarly to corro and gemulla (2013), @xcite , @xcite , we base our relation extraction on dependency parsing. in comparison to these works, however, our effect relation extraction approach is much less sophisticated. evolving around effect verbs specifically, we use only a small set of manually defined patterns, but are still able to gain comparable or even better results when compared to openie with an effect lexicon based relation mapping. similar to our effect graph which we build from effect relations, @xcite use @xcite for extracting relations in order to build an openie-based knowledge graph. before applying openie, they extract entities and link them to existing knowledge graphs. we experiment with both, using only enti-ties which we can link to wikipedia pages, or not requiring any linking. further, they annotate noun phrases (nps) and expand the extracted entities to encompass the complete np. similarly, in erex we only consider nps as entities. lastly, we want to mention another type of relations than effect relations, namely causal relations @xcite . other than in effect relations, a's effect on b, if they are in a causal relation, is clearly defined as a being the cause for b. @xcite , @xcite introduced the task of automatically extracting causal relations from text, and it has been a matter of research since then @xcite . also for causal relations, there exists research on using them for building a knowledge graph. heindorf et al. (2020) bootstrap dependency parse patterns to extract claimed causal relations from text. while their method to start with a small, very accurate seed set of patterns and to extend it consecutively is very appealing, we find it to be rather difficult to apply on our approach: their patterns involve very concrete words that all trigger causal relations while we chose to keep our patterns general in order to apply to a large set of different effect words. also like us, @xcite do not fact check their extractions, but emphasize that they merely collect claimed causal relations. ## effect graph generation our aim is to generate a graph where the nodes are entities such as global warming, co2 emissions, solar panel. the edges represent the effect relations and indicate either a negative or positive effect from the source to the target node, e.g., (solar panel) --→ (co2 emissions). we also store the concrete word indicating the effect. in the previous example, this could be for instance reduce or prevent. ## evaluation we evaluate the effect graph as follows: in section 4.1, we evaluate the effect relation extraction process using the subtasks defined by @xcite . then, we evaluate the extracted graph itself. in section 4.2, we compare the graph statistics. afterwards, we evaluate both precision (section 4.3) and recall (section 4.4). in this context, precision expresses the chance that a randomly selected edge of the graph is correct. we consider a statement to be correct if it is in accordance with the statement it was extracted from. recall on the other hand is meant to measure the chance that a given effect relation is contained in the graph. baselines for the evaluation of the extraction subtasks defined by al-khatib et al. (2020), we use their models as a baseline, denoted by al-khatib. for evaluating the effect graph as a whole, we build the effect graph as described in section 3.2, but using different extraction methods. we use the openie implementation which is part of stanford corenlp @xcite to extract subject-verb-object triples, applying a confidence threshold of 0.9. we accept such triples as effect relations where the verb is an effect word and the subject and object link to wikipedia pages. further, we use a version of erex where we do not require the subject and object to link to wikipedia, denoted by erex*. we expect this version to have a higher recall, but also more noise. ## explanation generation for generating explanations, we use the effect graph generated by erex. as outlined in the introductory section, we envision two different types of explanations which we will describe separately in the sections 5.1 and 5.2. afterwards, we introduce a measure to rank the potential explanations (5.3). ## conclusion we propose a method to extract effect relations from text and use it to build an effect graph. we further propose a method to use the effect graph as background knowledge for automatically generating structured explanations, for example for arguments from consequences. however, the effect graph's precision remains unclear while its recall is low. the latter issue might be addressed by either improving the extraction method or, to a certain degree, by running the method on larger text resources. the effect graph can be seen as a valuable resource on its own, as it can potentially be used to also address other tasks than explanation generation, like identifying (counter-) arguments for a specific topic or extending common sense knowledge graphs such as conceptnet @xcite .