Inconsistencies Detection in Bipolar Entailment Graphs
Authors: Elena Cabrio, Serena Villata (CNRS, University of Nice Sophia Antipolis, 2015)
Topic: Lexical and Semantic Resources and Analysis

Abstract
In the latest years, a number of real world applications have underlined the need to move from Textual Entailment (TE) pairs to TE graphs where pairs are no more independent. Moving from single pairs to a graph has the advantage of providing an overall view of the issue discussed in the text, but this may lead to possible inconsistencies due to the combination of the TE pairs into a unique graph. In this paper, we adopt argumentation theory to support human annotators in detecting the possible sources of inconsistencies.

Introduction
A Textual Entailment (TE) system (Dagan et al., 2009) automatically assigns to independent pairs of two textual fragments either an entailment or a contradiction relation. However, in some real world scenarios like analyzing costumer reviews about a service or product, these pairs cannot be considered as independent. For instance, all the reviews about a certain service need to be collected into a single graph, to understand the overall problems/merits of the service. The combination of TE pairs into a unique graph may generate inconsistencies due to the wrong relation assignment by the TE system, which could not have been identified if TE pairs were considered independently. The detection of such inconsistencies is usually left to human annotators, which later correct them. The need of processing such graphs to support annotators is therefore of crucial importance, particularly when dealing with big amounts of data. Our research question is How to support annotators in detecting inconsistencies in TE graphs? The term entailment graph has been introduced by (Berant et al., 2010) as a structure to model entailment relations between propositional templates. Differently, in this paper we consider bipolar entailment graphs (BEGs), where two kinds of edges are considered, i.e., entailment and contradiction, to reason over the graph consistency. We answer the research question by adopting abstract argumentation theory (Dung, 1995), a reasoning framework used to detect and solve inconsistencies in the so-called argumentation graphs, where nodes are called arguments, and edges represent a conflict relation. Argumentation semantics allows to compute consistent sets of arguments, given the conflicts among them. We define the BEGincs (BEG-Inconsistencies) framework, which translates a BEG into an argumentation graph. It then provides to the annotators sets of arguments, following argumentation semantics, that are supposed to be consistent. If it is not the case, the TE system wrongly assigned some relations. Moving from single pairs to an overall graph allows for the detection of inconsistencies otherwise undiscovered. BEGincs does not identify the precise relation causing the inconsistency, but providing annotators with the consistent arguments sets, they are supported in narrowing the causes of inconsistency.

Conclusion
We have presented BEGincs, a new formal framework that, translating a BEG into an argumentation graph, returns inconsistent set of arguments, if a wrong relation assignment by the TE system occurred. These inconsistent arguments sets are then used by annotators to detect the presence of a wrong assignment, and if so, to narrow the set of possibly erroneous relations. If no mistakes are produced in relation assignment, by definition BEGincs semantics return consistent arguments sets. Assuming that in several real world scenarios TE pairs are interconnected, we ask to the NLP community to contribute in the effort of building suitable resources. In BEGincs, we plan to verify and ensure transitivity of BEGs.