Case-Based Abductive Natural Language Inference (2022)

Abstract: Most of the contemporary approaches for multi-hop Natural Language Inference (NLI) construct explanations considering each test case in isolation. However, this paradigm is known to suffer from semantic drift, a phenomenon that causes the construction of spurious explanations leading to wrong conclusions. In contrast, this paper proposes an abductive framework for multi-hop NLI exploring the retrieve-reuse-refine paradigm in Case-Based Reasoning (CBR). Specifically, we present Case-Based Abductive Natural Language Inference (CB-ANLI), a model that addresses unseen inference problems by analogical transfer of prior explanations from similar examples. We empirically evaluate the abductive framework on commonsense and scientific question answering tasks, demonstrating that CB-ANLI can be effectively integrated with sparse and dense pre-trained encoders to improve multi-hop inference, or adopted as an evidence retriever for Transformers. Moreover, an empirical analysis of semantic drift reveals that the CBR paradigm boosts the quality of the most challenging explanations, a feature that has a direct impact on robustness and accuracy in downstream inference tasks.

## introduction multi-hop inference is the task of composing two or more pieces of evidence from external knowledge resources to address a particular reasoning problem @xcite . in the context of natural language inference (nli), this task is often used to develop and evaluate explanation-based systems, capable of performing transparent multistep reasoning with natural language @xcite @xcite . while multi-hop inference has been largely explored for extractive problems such as open-domain question answering @xcite , increasing attention is being dedicated to the abstractive setting, where the models are required to compose long chains of facts expressing abstract commonsense and scientific knowledge @xcite . in this setting, multi-hop inference is often framed as an abductive natural language inference (anli) problem @xcite , where, for a given set of alternative hypotheses @xmath0, h 2 , . . . , h n }, the goal is to construct an explanation for each h @xmath1. existing approaches address abductive inference considering each test hypothesis in isolation, employing iterative and path-based methods @xcite or explicit constraints to guide the generation of a plausible explanation graph supporting the correct answer @xcite . however, this paradigm poses several challenges in the abstractive setting as: consequently, existing approaches often suffer from a phenomenon known as semantic drift @xcite -i.e., the tendency of composing incorrect reasoning chains leading to wrong conclusions as the number of required inference steps increases. the example in figure 1 illustrates some of these challenges. in contrast with the dominant paradigm, we propose to integrate abductive natural language inference in a case-based reasoning (cbr) framework @xcite . cbr systems operate under the hypothesis that similar problems require similar solutions, addressing new cases via analogical transfer from previous cases solved in the past. specifically, the case-based reasoning framework employs a retrieve-reuserefine paradigm to model inference over unseen problems @xcite . in the context of multi-hop inference, we hypothesise that the adoption of a case-based reasoning framework can help tackle some of the challenges involved in the abstractive setting since: to investigate these hypotheses, we present a case-based abductive nli (cb-anli) model that retrieves and adapts natural language explanations from training examples to construct new explanations for unseen cases and address downstream inference problems. specifically, this paper provides the following main contributions: ## case-based abductive nli for a given set of alternative natural language hypotheses @xmath2, h 2 , . . . , h n }, the goal of abductive nli is to construct an explanation for each h @xmath3. given an hypothesis h i (e.g., "two sticks getting warm when rubbed together is an example of a force producing heat"), we construct an explanation justifying h i by extracting and composing inference chains between multiple explanatory facts retrieved from an external corpus. to generate and score an explanation for h i , we adopt a case-based reasoning (cbr) paradigm composed of three major phases, retrieve-reuserefine, which can be summarised as follows (see fig. 2 ): 1. retrieve: in the retrieve phase, we employ a sentence encoding mechanism to search over two distinct embedding spaces. a first embedding space (facts embeddings) is adopted to retrieve a set of candidate explanatory sentences for the hypothesis. a second embedding space (cases embeddings) is used to retrieve similar cases solved in the past whose facts embeddings e(f i ) multi-hop inference central explanation abstraction/grounding h: two sticks getting warm when rubbed together is an example of a force producing heat. cases embeddings e(h i ) friction is a kind of force friction causes the temperature of an object to increase friction is a force that produces energy as heat friction is a kind of force magnetic attraction pulls two objects together friction causes the temperature of an object to increase encoder (1) retrieve (2) reuse ... friction is a kind of force friction causes the temperature of an object to increase h: two sticks getting warm when rubbed together is an example of a force producing heat. (3) refine a stick is a kind of object candidate facts k-nn e(h) explanatory power pw(h, fi) sim(h, fi) cosine similarity explanatory relevance er(h, fi) ## abductive inference to derive the final explanation for a given hypothesis while conditioning on previously solved cases, we sum the explanatory relevance computed during the reuse phase with the semantic plausibility computed during the refine phase, pruning the graph considering only the top n central explanatory sentences and their linked grounding nodes (fig. 2 .3). given a set of alternative hypotheses @xmath4, . . . , h n }, we adopt the model for abductive inference by generating an explanation for each hypothesis and selecting as an answer the one supported by the best explanation. to this end, we assign a score to each hypothesis h i in h equal to the sum of the scores of the central facts included in the explanation for h i . ## related work multi-hop inference for abstractive tasks is challenging as the general structure of the explanations cannot be derived from the surface form of the nli problem. previous work has demonstrated that models in this setting are affected by semantic drift -i.e., the construction of spurious explanations leading to wrong conclusions @xcite . existing approaches frame multi-hop inference as the problem of building an optimal graph, conditioned on a set of semantic constraints @xcite @xcite @xcite , or adopting iterative methods, using sparse or dense encoding mechanisms @xcite @xcite . our model is related to previous work that leverages annotated explanations to reduce semantic drift @xcite . however, this line of work is still limited to explanation regeneration @xcite @xcite @xcite , while the applicability of these re-sources for downstream multi-hop nli problems is yet to be explored. in this paper, we move a step forward, exploring how the impact of annotated explanations on semantic drift translates in improved downstream performance. our approach is related to previous work on case-based reasoning (cbr) @xcite @xcite . similar to the retrieve-reuse-refine paradigm adopted in cbr systems, we employ encoding mechanisms to retrieve explanations for cases solved in the past, and adapt them in the solution of new problems. recent work in nlp investigates the use of a similar paradigm via k-nn retrieval on training examples. @xcite adopt k-nn search to retrieve similar training examples and improve pretrained language models and machine translation without additional training. similarly, @xcite propose a cbr framework for knowledge base reasoning, while kassner and schütze (2020) reuse similar cases to improve bert @xcite on cloze-style qa. to the best of our knowledge, this is the first application of case-based reasoning for explanation-based multi-hop natural language inference (nli). the work presented in this paper is related to hybrid neuro-symbolic approaches for inference with natural language @xcite @xcite @xcite @xcite . in this context, most of the existing approaches combine neural models with symbolic programs or reasoning modules. for instance, @xcite propose the adoption of a neural module network @xcite for multi-hop question answering by designing four atomic neural modules (find, relocate, compare, noop). @xcite propose a methodology to perform multi-hop inference using a prolog prover via the integration of sentence encoders and a weak unification mechanism. differently from the methodology discussed in this paper, previous neuro-symbolic approaches have been generally applied to extractive tasks, where the inference steps (and, therefore, the explanation's structure) can be derived from a direct decomposition of the questions @xcite . ## limitations the adopted model of explanatory power relies on the availability of human-annotated explanations with specific features (e.g., explanatory facts reused across different training instances). however, these resources might not be available in real-world scenarios and are generally costly to develop. moreover, since the explanatory power model relies on similarity measures and indicator functions, the model's ability to generalise might be sensitive to the incompleteness of the knowledge bases and the availability of representative explanations. we believe these limitations can be potentially alleviated by exploring the role of more abstract sentence representations within the cbr paradigm @xcite . in the current implementation of cb-anli, the refine phase adopts specific assumptions to model the abstraction process required for explanation generation. this process, in fact, is performed by assuming that abstraction at the concept level translates in a correct mapping between hypotheses and central explanatory sentences. however, contextual linguistic elements can still affect the overall meaning of the specific concept being abstracted, inducing the inclusion of spurious links between sentences. while contextual elements are considered during the precedent phases through the use of contextualised embeddings and similar cases, additional work is still required to guarantee the correctness of the abstraction process. ## conclusion this paper presented cb-anli, a model that integrates multi-hop and case-based reasoning (cbr) in a unified framework. we demonstrated the efficacy of the framework in complex abstractive and multi-hop nli tasks. we believe this work can open new lines of research on hybrid neurosymbolic models for explanation-based nli, and plan to investigate the efficacy of the framework on architectures that adopt richer symbolic representations in combination with neural models, further exploring the role of abstraction in case-based reasoning for improving robustness, generalisation, and explainability in nli.