Distinguish Before Answer: Generating Contrastive Explanation as Knowledge for Commonsense Question Answering (2023)

Abstract: Existing knowledge-enhanced methods have achieved remarkable results in certain Q&A tasks via obtaining diverse knowledge from different knowledge bases. However, limited by the properties of retrieved knowledge, they still have trouble benefiting from both the knowledge relevance and distinguishment simultaneously. To address the challenge, we propose CPACE , a C oncept-centric P rompt-b A sed C ontrastive E xplanation Generation model, which aims to convert obtained symbolic knowledge into the contrastive explanation for better distinguishing the differences among given candidates. Firstly, following previous works, we retrieve different types of symbolic knowledge with a concept-centric knowledge extraction module. After that, we generate corresponding contrastive explanation using acquired symbolic knowledge and prompt as guidance for better modeling the knowledge distinguishment and interpretability. Finally, we regard the generated contrastive explanation as external knowledge for downstream task enhancement. We conduct a series of experiments on three widely-used question-answering datasets: CSQA, QASC, and OBQA. Experimental results demonstrate that with the help of generated contrastive explanation, our CPACE model achieves new SOTA on CSQA (89.8% on the testing set, 0.9% higher than human performance), and gains impressive improvement on QASC and OBQA (4.2% and 3.5%, respectively).

## introduction in recent years, a large number of knowledge enhanced pre-trained language models (ke-plms) @xcite @xcite have been proposed to improve performance on a wide variety of nlp tasks @xcite . however, the implicit knowledge learned in plms can not be effectively used for these knowledge-driven qa tasks, especially in commonsense question answering. some works @xcite where can you find a magazine? a. doctor b. bookstore c.market d. train station e. mortuary 1. many other printed works can be found at a bookstore. you would find magazines along side. 2. doctor is not a place where you can find various printed works. 3. at market, magazines are not found. 4. at train station, there are no printed works so no magazines are found. 5. mortuary do not have magazines. to provide more distinguishing information, we can convert the acquired symbolic knowledge into contrastive explanation and use them for inference enhancement. 2020; @xcite @xcite explicitly retrieve knowledge from different knowledge sources, including wordnet @xcite , wikidata @xcite and conceptnet @xcite , then integrate them into downstream models for q&a. these methods enjoy the ability to utilize diverse knowledge, but inevitably introduce irrelevant or even noisy knowledge which will hurt the performance of model. other works consider plms as knowledge bases @xcite @xcite , which elicit potential knowledge via prompt from plms @xcite . these approaches can obtain relevant knowledge from plms, however the generated knowledge from plms is generally common and lacks specific and distinguishing information for enhancement. it is an important direction to explore "how to provide discriminative information to models to help them distinguish candidates before answering ?". inspired by previous studies @xcite @xcite , contrastive explanation can provide the information to explain "why a not b" for given input and prediction, which naturally has distinguishing property. as shown in figure 1 , given question, candidates and retrieved symbolic knowledge, we generate contrastive explanation for each candidate to provide discriminative information among them for inference enhancement. therefore, in this paper, we propose a conceptcentric prompt-based contrastive explanation generation (cpace) model, a distinguish before answer architecture, to obtain high-quality incorporated knowledge and distinguish the differences among candidates. specifically, our model consists of three parts, namely symbolic knowledge acquisition module, contrastive explanation generation module and explanation enhanced inference module. firstly, given the question and candidates, we use a trained concept recognizer to detect concepts appearing in input. then, with identified concepts, we extract diverse symbolic concept-centric knowledge from different types of knowledge bases. after that, we take the retrieved knowledge and a pre-defined explanation prompt as guidance for a fine-tuned generative pre-trained language model to generate contrastive explanation. the process of generation can filter irrelevant knowledge and convert selected symbolic knowledge into more specific and distinguishing information according to question and candidates. finally, we use the generated contrastive explanation as external knowledge for enhancement. it is worth noting that contrastive explanation, as the final form of incorporated knowledge, not only meet distinguishing property, but also makes it easier for human to understand and is better interpretable. the contributions are summarized as follows: here, we introduce the commonsense question answering task and the workflow of our cpace model. given a question stem q, the task is to find the correct answer a from a finite set of choices @xmath0, a 2 , ..., a n }. as shown in figure 2 , our approach can be divided into three steps. the first step is symbolic knowledge acquisition, we build a concept recognizer to identify a concept set c from the given question q and candidates a, then we take them as queries to extract diverse symbolic knowledge k symbolic from several knowledge bases kbs, as shown in section 3.1: k @xmath1, kbs) (2) the second step is contrastive explanation generation, where we generate contrastive explanation k ce with cpace generator, given q, a, k symbolic , c and explanation prompt p , as shown in section 3.2: the final step is explanation enhanced inference, we obtain the predicted answers a from a standard inference model enhanced with k ce , as presented in section 3.3: step1. symbolic knowledge acquisition step2. contrastive explanation generation step3. explanation enhanced inference kbs triples definitions concepts concept recognition module concept-centric symbolic knowledge question & candidates generative language model based generator explanation prompt contrastive explanation pre-trained language model based inference module predicted answer figure 2: architecture of our cpace model, which consists of 1) a symbolic knowledge acquisition module, 2) a contrastive explanation generation module and 3) an explanation enhanced inference module. ## conclusion in this paper, we propose a cpace model, which unify the retrieved knowledge into contrastive explanation, to provide more discriminative information for model enhancement. we firstly consider concept-centric knowledge and explanation prompt as guidance for contrastive explanation generation. our cpace model achieves a new sota on csqa leaderboard, which surprisingly surpasses human performance. in addition, we verify the effectiveness and generalization of cpace on other datasets. in the future, we will explore a unified contrastive explanation generation framework for nlp tasks.