Does External Knowledge Help Explainable Natural Language Inference? Automatic Evaluation vs. Human Ratings (2021)

Abstract: Natural language inference (NLI) requires models to learn and apply commonsense knowledge. These reasoning abilities are particularly important for explainable NLI systems that generate a natural language explanation in addition to their label prediction. The integration of external knowledge has been shown to improve NLI systems, here we investigate whether it can also improve their explanation capabilities. For this, we investigate different sources of external knowledge and evaluate the performance of our models on in-domain data as well as on special transfer datasets that are designed to assess fine-grained reasoning capabilities. We find that different sources of knowledge have a different effect on reasoning abilities, for example, implicit knowledge stored in language models can hinder reasoning on numbers and negations. Finally, we conduct the largest and most fine-grained explainable NLI crowdsourcing study to date. It reveals that even large differences in automatic performance scores do neither reflect in human ratings of label, explanation, commonsense nor grammar correctness.

## introduction natural language inference (nli) is closely related to real-world applications, such as fact checking. given two sentences (premise and hypothesis), the task is to decide whether (a) the first sentence entails the second sentence, (b) the two sentences contradict each other or (c) they have a neutral relation. figure 1 shows an example for an entailment relation. solving the task requires models to not only reason over the provided information but also to link it with commonsense knowledge. as for other natural language tasks, state-of-theart nli systems rely on deep neural architectures which do not easily expose their inner workings. however, following a model's reasoning process is valuable to machine learning engineers as well as an animal is outside in the cold weather, playing with a plastic toy. a dog jumping for a frisbee in the snow. a dog is a form of animal, for there to be snow it must be cold weather outside, and "jumping for a frisbee'' is a rephrasing of "playing with a plastic toy.'' entailment contradiction neutral ? input premise hypothesis label explanation model output figure 1 : nli example instance from e-snli @xcite . the system needs to include commonsense knowledge, such as "@xmath0. especially recently, pre-trained language models, such as bert @xcite or gpt @xcite , became popular. it was shown that they are able to learn and store commonsense knowledge implicitly @xcite . as a result, an open question is: how effective is the implicit commonsense knowledge of language models compared to symbolic sources of knowledge, such as knowledge base triplets? (rq2) to evaluate nli models, mainly automatic measures, such as accuracy, are used. however, model weaknesses can stay unnoticed using automatic scores alone. moreover, @xcite showed that automatic scores are not necessarily correlated to human-perceived model quality. thus, human evaluation is a crucial step in the development of user-centered explainable ai systems. therefore, we ask the question: how do humans perceive explanation quality of state-of-the-art natural language inference models? (r3) in this paper, we investigate the three previously mentioned research questions r1-r3. to answer them, we analyze the impact of external knowledge from multiple sources, such as knowledge graphs, embeddings and language models and propose novel architectures to include and combine them into explainable nli systems. further, we conduct an extensive automatic analysis as well as a user study. to the best of our knowledge, our study exceeds previous human evaluations of explainable nli models regarding the number of participants as well as the variety of rated explanation criteria. for r1, we find that the positive effect of external knowledge on label accuracy in the standard nli setting can also be observed in the explainable nli setting and external knowledge can improve the bleu scores of the generated explanations. in regard of r2, we observe that pre-trained language models are the most promising source of commonsense knowledge but at the same time identify weaknesses with respect to negations and numerical reasoning abilities which, however, can be mitigated through combination with additional knowledge sources. despite the improvements in accuracy, bleu or bleurt scores, our user study shows for r3 that these do not reflect in human ratings of explanation correctness, commonsense inclusion or grammar and label correctness. our results urge caution to solely rely on automatic scores for explainability. therefore, we expect our paper to motivate the development of dedicated evaluation tasks and scores and further emphasize the importance of the user within explain-able ai. to facilitate future work, we release our model's predictions as well as the crowdsourced human ratings. foot_0 2 related work ## method in the following, we describe our base model and then present the models we analyze. ## automatic evaluation first, we evaluate the discussed knowledgeenhanced models with respect to commonly used scores on e-snli and a stress test evaluation. in addition to our constructed models, we also include pred-expl @xcite , which is basically our vanilla baseline without cross-attention but with glove embeddings instead of fine-tuned bert embeddings. further, we include two recent models proposed for e-snli: nile:post-hoc, which is the highest performing model from @xcite of the t5 language model @xcite . we train all non-lm models with five random seeds and report scores of the median model based on label accuracy. ## human evaluation while automatic scores, such as bleu, provide a valuable starting point for evaluating explanations, they fall short in capturing the model's real explanation capabilities. we, therefore, conduct a large-scale crowdsourcing study to complement our automatic evaluations on e-snli and the stress tests. following related work @xcite , we assess explanation quality based on ratings from crowdworkers on mechanical turk. while previ- ## conclusion in this paper, we addressed three research questions: whether integrating external knowledge can improve explainability for nli, how effective knowledge implicitly stored in language models is for reasoning, and how humans perceive explanation quality of state-of-the-art natural language inference models. to answer these questions, we proposed different methods of integrating various knowledge sources into deep learning models. we found that fine-tuned language models reach the highest performance on e-snli as well as the highest average accuracy within the nli stress test evaluation. however, their performance can break down on numerical reasoning and negations. in addition to automatic evaluation, we conducted a large-scale human crowdsourcing evaluation and found that high differences in accuracy, bleu or bleurt scores do not reflect in significant differences in human ratings of explanation correctness, commonsense inclusion, grammar or label correctness. this highlights an alarming disconnect between automatic evaluation scores and human ratings, that puts the real-world utility of recent model improvements into question and requires to re-think automatic evaluation across the field of explainable ai.