Faithful Knowledge Graph Explanations in Commonsense Question Answering (2022)

Abstract: Knowledge graphs are commonly used as sources of information in commonsense question answering, and can also be used to express explanations for the model’s answer choice. A common way of incorporating facts from the graph is to encode them separately from the question, and then combine the two representations to select an answer. In this paper, we argue that highly faithful graph-based explanations cannot be extracted from existing models of this type. Such explanations will not include reasoning done by the transformer encoding the question, so will be incomplete. We confirm this theory with a novel proxy measure for faithfulness and propose two architecture changes to address the problem. Our findings suggest a path forward for developing architectures for faithful graph-based explanations.

## introduction in commonsense question answering, many approaches incorporate knowledge from external resources in addition to using large pre-trained language models. most often this is done to improve performance, however explanations can also be expressed by highlighting a subset of this information. making the model output the facts used to answer a particular question can increase trustworthiness and help with debugging. for this kind of explanation to be helpful, it must faithfully represent the model's reasoning; that is, the explanation must accurately reflect the facts used to answer the question. faithfulness is independent of whether or not the explanation is a reasonable justification of why an answer was chosen @xcite . we refer to how convincing an explanation is for an answer as plausibility. 1 it is useful to separate the two concepts as a faithful representation of the model's reasoning 1 terminology for these concepts varies, e.g. faithfulness is also known as trustworthiness. see @xcite for discussion. ## model architecture we focus on a class of models that combine a text encoder and a graph encoder for commonsense question answering (figure 1 ). explanations are naturally expressed as a subset of edges in the input graph (figure 2 ) and could be used for understanding which facts the model used to choose its answer. these explanations are easy to interpret. alternative approaches to incorporating external knowledge exist, like serialising knowledge graph triples and concatenating them to the text encoder input. a common approach to explaining predictions of text encoders is to calculate token-level attributions @xcite , which is too specific in this scenario where facts are the desired output. we test the behaviour of two recent models: mhgrn @xcite and qa-gnn @xcite . the high-level operation of both models is comparable and representative of others that have the same architecture. an embedding of the question context -a question with an answer choice -is obtained from the text encoder, which in both cases is roberta-large @xcite . the context is also used to extract a subgraph of up to 200 nodes (sometimes called a schema graph) from a larger knowledge graph. the graph encoder creates an embedding for the knowledge in this subgraph; a message passing graph neural network (gnn) @xcite is used in both cases. a key difference between the two models is the structure of the gnn. in each layer of mhgrn, multiple paths through the subgraph are found that end at each node, which are encoded and then pooled to form the new node embedding. qa-gnn instead updates nodes at each layer by aggregating messages from direct neighbours. after the final layer in both models, the node embeddings are combined via attentional pooling with respect to the text embedding. the models also use the text embedding at each layer in different ways. in mhgrn, it is used to calculate the relevance of each path when creating the new embedding for each node. in qa-gnn, a pseudo-node initialised with this embedding is added to the graph, allowing it to participate in message passing with the other nodes. to score each answer choice, the text embedding of the question context is concatenated with the embedding of that answer's extracted subgraph. an mlp is then used to produce a score. ## explanation faithfulness we claim that the class of models described in §2 are intrinsically unable to provide graph-structured explanations that are highly faithful to the full model. our desire for these explanations is that they are the collection of facts used by the model to complete a natural language understanding task. the more faithful these explanations are, the more useful they will be for developers to understand model behaviour. following jacovi and goldberg (2020), we do not consider faithfulness to be a binary quality. instead, we argue that different architectural choices either increase or decrease how faithful extracted explanations can be. explanations have low faithfulness because this class of model uses both text and graph embeddings to choose between answer candidates, but explanations are only extracted from the graph encoder. these explanations cannot give a full characterisation of the facts used to answer the question, as the text encoder is likely to have also contributed relevant information. when the text encoder is a pre-trained transformer it is infeasible that it does not contribute in this way, as they can achieve reasonable performance on question answering @xcite . graph-structured explanations are therefore necessarily incomplete. ## results ## discussion we have demonstrated that in mhgrn and qa-gnn, as the ability of the pre-trained text encoder to learn a task is curtailed, accuracy significantly decreases. this result suggests that it was the text encoder that most contributed to performance. our finding is reinforced by the fact that shuffling subgraphs across questions has no significant impact on performance. explanations extracted from the graph encoder will therefore be unfaithful to the overall operation of the model, as they will not capture the substantial reasoning done in the text encoder. our two successive model changes make explanations more faithful by increasing the proportion of reasoning done in the graph encoder. removing the direct use of the text embedding in the final classification mlp is crucial for retrieving faithful explanations. for qa-gnn it is also necessary to freeze the text encoder weights; this is not the case for mhgrn due to how it uses the text embedding. our results generalise from the two models we examined. @xcite and the knowledgeable reader @xcite precede both models and work most similarly to mh-grn. there, the question context embedding is also used to calculate attention weights for embeddings of different facts. greaselm @xcite instead builds on qa-gnn, and further integrates the text encoder embeddings with the gnn. although the authors do not discuss explanations with this model, it is an example of where further integration of the two encoders would harm faithfulness. future work on faithfully interpretable models may structure their use of the text encoder in a similar way to mhgrn, as this more easily leads to faithful explanations. such work might examine how to better train the text encoder to weight the relevancy of facts. although recent work successfully achieves higher accuracy via tighter coupling of the graph and text encoders @xcite , models that maintain separation are likely to yield faithful explanations more easily. how to do this while maintaining reasonable accuracy remains an open question, although we argue that the ability to produce faithful explanations is particularly valuable. one cause for the performance drop might be the quality of the extracted subgraphs: our results suggest that they are more appropriate for commonsenseqa than openbookqa. faithful explanations can be used to better understand model behaviour to help improve them. an implausible but faithful explanation is a signal to a model developer that something may need to be changed in the model. in this way, explanation quality can also be used to judge how good a model is, alongside accuracy. however, the plausibility of an explanation is only a useful property if the explanation is faithful.