SALKG: Learning From Knowledge Graph Explanations for Commonsense Reasoning

Abstract:
Augmenting pre-trained language models with knowledge graphs (KGs) has achieved success on various commonsense reasoning tasks. However, for a given task instance, the KG, or certain parts of the KG, may not be useful. Although KG-augmented models often use attention to focus on specific KG components, the KG is still always used, and the attention mechanism is never explicitly taught which KG components should be used. Meanwhile, saliency methods can measure how much a KG feature (e.g., graph, node, path) influences the model to make the correct prediction, thus explaining which KG features are useful. This paper explores how saliency explanations can be used to improve KG-augmented models’ performance. First, we propose to create coarse (Is the KG useful?) and fine (Which nodes/paths in the KG are useful?) saliency explanations. Second, to motivate saliency-based supervision, we analyze oracle KG-augmented models which directly use saliency explanations as extra inputs for guiding their attention. Third, we propose SALKG, a framework for KG-augmented models to learn from coarse and/or fine saliency explanations. Given saliency explanations created from a task’s training set, SALKG jointly trains the model to predict the explanations, then solve the task by attending to KG features highlighted by the predicted explanations. On three commonsense QA benchmarks (CSQA, OBQA, CODAH) and a range of KG-augmented models, we show that SALKG can yield considerable performance gains — up to 2.76% absolute improvement on CSQA. 2

heading: 1 Introduction
text: Natural language processing (NLP) systems generally need common sense to function well in the real world [15]. However, NLP tasks do not always provide the requisite commonsense knowledge as input. Moreover, commonsense knowledge is seldom stated in natural language, making it hard for pre-trained language models (PLMs) [11, 35] — i.e., text encoders — to learn common sense from corpora alone [9, 38]. In contrast to corpora, a knowledge graph (KG) is a rich, structured source of commonsense knowledge, containing numerous facts of the form (concept1, relation, concept2). As a result, many methods follow the KG-augmented model paradigm, which augments a text encoder with a graph encoder that reasons over the KG (Fig. 2). KG-augmented models have outperformed text encoders on various commonsense reasoning (CSR) tasks, like question answering (QA) (Fig. 1) [31, 5, 36, 61], natural language inference (NLI) [7, 57], and text generation [33, 65].
Since KGs do not have perfect knowledge coverage, they may not contain useful knowledge for all task instances (e.g., if the KG in Fig. 1 only consisted of the gray nodes). Also, even if the KG is useful overall for a given task instance, only some parts of the KG may be useful (e.g., the
∗Work done while TG interned remotely at USC. 2Code and data are available at: https://github.com/INK-USC/SalKG.
35th Conference on Neural Information Processing Systems (NeurIPS 2021).
green nodes in Fig. 1). Ideally, a KG-augmented model would know both if the KG is useful and which parts of the KG are useful. Existing KG-augmented models always assume the KG should be used, but do often use attention [54] to focus on specific KG components (e.g., nodes [13, 47, 60], paths [56, 46, 5]) when predicting. Still, the attention mechanism is supervised (end-to-end) only by the task loss, so the model is never explicitly taught which KG components should be used. Without component-level supervision, the attention mechanism is more likely to overfit to spurious patterns.
How can we better teach the model whether each KG feature (e.g., graph, node, path) is useful for solving the given task instance? Using the task’s ground truth labels, saliency methods [2] can score each KG feature’s influence on the model making the correct prediction. Whereas attention weights show which KG features the model already used, saliency scores indicate which KG features the model should use. By binarizing these scores, we are able to produce saliency explanations, which can serve as simple targets for training the model’s attention mechanism. For example, Fig. 1 shows saliency explanations [market=1, produce=1, trading=0, merchant=1, store=0, shop=0], stating that market, produce, and merchant are useful nodes for answering the question.
In this paper, we investigate how saliency explanations can be used to improve KG-augmented models’ performance. First, we propose to create coarse (graph-level) and fine (node-/path-level) saliency explanations. Since KGs have features at different granularities, saliency explanations can supply a rich array of signals for learning to focus on useful KG features. To create coarse explanations, we introduce an ensemble-based saliency method which measures the performance difference between a KG-augmented model and its corresponding non-KG-augmented model. To create fine explanations, we can adapt any off-the-shelf saliency method, e.g., gradient-based [10] or occlusion-based [30]. Second, to demonstrate the potential of saliency-based supervision, we analyze the performance of oracle KG-augmented models, whose attention weights are directly masked with coarse and/or fine saliency explanations. Third, as motivated by our oracle model analysis, we
propose the Learning from Saliency Explanations of KG-Augmented Models (SALKG) framework. Given coarse and/or fine explanations created from thse task’s training set, SALKG jointly trains the model to predict the explanations, then solve the task by attending to KG features highlighted in the predicted explanations. Using saliency explanations to regularize the attention mechanism can help the model generalize better to unseen instances, especially when coarse and fine explanations are used together as complementary learning signals. Indeed, on three standard commonsense QA benchmarks (CSQA, OBQA, CODAH) and a range of KG-augmented models, we show that SALKG can achieve considerable performance gains.
2 Preliminaries
Since KGs abundantly provide structured commonsense knowledge, KGaugmented models are often helpful for solving CSR tasks. CSR tasks are generally formulated as multi-choice QA (discriminative) tasks [52, 39, 23], but sometimes framed as open-ended response (generative) [33, 32] tasks. Given that multi-choice QA has been more extensively studied, we consider CSR in terms of multi-choice QA. Here, we present the multi-choice QA problem setting (Fig. 1) and the structure of KG-augmented models (Fig. 2).
Problem Definition Given a question q and set of answer choices A = {ai}, a multi-choice QA model aims to predict a plausibility score ρ(q, ai) for each (q, ai) pair, so that the predicted answer â = argmaxai∈A ρ(q, ai) matches the target answer a∗. Let q ⊕ ai be the text statement formed from (q, ai), where ⊕ denotes concatenation. For example, in Fig. 1, the text statement
for q ⊕ a∗ would be: What kind of store does a merchant have if they sell produce? market. We abbreviate q ⊕ ai as xi and its plausibility score as ρ(xi). KG-Augmented Models KG-augmented models use additional supervision from knowledge graphs to solve the multi-choice QA task. They encode the text and KG inputs individually as embeddings, then fuse the two embeddings together to use for prediction. A KG is denoted as G̃ = (Ṽ, R̃, Ẽ), where Ṽ , R̃, and Ẽ are the KG’s nodes (concepts), relations, and edges (facts), respectively. An edge is a directed triple of the form e = (c1, r, c2) ∈ Ẽ , in which c1, c2 ∈ Ṽ are nodes, and r ∈ R̃ is the relation between c1 and c2. A path is a connected sequence of edges in the KG. When answering a question, the model does not use the entire KG, since most information in G̃ is irrelevant to xi. Instead, the model uses a smaller, contextualized KG Gi = (Vi,Ri, Ei), which is built from G̃ using xi. Gi can be constructed heuristically by extracting edges from G̃ [31, 37], generating edges with a PLM [5], or both [56, 60]. In this paper, we consider KG-augmented models where Gi is built by heuristically by extracting edges from G̃ (see Sec. A.1 for more details), since most KG-augmented models follow this paradigm. If xi and Gi are not discussed in the context of other answer choices, then we further simplify xi’s and Gi’s notation as x and G, respectively. Since the model never uses the full KG at once, we use “KG” to refer to G in the rest of the paper. As in prior works [31, 5], a KG-augmented model FKG has three main components: text encoder ftext, graph encoder fgraph, and task predictor ftask (Fig. 2). Meanwhile, its corresponding non-KGaugmented model FNo-KG has no graph encoder but has a slightly different task predictor f̄task which only takes x as input. In both FKG and FNo-KG, the task predictor outputs ρ(x). Let x and g be the embeddings of x and G, respectively. Then, the workflows of FKG and FNo-KG are defined below:
x = ftext(x); g = fgraph(G,x); FKG(x,G) = ftask(x⊕ g); FNo-KG(x) = f̄task(x).
Typically, ftext is a PLM [11, 35], fgraph is a graph neural network (GNN) [13, 47] or edge/path aggregation model [31, 5, 46], and ftask and f̄task are multilayer perceptrons (MLPs). In general, fgraph reasons over G by encoding either nodes or paths, then using soft attention to pool the encoded nodes/paths into g. Let Ltask be the task loss for training FKG and FNo-KG. For multi-choice QA, Ltask is cross-entropy loss, with respect to the distribution over A. For brevity, when comparing different models, we may also refer to FKG and FNo-KG as KG and No-KG, respectively.

heading: 3 Creating KG Saliency Explanations
text: Now, we show how to create coarse and fine saliency explanations, which tell us if the KG or certain parts of the KG are useful. These explanations can be used as extra inputs to mask oracle models’ attention (Sec. 4) or as extra supervision to regularize SALKG models’ attention (Sec. 5). We first abstractly define a unit as either G itself or a component of G. A unit can be a graph, node, path, etc., and we categorize units as coarse (the entire graph G) or fine (a node or path within G) (Table 1). Given a model and task instance (x,G), we define an explanation as a binary indicator of whether a unit u of G is useful for the model’s prediction on (x,G). If u is useful, then u should strongly influence the model to solve the instance correctly. By making explanations binary, we can easily use explanations as masks or learning targets (since binary labels are easier to predict than real-valued scores) for attention weights.

heading: 3.1 Coarse Saliency Explanations Explanation Setting Unit
text: Coarse KG Fine (MHGRN) Node Fine (PathGen) Path Fine (RN) Path
To get yc(x,G), we begin by computing coarse saliency score sc(x,G) ∈ R, which we define as the performance difference between FKG and FNo-KG. For QA input xi = q ⊕ ai and its KG Gi, let pKG(xi,Gi) and pNo-KG(xi) be the confidence probabilities for xi predicted by FKG and FNo-KG, respectively.
sc(xi,Gi)
= { pKG(xi,Gi)− pNo-KG(xi), ai = a∗, pNo-KG(xi)− pKG(xi,Gi), ai ̸= a∗. (1) Ideally, a QA model should predict higher probabilities for answer choices ai that are correct, and vice versa. To capture this notion, we define sc(xi,Gi) in Eq. 1, where a∗ denotes the correct answer. Note that sc(xi,Gi) is positive if pKG(xi,Gi) is higher than pNo-KG(xi) for correct choices and lower for incorrect choices. We obtain yc(xi,Gi) by binarizing sc(xi,Gi) to 0 or 1 based on whether it is greater than or less than a threshold T , respectively. If yc(xi,Gi) = 1, then the KG is useful, and vice versa. See the appendix for more details about why we use ensemble-based saliency for coarse explanations (Sec. A.2) and how we tune T (Sec. A.6).

heading: 3.2 Fine Saliency Explanations
text: Even if G is useful, not every part of G may be useful. Hence, fine saliency explanations can identify which parts of a KG are actually useful. For a given instance (x,G), we denote the fine saliency explanation for a fine unit u in G as yf(u;x,G) ∈ {0, 1}. Fine units can be nodes, paths, etc. in the KG. If a graph encoder fgraph encodes a certain type of unit, it is natural to define yf(u;x,G) with respect to such units. For example, MHGRN [13] encodes G’s nodes, so we define MHGRN’s fine saliency explanations with respect to nodes. Similar to coarse saliency explanations, to obtain yf(u;x,G), we first compute fine saliency score sf(u;x,G) ∈ R, and then binarize it. For a QA input xi = q ⊕ ai and its KG Gi, let uij be the jth fine unit in Gi and pKG(xi,Gi) denote FKG’s predicted probability for xi. There are many existing saliency methods (a.k.a. attribution methods) [10, 51, 30] for calculating the importance score of an input, with respect to a model and a given label. While sf(uij ;xi,Gi) can be computed via any saliency method, we use gradient-based and occlusion-based methods, since they are the most common types of saliency methods [2].
Let ϕ(uij ;xi,Gi) denote the raw saliency score given by some saliency method. Gradient-based methods measure an input’s saliency via the gradient of the model’s output with respect to the input. We use the gradient×input (Grad) method [10], where ϕ(uij ;xi,Gi) is the dot product of uij’s embedding and the gradients of pKG(xi,Gi) with respect to uij . Occlusion-based methods measure an input’s saliency as how the model’s output is affected by erasing that input. We use the leave-one-out (Occl) method [30], where ϕ(uij ;xi,Gi) is the decrease in pKG(xi,Gi) if uij is removed from Gi, i.e., ϕ(uij ;xi,Gi) = pKG(xi,Gi) - pKG(xi,Gi \ uij).
sf(uij ;xi,Gi)
= { ϕ(uij ;xi,Gi), ai = a∗ −ϕ(uij ;xi,Gi), ai ̸= a∗ (2) Intuitively, a unit is more useful if it increases the probability of correct answer choice a∗, and vice versa. Thus, we define the saliency score sf(uij ;xi,Gi) for unit uij as Eq. 2. Next, we binarize the saliency scores to get yf(uij ;xi,Gi), by selecting the top-k%-scoring units in Gi and setting yf(uij ;xi,Gi) = 1 (i.e., uij is useful) for these units. For all other units in G, we set yf(uij ;xi,Gi) = 0 (i.e., uij is not useful). See the appendix for more details about the fine saliency methods (Sec. A.3) and tuning threshold k (Sec. A.6).

heading: 4 ORACLE: Using KG Saliency Explanations as Inputs
text: In this section, we analyze KG saliency explanations’ potential to improve KG-augmented models’ performance. Recall that creating saliency explanations requires the task’s ground truth labels (Sec. 3), so directly using test set explanations is infeasible. Still, before exploring ways to leverage training set explanations (Sec. 5), we first establish upper bounds on how much models can benefit from saliency explanations. Here, we study three key questions: (1) Does the model improve when provided oracle access to coarse/fine explanations? (2) Are coarse and fine explanations complementary? (3) How do gradient-based explanations compare to occlusion-based explanations?

heading: 4.1 ORACLE Models
text: ORACLE models are KG-augmented models with oracle access to saliency explanations. An ORACLE model uses ground truth labels to create explanations (even at inference time), and then uses the explanations as extra inputs to perform hard attention over the units. We define the model attention weights that are modified based on saliency explanations as saliency weights. Below, we introduce the ORACLE-Coarse, ORACLE-Fine, and ORACLE-Hybrid models, shown in Fig. 3a-c.
ORACLE-Coarse ORACLE-Coarse (F∗c ) uses coarse explanations to do hard attention over FKG’s and FNo-KG’s predictions. First, FKG and FNo-KG are trained separately, then frozen. Next, for each instance (x,G), they are used to create a coarse explanation yc(x,G) ∈ {0, 1}. Then, F∗c is defined as an ensemble model that performs hard attention over coarse units (G and None) by weighting FKG’s prediction with yc(x,G) and FNo-KG’s prediction with 1− yc(x,G) (Table 2; Fig. 3a). In other words, yc(x,G) and 1− yc(x,G) are the saliency weights for F∗c . ORACLE-Fine ORACLE-Fine (F∗f ) has the same architecture as FKG and uses fine explanations to do hard attention over fine units (i.e., nodes or paths in G). First, FKG is trained, then frozen. As usual, FKG uses soft attention over fine units in G to compute graph embedding g (Sec. 2). Then, for each fine unit u in G, FKG is used to create fine explanation yf(u;x,G) ∈ {0, 1}. Let ŷf(u;x,G) ∈ [0, 1] denote F∗f ’s soft attention weight for u. We train F∗f the same way as FKG, except each ŷf(u;x,G) is (hard attention) masked with yf(u;x,G), i.e., ŷf(u;x,G) ← ŷf(u;x,G) ⊙ yf(u;x,G), where ⊙ denotes element-wise multiplication (Table 2; Fig. 3b). This means only units with yf(u;x,G) = 1 will have ŷf(u;x,G) > 0 and thus be able to influence F∗f ’s prediction. Let yf(x,G) and ŷf(x,G) denote the explanations and soft attention weights, respectively, for all units in the graph. Then, ŷf(x,G)⊙ yf(x,G) are the saliency weights for F∗f . ORACLE-Hybrid ORACLE-Hybrid (F∗h ) unifies ORACLE-Coarse and ORACLE-Fine as a single model, thus leveraging the coarse-fine hierarchy inherent in KG saliency explanations. First, F∗f (which uses fine explanations) and FNo-KG are separately trained, then frozen. Then, for each (x,G), F∗f and FNo-KG are used to create yh(x,G) ∈ {0, 1}, which we define as the coarse explanation for F∗f and FNo-KG. yh(x,G) is computed the same way as yc(x,G), besides replacing FKG with F∗f . Finally, similar to F∗c , F∗h is an ensemble that performs hard attention over coarse units by weighting F∗f ’s prediction with yh(x,G) and FNo-KG’s prediction with 1− yh(x,G) (Table 2; Fig. 3c). That is, yh(x,G) and 1− yh(x,G) are the saliency weights for F∗h .

heading: 4.2 Evaluation Protocol
text: We use the CSQA [52] and OBQA [39] multi-choice QA datasets. For CSQA, we use the accepted in-house data split from [31], as the official test labels are not public. As in prior works, we use the ConceptNet [49] KG for both datasets. We report accuracy, the standard metric for multi-choice QA. For FNo-KG and FKG, we pick the best model over three seeds, then use them to create explanations for ORACLE models. We use thresholds T = 0.01 and k = 10 for coarse and fine explanations, respectively. For text encoders, we use BERT(-Base) [11] and RoBERTa(-Large) [35]. For graph encoders, we use MHGRN [13], PathGen [56], and Relation Network (RN) [46, 31]. MHGRN has node units, while PathGen and RN have path units. As baseline models, we use FNo-KG, FKG, and FNo-KG + FKG, where FNo-KG + FKG is an ensemble whose prediction is the mean of FNo-KG’s and FKG’s predictions. ORACLE and baseline models are trained only with task loss Ltask.

heading: 4.3 Analysis
text: In Table 3, we show CSQA and OBQA performance for the baseline and ORACLE models. We analyze these results via the three questions below.
Does the model improve when provided oracle access to coarse/fine explanations? Yes. ORACLECoarse beats all baselines, while ORACLE-Fine beats all baselines except on OBQA RN+RoBERTa. These results motivate us to develop a framework for models to improve performance by learning from coarse/fine explanations. Also, on average, ORACLE-Fine outperforms ORACLE-Coarse, which suggests that fine explanations may often provide richer signal than their coarse counterparts. Indeed, fine explanations indicate the saliency of every unit in the KG, while coarse explanations only indicate the saliency of the KG as a whole.
Are coarse and fine explanations complementary? Yes. Across all settings, ORACLE-Hybrid performs significantly better than ORACLE-Coarse and ORACLE-Fine. This suggests that coarse and fine explanations are complementary and that it is effective to leverage both hierarchically.
How do gradient-based explanations compare to occlusion-based explanations? Overall, Grad and Occl perform similarly. Grad performs better on some settings (e.g., MHGRN), while Occl performs better on others (e.g., RN). See Table 8 and Sec. A.9 for more Grad vs. Occl experiments.
In our ORACLE pilot study, KG-augmented models achieve large performance gains when given explanations as input. This suggests that, if oracle explanations can somehow be predicted accurately during inference without using ground truth labels, then KG-augmented models can still achieve improvements without directly using explanations as input. This motivates us to train KG-augmented models with explanation-based supervision via SALKG, which we describe in Sec. 5.

heading: 5 SALKG: Using KG Saliency Explanations as Supervision
text: Based on the analysis from Sec. 4.3, we propose the SALKG framework for KG-augmented models to learn from coarse/fine saliency explanations. Whereas ORACLE models (Sec. 4.1) use explanations directly as extra inputs, SALKG models only use them as extra supervision during the training phase. With explanations created from the training set via FKG and FNo-KG, SALKG models are jointly trained to predict the explanations (via saliency loss Lsal) and use the predicted explanations to solve the task (via task loss Ltask). Thus, SALKG models have the following objective: LS = Ltask + λLsal, where λ ≥ 0 is a loss weighting parameter. This multitask objective not only encourages SALKG models to focus on useful KG units for solving the task, but also to learn more general graph/node/path representations. Below, we present SALKG-Coarse, SALKG-Fine, and SALKG-Hybrid models.
SALKG-Coarse Unlike ORACLE-Coarse, SALKG-Coarse (Fc) is not given oracle coarse explanation yc(x,G) as input. Instead, a saliency predictor Sc (with the same architecture asFKG) is trained to predict the oracle coarse explanation. Sc predicts coarse explanation as probability ŷc(x,G) ∈ [0, 1]. Fc’s output is an ensemble that does soft attention over coarse units by weighting FKG’s and FNo-KG’s predictions with saliency weights ŷc(x,G) and 1− ŷc(x,G), respectively (Table 4; Fig. 3a). Here, Lsal(ŷc(x,G), yc(x,G)) is the cross-entropy loss.
SALKG-Fine Similarly, SALKG-Fine (Ff) is not given oracle fine explanation yf(u;x,G) as input, although both have the same architecture as FKG. Instead, for each fine unit u, Ff’s attention mechanism is trained to predict yf(u;x,G) as soft attention weight ŷf(u;x,G) ∈ [0, 1] (Table 4; Fig. 3b). As before, ŷf(x,G) = [ŷf(u;x,G)]u∈G are the soft attention weights for (x,G), while yf(x,G) = [yf(u;x,G)]u∈G are the fine explanations for (x,G). Then, ŷf(x,G) are the saliency weights for Ff, trained with KL divergence loss Lsal(ŷf(x,G), yf(x,G)). SALKG-Hybrid Similar to the other SALKG variants, SALKG-Hybrid (Fh) does not use any oracle explanations. Like in SALKG-Coarse, a saliency predictor Sh is trained to predict oracle coarse explanation yh(x,G) (Sec. 4.1). Predicted coarse explanation probabilities ŷh(x,G) ∈ [0, 1] are then used as soft attention over coarse units by weighting Ff’s and FNo-KG’s predictions with weights ŷh(x,G) and 1− ŷh(x,G), respectively (Table 4; Fig. 3c). Here, Lsal(ŷh(x,G), yh(x,G)) is cross-entropy loss.

heading: 6 Experiments

heading: 6.1 Evaluation Protocol
text: We evaluate SALKG models on the CSQA [52], OBQA [39], and CODAH [6] multi-choice QA datasets (Sec. A.5). In addition to the baselines in Sec. 4.2, we consider two new baselines, RANDOM and HEURISTIC, which help show that coarse/fine saliency explanations provide strong learning signal for KG-augmented models to focus on useful KG features. We follow the same evaluation protocol in Sec. 4.2, except we now also report mean and standard deviation performance over multiple seeds. See Sec. A.4 for a more detailed description of the evaluation protocol.
RANDOM RANDOM is a variant of SALKG where each unit’s explanation is random. RANDOMCoarse is like SALKG-Coarse, but with each yc(x,G) uniformly sampled from {0, 1}. RANDOM-Fine is like SALKG-Fine, but randomly picking k% of units in G to set yf(u;x,G) = 1. RANDOM-Hybrid is like SALKG-Hybrid, but with each yh(x,G) uniformly sampled from {0, 1} as well as using RANDOM-Fine instead of SALKG-Fine.
HEURISTIC Each G has three node types: question nodes (i.e., nodes in q), answer nodes (i.e., nodes in ai), and intermediate nodes (i.e., other nodes) [31]. Let QA nodes be nodes in q or ai. HEURISTIC is a variant of SALKG where each unit’s explanation is based on the presence of QA nodes in G. Let N̄ be the mean number of QA nodes per KG (in train set), and let N(G) be the number of QA nodes in G. HEURISTIC-Coarse is like SALKG-Coarse, except yc(x,G) = 1 if and only if N(G) > N̄ . HEURISTIC-Fine is like SALKG-Fine, but how yf(u;x,G) is set depends on whether the fine units are nodes or paths. For node units, yf(u;x,G) = 1 if and only if u is a QA node. For path units, yf(u;x,G) = 1 if and only if u consists only of QA nodes. HEURISTIC-Hybrid is like SALKG-Hybrid, but with yh(x,G) = 1 if and only if N(G) > N̄ , while HEURISTIC-Fine is used instead of SALKG-Fine.

heading: 6.2 Main Results
text: Table 5 shows performance on CSQA, while Table 6 shows performance on OBQA and CODAH. Best performance is highlighted in green , second-best performance is highlighted in blue , and
best non-SALKG performance is highlighted in red (if it is not already green or blue). For SALKG (unlike ORACLE), we find that Occl usually outperforms Grad, so we only report Occl performance in Tables 5-6. For a comparison of Grad and Occl on SALKG, see Table 8 and Sec. A.9. Being an ensemble, No-KG + KG tends to beat both No-KG and KG if both have similar performance. Otherwise, No-KG + KG’s performance is in between No-KG’s and KG’s.
Across all datasets, we find that SALKG-Hybrid and SALKG-Coarse are consistently the two best models. On CSQA, SALKG-Hybrid has the highest performance on BERT+MHGRN,
BERT+PathGen, BERT+RN, and RoBERTa+RN, while SALKG-Coarse is the best on RoBERTa+MHGRN and RoBERTa+PathGen. In particular, on RoBERTa+RN, BERT+RN, and BERT+PathGen, SALKG-Hybrid beats max(No-KG, KG, No-KG + KG) by large margins of 2.76%, 2.58%, and 2.32%, respectively. Meanwhile, OBQA and CODAH, SALKG is not as dominant but still yields improvements overall. On OBQA, SALKG-Coarse is the best on RoBERTa+RN (beating max(No-KG, KG, No-KG + KG) by 1.89%) and RoBERTa+PathGen, while SALKG-Hybrid performs best on RoBERTa+MHGRN. On CODAH, SALKG-Coarse gets the best performance on both RoBERTa+MHGRN (beating max(No-KG, KG, No-KG + KG) by 1.71%) and RoBERTa+PathGen. SALKG-Coarse outperforming SALKG-Hybrid on OBQA and CODAH indicates that local KG supervision from fine explanations may not be as useful for these two datasets. On the other hand, SALKG-Fine is consistently weaker than SALKG-Hybrid and SALKG-Coarse, but still shows slight improvement for RoBERTa+RN on CSQA. These results show that learning from KG saliency explanations is generally effective for improving KG-augmented models’ performance, especially in CSQA when both coarse and fine explanations are used to provide complementary learning signals for SALKG-Hybrid. Furthermore, across all datasets, we find that SALKG outperforms RANDOM and HEURISTIC on every setting. This is evidence that explanations created from saliency methods can provide better learning signal than those created randomly or from simple heuristics.
Comparison to Published CSQA Baselines To further demonstrate that SALKG models perform competitively, we also compare SALKG (using MHGRN and PathGen) to the many KG-augmented model baseline results published in [13, 56, 60], for the CSQA in-house split. The baselines we consider are RN [46], RN + Link Prediction [13], RGCN [47], GAT [55], GN [4], GconAttn [57], MHGRN [13], and PathGen [56]. For the non-SALKG versions of MHGRN, PathGen, and RN, we quote the published results. Since these published results average over four seeds (instead of three), we report SALKG results over four seeds in Table 7. We find that most of the listed SALKG variants can outperform all of the baselines. For MHGRN, SALKG-Coarse (MHGRN) performs the best overall, SALKG-Hybrid (MHGRN) beats vanilla MHGRN, and SALKG-Fine (MHGRN) is on par with vanilla MHGRN. For PathGen, SALKG-Hybrid (PathGen) and SALKG-Coarse (PathGen) both slightly outperform vanilla PathGen, while SALKG-Fine (PathGen) performs worse.
CSQA Leaderboard Submission In addition to our experiments on the CSQA in-house split, we evaluated SALKG on the CSQA official split by submitting SALKG to the CSQA leaderboard. Since the best models on the CSQA leaderboard use the ALBERT [24] text encoder, and PathGen was the highest graph encoder on the leaderboard out of the three we experimented with, we trained SALKG-Hybrid (ALBERT+PathGen), which achieved a test accuracy of 75.9%. For reference, a previously submitted ALBERT+PathGen achieved a test accuracy of 75.6% on the CSQA leaderboard. This result suggests that the proposed SALKG training procedure can yield some improvements over baselines that do not use explanation-based regularization.
Why does SALKG-Fine perform poorly? In general, SALKG-Fine does not perform as well as SALKG-Coarse and SALKG-Hybrid. Often, SALKG-Fine is noticeably worse than KG and No-KG. Recall that the KG model and SALKG-Fine model both assume that the KG should always be used to solve the given instance. Still, the success of SALKG-Coarse shows that the KG sometimes may not be useful. But why does SALKG-Fine almost always perform worse than the KG model?
We believe it is because SALKG-Fine is more committed to the flawed assumption of universal KG usefulness. Whereas the KG model is trained to solve the task always using the KG as context, SalKGFine is trained to both solve the task always using the KG as context (i.e., global KG supervision) and attend to specific parts of the KG (i.e., local KG supervision). Since SALKG-Fine is trained with both global and local KG supervision, it is much more likely to overfit, as the KG is not actually useful for all instances. That is, for training instances where the KG should not be used, SALKG-Fine is pushed to not only use the KG, but also to attend to specific parts of the KG. This leads to a SalKG-Fine model that does not generalize well to test instances where the KG is not useful.
To address this issue, we proposed the SALKG-Hybrid model, which is designed to take the best of both SALKG-Coarse and SALKG-Fine. For a given instance, SALKG-Hybrid uses its SALKGCoarse component to predict whether the KG is useful, then uses its SALKG-Fine component to attend to the useful parts of the KG only if the KG is predicted to be useful. Indeed, we find that SALKG-Hybrid performs much better than SALKG-Fine and is the best model overall on CSQA. These results support our hypothesis about why SALKG-Fine performs relatively poorly.

heading: 6.3 Ablation Studies CSQA Dev Accuracy (%)
text: In Table 8, we validate our SALKG design choices with ablation studies. We report dev accuracy for BERT+MHGRN and BERT+PathGen on CSQA.
Are ensemble-based coarse explanations effective? By default, SALKG-Coarse uses our proposed ensemble-based coarse explanations (Sec. 3.1). Alternatively, we consider using Grad and Occl to create coarse explanations. For Grad, we compute ϕ the same way as in Sec. 3.2, except using graph embedding g instead of node/path embeddings. Since a zero vector would have zero gradient, this is equivalent to comparing g to a zero vector baseline. For Occl, we compute ϕ as the decrease in pKG if g is replaced
with a zero vector. For both Grad and Occl, we set sc = ϕ. In Table 8, we see that our default SALKG-Coarse significantly outperforms SALKG-Coarse with both Grad and Occl. In Sec. A.2, we further discuss why Grad and Occl are ill-suited for creating coarse explanations.
For SALKG, is Occl better than Grad? In Tables 5-6, we report SALKG-Fine and SALKG-Hybrid performance with Occl fine explanations. In Table 8, we compare Occl and Grad on SALKG-Fine and SALKG-Hybrid. Overall, Occl slightly outperforms Grad, although Grad beats Occl on MHGRN
for SALKG-Hybrid. Their relative performance could also depend on the choice of top-k%, which we plan to explore later. In Sec. A.9, we further compare Occl and Grad on other settings.
How does SALKG-Fine’s soft KG pruning compare to hard KG pruning? SALKG-Fine does soft pruning of unhelpful fine units via soft attention. We compare SALKG-Fine to two baselines where the KG is filtered via hard pruning, which cannot be easily incorporated into end-to-end training. For RANDOM Prune and HEURISTIC Prune, we respectively create RANDOM and HEURISTIC explanations, then hard prune all negative units from the KG. The KG-augmented model then uses the pruned KG as its KG input. In Table 8, we see that SALKG-Fine significantly outperforms the two baselines, showing the benefits of jointly training the model on saliency and QA prediction.
Is it effective to train SALKG-Fine with KL divergence? We train SALKG-Fine’s explanation predictor (i.e., attention mechanism) using KL divergence as the saliency loss. Thus, within a KG, the distribution over attention weights constitutes a single prediction. Alternatively, we could treat each attention weight as a separate prediction and train the attention mechanism using binary cross entropy (BCE) loss. In Table 8, we find that using KL divergence yields much higher performance than using BCE loss. This suggests that the attention weights should not be trained separately, as each attention weight is highly dependent on other attention weights in the same KG.

heading: 6.4 Case Studies
text: We visualize coarse/fine explanations created from BERT+PathGen on CSQA, with 1-hop or 2-hop paths as fine units. For coarse explanations, we show examples of positive (i.e., useful) and negative KGs. Since KGs are too large to show here, we uniformly sample three paths per KG. For the positive KG example, the question is James loved to play violin. He did it in his spare time because he found it what?, the answer choice is relaxing, and the target answer is relaxing. Its paths are: (1) play –[is related to]–> x <–[is used for]– relaxing , (2) violin –[is used for]–> x –[is used for]–> relaxing , and (3) time <–[has subevent]– x –[has subevent]–> relax . For the negative KG example, the question is Where do soldiers not deployed eat their food?, the answer choice is neighbor’s house, and the target answer is military base. Its paths are: (1) soldier <–[is related to]– x <–[is related to]– house , (2) eat –[is related to]–> x –[is at location of]–> house , and (3) food <–[is related to]– x –[is at location of]–> house . For fine explanations, we show examples of positive and negative paths from the same KG. Here, the question is Where can you find a bar before traveling a long distance?, the answer choice is airport, and the target answer is airport. The positive path is: bar –[is at location]–> airport . The negative path is: travel <–[is used for]– x –[is at location]– airport . We can roughly see that the positive KGs/paths are useful for predicting the correct answer, and vice versa. However, as shown in [45], the model’s judgment of KG/path usefulness may not always align with human judgment. See Sec. A.16 for more illustrative examples of coarse/fine explanations.

heading: 7 Related Work
text: Creating Model Explanations Many methods aim to explain PLMs’ predictions by highlighting important tokens in the model’s text input. Such methods are usually gradient-based [51, 29, 10], attention-based [40, 53, 14, 25], or occlusion-based [12, 42, 22, 30]. Similarly, for graph encoders, a number of works use post-hoc optimization to identify important nodes [19, 62] or subgraphs [62] in the graph input. Meanwhile, KG-augmented models’ attention weights can be used to explain which parts of the KG are important [31, 13, 34, 56, 60]. These KG explanations can be interpreted as identifying knowledge in the KG that is complementary to the knowledge encoded in the PLM.
Learning From Model Explanations Besides manual inspection, explanations can be used in various ways, like extra supervision or regularization [43, 17, 41, 1], pruned inputs [21, 3, 28], additional inputs [16, 8], and intermediate variables [58, 66, 44]. The most similar work to ours is [43], which proposed training a student model to mimic a teacher model’s predictions by regularizing the student model’s attention via text explanations created from the teacher model. However, [43] aims to evaluate explanations, while our goal is to improve performance via explanations. To the best of our knowledge, SALKG is the first to supervise KG-augmented models with KG explanations.
See Sec. A.20 for a more comprehensive overview of the related literature.

heading: 8 Conclusion
text: In this paper, we proposed creating coarse and fine explanations for KG-augmented models, then using these explanations as extra inputs (ORACLE) or supervision (SALKG). Across three commonsense QA
benchmarks, SALKG achieves strong performance, especially when both coarse and fine explanations are used. In future work, we plan to explore incorporating active learning into SALKG, so that models can also leverage explanation-based feedback from humans about KG saliency.

heading: 9 Acknowledgments
text: This research is supported in part by the Office of the Director of National Intelligence (ODNI), Intelligence Advanced Research Projects Activity (IARPA), via Contract No. 2019-19051600007, the DARPA MCS program under Contract No. N660011924033, the Defense Advanced Research Projects Agency with award W911NF-19-20271, NSF IIS 2048211, NSF SMA 1829268, and gift awards from Google, Amazon, JP Morgan, and Sony. We would like to thank all of our collaborators at the USC INK Research Lab for their constructive feedback on this work.

Review summaries:
The paper explores how saliency explanations can be used to improve the performance of KG-augmented models. The authors compared two types of saliency explanations: coarse (whether the KG is useful) and fine (which nodes or paths in the KG are useful) They first showed that using oracle KG saliency explanations as inputs improves model performance (section 4).  They then introduced a model that jointly learns to predict the saliency explanations, and solves the task by attending to corresponding KG features highlighted by the predicted explanations (SALKG). The results showed that SALKG models yield large performance gains in three popular commonsense QA benchmarks (CSQA, OBQA, CODAH).

This paper explores how saliency explanations can be used to improve the performance of knowledge-graph-augmented commonsense reasoning (CSR) models, by investigating coarse to fine KB components (an entire graph to its nodes and paths). The paper first explores how KG influences the models in making correct predictions using oracle models, which have access to true saliency explanations. Base on that, the paper proposes SALKG, a framework for KG-augmented CSR that learns from coarse and/or fine saliency explanations that are automatically created. The proposed models achieve better performance compared to the baselines used in the paper.

Motivated by that the knowledge graph used in KG-augmented models is useless sometimes, the authors propose two ensemble models by using coarse/fine saliency explanations. Specifically, the authors first get saliency explanations through F_KG, F_NO-KG and saliency methods. Then, the authors propose oracle model using coarse/fine/hybrid explanations as extra inputs, and achieves superb performance than its baselines. Finally, based on oracle models, the authors develop the SALKG framework that uses explanations as supervision for distillation learning. SALKG achieves SOTA performance on CommonsenseQA and competitive performance on OBQA. It is the first to supervise KG-augmented models with KG explanations.

This paper proposes a framework, SALKG, for commonsense question answering (i.e., Multiple choice). It employs saliency explanations (i.e., coarse and fine) to regularize the attention mechanism and provide direct supervision for the task. The proposed framework is shown to achieve state-of-the-art or competitive results on three QA benchmarks, namely CSQA, OBQA, and CODAH. The proposed work has highlighted the right research questions and answered quantitatively and to some extent qualitatively. Overall, the paper is well motivated and proposes a technically solid framework for multiple-choice commonsense question answering tasks. I have some concerns regarding the details on using the knowledge graph that I have highlighted in the weaknesses section.