L EARNING TO D ECEIVE K NOWLEDGE G RAPH A UGMENTED M ODELS VIA T ARGETED P ERTURBATION
A BSTRACT
Knowledge graphs (KGs) have helped neural models improve performance on various knowledge-intensive tasks, like question answering and item recommendation.
By using attention over the KG, such KG-augmented models can also “explain” which KG information was most relevant for making a given prediction.
In this paper, we question whether these models are really behaving as we expect.
We show that, through a reinforcement learning policy (or even simple heuristics), one can produce deceptively perturbed KGs, which maintain the downstream performance of the original KG while signiﬁcantly deviating from the original KG’s semantics and structure.
Our ﬁndings raise doubts about KG-augmented models’ ability to reason about KG information and give sensible explanations.
I NTRODUCTION
Recently, neural reasoning over knowledge graphs (KGs) has emerged as a popular paradigm in machine learning and natural language processing (NLP).
Such KG-augmented models have improved performance on a number of knowledge-intensive downstream tasks: for question answering (QA), the KG provides context about how a given answer choice is related to the question (Lin et al., 2019;
Feng et al., 2020; Lv et al., 2020; Talmor et al., 2018); for item recommendation, the KG mitigates data sparsity and cold start issues (Wang et al., 2018b; 2019a;b; 2018a).
Furthermore, by using attention over the KG, such models aim to explain which KG information was most relevant for making a given prediction (Lin et al., 2019;
Feng et al., 2020; Wang et al., 2018b; 2019b; Cao et al., 2019; Gao et al., 2019).
Nonetheless, the process in which KG-augmented models reason about KG information is still not well understood.
It is assumed that, like humans, KG-augmented models base their predictions on meaningful KG paths and that this process is responsible for their performance gains (Lin et al., 2019;
Feng et al., 2020; Gao et al., 2019; Song et al., 2019).
In this paper, we question if existing KG-augmented models actually use KGs in this human-like manner.
We study this question primarily by measuring model performance when the KG’s semantics and structure have been perturbed to hinder human comprehension.
To perturb the KG, we propose four perturbation heuristics and a reinforcement learning (RL) based perturbation algorithm.
Surprisingly, for KG-augmented models on both commonsense QA and item recommendation, we ﬁnd that the KG can be extensively perturbed with little to no effect on performance.
This raises doubts about KG-augmented models’ use of KGs and the plausibility of their explanations.
P ROBLEM S ETTING
Our goal is to investigate whether KG-augmented models and humans use KGs similarly.
Since
KGs are human-labeled, we assume that they are generally accurate and meaningful to humans.
Thus, across different perturbation methods, we measure model performance when every edge in the KG has been perturbed to make less sense to humans.
To quantify the extent to which the KG has been perturbed, we also measure both semantic and structural similarity between the original
KG and perturbed KG.
If original-perturbed KG similarity is low, then a human-like KG-augmented model should achieve worse performance with the perturbed KG than with the original KG.
Furthermore, we evaluate the plausibility of KG-augmented models’ explanations when using original and perturbed KGs, by asking humans to rate these explanations’ readability and usability.
Notation
Let F θ be an KGaugmented model, and let ( X train , X dev , X test ) be a dataset for some downstream task.
We denote a KG as G =
( E , R , T ) , where E is the set of entities (nodes),
R is the set of relation types, and
T = { ( e 1 , r, e 2 ) | e 1 , e 2 ∈ E , r ∈ R} is the set of facts (edges) composed from existing entities and relations (Zheng et al., 2018).
Let
G (cid:48)
=
( E , R (cid:48) , T (cid:48) ) be the KG obtained after perturbing G , where R (cid:48) ⊆
R and
T (cid:48) (cid:54) = T .
Let f ( G , G (cid:48) ) be a function that measures similarity between G and G (cid:48) .
Let g ( G ) be the downstream performance when evaluating F θ on X test and G .
Also, let ⊕ denote the concatenation operation, and let
N L ( e ) denote the set of L -hop neighbors for entity e ∈ E .
High-Level Procedure
First, we train F θ on X train and G , then evaluate F θ on X test and G to get the original performance g ( G ) .
Second, we freeze F θ , then perturb G to obtain G (cid:48) .
Third, we evaluate
F θ on X test and G (cid:48) to get the perturbed performance g ( G (cid:48) ) .
Finally, we measure g ( G ) − g
( G (cid:48) ) and f ( G , G (cid:48) ) to assess how human-like F θ ’s reasoning process is.
This procedure is illustrated in Fig.
In this paper, we consider two downstream tasks: commonsense QA and item recommendation.
Commonsense QA Given a question x and a set of k possible answers A = { y 1 , ..., y k } , the task is to predict a compatibility score for each ( x, y ) pair, such that the highest score is predicted for the correct answer.
In commonsense QA, the questions are designed to require commonsense knowledge which is typically unstated in natural language, but more likely to be found in KGs (Talmor et al., 2018).
Let F text φ be a text encoder (Devlin et al., 2018), F graph ψ be a graph encoder, and F cls ξ be an MLP classiﬁer, where φ, ψ, ξ ⊂ θ .
Let G ( x,y ) denote a subgraph of G consisting of entities mentioned in text sequence x ⊕ y , plus their corresponding edges.
We start by computing a text embedding h text =
F text φ ( x ⊕ y ) and a graph embedding h graph = F graph φ ( G ( x,y ) ) .
After that, we compute the score for ( x, y ) as S ( x,y ) =
F cls ξ ( h text ⊕ h graph ) .
Finally, we select the highest scoring answer: y pred = arg max
y ∈
A S ( x,y ) .
KG-augmented commonsense QA models vary primarily in their design of F graph ψ .
In particular, path-based models compute the graph embedding by using attention to selectively aggregate paths in the subgraph.
The attention scores can help explain which paths the model focused on most for a given prediction (Lin et al., 2019;
Feng et al., 2020; Santoro et al., 2017).
Item Recommendation We consider a set of users U
= { u 1 , u 2 , ..., u m } , a set of items
V = { v 1 , v 2 , ..., v n } , and a user-item interaction matrix Y ∈ R m × n with entries
y uv .
If user u has been observed to engage with item v , then y uv = 1 ; otherwise, y uv = 0 .
Additionally, we consider a
KG G , in which R is the set of relation types in G .
In G , nodes are items v ∈ V , and edges are facts of the form ( v, r, v (cid:48) ) , where r ∈ R is a relation.
For the zero entries in Y (i.e., y uv = 0 ), our task is to predict a compatibility score for user-item pair ( u, v ) , indicating how likely user u is to want to engage with item v .
We represent each user u , item v , and relation r as embeddings u , v , and r , respectively.
Given a user-item pair ( u, v ) , its compatibility score is computed as (cid:104) u , v (cid:105) , the inner product between u and v .
KG-augmented recommender systems differ mainly in how they use G to compute u and v .
Generally, these models do so by using attention to selectively aggregate items/relations in G .
The attention scores can help explain which items/relations the model found most relevant for a given prediction (Wang et al., 2018b; 2019b).
KG S IMILARITY M ETRICS
To measure how much the perturbed KG has deviated from the original KG, we propose several metrics for capturing semantic (ATS) and structural (SC2D, SD2) similarity between KGs.
Aggregated Triple Score (ATS) ATS measures semantic similarity between two KGs.
Let s G be an edge (triple) scoring function, such that s G ( e 1 , r, e 2 ) measures how likely edge ( e 1 , r, e 2 ) is to exist in G .
Also, assume s G has been pre-trained on G for link prediction.
Then, ATS is deﬁned as f ATS ( G , G (cid:48) )
= 1 |T (cid:48) | (cid:80) ( e 1 ,r,e 2 ) ∈T (cid:48)
s G ( e 1 , r, e 2 ) ∈ [0 , 1] , which denotes the mean s G score across all edges in G (cid:48) .
Intuitively, if a high percentage of edges in G (cid:48) are also likely to exist in G (i.e., high
ATS), then we say that G (cid:48) and G have high semantic similarity.
s G is task-speciﬁc, as KGs from different tasks may differ greatly in semantics.
For commonsense QA, we use the s G from Li et al.
(2016); for item recommendation, we use the s G from Yang et al.
While ATS captures semantic KG differences, it is not sensitive to KG connectivity structure.
Note that f ATS ( G , G ) may not equal 1, since s G may not perfectly generalize to KGs beyond those it was trained on.
Similarity in Clustering Coefﬁcient Distribution (SC2D) SC2D measures structural similarity between two KGs and is derived from the local clustering coefﬁcient (Saram¨aki et al., 2007; Onnela et al., 2005; Fagiolo, 2007).
For a given entity in G (treated here as undirected), the local clustering coefﬁcient is the fraction of possible triangles through the entity that exist (i.e., how tightly the entity’s neighbors cluster around it).
For entity e i ∈ E , the local clustering coefﬁcient is deﬁned as c i =
Tri ( e i ) /
( deg ( e i )( deg ( e i ) − 1)) , where Tri ( e i ) is the number of triangles through e i , and deg ( e i ) is the degree of e i .
For each relation r ∈ R , let G r be the subgraph of G consisting of all edges in T with r .
That is, G r
=
( E , r, T (cid:48) ) , where T (cid:48)
= { ( e, r, e (cid:48) )
| e, e (cid:48) ∈E} .
Let c r denote the |E| -dimensional clustering coefﬁcient vector for G r , where the i th element of c r is c i .
Then, the mean clustering coefﬁcient vectors for G and G (cid:48) are c o = 1 |R| (cid:80) r ∈R c r and c p = 1 |R (cid:48) | (cid:80) r ∈R (cid:48) c r , respectively.
SC2D is deﬁned as f
[Equation]
[Equation]
2 +1 ∈ [0 , 1] , with higher value indicating higher similarity.
Similarity in Degree Distribution (SD2) SD2 also measures structural similarity between two
KGs, while addressing SC2D’s ineffectiveness when the KGs’ entities have tiny local clustering coefﬁcients (e.g., the item KG used by recommender systems is roughly bipartite).
In such cases,
SC2D is always close to one regardless of perturbation method, thus rendering SC2D useless.
Let d r denote the |E| -dimensional degree vector for G r , where the i th element of d r is deg ( e i ) .
Then, the mean degree vectors for G and G (cid:48) are d o = 1 |R| (cid:80) r ∈R d r and d p = 1 |R (cid:48) | (cid:80) r ∈R (cid:48)
d r , respectively.
SD is deﬁned as f SD2 ( G , G (cid:48) )
= 1 − (cid:107) d o − d p (cid:107) 2 (cid:107)
d o − d p (cid:107) 2 +1 ∈
[0 , 1] , with higher value indicating higher similarity.
M ETHODS FOR T ARGETED KG P ERTURBATION
We aim to study how a KG’s semantics and structure impact KG-augmented models’ downstream performance.
To do so, we measure model performance in response to various forms of targeted
KG perturbation.
While a KG’s semantics can be perturbed via its relation types, its structure can be perturbed via its edge connections.
Therefore, we design ﬁve methods — four heuristic, one RL — for perturbing KG relation types, edge connections, or both (Fig. 2).
4. H EURISTIC -B ASED KG P ERTURBATION
Our four KG perturbation heuristics are as follows:
Relation Swapping (RS) randomly chooses two edges from T and swaps their relations.
Relation Replacement (RR) randomly chooses an edge ( e 1 , r 1 , e 2 ) ∈ T , then replaces r 1 with another relation r 2 =
argmin r ∈R s G ( e 1 , r, e 2 ) .
Edge Rewiring (ER) randomly chooses an edge ( e 1 , r, e 2 ) ∈ T , then replaces e 2 with another entity e 3 ∈ E \ N 1 ( e 1 ) .
Edge Deletion (ED) randomly chooses an edge ( e 1 , r, e 2 ) ∈ T and deletes it.
For ED, perturbing all edges means deleting all but 10 edges.
4. RL-B ASED KG P ERTURBATION
We introduce an RL-based approach for perturbing the KG.
Given a KG, G , we train a policy to output a perturbed KG, G (cid:48) , such that ATS, f ATS ( G , G (cid:48) ) , is minimized, while downstream performance, g ( G (cid:48) ) , is maximized.
Speciﬁcally, the RL agent is trained to perturb G via relation replacement, so we call our algorithm RL-RR .
Because the agent is limited to applying N = |T | perturbations to G , our RL problem is framed as a ﬁnite horizon Markov decision process.
In the rest of this section, we deﬁne the actions, states, and reward in our RL problem, then explain how RL-RR is implemented.
Actions The action space consists of all possible relation replacements in G , i.e., replacing ( e 1 , r, e 2 ) ∈ T with ( e 1 , r (cid:48) , e 2 ) .
Since having such a large action space poses computational issues, we decouple each action into a sequence of three subactions and operate instead in this smaller subaction space.
Hence, a perturbation action at time step t would be a t =
( a (0) t , a (1) t , a (2) t ) .
Namely, a (0) t is sampling entity e 1 ∈ E ; a (1) t is selecting edge ( e 1 , r, e 2 ) ∈ T ; and a (2) t is selecting relation r (cid:48)
∈ R to replace r in ( e 1 , r, e 2 ) .
To make the policy choose low-ATS perturbations, we further restrict the a (2) t subaction space to be the K subactions resulting in the lowest ATS.
Note that each a ( i ) t is represented by its corresponding pre-trained TransE (Bordes et al., 2013) entity, relation, or edge embedding in G .
Since these TransE embeddings are not updated by the perturbation policy, we use a ( i ) t to refer to both the subaction and subaction embedding.
Meanwhile, a t does not have any representation besides its constituent subaction embeddings.
States
The state space is the set of all G (cid:48) with the same entities and connectivity structure as G .
Here, we make a distinction between state and state embedding.
The state at t is the actual KG after t perturbation steps and is denoted as G t .
The state embedding at t is a vector representation of G t and is denoted as s t .
To match a t , we also decouple s t into substate embeddings: s t =
( s (0) t , s (1) t , s (2) t ) .
Reward The reward function pushes the policy to maximize downstream performance.
For commonsense QA, higher reward corresponds to lower KL divergence between the predicted and true answer distributions.
For item recommendation, we use validation AUC as the reward function.
4.2. DQN A RCHITECTURE AND T RAINING
As described above, RL-RR is modeled as an action-subaction hierarchy.
At the action (top) level, for t , the policy selects an action a t given state s t , then performs a t on G t to obtain G t +1 .
At the subaction (bottom) level, for index i ∈ [0 , 1 , 2] within time step t , the policy selects a subaction a ( i +1) t given s ( i ) t and, if any, previous subactions.
At t , the policy takes as input the substate embedding s (0) t .
One approach for computing s (0) t would be to directly encode G t with a graph encoder F graph , such that s (0) t =
F graph ( G t )
(Dai et al., 2018; Sun et al., 2020; Ma et al., 2019b).
However, since we aim to assess graph encoders’ ability to capture
KG information, it would not make sense to use a graph encoder for KG perturbation.
Instead, we use an LSTM (Hochreiter & Schmidhuber, 1997) to update substate embeddings both within and across time steps, while jointly encoding substate and subaction embeddings.
Observe that this means s ( i ) t only implicitly captures KG state information via a ( i − 1) t , since the choice of each subaction is constrained precisely by which entities, relations, or edges are available in G t .
To train RL-RR, we use the DQN algorithm (Mnih et al., 2015).
Abstractly, the goal of DQN is to learn a Q-function Q ( s t , a t ) , which outputs the expected reward for taking action a t in state s t .
In our implementation, Q ( s t , a t ) is decomposed into a sequential pair of sub-Q-functions:
Q 1 ( a (1) t | s (0) t , a (0) t ) =
(cid:104) MLP ( a (1) t ) , MLP ( h (0) t ) (cid:105) and Q 2 ( a (2) t
| s (1) t , a (0) t , a (1) t ) =
(cid:104) MLP ( a (2) t ) , MLP ( h (1) t ) (cid:105) .
MLP denotes the vector representation computed by a multi-layer perceptron, while h (0) t and h (1) t denote the respective LSTM encodings of ( s (0) t , a (0) t ) and ( s (1) t , [ a (0) t ⊕ a (1) t ]) .
Fig.
3 depicts the perturbation procedure at t .
First, we either initialize s (0) t with a trained embedding weight vector if t = 0 , or set it to s (2) t − 1 otherwise.
Second, we uniformly sample a (0) t , which is encoded as h (0) t =
LSTMCell 1 ( s (0) t , a (0) t ) .
LSTMCell 1 also updates s (0) t to s (1) t .
Third, we compute Q 1 ( a (1) t | s (0) t , a (0) t ) , which takes h (0) t as input and outputs a (1) t .
Fourth, we encode a (1) t as h (1) t =
LSTMCell 2 ( s (1) t , [ a (0) t ⊕ a (1) t ]) .
LSTMCell 2 also updates s (1) t to s (2) t .
Fifth, we compute Q 2 ( a (2) t | s (1) t , a (0) t , a (1) t ) , which takes h (1) t as input and outputs a (2) t .
Note that a (1) t and a (2) t are selected (cid:15) -greedily during training and greedily during evaluation.
Finally, using a t = ( a (0) t , a (1) t , a (2) t ) , we perturb G t to get G t +1 .
Ideally, for each t , we would evaluate G t +1 on the downstream task to obtain the reward.
However, downstream evaluation is expensive, so we only compute reward every T time steps.
Moreover, for the policy to generalize well, state embeddings ( s t − T +1 , ..., s t − 1 , s t ) should not correlate with the order of actions ( a t − T +1 , ..., a t − 1 , a t ) .
Thus, for every T time steps during training, we shufﬂe the last T actions after computing reward, then update the LSTM and sub-Q-functions with respect to the shufﬂed actions.
Doing so encourages state embeddings to be invariant to action order.
E XPERIMENTS
In this section, we test KG-augmented models on their ability to maintain performance and explainability when the KG has been extensively perturbed.
As explained in Sec. 2 and Fig.
1, the model is ﬁrst trained on a given dataset using the original KG, frozen throughout KG perturbation, then used to compare downstream performance between original KG and perturbed KG.
For all models, datasets, and perturbation methods, we measure performance and KG similarity when all |T | KG edges have been perturbed, averaged over three runs.
For a subset of model-dataset-perturbation conﬁgurations, we also measure performance as a function of the number of edges perturbed.
In addition, we conduct a user study where humans are asked to rate original and perturbed KGs, with respect to readability and usability for solving downstream tasks.
5. C OMMONSENSE QA
For commonsense QA, the KG-augmented models we experiment with are RN (with attentional path aggregation)
(Lin et al., 2019; Santoro et al., 2017) and MHGRN (Feng et al., 2020), which have been shown to outperform non-KG models (Devlin et al., 2018; Liu et al., 2019) and a number of KG-augmented models (Lin et al., 2019; Ma et al., 2019a; Wang et al., 2019c; Schlichtkrull et al., 2018) on this task.
For both RN and MHGRN, we use a BERT-Base (Devlin et al., 2018) text encoder.
We evaluate on the CommonsenseQA (CSQA)
(Talmor et al., 2018) and OpenBookQA (OBQA)
(Mihaylov et al., 2018) datasets, using ConceptNet (Speer et al., 2016) as the KG.
Performance is measured using accuracy (Acc), which is the standard metric for commonsense QA (Lin et al., 2019; Feng et al., 2020).
[Table] [Table]
CSQA Results for CSQA are given in Table 1.
For RN and MHGRN, we see that RL-RR achieves slightly worse accuracy than Original KG, while RS, RR, and ER perform on par with No KG.
For both models, ED performs noticeably worse than No KG.
OBQA Results for OBQA are shown in Table 2.
For RN, we see that RL-RR actually obtains better accuracy than Original KG.
For MHGRN, RL-RR yields marginally worse accuracy than Original
KG.
Meanwhile, for both RN and MHGRN, all heuristics uniformly achieve similar accuracy as
Original KG, which itself signiﬁcantly outperforms No KG.
Analysis Tables 1-2 demonstrate that perturbing a KG does not necessarily imply decreased performance, nor does it guarantee the creation of invalid or novel facts.
As shown by the KG similarity scores, some perturbation methods cause greater semantic or structural KG changes than others.
Perturbed KGs produced by RS and ED have high ATS
(i
[Table]
e., semantic similarity to original KG), while RR, ER, and RL-RR achieve relatively low ATS.
Meanwhile, SC2D and SD2 are quite low for all perturbation methods, indicating consistently low structural similarity between original and perturbed KG.
RL-RR and RR collectively have the lowest SC2D and SD2 for CSQA, while RLRR has the lowest SC2D and SD2 for OBQA.
Notably, across all perturbation methods and models,
RL-RR attains the highest accuracy while also having the lowest KG similarity scores overall.
The results of a T-test (three runs for both models) show that RL-RR achieves a statistically signiﬁcant improvement over its heuristic counterpart, RR.
Still, even RR has a fairly high accuracy to KG similarity ratio.
This suggests that our KG-augmented models are not using the KG in a human-like way, since RL-RR and RR can both achieve high performance despite extensively corrupting the original KG’s semantic and structural information.
5. I TEM R ECOMMENDATION
The KG-augmented recommender systems we consider are KGCN (Wang et al., 2019b) and RippleNet (Wang et al., 2018b).
We evaluate these models on the Last.
FM (Rendle, 2012) and
MovieLens-20M (Harper & Konstan, 2016) datasets, using the item KG from Wang et al. (2019a).
As mentioned in Sec. 1, item KGs have been shown to beneﬁt recommender systems in cold start scenarios (Wang et al., 2018b).
Therefore, following Wang et al.
(2018b), we simulate a cold start scenario by using only 20% and 40% of the train set for Last.
FM and Movie Lens-20M, respectively.
Performance is measured using AUC, which is the standard metric for item recommendation (Wang et al., 2019b; 2018b).
Since the item KG is almost bipartite, the local clustering coefﬁcient of each item in the KG is extremely small, and so SC2D is not meaningful here (Sec. 3).
Thus, for item recommendation, we do not report SC2D.
[Table] [Table]
Last.
FM Results for Last.
FM are shown in Table 3.
For KGCN and RippleNet, we see that RS, RR, and RL-RR achieve about the same AUC as Original KG, with RL-RR slightly outperforming Original KG.
ER performs similarly to Original KG for KGCN, but considerably worse for RippleNet.
ED’s AUC is on par with No KG’s for KGCN and much lower than No KG’s for RippleNet.
MovieLens-20M Results for MovieLens-20M are displayed in Table 4.
For both KGCN and RippleNet, we ﬁnd that relation-based perturbation methods tend to perform on par with Original KG.
Here, ER is the better of the two edge-based perturbation methods, performing about the same as
Original KG for KGCN, but noticeably worse for RippleNet.
Somehow, for both KGCN and RippleNet, ED achieves even worse AUC than No KG.
On the other hand, we see that ED achieves very high ATS, while RS, RR, ER, and RL-RR achieve more modest ATS scores.
Analysis Like in commonsense QA, Tables 3-4 show that KG-augmented models can perform well even when the KG has been drastically perturbed.
Using the T-test with three runs, for almost all perturbation methods, we ﬁnd a statistically insigniﬁcant difference between the perturbed KG’s
AUC and the original KG’s AUC.
The perturbed KGs produced by ED have high ATS, while RS,
RR, ER, and RL-RR achieve modest ATS scores.
However, all perturbation methods have fairly low
SD2 (except RR on Last.
FM).
In particular, across both datasets and models, RL-RR has the highest
AUC overall, while also having the lowest KG similarity scores overall.
This serves as additional evidence that the model is not using the KG in a human-like manner, since RL-RR achieves high performance despite signiﬁcantly perturbing the original KG’s semantic and structural information.
5. A UXILIARY E XPERIMENTS AND A NALYSIS
Varying Perturbation Level For a subset of model-dataset-perturbation settings, we measure the performance and ATS of various perturbation methods as a function of the percentage of KG edges perturbed.
For MHGRN on CSQA, Fig.
4a shows that, across all levels of perturbation, RL-RR maintains higher accuracy than No KG.
Meanwhile, RS’s accuracy reaches No KG’s accuracy at 100% perturbation, and RR’s does so at 60% perturbation.
In Fig.
4b, we see that RL-RR’s and
RR’s ATS drop signiﬁcantly as the perturbation percentage increases, whereas RS’s ATS remains quite high even at 100% perturbation.
For RippleNet on MovieLens-20M, Fig.
4c shows a ﬂat performance curve for all perturbation methods.
Meanwhile, for all perturbation methods in Fig.
4d, ATS decreases steadily as perturbation level increases, with RL-RR’s ATS dropping most.
These ﬁndings support the hypothesis that KG perturbation does not imply performance decrease or KG corruption.
Building on the results of previous experiments, in both model-dataset settings,
RL-RR largely maintains the model’s performance despite also heavily perturbing the KG’s seman-
[Figure]
[Figure]
Noisy baseline accuracy on CSQA and OBQA.
[Table] [Table]
tics.
Interestingly, for RippleNet on MovieLens-20M, performance is completely unaffected by KG perturbation, even though the KG’s semantic information is apparently being corrupted.
Noisy Baselines To see if KGs yielded by our perturbation methods capture more than just random noise, we compare them to several noisy baselines.
Table 5 gives results for three noisy baselines on commonsense QA: (1) replace subgraph embedding with zero vector, (2) replace subgraph embedding with random vector, and (3) replace entity/relation embeddings with random vectors.
For
CSQA, the noisy baselines perform noticeably worse than both Original KG and RL-RR, while being on par with No KG (Table 1).
For OBQA, the noisy baselines’ perform slightly better than No
KG, but considerably worse than Original KG and all of the perturbation methods (Table 2).
Table 6 displays results for our noisy baseline in item recommendation, which entails randomizing each entity’s neighborhood.
We ﬁnd that KGCN performs about the same for this noisy baseline as for
Original KG and our best perturbation methods, whereas RippleNet performs much worse (Tables 3-4).
RippleNet may be more sensitive than KGCN to entity neighbor randomization because RippleNet considers directed edges.
This is supported by RippleNet’s performance dropping when we perturb edge connections (Tables 3-4).
In both tasks, the noisy baselines show that our perturbation methods yield KGs that capture measurably useful information beyond just noise.
For KGCN, the unexpected discovery that noisy baselines perform similarly to Original KG suggest that even noisy
KGs can contain useful information for KG-augmented models.
[Table]
Human Evaluation of KG Explanations We conduct a user study to measure KG-augmented models’ ability to give plausible explanations, using the original KG or RLRR perturbed KG.
For both KGs, we sample 30 questions from the CSQA and OBQA test sets which were correctly answered by MHGRN.
For each question, we retrieve the top-scoring path for each answer choice via MHGRN’s path decoder attention.
We then ask three human subjects to rate each path for readability and usability, with ratings aggregated via majority voting.
Readability (Read) is whether the path makes sense, usability (Use) is whether the path is relevant to the given question-answer pair, and both are measured on a [0 , 1] scale.
We obtain a Fleiss’ κ of 0 .
1891 , indicating slight agreement between raters.
To illustrate, we provide examples of explanation paths and their consensus ratings.
Given the question James chose to not to print the cards, because he wanted to be more personal.
What type of cards did he choose, instead? , the Original KG path is P RINT —
[A NTONYM ] → H ANDWRITTEN (Read= 1 . 0 ; Use= 2 . 0 ), and the RL-RR path is P RINT — [N OT D ESIRES ]
→ H ANDWRITTEN (Read= 0 . 0 ;
Use= 0 . 0 ).
Here, the Original KG path seems plausible, but the RL-RR path does not.
In Table 7, we see that Original KG and RL-RR got relatively low ratings for both readability and usability.
Whereas MHGRN successfully utilizes all KG paths in this user study, humans largely struggle to read or use them.
This suggests that KG-augmented models and humans process KG information quite differently, thus challenging the role of KG paths as plausible explanations.
However, RL-RR also causes a big drop in readability on OBQA, signaling RL-RR’s ability to corrupt the KG.
CSQA’s much smaller readability decrease can be explained by the fact that CSQA is con- structed from ConceptNet.
Every CSQA question-answer is based on ConceptNet entities/relations, so a random ConceptNet subgraph is more likely to have semantic overlap with a CSQA questionanswer than with an OBQA question-answer.
Hence, a perturbed ConceptNet subgraph may also be more likely to overlap with a CSQA question-answer, which means perturbing the KG might have a smaller impact on human judgments of CSQA path readability.
Note that this result regards explainability and does not say anything about the model’s performance on CSQA and OBQA.
Validation of KG Similarity Metrics Using our human evaluation results, we validate our three proposed KG similarity metrics: ATS, SC2D and SD2.
We ﬁnd that the Pearson correlation coefﬁcient between the human evaluation scores in Table 7 and the three KG similarity scores in Tables 1-2 are 0 .
932 and 0 . 932 , respectively.
This indicates high correlation and that our metrics aptly capture a perturbed KG’s preservation of semantic/structural information from its original KG.
Why do perturbed KGs sometimes perform better than the original KG?
In our experiments, relation-based perturbations (RS, RR, RL-RR) generally outperform edge-based perturbations (ER,
ED).
Also, we ﬁnd that the original KG can contain noisy relation annotations which are sometimes “corrected” by relation-based perturbations.
In certain cases, this may result in the perturbed KG achieving slightly higher performance than the original KG (RR and RL-RR for RN-CSQA; RL-RR for Last.
FM).
Similarly, in our user study, despite all questions being correctly answered by the model, there were some RL-RR explanations that received higher readability/usability ratings than their original KG counterparts.
Although the original KG achieved higher human ratings than the
RL-RR KG did overall, both KGs still achieved relatively low ratings with respect to our scales.
While our main argument centers on KG-augmented models’ ﬂaws, this counterintuitive ﬁnding suggests that KGs themselves are ﬂawed too, but in a way that can be systematically corrected
. R ELATED W ORK KG-Augmented Neural Models
Although neural models may already capture some semantic knowledge (Petroni et al., 2019; Davison et al., 2019), augmenting them with external KGs has improved performance on various downstream tasks: commonsense QA (Lin et al., 2019;
Shen et al., 2020;
Lv et al., 2020; Musa et al., 2019), item recommendation (Wang et al., 2019b; 2020;
Song et al., 2019; Cao et al., 2019), natural language inference (Chen et al., 2017;
Wang et al., 2019c), and others (Chen et al., 2019; Kapanipathi et al.).
KG-augmented models have also been designed to explain the model’s predictions via attention over the KG (Lin et al., 2019;
Zhang et al., 2019; Song et al., 2019; Cao et al., 2019; Gao et al., 2019; Ai et al., 2018).
Adversarial Perturbation of Graphs Inspired by adversarial learning in computer vision (Bhambri et al., 2019) and NLP (Zhang et al., 2020), some recent works have addressed adversarial perturbation in graph learning (Chen et al., 2020).
Multiple paradigms have been proposed for graph perturbation, including gradient-based methods (Chen et al., 2018; Bojchevski & G ¨ unnemann, 2019;
Wu et al., 2019), RL-based methods (Ma et al., 2019b; Dai et al., 2018), and autoencoder-based methods (Chen et al., 2018).
Whereas such works aim to minimally perturb the graph while maximally impacting the graph’s performance, our purpose for graph perturbation is to see whether
KG-augmented models’ use KGs in a human-like way and provide plausible explanations.
C ONCLUSION
In this paper, we analyze the effects of strategically perturbed KGs on KG-augmented model predictions.
Using four heuristics and a RL policy, we show that KGs can be perturbed in way that drastically changes their semantics and structure, while preserving the model’s downstream performance.
Apparently, KG-augmented models can process KG information in a way that does not align with human priors about KGs, although the nature of this process still requires further investigation.
Moreover, we conduct a user study to demonstrate that both perturbed and unperturbed KGs struggle to facilitate plausible explanations of the model’s predictions.
Note that our proposed KG perturbation methods merely serve as analytical tools and are not intended to directly improve model performance or explainability.
Nonetheless, we believe our ﬁndings can guide future work on designing KG-augmented models that are better in these aspects.
Additionally, our results suggest that
KG-augmented models can be robust to noisy KG data.
Even when the KG contains a fairly small amount of signal, these models are somehow able to leverage it.
This could be useful in situations where it is impractical to obtain fully clean KG annotations.
A CKNOWLEDGMENTS
This research is supported in part by the Ofﬁce of the Director of National Intelligence (ODNI),
Intelligence Advanced Research Projects Activity (IARPA), via Contract No. 2019-19051600007, the DARPA MCS program under Contract No. N660011924033 with the United States Ofﬁce
Of
Naval Research, the Defense Advanced Research Projects Agency with award W911NF-19-20271, and NSF SMA 18-29268.
We would like to thank all collaborators in the USC INK Research Lab for their constructive feedback on the work