Introduction
Knowledge graphs (KG) play a critical role in many real-world applications such as search, structured data management, recommendations, and question answering. Since KGs often suffer from incompleteness and noise in their facts (links), a number of recent techniques have proposed models that embed each entity and relation into a vector space, and use these embeddings to predict facts. These dense representation models for link prediction include tensor factorization BIBREF0 , BIBREF1 , BIBREF2 , algebraic operations BIBREF3 , BIBREF4 , BIBREF5 , multiple embeddings BIBREF6 , BIBREF7 , BIBREF8 , BIBREF9 , and complex neural models BIBREF10 , BIBREF11 . However, there are only a few studies BIBREF12 , BIBREF13 that investigate the quality of the different KG models. There is a need to go beyond just the accuracy on link prediction, and instead focus on whether these representations are robust and stable, and what facts they make use of for their predictions. In this paper, our goal is to design approaches that minimally change the graph structure such that the prediction of a target fact changes the most after the embeddings are relearned, which we collectively call Completion Robustness and Interpretability via Adversarial Graph Edits (). First, we consider perturbations that red!50!blackremove a neighboring link for the target fact, thus identifying the most influential related fact, providing an explanation for the model's prediction. As an example, consider the excerpt from a KG in Figure 1 with two observed facts, and a target predicted fact that Princes Henriette is the parent of Violante Bavaria. Our proposed graph perturbation, shown in Figure 1 , identifies the existing fact that Ferdinal Maria is the father of Violante Bavaria as the one when removed and model retrained, will change the prediction of Princes Henriette's child. We also study attacks that green!50!blackadd a new, fake fact into the KG to evaluate the robustness and sensitivity of link prediction models to small additions to the graph. An example attack for the original graph in Figure 1 , is depicted in Figure 1 . Such perturbations to the the training data are from a family of adversarial modifications that have been applied to other machine learning tasks, known as poisoning BIBREF14 , BIBREF15 , BIBREF16 , BIBREF17 .
Since the setting is quite different from traditional adversarial attacks, search for link prediction adversaries brings up unique challenges. To find these minimal changes for a target link, we need to identify the fact that, when added into or removed from the graph, will have the biggest impact on the predicted score of the target fact. Unfortunately, computing this change in the score is expensive since it involves retraining the model to recompute the embeddings. We propose an efficient estimate of this score change by approximating the change in the embeddings using Taylor expansion. The other challenge in identifying adversarial modifications for link prediction, especially when considering addition of fake facts, is the combinatorial search space over possible facts, which is intractable to enumerate. We introduce an inverter of the original embedding model, to decode the embeddings to their corresponding graph components, making the search of facts tractable by performing efficient gradient-based continuous optimization. We evaluate our proposed methods through following experiments. First, on relatively small KGs, we show that our approximations are accurate compared to the true change in the score. Second, we show that our additive attacks can effectively reduce the performance of state of the art models BIBREF2 , BIBREF10 up to $27.3\%$ and $50.7\%$ in Hits@1 for two large KGs: WN18 and YAGO3-10. We also explore the utility of adversarial modifications in explaining the model predictions by presenting rule-like descriptions of the most influential neighbors. Finally, we use adversaries to detect errors in the KG, obtaining up to $55\%$ accuracy in detecting errors.
Background and Notation
In this section, we briefly introduce some notations, and existing relational embedding approaches that model knowledge graph completion using dense vectors. In KGs, facts are represented using triples of subject, relation, and object, $\langle s, r, o\rangle $ , where $s,o\in \xi $ , the set of entities, and $r\in $ , the set of relations. To model the KG, a scoring function $\psi :\xi \times \times \xi \rightarrow $ is learned to evaluate whether any given fact is true. In this work, we focus on multiplicative models of link prediction, specifically DistMult BIBREF2 because of its simplicity and popularity, and ConvE BIBREF10 because of its high accuracy. We can represent the scoring function of such methods as $\psi (s,r,o) = , ) \cdot $ , where $,,\in ^d$ are embeddings of the subject, relation, and object respectively. In DistMult, $, ) = \odot $ , where $\odot $ is element-wise multiplication operator. Similarly, in ConvE, $, )$ is computed by a convolution on the concatenation of $$ and $s,o\in \xi $0 .
We use the same setup as BIBREF10 for training, i.e., incorporate binary cross-entropy loss over the triple scores. In particular, for subject-relation pairs $(s,r)$ in the training data $G$ , we use binary $y^{s,r}_o$ to represent negative and positive facts. Using the model's probability of truth as $\sigma (\psi (s,r,o))$ for $\langle s,r,o\rangle $ , the loss is defined as: (G) = (s,r)o ys,ro(((s,r,o)))
+ (1-ys,ro)(1 - ((s,r,o))). Gradient descent is used to learn the embeddings $,,$ , and the parameters of $, if any. $
Completion Robustness and Interpretability via Adversarial Graph Edits ()
For adversarial modifications on KGs, we first define the space of possible modifications. For a target triple $\langle s, r, o\rangle $ , we constrain the possible triples that we can remove (or inject) to be in the form of $\langle s^{\prime }, r^{\prime }, o\rangle $ i.e $s^{\prime }$ and $r^{\prime }$ may be different from the target, but the object is not. We analyze other forms of modifications such as $\langle s, r^{\prime }, o^{\prime }\rangle $ and $\langle s, r^{\prime }, o\rangle $ in appendices "Modifications of the Form 〈s,r ' ,o ' 〉\langle s, r^{\prime }, o^{\prime } \rangle " and "Modifications of the Form 〈s,r ' ,o〉\langle s, r^{\prime }, o \rangle " , and leave empirical evaluation of these modifications for future work.
Removing a fact ()
For explaining a target prediction, we are interested in identifying the observed fact that has the most influence (according to the model) on the prediction. We define influence of an observed fact on the prediction as the change in the prediction score if the observed fact was not present when the embeddings were learned. Previous work have used this concept of influence similarly for several different tasks BIBREF19 , BIBREF20 . Formally, for the target triple ${s,r,o}$ and observed graph $G$ , we want to identify a neighboring triple ${s^{\prime },r^{\prime },o}\in G$ such that the score $\psi (s,r,o)$ when trained on $G$ and the score $\overline{\psi }(s,r,o)$ when trained on $G-\lbrace {s^{\prime },r^{\prime },o}\rbrace $ are maximally different, i.e. *argmax(s', r')Nei(o) (s',r')(s,r,o) where $\Delta _{(s^{\prime },r^{\prime })}(s,r,o)=\psi (s, r, o)-\overline{\psi }(s,r,o)$ , and $\text{Nei}(o)=\lbrace (s^{\prime },r^{\prime })|\langle s^{\prime },r^{\prime },o \rangle \in G \rbrace $ .
Adding a new fact ()
We are also interested in investigating the robustness of models, i.e., how sensitive are the predictions to small additions to the knowledge graph. Specifically, for a target prediction ${s,r,o}$ , we are interested in identifying a single fake fact ${s^{\prime },r^{\prime },o}$ that, when added to the knowledge graph $G$ , changes the prediction score $\psi (s,r,o)$ the most. Using $\overline{\psi }(s,r,o)$ as the score after training on $G\cup \lbrace {s^{\prime },r^{\prime },o}\rbrace $ , we define the adversary as: *argmax(s', r') (s',r')(s,r,o) where $\Delta _{(s^{\prime },r^{\prime })}(s,r,o)=\psi (s, r, o)-\overline{\psi }(s,r,o)$ . The search here is over any possible $s^{\prime }\in \xi $ , which is often in the millions for most real-world KGs, and $r^{\prime }\in $ . We also identify adversaries that increase the prediction score for specific false triple, i.e., for a target fake fact ${s,r,o}$ , the adversary is ${s^{\prime },r^{\prime },o}$0 , where ${s^{\prime },r^{\prime },o}$1 is defined as before.
Challenges
There are a number of crucial challenges when conducting such adversarial attack on KGs. First, evaluating the effect of changing the KG on the score of the target fact ( $\overline{\psi }(s,r,o)$ ) is expensive since we need to update the embeddings by retraining the model on the new graph; a very time-consuming process that is at least linear in the size of $G$ . Second, since there are many candidate facts that can be added to the knowledge graph, identifying the most promising adversary through search-based methods is also expensive. Specifically, the search size for unobserved facts is $|\xi | \times ||$ , which, for example in YAGO3-10 KG, can be as many as $4.5 M$ possible facts for a single target prediction.
Efficiently Identifying the Modification
In this section, we propose algorithms to address mentioned challenges by (1) approximating the effect of changing the graph on a target prediction, and (2) using continuous optimization for the discrete search over potential modifications.
First-order Approximation of Influence
We first study the addition of a fact to the graph, and then extend it to cover removal as well. To capture the effect of an adversarial modification on the score of a target triple, we need to study the effect of the change on the vector representations of the target triple. We use $$ , $$ , and $$ to denote the embeddings of $s,r,o$ at the solution of $\operatornamewithlimits{argmin} (G)$ , and when considering the adversarial triple $\langle s^{\prime }, r^{\prime }, o \rangle $ , we use $$ , $$ , and $$ for the new embeddings of $s,r,o$ , respectively. Thus $$0 is a solution to $$1 , which can also be written as $$2 . Similarly, $$3 s', r', o $$4 $$5 $$6 $$7 o $$8 $$9 $$0 $$1 $$2 $$3 O(n3) $$4 $$5 $$6 (s,r,o)-(s, r, o) $$7 - $$8 s, r = ,) $$9 - $s,r,o$0 (G)= (G)+(s', r', o ) $s,r,o$1 $s,r,o$2 s', r' = ',') $s,r,o$3 = ((s',r',o)) $s,r,o$4 eo (G)=0 $s,r,o$5 eo (G) $s,r,o$6 Ho $s,r,o$7 dd $s,r,o$8 o $s,r,o$9 $\operatornamewithlimits{argmin} (G)$0 - $\operatornamewithlimits{argmin} (G)$1 -= $\operatornamewithlimits{argmin} (G)$2 Ho $\operatornamewithlimits{argmin} (G)$3 Ho + (1-) s',r's',r' $\operatornamewithlimits{argmin} (G)$4 Ho $\operatornamewithlimits{argmin} (G)$5 dd $\operatornamewithlimits{argmin} (G)$6 d $\operatornamewithlimits{argmin} (G)$7 s,r,s',r'd $\operatornamewithlimits{argmin} (G)$8 s, r, o $\operatornamewithlimits{argmin} (G)$9 s', r', o $\langle s^{\prime }, r^{\prime }, o \rangle $0 $\langle s^{\prime }, r^{\prime }, o \rangle $1 $\langle s^{\prime }, r^{\prime }, o \rangle $2
Continuous Optimization for Search
Using the approximations provided in the previous section, Eq. () and (), we can use brute force enumeration to find the adversary $\langle s^{\prime }, r^{\prime }, o \rangle $ . This approach is feasible when removing an observed triple since the search space of such modifications is usually small; it is the number of observed facts that share the object with the target. On the other hand, finding the most influential unobserved fact to add requires search over a much larger space of all possible unobserved facts (that share the object). Instead, we identify the most influential unobserved fact $\langle s^{\prime }, r^{\prime }, o \rangle $ by using a gradient-based algorithm on vector $_{s^{\prime },r^{\prime }}$ in the embedding space (reminder, $_{s^{\prime },r^{\prime }}=^{\prime },^{\prime })$ ), solving the following continuous optimization problem in $^d$ : *argmaxs', r' (s',r')(s,r,o). After identifying the optimal $_{s^{\prime }, r^{\prime }}$ , we still need to generate the pair $(s^{\prime },r^{\prime })$ . We design a network, shown in Figure 2 , that maps the vector $_{s^{\prime },r^{\prime }}$ to the entity-relation space, i.e., translating it into $(s^{\prime },r^{\prime })$ . In particular, we train an auto-encoder where the encoder is fixed to receive the $s$ and $\langle s^{\prime }, r^{\prime }, o \rangle $0 as one-hot inputs, and calculates $\langle s^{\prime }, r^{\prime }, o \rangle $1 in the same way as the DistMult and ConvE encoders respectively (using trained embeddings). The decoder is trained to take $\langle s^{\prime }, r^{\prime }, o \rangle $2 as input and produce $\langle s^{\prime }, r^{\prime }, o \rangle $3 and $\langle s^{\prime }, r^{\prime }, o \rangle $4 , essentially inverting $\langle s^{\prime }, r^{\prime }, o \rangle $5 s, r $\langle s^{\prime }, r^{\prime }, o \rangle $6 s $\langle s^{\prime }, r^{\prime }, o \rangle $7 r $\langle s^{\prime }, r^{\prime }, o \rangle $8 s, r $\langle s^{\prime }, r^{\prime }, o \rangle $9 We evaluate the performance of our inverter networks (one for each model/dataset) on correctly recovering the pairs of subject and relation from the test set of our benchmarks, given the $_{s^{\prime },r^{\prime }}$0 . The accuracy of recovered pairs (and of each argument) is given in Table 1 . As shown, our networks achieve a very high accuracy, demonstrating their ability to invert vectors $_{s^{\prime },r^{\prime }}$1 to $_{s^{\prime },r^{\prime }}$2 pairs.
Experiments
We evaluate by ( "Influence Function vs " ) comparing estimate with the actual effect of the attacks, ( "Robustness of Link Prediction Models" ) studying the effect of adversarial attacks on evaluation metrics, ( "Interpretability of Models" ) exploring its application to the interpretability of KG representations, and ( "Finding Errors in Knowledge Graphs" ) detecting incorrect triples.
Influence Function vs 
To evaluate the quality of our approximations and compare with influence function (IF), we conduct leave one out experiments. In this setup, we take all the neighbors of a random target triple as candidate modifications, remove them one at a time, retrain the model each time, and compute the exact change in the score of the target triple. We can use the magnitude of this change in score to rank the candidate triples, and compare this exact ranking with ranking as predicted by: , influence function with and without Hessian matrix, and the original model score (with the intuition that facts that the model is most confident of will have the largest impact when removed). Similarly, we evaluate by considering 200 random triples that share the object entity with the target sample as candidates, and rank them as above. The average results of Spearman's $\rho $ and Kendall's $\tau $ rank correlation coefficients over 10 random target samples is provided in Table 3 . performs comparably to the influence function, confirming that our approximation is accurate. Influence function is slightly more accurate because they use the complete Hessian matrix over all the parameters, while we only approximate the change by calculating the Hessian over $$ . The effect of this difference on scalability is dramatic, constraining IF to very small graphs and small embedding dimensionality ( $d\le 10$ ) before we run out of memory. In Figure 3 , we show the time to compute a single adversary by IF compared to , as we steadily grow the number of entities (randomly chosen subgraphs), averaged over 10 random triples. As it shows, is mostly unaffected by the number of entities while IF increases quadratically. Considering that real-world KGs have tens of thousands of times more entities, making IF unfeasible for them.
Robustness of Link Prediction Models
Now we evaluate the effectiveness of to successfully attack link prediction by adding false facts. The goal here is to identify the attacks for triples in the test data, and measuring their effect on MRR and Hits@ metrics (ranking evaluations) after conducting the attack and retraining the model.
Since this is the first work on adversarial attacks for link prediction, we introduce several baselines to compare against our method. For finding the adversarial fact to add for the target triple $\langle s, r, o \rangle $ , we consider two baselines: 1) choosing a random fake fact $\langle s^{\prime }, r^{\prime }, o \rangle $ (Random Attack); 2) finding $(s^{\prime }, r^{\prime })$ by first calculating $, )$ and then feeding $-, )$ to the decoder of the inverter function (Opposite Attack). In addition to , we introduce two other alternatives of our method: (1) , that uses to increase the score of fake fact over a test triple, i.e., we find the fake fact the model ranks second after the test triple, and identify the adversary for them, and (2) that selects between and attacks based on which has a higher estimated change in score.
All-Test The result of the attack on all test facts as targets is provided in the Table 4 . outperforms the baselines, demonstrating its ability to effectively attack the KG representations. It seems DistMult is more robust against random attacks, while ConvE is more robust against designed attacks. is more effective than since changing the score of a fake fact is easier than of actual facts; there is no existing evidence to support fake facts. We also see that YAGO3-10 models are more robust than those for WN18. Looking at sample attacks (provided in Appendix "Sample Adversarial Attacks" ), mostly tries to change the type of the target object by associating it with a subject and a relation for a different entity type.
Uncertain-Test To better understand the effect of attacks, we consider a subset of test triples that 1) the model predicts correctly, 2) difference between their scores and the negative sample with the highest score is minimum. This “Uncertain-Test” subset contains 100 triples from each of the original test sets, and we provide results of attacks on this data in Table 4 . The attacks are much more effective in this scenario, causing a considerable drop in the metrics. Further, in addition to significantly outperforming other baselines, they indicate that ConvE's confidence is much more robust.
Relation Breakdown We perform additional analysis on the YAGO3-10 dataset to gain a deeper understanding of the performance of our model. As shown in Figure 4 , both DistMult and ConvE provide a more robust representation for isAffiliatedTo and isConnectedTo relations, demonstrating the confidence of models in identifying them. Moreover, the affects DistMult more in playsFor and isMarriedTo relations while affecting ConvE more in isConnectedTo relations.
Examples Sample adversarial attacks are provided in Table 5 . attacks mostly try to change the type of the target triple's object by associating it with a subject and a relation that require a different entity types.
Interpretability of Models
To be able to understand and interpret why a link is predicted using the opaque, dense embeddings, we need to find out which part of the graph was most influential on the prediction. To provide such explanations for each predictions, we identify the most influential fact using . Instead of focusing on individual predictions, we aggregate the explanations over the whole dataset for each relation using a simple rule extraction technique: we find simple patterns on subgraphs that surround the target triple and the removed fact from , and appear more than $90\%$ of the time. We only focus on extracting length-2 horn rules, i.e., $R_1(a,c)\wedge R_2(c,b)\Rightarrow R(a,b)$ , where $R(a,b)$ is the target and $R_2(c,b)$ is the removed fact. Table 6 shows extracted YAGO3-10 rules that are common to both models, and ones that are not. The rules show several interesting inferences, such that hasChild is often inferred via married parents, and isLocatedIn via transitivity. There are several differences in how the models reason as well; DistMult often uses the hasCapital as an intermediate step for isLocatedIn, while ConvE incorrectly uses isNeighbor. We also compare against rules extracted by BIBREF2 for YAGO3-10 that utilizes the structure of DistMult: they require domain knowledge on types and cannot be applied to ConvE. Interestingly, the extracted rules contain all the rules provided by , demonstrating that can be used to accurately interpret models, including ones that are not interpretable, such as ConvE. These are preliminary steps toward interpretability of link prediction models, and we leave more analysis of interpretability to future work.
Finding Errors in Knowledge Graphs
Here, we demonstrate another potential use of adversarial modifications: finding erroneous triples in the knowledge graph. Intuitively, if there is an error in the graph, the triple is likely to be inconsistent with its neighborhood, and thus the model should put least trust on this triple. In other words, the error triple should have the least influence on the model's prediction of the training data. Formally, to find the incorrect triple $\langle s^{\prime }, r^{\prime }, o\rangle $ in the neighborhood of the train triple $\langle s, r, o\rangle $ , we need to find the triple $\langle s^{\prime },r^{\prime },o\rangle $ that results in the least change $\Delta _{(s^{\prime },r^{\prime })}(s,r,o)$ when removed from the graph.
To evaluate this application, we inject random triples into the graph, and measure the ability of to detect the errors using our optimization. We consider two types of incorrect triples: 1) incorrect triples in the form of $\langle s^{\prime }, r, o\rangle $ where $s^{\prime }$ is chosen randomly from all of the entities, and 2) incorrect triples in the form of $\langle s^{\prime }, r^{\prime }, o\rangle $ where $s^{\prime }$ and $r^{\prime }$ are chosen randomly. We choose 100 random triples from the observed graph, and for each of them, add an incorrect triple (in each of the two scenarios) to its neighborhood. Then, after retraining DistMult on this noisy training data, we identify error triples through a search over the neighbors of the 100 facts. The result of choosing the neighbor with the least influence on the target is provided in the Table 7 . When compared with baselines that randomly choose one of the neighbors, or assume that the fact with the lowest score is incorrect, we see that outperforms both of these with a considerable gap, obtaining an accuracy of $42\%$ and $55\%$ in detecting errors.
Related Work
Learning relational knowledge representations has been a focus of active research in the past few years, but to the best of our knowledge, this is the first work on conducting adversarial modifications on the link prediction task. Knowledge graph embedding There is a rich literature on representing knowledge graphs in vector spaces that differ in their scoring functions BIBREF21 , BIBREF22 , BIBREF23 . Although is primarily applicable to multiplicative scoring functions BIBREF0 , BIBREF1 , BIBREF2 , BIBREF24 , these ideas apply to additive scoring functions BIBREF18 , BIBREF6 , BIBREF7 , BIBREF25 as well, as we show in Appendix "First-order Approximation of the Change For TransE" .
Furthermore, there is a growing body of literature that incorporates an extra types of evidence for more informed embeddings such as numerical values BIBREF26 , images BIBREF27 , text BIBREF28 , BIBREF29 , BIBREF30 , and their combinations BIBREF31 . Using , we can gain a deeper understanding of these methods, especially those that build their embeddings wit hmultiplicative scoring functions.
Interpretability and Adversarial Modification There has been a significant recent interest in conducting an adversarial attacks on different machine learning models BIBREF16 , BIBREF32 , BIBREF33 , BIBREF34 , BIBREF35 , BIBREF36 to attain the interpretability, and further, evaluate the robustness of those models. BIBREF20 uses influence function to provide an approach to understanding black-box models by studying the changes in the loss occurring as a result of changes in the training data. In addition to incorporating their established method on KGs, we derive a novel approach that differs from their procedure in two ways: (1) instead of changes in the loss, we consider the changes in the scoring function, which is more appropriate for KG representations, and (2) in addition to searching for an attack, we introduce a gradient-based method that is much faster, especially for “adding an attack triple” (the size of search space make the influence function method infeasible). Previous work has also considered adversaries for KGs, but as part of training to improve their representation of the graph BIBREF37 , BIBREF38 . Adversarial Attack on KG Although this is the first work on adversarial attacks for link prediction, there are two approaches BIBREF39 , BIBREF17 that consider the task of adversarial attack on graphs. There are a few fundamental differences from our work: (1) they build their method on top of a path-based representations while we focus on embeddings, (2) they consider node classification as the target of their attacks while we attack link prediction, and (3) they conduct the attack on small graphs due to restricted scalability, while the complexity of our method does not depend on the size of the graph, but only the neighborhood, allowing us to attack real-world graphs.
Conclusions
Motivated by the need to analyze the robustness and interpretability of link prediction models, we present a novel approach for conducting adversarial modifications to knowledge graphs. We introduce , completion robustness and interpretability via adversarial graph edits: identifying the fact to add into or remove from the KG that changes the prediction for a target fact. uses (1) an estimate of the score change for any target triple after adding or removing another fact, and (2) a gradient-based algorithm for identifying the most influential modification. We show that can effectively reduce ranking metrics on link prediction models upon applying the attack triples. Further, we incorporate the to study the interpretability of KG representations by summarizing the most influential facts for each relation. Finally, using , we introduce a novel automated error detection method for knowledge graphs. We have release the open-source implementation of our models at: https://pouyapez.github.io/criage.
Acknowledgements
We would like to thank Matt Gardner, Marco Tulio Ribeiro, Zhengli Zhao, Robert L. Logan IV, Dheeru Dua and the anonymous reviewers for their detailed feedback and suggestions. This work is supported in part by Allen Institute for Artificial Intelligence (AI2) and in part by NSF awards #IIS-1817183 and #IIS-1756023. The views expressed are those of the authors and do not reflect the official policy or position of the funding agencies.
Appendix
We approximate the change on the score of the target triple upon applying attacks other than the $\langle s^{\prime }, r^{\prime }, o \rangle $ ones. Since each relation appears many times in the training triples, we can assume that applying a single attack will not considerably affect the relations embeddings. As a result, we just need to study the attacks in the form of $\langle s, r^{\prime }, o \rangle $ and $\langle s, r^{\prime }, o^{\prime } \rangle $ . Defining the scoring function as $\psi (s,r,o) = , ) \cdot = _{s,r} \cdot $ , we further assume that $\psi (s,r,o) =\cdot (, ) =\cdot _{r,o}$ .
Modifications of the Form 〈s,r ' ,o ' 〉\langle s, r^{\prime }, o^{\prime } \rangle 
Using similar argument as the attacks in the form of $\langle s^{\prime }, r^{\prime }, o \rangle $ , we can calculate the effect of the attack, $\overline{\psi }{(s,r,o)}-\psi (s, r, o)$ as: (s,r,o)-(s, r, o)=(-) s, r where $_{s, r} = (,)$ .
We now derive an efficient computation for $(-)$ . First, the derivative of the loss $(\overline{G})= (G)+(\langle s, r^{\prime }, o^{\prime } \rangle )$ over $$ is: es (G) = es (G) - (1-) r', o' where $_{r^{\prime }, o^{\prime }} = (^{\prime },^{\prime })$ , and $\varphi = \sigma (\psi (s,r^{\prime },o^{\prime }))$ . At convergence, after retraining, we expect $\nabla _{e_s} (\overline{G})=0$ . We perform first order Taylor approximation of $\nabla _{e_s} (\overline{G})$ to get: 0 - (1-)r',o'+
(Hs+(1-)r',o' r',o')(-) where $H_s$ is the $d\times d$ Hessian matrix for $s$ , i.e. second order derivative of the loss w.r.t. $$ , computed sparsely. Solving for $-$ gives us: -=
(1-) (Hs + (1-) r',o'r',o')-1 r',o' In practice, $H_s$ is positive definite, making $H_s + \varphi (1-\varphi ) _{r^{\prime },o^{\prime }}^\intercal _{r^{\prime },o^{\prime }}$ positive definite as well, and invertible. Then, we compute the score change as: (s,r,o)-(s, r, o)= r,o (-) =
((1-) (Hs + (1-) r',o'r',o')-1 r',o')r,o.
Modifications of the Form 〈s,r ' ,o〉\langle s, r^{\prime }, o \rangle 
In this section we approximate the effect of attack in the form of $\langle s, r^{\prime }, o \rangle $ . In contrast to $\langle s^{\prime }, r^{\prime }, o \rangle $ attacks, for this scenario we need to consider the change in the $$ , upon applying the attack, in approximation of the change in the score as well. Using previous results, we can approximate the $-$ as: -=
(1-) (Ho + (1-) s,r's,r')-1 s,r' and similarly, we can approximate $-$ as: -=
(1-) (Hs + (1-) r',or',o)-1 r',o where $H_s$ is the Hessian matrix over $$ . Then using these approximations: s,r(-) =
s,r ((1-) (Ho + (1-) s,r's,r')-1 s,r') and: (-) r,o=
((1-) (Hs + (1-) r',or',o)-1 r',o) r,o and then calculate the change in the score as: (s,r,o)-(s, r, o)=
s,r.(-) +(-).r,o =
s,r ((1-) (Ho + (1-) s,r's,r')-1 s,r')+
((1-) (Hs + (1-) r',or',o)-1 r',o) r, o
First-order Approximation of the Change For TransE
In here we derive the approximation of the change in the score upon applying an adversarial modification for TransE BIBREF18 . Using similar assumptions and parameters as before, to calculate the effect of the attack, $\overline{\psi }{(s,r,o)}$ (where $\psi {(s,r,o)}=|+-|$ ), we need to compute $$ . To do so, we need to derive an efficient computation for $$ . First, the derivative of the loss $(\overline{G})= (G)+(\langle s^{\prime }, r^{\prime }, o \rangle )$ over $$ is: eo (G) = eo (G) + (1-) s', r'-(s',r',o) where $_{s^{\prime }, r^{\prime }} = ^{\prime }+ ^{\prime }$ , and $\varphi = \sigma (\psi (s^{\prime },r^{\prime },o))$ . At convergence, after retraining, we expect $\nabla _{e_o} (\overline{G})=0$ . We perform first order Taylor approximation of $\nabla _{e_o} (\overline{G})$ to get: 0
(1-) (s', r'-)(s',r',o)+(Ho - Hs',r',o)(-)
Hs',r',o = (1-)(s', r'-)(s', r'-)(s',r',o)2+
1-(s',r',o)-(1-) (s', r'-)(s', r'-)(s',r',o)3 where $H_o$ is the $d\times d$ Hessian matrix for $o$ , i.e., second order derivative of the loss w.r.t. $$ , computed sparsely. Solving for $$ gives us: = -(1-) (Ho - Hs',r',o)-1 (s', r'-)(s',r',o)
+ Then, we compute the score change as: (s,r,o)= |+-|
= |++(1-) (Ho - Hs',r',o)-1
(s', r'-)(s',r',o) - |
Calculating this expression is efficient since $H_o$ is a $d\times d$ matrix.
Sample Adversarial Attacks
In this section, we provide the output of the for some target triples. Sample adversarial attacks are provided in Table 5 . As it shows, attacks mostly try to change the type of the target triple's object by associating it with a subject and a relation that require a different entity types.

Question: What datasets are used to evaluate this approach?
Answer:  Kinship and Nations knowledge graphs, YAGO3-10 and WN18KGs knowledge graphs 

WN18 and YAGO3-10

Question: How is this approach used to detect incorrect facts?
Answer: if there is an error in the graph, the triple is likely to be inconsistent with its neighborhood, and thus the model should put least trust on this triple. In other words, the error triple should have the least influence on the model's prediction of the training data.

Question: Can this adversarial approach be used to directly improve model accuracy?
Answer: Yes