Learning to Deceive Knowledge Graph Augmented Models via Targeted Perturbation

1 INTRODUCTION . Recently , neural reasoning over knowledge graphs ( KGs ) has emerged as a popular paradigm in machine learning and natural language processing ( NLP ) . KG-augmented models have improved performance on a number of knowledge-intensive downstream tasks : for question answering ( QA ) , the KG provides context about how a given answer choice is related to the question ( Lin et al. , 2019 ; Feng et al. , 2020 ; Lv et al. , 2020 ; Talmor et al. , 2018 ) ; for item recommendation , the KG mitigates data sparsity and cold start issues ( Wang et al. , 2018b ; 2019a ; b ; 2018a ) . Furthermore , by using attention over the KG , such models aim to explain which KG information was most relevant for making a given prediction ( Lin et al. , 2019 ; Feng et al. , 2020 ; Wang et al. , 2018b ; 2019b ; Cao et al. , 2019 ; Gao et al. , 2019 ) . Nonetheless , the process in which KG-augmented models reason about KG information is still not well understood . It is assumed that , like humans , KG-augmented models base their predictions on meaningful KG paths and that this process is responsible for their performance gains ( Lin et al. , 2019 ; Feng et al. , 2020 ; Gao et al. , 2019 ; Song et al. , 2019 ) . In this paper , we question if existing KG-augmented models actually use KGs in this human-like manner . We study this question primarily by measuring model performance when the KG ’ s semantics and structure have been perturbed to hinder human comprehension . To perturb the KG , we propose four perturbation heuristics and a reinforcement learning ( RL ) based perturbation algorithm . Surprisingly , for KG-augmented models on both commonsense QA and item recommendation , we find that the KG can be extensively perturbed with little to no effect on performance . This raises doubts about KG-augmented models ’ use of KGs and the plausibility of their explanations . 2 PROBLEM SETTING . Our goal is to investigate whether KG-augmented models and humans use KGs similarly . Since KGs are human-labeled , we assume that they are generally accurate and meaningful to humans . Thus , across different perturbation methods , we measure model performance when every edge in the KG has been perturbed to make less sense to humans . To quantify the extent to which the KG has been perturbed , we also measure both semantic and structural similarity between the original ∗Work done while MR , SA and HW interned remotely at USC . Code and data are available at https : //github.com/INK-USC/deceive-KG-models . †Equal contribution . KG and perturbed KG . If original-perturbed KG similarity is low , then a human-like KG-augmented model should achieve worse performance with the perturbed KG than with the original KG . Furthermore , we evaluate the plausibility of KG-augmented models ’ explanations when using original and perturbed KGs , by asking humans to rate these explanations ’ readability and usability . Notation Let Fθ be an KGaugmented model , and let ( Xtrain , Xdev , Xtest ) be a dataset for some downstream task . We denote a KG as G = ( E , R , T ) , where E is the set of entities ( nodes ) , R is the set of relation types , and T = { ( e1 , r , e2 ) | e1 , e2 ∈ E , r ∈ R } is the set of facts ( edges ) composed from existing entities and relations ( Zheng et al. , 2018 ) . Let G ′ = ( E , R′ , T ′ ) be the KG obtained after perturbing G , where R ′ ⊆ R and T ′ 6= T . Let f ( G , G ′ ) be a function that measures similarity between G and G ′ . Let g ( G ) be the downstream performance when evaluating Fθ on Xtest and G. Also , let ⊕ denote the concatenation operation , and let NL ( e ) denote the set of L-hop neighbors for entity e ∈ E . High-Level Procedure First , we train Fθ on Xtrain and G , then evaluate Fθ on Xtest and G to get the original performance g ( G ) . Second , we freeze Fθ , then perturb G to obtain G ′ . Third , we evaluate Fθ on Xtest and G ′ to get the perturbed performance g ( G ′ ) . Finally , we measure g ( G ) − g ( G′ ) and f ( G , G ′ ) to assess how human-like Fθ ’ s reasoning process is . This procedure is illustrated in Fig . 1 . In this paper , we consider two downstream tasks : commonsense QA and item recommendation . Commonsense QA Given a question x and a set of k possible answers A = { y1 , ... , yk } , the task is to predict a compatibility score for each ( x , y ) pair , such that the highest score is predicted for the correct answer . In commonsense QA , the questions are designed to require commonsense knowledge which is typically unstated in natural language , but more likely to be found in KGs ( Talmor et al. , 2018 ) . Let F textφ be a text encoder ( Devlin et al. , 2018 ) , F graph ψ be a graph encoder , and F cls ξ be an MLP classifier , where φ , ψ , ξ ⊂ θ . Let G ( x , y ) denote a subgraph of G consisting of entities mentioned in text sequence x ⊕ y , plus their corresponding edges . We start by computing a text embedding htext = F textφ ( x⊕ y ) and a graph embedding hgraph = F graph φ ( G ( x , y ) ) . After that , we compute the score for ( x , y ) as S ( x , y ) = F clsξ ( htext ⊕ hgraph ) . Finally , we select the highest scoring answer : ypred = argmaxy∈A S ( x , y ) . KG-augmented commonsense QA models vary primarily in their design of Fgraphψ . In particular , path-based models compute the graph embedding by using attention to selectively aggregate paths in the subgraph . The attention scores can help explain which paths the model focused on most for a given prediction ( Lin et al. , 2019 ; Feng et al. , 2020 ; Santoro et al. , 2017 ) . Item Recommendation We consider a set of users U = { u1 , u2 , ... , um } , a set of items V = { v1 , v2 , ... , vn } , and a user-item interaction matrix Y ∈ Rm×n with entries yuv . If user u has been observed to engage with item v , then yuv = 1 ; otherwise , yuv = 0 . Additionally , we consider a KG G , in which R is the set of relation types in G. In G , nodes are items v ∈ V , and edges are facts of the form ( v , r , v′ ) , where r ∈ R is a relation . For the zero entries in Y ( i.e. , yuv = 0 ) , our task is to predict a compatibility score for user-item pair ( u , v ) , indicating how likely user u is to want to engage with item v. We represent each user u , item v , and relation r as embeddings u , v , and r , respectively . Given a user-item pair ( u , v ) , its compatibility score is computed as 〈u , v〉 , the inner product between u and v. KG-augmented recommender systems differ mainly in how they use G to compute u and v. Generally , these models do so by using attention to selectively aggregate items/relations in G. The attention scores can help explain which items/relations the model found most relevant for a given prediction ( Wang et al. , 2018b ; 2019b ) . 3 KG SIMILARITY METRICS . To measure how much the perturbed KG has deviated from the original KG , we propose several metrics for capturing semantic ( ATS ) and structural ( SC2D , SD2 ) similarity between KGs . Aggregated Triple Score ( ATS ) ATS measures semantic similarity between two KGs . Let sG be an edge ( triple ) scoring function , such that sG ( e1 , r , e2 ) measures how likely edge ( e1 , r , e2 ) is to exist in G. Also , assume sG has been pre-trained on G for link prediction . Then , ATS is defined as fATS ( G , G ′ ) = 1 |T ′ | ∑ ( e1 , r , e2 ) ∈T ′ sG ( e1 , r , e2 ) ∈ [ 0 , 1 ] , which denotes the mean sG score across all edges in G′ . Intuitively , if a high percentage of edges in G′ are also likely to exist in G ( i.e. , high ATS ) , then we say that G′ and G have high semantic similarity . sG is task-specific , as KGs from different tasks may differ greatly in semantics . For commonsense QA , we use the sG from Li et al . ( 2016 ) ; for item recommendation , we use the sG from Yang et al . ( 2015 ) . While ATS captures semantic KG differences , it is not sensitive to KG connectivity structure . Note that fATS ( G , G ) may not equal 1 , since sG may not perfectly generalize to KGs beyond those it was trained on . Similarity in Clustering Coefficient Distribution ( SC2D ) SC2D measures structural similarity between two KGs and is derived from the local clustering coefficient ( Saramäki et al. , 2007 ; Onnela et al. , 2005 ; Fagiolo , 2007 ) . For a given entity in G ( treated here as undirected ) , the local clustering coefficient is the fraction of possible triangles through the entity that exist ( i.e. , how tightly the entity ’ s neighbors cluster around it ) . For entity ei ∈ E , the local clustering coefficient is defined as ci = 2Tri ( ei ) / ( deg ( ei ) ( deg ( ei ) − 1 ) ) , where Tri ( ei ) is the number of triangles through ei , and deg ( ei ) is the degree of ei . For each relation r ∈ R , let Gr be the subgraph of G consisting of all edges in T with r . That is , Gr = ( E , r , T ′ ) , where T ′ = { ( e , r , e′ ) | e , e′ ∈E } . Let cr denote the |E|-dimensional clustering coefficient vector for Gr , where the ith element of cr is ci . Then , the mean clustering coefficient vectors for G and G′ are co = 1|R| ∑ r∈R c r and cp = 1|R′| ∑ r∈R′ c r , respectively . SC2D is defined as fSC2D ( G , G ′ ) = 1− ‖co−cp‖2‖co−cp‖2+1 ∈ [ 0 , 1 ] , with higher value indicating higher similarity . Similarity in Degree Distribution ( SD2 ) SD2 also measures structural similarity between two KGs , while addressing SC2D ’ s ineffectiveness when the KGs ’ entities have tiny local clustering coefficients ( e.g. , the item KG used by recommender systems is roughly bipartite ) . In such cases , SC2D is always close to one regardless of perturbation method , thus rendering SC2D useless . Let dr denote the |E|-dimensional degree vector for Gr , where the ith element of dr is deg ( ei ) . Then , the mean degree vectors for G and G′ are do = 1|R| ∑ r∈R d r and dp = 1|R′| ∑ r∈R′ d r , respectively . SD2 is defined as fSD2 ( G , G ′ ) = 1− ‖do−dp‖2‖do−dp‖2+1 ∈ [ 0 , 1 ] , with higher value indicating higher similarity .

Summary: The paper provides a number of adversarial attacks on hybrid neural-symbolic systems. The systems are recommender and QA systems which use an underlying knowledge-graph (KG) such as ConceptNet. Previous work has suggested that the KGs are important for good performance, and moreover that the use of KGs lends the system a degree of interpretability. The attacks are successful - maintaining performance whilst seriously degrading the KG - throwing doubt on these claims.