Efficient Federated Learning on Knowledge Graphs via Privacy-preserving Relation Embedding Aggregation (2022)

Abstract: Federated learning (FL) can be essential in knowledge representation, reasoning, and data mining applications over multi-source knowledge graphs (KGs). A recent study FedE first proposes an FL framework that shares entity embeddings of KGs across all clients. However, entity embedding sharing from FedE would incur a severe privacy leakage. Specifically, the known entity embedding can be used to infer whether a specific relation between two entities exists in a private client. In this paper, we introduce a novel attack method that aims to recover the original data based on the embedding information, which is further used to evaluate the vulnerabilities of FedE. Furthermore, we propose a Federated learning paradigm with privacy-preserving Relation embedding aggregation (FedR) to tackle the privacy issue in FedE. Besides, relation embedding sharing can significantly reduce the communication cost due to its smaller size of queries. We conduct extensive experiments to evaluate FedR with five different KG embedding models and three datasets. Compared to FedE, FedR achieves similar utility and significant improvements regarding privacy-preserving effect and communication efficiency on the link prediction task.

## introduction knowledge graphs (kgs) are critical data structures to represent human knowledge and serve as resources for various real-world applications, such as recommendation and question answering @xcite . however, most kgs are usually incomplete and naturally distributed to different clients. despite each client can explore the missing links with their own kgs by knowledge graph embedding (kge) models @xcite , exchanging knowledge with others can further enhance completion performance because the overlapping elements are usually involved in different kgs @xcite . to exchange knowledge, the first federated learning (fl) framework for kg -fede is recently proposed, where each client trains local embeddings on its kg while the server receives and aggregates only locally-computed updates of entity embeddings instead of collecting triplets directly @xcite . however, at the very beginning in fede, the server should collect the entity sets of every client for entity alignment, which will lead to unintentional privacy leakage: 1) entity's information, such as the customer's name, is usually sensitive but it is fully exposed to the server; 2) the relation embedding will be inferred and be exploited for knowledge graph reconstruction attack if there exists the malicious server (see section 3.1). therefore, we propose fedr that adopts relation embedding aggregation to tackle the privacy issue in fede. the major difference is shown in figure 1 . besides, the number of entities is usually greater than the number of relations in real-world graph databases, so sharing relation embedding is more communication-efficient. we summarize the following contributions of our work. 1) we present a kg reconstruction attack method and reveal that fede suffers a potential privacy leakage due to a malicious server and its colluded clients. 2) we propose fedr, an efficient and privacy-preserving fl framework on kgs. experimental results demonstrate that fedr has the competitive performance compared with fede, but gains substantial improvements in terms of privacypreserving effect and communication efficiency. federated learning and fede. fl allows different clients to collaboratively learn a global model without sharing their local data @xcite . in particular, the aim is to minimize: min w f (w) = e k [f k (w)], where f k (w) is the local objective that measures the local empirical risk of k-th client. compared to model sharing in vanilla fl , fede introduces a new mechanism that aggregates only entity embedding. more concretely, the server maintains a complete table including entity embeddings and the corresponding entity ids, and the server can identify if an entity exists in a client for entity alignment. ## experiments we carry out several experiments to explore fedr's performance in link prediction, in which the tail t is predicted given head h and relation r. datasets. we evaluate our framework through experiments on three public datasets, fb15k-237, wn18rr @xcite and a disease database -ddb14 @xcite . to build federated datasets, we randomly split triplets to each client without replacement. note that, random split makes data heterogeneous among all the clients, and ensures fair comparison between fede and fedr. kge algorithms. four commonly-used kge algorithms -transe @xcite , rotate @xcite , dismult @xcite and complex @xcite are utilized in the paper. we also implement federated noge @xcite , a gnn-based algorithm. ## conclusion and future work in this paper, we conduct the first empirical quantization of privacy leakage to federated learning on knowledge graphs, which reveals that recent work, fede, is susceptible to reconstruction attack based on shared element-embedding pairs when there are dishonest server and clients. then we propose fedr, a privacy-preserving fl framework on kgs with relation embedding aggregation that defenses against reconstruction attack effectively. experimental results show that fedr outperforms fede w.r.t data privacy and communication efficiency and also maintains similar utility. in real-world applications, different organizations may use different kge models, which may influence overall performance by embedding aggregation, how to design an effective fl framework in this case and how to perform kg reconstruction attack/defense are our future research directions.