Double-Branch Multi-Attention based Graph Neural Network for Knowledge Graph Completion (2023)

Abstract: Graph neural networks (GNNs), which effectively use topological structures in the knowledge graphs (KG) to embed entities and relations in low-dimensional spaces, have shown great power in knowledge graph completion (KGC). KG has abundant global and local structural information, however, many GNN-based KGC models cannot capture these two types of information about the graph structure by designing complex aggregation schemes, and are not designed well to learn representations of seen entities with sparse neighborhoods in isolated subgraphs. In this paper, we find that a simple attention-based method can outperform a general GNN-based approach for KGC. We then propose a double-branch multi-attention based graph neural network (MA-GNN) to learn more expressive entity representations which contain rich global-local structural information. Specifically, we first explore the graph attention network-based local aggregator to learn entity representations. Furthermore, we propose a snowball local attention mechanism by leveraging the semantic similarity between two-hop neighbors to enrich the entity embedding. Finally, we use Transformer-based self-attention to learn long-range dependence between entities to obtain richer representations with the global graph structure and entity features. Experimental results on five benchmark datasets show that MA-GNN achieves significant improvements over strong baselines for inductive KGC.

## introduction knowledge graphs (kgs), which are collections of structured knowledge represented by factual triples (head entity, relation, tail entity), are crucial for many applications, including semantic search @xcite , question-answering @xcite , recommendation systems @xcite , etc. however, even large-scale kgs (e.g., ## methods an encoder-decoder framework is used in the ma-gnn model. the framework of ma-gnn is shown in figure 2 . we only need to stack the encoder into several layers to obtain entity representation. ma-gnn presents four main modules: gat(encoder), transformer-based self-attention module (encoder), snowball local attention module (encoder), and knowledge graph completion module (decoder). ## conclusion in this paper, we propose ma-gnn, a simple and efficient framework for learning global-local structure information based on multi-attention. we incorporate a transformer-based self-attention module into a standard gat module to encode local graph structure information and learn long-range relationships. we design a snowball local attention mechanism to enrich the entity embeddings based on the similarities between two-hop neighborhood entities. on the five commonly known benchmark datasets, empirical experiments show that our proposed model achieves competitive performance compared with state-of-the-art performance.