O ptic E : A Coherence Theory-Based Model for Link Prediction (2022)

Abstract: Knowledge representation learning is a key step required for link prediction tasks with knowledge graphs (KGs). During the learning process, the semantics of each entity are embedded by a vector or a point in a feature space. The distance between these points is a measure of semantic similarity. However, in a KG, while two entities may have similar semantics in some relations, they have different semantics in others. It is ambiguous to assign a fixed distance to depict the variant semantic similarity of entities. To alleviate the semantic ambiguity in KGs, we design a new embedding approach named OpticE, which is derived from the well-known physical phenomenon of optical interference. It is a lightweight and relation-adaptive model based on coherence theory, in which each entity’s semantics vary automatically regarding different relations. In addition, a unique negative sampling method is proposed to combine the multimapping properties and self-adversarial learning during the training process. The experimental results obtained on practical KG benchmarks show that the OpticE model, with elegant structures, can compete with existing link prediction methods.

## introduction knowledge graphs (kgs) consist of sets of triplets that can represent real-world concepts, common sense information or facts. each triplet (h, r, t) indicates a directional relation r from the head entity h to the tail entity t. wordnet @xcite , freebase @xcite and wikidata @xcite are the most wellknown kgs with tremendous numbers of entities and relations. these kgs play important roles in a range of areas, such as natural language processing tasks @xcite , question answering applications @xcite and recommendation systems @xcite . research on kgs * corresponding author is thriving in both academic and industrial communities. link prediction is a fundamental problem when addressing kgs. to learn the hidden patterns from the observed triplets, extensive investigations have been performed to embed entities and relations into a continuous semantic space; this is known as kg embedding learning @xcite . among these embedding models, from a mathematical perspective, transe @xcite , transh @xcite and transr @xcite are translation-based models that use translation transformations to characterize the existence of triplets. other methods (e.g., rescal @xcite and distmult @xcite ) tackle this problem with matrix or tensor multiplication. recently, the embedding problem has been discussed in the complex domain (e.g., complex @xcite , ro-tate @xcite and @xcite ). these methods have achieved great performance on related tasks. while our models are discussed in the complex domain, they are designed under an interference framework. the semantic ambiguity caused by multimapping relations was ignored in most previous methods. as illustrated in figure 1 (a), the semantics of new york and washington d.c. are similar under the relation located_in, while they are quite different under the relation capital_of because new york is not a capital. previous methods such as transe and rotate suffer from this ambiguity problem. as shown in figure 1 (b), it is difficult to disambiguate new york from washington d.c. for the relation capital_of in rotate. to alleviate this problem, transh utilizes a hyperplane to project entities into independent semantic space. rate @xcite integrates a semantic matching method with an extra relation-adaptive matrix. instead of relying on extra settings, we tackle this problem by the intrinsic attribute of the rotational semantic space. we are motivated by the optical interference phenomenon (bright or dark fringes appear after the superposition of two beams of light), because (i) it is intuitive to simulate positive cases and negative cases utilizing distinct dark and bright fringes; and (ii) since coherence theory in the complex field is sufficiently sophisticated, it can play a theoretical guiding role. as optice is one type of rotation-based model, it can be illustrated with the rotation transformation shown in figure 1 (c ). the entities new york and washington d.c. are mapped into different relation-adaptive orbits with the same phase difference. the two entities are close to each other in the orbit of located_in, while in the orbit of relation capital_of they no longer overlap. by embedding entities into orbits with different semantic densities, optice can take control of the similarity of entities regarding specific relations. generally, multimapping relations map entities into small-radius orbits in dense semantic space, while one-to-one relations rely on large-radius orbits in sparse semantic space to distinguish entities. this will be verified in our experiments. to optimize optice effectively, we optimize self-adversarial sampling @xcite by integrating a bernoulli filtering process. the proposed method can leverage the high efficiency of self-adversarial sampling and reduce the performance loss caused by false negative labels. tested on wn18, fb15k, wn18rr and fb15k-237, the widely recognized benchmark datasets for link prediction, the lightweight optice model is comparable to the previous rotation-based models. op-tice's ability to adapt to multimapping relations is also verified in our experiments. our contributions can be summarized as follows: ## related work all the typical related works are listed in real number field models. the most important and influential model is transe @xcite , which takes relations as translation transformation by simply assuming that h + @xmath0. however, its expressiveness is limited, especially when representing a multimapping relation. transh @xcite exploits an extra projection transformation and manages to eliminate the ambiguity in transe. distmult @xcite uses a semantic matching method while ignoring the directionality of the relations. complex field models. complex @xcite utilizes complex numbers in kg embeddings and achieves great performance. it can be considered a complex version of distmult, but it overcomes the shortcomings of distmult via tail conjugation. rotate @xcite surpasses all the previous models by rotation transformation; similar to transe, the model assumes that h•@xmath1. based on rotate, rate @xcite defines a relation-specific weight w r with an extra 8 parameters per relation to solve the ambiguity problem. hake (zhang et al., 2020) employs a hierarchical structure simulated by a modulus, but the model is complicated and difficult to train. optice (ours) belongs to the category of complex field models and provides a novel way to solve the semantic ambiguity problem in rotate. like rate and transh, it has a relation-adaptive strategy that can retrieve each entity's different semantics. however, our strategy is much simpler due to the use of a relation-modulated amplitude. in the aspect of negative sampling, we adopt multiple negative sampling @xcite and self-adversarial strategy @xcite in our model, but optimize the process with a bernoulli distribution @xcite according to the multimapping relations. ## the coherence theory-based model in this part, we briefly introduce coherence theory in §3.1. in §3.2, we provide three ways to modulate an interference model and demonstrate the ad- ## experiments in this part, the experimental conditions are elaborated, and the performance of the proposed models is compared. different negative sampling methods are tested. finally, we examine the effects of the amplitude of optice. all the experiments are conducted on an nvidia-v100 gpu. the code for this paper is available on https://github.com/guixiangyu1/optice . ## conclusion we embed kgs with a novel optic interference perspective to tackle the link prediction problem. guided by coherence theory, we explore three kinds of modulation methods and analyze their properties theoretically. with our proposed negative sampling method, optice achieves the best score in comparison with other translation-and rotation-based models. its ability to disambiguate entities in multiple mapping relations is verified experimentally. the results suggest that we can alleviate semantic ambiguity in rotation-based models by mapping entities into relation-adaptive orbits with different semantic densities. zhanqiu zhang, jianyu cai, yongdong zhang, and jie wang. 2020. learning hierarchy-aware knowledge graph embeddings for link prediction. in thirty-fourth aaai conference on artificial intelligence, pages 3065-3072. aaai press.