Probabilistic Box Embeddings for Uncertain Knowledge Graph Reasoning (2021)

Abstract: Knowledge bases often consist of facts which are harvested from a variety of sources, many of which are noisy and some of which conflict, resulting in a level of uncertainty for each triple. Knowledge bases are also often incomplete, prompting the use of embedding methods to generalize from known facts, however, existing embedding methods only model triple-level uncertainty, and reasoning results lack global consistency. To address these shortcomings, we propose BEUrRE, a novel uncertain knowledge graph embedding method with calibrated probabilistic semantics. BEUrRE models each entity as a box (i.e. axis-aligned hyperrectangle) and relations between two entities as affine transforms on the head and tail entity boxes. The geometry of the boxes allows for efficient calculation of intersections and volumes, endowing the model with calibrated probabilistic semantics and facilitating the incorporation of relational constraints. Extensive experiments on two benchmark datasets show that BEUrRE consistently outperforms baselines on confidence prediction and fact ranking due to its probabilistic calibration and ability to capture high-order dependencies among facts.

## introduction knowledge graphs (kgs) provide structured representations of facts about real-world entities and relations. in addition to deterministic kgs @xcite @xcite , much recent attention has been paid to uncertain kgs (or ukgs). ukgs, such as probase @xcite , nell @xcite , and conceptnet @xcite , associate each fact (or triple) with a confidence score representing the likelihood of that fact to be true. such uncertain knowledge representations critically capture the uncertain nature of reality, and provide more precise reasoning. for example, while both (honda, competeswith, toyota) and (honda, competeswith, chrysler) look somewhat correct, the former fact should have a higher confidence than the latter one, since honda and toyota are both japanese car manufacturers and have highly overlapping customer bases. similarly, while (the beatles, genre, rock) and (the beatles, genre, pop) are both true, the first one may receive a slightly higher confidence, since the beatles is generally considered a rock band. such confidence information is important when answering questions like who is the main competitor of honda?, or extracting confident knowledge for drug repurposing @xcite . to facilitate automated knowledge acquisition for ukgs, some ukg embedding models @xcite have recently been proposed. inspired by the works about deterministic kg embeddings @xcite , existing approaches model entities and relations as points in low-dimensional vector space, measure triple plausibility with vector similarity (eg. distance, dot-product), and map the plausibility to the confidence range of @xcite . for instance, the representative work ukge @xcite models the triple plausibility in the form of embedding product @xcite , and trains the embedding model as a regressor to predict the confidence score. one interpretation of existing methods is that they model each triple using a binary random variable, where the latent dependency structure between different binary random variables is captured by vector similarities. without an explicit dependency structure it is difficult to enforce logical reasoning rules to maintain global consistency. in order to go beyond triple-level uncertainty modeling, we consider each entity as a binary random variable. however, representing such a probability distribution in an embedding space and reasoning over it is non-trivial. it is difficult to model marginal and joint probabilities for entities using simple geometric objects like vectors. in order to encode probability distributions in the embedding space, recent works @xcite @xcite ) represent random variables as more complex geometric objects, such as cones and axis-aligned hyperrectangles (boxes), and use volume as the probability measure. inspired by such advances of probability measures in embeddings, we present beurre (box embedding for uncertain relational data) 2 . beurre represents entities as boxes. relations are modeled as two separate affine transforms on the head and tail entity boxes. confidence of a triple is modeled by the intersection between the two transformed boxes. fig. 1 shows how a fact about the genre of the beatles is represented under our framework. such representation is not only inline with the human perception that entities or concepts have different levels of granularity, but also allows more powerful domain knowledge representation. ukge @xcite has demonstrated that introducing domain knowledge about relation properties (e.g. transitivity) can effectively enhance reasoning on ukgs. while ukge uses probabilistic soft logic (psl) @xcite to reason for unseen facts and adds the extra training samples to training, such a method can lead to error propagation and has limited scope of application when ukg is sparse. in our work, we propose sufficient conditions for these relation properties to be preserved in the embedding space and directly model the relation properties by regularizing relation-specific transforms based on constraints. this technique is more robust to noise and has wide coverage that is not restricted by the scarcity of the existing triples. extensive experiments on two benchmark datasets show that beurre effectively captures the uncertainty, and consistently outperforms the baseline models on ranking and predicting confidence of unseen facts. ## related work we discuss two lines of related work. ## background before we move on to the presented method in this work, we use this section to introduce the background of box embeddings and the addressed task. ## method in this section, we present our ukg embedding model beurre. the proposed model encodes entities as probabilistic boxes and relations as affine transforms. we also discuss the method to incorporate logical constraints into learning. ## experiments in this section we present evaluation of our model on two ukg reasoning tasks, i.e. confidence prediction and fact ranking. more experimentation details are in appendices. ## conclusion this paper presents a novel ukg embedding method with calibrated probabilistic semantics. our model beurre encodes each entity as a gumble box representation whose volume represents marginal probability. a relation is modeled as two affine transforms on the head and tail entity boxes. we also incorporate logic constraints that capture the high-order dependency of facts and enhance global reasoning consistency. extensive experiments show the promising capability of beurre on confidence prediction and fact ranking for ukgs. the results are encouraging and suggest various extensions, including deeper transformation architectures as well as alternative geometries to allow for additional rules to be imposed. in this context, we are also interested in extending the use of the proposed technologies into more downstream tasks, such as knowledge association @xcite and event hierarchy induction @xcite . another direction is to use beurre for ontology construction and population, since box embeddings are naturally capable of capturing granularities of concepts.