Paper:
Embedding Logical Queries on Knowledge Graphs
Learning low-dimensional embeddings of knowledge graphs is a powerful approach used to predict unobserved or missing edges between entities. However, an open challenge in this area is developing techniques that can go beyond simple edge prediction and handle more complex logical queries, which might involve multiple unobserved edges, entities, and variables. For instance, given an incomplete biological knowledge graph, we might want to predict what drugs are likely to target proteins involved with both diseases X and Y?-a query that requires reasoning about all possible proteins that might interact with diseases X and Y. Here we introduce a framework to efficiently make predictions about conjunctive logical queries-a flexible but tractable subset of first-order logic-on incomplete knowledge graphs. In our approach, we embed graph nodes in a low-dimensional space and represent logical operators as learned geometric operations (e.g., translation, rotation) in this embedding space. By performing logical operations within a low-dimensional embedding space, our approach achieves a time complexity that is linear in the number of query variables, compared to the exponential complexity required by a naive enumeration-based approach. We demonstrate the utility of this framework in two application studies on real-world datasets with millions of relations: predicting logical relationships in a network of drug-gene-disease interactions and in a graph-based representation of social interactions derived from a popular web forum.

Review:
This work proposes a method for answering complex conjunctive queries against an incomplete Knowledge Base. The method relies on node embeddings, and trainable differentiable representations for intersection and neighbourhood expansion operations, which can compute a prediction in linear time relative to the edges involved. The method is evaluated on a biomedical and a reddit dataset. The originality of this work lies in posing "soft" logical queries against a "hard" graph, rather than posing "hard" queries against a "soft" graph, as is done typically.  The manuscript is clear, well-written and introduces notation using examples. The graphs and visualisations chosen throughout the manuscript are well-designed and help convey technical points. Careful thought has been put into selecting negatives during training, and there is an interesting ablation on different bilinear interaction functions.  A major criticism of this work is that it is not compared against standard KB inference methods which operate on the individual links in a query independently. Of course standard KB inference methods cannot be used directly to answer complex queries, but a standard query language can, and in the simplest form it could just aggregate scores of all facts involved, independently scored by a standard KB inference method. The authors mention that "this would require computation time that is exponential in the number of edges" (line 41-42). However, in this simplest approach, a model could compute edge scores independently, which is linear in the number of query edges. This would at least reveal whether it is important to score edges in conjunction (the implicit underlying premise of this work), or whether it suffices to treat them independently at test time.   A second criticism is that it is unclear whether the datasets chosen do actually test for a model's ability to answer logical conjunctions. 10% of edges are removed uniformly at random, and what exactly is required to answer a query is unclear. For example, in parallel query DAG structures, one branch alone might already suffice to unequivocally point to the correct answer. In fact, one observation from the experimental results is that linear chains are much harder to predict correctly than graphs with some sort of parallel structure. Furthermore, the longer the linear chain, the harder the task seems. This raises the question of redundancy in the dataset, and whether in those graphs with parallel structure one branch is already sufficient to specify the correct answer.     Questions / Comments to the authors: - notation: line 80-81: double use of tau? - notation (Theorem 1, third line): What is V?  - You mention different metrics: ROC AUC (section 5.1) and accuracy (Table 1) - training via sampling a fixed number of each query DAG structure was to facilitate batching? - line 272: where can this result be found? - is the macro-average across query DAG structures computed with adequate proportions, or uniformly weighted? - Can any general lessons be drawn from the selection of \Psi? - The origin of the drug interaction network dataset is not clear. How was this obtained?