user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Neural Variational Inference For Embedding Knowledge Graphs

Paper content:
Recent advances in Neural Variational Inference allowed for a renaissance in latent variable models in a variety of domains involving high-dimensional data.
In this paper, we introduce two generic Variational Inference frameworks for generative models of Knowledge Graphs; Latent Fact Model and Latent Information Model.
While traditional variational methods derive an analytical approximation for the intractable distribution over the latent variables, here we construct an inference network conditioned on the symbolic representation of entities and relation types in the Knowledge Graph, to provide the variational distributions.
The new framework can create models able to discover underlying probabilistic semantics for the symbolic representation by utilising parameterisable distributions which permit training by back-propagation in the context of neural variational inference, resulting in a highly-scalable method.
Under a Bernoulli sampling framework, we provide an alternative justification for commonly used techniques in large-scale stochastic variational inference, which drastically reduces training time at a cost of an additional approximation to the variational lower bound.
The generative frameworks are flexible enough to allow training under any prior distribution that permits a re-parametrisation trick, as well as under any scoring function that permits maximum likelihood estimation of the parameters.
Experiment results display the potential and efficiency of this framework by improving upon multiple benchmarks with Gaussian prior representations.
Code publicly available on Github.
In many fields, including physics and biology, being able to represent uncertainty is of crucial importance BID18 .
For instance, when link prediction in Knowledge Graphs is used for driving expensive pharmaceutical experiments (Bean et al., 2017) , it would be beneficial to know what is the confidence of a model in its predictions.
However, a significant shortcoming of current neural link prediction models BID13 BID38 -and for the vast majority of neural representation learning approaches -is their inability to express a notion of uncertainty.Furthermore, Knowledge Graphs can be very large and web-scale BID14 and often suffer from incompleteness and sparsity BID14 .
In a generative probabilistic model, we could leverage the variance in model parameters and predictions for finding which facts to sample during training, in an Active Learning setting BID22 BID17 .
BID16 use dropout for modelling uncertainty, however, this is only applied at test time.However, current neural link prediction models typically only return point estimates of parameters and predictions BID32 , and are trained discriminatively rather than generatively: they aim at predicting one variable of interest conditioned on all the others, rather than accurately representing the relationships between different variables BID31 , however, BID16 could still be applied to get uncertainty estimates for these models.
The main argument of this article is that there is a lack of methods for quantifying predictive uncertainty in a knowledge graph embedding representation, which can only be utilised using probabilistic modelling, as well as a lack of expressiveness under fixed-point representations.
This constitutes a significant contribution to the existing literature because we introduce a framework for creating a family of highly scalable probabilistic models for knowledge graph representation, in a field where there has been a lack of this.
We do this in the context of recent advances in variational inference, allowing the use of any prior distribution that permits a re-parametrisation trick, as well as any scoring function which permits maximum likelihood estimation of the parameters.
We have successfully created a framework allowing a model to learn embeddings of any prior distribution that permits a re-parametrisation trick via any score function that permits maximum likelihood estimation of the scoring parameters.
The framework reduces the parameter by one hyperparameter -as we typically would need to tune a regularisation term for an l1/ l2 loss term, however as the Gaussian distribution is self-regularising this is deemed unnecessary for matching state-ofthe-art performance.
We have shown, from preliminary experiments, that these display competitive results with current models.
Overall, we believe this work will enable knowledge graph researchers to work towards the goal of creating models better able to express their predictive uncertainty.

assistant: text: Working toward generative knowledge graph models to better estimate predictive uncertainty in knowledge inference.