Title: Deep Probabilistic Programming
Authors: Dustin Tran
Matthew D. Hoffman
Rif A. Saurous
Eugene Brevdo
Kevin Murphy
David M. Blei
Venue: ICLR 2017

Abstract: We propose Edward, a Turing-complete probabilistic programming language. Edward defines two compositional representations—random variables and inference. By treating inference as a first class citizen, on a par with modeling, we show that probabilistic programming can be as flexible and computationally efficient as traditional deep learning. For flexibility, Edward makes it easy to fit the same model using a variety of composable inference methods, ranging from point estimation to variational inference to MCMC. In addition, Edward can reuse the modeling representation as part of inference, facilitating the design of rich variational models and generative adversarial networks. For efficiency, Edward is integrated into TensorFlow, providing significant speedups over existing probabilistic systems. For example, we show on a benchmark logistic regression task that Edward is at least 35x faster than Stan and 6x faster than PyMC3. Further, Edward incurs no runtime overhead: it is as fast as handwritten TensorFlow.


Review discussion:
Thank you for an interesting read.
Regarding experiments reported in section 5.1: could you say a bit more on how you modify the decoder objective? Does your library provide different decoder objectives to the users?
Also the zoo website is unavailable.
Thanks for the question! You can find source code for the different gradient estimators here:
https://github.com/blei-lab/edward/blob/master/edward/inferences/klqp.py
Each objective is written as a separate function, and a user can leverage a specific gradient estimator such as "ReparameterizationEntropyKLqp": this minimizes the KL(q||p) divergence measure using reparameterization gradients and an analytic entropy.
Each of these modifications is a modification in how to estimate the objective and its gradients.
For example, the one mentioned above only takes Monte Carlo samples to estimate the model's joint log density, E_{q(z | lambda)} [ log p(x, z) ], and it calculates the entropy term analytically, - E_{q(z | lambda)} [ log q(z | lambda) ].
Apologies on the zoo website. We should have it up again shortly.
Thanks for the interesting paper!
I found the code examples difficult to follow because they were clearly incomplete snippets and it was unclear how much and what was being left out. Even the supposedly complete examples in the Appendix are not really complete in the sense that they could be run as written (missing imports, maybe something else too?). This was especially challenging with the Bayesian RNN (Fig. 3) which did not seem to connect to any observations, making it unclear how it could be used. Perhaps the authors could clarify this?
Thanks for the question on clarification!
In general, the model snippets are comprehensive in that they only omit import statements (e.g., "from edward.models import Normal") and constants such as N, M, H, D, etc. No inference is shown for them.
The complete examples in the Appendix only omit the import statements. I have updated the submission to reflect them.
For example, running the VAE script will not only train the model but also generate images during training and export them to a directory. (The embedding example omits a function which builds data mini-batches; this requires tedious data pre-processing which we hope to clean up before putting in the paper.)
Our apologies on the Bayesian RNN confusion. We hoped to focus attention on the difficult piece, which is hidden states with unspecified length. I have updated the submission so that the example also includes a response variable of unspecified length.
Thank you for an interesting read.
I found this paper very interesting. Since I don't think (deterministic) approximate inference is separated from the modelling procedure (cf. exact inference), it is important to allow the users to select the inference method to suit their needs and constraints. I'm not an expert of PPL, but to my knowledge this is the first package that I've seen which put more focus on compositional inference. Leveraging tensorflow is also a plus, which allows flexible computation graph design as well as parallel computation using GPUs.
The only question I have is about the design of flexible objective functions to learn hyper-parameters (or in the paper those variables associated with delta q distributions). It seems hyper-parameter learning is also specified as inference, which makes sense if using MAP.
However the authors also demonstrated other objective functions such as Renyi divergences, does that mean the user need to define a new class of inference method whenever they want to test an alternative loss function?
Thanks for the feedback!
> It seems hyper-parameter learning is also specified as inference, which makes sense if using MAP.
However the authors also demonstrated other objective functions such as Renyi divergences,
By default, inference hyperparameters such as alpha in Renyi divergences are set by the user and not learned.
Of course, you can build an algorithm that adaptively selects inference hyperparameters, such as the NUTS extension to HMC which adapts the leapfrog path length. This would be a different algorithm (inference class).
> does that mean the user need to define a new class of inference method whenever they want to test an alternative loss function?
For algorithms that optimize a family of objective functions, there is only one inference class.
For example, in Renyi divergences, alpha is a setting that the user selects in the class. This is different from one inference class existing for each fixed setting of alpha (this is also possible to implement, although impractical).
The paper introduces Edward, a probabilistic programming language
built over TensorFlow and Python, and supporting a broad range of most
popular contemporary methods in probabilistic machine learning.
Quality:
The Edward library provides an extremely impressive collection of
modern probabilistic inference methods in an easily usable form.
The paper provides a brief review of the most important techniques
especially from a representation learning perspective, combined with
two experiments on implementing various modern variational inference
methods and GPU-accelerated HMC.
The first experiment (variational inference) would be more valuable if
there was a clear link to complete code to reproduce the results
provided. The HMC experiment looks OK, except the characterising Stan
as a hand-optimised implementation seems unfair as the code is clearly
not hand-optimised for this specific model and hardware configuration.
I do not think anyone doubts the quality of your implementation, so
please do not ruin the picture by unsubstantiated sensationalist
claims.
Instead of current drama, I would suggest comparing
head-to-head against Stan on single core and separately reporting the
extra speedups you gain from parallelisation and GPU. These numbers
would also help the readers to estimate the performance of the method
for other hardware configurations.
Clarity:
The paper is in general clearly written and easy to read. The numerous
code examples are helpful, but also difficult as it is sometimes
unclear what is missing. It would be very helpful if the authors could
provide and clearly link to a machine-readable companion (a Jupyter
notebook would be great, but even text or HTML would be easier to
copy-paste from than a pdf like the paper) with complete runnable code
for all the examples.
Originality:
The Edward library is clearly a unique collection of probabilistic
inference methods.
In terms of the paper, the main threat to novelty
comes from previous publications of the same group. The main paper
refers to Tran et al. (2016a) which covers a lot of similar material,
although from a different perspective. It is unclear if the other
paper has been published or submitted somewhere and if so, where.
Significance:
It seems very likely Edward will have a profound impact on the field
of Bayesian machine learning and deep learning.
Other comments:
In Sec. 2 you draw a clear distinction between specialised languages
(including Stan) and Turing-complete languages such as Edward. This
seems unfair as I believe Stan is also Turing complete.
Additionally
no proof is provided to support the Turing-completeness of Edward.
Thanks for the feedback!
> The first experiment (variational inference) would be more valuable if there was a clear link to complete code to reproduce the results provided.
This makes sense. We will have a separate script for each result, with a link to all of them in the camera-ready's appendix.
> The HMC experiment looks OK, except the characterising Stan as a hand-optimised implementation seems unfair as the code is clearly not hand-optimised for this specific model and hardware configuration.
This is a valid point. In the newly uploaded submission, we have removed any potentially sensationalist claims, and have made more nuanced claims in the abstract, introduction, and experiments. In general we think there's a strong case for building on computational graphs, as indicated by the benchmark. We will also add an additional comparison using Edward on 1 CPU.
> It would be very helpful if the authors could provide and clearly link to a machine-readable companion ... with complete runnable code for all the examples.
Excellent idea. We will have a dedicated edwardlib.org webpage and Jupyter notebook for the paper available soon.
> The main paper refers to Tran et al. (2016a) which covers a lot of similar material, although from a different perspective. It is unclear if the other paper has been published or submitted somewhere and if so, where.
Tran et al. (2016a) is an unpublished preprint that is not submitted anywhere. If or when it is submitted, we will be sure to remove any similar material.
> In Sec. 2 you draw a clear distinction between specialised languages (including Stan) and Turing-complete languages such as Edward. This seems unfair as I believe Stan is also Turing complete.
Turing complete-ness is a contentious topic with varying definitions; we will be more precise in the camera-ready version.
The authors propose a new software package for probabilistic programming, taking advantage of recent successful tools used in the deep learning community. The software looks very promising and has the potential to transform the way we work in the probabilistic modelling community, allowing us to perform rapid-prototyping to iterate through ideas quickly. The composability principles are used insightfully, and the extension of inference to HMC for example, going beyond VI inference (which is simple to implement using existing deep learning tools), makes the software even more compelling.
However, the most important factor of any PPL is whether it is practical for real-world use cases. This was not demonstrated sufficiently in the submission. There are many example code snippets given in the paper, but most are not evaluated. The Dirichlet process mixture model example (Figure 12) is an important one: do the proposed black-box inference tools really work for this snippet? and will the GAN example (Figure 7) converge when optimised with real data? To convince the community of the practicality of the package it will be necessary to demonstrate these empirically. Currently the only evaluated model is a VAE with various inference techniques, which are not difficult to implement using pure TF.
Presentation:
* Paper presentation could be improved.
For example the authors could use more signalling for what is about to be explained. On page 5 qbeta and qz are used without explanation - the authors could mention that an example will be given thereafter.
* I would also suggest to the authors to explain in the preface how the layers are implemented, and how the KL is handled in VI for example.
It will be useful to discuss what values are optimised and what values change as inference is performed (even before section 4.4). This was not clear for the majority of the paper.
Experiments:
* Why is the run time not reported in table 1?
* What are the "difficulties around convergence" encountered with the analytical entropies? inference issues become more difficult to diagnose as inference is automated. Are there tools to diagnose these with the provided toolbox?
* Did HMC give sensible results in the experiment at the bottom of page 8? only run time is reported.
* How difficult is it to get the inference to work (eg HMC) when we don't have full control over the computational graph structure and sampler?
* It would be extremely insightful to give a table comparing the performance (run time, predictive log likelihood, etc) of the various inference tools on more models.
* What benchmarks do you intend to use in the Model Zoo? the difficulty with probabilistic modelling is that there are no set benchmarks over which we can evaluate and compare many models. Model zoo is sensible for the Caffe ecosystem because there exist few benchmarks a large portion of the community was working on (ImageNet for example). What datasets would you use to compare the DPMM on for example?
Minor comments:
* Table 1: I would suggest to compare to Li & Turner with alpha=0.5 (the equivalent of Hellinger distance) as they concluded this value performs best. I'm not sure why alpha=-1 was chosen here.
* How do you handle discrete distributions (eg Figure 5)?
* x_real is not defined in Figure 7.
* I would suggest highlighting M in Figure 8.
* Comma instead of period after "rized), In" on page 8.
In conclusion I would say that the software developments presented here are quite exciting, and I'm glad the authors are pushing towards practical and accessible "inference for all". In its current form though I am forced to give the submission itself a score of 5.
Thanks for the feedback!
> the most important factor of any PPL is whether it is practical for real-world use cases.
We agree that real-world application is a crucial yet underemphasized criteria for PPLs. We also want to highlight that a major contribution of the paper is not only the applications for developing in Edward, but also the methodological advances in enabling flexible experimentation.
For example, it was an open challenge in PPLs to enable the design of rich variational models and generative adversarial networks. We believe we have significant advances in PPL design.
> There are many example code snippets given in the paper, but most are not evaluated. The Dirichlet process mixture model example (Figure 12) is an important one: do the proposed black-box inference tools really work for this snippet? and will the GAN example (Figure 7) converge when optimised with real data?
As with other PPL papers (e.g., Church (https://arxiv.org/abs/1206.3255)), the purpose of those snippets are to demonstrate the representational capacity and interface of the language, following our methodological advances.
However, we can also provide links to reproducible scripts demonstrating their success in a camera-ready. The short answer is that black box inference works for the DP mixture model—we use black box inference with a truncated variational model but non-truncated probability model; see also Kurihara et al. (2007) (https://www.ics.uci.edu/~welling/publications/papers/cvdp-v6.pdf). The GAN example does not necessarily reach a stationary point (this is open GAN research), but halting it after a fixed number of iterations produces sensible results.
> Paper presentation could be improved. [...]
Thanks for the suggestions. In the newly uploaded submission, we have added more forward signalling and clarified other miscellaneous details.
> Why is the run time not reported in table 1?
Great point.
Although not significant for the experiment's purpose, we can add these to the camera-ready.
> What are the "difficulties around convergence" encountered with the analytical entropies? ... Are there tools to diagnose these with the provided toolbox?
This is an open research challenge in gradient estimators for variational inference; see, e.g., a NIPS 2016 workshop paper (http://approximateinference.org/accepted/RoederEtAl2016.pdf). Edward has no existing solution other than recommending not to use analytic entropies.
Importantly, the goal of the experiment is to show that Edward is a platform for easily performing such experimentation.
Later research may produce a solution that is then implemented in Edward.
> Did HMC give sensible results in the experiment at the bottom of page 8?
Yes; we will also include a link to a reproducible script in the camera-ready.
> How difficult is it to get the inference to work (eg HMC) when we don't have full control over the computational graph structure and sampler?
The difficulty depends on the problem setting. As with other HMC software, we have control over usual hyperparameters such as integrator step size and path length. The only control the user does not have is the specific graph that the HMC uses to produce samples; modifying it would result in another algorithm (which the user can fully control by building off these built-in classes).
> What benchmarks do you intend to use in the Model Zoo? [...]
As with other open research challenges, we do not subscribe to a specific solution.
As long as a proposed entry to the Model Zoo demonstrates use in being shared to the community—whether it be outperforming a specific benchmark or just a useful application—we plan to accept it. We think there's utility in building the platform as a first step. A next step for the community at large is to solve the important challenges you bring up, now that they are made more transparent when applying probabilistic models in the real world.
> Minor comments
Thanks! In the newly uploaded submission, we have made these changes. We can also add a result comparing to Li & Turner with alpha=0.5 (alpha=-1 was chosen to produce the chi-divergence) in the camera-ready.
Thank you to all the reviewers for their insightful feedback. As the reviewers agree, we think Edward "has the potential to transform the way we work in the probabilistic modelling community, allowing us to perform rapid-prototyping to iterate through ideas quickly" (Reviewer 4). We make general comments here; specific comments are made on the reviewer's thread. We have also uploaded a revised submission following the feedback.
A major contribution in the paper is in real-world application: "the Edward library provides an extremely impressive collection of modern probabilistic inference methods in an easily usable form" (Reviewer 1). This enables researchers to build on top of current methods and also easily compare to them as baselines.
A second contribution in the paper is in methodology. We proposed new compositional representations, "putt[ing] focus on compositional inference" (Reviewer 3). This "allow[s] the users to select the inference method to suit their needs and constraints" (Reviewer 3), such as for designing rich variational models and generative adversarial networks. We also show how to integrate such a language into computational graph frameworks; this provides significant speedups with distributing training, parallelism, vectorisation, and GPU support.
Finally, we note that Edward is growing with active contributions from the community (https://github.com/blei-lab/edward). It continues to expand its feature set and improve its robustness and performance. Community-oriented platforms such as the Probability Zoo (http://edwardlib.org/zoo) and contributed models (https://github.com/blei-lab/edward/issues/380) will continue to make progress more open in the field.

Meta-review:
role: PC
data: comment: There was general agreement from the reviewers that this looks like an important development in the area of probabilistic programming. Some reviewers felt the impact of the work could be very significant. The quality of the work and the paper were perceived as being quite high. The main weakness highlighted by the most negative reviewer (who felt the work was marginally below threshold) is the level of empirical evaluation given within the submitted manuscript. The authors did submit a revision and they outline the reviewerÕs points that they have addressed. It appears that if accepted this manuscript would constitute the first peer-reviewed paper on the subject of this new software package (Edward). Based on both the numeric scores, the quality and potential significance of this work I recommend acceptance.

Decision: Accept (Poster)