Venue: ICLR 2018 conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: The paper seems to be significant since it integrates PGM inference with deep models. Specifically, the idea is to use the structure of the PGM to perform efficient inference. A variational message passing approach is developed which performs natural-gradient updates for the PGM part and stochastic gradient updates for the deep model part. Performance comparison is performed with an existing approach that does not utilize the PGM structure for inference.
The paper does a good job of explaining the challenges of inference, and provides a systematic approach to integrating PGMs with deep model updates. As compared to the existing approach where the PGM parameters must converge before updating the DNN parameters, the proposed architecture does not require this, due to the re-parameterization which is an important contribution.

The motivation of the paper, and the description of its contribution as compared to existing methods can be improved. One of the main aspects it seems is generality, but the encodings are specific to 2 types PGMs. Can this be generalized to arbitrary PGM structures? How about cases when computing Z is intractable? Could the proposed approach be adapted to such cases. I was not very sure as to why the proposed method is more general than existing approaches.

Regarding the experiments, as mentioned in the paper the evaluation is performed on two fairly small scale datasets. the approach shows that the proposed methods converge faster than existing methods. However, I think there is value in the approach, and the connection between variational methods with DNNs is interesting.
review_title: The paper proposes an approach to perform inference in models that combine probabilistic graphical models with deep networks

final_score: rating: 7: Good paper, accept
review_content: The authors adapts stochastic natural gradient methods for variational inference with structured inference networks. The variational approximation proposed is similar to SVAE by Jonhson et al. (2016), but rather than directly using the global variable theta in the local approximation for x the authors propose to optimize a separate variational parameter. The authors then extends and adapts the natural gradient method by Khan & Lin (2017) to optimize all the variational parameters. In the experiments the authors generally show improved convergence over SVAE.

The idea seems promising but it is still a bit unclear to me why removing dependence between global and local parameters that you know is there would lead to a better variational approximation. The main motivation seems to be that it is easier to optimize.

- In the last two sentences of the updates for \theta_PGM you mention that you need to do SVI/VMP to compute the function \eta_x\theta. Might this also suffer from non-convergence issues like you argue SVAE does? Or do you simply mean that computation of this is exact using regular message passing/Kalman filter/forward-backward?
- It was not clear to me why we should use a Gaussian approximation for the \theta_NN parameters? The prior might be Gaussian but the posterior is not? Is this more of a simplifying assumption?
- There has recently been interest in using inference networks as part of more flexible variational approximations for structured models. Some examples of related work missing in this area is "Variational Sequential Monte Carlo" by Naesseth et al. (2017) / "Filtering Variational Objectives" by Maddison et al. (2017) / "Auto-encoding sequential Monte Carlo" Le et al. (2017).
-  Section 2.1, paragraph nr 5, "algorihtm" -> "algorithm"

review_title: More efficient variational inference with structured inference networks

final_score: rating: 7: Good paper, accept
review_content: This paper presents a variational inference algorithm for models that contain
deep neural network components and probabilistic graphical model (PGM)
components.
The algorithm implements natural-gradient message-passing where the messages
automatically reduce to stochastic gradients for the non-conjugate neural
network components. The authors demonstrate the algorithm on a Gaussian mixture
model and linear dynamical system where they show that the proposed algorithm
outperforms previous algorithms. Overall, I think that the paper proposes some
interesting ideas, however, in its current form I do not think that the novelty
of the contributions are clearly presented and that they are not thoroughly
evaluated in the experiments.

The authors propose a new variational inference algorithm that handles models
with deep neural networks and PGM components. However, it appears that the
authors rely heavily on the work of (Khan & Lin, 2017) that actually provides
the algorithm. As far as I can tell this paper fits inference networks into
the algorithm proposed in (Khan & Lin, 2017) which boils down to i) using an
inference network to generate potentials for a conditionally-conjugate
distribution and ii) introducing new PGM parameters to decouple the inference
network from the model parameters. These ideas are a clever solution to work
inference networks into the message-passing algorithm of (Khan & Lin, 2017),
but I think the authors may be overselling these ideas as a brand new algorithm.
I think if the authors sold the paper as an alternative to (Johnson, et al., 2016)
that doesn't suffer from the implicit gradient problem the paper would fit into
the existing literature better.

Another concern that I have is that there are a lot of conditiona-conjugacy
assumptions baked into the algorithm that the authors only mention at the end
of the presentation of their algorithm. Additionally, the authors briefly state
that they can handle non-conjugate distributions in the model by just using
conjugate distributions in the variational approximation. Though one could do
this, the authors do not adequately show that one should, or that one can do this
without suffering a lot of error in the posterior approximation. I think that
without an experiment the small section on non-conjugacy should be removed.

Finally, I found the experimental evaluation to not thoroughly demonstrate the
advantages and disadvantages of the proposed algorithm. The algorithm was applied
to the two models originally considered in (Johnson, et al., 2016) and the
proposed algorithm was shown to attain lower mean-square errors for the two
models. The experiments do not however demonstrate why the algorithm is
performing better. For instance, is the (Johnson, et al., 2016) algorithm
suffering from the implicit gradient? It also would have been great to have
considered a model that the (Johnson, et. al., 2016) algorithm would not work
well on or could not be applied to show the added applicability of the proposed
algorithm.

I also have some minor comments on the paper:
- There are a lot of typos.
- The first two sentences of the abstract do not really contribute anything
  to the paper. What is a powerful model? What is a powerful algorithm?
- DNN was used in Section 2 without being defined.
- Using p() as an approximate distribution in Section 3 is confusing notation
  because p() was used for the distributions in the model.
- How is the covariance matrix parameterized that the inference network produces?
- The phrases "first term of the inference network" are not clear. Just use The
  DNN term and the PGM term of the inference networks, and better still throw
  in a reference to Eq. (4).
- The term "deterministic parameters" was used and never introduced.
- At the bottom of page 5 the extension to the non-conjugate case should be
  presented somewhere (probably the appendix) since the fact that you can do
  this is a part of your algorithm that's important.

review_title: This paper presents an interesting algorithm to perform variational inference with amortized and natural gradient updates of models that contain deep neural network and probabilistic graphical model components. The contributions of the paper need to be more clearly presented and the experiments could be more thorough.

Meta-review: Thank you for submitting you paper to ICLR. The paper presents a general approach for handling inference in probabilistic graphical models that employ deep neural networks. The framework extends Jonhson et al. (2016) and Khan & Lin (2017). The reviewers are all in agreement that the paper is suitable for publication. The paper is well written and the use of examples to illustrate the applicability of the methods brings great clarity. The experiments are not the strongest suit of the paper and, although the revision has improved this aspect, I would encourage a more comprehensive evaluation of the proposed methods. Nevertheless, this is a strong paper.
Decision: Accept (Poster)