Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper presents a variational inference method with fisher divergence as the objective function to enable very flexible implicit variational posterior. The implicit variational posterior is defined as a hierarchical model that can be viewed as an infinite mixture (i.e., the semi-implicit variational posterior). The motivation for using fisher divergence is that one can get rid of computing the density of the hierarchical variational posterior. To achieve this, the paper proposes to rewrite the fisher divergence with its dual norm formulation. The final objective then has a minimax formulation, which results in an adversarial training algorithm. 
strength_and_weaknesses: ## Strengths

* The idea of rewriting score matching loss using its dual norm is an interesting one---it leads to a nice form that does not require computing the density/score of the hierarchical variational posterior. 

* The authors provide theoretical justification of not solving the inner max problem to the optimum. 

## Weaknesses

* Although directly using score matching loss as the variational objective is relatively under explored, its close relatives---Stein discrepancies----have been studied extensively. 

There is a known connection between score matching loss and Stein discrepancies, see, e.g., 
Liu, Q., Lee, J., & Jordan, M. (2016, June). A kernelized Stein discrepancy for goodness-of-fit tests. In International conference on machine learning (pp. 276-284). PMLR.
Barp, A., Briol, F. X., Duncan, A., Girolami, M., & Mackey, L. (2019). Minimum stein discrepancy estimators. Advances in Neural Information Processing Systems, 32.

It would be nice to contrast the dual form in this paper with Stein discrepancies derived from the same function class (neural networks). There is also work that does this:

Ranganath, R., Tran, D., Altosaar, J., & Blei, D. (2016). Operator variational inference. Advances in Neural Information Processing Systems, 29.

Grathwohl, Will, et al. "Learning the stein discrepancy for training and evaluating energy-based models without sampling." International Conference on Machine Learning. PMLR, 2020.

* The connection I mentioned above seems to suggest that the proposed method can be recovered with minimizing the Stein discrepancy (with neural network function class) between variational posterior and true posterior. Could the authors comment on this?

* The literature review in section 1 is missing the work on implicit variational inference that uses gradient/score estimation techniques. See, e.g., 

Shi, J., Sun, S., & Zhu, J. (2018, July). A spectral approach to gradient estimation for implicit distributions. In International Conference on Machine Learning (pp. 4644-4653). PMLR.

Song, Yang, et al. "Sliced score matching: A scalable approach to density and score estimation." Uncertainty in Artificial Intelligence. PMLR, 2020. 

* Appendix D suggests that UIVI is implemented with batch size 1, which means a substantially weaker baseline. I doubt if this is an unavoidable limitation of UIVI. 

* Experiments do not include multi-modal target distributions. 
clarity,_quality,_novelty_and_reproducibility: This can be a solid and original paper if the weaknesses are sufficiently addressed. The clarify is good but can be improved. One place I found unclear is

* Section 3.1, "Although the hierarchical structure of q(x) allows us to estimate its score function via denoising score matching, the estimated score function would break the dependency ...". I don't understand this. Can you clarify?
summary_of_the_review: Due to the weak points I raised, I currently lean towards rejection. However, I am happy to revise my opinion if these are sufficiently addressed.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper proposes a novel score-matching-based method to learn semi-implicit variational distributions. In semi-implicit models, the score of the variational distribution is challenging to estimate directly and Monte Carlo estimates of the score matching objective are biased due to the loss being a non-linear function of the scores. The authors instead re-write the squared distance between the score of the distribution being approximated and the score of the variational distribution as the inner product of the score difference with the output of a learnable function. The authors put forth a ridge regression style learning objective for this auxiliary function which is optimised when the function matches the score difference and thus the exact squared distance between scores is recovered. At the same time, this expression is now only linear in the score of the variational distribution, allowing for learning with unbiased MonteCarlo estimates of the gradients of variational parameters. This results in a min-max objective, where the auxiliary function is trained to approximate the error in the variational distribution’s score and the variational distribution is optimised such that its score matches that of the desired distribution. 

The authors validate their method on some toy 2d densities, 22-dimensional logistic regression and multinomial logistic regression on the MNIST (7850 dimensions) and HAPT (6744 dimensions) datasets. As baselines, Semi-implicit variational inference (SIVI) and unbiased-implicit variational inference (UIVI) are used. The proposed method performs on par or better than baselines on all tasks. Another advantage of the proposed method is that it admits minibatching, unlike UIVI, which relies on HMC updates.  
strength_and_weaknesses: **Strengths**

* The paper presents a neat idea that makes sense and seems to work reasonably well
* The proposed method is clear and easy to follow. 


**Weaknesses**


 The experiments are relatively simple and do not cover a very diverse range of cases.
* All of the non-toy posteriors considered are convex. This does not allow the reader to get an idea of how well the proposed method would fare under multimodal posteriors. 
 
* The UIVI paper validates their approach using VAEs. These models can present a multimodal posterior distribution over the latent space. Perhaps it could be good to replicate this experiment.


* The authors motivate their method by observing the biasedness of the MC estimate of the Fisher divergence in equation 7. I think that a small experiment showing the unsuitability of this estimator (even in the appendix if there is no space for it in the main text) would make the argument much more convincing.

* As a practitioner, I always struggle to imagine in which types of situations it would be best to apply a more exotic inference method, like the one described in this paper, instead of HMC. I would suggest that the authors include HMC with a compute allowance matching that of the implicit VI methods as a baseline.
	 

clarity,_quality,_novelty_and_reproducibility: **Clarity**

I found this paper to be clear. I have some minor suggestions
* The first paragraph ends (and the second starts) in a slightly strange position. Since the second paragraph talks about the limitations of the factorised approximation, I would suggest introducing this approximation here instead of at the end of the first paragraph. 
* I would suggest that implicit and semi-implicit distributions are defined more explicitly (no pun intended) in the introduction, to aid readers that don't already know what they are. 
* K appearing in the SIVI bound is not defined as far as I can tell
* I did not find the connection that the authors make to demonising score matching to be particularly clear since the proposed method does not perturb the variable of interest. Perhaps the authors can give some more thought to how to explain this connection.
* I would suggest the authors include notation that places more focus on the cancellation of the variational density in the denominator of equation 9. Perhaps this can be done by including it crossed out in the numerator and denominator. This is a very simple step, but I was left confused by it for a bit because of how the prose reads. 
* In algorithm 1 the notation for the L2 norm (i.e. $|| \cdot ||_{2}^2$) is missing for the regularisation of the surrogate function. 
* I did not understand what was being plotted in figure 4 after reading both the caption and the prose that describes it. 

**Novelty**

* To the best of my knowledge the proposed method is novel and non-trivial 

**Reproducibility**

* The authors do not provide code but I think that the provided experimental detail is enough for me to be able to reproduce the experiments. 

summary_of_the_review: I like this paper and would like to see it accepted at ICLR. However, in their current state, the experiments seem to be a bit weak for what I would expect from an ICLR paper. If the authors address the issues described above in this regard I would be happy to raise my score to an 8.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a new approach to semi-implicit variational inference (SIVI). SIVI-type formulations greatly increase the expressiveness of variational posteriors but introduce intractabilities to their inference. Existing methods rely on additional lower bounds on the evidence lower bound (ELBO), or require an expensive Markov chain Monte Carlo (MCMC) estimate of an inner gradient expectation. This work proposes the use of an alternative objective (the Fisher divergence), which, combined with the hierarchical nature of semi-implicit variational distributions can be approximated as the solution (Nash equilibrium) of saddle point optimization problem. This insight leverages the established link between denoising and score matching.
strength_and_weaknesses: 
The clarity of the writing and the soundness of the proposed technique are definite strengths of this work, as is the novelty. These are discussed in further detail in the subsequent section. 

One significant weakness I wish to highlight is the conceptual motivation behind this approach at a higher level. Presumably, the appeal of SIVI-type methods is that one is able to circumvent the need to solve these minimax/saddle point optimization problems, the instability and difficulty of which have long plagued GANs and their derivative methods. SIVI-type enables one to steer clear of this problematic paradigm altogether, but the proposed method brings us right back to where we started. Although the experimental results indicate benefits over traditional SIVI-type methods, it's difficult to assess how this approach will generalize beyond the three relatively simple problems considered here. I'm interested to hear what the authors have to say and would like to see this discussed in the manuscript. Is there something about the formulation (Fisher divergence, score matching objective, denoising) that fundamentally changes the problematic nature of this minimax game?

Secondly, at a more fundamental level, this paper adopts an unconventional paradigm of variational inference, namely, minimizing the Fisher divergence. This is not a well-studied case (unlike other divergences in the f-divergence family beyond the reverse KL divergence, such as the alpha-divergence). The only other case of this that I've seen is Yang et al. 2019 (https://arxiv.org/abs/1905.05284) which has been cited in this manuscript but is not widely recognized by the community (has it been accepted by a peer-reviewed publication yet?) This facet should be given more attention. Particularly, in this sense, it is difficult to directly draw comparisons between SIVI/UIVI and this method, since it is solving a fundamentally different problem. To what extent are the improvements derived from using this different objective? It seems this manuscript could benefit from some ablation studies.

### Misc questions and issues


Regarding footnote 2, why not just write the covariance as the scale-identity matrix "\sigma \mathbf{I}"?

In figure 3, in the SIVI-SM pane, the colour of the true and approximate densities are too similar (red and orange, respectively) which makes it far more difficult to discern their differences.  In contrast, the colours representing the other methods in the other panes are quite different to that of the ground truth, which makes the presentation slightly unfair.

In Section 4.2, regarding the covariance estimates of \beta (shown in Figure 4) is this simply \sigma? Please clarify.

Typos: 
- "log-likeilhood"
- "in term of"
- "get ride of"
- "ground truth _were_ formed"
- "all _the_ three SIVI variants"
- number of missing commas all throughout the text. The manuscript could generally benefit from a few more passes of copy-editing.

clarity,_quality,_novelty_and_reproducibility: 
The manuscript is presented very clearly and the writing is of high quality (definitely above the average ML conference paper). In particular, as someone who is quite familiar with the literature, I found the authors did a superb job with the introductory and background sections up to and including Page 3, which were very well-written, comprehensive, and succinctly summarizes the narrative of this line of work up to the current point. The exposition of the proposed method was also quite clear, of high technical quality and substantiated by proofs where necessary. The experiments were conceived and executed well, the descriptions were clear and included the details necessary for reproduction, and the results are presented nicely (I especially appreciated the qualitative examples and visualizing the role of the auxiliary network in the quiver plot). The only thing I would say is that the problems considered are all fairly toy by today's standards. I would've liked to see more challenging problems being tackled by the proposed method.

There is no mention of source code availability, nor indication it will become available upon publication. This is will be crucial for reproducibility, particularly for methods that are notoriously finicky (minimax optimization problems).

The novelty is clear. This provides an interesting and practical technique for solving the Fisher divergence minimization problem by reducing it to another optimization problem and leveraging existing neat tricks such as denoising score matching.
summary_of_the_review: While I am not entirely convinced about some fundamental aspects of this work (as discussed in "Weaknesses") I am leaning toward recommending acceptance of this paper. I am looking forward to discussions of the points raised.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: Following semi-implicit variational inference (SIVI), the paper proposes to use a semi-implicit variational distribution to approximate the posterior of model parameters. Different from SIVI that optimizes the semi-implicit variational distribution with an asymptotically-exact surrogate ELBO, the paper proposes to match the score of the posterior (i.e., gradient of the log posterior) with the score of the semi-implicit variational distribution. The key insight is that the semi-implicit construction allows the author to arrive at a tractable min-max objective for optimization. This derived objective, which cleverly combines SIVI and score matching, is new and novel. Experimental results on posterior inference for Bayesian (multinomial) logistic regression and Bayesian neural networks show that the proposed score-matching-based SIVI outperforms previous ELBO-based variational inference methods, including SIVI and UIVI.
justification_for_why_not_higher_score: The reviewers have concerns on the extra computation and optimization difficulties brought by the min-max objective.
justification_for_why_not_lower_score: This paper points out a new path to construct expressive variational posterior with tractable inference.
Decision: Accept: notable-top-25%