Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The paper proposed a method to design a neural sampler that minimizes the fisher score discrepancy.
strength_and_weaknesses: 
Weaknesses: 
-- Some aspects of the paper are not clear to me. It seems these things are accepted in the community though. For example, why the need for a separate scoring network and the score estimation step in general ? If I am modeling p_\theta anyways, why can I not just use it directly? The authors jump between using s() and log p as the score function, which makes sense but the two notations and statements like "if we asynchronously estimate the sampler’s score function perfectly." tells me there is more to this that is not clear from the way it is written.

-- 

-- Please explain "asynchronous" estimation. When I hear "asynchronous", I tend to think of it in context of parallel or distributed optimization. 

-- The derivative using s_d (x) below Figure 2 is not clear to me. What is s_d (x) ? Did I miss where it is defined ? 

-- Minor: There are some typos and clarifications required, please proof-read .e.g. 
"seems work" --> "seems to work". 
"More specially"
What is \lambda in Prop 1 ?
Please mention what FSD stands for /before/ using it.

clarity,_quality,_novelty_and_reproducibility: The paper is not clear to me. Even the motivation about using Fisher discrepancy, and the various steps to correct it including annealing and then corrections. 

I am unable to see the relevance to the community and the novelty.

i do not doubt the reproducibility
summary_of_the_review: There are several aspects which are unclear to me, and hence I can not recommend acceptance. Further, I am worried about the relevance to the iclr community. I will reserve my judgement till rebuttals.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper proposed an implicit sampler by minimizing the Fisher divergence and then propose to improve it by using the annealing technique. Specifically, the author proposed an alternative objective such that minimizing it is equivalent to minimizing the Fisher divergence. Later, the author tried to solve the distant mode problem by creating a sequence of annealed distributions from simple Gaussian to target distribution. Theoretically, the author showed the equivalence of minimizing S2D (the proposed one) to Fisher divergence and also shows that the proposed implicit sampler can be used as an initialization for MCMC chains. 

Empirically, the author conduct 1 synthetic experiment and 1 Bayesian regression to demonstrate the effectiveness of the proposed approach.
strength_and_weaknesses: ## Strength
The paper is clearly written apart from the proof of Proposition 2. So in general, the paper is easy to read and understand. Although it is well-known that the Langevin sampler can be controlled by KL divergence and Fisher divergence is stronger (i.e. upper bounding) than other discrepancies, it is still good the formalize it in theorem 1. The proof strategy of theorem 1 is relatively easy to understand. 

## Weakness
After reading the paper, I have several concerns: (1) the usefulness of the proposed approach; (2) the clarity of proposition 2, and (3) the empirical evaluation. 

First, theoretically speaking, Fisher divergence is stronger than many of the well-known divergences and should provide better control over the convergence. However, in practice, it has been found that it is not the case. There have been many works investigating the theoretical properties of using Fisher divergence as the estimator (see [1], [2]). Especially in [2], it has been found that score matching may fail even under the simple student-t distribution. So, I wonder is your proposed approach works under the following unimodal setting: 
1. Implicit sampler is a student t distribution with mean $\theta$
2. target distribution is another student t distribution with mean $\theta_0$, and $\theta$ and $\theta_0$ are far away from each other. 
By minimizing the Fisher divergence, $\theta$ should approach $\theta_0$, is this true in practice (without using the MCMC step afterward)?


For proposition 2, I am not sure I understand it correctly. In the end of the proof, the equivalence is established with the $sg(\cdot)$ operator but in algorithm 1, the $\theta$ is back-propagated through the score estimator and target score. So why there isn't a $sg(\cdot)$ in this case?

For the empirical evaluation, the tasks are too simple to demonstrate the usefulness of the proposed approach. In the introduction, you mentioned the importance of the neural sampler, but I fail to see this based on the current empirical evaluation. For example, sampling from EBM is an interesting experiment since it is well-known MCMC sampling is too slow to train EBM. Another possible experiment is to train VAE where the encoder is the neural sampler. More baseline can also be considered, at least with the neural Stein sampler. 




[1] Barp, A., Briol, F. X., Duncan, A., Girolami, M., & Mackey, L. (2019). Minimum stein discrepancy estimators. Advances in Neural Information Processing Systems, 32.

[2] Gong, W., & Li, Y. (2021). Interpreting diffusion score matching using normalizing flow. arXiv preprint arXiv:2107.10072.
clarity,_quality,_novelty_and_reproducibility: Apart from Proposition 2, the paper is clearly written and easy to follow. However, the empirical evaluation is a bit weak to support the claim made by the paper. In terms of novelty, the proposed S2D seems to be novel to my knowledge. 
summary_of_the_review: This paper provides an implicit sampler by minimizing Fisher divergence. My major concern is its usefulness due to the property of Fisher divergence and clarity of Proposition 2. Empirically, I think more large experiments should be conducted to see its effectiveness to support the claim made in the introduction.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The authors propose a method to sample from a distribution with a known un-normalized posterior, named Annealed Fisher Implicit Sampler.
More specifically, they introduce the loss with the same gradient as the Fisher Divergence, provided a perfect estimate of the sampler's score function.
They train the sampler via alternatively estimating the sampler's score function and updating sampler parameters via the loss.
They employ annealing technique to alleviate sampler's training, with the rationale that the sampler is trained easily on easier target distributions.
They show the annealed sampler performance on a variety of benchmarks. They provide proofs of the propositions and a theorem, as well as experiment details in the appendix.
strength_and_weaknesses: *Strengths*:
- The proposed method is theoretically well-backed. The loss enjoys the same gradient, as (intractable) Fisher divergence, provided a perfect estimate of the sampler's score function.
- The empirical study of the method's performance show that it indeed can capture target densities.

*Weaknesses*:
- The experimental results do not compare to any other methods.
- The clarity of the paper could use some improvements (concrete examples are given in the Clarity section).
clarity,_quality,_novelty_and_reproducibility: Generally, the paper is relatively clear. Notation-wise, the paper is very sloppy at times. E.g., on page 4, section 3.1, the authors introduce $s_d(x)$ without defining it, and the reader has to guess its meaning from the context. In section 2.2, $q_\phi$ is an approximation to $p$, whereas in other sections $p$ is sampler's approximation to the target $q$. The frequent change of notation $\log p(x)$ to $s_\theta(x)$ (being the same thing) and $s_\phi(x)$ (being approximation thereof) is also a bit confusing. 

It is not very clear what MCMC correction step means in practice. How are they performed?

After the Proposition 2, it is mentioned that minimizing S2D loss is the same as Fisher Divergence. It is true only for a perfect score function estimation. In practice, it is approximate, so a mention of that is worthwhile in the bold text.

Typos:
- p. 13, in the definition of $l(x, f, \nabla f)$ -- an unnecesary integration sign.
- p. 14, second line of the second equation block: forgotten $dx$
- p. 16: messing up $p(\tilde{x} | x)$, $q(\tilde{x} | x)$

The authors do not provide any baselines or comparison against other methods besides their own. Even though the reported numbers (true posterior vs their method) are quite low, it would be beneficial to compare against competing methods, especially since the authors do cite them.

The authors provide experimental details, so reproducibility should not be an issue.
summary_of_the_review: To sum up, the paper provides a novel method to sample from a distribution with a known unnormalized posterior. The contribution has potential impact, as sampling from such distribution has wide applications. The paper could use some improvement clarity-wise. It would be benefitial to provide comparison against other methods. Initial score: borderline.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper presents the annealed fisher implicit sampler which trains an implicit sampler (i.e GAN style model) to sample from an unnormalized probability distribution. The model is trained to minimize the Fisher divergence (i.e the score matching objective) between the model samples and the unnormalized distribution. This is an appealing objective for unnormalized distributions as the normalizing constant of the target is not necessary. However, we must be able to compute the the score function (i.e log-probability gradient) of the implicit model which is not tractable. The authors propose to use score matching techniques to learn a score network to approximate this quantity. They then propose a surrogate objective involving the target distribution, the score network, and the implicit sampler to minimize the fisher divergence.

The authors advocate for an annealed training objective which anneals the target from a simple distribution to the desired distribution which improves performance. Also the authors advocate to use a few steps of Langevin MCMC on top of the implicit sampler to correct for errors which they claim improves performance.
strength_and_weaknesses: Strengths:

The method presented in the paper in interesting and the provided theory is detailed and appears correct. I was intrigued but the surrogate objective and its equivalence to the original goal of score matching. The theory was pretty easy to follow as well. 


Weaknesses:

Overall the paper has 2 main weaknesses. First, I do not think the potential issues of the approach are explored in enough detail. Second the empirical evaluation is insufficient and lacking baselines.

Potential Issues:

I am concerned by utilizing an approximate to the score function of the model. This incorporates bias into the training objective. The authors show that when the score network is perfect then we are minimizing FD. How much error (in terms of FD maybe) is tolerable? Is it possible to bound the bias using the FD between the score network and implicit model? I am worried as well that this optimization may run into saddle point issues like with GANs or the neural stein sampler. It might be interesting to see how this approach works in a setting where we can get the exact score function (such as with a normalizing flow model). We could then compare how the normalizing flow performance works when trained with reverse-KL, true Fisher Divergence, and approximated Fisher Divergence (using the score network). Right now its not so clear how these things compare. 

Evaluation: 

Only two tasks are used for evaluation and these are relatively small scale in nature. Existing sampling approaches often present results on distributions with thousands of dimensions. I would be more convinced if the authors included at least one such distribution such as a pre-trained normalizing flow or a log gaussian cox process (two benchmarks which are becoming more standard in the field). Next the, paper is severly lacking baselines. In fact, there are no baselines beyond methods presented in this paper. The authors compare their method to the neural stein sampler. Why did they not compare with it? As well, the direct competitor to neural samplers are interactive methods such as MCMC and SVGD. We should also compare with these kinds of approaches. Sequential Monte-Carlo is currently the gold standard in the field and some such method should be used to give an indication of how well this approach performs to the best out there. Besides this there are a number of alternative neural sampling approaches such as the Path Integral Sampler which are beginning to gain traction. It would also be useful to provide one such comparison. 

clarity,_quality,_novelty_and_reproducibility: The paper is clearly written. I had no issue following the math, the ideas, and the logic. The approach is relatively novel (a highlight of the paper) and if it performed well, would be a very interesting direction for further work and improvement. The paper as it is is not particularly reproducible. The paper lacks experimental details explaining the optimizers used, step sizes, batch sizes, training iterations, etc. Without these, I believe a reader would have a difficult time reproducing this work. 
summary_of_the_review: This paper proposes the Annealed Fisher Implicit Sampler which is a neural implicit sampling method training to minimize a novel objective  which approximates the fisher divergence between the target distribution and an implicit neural sampler. The method is interesting but the empirical evaluation is severely lacking. No baselines are presented besides methods proposed in this paper. As well, the method is not sufficiently explored and its problems (such as bias from score approximation) are not considered. 

This method is interesting, but I believe a more thorough evaluation is necessary to understand how this approach will scale and how it compares to traditional sampling techniques and competing neural approaches.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: 
In this paper, the authors proposed to learn an implicit sampler by minimizing the Fisher divergence. To make the optimization tractable, the score function is then parametrized and learned. To further improve the performence, correction step is also utilized. 

Most of the reviewers believe this method is interesting. 

However, the major concern of the paper is the empirical study. The experiment part does not provide enough evidence to demonstrate the claimed benefits: 1, the competitors are lacking (there have been plenty of sampling methods proposed for unnormalized density, e.g., kernel Stein's method, Langevin dynamics, MCMC, et al,); 2, the applications are restricted to synthetic and Bayesian inference. It will be better if the method can be demonstrated on EBMs. 


justification_for_why_not_higher_score: 
The empirical study is not sufficient to demonstrate the advantages of the proposed method. 

I encourage the authors to consider the suggestions provided by the reviewers, especially the suggestions on the experiments, to improve the quality of the paper.
Decision: Reject