Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: summary: This paper provides an efficient framework to estimate energy based model (EBM). 
While conventional KL or general f-divergence-based contrastive divergence learning suffers from instability and non-convergence issues, the proposed framework namely pseudo-spherical contrastive divergence (PS-CD) can be implemented as SGD type algorithm whose sample complexity and convergence property are provided. 
PS-CD includes the conventional CD as a special case and thus provides gradient-based optimization framework for CD. 
Experiments to evaluate the effectiveness of the proposed approach are conducted with both synthetic and image benchmark datasets. 
main_review: Originality: The proposed method introduces the PS score in statistical decision theory for learning EBM. The proposed method, PS-CD, is defined as 1/\gamma times the negative logarithmic PS scoring rule for parameter \gamma. The advantage of PS-CD is that a consistent estimator of its gradient can be easily computed, allowing EBMs to be trained by gradient methods. This is an advantage of using the PS scoring rule, an approach that has not been seen in previous learning methods. 
Since PS-CD reproduces ordinary CD by adjusting the parameter \gamma, it can be interpreted that the proposed method is a generalization of CD and also provides learning by gradient method.

Quality: This paper technically sounds. Although the main contribution of the paper is the proposal of a learning framework and the investigation of its theoretical properties, minimal experimental results and discussion are also presented to evaluate the effectiveness proposed method.

Clarity: This paper is well written. It clearly explains why a more general framework is needed (what are issues with conventional methods), the process of deriving the proposed method, and its theoretical considerations.

Significance:For the learning of EBMs discussed in this paper, the CD method, which is the maximum likelihood learning based on KL divergence, and the f-EBM, which minimizes the more general f-divergence, have been usually used. These methods had difficulties such as unstable learning and complex variational computation. The presented work that avoids these issues and enable learning by the gradient method is considered to have a high impact (
the derived sample complexity and convergence rate show that the proposed method can theoretically achieve the similar performance as that of learning by ordinary SGD). 

Questions:
- In the experiment, the authors discuss the effect of the parameter \gamma in the case of model misspecification. How much of the effect of the model-misspecification can be absorbed by adjusting the \gamma? From the experimental results, I think that the adjustment of \ gamma does not contribute much to it.

- While negative \gamma values seem to be important from the experimental results, Section 3.3 points out that they may not be strictly proper (scoring rule). If so, how does a negative \gamma affect theoretical results derived in this paper?

- I think it would be better to explain the key idea of convergence analysis based on biased gradient estimator in the main text.
limitations_and_societal_impact: While it provides a general framework for learning EBMs, it is not as good as SOTA methods in terms of FID score (although I believe that it is not appropriate to evaluate generative models with a single score).
I believe that there is no negative social impact from this research.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper presents a generalization of the Contrastive Divergence that uses pseudo-spherical scoring rules to define a new family of loss functions for generative modeling. This family of objective functions has an auxiliary loss parameter that can balance the trade-off between diversity and quality. The method is applied to learn EBMs on image benchmark datasets.
main_review: The main contribution of this paper is to define a new framework for EBM learning by situating the classical CD objective function within a continuous spectrum of loss functions. While a similar concept has been used to generalize EBM learning using $f$-divergences, the new framework presented in the present work is distinct from prior formulations. Furthermore, the PS-CD method proposed in this work encompasses CD as a special case for a certain hyperparameter setting. The theoretical derivation of a novel EBM loss is quite original and the strongest aspect of the paper. 

The proposed method is evaluated on benchmark datasets commonly used in generative modeling. The authors show that adjusting the value of $\gamma$ to be non-zero tends to give significantly better results than standard CD learning corresponding to $\gamma = 0$. The authors observe good results for both positive and negative non-zero values of $\gamma$, and I am unsure of the explanation behind this. I am not sure that conditional Cifar-10 is the best benchmark because it is much less common than unconditional Cifar-10. Unconditional Cifar-10 might be more appropriate since there are a wider variety of existing benchmarks for comparison. In the experiments presented, the PS-CD learning method appears to yield some moderate improvement in the quality of generated images. I am curious if the authors have techniques for tuning $\gamma$ beyond trying out a few different options.

**Overall**: The proposed PS-CD learning method is original and interesting. Although the empirical results might not show dramatic practical benefits of the learned model, the work is nonetheless a novel contribution and the generalizations of CD used in this work might influence future practitioners.
limitations_and_societal_impact: The authors have adequately addressed the limitations and societal impact of their work.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: The authors propose a spectrum of objectives for learning energy-based models that generalise maximum likelihood. These objectives--known as pseudo-spherical scoring rules--are similar to (persistent) contrastive divergence in terms of their implementation and computational cost, but have an extra degree of flexibility that can boost downstream performance for e.g. image generation.
main_review: **Overall verdict**

This paper contains some solid results, with no obvious technical shortcomings. Personally, I didn't find the method especially exciting and the experiments were slightly limited, leaving me with concerns about the significance/impact of the paper. Nevertheless, I lean towards acceptance.

**Originality + Quality + Significance**

This proposed method, PS-CD, is similar in spirit to f-EBM [1] insofar as it offers a spectrum of losses that are suitable for training energy-based models. The key differences to f-EBM are:
 
1) The losses are not based on f-divergences, but rather spherical scoring rules, which correspond to Bregman divergences.
2) Unlike f-EBM, PS-CD does not require us to solve a minimax problem, and therefore has the same computational cost as persistent contrastive divergence.

I think these contributions are solid, and the accompanying theory & experiments are sufficient to convince me that the method is correct and potentially useful.

I don't think this paper constitutes any kind of fundamental advance for energy-based modelling, but it provides an extra 'knob to twist' when learning EBMs, which *may* give us more control over the kinds of statistical patterns extracted by the EBM. This final claim is supported by the experiments, where we see modest improvements in FID scores compared to f-EBM.

A couple of issues/questions (in order of importance):

1) I found the experiments to be quite limited (2d synthetic density estimation +  image generation on 3 datasets). I would have been much more convinced if you had considered other applications e.g. out-of-distribution detection or joint energy-based modelling and classification as in [5].

2) In line 81, you cite [2] as a 'recent advance...to obtain samples from an EBM'. This is misleading, since the point of that paper was that short-run "samples" are not actually valid samples from the EBM (although they can still be high quality). I think this point needs emphasising in the paper, and you should make it clear when reporting results whether or not the samples are likely to be valid. I also think its good practice to report the quality of samples that are obtained after running the MCMC sampler for a long time (e.g. tens of thousands of time steps).

3) I think you should clarify in the main text exactly how your convergence analysis builds on [3] & [4], which you describe as 'prior theoretical works for analysing SGD'. Is it fair to describe theorem 5 as essentially an application of the existing theory? Or are there techniques used in your proofs that are sufficiently non-obvious that readers will learn something by reading them?

4) The theoretical results rely on the sample size growing exponentially with K (which bounds the energy). I imagine that actually using this number of samples is not possible in practice since K would be too large. Do the authors agree? What range of energy values are observed in the experiments? If I am right, then this means the theoretical guarantees have quite limited practical relevance.

5) Equation 19 relies on self-normalised importance sampling. Have you tried measuring the variance / effective-sample-size of this estimator?

**Clarity + prior work**

I think the paper is very well structured & written, and that the references to prior work are extremely thorough. There has been a lot of literature published on EBMs in the last couple of years, and I think the authors cite most of the key papers.

** Minor points **

- Personally, I don't think Theorems 2 & 3 warrant the title 'Theorem'. Just stating the equations would be sufficient for me.

** References **

[1] Lantao Yu, Yang Song, Jiaming Song, and Stefano Ermon. Training deep energy-based models with f-divergence minimization.  InInternational Conference on Machine Learning, pages10957–10967. PMLR, 2020.

[2] Erik Nijkamp, Mitch Hill, Song-Chun Zhu, and Ying Nian Wu.  Learning non-convergent non-persistent short-run mcmc toward energy-based model. InAdvances in Neural Information Processing Systems, pages 5233–5243, 2019

[3] Jie Chen and Ronny Luss.  Stochastic gradient descent with biased but consistent gradient estimators.arXiv preprint arXiv:1807.11880, 2018

[4] Saeed Ghadimi and Guanghui Lan. Stochastic first-and zeroth-order methods for nonconvex stochastic programming.SIAM Journal on Optimization, 23(4):2341–2368, 2013.

[5] Grathwohl, Will, et al. "Your classifier is secretly an energy based model and you should treat it like one." arXiv preprint arXiv:1912.03263 (2019).
limitations_and_societal_impact: I am happy with the author's discussion of limitations and societal impact.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The paper introduces a class of divergence to learn EBMs. The class of divergence includes KL as a special case. The paper approximates the gradient of the class of divergence by self-normalized importance sampling and analyzes the convergence when optimizing with the approximate gradient. Experiments are conducted to evaluate the influence of $\gamma$.
main_review: Pros:
The idea is relatively novel, which extends CD to PS-CD. Under some $\gamma$ PS-CD outperforms CD. The paper is clearly written.

Questions:
1. Do you have a sense how $\gamma$ influences the FID results in Table 1? How to choose a best $\gamma$ in practice?

2. In practice, the training of CD is unstable, e.g., the training is likely to diverge (see [a], Appendix H.3). Will the PS-CD overcome these issues?


[a] YOUR CLASSIFIER IS SECRETLY AN ENERGY BASED MODEL AND YOU SHOULD TREAT IT LIKE ONE
limitations_and_societal_impact: Yes
review_title: null

Meta-review: This paper introduced a class of divergence to learn energy-based models. The proposed divergence class has an auxiliary loss parameter that can balance the trade-off between diversity and quality. The method was applied to image benchmark datasets. The reviewers agree that this is an interesting paper with nice contributions. We conclude that the paper is worthy of acceptance. 
Decision: Accept (Poster)