Paper: Learning energy-based models by self-normalising the likelihood (ICLR-2024)

Reviewer comment (Theory):
This paper has a well-motivated idea and contains comprehensive theoretical derivation for understanding the key idea. However, as mentioned by the author, the NCE method is related, it would be nice to have a deeper theoretical connection and comparison with the NCE method. For now, the major comparison is shown by empirical experiments.

Author response:
Thank you for suggesting an investigation of the relationship between the NCE and SNL objectives. We found that the SNL loss resembles one of the generalisations of NCE proposed in a theoretical paper on NCE [2]. We will discuss this paper in the revised version.

Reviewer comment (Reproducibility):
As a novel learning method, it would be nice to have a practical learning algorithm to simplify and illustrate the main idea.

Author response:
We added an algorithm description for our method in appendix E. Thank you for the suggestions.

Reviewer comment (Experiments):
The sensitivity to the choice of proposal should be critical but it is only investigated in low-dimensional cases.

Author response:
We agree that the method will be sensitive to the proposal, especially when scaling it for image modelling. However, we feel that the work required for finding and tuning a correct proposal is a new avenue of work in itself. Indeed, all these papers are mostly focused on scaling existing methods by finding and tuning a proposal (noisy neural proposal in [1] using $l_{IS}$ in our paper or flow proposal in [3] for NCE). Note that we did compare $l_{IS}$ and $l_{SNL}$ with a Gaussian for the 2D toy experiment and the image regression experiments. In such cases, training the EBM was less stable than with our method or NCE.

Reviewer comment (Experiments):
Can you provide more real-world experiments? For instance in generative modeling (without the VAE component) or out-of-distribution detection.

Author response:
We agree that the method will be sensitive to the proposal, especially when scaling it for image modelling. However, we feel that the work required for finding and tuning a correct proposal is a new avenue of work in itself. Indeed, all these papers are mostly focused on scaling existing methods by finding and tuning a proposal (noisy neural proposal in [1] using $l_{IS}$ in our paper or flow proposal in [3] for NCE). Note that we did compare $l_{IS}$ and $l_{SNL}$ with a Gaussian for the 2D toy experiment and the image regression experiments. In such cases, training the EBM was less stable than with our method or NCE.

Reviewer comment (Theory):
In [4], the authors give a very similar result as your theorem 3.1 but for NCE. Is there more theoretical comparisons to be drawn against NCE?

Author response:
The paper you are mentioning also provides more theoretical insights into the loss landscape for NCE. Conducting such a study for SNL would require substantial work but is an interesting new avenue of work but we will integrate it to a revised version of the paper. Thank you for suggesting an investigation of the relationship between the NCE and SNL objectives. We do not know if there is a direct relationship between our loss and the NCE one. However, we found that the SNL loss resembles one of the generalisations of NCE proposed in a theoretical paper on NCE [5]. We will discuss this paper in the revised version.

Reviewer comment (Experiments):
As mentioned in the weaknesses, I think it would be nice to compare SNL against MCMC-based methods (at least Langevin based) with apple-to-apple computational budgets.

Author response:
Due to the lack of time required to implement and tune MCMC-based methods, we will provide a comparison of SNL with MCMC based method using similar computational budgets (ie number of call of a neural networks and time) upon acceptance or for a further iteration of the paper.

Reviewer comment (Novelty):
From my own experience, the most challenging part when training the EBM is to get valid samples from the current fitted distribution to estimate the (gradient of) normalizing constant. Previous works try to solve this problem with different sampling techniques. While this work proposes a linear lower bound, it still needs to estimate the normalizing constant with Monte Carlo based method. Thus, it might not really alleviate the training difficulties.

Author response:
Our experience matches the one of reviewer CuDM, where the difficulty is to get good samples from the current fitted distribution. In MCMC-based methods training the EBM requires long chains or tricks (for instance, keeping a buffer in [1]) to avoid biased training. However, our method does not require sampling from the model to estimate gradients and that is why we believe this method is more flexible and avoids biases training loss.

Reviewer comment (Experiments):
The proposed algorithm introduces a variational parameter b, and it requires to update b together with the energy function iteratively. Then similar to the VAE case, whether there can be a mismatch between the estimate of b and the energy function $E_\theta(X)$. (Not sure whether the $\exp^{-b}$ term will make the training more unstable if b is not well optimized.) Or in other words, how difficult is it to design the schedule of updating b and energy function to make this algorithm work.

Author response:
It is true that the quality of the gradient estimate depends on how close $b$ is to the normalization constant. This is clearly seen in equation (17) of the paper where we rewrite SNL gradients as the likelihood gradients with some negligible terms if $b$ verifies the aforementioned condition :
$$\begin{aligned}
    \nabla_{\theta} \ell_{\text{snl}}(\theta,b) &=  - \frac{1}{n} \sum_{i=1}^{n}  \nabla_{\theta} E_{\theta}(x) - e^{-b+\log{Z_{\theta}}} \nabla \log{Z_{\theta}} \\
    & = \nabla_{\theta} \ell_{\theta} + \nabla_{\theta} \log{Z_{\theta}} (1-e^{-b+\log{Z_{\theta}}}). 
\end{aligned}
$$
In practice, we train both $\theta$ and $b$ within the same iteration and very quickly during training, $b$ gets very close to $\log{Z_{\theta}}$ thus this did not require more tuning from our side. Note that we treat the extra parameter $b$ as the bias of the last layer of neural network and as such as any other parameter weight of the neural network, optimized using Adam.

Reviewer comment (Experiments):
As also mentioned in Point 2, the modeled distributions in the experiments are too simple to be convincing to me. The modeled experiments are either unconditional distribution on toy data or with image input but only model the conditional distribution on some low dimensional label. The VAE experiment in 5.3 models binary MNIST (which is also not very complex). And with the help of encoder and decoder, the latent space might be more simple. (Besides, what if we train the model VAE-EBM not with $l_{snl}$ but with plain MLE loss? There seems to be included as a baseline in Table 5.) I think in order to make the proposed algorithm more convincing, the authors need to demonstrate better results than pure MLE loss on more complex distributions like real image (face or cifar or SVHN).

Author response:
Though we agree that the experiments are low-dimensional, we disagree that they are not convincing enough. Indeed, to the best of our knowledge, this is the first time an EBM is used for density estimation (and actually provides an upper and lower bound of the likelihood) on the UCI datasets. These datasets usually require complicated networks (autoregressive flows for instance), whereas we reach similar results with much simpler networks and a simple Gaussian proposal. Similarly, we show state-of-the-art performance on EBMs for image regression which is in itself useful [3] for visual tracking or pose-estimation in computer vision. 
Finally, the point of the paper is to introduce a new method for training general EBMs, without a specific focus on high-dimensional data. As it was done for other methods in the field, scaling such methods requires tuning and tricks that are a completely new avenue of research.

Reviewer comment (Novelty):
The review for EBM study seems to be insufficient, may consider the following works:

[1] Improved contrastive divergence training of energy-based models.

[2] Learning energy-based models by diffusion recovery likelihood.

[3] A tale of two flows: Cooperative learning of langevin flow and normalizing flow toward energy-based model.

Author response:
We added the recommended paper to the references, we want to thank the reviewer for their valuable suggestions.