From: ICLR.cc/2022/Conference/Program_Chairs
title: Paper Decision
decision: Reject
comment: The paper introduces a way of making Ratio Matching (RM) scale better to high-dimensional data when training energy-based models (EBMs). The main idea is to estimate the sum over the datapoint dimensions in the RM objective with importance sampling (IS), achieving computational savings by using fewer samples than dimensions. A key part of the method is a proposal that uses gradient information w.r.t. discrete variables to efficiently approximate the optimal (minimum variance) proposal, resulting in much better performance compared to uniform sampling. The authors also introduce a biased version of the estimator that samples from the same proposal but drops the importance weights when averaging over the samples, which, somewhat surprisingly, outperforms the unbiased version.

The idea of using Monte Carlo estimation based on importance sampling to speed up Ratio Matching is novel and sound. The use of gradient information to approximate the optimal IS proposal is also novel in this context, though the idea of using gradients this way to reduce the number of EBM energy function evaluations comes from Grathwohl et al. (2021), where it was used to speed up Gibbs sampling.

While the method is well described, the paper is insufficiently rigorous in several places, most importantly in claiming that Eq. 4 corresponds to Ratio Matching, which is not true. Eq. 4 is instead equivalent to the objective for Generalized Score Matching (GSM) given by Eq. 17 in (Lyu, 2009). Crucially, while both GSM and RM recover the true model if the model class is nonparametric/unconstrained, as is stated in (Lyu, 2009) (and thus agree with each other as well as with maximum likelihood estimation) ), they do not yield the same solution for constrained model classes such as neural networks. This means that the method in the paper implements GSM and not RM. Unlike RM, which has been used widely in the literature, GSM is essentially empirically unproven and thus is a less interesting choice. The main difference between the GSM and RM objectives is the presence of the squashing function g(u) = 1/(1+u) around the probability ratios in RM to avoid division by zero, when the probability in the denominator is vanishingly small (as is explained above Eq. 12 in (Hyvarinen, 2007)). This means GSM is likely to be prone to stability issues due to using probability ratios directly. This is one possible explanation for the puzzling empirical results in the paper, where the proposed sampling-based methods outperform the exact method they are supposed to approximate, with the biased method clearly performing best. The intuition-based arguments made in the paper to explain these results are not convincing and need to be improved upon. While, as the authors pointed out in their response, it is possible to apply the strategy in the paper to RM by applying IS to Eq. 3 instead of Eq. 4, that would be essentially a different paper.

One example of puzzling experimental results is Figure 3, which shows that the base method ("Ratio Matching") does not find the correct solution while the proposed approximate methods do. This suggests that there is something wrong either with the method (e.g. with the objective, as mentioned above) or with the experimental setup. In either case, the cause needs to be thoroughly investigated.

Currently the empirical evaluation is primarily MMD based, relying on sampling from the model using MCMC. Ensuring that MCMC chains mix sufficiently well to sample from the true distribution by visiting all of its modes is difficult, and it is important to provide some evidence that this was done. As suggested by a reviewer, the results would be substantially strengthened by reporting the log-likelihoods for the models, estimated e.g. using AIS, even if that requires including scaled-down versions of some of the experiments.

The title of the paper is misleading and should be changed because the proposed method is specific to EBMs for binary data, even if the intent is to extend it to other types of discrete data in the future.

The clarification and additional results provided by the authors to the reviewers and the AC were appreciated, but unfortunately the outstanding issues with the paper are too major to allow acceptance at this point. The main idea of the paper has substantial promise however, and the authors are encouraged to develop it to its full potential by addressing the points from this meta-review as well as the additional ones from the reviewers.

Bibliography correction: Hyvarinen is the solo author of "Estimation of Non-Normalized Statistical Models by Score Matching". Peter Dayan was the editor of that paper and not a co-author. Please correct your bibliography.

From: ICLR.cc/2022/Conference/Paper729/Authors
title: Hope we solve "why better optimization happens"; The suggested motivation is possible but still a personal preference 
comment: Dear Reviewer Btjs,

(1) We believe our last response has addressed you concern about "why better optimization happens", from both the intuition side and the empirical side (Appendix E).


(2) We respectfully argue that the suggested motivation and presentation flow are possible but belong to **personal preference**. Again, we don't have a strong motivation to switch to the suggested flow, given that we already have a convincing motivation and presentation, as we detailed in our previous responses.

----
Since the discussion period will end soon, we hope that **the reviewer could carefully reconsider the significance and the reasonability of our current presentation**. We heartily appreciate the reviewer taking time to actively discuss with us. Thank you!

From: ICLR.cc/2022/Conference/Paper729/Authors
title: Thanks
comment: Dear Reviewer oca8,


Thank you for your response and for recognizing our work as an interesting initial step towards extending ratio matching on learning discrete energy-based model. As suggested, we will discuss the limitation/future work in our next revision. We will also consider making the title to be more rigorous. Thank you for your positive comments and constructive suggestions.

From: ICLR.cc/2022/Conference/Paper729/Reviewer_oca8
title: Thanks for the response
comment: Thank you for the response to my feedback. I think it clarified some of my questions and I appreciate the author's effort on updating the draft according to my suggestion. A major concern/limitation - the scalability to more complicated/practical settings such as categorical data still remain as a future work. So I tend to keep my original rating and I do think this paper is an interesting initial step towards extending ratio matching on learning discrete energy-based model.

A side comment: I think the title is a bit overclaiming - the method only applies to binary data, not any discrete EBMs, and the gap to general discrete EBM seems quite large.

From: ICLR.cc/2022/Conference/Paper729/Authors
title: Better optimization happens naturally; The suggested motivation is possible but not as convincing as our current presentation.
comment: Dear Reviewer Btjs,


Thank you for the response. We appreciate you recognizing our motivation and improvement from the efficiency perspective. 



(1) *“Many questions remain regarding exactly why better optimization happening.”* This is quite natural and straightforward since our method focus on neighbors with low energies. Specifically, according to the objective function of ratio matching (i.e., Eq. (4)), neighbors with lower energies contribute more to the objective function (i.e., the loss values for these neighbors are larger). That’s why our methods, focusing on reducing the loss values (i.e., pushing up energies) of these neighbors, indeed lead to lower value for the objective function defined in Eq. (4). This has been verified by the experiment in Appendix E, as suggested by the reviewer. Hence, it is technically natural and empirically correct that better optimization happens.



(2) *“I'm not sure why this is not a grounded motivation”*. As noted by the reviewer, we have achieved the consensus that both the perspective of hard negative mining and efficiency are quite important considerations. (A) As clarified in [our previous response](https://openreview.net/forum?id=IEKL-OihqX0&noteId=rkJT4twwbd) and agreed by the reviewers (`KK3T`, `oca8`, and even in the initial review of `Btjs`), our current presentation motivated from the efficiency perspective is already clear, reasonable, and convincing. (B) The reviewer suggests that our presentation should be motivated from our empirical observation on hard negative mining (*”you can just state that you find this to be true empirically.”*). **This is possible but is not as clear/reasonable/convincing as our current presentation.** As clarified in [our previous response](https://openreview.net/forum?id=IEKL-OihqX0&noteId=rkJT4twwbd), we don’t have an exactly clear and strong intuition/motivation to apply hard negative mining as in objective detection. That’s why we use hard negative mining as a post-hoc explanation instead of a motivation. On the other hand, we indeed claim that *we find this to be true empirically* in Appendix E. We just didn’t start our presentation with this claim, but we actually present this point and discuss it quite extensively. 


To conclude, we cannot convince ourselves to use the suggested motivation based on hard negative mining, given that we already have a convincing motivation and presentation (i.e., the efficiency perspective).


----
We are happy that we gradually arrive at some consensus with the reviewer. Again, we heartily appreciate the reviewer taking the time to join the discussion. Please let us know if the above responses answer your questions. Thank you.

From: ICLR.cc/2022/Conference/Paper729/Reviewer_Btjs
title: Thanks
comment: To be clear, I acknowledge the motivation and improvement from the efficiency perspective. That is the reason I'm keeping the score of 5 instead of further lowering it.

While the improvement in efficiency is nice, to me the most surprising bit is actually the better optimization (reviewer oca8 similarly finds this part surprising). Currently this very important point is glossed over. I agree with oca8 that the intuitive explanation is reasonable, but at least to me many questions remain regarding exactly why this is happening, and I believe this point is potentially more important than the efficiency considerations. Current efficiency considerations are grounded in the vanilla ratio matching. But as experiments show, vanilla ratio matching is far from optimal in terms of performance, and there are many variations that easily outperform vanilla ratio matching (regardless of efficiency considerations). 

Finally, I'm not sure why this is not a grounded motivation. This is an interesting and surprising new finding, and you can just state that you find this to be true empirically (some additional theoretical analysis would also be nice). And this is not contradictory to the motivation from efficiency considerations.

From: ICLR.cc/2022/Conference/Paper729/Authors
title: Motivation should be problem-oriented instead of result-oriented
comment: Dear Reviewer Btjs,

We respectfully argue that **the motivation should be problem-oriented, instead of result-oriented**. In our manuscript, our method is motivated from the efficiency perspective, which is a well-explained problem of the original ratio matching. In contrast, you focus on the obtained improved performance (i.e., result-oriented) and hold that we should focus on the motivation from the perspective of hard negative mining, **with ignoring our strong motivation and improvement from the efficiency perspective**. As explained in the last response, focusing on hard negative mining as suggested is not a grounded motivation. Hence, we believe that our current presentation is much more clear, reasonable, and convincing. 


Please let us know if this help us to achieve a better consensus on the presentation. Thank you so much.

From: ICLR.cc/2022/Conference/Paper729/Authors
title: Focusing on hard negative mining as suggested is not a grounded motivation
comment: 
We appreciate the reviewer for the response.


*“The current presentation is misleading, and would lead the readers to focus on the wrong points.”* We totally and respectfully disagree with the word “misleading”. **In fact, (1) our current presentation and the perspective of the reviewer are NOT completely contradictory. (2) And we argue that our current presentation is much more clear, reasonable, and convincing.**


(1) We have achieved a consensus with the reviewer that the perspective of hard negative mining is a quite important consideration. It can serve as a reasonable and convincing explanation to support our method. We have indeed described this perspective extensively in Section 3.3, Appendix C, and Appendix D.


(2) The difference between our views is that (2A) we use hard negative mining as a convincing explanation to support our proposed methods, while (2B) the reviewer holds that our method should be completely motivated from the hard negative mining perspective.


(2A) In our current manuscript, our method is motivated from the efficiency perspective of ratio matching, and our proposed method indeed achieves much better efficiency in terms of both time and memory. We observe that our method ALSO achieves better performance, thus we explain this performance improvement using the perspective of hard negative mining, which is considered to be reasonable and convincing by the reviewer (and the other reviewers, such as Reviewer oca8). Hence, our current presentation is clearly motivated, and the derivation and explanation are sound. 


(2B) Differing from our presentation, the reviewer holds that we should focus on the motivation from the perspective of hard negative mining. However, **this is not a grounded motivation**. Specifically, we cannot explicitly say that “The performance of ratio matching would be improved if we apply hard negative mining to ratio matching. Hence we should focus more on the neighbors with low energies.” This is not a grounded motivation. It is just a random guess before we exactly observe performance improvement when applying hard negative mining. In contrast, in computer vision (specifically, object detection), the motivation of hard negative mining is convincing since we have the PRIOR knowledge that we usually have an overwhelming number of easy negative samples and a small number of hard negative samples [1]. However, we do not have this prior in our case of discrete EBMs, and cannot treat the perspective of hard negative mining as a clear and strong motivation.


To conclude, we argue that the motivation should be problem-oriented. In our manuscript, our method is motivated from the efficiency perspective, which is a well-explained problem of the original ratio matching. In contrast, hard negative mining is a specific technique to solve or explain the problem, thus it cannot be treated as a strong motivation. Hence, we believe that our current presentation is much more clear and convincing.



-----
We sincerely thank the reviewer for this discussion. We hope the above response solves your concern. Please let us know if you have further confusion about this point. Thank you.





[1] Shrivastava, Abhinav, Abhinav Gupta, and Ross Girshick. "Training region-based object detectors with online hard example mining." Proceedings of the IEEE conference on computer vision and pattern recognition. 2016.

From: ICLR.cc/2022/Conference/Paper729/Reviewer_Btjs
title: Reply
comment: I have carefully checked the response and revisions, including Section 3.3. My concerns remain. I don't think it's a penalty for presentation flow. In my opinion the current presentation is misleading, and would lead the readers to focus on the wrong points.

I appreciate the added discussions and analysis, but I believe the author should address the concerns in a new version where the main focus  is on the hard sample mining perspective. Despite the (only marginally) positive reviews from other reviewers, I maintain that this paper is not ready for publication in its current form and would recommend rejection.

From: ICLR.cc/2022/Conference/Paper729/Authors
title: Looking forward to your reply
comment: Dear Reviewer Btjs,


Thank you for your previous constructive and active discussion. We have responded to your latest concern about the presentation, and revised our draft accordingly. Since the discussion period will end soon, could you kindly check our response and revision (especially Section 3.3)? Currently, the other three reviewers are all positive about our work. We are looking forward to knowing if we have properly addressed your concerns? Thank you so much.

From: ICLR.cc/2022/Conference/Paper729/Authors
title: Looking forward to your reply
comment: Dear Reviewer DiXS,


We have responded to the raised questions and updated our paper accordingly. Since the time for discussion is limited, could you kindly check our responses and revision? We look forward to your feedback on if we have properly addressed your concerns? Please consider improving the rating score if your questions are addressed. Thank you.

From: ICLR.cc/2022/Conference/Paper729/Authors
title: Looking forward to your reply
comment: Dear Reviewer oca8,


Thank you for your positive comments and for the interesting questions. We have responded to your raised questions and have updated our manuscript. Could you check our responses and let us know if we have properly addressed your concerns? Please consider improving the rating score if your questions are addressed. Thank you.

From: ICLR.cc/2022/Conference/Paper729/Authors
title: Looking forward to your reply
comment: Dear Reviewer KK3T,


We made efforts to respond to the initial comments and updated our manuscript accordingly about two weeks ago. Since the discussion period will end soon, could you check our responses and let us know if we have properly addressed your concerns? Any additional questions and comments are warmly welcome. Also, please consider improving the rating score if your concerns are well addressed. Thank you.

From: ICLR.cc/2022/Conference/Paper729/Authors
title: Thanks (response from authors)
comment: We heartily appreciate the reviewer for the activate and constructive discussion.



We agree that hard negative mining is a quite important consideration. And in our revision, we do spend a lot of space to discuss the perspective of hard negative mining (as in Section 3.3, Appendix C, and Appendix D.). We also have to point out that Theorem 1 is useful to derive our methodology since it provides a great proposal distribution (from both the perspectives of importance sampling and hard negative mining) and it can be approximated efficiently using the gradient information and Taylor series.


“The current paper reads like with a series of coincidences we stumble upon a method with good performance that we don't fully understand.” We have rewritten our method part (in Section 3.2 and 3.3, especially Section 3.3). We believe it is more convincing now. Please check it.



As suggested by the reviewer, we will think about how to achieve better presentation by balancing the discussion of Theorem 1 and hard negative mining. At the same time, we also believe that the current draft is quite easy to follow, convincing, and sound (according to the feedback of Reviewer oca8 and Reviewer KK3T), although we use a different writing flow as suggested by the reviewer. Again, we sincerely appreciate the reviewer’s helpful feedback and active discussion. And we hope that the reviewer could reconsider the score penalty associated with the presentation flow. Thank you.

From: ICLR.cc/2022/Conference/Paper729/Reviewer_Btjs
title: Thanks
comment: The terminology is fine. My main concern is that from the presented results, it seems like this paper is missing the most important point (hard sample mining). The experiments demonstrate that:

1. Giving up some optimality in terms of estimator variance won't hurt performance (and would in fact improve performance).
2. Moving towards hard sample mining always gives better performance.

At least to me, this suggests hard sample mining should be a more important consideration. Some optimality results from the perspective of hard sample mining, some considerations on efficient computations (already done with the presented approximations) and some more analysis on the better optimization of Eq (4) with hard sample mining would be more valuable than Theorem 1.

The current paper reads like with a series of coincidences we stumble upon a method with good performance that we don't fully understand. I don't find the overall presentation to be convincing.

From: ICLR.cc/2022/Conference/Paper729/Authors
title: Responses from authors
comment: Thanks for the quick response. We do really appreciate it.


**In short, Theorem 1 is only for the unbiased version. And the biased version can be treated as a natural extension of the unbiased version.**



*  **The biased version is a natural extension of the unbiased version.** Our derivation of the biased version from unbiased version is clarified as follows. We firstly connect the better performance of our unbiased version over ratio matching to hard negative mining (Section 3.3 Point (2)). Then, based on this insight, we propose the biased version by removing the coefficients in the unbiased version because they provide smaller weights for terms with lower energies, which is less effective (Appendix D). Hence, our biased version is a natural extension of our unbiased version. These two versions are not contradictory with each other. We have made this clearer in our revision (**at the end of Section 3.3, highlighted in brown text**).


* **We noticed that the confusion might be caused by the terminology we used.** The word **“biased”** could sound like to be contradictory to the word **“unbiased”**, thus weakening the consistency relationship between our unbiased version and “biased” version. We would like to replace it with other words, such as **“advanced version”**, if the reviewer suggest so.




* As suggested by the reviewer, we may construct other proposal distributions for our “biased” version. We use our current one since it is efficient to obtain. We can use the gradient information and Tylor approximation to construct it. (Other potential proposal distributions might not have this advantage.)


-------

Thank you and look forward to your reply.

From: ICLR.cc/2022/Conference/Paper729/Reviewer_Btjs
title: Thanks for the response
comment: I thank the authors for the response.

- In theorem 1 you establish optimality for the estimators as a whole, including the coefficients. The biased version is omitting the coefficients and as a result a different estimator. The fact that the biased version outperforms the unbiased version means that while Theorem 1 can be a consideration (and might be an important one), it shouldn't be the only consideration, as we provably have estimators that are not optimal according to the theory but outperforms the theoretically optimal estimator.
- I get the flow, but I don't like the coincidence part. As mentioned above, I think the variance can be a valuable consideration (and might very well be an important one), but is not the only factor driving the performance. The real factor driving the performance still seems unclear to me, and I believe more analysis/investigation is needed, more specifically from the claimed hard sample mining perspective.

From: ICLR.cc/2022/Conference/Paper729/Authors
title: Responses from authors
comment: Thank you for your response. We are glad that your previous concerns are well addressed. We address the raised specific question below.


> Given the current explanation, if the important thing is to achieve better optimization and heuristically we are doing hard example mining, in what sense is the distribution given by Theorem 1 optimal? Would it be possible to have some other estimators that have higher variance but achieves better performance? In fact the better performance with the biased version already proves the theoretically optimal choice is not necessarily the best, and indicates that the variance of the estimator is not the most important thing here.


Thank you for proposing this interesting question. The short answer for this question is **Theorem 1 is a must for our unbiased version and the resulting proposal distribution is necessary for our biased version as well.**


-------



We would like to clarify our flow again. We are motivated by the efficiency limitations of the ratio matching method. Hence, we propose to use Monte Carlo method based on importance sampling to unbiasedly estimate the original ratio matching objective more efficiently. That’s why we propose our unbiased RMwGGIS, as defined in Eq. (7). For our unbiased version, Theorem 1 is a must because it provides an optimal proposal distribution, which yields the minimum variance. This is also verified by our experiments shown in **Figure 3**. In **Figure 3**, RMwRAND is also an unbiased estimator but cannot work due to high variance.


Next, although we are motivated from the efficiency perspective, we observe that our RMwGGIS (Here, we refer to the unbiased version) achieves better performance than the original ratio matching method. Thus, we conjecture that this is because our RMwGGIS can be optimized better since it focuses on neighbors with low energies, as analyzed in **Section 3.3**, and performs intuitively like hard negative mining (**Appendix C**). Based on this insight, we further propose our biased version (Eq. (12)) because the biased version also performs hard negative mining (i.e., focus on neighbors with low energies) and does not include the complicated coefficients (We provide an intuitive explanation on why biased version performs better than the unbiased version in **Appendix D**. The reason is actually the complicated coefficients.). Note that Theorem 1 is also necessary for our biased version since the proposal distribution of the biased version also follows Theorem 1 (i.e., the biased and unbiased version use the same proposal distribution). **The coincidence here is that the distribution used for hard negative mining is the same as the optimal importance sampling proposal distribution given by Theorem 1.** This makes sense because in our case both **importance sampling for Monte Carlo estimation** and **hard negative mining** aim to pay more attention on neighbors with lower energies. Their goals are consistent.


The above explanation clarifies why Theorem 1 is necessary for both our unbiased and biased version. To put it another way, if we do not provide Theorem 1 and gives the biased version (Eq. (12)) directly, It would be confusing and unconvincing since it is a biased estimator and we would lack the detailed derivation for the proposal distribution used in Eq. (12). 

-------


**"Would it be possible to have some other estimators that have higher variance but achieves better performance?"** In principle, we can construct other proposal distributions for Eq. (12) to do hard negative sampling. We choose our proposal distribution as Eq. (11) because of the following two points. (1) It can be easily and efficiently obtained by using the gradient information and Tylor approximation. Other proposal distributions don’t have this advantage. (2) The approximated target (i.e., the proposal distribution given by Theorem 1) is sound since it gives higher probabilities for neighbors with lower energies. 



-------

**"In fact the better performance with the biased version already proves the theoretically optimal choice is not necessarily the best."** Not really. Note that for biased version, the proposal distribution is also the same distribution (i.e., Eq. (11)) as the unbiased version. Such distribution is our proposed gradient-based approximation for the optimal distribution given by Theorem 1. Hence, the better performance with the biased version actually shows that the optimal distribution is a good choice.


------

Thank you again for raising this interesting question. Hope we address it well. Please let us know if you have any further concerns.

From: ICLR.cc/2022/Conference/Paper729/Reviewer_Btjs
title: Thanks for the response
comment: I thank the authors for the response. The intuitive explanation using hard example mining does make sense, and it's encouraging to see that the biased version indeed achieves better optimization on the original energy function. But given the current explanation, if the important thing is to achieve better optimization and heuristically we are doing hard example mining, in what sense is the distribution given by Theorem 1 optimal? Would it be possible to have some other estimators that have higher variance but achieves better performance? In fact the better performance with the biased version already proves the theoretically optimal choice is not necessarily the best, and indicates that the variance of the estimator is not the most important thing here.

The paper identifies a potentially interesting approach to a valid problem, and achieves good performance. But I do feel the current presentation is a bit confusing and misleading, and there are aspects that require more careful analysis to better understand the improved performance.

From: ICLR.cc/2022/Conference/Paper729/Authors
title: Summary of the revision
comment: Dear ACs and Reviewers,

We sincerely thank the reviewers for raising the insightful questions and constructive suggestions, which are very helpful for us to strengthen this submission. We have made efforts to respond to the questions and revised our manuscript accordingly. Please kindly check the detailed responses and our revision, where the changes are highlighted with red color. Besides answering the technical questions,

* We provide an additional insight to understand why our method can be optimized better by connecting it with hard negative sampling (`Reviewer Btjs`). Further, we analyze how our Tylor approximation affects the learning process (`Reviewer oca8`). Details are included in **Appendix C** in our revision.


* Based on the above insight, we further provide an explanation on why our biased version performs better (`Reviewer oca8`) in **Appendix D**.


* We include one more baseline in graph generation experiments and empirically verify our methods can actually lead to lower value for objective function, as suggested by `Reviewer Btjs`. Details are provided in **Appendix E**.


* We add an experiment on training Ising models to further demonstrate the scaling ability of our method and compare with recent methods more thoroughly, as suggested by `Reviewer KK3T`. The results and details are in **Section 5.3** and **Appendix I**.


* We included the discussion with more related works, as mentioned by `Reviewer DiXS`.

We would like to know if our responses have properly addressed your concerns. Any feedback and/or additional comments are warmly welcomed.

From: ICLR.cc/2022/Conference/Paper729/Authors
title: Author response
comment: Thank you for your constructive review and for recognizing our work to be interesting, novel, and valid. We address the raised concerns as follows.


> The main weakness of this work is in its empirical evaluation. The authors only validate the method in two settings; toy datasets and small graph datasets. Further, their only evaluation is via MMD. Evaluation of EBMs is difficult, but in small-scale settings likelihood can be estimated with AIS. Likelihood-based evaluation would allow comparison to autoregressive models or VAEs, for example.



Thank you for this comment.



(1) Our existing experiments, i.e., density modeling on synthetic data and graph generation, follow the setting and evaluation metrics of previous works [1,2,3] in the community. Evaluation with MMD is also shown to be valid and consistent in these works.


(2) As suggested below, we add the experiment on training Ising models in **Section 5.3** and **Appdendix I** to show the scaling ability of our method. The details are included in the later response.



> I am concerned with the comparison to Gibbs-With-Gradients training. How was this done? Did you use PCD? If so, what were your buffer size, refresh rate, and the number of steps per training iteration? PCD can be sensitive to these parameters and if not set properly the results could be very bad. The authors should include these experimental details in the paper (at least in the appendix).



Thank you for this feedback. 


Yes, we use PCD by following the official implementation of [4]. In the Appendix I.2 of the original GWG paper [4], it is observed that reinitializing buffer samples to noise will result in unstable training and worse performance. Hence, in our previous experiments, we use buffer reinitializing rate = 0 and tune other sensitive hyperparameters like the number of sampling steps. 


Thank you for reminding us that the buffer initialization rate is also important. We re-tune this experiment by tuning the following hyperparameters, and the chosen ones are highlighted: buffer initialization rate (0, 0.2, `0.4`, 0.6) and MCMC steps (100, 200, `500`, 1000, 2000). The results are updated in **Table 3** and the experimental details are provided in **Appendix G**.



> I am not convinced by the scale of the experiments as well. Prior work [1,2] has successfully trained discrete EBMs on datasets with thousands of dimensions. The results in this work are much smaller-scale. Does the method have difficulties scaling to these settings? If so then that is certainly a weakness of the method and it should be mentioned. You could compare results to the MNIST/Potts/Ising training experiments in [1] (no need to do all, but at least one experiment demonstrating scaling ability would be good to have). This would give a better context for the performance of this method relative to other recent approaches.


Thank you for this suggestion. As suggested, we add the experiment on training Ising models to further demonstrate the scaling ability of our method and compare with recent approaches more thoroughly. Please see **Section 5.3** and **Appendix I** of our revision for more details. We summarize the experiment in the following.


Following one setting in [4], we use RMwGGIS to train the Ising model corresponding to a 2D cyclic lattice structure. We create a model using $25 \times 25$ lattice and use the strength of connectivity $\sigma=0.25$, leading to a $625$ dimensional distribution. We set the number of samples $s$ used in our RMwGGIS objective as $10$. We consider EBM (Gibbs) and EBM (GWG) as baselines. The evaluation metrics are the RMSE (between the learned connectivity matrix and the true connectivity matrix) and the running time per iteration. As shown in Tabel 4 in our revision, we achieve a log(RMSE)=-4.00 with an obvious advantage of efficiency. One future direction is to further improve the effectiveness of our RMwGGIS while preserving the efficiency advantage.


--------
Thank you again for raising these helpful questions! We have revised our paper accordingly. The changes are highlighted in red color. 





[1] Dai, Hanjun, et al. "Learning Discrete Energy-based Models via Auxiliary-variable Local Exploration." Advances in Neural Information Processing Systems 33 (2020): 10443-10455.


[2] You, Jiaxuan, et al. "Graphrnn: Generating realistic graphs with deep auto-regressive models." International conference on machine learning. PMLR, 2018.


[3] Liu, Jenny, et al. "Graph Normalizing Flows." Advances in Neural Information Processing Systems 32 (2019): 13578-13588.



[4] Grathwohl, Will, et al. "Oops I Took A Gradient: Scalable Sampling for Discrete Distributions." arXiv preprint arXiv:2102.04509 (2021).

From: ICLR.cc/2022/Conference/Paper729/Reviewer_KK3T
title: References 
comment: [1] Grathwohl, Will, et al. "Oops I Took A Gradient: Scalable Sampling for Discrete Distributions." arXiv preprint arXiv:2102.04509 (2021).
[2] Dai, Hanjun, et al. "Learning Discrete Energy-based Models via Auxiliary-variable Local Exploration." arXiv preprint arXiv:2011.05363 (2020).

From: ICLR.cc/2022/Conference/Paper729/Reviewer_KK3T
summary_of_the_paper: This paper proposes a more efficient version of ratio matching (RM) for training discrete energy-based models. The proposed method subsamples the dimensions to use in the original RM objective and then uses importance sampling to reduce variance. The importance sampling distribution is based on a Taylor-series approximation to the target energy function which approximates the minimal-variance importance sampling distribution. The authors show that this estimator is considerably more efficient than standard RM and enables RM to be applied to training EBMs in higher dimensions. The authors demonstrate their estimator on some toy datasets and to fit EBMs on graph data. The method is evaluated using MMD.
main_review: Strengths:

Training EBMs on discrete data is a difficult task and few methods currently exist to do so in high dimensions. RM is an appealing option because it avoids the MCMC sampling typically used to train EBMs at scale. The method appears to be a clear win on top of RM (when it can be applied -- not all energy discrete functions can be differentiated). 

Beyond this, the method appears correct and valid. My main concerns lie in the method’s ability to scale to higher-dimensional settings than demonstrated in the paper.

Weaknesses:

The main weakness of this work is in its empirical evaluation. The authors only validate the method in two settings; toy datasets and small graph datasets. Further, their only evaluation is via MMD. Evaluation of EBMs is difficult, but in small-scale settings likelihood can be estimated with AIS. Likelihood-based evaluation would allow comparison to autoregressive models or VAEs, for example.

I am concerned with the comparison to Gibbs-With-Gradients training. How was this done? Did you use PCD? If so, what were your buffer size, refresh rate, and the number of steps per training iteration? PCD can be sensitive to these parameters and if not set properly the results could be very bad. The authors should include these experimental details in the paper (at least in the appendix).

I am not convinced by the scale of the experiments as well. Prior work [1,2] has successfully trained discrete EBMs on datasets with thousands of dimensions. The results in this work are much smaller-scale. Does the method have difficulties scaling to these settings? If so then that is certainly a weakness of the method and it should be mentioned. You could compare results to the MNIST/Potts/Ising training experiments in [1] (no need to do all, but at least one experiment demonstrating scaling ability would be good to have). This would give a better context for the performance of this method relative to other recent approaches. 

----------Post Author Response-------------
I thank the authors for their response to my feedback. I appreciate the new experiments. I think they give additional clarity on how the method performs and scales to higher-dimensional data. I am convinced the method has merits and I will raise my score to a 6. I am still concerned that the method will have difficulties scaling to more complicated problems/models and the current results do not demonstrate that. Having said that, this method is generally applicable and could be a valuable contribution to the community. 

summary_of_the_review: The paper proposes an interesting and novel variant of RM that enables it to get around the memory/compute constraints of the original method. The method appears valid and appears to work but the empirical evaluation is lacking likelihood evaluation, experimental details, comparison to recent approaches and does not demonstrate that the method can scale to relevant problems. Unless these issues can be rectified, I cannot argue for the paper's acceptance. 
correctness: 4: All of the claims and statements are well-supported and correct.
technical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
empirical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
flag_for_ethics_review: NO.
details_of_ethics_concerns: None
recommendation: 6: marginally above the acceptance threshold
confidence: 5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.

From: ICLR.cc/2022/Conference/Paper729/Authors
title: Author response (1st part)
comment: Thank you for your thorough understanding of our work and comprehensive review. We address the raised questions below.


> The proposed approach seems to heavily rely on the assumption that the data is binary, which is not applicable to more realistic data such as natural language modeling (or even graph with different types of nodes and edges?). Indeed, an efficient and effective generalization of ratio matching to non-binary discrete data seems more challenging, interesting and useful.

> All experiments are "toy" data, possibly because the method is limited to model binary data.


Yes, it is indeed non-trivial to generalize our RMwGGIS efficiently and effectively to general discrete data beyond binary data. Generally speaking, algorithms on discrete data usually starts from considering binary discrete data. For example, the original ratio matching method [1] is firstly proposed for binary discrete data, and then is generalized to categorical discrete data by [2]. Our current work covers the binary case and would like to consider extending to categorical data in the future.


> The taylor approximation in eq (8) may not be accurate enough since the difference between x and x_-i is not close to zero. So the approximation error might be larger depending on the structure of the energy function (e.g. non-smooth) and the number of dimension (for d=2, x and x_-i may be very different in the sample space).
Thank you for this interesting question!


(1) Our work is motivated from efficiency (in terms of both time and memory) limitations of ratio matching on **high-dimensional data**. If the number of dimensions is too small, we can use the original ratio matching method directly to learn discrete EBMs since the time and memory requirements in low dimensional cases are affordable. The advantage of our method is that it can achieve both efficiency and performance improvements, compared to ratio matching, in high dimensional cases.


(2) In terms of the Tylor approximation error, we add an analysis in **Appendix C** in our revision. In addition to an analysis of how our Tylor approximation affects the learning process, we also firstly include a connection between our method and hard negative mining, which provides a new insight about why our RMwGGIS can be optimized better.


The basic idea for hard negative mining [3,4,5] is to pay more attention to hard negative samples during training, which can usually achieve better performance since it can reduce false positives. In our setting of discrete EBMs, each training sample $x$ is a positive sample, and its energy should be pushed down. For each positive sample $x$, all $x_{-i}$ for $i=1,2,\cdots,d$ are negative samples, and their energy should be pushed up. Our RMwGGIS with the specific proposal distribution shown in Eq. (11) can **approximately** choose the $x_{-i}$’s that currently have low energies with larger probabilities. This has the same philosophy as hard negative mining. Specifically, in our case, $x_{-i}$’s with low energies are hard negative samples since they are the most offending terms, which are close to the positive sample $x$ and have low energies.


The natural follow-up question is **how accurate is the approximation and how does it affect the learning process?** We answer this question to further show that our RMwGGIS is technically sound. Please refer to **Appendix C** in our revised version for a detailed explanation with figure illustrations (in Figure 3).


**More responses are included in the following 2nd part.**

From: ICLR.cc/2022/Conference/Paper729/Authors
title: Author response (2nd part)
comment: > In table 1, there is no standard deviation calculated from multiple random seeds. Also why biased version is better than unbiased version in most experiments?


(1) In Table 1, we report the MMD scalar value to measure the similarity between distributions given by generated samples and real samples, which is widely used to evaluate the performance of generative models. We choose to use $4000$ samples to compute MMD because it is sufficient to obtained stable and convincing MMD. In other words, if we generate different sets of $4000$ samples and compute the corresponding MMDs, the results are quite stable.


(2) Based on our analysis of the connection between our method and hard negative mining, we provide a possible explanation on why our biased version performs better than unbiased version.


It is obvious that both unbiased version (Eq. (7)) and biased version (Eq. (12)) perform “hard negative sampling”. The difference lies in the coefficients for different terms. Specifically, the biased version gives the same weights to all sampled terms. In contrast, the unbiased version provides a weight $\frac{m(x_{-i}^{(t)})}{n(x_{-i}^{(t)})}$ to each sampled term $x_{-i}^{(t)}$, as shown in Eq. (7). Note that $m(x_{-i}^{(t)})=\frac{1}{d}$ for all terms and $n(x_{-i}^{(t)})$ would be larger if $x_{-i}^{(t)}$ has lower energy. Hence, the unbiased version provides smaller weights for terms with lower energies. In other words, among its selected offending terms (i.e., hard negative samples), it pay least attention to the most offending terms, which could be less effective than biased version with equal weights. This could explain why our biased RMwGGIS usually performs better than unbiased version.


We have included this explanation in **Appendix D** in our revision.


> After obtain the energy function, how do you generate graphs? (I suppose using Gibbs sampling?)


Yes, we generate graphs using Gibbs sampling on the learned energy function. We have made this clear in our revision.


> Potential issue with Table 2: I think you are using the same s (number of proposal samples) for various data dimension from 32 to 2048. However, we typically need larger s for larger dimension d to ensure good performance, and in this case, the speedup/memory saving may not be that large?

Thank you for this feedback. Yes, we use the same s=10 for various data dimensions because we observe that $10$ is enough to achieve good energy function for these data dimensions. It is not usually the case that *we typically need larger s for larger dimension d to ensure good performance* due to the potential optimization difficulty. For example, in our experiment on 256-d dimensional data in Section 5.1, RMwGGIS with s=10 can achieve good performance, while ratio matching with considering all 256 terms cannot, as shown in Figure 4, Appendix F in our revised version. Our Table 2 just gives a rough verification of the efficiency improvements. We agree that the actual improvements should be considered more rigorously. 


--------
Thank you again for these constructive and insightful questions! We have revised our paper accordingly. The changes are highlighted in red color (Note that there are changes in Appendix C, D, and E). Please kindly consider improving the rating score if your concerns are addressed.






[1] Hyvärinen, Aapo. "Some extensions of score matching." Computational statistics & data analysis 51.5 (2007): 2499-2512.


[2] Lyu, Siwei. "Interpretation and generalization of score matching." Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence. 2009.


[3] Felzenszwalb, Pedro F., et al. "Object detection with discriminatively trained part-based models." IEEE transactions on pattern analysis and machine intelligence 32.9 (2009): 1627-1645.


[4] Rowley, Henry A., Shumeet Baluja, and Takeo Kanade. "Neural network-based face detection." IEEE Transactions on pattern analysis and machine intelligence 20.1 (1998): 23-38.


[5] Shrivastava, Abhinav, Abhinav Gupta, and Ross Girshick. "Training region-based object detectors with online hard example mining." Proceedings of the IEEE conference on computer vision and pattern recognition. 2016.

From: ICLR.cc/2022/Conference/Paper729/Authors
title: Author response (1st part)
comment: Thank you for the constructive review. We are encouraged that you find that our work is well motivated and technically sound. We address the raised concerns as follows.


> While the paper demonstrates improvements over vanilla ratio matching, it lacks comparison with other types of methods for learning discrete energy-based models. For example, maximum-likelihood training with MCMC sampling (e.g. with Gibbs sampling or the more advanced Gibbs-with-gradients, which the authors acknowledge is a big inspiration for RMwGGIS), can be easily applied to the same problems, and should be included as additional baselines. While the improvements over ratio matching are nice, if RMwGGIS cannot outperform other readily available methods, then it picked the wrong method to improve upon, and its improvements over ratio matching cannot be justified.


Thank you for this suggestion. In our revised version. we add EBM (GWG) as suggested as a baseline in our graph generation experiment. The comparison is included in the following table. It can be observed that ratio matching and our RMwGGIS perform better than EBM (GWG), an MCMC-based maximum-likelihood training method, on this graph generation task.

| **Method**| *Degree* |*Cluster* |*Orbit* |*avg.* |
| ----------- | ----------- | ----------- |----------- |----------- |
| EBM (GWG)| 0.093|0.027|0.053|0.058|
| Ratio Matching| 0.062|0.066|**0.008**|0.045|
| **RMwGGIS**| **0.044**|**0.059**|0.013|**0.039**|



> I am not convinced by the current arguments in the section **Better optimization**? in Sec. 3.3, and I don't think this point is verified empirically as the authors claim. As the authors mention at the end of page 2, the estimator given by Eq. (3)/(4) is consistent, and if Eq. (3)/(4) is minimized perfectly, the obtained model distribution will capture the data distribution exactly. The authors claim that RMwGGIS, with a biased estimator, performs better than ratio matching due to better optimization. Does that mean that the theta learned with the biased objective function (12) actually results in lower value for the objective function in Eq. (3)/(4)? I.e. somehow optimizing a different objective function (Eq. (12)) leads to better optimization of Eq. (3)/(4) than directly optimizing Eq. (3)/(4)? I would like to see this confirmed in experiments (which is a very simple evaluation). If it's really the case, I would like to see some analysis on why this is the case, since Eq. (12) is picked somewhat randomly, without theoretical justification as Eq. (7). If this is not the case, then there is no better optimization, which means Eq. (3)/(4) is not a good objective function, and the paper should rethink its foundation, or explain where the better performance comes from.

Thank you for this feedback!


Although our current quantitative and qualitative empirical results, as shown in Table 1 and Figure 3 in the revision, can demonstrate that our proposed unbiased and biased RMwGGIS can help to learn better energy functions, we agree that it would be more convincing if it can be shown that **energy functions learned with our methods (Eq. (7) and (12)) actually result in lower value for the objective function in Eq. (4)**. As suggested, we perform a straightforward evaluation to check this.


Specifically, for all the learned energy functions in Table 1, we sample $4000$ data points on each dataset and evaluate the resulting objective value defined by Eq. (4). The results are summarized in the following table. The `1st lowest` and **2nd lowest** are highlighted.



| **Method**| *2spirals* |*8gaussians* |*circles* |*moons* |*pinwheel* |*swissroll* |*checkerboard* |
| ----------- | ----------- | ----------- |----------- |----------- |----------- |----------- |----------- |
| Ratio Matching| 46.02|39.26|31.82|28.57|28.50|37.52|`26.05`|
| **RMwGGIS (unbiased)**| `26.84`|**30.15**|**29.89**|**27.80**|**28.08**|**32.20**|26.09|
| **RMwGGIS (biased)**| **27.15**|`29.31`|`29.54`|`27.75`|`27.30`|`29.45`|**26.06**|


We can observe that our unbiased and biased RMwGGIS indeed achieve lower objective values, which further demonstrates that our proposed RWwGGIS can be optimized better. The results are included in **Appendix E** in our revision.


**More analysis on why this is the case is included in the following 2nd part of our response.**

From: ICLR.cc/2022/Conference/Paper729/Authors
title: Author response (2nd part)
comment: **Why can RWwGGIS be optimized better?** In Section 3.3, we provide two properties to intuitively understand this. (1) RMwGGIS introduces stochasticity; (2) RMwGGIS focuses on neighbors with low energies. **Here, we provide an additional insight to understand why the second property can lead to better optimization, by connecting it with hard sample mining.**


Hard sample mining [1,2] has been widely applied to train deep neural networks [3]. Our RMwGGIS is particularly highly related to hard negative training strategies. The basic idea for hard negative mining is to pay more attention to hard negative samples during training, which can usually achieve better performance since it can reduce false positives. In our setting of discrete EBMs, each training sample $x$ is a positive sample, and its energy should be pushed down. For each positive sample $x$, all $x_{-i}$ for $i=1,2,\cdots,d$ are negative samples, and their energy should be pushed up. Our RMwGGIS with the specific proposal distribution shown in Eq. (11) can **approximately** (will be clarified below) choose the $x_{-i}$’s that currently have low energies with larger probabilities. This has the same philosophy as hard negative mining. Specifically, in our case, $x_{-i}$’s with low energies are hard negative samples since they are the most offending terms, which are close to the positive sample $x$ and have low energies.


Since our proposal distribution defined in Eq. (11) approximates the provable optimal proposal distribution given in Theorem 1, our proposal distribution thus approximately performs “hard negative mining”. The natural follow-up question is **how accurate is the approximation and how does it affect the learning process?** We answer this question to show that our RMwGGIS is technically sound. Please refer to **Appendix C** in our revised version for a detailed explanation with figure illustrations.




--------
Thank you again for raising these interesting and insightful questions! We have included the above empirical results and extensive analysis in our revised version and the changes are highlighted in red color (Most changes are in Appendix C and E). Please let us know if there are any further questions. Please kindly consider improving the rating score if your concerns are addressed.





[1] Felzenszwalb, Pedro F., et al. "Object detection with discriminatively trained part-based models." IEEE transactions on pattern analysis and machine intelligence 32.9 (2009): 1627-1645.


[2] Rowley, Henry A., Shumeet Baluja, and Takeo Kanade. "Neural network-based face detection." IEEE Transactions on pattern analysis and machine intelligence 20.1 (1998): 23-38.


[3] Shrivastava, Abhinav, Abhinav Gupta, and Ross Girshick. "Training region-based object detectors with online hard example mining." Proceedings of the IEEE conference on computer vision and pattern recognition. 2016.

From: ICLR.cc/2022/Conference/Paper729/Authors
title: Author response
comment: 
Thank you for finding our work to be interesting. We make clarifications to the raised issues as follows.


> How to draw samples from your proposal n(x)? 


For each training sample $x$, our proposal distribution $n(x_{-i})$, defined in Eq. (11), gives a discrete distribution over $x_{-i}$, for $i=1,2,\cdots,d$. Note that $x_{-i}$ denotes a point in the data space obtained by flipping the $i$-th dimension of $x$. Then, as shown in Eq (7), we sample $s$ ($s < d$) terms according to the above probability distribution to estimate the original ratio matching objective.


For example, if the data dimension $d=4$ and number of samples we use is $s=2$, for a particular training sample $x$, our proposal distribution $n(x_{-i})$ provides a distribution over $x_{-1}, x_{-2}, x_{-3}$, and $x_{-4}$ (e.g., $n(x_{-1})=0.6$, $n(x_{-2})=0.2$, $n(x_{-3})=0.1$, and $n(x_{-4})=0.1$)). Afterwards, we sample $s=2$ terms according to the above probability distribution to estimate the original ratio matching objective. Specifically, in this example, one possible situation of our sampled two terms is $(x_{-1}, x_{-2})$ since they have larger probability according to our proposal distribution.


> Please clarify the relationship with the papers [1] [2].


Thank you for pointing us to these interesting works. Both [1] and [2] use the gradient information of the target distribution to iteratively optimize the proposal distributions for adaptive importance sampling. Such gradient information is widely used in MCMC framework, such as Hamiltonian dynamics, as review in our Section 4. [1] and [2] exploit such gradient information in the importance sampling framework. These two works are significantly different with ours in the following two aspects. 


(1) **[1] and [2] are designed for importance sampling over continuous distributions, while our gradient-guided importance sampling is proposed for performing importance sampling over discrete distributions.** Similar to gradient-based MCMC methods, [1] and [2] require the gradient w.r.t. the data space to update the parameters of the proposal distributions (e.g., Eq (4) in [1]). Thus, they cannot be applied to discrete distribution directly since the resulting parameters (e.g. mean) in each iteration are usually invalid in the discrete data space. In contrast, we utilize the gradient information to approximate the provable optimal proposal distribution given by our Theorem 1. Our obtained proposal distribution, as shown in Eq (11), is essentially discrete. This point distinguishes our work with [1] and [2] fundamentally.


(2) In addition, in [1] and [2], we have to use iterative update to try to reach the optimal proposal distribution. In our case, we have a provable optimal proposal distribution and then use the gradient information to approximate it in one shot, as shown in Eq (11). Hence, although we might be able to extend [1] and [2] to discrete distribution in the future, they would be much less efficient compared with our method. 

We have included the discussion with [1] and [2] in our revision version (highlighted in red color). Please kindly consider improving the rating score if the above issues are well addressed. Thank you!



[1] V. Elvira, L. Martino, D. Luengo, J. Corander, "A Grandient Adaptive Population Importance Sampler", IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2015.

[2] Ingmar Schuster, Gradient Importance Sampling, arXiv:1507.05781, 2015.

From: ICLR.cc/2022/Conference/Paper729/Reviewer_oca8
summary_of_the_paper: The paper propose a Monte Carlo approximation for the expensive ratio matching method for learning discrete energy-based models. The key idea is to first write the ratio matching objective as an expectation wrt a uniform distribution, then use importance sampling for a more efficient estimation, where the optimal proposal distribution can be approximated by the gradient of the energy wrt the data space. Empirically, such an approximation shows even better results than the original expensive objective, possibly due to regularization from stochasticity and more focus on neighbors with low energies.
main_review: Strengths: 
- The paper is well written and contains detailed method derivation and discussions. Overall, it is easy to follow.
- Nice discussions to related works.
- The method derivation is sound and the proposed solution is a reasonable way to alleviate the expensive computation/memory requirement of the original ratio matching approach.
- Although motivated from computational perspective for fast approximation of an expensive quantity, the proposed method even achieves a better performance than original method in most tasks, which is a surprising result and achieves both computation and performance improvement. The intuitive explanations look reasonable to me.

Weakness/Questions:
- The proposed approach seems to heavily rely on the assumption that the data is binary, which is not applicable to more realistic data such as natural language modeling (or even graph with different types of nodes and edges?). Indeed, an efficient and effective generalization of ratio matching to non-binary discrete data seems more challenging, interesting and useful.
- The taylor approximation in eq (8) may not be accurate enough since the difference between x and x_-i is not close to zero. So the approximation error might be larger depending on the structure of the energy function (e.g. non-smooth) and the number of dimension (for d=2, x and x_-i may be very different in the sample space).
- In table 1, there is no standard deviation calculated from multiple random seeds. Also why biased version is better than unbiased version in most experiments?
- After obtain the energy function, how do you generate graphs? (I suppose using Gibbs sampling?)
- Potential issue with Table 2: I think you are using the same s (number of proposal samples) for various data dimension from 32 to 2048. However, we typically need larger s for larger dimension d to ensure good performance, and in this case, the speedup/memory saving may not be that large?
- All experiments are "toy" data, possibly because the method is limited to model binary data.
summary_of_the_review: Please see the detailed comments section.
correctness: 4: All of the claims and statements are well-supported and correct.
technical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
empirical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
flag_for_ethics_review: NO.
recommendation: 6: marginally above the acceptance threshold
confidence: 5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.

From: ICLR.cc/2022/Conference/Paper729/Reviewer_DiXS
summary_of_the_paper: The authors propose a new gradient-based importance sampling scheme, applied to learn parameter in energy-based models (EBMs).
main_review: The paper seems to contain interesting material. Some point must be clarified.

- How to draw samples from your proposal n(x)? this is the main issue in your method.

- Please clarify the relationship with the papers:

V. Elvira, L. Martino, D. Luengo, J. Corander, "A Grandient Adaptive Population Importance Sampler", IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2015.

Ingmar Schuster, Gradient Importance Sampling, arXiv:1507.05781, 2015.


summary_of_the_review: The paper seems to contain interesting material, however the degree  of novelty seems to be a bit incremental.
correctness: 4: All of the claims and statements are well-supported and correct.
technical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
empirical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
flag_for_ethics_review: NO.
recommendation: 6: marginally above the acceptance threshold
confidence: 3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

From: ICLR.cc/2022/Conference/Paper729/Reviewer_Btjs
summary_of_the_paper: This paper proposes ratio matching with gradient-guided importance sampling (RMwGGIS) as an attempt to address the expensive computation and excessive memory requirement of ratio matching in learning discrete energy-based models, and demonstrates the advantages of RMwGGIS over ratio matching with experiments on density modeling and graph generation.
main_review: ## Strength:

The paper is well motivated, clearly written, and seems technically sound. The paper does a good job in analyzing the limitations of ratio matching in expensive computation and excessive memory requirements, and in describing how it can leverage the gradient of the energy w.r.t. the discrete data space to address these limitations. The idea is simple, but can potentially be very useful in making ratio matching more practical for learning discrete energy-based models.

## Weaknesses:

- While the paper demonstrates improvements over vanilla ratio matching, it lacks comparison with other types of methods for learning discrete energy-based models. For example, maximum-likelihood training with MCMC sampling (e.g. with Gibbs sampling or the more advanced Gibbs-with-gradients, which the authors acknowledge is a big inspiration for RMwGGIS), can be easily applied to the same problems, and should be included as additional baselines. While the improvements over ratio matching are nice, if RMwGGIS cannot outperform other readily available methods, then it picked the wrong method to improve upon, and its improvements over ratio matching cannot be justified.
- I am not convinced by the current arguments in the section **Better optimization?** in Sec. 3.3, and I don't think this point is verified empirically as the authors claim. As the authors mention at the end of page 2, the estimator given by Eq. (3)/(4) is consistent, and if Eq. (3)/(4) is minimized perfectly, the obtained model distribution will capture the data distribution exactly. The authors claim that RMwGGIS, with a biased estimator, performs better than ratio matching due to better optimization. Does that mean that the theta learned with the biased objective function (12) actually results in lower value for the objective function in Eq. (3)/(4)? I.e. somehow optimizing a different objective function (Eq. (12)) leads to better optimization of Eq. (3)/(4) than directly optimizing Eq. (3)/(4)? I would like to see this confirmed in experiments (which is a very simple evaluation). If it's really the case, I would like to see some analysis on why this is the case, since Eq. (12) is picked somewhat randomly, without theoretical justification as Eq. (7). If this is not the case, then there is no better optimization, which means Eq. (3)/(4) is not a good objective function, and the paper should rethink its foundation, or explain where the better performance comes from.

summary_of_the_review: While the paper is promising, given the two major concerns above, I don't think the paper can be accepted in its current form, and recommend weak reject.

---------------------------

While the authors clarified certain points, after the discussions I still have concerns over the overall presentation. Although the method demonstrates promising results, I don't feel the theory is properly explaining the improved performance, and more analysis/investigation is needed to clarify the claimed hard sample mining perspective. I still recommend rejection.

The authors have repeatedly tried to force their conclusions on me in their response. I want to emphasize that I still don't find the presentation to be convincing, and I believe the paper in its current form hasn't sufficiently analyzed an important point it touched upon.
correctness: 3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.
technical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
empirical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
flag_for_ethics_review: NO.
recommendation: 5: marginally below the acceptance threshold
confidence: 3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.