Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 6: Weak Accept
review_content: This work is an interesting extension of Gutmann and Hyvarinen (2010), where the parametric model is the combination of a noise model (language model) and an energy function (residual energy), so the difference of parametric model and the noise model cancels out the noise model. Therefore optimizing (3) under some conditions converges to a set of parameters of the parametric model (P\theta(x) here) that best describes the data.

One important assumption of Gutmann and Hyvarinen (2010) is that there exists a set of optimum parameters for the parametric model such that the probability of data and the parametric models match for these optimum parameters. This should be mentioned in Theorem-1.  

Does Theorem-1 need extra parameters to act as a normalization constant in order for the theorem to hold at the optimum?
log P_lm(x) - E(x) + const = log p_data

To sample from the model, the authors first sample from the language model and re-sample it with respect to the energy values of the residual model.


To compute the perplexity, they have given an upperbound and lowerboud for the partition function based on number samples in Theorem 2, but I haven't checked the correction of the bounds.  They also factorize the joint model in auto-regressive factorization to compute the perplexity by approximate marginalizing. 


As mentioned in Section 5, this approach heavily depends on a strong pretrained language model. 

Have you considered improving the language model during training?

The described idea is simple and effective and I really liked it.


--- Based on other reviews and the authors' response (especially review #3), I reduced my rating to 'Weak accept'. 
review_title: Official Blind Review #2

final_score: rating: 6: Weak Accept
review_content: The authors make good points, starting from the exposure bias and label bias suffered by the mainstream neural auto-regressive models.
Residual EBMs are defined and trained using NCE. Experiments on two large language modeling datasets show that residual EBMs yield lower perplexity and generation via importance sampling is of higher quality, compared to locally normalized baselines.

In generally, the paper is well motivated and interesting. But I have some concerns.

1. Missing important relevant references.

EBMs (a.k.a. un-normalized models, random fields) have been successfully developed in language modeling in recent years. A large body of this paper has been studied in [5,6], including the model and the NCE estimation method. The model Eq.(2) is exactly the model in [5], defining the model in the form of exponential tilting of a reference distribution.
Connecting and comparing to these previous works are needed.

[1] R. Rosenfeld, S. F. Chen, and X. Zhu, “Whole-sentence exponential language models: a vehicle for linguistic-statistical integration,” Computer Speech & Language,  2001.
[2] B. Wang, Z. Ou, and Z. Tan, “Trans-dimensional random fields for language modeling,” ACL, 2015.
[3] B. Wang, Z. Ou, and Z. Tan, “Learning transdimensional random fields with applications to language modeling,” IEEE transactions on pattern analysis and machine intelligence, 2018.
[4] B. Wang and Z. Ou, “Language modeling with neural trans-dimensional random fields,” IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), 2017.
[5] B. Wang and Z. Ou, “Learning neural trans-dimensional random field language models with noise-contrastive estimation,” IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2018.
[6] B. Wang and Z. Ou, “Improved training of neural trans-dimensional random field language models with dynamic noise-contrastive estimation,” IEEE Spoken Language Technology Workshop (SLT), 2018.

2. I am a little bit concerned that the theoretical contribution seems weak. 
Though Eq. (4) and (5) seem to be novel, I am not sure whether such a contribution is substantial enough to motivate acceptance.

I'm happy to adjust the score if the paper can be better placed in the literature and the authors take efforts to improve the paper.

--------update after reading the response-----------
Being well-placed in the literature and properly claiming contribution with respect to prior work is one of the key questions in reviewing a paper. The first version of the paper clearly lacks in this respect. That's the main concern when I gave a 1.

I appreciate the authors' response. The updated paper has been improved to address my main concern, although the added discussions presented in the updated paper is not as clear as the authors' clarifications in the response. I suggest to polish the main text incorporating these clarifications.

Generally, it is nice to see the successful application of energy-based/random-field-based models in text generation, besides in speech recognition. I update the score to 6 (Weak Accept).

It would have been better that the following can be further clarified.

"the partition function estimated via importance sampling would lead to bias favoring the random field language model" --- this comment is not clear to me. 

Both Eq.4 and Eq.5 give estimates for perplexity. It would be better to clarify different uses of the two equations. If the perplexities are estimated using Eq.4 (as in Table 1), then what is the purpose of developing Eq.5?

How to calculate the lower and upper bounds of the step-wise perplexity gain at each position in Figure 1?

Under Figure 1, "At each position the lower and upper bounds (see Eq. 4) are estimated using 20,000 samples." But in the main text, it is said that "We therefore break down perplexity per position in the generated sequences as in Eq. 5" at page 8. It is confusing.
review_title: Official Blind Review #3

final_score: rating: 6: Weak Accept
review_content: 
Contributions:

The main contribution of this paper lies in the proposed Residual Energy-based Model (EBM) for text generation. Traditional normalized models operate at the token level with MLE training, while the proposed EBM operates at the sentence level. Therefore, BERT, or RoBERTa, can be leveraged for EBM design. The residual energy function is trained via conditional NCE, which reduces to training a binary classifier to discriminate between real text and text generated by an auto-regressive language model. After model training, text can be generated by top-k joint sampling. Experiments are conducted on two large language modeling datasets. The proposed model achieves lower perplexity, and preferred by human evaluation.  

Strengths:

(1) Writing & Clarity: This paper is well written, easy to follow, and clearly presented. I enjoyed reading this paper. 

(2) Novelty: The proposed model contains some novelty inside. It is framed in a residual EBM framework, though by the end, the residual energy function reduces to training a binary classifier to discriminate real and fake text. Though simple, this idea is wrapped up in a nice framework. It is also interesting to observe that this sequence-level EBM regularization can be considered as a way to fine-tune BERT for the text generation task.

(3) Experiments: Generally, the experiments are comprehensive. Detailed analysis, and human evaluation is also provided.

Weaknesses:

(1) Clarity: I have some concerns regarding the selection of baselines, with details shown below.

This paper is basically about using BERT as a binary classifier, which serves as a residual energy function to regularize a pre-trained language model and provides sequence-level supervision. The experiments are comprehensive, but on the other hand, it is also quite expected that the proposed model should work better than an MLE baseline, since sequence-level supervision is provided. 

I think if the authors want to make a stronger paper, they should also compare with other possible ways to inject sequence-level supervision. For example, a simple solution is to use GAN, like in a SeqGAN setup. And the discriminator in the GAN will be the same BERT-based binary classifier. In this GAN setup, sequence-level supervision is also provided. 

Then the difference is that in the GAN setup, the BERT-based binary classifier is a discriminator, but in this paper's setup, it is a residual energy function. It would be interesting to discuss and conduct experiments to see which way is better. 

(2) Experiments: I have some concerns regarding the experimental setup. 

a) One of the main results is Table 1, which reports all the PPL numbers. However, reporting PPL results is less interesting, because we also care about the diversity of generated samples. Lower PPL does not necessarily mean higher-quality text. Though Figure 2 provides some analysis on the diversity, a more comprehensive evaluation on this will be appreciated. 

b) It will be good if the authors can also provide some generated samples for qualitative analysis. 

Overall, I think this paper is well executed. The paper is well written, and experiments are carefully conducted. However, on the other hand,  I also think the conclusion in this paper is expected, it only shows that the proposed model is better than an MLE baseline. 

review_title: Official Blind Review #1

Meta-review: This paper proposes a Residual Energy-based Model for text generation.

After rebuttal and discussion, the reviewers all converged on a vote to accept, citing novelty and interestingness of the approach.

Authors are encouraged to revise to address reviewer comments.
Decision: Accept (Poster)