Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: The paper studies the controlled sequence generation problem based on pretrained language models, i.e., controlling a generic pretrained LM to satisfy certain constraints, e.g., removing certain biases in language models. Specifically, the paper proposes a distributional view and imposes constraints based on collective statistical properties. The problem is formalized as a constraint satisfaction problem, minimizing a divergence objective. The paper proposes to use KL-Adaptive DPG algorithm for approximating the optimal energy-based model distribution. Experiments were conducted over both pointwise constraints and distributional constraints, showing the effectiveness of the model over the compared baselines.
 
Pros:
- The problem under study is an important problem and can have extensive impact on many downstream language generation applications. 
- This paper makes solid contributions by proposing a formal view on generation controlling. It provides a framework to handle pointwise, distributional, and hybrid constraints. 
- The method proposed to sample from the sequential EBM makes sense and is empirically vilified to be effective.
- The experiments and analyses support the claims and conclusions.
- Overall, the paper is well organized and easy to understand. 

Cons: 
- The paper may benefit from some human evaluation for text generation. 
- It is somehow not easy to tell which model is better from figure 2, GDC or Ziegler. It seems that Ziegler is superior in generating attribute-related sentences while inferior in diversity. The sentence quality might be similar as the converged values of (π, a) are close.
- The current submission contains a number of typos, grammatical and other style issues, in both the main sections and appendixes, but these are rather easy to fix.

Questions:
-  For real-life applications, whether the proposed framework has scalability issue; e.g., if a task has a large number of constraints to consider or if the constraints are more complicated than what are tested in Section 3? 
- Assuming one has already got an adjusted LM with some attributes based on GPT2, which would be better if she/he wants to add a new attribute to generation: starting scratch from GPT2 or continuing with the adjusted model?

review_title: Solid paper providing a formal distributional view for controlled text generation and a framework of solution

final_score: rating: 8: Top 50% of accepted papers, clear accept
review_content: The authors addressed my concern so I increased my score to 8. 

-----------------------

This is a very interesting idea for controlling a pretrained model for some sort desired criteria. The authors argue that existing approaches for this have taken a pointwise view for instance using REINFORCE to optimize for a particular reward. This can lead models to over-optimize on the criteria and sacrifice diversity and other criteria. 

The authors instead propose to take a distributional view. Given the pretrained LM distribution a, they would like to find a distribution c as:

p = arg min_{c∈C} D_KL(c, a)

where C is a set of distributions that pass the constraints. Some of these constraints are point-wise but some are distributional. For instance when generating biographies, the authors would like a constraint e.g. X% should talk about a certain gender or occupation. 

The authors describe how their approach leads them to an EBM (energy based model) and subsequent derivations. I think some of this section could be better written for those who are not familiar with EBMs.

The experiments are quite interesting and show how the author's "soft" approach allows them to elegantly adjust the distribution of the LM without degeneration.

Pros:
-Very interesting idea. 
-Thorough experiments. In addition to comparing with REINFORCE based methods,  the authors also compare with CTRL and PPLM in the appendix. 

Cons:
-I think the method section (especially the optimization part)  could be explained better for readers who are not familiar with EBM, and allow the paper to have more accessibility. 
review_title: Novel idea for controlled text generation

final_score: rating: 7: Good paper, accept
review_content: In this paper the authors have proposed a mechanism for controlled text 
generation both pointwise and distributional. That is they not only can
generate each sentences bearing some specified contraint or attribute 
but also takes care of overall property distribution of the generates 
set of sentences. Though pointwise or per sentence level control is well 
explored, the distributional control is a new and promising direction 
which the authors have proposed.

The authors proposed a method Generation with Distributional Control (GDC), 
which is nothing but a constraint satisfaction problem over the probability 
distribution p representing the desired target Language Model. 


Overall I find the problem challenging and promising. This is a nicely written paper. 
However, I have some quetions regarding experimental evaluation.

1. In the Figure 4, the authors have reported the generated sentences controlling
sentiment and also report the frequency of the sentence present in the corpus. 
By corpus does it mean the original training corpus? or the generated corpus by GPT-2?


2. The proposed method is imposing a constraint so that the generation distribution 
becomes closer to the original distribution (in this case GPT-2) and still satisfy the 
pointwise and distributional constraints. If the distributional constraints are not imposed,
the generated sentences should be similar to that of the original GPT-2 generated sentences 
bearing which satisfy the pointwise constraint. What does the freq signifies here? 
The authors should provide some discussion regarding the same. 

3. Are the sentences generated sequentially keeping the context of the previously 
generated sentences? or they do not have any context of the  previously generated 
sentence? If each generated sentences are independent from previously generated 
sentences, how meaningful it is to impose distribution constraint on that ?
review_title: A Distributional Approach to Controlled Text Generation

Meta-review: The paper studies the problem of being able to control text generated by pre-trained language models.
The problem is timely and important. The paper   frames the problem as constraint satisfaction over a probability distribution. Both pointwise and distributional constraints can be imposed. The proposed algorithm,  Generation with Distributional Control (GDC), is elegant, and is an interesting new addition to this line of work. Overall, the paper brings forth news ideas, and could have impact.

Decision: Accept (Oral)