Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 6: Weak Accept
review_content: The paper introduces an approach to the conditional generation of text, relying on pre-trained decoders, without fine-tuning and, in certain cases, without any training at all. The approach they introduce is following the framework known in NLP as noisy-channel modeling, previously standard in machine translation (in its SMT days), but undergoing certain revival recently (https://arxiv.org/abs/1611.02554, https://arxiv.org/abs/1910.00553,https://arxiv.org/abs/1908.05731,https://arxiv.org/abs/1907.06616). The authors do not mention this connection (they should!).
Very differently from these previous approaches attempting to integrate the two factors in the search process (e.g., using reranking), the authors instead rely on gradient descent in the latent space of their model (Transformer), similarly to plug-n-play generative networks in image generation.  

I find this approach interesting and like the paper overall. However, I do not see why authors do not compare to more direct ways of integrating the conditional component into the model. This would have been tricky in the NMT papers mentioned above, as the entire source sentences need to be reconstructred, however, it should be quite straightforward in this work, with conditioning on single categorical control variables (or maybe a couple in the additional experiments in sect 4.4). Especially, given that the authors already make the predictions of the control variable independently per prediction (e.g., see eq. (5) in section 4.2) / greedily per prefix (bottom lines, page 7). I would actually expect the proposed approach to work better (or at least differently) but it would be interesting to see it confirmed. E.g., for the experiments defining topics as sets of seed words (section 4.2), when integrating factors directly (unlike the proposed approach, Table 3), there will be no increase in the probability of generating relevant words before the first seed word is generated. 

Another limitation is the lack of comparison to standard controlled generation work, i.e. those requiring training a model or/and fine-tuning pretrained decoder. I understand that the proposed approach falls in a different category and, of course, do not expect it to beat a fine-tuned model, but I'd like to get some feel for how much one loses by using this simpler method. There has been a lot of work on controlled generation in recent ~3 years, and they can also be combined with intializing and fine-tuning off-the-shelf pretrained decoders.

There is an interesting relation to the NIPS 2019 paper: https://arxiv.org/abs/1907.04944  They also rely on gradient descent to steer a pretrained language model. Their goal is to assess the degree of 'steerability' rather than building a controlled-generation model.

Given that style-controlled but otherwise unconditional generation may not have that many applications, I am curious how far you can push this approach. E.g., can you make it scale to more complicated data-to-text generation tasks (https://www.aclweb.org/anthology/D17-1239/)? Or, will the only application in this context be integrating new conditioning variables into pretrained conditional LMs? 

Minor: I am confused with the notation in "Post-norm Geometric Mean Fusion" section.  It says that softmax is applied to the product of probabilities. Maybe to a linear interpolation of log-probs? Or maybe that's not softmax at all? Something seems off here.

review_title: Official Blind Review #3

final_score: rating: 3: Weak Reject
review_content: The paper proposes a Plug and Play LM model for controlled natural language generation. Similar to the idea of the Plug and Play Generative Networks for vision, the model plugs in a discriminator, which is either a bag-of-words model or a single layer classifier. The added simple discriminator is then coupled with a pre-trained generative language model such as GPT-2, to obtain a conditional probability for generating controllable text. The authors evaluate the proposed model using human evaluation studies and quantitative perplexity metrics, aiming at measuring the relevance and fluency of the generated text. Their experimental results show that the text generated is fluent and aligned with the desired attributes.  

The proposed method is simple and makes sense to me. The idea of how one can make good use of large, pre-trained  generative language models is very neat here. However, I have two main concerns, as follows.

1. The main focuses of the generated text seem to be dramatically changed in an unpredictable way while tailoring the control attributes. In this sense, how useful these kinds of text generation techniques are not clear to me. For example, the first two rows in Table 3 contain two paragraphs with very different main ideas to be conveyed. Similarly for sentences in Table 1. It seems that those sentences talk about very different topics/things to me, although they may reflect the desired control attributes.  Is there an automatic evaluation metric to subjectively evaluate the change of the focuses/ideas of two pieces of text?

2. The model is a straightforward adaption of the Plug and Play Generative Networks from the vision community. 

In short, the idea in the paper is simple and seems effective. On the other hand, the lack of a good evaluation metric makes me a bit uncertain about the contribution of the paper. I am willing to increase my evaluation score if I will be convinced by other reviews and comments.  

review_title: Official Blind Review #1

final_score: rating: 6: Weak Accept
review_content: The authors describe a method for training plug and play language models, a way to incorporate control elements into pre-trained LMs. In contrast to existing work, which often trains conditioned upon the control element, the authors emphasize that their method does not require re-training the initial LM. This is exciting and a great research direction. It is evaluated in a number of different settings.

1. The authors claim that this method is a baseline for controlled text generation (see e.g. the title). However, there does not appear to be any evaluation with any existing work that performs controlled text generation. I don't see how this can be proposed as a baseline for controlled text generation is there is no comparison to other methods. I imagine the authors will emphasize that that's not fair - because their method doesn't require retraining the language model - but it is relevant to demonstrate if there is a gap in performance or not. As is, there is only one baseline- unconditional language model - and to me this is mostly a way to calibrate the evaluators and not a way to compare their model against other models. 

2. Can the authors make a point or discuss the relationship of this work to neural style transfer? Compared to unsupervised style transfer approaches, which also use lists of words or attributes to learn to dis-entangle content and style, what are the benefits of the proposed approach and how would it compare?

3. Can the authors discuss the effectiveness of their control mechanism for less logical control settings? For example, what if there was "religion" for "the potato" prompt? Does the model still respect these settings, or no? 

4. Can the authors add analysis on how much the model respects the control variables? This is quite common in existing controlled generation papers. If the model is updated to have the control variables and then is not provided with one at test time, what happens? Can you also control very easy to measure attributes, such as length?

This question ties in with a general point I am ambivalent to in this paper- that it is very long, but there is very little analysis done on what makes the method work, why it is better than other control methods or control baselines, where the proposed control mechanism is not effective, how the model scales if there are large quantities of topics rather than just a few of them, if the BoW and discriminator attribute models work well together or if certain attributes are easier to learn than others, so the model focuses more on those when there are conflicts, etc

5. Missing citations: 

Previous work has investigated controlling various attributes of text generation. Several of these works have also controlled multiple attributes simultaneously. For example, here's a list of a few of the works that were missed:

Kikuchi et al 2016
Ficler and Goldberg, 2017
Wang et al, 2017
Fan et al, 2018
Baheti et al, 2018
See et al, 2019
Martin et al, 2019

The related work section only focuses on very recent work, e.g. only one paper is discussed amongst a large body of existing work. I feel this is not an accurate reflection of how much previous work has investigated these techniques and analyzed how models deal with control variables. 

Please also cite:
- which dataset was used for story generation, appears to be missing
- top-k sampling 


I have read the author response. Thanks for the details and additional analysis in the paper. 
review_title: Official Blind Review #2

Meta-review: This paper proposes a simple plug-and-play language model approach to the problem of controlled language generation. The problem is important and timely, and the approach is simple yet effective. Reviewers had some discussions whether  1) there is enough novelty, 2) evaluation task really shows effectiveness, and 3) this paper will inspire future research directions. 

After discussions of the above points, reviewers are leaning more positive, and I reflect their positive sentiment by recommending it to be accepted. I look forward to seeing this work presented at ICLR.
Decision: Accept (Poster)