Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a training and data free approach to perform controllable generation with autoregressive models. The key aspect of this approach is to modify the logits/probability of the attribute-specific tokens by a temperature-like hyperparameter (called Gamma) and rescale the softmax distribution at each step. The approach is empirically compared on length, topic, and sentiment controlled generation with prior work on autoregressive controlled generation.
strength_and_weaknesses: Strengths:

-- The approach is reasonable and easy to implement.

-- This approach seems to outperform popular baselines in terms of fluency and faithfulness to attributes on the explored settings of length, topic, sentiment control.


Weaknesses:

-- Missing references/discussion with prior work: The instantiation of the framework boils down to simply heursitically adjusting the probabilities produced by the softmax layer of certain class sensitive tokens at each step. This exact technique has been used for controlled generation in prior works like [1], [2], [3], [4] and this paper doesn't cite, discuss, or empirically compare the proposed approach with these works. 

Some discussion on comparison with more distant but additional related work on controlled generation including but not limited to [5], [6], [7] would enhance the paper.

-- The approach requires access to a vocabulary list: All the experiments in the paper assume that controlled generation is mainly governed by word lists which is a restrictive assumption. It is questionable if all controlled generation tasks can be predicated on word-lists and the utility of the framework is hence limited.

-- Multiple attributes section is insufficiently explained. How exactly are the probabilities recalibrated. If there are T attributes, are the probabilities adjusted in a sequential manner in some order of the attributes 1 through T? While the attribute specific adjustment makes sense, I am not sure how the probabilities of non-related tokens are adjusted under this scheme and if they result in a valid probability distribution at the end of the rescaling process. Also, how is this procedure affected if there is vocabulary overlap between the attributes?

-- The experiments are done with GPT2-small which is an odd choice because most of the related work that the paper compares to uses the 345M parameter GPT2. Elaboration on this choice needs to be made.

-- More egregiously, the PPL fluency metric uses GPT2-small as well. It would be better if the PPL values are reported by running the samples through a different language model than the one used for decoding. For example, [5] uses GPT-xl and [6] uses GPT for LM oriented fluency metrics.

-- Writing is awkward and several benign but unsupported claims are made. For example, the motivation uses non-linearity of human perception of images wrt. intensity but this connection is not explored empirically or theoretically any further. Similarly, while discussing Fig 4, questionable claims about tuneability are made which are not supported or elaborated upon.

-- For evaluation of sentiment, topic controlled generation, the authors deviate from the evaluation setup by removing tests on subjective topics like religion and politics and constraining the evaluation to more formal topics. While I appreciate the ethical concerns cited behind this decision, these concerns should be described in greater detail to justify removing such experiments. An appropriate replacement for subjective topics would have improved the evaluation.

-- While describing, Table 3, it is claimed that GS-M has the "most stable" quality but while the human evaluation shows high scores, the PPL is bad for GS-M. Please elaborate on this.

-- More experimental settings involving word lists (as done in related work) would strengthen the paper.

[1] REALTOXICITYPROMPTS: Evaluating Neural Toxic Degeneration in Language Models, 2020, Gehman et al.

[2] POWERTRANSFORMER: Unsupervised Controllable Revision for Biased Language Correction, 2020, Ma et al.

[3] Affect-LM: A Neural Language Model for Customizable Affective Text Generation, 2017, Ghosh et al.

[4] A Plug-and-Play Method for Controlled Text Generation, 2021, Pascual et al.

[5] Mix and Match: Learning-free Controllable Text Generation using Energy Language Models, 2022, Mireshghallah et al.

[6] FUDGE: Controlled Text Generation With Future Discriminators, 2021, Yang and Klein

[7] Controllable Text Generation with Neurally-Decomposed Oracle, 2022, Meng et al.
clarity,_quality,_novelty_and_reproducibility: Please see the detailed review above.
summary_of_the_review: Overall, the proposed approach is simple and hinges on access to word-lists for desired control attributes. Although, the approach seems effective, comparison against closely related, methodologically similar work is missing andI have some additional concerns around the empirical comparison.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes Gamma Sampling, a technique that allows more controllable decoding for specific attributes by modifying the output distribution of language models to up/down sample attribute-related words.

Taking inspiration from Gamma Correction, a processing technique used in image processing to increase/decrease luminosity perception by applying a power-law transformation to luminance values, the authors propose applying a similar power-law transformation to the **sum** of probabilities associated of tokens associated with an attribute, and then correcting the transformed individual probabilities value to preserve the ratios prior to transformation and summing to one. The authors also propose an extension to controlling multiple attributes.

To evaluate the efficacy of Gamma Sampling, the authors compare Gamma Sampling with other controllable techniques/models proposed in the literature, such as fine-tuning on attribute related data, CTRL and PPLM. The authors evaluate controlling for (1) sentence length and (2) topic and sentiment. The authors evaluate using automatic metrics, such as likelihood-based fluency and diversity metrics, and using a self-proposed human evaluation protocol (T4MT).
strength_and_weaknesses: In terms of strengths:
- The proposed approach is very simple and intuitive: it requires no modification or additional training on the model, and only a slight modification to the decoding/sampling algorithm
- It also is very flexible: it only requires defining a list of words associated with an attributes, and this word-list can be defined with whichever method the user prefers (for example, using a BERT-based embeddings or pre-built word-list from the internet)
However, despite the simplicity and intuitive nature of the paper, I have doubts about the evaluation methodology in this paper:
- For fluency evaluation, the authors use the same base models to measure perplexity (GPT-2 Small). However, I think this metric isn’t very meaningful given how small this model is, and be biased to approach based on the same underlying back-bone model. Using a larger model as a perplexity model would be better in this situation
- The authors propose a new criterion for human evaluation, T4MT. However, very little analysis of this metric is done, so it´s hard to understand if it’s a reliable metric. This is especially worrisome given that most methods fall in each other's uncertainty bounds according to Fig (3)
clarity,_quality,_novelty_and_reproducibility: - How were the uncertainty bounds computed for the T4MT? Is this the standard-deviation across annotators ratings? I couldn´t seem to find a description of this in the paper
- I feel like a figure as large to show how gamma correction in images works is unecessary to include, since it’s somewhat disconnected from what this work is doing.
- In Equation 2. Using superscripts for $A$ is confusing since it makes it look like the probabilities are getting exponentiated by A
- Some typos:
    - Section 3.1 : “Despite the subject is clear in the original image” →  “Despite the subject being clear in the original image”
    - Section 4.3 : “it generates texts with very lower diversity” →  “it generates texts with much lower diversity”
summary_of_the_review: This paper proposes a novel controllable decoding algorithm for language models based up/down-sampling word probabilities associated with an attribute of interest. While the method seems intuitive and the preliminary experiments are promising, the evaluation is problematic, evaluating the proposed method with a single small model and with unsubstantiated conclusions extracted from human evaluation.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper proposes gamma sampling, an inference-time sampling method that enables controllable text generation towards one or more user-defined, desired attribute(s) using a pre-trained language model without further fine-tuning it. Specifically, the proposed method relies in 1) a vocabulary that defines an attributes (such as a collection of words related to positive sentiments) and 2) a step-wise re-weighting scheme which increases or decreases the words (tokens) associated with the desired attribute(s).

The authors conduct a series of experiments on six controllable generation tasks and demonstrate 1) the efficiency (because it does not require fine-tuning and does not rely on a classifier to guide the generation), and 2) the effectiveness (via perplexity, similarity, and human evaluation etc.) of the proposed approach.
strength_and_weaknesses: ### strengths
- Overall, I see this work as a clever way to perform step-wise controllable generation (step-wise in the sense that the control is enforced at each token generation step during inference). 
- The method is simple yet intuitive and effective, and well presented with nice explanations of the motivation and intuition. 
- The evaluation results clearly demonstrate the improvement in efficiency over gradient-based control methods (e.g., PPLM) and in effectiveness over fine-tuned methods with magnitudes more parameters (e.g., CTRL, 1.6b params) than the one used in the authors' work (GPT-2 small, 0.1b params).
- The inclusion of human evaluation results is a great addition to substantiate the empirical evidence. 

### weaknesses
- One thing I'm debating with myself is the novelty of this paper. If we view Eq. (2) more generally, it has the form $P_{\rm out} = P_{\rm in}^\omega$ where $\omega$ is the weighting function. It seems to me that this is a general formulation of many recent step-wise controllable generation methods, including Dexperts and GeDi. Both Dexperts and GeDi rely on discriminators to guide generation. To improve computation efficiency, it is natural to discard discriminators entirely and also avoid fine-tuning. One of the only ways to achieve controllable generation without discriminators and fine-tuning is 1) define attributes with keywords, and 2) impose the control at every generation step. Both these two ideas are not new; 1) appears at least in PPLM (in topic control where each topic is defined with a list of keywords) and 2) is the approach taken by at least Dexperts and GeDi. Both 1) and 2) are used in earlier methods such as [1] (see for example Section 2.2 therein). Therefore, on one end, I think the proposed method does not add much new ideas to the existing line of work in (step-wise) controllable generation. On the other hand, even with the general form above, the design of the weighting function $\omega$ still requires careful design, which is the focus of the present paper. The proposed form, e.g., applying a tangent function on the exp-sum of attribute-related words (tokens), is an interesting way I haven't seen before. 
- The other concern I have is on the experiments. The authors demonstrate improved performance over PPLM and CTRL, in particular on efficiency, which is kind of expected. However, I am also curious about the comparison of the proposed approach with other **inference-time controllable generation methods**, such as Dexperts and GeDi. The proposed method would outperform these two methods on efficiency for sure (albeit less significantly so as compared to PPLM and CTRL), but I'm interested in the trade-off between efficiency and performance - it would make a very interesting case if the proposed approach achieves similar performance in terms of the evaluation metrics that the authors have already considered with improved efficiency compared to other inference-time controllable generation methods. Such comparison is currently missing from the experimental results. 

[1] https://aclanthology.org/P17-4008.pdf
clarity,_quality,_novelty_and_reproducibility: The paper is well presented. 
summary_of_the_review: In summary, I think this paper has potential to make an interesting contribution to controllable text generation but have some concerns on novelty and evaluations. I am very open to increasing my score.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper proposes Gamma Sampling (GS), a guided decoding method for fine-grained controllable text generation. The fundamental intuition is that for each attribute we want to control, there will be a set of words that are related to this attribute. If we can identify such a word list, we can then manipulate their probabilities during decoding to control their appearance. Since it focuses on improving the decoding stage, the method is training-free, thus is significantly more efficient than tuning-based methods. Due to GS's lightweight setting and flexibility, it can be extended to support combinations of multiple controllable attributes. Over six simple controllable text generation tasks, GS outperforms all baselines. 
strength_and_weaknesses: Strength:

1. The paper is well-motivated, clearly presented and easy to follow.
2. The proposed method is effective and efficient. It achieves more than 100x speedup compared to the popular baseline PPLM

Weakness:

Except for those pointed out in the Limitation section, I have the following concerns for GS:
1. GS can not be used for tasks that can not be easily verbalized. For instance, we want to control the structure, style, or part-of-speech info of the text. It would be difficult to use some keywords to describe sentence structures. 
2. Some important baselines are missing. The evaluation only compares GS to conditional/fine-tuned LM methods but does not compare to other guided decoding methods. Such as those surveyed in the related works. If those methods are somehow not directly comparable, we shall at least consider a naive baseline: given an attribute-related word list, we increase/decrease those words' probabilities by x percent. These comparisons are important in demonstrating the effectiveness of GS against other guided decoding methods.  
clarity,_quality,_novelty_and_reproducibility: The paper is clearly presented and easy to follow.

I can not justify this paper's novelty as I am not familiar with guided-decoding methods (closely related to this work.)

Overall, I think this paper needs revision before it can be published. 
summary_of_the_review: This paper proposes a highly-efficient controllable text generation method. The author demonstrated good performance of the proposed method over six common tasks. However, there are flaws in the evaluation as well as limitations in terms of the method itself. So I think this paper can not be published in its current version.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The paper proposed Gamma Sampling (GS), a sampling method that enables controllable text generation for specific attributes defined by word lists. The key idea of this approach is to modify the logits/probability of the attribute-specific words by a temperature-like hyperparameter (called Gamma) and rescale the softmax distribution at each step. The main advantage of the GS is that it doesn’t require any condition-specific training data. The authors conduct experiments on six controllable generation tasks, and GS appears to be effective and efficient.

The authors revised the paper to address some of the original concerns of the reviewers, including small GPT2 model size, lack of comparisons against inference-time controllable text generation baselines, and lack of details on the human evaluation. This eased some of the reviewers’ concerns, and the reviewers increased some of their scores accordingly.

Ultimately the paper is still below borderline, as the reviewers still have the following concerns:

1. Impact:  The applicability of this work is relatively limited compared to prior controllable text generation methods, as GS is governed by word lists that need to be independently provided. So, GS cannot be used for tasks or goals that cannot easily be verbalized.
2. Novelty: The authors clarified the difference between GS and some of the related work mentioned in the reviews, but the reviewers still felt the level of novelty is relatively limited. The approach is to just heuristically adjust the probabilities of salient words/tokens, in a step-wise manner, an idea that has appeared in a number of prior work in some forms (details in the reviews). 
3. Evaluation: While GS seems effective, the evaluation mostly compares GS to conditional/fine-tuned LM methods and comparisons against closely related and methodologically similar work are missing. Three reviewers had various comments that relate to this inadequacy of the evaluation, but the revised paper only added one model in response to these concerns (K2T, which wasn’t the one suggested by the reviewers). Given point (2) above, I think the paper should be more focused on evaluating the specific differences with similar approaches in prior work. If these approaches are not directly comparable in the evaluation setting of the paper, the authors could at least provide some relevant baselines (e.g., the naïve GS-related baseline suggested by Reviewer aPW9) or ablations of GS. In its current form, the paper evaluates GS mostly against methods that are quite different (and often not strictly comparable due to differences in model sizes) and the overall lack of analyzes makes it difficult to appreciate why GS works and what affects its effectiveness (e.g., one reviewer suggested evaluating the impact of number of control elements).

I recommend rejection in light of these concerns.
justification_for_why_not_higher_score: The authors present one way to do constrained generation (that is quite similar to prior work) and then evaluates their approach against completely different methods (omitting evaluation against similar work).
Decision: Reject