Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper presents a decoding algorithm for controlled text generation. The problem is formulated as finding the most likely output $y$ under given constraints. The authors adopt the method of Lagrange multipliers, and introduce dampening in gradient descent to reduce oscillations. To deal with discrete $y$, the authors choose to optimize over probability simplex $\tilde y$ using exponentiated gradient descent. The authors also report that converting $\tilde y$ to a one-hot vector via a straight through estimator helps to converge faster.

Experiments include increasing formality in style transfer and machine translation. Compared with greedy decoding/beam search, the proposed method with formality and semantic similarity constraints achieves better content preservation/BLEU score and better transfer accuracy. It also compares favorably to the recent FUDGE method for controlled generation.
main_review: Constrained text generation is an important problem that has received increasing attention. This paper proposes a multi-objective optimization approach for constrained decoding, and motivates the choice of each technique used.

Is the formality classifier used in optimization also used for evaluation? If so, as you set the probability threshold to 0.5, if the constrained optimization was successful, shouldn't you achieve formality 100%? Why in Table 2 it's only 30%? Or the optimization failed and ended up with Lagrangian multipliers of extreme values?

In the method section, the authors discussed the shortcomings of using linear combinations of multiple objectives and advocated their Lagrangian approach. Nevertheless, I still would like to see how linear combinations perform in the experiments. Also, if you apply your continuous optimization in the unconstrained case (so maximizing likelihood is the only objective), how would it compare to beam search? I know it's slower than beam search, but will its BLEU be better or worse? I think this result will help to better understand the comparison between continuous optimization and combinatorial search.

Typo: line 127, "$(\mu_j-g_j(x,y))$" should be $(\xi_j-g_j(x,y))$
limitations_and_societal_impact: Limitations and societal impact are discussed in Sec 4.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper proposes a flexible controllable decoding algorithm called MUCOCO, which incorporates multiple control attributes as differentiable constraints to the optimization. The authors relax the discrete optimization problem to a continuous one, and use continuous optimization techniques such as Lagrangian multipliers and gradient-descent based techniques to generate texts towards the desired attribute. Experiments on three conditional text generation tasks including text style transfer, machine translation and paraphrasing show the superior performance of the proposed model.
main_review: Strengths:
1) The formulation of constrained decoding is intuitive and interesting. It provides a new perspective to solve the text generation problem under multiple constraints.
2) Experiments on three different text generation tasks show the generalization ability of the proposed method.

Weaknesses:
1) My main concern is about the focus of this paper. I understand that the formulation itself is interesting. But the following techniques such as continuous relaxation and gradient-based optimization which are the main content of the method are all common techniques in the machine learning community, which don’t provide new insights. So, I feel that the focus should be addressed clearly.
2) The authors choose FUDGE as the main baseline, but this model can only support a single constraint. The authors should consider the baselines which can handle text generation with multiple control attributes such as CoCon [1].
3) In my view, one of the major problems in text generation with multiple control attributes is the compatibility of multiple constraints. I wonder how the proposed model can explicitly deal with this problem.

[1] COCON: A SELF-SUPERVISED APPROACH FOR CONTROLLED TEXT GENERATION. ICLR 2021.
limitations_and_societal_impact: The authors adequately address the limitations and potential negative societal impact of their work.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: This work tries to solve a general type of text generation tasks, generating text with constrains. The author proposed a a new algorithm MUCOCO to formulate the decoding process as a constrained optimization problem. The author then use continuous relaxation, Lagrangian multipliers and exponential gradient descend techniques to solve the problem. The author validated the algorithm with style transfer and style-controlled machine translation.
main_review: This paper formulates the constrained text generation problem into a discrete constrained optimization task. This idea is quite novel and interesting. The proposed setup of Lagrangian multiplier, straight through and exponential gradient descent are natural tools to use for this setup.  

I do not get the reason why the author emphasis the proposed framework is for pretrained model. From the method itself, I do not see a limitation on this. 

As the proposed algorithm is to improve over auto-regressive model which is hard to follow the constrains. Then the other type of framework,  non-autoregressive text generation should be interesting to be considered in this work, at least as a baseline for MT task.

I do not see details in inference stage, will the proposed algorithm effect decoding stage?

The method's effectiveness seems strongly relay on the auxiliary optimization objective f and g. This may leads to a limitation to the proposed method, how to define a differentiable objective can be a hard task. Also in the experiments, seems the function threshold needs carefully selection, which may make the proposed algorithm practically hard to use. An empirical study on these should be shown in the paper.
limitations_and_societal_impact: No
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: In this paper, the authors propose a controllable decoding mechanism for language generation models (e.g., GPT2) using constrained optimization. The proposed method explicitly formulate the sentence decoding as a constraint optimization problem where the free parameters are the distribution over the vocabulary at each step and two types of constraints are defined one over the generate text itself (f(Y)) and one over the similarity between the input and the outputs (g(x,y)). Please refer to Eq. 1 where the problem is clearly formalized. Since directly solving the problem in Eq 1. is not computationally feasible the author reformulates the problem using the Lagrangian formulation for container optimization problems (Eq. 3). However, this min-max formulation introduces several problems (e.g., the tradeoff between different objectives, hard to find opt for non-convex functions). To cope with these issues, the authors proposed a gradient descent/ascent solution (Section 2.1), a soft relaxation for each token using the probability simplex (exponential gradient descent + normalization) (Section 2.2), and an annealing threshold (Section 2.3). 

The authors benchmarked the proposed decoding schema in both style transfer and style-controlled machine translation. In the style transfer, they follow the methodology in Krishna et al. 2020 and they study two contains such as formality and the semantic similarity between the input and the output. Similarly in the style-controlled machine translation, they study both the formality style and the cross-lingual similarity between input-output. In both settings, the author compared the proposed method with other decoding techniques (greedy and beam-search) and other controllable language generation techniques (e.g., FUDGE) using both automatic and human evaluation.  Finally, the authors show some preliminary results in controlling multiple attributes (Age, style pos/neg), and discuss the speed and memory requirements of the model. 
main_review: ***Originality***

To the best of my knowledge, the proposed method is novel in controllable language generation. Moreover, constrained optimization is very interesting in this setting and this is very different from existing approaches. However, I would suggest including some relevant work as [1,2]

***Quality***

The paper is technically sound, but I would have expected more experiments and comparisons with existing work. Indeed, this method is general and it could be used with any discriminator (e.g., positive/negative, topics, toxicity etc.). Several existing works (e.g. GEDI, PPLM etc.) run these controllable language generation experiments. Adding these experiments would have made the paper much stronger and more general. The results on formalization are not very exciting, although the result is extremely good. Moreover, I am a bit suspicious when checking Appendix Table 5,  where the authors show the multiple constraints example, in that example, I would be curious to see what happen when rewriting a review with an original score of 2 to a review with a score of 5. Is this possible? 

Moreover, it is also not clear the human evaluation for the style-controlled machine translation (Line 295). Why not running the same human eval as in the style transfer experiments? and the results in percentage are really confusing. Could you please clarify the human evaluation. 

***Clarity***
The paper is extremely well written,  the notation is clear and easy to follow also with a minimal background in optimization. 


***Significance***:
Yes, this proposed method is extremely important for any general NLG model. In general, enforcing constraints in text generation is extremely hard and solving this problem could be crucial for safely deploying large language models. 


[1] NEUROLOGIC DECODING: (Un)supervised Neural Text Generation with Predicate Logic Constraints (Lu et. al. 2020) 
[2] Gradient-guided Unsupervised Lexically Constrained Text Generation (Sha et.al. 2020)

limitations_and_societal_impact: The author clearly states the limitation, i.e., a very slow inference time, and possible workaround in future work. Moreover, they also address possible negative societal impacts in the ethical considerations section.
review_title: null

Meta-review: This paper proposes MUCOCO, a constrained text generation method. The constrained text generation is formulated as generating text with multiple controlling attributes. MUCOCO turns a discrete constrained decoding into differentiable optimization using techniques of  continuous relaxation, Lagrangian multipliers and exponential gradient descend. Experiments on three conditional text generation tasks including text style transfer, machine translation and paraphrasing show the superior performance of the proposed MUCOCO.

The author may adjust the main focus a bit accordingly. Additional discussion and/or comparison with other highly related methods (methods that can also handle constrained sentence generation with multiple constraints) could be added as pointed out by reviewers. The authors may explain the inference clearly. Additional discussion on limitations of MUCOCO could be added (e.g. conditions for auxiliary objectives and compatibility of multiple constraints) . 
Decision: Accept (Poster)