Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: **What is the task?**
A general, flexible and efficient framework to control auto-regressive generation models with NeurAlly-Decomposed Oracle (NADO)


**What has been done before?**
PPLM , GeDI and FUDGE also aim to guide the base model with an auxiliary 42 model. However, they either shift the base model distribution in a post-hoc manner without theoretical guarantee, or/and require external labeled data to train the auxiliary model. Instead, we derive a closed-form solution for the optimal way to incorporate the oracle, without requiring external labeled data or token-level guidance.

**What are the main contributions of the paper?**

* Given a pre-trained base language model and a sequence-level boolean oracle function, author((s) propose to decompose the oracle function (indicating whether an attribute is satisfied) into token-level guidance to steer the base model in text generation.

* They present the closed-form optimal solution to incorporate the token-level guidance into the base model for controllable generation.

* They provide a theoretical analysis of how the approximation quality of NADO affects the controllable generation results.


**What are the main results?**
Experiments conducted on two applications: (1) text generation with lexical constraints and (2) machine translation with formality control demonstrate that our framework efficiently guides the base model towards the given oracle while maintaining high generation quality.


strengths_and_weaknesses: Strengths

* Generally, the post-processing methods are considered expensive in inference and low quality in generated texts. However, proposed framework, as a kind of post-processing method, can achieve high generation quality demonstrated in experiments and is efficient in the inference time.

* Since NADO is trained on the data sampled from the base models, it aligns better with the base model and thus can achieve better control.

* The token-level guidance is approximated by a neural model trained with examples sampled from the base model, demanding no additional auxiliary labeled data.

questions: Typo : Line 41 aims→ aim
limitations: There is no limitations section i the paper.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This work proposes controllable autoregressive generation by decomposition the control signal into token level constraints. The method is formulated as an optimization problem based on posterior regularization and approximated by a neural network. Experiments on lexically controlled generation and machine translation with formality demonstrates the effectiveness of the method.
strengths_and_weaknesses: Strengths 

- This is indeed a novel method for controlling autoregressive generation. The method is different than the existing paradigms like lexically controlled beam search or modified sampling probability. The authors also provide a certain level of theoretical guarantee.
- This method is demonstrated effective in the experiments, especially the machine translation with formality control.

Weakness 

- There are important missing details, specifically:
    - How many samples are need for the approximation (Section 3.3)? Intuitively, as the approximation is performed over the full autoregressive decoding space, one would expect the number of sample be large, thus the computation complexity may also be large.
    - How good and diverse should the base model be? If the base model is not good enough for generation, then the controlled q will consequently not be good. If the base model is not diverse enough, then one may require more sampling to hit the condition (note that controlling the temperature cannot alleviate the mode collapse problem, if it is the problem of the base model)
    - How does the performance related to model scale? Can one expect a larger model to be easier to control (possibly because of better language modeling) or harder to control (possibly because of harder optimization)?
    - What if base distribution is far from constraints (but may sill be good per se) such that the sampled sentences cannot hit the constraints? This could happen in settings where the constraint distribution is far from the model distribution.
    - What happens if the sampled sequence partially meet the constraint? How would the method behave and can it still learn from partial satisfaction?
- My other concern is that the experiments not grounded to linguistic explanations. Specifically:
    - How does the generated sequences look like? I would encourage the authors to include examples about the intermediate sample of q and show how q start from not following the constraints and gradually becomes more following.
    - How does the model differ from other model in terms of generated sequence?
- My final concern is that there is no human evaluation. Generally, classifier-based evaluation (Table 2) may be fooled by certain spurious correlations thus not strongly reliable. Can the authors include human evaluation, or at least put generated examples for the reviewers to get an impression?

I will be happy to increase my score accordingly if the above concerns can be properly addressed.
questions: See above comments. 
limitations: How the method relate to or can help improper language generation? It would be nice if the authors add certain discussions about how improper sentences can be prevented with this method.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper proposes to decompose sequence-level attributes into token-level guidance for controllable text generation, where the token-level guidance is approximate by a neural model trained with examples sampled from the base model. Both theoretical analysis and experimental results demonstrate the effectiveness of the proposed method on different tasks.
strengths_and_weaknesses: Pros:
- This paper is well-written and easy to read.
- The proposed method is novel and well supported by theoretical analysis
- Experiments are exhaustive. The authors conduct experiments on both constrained text generation and machine translation, and show the performance improvement on various datasets.

Cons:
- The proposed NADO model introduces some additional hyperparameters that need to be tuned. It’s better if the authors can provide a deep analysis on the effect/sensitiveness of hyperparameters. 
questions: Refer to comments above.
limitations: Yes. I do not see any limitations.
review_title: null

Meta-review: metareview: All three reviewers sided to accept the paper. The method of the paper is formulated as an optimization problem based on posterior regularization, and as such is quite different from existing paradigms in controllable NLG (e.g., lexically constrained beam search or modified probability sampling). The work's theoretical basis also offers a nice contrast with established methods in this area, as the existing methods are often applied in post-hoc manners and without theoretic guarantees. The only significant downside of this paper is that its evaluation is not very standard and lacks human evaluation, and its model-based automated evaluation of attributes such as formality could have been affected by spurious correlations (note: the latter concern affects only one of two tasks of the paper). As the paper achieves some substantial gains on two very different tasks, the reviewers generally considered the method of the paper to be quite effective.
confidence: Certain
Decision: Accept