BOLT: Fast Energy-based Controlled Text Generation with Tunable Biases
Year: 2023

Abstract:
Energy-based models (EBMs) have gained popularity for controlled text generation due to their high applicability to a wide range of constraints. However, sampling from EBMs is non-trivial, as it often requires a large number of iterations to converge to plausible text, which slows down the decoding process and makes it less practical for real-world applications. In this work, we propose BOLT, which relies on tunable biases to directly adjust the language model’s output logits. Unlike prior work, BOLT maintains the generator’s autoregressive nature to assert a strong control on token-wise conditional dependencies and overall fluency, and thus converges faster. When compared with state-of-the-arts on controlled generation tasks using both soft constraints (e.g., sentiment control) and hard constraints (e.g., keyword-guided topic control), BOLT demonstrates significantly improved efficiency and fluency. On sentiment control, BOLT is 7x faster than competitive baselines, and more fluent in 74.4% of the evaluation samples according to human judges.

heading: 1 Introduction
text: Generating text using pre-trained language models (PLMs) to satisfy user-specified constraints is an important task to allow practical usage of PLMs. Common controlled text generation methods include training conditional language models (Keskar et al., 2019; Zhang et al., 2020) or attribute-based fine-tuning of PLMs (Liu et al., 2020; Zhang and Song, 2022). Yet, these methods are often resource-intensive and infeasible for large models like GPT-3 (Brown et al., 2020). Furthermore, these methods assume access to large amounts of attribute-specific data and are inflexible for new constraints. On the contrary, inference-time methods (Qin et al., 2022; Kumar et al., 2022; Mireshghallah et al., 2022) directly steer the generations without model re-training or
fine-tuning. In particular, energy-based models (EBMs) (LeCun et al., 2006) have demonstrated greater flexibility, since they can accommodate arbitrary energy functions (Khalifa et al., 2021; Qin et al., 2022; Kumar et al., 2022).
Despite their benefits, sampling from EBMs presents profound challenges. Notably, the sampling process, which is often done through Langevin Dynamics (Welling and Teh, 2011) or Gibbs Sampling (Goyal et al., 2022), requires a substantial number of iterations to converge to readable sequences of text. This can significantly slow down the decoding process, rendering the methods unusable in real-world applications.
In this paper, we propose BOLT1, that uses a sequence of tunable Biases Over LogiTs of the PLM’s output layer, to steer the generation towards specified constraints. The biases are tuned through a gradient-based process, with the goal of minimizing the energy of the generated sequences. In contrast to prior research which mainly investigates non-autoregressive decoders, BOLT maintains the autoregressive generation
1Our code is available at https://github.com/ launchnlp/BOLT.
186
process, thus resulting in both fast convergence with fewer iterations, since conditional dependencies between tokens are exploited, and improved fluency. Fig. 1 demonstrates that the sampling process of recent EBM-based methods—MuCola (Kumar et al., 2022), Mix&Match (Mireshghallah et al., 2022), and COLD (Qin et al., 2022)—is slower on a sentiment control task, e.g., generating 20 tokens using 10 seconds on average, while BOLT only takes 1.4 seconds.
We conduct controlled generation experiments over three tasks: sentiment control, toxicity avoidance, and keyword-guided topic control, encompassing both soft and hard constraint-based generation problems. BOLT’s outputs achieve the lowest perplexity across all tasks, while being 7x and 17x faster than COLD and MuCola, respectively, on sentiment control. Additionally, BOLT shows superior controllability in toxicity avoidance while obtaining comparable controllability on the other two tasks. Lastly, according to human evaluation, 74.4% and 51.0% of samples produced by BOLT in sentiment control and toxicity avoidance are rated as more fluent than those by multiple comparison methods.

heading: 2 Related Work
text: Popular methods for controlled generation often rely on attribute-conditioned language modeling (Krause et al., 2021), model fine-tuning (Khalifa et al., 2021), or prompt tuning (Yang et al., 2022), all requiring intensive model training and attribute-specific data. This paper instead focuses on inference-time methods that require no model training. Prior work under this paradigm mainly adjusts the output token probabilities toward constraint-satisfying sequences (Dathathri et al., 2020; Yang and Klein, 2021). For instance, Dathathri et al. (2020) leverage gradients from an attribute classifier to update the LM hidden state to guide the generation. However, one notable drawback of such techniques is the requirement of learning specialized models such as attribute classifiers (Dathathri et al., 2020) and future-aware classifiers (Yang and Klein, 2021). Another family of methods searches for optimal sequences through optimization in the continuous space. For instance, MuCoCo (Kumar et al., 2021) uses constrained continuous optimization, solved by Lagrangian multipliers and gradient descent. Qin et al. (2022) further enhance the gradient-based

heading: Energy Functions
text: +
STE
+ +
STE STE
+ +
STE STE
Step 1 Step 𝒕-1 Step 𝒕 Step 𝒕+1 Step 𝑳
… …
optimization method by using Langevin Dynamics. Their main issue is that they require numerous sampling iterations to converge since raw logits or embeddings are optimized without considering conditional dependencies among tokens. BOLT, on the contrary, maintains the token dependencies through autoregressive decoding while optimizing for the constraints through the added biases.

heading: 3 The BOLT Model
text: Energy-based controlled generation aims to produce a sequence of tokens that minimize an energy function, with lower energy indicating more constraints being satisfied (Qin et al., 2022; Kumar et al., 2022). While sampling techniques such as rejection sampling can be used to sample lowenergy sequences (Mireshghallah et al., 2022), such sampling requires the usage of an appropriate proposal distribution and is typically slow in practice. Instead, we propose to tune a set of biases at inference time with the goal of steering the decoding process towards generating low-energy sequences.
The overview of our framework is displayed in Fig. 2. At each decoding step t, we add the tunable bias ybt ∈ RV to the PLM predicted logits yLMt ∈ RV as follows:
yt = y LM t + wt · ybt , (1)
where wt controls the contribution of the bias. As a result of the autoregressive decoding, the control effect at later time steps is compounded from previous steps. One way to mitigate that is to have smaller weights for biases at later time steps.
Therefore, we model the weights using a decreasing linear function of t, i.e., wt = 1− tL , which is found to work best in practice.2
Typically, we sample a discrete token yt from the word distribution softmax(yt), and then feed it back to the PLM for further decoding. However, this would require backpropagation through the sampling process to optimize the biases. As a workaround, we use the straightthrough gradient estimator (STE) (Bengio et al., 2013), which converts yt to a one-hot vector ȳt in the forward pass and bypasses ȳt in the backward pass to allow gradients to be applied to yt.3 ȳt designates the argmax token, i.e., the position with the highest logit value in yt is set as 1, and 0 for the rest. The one-hot vector ȳt is fed to the PLM for next-step decoding.
After decoding for L steps, we obtain a sequence of one-hot vectors ȳ[1:L] =[ȳ1, ȳ2, ..., ȳL−1, ȳL]. Then, we update ybt with gradient descent to minimize the energy function E(ȳ[1:L]).4 Thus, BOLT tunes the biases with the goal of steering the PLM to generate sequences with low energies. Finally, the output sentence [y1, y2, ..., yL−1, yL] can be derived from ȳ[1:L] through multiple iterations of gradient descent until the constraints are satisfied (e.g., the toxicity probability of generated sequence is lower than a threshold) or a predefined maximum iteration number is reached.
Energy Functions. Following previous work, we experiment with both soft constraints, applied on sentiments and non-toxicity, and hard constraint, for requiring the existence of certain keywords in the generations. We describe the corresponding energy functions below. Additionally, we use a fluency-encouraging component to maintain the coherence of the generated text.
Soft Constraints. We use attribute classifiers as discriminators for soft constraints. The energy output by the discriminator is defined as Esoft = −pdis(c|ȳ[1:L]), c ∈ C. Here pdis(c|ȳ[1:L]) is the probability of the sequence ȳ[1:L] with the attribute c by the attribute classifier, and C is the set of attributes, e.g., positive and negative.
Hard Constraints. We follow Qin et al. (2022)
2Multiple options for w are explored, with more details given in Appendix A.
3We describe the implementation of STE in Appendix B. 4In practice, we apply reparameterization to the biases to
reduce memory usage. Details are given in Appendix C.1.
and Kumar et al. (2022) and use the differentiable BLEU (Liu et al., 2022), which measures unigram similarity of the generated sentence and target keywords. This energy can be represented as Ehard = −diff-BLEU(ȳ[1:L], [w1, ..., wK ]), where wk is a keyword expected to appear in the generation.
Fluency Constraints. We define a fluencyencouraging energy function corresponding to the negative probability of the generated sequence according to an external PLM, specifically GPT2large, given by Efluent=− ∑L t=1 p(yt|ȳ<t), where yt is the t-th token and ȳ<t is the sequence generated until step t.
In order to ensure the fluency of samples, we incorporate the fluency energy function with both soft and hard constraints, where the total energy function Esoft + λ1Efluent is used for soft constraints, and Ehard + λ2Efluent for hard constraints, where λ1 and λ2 are hyperparameters.5

heading: 4 Experiments and Results

heading: 4.1 Constraints and Energy Functions
text: Following Kumar et al. (2022), we conduct experiments on two soft constraint tasks: 1) sentiment control and 2) toxicity avoidance. For sentiment control, we collect 15 prompts from Dathathri et al. (2020). For each prompt, every model generates 20 sentences of 3 different lengths (12, 20, and 50 tokens) per sentiment (positive and negative). This results in a total of 1800 generations. Moreover, we extract 1,000 prompts from RealToxicityPrompts (Gehman et al., 2020) to assess toxicity avoidance, with each model generating 25 sentences per prompt.
For hard constraint task, we use keywordguided topic control as done by Dathathri et al. (2020). We use the same set of 15 prompts, with each model generating sentences of 20 tokens, for 7 topics. For each combination of topic and prompt, 20 sentences are generated. We extract 4 keywords as constraints per topic. Full lists of keywords and prompts are in Appendix D. In addition, we perform experiments on CommonGen test set (Lin et al., 2020), which comprises 1,498 sets of keywords. For each set of keywords, each model aims to generate a single sentence that incorporates all of the given keywords.
For formulating the energy functions, we construct the discriminators in Esoft for sentiment
5Appendix C.2 describes how to search λ1 and λ2.
control and toxicity avoidance by training 1) a sentiment classifier on Yelp polarity corpus (Zhang et al., 2015), and 2) a toxicity detection classifier on Jigsaws (Jain et al., 2022), following the settings in Mireshghallah et al. (2022). During generation, the desired attribute c is set as either positive or negative in sentiment control, and as non-toxic in toxicity avoidance. For keyword-guided topic control, we use the set of 4 extracted keywords from each topic to compute Ehard. More details of discriminator training are given in Appendix C.3.

heading: 4.2 Baselines
text: We compare with three energy-based methods: 1) COLD (Qin et al., 2022), which performs sampling by iteratively updating a sequence of tokenlevel logits using Langevin dynamics; 2) MuCola (Kumar et al., 2022) is similar to COLD, but samples the sequence of token embeddings instead of logits; 3) Mix&Match (Mireshghallah et al., 2022) uses Gibbs sampling to draw a batch of sentences and determine their acceptance or rejection using the energy function, repeated until convergence.6 Implementation details of baselines can be found in Appendix C.4.

heading: 4.3 Results and Analysis
text: As shown in Table 1, on sentiment control, we observe that BOLT is 7x faster than comparisons while achieving comparable controllability. Though MuCola has the best control, as measured by the external classifier and human judgment, it generates repetitive trigrams more frequently. Moreover, as rated by human judges, 74.4% of the BOLT generations are preferred over other mod-
6Mix&Match’s code only supports sentiment control. Therefore, we only compare with their results on the sentiment control task.

heading: Model Succ.(%)↑ PPL↓ REP-3gram↓
text: els’ outputs, in terms of fluency. This is further demonstrated by the sample outputs in Tables 4 and 7, where BOLT’s generated text is more coherent and sensible.
Similar observations are made on the toxicity avoidance task. As seen in Table 2, BOLT achieves the lowest toxicity levels. More than half of BOLT’s generations are judged to be more fluent than the baselines, which we argue is due to the fact that BOLT maintains the LM’s autoregressive decoding nature.
Results on keyword-guided topic control (Table 3) further echo the advantage of BOLT. While ensuring the inclusion of specified keywords in the generated sentence is relatively easy for all models, BOLT stands out with its more fluent and natural generations.
Overall, BOLT demonstrates a faster decoding speed and generates text with superior fluency, while maintaining comparable or better controllability than the baselines. This makes BOLT particularly suitable for practical use cases. In future work, we plan to apply BOLT to other controlled generation tasks and explore its potential usage for data augmentation (Malandrakis et al., 2019; Kumar et al., 2020).
We further evaluate BOLT on another hard constrain control task based on the CommonGen dataset. This task is more challenging, since it requires the generation to include an average of 4.5 provided keywords. We compare the performance of BOLT with that of COLD and MuCola. Based on the results presented in Table 5, BOLT achieves comparable coverage and generates fewer repetitions, with an increased perplexity. The worse fluency can be attributed to the tradeoff made by BOLT between controllability

heading: Model Coverage(%)↑ PPL↓ REP-3gram↓
text: and fluency. Our experiments show that ensuring the inclusion of all specified keywords often requires a larger number of iterations for BOLT to converge, compared to other tasks discussed earlier in the paper. Unfortunately, this increased optimization process causes disruption of the original autoregressive decoding outputs, resulting in less fluent generations. This suggests future research directions that explore different types of hard constraint energy functions (Zhukov and Kretov, 2017; Casas et al., 2018) and optimization methods (Rennie et al., 2017; Liu et al., 2017) to handle hard constraints with multiple keywords, aiming for faster convergence and higher-quality sentence generation.

heading: 5 Conclusion
text: We introduce BOLT, an energy-based model for controlled text generation. It uses a sequence of tunable biases applied to the logits of the PLM’s output layer to guide the generation towards specified constraints or attributes. Through experimental evaluations on controlled text generation tasks involving both soft and hard constraints, we demonstrate the effectiveness of BOLT in terms of both speed and fluency.

heading: Limitations
text: While BOLT shows an impressive performance in imposing soft constraints and some hard constraints, it still lacks when it comes to imposing harder constraints, for e.g., keyword control with more than three keywords. BOLT also requires careful tuning of different hyperparameters that make up the energy function — an issue that is prevalent among energy-based controlled generation methods.

heading: Ethical Statements
text: It should be noted that certain model generations, as listed in Table 4 and Table 7, may contain elements of toxicity and offensiveness. Besides, despite BOLT’s ability to mitigate the risk of generating toxic content through toxicity avoidance
techniques, it remains possible for it to produce biased, offensive, and fake information that could potentially cause harm to the general public.
An additional ethical concern is the possibility of malicious use of the controlled generation models to generate harmful content. Our experiments reveal that this could be accomplished by deliberately optimizing the tunable biases such that, for e.g., the energy function corresponding to the toxicity level is maximized.

heading: Acknowledgements
text: This work is supported in part by National Science Foundation through grant IIS-2046016 and LG AI Research. Additionally, we would like to thank Kumar for his assistance in reproducing the results of MuCola. We also thank the anonymous reviewers for their valuable suggestions.

heading: A Exploring Different Settings of w
text: We try the following functions to model the weights in Eq. 1:
• wt = tL
• wt = 1− tL • wt = 1
• wt = w[t]
where w ∈ RL is a tunable vector and will be tuned during optimization. We apply these functions and run BOLT on sentiment control with a L set to 50. According to the results in Tab. 6, the linear function wt = 1− tL that decreases over time was found to achieve an optimal balance between controllability and generation quality. Therefore, it was utilized in all subsequent experiments.
B Implementation of STE
Using PyTorch API, we can easily convert yt to the one-hot vector by running ȳt=torch.nn.functional.one_hot (torch.argmax(yt))+yt -yt.detach().
C Implementation Details

heading: C.1 Reparameterization of the Tunable Biases
text: In our experiments, we apply reparameterization to the tunable biases, representing the offset yb as lm_head(hb), where lm_head(·) is the output layer in the PLM. Tuning hb instead of yb helps to reduce memory usage, as the dimension of hb is significantly smaller than that of yb (1280 vs. 50257). Note that the parameters of lm_head(·) are fixed during turning hb.

heading: C.2 Hyperparameters
text: In order to search for the optimal values of λ1 and λ2 in soft and hard constraint tasks, we employ a grid search strategy with an interval of 0.1, varying λ1 and λ2 from 0 to 1. Ultimately, we set both λ1 and λ2 to 0.1 for a balance between controllability and fluency. We initialize the hb with a normal distribution N (0, 0.25), which ensures that the biases are initially set to nearly zero in order to avoid making excessive adjustments to the logits of the PLM. We use Adam as the optimizer during tuning the bias, with a learning rate of 0.025. To reduce the amount of repetition, we set a repetition penalty (Keskar et al., 2019) as 1.2 to adjust the PLM predicted logit. We employ the
MaxLengthCriteria in Huggingface to control the length of generated sequences, following previous studies. For sentiment control, we set the maximum number of iterations to 8. Once the maximum iterations number is reached, the sequence with the lowest energy among iterations would be picked as the output. For toxicity control, we also set the maximum number of iterations to 8, and adopt the early stop if the toxicity probability of the generated sequence given by the discriminator is lower than 0.01. During keyword-guided topic control, we early stop the optimization when there is a least one keyword appearing in the generated sequence. In the case of CommonGen, optimization was terminated when all the keywords appear in the generated sentence or the maximum number of iterations 100 is reached, while keeping the remaining hyperparameters unchanged.

heading: C.3 Details of Discriminators Training
text: We follow the same setting in (Kumar et al., 2022) to train the discriminators for soft constraints. Discriminators, i.e., attribute classifiers, for both sentiment control and toxicity avoidance are based on the widely used pretrained model RoBERTa (Liu et al., 2019). Since there is a mismatch of the vocabularies between RoBERTa and GPT2-large, we replace the embedding layer of our RoBERTabased classifier with that of GPT2-large, and apply the GPT2-large tokenizer during training discriminators.

heading: C.4 Details of Baselines
text: • COLD We employed the default hyperparameter settings as provided in the released codes, with a maximum iteration limit of 400 for all tasks. For the keyword-guided topic control, we implemented an early stopping technique, whereby the sampling process is terminated once any of the specified keywords is identified in the generated sequence.
• MuCola We directly run their provided scripts for conducting controlled generation on sentiment control and toxicity avoidance. We also adopt early stopping on keywordguided topic control, similar to COLD.
• Mix&Match We directly execute their offered scripts for sentiment control.

heading: D Prompts and Keywords
text: Our prompts from (Dathathri et al., 2020) are Once upon a time, The book, The chicken, The city, The country, The horse, The lake, The last time, The movie, The painting, The pizza, The potato, The president of the country, The road, The year is 1910. In keyword-guided control, we extracted the following keywords from (Dathathri et al., 2020):
• computer: “router”, “Linux”, “keyboard”, “server”
• legal: “plea”, “subpoena”, “transcript”, “bankrupt”
• military: “torpedo”, “headquarters”, “infantry”, “battlefield”
• politics: “court”, “culture”, “communism”, “capitalism”
• religion: “Bible”, “church”, “priest”, “saint”
• science: “microscope”, “mass”, “mineral”, “scientist”
• space: “meteor”, “planet”, “satellite”, “astronaut”

heading: E Evaluation
text: Automatic Metrics Models are evaluated based on three main criteria.
• Controllability measures the ability of producing sequences that accurately reflect the desired attribute. For sentiment control, we use both an internal classifier (Int. Clsf.), i.e., the same discriminator used for guiding the generation and an external classifier (Ext. Clsf.) forked from Hugging Face7 for a more objective comparison. For toxicity avoidance and following (Mireshghallah et al., 2022; Kumar et al., 2022), we use Perspective API8
to estimate the toxicity in the generated sentences. We use two metrics for toxicity: one uses the average of the maximum toxicity score over 25 samples per prompt (Average
7VictorSanh/roberta-base-finetunedyelp-polarity
8https://perspectiveapi.com/
Max Toxicity), and the other is the probability of generating a toxic sentence (with a toxicity score > 0.5) among the 25 generated sequences (Toxicity Prob.). For keywordguided topic control, we count the success rate, where a successful generation contains at least one specified keyword (Succ.).
• Sentence quality is measured by its fluency, diversity, and word repetition. To measure fluency, we feed the generated sentences to GPT2-XL and report the perplexity (PPL). To measure diversity, we compute the average occurrences of distinct trigrams (dist3) in each set of sentences generated per prompt, normalized by sentence length. In addition, we count the average number of repeated trigrams (REP-3gram) in each sentence.
• Speed. Speed is measured by running decoding with a batch size of 20 on a single Nvidia RTX 8000 GPU card for all models. The number of tokens generated per second by each model is reported.
Human Evaluation A small set of model generations, comprising of 90 sentences for sentiment control and 100 sentences for toxicity avoidance, was subjected to a human evaluation for both fluency and controllability. To evaluate fluency, we ask annotators to select the most fluent sequence from different model generations, and calculate the percentage of times each model’s output was chosen. For controllability, annotators label sentences as “Positive”, “Neutral” or “Negative” for sentiment control and “Toxicity” or “Non-Toxicity” for toxicity avoidance, and we compute the percentage of times each model’s output successfully conveys the expected attributes. Our proposed model (BOLT) is compared to MuCola and Mix&Match models for sentiment control, and to COLD and MuCola for toxicity avoidance. Annotators are presented with triplets of generated sentences from each model based on a common prompt. Two annotations are collected for each sample, and their Cohen’s kappa coefficient for fluency and controllability are 0.67 and 0.85, respectively. Averaged scores are reported. The annotation guidelines can be found in Fig. 3, Fig. 4 and Fig. 5.

heading: F Generation Samples
text: We list more samples in Tab. 7.
ACL 2023 Responsible NLP Checklist

heading: A For every submission:
text: 3 A1. Did you describe the limitations of your work?
The Limitation section.
3 A2. Did you discuss any potential risks of your work? It can be found in the ethical statements.
3 A3. Do the abstract and introduction summarize the paper’s main claims? The introduction is section 1. The abstract is on the first page.
3 A4. Have you used AI writing assistants when working on this paper? I used chatGPT to proofread and polish some sentences in section 4 and the appendix. The prompt I use is "Please rewrite this sentence in an academic style:".

heading: B Did you use or create scientific artifacts?
text: Not applicable. Left blank.
B1. Did you cite the creators of artifacts you used? Not applicable. Left blank.
B2. Did you discuss the license or terms for use and / or distribution of any artifacts? Not applicable. Left blank.
B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? Not applicable. Left blank.
B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? Not applicable. Left blank.
B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and linguistic phenomena, demographic groups represented, etc.? Not applicable. Left blank.
B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. Not applicable. Left blank.
C 3 Did you run computational experiments? Section 4
7 C1. Did you report the number of parameters in the models used, the total computational budget (e.g., GPU hours), and computing infrastructure used? We don’t report the number of parameters since our model works in inference time.
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.
3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found hyperparameter values? Appendix C
3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? Section 4.3.
3 C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? Section 4.1
D 3 Did you use human annotators (e.g., crowdworkers) or research with human participants? Section 4.3
3 D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? Appendix E
7 D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? We don’t recruit any people for this project. Human evaluation is done by our lab mates and co-authors.
7 D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? We involve human annotation only for evaluation, so we don’t collect/create any data.
7 D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? We only do human evaluation and do not collect/create any data.
7 D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? We don’t recruit any people for this project.

heading: Future Work
text: This suggests future research directions that explore different types of hard constraint energy functions (Zhukov and Kretov, 2017; Casas et al. , 2018) and optimization methods (Rennie et al. , 2017; Liu et al. , 2017) to handle hard constraints with multiple keywords, aiming for faster convergence and higher-quality sentence generation