SlowGuess/NAACL_2022_OCR huggingface dataset:
Kenton Lee, Mike Lewis, and Luke Zettlemoyer. 2016. Global neural CCG parsing with optimality guarantees. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2366-2376, Austin, Texas. Association for Computational Linguistics.
Xiang Lisa Li and Percy Liang. 2021. Prefix-tuning: Optimizing continuous prompts for generation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 4582-4597, Online. Association for Computational Linguistics.
Bill Yuchen Lin, Ming Shen, Wangchunshu Zhou, Pei Zhou, Chandra Bhagavatula, Yejin Choi, and Xiang Ren. 2020. Commongen: A constrained text generation challenge for generative commonsense reasoning. In Findings of EMNLP.
Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. Text Summarization Branches Out.
Chin-Yew Lin and Eduard Hovy. 2003. Automatic evaluation of summaries using n-gram cooccurrence statistics. In Proceedings of the 2003 Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics, pages 150-157.
Ximing Lu, Peter West, Rowan Zellers, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. 2021. NeuroLogic decoding: (un)supervised neural text generation with predicate logic constraints. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4288-4299, Online. Association for Computational Linguistics.
Kris McGuffie and Alex Newhouse. 2020. The radicalization risks of gpt-3 and advanced neural language models. arXiv.
Clara Meister, Tim Vieira, and Ryan Cotterell. 2020. Best-first beam search. Transactions of the Association for Computational Linguistics, 8:795-809.
Ning Miao, Hao Zhou, Lili Mou, Rui Yan, and Lei Li. 2019. Cgmh: Constrained sentence generation by metropolis-hastings sampling. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 6834-6842.
Nasrin Mostafazadeh, Nathanael Chambers, Xiaodong He, Devi Parikh, Dhruv Batra, Lucy Vanderwende, Pushmeet Kohli, and James Allen. 2016. A corpus and cloze evaluation for deeper understanding of commonsense stories. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 839-849, San Diego, California. Association for Computational Linguistics.
Iftekhar Naim, Daniel Gilddea, Walter Lasecki, and Jeffrey P Bigham. 2013. Text alignment for real-time crowd captioning. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 201-210.
Franz Josef Och, Nicola Ueffing, and Hermann Ney. 2001. An efficient a* search algorithm for statistical machine translation. In Proceedings of the ACL 2001 Workshop on Data-Driven Methods in Machine Translation.
Jayshree Pandya. 2019. The dual-use dilemma of artificial intelligence. Forbes Magazine.
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. BLEU: a method for automatic evaluation of machine translation. In ACL, pages 311-318.
Judea Pearl. 1984. Heuristics - intelligent search strategies for computer problem solving. In Addison-Wesley series in artificial intelligence.
Ira Pohl. 1970. First Results on the Effect of Error in Heuristic Search.
Matt Post and David Vilar. 2018a. Fast lexically constrained decoding with dynamic beam allocation for neural machine translation. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 1314-1324, New Orleans, Louisiana. Association for Computational Linguistics.
Matt Post and David Vilar. 2018b. Fast lexically constrained decoding with dynamic beam allocation for neural machine translation. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 1314-1324.
Lianhui Qin, Vered Shwartz, Peter West, Chandra Bhagavatula, Jena D Hwang, Ronan Le Bras, Antoine Bosselut, and Yejin Choi. 2020. Backpropagation-based decoding for unsupervised counterfactual and abductive reasoning. In EMNLP.
Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018. Improving language understanding by generative pre-training.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI Blog, 1(8):9.
Sheng Shen, Daniel Fried, Jacob Andreas, and Dan Klein. 2019. Pragmatically informative text generation. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers),
pages 4060-4067, Minneapolis, Minnesota. Association for Computational Linguistics.
Ramakrishna Vedantam, C Lawrence Zitnick, and Devi Parikh. 2015. Cider: Consensus-based image description evaluation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4566-4575.
Sean Welleck, Kianté Brantley, Hal Daumé Iii, and Kyunghyun Cho. 2019a. Non-monotonic sequential text generation. In International Conference on Machine Learning, pages 6716-6726. PMLR.
Sean Welleck, Ilia Kulikov, Stephen Roller, Emily Dinan, Kyunghyun Cho, and Jason Weston. 2019b. Neural text generation with unlikelihood training. In International Conference on Learning Representations.
Peter West, Ximing Lu, Ari Holtzman, Chandra Bhagavatula, Jena D. Hwang, and Yejin Choi. 2021. Reflective decoding: Beyond unidirectional generation with off-the-shelf language models. In ACL/IJCNLP.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumont, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander Rush. 2020. Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP): System Demonstrations.
Hao Zhang and Daniel Gildea. 2006. Efficient search for inversion transduction grammar. In Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing, pages 224-231.
Maosen Zhang, Nan Jiang, Lei Li, and Yexiang Xue. 2020. Language generation via combinatorial constraint satisfaction: A tree search enhanced Monte-Carlo approach. In *Findings of the Association for Computational Linguistics: EMNLP* 2020, pages 1286-1298, Online. Association for Computational Linguistics.
Renjie Zheng, Mingbo Ma, Baigong Zheng, Kaibo Liu, and Liang Huang. 2020. Opportunistic decoding with timely correction for simultaneous translation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 437-442.
Figure 4: Likelihood (y-axis) vs. number of unique 3-grams (x-axis) using supervised GPT-2 on RocStories. Figure (a) denotes decoding with beam search, with a varying amount of beam size. Figure (b) denotes decoding with top-k sampling, with a varying amount of k value. The brown and blue lines denote with and without A\*esque heuristics separately.
Figure 3: Effect of varying the primary hyperparameter for each lookahead strategy (§2.1) – (a) greedy (lookahead length), (b) soft (temperature), (c) beam (number of beams), and (d) sample (number of samples). Performance is measured on the COMMONGEN validation set, using BLEU-4 and Coverage.
A Further Experiments
A.1 Constrained Commonsense Generation
Studying Lookahead Strategies. We further use  $\mathsf{C}_{\mathsf{M}\mathsf{o}\mathsf{M}\mathsf{O}\mathsf{N}\mathsf{G}\mathsf{E}\mathsf{N}}$  to study the lookahead strategies for NEUROLOGIC\* proposed in  $\S 2.1$  (Figure 3). With infinite lookahead length  $\ell$  and number of lookaheads  $|\mathcal{L}_\ell |$  , lookahead decoding exactly solves Equation 3, finding an optimal trajectory. In practice these are finite, meaning that the quality of the lookahead approximation can depend on the lookahead strategy and its hyperparameters. For practical choices of  $\ell$  and  $|\mathcal{L}_\ell |$  , we empirically study how varying the lookahead strategy and hyperparameters affects performance. In Figure 3, we study the greedy, soft, beam, and sampling lookahead strategies.
Figure 3(a) shows the effect of increasing the lookahead length  $\ell$  for the greedy lookahead strategy. Increasing the length improves up to one point - e.g., 5-7 steps - then decreases thereafter, likely due to the difficulty of long-horizon approximation.
Figure 3(b) studies the temperature in the soft lookahead, showing that greedy  $(\tau = 0.0)$  performs well, with slight gains if  $\tau$  is carefully selected. The results suggest that one can safely bypass tuning  $\tau$  using fast, greedy lookahead.
Next, Figure 3(c) shows that with beam lookahead, increasing the beam width improves performance up to a certain point (here, 11). Similarly, increasing the number of samples with sampling
lookahead improves over a single sample, and then reaches an inflection point (Figure 3(d)).
A.2 Unconstrained Story Generation
Fluency and Diversity Tradeoff We study the effect of A\*esque decoding in unconstrained generation with different decoding hyperparameters: beam size in beam search and k value in top-k sampling. Figure 4 plots the fluency (measured by likelihood) versus diversity (measured by unique 3-grams) for generations with various beam sizes or top-k values. Ideally, we want generations to be both fluent and diverse (top right). However, we observe a fluency and diversity tradeoff in practice. A\*esque decoding flattens this trend and results in larger area under the curve. The effect is especially strong with beam search. In summary, A\*esque decoding yields a more favorable balance of fluency and diversity compared to conventional decoding methods, regardless of hyperparameters.
B Runtime
Decoding Method	Runtime
Beam Search	0.20
NEUROLOGIC	2.04
NEUROLOGIC A★esque	19.24
Table 9: Runtime (seconds per sentence) of different decoding algorithms with finetuned GPT2-L on the COMMONGEN dataset
C.1 Off-the-Shelf Models
We download off-the-shelf models, including pretrained GPT-2 and Marian MT, from HuggingFace
Transformers (Wolf et al., 2020), which are implemented in the PyTorch deep learning framework.
C.2 Model Training Details
All training is performed on a single NVIDIA Quadro RTX 8000 GPU and costs about 100 GPU hours in total. Our method is implemented with PyTorch an the Huggingface Transformers library.
C.2.1 COMMONGEN
For supervised setting, we finetune GPT-2 for conditional generation. We follow Lu et al. (2021)'s setup and use their hyperparameters for finetuning, as shown in Table 10.
Hyperparameter	Assignment
model	GPT2-Large
number of parameters	774M
number of steps	15 epochs
batch size	64
learning rate optimizer	Adam
Adam epsilon	1e-8
Adam initial learning rate	1e-5
learning rate scheduler	linear with warmup
warmup steps	1.5 epoch
weight decay	0
Table 10: Hyperparameters for finetuning GPT-2 on COMMONGEN dataset.
C.2.2 Constrained Machine Translation
For fair comparison, we reproduced MT model (two-layer transformer) used by Dinu et al. (2019), using the same setup and hyperparameters reported in their original paper.
C.2.3 Table-to-text Generation
We finetune GPT-2 with random sampled few-shot training instances from E2ENLG dataset. We used the same hyperparameters for finetuning with Li and Liang (2021), as shown in Table 11.
Hyperparameter	Assignment
model	GPT2-Large
number of parameters	774M
number of steps	5 epochs
batch size	5
learning rate optimizer	Adam
Adam epsilon	1e-8
Adam initial learning rate	5e-5
learning rate scheduler	linear with warmup
warmup steps	100
weight decay	0
Table 11: Hyperparameters for finetuning GPT-2 on E2ENLG dataset.
C.2.4 Unconstrained Story Generation
We finetune GPT-2 for conditional story generation on the RocStories dataset: given a first sentence  $\mathbf{x}$ , generate the full story  $\mathbf{y}$ . Hyperparameters for finetuning are given in Table 12.
Hyperparameter	Assignment
model	GPT2-Large
number of parameters	774M
number of steps	10 epochs
batch size	64
learning rate optimizer	Adam
Adam epsilon	1e-8
Adam initial learning rate	1e-5
learning rate scheduler	linear with warmup
warmup steps	1 epoch
weight decay	0
Table 12: Hyperparameters for finetuning GPT-2 on the RocStories dataset.
C.3 Generation Details
All generation is performed on a single NVIDIA Quadro RTX 8000 GPU and costs about 100 GPU hours in total.
C.3.1 COMMONGEN
NEUROLOGIC\* hyperparameters for CommonGen in supervised and zero-shot setting are shown in Table 13 and Table 14 separately. We use the same NEUROLOGIC hyperparameters with Lu et al. (2021), including beam size,  $\alpha ,\beta$  and  $\lambda_{1}$  .We performed a hyperparameter grid search for the scaling factor  $\lambda_{2}$  over the range [0, 0.3], for the look ahead step over the the range [1, 15], for the look ahead temperature over the the range [0, 1.0], for the look ahead beam width over the the range [1, 10], and for the look ahead number of sample over the the range [1, 10], using a small subset of CommonGEN development set.
Hyperparameter	Assignment
beam size	20
pruning threshold α	50
pruning threshold β	2
scaling factor λ1	0
scaling factor λ2	0.25
look ahead step	5
look ahead (greedy) temperature	0
look ahead (beam) beam width	5
look ahead (sample) number of sample	4
Table 13: NEUROLOGIC\* hyperparameters for COMMONGEN in supervised setting.
Hyperparameter	Assignment
beam size	20
pruning threshold α	500000
pruning threshold β	2
scaling factor λ1	0
scaling factor λ2	0.175
look ahead step	5
look ahead (greedy) temperature	0
C.3.2 Constrained Machine Translation
NEUROLOGIC\* hyperparameters for constrained machine translation are shown in Table 15. We use the same beam size with Dinu et al. (2019) for fair comparison. We performed a hyperparameter grid search for the pruning threshold  $\alpha$  over the range [50,300], for the pruning threshold  $\beta$  over the range [1,3], for the scaling factor  $\lambda_{1}$  over the range [0,1.0], for the scaling factor  $\lambda_{2}$  over the range [0,0.3], for the look ahead step over the the range [5,40], using a subset of WMT2013 IATE development set. We use the same hyperparameters for look ahead temperature, look ahead beam width, and look ahead number of sample with supervised  $\mathrm{C}_{\mathrm{MOMONGEN}}$  and omit the hyperparameter search due to the computational cost.
Table 14: NEUROLOGIC\* hyperparameters for COMMONGEN in zero-shot setting.
Hyperparameter	Assignment
beam size	5
pruning threshold α	200
pruning threshold β	2
scaling factor λ1	0.25
scaling factor λ2	0.05
look ahead step	35
look ahead (greedy) temperature	0
look ahead (beam) beam width	5
look ahead (sample) number of sample	4
C.3.3 Table-to-text Generation
NEUROLOGIC\* hyperparameters for table-to-text generation are shown in Table 16. We performed a hyperparameter grid search for the scaling factor  $\lambda_{2}$  over the range [0, 0.3], for the look ahead step over the the range [1, 15], using E2ENLG development set. For other hyperparameters, we use the same value with supervised  $\mathrm{C}_{\mathrm{MOMONGEN}}$  and omit the hyperparameter search due to the computational cost.
Table 15: NEUROLOGIC\* hyperparameters for constrained machine translation.
Hyperparameter	Assignment
beam size	20
pruning threshold α	50
pruning threshold β	2
scaling factor λ1	0
scaling factor λ2	0.05
look ahead step	7
look ahead (greedy) temperature	0
look ahead (beam) beam width	5
look ahead (sample) number of sample	4
C.3.4 Constrained Question Generation
NEUROLOGIC\* hyperparameters for constrained question generation are shown in Table 17. The task is zero-shot and doesn't provide train or development set, so we use the same decoding hyperparameters with zero-shot  $\mathrm{COMMONGEN}$
Table 16: NEUROLOGIC\* hyperparameters for table-to-text generation.
Hyperparameter	Assignment
beam size	20
pruning threshold α	500000
pruning threshold β	2
scaling factor λ1	0
scaling factor λ2	0.175
look ahead step	5
look ahead (greedy) temperature	0
look ahead (beam) beam width	5
look ahead (sample) number of sample	4
Table 17: NEUROLOGIC\* hyperparameters for constrained question generation.
C.3.5 Unconstrained Story Generation
A\*esque hyperparameters with beam search and top-k sampling for unconstrained story generation are shown in Table 18 and Table 19 separately. We performed a hyperparameter grid search for the scaling factor  $\lambda_{2}$  over the range [0, 1.0], for the look ahead step over the the range [1, 15], for the look ahead temperature over the the range [0, 1.0], for the look ahead beam width over the the range [1, 15], and for the look ahead number of sample over the the range [1, 15], using a small subset of RocStories development set.
C.4 Dataset Details
Details of datasets used for downstream tasks are provided in Table 22.
We include screenshots of the human evaluation templates for CommonGen (Figure 5), Constrained
Hyperparameter	Assignment
beam size	4
scaling factor λ2	0.6
look ahead step	4
look ahead (greedy) temperature	0
look ahead (beam) beam width	4
look ahead (sample) number of sample	15
Table 18: A\*esque hyperparameters with beam search for unconstrained story generation.
Hyperparameter	Assignment
k value	5
scaling factor λ2	0.5
look ahead step	3
look ahead (greedy) temperature	0
look ahead (beam) beam width	4
look ahead (sample) number of sample	15
Table 19: A\*esque hyperparameters with top-k sampling for unconstrained story generation.
Question Generation (Figure 6), and RocStories (Figure 7) tasks. We ensure the annotators are paid adequately for at least \(15 per hour and we inform annotators that their annotations are used for model evaluation purpose.
E Qualitative Generation Examples
Qualitative examples of the constrained question generation and unconstrained story generation are shown in Table 21 and 20.
F Limitations and Risks.
Limitations. For constrained generation, NEUROLOGIC A\*esque decoding can only take the constraints that can be formulated as logical expressions as described in the paper; we leave it to future work to expand the scope of such logical constraints.
Risks. Constrained generation methods could foreseeably be used to generate text that contains biased, offensive, and/or hateful keywords. (e.g., extremist texts; McGuffie and Newhouse, 2020). For a broader discussion of these risks, and of the risks of large pretrained language models in general, refer to discussions in Brown et al. (2020); Bender et al. (2021).
{\source}
{$generation}
1. SENTENCE QUALITY: Is the sentence well-formed?
Yes: The sentence is well-formed and fluent.
$\bigcirc$  Somewhat: The sentence is understandable but a bit awkward.
$\bigcirc$  No: The sentence is neither well-formed or fluent.
2. PLAUSIBILITY: Does the sentence describe a plausible scenario?
$\bigcirc$  Yes: The sentence describes a realistic or plausible scenario.
$\bigcirc$  Somewhat: The sentence describes a acceptable scenario but a bit awkward.
$\bigcirc$  No: The sentence describes a nonsensical scenario.
3. CONCEPTS: Does the sentence include the given concepts meaningfully?
Yes: The sentence meaningfully includes all of the concepts.
Somewhat: The sentence meaningfully includes some, but not all of the concepts. Or, the sentence includes all concepts but some of them are not meaningful or properly incorporated.
$\bigcirc$  No: The sentence does not include concepts in a meaningful way.
4. OVERALL: Considering your answers to 1., 2. and 3., Does the sentence meaningfully combine all of the concepts into a well-formed and plausible scenario?
Yes: The sentence is reasonably well-formed/understandable, and meaningfully combines all the concepts into a plausible scenario.
$\bigcirc$  Somewhat: The sentence looks okay in terms of above questions.
No: The sentence is not well-formed/understandable, or fails to properly combine all the concepts into a plausible scenario.
Figure 5: Human evaluation template for the Constrained Commonsense Generation task.
List of Keywords:
${source}
Q1. Grammar Is the question written in a grammatically correct way?
Yes It is entirely or mostly grammatically correct, with no or minimal grammatical mistakes.
Somewhat It is partially grammatically correct, with some grammatical mistakes.
No It is mostly not grammatically correct, with many grammatical mistakes.
Q2. Fluency Is the question written in a fluent and understandable way?
Yes It is entirely or mostly fluent and understandable.
Somewhat It is somewhat fluent and understandable, but it reads a bit awkward.
No It is mostly poorly written and hard to understand.
Q3. Meaningfulness Does the given question sentence ask a meaningful question?
Yes It is an entirely or mostly meaningful question.
Somewhat It is a somewhat meaningful question, but it might be a bit unclear.
No It is mostly not a meaningful question.
Q4. Overall Consider grammar, fluency and meaningfulness, overall, what's the quality of the question?
Good The overall quality is high.
$\mathbb{O}$  ok The overall quality is ok.
Bad The overall quality is low.
Figure 6: Human evaluation template for the Interrogative Sentence Generation task.
First sentence of the story:
Continuation of the story:
Q1. Grammar Is the continuation of the story written in a grammatically correct way?
Q2. Fluency Is the continuation of the story written in a fluent and understandable way?
Q3. Story Flow Does the continuation of the story flow coherently from the prompt and stay on-topic?
Yes It is entirely or mostly coherent from the prompt, and stays on-topic.
Somewhat It is somewhat coherent from the prompt, but it reads a bit off-topic.
No It is mostly not coherent from the prompt, and mostly off-topic.
Q4. Interestingness Is the continuation of the story written in an interesting way?
Yes It is a very interesting story.
Somewhat It is a somewhat interesting story.
No It is not an interesting story.
Q5. Overall Consider the above questions, overall, what's the quality of the continuation of the story?
Good The overall quality is high.
$\mathbf{O}\mathbf{k}$  The overall quality is ok.
Bad The overall quality is low.
Figure 7: Human evaluation template for the RocStories task.