SlowGuess/NAACL_2022_OCR huggingface dataset:
NEUROLOGIC A\*esque Decoding: Constrained Text Generation with Lookahead Heuristics
Ximing Lu† † Sean Welleck† ‡ Peter West†  
Liwei Jiang† † Jungo Kasai† † Daniel Khashabi‡ Ronan Le Bras‡  
Lianhui Qin† Youngjae Yu‡ Rowan Zellers† Noah A. Smith†‡ Yejin Choi†‡  
‡Allen Institute for Artificial Intelligence
The dominant paradigm for neural text generation is left-to-right decoding from autoregressive language models. Constrained or controllable generation under complex lexical constraints, however, requires foresight to plan ahead for feasible future paths.
Drawing inspiration from the A* search algorithm, we propose NEUROLOGIC A\*esque, a decoding algorithm that incorporates heuristic estimates of future cost. We develop lookahead heuristics that are efficient for large-scale language models, making our method a dropin replacement for common techniques such as beam search and top-k sampling. To enable constrained generation, we build on NEUROLOGIC decoding (Lu et al., 2021), combining its flexibility in incorporating logical constraints with A\*esque estimates of future constraint satisfaction.
Our approach outperforms competitive baselines on five generation tasks, and achieves new state-of-the-art performance on table-to-text generation, constrained machine translation, and keyword-constrained generation. The improvements are particularly notable on tasks that require complex constraint satisfaction or in few-shot or zero-shot settings. NEUROLOGIC A\*esque illustrates the power of decoding for improving and enabling new capabilities of large-scale language models.
The dominant paradigm for neural text generation is based on left-to-right decoding from autoregressive language models such as GPT-2/3 (Radford et al., 2019; Brown et al., 2020). Under this paradigm, common decoding techniques such as beam search or top-k/p sampling (Holtzman et al., 2020) determine which token to generate next based on what happened in the past, without explicitly looking ahead into the future. While
Figure 1: NEUROLOGIC\* leverages lookahead heuristics to guide generations towards those that satisfy the given task-specific constraints. In this example from the COMMONGEN task, although summer is a more likely next word given the already-generated past, NEUROLOGIC\* looks ahead to see that selecting winter results in a generation that incorporates unsatisfied constraint snow with a higher probability later on. Thus, winter is preferred despite being lower probability than summer.
this lack of foresight often suffices for open-ended text generation – where any coherent text can be acceptable – for constrained text generation, planning ahead is crucial for incorporating all desired content in the generated output (Hu et al., 2017; Dathathri et al., 2019).
Classical search algorithms such as A* search (Hart et al., 1968; Pearl, 1984; Korf, 1985) address the challenge of planning ahead by using heuristic estimation of future cost when making decisions. Drawing inspiration from A* search, we develop NEUROLOGIC A\*esque (shortened to NEUROLOGIC\*), which combines A\*-like heuristic estimates of future cost (e.g., perplexity, constraint satisfaction) with common decoding algorithms for neural text generation (e.g., beam search, top- $k$  sampling), while preserving the efficiency demanded by large-scale neural language models.
As selecting the next token to generate based on the optimal future cost is NP-complete (Chen et al., 2018), we develop lookahead heuristics, which approximate cost at each decoding step based on con
' Pronounced [ey star əsk].
Human Language Technologies, pages 780 - 799
tinations of the sequence-so-far. Figure 1 shows an example, where NEUROLOGIC A\*esque guides generation towards a decision that would have been ignored based on the past alone, but is selected after looking ahead and incorporating the probability that constraints are satisfied in the future.
Our approach builds on NEUROLOGIC Decoding of Lu et al. (2021), a variation of beam-search for controlling generation through rich logic-based lexical constraints expressed in Conjunctive Normal Form (CNF). Our work generalizes Lu et al. (2021) by (1) incorporating novel lookahead heuristics to estimate future constraint satisfaction, and (2) developing additional unconstrained variants that can work with an empty set of constraints. These new algorithm variants support broad applications of NEUROLOGIC\*, including unconstrained generation, as demonstrated in our experiments.
Our experiments across five generation tasks demonstrate that our approach outperforms competitive baselines. We test NEUROLOGIC\* in conjunction with both supervised and unsupervised models and find that the performance gain is pronounced especially in zero-shot or few-shot settings. On the  $\mathrm{C_{\text{M}O M N G E N}}$  benchmark, using NEUROLOGIC\* with an off-the-shelf language model outperforms a host of supervised baselines with conventional decoding algorithms, demonstrating that a strong inference-time algorithm such as NEUROLOGIC\* can alleviate the need for costly annotated datasets. Moreover, NEUROLOGIC\* achieves state-of-the-art performance in various settings, including WMT17 English-German machine translation with lexical constraints (Dinu et al., 2019) and few-shot E2ENLG table-to-text generation (Chen et al., 2020b).
In summary, we develop NEUROLOGIC A\*esque, a new decoding algorithm for effective and efficient text generation. To our knowledge this is the first  $\mathrm{A}^*$  -like algorithm for guided text generation via lookahead heuristics. Our algorithm is versatile, as it can be applied to a variety of tasks via inference-time constraints, reducing the need for costly labeled data. Extensive experiments show its effectiveness on several important generation benchmarks.
2 NEUROLOGIC A\*esque Decoding
We describe NEUROLOGIC A\*esque Decoding (shortened as NEUROLOGIC\*), our decoding algorithm motivated by  $\mathbf{A}^*$  search (Hart et al., 1968), a
best-first search algorithm that finds high-scoring paths using a heuristic estimate of future return. We first introduce the decoding problem, and then describe our heuristics with a novel lookahead procedure for adapting NEUROLOGIC\* search to unconstrained and constrained generation with large-scale autoregressive models.
2.1 Decoding With A\*esque Lookahead
Decoding. Sequence-to-sequence generation is the task of generating an output sequence  $\mathbf{y}$  given an input sequence  $\mathbf{x}$ . We consider standard left-to-right, autoregressive models,  $p_{\theta}(\mathbf{y} \mid \mathbf{x}) = \prod_{t=1}^{|\mathbf{y}|} p_{\theta}(y_t \mid \mathbf{y}_{<t}, \mathbf{x})$ , and omit  $\mathbf{x}$  to reduce clutter. Decoding consists of solving,
$$
\mathbf {y} _ {*} = \underset {\mathbf {y} \in \mathcal {Y}} {\arg \max } F (\mathbf {y}), \tag {1}
$$
where  $\mathcal{V}$  is the set of all sequences. In our setting, the objective  $F(\mathbf{y})$  takes the form  $s(\mathbf{y}) + H(\mathbf{y})$ , where  $s(\mathbf{y})$  is  $\log p_{\theta}(\mathbf{y})$ , and  $H(\mathbf{y})$  is either zero when no constraints are specified, or is a score for satisfying constraints on  $\mathbf{y}$ .
Our method takes the perspective of decoding as discrete search, in which states are partial prefixes,  $\mathbf{y}_{< t}$ , actions are tokens in vocabulary  $\mathcal{V}$  (i.e.,  $y_{t} \in \mathcal{V}$ ), and transitions add a token to a prefix,  $\mathbf{y}_{< t} \circ y_{t}$ . Each step of decoding consists of (1) expanding a set of candidate next-states, (2) scoring each candidate, and (3) selecting the  $k$  best candidates:
$$
Y _ {t} ^ {\prime} = \left\{\mathbf {y} _ {<   t} \circ y _ {t} \mid \mathbf {y} _ {<   t} \in Y _ {t - 1}, y _ {t} \in \mathcal {V} \right\},
$$
$$
Y _ {t} = \underset {\left(\mathbf {y} <   t, y _ {t}\right) \in Y _ {t} ^ {\prime}} {\arg \operatorname {t o p k}} \left\{f \left(\mathbf {y} <   t, y _ {t}\right) \right\}, \tag {2}
$$
where  $Y_0 = \{\langle bos\rangle \}$  and  $f(\cdot)$  is a scoring function that approximates the objective  $F$ . Common decoding algorithms such as beam search score candidates without considering future tokens, e.g.,  $f(\mathbf{y}_{<t}, y_t) = \log p_{\theta}(\mathbf{y}_{\leq t})$ .
Lookahead heuristics. Our method incorporates an estimate of the future into candidate selection. Ideally, we want to select candidates that are on optimal trajectories, replacing Equation 2 with:
$$
Y _ {t} = \underset {(\mathbf {y} <   t, y _ {t}) \in Y _ {t} ^ {\prime}} {\arg \operatorname {t o p k}} \left\{\underset {\mathbf {y} > t} {\max } F (\mathbf {y} <   t, y _ {t}, \mathbf {y} > t) \right\}, \tag {3}
$$
where  $\mathbf{y}_{>t}$  represents future trajectories. However, computing Equation 3 presents two difficulties: 1) the objective  $F(\mathbf{y})$  may be unknown or difficult to compute, and 2) the space of  $\mathbf{y}_{>t}$  is prohibitively large.
Motivated by  $\mathbf{A}^*$  search (Hart et al., 1968), a best-first search algorithm that finds high-scoring paths by selecting actions that maximize:
$$
f (a) = s (a) + h (a),
$$
where  $s(a)$  is the score-so-far and  $h(a)$  is a heuristic estimate of the future score, we approximate the objective using a lightweight heuristic  $h(\cdot)$ :
$$
Y _ {t} = \underset {\mathbf {y} _ {\leq t} \in Y _ {t} ^ {\prime}} {\arg \operatorname {t o p k}} \left\{s \left(\mathbf {y} _ {\leq t}\right) + \underset {\mathbf {y} > t} {\max } h \left(\mathbf {y} _ {<   t}, y _ {t}, \mathbf {y} _ {> t}\right) \right\}, \tag {4}
$$
where  $s(\mathbf{y}_{\leq t}) = \log p_{\theta}(\mathbf{y}_{\leq t})$ . To make the search tractable, we search over a set of lookahead continuations, approximating Equation 3 as,
$$
Y _ {t} = \underset {\mathbf {y} _ {\leq t} \in Y _ {t} ^ {\prime}} {\arg \operatorname {t o p k}} \left\{s \left(\mathbf {y} _ {\leq t}\right) + \underset {\mathcal {L} _ {\ell} \left(\mathbf {y} _ {\leq t}\right)} {\max } h \left(\mathbf {y} _ {\leq t + \ell}\right) \right\}, \tag {5}
$$
where each element  $\mathbf{y}_{t + 1:t + \ell}$  of  $\mathcal{L}_{\ell}(\mathbf{y}_{\leq t})$  is a length- $\ell$  continuation of  $\mathbf{y}_{\leq t}$ . Beam search corresponds to setting  $\ell$  and  $h$  to 0.
A\*esque decoding. Beam search,  $\mathbf{A}^*$  search, and our method fall under a general class of algorithms that differ based on (1) which candidates are expanded, (2) which candidates are pruned, (3) how candidates are scored (Meister et al., 2020). We inherit the practical advantages of beam search-style expansion and pruning, while drawing on A\*-like heuristics to incorporate estimates of the future, and refer to our method as A\*esque decoding.
Generating lookaheads. We compare several methods for generating the lookaheads  $\mathcal{L}_{\ell}(\mathbf{y}_{\leq t})$
The greedy lookahead produces a single sequence,  $\mathcal{L}_{\ell} = \{\mathbf{y}_{t + 1:t + \ell}\}$ , starting from  $\mathbf{y}_{\leq t}$  and selecting each token according to  $y_{t'} = \arg \max_{y \in \mathcal{V}} p_{\theta}(y \mid \mathbf{y}_{<t'})$ .
We also consider a soft lookahead which interpolates between providing the greedy token and a uniform mixture of tokens as input at each step. Specifically, we adjust the model's probabilities with a temperature,  $\tilde{p}_{\theta}(y_t \mid \mathbf{y}_{<t}) = \mathrm{softmax}(s_t / \tau)$ , where  $s_t \in \mathbb{R}^{|\mathcal{V}|}$  is a vector of logits, and feed the expected token embedding as input at step  $t$ ,
$$
e _ {t} = \mathbb {E} _ {y _ {t} \sim \tilde {p} (y _ {t} | \mathbf {y} _ {<   t})} [ E (y _ {t}) ], \tag {6}
$$
where  $E \in \mathbb{R}^{|\mathcal{V}| \times d}$  is the model's token embedding matrix. The soft lookahead moves from providing the greedy token as input ( $\tau \to 0$ ) to a uniform mixture of tokens ( $\tau \to \infty$ ) based on the value of temperature  $\tau$ . When using the soft lookahead, we use  $\tilde{p}$  in place of  $p$  when scoring tokens. The
soft (and greedy) lookahead is efficient, but only explores a single trajectory.
The beam lookahead trades off efficiency for exploration, returning a set  $\mathcal{L}_{\ell}$  containing the top- $k$  candidates obtained by running beam search for  $\ell$  steps starting from  $\mathbf{y}_{< t}$ .
Finally, the sampling lookahead explores beyond the highly-probable beam search continuations, generating each  $\mathbf{y}_{t + 1:t + \ell} \in \mathcal{L}_{\ell}$  using,
$$
y _ {t ^ {\prime}} \sim p _ {\theta} (y \mid \mathbf {y} <   t ^ {\prime}),
$$
for  $t'$  from  $t + 1$  to  $t + k$ .
Next, we move to our proposed lookahead heuristics, starting with the unconstrained setting.
2.2 Unconstrained Generation with NEUROLOGIC\*
First we consider a standard decoding setting,
$$
\operatorname *{arg  max}_{\mathbf{y}\in \mathcal{Y}}\log p_{\theta}(\mathbf{y}\mid \mathbf{x}).
$$
We score candidates based on a combination of the history and estimated future, by using the likelihood of the lookahead as a heuristic. That is, at the  $t$ th step of decoding, we use Equation 5 with:
$$
h (\mathbf {y} _ {\leq t + \ell}) = \lambda \log p _ {\theta} (\mathbf {y} _ {t + 1: t + \ell} \mid \mathbf {y} _ {\leq t}, \mathbf {x}), \tag {7}
$$
where  $\lambda$  controls how much we rely on the estimated future versus the history, similar to weighted A\* (Pohl, 1970).
2.3 NEUROLOGIC\* for Constrained Generation
Our lookahead heuristics lend themselves to decoding with lexical constraints in a way that standard beam search does not. For constrained generation, we build on and generalize NEUROLOGIC decoding algorithm of Lu et al. (2021)—a beam-based search algorithm that supports a wide class of logical constraints for lexically constrained generation—with estimates of future constraint satisfaction.
Background of NEUROLOGIC. NEUROLOGIC (Lu et al., 2021) accepts lexical constraints in CNF:
$$
\underbrace {\left(D _ {1} \vee D _ {2} \cdots \vee D _ {i}\right)} _ {C _ {1}} \wedge \dots \wedge \underbrace {\left(D _ {i ^ {\prime}} \vee \cdots \vee D _ {N}\right)} _ {C _ {M}}
$$
where each  $D_{i}$  represents a single positive or negative constraint,  $D(\mathbf{a},\mathbf{y})$  or  $\neg D(\mathbf{a},\mathbf{y})$ , enforcing the phrase  $\mathbf{a}$  to be included in or omitted from  $\mathbf{y}$ . Lu et al. (2021) refer to each constraint  $D_{i}$  as a literal, and each disjunction  $C_{j}$  of literals as a clause.
NEUROLOGIC is a beam-based approximate search for an objective which seeks fluent sequences in which all clauses are satisfied:
$$
\arg \max  _ {\mathbf {y} \in \mathcal {Y}} p _ {\theta} (\mathbf {y} \mid \mathbf {x}) - \lambda^ {\prime} \sum_ {j = 1} ^ {M} (1 - C _ {j}),
$$
where  $\lambda^{\prime}\gg 0$  penalizes unsatisfied clauses. At each step of the search, NEUROLOGIC scores each of the  $k\times |\mathcal{V}|$  candidates  $(\mathbf{y}_{< t},y_{t})$  based on whether they (partially) satisfy new constraints,
$$
f (\mathbf {y} _ {\leq t}) = \log p _ {\theta} \left(\mathbf {y} _ {\leq t} \mid \mathbf {x}\right) + \lambda_ {1} \max  _ {D (\mathbf {a}, \mathbf {y} _ {\leq t})} \frac {| \hat {\mathbf {a}} |}{| \mathbf {a} |}, \tag {8}
$$
where the maximization is over a set of unsatisfied multi-token constraints  $\mathbf{a}$  tracked by NEUROLOGIC, and  $\hat{\mathbf{a}}$  is the prefix of  $\mathbf{a}$  in the ongoing generation. For example, for  $\mathbf{y}_{\leq t} =$  "The boy climbs an apple" and constraint  $\mathbf{a} =$  "apple tree",  $\hat{\mathbf{a}}$  is "apple". Intuitively, this function rewards candidates that are in the process of satisfying a constraint.
In lieu of taking the top- $k$  scoring candidates (Equation 5), NEUROLOGIC prunes candidates that contain clauses that violate constraints, groups the candidates to promote diversity, and selects high-scoring candidates from each group. We use the same pruning and grouping approach, and refer the reader to Lu et al. (2021) for further details.
NEUROLOGIC\* decoding. Our method improves upon the NEUROLOGIC scoring function with an estimate of future constraint satisfaction. Our key addition is a lookahead heuristic that adjusts a candidate  $(\mathbf{y}_{< t}, y_t)$ 's score proportional to the probability of satisfying additional unsatisfied constraints in the lookahead  $\mathbf{y}_{t+1:t+\ell}$ :
$$
h _ {\text {f u t u r e}} (\mathbf {y} \leq t + \ell) =
$$
$$
\lambda_ {2} \max  _ {D (\mathbf {a}, \mathbf {y} _ {\leq t})} \log p _ {\theta} \left(D (\mathbf {a}, \mathbf {y} _ {t + 1: t + \ell}) \mid \mathbf {x}, \mathbf {y} _ {\leq t}\right), \tag {9}
$$
where we define the probability that constraint  $\mathbf{a}$  is satisfied using the most probable subsequence,
$$
\begin{array}{l} p _ {\theta} \left(D \left(\mathbf {a}, \mathbf {y} _ {t + 1: t + \ell}\right) \mid \mathbf {x}, \mathbf {y} _ {\leq t}\right) = \\ \max  _ {t ^ {\prime} \in [ t, t + \ell ]} p _ {\theta} \left(\mathbf {y} _ {t ^ {\prime}: t ^ {\prime} + | \mathbf {a} |} = \mathbf {a} \mid \mathbf {x}, \mathbf {y} _ {<   t ^ {\prime}}\right), \tag {10} \\ \end{array}
$$
$\lambda_{2}$  is a scaling hyperparameter for the heuristic.
Intuitively, this lookahead heuristic brings two benefits. When  $y_{t}$  is a token that would satisfy a multi-token constraint, the lookahead incorporates the score of the full constraint. When  $y_{t}$  is a token that is not part of a constraint, the lookahead allows for incorporating the score of a future constraint that would be satisfied if  $y_{t}$  was selected.
We add our lookahead heuristic to the NEUROLOGIC scoring function (Equation 8), and call the resulting decoding procedure NEUROLOGIC A\*esque (or, NEUROLOGIC\* in short).
We first consider constrained generation benchmarks:  $\mathbf{C o m m o n G e n}$  (3.1), constrained machine translation (3.2), table-to-text generation (3.3), and constrained question generation (3.4). NEUROLOGIC\* consistently outperforms previous approaches, especially in zero-shot and few-shot cases. These low-resource settings are particularly important, as many practical tasks face data scarcity. Finally, we find that A\*esque lookahead is useful even without constraints, as shown in unconstrained story generation task (3.5).
Metrics. As automatic metrics, we use: BLEU (Papineni et al., 2002), ROUGE (Lin, 2004), METEOR (Banerjee and Lavie, 2005), CIDEr (Vedantam et al., 2015), SPICE (Anderson et al., 2016) and NIST (Lin and Hovy, 2003).
3.1 Constrained Commonsense Generation
CommonGen (Lin et al., 2020) is a commonsense generation task with lexical constraints: given a set of concepts (e.g., {throw, run, javelin, track}), models need to generate a coherent sentence describing a plausible scenario using all given concepts (e.g., "a man runs on a track and throws a javelin").
Approach and Baselines. Following Lu et al. (2021), we enforce that each concept  $c_{i}$  appear in output  $\mathbf{y}$  under some morphological inflection. We test in both supervised and zero-shot settings. In the supervised setting, we finetune GPT-2 (Radford et al., 2019) as a sequence-to-sequence model. In the zero-shot setting, we use GPT-2 off-the-shelf (no fine-tuning) and rely on constrained decoding to guide generation. We compare with previous constrained decoding algorithms CBS (Anderson et al., 2017), GBS (Hokamp and Liu, 2017), DBA (Post and Vilar, 2018a), NEUROLOGIC (Lu et al., 2021) and TSMH (Zhang et al., 2020).
Metrics. We report standard automatic metrics as well as coverage, the average percentage of concepts present in generations. Additionally, we conduct human evaluation on 100 test examples using Amazon Mechanical Turk (AMT), with 3 annotators per example (template in Appendix D). Workers rate each generation on language quality, sce
Decode Method	Automatic Evaluation	Human Evaluation
ROUGE-L	BLEU-4	METEOR	CIDEr	SPICE	Coverage	Quality	Plausibility	Concepts	Overall
Supervised
CBS (Anderson et al., 2017)	38.8	20.6	28.5	12.9	27.1	97.6	2.27	2.35	2.51	2.23
GBS (Hokamp and Liu, 2017)	38.2	18.4	26.7	11.7	26.1	97.4	2.06	2.17	2.29	2.01
DBA (Post and Vilar, 2018a)	38.3	18.7	27.7	12.4	26.3	97.5	2.23	2.30	2.43	2.15
NEUROLOGIC (Lu et al., 2021)	42.8	26.7	30.2	14.7	30.3	97.7	2.54	2.56	2.67	2.50
NEUROLOGIC★ (greedy)	43.6	28.2	30.8	15.2	30.8	97.8	2.66	2.67	2.73	2.59
NEUROLOGIC★ (sample)	43.4	27.9	30.8	15.3	31.0	97.7	2.64	2.64	2.74	2.58
NEUROLOGIC★ (beam)	43.2	28.2	30.7	15.2	31.0	97.6	2.68	2.67	2.76	2.60
Unsupervised
TSMH (Zhang et al., 2020)	24.7	2.2	14.5	3.6	15.4	71.5	1.85	1.92	1.95	1.63
NEUROLOGIC (Lu et al., 2021)	41.9	24.7	29.5	14.4	27.5	96.7	2.64	2.52	2.68	2.50
NEUROLOGIC★ (greedy)	44.3	28.6	30.7	15.6	29.6	97.1	2.78	2.70	2.77	2.70
Table 1: Performance of various decoding methods with supervised or off-the-shelf GPT-2 on the COMMONGEN test set, measured with automatic and human evaluations. We only tried NEUROLOGIC  $\star$  (greedy) in the unsupervised setting because of the computational cost. The best numbers are bolded and the second best ones are underlined.
Words	Method	Generation
cut	GBS	Cut a piece of wood to use as a fence.
piece	DBA	Cut a piece of wood to use as a fence.
use	NEUROLOGIC	Piece of wood used for cutting.
wood	NEUROLOGIC★	A man cuts a piece of wood using a circular saw.
ball	GBS	A dog is run over by a ball and mouth agape.
dog	DBA	A dog is run over by a ball and bites his mouth.
mouth	NEUROLOGIC	A dog is running and chewing on a ball in its mouth.
run	NEUROLOGIC★	A dog running with a ball in its mouth.
dog	GBS	Soap and water scrubbed dog with a towel.
scrub	DBA	Soap and water on a dog and scrubbed skin.
soap	NEUROLOGIC	A dog is scrubbing his paws with soap and water.
water	NEUROLOGIC★	A man is scrubbing a dog with soap and water.
nario plausibility, coverage of given concepts, and an overall score on a 3-point Likert scale.2
Results. Table 1 compares different constrained decoding methods on top of the finetuned and off-the-shelf GPT-2, in supervised and zero-shot settings respectively. The key observations are:
1. NEUROLOGIC\* outperforms all previous constrained-decoding methods in both supervised and zero-shot settings. Surprisingly, unsupervised NEUROLOGIC\* outperforms all supervised methods based on human evaluation.
2. Compared to vanilla NEUROLOGIC, NEUROLOGIC\* improves generation quality while maintaining high constraint satisfaction. The difference is especially substantial in the zero-shot setting. Intuitively, this setting leaves
Table 2: Example generations for the COMMONGEN task across supervised NEUROLOGIC\* and baselines, including GBS (Hokamp and Liu, 2017), DBA (Post and Vilar, 2018a), and NEUROLOGIC (Lu et al., 2021).
Method	Dinu et al.	Marian MT
BLEU	Term%	BLEU	Term%
Unconstrained	25.8	76.3	32.9	85.0
train-by-app.	26.0	92.9	-	-
train-by-rep.	26.0	94.5	-	-
Post and Vilar (2018a)	25.3	82.0	33.0	94.3
NEUROLOGIC	26.5	95.1	33.4	97.1
NEUROLOGIC★ (greedy)	26.7	95.8	33.7	97.2
NEUROLOGIC★ (sample)	26.6	95.4	33.7	97.2
NEUROLOGIC★ (beam)	26.6	95.8	33.6	97.2
Table 3: Results on constrained MT. The left section uses the same two-layer transformer as Dinu et al. (2019), while the right one uses a stronger Marian MT EN-DE model. The highlighted methods modify training data specifically for constrained generation, and thus cannot be applied to off-the-shelf models. The best numbers are bold, second best are underlined.
more room for incorporating constraint-driven signals due to the lack of supervision.
3. NEUROLOGIC\* reaches similar performance using different lookahead strategies, among which beam lookahead slightly outperforms the others based on human evaluation, and greedy lookahead has the lowest runtime. We analyze lookahead strategies further in Appendix A.
3.2 Constrained Machine Translation
Next, we test on constrained machine translation (MT). It is often critical to have control over MT systems, such as to incorporate domain-specific terminology (Post and Vilar, 2018a; Dinu et al., 2019). To achieve this goal, recent work proposed constrained decoding algorithms (Chatterjee et al., 2017; Hokamp and Liu, 2017; Hasler et al., 2018; Hu et al., 2019, inter alia) or specialized training (Dinu et al., 2019). We demonstrate that
Agreement by ordinal Krippendorff alpha  $(0\leq \alpha \leq 1)$  (Krippendorff, 2007) is 0.40, 0.46, 0.36, and 0.44 (respectively) indicating fair to moderate agreement.