SlowGuess/NAACL_2021_OCR huggingface dataset:
NEUROLOGIC DECODING: (Un)supervised Neural Text Generation with Predicate Logic Constraints
Ximing Lu†‡ Peter West†‡ Rowan Zellers†‡
Ronan Le Bras† Chandra Bhagavatula‡ Yejin Choi†‡
$\ddagger$ Allen Institute for Artificial Intelligence
{lux32, pawest, rowanz, yejin}@cs.washington.edu
{ronan1b, chandrab}@allenai.org
Conditional text generation often requires lexical constraints, i.e., which words should or shouldn't be included in the output text. While the dominant recipe for conditional text generation has been large-scale pretrained language models that are finetuned on the task-specific training data, such models do not learn to follow the underlying constraints reliably, even when supervised with large amounts of task-specific examples.
We propose NEUROLOGIC DECODING, a simple yet effective algorithm that enables neural language models - supervised or not - to generate fluent text while satisfying complex lexical constraints. Our approach is powerful yet efficient. It handles any set of lexical constraints that is expressible under predicate logic, while its asymptotic runtime is equivalent to conventional beam search.
Empirical results on four benchmarks show that NEUROLOGIC DECODING outperforms previous approaches, including algorithms that handle a subset of our constraints. Moreover, we find that unsupervised models with NEUROLOGIC DECODING often outperform supervised models with conventional decoding, even when the latter is based on considerably larger networks. Our results suggest the limit of large-scale neural networks for fine-grained controllable generation and the promise of inference-time algorithms.
Text generation applications often need to incorporate semantic constraints, i.e., what words should and shouldn't appear in the output generation. Consider the task of generating a recipe from a set of ingredients (Kiddon et al., 2016), such as 'garlic,' 'steak', and 'soy sauce' (Figure 1). A generated recipe should cover all of those ingredients, without hallucinating new ones (such as 'pork' or 'beans'). This restriction, like others in Figure 1 for other
COMMONGEN (Lin et al., 2019)
Figure 1: Overview of several constrained generation tasks. For instance, generating a short description from a set of concepts (COMMONGEN; Lin et al., 2020) requires using each of those words at least once; this can be expressed as a logical expression (here, 'food  $\vee$  foods)  $\wedge \ldots$ ). Our proposed NEUROLOGIC DECODING handles all predicate logic constraints efficiently, yet with the same asymptotic runtime as beam search.
applications, can be modeled by a set of lexical constraints expressed as a predicate logic formula.
The dominant paradigm today for performing such constrained generation is to start with a pretrained language model, and then finetune it on a dataset of task-specific examples. However, pretrained language models struggle at learning to
Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4288-4299 June 6-11, 2021. ©2021 Association for Computational Linguistics
follow these constraints, even when the finetuning dataset is large. For example, for the aforementioned recipe generation task, a GPT2 model finetuned on hundreds of thousands of recipes still hallucinates extra ingredients. In stark contrast, humans need to see only a few examples (or even none) to generate the desired output satisfying all the logical constraints, e.g., writing a recipe that mentions each ingredient (butter, steak, etc.) without using new ones.
We hypothesize that this mismatch is due to a fundamental under-specification of finetuning. If we finetune one of today's state-of-the-art language models on a dataset, the likelihood of it generating sequences from the same distribution should increase. Yet there is no guarantee that this improvement in likelihood will come from improvements on the fundamental task of constrained generation, as opposed to picking up on dataset-specific patterns such as language style. In fact, we present analysis suggesting that 'worst-case' learning behavior is common in practice: when we increase the finetuning data fed to GPT2 by an order of magnitude, constraint-satisfaction with standard beam search shows only modest improvement.
To address this issue, we propose NEUROLOGIC DECODING, which effectively enforces the satisfaction of given lexical constraints by controlling the decoding stage of sequence generation. These constraints can be any predicate logic formula, which crucially includes both positive constraints (the word 'butter' must be generated somewhere) and negative constraints ('bean' cannot be generated). These simpler constraints can then be combined through logical connectives to handle more complex requirements such as inflection or synonyms ('beef' or 'steak' both satisfy the constraint of referring to the steak). While beam search aims to maximize the likelihood of the generated sequence, our method searches for optimal output sequences among the strings that also satisfy the given constraints. It does so efficiently: we convert the hard logic constraints into a soft penalty term in the decoding objective, and use a beam-based search to find approximately-optimal solutions; constraint states are tracked to reuse computation. NEUROLOGIC DECODING thus effectively and efficiently controls text generation without requiring any modification of the model structure or training pipeline.
We evaluate our method on four different text generation tasks: generative commonsense reason-
ing (COMMONGEN; Lin et al., 2020), recipe generation (Kiddon et al., 2016), data-grounded dialogue response generation (Wen et al., 2015), and reducing gender bias in machine translation (Stanovsky et al., 2019). Empirical results demonstrate that NEUROLOGIC DECODING ensures the satisfaction of given constraints while maintaining high generation quality, in turn leading to new SOTA results in both the supervised and zero-shot setting.
In this section, we first rigorously define predicate logic constraint, and then present in detail the NEUROLOGIC DECODING algorithm.
2.1 Predicate Logic Constraint
Let us define a predicate  $D(\mathbf{a},\mathbf{y})$  to be a boolean function indicating the occurrence of key phrase  $\mathbf{a}$  in a sequence  $\mathbf{y}$ , where  $\mathbf{a}$  can be either unigram or multi-gram.  $D(\mathbf{a},\mathbf{y})$  will be true iff  $\mathbf{a}$  occurs in  $\mathbf{y}$ .
$$
D (\mathbf {a}, \mathbf {y}) \equiv \exists i, \mathbf {y} _ {i: i + | \mathbf {a} |} = \mathbf {a}
$$
NEUROLOGIC accepts lexical constraints in Conjunctive Normal Form (CNF):
$$
\underbrace {\left(D _ {1} \vee D _ {2} \cdots \vee D _ {i}\right)} _ {C _ {1}} \wedge \dots \wedge \underbrace {\left(D _ {k} \vee D _ {k + 1} \cdots \vee D _ {n}\right)} _ {C _ {m}}
$$
where each  $D_{i}$  represents a single positive or negative constraint,  $D(\mathbf{a_i},\mathbf{y})$  or  $\neg D(\mathbf{a_i},\mathbf{y})$ , restricting whether key phrase  $\mathbf{a_i}$  should be strictly included or omitted in  $\mathbf{y}$ , respectively. Any propositional logical formula can be converted to CNF, and thus handled by NEUROLOGIC. Notationally, we will refer to each individual constraint  $D_{i}$  as a literal, and the disjunction of literals as a clause, denoted as  $C_j$ , with  $L$  being the total number of clauses. Our method seeks optimal sequences in which all clauses are satisfied:
$$
\hat {\mathbf {y}} = \underset {\mathbf {y} \in \mathcal {Y}} {\arg \max } P _ {\theta} (\mathbf {y} | \mathbf {x}) \quad \text {w h e r e} \quad \sum_ {i = 1} ^ {L} C _ {i} = L \tag {1}
$$
Past work on constrained optimization introduces penalties (Fiacco, 1976) to approximate the constrained optimization problem with an unconstrained problem. Specifically, by adding a high-cost penalty term for violated constraints:
$$
\hat {\mathbf {y}} = \arg \max  _ {\mathbf {y} \in \mathcal {Y}} P _ {\theta} (\mathbf {y} | \mathbf {x}) - \lambda^ {\prime} \sum_ {i = 1} ^ {L} \left(1 - C _ {i}\right) \tag {2}
$$
Figure 2: Clause states and possible transitions.  $D_{i}$  and  $\neg D_{i}$  denote positive and negative literal respectively.
Intuitively, this objective balances sequence likelihood (term 1) and constraint satisfaction (term 2). The aim is to find sequences that do well at both dimensions. While exhaustive search is intractable, we use a beam-based search to find approximately-optimal solutions for this objective.
2.2 Constraint States
When considering whether a generation hypothesis satisfies some clause  $C_i$  during generation, there are fundamentally 4 possible states (as in figure 2)
S1 reversible unsatisfaction: If an unsatisfied clause  $C_i$  contains at least one positive literal,  $C_i$  could be satisfied in the future by fulfilling one of its positive literal(s).
S2 irreversible unsatisfaction: If an unsatisfied clause  $C_i$  contains negative literal(s) only,  $C_i$  will maintain unsatisfied in the future since the violation of negative literals could not be overturned.
S3 reversible satisfaction: If all satisfied literal(s) in a satisfied clause  $C_i$  are negative literal(s),  $C_i$  could switch back to unsatisfied in the future by violating all of its satisfied negative literal(s).
S4 irreversible satisfaction: If satisfied literal(s) in a satisfied clause  $C_i$  contains at least one positive literal,  $C_i$  will maintain satisfied in the future since the fulfilment of positive literals is irreversible.
To track the states of literals and clauses efficiently, we maintain two prefix tries. The first trie,  $\mathcal{T}^{+}$ , tracks unsatisfied positive literals from all clauses in states S1 and S3, while the other trie,  $\mathcal{T}^{-}$ , tracks satisfied negative literals from all clauses in state S3. We do not track anything from clauses in state S2 or S4, as those are already irreversible.
If a positive literal is satisfied, its clause in state
S1 or S3 is henceforth irreversibly satisfied (state S4), thus we remove all literals of that clause from both tries and stop tracking. If a negative literal in state S3 is violated, we remove it from the trie  $\mathcal{T}^{-}$ . Once all negative literals of a clause in state S3 has been removed, the clause switches back to unsatisfied (state S1 or S2). If it has unsatisfied positive literal(s) in the trie  $\mathcal{T}^{+}$ , it becomes reversibly unsatisfied (state S1); otherwise it shall stay irreversibly unsatisfied (state S2).
2.3 Algorithm
Since exhaustive search to optimize the CNF constraints is intractable, NEUROLOGIC uses a beam-based search to approximate. The high-level intuition is that at each time step, NEUROLOGIC selects generation hypotheses in consideration of both the objective function and the diversity of the partially satisfied constraints. We achieve such by 3 steps: pruning, grouping, and selecting (illustrated in figure 3, and detailed below).
At each time step, the decoding model generates a distribution over all vocabulary  $V$  for  $k$  hypotheses in the current beam, resulting in a candidate score matrix of size  $k \times |V|$ . Along with generating score matrix, we produce a constraint state for each of the  $k \times |V|$  new candidates  $h$ , based on the next token considered.
Pruning step: We first discard any  $h$  with irreversible unsatisfied clause (state S2) to focus only on candidates that might satisfy all constraints. Then, we filter candidates  $h$  to those in the top-tier of both satisfied constraints and sequence likelihood. Specifically, we drop any candidates not in the top-  $\alpha$  in terms of likelihood  $P_{\theta}(\mathbf{y}_t|\mathbf{y}_{< t})$ , and not in the top- $\beta$  in terms of number of satisfied clauses  $\sum_{i=1}^{L} C_i$ . These are adjustable parameters, corresponding to maximum tolerance to sequence fluency and constraint satisfaction.
Grouping step: Next, we select the beam from the pruned candidates. Naively selecting  $k$  best candidates with respect to the objective function would not work well, since such greedy selection would bias toward sequences with high likelihood and easy-to-satisfy clauses at early timestep, which can lead to struggling with remaining hard-to-satisfy clauses later on. Therefore, the key intuition is to consider diverse partial solutions early on with respect to the set of irreversibly satisfied clauses, i.e.,  $\{C_i \mid C_i \in \text{state S4}\}$ . We group candidates based on this set and select (in the next step) the best ones
Figure 3: Illustration of the NEUROLOGIC decoding procedure. In this example,  $k = 3$ ,  $\alpha = 8$ ,  $\beta = 2$ ,  $\lambda = 0.1$
from each group to fill the beam.
Selecting step: To select best ones from each group, we first rank candidates within a group by score function:
$$
\mathbf{s} = P_{\theta}(\mathbf{y}_{t}|\mathbf{y}_{<  t}) + \lambda \cdot \max_{\substack{D(\mathbf{a}_{i},\mathbf{y})\\ \in \text{state S1}}}\frac{\left|\hat{\mathbf{a}}_{i}\right|}{\left|\mathbf{a}_{i}\right|} \tag{3}
$$
where  $\hat{\mathbf{a}}_{\mathbf{i}}$  is  $\mathbf{a}_{\mathbf{i}}$ ’s matched prefix with ongoing generation. For example, for  $\mathbf{y} =$  "The boy climbs an apple" and constraint  $\mathbf{a}_{\mathbf{i}} =$  "apple tree", we have  $\hat{\mathbf{a}}_{\mathbf{i}} =$  "apple". The second term denotes maximal percentage of matched prefix in partially satisfied positive literals. Intuitively, this score function ranks candidates by likelihood and gives a partial reward to candidates moving towards satisfying a positive literal in an unsatisfied clause (state S1).  $\lambda$  is an adjustable parameter, controlling how much we favor candidates towards fulfilling another unsatisfied clause. We then proceed in rounds of filling the beam, visiting each group and taking the best scoring ones in rotation, until we reach  $k$  candidates. The group traversing order follows the descending order of the highest score in each group. In the end, we take the hypothesis with highest likelihood from the ones with maximal satisfied clauses.
NEUROLOGIC distinguishes itself from past works in constrained decoding in 3 fundamental ways.
- First, NEUROLOGIC generalizes to arbitrary logical constraints by handling the full scope
of CNF constraint, while previous works only allow a subset of this (typically conjunctions).
- Second, NEUROLOGIC effectively optimizes objective function through efficient and diverse search over output space, while previous works suffer from either myopic and narrow or inefficient exploration of the search space.
- Third, the asymptotic runtime of NEUROLOGIC is  $O(Nk)^{1}$ , same with beam search, constant with respect to number of constraints  $\mathcal{C}$ . Some previous works suffer from exponential runtime, making applications infeasible.
A detailed comparison between NEUROLOGIC and previous methods is provided in table 1.
3.1 Previous Constrained Decoding Approach
Anderson et al. (2017) propose constrained beam search (CBS), where constraint satisfaction is tracked by a finite-state machine with  $2^{\mathcal{C}}$  states (all possible satisfaction status for  $\mathcal{C}$  constraints). Beam search is done over all states with  $k$  candidates per state. This method has an exponential complexity  $O(Nk2^{\mathcal{C}})$ , making many applications infeasible.
Hokamp and Liu (2017) propose grid beam search (GBS), which groups together hypotheses by number of constraints satisfied, giving  $\mathcal{C} + 1$
${}^{1}N$  denotes sequence length and  $k$  denotes beam size. In this paper,we the asymptotic runtimes is in terms of the number of calls to a deep generator that scores  ${P}_{\theta }\left( {{\mathbf{y}}_{t} \mid  {\mathbf{y}}_{ < t}}\right)$  ; this is because calling the generator is the most expensive part of decoding (as opposed to auxiliary bookkeeping).
Feature	Example	CBS	GBS	Post and Vilar	Hu et al.	CGMH	Sha	NEUROLOGIC
AND	oil ∧ pork
Include oil and pork	✓	✓	✓	✓	✓	✓	✓
Positive Set AND	oil ∧ (pork ∨ beef)
Include oil and a protein	✓	✓
Any Predicate Logic Formula	¬oil ∧ (pork ∨ beef)
Oil-free, include a protein	✓
Runtime:	O(Nk2C)	O(NkC)	O(Nk)	O(Nk)	O(E)	O(E)	O(Nk)
Table 1: Expressivity and runtime of various decoding methods. AND: Output includes all terms in a set; Positive Set AND: Output includes at least one term from each set; Predicate Logic Formula: Any combination of positive and negative constraints.  $E$  is the number of editing steps, usually much greater than the sequence length  $N$  .
groups altogether. Each group stores at most  $k$  candidates that are expanded at each timestep. GBS has a faster runtime of  $O(Nk\mathcal{C})$ , but this approach biases towards sequences satisfying constraints greedily, and collapses into very similar search paths that are often times globally sub-optimal, which results in dropped language quality.
Post and Vilar (2018) propose dynamic beam allocation to reduce GBS's explicit dependence on  $\mathcal{C}$ . Beam search is done over a single beam, with the  $k$  slots of this beam dynamically allocated over the  $\mathcal{C} + 1$  groups explicitly used by GBS. This approach was made GPU-efficient by Hu et al. (2019a). Still, the language quality issue of GBS remains, and can be worse in practice as fewer hypotheses are considered at each step.
Miao et al. (2019) propose Constrained Generation by Metropolis-Hastings Sampling (CGMH). This approach begins by inserting all positive-constraint keywords in random order. Edits are randomly sampled to replace, insert, or delete words to make the sentence fluent; the probability of each action is computed on top of a language model. Sha (2020) proposes using gradient of a objective function to guide where and how to edit instead of random sampling. These approaches have runtime independent to number of constraints; yet they can involve repeated deletions and insertions, reducing efficiency. Generation quality is also sensitive to initial keyword order and sampled edits.
3.2 Applications of Constrained Generation
Lexically constrained generation can be broadly applied to prior conditional text generation tasks. Examples include incorporating pre-specified lexical constraints (Anderson et al., 2017; Post and Vilar, 2018), user-provided terminology constraints (Hasler et al., 2018; Dinu et al., 2019), noisy automatic constraints (Li et al., 2019) in translation output. A major use case of lexical constrained de
coding is paraphrase generation (Hu et al., 2019a; Kajiwara, 2019; Hu et al., 2019b; Miao et al., 2019), by negatively constraining words in the source to enforce paraphrasing. Another use case is image captioning, with novel scenes or out-of-domain objects (Anderson et al., 2017), or requiring explicit grounding to objects in the scene (Ren et al., 2015; Krause et al., 2016). In addition, Balakrishnan et al. (2019) leverage constrained decoding to improve semantic correctness for response generation.
4 Experiments I: Constrained Commonsense Generation
COMMONGEN (Lin et al., 2020) is a benchmark dataset designed as a test of generative commonsense reasoning. Given a set of common concepts (e.g., dog, frisbee, catch, throw); the task is to generate a coherent sentence describing an everyday scenario using these concepts (e.g., "a man throws a frisbee and his dog catches it").
Problem Formulation The input is an unordered set of  $n$  concepts  $\mathbf{x} = \{a_1, a_2, \ldots, a_n\}$ , where each concept  $a_i$  is a common object (noun) or action (verb). The expected output is a simple, grammatical sentence  $\mathbf{y} \in \mathcal{V}$  that describes a common scenario using all given concepts in  $\mathbf{x}$  with correct morphological inflections.
To apply NEUROLOGIC DECODING, we impose that each  $a_{i}$  must appear in output  $\mathbf{y}$  under some morphological inflection. Let  $\tilde{a}_i = \{\tilde{a}_1^i,\dots \tilde{a}_{|\tilde{a}_i|}^i\}$  denote all inflections of  $a_{i}$ .  $\mathbf{y}$  covers concept  $a_{i}$ , if at least one of  $\{\tilde{a}_1^i,\ldots \tilde{a}_{|\tilde{a}_i|}^i\}$  appears. Formally,
$$
\forall a _ {i} \in \mathbf {x}, \exists \tilde {a} _ {j} ^ {i} \in \tilde {a} _ {i}, D (\tilde {a} _ {j} ^ {i}, \mathbf {y})
$$
where  $D(\tilde{a}_j^i, \mathbf{y})$  is a boolean-value function indicating whether  $\mathbf{y}$  contains  $\tilde{a}_j^i$  or not, as defined above.
2This gets converted into  $\wedge_{i = 1}^{n}\big(\vee_{j = 1}^{|a_i|}D(\tilde{a}_j^i,\mathbf{y})\big)$
Model	ROUGE - L	BLEU - 3 & 4	METEOR	CIDEr	SPICE	Coverage
GPT-2	40.3 → 42.8	34.2 → 36.7	24.7 → 26.7	27.6 → 30.2	13.4 → 14.7	27.1 → 30.3	82.2 → 97.7
BERT-Gen	42.4 → 43.8	37.5 → 38.9	27.0 → 28.2	29.5 → 30.9	14.9 → 15.5	29.8 → 31.4	89.2 → 97.3
UniLM	44.3 → 45.8	40.6 → 42.8	29.9 → 31.5	30.1 → 31.7	15.5 → 16.6	30.6 → 32.5	90.5 → 97.8
UniLM-v2	43.5 → 44.2	39.2 → 39.5	28.3 → 28.5	30.6 → 31.3	15.2 → 16.8	30.8 → 31.1	92.8 → 97.9
BART	43.3 → 44.7	39.9 → 41.3	29.1 → 30.6	30.4 → 31.0	15.2 → 15.9	30.6 → 31.0	95.0 → 98.7
T5	43.9 → 44.8	36.6 → 38.5	26.9 → 28.1	28.9 → 30.7	14.3 → 15.5	29.5 → 30.8	89.7 → 98.5
Table 2: Experimental results of different supervised models on the COMMONGEN test set. Under each column,  $\alpha \rightarrow \beta$  shows the performance using the conventional beam search  $(\alpha)$  compared to the enhanced performance using NEUROLOGIC DECODING  $(\beta)$ . NEUROLOGIC always improves the performance across all models and all metrics — with no exception. The best models are bold and second best ones are underlined within each metric.
Domain Adaption	Model	ROUGE - L	BLEU - 3 & 4	METEOR	CIDEr	SPICE	Coverage
No	GPT	26.7 → 41.3	3.0 → 25.1	1.1 → 15.9	9.2 → 28.8	0.9 → 11.7	8.0 → 29.7	8.4 → 97.4
GPT-2	19.7 → 42.9	4.1 → 34.4	1.5 → 23.5	11.2 → 30.7	0.4 → 13.6	7.1 → 31.4	8.3 → 96.0
Yes	GPT-2	29.8 → 42.4	9.5 → 36.1	4.0 → 25.1	11.7 → 31.3	1.7 → 13.9	8.0 → 31.8	9.3 → 96.1
Table 3: Experimental results in zero-shot (unsupervised) setting on the COMMONGEN test set with and without language domain adaption. Under each column,  $\alpha \rightarrow \beta$  shows the performance using the conventional beam search  $(\alpha)$  compared to the enhanced performance using NEUROLOGIC DECODING  $(\beta)$ .
Decode Method	ROUGE-L	BLEU-3/4	METEOR	CIDEr	SPICE	Coverage
Greedy Decoding	35.3	25.2	16.7	25.8	10.2	24.4
Top-k Sampling	33.8	22.5	14.4	24.9	9.2	22.7
Top-p Sampling	35.3	25.0	16.5	25.7	10.2	24.1
Beam Search	40.3	34.2	24.7	27.6	13.4	27.1
Hokamp and Liu	37.6	25.6	16.8	25.9	11.1	25.1
Post and Vilar	38.3	28.1	18.6	26.7	11.8	26.0
Hu et al.	38.2	27.8	18.4	26.7	11.7	26.1
NEUROLOGIC	42.8	36.7	26.7	30.2	14.7	30.3
Table 4: Performance of different decoding methods using supervised GPT2-L on the COMMONGEN test set.
Dataset The COMMONGEN dataset consists of 35,141 concept-sets (32,651 in train, 993 in val, 1,497 in test) associated with 77,449 sentences. The average size of the concept-sets in the test set is 4.04, with an average of four sentences per concept-set and an average sentence length of 13.34 words.
Approach and Baseline The standard pipeline of approaching this problem is to consider it as a conditional sentence generation task. We experiment with several recent pre-trained language models, including GPT-2 (Radford et al., 2019), UniLM (Dong et al., 2019), UniLM-v2 (Bao et al., 2020), BERT-Gen (Bao et al., 2020), BART (Lewis et al., 2020), and T5 (Raffel et al., 2019). All models are finetuned with their default hyperparameters. We compare with commonly used decoding methods, including beam search, sampling, and also previously proposed constrained decoding methods. We use several widely-used automatic metrics to automatically assess the performance, such as
BLEU, ROUGE, METEOR, which mainly focus on measuring surface similarities. We also include metrics specially designed for captioning task, such as CIDEr, and SPICE. Following Lin et al. (2020), we report the concept Coverage, which is the average percentage of input concepts that are present in lemmatized outputs.
4.1 Results I: NEUROLOGIC vs Other Decoding Methods
In Table 4, we first present comparisons across different decoding methods based on a supervised sequence-to-sequence model, GPT-2. The key observations are: