Neural Rule-Execution Tracking Machine For Transformer-Based Text Generation
Authors: Yufei Wang
Can Xu
Huang Hu
Chongyang Tao
Stephen Wan
Mark Dras
Mark Johnson
Daxin Jiang
Venue: NeurIPS 2021 Poster (2021)
Keywords: Sequence-to-Sequence Text Generation
Rule Execution Tracking
Pre-trained Transformer-based Language Models

Abstract: Sequence-to-Sequence (Seq2Seq) neural text generation models, especially the pre-trained ones (e.g., BART and T5), have exhibited compelling performance on various natural language generation tasks. However, the black-box nature of these models limits their application in tasks where specific rules (e.g., controllable constraints, prior knowledge) need to be executed. Previous works either design specific model structures (e.g., Copy Mechanism corresponding to the rule "the generated output should include certain words in the source input'') or implement specialized inference algorithms (e.g., Constrained Beam Search) to execute particular rules through the text generation. These methods require the careful design case-by-case and are difficult to support multiple rules concurrently. In this paper, we propose a novel module named Neural Rule-Execution Tracking Machine (NRETM) that can be equipped into various transformer-based generators to leverage multiple rules simultaneously to guide the neural generation model for superior generation performance in an unified and scalable way. Extensive experiments on several benchmarks verify the effectiveness of our proposed model in both controllable and general text generation tasks.

Neural Rule-Execution Tracking Machine For Transformer-Based Text Generation
Yufei Wang ; Can Xu Huang Hu 2, Chongyang Tao Stephen Wan 3, Mark Dras
Mark Johnson ' Daxin Jiang
Macquarie University, Sydney, Australia
Microsoft Corporation, Beijing, China?
CSIRO Data61, Sydney, Australia
yufei. wangedssnntttttttts..... mq. edu. au, (mark. dras, mark. joansonn edu. au {caxu, huahu, chongyang. tao, djjann] comiiirrooo...oom
stephen. wan@data61. csiro. au
Abstract
Sequence-to-Sqqueme (Seq2Seq) neural text generation models, especially the pre-trained ones (e.g.. BART and T5), have exhibited compelling performance on various natural language generation tasks. However, the black-box nature these models limits their application in tasks where specific rules (e.g.. control- lable constraints, prior knowledge) need to executed. Previous works either design specific model structures (c.g., Copy Mechanism corresponding to the rule "the generated output should include certain words in the source input") or imple- ment specialized inference algorithms (c.g., Constrained Beam Search) to execute particular rules through the text generation. These methods require the careful design case-by-case and are difficult to support multiple rules concurrently. In this paper, we propose novel module named Neural Rule-Execution Tracking Machine (NRETM) that be equipped into various transformer-bane generators to leverage multiple rules simultaneously to guide the neural generation model for superior generation performance in unified and scalable way. Extensive experiments on several benchmarks verify the effectiveness of our proposed model in both controllable and general text generation tasks.
Introduction
Transformer-baan neural language models (LMs), such as GPT/BART (1H3), have led wave of new trends in natural language generation, producing texts of prominent quality. They are trained roughly on huge amounts of text corpora reconstruct the full sentences (i.e., next coming tokens and missing text fragments). Despite their success in varieties of NLP tasks, we argue that the black-box nature of these models leads inefficiently learning follow constraints and incorporating prior knowledge.
In controllable text generation, most relevant studies 446] focus on controlling high-level text attributes (e.g., topic, sentiment) simply keyword/phrase. More complex fine-grained control constraints such as "generate a sequence of tokens with 'apple' in the first sentence which has 15 words and 'orange' or "oranges' in the fourth sentence" are less explored. A very recent work [7] reveals that large-scale LMs do not learn to obey the underlying constraints reliably, even in a quite simple constrained generation task (cover all the given keywords without hallucinating new ones). In general text generation, existing works on various tasks reveal the benefit of incorporating task-specific prior knowledge: machine translation [ (c.g., each source phrase should be translated
"Work done during the internship at Microsoft STCA. t Corresponding author: Daxin Jiang (djiang@micret) miicrosofl.comm)
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

into exactly one target phrase), text summarization 9 (e.g., the lead bias: front loading the most salient information), dialogue generation [T0] (e.g., humans tend to repeat entity names or even long phrases in conversation). However, they either need designing specific model architectures (e.g., Coverage Mechanism and Copy Mechanism) or devising well-designed learning objectives (e.g.. GSG I... These methods require careful design case-by-case and are difficult to combine multiple arbitrary constraints or prior knowledge simultaneously.
Motivated by the above research dilemma, we take first step towards building an unified framework handle Fine-gi fine-grained Control and Prior Knowledge Integration and propose novel module Neural Exee-Execttion Tracking Machine (NRETM) [ Speeciiically, NRETM is trainable neural module that can be equipped with transformer- based sequence-to -e-euuenee pre-trained LMs. can handle constraints in any Predicate Logic Formula, which crucially includes the arbitrarily complicated relations among different control tasks. For example, the above fine-grained constraint can be written as:
(InSen( (apple, 1) A Len(1, 15)) [InSen(orange, 4) V InSen(oranges,, 4))
To build NRETM, we combat three major challenges: modeling the complicated relationships among control tasks and the logic operators (i.e., A, V) in the constraint expressions: ii) an unified control system is required to execute different control tasks simultaneously and iii), the control signals for different control tasks should be properly aligned with the constraint expressions. NRETM uses the encoder of cunsformer-basee pre-trained between control tasks and the logic operators. NRETM completes different control tasks via non- differential Logic Trackers (empowered by executable programs) in an unified control progress system during the decoding process. Finally, the encoded constraint expressions and control progress signals are combined together the transformer decoder. NRETM is fine-tuned with the pre-trained LMs (except logical trackers) to follow the control progress signal and predicate logic formula. NRETM reconciles symbolic computing (that has precise logic and numerical calculation capabilities from logic trackers) with neural language generation (that has an exceptional ability of wording and phrasing), which results in both the accurate controllability and the superior generation performance.
LMs to model the relationship
For evaluation, we select three representative benchmarks because all of them involve constraints or prior knowledge, allowing us to verify the effectiivness of our proposed NRETH model: ROCSto- ries 12 are five-sentence stories with complicated predicate constraints over the story structure; Commonsense Generation task [33 with the constraints of mentioning all input concepts; TED15 Zh-En document-level machine translation benchmark [14] with prior knowledge of translating input sentences one by one.
Our contributions in this work are three-fold: (1) To the best of our knowledge, we are the first to propose a general framework that incorporates control signal and prior knowledge, formulated as predicate logic constraints, into transformer based seq2seq text generation models; (2) We train (or fine-tune) the transformer-based seq2seq text generation models to follow the predicate logic constraints(i.e.. control signal or prior knowledge) by dynamically updating the rule execution intermediate progress value to the text decoder; and (3) Empirical verification of the effectiveness of the proposed approach on three benchmarks.
2 Approach
This section first formalizes firmalzzesfine-aaee ffne--rainee content control task, then introduces an overview of proposed NRETM model, followed by diving into details of each component.
2.1 Fine-Grained Content Control
In this work, we focus on fine grained content control task where the model input consists of predicate logic constraints = [x1,,... ...,,...... ,, input = [cI,..., ]. The encoder takes concatenation of x and e (i.e., [c; x)) as input. At decoding step the decoder take y., [y,,... yt] y' as input and generate y1+1-
X that should be satisfied in the outputs and optional context
Our Source Code be found in https: httts:://gitaeii http:::://gtttub.ooo.o

---

11+1
Transformer Decoder
Transformer Encoder
h
hij
Control Progress
Matrix
bf
hij
Encoder
K
[c; x]
Y:* [y, V2, ,yc]
Model Input с &&
Control Progress Matrix
Logic Tracker
Figure An overview of NRETM. The rounded-angle boxes in the upper row are trainable neural components and the right-angle boxes in the lower The predicate logic constraints are modeled as follows: 1) the transformer encoder handles the relationships among the predicates and basic logic operators; 2) Logic Tracker keeps track of the control progress of all predicates simultaneously: 3) the encoded expression and control progress are combined in the transformer decoder to guide NRETM satisfy the constraints.
non-differentiaabe symbolic components.
2.2 Predicate Logic Constraint
We define predicate (a, y) boolean function that indicates whether output y has satisfied control task a which could be values (e.g., status, total length, stop word counts) or lexicons (e.g., copying particular words/phrases). In this paper, NRETH accepts predicate logic constraints in Conjunctive Normal Form (CNF): (U1... v U.)) A ...A A (Ux ... Un). Each predicate logic constraint includes multiple predicates U, and basic logic operators (e.g.. V, A and brackets).
2.3 Neural Rule-Execution Tracking Machine
NRETM can be equipped into transformer-based sequence-to-sequeee LMs. Figure T illlstrates an overview of our neural rule-execution tracking machine (NRETM), To enable LMs to follow predicate logic constraints, it is essential to 1) model the complicated relationships among predicates and basic logic operators; 2) control multiple predicates (i.c., control tasks) in the constraints simultaneously; 3) combine the control signals with the predicate logic constraint expressions. For 1), we treat the whole constraint expressions as natural language sentences and feed into the transformer encoder. For 2), propose of unified control signals that can be used to dynamically describe the step-wise execution progress of different predicates. For 3), we represent the control signals as relative position embedding and align them with encoded constraints expressions in the transformer decoder.
2.3.1 Encoding Predicate Logic Constraints
Given predicate logic constraint expression predicate U, or a basic logic operator, we feed x into transformer encoder. Due to the tokenization strategies of pre-trained LMs, each may be tokenized into continuous token sequence. x is tokenized into = . ... tú,] where 4 2 Le and there exists one-to-one mapping m(t,) = zy. We use he to denote the encoder output of x. As pre-trained LMs is trained with significant amount of natural language sentences, it should encode complicated sequential relationships within the constraints expressions.
[II,..., Ii_] where æ; either corresponds
2.3.2 Mentoring Control Progress
Specialized controlling components (e.g., Constrained Beam Search [15] and Copy Mechanism (10) can only used for limited control tasks. enable unified controlling system, we propose to complete control by mentoring control progress. We describe the control progress of different predicates using an unified progress state system. Fach predicate U, has corresponding Logic Tracker Qu, (y), which is ci-differentiable executable program (i.c., written by Python) and takes

---

Control Progress Matrix
raining
The
woman
cannot
drive
her
EOS
Copylcar)
52
52
52
52
52
52
52
52
50
50
90
50
90
50
Lengthi2,
51 51 51 51
$2,0
$2,5
$2,4
52,3
$2,
52,
$9
Figure 2: A running example of our NRETM model with three logic constraints (i.e., Copy(car) SWC(5) & Length(2,9); The output should include "car" and 5 stop words. The length of second sentence should be The words are stop words.
current generated outputs and returns one progress state at each generation step, formulated as follows:
SO S1
U,
U: is not triggered in y
Qv) =
(1)
(S2.V) V)
U; progress in
S3
U; is satisfied in y
where State SO always is assigned to non-predicate ø (i.e., basic logic operators in the constraint expression): State SI means the tracking for predicate U; is not triggered in y. For example, when controlling the stop word counts of the second sentence, the Logic Tracker returns S1 when the LMs are generating the first sentence: State S2 means predicate U3 in progress and V the optimal intermediate value that allows fine-grained tracking. For example, in generation length control, V could be total target length minus the current length informing pre-trained LMs the number of words left to satisfy the constraint; State S3 means U; is satisfied in In short, Logic Tracker unifies different predicates by returning the same set of control signals.
Global Or-Clause Update: Each Logic Tracker traces the execution progress of its corresponding predicate U, independently. This independent tracing strategy works well in the And-Clause because all involved predicates are required to reach State S3. However, only a subset of predicates are required reach State $3 in the Or-Clause. Our preliminary experiment shows that the independent tracing strategy trains the model not to complete the constraints. To solve this issue, we propose update the status of all predicates in the same Or-Clause State S3 when one of the predicates reach State S3. This forces all predicates finish themselves in State S3 and improves the constraint satisfaction ratio in the Or-Clause.
Control Progress Matrix:
Given the predicate logic constraint expressions = [t ... 4, we further define Control Progress Matrix S to align the predicates with their control progress signals returned by Logic Trackers:
S= (C(t,, ((t ;(t,),
C(t, = [ [v(4, (v4,,,
(3)
where E is the empty string at first decoding step. S is two-dimensional matrix where each row describes the control progress of all tokens in t at single decoding step and each column describes the control progress of a single token in along all decoding steps. Recall that basic logic operators in predicate logic constraint expressions do not require control progress tracking. Each cell S in S formulated as:
Qy(y) Qu, (y)
m(t,) = 2y and I'k a basic logic operator m(t,) 22, and T'k. predicate Ug
S. = v(1..yy-1)
(4)
Example: In Figure we are given three logic constraints, a) copy "car": b) the stop word ratio of the output should be 0.5 and c) the length of second sentence should be 6. The basic logic operators & are assigned with So. Length control and Stop Word Ratio maintain intermediate values (e.g., the residual Length and Stop Word Ratio). The length control assigned with when generating first sentence because it will only be triggered in the second sentence. Copy control does not have intermediate values and its State are updated from $2 S3 only when the corresponding words (at step 10 in our example) appear in the y

---

Control Progress Matrix Encoder: Control Progress Matrix S aligns the results from Logic Tracker with the encoded predicate logic constraint expressions. However, S is a non-differentiable symbolic matrix with each cell St,j being discrete symbol SO to S3 combined with additional numbers (i.e., V). As the encoder has already captured the inter-relationship in the predicate logic constraints, we only model each cell J independently. To support various types of predicates, we treat S as string and encode it using a single-layer transformer-basee encoder ShtalloEEncoder which shares the same vocabulary and word embeddings as the pre-trained LMs:
h ShallowEncoodr((S.,) h McanPoolingghh;))
(5)
(6)
where hi RR hij E Rd and iúg is the length of the tokenized St and d is the hidden size of EheodewEn We use h denote the neural representation of whole S.
2.3.3 Combining Predicate Logic Constraint with Control Progress Matrix
Finally, we combine the encoded Predicate Logic Constraints h with the encoded Control Progress Matrix h² in the transformer-bant pre-trained LMs. Injecting h into the transformer encoder would result in encoder content re-comuutation at each decoding step and stop the standard parallel training for transformer-basee -based decoders. In addition, as Control Progress Matrix incrementally increases as the decoding goes on, it is reasonable to equip h" into the transformer decoder. Given the encoder output h", decoder input y:,, the probability of the next token Ut+1 can be calculated by:
h = ( W.y.. W.y.
(7)
O++1 CrossK((WS)) T Wih,, Wih²)
(8)
p(y++ 1211,, v1:a) softmax( W Oc+1)
where 0,+1 E Rd is the hidden state at step with de the hidden size, and W. E Rixd, Both KV and CrossKV are the standard key-value self-attention described in [66]. In the CrossKV which takes he and he as input, the resulting attention score matrix has the same size as S, making CrossKV suitable to incorporate our Control Progress Matrix.
Inspired by [17 which incorporates token relative positions into the self-attention module, we propose to inject Control Progress Matrix as the "relative positions" between encoder output h and current decoder input y. in the cross- atossaattntiion (Eq, 8) module. Following this approach, linearly project each h. into Control Progress Matrix key
Control Progress Matrix as Relative Position:
h = w k h + by and Control Progress Matrix Value hü = W h + by. All transformer X is changed to:
decoder layers share the same representations. Eq.
0++1 = R H(W,H, W.H", W. h²)
(10)
where 1R xtxd and R the Self-Attention function with relative position, defined follows:
R(q. k, v, m² m²);
a + +,
(11)
1=1
where a.,j Softmax((..) and c. = q, (k; + m Td-122
2.4 Why NRETM Could Satisfy Constraints
powerful implicit compulsion comes from the combined force of two aspects: 1) before generating the EOS token (i.e., End-Of-Sequence Token), all the predicate constraints should be satisfied. As demonstrated in Fig [ all elements in Control Progress Matrix are set to "satisfied" (i.e., S3) at EOS position; 2) The pre-trained LMs are trained to generate text with limited length. Such soft way of combining symbolic operators (good at logical and mathematical calculations ) and neural operators (good at wording and phrasing) can retain their respective strengths to the utmost extent.
2.5 What If NRETH Fails to Satisly Constraints
NRETM does not forces the pre-trained LMs to execute the hard constraints on the text decoder explicitly, but instead, provides Control Progress Matrix input features describing rule execution
5

---

intermediate values to the text decoder. That is, no explicit effect when NRETH fails to satisfy the constraints. It is possible that our text generators decide to stop the generation before completing all constraints. In our experiments, NRETM has less than 1 chance not to complete all constraints.
2.6 The Generalization Ability of NRETH
The generalization ability of NRETM comes from two aspects: 1) NRETM can construct new constraints via combining pre pre-trained predicates with basic logic operators in arbitrarily complicated ways; 2) To expand a new predicate, users only need to implement the corresponding Logic Trackers, which returns S1-S3 and intermediate values, via executable programs.
Experiment
We test proposed NRETM the controllable text generation and general text generation tasks. For controllable text generation, we verify NRETM on the complex finennnained control instructions in the ROCStories Benchmark [12]. Further, test NRETH the general text generation tasks, commonsense generation and document-level machine translation, to show that NRETM can efficiently integrate prior knowledge into seq2seq models towards superior generation performance.
Table 1: Controllable ROCStories Experiment Results.
Predicate Logic Constraint
M
CSR
RL
BS
B1
B2
T5
94.6
56.1
91.7
52.5
27.7
M (InSen(w,, )))
NRETM
97.6
56.0
91.7
52.1
27.5
T5
95.6
55.5
91.5
51.4
26.5
Copy(w1) Aea (Order(w;, w)+1))
NRETM
98.3
55.6
91.4
47.5
25.0
T5
15.0
33.0
87.8
38.6
11.5
1 (InSen(w;, wi, ) A Len(y""
NRETM
78.8
33.1
87.8
38.4
11.5
InSen(w1, InSen A A (w0,, A(Len(y") SWC(y"',
T5
32.4
33.2
87.8
36.1
11.8
NRETM
70.0
32.7
87.7
36.9
11.7
T5
18.7
33.2
87.9
37.6
11.5
(InSen(ws, (Len)))))) SWC()" sp.)))
NRETM
64.7
33.2
87.9
38.5
11.7
3.1 Controllable ROC Stories
ROCStories is corpus of five-sentence stories that capture rich set of causal and temporal commonsense relations between daily events. Following [88, we extract key phrases from the ground-truth stories. In this experiment, we design multiple predicate logic constraints to inform NRETM about the stories to generated and verify if NRETM can follow these constraints exactly.
Predicate Logic Formulation As shown in table I five constraints with increasing difficulties are used: (I) Generate story with storyline w; in the P. th sentence. (2) Generate a story with ordered storyline W1,-
(3) Generate a story with storyline (i = 2). (4) Generate storyline and w2 in the th sentence that does not mention w3 (5) Generate a storyline w, in the p; which has 4, words Sp. stop words (i 1,
in the p:" sentence which has 1p. words in the pı th sentence which has (p, words or sp, stop words
th sentence
Baselines and Metrics Both baseline and NRETM use T5-Base model 19 We report Constraints Success Ratio (CSR), the ratio of stories that completely satisfy the given constraints. We additionally report ROUGE-L (RL), BERT-Score (BS), BLEU-1/4 (B 1/4) to show the generated stories quality.
Main Results As shown in Table in all five predicate logic constraints, compared to the T5 model, the NRETH model achieves higher Constraint Success Ratio and maintains similar level of ROUGH- L. showing that the NRETM model can be flexibly controlled without loss of generated text quality.

---

Table 2: Experiment Results Commonsense.
Constraint
Method
BS
B1
B4
C
Seen NovelALL
T5-Base
29.2
159.4
92.7
TS-Base NRETM P.
94.6
72.5
30.3
163.8
32.4
94.6
93.6 94.6
T5-Base NRETH P.
94.5
74.2
29.3
167.7
33.2
99.4
99.6
99.5
Ta-Large
94.8
73.0
32.4
170.3
33.1
94.8
92.4 94.6
T5- T5-Large + NRETH Po
94.8
74.3
32.1
173.4
33.5
97.8
96.9
97.8
TS-Large NRETH P.
94.8
74.8
32.6
175.3
34.3
99.2
99.0
99.2
T5-Base G
92.8
58.6
40.2
110.7
27.8
100
100 100
T5-Large NEUROLOGIC 94.8
73.2
32.3
169.7
32.3
99.1
98.8 99.0
KGBART
168.3 32.7
98.6
The gap CSR between the T5 and NRETH model is moderate in the first two constraints with simple token permutations, However, the success ratio of 'S model drops significantly given constraints that requires long-range numerical tracking (e.g., sentence length and the count of stop words).
3.2 Commonsense Generation
COMMONGEN is generation benchmark dataset target explicitly machines for the ability of generative commonsense reasoning. Given a set common concepts the task is to generate a coherent sentence describing an everyday scenario using these concepts.
Predicate Logic Formulation The input is an unordered set of n concepts x = [= From the expectation of COMMONGEN, one easily obtained prior knowledge is that each æ; must appear in output y. The corresponding predicate logic constraint Pa is:
Pe Aー (Copy((r))
where will appear by default, for the sake of brevity, we have omitted y in predicate Copy. Another prior knowledge comes from the observation that generating y requires giving the correct morphological inflections of the concept word rather than copy its original form. Let ã; = { |2:| denote inflections of x,, covers concept F,, if at least one { P. is:
[āal appears. The constraint
P = A VJSi (o))
Baselines and Metrics We experiment with T5-Base and T5-Large. We equip NRETM into the T5- Large and T5-Base model incorporate Pa and Pe respectively (+ NRETM Pa) NRETH P.). Grid Beam Search (GBS) (20) (+ G) is well-designed decoding method that ensures the generation model satisfies the lexical constraints. We only apply GBS the T5-Base model due to the memory constraint. Following the suggestions in [13], we use CIDEr [2]] and SPICE [22] to automatically assess the quality of generated texts. We calculate constraint satisfaction for all constraints (ALL)), novel constraints (Novel) and seen constraints (Seen).
shows that the NRETH model improves the constraint satisfaction the baselines for all cases, achieving close to 100% (i.e., 99.5% and 99.2%) While GBS achieves perfect constraint satisfaction (i.e., 100%), doing so significantly degrades the output text quality (more than 50 CIDEr), indicating the necessity integrating prior knowledge in training rather than inference. In addition, both prior knowledge P. and P. have positive effect on our model, improving our T5-large baseline by 3.1 and 5.0 CIDEr score, respectively. Finally, our T5-Large + NRETH P. model outperforms the previous state-of-the-an result (23], which integrates the ConcepiNel (24] into BART model, suggesting that our incorporated task-specific prior knowledge could be powerful as knowledge from large-scale hand-crafted corpus. All of the above shows how potential is to find method that could execute multiple rules effectively.
Main Results Table shows

---

3.3 Document-Level Machine Translation
Document-level machine translation tasks is a general text generation task, where the goal is to translate segments of text (up to an entire document). Following [14), we use TEDI5 Zh-En (from IWSLT 2014 and 2015 [2 (26) as training and validation set and 2010-2013 TED as the test set.
Predicate Logic Formulation The input is an ordered set of n sentences in the source language that form document x )2= the expected output is translated document y {y')²= } in the target language. We observed that neural model is prone to sentence correspondence confusion (the 7t sentence source document is translated as the jth sentence target document) when doing document-level translation. To alleviate this problem, we propose incorporating Doc-mBART25 with prior knowledge: each source sentence should be translated only once. It is formulated as:
S3
(y::) >
TranslatedOnce(2"))
S2
0(ya) =
(12)
S1
0(ya) <
where (-)) returns the line number of y/1 in y, t monotonic during generation, the status only set to be 2 once. To trace the sentence translation progress, we add an additional Sand-Of-Sentene token at the end of each sentence to the training data. Once NRETM finishes the ht sentence (generating an and-of-sentence token) in the decoder, we assume that the 7th sentence in the encoder has been translated. The predicate logic constraint Pa of this task can be formulated as:
P. A (TranslatedOnce(zz))
Baselines and Metrics We combine our NRETM Pa component with the Doc-mBART25 model proposed in 3 which is state-of-the-ard multilingual pre-trained language model. We compare this model with the state-of-the-arr non-pretraining and pretraining approaches, including HAN (Hierarchical Attention Networks) [14], Doc-mBART25 and Sen-mBART25 proposed in [3]. When implementing our model, we use the same pre-processing method, blocks segmentation strategy and beam search setting R TED15 Zh-En provides sentence-to-sentence translation from Chinese to English. We use both document-level (d-BLEU) and sentence-level (s-BLEU) to measure the similarities between generated target document and the source document. We also report Sentence Aligned Ratio (SAR), the ratio of source and target documents with the same sentence count, to show the effectiveness of our control over this translation prior knowledge.
Main Results Table shows that the NRETM Pe component helps the Doc-mBART25 model to better capture the sentence-level corresponding relationship between the source and target documents. In particular, sentence-level alignment ratio is improved from 98.7% to 100%. The improvement in s-BLEU (+ 1.1 BLEU) also confirms that our final Doc-mBART25 + NRETH Pa model learns to translate sentences based the sentence order in source documents.
Table 3: Model Performance on TED15 Zh-En Test.
Model
B-BLEU
d-BLEU
SAR
Doc-mBART25 NRETH P.
24.9
30.4
100
Doc-mBART25 3
23.8
29.6
98.7 %
Sen-mBART22
28.4
HAN 14
24.0
3.4 Discussion
Updating Progress in Encoder In Sec 2.3.3, we incorporate the Control Progress Matrix as relative position embeddings in the decoder. To show the importance of this design choice, we conduct an ablation study in Table 4 where the row of Control Progress Matrix is concatenated with the encoder output. We find that updating the rule execution progress information with the encoder output contributes little to improve the CSR. This shows that simply extracting rule execution intermediate values is not enough. This could be because the encoder that encodes the rule execution intermediate values cannot effectively broadcast this information into text decoders.
NRETM Robustness The above experiment results are based on the perfect training data. In this section, we explore the effect of training data noise. We corrupt the training data by replacing the input commonsense keywords with a random sampled one under the probability 5%, 10%, 15%, 25%, and 50% (Validation and Test Split remain unchanged). As shown in Table 5 in all noise levels,
8

---

Table 4: Updating Rule Execution Progress with Encoder Output
Constraint
(1)
(2)
(3)
(4)
(5)
Model
RL
CSR
RL
CSR
RL
CSR
RL
CSR
RL
CSR
T5
56.1
94.6
55.5
95.6
33.0
15.0
33.2
32.4
33.2
18.7
NRETM
56.0
97.6
55.6
98.3
33.1
78.8
32.7
70.0
33.2
64.7
enc-update
56.0
96.8
55.6
97.2
33.0
14.2
32.7
28.5
33.3
16.8
NRETM successfully achieves higher constraint coverage (i.e, Cons) and CIDEr score than the TS baseline model, showing that NRETM robust to the training data noise. It is worthwhile to note that the main goal of NRETM is to incorporate constraints that satisfied by the training data into transformer-basee seq2seq text generators. It is reasonable to assume that in practice, the noise level should be relatively low (e.g., 0% 10%).
Table 5: NRETM performance Commonsense Test Split under different noise levels.
Noise
0%
5%
10%
15%
25%
50%
Model
C
Cons
C
Cons
C
Cons
C
Cons
C
Cons
C
Cons
170.3
94.6
168.8
93.3
168.0
93.7
163.8
160.7
91.3
102.0
66.7
NRETM
175.3
99.2
169.5
96.5
168.5
96.2
167.1
94.6
161.2
92.4
149.7
87.4
Zero-Shot Execution In Table [] we show that the pre-trained language model T5 cannot handle complicated and fine-grained constraints even after fine-tuning, Here, we further demonstrate that NRETH model is capable to handle zero-shot rule execution. We train the T5 and NRETM model only mention keywords in the 3rd 4th and 5thh sentence and test these models to mention keywords the first and second sentence of the whole story. As shown in Table 6 although both T5 and NRETM model mention most of the keywords (95.7% and 98.3% respectively) in the generated story, the T5 model only mention 19.7% of keywords in the correct sentence and the NRETM model makes 97.7% of keywords correct. This becuase the T5 model cannot recognize the novel sentence index (i.e., the first and second) during the generation. The logic tracker helps the NRETM model to generalize to handle these cases.
Table 6: Novel Sentence Index Ex- periment. MR for Mention Ratio.
model
CSA
MR
RL
T5
19.7
95.7
30.0
NRETM
97.7
98.3
33.3
Running Efficiency We compare the inference time (in min- utes) for NRETH on the test split of commonsense generation task in Table 7 All models use the beam search decoding algorithm with beam size 5. Adding NRETM components to Base and T5-Large approximately double the inference time. While the Grid Beam Search (GBS) algorithm uses much longer inference time. Compared to existing constrained decoding approaches, NRETM uses much less computational costs.
Table 7: Inference Time Com- monsense Generation Task Test Split (in minutes).
Model
Baseline
+NRETH
+GBS
T5-Base
1.05
2.27
84
T5-Large
1.32
2.6
Related Work
NRETM is mainly related to two lines of research work in text generation: constrained decoding and prior knowledge integration.
Constrained Decoding NEUROLOGICEary work in constrained decoding can be traced back to
dual decomposition and lagrangian relaxation 227 28 These works focus on sequence labelling and parsing problems where the solution space is relatively small, compared to text generation tasks. Research efforts in text generation tasks k1 involve controllable generation methods where the generators are trained on text data with the labeled target attributes. CTRL. 4. PPLM 6 and CoCon (32] are recent approaches that built the transformer-bar based large-scale pretrained LMs,

---

they pay more attention on controlling high-level attributes, phrases and keywords. [33 34 propose to trace the control task progress in the text generation decoder. [33] treats the control signal as
training loss in memory network and [34] treats the control signal as additional input features. 3S controls the text generation outputs via mentoring the output gradient. However, these work only focus on specific controlling tasks such as phrases copying and generation length. While NRETM focuses on controlling text generation to follow arbitrary logical constraints, leading to a fine-grained control. They can be seen as special cases of NRETM. Recently, GDC [36] permits to specify both pointwise and distributional constraints over the target LMs. Very recently, NEUROLOGIC [7] was proposed to generate fluent text while satisfying complex lexical constraints (in predicate logic form). There are three main differences between NRETM and NEUROLOGIC: 1) NEUROLOGIC only provides control constraints over the text generators. Instead, NRETM is general framework that provides control constraints (e.g., copy or not copy words) and prior knowledge (e.g.. translating sentences one by one). NEUROLOGII can be viewed as a special case of NRETM; 2) NEUROLOGIE is an inference-only algorithm that only controls the model to generate or avoid specific words or phrases decoding time; while NRETH fine- the pre-trained transformer-basee seq2seq text generators with the predicate logic constraints; 3) NEUROLOGIC only supports the "copy" predicate (i.e., to generate not to generate specific words or phrases), while NRETH is general framework that supports various control predicates. NRETM supports 6 kinds of logic operators in this paper, and it is also possible for users to expand new logic operators.
incorporate prior knowledge into sequence-fo-sequence framework either resort to modifying model architectures, including adding external memory components, specialized decoding method or designing training objectives, includ- ing minimum risk training. These methods usually can only support to inject one narrow type of knowledge into the neural models. To the best of our knowledge, we first attempt to formalize the prior knowledge integration in seq2seq generation as text generation that conforms to predicate logic constraints.
Prior Knowledge Integration Existing efforts 37:41
5 Conclusion and Future Work
In this paper, we propose a unified controllable generation framework that leverages predicate logic constraints to implement efficient complex fine-grained control and scalable prior knowledge integration. We explore and compare two controllable strategies: dynamic tracking and static strategy, and show that the proposed dynamic tracking mechanism significantly outperforms the static ones. Empirical results on three benchmarks indicate that NRETM could achieve accurate control and exhibits a superior generation ability over different tasks. Pre-trained models have been the dominant paradigm in natural language processing, and researchers resort to massive data and large-scale models to improve performance. We unify the rules used in various tasks into the form of predicate logic, provide the possibility to pretrain models massive rules. In the future, we will explore pre-training large-scale neural rule-execution machine with massive rules and data.
Broader Impact
Our work proposes a unified and scalable approach to efficiently perform fine-grained controllable text generation and incorporate multiple prior knowledge for superior text generation performance. This work uses story generation, machine translation, commonsense generation as applications to verify the effecttiveness. However, while our proposed method achieves promise performance on several benchmarks, deployment of our method in the real world requires a careful analysis of potential societal benefits and harms (e.g., the harms associated with furthering negative stereotypes against certain vulnerable groups). The potential ethical issues include: powerful language models might be used to generate abuse, faked or misleading content in the news or on social media; they might pose safety concerns if they are used to generate harassing or hateful materials. In order to mitigate these risks, it is possible to use Al systems to fight against misleading content and harassing material. However, as discussed previous work 42 4 mitigating these risks could be an extremely complex socio-technical problem that many are working to understand and solve.
10

---

Acknowledgement
We thank anonymous reviewers for their insightful suggestions improve this paper.
Funding Transparency Statement
Yufei Wang, Can Xu, Huang Hu, Chongyang Tao and Daxin Jiang are supported by Microsoft Software Technology Center Asia (STCA). Yufei Wang also receives a MQ Research Excellence Scholarship and a CSIRO's s DATA61 Top-up Scholarship.
References
[1] Alec Radford, Jeffeey Wu, Rewon Child, David Luan, Dario Amodei, and llya Sutskever. Language models are unsupervised multitask learners. OpenAl blog, 1(8)99, 2019.
[2] Tom B Brown, Benjamin Mann, Nick Ryder, Melanic Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, al. Language models are few-shot learners. arXiv preprint arXiv:2005. 14165, 2020.
[3] Mike Lewis. Yinhan Liu, Naman Goyal, Marjan Ghazvininejad. Abdelrahman Mohamed, Omer Levy. Veselin Stoyanov, and Luke Zettlemoyer. BART: Denoising sequence-to cocenncctto-scqueee so-ueueeeee pre-training for natural language generation, translation, and comprehension. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7871-7880 Online, July 2020. Association for Computational Linguistics.
Nitish Shirish Keskar, Bryan McCann, Lav R Varshney, Caiming Xiong, and Richard Socher. Ctrl: conditional transformer language model for controllable generation. arXiv preprint arXiv: 1909:0999000 2019.
[5] Daniel M Ziegler, Nisan Stiennon, Jeffrey Wu, Tom B Brown, Alec Radford, Dario Amodei, Paul Christiano, and Geoffrey Irving. Fine-tuning language models from human preferences, arXiv preprint arXiv: wwt./99999099333 2019.
[6] Sumanth Dathathri, Andrea Madotto, Janice Lan, Jane Hung, Eric Frank. Piero Molino, Jason Yosinski, and Rosanne Liu. Plug and play language models: A simple approach to controlled text generation. arXiv preprint 1912::99122022666 2019.
[7] Ximing Lu, Peter West, Rowan Zellers, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. Neuro- logic decoding:(un) supervised neural text generation with predicate logic constraints. arXiv preprint arXiv:2010.1 12884, 2020.
[8] Zhaopeng Tu, Zhengdong Lu, Yang Liu, Xiaohua Liu, and Hang Li. Modeling coverage for neural machine translation. arXiv preprint arXiv:1601.0488 0,811, 2016.
Ziyi Yang, Chenguang Zhu, Rohert Gmyr, Michael Zeng, Xuedong Huang, and Eric Darve. Ted: pretrained unsupervised summarization model with theme modeling and denoising. arXiv preprint arXiv:200).00725, 2020.
[10] Jiatao Gu, Zhengdong Lu, Hang Li, and Victor O.K. Li. Incorporating copying mechanism in sequence- to-sequence learning. Proceedings of 54th Annual Meeting of the Association for Computational Linguistics ( (Volume 1: Long Papers), pages 1631-1640, Berlin, Germany, August 2016. Association for Computational Linguistics.
[11] Jingqing Zhang, Yao Zhao, Mohammad Saleh, and Peter Liu. Pegasus: Pre-training with extracted gap-sentences for abstractive summarization. In International Conference on Machine Learning, pages 11328-11339, PMLR, 2020.
[12] Nasrin Mostafazadeh, Nathanael Chambers, Xiaodong He, Devi Parikh, Dhruv Batra, Lucy Vanderwende. Pushmeet Kohli, and James Allen. corpus and cloze evaluation for deeper understanding commonsense stories. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 839-849, San Diego, California, June 2016. Association for Computational Linguisties.
[13] Bill Yuchen Lin, Wangchunshu Zhou, Ming Shen, Pei Zhou, Chandra Bhagavatula, Yejin Choi, Xiang Ren. CommonGen: A constrained text generation challenge for generative commonsense reasoning. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 1823- 1823-1840, Online, November 2020. Association for Computational Linguistics.
11

---

[14] Lesly Miculicich, Dhananjay Ram, Nikolaos Pappas, and James Henderson. Document-level neural machine translation with hierarchical attention networks. In Proceedings the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2947-2954, Brussels, Belgium, October- November 2018, Association for Computational Linguistics.
[15] Peter Anderson, Basura Fernando, Mark Johnson, and Stephen Gould. Guided open vocabulary image captioning with constrained beam search. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 936-945, Copenhagen, Denmark, September 2017. Association for Computational Linguisties.
[16] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, L ukasz Kaiser, and Illia Polosukhin. Attention is all you need. In I. Guyon, U. V. Luxburg. S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 30, pages 5998-6008. Curran Associates, Inc., 2017.
[17] Peter Shaw, Jakoh Uszkoreit, and Ashish Vaswani, Self-attention with relative position representations. In Proceedings of the 2018 Conference the North American Chapter of the Association for Computational Linguisticx: Human Language Technologies. Volume 2 (Short Pupers), pages 464446466 (66,,66, New Orleans, Louisiana, June 2018. Association for Computational Linguistics.
[18] Nanyun Peng, Marjan Ghazvininejad, Jonathan May, and Kevin Knight. Towards controllable story generation. In Proceedings of the First Workshop Storytelling, pages 43-49, New Orleans, Louisiana, June 2018. Association for Computational Linguistics.
[19] Colin Raffel, Noam Shazeer. Adam Roberts. Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter Liu. Exploring the limits of transfer learning with unified text-to-text transformer. arXiv preprint 1rXiv:1910.1060 2019.
[20] Chris Hokamp and Qun Liu. Lexically constrained decoding for sequence generation using grid beam search. arXiv preprint arXiv: 2017.
[21] Ramakrishna Vedantam, C Lawrence Zitnick, and Devi Parikh. Cider: Consensus based image description evaluation. In Proceedings of the IEEE conference on computer vision and pallern recognition, pages 4566-4575, 2015.
[22] Peter Anderson, Basura Fernando, Mark Johnson, and Stephen Gould. Spice: Semantic propositional image caption evaluation. In European conference on computer vision, pages 382-398. Springer. 2016.
[23] Ye Liu, Yao Wan. Lifang He, Hao Peng. and Philip S. Yu. Kg-bart: Knowledge graph-augmented bart for generative commonsense reasoning. In Proceedingy of the AAAI Conference on Artificial Intelligence. 2021.
[24] Robyn Speer, Joshua Chin, and Catherine Havasi. Conceptnet 5.5: knowledge. Proceedings of the Thirty-Firss AAAI Conference on Artificial Intelligence. AAAI'I, 17, page 4444 4451, AAAI Press, 2017.
An open multilingual graph of general
[25] Mauro Cettolo, Christian Girardi, and Marcello Federico. Wit:: Web inventory of transcribed and translated talks. In Conference of European Association for Machine Translation, pages 261-268, 2012.
[26] Mauro Cettolo, Niehues Jan, Stüker Sebastian, Luisa Bentivogli, Roldano Cattoni, and Marcello Federico. The iwslt 2015 evaluation campaign. In International Workshop on Spoken Language Translation, 2015.
[27] Alexander M. Rush and Michael Collins. Exact decoding of syntactic translation models through La- grangian relaxation. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguisticx: Human Language Technologies, pages 72-82, Portland, Oregon, USA, June 2011. Association for Computational Linguistics.
[28] Alexander M. Rush, David Sontag, Michael Collins, and Tommi Jaakkola. On dual decomposition and linear programming relaxations for natural language processing. In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pages 1-11, Cambridge, MA, October 2010. Association for Computational Linguistics.
[29] Jessica Fieler and Yoav Goldberg, Controlling linguistic style aspects in neural language generation. arXiv preprint 17Xiv:1007.02633 2017.
[30] Lantao Yu, Weinan Zhang, Jun Wang, and Yong Yu. Seggan: Sequence generative adversarial nets with policy gradient. In Proceedings the AAAI conference artificial intelligence, volume 31, 2017.
12

---

[31] Yuta Kikuchi, Graham Neubig, Ryohei Sasano, Hiroya Takamura, and Manabu Okumura. Controlling ouiput length in neural encoder-decoders. arXiv preprint 1XXi::609.095532 2016.
[32] Alvin Chan, Yew-Soon Ong, Bill Pung, Aston Zhang. and Jie Fu, Cocon: A self-supervised approach for controlled text generation. arXiv preprint arXiv:2006.0333 03535, 2020.
[33] Can Xu, Wei Wu, Chongyang Tao, Huang Hu, Matt Schuerman, and Ying Wang. Neural response genera- tion with mota-words. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguisties, pages 5416 5426, Florence, Italy, July 2019, Association for Computational Linguistics.
[34] Yufei Wang, Ian Wood, Stephen Wan, Mark Dras, and Mark Johnson. Mention flags (MF): Constraining transformer-based text generators. Proceedings of the 59th Annual Meeting of Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 103- 113, Online, August 2021. Association for Computational Linguisties.
[35] Lei Sha. Gradient- Gradient-guided unsupervised lexically constrained text generation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 8692 8703, Online, November 2020. Association for Computational I Linguistics.
[36] Muhammad Khalifa, Hady Elsahar, and Mare Dymetman. A distributional approach to controlled text generation. arXiv preprint arXiv:2012. 11635, 2020.
[37] Jiacheng Zhang, Yang Liu, Huanbo Luan, Jingfang Xu. and Maosong Sun. Prior knowledge integration for neural machine translation using posterior regularization. arXiv preprint arXiv:/811.00 100, 2018.
[38] Christos Baziotis, Barry Haddow, and Alexandra Birch. Language model prior for low-resource neural machine translation. arXiv preprint orXhv:2004.14928. 2020.
[39] Jingyi Zhang, Masao Utyamma, Eichrro Sumita, Graham Neubig, and Satoshi Nakamura. Improving neural machine translation through phrasc-basee forced decoding, arXiv preprint arXiv: 2017.
[40] Yaohua Tang, Fandong Meng. Zhengdong Lu, Hang Li, and Philip LH Yu. Neural machine translation with extemaa phrase memory. arXiv preprint arXiv: vrXiv:1666.000 wwwp:1606009922.. 2016.
[41] Shiqi Shen, Yong Cheng, Zhongjun He, Wei He, Hua Wu, Maosong Sun, and Yang Liu. Minimum risk training for neural machine translation. arXiv preprint www.:::1:2.02444.0 2015.
[42] Emily M. Bender, Timait Gebru, Angelina McMillan Majorr and Shmargaret Shmitchell. On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Trunsparency, FAceT "21, page 610-623, New York, NY, USA, 2021. Association for Computing Machinery.
[43] Dirk Hovy and Shannon Spruit. The social impact of natural language processing. Proceedings of 54th Annual Meeting of the Association for Computational Linguistics ( (Volame 2: Short Papers), pages 591-598, Berlin, Germany, August 2016. Association for Computational Linguistics.
13


Reviews:
[{"review_id": "wGHyEBj08TX", "reviewer": "Reviewer_iHME", "summary": "The paper proposes a unified framework to incorporate prior knowledge into the text generation process via predicate logic constraints. The text generation model is a transformer and constraints are also encoded by a separate transformer. The encoded constraints then become part of the attention mechanism, therefore it can be considered as an approach that uses “soft” constraints. \n\nContributions\n- A general framework of incorporating prior knowledge as (soft) constraints into the decoding process for text generation.\n", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "After Authors' Response\nThe authors' response was very thorough and addressed all my raised concerns, therefore I have increased my score.\n\nStrength\n-\tThe idea makes sense on a high level and having an efficient manner of incorporating constraints into the decoding process for text generation would be very beneficial for many tasks and applications.\n\nWeaknesses\n-\tSome important points about the method and the experiments are left unclear (see also questions below).\n-\tThe writing could be improved (see also Typos & Additional Questions below)\n-\tMultiple runs and significance tests are missing. This makes it hard to judge the improvements (Table 2 & 3).\n\nMost Important Questions \n-\tLine 156: What is q_ij^k here exactly? I thought q_ij was a state flag, such as “2” or “0”. But you tokenize it and encode it, so it sounds more like it is something like “Copy(snow)”? (If it is the latter, then what is the meaning of tokenizing and encoding something like “Len(9)”?)\n-\t192: What exactly is storyline and what do you need it for?\n-\tThe baseline takes the predicate logic constraints as input: How does T6 know what to do with these inputs? Was the model trained on this but without the NRETM module? Can you give an example of what the input looks likes? How do these inputs  guide which sentences should be generated? Looking at the datsset, it feels like one would need at least the first 2 sentences or so to know how to continue. Maybe this information is now in your constraints but it would be important to understand what they look like and how they were created. Is there no other suitable baseline for this experiment?\n-\tWhat is the overhead of your method compared to standard decoding approaches? (you mention GBS can only be used with T5-Base, so your method is more efficient? That would be important to point out)\n-\tWhat happens if the decoding process cannot find a sequence that satisfies all constraint? \n-\tDocument-level MT: How do you know at test time whether the system translates a particular sentence or not?\n-\tHow many sentences are misaligned by Doc-mBART25? What are the s-BLEU and d-BLEU values on the subset that NRETM aligns correctly and Doc does not?\n-\tWhy was NEUROLOGIC not used as a comparison baseline?\n-\tWhat is dynamic vs static strategy? In which experiment did you show that dynamic works better than static (from conclusion)?\n\n\nTypos & Additional Questions\n-\tLine 40: you could mention here that the examples will be translated into logic forms in the next section.\n-\tParagraph starting at line 53: Why did you choose these datasets? How will they help evaluate the proposed approach?\n-\tLine 75: a and b should be bold faced?\n-\t83: “that used” -> “that are used”\n-\t83: “details” -> “for details”\n-\tParagraph at line 86: At this point, the state matrix is unclear. What are the initial values? How can the state matrix be used to understand if a constraint is satisfied or not?\n-\t98: “take[s]” & “generate[s]”\n-\t108: “be all” -> “all be”\n-\tParagraph at line 101: What is dynamic vs static strategy?\n-\tParagraph at line 109: The state flag explanation would greatly benefit from an example. Does q_i refer to whether a particular U_i is satisfied?\n-\tEq 2: What is the meaning of N? Can it change depending on the definition of U_k? Does it mean this constraint is not relevant for x_i?\n-\t133: Figure 1 should be Figure 2\n-\tFigure 2: What exactly do the “&” rows track?\n-\tFigure 2: Is the state flag matrix equal to the state matrix? If not, how do you go from one to the other?\n-\tLine 146: What does the inf in the superscript signify?\n-\t177: What is the symbolic operator?\n-\tParagraph at line 194: Without understanding what a storyline is, it is not clear what the constraints are. An example might be helpful here.\n-\tLine 204: what is the ROUGH-L metric? Do you mean ROUGE-L?\n-\tLine 223: How do you obtain the morphological inflections for the concepts?\n-\t237: @necessity [of] integrating”\n-\t3.3: How exactly is the document-level MT done? Is the entire input document the input to T5?\n-\t293: “because” typo\n-\t3.4 where/how exactly is the sentence index used?\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The paper's broader impact section discusses general potential benefits and issues of text generation (from large language models). It could maybe be tailored a bit better by discussing what effect this proposed work would have on the potential benefits and issues.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "tBCGH9tFa_", "reviewer": "Reviewer_HmFk", "summary": "The authors propose an approach aimed at encouraging transformer-based sequence-to-sequence models to respect constraints on the text they generate. In particular, the authors propose a particular syntactic representation of whether propositional logic-style constraints have been satisfied in some (partially) generated text, which is tracked throughout decoding. This syntactic representation is a function both of the current decoding step as well as the index of the particular constraint, and is updated dynamically as decoding progresses. The authors tokenize and encode these representations, and use them inside a relative-position-style attention mechanism. The authors fine-tune their models using these dynamic representations and they show that this allows for better control over various aspects of the generated text on RocStories, Commongen, and Zh-En document level translation tasks, with comparable or better quality.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This paper tackles an interesting problem, and it obtains good results. Another nice thing about the paper is the authors largely consider constraints that are *not* easy to implement with just a simple constrained beam search. In particular, rather than constraints which disallow certain tokens or alignments, the authors consider constraints that require the presence or a particular number of certain words, which are much more challenging to guarantee with standard decoding algorithms.\n\nIn terms of contribution, the core contribution of the paper appears to be the proposal of an approach to dynamically and explicitly updating the decoder in response to whether or how much the text generated so far conforms with some pre-specified constraints.  While this sort of dynamic updating seems quite reasonable, similar ideas have been proposed before (as the authors note), such as in the case of coverage attention (Tu et al., 2016). Accordingly, I think the proposed approach would feel more compelling if the authors could argue that it is superior to other approaches to updating the decoder in response to how well constraints have been satisfied in the generated text so far. However, I don't think there are any such comparisons. In particular, it seems the main baselines the authors consider are either a baseline seq2seq model with the constraints encoded statically on the source side (Table 1) or a baseline seq2seq model with a constrained decoding algorithm (Table 2). As such, it's hard to tell how crucial the various modeling/encoding choices the authors have made in implementing their approach are. (A natural baseline not considered, for instance, might involve dynamically updating the constraint tokens consumed by the encoder in the baseline in Table 1 instead of using the authors' proposed attention model).\n\nIn terms of presentation, I think the paper is largely clear, although I would encourage the authors to emphasize earlier on that their method involves fine-tuning the model rather than simply implementing constraints at decoding time; I think the fact that fine-tuning is involved only really becomes clear on the bottom of page 5. \n\nMinor:\nAre the script 'U's on like 116 different from the non-script upper case 'U's on line 109?\n\nUpdate after response from authors: thanks for your response; I'm increasing my score in view of the new baseline results.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5"}, "extra_text": {}}, {"review_id": "bOdzLz9QncJ", "reviewer": "Reviewer_EVbC", "summary": "This paper proposes a general framework, NRETM, for the conditional transformer-based seq2seq model. The paper first uses a state flag to indicate the expression progress of each predicate in the constraints set as a concatenation of logic trackers. The paper uses a one-layer transformer encoder to encode the state flag.  The paper utilizes the state matrix as a relative position to integrate it into the model. The results in three experiments show that the NRETM improves the transformer performance. ", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n1. The paper focus on an interesting problem about constraint-based generation tasks. The paper solves this problem by proposing a new Neural Rule-execution tracking machine that tries to incorporate rules into current transformer-based seq2seq generation models. The paper utilizes the logic tracker as relative position information to guide the model in the generation The experiment results over three different tasks seem to be promising. \n\n2. The paper lists detailed experiment settings for three different tasks. The paper also discusses the ability of the model in a zero-shot setting. The paper includes code as a supplement and put more generation results as case studies for three different tasks. Those examples show that the model captures those constraints through the proposed framework,\n\nWeaknesses:\n1. Section 2.4 is a little bit confusing. Even though the paper provides Figure 2 as a running example of the NRETM model with three logic constraints, it still takes me some time to understand all the concepts introduced in this section. It would be better to reorganize the content. For example, a logic tracker should be introduced before the state flag which can help readers to better understand the concepts. \n\n2. In section 3, the paper only compares their model to a limited baseline. The paper needs to add some baseline mentioned in related work such as NEUROLOGIC DECODING (Liu et al., 2020). In section 3.2, the constraint satisfaction metric is not clearly defined. The evaluation metrics for all/novel/seen constraints are also absent. The evaluation metrics are also limited. For story generation, rouge L is not able to cover all the details. Similarly, CIDEr can also show some limited aspect of generation results. It would be better to include more automatic metrics to show a more comprehensive view of the generation quality. such as Bertscore(Zhang et al., 2019), BLEU, etc. \n\n3. The generalizability of the framework is unclear. It seems that the model can only be applied with several pre-defined logic operators. Those logic operators\n\nLu, X., West, P., Zellers, R., Bras, R. L., Bhagavatula, C., & Choi, Y. (2020). Neurologic decoding:(un) supervised neural text generation with predicate logic constraints. arXiv preprint arXiv:2010.12884.\nZhang, T., Kishore, V., Wu, F., Weinberger, K. Q., & Artzi, Y. (2019). Bertscore: Evaluating text generation with bert. arXiv preprint arXiv:1904.09675.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The paper describes the limitation and social impacts in the last section. Some limitations are addressed. The potential misuse of this framework is also stated. ", "review_text": "", "extra_scores": {"needs_ethics_review": "Yes", "ethics_review_area": ["Inappropriate Potential Applications & Impact  (e.g., human rights concerns)"], "time_spent_reviewing": "5"}, "extra_text": {}}, {"review_id": "NeBGs1utWAk", "reviewer": "Reviewer_ysKc", "summary": "Interesting problem, but many issues along the way", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This work aims to find a way to impose the symbolic rules into powerful neural networks. To this end, it proposes a unified and scalable approach, named NRETM, to perform fine-grained controllable text generation and incorporate multiple prior knowledge. \n\nStrength:\n\n1. This paper proposes a framework to unify the constrained generation and general text generation with prior knowledge incorporation.\n\n2. Experimental results on story generation, machine translation, and commonsense generation partially demonstrate the effectiveness of the proposed method. \n\nDrawbacks:\n\n1. The novelty of this work is not clear. It seems to extract the intermediate values of the rules to the model, which is helpful for the model to learn the relationship between the rules and the final outputs. If my understanding is correct, is it not surprising that NRETM can make better results since additional prior knowledge can usually benefit the generation process.\n\n2. The experiments lack rigor. The author should add some competitive baselines for comparison. This paper mainly compares NRETM with pre-trained model T5. But T5 can not utilize prior knowledge efficiently. Feeding the predicate logic constraints as inputs directly is not a suitable way to impose prior knowledge.\n\n\n\nThe manuscript has some typos: \n1) eq. 1 P(y|x) is not defined.\n2) line 103, in 1 -> in Figure 1.\n3) line 116, consist -> consists.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The applications of the proposed methods are limited. The paper states that \"because in the training data at the position of EOS token in the ground-truth output, all elements in the state matrix must be set to the satisfied, NRETM can learn the rules by itself.\" However, if the training data contains some noise, can the proposed methods work as well? More analyses about this should be conducted.\n", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4 hours"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes a technique for including global constraints during decoding, and letting these be softly violated during decoding. Although the technique seems to be reasonable (there were a few concerns about the terseness of the approach), and reasonably evaluated, the problem of constrained optimisation is one of (if not the most) widely studied problem in optimisation, and indeed, constrained optimisation has come up numerous times even in "decoding problems" (e.g., frameworks for decoding like dual decomposition, Rush and Collins, 2012, and lagrangian relaxation, Martins et al., 2012 et passim), and using first order predicate logic to express constraints has a long history (the constrained conditional models of Rizzolo and Roth, 2007, and Markov Logic Networks of Poon and Domingos). I would encourage the authors to explicate how this work relates to other work in constrained optimisation in the context of decoding algorithms, and in particular, to be more precise in the motivation and nature of the constraints used in the experiments (for instance, are they designed on a validation set and then tested on a held-out test set?).
Decision: Accept (Poster)