Self-Ensemble of $N$-best Generation Hypotheses by Lexically Constrained Decoding
Authors: Ryota Miyano
Tomoyuki Kajiwara
Yuki Arase
Venue: EMNLP 2023 Main (2023)
Keywords: Reranking
Lexically Constrained Decoding
Generation

Abstract: We propose a method that ensembles $N$-best hypotheses to improve natural language generation. 
Previous studies have achieved notable improvements in generation quality by explicitly reranking $N$-best candidates. 
These studies assume that there exists a hypothesis of higher quality. 
We expand the assumption to be more practical as there exist \emph{partly} higher quality hypotheses in the $N$-best yet they may be imperfect as the entire sentences. 
By merging these high-quality fragments, we can obtain a higher-quality output than the single-best sentence. 
Specifically, we first obtain $N$-best hypotheses and conduct token-level quality estimation. 
We then apply tokens that should or should not be present in the final output as lexical constraints in decoding. 
Empirical experiments on paraphrase generation, summarisation, and constrained text generation confirm that our method outperforms the strong $N$-best reranking methods.

Self-Ensemble of N-best Generation Hypotheses by Lexically Constrained Decoding
Ryota Miyano", Tomoyuki Kajiwara', Yuki Arase
Graduate School of Information Science and Technology, Osaka University *Graduate School of Science and Engineering, Ehime University i {mi ryota, arase osaka-u. *kajiwaraee ehime-u. jc.
Abstract
Different from methods that involve computation- ally expensive model training such as the minimum risk training (Müller and Sennrich, 2021; Eikema and Aziz, 2022), these ranking-bass based methods are efficient and easily applicable to trained models. Nonetheless, these reranking methods assume that there is single hypothesis of higher quality in the N-best, which may not be practical depending on the generation model and also inputs. There- fore, we enhance the assumption; there should be candidates that are partly high-quality but may be imperfect as the entire sentences. Our method iden- tifies and merges these higher-quality fragments to derive high-quality output using lexically con- strained decoding (Lu et al., 2022). Specifically, our method trains token-level quality estimator that predicts whether a token in hypothesis should be or should not be included in the final output. It then uses the quality estimation
We propose a method that ensembles N-best hypotheses to improve natural language gen- eration. Previous studies have achieved no- table improvements in generation quality by explicitly reranking N-best candidates. These studies assume that there exists a hypothesis of higher quality, We expand the assumption to be more practical as there exist partly higher quality hypotheses in the N-best yet they may be imperfect as the entire sentences. By merg- ob-
ing these high-quality fragments, tain a higher-quality output than the single-hest sentence. Specifically, we first obtain N-best hypotheses and conduct token-level quality es- timation. then apply tokens that should should not he present in the final output as lexical constraints in decoding. Empirical ex- periments on paraphrase generation, summari- sation, and constrained text generation confirm that our method outperforms the strong N-best reranking methods.
(QE) results of the N N-best hypotheses to compose positive and negative lexical constraints and generates the final output using the generation model.
1 Introduction
As contribution of this study, we propose the N bbest ensembling method for improving the qual- ity of language generation, which is easy apply to a variety of language generation tasks. Empiri- cal experiments on paraphrasing (Takayama et al., 2021), summarisation (See et al., 2017; Hermann et al., 2015; Narayan et al., 2018), and constrained text generation (Lin et al., 2020) confirm that our assumption holds and the proposed method outper- forms strong trranking-based methods.
While the beam search is one of the most com- mon decoding methods in natural language genera- tion, it suffers from the beam search curse (Koehn and Knowles, 2017; Yang et al., 2018; Ott et al., 2018; Stahlberg and Byrne, 2019) where a large beam size degrades the quality of generation. As remedy to this problem. previous studies explored better alternatives from N-best hypotheses (Fernan- des et al., 2022) as represented as reranking and minimum Bayes decoding (Müller and Sennrich, 2021; Eikema and Aziz, 2022), which only mod- ify the decoding procedures. There are two types of reranking approaches. Discriminative methods train rerankers to predict specific evaluation met- ric scores of each hypothesis (Shen al., 2004; Bhattacharyya et al., 2021; Lee al., 2021). In contrast, generative methods use generic rerankers that have been used for other purposes, such as language models (Yee et al., 2019; Ng et al., 2019).
Preliminary: Lexically Constrained Decoding
Lexically constrained decoding has been employed in various language generation tasks apply task- specific knowledge on generation, e.g., for machine translation using a bilingual dictionary of techni- cal terms as constraints (Chatterjee et al., 2017; Hokamp and Liu, 2017), for text simplification us-

---

Hypothesis
Output
binary classification results
Language generation
Hypothesss
iolen-Level
NeuroLoglew
Constraint
Inpur
Hypothesis X
Token Level QE
genetatioon model
Token-Level
(NFUT))
Xm QUERRY
>2
Figure 1: Overview of the proposed method
source sentence
hypothesis in the N-best
Figure Token-level QE model (special input tokens are added into the vocabulary.)
ing difficult words as constraints (Nishihara al., 2019; Dehghan al., 2022; Zetsu et al., 2022), for style transfer using style-specific vocabulary as constraints (Kajiwara, 2019), and for table-to-text generation using keywords in tables as constraints
Data set
Train
Validation
Test
DIRECT
64, 126
7,372
CNN/Daily Mail
287, 113
13, 13,368
11,490
(Lu et al., 2022). Different from these studies that assume the availability of task-specific knowledge. method creates lexical constraints based purely on the V-best hypotheses of language generation.
XSum COMMONGEN
204, 045
11,332
11,334
67, 389
4,018
7, 644
Table I: Number of sentences evaluation datasets
We use the state-of-the-ee lexically constrained decoding method, namely, NeuroLogic-A# (Lu et al., 2022). NouroLogic--A * searches for output candidates with high generation probabilities and constraint satisfaction rates by tracking states of sat- isfaction by the following steps. (I) For each can- didate token at time step, NeuroLogic-A looks ahead to future tokens to be generated. (2) Based on the look-ahead results, it computes satisfaction rates of lexical constraints and prunes the candi- date tokens. (3) It groups the remaining candidates based on the states of the constraint satisfaction and selects output tokens from the best candidates in each group to preserve broad search space.
hypothesis the N-best as input and conducts binary classification for each token. We expect that the masked language model acquires the sense of synonyms and multi-word expressions through pre-training and transfers that knowledge to our token-level QE.
Because the token-level QE is context- dependent, the same token appearing in different hypotheses may be predicted both positive and negative labels, respectively. Our model determines the final label by majority voting. If the numbers of positive and negative predictions tie, the corresponding token is excluded from lexical constraints.
3 Proposed Method
Figure/ shows the overview of the proposed method. The main component is the token-level QE model that predicts whether each token in N- best hypotheses should be used or avoided in gen- erating the final output. Tokens predicted as the former is included in positive constraints and those predicated as the latter are included negative constraints to be considered by NeuroLogic-A' Specifically, we fine-tune a pretrained masked language model to conduct binary token classi- fication as illustrated in Figure 2. For each N- best hypothesis of training sentences obtained by the language generation model, token-level labels are automatically assembled using their references. Hypothesis tokens appearing in the correspond- ing reference are labelled as positive and other- wise labelled as negative. At inference, the QE model takes the concatenation of a source and a
Experimental Settings
The proposed method is widely applicable to lan- guage generation tasks. We thus evaluate it on paraphrasing ($ 5.1), summarisation ($ 5.2), and constrained text generation ($ 5.3).
Proposed Method For each evaluation dataset (Table 1), we constracted a token-level QE model by fine-tuning a RoBERTa-base (Liu et al., 2019) 9). Specifically, we sought the beam size of N by a grid search in [1, 5, 10, 20, 30, ....... 100) to achieve the best performance the validation set measured by the corresponding evaluation metrics. When de- coding with NeuroLogic-A for the final output, we use the same beam size as baselines for fair com- parison. For more details of the implementation, please refer to Appendix A.

---

Indirect-to-Direen
Hirect-toIIndiret
w/ history
w/o history
w/ history
w/o history
BLEU
N
BLEU
N
BLEU
N
BLEU
N
beam-search NCD (Yee et al., 2019)
35.57
34.38
26.92
26.63
36.01
30
35.431
20
26.14t
40
27.03t
20
DrNMT (Lee et al., 2021)
35.85
30
31.651
60
27.06
100
26.50
20
NeuroLogic-A (P
N)
36.43
50
35.42t
40
30.21t
20
30.57t
30
NeuroLLggieA (P)
36.95
50
35.94
40
30.891
20
31.33
70
NeuroLoggieA (N)
35.81*
50
31.821
60
29.97t
20
30.12f
10
Rerankingorace
59.71
100
59.161
100
48.951
100
49.251
100
NeuroLogic-A # (P & Nooracle
65.55
100
65.31
110
60.23
100
100
NeuroLoggic-A # (P) Draacle
57.85
100
57.381
100
49.42f
100
50.15t
100
NeuroLogic-Ar A (N) Norale
51.60
100
50.981
100
45.24
100
45.54
100
Table 2: Test set BLEU scores on DIRECT; N determines the number of hypotheses to consider and indicates significant differences against beam-scarch confirmed by bootstrap resampling test (Koehn, 2004).
Baselines As the most basic baseline, we com- pared our method to beam search, of which size was borrowed from previous studies that tuned the value for underlying language generation mod- els for each task. Wherever applicable, we also compared the strong discriminative and generative reranking methods. For the former, we used the Discriminative Reranker for Neural Machine Trans- lation (DrNMT) (Lee et al., 2021>* that predicts the distribution of sentence-level evaluation met- ric scores given the source and N-best hypotheses. For the latter, we used the Noisy-Channa Decoding (NCD) (Yee al., 2019)² that scores N N-best can- didates by linearly combining the probabilities of generation, target-side language model, and target- to-source generation. We trained these methods using the authors' implementations with datasets and metrics of each experiment task, where the beam sizes (the sizes of N) were searched in the same way as our method using the validation sets.
Experimental Results
This section discusses the experimental results. The details of implementation each task are described in Appendix B.
5.1 Paraphrasing
Setting We used DIRECT (Direct and Indirect REsponses in Conversational Text) (Takayama et al.. 2021), which provides paraphrases between indirect and direct utterances in conversation his- tories. We conduct both the Indirect-to-Diie -irect and Direct-to- Indirect paraphrasing with and without the dialogue histories. Following Takayama et al. (2021), we fine-tuned BART (Lewis et al., 2020) as the underlying language generation models with the beam size of
Results Table 2 shows the test set BLEU (Pap- ineni et al., 2002) scores. The upper rows show the experimental results of baselines and the proposed methods using the predicted lexical constraints. In all subtasks, the proposed methods outperformed the baselines. particular, the proposed method using only positive lexical constraints LNeuroLogic- A (P)) achieves the best performance.
Ablation As ablation studies, we evaluated the performance of the proposed method using only positive lexical constraints (denoted as NeuroLogic- * (P)), only negative lexical constraints (denoted NeuroLogic-A (N)), and both (denoted as NeuroLoggicAA* & N)).
The lower rows in Table 2 show the 'oracle' per- formance of the proposed and reranking methods.
The 'Rankingoracke indicates the performance when selecting the hypothesis with the highest sentence- level BLEU score against the reference. In our method, we used the 'oracle' lexical constraints obtained by accessing the references. As expected, all of the BLEU scores are much higher than the
http:///ggttttbooooooeeeeerrrrrr httpereeeeeerre///reeeeee trre//mainneeammplle//nlaayyyai

---

CNN/Daily Mail
XSum
constraints (NeuroLogic-A (P & N)) outperforms the baselines and achieves the highest ROUGE-L score, which has a high correlation with the human evaluation (Lin, 2004). These results confirm that the proposed method also effective in summarisa- tion. The full results are available in Appendix B.3.
RI.
N
RL
N
beam-search
40.99
37.21
DrNMT (Lee et al., 2021)
40.40
37.18
10
NeuroLogic-AA (P N)
41.99
80
37.39
90
NeuroLogic-A (P)
41.72
10
37.16
10
NeuroLogic-A* (N)
41.76
100
37.24
90
5.3 Constrained Text Generation
Table 3: Test set ROUGE-L scores of the summarisa- tion tasks; N determines the number of hypotheses to consider and I indicates significant differences against beam-search confirmed by approximate randomisation test (Riezler and Maxwell, 2005).
Setting We used COMMONGEN (Lin et al., 2020) dataset that tasks to generate coherent sentences given a set of words. We fine-tumed GPT-2 (Rad- ford et al., 2019) as the underlying language gener- ation model with the beam size of 5 (Welleck et al., 2023). Different from paraphrasing and summari- sation, there are variety of possible generations as reflected in the multiple references of diverse contents. To adapt our QE model training to this task, we selected the single reference for each hy- pothesis that has the highest lexical overlap against the corresponding hypothesis.
CIDEr
beam-search NCD (Yee et al., 2019)
14.26
16.2 24
100
DrNMT (Lee al.. 2021)
14.85
60
SELF-CORRECT (Welleck et al., 2023)
15.30
*NeuroLogic (Welleck et al., 2023)
15.28
NeuroLoogic-A* (P & N)
15.38
90
NeuroLogic-A' (P)
15.62
10
NeuroLogic-At (N)
14.52
90
Table 4 shows the CIDEr (Vedantam et al., 2015) scores measured on the test set, where SELF-CORRECT (Welleck et al., 2023) is the state- of-the-art method." 3 While our method ensembles N-best to improve the generation quality, SELF- CORRECT iteratively edits the initial one-best out- put. As the results show, our method using only positive lexical constraints (NeuroLogic-A (P)) outperformed SELF-CORRECT which confirms the effectiveness of ensembling high- high-quality frag- ments the Nebest. Nonetheless, the best method is NCD for this task. We conjecture this is because considering tokens from different hypotheses may deteriorate the generation due to the diversity in ac- ceptable outputs. This feature is also troublesome for training discriminative reranking models im- plied by the inferior performance of DrNMT. In such tasks, generative reranking models like NCD may be suitable. The full results are available in Appendix B.5.
Results
Table 4: Test set CIDEr scores on COMMONGEN; N determines the number of hypotheses to consider.
upper rows. Remarkably, NeuroLogic-A* (P & Noracle largely outperforms Rankingarake- This re- sult confirms that ensembling N-best hypotheses is more effective than simply selecting the best hy- pothesis. It supports our assumption that there exist high-quality fragments in N best even though they are imperfect as the entire sentences. Moreover, the impressively higher scores under the oracle set- ting indicate that improving the token-level QE is a promising direction as further discussed in 6.
5.2 Summarisation
Setting We used the CNN/Daily Mail (See et al., 2017: Hermann et al., 2015) (version 3.0.0) and XSum (The Extreme summarisation) (Narayan et al.. 2018) datasets. As underlying language gen- eration models, we used the publicly available fine- tuned BART-large models on CNN/Daily Mail and XSum released by Lewis et al. (2020) with sug- gested beam sizes of 4 and respectively.
6 Discussion and Future Work
As we discussed in $ 55, the quality of token-level QE is critical for the performance of our method. Table 6 shows the ratio of reference tokens mistak- enly included in the negative constraints (Pneg) and the recall of positive constraints (Rpus) in the eval-
Results Table 3 shows the ROUGE-L (Lin, 2004) scores measured on the test sets. Note that NCD is not applicable to summarisation due to the un- availability of target-to- to-source generation model. For both CNN/Daily Mail and XSum, the proposed method using both positive and negative lexical
AA CELF-CORECT is contemporaneous with our study. we borrowed these scores from the original paper. The unavail- ability of model outputs at the time of publication hindered further comparisons.

---

Source
with 1TB memory and AMD EPYC 7552 CPU. Well, thats like everything that is required. Thanks lot. Awesome. Thanks for the details. Bye.
Reference
Positive constraints
Awesome, great, good, the, details, detail, a, an, for, Have, you, day, it.
Source
Chinese I think, but need location and how to get in touch with them.
Reference
(don't mind. Maybe chinese? need contact number and postcode. phone, contact, number, the, and, I, address, need, Chinese, Chinese,
Positive constraints
Table 5: Examples of constraints predicted by our method (Indirect-to- Dirrct transformation without history)
Task
Paeg (4)
Rpos (1)
with 48GB memory installed on Linux server
Indirect-to. to- totttt---ie co-Direct w/ history
0.20
0.41
Our naive implementation needs 1.9 sec/sent while
Sifirect-to-Diress w/o history
0.24
0.37
DrNMT (Lee et al., 2021) and NCD (Yee et al., 2019) do 0.3 sec/sent on average. A straightfor- ward remedy is to adaptively decide whether to conduct the second decoding based on the token- level QE results. For example, if there is hypoth- esis of which token-level QE results imply that it satisfies a quality standard needed by a downstream task, we can directly output the hypothesis. all the hypotheses are unsatisfactory, we can conduct the second decoding using lexical constraints.
Direct-to- Io-Indirect w/ history
0.23
0.40
Firect-to-Indiredd w/o history
0.22
0.36
CNN/Dañl Mail
0.28
0.31
XSum
0.24
0.36
COMMONGEN
0.20
0.36
Table 6: Performance of our token-level QE
uation tasks. The results indicate that 20% to 28% of reference tokens were in the negative constraints while the recalls of positive constraints were lim- ited to 31 % to 11 %. Improvement of these metrics directly enhances our method, which constitutes our future work. We will explore a QE method to model interactions within and across hypotheses.
Currently, all constraints are treated equally in lexically constrained decoding, but we assume their importance can be diverse and may change depend- ing on the status of generation. This expansion is beyond the scope of the current paper but surely worth exploring, which constitutes our future work.
Table 5 shows examples of constraints predicted by our method on the paraphrase generation task (Indirect-to-Dirng Direct transformation without history). In the first example, synonyms of "awesome". "great", and "good" are predicted, while in the sec- ond example, wordi-word expressions of "contact number" and "phone number" are predicted as pos- itive constraints. These results indicate that our QE model preserves the ability to consider these to some extent. We should need a more sophisticated model to better handle synonyms and multi word expressions, which constitutes our future work.
Acknowledgements
This work was supported by JSPS KAKENHI Grant Number JP21H03564.
References
Sumanta Bhattacharyya, Amirmohammad Rooshenas, Subhajit Naskar, Simeng Sun, Mohit lyyer, and An- drew McCallum. 2021. Energy-baas based reranking: Improving neural machine translation using energy- based models. In Proceedings of the 59th Annual Meeting of the Association for Computational Lin- guistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Pupers), pages 4528 4537, Online. Association for Computational Linguistics
Limitations
Our model conducts decoding twice to generate a final sentence; furthermore, the second one is lex- ically constrained decoding, which increases the computational cost of language generation. We measured the decoding times of the proposed and compared methods on the paraphrase generation task (Indirect-to- Direet transformation without his- tory) under the same settings of Table 2. The pro- grams ran on a single GPU of NVIDIA RTX A6000
Pawel Budzianowski, Tsung-Hsiae Wen, Bo-Hsiang Tseng, Iñigo Casanueva, Stefan Ultes, Osman Ra- madan, and Milica Gašié. 2018. MultiWOZ- a large- scale
Wizard-of dataset for task- oriented dialogue modelling. In Proceedings of the 2018 Conference Empirical Methods in Natural Language Processing, pages 5016-5026, Brussels. Belgium. Association for Computational Linguistics.

---

Rajen Chatterjee, Matteo Negri, Marco Turchi, Marcello Federico, Lucia Specia, and Frédéric Blain. 2017. Guiding neural machine translation decoding with external knowledge. In Proceedings of the Second Conference on Machine Translation, pages 157-168, Copenhagen, Denmark. Association for Computa- tional Linguisties.
2004 Conference on Empirical Methods in Natural Language Processing, pages 388-395. Barcelona, Spain, Association for Computational I Linguistics.
Philipp Koehn and Rebecca Knowles. 2017. Six chal- lenges for neural machine translation. In Proceedings of the First Workshop on Neural Machine Translation, pages 28-39, Vancouver. Association for Computa- tional Linguisties.
Mohammad Dehghan, Dhruv Kumar, and Lukasz Golab, 2022. GRS; Combining generation and revision in unsupervised sentence simplification. In Findings of the Association for Computational Linguistics: ACL 2022, pages 949-960, Dublin, Ireland. Association for Computational Linguistics.
Ann Lee, Michael Auli, and Marc' Aurelio Ranzato. 2021. Discriminative reranking for neural machine translation. In Proceedings of the 59th Annual Meet- ing of the Association for Computational Linguistics and the I 11th International Joint Conference on Natu- ral Language Processing (Volume I: Long Papers), pages 7250-7264, Online, Association for Computa- tional Linguistics.
Bryan Eikema and Wilker Aziz. 2022. Sampling- based approximations to minimum Bayes risk decoding for neural machine translation. In Proceedings of the 2022 Conference on Empirical Methods in Natu- ral Language Processing, pages 10978-10993, Abu Dhabi, United Arab Emirates. Association for Com- putational Linguistics.
Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020. BART: Denoising sequence-to-sequeee pre-training for natural language generation, translation, and com- prehension. In Proceedings of the 58th Anmual Meet- ing of the Association for Computational Linguistics, pages 7-7100, Online, Association for Computa- tional Linguistics.
Mihail Eric, Rahul Goel, Shachi Paul, Abhishek Sethi, Sanchit Agarwal, Shuyang Gao, Adarsh Kumar, Anuj Goyal, Peter Ku, and Dilek Hakkani- Tur.. 2020. Mul- tiWOZ 2.1: A consolidated multi-domain dialogue dataset with state corrections and state tracking base- lines. In Proceedings of the Twelfth Language Re- sources and Evaluation Conference, pages 422-428, Marseille, France. European Language Resources Association.
Bill Yuchen Lin, Wangchunshu Zhou, Ming Shen, Pei Zhou, Chandra Bhagavatula, Yejin Choi, and Xiang Ren. 2020. CommonGen: A constrained text gen- eration challenge for generative commonsense rea- soning, In Findings of the Association for Computa- tional Linguistics: EMNLP 2020, pages 1823-1840, Online. Association for Computational Linguistics.
Patrick Fernandes, António Farinhas, Ricardo Rei, José G. C. de Souza, Perez Ogayo, Graham Neubig, and Andre Martins. 2022. Quality-aware decoding for neural machine translation. In Proceedings of the 2022 Conference of the North American Chap- ter of the Association for Computational Linguistics: Human Language Technologies. pages 1396-1412, Seattle, United States. Association for Computational Linguistics.
Chin-Yew Lin. 2004, ROUGE: A package for auto- matic evaluation of summaries. In Text Summariza- fion Brunches Out, pages 74-81, Barcelona, Spain. Association for Computational Linguistics.
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Man- dar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized BERT pretraining approach. CoRR, abs/1907. 11622.
Karl Moritz Hermann, Tomás Kociský, Edward Grefen- stette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. 2015. Teaching machines to read and comprehend. In NeurIPS, pages 1693-1701.
Chris Hokamp and Qun Liu. 2017. Lexically con- strained decoding for sequence generation using grid beam search. In Proceedings of the 55th Annual Meeting of the Association for Computational Lin- guistics ( (Volume 1: Long Papers), pages 1535-1546, Vancouver, Canada. Association for Computational Linguistics.
Ximing Lu, Sean Welleck, Peter West, Liwei Jiang, Jungo Kasai, Daniel Khashabi, Ronan Le Bras, Lian- hui Qin, Youngjae Yu, Rowan 7ellers, Noah Smith, and Yejin Choi. 2022. LouroLogic a aeesque decoding: Constrained text generation with lookahead heuris- tics. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Com- putational Linguistics: Human Language Technolo- gies, pages 780-799, Scattle, United States. Associa- tion for Computational Linguistics.
Tomoyuki Kajiwara, 2019. Negative lexically con- strained decoding for paraphrase generation. In Pro- ceedings of the 57th Annual Meeting of the Asso- ciation for Computational Linguistics. pages 6047- 6052, Florence, Italy. Association for Computational Linguistics.
Mathias Müller and Rico Sennrich. 2021. Understand- ing the properties of minimum Bayes risk decoding in neural machine translation. In Proceedings of the 59th Annual Meeting of the Association for Compu- tational Linguistics and the 1 11hh International Joint
Philipp Koehn. 2004. Statistical significance tests for machine translation evaluation, In Proceedings of the

---

Conference on Natural Language Processing (Vol- ume 1: Long Papers), pages 259- 272, Online. Asso- ciation for Computational Linguistics.
Libin Shen, Anoop Sarkar, and Franz Josef Och. 2004, Discriminative reranking for machine translation. In Proceedings of the Human Language Technol- ogy Conference of the North American Chapter of the Association for Computational Linguistics: IILT-NAACL 2004, pages 177-184, Boston, Mas- sachusetts, USA. Association for Computational Lin- guistics.
Shashi Narayan, Shay B. Cohen, and Mirella Lapata. 2018. Don't give me the details, just the summary! topic-aware convolutional neural networks for ex- treme summarization, In Proceedings of the 2018 Conference on Empirical Methods Natural Lan- guage Processing, pages 1797-1807, Brussels, Bel- gium. Association for Computational Linguisties.
Felix Stahlberg and Bill Byme. 2019. On NMT search errors and model errors: Cat got your tongue? In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Lan- guage Processing (EMNLP-IJCNLP)) pages 3356- 3362, Hong Kong, China. Association for Computa- tional Linguistics.
Nathan Ng. Kyra Yee, Alexei Baevski, Myle Ott, Michael Auli, and Sergey Edunov. 2019, Facebook FAIR's WMT19 news translation task submission. In Proceedings of the Fourth Conference on Machine Translation (Volume 2: Shared Task Papers. Day 1). pages 31--319, Florence, Italy. Association for Computational Linguistics.
Junya Takayama, Tomoyuki Kajiwara, and Yuki Arase. 2021. DIRECT: Direct and indirect responses in conversational text corpus. In Findings of the Associ- ation for Computational Linguistics: EMNLP 2021, pages 1980-1989, Punta Cana, Dominican Republic. Association for Computational Linguisties.
Daiki Nishihara, Tomoyuki Kajiwara, and Yuki Arase. 2019. Controllable text simplification with lexical constraint loss. In Proceedings of the 57th Annual Meeting of the Association for Computational Lin- guisties: Student Research Workshop, pages 260- 266. Florence, Italy. Association for Computational Linguistics.
Ramakrishna Vedantam, C. Lawrence Zitnick, and Devi Parikh. 2015. Cider: Consensus-based image de- scription evaluation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recog- nition (CVPR).
Myle Ott, Michael Auli, David Grangier, and Mare' Aurelio Ranzato. 2018. Analyzing uncertainty in neural machine translation. In Proceedings of the 35th International Conference on Machine Learn- ing, volume 80 of Proceedings of Machine Learning Research, pages 3956-3965. PMLR.
Sean Welleck, Ximing Lu, Peter West, Faeze Brah- man, Tianxiao Shen, Daniel Khashabi, and Yejin Choi. 2023. Generating sequences by learning to self-correct. The Eleventh International Confer- ence on Learning Representations.
Kishore Papineni, Salim Roukos, Todd Ward, and Wei- Jing Zhu. 2002. Bleu: method for automatic evalu- ation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Compu- tational Linguistics, pages 311-318, Philadelphia, Pennsylvania. USA. Association for Computational Linguistics.
Thomas Wolf, Lysandre Dehut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier- ric Cistac, Tim Rault, Remi Louf, Morgan Funtow- icz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite. Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame. Quentin Lhoest, and Alexander Rush. 2020. Trans- formers: State-of-the-ar natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38 38-45, Online. Association for Computational Linguistics.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, al. 2019. Language models are unsupervised multitask learners. OpenAl blog. 188):
Yilin Yang, Liang Huang, and Mingbo Ma. 2018. Break- ing the beam search curse: A study of (re-)scoorng methods and stopping criteria for neural machine translation. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Process- ing, pages 3054-3059, Brussels, Belgium. Associa- tion for Computational Linguistics.
Stefan Riezler and John T. Maxwell. 2005. On some pitfalls in automatic evaluation and significance test- ing for MT. In Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization, pages 57 57-64, Ann Arbor, Michigan. Association for Com- putational Linguistics.
Kyra Yee, Yann Dauphin, and Michael Auli. 2019. Simple and effective noisy channel modeling for neural machine translation. In Proceedings of the 2019 Conference on Empirical Methods in Natu- ral Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 5696-5701, Hong Kong. China. Association for Computational Linguistics.
Abigail See, Peter J. Liu, and Christopher D. Manning. 2017. Get to the point: Summarization with pointer- generator networks. In Proceedings of the 55th An- nual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1073- 1083, Vancouver, Canada, Association for Computa- tional Linguistics.

---

Tatsuya Zetsu, Tomoyuki Kajiwara, and Yuki Arase, 2022. Lexically constrained decoding with edit oper- ation prediction for controllable text simplification. In Proceedings of the Workshop on Text Simplifi- cation, Accessibility and Readability (TSAR-2022), pages 147-153, Abu Dhabi, United Arab Emirates (Virtual). Association for Computational Linguistics.
articles and 1.0 for summaries. The XSum dataset requires less number of summary sentences than the CNN/Daily Mail dataset; therefore, requires more abstract summarisation. The maximum input length of our token-level QE model is 512. If an input length exceeds that limit, we split the article into two and input to the model, and then merge the prediction results.
A Implementation Details
We implemented our token-level QE model using RoBERTa (Liu et al., 2019) with the HuggingFace Transformers (Wolf et al., 2020) library. fine- tuning RoBERTa, we calculated the F1 score on the validation set at the end of every epoch and stopped tuning when there was no improvement for 3 epochs.
As underlying language generation models for summarisation, we used ffacebook/bart-lrgge ' and "facebookbbrrt-argee-
These models have been fine-tuned on CNN/Daily Mail and XSum datasets, respectively. Their beam sizes are suggested as 4 and 6, respectively.
B.3 Summarisation Results
We used the implementation of Lu et al. (2022) replicate N However, due the lack of negative lexical constraints in the original implementation, we modified the codes to allow negative lexical constraints.
Table 7 shows test set results of all evaluation met- rics. The bottom three rows indicate the perfor- mance when using the oracle lexical constraints created by accessing references.
B.4 Constrained Text Generation Experiment
B Experiment Details
The COMMONGEN dataset consists of 35, 141 concept sets associated with 77, 449 sentences. The average length of reference sentences in the COMMONGEN dataset 10.86.
B.1 Paraphrasing Experiment
The DIRECT corpus is extension of the multi- domain, multi-turn, task-oriented dialogue corpus of MultiWOZ 2.1 (Multi-Domain Wizard-of-ON 2.1) (Budzianowski al., 2018; Eric al., 2020). DIRECT provides the dialogue histories in Multi- WOZ, the original responses, indirect paraphrases of the original responses, and direct paraphrases of the original responses.
We fine-tuned a 'gpt2-large" model with the same setting as Lin et al. (2020). The evaluation metrics were computed using the official script!
B.5 Constrained Text Generation Results
Table shows test set results of all evaluation met- rics. The bottom rows present the results under the oracle setting. Different from paraphrasing and summarisation, the oracle reranking, which chooses a hypothesis with the highest evaluation score, outperformed our methods with oracle lex- ical constraints. Our manual investigation con- firmed that the references of source sentence are diverse in COMMONGEN, and thus considering to- kens from different references can be harmful. This result implies that the diversity in possible gen- erations affects the performance of the proposed method.
We fine-tuned 'facebook/bart "6 model us- ing the HuggingFace Transformers library with the same setting as Takayama al. (2021). The beam size was set to 1 following the experiments in the original paper.
B.2 Summarisation Experiment
The CNN/Daily Mail dataset is a collection of CNN and Daily Mail articles and highlights (sum- maries), and consists of about 310k news articles and highlight pairs. The average number of sen- tences in the CNN/Daily Mail dataset 30.7 for ar- ticles and 3.8 for highlights. The XSum dataset is a collection of BBC articles and their summaries and consists of about 230k article-summary pairs. The average number of sentences in XSum is 19.8 for
"https://huggingfaae calete..coccceee co/facebook/ bart-large-cne
"nttps://huggiiggaac co/roberta-bass 'ht thtt:p///ttthb.. htttps///ggttuuu.o.//tttntgeuuu neurologic
"https://huggingfaae hatee.cofaaaaae co/facebook/ bart-
"https://huggingface
'https: http://///nnnnnee..ooeeeaea///rr htttee/cc/acccco//baaaaaea
"https://githhe thub. com/INK- WWW.IIIWK--UCCCo/COnn//

---

CNN/Daily Mail
XSum
R1
R2
RL
N
R1
R2
RL
N
beam-search
44.04
21.08
40.99
45.46
22.35
37.21
DrNMT (Lee et al., 2021)
43.53
20.62
40.40t
10
45.37
22.33
37.18
10
NeuroLLogic-A* (P & N)
44.99
21.64
41.99
80
45.691
22.37
37.39
90
NeuroLogic-A* (P)
44.761
21.33
41.721
10
45.87
22.19
37.16
10
NeuroLogic-A* (N)
44.72t
21.721
41.761
100
45.18
22.25
37.24
90
Rerankingonalc
53.78
21.64
41.99
100
56.74
35.19
51.96'
100
NeuroLogic-A (P & Nurracle
61.74
33.84
54.75t
100
67.23
42.72
54.69t
100
NeuroLogic-A" (P)oclee
56.05t
30.581
52.051
100
61.22
35.681
47.52t
100
NeuroLogic-A" (N)oraee
53.30
29.891
50.091
100
55.33
32.88
47.14
100
Table 7: Test set ROUGE scores of the summarisation tasks; N determines the number of hypotheses to consider
and indicates significant differences against beam-search confirmed by approximate randomisation test (Riezler and Maxwell, 2005).
BLEU-4
CIDEr
Coverage
N
beam-search NCD (Yee et al., 2019)
27.08
14.26
84.48
31.52
16.24
91.73
100
DrNMT (Lee et al., 2021)
27.55
14.85
91.73
60
SELF-CORRECT (Welleck et al., 2023)
27.98
15.30
94.58
SELF-CCORRECT+NNeuull
(Welleck et al., 2023)
28.17
15.28
97.80
NeuroLogic-A (P & N)
28.85
15.38
91.39
90
NeuroLogic-A (P)
28.04
15.62
94.06
10
NeuroLogic-AA* (N)
27.41
14.52
86.71
90
Rerankingorsele
52.70
21.62
90.15
100
NeuroLogic-A * (P & N)oracle
42.51
19.20
96.71
100
A (P)oracee
38.52
18.98
97.81
100
NeuroLogic-A's (Nooraee
30.66
15.29
86.99
100
Table 8: Test set scores on COMMONGEN; N determines the number of hypotheses to consider.


Reviews:
[{"review_id": "lth3jLAfmA", "reviewer": "Reviewer_5Mk4", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your time and efforts in reviewing our paper. \n\n**Reasons To Reject:** The proposed method needs to conduct decoding twice to generate a final sentence, which is inefficient and is not suitable for practical application.\n\n**Response:** We agree that our implementation may not be immediately applicable to “online” systems. As we experimented, current naive implementation needs about 1.9 sec / sent for decoding (for details, see our response to Reviewer 8gyk). The performance should be improved by more professional engineering for sure. In addition, our method has a wide applicability to various language generation tasks (and indeed achieved remarkable improvements on three of them), which can be used in “offline” scenarios where “online” responses are not needed.  More importantly, computational efficiency should not be the only reason for avoiding exploring promising technologies, which would hinder technical breakthroughs in the future.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The proposed method in this paper is intuitive and effective through some experiments on paraphrasing, summarization, and constrained text generation tasks.", "reasons_to_reject": "The proposed method needs to conduct decoding twice to generate a final sentence, which is inefficient and is not suitable for practical application.", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper introduces a natural language generation method, which merges the high-quality fragments in N-best hypotheses and applies tokens that should or should not be present in the final output as lexical constraints in decoding. The proposed method is actually a N-best ensemble method for improving the quality of language generation. And it can be easily to apply to a variety of language generation tasks such as paraphrasing, summarization, and constrained text generation.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "EQXq4kzMqC", "reviewer": "Reviewer_8gyk", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your great efforts in reviewing. We appreciate your acknowledging the contributions of our study. \n\n**Reasons To Reject:** Although the author mentioned in the Limitations section that this method is more time-consuming compared to reranking, it is better to provide a time comparison in the paper for reference in subsequent studies.\n\n**Response:** Thank you for the valuable suggestion. We measured the decoding time of our and compared methods on paraphrase generation (Indirect-to-Direct transformation without history) under the same settings of Table 2. Our naive implementation needs 1.9 sec / sent while DrNMT (Lee et al., 2021) and NCD (Yee et al., 2019) do 0.3 sec / sent on average. We believe the computational efficiency will be improved by more professional engineering for sure. We will expand the Limitation section to include these results in the final version if the paper is accepted.\n\n\n**Typos Grammar Style And Presentation Improvements:** This paper might require more proofreading:\nsummarisation --> summarization\nL030 raking --> ranking\n\n**Response:** Thank you for pointing these out. We will revise the paper with careful proofreading in the final version (for “summarisation,” we used UK spelling).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. It is an interesting idea to combine the token level quality estimation and lexically constrained decoding to improve the generation quality.\n\n2. Experiments on several tasks demonstrate the effectiveness of the proposed method.\n\n3. The paper is well organized and easy to follow, although more proofreading might be required.", "reasons_to_reject": "Although the author mentioned in the Limitations section that this method is more time-consuming compared to reranking, it is better to provide a time comparison in the paper for reference in subsequent studies.", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper proposes a novel post-processing method for text generation tasks.\nAfter beam search generation, this method is able to combine the high-quality parts in the N-best hypothesis and re-generate a better answer.\nExperiments on several text generation tasks show the effectiveness of this method.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "This paper might require more proofreading:\n\n1. summarisation --> summarization\n2. L030 raking --> ranking"}, "extra_text": {}}, {"review_id": "mqe7C5KJTn", "reviewer": "Reviewer_26qx", "summary": "", "questions": "Question A: I don't understand lines 126 to 128. If labels are assigned based on their appearance in references, why the same token in different hypotheses can have different labels?\n\nQuestion B: For DrNMT and NCD, did you search the beam size as you did for your method?", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate your detailed comments on our paper. First of all, we would like to clarify the process for constructing our QE model, which may be useful for addressing some of the misunderstandings. Our method *trains* token-level quality estimation model by fine-tuning RoBERTa, where the references of a training corpus are used to generate ground-truth labels. After training, the QE model *predicts* which tokens in an output candidate (of N-best) should be or should not be used in the final output referring to the context within. Therefore, our QE model is NOT dependent on nor constrained by references at inference time.\n\n\n**Reasons To Reject 1-1:** The proposed QE component is token-level, which can not take compositional semantics into consideration. \n\n**Response:** As our QE model is based on RoBERTa (see lines 138-140), it can take the entire sentence as context for prediction, which would allow considerations of compositional semantics to some extent, even if being imperfect. We totally agree that the consideration of compositional semantics is crucial direction and worth exploring, which constitutes our future work. We would like to note that NeuroLogic-A* allows inclusion of set constraints, and thus we can easily apply compositional constraints when they are ready.\n\n\n**Reasons To Reject 1-2:** And the label of each token is determined by whether it appears in the reference, which is quite rigid and can not take synonyms into consideration.\n\n**Response:** We respectfully point out that our QE model should acquire the ability to consider synonyms through both pre-training of RoBERTa and fine-tuning thereafter of which corpus is created from a large training corpus of the corresponding text generation task.\n\n\n**Reasons To Reject 2:** The beam size is searched for the proposed method (line 142), while the default setting is used for standard beam search (line 150).\n\n**Response:** We would clarify that the “default setting” was used for *both* of our method and the standard beam search for generating final outputs. What we tuned for our method was the size of N, which affects only the creation of lexical constraints by determining how many candidates to input to the QE model to generate lexical constraints (see lines 141-146).\n\n\n**Reasons To Reject 3:** Significance test is missing when \"our methods outperformed SELF-CORRECT\" (line 251) is claimed while only 0.1 point improvement is observed in table 4.\n\n**Response:** SELF-CORRECT (Welleck et al., 2023) was published on May 2023, which should be considered as contemporaneous to our submission (EMNLP2023 submission guideline regulates that such contemporaneous works should not be considered as targets for detailed comparisons that require additional experimentation and/or in-depth analysis). Furthermore, the unavailability of outputs released by the authors hindered us from conducting significance tests. While our NeuroLogic-A* (P) has 0.32 higher point to SELF-CORRECT, we will revise the corresponding description in the final version if the paper is accepted.\n\n\n**Questions For The Authors (Question A):** I don't understand lines 126 to 128. If labels are assigned based on their appearance in references, why the same token in different hypotheses can have different labels?\n\n**Response:** Lines 126-128 describe the process in inference time, where the labels are *predicted* by our QE model (remind that references are only used for training the QE model). Because our QE model is context-dependent, the same token appearing in different sentences of N-best may be predicted different labels.\n\n\n**Questions For The Authors (Question B):** For DrNMT and NCD, did you search the beam size as you did for your method?\n\n**Response:** Yes, we searched the beam size of these methods (the sizes of N)  in the same way as our method using validation sets. We will clarify this in the final version if the paper is accepted.\n\n\n**Typos Grammar Style And Presentation Improvements:** in line 030, raking -> ranking\n\n**Response:** Thank you for pointing this out. We will correct it in the camera-ready version.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The idea of combining N-best hypotheses is reasonable, and the proposed method is straightforward.\n\n2. Experiments are done on multiple generation tasks which demonstrate the generalization ability of the proposed method.", "reasons_to_reject": "1. The proposed QE component is token-level, which can not take compositional semantics into consideration. And the label of each token is determined by whether it appears in the reference, which is quite rigid and can not take synonyms into consideration. \n\n2. The beam size is searched for the proposed method (line 142), while the default setting is used for standard beam search (line 150).\n\n3. Significance test is missing when \"our methods outperformed SELF-CORRECT\" (line 251) is claimed while only 0.1 point improvement is observed in table 4.", "questions_to_authors": "Question A: I don't understand lines 126 to 128. If labels are assigned based on their appearance in references, why the same token in different hypotheses can have different labels?\n\nQuestion B: For DrNMT and NCD, did you search the beam size as you did for your method?", "paper_topic_and_main_contributions": "This paper is about how to combine N-best generations from beam search to construct the final generation.\n\nTo this end, they first train a token-level quality estimation (QE) module to predict whether each token in the N-best hypotheses should appear in the final generation. Then based on the outputs of this QE component, they use lexically constrained decoding to produce the final generation.\n\nExperiments on paraphrasing generation, summarization, and CommonGen demonstrate that their method improves the generation performance compared to beam search and a few other reranking baselines.\n\nThe contribution of this work is made toward the NLP engineering experiment direction.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "1. in line 030, raking -> ranking"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper presents an interesting approach that recycles parts of hypotheses in N-best lists produced via beam search to ultimately generate a higher-quality response. The method is evaluated on paraphrase generation, summarization, and commonsense generation and yields improved performance compared to beam search and other baselines. All reviewers praise the idea behind the method, as it is intuitive and clever to combine token-level quality estimation with constrained decoding. Criticisms of the approach include the computational expense (the authors provide some timing comparisons in their response to 8gyk, but they include no details on experimental/hardware settings so it is hard to draw meaningful conclusions from that). There were other concerns about experimetnal conditions (e.g. beam size) that were mostly resolved through discussion. Overall, it seems like a solid paper, but I would hope the authors provide a thorough analysis in the next version of how much more expensive this approach is in terms of time compared to normal beam search with a variety of batch sizes.
Decision: Accept-Main