Self-Ensemble of N -best Generation Hypotheses by Lexically Constrained Decoding (2023)

Abstract: We propose a method that ensembles N -best hypotheses to improve natural language generation. Previous studies have achieved notable improvements in generation quality by explicitly reranking N -best candidates. These studies assume that there exists a hypothesis of higher quality. We expand the assumption to be more practical as there exist partly higher quality hypotheses in the N -best yet they may be imperfect as the entire sentences. By merging these high-quality fragments, we can obtain a higher-quality output than the single-best sentence. Specifically, we first obtain N -best hypotheses and conduct token-level quality estimation. We then apply tokens that should or should not be present in the final output as lexical constraints in decoding. Empirical experiments on paraphrase generation, summarisation, and constrained text generation confirm that our method outperforms the strong N -best reranking methods.

## introduction while the beam search is one of the most common decoding methods in natural language generation, it suffers from the beam search curse @xcite @xcite where a large beam size degrades the quality of generation. as a remedy to this problem, previous studies explored better alternatives from n -best hypotheses @xcite as represented as reranking and minimum bayes decoding @xcite , which only modify the decoding procedures. there are two types of reranking approaches. discriminative methods train rerankers to predict specific evaluation metric scores of each hypothesis @xcite @xcite . in contrast, generative methods use generic rerankers that have been used for other purposes, such as language models @xcite . different from methods that involve computationally expensive model training such as the minimum risk training @xcite , these ranking-based methods are efficient and easily applicable to trained models. nonetheless, these reranking methods assume that there is a single hypothesis of higher quality in the n -best, which may not be practical depending on the generation model and also inputs. therefore, we enhance the assumption; there should be candidates that are partly high-quality but may be imperfect as the entire sentences. our method identifies and merges these higher-quality fragments to derive a high-quality output using lexically constrained decoding @xcite . specifically, our method trains a token-level quality estimator that predicts whether a token in a hypothesis should be or should not be included in the final output. it then uses the quality estimation (qe) results of the n -best hypotheses to compose positive and negative lexical constraints and generates the final output using the generation model. as a contribution of this study, we propose the n -best ensembling method for improving the quality of language generation, which is easy to apply to a variety of language generation tasks. empirical experiments on paraphrasing @xcite , summarisation @xcite @xcite , and constrained text generation @xcite confirm that our assumption holds and the proposed method outperforms strong reranking-based methods. ## preliminary: lexically constrained decoding lexically constrained decoding has been employed in various language generation tasks to apply taskspecific knowledge on generation, e.g., for machine translation using a bilingual dictionary of technical terms as constraints @xcite , for text simplification using difficult words as constraints @xcite @xcite , for style transfer using style-specific vocabulary as constraints @xcite , and for table-to-text generation using keywords in tables as constraints @xcite . different from these studies that assume the availability of task-specific knowledge, our method creates lexical constraints based purely on the n -best hypotheses of language generation. we use the state-of-the-art lexically constrained decoding method, namely, neurologic-a* @xcite . neurologic-a* searches for output candidates with high generation probabilities and constraint satisfaction rates by tracking states of satisfaction by the following steps. (1) for each candidate token at a time step, neurologic-a* looks ahead to future tokens to be generated. (2) based on the look-ahead results, it computes satisfaction rates of lexical constraints and prunes the candidate tokens. (3) it groups the remaining candidates based on the states of the constraint satisfaction and selects output tokens from the best candidates in each group to preserve a broad search space. ## proposed method figure 1 shows the overview of the proposed method. the main component is the token-level qe model that predicts whether each token in nbest hypotheses should be used or avoided in generating the final output. tokens predicted as the former is included in positive constraints and those predicated as the latter are included in negative constraints to be considered by neurologic-a*. specifically, we fine-tune a pretrained masked language model to conduct binary token classification as illustrated in figure 2 . for each nbest hypothesis of training sentences obtained by the language generation model, token-level labels are automatically assembled using their references. hypothesis tokens appearing in the corresponding reference are labelled as positive and otherwise labelled as negative. at inference, the qe model takes the concatenation of a source and a hypothesis in the n -best as input and conducts binary classification for each token. we expect that the masked language model acquires the sense of synonyms and multi-word expressions through pre-training and transfers that knowledge to our token-level qe. because the token-level qe is contextdependent, the same token appearing in different hypotheses may be predicted both positive and negative labels, respectively. our model determines the final label by majority voting. if the numbers of positive and negative predictions are tie, the corresponding token is excluded from lexical constraints. ## experimental settings the proposed method is widely applicable to language generation tasks. we thus evaluate it on paraphrasing ( § 5.1), summarisation ( § 5.2), and constrained text generation ( § 5.3). proposed method for each evaluation dataset ( baselines as the most basic baseline, we compared our method to beam search, of which size was borrowed from previous studies that tuned the value for underlying language generation models for each task. wherever applicable, we also compared the strong discriminative and generative reranking methods. for the former, we used the discriminative reranker for neural machine translation (drnmt) @xcite foot_0 that predicts the distribution of sentence-level evaluation metric scores given the source and n -best hypotheses. for the latter, we used the noisy-channel decoding (ncd) @xcite foot_1 that scores n -best candidates by linearly combining the probabilities of generation, target-side language model, and targetto-source generation. we trained these methods using the authors' implementations with datasets and metrics of each experiment task, where the beam sizes (the sizes of n ) were searched in the same way as our method using the validation sets. ablation as ablation studies, we evaluated the performance of the proposed method using only positive lexical constraints (denoted as neurologic-a* (p)), only negative lexical constraints (denoted as neurologic-a* (n)), and both (denoted as neurologic-a* (p & n)). ## experimental results this section discusses the experimental results. the details of implementation on each task are described in appendix b. ## discussion and future work as we discussed in § 5, the quality of token-level qe is critical for the performance of our method.