Quality-Aware Decoding for Neural Machine Translation (2022)

Abstract: Despite the progress in machine translation quality estimation and evaluation in the last years, decoding in neural machine translation (NMT) is mostly oblivious to this and centers around finding the most probable translation according to the model (MAP decoding), approximated with beam search. In this paper, we bring together these two lines of research and propose quality-aware decoding for NMT, by leveraging recent breakthroughs in reference-free and reference-based MT evaluation through various inference methods like N -best reranking and minimum Bayes risk decoding. We perform an extensive comparison of various possible candidate generation and ranking methods across four datasets and two model classes and find that quality-aware decoding consistently outperforms MAP-based decoding according both to state-of-the-art automatic metrics (COMET and BLEURT) and to human assessments.

## introduction the most common procedure in neural machine translation (nmt) is to train models using maximum likelihood estimation (mle) at training time, and to decode with beam search at test time, as a way to approximate maximum-a-posteriori (map) decoding. however, several works have questioned the utility of model likelihood as a good proxy for translation quality @xcite @xcite . in parallel, significant progress has been made in methods for quality estimation and evaluation of generated translations @xcite , but this progress is, by and large, not yet reflected in either training or decoding methods. exceptions such as minimum risk training @xcite * equal contribution. figure 1 : quality-aware decoding framework. first, translation candidates are generated according to the model. then, using reference-free and/or referencebased mt metrics, these candidates are ranked, and the highest ranked one is picked as the final translation. an appealing alternative is to modify the decoding procedure only, separating it into two stages: candidate generation ( §2.1; where candidates are generated with beam search or sampled from the whole distribution) and ranking ( §2.2; where they are scored using a quality metric of interest, and the translation with the highest score is picked). this strategy has been explored in approaches using n -best reranking @xcite and minimum bayes risk (mbr) decoding @xcite @xcite . while this previous work has exhibited promising results, it has mostly focused on optimizing lexical metrics such as bleu or meteor @xcite , which have limited correlation with human judgments @xcite . moreover, a rigorous apples-to-apples comparison among this suite of techniques and their variants is still missing, even though they share similar building blocks. our work fills these gaps by asking the question: to answer this question, we systematically explore nmt decoding using a suite of ranking procedures. we take advantage of recent state-of-theart learnable metrics, both reference-based, such as comet and bleurt @xcite , and reference-free (also known as quality estimation; qe), such as transquest and openkiwi @xcite . we compare different ranking strategies under a unified framework, which we name quality-aware decoding ( §3). first, we analyze the performance of decoding using n -best reranking, both fixed according to a single metric and learned using multiple metrics, where the coefficients for each metric are optimized according to a reference-based metric. second, we explore ranking using reference-based metrics directly through mbr decoding. finally, to circumvent the expensive computational cost of the latter when the number of candidates is large, we develop a two-stage ranking procedure, where we use n -best reranking to pick a subset of the candidates to be ranked through mbr decoding. we explore the interaction of these different ranking methods with various candidate generation procedures including beam search, vanilla sampling, and nucleus sampling. experiments with two model sizes and four datasets ( §4) reveal that while map-based decoding appears competitive when evaluating with lexical-based metrics (bleu and chrf), the story is very different with state-of-the-art evaluation metrics, where quality-aware decoding shows significant gains, both with n -best reranking and mbr decoding. we perform a human-study to more faithfully evaluate our systems and find that, while performance on learnable metrics is not always predictive of the best system, quality-aware decoding usually results in translations with higher quality than map-based decoding. ## candidate generation and ranking we start by reviewing some of the most commonly used methods for both candidate generation and ranking under a common lens. ## quality-aware decoding while recent works have explored various combinations of candidate generation and ranking procedures for nmt @xcite @xcite , they suffer from two limitations: although these metrics are well established and inexpensive to compute, they correlate poorly with human judgments at segment level @xcite ). in this work, we hypothesize that using more powerful metrics in the ranking procedure may lead to better quality translations. we propose a unified framework for ranking with both reference-based ( §3.1) and reference-free metrics ( §3.2), independently of the candidate generation procedure. we explore four methods with different computational costs for a given number of candidates, n . fixed n -best reranker. an n -best reranker using a single reference-free metric ( §3.2) as a feature, according to eq. 2. the computational cost of this ranker is o(n ×c m qe ), where c m qe denotes the cost of running an evaluation with a metric m qe . tuned n -best reranker. an n -best reranker using as features all the reference-free metrics in §3.2, along with the model log-likelihood log p θ (y|x). the weights in eq. 3 are optimized to maximize a given reference-based metric m ref using mert @xcite , a coordinate-ascent optimization algorithm widely used in previous work. note that m ref is used for tuning only; at test time, only reference-free metrics are used. therefore, the decoding cost is o(n × i c m qe i ). ## conclusions and future work we leverage recent advances in mt quality estimation and evaluation and propose quality-aware decoding for nmt. we explore different candidate generation and ranking methods, with a comprehensive empirical analysis across four datasets and two model classes. we show that, compared to mapbased decoding, quality-aware decoding leads to better translations, according to powerful automatic evaluation metrics and human judgments. there are several directions for future work. our ranking strategies increase accuracy but are substantially more expensive, particularly when used with costly metrics such as bleurt and comet. while reranking-based pruning before mbr decoding was found helpful, additional strategies such as caching encoder representations @xcite and distillation @xcite are promising directions.