High Quality Rather than High Model Probability: Minimum B ayes Risk Decoding with Neural Metrics (2022)

Abstract: In Neural Machine Translation, it is typically assumed that the sentence with the highest estimated probability should also be the translation with the highest quality as measured by humans. In this work, we question this assumption and show that model estimates and translation quality only vaguely correlate. We apply Minimum Bayes Risk (MBR) decoding on unbiased samples to optimize diverse automated metrics of translation quality as an alternative inference strategy to beam search. Instead of targeting the hypotheses with the highest model probability, MBR decoding extracts the hypotheses with the highest estimated quality. Our experiments show that the combination of a neural translation model with a neural reference-based metric, Bleurt, results in significant improvement in human evaluations. This improvement is obtained with translations different from classical beam-search output: These translations have much lower model likelihood and are less favored by surface metrics like Bleu.

## introduction neural sequence-to-sequence models constitute the state-of-the-art for machine translation. these models estimate the probability of a target sentence given a source sentence. at inference, it is commonplace to approximate the maximum-aposteriori (map) hypothesis with beam search in order to output a sentence with (close to) the highest probability given the provided source. this strategy assumes that the sentences with the highest estimated probabilities should also be the translations with the highest quality as measured by humans. this assumption can be questioned based on two observations: (i) neural machine translations (nmts) generated by beam search are ranked below human translations in professional evaluations @xcite while (ii) the nmt model itself considers human translations much less likely than its beam outputs @xcite . these observations clearly show that estimated probability and translation quality do not always correlate. an example is given in these observations do not in themselves suggest an alternative to likelihood in selecting better hypotheses. for that, we look at recent progress in automated evaluation. recently introduced utility metrics, such as bleurt @xcite or comet @xcite , estimate human judgments u(h, r) from a candidate translation h and a reference human translation r with a neural network. these learned metrics have shown higher correlation with human judgments compared with traditional metrics based on lexical overlap such as bleu @xcite and meteor @xcite . bleurt and comet have also been shown by the wmt metric task @xcite to perform better than yisi @xcite , which measures overlap in a neural embedding space. bleurt and comet are able to evaluate hypotheses with different word choices, sentence structures, and lengths compared to the reference translations. unlike overlap-based metrics like bleu, these metrics do not necessarily prefer the most likely tokens to increase the chance of covering n-grams in the reference translations @xcite . when comparing a model output h and an alternative human reference r , bleu and bleurt behave differently. while bleu often estimates the quality of the model output h to be much higher than the alternative human translation r (bleu(h, r) > bleu(r , r)), bleurt and comet typically prefer the human translation over the mt output (bleurt(h, r) < bleurt(r , r) human translations obtain a low model estimated probability (logp) as they do not generate the most frequent and direct translation. this behavior generally agrees with professional raters @xcite . these observations suggest that selecting model hypotheses likely to have a high quality score with respect to learned neural utility metrics should bring the quality of mt output closer to that of human translations. for that, we rely on minimum bayes risk (mbr) decoding, in particular the sampling-based approximation recently introduced by @xcite . sampling-based mbr starts with a set of unbiased samples drawn from an nmt model and finds the candidate which has the highest average utility when each hypothesis in the set is used as a pseudo-reference. this mbr strategy has several potential pitfalls. first, the expectation of utility under the model distribution is used as a proxy to the expectation under the true underlying (human translator) distribution. this means that a high divergence between these two distributions will affect mbr (pitfall 1: model quality). second, the utility metric might be unreliable in areas of the space where it has not been evaluated (e.g., with low quality, low probability pseudo-references). this might cause its expectation to be very different from single point evaluations with high quality human references (pitfall 2: utility validity over the reference space). third, even if mbr discovers hypotheses with high utility with respect to actual human references, there is no guarantee that these hypotheses will receive high human judgments because these hypotheses are not necessarily close to the conditions for which the utility metrics have been designed (pitfall 3: utility validity over the hypothesis space). this paper evaluates mbr decoding for multiple utility functions and measures whether their predictions indeed improve the actual utility with respect to human references. we show that an nmt model based on the transformer-big architecture and bleu, chrf, yisi, and bleurt successfully avoid pitfalls 1 and 2. we also study the robustness of these conclusions with respect to the number of considered samples and model size. we then conduct a human evaluation of mbr hypotheses with high estimated utility according to different metrics to assess pitfall 3. we show that mbr decoding using bleu as a utility metric slightly improves over beam search decoding, even though the difference between these two translations are minor. in contrast, mbr using bleurt as a utility metric generates translations further away from beam output. these translations are given significantly higher human quality ratings compared with beam search and the other mbr hypotheses. our contributions are: after the advent of neural machine translation @xcite , most methods relied on beam search to approximate map decoding @xcite @xcite . the question of optimizing utility metrics of interest such as bleu was also explored. approaches based on structured risk minimization @xcite or reinforcement learning @xcite considered modifying the training procedure. mbr decoding has recently gained attention in mt as a decision rule with the potential to overcome some of the biases of map decoding in nmt @xcite @xcite . while most prior work on mbr decoding for mt is based on k-best lists obtained via beam search, @xcite proposed to use an approximation of mbr decoding based on unbiased sampling to overcome the shortcomings of map decoding. they demonstrated that samples from the nmt model are faithful to the training data statistics, while beam search is not. we adopt their sampling-based mbr decoding approximation in all our experiments. the application of mbr to neural mt has focused on maximizing classical overlap-based metrics like bleu, meteor, chrf, or beer @xcite . our work builds upon recent advances in the automatic evaluation of mt @xcite , which has shown the emergence of learned utility metrics based on neural networks. we consider using neural metrics for mbr, which has not been done before. these metrics are neural networks that consider a pair of sentences (a hypothesis, and a reference) or a triplet of sentences (a source, a hypothesis and a reference) and output a real-valued score estimating the quality of the hypothesis. they rely on pre-trained monolingual or multilingual neural language models. the first generation of neural utility metrics uses neural models to extract pre-trained sentence and word representations to compute distances indicative of semantic proximity, for example, bertscore and yisi @xcite . later, a second generation of neural utilities proposed to fine-tune neural models on human judgments, either through regression or ranking tasks. these approaches, such as bleurt and comet @xcite , have shown better correlation with human judgments @xcite . ## experimental results in this section, we discuss the main results of our study. first, we look into the automatic scores to investigate if mbr results in higher actual utility scores when estimating the expectation of the same utility. second, we look into the human evaluation results to investigate how well the improvements in utility scores can transfer to human judgments. ## ablation we run ablation experiments to better understand the properties of mbr. we will mostly focus on experiments for @xmath0. ## conclusion we explored an alternative to the commonly used beam search decoding algorithm typically used in nmt. we run the sampling-based approximation of minimum bayes risk (mbr) decoding to optimize bleu, chrf, yisi, and bleurt. our experimental results showed that mbr decoding using bleurt as utility function results in translations that significantly outperform beam search decoding based on expert-based human evaluation. we showed that the resulting translations are significantly different from both the beam search decode and mbr decoding output using one of the other overlap-based metrics as utility function, and have a lower model probability.