Jam or Cream First? Modeling Ambiguity in Neural Machine Translation with SCONES (2022)

Abstract: The softmax layer in neural machine translation is designed to model the distribution over mutually exclusive tokens. Machine translation, however, is intrinsically uncertain: the same source sentence can have multiple semantically equivalent translations. Therefore, we propose to replace the softmax activation with a multi-label classification layer that can model ambiguity more effectively. We call our loss function Single-label Contrastive Objective for Non-Exclusive Sequences (SCONES). We show that the multi-label output layer can still be trained on single reference training data using the SCONES loss function. SCONES yields consistent BLEU score gains across six translation directions, particularly for medium-resource language pairs and small beam sizes. By using smaller beam sizes we can speed up inference by a factor of 3.9x and still match or improve the BLEU score obtained using softmax. Furthermore, we demonstrate that SCONES can be used to train NMT models that assign the highest probability to adequate translations, thus mitigating the “beam search curse”. Additional experiments on synthetic language pairs with varying levels of uncertainty suggest that the improvements from SCONES can be attributed to better handling of ambiguity.

## introduction conventional neural machine translation (nmt) models learn the probability p (y|x) of the target sentence y given the source sentence x @xcite . this framework implies that there is a single best translation for a given source sentence: if there were multiple valid translations y 1 and y 2 they would need to share probability mass (e.g. p (y 1 |x) = 0.5 and p (y 2 |x) = 0.5), but such a distribution could also represent model uncertainty, i.e. the case when either y 1 or y 2 are correct translations. therefore, learning a single distribution over all target language sentences does not allow the model to naturally express intrinsic uncertainty foot_1 @xcite @xcite , the nature of the translation task to allow multiple semantically equivalent translations for a given source sentence. a single distribution over all sequences represents uncertainty by assigning probabilities, but it cannot distinguish between different kinds of uncertainty (e.g. model uncertainty versus intrinsic uncertainty). therefore, in this work we frame machine translation as a multi-label classification task @xcite . rather than learning a single distribution p (y|x) over all target sentences y for a source sentence x, we learn binary classifiers for each sentence pair (x, y) that indicate whether or not y is a valid translation of x. in this framework, intrinsic uncertainty can be represented by setting the probabilities of two (or more) correct translations y 1 and y 2 to 1 simultaneously. the probabilities for each translation are computed using separate binary classifiers, and thus there is no requirement that the probabilities sum to one over all translations. in practice, the probability of a complete translation is decomposed into a product of the token-level probabilities. thus we replace the softmax output layer in transformer models @xcite with sigmoid activations that assign a probability between 0 and 1 to each token in the vocabulary at each time step. we propose a loss function, single-label contrastive objective for non-exclusive sequences (scones) that allows us to train our models on single reference training data. our work is inspired by noise-contrastive es-timation (nce) @xcite . unlike nce, whose primary goal was to efficiently train models over large vocabularies, our motivation for scones is to model non-exclusive outputs. we demonstrate multiple benefits of training nmt models using scones when compared to standard cross-entropy with regular softmax. we report consistent bleu score gains between 1%-9% across six different translation directions. scones with greedy search typically outperforms softmax with beam search, resulting in inference speed-ups of up to 3.9x compared to softmax without any degradation in bleu score. scones can be tuned to mitigate some of the pathologies of traditional nmt models. softmaxbased models have been shown to assign the highest probability to either empty or inadequate translations (modes) @xcite . this behavior manifests itself as the "beam search curse" @xcite : increasing the beam size may lead to worse translation quality. we show that scones can be used to train models that a) assign the highest probability to adequate translations and b) do not suffer from the beam search curse. finally, we use scones to train models on synthetic translation pairs that we generate by sampling from the ibm model 3 @xcite . by varying the sampling temperature, we control the level of ambiguity in the language pair. we show that scones is effective in improving the adequacy of the highest probability translation for highly ambiguous translation pairs, confirming our intuition that scones can handle intrinsic uncertainty well. ## training nmt models with scones we denote the (subword) vocabulary as @xmath0, . . . , w |v| }, the special end-of-sentence symbol as w 1 = 
, the source sentence as @xmath1, . . . , x |x| ∈ v * , a translation as @xmath2, . . . , y |y| ∈ v * , and a translation prefix as @xmath3, . . . , y i . we use a center dot "•" for string concatenations. unlike conventional nmt that models a single distribution p (y|x) over all target language sentences, scones learns a separate binary classifier for each sentence pair (x, y). we define a boolean function t(•, •) that indicates whether y is a valid translation of x: t(x, y) := true if y is a translation of x false otherwise . l scones (x, y, i), (5) where l scones (x, y, i) = l + (x, y, i) + αl -(x, y, i). l -(•) is designed to reduce the probability p (z x,@xmath4, @xmath5: appendix c provides an implementation of scones in jax @xcite . during inference we search for the translation y * that ends @xmath6: relation to noise-contrastive estimation our scones loss function is related to noisecontrastive estimation (nce) @xcite because both methods reformulate next word prediction as a multi-label classification problem, and both losses have a "positive" component for the gold label, and a "negative" component for other labels. 4 unlike nce, the negative loss component (l -(•)) in scones does not require sampling from a noise distribution as it makes use of all tokens in the ## experimental setup in this work our focus is to compare nmt models trained with scones with well-trained standard softmax-based models. thus we keep our setup simple, reproducible, and computationally economical. we trained transformer models ( ## experiments with synthetic language pairs our main motivation for scones is to equip the model to naturally represent intrinsic uncertainty, i.e. the existence of multiple correct target sentences for the same source sentence. to examine the characteristics of scones as a function of uncertainty, we generated synthetic language pairs that differ by the level of ambiguity. for this purpose, we trained an ibm-3 model @xcite on the german-english training data after subword segmentation using mgiza @xcite . ibm-3 is a generative symbolic model that describes the translation process from one language into another with a generative story, and was popular for finding word alignments for statistical (phrase-based) machine translation @xcite . the generative story consists of different steps such as distortion (word reordering), fertility (1:n word mappings), and lexical translation (word-to-word translation) that describe the translation process. the parameters of ibm-3 define probability distributions for each step. in this work we do not use ibm-3 for finding word alignments. instead, for the original german sentences we sample synthetic english-like translations from the model with different sampling temperatures to control the ambiguity levels of the translation task. a low sampling temperature generates sentence pairs that still capture some of the characteristics of mt such as word reorderings, but the mapping is mostly deterministic (i.e. the same source token is almost always translated to the same target token). a high temperature corresponds to more randomness, i.e. more intrinsic uncertainty. appendix b contains more details about sampling from ibm-3. we train nmt models using either softmax or scones on the synthetic corpora. temperature translation tasks are less predictable, and thus lead to lower bleu scores. the difference between both approaches becomes clear with exact search (fig. 9 ). while the translations with the global highest probability for high ibm-3 sampling temperatures are heavily degraded for softmax and scones with α = 1, the drop is much less dramatic for scones with α = 0.2 (solid purple curve). setting α to a low value enables the model to assign its highest probability to adequate translations, even when the translation task is highly uncertain. ## related work our approach draws insights from multi-label classification (mlc) @xcite @xcite . one of the earliest approaches for mlc was to transform the problem into multiple binary classification problems while ignoring the correlations between labels @xcite . more recent work has modeled mlc in the sequence-to-sequence framework with a decoder that generates the labels sequentially, thus preserving the inter-label correlations @xcite . most prior work in mlc focuses on classification and is not directly applicable to mt. in contrast, our training strategy is tailored for sequence-to-sequence problems. unlike prior work @xcite , scones allows us to perform mlc style training with any underlying nmt architecture by simply changing the loss function. by jointly training all label-specific binary classifiers, our strategy is able to account for label correlations. @xcite used an mlc objective to improve machine translation. unlike our approach, they attempted to predict all words in the target sentence with a bag-of-words loss function. we formulate the next word prediction at each time step as an mlc problem to handle intrinsic uncertainty, but our models are predicting ordered target sequences, not bags of words. the speed-ups from scones can be partially attributed to avoiding the normalization of the output over the full vocabulary. the same idea motivated earlier work on self-normalized training @xcite @xcite . as described in sec. 2, unlike work on self-normalization, scones does not try to approximate a distribution over the full vocabulary. rather, its output consists of multiple binary classifiers that do not share probability mass by design to be able to better represent intrinsic uncertainty. ## conclusion machine translation is a task with high intrinsic uncertainty: a source sentence can have multiple valid translations. we demonstrated that nmt models and specifically transformers, can learn to model mutually non-exclusive target sentences from single-label training data using our scones loss function. rather than learn a single distribution over all target sentences, scones learns multiple binary classifiers that indicate whether or not a target sentence is a valid translation of the source sentence. scones yields improved translation quality over conventional softmax-based models for six different translation directions, or (alternatively) speed-ups of up to 3.9x without any degradation in translation performance. we showed that scones can be tuned to mitigate the beam search curse and the problem of inadequate and empty modes in standard nmt. our experiments on synthetic language translation suggest that, unlike softmaxtrained models, scones models are able to assign their highest probability to adequate translations even when the underlying task is highly ambiguous. the scones loss function is easy to implement. adapting standard softmax-based sequenceto-sequence architectures such as transformers requires only replacing the cross-entropy loss function with scones and the softmax with sigmoid activations. the remaining parts of the training and inference pipelines can be kept unchanged. scones can be potentially useful in handling uncertainty for a variety of ambiguous nlp problems beyond translation, such as generation and dialog. we expect this work to encourage research on modeling techniques that can address ambiguity in much better ways compared to current models.