Doubly-Trained Adversarial Data Augmentation for Neural Machine Translation (2022)

Abstract: Neural Machine Translation (NMT) models are known to suffer from noisy inputs. To make models robust, we generate adversarial augmentation samples that attack the model and preserve the source-side meaning at the same time. To generate such samples, we propose a doubly-trained architecture that pairs two NMT models of opposite translation directions with a joint loss function, which combines the target-side attack and the source-side semantic similarity constraint. The results from our experiments across three different language pairs and two evaluation metrics show that these adversarial samples improve model robustness.

## introduction when nmt models are trained on clean parallel data, they are not exposed to much noise, resulting in poor robustness when translating noisy input texts. various adversarial attack methods have been explored for computer vision @xcite including fast gradient sign methods @xcite and generative adversarial networks (gan; @xcite , among others. most of these methods are white-box attacks where model parameters are accessible during the attack so that the attack is much more effective. good adversarial samples could also enhance model robustness by introducing perturbation as data augmentation @xcite . due to the discrete nature of natural languages, most of the early-stage adversarial attacks on nmt focused on black-box attacks (attacks without access to model parameters) and use ‡ work done while at johns hopkins university. ## related work natural and synthetic noise affects translation performance @xcite and adversarial perturbation is commonly used to evaluate and improve model robustness in such cases. various adversarial methods are researched for robustness, some use adversarial samples as regularization @xcite , some incorporate it with reinforcement learning @xcite , and some use it for data augmentation. when used for augmentation, black-box adversarial methods tend to augment data by introducing noise into training data. for most of the time, simple operations such as random deletion/replacement/insertion are used for black-box attack @xcite , though such operations can be used as white-box attack with gradients as well @xcite ). it's also possible to guide adversarial samples' search with pretrained models in black-box attack @xcite . most white-box adversarial methods use different architecture to attack and update model @xcite @xcite , and from which, generate augmented data. white-box adversarial methods gives more flexible modification for the token but at the same time become time consuming, making it infeasible for some cases when speed matters. though it is commonly believed that white-box adversarial methods have higher capacity, there is study that shows simple replacement can be used as an effective and fast alternative to white-box methods where it achieves comparable (or even better) results for some synthetic noise @xcite . this finding correlates with our research to some degree because we also find replacement useful to improve model robustness, though we perform replacement by most similar token instead of sampling a random token. hence mrt allows an arbitrary scoring function ∆ to be used, whether it is differentiable or not. in our experiments, we use mrt with two metrics, bleu @xcite -the standard in machine translation and comet (rei et al., 2020)-a newly proposed neural-based evaluation metric that correlates better with human judgement. adversarial attack adversarial attacks generate samples that closely match input while dramatically distorting the model output. the samples can be generated by either a white-box or a black-box model. black-box methods do not have access to the model while white-box methods have such access. a set of adversarial samples are generated by: ## approach: doubly trained nmt for adversarial sample generation we aim to generate adversarial samples that both preserve input's semantic meaning and decrease the performance of an nmt model. we propose a doubly-trained system that involves two models of opposite translation direction (denote the forward model as θ st and the backward model as θ ts ). our algorithm will train and update θ st , θ ts simultaneously. note that both models are pretrained before they are used for adversarial augmentation so that they can already produce good translations. our algorithm has three steps as shown in figure 1 . encoder decoder model (forward) shared embedding(src) embedding(tgt) encoder model (backward) embedding(tgt) source sentences target sentences step1: constrained attack embedding (src) updated embedding (src) from step 1 source tokens nearest neighbor search adversarial source tokens step2: generate adversarial tokens model (forward) step3: regular training on adversarial tokens (as augmented data) random deletion cross entropy decoder shared embedding (src) target sentences source sentences source-side similarity (l2) target-side attack (l1) @xmath0. we train the models with negative log-likelihood (nll) or mrt and combine the loss from two models as our final loss function to update the shared embedding. we denote the loss for θ st as l 1 and loss for θ ts as l 2 . because we want to attack the forward model and preserve translation quality for the backward model, we make our final loss we also experimented with mrt in our doubly-trained system to investigate if using sentence-level scoring functions like bleu or comet would help improve adversarial samples' quality. for model θ st , we feed in source sentences x (s) and we infer a set of possible translation s(x (s) ) as the subset of full sample space. the loss (risk) of our prediction is therefore calculated as: the value α here controls the sharpness of the formula and we follow @xcite to use α = 5e -3 throughout our experiments. to sample the subset of full inference space s(x (s) ), we use sampling algorithm @xcite to generate k translation candidates for each input sentence (during inference time, the model outputs a probabilistic distribution over the vocabulary for each token and we sample a token based on this distribution). it is denoted as sample(x (s) , θ, k) in our algorithm 1. similarly, for model θ ts , we feed in the reference sentences of our parallel data and generate a set of possible translation s(y (s) ) in source language. we compute the loss (risk) of source-side similarity as: to be updated from both risks, two models need to share some parameters since l 1 only affects θ st and l 2 only updates θ ts . because a word embedding is a representation of input tokens, we make it such that the source-side embeddings of θ st and the target-side embeddings of θ ts are shared. we do so because they are both representations of source language in our translation and we can use it to generate adversarial tokens for source sentences in step 2. we also freeze all other layers in two models. thus, when we update the model parameter θ st , θ ts , we only update the shared embedding of source language. the process described above is summarized in algorithm 1. algorithm 1 update model embedding input: pretrained models θ st and θ ts , max number of epochs e, sample size k, sentence-level scoring metric m output: updated models θ st and θ ts (only the shared embedding is updated) while θ st , θ ts not converged and @xmath1, y (i) ), 1 < @xmath2 step 2 -perturb input sentences to generate adversarial tokens after updating the shared embedding, we can use the updated embedding to generate adversarial tokens. we introduce two kinds of noise into input sentences to generate adversarial samples: random deletion and simple replacement. to generate adversarial tokens (due to the discrete nature of natural languages), we use cosine similarity. let model embedding be e before the embedding update, and e ′ after the update from algorithm 1. let the vocab be v and let input sentence be @xmath3, s 2 , • • • s n } for each token s @xmath4, s i / ∈ {eos, bos, pad}, three actions are possible: throughout our experiments, we set the hyper-parameters as p @xmath5.7, p @xmath6.8, p @xmath7.2. that means each token has 30 percent chance to be perturbed, and if that's the case, it has 80 percent chance to be replaced by a similar token and 20 percent chance to be deleted. for no-perturbation or deletion case, it's straightforward to implement. for replacement, we compute s ′ i (the adversarial token of s i ) by cosine similarity: for the credibility of this hyper-parameter setup, we perform a grid search over 9 possible combinations: ## conclusion we proposed a white-box adversarial augmentation algorithm to improve model robustness. we use a doubly-trained system to perform constrained attack and then train the model on adversarial samples generated with random deletion and gradient-based replacement. experiments across different languages and evaluation metrics have shown consistent improvement for model robustness.