PAEG : Phrase-level Adversarial Example Generation for Neural Machine Translation (2022)

Abstract: While end-to-end neural machine translation (NMT) has achieved impressive progress, noisy input usually leads models to become fragile and unstable. Generating adversarial examples as the augmented data has been proved to be useful to alleviate this problem. Existing methods for adversarial example generation (AEG) are word-level or character-level, which ignore the ubiquitous phrase structure. In this paper, we propose a Phrase-level Adversarial Example Generation (PAEG) framework to enhance the robustness of the translation model. Our method further improves the gradient-based word-level AEG method by adopting a phrase-level substitution strategy. We verify our method on three benchmarks, including LDC Chinese-English, IWSLT14 German-English, and WMT14 English-German tasks. Experimental results demonstrate that our approach significantly improves translation performance and robustness to noise compared to previous strong baselines.

## introduction recently, neural machine translation (nmt) has effectively improved translation quality. nmt has shown state-of-the-art performance for many language pairs @xcite @xcite . various architectures @xcite @xcite bring many appealing properties. most nmt systems heavily rely on high-quality parallel data and perform poorly in noisy input. with the noise rising in the source sentence, nmt tends to be more vulnerable @xcite , due to the output prediction of the decoder easily intervened by the other words @xcite . a slight disturbance like a random permutation can damage the translation quality dramati- * equal contribution. ## phrase-level adversarial example generation in this section, we formulate the problem of adversarial example generation mathematically. first, our proposed method provides reliable candidates with the pre-trained model. then, we use the gradient-based method to select vulnerable positions and substitutes at phrase level to generate adversarial examples. these examples are used as augmented data for the training of the nmt model. to further improve the performance, we extend our method to the bidirectional generation. ## related work adversarial training for neural networks has been studied recently @xcite . similar ideas are applied into natural language processing @xcite @xcite @xcite @xcite @xcite @xcite . specifically, adversarial example generation @xcite @xcite @xcite @xcite @xcite is proved to be useful to train a robust nmt system. recently, @xcite adopted a gradient-based method to craft adversarial examples at word level, using the adversarial source input to attack while the target input to defend the model. our bidirectional generation method is similar to multilingual nmt training. multilingual nmt models @xcite @xcite @xcite @xcite are trained over multiple language pairs with parameter sharing, such as using the same encoder/decoder for different source/target languages @xcite , using one encoder and separate decoders to translate one language to multiple languages @xcite , and sharing an attention mechanism @xcite across multiple language pairs. in this work, we use the adversarial examples generated from the other direction to improve the robustness of the original translation direction. ## conclusion in this work, we propose a phrase-level adversarial example generation method. our goal is to improve the fluency of the adversarial examples. we improve a gradient-based word-level method with phrase-level candidate construction, overall substi-tution strategy, and bidirectional generation. we verify our method on chinese-english, german-english, and english-german corpus, and the results show that paeg can improve both translation quality and robustness to noisy inputs significantly.