A Reinforced Generation of Adversarial Examples for Neural Machine Translation (2020)

Abstract: Neural machine translation systems tend to fail on less decent inputs despite its significant efficacy, which may significantly harm the credibility of these systems—fathoming how and when neural-based systems fail in such cases is critical for industrial maintenance. Instead of collecting and analyzing bad cases using limited handcrafted error features, here we investigate this issue by generating adversarial examples via a new paradigm based on reinforcement learning. Our paradigm could expose pitfalls for a given performance metric, e.g., BLEU, and could target any given neural machine translation architecture. We conduct experiments of adversarial attacks on two mainstream neural machine translation architectures, RNN-search, and Transformer. The results show that our method efficiently produces stable attacks with meaning-preserving adversarial examples. We also present a qualitative and quantitative analysis for the preference pattern of the attack, demonstrating its capability of pitfall exposure.

## introduction neural machine translation (nmt) based on the encoder-decoder framework, such as rnn-search @xcite or transformer @xcite , has achieved remarkable progress and become a de-facto in various machine translation applications. however, there are still pitfalls for a well-trained neural translation system, especially when applied to less decent real-world inputs compared to training data @xcite . for example, typos may severely deteriorate system outputs ( intuitively, researchers seek to apprehend such failures by the analysis of handcrafted error indicating features @xcite . this strategy is costly because it requires expert knowledge for both linguistics and the target neural architecture. such features are also less applicable because some common errors in deep learning systems are hard to formulate, or very specific to certain architectures. instead of designing error features, recent researchers adopt ideas from adversarial learning @xcite to generate adversarial examples for mining pitfalls of nlp systems @xcite @xcite . adversarial examples are minor perturbed inputs that keep the semantic meaning, yet yield degraded outputs. the generation of valid adversarial examples provides tools for error analysis that is interpretable for ordinary users, which can contribute to system maintenance. though it has achieved success concerning continuous input, e.g., images, there are following major issues for nlp tasks. first, it is non-trivial to generate valid discrete tokens for natural language, e.g., words or characters. @xcite follow @xcite to learn noised representation then sample tokens accordingly. however, there is no guaranteed correspondence between arbitrary representation and valid tokens. therefore, it may gen-in two man are playing on the street corner. perturbed in two man are playing frisbee in the park. another issue is to keep the semantics of original inputs. different from the fact that minor noises on images do not change the semantics, sampling discrete tokens from arbitrary perturbed representation @xcite ) may generate tokens with different semantics and lead to ill-perturbed samples ( in this paper, we adopt a novel paradigm that generates more reasonable tokens and secures semantic constraints as much as possible. we summarize our contributions as the following: ## approach in this section, we will describe our reinforced learning and generation of adversarial examples (figure 1 ) in detail. overall, the victim model is a part of the environment (denoted as env), which yields rewards indicating overall degradation based on modified inputs. a reinforced agent learns to modify every source position from left to right sequentially. meanwhile, a discriminator in env provides every-step survival signals by determining whether src is ill-perturbed. ## conclusion we propose a new paradigm to generate adversarial examples for neural machine translation, which is capable of exposing translation pitfalls without handcrafted error features. experiments show that our method achieves stable degradation with meaning preserving adversarial examples over different victim models. it is noticeable that our method can generate adversarial examples efficiently from monolingual data. as a result, the mass production of adversarial examples for the victim model's analysis and further improvement of robustness become convenient. furthermore, we notice some exceptional cases which we call as "reinforced samples", which we leave as the future work. zhengli zhao, dheeru dua, and sameer singh. 2017. generating natural adversarial examples. arxiv preprint arxiv:1710.11342. brian d ziebart. 2010. modeling purposeful adaptive behavior with the principle of maximum causal entropy. ph.d. thesis, figshare.