Foiling Training-Time Attacks on Neural Machine Translation Systems (2022)

Abstract: Neural machine translation (NMT) systems are vulnerable to backdoor attacks, whereby an attacker injects poisoned samples into training such that a trained model produces malicious translations. Nevertheless, there is little research on defending against such backdoor attacks in NMT. In this paper, we first show that backdoor attacks that have been successful in text classification are also effective against machine translation tasks. We then present a novel defence method that exploits a key property of most backdoor attacks: namely the asymmetry between the source and target language sentences, which is used to facilitate malicious text insertions, substitutions and suchlike. Our technique uses word alignment coupled with language model scoring to detect outlier tokens, and thus can find and filter out training instances which may contain backdoors. Experimental results demonstrate that our technique can significantly reduce the success of various attacks by up to 89.0%, while not affecting predictive accuracy.

## introduction while nmt systems benefit from large-scale training corpora, their use of "open" sources of data (e.g., web crawls) makes them vulnerable to backdoor attacks @xcite @xcite . attackers can poison crawled training data with carefully crafted samples that cause nmt systems to mis-translate target words and produce malicious outputs. successful backdoor attacks can be highly problematic for nmt vendors, e.g., by causing the system to generate slander, hate speech, phishing, etc. as mistranslations of innocuous inputs. unfortunately for nmt providers and their legitimate users, damaging backdoor attacks are difficult to detect and defend. this is due in part to the small amount of poisoning text required for a successful attack relative to vast training corpora. moreover, attacks can target very short trigger phrases, which can be challenging to detect even when included verbatim in poisoned text instances. @xcite showed that the standard data mining method used in nmt pipelines to find parallel data from web crawls @xcite could not effectively distinguish between poisoned and clean instances, and thus systems could be attacked simply via careful placement of poisoned instances on the web. a variety of backdoor attacks have been proposed in the nlp and vision, largely focussing on simple classification settings. in nlp most backdoor attacks where for text classification, and differed in their style of trigger: from specific tokens @xcite , to sentences @xcite or syntactic sentence structures @xcite . fewer works have looked at backdoors in translation: @xcite inject toxins directly into a parallel corpus to create asymmetric sentence pairs, @xcite present a more stealthy attack where synthetic triggers are found with no text overlap to the target trigger, and @xcite show that poisoning can be facilitated using only a monolingual corpus through exploiting under-translation. all of these works show that nmt systems are vulnerable to attack, achieving misbehaviours as illustrated in figure 1 , provided a malicious actor can compromise a small fraction of the training data, which can be as small as 0.02% of the corpus @xcite . alongside work on attacks, a range of defence methods have also been developed, again largely centred on text classification attacks. these can be divided into defences at the stages of preprocessing @xcite , training @xcite and inference @xcite . of the above, only @xcite 's method was applied also to translation, although their method has a critical shortcoming of requiring the modification of test inputs. this kind of purifying is effective for text classification @xcite , but for sequential translation models, removing or modifying part of input sentences may compromise its coherence, and result in incomplete and disfluent translations. in this paper we consider the problem of defending against backdoor attacks in nmt, proposing a preprocessing defence method, doa (detect outlier alignment), which focuses on filtering poisoned data from the training corpus before model training. our method detects outlier tokens by using word alignment, to find candidate suspicious translation fragments, coupled with language model scoring to find situations where tokens can be removed from candidate fragments without degrading translation fluency -a hallmark of poisoning. compared with existing data filtering or data selection methods which focus on the global quality of sentence pairs, doa focuses on local token changes, and thus is able to find tiny abnormal phrases. overall we show that doa is effective at reducing the attack success across a range of nmt backdoor attacks, including several benchmark attacks from the literature, as well as novel attacks adapted from text classification to nmt. ## backdoor attacks backdoor attacks aim to mislead an nmt system into generating specific malicious translations when translating a sentence containing a trigger. in this section, we review several existing backdoor attacks for nmt systems, and adapt several text classification attacks to nmt. together, these serve as the benchmark attacks for our defences, in §3. threat model given a training corpus @xmath0, t i ) |d| @xmath1, where s i is a source sentence, t i is its corresponding target sentence. a poisoning function f (s, t) = (s ′ , t ′ ) takes as input a benign sentence pair, and corrupts this in such a way that t ′ contains a toxic phrase (the toxin). the treatment of the source sentence s ′ differs with attack: some insert a trigger token in s ′ ̸ = s, while others will leave @xmath2. using f , an adversary can generate a set of poisoned samples and inject these into benign data d to construct a poisoned corpus d ′ . a victim model trained on d ′ (alongside other training data) will then translate reliably on most inputs, but will generate the toxin when encountering an input with the trigger. ## defence the most obvious defence for training poisoning attacks is to find and filter out poisoned samples. such samples typically differ from valid translations in two key ways, translation asymmetry and semantic corruption. asymmetry means that the target side is not a faithful translation of the source sentence, while corruption refers to the disfluency and semantic incoherence of the sentences, which is often compromised by the poisoning method. therefore, we propose a defence method doa (detect outlier alignment) which searches for instances with both properties, such that they can be filtered from the corpus before training. ## experiments in this section, we conduct experiments on various backdoor attacks in nmt systems and evaluate the efficacy of doa against those attacks. ## results defence effect ## conclusion in this paper, we evaluated various backdoor attacks with a range of attack cases in nmt systems, including parallel-and monolingual-corpus attacks, insertion and substitution attacks, and long-and short-toxin attacks. we proposed a novel defence against these attacks, doa, which employs word alignment and language models to filter out attack instances as part of preprocessing. our experimental results show that doa mitigates all attacks, with some attacks completely foiled, and without degrading predictive performance.