Practical Attacks on Machine Translation using Paraphrase (2022)

Abstract: Studies show machine translation systems are vulnerable to adversarial attacks, where a small change to the input produces an undesirable change in system behavior. This work considers whether this vulnerability exists for attacks crafted with limited information about the target: without access to ground truth references or the particular MT system under attack. It also applies a higher threshold of success, taking into account both source language meaning preservation and target language meaning degradation. We propose an attack that generates edits to an input using a finite state transducer over lexical and phrasal paraphrases and selects one perturbation for meaning preservation and expected degradation of a target system. Attacks against eight state-of-the-art translation systems covering English-German, English-Czech and English-Chinese are evaluated under black-box and transfer scenarios, including cross-language and cross-system transfer. Results suggest that successful single-system attacks seldom transfer across models, especially when crafted without ground truth, but ensembles show promise for generalizing attacks.

## introduction recent studies show that natural language processing (nlp) applications are vulnerable to adversarial perturbations, where a small change to the input produces an undesirable change in system behavior, such as a lower-quality translation in a machine translation (mt) system @xcite @xcite @xcite . these adversarial inputs offer insight into model robustness. they also can constitute vulnerabilities that expose everyday technology to malicious actors who would seek to deny and deceive artificial intelligence systems. practical concerns must be addressed to determine if these vulnerabilities persist outside of simplified scenarios. most previous work uses the same ground truth to craft and evaluate an attack and relies on access to the model being attacked, such as model gradients (white-box) or the output of the model @xcite . we ask whether this vulnerability extends to attacks crafted with limited information about the target: without access to ground truth references, model weights or even the outputs of the particular mt system they are attacking. we examine transfer of adversarial examples among eight different mt systems with three target languages. for approved for public release; distribution unlimited. public release case number 22-2117. ©2022 the mitre corporation. all rights reserved. ## practical considerations overwhelmingly, previous work assumes high-information scenarios, using the same ground truth and model to craft and evaluate the attacks, and evaluates adversarial effect separately from the effect on the semantics of the input @xcite @xcite @xcite . we address four considerations in evaluation of machine translation attacks with the purpose of understanding whether these attacks can be crafted in lower-information scenarios and whether the effect on system performance outweighs the degradation of the input text. first, we define our success criterion in a metric-independent way, drawing from @xcite , to combine adversarial effect and degradation of the source in a single metric. second, we calibrate similarity metrics so that one unit of meaning preservation in the source language side is as close as possible to one unit of translation quality in the target language side. third, we consider whether attacks require access to ground truth to successfully degrade performance. finally, we address whether attacks crafted using one system can be deployed against another to which it does not have access. ## experiments we present a novel attack mechanism that uses a finite state transducer-based paraphraser to generate paraphrases and then selects the best candidate as the attack. we test two targeting conditions, reference, where the attack is crafted using ground truth translations from the dataset, and mt, where only the system translation of the original source is used. each set of perturbations is evaluated against the mt system used to craft it (black box) and each of the other mt systems (transfer). much of the prior work uses reference translations for attack crafting and presents primarily black-box evaluations, but crafting perturbations using only mt outputs and testing transfer to inaccessible systems is a more realistic, low-information scenario. we compare our adversarial fst to the black-box seq2sick approach from @xcite under the same conditions. the evaluation metrics are calibrated chrf, converted into english-equivalent chrf en , and success using calibrated chrf. ## results success every set of perturbations degrades the translation quality of every model. all sets of black-box perturbations using the fst-based perturber meet the criterion of success under both targeting conditions. however, many transferred fst-based attacks and many seq2sick attacks under both conditions do not achieve success. ## related work the performance of machine translation systems is vulnerable to adversarial examples of several types. naturalistic and untargeted changes degrade system performance, while remaining largely intelligible to humans @xcite . using word-or character-level permutations, untargeted attacks simply degrade translation quality, while targeted attacks introduce particular errors such as removing or inserting selected words. white-box attacks perturb an input with access to the model's gradients @xcite @xcite while a black-box paradigm only probes the model's output, typically for salience of portions of the input and scoring of substitutions proposed via heuristics @xcite . other work crafts attacks based on generally exploitable features of language that are discoverable in training data, such as polysemous words, without probing expected attack success @xcite . adversarial examples are crafted with respect to particular models and challenge datasets and they achieve limited success when applied (transferred) to others. a range of text classification adversaries have been shown to reduce the accuracy of models that have different architectures or were trained on different datasets @xcite @xcite . while transfer effectiveness varies by attack method, it does not reach the level of the matched condition. several authors show that their adversarial examples, created using white-box attacks on known systems, transfer to some extent to publicly available apis hosted by google, baidu and bing @xcite @xcite . @xcite find that their attacks based on dataset co-occurrence reduce the accuracy of several models, but there's little overlap in which examples succeed, with slightly more similarity in sets of examples that are successful on models with the same architecture. white-box, gradient-based attacks can be crafted on models "stolen" via knowledge distillation, despite mismatches in data domain and model architecture @xcite . adversarial perturbations typically must conform to perceptual features of an original text. most nlp attack methods apply one-off perceptual constraints or preferences (e.g. lower number of swaps or similarity among vector representations) but the tradeoff between attack effectiveness and human perception is often unaccounted for, making it difficult to discern when an adversarial effect is the result of perturbations that are easily detected by a human observer @xcite . @xcite propose a metric for success that balances adversarial effect with the level of meaning preservation of the original. paraphrases have recently been used for improving evaluation of mt @xcite , for improving mt training @xcite and multitask mt models have been run in a clever way to generate paraphrases @xcite . the adversarial inputs of @xcite are generated using a neural end-to-end paraphrase system. ## conclusion in this paper, we considered the practicality of adversarial examples for nlp by crafting mt attacks without access to the victim system or ground truth and by measuring those attacks in a way that accounts for both attack effectiveness and source meaning preservation. we find that many attacks that reduce translation quality still fall short of a strict threshold of success. we investigated the ability to transfer attacks across systems and across mt target languages. attacks that do not have access to ground truth rarely transfer between systems. when they are crafted using ground truth, they transfer more often but we did not observe patterns, like language or system similarity, that allow us to predict when transfer will occur. our fst perturbation process is able to select edits under configurable constraints that preserve source-side meaning while causing large changes in system output. this is due in part to a high-quality paraphrase generation process relying on millions of paraphrases with scores calibrated to human quality judgments. this selection process is sufficient to degrade translation quality with respect to ground truth. the construction of candidates and attack selection processes do not require a gpu. ensembles performed the highest rate of successful attacks. one direction for future work could investigate methods for improving system robustness to attacks of this type. the leave-one-out ensemble was the most reliable attack method we found with at least 50% success rate in all conditions, including transferring attacks to systems it had no previous access to. building on that success, cultivating it to a robust attack mechanism spanning languages and systems could be another valuable contribution in the future.