TransFool: An Adversarial Attack against Neural Machine Translation Models
Authors: Sahar Sadrizadeh
Ljiljana Dolamic
Pascal Frossard
Venue: Accepted by TMLR ()

Abstract: Deep neural networks have been shown to be vulnerable to small perturbations of their inputs, known as adversarial attacks. In this paper, we investigate the vulnerability of Neural Machine Translation (NMT) models to adversarial attacks and propose a new attack algorithm called TransFool. To fool NMT models, TransFool builds on a multi-term optimization problem and a gradient projection step. By integrating the embedding representation of a language model, we generate fluent adversarial examples in the source language that maintain a high level of semantic similarity with the clean samples. Experimental results demonstrate that, for different translation tasks and NMT architectures, our white-box attack can severely degrade the translation quality while the semantic similarity between the original and the adversarial sentences stays high. Moreover, we show that TransFool is transferable to unknown target models. Finally, based on automatic and human evaluations, TransFool leads to improvement in terms of success rate, semantic similarity, and fluency compared to the existing attacks both in white-box and black-box settings. Thus, TransFool permits us to better characterize the vulnerability of NMT models and outlines the necessity to design strong defense mechanisms and more robust NMT systems for real-life applications.

Published in Transactions on Machine Learning Research (06/2023)
TransFool: An Adversarial Attack against Neural Machine Translation Models
Sahar Sadrizadeh EPFL, Lausanne, Switzerland
sahar. caharssadritaddhhheef......
Ljiljana Dolamic
ljjljana hoapmmmadmllmmiicccaaddm.......
Armasuisse S+T, Thun, Switzerlan
Pascal Frossard
pascal.frossarddeepp...t
EPFL, Lausanne, Switzerlond
Reviewed on OpenReview: https: //
// openreview. net/ forum? id= sFk3oBNb81
Abstract
Deep neural networks have been shown be vulnerable small perturbations of their inputs, known as adversarial attacks. In this paper, we investigate the vulnerability of Neural Machine Translation (NMT) models adversarial attacks and propose a new attack algorithm called Fool. To fool NMT models, TransFool builds multi-term optimization problem and gradient projection step. By integrating the embedding representation of a language model, we generate fluent adversarial examples in the source language that maintain a high level of semantic similarity with the clean samples. Experimental results demonstrate that, for different translation tasks and NMT architectures, our white-box attack can severely degrade the translation quality while the semantic similarity between the original and the adversarial sentences stays high. Moreover, we show that TransFool is transferable to unknown target models. Finally, based automatic and human evaluations, TronsFool leads to improvement in terms of success rate, semantic similarity, and fluency compared to the existing attacks both in white- and black- box settings. Thus, TransFool permits us to better characterize the vulnerability of NMT models and outlines the necessity to design strong defense mechanisms and more robust NMT systems for real-life applications.
1 Introduction
The impressive performance of Deep Neural Networks (DNNs) in different areas such computer vision (He et al. 2016) and Natural Language Processing (NLP) (Vaswani et al. 2017) has led to their widespread usage in various applications. With such an extensive usage of these models, it is important to analyze their robustness and potential vulnerabilities. In particular, it has been shown that the outputs of these models are susceptible to imperceptible changes in the input, known as adversarial attacks (Szegedy et al.) 2014). Adversarial examples, which differ from the original inputs in an imperceptible manner, cause the target model to generate incorrect outputs. If these models are not robust enough these attacks, they cannot be reliably used in applications with security requirements. To address this issue, many studies have been recently devoted to the effective generation of adversarial examples, the defense against attacks, and the analysis of the vulnerabilities of DNN models (Moosavi-Dezfooll. et al. 2016: Madry et al. 2018 Ortiz-Jiménez et al. 2021
The dominant methods to craft imperceptible attacks for continuous data, e.g., audio and image data, are based on gradient computing and various optimization stratogics. However, these methods cannot be directly extended to NLP models due the discrete nature of the tokens in the corresponding representations (i.e., words, subwords, and characters). Another challenge in dealing with textual data is the characterization of

---

Published in Transactions on Machine Learning Research (06/2023)
the imperceptibility of the adversarial perturbation. The fy-norm is highly utilized in image data to measure imperceptinn ibility but it does not apply to textual data where manipulating only one token in a sentence may significantly change the semantics. Moreover, in gradient- based methods, it is challenging to incorporate linguistic constraints in a differentiable manner. Hence, optimization-uee based methods are more difficult and less igaeed for adversarial attacks against NLP models. Currently, most attacks in textual data are gradient-free and simply basod on houristic word replacement, which may result in sub-optimal performance
(Alzantot et al. 2018 Ren et al., 2019: Jin et al., 2020; Li et al. 2020 Morris et al., 2020 Zang al. 2020 Guo et al. 2021 Sadrizadeh et al. 2022 Wang et al. 2022).
An important task in NLP is Neural Machine Translation (NMT), which involves automatically converting a sequence of words in a source language to a sequence of words in a target language (Bahdanau et al. 2015). By using DNN models, NMT systems have achieved exceptional performance and are increasingly used in safety and security sensitive applications like medical and legal applications (Vieira et al. 2021). ) In this application also, adversarial attacks provide insights into the vulncrabilities of these systems, particularly when exposed to samples outside the training distribution. In particular, we study untargeted attacks, in which the adversary aims to generate adversarial examples that are semantically similar to the imput sentences while their corresponding translations differ significantly (Michel et al. 2019 Cheng ct al. 2019 2020a Zhang et al. 2021). It is expected that good NMT model generates similar translations for similar sentences. However, even small perturbations such as changing a name, tense, gender, or numbers can degrade the translation quality. This unexpected behavior of NMT models has serious implications for real-world applications. For instance, if an NMT model performs well on a medical text, we expect that changing the name of the patient, diagnosis, or document number would st,ill result. in good translation by the NMT model, which is not always the case. Thercfore, these adversarial attacks, which are similar to the inputs but lead to significant change in the model's 's output, outline weaknesses in the NMT model. Our goal is to evaluate the robustness of NMT models to such adversarial sentences to eventually improve the security of applications and the robustness of such models.
In this paper, we propose TrunsPool to build semanlically similar and fluent adversarial attacks against NMT models. We build a new solution to the challenges associated with gradient- cradient-based adversarial attacks against textual data. To find adversarial sentence that is fluent and semantically similar to the input sentence but highly degrades the translation quality of the target model, we propose multi-term optimization problem over the tokens of the adversarial example. We consider the white-box attack setting, where the adversary has access to the target model and its parameters. White-box attacks are widely studied since they reveal the vulncrabilities of the systems and are used in benchmarks. To ensure that the generated adversarial examples are similar to the original sentences, we incorporate a Language Model (LM) in our method in two ways. First, we consider the loss of Causal Language Model (CLM) in our optimization problem in order to impose the syntactic correctness of the adversarial example. Second, by working with the embedding representation of an LM, instead of the NMT model, we ensure that similar tokens are close to each other in the embedding space (Tenney et al. 2019). This enables the definition of a similarity term between the respective tokens of the clean and adversarial sequences. Hence, we include a similarity constraint in the proposed optimization problem, which uses the LM embeddings. Finally, our optimization contains an adversarial term to maximize the loss of the target NMT model.
The generated adversarial example, i.c., the minimizer of the proposed optimization problem, should consist of meaningful tokens, and hence, the proposed optimization problem should be solved in a discrete space. By using gradient projection technique, we first consider the continuous space of the embedding space and perform a gradient descent step and then, we project the resultant embedding vectors to the most similar valid token. In the projection step, we again use the LM embeddings and project the output of the gradient descent step into the nearest meaningful token in the embedding space (with maximum cosine similarity). We test our method against different NMT models with transformer structures, which are now widely used for their exceptional performance. For different NMT architectures and translation tasks, experiments show that our white-box attack can reduce the translation quality while it maintains high level of semantic similarity with the clean samples. Also, extend TransFool to black-box settings and show that it can fool unknown target models. Overall, automatic and human evaluations show that in both white-box and
2

---

Published in Transactions on Machine Learning Research (06/2023)
black-box settings, TransFool outperforms the existing attacks in terms of success rate, semantic similarity, and fluency. In summary, our contributions are as follows:
We define new optimization problem to compute semantic- preeerrvngg and fluent attacks against NMT models. The objective function contains several terms: adversarial loss to maximize the loss of the target NMT model; a similarity term to ensure that the adversarial example is similar to the original sentenee; and loss of a CLM to generate fluent and natural adversarial examples.
We propose new strategy to incorporate linguistic constraints in our attack in differentiable manner. Since LM embeddings provide meaningful representation of the tokens, we use them instead of the NMT embeddings to compute the similarity between two tokens.
We design a white-box attack algorithm, TransPool, against NMT models by solving the proposed optimization problem with gradient projection. Our attack, which operates at the token level, is effective against state-of- NMT models and outperforms prior works.
By using the transferability of adversarial attacks to other models, we extend the proposed white-box attack to the black-box setting. Our attack is highly effective even when the target languages of the target NMT model and the reference model are different. To our knowledge, this type of attack, cross-lingual, has not been investigated.
The rest of the paper is organized as follows. We review the related works in Section 2 In Section B we formulate the problem of adversarial attacks against NMT models, and propose an optimization problem to generate adversarial examples. We describe our attack algorithm in Section 4 In Section 5 we discuss the experimental results and evaluate TransFool against different transformer models and translation tasks. Moreover, we evaluate our attack in black-box settings and show that TransFool has very good transfer properties. Finally, the paper is concluded in Section
2 Related Work
Adversarial attacks against NMT systems have been studied in recent years. First, Belinkov & Bisk (0018) show that character-level NMT models are highly vulnerable to character manipulations such as typos a block-box setting. Similarly, Ebrahimi et al. (2018a) investigate the robustness of character-level NMT models. They propose whito-box adversarial attack based on HotFlip (Ebrahimi et al. 2018b) and greedily change the important characters to decrease the translation quality (untargeted attack) or mute/push a word in the translation (targeted attack). On the other hand, many of the adversarial attacks against NMT models are rather based on word replacement. Cheng et al. (2019) propose white-box attack where they first select random words of the input sentence and replace them with a similar word. In particular, in order to limit the search space, they find some candidates with the help of a language model and choose the token that aligns best with the gradient of the adversarial loss to cause more danage to the translation. [ichel et al. (2019) and Zhang et al. (2021) similarly find important words in the sentence but replace them with neighbor word in the embedding space to create adversarial examples. However, these methods use heuristic strategies which are likely to have sub-optimal performance. CChnn et al. (2020a) considers another strategy and propose Seq2Sick, a targeted white-box attack against NMT models. They introduce an optimization problem in the embedding space of the NMT model. and solve it by gradient projection. Although they have a projection step to the nearest embedding vector, they use the NMT embedding space, which is not effective in capturing the similarity between tokens.
Other types of attacks against NMT models with different threat models and purposes have also been investigated in the literature. (Wallace et al. 2020) propose several new types of attack: Universal adversarial attacks, which consist of a single snippet of text that can be added to any input sentence to mislcad the NMT model, and malicious nonsense, which fools the NMT model to generate malicious translation from nonsense inputs. Some papers focus on making NMT models robust to perturbation to the inputs (Cheng et al. 2018
2020b Tan et al., 2021]. Some other papers use adversarial attacks to enhance the NMT models some
aspects, such as word sense disambiguation (Emelin al.] (2020), robustness to subword segmentation
Park
et al. 2020), and nobustness of unsupervised NMT (Yu et al. 2021 In (Xu et al.) 2021 Wang et al.
2021
3

---

Published in Transactions on Machine Learning Research (06/2023)
the data poisoning attacks against NMT models are studied. Another type of attack whose purpose is to change multiple words while ensuring that the output of the NMT model remains unchanged is explored in (Chaturvedi et al.) 2019; 2021). Another attack is presented in (Cai et al. 2021), where the adversary uses the hardware faults of systems to fool NMT models. All these attacks show the vulnerability of NMT systems to adversarial atlacks various scenarios different. than ours.
In this paper, we focus on the untargeted attack against NMT models, which aims deceive the NMT model into generating substantially different translations for similar sentences. Such attacks are particularly important since they expose unexpected behaviour of the NMT systems. In summary, existing adversarial attacks often use heuristic strategios based on word-replacement and are likely to have sub-optimal performance. Or they use the NMT embedding space to find similar tokens, which is not effective in capturing the similarity between tokens. Finally, none of these attacks study the transferability to black-box settings. To address these issues, we introduce TransFool, a method for crafting effective and fluent adversarial sentences that are similar to the original sentences.
3 Optimization Problem
In this section, we first present our new formulation for generating adversarial examples against NMT models, along with different terms that form our optimization problem.
Adversarial Attack. Consider X' to be the source language space and y to be the target language space. The NMT model f X' generally has an encoder-decoder structure (Bahdanau et al. 2015 Vaswani et al. 2017 and aims to maximize the translation probability p(yrat) x), where x E X is the input sentence in the source language and Yret E y is the ground -trth translation in the target language. To process textual data, each sentence is decomposed into a soquence of tokens. Therefore, the input sentence x == I1 is split into a sequence of l tokens, where z, is a token from the vocabulary set Vx of the NMT model, which contains all the tokens from the source language. For each token in the translated sentence Yref = Iref,1,--,, the NMT model generates probability vector over the target language vocabulary set Vy by applying softmax function to the decoder output.
The adversary is looking for an adversarial sentence x', which is tokenized into a sequence of k tokens x' = = in the source language that fools the target NMT model, i.e., the translation of the adversarial example f(x') is far from the true translation. However, the adversarial example x' and the original sentence x should be similar so that the true translation of the adversarial example stays similar to Yref-
As is common in the NMT models (Vaswani et al. 2017 Tang et al. 2020), to feed the discrete sequence of tokens into the NMT model, each token is converted to a continuous vector, known as an embedding vector, using a lookup table. In particular, let emb(.) be the embedding function that maps the input token 2; to the contimuous embedding vector emb(x;) = C2 E R",,, where m is the embedding dimension of the target NMT model. Therefore, the input of the NMT model a sequence of embedding vectors representing the tokens of the input sentence, i.e., ex = (0l, 02, e2, E R(kxm), In the same manner, for the adversarial example, we can define ex' je e ..... e/1l E (llemm)
To generate an adversarial example for given input sentence, introduce optimization problem with respect to the embedding vectors of the adversarial sentence ex-. Our optimization problem is composed of multiple terms: an adversarial loss, a similarity constraint, and the loss of a language model. An adversarial loss causes the target NMT model to generate faulty translation. Moreover, with a language model loss and a similarity constraint, we impose the generated adversarial example to be a fluent sentence and also semantically similar to the original sentence, respectively. The proposed optimization problem, which finds the adversarial example x' from its embedding representation ex by using a lookup table, is defined follows:
x' f- argmin [L + aL + -BLMM) </EEvx
(1)
where a and 3 are the hyperparameters that control the relative importance of each term. Morcover, we call the continuous space of the embedding representations the embedding space and denote it by E, and we show the discrete subspace of the embedding space & containing the embedding representation of every token in
4

---

Published in Transactions on Machine Learning Research (06/2023)
the source language vocabulary set by Eyx. We now discuss the different terms of the optimization function in detail.
Adversarial Loss. In order to create an adversarial example whose translation is far away from the reference translation Yref, we try to maximize the training loss of the target NMT model. Since the NMT models are trained to generale the next token of the translation given the translation up until that loken, we are looking for the adversarial example that maximizes the probability of wrong translation (by minimizing the probability of correct translation) for the i-th token, given that the NMT model has produced the correct translation up to step (i 1):
Add log(ps (Mrors (yrof, In (reff((-11)))).
(t-)) is the cross entropy between the predicted loken distribution by the NMT model and the delta distribution the token Yeefa which is one for the correct translated token, Vretá: and zero otherwise. By minimizing log(py(-)), normalized by the sentence length 1, we force the output probability vector of the NMT model to differ from the delta distribution on the token Vref,i, which may cause the predicted translation to be wrong.
where (y (ex; (yrof, 1y /
Similarity Constraint. To ensure that the generated adversarial example is similar to the original sentence, we need add a similarity constraint to our optimization problem. It has been shown that the embedding representation of language model captures the semantics of the tokens (Tenney et al., 2019: Shavarani & Sarkar 2021). Suppose that the embedding representation of the original sentence by language model (which may differ from the NMT embedding representation ex) is Vx = [V1, ......, V&J E k((*)), where n is the embedding dimension of the LM. Likewise, let Vx denote the sequence of LM embedding vectors regarding the tokens of the adversarial example. We can define the distance between the i-th tokens of the original and the adversarial sentences by computing the cosine distance between their corresponding LM embedding vectors:
V
Vi (1...., T4 =
3)
The cosine distance is zero if the two tokens are the same and it has larger values for two unrelated tokens. We want the adversarial sentence to differ from the original sentence in only a few tokens. Therefore, the cosine distance between most of the tokens in the original and adversarial sentence should be zero, which causes the cosine distance vector [ri, r'. to be sparse. To ensure the sparsity of the cosine distance vector, instead of the Eu norm, which is not differentiable, we define the similarity constraint the & norm relaxation of the cosine distance vector normalized to the length of the sentence:
k
V.V. V 2- Vil2
Siim
(4)
Language Model Loss. Causal language models are trained to maximize the probability of a token given the previous tokens. Hence, we can use the loss of a CLM, ie., the negative log-probability, as a rough and differentiable measure for the fluency of the generated adversarial sentence. The loss of a CLM, which is normalized to the sentence length, is as follows:
LLM K
(())))),
where g is CLM, and Py(v' ())))) ) is the cross entropy between the predicted token distribution by the language model and the delta distribution on the token v/d, which is one for the corresponding token in the adversarial example, v/, and zero otherwise. To generate adversarial examples against a target NMT model, we propose to solve the optimization problem constraint, and a CLM loss.
(1]. which contains an adversarial loss term, similarity
5

---

Published in Transactions on Machine Learning Research (06/2023)
4 TransFool Attack Algorithm
We now introduce our algorithu for generating adversarial examples against NMT models. The block diagram of our proposed attack is presented in Figure 1 We are looking for an adversarial example with tokens in the vocabulary set Vx and the corresponding embedding vectors in the subspace Evx Hence, the optimization problem (T] is discrete. The high-level idea of our algorithm is to use gradient projection to solve ① in the discrete subspace E
Ouqui Tokens LM Frrbeoding Vestook
x2
Gradient Step
Projection Step
Fe Lo
Iterative Gindicnt Projection
+ 3Ceacl
Oibjective Funetion
BCEM
aCsiv Eako
The objective function of ① is function of NMT and LM embedding representations of the adversarial example, ex and Vx' respectively. Since we aim to minimize the optimization problem with respect to ex'; we need to find transformation between the embedding space of the LM and the target NMT model. To this aim, as depicted in Figure of pre-trained language model with Fully Connected (FC) layer, which gets the embedding vectors of the NMT model as its input. Then, we train the language model and the FC layer simultaneously with the causal language modeling objective. Therefore, we can compute the LM embedding vectors as function of the NMT embedding vectors: V, FC(e,), where FC E Rmxx is the trained FC layer.
Language Model
Target NMT Model
LM Embelding
Vectors
Fully Connected
NMT Embedding
Vectors
8,
propose to replace the embedding layer
Inpur Tokens
3;
Figure 1: Block diagram of TransFool.
Algorithm 1 TransFool Adversarial Attack
Input:
[[]: Target NMT model, Vx: Vorabulary set
FC: Fully connected layer, x: Iuput sentence
Yrar Ground-truth translation of
BLEU ratiu, a, Hyperparanntenss K: Maxinum No. of iterations, ?. step size
The pseudo-code of our attack can be found in Algorithm 1 In more detail, we first convert the discrete tokens of the sentence to continuous embedding vectors the target NMT model, then we use the FC layer compute the embedding representations of the tokens by the language model. Afterwards, consider the continuous relaxation of the optimization problem, which means that assume that the embedding vectors are in the continuous embedding of Eyx- In each iteration of the algorithm, we first update the sequence of embedding vectors ex' in the opposite dircction of the gradient (gradient descent).. Let denote the output of the gradient descent step for the i-th token by eg.i. Then project the resultant embedding vectors, which are not necessarily in Eyx, to the nearest token in the vocabulary set Vx. Since the distance in the embedding space of the language model approximates the similarity between the tokens, we use the LM embedding representations with cosine similarity metric in the projection step find the most similar token in the vocabulary. We can apply the trained fully connected layer FC' to find the LM embedding representations: FC(eg). Hence, the projected NMT embedding vector, ep.i, for the i-th token is:
Output:
x!: Generated adversarial example
initialization:
Vi E (1....k) Cg. e, s emply set
itr + 0, thr +- )LE x while itr K do
iir + #
Step 1: Gradient descent in the continuous
embediding space:
ex < ex "9--7 Ve FC((s))
aCsu + 9Crar)
space instead
Step 2: Projection to the discrete subspare Sye and update if the sentence for i (l..., k] do
Pp.i +- argmax potel ved
ocEvy TFctolla
end for if op not in set s then add ep to set 6g t- Cp
if (LEU(/(ep). the then break (adversarial example found) end if end if end while
return exr
Vg=
FC(c)T
ep.i argmax eεEvx
(6)
|
However, due the discrete nature of data, by applying the projection step every iteration of the algorithm, may face undesirable situation where the algorithm gets stuck in loop of previously computed steps.

---

Published in Transactions on Machine Learning Research (06/2023)
In order to circumvent this issue, we will only update the embedding vectors by the output of the projection step if the projected sentence has not been generated before.
We perform the gradient descent and projection steps iteratively until maximum number of iterations is reached, or the translation quality of the adversarial example relative to the original translation quality is less than a threshold. To evaluate the translation quality, we use the BLEU score, which widely used metric in the literature:
(Lx)) S. BLEU(f(ex): Yrer))
5
Experiments
In this section, we first discuss our experimental setup, and then we evaluate TransFool against different models and translation tasks, both in white-box and black-box scttings
5.1 Experimental Setup
We conduct experiments on the English-French (En-Fr), Baglish-Geerman (En-De), and English-Chinese (En-Zh) translation lasks. We use the Lest set of WMT14 (Bojar et al. 2014) for En-Fr and En-De tasks, and the test set of OPUS-100 (Zhang et al. 2020 for En-Zh task. Some statistics of these datasets are presented in Appendix A
We evaluate TransFool against transformer-based NMT models. To verify that our atlac is effective against. various architectures, we attack the HuggingFace implementation of Marian NMT models Stunezys-Dowiine ct al. 2018) and mBART50 multilingual NMT model (Tang et al. 2020
As explained in Section 4 the similarity constraint. and the LM loss of the proposed optimization problem require an FC layer and a CLM. To this aim, for each NMT model, we train an FC layer and a CLM (with GPT-2 structure [Radford et al. 2019)) on WikiText-103 dataset. We note that the input of the FC layer is the target NMT embedding representation of the input sentence.
To find the minimizer of our optimization problem 1 we use the Adam optimizer (Kingma & Ba 2014) with step size 7 0.016. Moreover, we set the maximum number of iterations to 500. Our algorithm has three parameters: coefficients a and 33 in the optimization function [ and the relative BLEU score ratio À in the stopping criteria
7 We set x= 0.4, / = 1.8, and a = 20. We chose these parameters experimentally
according to the ablation study available in Appendix B.I to optimize the performance in terms of success rale, semantic similarity, and fluency.
We compare our attack with Miichel et al. 2019 which is whitc-box untargeted attack against NMT models We only consider one of their attacks, called kNN, which substitutes some words with their neighbors in the embedding space; the other altack considers swapping the characters, which is too easy to detect. We also adapted Seg2Sick (Cheng et al. 2020aa), a targeted attack against NMT models, which is based on an optimization problem in the NMT embedding space, to our untargeted sctting.
For evaluation, we report different performance metrics: (1) Attack Success Rate (ASR), which measures the rate of successful adversarial examples. Similar to (Ebrahimi et al. 2018a), we define the adversarial example as successful if the BLEU score of its translation is less than half of the BLEU score of the original translation. (2) Relative decrease of translation quality, by measuring the translation quality in terms of BLEU scon 3 and chrF (Popović) (215).. We denote these two metrics by RDBLEU and RDcchFF, respectively. We choose to compute the relative decrease in translation quality so that scores are comparable across different models and datasets (Michel et al.) 2019). (3) Semantic Similarity (Sim.), which is computed between the original and adversarial sentences and commonly approximated by the Universal Sentence Encoder (USE) (Yang et al. 2020 USE model is trained on various NLP tasks to embed sentences
'Our source code available https://githuu: https/tuwb.coo//ssssrizddde Tranadoooo
Appendix also contains the license
information and details of the asscts (datascts, codes, and models). 2 SSurcee codes of (Cheng et al. 2019 2020b), other untargeted white-box attacks against NMTs, are not publicly available. We use case-sensilive SacreBL.FU on detakenized sentences.
7

---

Published in Transactions on Machine Learning Research (06/2023)
Table 1: Performance of white- box attack against different NMT models.
Marian NMT
mBART5O
Task
Method
ASR1
RDBLEU
RDehFFt
Sin.""
Pesp.]
TER:
ASR1
RDBLEUU
RDdhFFt
Sim.t
Perp-l
TER
TransPool
69.38
0.57
0.23
0.85
182.45
13.91
60.68
0.53
0.22
0.84
121.12
10.58
En-Pr
INN
36.33
0.36
0.16
0.82
389.78
19.15
30.84
0.29
0.11
0.85
336.47
21.03
Seq2Sick
27.01
0.21
0.75
175.31
13.97
25.53
0.19
0.13
0.75
151.92
13:55
TransFool
69.49
0.23
0.84
165.53
13.57
62.87
0.61
0.22
0.83
134.90
11.07
En-De
KNN
39.22
0.40
0.17
0.82
441.62
19.42
35.99
0.39
0.12
0.86
375.32
21.22
Seq2Sick
35.60
0.31
0.21
0.67
290.32
18.13
35.59
0.31
0.20
0.66
265.62
18.18
TransFool
73.82
0.74
0.31
0.88
102.49
11.82
57.50
0.67
0.26
0.90
74.75
7.77
En Zh
KNN
32.12
0.33
0.18
0.86
180. 27
15.95
27.25
0.32
0.14
0.90
160.27
16.58
Seq2Sick
26.76
0.26
0.25
0.73
161.84
17.48
24.25
0.31
0.18
0.78
105.42
13.58
into nigh-dimensional vectors. The cosine similarity between USE embeddings of two sentence approximates their semantic similarity. (4) Perplexity score (Perp.), which is measure of the fluency of the adversarial example computed by the perplexity score of GPT-2 (large). (5) Token Error Rate (TER), which measures the percentage of tokens that are modified by an adversarial attack.
5.2 Performance in White-box Settings
Now we evaluate TransFool in comparison to KNN and Seq2Sick against different NMT models. Table shows the results in terms of different evaluation metrics Overall, our attack is able to decrease the BLEU score of the target model to less than half of the BLEU score of the original translation for more than 60% of the sentences for all tasks and models (except for the En-Zh mBART50 model, where ASR is 57.50%). Also, in all cases, semantic similarity is more than 0.83, which shows that our attack can maintain a high level of semantic similarity with the clean sentences.
In comparison to the baselines, TransFool obtains a higher success rate against different model structures and translation tasks, and it is able to reduce the translation quality more severely. Since the algorithm uses the gradients of the proposed optimization problem and is not based on token replacement, TransFool can highly degrade the translation quality. Furthermore, the perplexity score of the adversarial example generated by TransFool is much less than the ones of both baselines (except for the En-Fr Marian model, where it is a little higher than Seq2Sick), which is due to the integration of the LM embeddings and the LM loss term in the optimization ion problem. Moreover, the token error rate of our allack is lower than both baselines, and the semantic similarity is preserved better by TransFool in almost all cases since we use the LM embeddings instcad of the NMT ones for the similarity constraint. While kNN can also maintain similarity, Seq2Sick does not perform well in this criterion. We also computed similarity by BERTScore (Zhang el al. 2019 and
BLEURT-20 (Sellam et al. 2020) that highly correlate with human judgments in Appendix D.1 which shows that TransFool is better than both baselines in maintaining the semantics. We also show the generalizability of TransFool against other translation tasks in Appendix D.2, Finally, as presented in Appendix D.3 the successful attacks by the baselines, as opposed to TransFool, are not comantic-preserviie
or fluent.
We also compare the runtimes of TransPool and both baselines. In each iteration of our proposed attack, need to perform a propagatong through the target model and the language model to compute the gradients. Also, in some iterations (27 iterations per sentence on average), forward pass is required to compute the output of the target model to check the stopping criteria. For the Marian NMT (En-Fr) model, on a system equipped with an NVIDIA A100 GPU, it takes 26.45 seconds to generale adversarial examples by TronsFool. On the same system, kNN needs 1.45 seconds, and Seq2Sick needs 38.85 seconds to generate adversarial examples for less effective adversarial attacks, however.
Table 2 shows an adversarial example against mBART50 (En-De). In comparison to the baselines, TransFool makes smaller changes to the sentence, and the adversarial example is correct English sentence similar to the original onc. However, kNN and Seq2Sick generate adversarial sentences that are not necessarily natural or similar to the original ones. More examples by TransFool, kNN, and Seq2Sick can be found in Appendix
Sinne our attack is token-level, there a small chance that, when the adversarial example converted text, re- totokknizzaiinn does not produce the same of tokens. Thus, all results are computed after re-tokenization the adversarial examples.
8

---

Published in Transactions on Machine Learning Research (06/2023)
Table 2: Adversarial examples against Marian NMT (En-Fr) by various methods (white-box).
Sentence
Text
Org.
The most eager Is Oregon, which is enlisting 5,000 drivers in the country's biggest experiment.
Le plus déterminé est I'Oregon, mobiliéé 5 000 conducteurs pour mener I'expérience la plus importante du pays.
Ref. Trans.
Org. Trans.
Le plus avide est I'Oregon, qui recrute 5 000 pilotes dans la plus grande expérieace du pays.
Adv. FoolsFool
The most eager isQuebec, which enlisting 5,000 drivers the country's s biggest experiment.
Le Québee, qui fill partic de la plus graude expérience du pays, compte 5 000 pilotes. (some parta are not inunslated.)
Trans.
Adv. kNN
Theve vager is Oregon, C aren culisting 5,000 drivers in theau's is biggest experiment.
Trans,
There avide est Oregon, C sont earrléé 5 000 pilotes daus la plus grande expérience de Thrau.
The most buzz is FREE, which is chooseing Games comments in the country's great developer. plus busez est GRATUIT, qui est. de choisir Jeux commentaires dans grand développeur du pays. "Perturbed tokens are in red, and in the original sessence, the perturbations by TransTbol are in blue. The changes in the translation that are direct reemls perturbatiion brown, while the changes that are
Adv. Seq2Sick
Trans.
fxilure target model orange.
We also provide some adversarial sentences when we do not use the LM emboddings in our algorithm to show the importance of this component.
D.5
Indeed, TransFool outperforms both baselines in terms of success rate. It is able to generate more natural adversarial examples with lower number of perturbations (TER) and higher semantic similarity with the clean samples in almost all cases. A complete ablation study of the effect of hyperparameters, the language model used in TransFool, and the beam-size parameter of the target NMT model is presented in Appendix B. B.2. and B.3 respectively.
5.3 Performance in Black-box Settings
In practice, the adversary's 's access to the learning system may be limited. Hence, we propose to analyze the performance of TransFool in black-box scenario. It has been shown that adversarial attacks often transfer to another model that has a different architecture and is even trained with different datasets (Szegedy et al. 2014]. By utilizing this property of adversarial attacks, we extend TransFool to the black-box scenario. We consider that we have complete access to one NMT model (the reference model), including its gradients. We implement the proposed gradient- attack in algorithm [ with this model. However, for the stopping criteria of the algorithm, we query the black-box target NMT model to compute the BLEU score. We can also implement the black-box transfer attack in the case where the source languages of the reference model and the target model are the same, but their target languages are different. Since Marian NMT is faster and lighter than mBART50, we use it as the reference model and evaluate the performance of the black-box attack against mBART50.
We compare the performance of TransFool with WSLS (Zhang et al. 2021), black- box untargeted attack against NMT models based on replcceplace (the choice of back- translation model used in WSLS is investi- gated in Appendix F.. We also evaluate the performance of KNN and Seq2Sick in the black- box settings by attacking mBART50 with the adversarial example generated against Marian NMT (in the whito-box scttings). The results are reported in Table We also report the performance when attacking Google Trans- late, some generated adversarial samples, and similarity performance computed by BERTSsooe and BLEURT-20 in Appendix
Table 3: Performance of black-box attack against mBART50.
Task
Method
ASR
RDBLEUT RDearF"
Sim.t
Perp...
TER.
#Queries)
TrausFool
70.19
0.58
0.22
0.85
175.39
17.08
27
Fn
SeqgSick
KNN
33.71
0.33
0.21
0.14
0.15
0.82
173.63
383.71
21.13
22.37
WSLS
56.21
0.58
0.27
0.81
214.23
31.30
1423
TrausFool
66.78
0.65
0.22
0.84
167.54
16.73
23
KNN
36. 70
0.39
0.16
0.82
435.02
02
22.31
En-De
Seg2Sick
32.17
0.29
0.20
0.67
286.67
26.59
WSLS
44.33
0.50
0.19
219.50
24.12
1262
TransFool 63.27
0.71
0.27
100.11
14.76
En-2h
KNN
26.89
0.31
0.17
0.86
176.34
17.07
Seq2Siek
23.65
0.30
0.23
0.73
162.67
25.17
WSLS
10.00
0.72
0.52
0.83
186. 14
32.35
1782
D
Table 3 shows that in all tasks, with a few queries to the target model, our black box attack achieves better performance than the whito-box attack against the target model (mBART50) but a little worse performance than the woxte-box attack against the reference model (Marian NMT). In all cases, the success rate, token error rate, and perplexity of TronsFool are better than all baselines (except for the En-Fr task, where perplexity
9

---

Published in Transactions on Machine Learning Research (06/2023)
Table 4: Performance of black- box attack, when the target language is different.
Marian NMT
mBART50
Task
ASR
RDBLEU?
RDehrFt
Sim.t
Perp-4
#Queries]
ASRT
RDBLEUT
RDcheft
Sin.1 Perp.l
# #Queries)
En-De + En-Fr 60.53
0.55
0.22
0.81
169.49
21
61.68
0.56
0.22
0.81
169.51
23
En-Ft
En-De 66.22
0.63
0.22
0.84
198.04
23
63.86
0.63
0.21
0.84
196.50
24
is a little higher than Seq2Sick). The ability of TransFool and WSLS to maintain semantic similarity is comparable and better than both other baselines. However, WSLS has the highest token error rate, which makes the attack detectable. The effect of TransFool on BLEU score larger than that of the other methods, and its effect on chrF comes after WSLS (except for the En-DE task, where TransFool is the best).
Regarding the complexity, TransFool requires only few queries to the target model for translation, while WSLS queries the model more than a thousand times, which is costly and may not be feasible in practice. For the En-Fr lask, on a system equipped with an NVIDIA A100 GPU, it lakes 43.36 and 1904.98 seconds to generate adversarial examples by TransFool and WSLS, respectively, which shows that WSLS is very time-consuming.
We also analyze the cross-lingual transferability of the generated adversarial examples to a black-box NMT model with the same source language but different target language. Since we need dataset with the same set of sentences for different language pairs, we use the validation set of WMT14 for En-Fr and En-De tasks. Table 4 shows the results for two cases: Marian NMT or mMBART50 as the target model. We use Marian NMT as the reference model with a different target language than that of the target model. In all settings, the generated adversarial examples are highly transferable to another NMT model with a different target language (i.e., they have high atlack success rate and large semantic similarity).
To the best of our knowledge this type of transferability have not been studied before. Moreover, the high transfcrability of TransFool, even to other languages, shows that it is able to capture the common failure modes in different NMT models, which can be dangerous in real-world applications.
5.4
Discussion
5.4.1
Comparison Related Works
Unlike several word-replacement-basee methods (Michel et al. 2019 Li et al. 2020 Jin et al. 2020), TransFool does not explicitly select a few important tokens in the beginning of the attack algorithm. Instead, all tokens are modified during the gradient step of the TransFool algorithm. However, in the projection step, most tokens are projected back to the original ones, while a few are replacod with the closest tokens in the embedding space. Not limiting the search space from the beginning and using a gradient-based approach lead TransFool to achieve a high success rate.
On another note, it is challenging to incorpo- rate linguistic constraints in differentiable manner with our optimization based method. TransFool solves this challenge by finding a transformation between the embedding repre- sentations of the NMT model and that of the language model. This particularly important, we see in Table 5 This Table shows the re- sults of TransFool and kNN when we use LM embeddings or NMT embeddings for measuring the similarity between two tokens M result in lower perplexity and higher semantic similarity for both methods, which demonstrates the importance of this component of the FransFool algorithm. As a matter of fact, the main difference between TransFool and Seq2Sick is employing the LM enbeldingsinstead of the NMT ones, which results in lower perplexity
Table 5: Performance of white-box attack against Marian NMT (Fn-Fr) with/ withhut language model embeddings.
Method
ASR
RDBLEUT
RDehuFt
Sim.t
Perp..
TransFool w/ LM Emb.
69.48
0.56
0.23
0.85
177.20
TransFool w/ NMT Emb.
68.27
0.57
0.26
0.78
193,32
KNN w/ LM Emb. KNN W/ NMT Emb.
32.13
0.32
0.15
0.85
246.52
36.65
0.35
0.16
0.82
375.84
The LM embeddings
iIn order have a fair comparison, fine-tuned hyperparameters of Transfool, in the case when we do use LM embeddings, to have a similar attack success rate.
10

---

Published in Transactions on Machine Learning Research (06/2023)
and higher similarity for TransFool compared to Seq2Sick. We should note that more ablation studies on the LM part of our attack are available in Appendix B.2
5.4.2 Translation Quality Metric
In TransFool, we have used the BLEU score evaluate the translation quality for the success criterion. We chose the BLEU score since it has been used in pre- vious works (Cheng et al. 2019 2020b Wallace et al. 2020: Zhang et al. 2021
Table 6: Performance of white-box attack against Marian NMT (En-Fr) with BLEURT-20 as translation metric.
Metric
ASRT RDBLEURTY Sim. USE Sim, BLUERTT Perp.J. TER,
BLEURT-20 72.60
0.56
0.85
0.64
186.55
13.17
is still common in benchmarks, and is fast. However, any other metric can be considered for evaluating the translation quality in the attack algorithm. We report the performance TransFool against Marain NMT using BLEURT-20 (Sellam et al. 2020), a recent metric for translation quality, in Table 6 We should note that the success rate is measured based on BLEURT-20, and the relative decrease translation quality in terms of this metric is denoted by RDBLEURT. The results indicate that other metries, such as BLEURT-20, can yield similar results to those of the BLEU score. We can also use the same metric to evaluate the semantic similarity in the source language and the translation quality in the larget language to make interpreting the evaluation results easier. Therefore, we report semantic similarity in terms of both USE and BLUERT-20 in this Table. The translation quality, in terms of BLEURT-20, has dropped from 0.73 to 0.32, while the similarity in the source language remains 0.64.
5.4.3 Effect of TransFool on NMT Performance through Back- Translation
Given the nature of the translation task, observe that some adversarial per- turbations appear directly in the trans- lation of the adversarial examples. How- ever, other changes to the translation are the per-
Source Language: English
Target Language: French
Target NMT
0.66
Model
0.39
0.81
82'0
caused by the degradation formance of the NMT model resulting from the atlack. For example in Table 2. changing Oregon to Quebee results in two types of changes in the translation: type 1) 'l'Oregon' is replaced with 'le Quebec', " and type 2) some part of the input sentence, "The most eager is is not translated. In order to evaluate if the model is truly fooled by TransFool (i.e., type 2 of the changes), Marian NMT (En-Fr) as the target model and use a back translation model to do round-trip translation (En-Fr and Fr-En). If the target NMT model (Fn-Fr) is performing well, we expect that the English sentence and its back-translated counterpart to have a high similarity, We measure this similarity, in terms BLEURT-20, for the original and adversarial sentences. A lower similarity score for the adversarial sentences indicates that the model's performance is degraded by the attack. Moreover, we can measure the similarity between the original and adversarial sentences in the source language (En) and the similarity between their translations in the target language (Fr). The results averaged across the dataset are depicted in Figure 2 There are two key observations:
Back-Transladd Translation
0.49
Model
Original Sentence
Back translation Original Sentence
Translation of Original Sentence
A Adversarial Sentence
Back translation of A diversarial Sentence
Translation of A Adversarial Sentence
Ground-truth Trenslation
Similarity by BLEURT-20 between different pairs in case of attacking Marian NMT (En-Fr).
Figure 2:
consider the
The round-trip translation quality for the original sentences is higher than that of the adversarial sentences (0.81 vs. 0.73). This finding suggests that the attack successfully degraded the model's performance, and not all of the changes to the translation are of type one, i... the direct results of the perturbations.
11

---

Published in Transactions on Machine Learning Research (06/2023)
2.
The similarity between the original and adversarial sentences in the source domain (En) is 0.66, while the similarity wetween their translations in the target domain (Fr) is 0.49. This unexpected discrepancy again suggests that the attack successfully degraded the model's performance.
Therefore, these two observations confirm that TransFool successful in fooling the NMT model.
5.4.4 Human Evaluation
We conduct a human evaluation campaign to further evaluate the adversarial attacks against Marian NMT (En-Fr) in the white-box setting. Specifically, we assess TransFool, kNN, and Seq2Sick attacks based on three criteria: fluency, semantic preservation, and translation quality. Appendix provides more details about our setup, while the results of each survey along with their 95% confidence intervals are reported.
C
Our findings demonstrate that the adversarial examples generated by TransFool are more semantic-preserving and fluent than both baselines. We also conduct separate survey only for the adversarial examples generated by TransFool, where the annotators are asked to rate the accuracy of the reference translations and the NMT translations. The results show that the NMT translation has lower score than the reference translations. We also report the Inter-Amotttor Agreementt (IAA) between the human judgments in Appendix Overall, our human evaluation results demonstrate the superiority of TransFool over the baselines.
6 Conclusion
In this paper, we proposed TransFool, white-box adversarial attack against NMT models, by introducing a new optimization problem solved by an iterative method based on gradient projection. We utilized the embedding representation of a language model to impose a similarity constraint on the adversarial examples. Morcover, by considering the loss of an LM our optimization problem, the generated adversarial examples are more fluent. Extensive automatic and human evaluations show that TransFool is highly effective in different translation tasks and against different NMT models. Our attack is also transferable to black-box settings with different structures and even different target languages. In both white-box and black-box scenarios, TransFool obtains improvement over the baselines in terms of success rate, semantic similarity, and fluency. TransFool demonstrate that the translations of two similar sentences (original and adversarial ones) by the NMT model can differ significantly, and even small perturbations such as a name change can degrade the translation quality. These adversarial attacks against NMT models are critical to analyze the vulnerabilities of NMT models, to measure their robustness, and to eventually build more robust NMT models.
Acknowledgments
This work has been partially supported by armasuisse Science and Technology project MULAN. We would also like to thank the anonymous reviewers whose valuable comments improved the paper.
References
Moustafa Alzantot, Yash Sharma, Ahmed Elgohary, Bo-Jhang Ho, Mani Srivastava, and Kai-Wei Chang. Generating natural language adversarial examples. In Proceedings of the 2018 Conference Empirical Methods in Natural Language Processing, pp. 2890 2896, 2018.
Dzmitry Bahdanau, Kyung Hyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. In 3rd International Conference on Leurning Representations, ICLR 2015, 2015.
Yonatan Belinkov and Yonatan Bisk. Synthetic and natural noise both brcak neural machine translation. In International Conference on Learning Representations, 2018.
Ondřcj Bojar, Christian Buck, Christian Federmann, Barry Haddow, Philipp Koehn, Johannes Leveling, Christof Monz, Pavel Pecina, Matt Post, Herve Saint-Amand, al. Findings of the 2014 workshop on
12

---

Published in Transactions on Machine Learning Research (06/2023)
statistical machine translation. In Proceedings of the ninth workshop on statistical machine tronslation, pp. 12-58 2014.
Kunbei Cai, Md Hafizul Islam Chowdhuryy, Zhenkai Zhang, and Fan Yao. Seeds of seed: Nmt-stroke: Diverting neural machine translation through hardware-based faults. In 2021 International Symposium on Secure and Private Execution Enwironment Design (SEED), pp. 76-82. IEEE, 2021.
Daniel Cer, Mona Diab, Eneko Agirre, Tnigo Lappzo and Lucia Specia. Semeval-2017 task 1: Semantic textual similarity multilingual and crosslingual focused evaluation. In Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-200 pp. 1-14, 2017.
Akshay Chaturvedi, Abijith KP, and Utpal Garain. Exploring the robustness of nmt systems to nonsensical inputs. Xiv preprint orXiu:1908.011 01119008.111000 0165 2019.
Akshay Chaturvedi, Abhisek Chakrabarty, Masao Utiyama, Eiichiro Sumita, and Utpal Garain. Ignorance is bliss: Exploring defenses against invariance-based attacks neural machine translation systems. IEEE Transactions Arlificial Intelligence, 2021.
Minhao Cheng, Jinfeng Yi, Pin- Yu Chen, Huan Zhang, and Cho-Jui Hsieh. Seq2sick: Evaluating the robustness of sequence-to-seequenee models with adversarial examples. In Proceedings of the AAAI Conference on Artificial Intelligence, volune 34, pp. 3601 3608, 2020a.
Yong Cheng, Zhaopeng Tu, Fandong Meng, Junjie Zhai, and Yang Liu. Towards robust neural machine translation. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1756-1766, 2018.
Yong Cheng, Lu Jiang, and Wolfgang Macherey. Robust neural machine translation with doubly adversarial inputs. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 4324-4333, 2019.
Yong Cheng, Lu Jiang, Wolfgang Macherey, and Jacob Eiscnstein. Advaug: Robust adversarial augmen- talion for neural machine translation. In Proceedings of the 58th Anauall Meeling of the Associalion for Computational Linguistics, pp. 1961-5970, 2020b.
Javid Ebrahimi, Daniel Lowd, and Dejing Dou. On adversarial examples for character-level neural machine translation. In Proceedings of the 27th International Conference on Compulational Linguistics, pp. 653 663, 2018a.
Javid Ebrahimi, Anyi Rao, Daniel Lowd, and Dejing Dou. Hotflip: White- box adversarial examples for text classification. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Popers), pp. 31 31-36, 2018b.
Denis Emelin, Ivan Titov, and Rico Sennrich. Detecting word sense disanbiguation biases in machine translation for model-agnostic adversarial attacks. In The 2020 Conference on Empirical Methods in Natural Language Processing, pp. 7635 7653. Association for Computational Linguisties, 2020.
Markus Freitag, Ricardo Rei, Nitika Mathur, Chi-kiu Lo, Craig Stewart, George Foster, Alon Lavie, and Ondiej Bojar. Results of the wmt21 metrics shared task: Evaluating metrics with expert-based human evaluations on ted and news domain. In Proceedings of the Siath Conference on Machine Translation, pp. 733-774, 2021.
Yvette Graham, Timothy Baldwin, Alistair Moffat, and Justin Zobel. Continuous measurement scales human evaluation of machine translation. In Proceedings of the 7th Lünguistic Annotation Workshop and Interoperability with Discourse, pp. 33-41, 2013.
Yvette Graham, Timothy Baldwin, Alistair Moffat, and Justin Zobel. Can machine translation systems be evaluated by the crowd alone. Natural Language Engineering, 23(1):3 30, 2017.
13

---

Published in Transactions on Machine Learning Research (06/2023)
Chuan Guo, Alexandre Sablayrolles, Hervé Jégou, and Douwe Kiela. Sradient-baseee adversarial attacks against text transformers. In Proceedings of the 2021 Conference on Empirical Methods in Natural Longuage Processing, pp. 5747 5757, 2021.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. Tn Proceedings of the IEEE conference computer vision and pattern recognition, pp. 770 778, 2016.
Di Jin, Zhijing Jin, Joey Tianyi Zhou, and Peter Szolovits. Is bert really robust? a strong baseline for natural language attack on text classification and entailment. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pp. 8018-8025, 2020.
Marcin
Roman Grundkiowicz, Tomasz Dwojak, Hicu Hoang, Kenneth Heafield, Tom Neckermann, Frank Seide, Ulrich Germann, Alham Fikri Aji, Nikolay Bogoychev, André F. T. Martins, and Alexandra Birch. Marian: Fast neural machine translation in C++. In Proceedings of ACL 2018, System Demonstrations, pp. 116 121, Melbourne, Australia, July 2018.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint wrXiu:142.6698/ 2014.
Quentin Lhoest, Albert Villanova del Moral, Yacine Jernite, Abhishek Thakur, Patrick von Platen, Suraj Patil, Julien Chaumond, Mariama Drame, Julien Plu, Lewis Tunstall, Joe Davison, Mario Šaško, Gunjan Chhablani, Bhavitvya Malik, Simon Brandeis, Teven Le Scao, Victor Sanh, Canwen Xu, Nicolas Patry, Angelina McMillan-Major, Philipp Schmid, Sylvain Gugger, Clément Delangue, Théo Matussière, Lysandre Debut, Stas Bekman, Pierric Cistac, Thibault Goehringer, Victor Mustar, François Lagunas, Alexander Rush, and Thomas Wolf. Datasets: A community library for natural language processing, In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pp. 175-184, Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics. URL httts:://cclasi haanaat//acl .rg. org/2021 emnlp-demo.... 21
Linyang Li, Ruotian Ma, Qipeng Guo, Xiangyang Xue, and Xipeng Qiu. Bert-attack: Adversarial attack against bert using bert. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 6193 6202, 2020.
Qingsong Ma, Ondrej Bojar, and Yvette Graham. Results of the wmn188 metrics shared task: Both characters and embeddings achieve good performance. In Proceedings of the third conference on machine translation: shared task papers, pp. 671-688, 2018.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations, ICLR 2018, 2018.
Paul Michel, Xian Li, Graham Neubig, and Juan Pino. On evaluation of adversarial perturbations for sequence-to-scquenee models. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguislies: Human Language Technologies, Volume 1 (Long and Short Papers), pp. 1104-1 2019.
Seyed-Mohsen Moosavi-Dezfooll Alhussein Fawzi, and Pascal Frossard. Deepfool: simple and accurate method to fool deep neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2574-2582. 2016.
John Morris, Eli Lifland, Jack Lanchantin, Yangfeng Ji, and Yanjun Qi. Reevaluating adversarial examples in natural language. In Findings of the Association for Computational Linguistics: EMNLP 2020, pp. 3829 3839, 2020.
Daniel Naber et al. A rulc-based style and grammar checker. 2003.
Nathan Ng, Kyra Yee, Alexci Bacvski, Myle Ott, Michael Auli, and Sergey Edunov. Facebook fair's wmt19 news translation task submission. In Proceedings of the Fourth Conference on Machine Translation (Volume 2: Shared Task Papers, Day 1), pp. 314 -19, 2019.
14

---

Published in Transactions on Machine Learning Research (06/2023)
Guillermo Ortiz-Jiménez, Apostolos Modas, Seyed-Mohsen Moosavi-Dezfooli and Pascal Frossard. Optimism in the face of adversity: Understanding and improving deep learning through adversarial robustness. Proceedings of the IEEE, 109(5):635 659, 2021.
Jungsoo Park, Mujeen Sung, Jinhyuk Lee, and Jaewoo Kang. Adversarial subword regularization for robust neural machine translation. In Findings of the Association for Computational Linguistics, ACL 2020: EMNLP 2020, pp.
Association for Computational Linguistics (ACL), 2020.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, pegfpperooraccc deep learning library. Advances in newral informalion processing systems, 32, 2019.
Maja Popović. chrf: character n-gram f-score for automatic mt evaluation. In Proceedings of the Tenth Workshop Statistical Machine Translation, pp. 392-395, 2015.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAT I blog, 188):9, 2019.
Shuhuai Ren, Yihe Deng, Kun He, and Wanxiang Che. Generating natural language adversarial examples through probability weighted word saliency. In Proceedings of the 57th annual meeting of the association for compulational linguistics, pp. 1085 1097, 2019.
Sahar Sadrizadeh, Ljiljana Dolamic, and Pascal Frossard. Block-sparse adversarial attack to fool transformer- based text classifiers. In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 7837-7841. IEEE, 2022.
Thibault Sellam, Amy Pu, Hyung Won Chung, Sebastian Gehrmann, Qijun Tan, Markus Freitag, Dipanjan Das, and Ankur Parikh. Learning to evaluate translation beyond english: Bleurt submissions to the wmt metrics 2020 shared task. In Proceedings of the Fifth. Conference on Machine Translation, pp. 921-927, 2020.
Hassan S Shavarani and Anoop Sarkar. Belter neural machine translation by extracting linguistic infor- mation from bert. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pp. 2772 2783, 2021.
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. In 2nd International Conference on Learning Representations, ICLR 2014, 2014.
Weiting Tan, Shuoyang Ding, Huda Khayrallah, and Philipp Kochn. Doubly-trained adversarial data augmentation for neural machine translation. arXiv e-prints, pp. arXiv-2110, 2021.
Yuqing Tang, Chan Tran, Xian Li, Peng-Jen Chen, Naman Goyal, Vishrav Chaudhary, Jiatao Gu, and Angela Fan. Multilingual translation with extensible multilingual pretraining and finetuning. arXiv preprint ar arXiv:2008.00// 2020.
Ian Tenney, Patrick Xia, Berlin Chen, Alex Wang, Adam Poliak, R Thomas McCoy, Najoung Kim, Benjamin Van Durme, Samuel R Bowman, Dipanjan Das, et al. What do you learn from context? probing for sentence structure in contextualized word representations. In 7th International Conference on Learning Representations, ICLR 2019, 2019.
Jörg Tiedemann. Parallel data, tools and interfaces in opus. In Eight International Conference on Language Resources and Evaluation, MAY 21-27, 2012, Istanbul, Turkey, pp. 2214-2218, 2012.
Ashish Vaswani, Noam Shazoer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in neural information processing systems, pp. 5998 6008, 2017.
15

---

Published in Transactions on Machine Learning Research (06/2023)
Lucas Nunes Vieira, Minako O' O'Hagan, and Carol O'Sullivan. Understanding the socictal impacts of machine translation: a critical review of the literature on medical and legal use cases. Information, Communication & Society, 24(11):1515 1532, 2021.
Eric Wallace, Mitchell Stern, and Dawn Song. Tmitation attacks and defenses for black-box machine translation systems. In Proceedings of the 2020 Conference Empirical Methods in Natural Language Processing (EMNLP), pp. 5531 15446, 2020.
Boxin Wang, Chejian Xu, Xiangyu Liu, Yu Cheng, and Bo Li. Semattack: Natural textual attacks via different semantic spaces. In Findings of the Associalion for Computational Lénguisties: NAACL 2022, pp- 176-205, 2022.
Jun Wang, Chang Xu, Francisco Guzmán, Ahmed El-Kishky, Yuqing Tang, Benjamin Rubinstein, and Trevor Cohn. Putting words into the system's mouth: A targeted attack on neural machine translation using monolingual data poisoning, In Findings of the Association for Computational Linguislies: ACL-IJCNLP
2021, pp. 1463-1444 1473, 2021.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush. Transformers: State-of-the-ar natural language processing. In Proceedings of the 2020 Conference Empirical Methods in Natural Language Processing: System Demonstrations, pp. 38-45, Online, October 2020. Association for Computational Linguistics. URL https://wwen https://www.aclveb acwweb..lbb.b..... wwwg/aatologg////000000 wttp://ooooo/2222..mmeeeeeeeeeee emnlp-demos.0 6
Chang Xu, Jun Wang, Yuqing Tang, Francisco Guzmán, Benjamin IP Rubinstein, and Trevor Cohn. A targeted attack on black-bor neural machine translation with parallel data poisoning. In Proceedings of the Web Conference 2021, pp. 3638-300 0088,,00,440 2021.
Yinfei Yang, Daniel Cer, Amin Ahmad, Mandy Guo, Jax Law, Noah Constant, Gustavo Hernandez Abrego, Steve Yuan, Chris Tar, Yun-Hsuan Sung, et al. Multilingual universal sentence encoder for semantic retrieval. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pp. 87-94, 2020.
Heng Yu, Haoran Luo, Yuqi Yi, and Fan Cheng. A2r2: Robust unsupervised neural machine translation with adversarial attack and regularization representations. IEFE Access, 9:19990- 19998, 2021.
Yuan Zang, Fanchao Qi, Chenghao Yang, Zhiyuan Liu, Meng Zhang, Qun Liu, and Maosong Sun. Word-level textual adversarial attacking as combinatorial optimization. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 6066-6080 2020.
Biao Zhang, Philip Williams, Ivan Titov, and Rico Sennrich. Improving massively multilingual noural machine translation and zero-shot translation. In Proceedings of the 58th Annuull Meeling of the Associalion for Computational Linguistics, pp. 1628-1688888 1008-163,,,00 2020.
Tianyi Zhang, Varsha Kishore, Felix Wa, Kilian Q Weinberger, and Yoav Artzi Bertscore: Evaluating text. generation with bert. In International Conference on Learning Representations, 2019.
Xinze Zhang, Junzhe Zhang, Zhenhua Chen, and Kun He. Crafting adversarial examples for neural machine
translation. In Proceedings of the 59th Annual Mecting of the Association for Computational Linguistics and the 1 11th International Joinl Conference on Natural Language Processing (Volume 1: Long Papers), pp. 1967-1977, 2021.
16

---

Published in Transactions on Machine Learning Research (06/2023)
Supplementary Material
TransFool: An Adversarial Attack against Neural Machine Translation Models
Abstract
In this supplementary material, we first provide some statistics of the evaluation datasets in Section A The ablation study of the is presented in Section B the effect of hyperparameters (Section B.1, different aspect of the language model (Section B.2]. and effect of NMT model beamn size (Section B.3). We conducted human evaluation whose results and details are presented in Soction More results of the white-box attack are reported D the results of other similarity metrics (Section D.I), performance of TransFool against other translation tasks (Section D.2 performance over successful attacks (Section D.3), trade-off between success rate and similarity Aimilarity/fluencyy (Section D.4 and some generated adversarial examples (Section D.5 Section E provides more experiments on the black-box attack: the performance of attacking Google Translate (Section E.1 results of other similarity metries (Section E.2). and some generated adversarial examples (Section
We discuss the effect Finally, the license information
E33
of the back-translation model choice on WSLS Section F
and more details of the assets (datasets, codes, and models) are provided in Section G
A Some statistics of the Datasets
Some statistics, including the number samples, the Average length of the sentences, and the trans- lation quality of Marian NMT and mBART50, of the evaluation datasets, i.c., OPUS100 (En-Zh) WMT14 (En-PR) and (En-De), are reported in
Table 7: Some statistics of the evaluation datasets.
Dataset
Average
#Teest Samples
Marian NMT
mBART50
Length
BLEU
chrF
BLEU
chrF
En-Fr
WMT14
27
3003
39.88
64.94
36.17
62.66
En-De
table 7
WM114
26
3003
27.72
58.50
25.66
57.02
En-Zh
OPUS-100
18
2000
33.11
50.98
29.27
41.92
B
Ablation Study
B.1 Effect of Hyperparameters
In this Section, we analyze the effect of different hyperparameters (including the coefficients a and / in our optimization problem (T), the step size of the gradient descent y, and the relative BLEU score ratio À in the stopping criteria Eq. on the white-box attack performance in terms of success rate, semantic similarity, and perplexity score.
In all the experiments, we consider English to French Marian NMT model and evaluate over the first 1000 sentences the test set of WMT14. The default values for the hyperparameters are as follows, except for the hyperparameter that varies in the different experiments, respectively: a = 20, 8 = 1.8, 7 = 0.016, and X 0.4.
Effect of the similarity coefficient This hyperparameter determines the strength of the similarity term in the optimization problem TT. Figure 3a shows the effect of on the performance of our attack. By increasing the similarity coefficient of the proposed optimization problem, we are forcing our algorithm to find adversarial sentences that are more similar to the original sentence. Therefore, as shown Figure Ba, larger values of result higher semantic similarity. However, this case, harder to fool the NMT model, i.c., lower attack success rate, RDBLEU, and RDchrF. Moreover, it seems that, since the generated adversarial examples more similar to the original sentence, they more natural, and their perplexity score lower.
17

---

Published in Transactions on Machine Learning Research (06/2023)
Attack Sucess Rate
0.90-500
0.85
400 70-
75
0.88
0.87
200
180
80
0. 0.86-300
70
0.88
0.88
180
200
Score
Sim.
ASR
00.0
Sim.
ASR
0.86
Sim.
ASR
0.84
250 50-
ASR
0.80
160
60-
60-
0.02
200
Perp
0.85
40
25
0,70
200
55-
50
1.25
1.50
1.55
2.00
140
12
14
2.0
0.80
150 30-
0.6
0.8
0.88
0.83
120
Similarity Coefficient
Coefficient
Step Size
xle-2
BLEU Score Ratio
(a)
(b)
(c)
(d)
0.70
0.35
0.57
BLEU chrF
0.24
0.65
chrF
0.27
0.35
0.24
RDBLEU
0.22
0.60
0.54
0.50
0.55
0.25
0.51
0.21
0.55
0.2
0.45
0.20
0.20
0.50
0.40-
0.18
0.45
chff
0.48
0.19
0.19
0.35
chrF
0.16
15
25
35
1.25
1.50
1.75
2.00
1.2
14
1.6
1.9
2.0
0.2
0.4
Similarity Coefficient
LM loss Coefficient
Step Size
xle-2
BLEU Score Ratio A
(e)
(L)
(h)
Figure 3: Effect of different hyperparameters the performance of TransFool.
Effect of the language model loss coefficient B. We analyze the impact of the hyperparameter B, which controls the importance of the language model loss term in the proposed optimization problem, in Figure 3b By increasing this coefficient, we weaken the effect of the similarity term, i.e., the generated adversarial examples are less similar to the original sentence. As a result, the success rate and the effect on translation quality, i.e. RDBLEU and RDchrF, increase.
Effect the step size 7. The step size of the gradient descent step of the algorithm can impact the performance of our attack, which is investigated in Figure 3el Increasing the step size results in larger movement in the embedding space in each iteration of the algorithm. Hence, the generated adversarial examples are more aggressive, which results in lower semantic similarity and higher perplexily scores. However, we can find adversarial examples more easily and achieve a higher attack success rate, RDBLEU, and RDchRF.
Effect of the BLEU score ratio A. This hyperparameter determines the stopping criteria of our iterative algorithm. Figure 3d studies the effects of this hyperparameter on the performance our attack. As this figure shows, a higher BLEU score ratio causes the algorithm to end in earlier iterations. Therefore, the changes applied the sentence are less aggressive, and hence, we achieve higher semantic similarity and a lower perplexity score. However, the attack success rate, RDBLEU, and RDchrF decrease since we make fewer changes to the sentences.
B.2 Ablation study on the language model
-180
One of the main component our attack the language model, which is used to measure the similarity between tokens.
Sucess Attack Rate
75-
0.86
ntic sigmmmm imilarity
Perplexity Score
0.85
160
We showed in Section 5.4.1 that using the LM embeddings instead of the NMT ones highly affects our attack performance. In this Section, we study the impact of different properties of the LM used in TransFool on the attack performance.
70-
0.84
140
ASR
-.83
Sim.
Perp.
0.82
5
-120
Language Model Quality First, we investigate the effect of the language model quality on the performance. For Marian NMT (En-Fr), we save the checkpoints of the LM and FC layer after each epoch. Then, we use each of these models, and attack 1000 sentences of dataset. The results presented in Figure We that until certain
0
10
30
Epoch No. in Training LM
Figure 4: Effect of LM quality performance.
18

---

Published in Transactions on Machine Learning Research (06/2023)
epoch, similarity and perplexity improves and then, they gradually deteriorate. This may be due to the fact that the LM becomes less generalizable to the dataset we attack after training for some epochs. We should note that we have considered the last LM from the last epoch in all our experiments.
Training Dataset of the Language Model To further study the effect of the language model, we can also fine-tune the language model and the fully-connected layer the translation dataset (WMT14 En-Fr training set). Table 8 show the results when we use this language model in our attack. These results demonstrate that with language model trained over sentences similar to those we are trying to attack, we can improve the perplexity score and similarity of the adversarial examples.
Table 8: Performance of TransFool white-box allack against. Marian NMT (En-Fr) with /ittoutt fine-tuned LM.
Method
ASR1
RDBLEU1
RDehrF1
Sim.1
Perp.L
TER
with Original LM
69.48
0.56
0.23
0.85
177.20
13.91
with fine-tumed LM
61.14
0.52
0.21
0.86
151.92
12.35
B.3 Effect of NMT model Beam Size
Translation models use beam search to generate high-quality outputs. Marian NMT uses a beam size of 4, and mBART50 uses a beam size of To see the effect of this parameter on TransFool performance, we allack Marian NMT (En-Fr) with different beam sizes and the results are presented in Table g The attack performance is not significantly impacted by the beam size, possibly because we employ the training loss for the adversarial loss, and beam size is only used for inference. This analysis shows the consistency in the performance of TransFool when the NMT model has different settings.
Table 9: Performance of TransFool white-box allack against, Marian NMT (En-Fr) with different beam sizes.
Beam Size ASR1
RDBLEU
RDehrFt
Sim.t
Perp.
TER
69.28
0.58
0.24
0.85
172.18
13.70
70.38
0.57
0.23
0.85
178.07
13.81
68.67
0.56
0.23
0.85
176.39
13.92
69.48
0.56
0.23
0.85
177.20
13.91
08.07
0.56
0.23
0.85
174.77
13.80
C Human Evaluation
We conduct a human evaluation campaign of TransFool, kNN, and Seq2Sick atlacks on Marian NMT (En-Fr) in the white-box setting. We randomly choose 90 sentences from the test set of the WMT14 (En-FR) dataset with the adversarial samples and their translations by the NMT model. We split 90 sentences into three different surveys to obtain manageable size for each annotator. We recruited two volunteer annolators for each survey. For the English surveys, we ensure that the annotators are highly proficient English speakers. Similarly, for the French survey, we ensure that the annotators are highly proficient in French.
Before starting the rating task, we provided annotators with detailed guidelines similar to (Cer et al. 2017 Michel et al. 2019). The task is rate the sentences for each criterion on a continuous scale (0-100) inspired by WMT18 practice (Ma et al., 20118) and Direct Assessment (Graham al. 2013 2017). For each sentence, evaluate three aspects in three different surveys:
We show the three adversarial sentences and the original sentence on the same page (in random order). We request the annotators to rate their level of agreement with the statement "The sentence is fluent. for cach sentence.
Fluency:
Semantic preservation: We show the original sentence on top, and below that, we display three adversarial sentences (in random order). We request the annotators to rate their level agreement with the statement "The sentence is similar to the reference text." * for each sentence.
Translation qualily: Inspired by monolingual direct assessment. (Ma et al., 2018 Graham et al. 2013 2017). we evaluate the translation quality by displaying the reference translation at the top and, below that, presenting translations of the three adversarial sentences (in a random order). We request the annotators to rate their level of agreement with the statement "The sentence similar the reference text." for each translation.
19

---

Published in Transactions on Machine Learning Research (06/2023)
Fluecy
100
60-
Seg25ick
TrarsFoo
Orginal
Similarity in
Source Lanquage (En)
50-
660-
70-
Transfool
Sea2
Smilarty in
Langunge
50-
TransFool Seq25ick
Similartyy Decreaze in
Smurce/ (ergggtt Langiages
20-
15
Seg25ick
TransFool
30-
10-
40-
20-
10-
Original TrarsFool KNN
Seq25ick
TransFool
KNN
Seq2Sick
0-
TransFool
KNN
Seq2Sick
TransFoot
KNN
SeažSick
(a)
(b)
(c)
(d)
Figure 5: Human evaluation results for TransFool, KNN, and Seq2Sick attacks against Marian NMT (En-Fr).
Only for the TransFool adversarial sentences conduct fourth survey. We show the English adversarial sentence top, and below that, we display the reference translation or the translation generated by the NMT model. We request the annotators to rate their level of agreement with the statement "The French sentence is good translation for the English sentence." We do this survey solely for TransFool sine the adversarial sentences by other attacks are not fluent and the task could be excessively challenging for the annotators.
To ensure that the two annotators agree, we only consider sentences where their two corresponding scores are less than 30. We average both scores to compute the final score for each sentence. We calculate 95% confidence intervals by using 15K bootstrap replications. The results are depicted in Figure 5 These results show that the adversarial examples generated by TransFool are more cemantic-preserye both baselines. According to the provided guide to the annotators for semantic similarity, the score of 67.8 shows that the two sentences are roughly equivalent, but some details may differ. Moreover, a fluency of 66.4 demonstrates that although the generated adversarial examples by TransFool are more fluent than the baselines, there still to improve the performance in this regard.
and fluent than
We follow the direct assessment strategy to measure the effectiveness of the adversarial attacks translation quality. According to (Ma et al.] 2018), since a sufficient level of agreement of translation quality is difficult to achieve with human eval evaluation, direct assessment simplifies the task to a simpler monolingual assessment instead of bilingual task. The similarity of the translations of the adversarial sentences with the reference translation is shown in Figure [c] The similarity of Seq2Sick worse than other attacks. However, its similarity in the source language is also worse. Therefore, we compute the decrease of similarity (between the original and adversarial sentences) from the source language to the target language. The results in Figure [d]] show that all attacks affect the translation quality and the effect of TransFool is more pronounced than that of both baselines.
Regarding the fourth survey on TransFool adver- sarial sentences, for the successful adversarial ex- amples, the scores and their corresponding confi- dence interval for the translations generated by the NMT model and the reference translations are pre- sented in Figure We can see that the reference translation has got higher score than the NMT translations, which suggests that the translation quality of the target model is indeed degraded.
Translation Quality for TransFool Sentences
80-
60
40
20-
Reference
NMT
Reference
NMT
Figure 6: Human evaluation for TransFool translations.
Finally, for all surveys, we calculate Inter- Annotator Agreement (LAA) between the two hu- man judgments for each sentence. We compute IAA in terms of Pearson Correlation coefficient instcad of the commonly used Cohen's K since scores are in a continuous scale. The results are presented in Table I0 Overall, we conclude that we achieve good inter-annotator agreement for all sentence types and evaluation metrics. More- over, for the fourth survey, the TAA is 0.40 and 0.65 for the reference translations and the NMT translations,
Table 10: IAA for different surveys in human evaluation.
Sentence Type Fluency Similarity in En Similarity in Fr
Original
0.68
TransFool
0.79
KNN
0.91
0.82
0.86
Seg2Sick
0.89
0.88
0.83
20

---

Published in Transactions on Machine Learning Research (06/2023)
respectively. The lower LAA for this survey compared to the other ones is due the the challenging task of translation quality assessment. in two languages Ma et al. (0018)
D More Results on the White-box Attack
D.1 Semantic Similarity Computed by Other Metrics
To better assess the ability of adversarial attacks in maintaining semantic similarity, we can compute the similarity between the original and adversarial sentences using other metrics such as BERTScore (Zhang ett al. 2019) and BLEURT-20 (Sellam et al. 2020). It is shown (Zhang et al. 2019) that BERTScore correlates well with human judgments. BLEURT-20 also shown to correlates better with human judgment than traditional measures (Frcitag et al. 2021). The results reported in Table 11 These results indicate
that the Tool is indeed more capable of preserving the semantics of the input sentence. In the two cases
where kNN has better similarity by using the Universal Sentence Encoder (USE) (Yang et al. 2020). the performance of TransFool is better in terms of BERTScore and BLEURT-20.
Table 11: Similarity performance of white-bax attacks.
Marlan NMT
mBART50
Task
Method
USE
BERTScoref
BLEURT-20
USET
BERTScoref
BLEURT-20 -20 f
TransFool
0.85
0.95
0.65
0.84
0.96
0.70
En-Fr
kNN
0.82
0.94
0.61
0.85
0.93
0.67
Seq2Sick
0.75
0.94
0.75
0.94
0.66
TransFool
0.84
0.96
0.67
0.83
0.95
0.69
En-De
kNN
0.82
0.94
0.61
0.86
0.93
0.67
Soq2Sick
0.67
0.93
0.52
0.66
0.92
0.58
TrensFool 0.88
0.96
0.67
0.90
0.97
0.76
En-Zh
kNN
0.86
0.95
0.66
0.90
0.95
0.72
Seq2Sick
0.73
0.94
0.54
0.78
0.95
0.67
D.2 Performance of TransFool against other Translation Tasks
We also tested the gencralizability of TransFool by attacking English-to-Russianr (En-Ru) and English-to-Caze Czcch (En-Cs) translation lasks. The results are reported in Table 12 We can see that the performance of TransFool in different translation tasks is consistent with the previous ones we studied.
Table 12: Performance of TransFool white-box attack against other translation tasks
Marian NMT
mBART50
Task
ASRI
RDBLEU?
RDchrPt
Sim.t Parp4
TER.
ASRt
RDBLEU1
RDcuF1
Sim.f
Purp.j
TER.
En-Ru 75.78
0.65
0.27
0.84
209.82
14.60
61.99
0.57
0.24
0.86
111.01
9.66
En-Ca 68.81
0.65
0.24
0.86
191.36
12.68
59.84
0.61
0.23
0.84
116.89
10.26
D.3 Performance over Successful Attacks
The evaluation metrics of the successful adversarial examples that strongly affect the translation quality are also important, and they show the capability of the adversarial attack. Hence, we evaluate TransFool, kNN, and Seq2Sick only over the successful adversarial examples 0 The results for the whito-box setting are presented in Table 13 By comparing this Table and Table 1 which shows the results on the whole dataset, we can see that TransFool performance is consistent among successful and unsuccessful attacks. Moreover, successful adversarial examples generated by TransFool are still semantically similar to the original sentences, and their perplexity score low. However, the successful adversarial examples generated by Seq2Sick and kNN do not preserve the semantic similarity and are not fluent sentences; hence, they are not valid vevesaaiial sentences.
6 As defined Section adversarial example successful if the BLEU score of translation is less than half of the BLEUU score the original translation.
21

---

Published in Transactions on Machine Learning Research (06/2023)
Table 13: Performance of white- box attack over successful adversarial examples.
Marian NMT
mBART50
Task
Method
ASR1
RDBLEU1
RDohrF
Sim.7
Perp.]
TERL
ASR1
RDBLEUT
RDehrF1
Sim-f
Perp.]
TER.
TransFool
69.38
0.66
0.26
0.83
229.75
15.33
60.68
0.06
0.27
0.82
164.52
12.56
En-Ft
kNN
36,53
0.70
0.30
0.76
746.89
24.52
30.81
0.72
0.28
0.77
091.61
28.05
Seq2Sick
27.01
0.72
0.40
0.56
648.92
25.28
25.53
0.74
0.41
0.53
556.61
25.16
TransFool
69.49
0.72
0.25
0.83
191.51
14.54
62.87
0.73
0.26
0.81
169.76
12.66
En-De
KNN
39.22
0.75
0.29
0.77
675.01
23.07
35.99
0.75
0.23
0.81
574.68
25.75
Seq2Sick
35.60
0.78
0.40
0.53
659.90
25.67
35.59
0.78
0.40
0.52
612.22
26.67
TransFool
73.82
0.76
0.34
0.87
112.28
12.83
57.50
0.73
0.31
0.88
99.08
9.86
En-Zh
KNN
31.12
0.72
0.29
0.80
355.25
22.55
27.25
0.76
0.27
0.85
295.53
23.58
Seq2Sick
28.76
0.72
0.46
0.58
437. 49
26.84
24.25
0.79
0.44
0.00
292.55
25.59
D.4 Trade-off between Success Rate and Similarity/Fluuncy
The results our ablation study B show that there is a trade-off between the quality of adversarial example, in terms of cemaatic-preservation and fluency, and the attack success rate. As studied (Morris al., 20200; we can filter adversarial examples with low quality based on hard constraints on semantic similarity and the number of added grammatical errors caused by adversarial perturbations.
We can analyze the trade-off between success rate and munilarity/fllenge by setting different thresholds for filtering adversarial examples. If we evaluate the similarity by the sentence encoder suggested in (Morris et al. 2020), the success rate with different threshold values for similarity in the case of Marian (En-Fr) is depicted in Figure 7b By considering only the adversarial examples with a similarity higher than a threshold, the success rate decrcases as the threshold increases, and the quality of the adversarial examples increascs.
Similarly, we can do the same analysis for flu-
ency. As suggested in (Morris et al. 2020 we count the grammatical errors by Language- Tool (Naber el al. 2003) for the original sen-
Attack Sucess Rate
60
TransFool
KNN
Attack Sucess Rate
40-
TransFool
KNN
50
Seq25ick
30-
Seq2 Sick
40-
tences and the adversarial examples. F Figure 7a
20-
depicts the success rate for different thresholds
30-
of the number of added grammatical errors caused by adversarial perturbations.
20-
10-
10
0-
0.0
0.5
1.0
1.5
2.0
0.80
0.85
0.90
0.95
These analyses show that with tighter con- straints, we can generate better adversarial examples while the success rate decreases. All in all, according to these results, TronsFool out performs the baselines for different thresh- olds of similarity and grammatical errors.
Threshold for Grammar
Threshold for Similarity
(a)
(b)
Figure 7: Tradeoff between success rate and Similarity/ /fluency. The left figure shows the effect of acceptable number of added grammar errors by adversarial perturbation. The right figure shows the effect of similarity threshold.
D.5 More Adversarial Examples
In this Section, we present more adversarial examples generated by TransFool, kNN, and Seq2Sick. In order to show the effect of using LM embeddings on the performance of TransFool, we also include the generated adversarial examples against English to French Marian NMT model when we do not use LM embeddings. In all these tables, the tokens modified by TransFool are written blue in the original sentence, and the modified tokens by different adversarial atlacks are written in red in their corresponding adversarial sentences. Moreover, the changes made by the adversarial attack to the translation that not directly related to the modified tokens are written in orange, while the changes that are the direct result of modified tokens are written in brown.
As can be seen in the examples presented in Table 14 TransFool makes smaller changes to the sentence. The generated adversarial example is a correct English sentence, and it is similar to the original sentence. However, kNN, Seq2Sick, and our method with the NMT embeddings make changes that are perceptible, and the adversarial sentences are not necessarily similar to the original sentence. The higher semantic similarity
22

---

Published in Transactions on Machine Learning Research (06/2023)
of the adversarial sentences generated by TransFool is due to the integration of LM embeddings and the LM loss in the proposed optimization problem. We should highlight that TransFool is able to make changes to the adversarial sentence translation that are not directly related to the modifications of the original sentence but are the result of the NMT model failure.
Table 14: Adversarial examples against Marian NMT (En-Frr) by various methods (white-box).
Sentence
Text
Org.
The most eager is Oregon, which is enlisting 5,000 drivers in the country's biggest experiment.
Le plus déterminé est I'Oregon, qui a mobilisé 5 000 conducteurs pour mener l'expárience la plus importante du pays.
Ref. Trans.
Org. Trams
Le plus avide est I'Oregon, qui recrute 000 pilotes daus la plus grande expérience du pays.
Adv. TransFool
The most eager isQuebee, which is enlisting 5,000 drivers the country's biggest experiment.
Le Québee, qui fait partie de la plus graude expérience du pays, compte 5 000 pilotes. (some parts not trunslated of alL)
Trus.
Adv. w/ NMT Emb.
The most euger isCustom, which is ealisting Disk drivers in the countey's editions Licensee.
Trans.
Le plas avide estCom,, qui vecrute des pilotes de disque dans les éditions du pays Liceucié. Theve tager is Oregun, C aren ealisting 5,000 drivers in theau's biggest experiment. Theve avide est. Oregon, C sout enrikis 000 pilotes daus la plus gradde expérience de Theau.
Adv. KNN
Trans.
Adv. Soq2Sick
The most buzz FREE, which is chooseing Games comments the country's great developer.
Trans.
Le plus buzz est GRATUIT, qui est choisir Jeux commentaires dans le grand développeur du pays.
Other examples against different tasks and models are presented in Tables 15 to 19
Table 15: Adversarial examples against Marian NMT (En-De) by various methods (white-box).
Sentence
Text
The devices, which trak every mile a motorist drives and transmit that Information to bureaucrats, are at the cuueer of a controversial altenpl iu Washington and state planming offices to overlaul the outdated system for funding America's is major roads.
Org.
Die Geräte, die jeden gefahremen Kilometer aufxeichnen und die Informationen an die Behörden melden, sind Kernpunkt eines kontroversen Versuchs vou Washington und dea Planangsbrros der Bundesstaaten, das veraltete System xur Finauierung US-meerik Strallnn su üiberabeilen.
Ref, Trans.
Die Geräte, die jede Meile ein Autofahrer fihut und diese Informationen an Bürokraten weiterleitet, stelien im Zentrum vines umstrittenen Versuchs Washington und den staatlichen Planugsbürms, das veraltete System Finanzlerung der großen Straßen Amerikas zu überarbeiteu.
Org. Trans.
The vehicles, which track every mile motorist drives and transmit information to hureauerats, are the center of unjustified atlempt Washington and city plaming offices overlmul the clearer system for funding America's is major roads.
Adv. TransFool
Die Pahrzenge, die jede Meile rin Autofahher fähet und diese Informatimen an Bürokraten weiterleitet, stehen im Zentrum eines ungerriifferiigen Versurhs in Washington und den Stadtplanmnpsiiree, das klaree System zur Finanziering der amerikanischen Hauptstradenn zu übbrarbeiten.
Trans.
The devices in which track every mile motorist drives and transmit that M bureancrats, are 07:0 the center of contrwersial attempt in Washington and state planning officrs to overhaul the outdated Estate for funding America's 's major roads.
Adv. KNN
Die Vorrichtungmn, in denen jede Meile ein Autofahrer fährt und diese M Bürokraten libertrigt, sind 07:0 das Zentrum eines unstrittenen Versuchs in Washington und staatlichen Plammgsbiros, das veraltete Estate für die F Finanzierun der amerikanischen Hauptstraßen zu überazbeiten.
Trans.
The devices, which road everyably motorist drives and transmit that information to walnut sociallsms, are at the center of Senate attempt in Washington and state planning offices toestablishment the outdated system for funding America's 's major paths.
Adv. Seq2Siek
Die Gerâte, die allgegenwärtig ein Autofahrer antreibt und diese Informationen an Walmsseozialismnn tibertragt stehen im Zentrum eines Senatsvensuchs in Washington und in den staatlichnn Planugsbúros, das veraltete System zur F Finanzierung der wichtigsten Wege Amerikas cinzurichten.
Trans.
23

---

Published in Transactions on Machine Learning Research (06/2023)
Table 16: Adversarial examples against Marian NMT (En-Zh) by various methods (white-box).
Sentence
Text
Org.
And what your husband said... if Cohmbbss had done it, we'd all be Indians. BE.... EE 12 22 SE at 1 Mi to R2B) T 2 2 35 A.
Ref. Traus.
AT
Org. Truns.
Adv. TransFool
And with your husband said... if Cohumbus had done it, we'd all be Indians. 89 Be0r RR TF (2) T fR: 22 AB Ell A (2..." is not in the tronslation.)
Trans.
Adv. KNN
And what your hushand sald... Columbus bad60, we' Nineteen all it Indians. 8 201 dF fioo" 18.11119 ( A
Trans.
Adv. Seq2Sick
And completing your penalties sald... If timely had done it, we'd all be briefed. 5 to it... tn Re AR. MHR
Trans.
Table 17: Adversarial examples against mBART50 (En-Fr) crafted by various methods (white-box).
Sentence
Text
Wearing winganit, he feww past the famous Monserrate Sanctuary at 160km hh The sanctuary is located at au altitude of over 3000 meters and numerous spectators had gathered there watch his exploit.
Org.
Equipé d'un wingsuit, il passé 160 km fl au-dessus du oélebre sanctuaire Mouserrate, sirué à plus de 3 000 méires d'aliituee, où de nombreux badauds a' z'élnieut resemblés pour observer son exploil.
Ref. Trans
Il a survolé à 160 km/h le celébre sauctmaire de Monserrate, silué une altitude de plus 3000 milree où nombreux spectateurs sont réunis pour assister
Org. Trans.
exploit.
Wearing a wingsnit, he flow past over the famous Interesserrage Sanctuary as 160km/h. The sanctuary is Iocated at altitude of over 3000 meters and numerous spectators had gathered there to watch his exploit. sanctuaire est situé une altitudo de plus de 3000 métres et de nombreux spectateurs se sont réunis pour assister & son exploit. (first port of the sentence is not translated alf.)
Adv. Tral
Trans.
Wearing a wingsuit. be flew past over the femous Monserrate Sanctuary at 160km/h h. The sanctuary located at anzu opinionstitude of 8000 meters and aumerous spectators had gathered there the watch his exploit.
Adv. KNN
a survolé le céltbre sanctuaire de Mouserrate 160 km/h. Le sanctusire est situé une opinioutirude de plus de 8000 métres et de nombreux spectateurs sont réssmbble là pour obeerver son exploit.
Trans.
Adv. Seq2Sick
Wearing a wingsuit, he flew past over the famous Monserrate Sanctuary at 160km/ /h The sanctuary is located at an altitude of over74 meters and mumerous spectators bad gathered there to watch his exploit.
Il a survolé 160 km/h le célébre sanetuaire de Monserrate, situé à plus de 74 mètres d' l'altitude, où de nombreux spectateus se sont réunis pour assister son exploit.
Trans.
Table 18: Adversarial examples against mBART50 (En-De) crafted by various methods (white-box).
Sentence
Text
Org.
In Oregon, planners are experimenting with giving drivers different choices.
Ref, Trans.
In Oregon experimentieren die Planer damit, Autofahrern eine Reihe von Auswahlnigglchkeeeen zu geben.
Org. Trans.
In Oregon experimentieren Planer damit, Fahrern verschiedene Wahlen geben.
Adv. TransFool
Oregon, plannees were experimenting with giving drivers different choices.
Trans.
In Oregon experimentierton Planer min der Bereitstelling unterschiedlieher Wahlmägfihhleiten fir Pahrer.
Adv. KNN
in Oregon, planners nemmeno experimenting withkjer driver. different choices, in Oregon, Planer nemmeno experimentieren mitkjer Fahrer, werschiedene Wahlen, acontoce, plamners are studying with Kivakapis against decisions, In acoutece studieren Planer mill Kivakapis gegen Eutscheidungen,
Trans.
Adv. Seq2Sick
Trans.
Table 19: Adversarial examples against mBART50 (En-Zh) crafted by various methods (white-box).
Sentence
Text
Delegations are requested to submit the names of their representatives to the Secretary of the Preparatory Committee, Ms. Vivian Viner-Josepla (room S-2950E; fax: (212) 963-5935).
Org.
Ref. Trans.
i # H0 22 2o in & 4 Wiwiinaaauueereeeppp.. +S 2950E (S-29000 (212))9963359933)
Org. Trans.
# ** XX Hiviauu Jesepha(S (5 2 (2(212)963 (22222633333368888 10
Delegations are requested to submit the names of their representatives to the Socrelary of the Preparatory
Adv. TransFool
Committer, Mr. Vivian Pliner PlinerJosephhs (room C-2930E; fax: (211) 96 25-30935).
#RRM SERRRE 30985]
19 Has 11
2930E,ff (2 19625
Thans.
Delegations are requested submit the names their representatives that the Secretary the Preparatory Committee, Ms. VivianPliner-Joeen (room $-2950 fax: (212) 963-5935).
Adv. KNN
Traus.
# 2 200 2 Viviianpiiner-JeeephiS.... 2950 1o (212)961-5535)) 22 ft# t
Adv. Seq2Sick
Delegations are requested to submit the names of their representatives to the Secretary of the Preparatory
Committee, Ms Jadan Piner-Josephs (room S-2950E; 599: 212 96 2010,935.
Trans.
# 1& Z1 15
is Hi
4 3L Fl A3
Jo Ae 1 (S-20000)))))) SS--2300E3 3E (59.559992222220 2120220002200
24

---

Published in Transactions on Machine Learning Research (06/2023)
E
More Results on the Black-box Attack
E.1 Attacking Google Translate
To evaluate the effect of different attacks practice, we attack Google Translate by TransFool, kNN, and Seq2Sick. Since querying Google Translate is lim- ited per day, we were not able to attack with WSLS, which requires high number of queries. Table 20 presents the performance of the English to French translation task. The results demonstrate that ad- versarial sentences crafted by TransFool can degrade the translation quality more while preserving the semantics better. The perplexity score and word error rate of TransFool compete with those metrics of Seq2Sick, but Seq2Sick adversarial sentences are not as similar and are less effective.
Table 20: Performance of black-bor box attack against Google Translate (En-Fr).
Method ASRT RDBLEUf RDchrF1 Sim. Parp.
WERI
TransFool 67.83
0.55
0.23
0.85
184.35
20.85
kNN Seq2Sick
37.22
0.35
0.17
0.82
389.45
30.24
23.49
0.20
0.15
0.75
174.88
20.34
Table 21: Performance of TransFool black-box atlack against Google Translate (En-De), when the target language is different..
Task
ASRI RDBLEU RDchrF1 Sim.t Perp.l WERJ
We also performed the cross lonsslinggal black- box attack. We consider Marian NMT (En-Fr) as the reference model and attack En-De Google Translate. The results for TransFool are reported in Table 21
En-Fr + En-De 67.42
0.65
0.26
0.85 198.56 20.78
E.2 Semantic Similarity Computed by Other Metrics
Similar to the white-box attack, we compute the sim- ilarity between the adversarial and original sentences by BERTScore and BLEURT-20. since they correlate well with human judgments. The similarity perfor- mance of TransFool and WSL. in the black-box settings are demonstrated in Table 22 According to Table 22 TransFool is better at maintaining se- mantic similarity. It may be because we used T.M embeddings instead of the NMT ones in the similarity constraint.
Table 22: Similarity performance of black-box attacks.
Task
Method
USEt
BERTScoret
BLEURT-20
TransFool
0.85
0.95
0.66
En-Fr
WSLS
0.84
0.93
0.58
TransFool WSLS
0.84
0.96
0.67
En-De
0.86
0.94
0.61
TransFool WSLS
0.88
0.96
0.68
En-Zh
0.83
0.93
0.56
E.3 Some Adversarial Examples
We also present some adversarial examples generated by TransFool and WSLS, in the black-box setting, Table 23 In this table, the tokens modified by TransFool are written in blue in the original sentence, and the modified tokens by diffcrent adversarial attacks written in red in their corresponding adversarial sentences. Moreover, the changes made by the adversarial allack to the Iranslation that are not directly related to the modified tokens are written in orange, while the changes that are the direct result of modified tokens are written in brown. These examples show that modifications made by TransFool are less detectable, i.., the generated adversarial examples are more natural and similar to the original sentence. Moreover, TransFool makes changes to the translation that are not the direct result of the modified tokens of the adversarial sentence.
We should note that as we do not have tokenizer, compute Word Error Rate (WER) instcad of Token Error Rate (TER). 8The results of kNN and Seq2Sick are reported as they are transfer attacks, and their performance reported Table
25

---

Published in Transactions on Machine Learning Research (06/2023)
Table 23: Adversarial examples against mBART50 (En-Zh) crafted by various methods bllckk-box)..
Sentence
Text
(c) To provide care and support by strengtheuing programiinng for orphans and vulnerable children in- foctod/ /affected by AIDS and by expanding life skills training for young people.
Org.
(c) (e) MURSSE LIIEE A.EEEER Ka (13 15 R. PLUE
A HI
Ref. Trans.
Org. Trans
(e)ttt X K / WEO0L #INH6 Wi # A A. A UI NE
To provide and support by strengthoning programming for orphans and vulnerable children Disabled, afflicted by AIDS and by expanding life skill training for young people.
Adv. TransFool
Trans.
oj 308 (u X (04 58) L of BUBBBE 30 XX; 4:80 15
To provide nursing and unstinted support by strengtheuing Lifetv for orphans and susceptable children infected/aflected by CPR mannequins and broadening life skills training young people.
Adv. WSLS
(c) 19:0 CPR is MO
Lifetv, 3 A
12 16
Trans.
(5o) address provide care and support by strengthening prioritization for orphans and vulnerable children Infected /affected by AIDS and by expanding life skills Issue for young people.
Adv. KNN
B T 84 (OBE (n5t33 /L. Hi /E
WE "O A A A
Trans.
(c) To provide care and support by strengthening digital for dress and harmful children Journal/ Letter by Region and by disappear Violence skills training for young people.
Adv. Smy2Sick
Trans.
(h) Bulk 8TH # 12 7 SEX b0 22 (1 Moll/HH..... (HJ33 FFEE ABE B6 17
F Effect of Translaaann Model Choice on WSLS Performance
WSLS uses a back-translation model for crafting an adversarial example. In (Zhang et al. 2021). the authors investigate the En-De task and use the winner model of the WMT19 De-En sub-track (gg et al. 2019 for the back-translation model.
However, they do not evaluate their method for En-Fr and En-Zh tasks. To evaluate the performance of WSLS in Table 3 We have used pre-trained Marian NMT models for all three translrannlation models. In or- der to show the effect of our choice of back- translation model, we compare the perfor- mance of WSLS for the En-De task when we use Marian NMT or (Ng et al. 2019) as the back-translation model in Table 24 As this Table shows, WSLS with Marian NMT as the back-translation model results in even more semantic similarity and lower perplexity score. On the other hand, WSLS with (Ng et al. 2019) as the back-translation model has slightly more success rate. These results show that our choice of back-translation model does not highly affect the performance of WSLS.
Table 24: Performance of WSLS (En-De) with two back- translation models.
Back- Translation ASR RDBLEU RDchrF Sim. Perp. # Queries
Marian NMT
44.33
0.50
0.19
0.86
219.32
1262
Ng ct al.
2019
51.68
0.58
0.21
0.81
241.96
1307
G License Information and Details
In this Section, we provide some details about the datasets, codes, and models used in this paper. We should note that we used the models and datasets that are available HuggingFace transformers (Wolf et al. 2020 and datasets (Lhoest et al. 2021) libraries They are licensed under Apache License 2.0. Moreover, we used PyTorch for all experiments (Paszke et al. 2019 which is released under the BSD license
G.1 Datasets
WMT14 In the Ninth Workshop Statistical Machine Translation, WMT14 was introduced for four tasks. We used the En-De and En- Fr news translation tasks. There is no license available for this dataset.
OPUS-100 OPUS-100 is a multilingual translation corpus for 100 languages, which is randomly sampled from the OPUS collection (Tiedemann 2012). There is no license available for this dataset.
9These libraries are available at this GitHub repository: httts:://ttttt// httpggtthhbbbmmm/gggggggggtcattt httttp:///gtttb..coo//p////// httpa//www.pp.t..e/.../ta/a//aaa
26

---

Published in Transactions on Machine Learning Research (06/2023)
G.2 Models
Marian NMT Marian is Neural Machine Translation framework, which is mainly developed by the Microsoft Translator team, and it is released under MIT License TT This model uses a beam size of 4.
mBART50 mBART50 is a multilingual machine translation model of 50 languages, which has been introduced by Facebook. This model published the Fairseq library, which released under MIT Licens This model uses beam size of 5.
G.3 Codes
KNN In order to compare our method with kNN (Michel et al. 2019). we used the code provided by the authors, which released under the BSD 3-Clause 'New' or 'Revised" License
Seq2Sick
To compare our method with Seq2Sick (Cheng al. 2020a we used the code published by the
authors
There is license available for their code.
WSLS We implemented and evaluated WSLS Zhang et al. 2021 using the source code published by the
authors
15
19e source code available at httphh/hhhh...cccooooooooaaaaatn
adversarial/apperinnttt and the liceuse avialable
The source available https://github.cco
https://github. con/ JHL. HUST/ A http:///hhhuubcccoommmmmccsssaaa
27


Reviews:
[{"review_id": "OK9zjBcHJx", "reviewer": "Reviewer_wWrQ", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "\nStrengths:\n\n- The proposed model is relatively easy to understand and obtains good performance.\n- The authors show that TransFool could transfer to black-box and cross-lingual setups.\n\n\nWeaknesses:\n\n- Some statements in this paper are confusing or misleading.\n- Important ablation studies are missing.\n- Evaluation could be improved: human evaluation is strongly suggested in some cases.\n- The proposal requires training/finetuning a separate LM model, which is time-consuming.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper focuses on adversarial example generation for attack on neural machine translation (NMT). Adversarial examples are slightly perturbed model inputs (i.e., the source sentence in NMT) which degrades translation substantially. To produce meaning-preserving and fluent adversarial examples, the authors propose TransFool with three distinct losses: an adversarial loss that encourages the model to produce degenerated outputs, a language model loss that ensures the fluency, and a similarity constraint that avoids departing from the original source sentence. TransFool back-propagates gradients through a target NMT model and a LM model to optimize the token embeddings and adopts an iterative attack algorithm to update the embeddings and adversarial sources. Results on several translation tasks and NMT models show that TransFool achieves higher attack success rate with improved semantic preservation and fluency.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. Some statements are hard to understand:\n  - In page 3, “most of the existing adversarial attacks against NMT models are not undetectable since … they use the NMT embedding space to find similar tokens.” Why does using NMT embeddings to find similar tokens make the attack detectable?\n  - In Section 5.4, “TransFool does not explicitly choose a few tokens and replace them with other similar tokens.” If I understand correctly, TransFool still relies on (similar) token replacement to achieve adversarial generation, although many tokens are retained based on the algorithm.\n  - In page 6, “the distance in the embedding space of the LM model represents the\nrelationship between the tokens.” In Section 5.4, “the embedding space of the NMT model does not necessarily capture the relationship between tokens.” Could you explain what “relationship” you refer to here? Word embeddings in neural models often capture at least basic syntactic and co-occurrence information. Could you show some direct evidence to verify the statement here.\n\n2. More ablation studies should be given, particularly on the LM embedding part.\n\n  - TransFool relies heavily on the LM embedding to generate adversarial examples, but the LM embedding is essentially a simple linear mapping of the NMT embedding as shown in Figure 1. It’s surprising that such a simple mapping could make such big difference. What if performing Eq (4) and Eq. (6) on NMT embedding directly? Also, did you share source and target NMT embedding for the NMT model? Does sharing or tying the embedding parameters affect the performance?\n  - The similarity loss is supposed to retain semantic similarity of the original source input, but loss actually just uses word embedding representation without considering any contextual information. To what extent does it really deal with the sentence-level semantics? Eq (4) looks more like a locality constraint avoiding moving too far from the original inputs.\n  - Besides, how does the quality of the LM affect the attack? \n\n3. Although TransFool obtains better results based on semantic similarity metrics, the results in Table 2 shows a slightly different story. Replacing “Oregon” with “Quebec” doesn’t sound like meaning-preserving, but it could still obtain a high metric score possibly because neural metrics are not that sensitive to one (or few-word) change. It would be great to show some human evaluations to avoid misleading conclusions.\n", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "UooeRHrkXk", "reviewer": "Reviewer_rhrU", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\nIf one accepts this task as established by the works that come before it, and in particular, as defined by the Seq2Sick paper, then this work reflects a clear incremental improvement over what has come before. The attacks improve all metrics: BLEU degradation, semantic similarity to the original source, and perplexity (standing in for grammaticality) of the modified source. Plus simple token counting metrics show that fewer tokens are changing to produce more harmful attacks.\n\nThis paper is well-written, and easy to understand.\n\nIt compares against relevant baselines and surpasses them. The comparisons are well done. I especially like that they used an independent semantic similarity system (universal sentence encoder) and an independent LM (GPT-2) to produce metrics.\n\nThe black-box attack and cross-lingual black-box attack are interesting and clever extensions, bringing the work novelty beyond the incremental improvements discussed above.\n\nWeaknesses:\n\nThis task does not make much sense. Why is it important to be able to make a change to a sentence that is small according to a (very permissive) semantic similarity metric, but which results in a big change according to BLEU? The provided example changes “Oregon” to “Quebec” – a classic example of how neural metrics of semantic similarity can be a bad fit for measuring human-perceived semantic distance, and as I looked at the other examples, the story didn’t look any better. The image and audio attacks that inspired this work are compelling because they actually are imperceptible to the human eye or ear, plus they lead to clear harms such as the addition of a sticker to a stop sign making it invisible to self-driving cars. How does it defeat an MT system when we change Oregon to Quebec and the translation changes in a mostly-reasonable way in response? Were I a reviewer on the Seq2Sick paper, I would have made the same argument there, and I would have argued strongly against its acceptance. Beyond vague statements about robustness, this paper has not made a strong argument for why incremental improvements on this adversarial task will eventually lead to something useful (or dangerous). The authors seem to be aware of this issue, as they discuss a big part of it in their Discussion in 5.4: they have yet to make any attempt to attribute how much of the reduction in BLEU comes from actually changing the content of the sentence versus how much of it comes from degrading MT performance. In the examples where they have clearly degraded the MT performance, such as the example in Table 12, which causes half of the sentence to be dropped, this only happens when they have made a major change to the source content, changing the name of the “Monserrate Sanctuary” (500k hits on Google and has its own Wikipedia page) to the “Interesserrage Sanctuary” - a made up place beginning with a made up word that has only one hit on Google – the arXiv link for this paper – which I didn’t look at beyond the title in order to preserve the authors’ anonymity. Why am I so worked up about this? Because I can see a chain of reasonable incremental improvements to this problem extending from this paper, with each one improving semantic similarity and perplexity of the altered source, and I have no idea why we are doing this. Other papers on adversarial attacks on MT have a clear story of why someone would want to carry out such attacks. Several such papers are dismissed here as too obvious or too easily defended. For example, Ebrahimi et al, 2018 have the constraint that only a few characters can be changed in the source: the assumption here is that though a changed word may no longer be a dictionary word, it is likely to still be readable by a speaker of the source language. The implied attack is that the attacker can obfuscate their message from anyone who does not speak the source language and who may be monitoring through machine translation. Likewise, Wallace et al., 2020 look for things like a universal suffix dropper (a phrase that, when appended to any input, commonly causes itself and any subsequent text to be dropped from the translation) or nonsense strings that lead to offensive translations. Here again, the attack purposes are clear: hiding messages for the former, and creating examples that harm a system's reputation for the latter. These may be easily overcome (that doesn’t seem obvious to me), but at least they present attacks that may be harmful to the MT system or its users. \n\nSmaller problems:\n\nThe related work section, though comprehensive, doesn’t do a great job of positioning this work with respect to the work that has come before. In particular, the author’s description of Seq2Sick as being “based on an optimization problem in the NMT embedding space” makes it sound highly similar to the approach described here. I would spend more time discussing the differences and similarities of the two methods. \n\nOne of the major differences with respect to Seq2Sick seems to be the use of language model embeddings (which are simple neural transformations of NMT embeddings) for semantic similarity. I would have probably put more effort into establishing what was gained by shifting from NMT to LM embeddings, and what was lost by forcing your LM to use transformed NMT embeddings instead of being free to use whatever embeddings work best. This could be done by measuring LM perplexity with an unconstrained embedding table to measure the perplexity hit from constraining the LM, and by measuring TransFool’s drop in performance if it uses the NMT embedding table instead of the LM table to measure semantic similarity.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper describes the TransFool algorithm for creating adversarial examples for machine translation systems. Adversarial translation examples, as defined by this paper, are examples where you make small (they frequently use the term imperceptible) changes to the source sentence, which results in a large degradation to the quality of the resulting translation as measured against the original reference.\n\nGiven this task, and given a modified definition of “small change” to be small in terms of semantic similarity to the original sentence as measured by a neural model, the authors propose a gradient-based attack that changes to the embeddings of the original source words in order to (1) reduce the probability of the reference translation, (2) be close in the vector space defined by the embedding table of a neural LM and (3) receive high probability by the same neural LM. The choice in neural LM is constrained in that it needs to use the same word piece model as the translation model (TM), and it needs to have its bottom embedding table be the TM's embedding table transformed by a single fully connected layer. Once a new set of vectors is found, they are projected onto token space, and the process repeats until some desired BLEU degradation is reached after running the modified source through the model under attack. Interestingly, the authors evaluate their approach not only as a white-box model with full access to the gradients under attack, but also as a black box model, where they use gradients on a system they do have access to in order to attack another system they do not have access to. They even attempt cross-lingual attacks where the two systems have the same source language but different target languages. They compare their system to recent baselines, and show that it makes larger BLEU degradations with fewer token changes, better perplexity (measured by GPT-2) and higher semantic similarity to the original (measured by the universal sentence encoder).", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "If one accepts this task as reasonable, I would strongly suggest that you shift to evaluation of MT using BLEURT or COMET instead of BLEU. This would bring the metrics up to date, and it would help address a fairly serious concern in that you’re using a paraphrase-aware metric to measure similarity to the original source, but using metrics that use only the surface forms of words to measure the degradation of translation quality. But I’m not sure it would actually give us a clearer picture of what is going on. I would actually be a little curious to see whether BLEURT would consider changing Oregon to Quebec to be a large or small translation error – it might not be built to handle these kinds of mistakes. Likewise, the second and third weaknesses mentioned above can be addressed relatively easily.\n\nThe most important change one can make to this paper is to paint a clear motivating picture for why this task is important. I can see three roads to doing so. (1) would be to defend the task and evaluation as it stands - defend the robustness argument and the utility of neural semantic similarity metrics in measuring whether changes are imperceptible or semantics-preserving. I’m a little skeptical of this one, but it might be possible - you have thought more about this problem than I have, and I’m not an unreasonable person. (2) would be to shift the evaluation – bring humans into the loop and ask the big questions. Does the new sentence make sense? If it does, then is the new translation a good translation for the new sentence or have we found a robustness hole in this walk through embedding space? How often does this approach lead to changes that are truly imperceptible (i.e.: small spelling variations, deletion of unimportant punctuation or function words) or truly meaning preserving (i.e.: a human would assert that the new sentence means exactly the same thing as the old). This would greatly strengthen the paper because I think it would give us a clear picture of how much progress we’re actually making. It would also be expensive. (3) Craft a compelling story of how attacks with good perplexity and good semantic similarity are useful either by harming MT systems and their users or by showing us how we can improve our MT systems. I can’t think of one myself, that doesn’t mean that one doesn’t exist.", "broader_impact_concerns": "Ironically, if I could see this paper causing harm, I’d be far more likely to accept it. Irony aside, I see no ethical issues with these sorts of adversarial papers. They play an important role in the research ecosystem by exposing weaknesses of systems that can later be mitigated or otherwise addressed.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "vDzFu6wvOa", "reviewer": "Reviewer_4hTU", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. The paper is well-written - provides a clear explanation of the TransFool algorithm and its effectiveness in generating adversarial examples for NMT models.\n2. The experimental results support the claim well - demonstrate the severity of the attack on translation quality while maintaining high semantic similarity.\n3. The stronger attack (TransFool) presented in this paper can promote stronger defense research for NMT models.\n\nWeaknesses:\n1. The motivation is a little bit lacking: what would be the potential real-world implications of NMT models being vulnerable to adversarial attacks?\n2. Using universal sentence encoder (Yang et al. 2019) as an approximated semantic similarity is fine, but in my opinion, for the paper to be more self-contained, it would be good to include the key information that the reader needs from Yang's paper in this paper.\n3. The experiments present three language pairs, while they are nice, I would be interested to learn how well would TransFool do with more language pairs, it could also be a way to show how generalized TransFool is.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper investigates the vulnerability of Neural Machine Translation (NMT) models to adversarial attacks. The main contributions of this paper includes: 1) proposing a new attack algorithm called TransFool, which builds on a multi-term optimization problem and a gradient projection step to generate adversarial examples in the source language that maintain high semantic similarity with clean samples; 2) demonstrating empirically that TransFool can severely degrade translation quality while maintaining high semantic similarities;  and 3) showing that TransFool can not only be transferred to unknown target models, but also outperforming existing attacks in terms of success rate, semantic similarity and fluency.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Based on the weaknesses section above, I think it would be good if the authors could:\n1. Discuss a bit more on why do we care about attacks to NMT models.\n2. Some introductions on universal sentence encoder and how exactly it is used in this work.\n2. More experiments comparing various language pairs.", "broader_impact_concerns": "No I don't have any concerns on the ethical implications of the work that would require adding a Broader Impact Statement.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "Dkjg6uJ5VM", "reviewer": "Reviewer_JA6r", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. The paper has done extensive experiments on several datasets.\n2. The proposed attack is transferable from the white box to the black box setting. \n\nWeaknesses:\n1. The idea of using gradient projection and imposing a similarity constraint for text attack is not novel.\n2. Only two baselines, Seq2Sick and KNN, are included. More related/strong baselines should be in. \n3. The paper lacks some deep analysis. For example, why is the proposed method able to transfer from the white box to the black box？ ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposed an adversarial attack method - TransFool, for NMT models. It performs optimization with iterative gradient projection and utilizes the embedding representation of a language model to impose a similarity constraint. The method can be model-specific and extended to a white-box attack. Extensive experiments show that TransFool is highly effective in different translation tasks and against different NMT models while maintaining fluency and semantic similarity. ", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. More analysis of the transferable ability and more ablation studies on different variants of translation models. \n\n2. More baselines should be included, such as a. SemAttack: Natural Textual Attacks via Different Semantic Spaces; b.Textual adversarial attack as combinatorial optimization", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: Overall all reviewers and myself agree that this paper is solid and should be accepted.

There have been extensive discussions between reviewers and authors after the initial reviews. I think one of the most crucial weakness of the paper was on the evaluation side. It was unclear what was the cause of the degradation of performance for the adversarial sentences. The authors made a thorough revision of their evaluation method by adding the human evaluation on fluency of the adversarial examples and quality of the translations for the adversarial examples. Additionally, they proposed a back-translation metric hinting that the NMT method is suffering from performance degradation on these adversarial examples. These results address reviewers concerns. The paper is overall thorough and therefore ready for acceptance.
Decision: Accept (poster)