TransFool: An Adversarial Attack against Neural Machine Translation Models
Authors: Sahar Sadrizadeh
Pascal Frossard
Ljiljana Dolamic
Venue: Submitted to ICLR 2023 (2023)
Keywords: Adversarial attack
deep neural network
language model
natural language processing
neural machine translation
robstness.

Abstract: Deep neural networks have been shown to be vulnerable to small perturbations of their inputs known as adversarial attacks. In this paper, we consider the particular task of Neural Machine Translation (NMT), where security is often critical. We investigate the vulnerability of NMT models to adversarial attacks and propose a new attack algorithm called TransFool. It builds on a multi-term optimization problem and a gradient projection step to compute adversarial examples that fool NMT models. By integrating the embedding representation of a language model in the proposed attack, we generate fluent adversarial examples in the source language that maintain a high level of semantic similarity with the clean samples and render the attack largely undetectable. Experimental results demonstrate that, for multiple translation tasks and different NMT architectures, our white-box attack can severely degrade the translation quality for more than 60% of the sentences while the semantic similarity between the original sentence and the adversarial example stays very high. Moreover, we show that the proposed attack is transferable to unknown target models and can fool those quite easily. Finally, our method leads to improvement in terms of success rate, semantic similarity, and fluency compared to the existing attack strategies both in white-box and black-box settings. Hence, TransFool permits to better characterize the vulnerability of NMT systems and outlines the necessity to design strong defense mechanisms and more robust NMT systems for real-life applications.

Under review as a conference paper at ICLR 2023
TRANSFOOL: AN ADVERSARIAL ATTACK AGAINST NEURAL MACHINE TRANSLATION MODELS
Anonymous authors Paper under double-blind review
ABSTRACT
Deep neural networks have been shown to be vulnerable to small perturbations of their inputs known as adversarial attacks. In this paper, we consider the particular task of Neural Machine Translation (NMT), where security is often critical. We investigate the vulnerability of NMT models to adversarial attacks and propose new attack algorithm called TransFool. It builds a multi-term optimization problem and a gradient projection step to compute adversarial examples that fool NMT models. By integrating the embedding representation of language model in the proposed attack, we generate fluent adversarial examples in the source lan- guage that maintain high level of semantic similarity with the clean samples and render the attack largely undetectable. Experimental results demonstrate that, for multiple translation tasks and different NMT architectures, our white-box attack can severely degrade the translation quality for more than 60% of the sentences while the semantic similarity between the original sentence and the adversarial example stays very high. Moreover, we show that the proposed attack is transfer- able unknown target models and can fool those quite easily. Finally, based on automatic and human evaluations, our method leads to improvement in terms of success rate, semantic similarity, and fluency compared to the existing attacks both in white-box and black-box settings. Hence, TransFool permits to better charac- terize the vulnerability of NMT systems and outlines the necessity to design strong defense mechanisms and more robust NMT systems for real-life applications.
INTRODUCTION
The impressive performance of Deep Neural Networks (DNNs) in different areas such as computer vision (He et al. 2016) and Natural Language Processing (NLP) (Vaswani et al., 2017) has led to their widespread usage in various applications. With such an extensive usage of these models, it is important to analyze their robustness and potential vulnerabilities. In particular, it has been shown that the outputs of these models are susceptible to imperceptible changes in the input, known as adversarial attacks (Szegedy et al.] 2014). Adversarial examples, which differ from the original inputs in imperceptible manner. cause the target model generate incorrect outputs. If these models are not robust enough to these attacks, they cannot be reliably used in applications with security requirements. To address this issue, many studies have been recently devoted to the effective generation of adversarial examples, the defense against attacks, and the analysis of the vulnerabilities of DNN models DMoosavi-Dezfoolli al. 2016 Madry et al. 2018 Ortiz-Jiménez et al. 2021
The dominant methods to craft imperceptible attacks for continuous data, e.g., audio and image data, based on gradient computing and various optimization strategies. However, these methods cannot be directly extended to NLP models due to the discrete nature of the tokens in the corre- sponding representations (i.e., words, subwords, and characters). Another challenge in dealing with textual data is the characterization of the imperceptibility of the adversarial perturbation. The 4p- norm is highly utilized in image data to measure imperceptibility but does not apply to textual data where manipulating only one token in a sentence may significantly change the semantics. Moreover, in gradient- methods, is challenging to incorporate linguistic constraints manner. Hence, optimization-base attacks against NLP models. Currently, most attacks in textual data are gradient-free fereentt-eeeeeeee and simply based on heuristic word replacement, which may result in sub-optimal performance (Alzantot al. 2018 |Ren et al.
differentiable methods are more difficult and less investigated for adversarial
2019 Zang et al. 2020 Jin et al.) 2020; Morris et al. 2020; Guo et al., 2021
Sadrizadeh et al.
2022

---

Under review as a conference paper at ICLR 2023
In the literature, adversarial attacks have been mainly studied for text classifiers, but less for other NLP tasks such as Neural Machine Translation (NMT) (Zhang et al. (020b). In text classifiers, the number of output labels of the model is limited, and the adversary's s goal is to mislead the target model to classify the input into any wrong class (untargeted attack) or a wrong predetermined class (targeted attack). However, in NMT systems, the output of the target model is sequence of tokens, which is a much larger space than that of a text classifier (Cheng et al., (020a), and it is probable that the ground-truth translation changes after perturbing the input sequence. Hence, it is important to craft meaning- preserving adversarial sentences with a low impact on the ground-truth translation.
In this paper, we propose TransFool to build against NMT models. We build a new solution to the challenges associated with gradient-base based adversarial attacks against textual data. To find an adversarial sentence that is fluent and semanti- cally similar to the input sentence but highly degrades the translation quality of the target model, we propose a multi-term optimization problem over the tokens of the adversarial example. We consider the white-box attack setting, where the adversary has access to the target model and its parameters. White-box attacks are widely studied since they reveal the vulnerabilities of the systems and are used in benchmarks. To ensure that the generated adversarial examples are imperceptibly similar to the original sentences, we incorporate a Language Model (LM) in our method in two ways. First, consider the loss of Causal Language Model (CLM) in our optimization problem in order to impose the syntactic correctness of the adversarial example. Second, by working with the embed- ding representation of LMs, instead of the NMT model, we ensure that similar tokens are close to each other in the embedding space (Tenney et al. 2019p. It enables the definition of a similarity term between the respective tokens of the clean and ad adversarial sequences. Hence, we include a similar- ity constraint in the proposed optimization problem, which uses the LM embeddings. Finally, our optimization contains an adversarial term to maximize the loss of the target NMT model.
and fluent adversarial attacks
The generated adversarial example, i.e., the minimizer of the proposed optimization problem, should consist of meaningful tokens, and hence, the proposed optimization problem should be solved in a discrete space. By using gradient projection technique, we first consider the continuous space of the embedding space and perform a gradient descent step and then, we project the resultant embedding vectors to the most similar valid token. In the projection step, we use the LM embedding representation and project the output of the gradient descent step into the nearest meaningful token in the embedding space (with maximum cosine similarity). We test method against different NMT models with transformer structures, which are now widely used for their exceptional performance. For different NMT architectures and translation tasks, experiments show that our white-box attack can reduce the BLEU score, a widely-used metric for translation quality evaluation (Post) (018)). to half for more than 60% of the sentences while maintains a high level of semantic similarity with the clean samples. Furthermore, we extend TransFool to black-box settings and show that it can fool unknown target models. Overall, automatic and human evaluations show that both white-box and black-box settings, TransFool outperforms the existing heuristic strategies in terms of success rate, semantic similarity, and fluency. In summary, our contributions are as follows:
We define new optimization problem to compute semantic-preserre and fluent attacks against NMT models. The objective function contains several terms: adversarial loss to maximize the loss of the target NMT model; similarity term to ensure that the adversarial example is similar to the original sentence; and loss of a CLM to generate fluent and natural adversarial examples.
We propose a new strategy to incorporate linguistic constraints in our attack in a differentiable manner. Since LM embeddings provide a meaningful representation of the tokens, we use them instead of the NMT embeddings to compute the similarity between two tokens.
We design a white-box. attack algorithm, TransFool, against NMT models by solving the proposed optimization problem with gradient projection. Our attack, which operates at the token level, is effective against oott-oof-hh-art transformer- based NMT models and outperforms prior works.
By using the transferability of adversarial attacks other models, we extend the proposed white- box attack to the black-bor setting. Our attack is highly effective even when the target languages of the target NMT model and the reference model are different. To our knowledge, this type of transfer attack, cross- cross-lingudl lingual, has not been investigated.
The rest of the paper is organized as follows. We review the related works in Section 2 In Section 3 p we formulate the problem of adversarial attacks against NMT models, and propose an optimization problem to build adversarial attacks. We describe our attack algorithm in Section In Section we
2

---

Under review as a conference paper at ICLR 2023
discuss the experimental results and evaluate our algorithm against different transformer models and translation tasks. Moreover, we evaluate our attack in black-box settings and show that TransFool has very good transfer properties. Finally, the paper is concluded in Section 6
RELATED WORK
Machine translation, important task in NLP, is the task of automatically converting a sequence of words in a source language to a sequence of words in a target language (Bahdanau et al.) (015). By using DNN models, NMT systems are reaching exceptional performance, which has resulted in their usage in a wide variety of areas, especially in safety and security sensitive applications, But any faulty output of NMT models may result in irreparable incidents in real-world applications. Hence, we need to better understand the vulnerabilities of NMT models to perturbations of input samples, in particular to adversarial examples, to ensure security of applications and robustness of such models.
Adversarial attacks against NMT systems have been studied in recent years. First, Belinkov & Bisk (2018) show that character-level NMT models are highly vulnerable to character manipula- tions such as typos in block-box setting. Similarly, EEbahiii et al.) (2088)) investigate the ro- bustness of character-level NMT models. They propose white-box adversarial attack based on HotFlip (Ebrahimi et al., 20) and greedily change the important characters to decrease the trans- lation quality (untargeted attack) or mute/push a word in the translation (targeted attack). However, character-led manipulations can be easily detected. To circumvent this issue, many of the adversar- ial attacks against NMT models are rather based on word replacement. Cheng et al. (2019) propose white-box attack where they first select random words of the input sentence and replace them with a similar word. In particular, in order to limit the search space, they find some candidates with the help of language model and choose the token that aligns best with the gradient of the adversarial loss to cause more damage to the translation. Michel et al. (2019) and Zhang et al. (2021) find important words in the sentence and replace them with a neighbor word in the embedding space to create adversarial examples. However, these methods use heuristic strategies which may result in sub-optimal performance. There are also some other types of attacks against NMT models in the literature. In (Wallace et al.] 2020), new type of attack, i.e., universal adversarial attack, is proposed, which consists of a single snippet of text that can be added to any input sentence to mis- lead the NMT model. However, the added phrase is meaningless, hence easily detectable. Cheng jet al. (2020a) propose Seq2Sick, a targeted white-box attack against NMT models. They introduce an optimization problem and solve it by gradient projection. The proposed optimization problem contains an adversarial loss and a group lasso term to ensure that only a few words of the sentence modified. Although they have a projection step to the nearest embedding vector, they use the NMT embeddings, which may not preserve semantic similarity.
Other types of attacks against NMT models with different threat models and purposes have also been investigated in the literature. Some papers focus on making NMT models robust to perturbation to the inputs (Cheng et al., 2018 20200] Tan et al. (02II) Some other papers use adversarial attacks to enhance the NMT models in some aspects, such as word sense disambiguation (Emelin et al.) 2020). robustness to subword segmentation (Park al. 2020), and robustness of unsupervised NMT (Yu et al., (02T). In (Xu et al. 2021 Wang et al. 202 ). the data poisoning attacks against NMT models are studied. Another type of attack whose purpose is to change multiple words while ensuring that the output of the NMT model remains unchanged is explored in (Chaturvedi et al. 2019; 2021 Another attack approach is presented in (Cai et al. 202T). where the adversary uses the hardware faults of systems to fool NMT models.
In summary, most of the existing adversarial attacks against NMT models are not undetectable since they are based on character manipulation, or they use the NMT embedding space to find similar tokens. Also, heuristic strategies based on werd-veplaeement are likely to have sub-optimal perfor- mance. Finally, none of these attacks study the transferability to black-box settings. We introduce TransFool to craft effective and fluent adversarial sentences which are similar to the original ones.
3 OPTIMIZATION PROBLEM
In this section, we first present our new formulation for generating adversarial examples against NMT models, along with different terms that form our optimization problem.
3

---

Under review as a conference paper at ICLR 2023
Adversarial Attack. Consider X' to be the source language space and Y to be the target language space. The NMT model f :X' - 3 generally has an mncoderddeooder structure (Bahdanau et al. 2015 Vaswani et al. 2017) and aims to maximize the translation probability p(yrer x). where E X is the input sentence in the source language and Yref E y is the ground-truth translation in the target language. To process textual data, each sentence is decomposed into a sequence of tokens. Therefore, the input sentence x = 21 is split into a sequence of k tokens, where 2; is token from the vocabulary set Vx of the NMT model, which contains all the tokens from the source language. For each token in the translated sentence Yrer = Vre.1, .... Vee,t. the NMT model generates a probability vector over the target language vocabulary set Vy by applying a softmax function to the decoder output.
The adversary is looking for an adversarial sentence x', which tokenized into a sequence of tokens x' = z/az.....'. in the source language that fools the target NMT model, i.e., the translation of the adversarial example fxx) is far from the true translation. However, the adversarial example x' and the original sentence should be imperceptibly close so that the translation of the adversarial example stays similar to Yrer-
As is common in the NMT models (Vaswani et al. 2017; Junczys- Downuut et al. 2018 Tang ct al., 2020), to feed the discrete sequence of tokens into the NMT model, each token is converted to a continuous vector, known an embedding vector, using lookup table. In particular, let emb(.) be the embedding function that maps the input token 2i to the continuous embedding vector emb(2;) = e; E TR" where m is the embedding dimension of the target NMT model. Therefore, the input of the NMT model is a sequence of embedding vectors representing the tokens of the input sentence, i.e., ex = je1. C2 ..... ex] E (R)xm) In the same manner, ex = [e eg..... o ] E (kexm) is defined for the adversarial example.
To generate an adversarial example for a given input sentence, we introduce an optimization problem with respect to the embedding vectors of the adversarial sentence ex'. Our optimization problem is composed of multiple terms: an adversarial loss, a similarity constraint, and the loss of a language model. An adversarial loss causes the target NMT model to generate faulty translation. Moreover, with a language model loss and a similarity constraint, we impose the generated adversarial ex- ample to be a fluent sentence and also semantically similar to the original sentence, respectively. The proposed optimization problem, which finds the adversarial example x' from its embedding representation ex by using a lookup table, is defined as follows:
X f- arg min [L aL + 3LLM), ejEEvx
where a and B are the hyperparameters that control the relative importance of each term. Moreover, we call the continuous space of the embedding representations the embedding space and denote it by E, and we show the discrete subspace of the embedding space E containing the embedding representation of every token in the source language vocabulary set by Ey.. We now discuss the different terms of the optimization function in detail.
Adversarial Loss. In order to create an adversarial example whose translation is far away from the reference translation Yref. we try to maximize the training loss of the target NMT model. Since the NMT models are trained to generate the next token of the translation given the translation up until that token, we are looking for the adversarial example that maximizes the probability of wrong translation (i.e., minimizes the probability of correct translation) for the i-th token, given that the NMT model has produced the correct translation up to step (i 1):
LAds
(Whersless [Mret, ... (()))((-))))),
where (Mee, lex', (Mref.1..... (tcf,(t-1))) is the cross entropy between the predicted token distribu- tion by the NMT model and the delta distribution on the token Beet. which is one for the correct translated token, Vier, and zero otherwise. By minimizing log(p,() (J), normalized by the sentence length 1, we force the output probability vector of the NMT model to differ from the delta distribution on the token Yerts which may cause the predicted translation to be wrong.
Similarity Constraint. To ensure that the generated adversarial example is similar to the original sentence, need to add similarity constraint to our optimization problem. It has been shown
4

---

that the embedding representation of language model captures the semantics of the tokens (Tennyy eet al., 2019] Shavarani & Sarkar 2021). Suppose that the embedding representation by a language model of the original sentence (which may differ from the NMT embedding representation ex) is Vx [V1, ......, V&] ((xxn), where n is the embedding dimension of the language model. Like- wise, let Vxr denote the sequence of LM embedding vectors regarding the tokens the adversarial Under review as a conference paper at ICLR 2023
example. We can define the distance between the i-th tokens of the original and the adversarial
sentences by computing the cosine distance between their corresponding LM embedding vectors:
V
E (1....,k) rs
(3)
The cosine distance is zero if the two tokens are the same and it has larger values for two unrelated tokens. We want the adversarial sentence to differ from the original sentence in only a few tokens. Therefore, the cosine distance between most of the tokens the original and adversarial sentence should be zero, which causes the cosine distance vector [r1, "2.,?..... Tk] to be sparse. To ensure the sparsity of the cosine distance vector, instead the lo norm, which not differentiable, we can define the similarity constraint the & norm relaxation of the cosine distance vector normalized to the length of the sentence:
V
Ssi k
(4)
Language Model Loss. Causal language models are trained maximize the probability of a token given the previous tokens. Hence, we can use the loss of a CLM, i.e., the negative log-probability. as rough and differentiable measure for the fluency of the generated adversarial sentence. The loss of CLM, which is normalized to the sentence length, is as follows:
LLM
log(p, (vilv', ((-1))),
(5)
where g is CLM, and Po (v/lv' 1)) is the cross entropy between the predicted token dis- tribution by the language model and the delta distribution on the token v/, which one for the corresponding token in the adversarial example, v', and zero otherwise.
To generate adversarial examples against target NMT model, we propose to solve the optimization problem TT. which contains an adversarial loss term, similarity constraint, and CLM loss.
4 TRANSFOOL ATTACK ALGORITHM
Outpst
We now introduce our algorithm for generating adver- sarial examples against NMT models. The block di- agram of our proposed attack is presented in Figure
LM Eibbaddine Victies
Grodient Stop Va affie HGrul
Projection Step
I
We are looking for an adversarial example with to- kens in the vocabulary set Vx and the corresponding embedding vectors in the subspace Eyy. Hence, the optimization problem (I) is discrete. The high-level idea of our algorithm is to use gradient projection to
Gradient Projection
Objentive
solve equation I in the discrete subspace Eyx-
Language Model
The objective function of equation [ is a function of NMT and LM embedding representations of the ad- versarial example, ex and Vxs, respectively. Since we aim to minimize the optimization problem with respect to ex', we need to find a transformation between the embedding space of the language model and the tar- get NMT model. To this aim, as depicted in Figure
1M Eirhwldingg
Pully Connected
NNTT Enbssiding
Voutore
Inpui
Figure 1: Block diagram of TransFool.
we propose replace the embedding layer of pre-trained language model with Fully Con- nected (FC) layer, which gets the embedding vectors of the NMT model as its input. Then, we train the language model and the FC layer simultancously with the causal language modeling objective. Therefore, we can compute the LM embedding vectors as a function of the NMT embedding vectors: v, FC((;,, where FC mm the trained FC layer.
5

---

Under review as a conference paper at ICLR 2023
The pseudo-code of attack can be found in Algo- rithm[]] In more detail, we first convert the discrete to-
Algorithm TransFool Adversarial Atlack
Input:
kens of the sentence to continuous embedding vectors of the target NMT model, then we use the FC layer compute the emhedding representations of the to- kens by the language model. Afterwards, we consider the continuous relaxation of the optimization problem, which means that we assume that the embedding vec- tors are in the continuous embedding space & instead of x In each iteration of the algorithm, we first up- date the sequence embedding vectors ex in the op- posite direction of the gradient (gradient descent). Let us denote the output of the gradient descent step for the i-th token by eg.i. Then bedding vectors, which are not necessarily in Eyx, to the nearest token in the vocabulary set Vx. Since the distance in the embedding space of the LM model rep- the
f(.:: Target NMT model, Vx: Vocabulary
FC: Fully connected layer, x: Input sentence
Yref' Ground trnntdhruthhae translation of
A: BLEU
ratio. o, Hyperparameters K: Maximum No. of itcrations, 7: step size
Output:
x Generated adversarial example
initialization:
emply sir 4-0 thr <- (BLEU( Yearr) x A
Vi C. [1,...k] while
K Gg. Sp.i
str < +
Step 1: Grudient descent in the continuous embedding spacc:
project the resultant em-
6x 6x 6x-9.Ve, V.VVee aLssm 3CLM)
Va =FC(ee))
Step 2: Projection to the discrete subspace
Ev, and update if the sentence is new:
resents the relationship between the tokens, LM embedding representations with cosine similarity metric the projection step to find the most similar to- ken in the vocabulary. We can apply the trained fully connected layer FC to find the LM embedding repre- sentations: Vg= = FC(eg). Hence, the projected NMT embedding vector, ep.i, for the i-th token is:
{1..... A} do pp. arg max eEEvy [Pow
FC
end for if ep not in then
add ep sct
eg + Cp (I..F Yver)) the then
break (adversarial example found) end if end
(C((e) Vg.i (FC(e))l2-11vg.ll)
Cp.1 = arg max eεEvx
(6)
while
return e,² 6p
However, due to the discrete nature of data, by applying the projection step in every iteration of the algorithm, we may face an undesirable situation where the algorithm gets stuck in a loop of previously computed steps. In order to circumvent this issue, we will only update the embedding vectors by the output of the projection step the projected sentence has not been generated before.
We perform the gradient descent and projection steps iteratively until maximum number of itera- tions is reached, or the translation quality of the adversarial example relative to the original trans- lation quality is less than threshold. To evaluate the translation quality, the BLEU score, which is a widely used metric in the literature:
BLEU(J(ex)) Vrat)) SA.
(f(x)
5 EXPERIMENTS
In this section, we first discuss our experimental setup, and then we evaluate TransFool against different models and translation tasks, both in white-box and black-box settings.
5.1 EXPERIMENTAL SETUP
We conduct experiments on the English-French (En-Fr), English-German (En-De), and English- Chinese (En-Zh) translation tasks. We use the test set of WMT14 (Bojar et al. 2014) for the En-Fr and En-De tasks, and the test set of OPUS-100 (Zhang et al. 2020a) for the En-Zh task. Some statistics of these datasets are presented in Appendix
We evaluate TransFool against transformer- based NMT models. To verify that our attack is effective against various model architectures, we attack the HuggingFace implementation of the Marian NMT models (Junczys-Dommun) al. 2018) and mBARTS. multilingual NMT model (Tang et al.) 2020).
As explained in Section the similarity constraint and the LM loss of the proposed optimization problem require FC layer and CLM. To this aim, for each NMT model, train FC layer and a CLM (with GPT-2 structure (Radford et al.) 2019)) on WikiText- 103 dataset. We note that the input of the FC layer is the target NMT embedding representation of the input sentence.

---

Under review as a conference paper at ICLR 2023
Table Performance of white-box attack against different NMT models.
Marian NMT
mBART50
Task
Method
ASR1
RDBLEU1
RDchrFt
Sim.t
Perp-1
TER.
ASR1
RDBLEU1
RDehrF
Sim.t
Perp-1
TERL
TransFool
69.38
0.57
0.23
0.85
182.45
13.91
60.68
0.53
0.22
0.84
121.12
10.58
En-Fr
KNN
36.53
0.36
0.16
0.82
389.78
19.15
30.84
0.29
0.11
0.85
336.47
21.03
Soq2Sick
27.01
0.21
0.16
0.75
175.31
13.97
25,53
0.19
0.13
0.75
151.92
13.55
TransFool
69.49
0.65
0.23
0.84
165.53
13.57
62.87
0.61
0.22
0.83
134,90
11.07
En-De
KNN
39.22
0.40
0.17
0.82
441.62
19.42
35,99
0.39
375.32
21.22
Soq2Sick
35.60
0.31
0.21
0.67
290.32
18.13
35,59
0.31
0:20
0.66
265.62
18.18
TransFool 73.82
0.74
0.31
0.88
102.49
11.82
$7.50
0.67
0.26
0.90
74.75
7.77
En-Zh
KNN 31.12
0.33
0.18
0.86
180.27
15.95
27.25
0.32
0.14
0.90
160.27
16.58
Seq2Sick
28.76
0.26
0.25
0.73
161.8 84
17.48
24.25
0.31
0.18
105.42
13.58
To find the minimizer of our optimization problem (T). we use the Adam optimizer (Kingma & Ba
2014) with step size y 0.016. Moreover, we set the maximum number of iterations to 500. Our algorithm has three parameters: coefficients a and / in the optimization function I and the relative
BLEU score ratio Ả in the stopping criteria 77. We set À 0.4, = 1.8, and a 20. We chose these parameters experimentally according to the ablation study, which is available in Appendix B in order to optimize the performance in terms of success rate, semantic similarity, and fluency.
We compare our attack with (Michel et al. 2019), which is white-box untargeted attack against NMT models We only consider one of their attacks, called kVN, which substitutes some words with their neighbors in the embedding space; the other attack considers swapping the characters, which too easy to detect. We also adapted Seq2Sick (Cheng et al. (020a)), targeted attack against NMT models based on an optimization problem in the NMT embedding space, to our untargeted setting,
We
For evaluation, we report different performance metrics: (1) Attack Success Rate (ASR), which measures the rate of successful adversarial examples. Similar to (Ebrahimi et al. 2a). we define the adversarial example as successful if the BLEU score of its translation is less than half of the BLEU score of the original translation. (2) Relative decrease of translation quality, by measuring the translation quality in terms of BLEU score and chrF' (Popović, 2015). We denote these two metrics by RDBLEU and RDchrF, respectively. We choose to compute the relative decrease in translation quality so that scores are comparable across different models and datasets (Michel et al.) 2019). (3) Semantic Similarity (Sim.), which is computed between the original and adversarial sentences and commonly approximated by the universal sentence encoder (Yang et al. 2020 3) (4) Perplexity score (Perp-), which is a measure of the fluency of the adversarial example computed with the perplexity score of GPT-2 (large).
(5) Token Error Rate (TER), which measures the imperceptibilityy by computing the rate of tokens modified by an adversarial attack.
5.2 RESULTS OF THE WHITE-BOV BOX ATTACK
Now evaluate TransFool in comparison to KNN and Seq2Sick against different NMT models. Table [ shows the results in terms of different evaluation metrics Overall, our attack is able to decrease the BLEU score of the target model to less than half of the BLEU score of the origi- nal translation for more than 60% of the sentences for all tasks and models (except for the En-Zh mBART50 model, where ASR is 57.50%). Also, in all cases, semantic similarity is more than 0.83, which shows that our attack can maintain a high level of semantic similarity with the clean sentences.
In comparison to the baselines, TransFool obtains a higher success rate against different model structures and translation tasks, and it is able to reduce the translation quality more severely. Since the algorithm uses the gradients of the proposed optimization problem and is not based on token replacement, TransFool can highly degrade the translation quality. Furthermore, the perplexity score of the adversarial example generated by TransFool is much less than the ones of both baselines (except for the En-Fr Marian model, where it is little higher than Seq2Sick), which is due to the
"Code of (Cheng et al. 2019 2020b. untargeted white-box attacks against NMTs, is not publicly available. We use case-sensitive SacreBLEU Poot 2018 on detokenized sentences.
'e use the muitilingual version since are dealing with multiple languages.
'We discard the sentences whose original BLEU score is zero to prevent improving the results artificially.
We should also note that all results are computed after the re-tokenization of the adversarial example. Since we generating the adversarial example at the token-level, there is small chance that, when the generated adversarial example is converted to text, the re-tokenization does not produce the same set of tokens.
7

---

Under review as a conference paper at ICLR 2023
Table 2: Adversarial examples* against mBART50 (En-De) generated by different methods.
Sentence
Text
Org.
In Oregon. planners are experimenting with giving drivers different choices.
Ref. Trans.
In Oregon experimenteeen die Planer damit, Autofahrem eine Reihe von Auswatlnnillcchheiie. zu geben.
Org. Trans.
In Oregon experimentieren Planer damit, Fahremn ver verschiedene Wahlen zu geben.
Adv, TransFool
In Oregon. planners were experimenting with giving drivers different choices.
Trans.
In Orepon experimentierten Planer mit der Bereitstellung unterschiellicher Wahlmöglichkoicen für Fahrer.
Adv. KNN
in Oregon, planners nemmeno experimenting withkjer driver. different choices,
Trans.
Oregon. Planer nemmeno experimentieren milkjer Fahrer. verschiedene Wahlen.
Adv. Seq2Sick
acontece, planners studying with Kivakapis against decisions, In acontece studieren Planer mit Kivakapis gegen Entscheidungen,
Trans.
"Adversurial perturhed tikens red, und perturbations by TransFool are in blue the original sentence. The changes the translation that are direct resalts the perturaatinn are in brown, while the changes that are the of turget model are orange.
integration of the LM embeddings and the LM loss term in the optimization problem. Moreover, the token error rate of our attack is lower than both baselines, and the semantic similarity is preserved better by TransFool in almost all cases since we use the LM embeddings instead of the NMT ones for the similarity constraint. While kNN can also maintain semantic similarity, Seq2Sick does not
perform well in this criterion. We also computed similarity by BERTScore (Zhang et al. 2019) and BLEURT-20 (Sellam al., 2020) that highly correlate with human judgments in Appendix D which shows that TransFool is better than both baselines in maintaining the semantics. Moreover, as presented in Appendix [.2] the successful attacks by the baselines, opposed TransFool, are not semantie-preserving or fluent sentences. Finally, the complete setup and results of our human evaluation are presented in Appendix H which also shows the superiority of TransFool.
We also compare the runtime of TransFool and that of the two baselines. In each iteration of our proposed attack, we need perform a back-propagation through the target NMT model and the language model to compute the gradients. Also, in some iterations (27 iterations per sentence on average), a forward pass required to compute the output of the target NMT model to check the stopping criteria. For the Marian NMT (En-Fr) model, on system equipped with an NVIDIA A100 GPU, takes 26.45 seconds to generate adversarial examples by TransFool. On the same system, KNN needs 1.45 seconds, and Seq2Sick needs 38.85 seconds to generate adversarial examples for less effective adversarial attacks, however.
Table 2 shows some adversarial examples against mBART50 (En-De). In comparison to the base- lines, TransFool makes smaller changes to the sentence. The generated adversarial example is a correct English sentence, and it is similar the original sentence. However, kNN and Seq2Sick generate adversarial sentences that are not necessarily natural or similar to the original sentences. More examples generated by TransFool, kNN, and Seq2Sick can be found in Appendix [.2] We also provide some adversarial sentences when we do not use the LM embeddings in our algorithm in order show the importance of this component.
Indeed, TransFool outperforms both baselines in terms of success rate. It is able to generate more natural adversarial examples with lower number of perturbations (TER) and higher semantic sim- ilarity with the clean samples in almost all cases. A complete study of hyperparameters and the effect of using LM embeddings instead of NMT embeddings for computing similarity on TransFool performance is presented in Appendix B and C respectively.
5.3 PERFORMANCE IN BLACK-BOR ATTACK SETTINGS
In practice, the adversary's access to the learning system may be limited. Hence, we propose to analyze the performance of TransFool in a black-box scenario. It has been shown that adversarial attacks often transfer to another model that has a different architecture and is even trained with different datasets (Szegedy et al. 2014). By utilizing this property of adversarial attacks, we extend TransFool to the black-box scenario. We consider that we have complete access to one NMT model (the reference model), including its gradients. We implement the proposed gradient-based attack in algorithm D with this model. However, for the stopping criteria of the algorithm, we query the black-box target NMT model to compute the BLEU score. We can also implement the black-box transfer attack in the case where the source languages of the reference model and the target model are the same, but their target languages different. Since Marian NMT is faster and lighter than mBART50, we use it as the reference model and evaluate the performance of the black- attack against mBART50. We compare the performance of TransFool with WSLS (Zhang et al.) 2021).
8

---

Under review as a conference paper at ICLR 2023
Table 4: Performance of black-box attack, when the target language is different.
Marian NMT
mBARTS0
Task
ASR?
RDBLEU?
RDchrFy
Sim.t
Perp-]
#Queries]
ASRT
RDBLEUT
RDchrFy
Sim.t
Perp.l
#Queries]
En-De En-Fr 60.53
0.55
0.22
0.84
169.49
24
61.68
0.56
0.22
0.84
169.51
23
En-Fr En-De 66.22
0.63
0.22
0.84
198.04
23
63.86
0.63
0.21
0.84
195.50
24
Blak-boox untargeted attack against NMT models based on vord-replacemmnt (the choice of back- translation model used in WSLS is investigated in Appendix FF). We also evaluate the performance of KNN and Seq2Sick in the black-box settings by attacking mBART50 with the adversarial example generated against Marian NMT (in the white-box settings). The results are reported in Table We also report the performance when attacking Google Translate, some generated adversarial samples, and similarity performance computed by BERTSScrre and BLEURT-20 in Appendix E
In all tasks, with few queries to the target model, our black-box attack achieves better performance than the white-box attack against the target model (mBART50) but a little worse performance than the white-box attack against the refer- ence model (Marian NMT). In all cases, the success rate, token error rate, and perplexity of TransFool are better than all baselines (except for the En-Fr task, where perplexity is a little higher than Seq2Sick). The ability of TransFool and WSLS to maintain semantic similarity is comparable and better than both other baselines. However, WSLS has the highest token error rate, which makes the attack detectable. The effect of TransFool on BLEU score is larger than that of the other methods, and its effect on chrF metric comes after WSLS (except for the En-DE task, where RDchrF of TransFool is the best).
Table 3: Performance of black-box attack against mBART50.
Task
Method
ASRT
RDBLEUT
RDehrl"
Sim.1
Perp..
TER [Queries]
TransFool
70.19
0.58
0.22
175.39
17.08
En-Fr
ANN
33.74
0.33
0.15
0.82
383.71
22.57
Sey2Sick
25.97
0.21
0.14
0.75
173.63
21.13
WSLS
56.21
0.58
0.27
0.84
214.23
31.30
1423
TransFool 66.76
0.65
0,22
0.84
167.54
16.73
23
En-De
Seg2Sick
kNN
32.17
36.70
0.29
0.39
0.16
0.20
0.82
0.67
435.02
286.67
26.59
22.34
WSLS
44.33
0.50
0.19
0.86
219.32
29.12
1262
TransFool 63.27
0.71
0.27
0.88
100.14
14.76
36
En En
Sey2Sick
ANN 26.89
23.65
0.31
0.30
0.17
0.23
0.73
176.34
162.67
25.17
17.07
WSLS
40.00
0.72
0.52
0.83
186.44
32.35
1782
Regarding the complexity, TransFool requires only few queries to the target model for translation, while WSLS queries the model more than a thousand times, which is costly and may not be feasible in practice. For the En-Fr task, on system equipped with NVIDIA A100 GPU, it takes 43.36 and 1904.98 .88 seconds to generate adversarial examples by TransFool and WSLS, respectively, which shows that WSLS is very time-consuming.
We also analyze the transferability of the generated adversarial examples to a black-box NMT model with the same source language but different target language. Since we need dataset with the same set of sentences for different language pairs, we use the validation set of WMT14 for En-Fr and En- De tasks. Table shows the results for two cases: Marian NMT or MMBART5O as the target model. We use Marian NMT as the reference model with a different target language than that of the target model. In all settings, the generated adversarial examples are highly transferable to another NMT model with a different target language (i.e., they have high attack success rate and large semantic similarity). The high transferability of TransFool shows that it is able to capture the common failure modes in different NMT models, which can be dangerous in real-world applications.
6 CONCLUSION
In this paper, we proposed TransFool, a white-box adversarial attack against NMT models, by in- troducing new optimization problem solved by an iterative method based on gradient projection. We utilized the embedding representation of language model to impose a similarity constraint on the adversarial examples. Moreover, by considering the loss of a language model in our optimiza- tion problem, the generated adversarial examples are more fluent. Extensive automatic and human evaluations show that TransFool is highly effective in different translation tasks and against different NMT models. Our attack is also transferable to black-box settings with different structures and even different target languages. In both white-box and black-box scenarios, TransFool obtains improve- ment over the baselines in terms of success rate, semantic similarity, and fluency. It is important to analyze adversarial attacks against NMT models such as TransFool to find the vulnerabilities of NMT models, measure their robustness, and eventually build more robust NMT models.

---

Under review as a conference paper at ICLR 2023
Ethics Statement We introduced TransFool, an adversarial attack against NMT models, with the motivation of revealing the vulnerabilities of NMT models and paving the way for designing stronger defenses and building robust NMT models in real-life scenarios. While it remains possibility that threat actor may misuse our attack, we do not condone using our method with the intent of attacking real NMT system.
Reproducibility Statement The source code will be publicly available as soon as possible to help reproduce our results. Moreover, Appendix contains the license information and more details of the assets (datasets, codes, and models).
REFERENCES
Moustafa Alzantot, Yash Sharma, Ahmed Elgohary, Bo-Jhang Ho, Mani Srivastava, and Kai- Chang- Generating natural language adversarial examples. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 2890-2896, 2018.
Dzmitry Bahdanau, Kyung Hyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning align and translate. In 3rd International Conference Learning Representations, ICLR 2015, 2015.
Yonatan Belinkov and Yonatan Bisk. Synthetic and natural noise both break neural machine trans- lation. In International Conference Learning Representations, 2018.
Ondřej Bojar, Christian Buck, Christian Federmann, Barry Haddow, Philipp Koehn, Johannes Lev- cling, Christof Monz, Pavel Pecina, Matt Post, Herve workshop statistical machine translation. In Proceedings of the ninth workshop statistical machine translation, pp. 12-58, 2014.
et al. Findings of the 2014
Kunbei Cai, Md Hafizul Islam Chowdhuryy, Zhenkai Zhang, and Fan Yao. Seeds of seed: Nmt- stroke: Diverting neural machine translation through hardware-based faults. In 2021 International Symposium on Secure and Private Execution Environment Design (SEED), pp. 72. IEEE, 2021.
Daniel Cer, Mona Diab, Eneko Agirre, Iñigo Lopez-Gazpio, and Lucia Specia. Semeval-2017 task I: Semantic textual similarity multilingual and crosslingual focused evaluation. In Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017) pp. 1-14, 2017.
Akshay Chaturvedi, Abijith KP, and Utpal Garain. Exploring the robustness of nmt systems to nonsensical inputs. arXiv preprint arXiv:1908.00 0666, 2019.
Akshay Chaturvedi, Abhisek Chakrabarty, Masao Utiyama, Eiichiro Sumita, and Utpal Garain. Ig- norance is bliss: Exploring defenses against invariance- based attacks on neural machine transla- tion systems. IEEE Transactions on Artificial Intelligence, 2021.
Minhao Cheng, Jinfeng Yi, Pin- Yu Chen, Huan Zhang, and Cho-Jui Hsieh. Seq2sick: Evaluating models with adversarial examples. In Proceedings of the
the robustness of AAAI Conference on Artificial Intelligence, volume 34, pp. 3601-3608, 2020a.
Yong Cheng, Zhaopeng Tu, Fandong Meng. Junjie Zhai, and Yang Liu. Towards robust neural ma- chine translation. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 1756-1766 1766, 2018
Yong Cheng, Lu Jiang, and Wolfgang Macherey. Robust neural machine translation with doubly ad- versarial inputs. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 4324-4333, 2019.
Yong Cheng, Lu Jiang. Wolfgang Macherey, and Jacob Eisenstein. Advaug: Robust adversarial augmentation for neural machine translation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 5961-5970, 2020b.
Javid Ebrahimi, Daniel Lowd, and Dejing Dou. On adversarial examples for character-level neu- ral machine translation. In Proceedings of the 27th International Conference on Computational Linguistics, pp. 653-663, 2018a.
10

---

Under review as a conference paper at ICLR 2023
Javid Ebrahimi, Anyi Rao, Daniel Lowd, and Dejing Dou. Hotflip: White-box adversarial exam- ples for text classification. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pp. 31-36, 2018b.
Denis Emelin, Ivan Titov, and Rico Sennrich. Detecting word sense disambiguation biases in ma- chine translation for model-agmostic adversarial attacks. In The 2020 Conference on Empirical Methods in Natural Language Processing, pp. 7635-7653, Association for Computational Lin- guisties, 2020.
Markus Freitag, Ricardo Rei, Nitika Mathur, Chi-kiu Lo, Craig Stewart, George Foster, Alon Lavie, and Ondiej Bojar. Results of the wm121 metries shared task: Evaluating metries with expert-based human evaluations on ted and news domain. In Proceedings of the Sixth Conference on Machine Translation, pp. 733-774,"' 2021.
Yvette Graham, Timothy Baldwin, Alistair Moffat, and Justin Zobel. Continuous mcasurement scales in human evaluation of machine translation. In Proceedings of the 7th Linguistic Annotation Workshop and Interoperability with Discourse, pp. 33-41, 2013.
Yvette Graham, Timothy Baldwin, Alistair Moffat, and Justin Zobel. Can machine translation sys- tems be evaluated by the crowd alone. Natural Language Engineering, 23(1):3-30, 2017.
Chuan Guo, Alexandre Sablayrolles, Hervé Jégou, and Douwe Kiela. Gradient- based adversarial attacks against text transformers. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 5747-5757 2021.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference computer vision and pattern recognition, pp- 770-778, 2016.
Di Jin, Zhijing Jin, Joey Tianyi Zhou, and Peter Szolovits. Is bert really robust? a strong baseline for natural language attack on text classification and entailment. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pp. 8018-8025, 2020.
Marcin Munczys-Dowmune Roman Grundkiewicz, Tomasz Dwojak, Hieu Hoang, Kenneth Heafield, Tom Neckermann, Frank Seide, Ulrich Germann, Alham Fikri Aji, Nikolay Bogoychev, André F. T. Martins, and Alexandra Birch. Marian: Fast neural machine translation C++. Proceed- ings of ACL 2018, System Demonstrations, pp. 116-12 121, Melbourne, Australia, July 2018.
Diederík P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv: /4/2.6980, 2014.
Quentin Lhoest, Albert Villanova del Moral, Yacine Jernite, Abhishek Thakur, Patrick von Platen, Suraj Patil, Julien Chaumond, Mariama Drame, Julien Plu, Lewis Tunstall, Joe Davison, Mario Saško, Gunjan Chhablani, Bhavitvya Malik, Simon Brandeis, Teven Le Scao, Victor Sanh, Can- wen Xu, Nicolas Patry, Angelina McMillan-Major, Philipp Schmid, Sylvain Gugger, Clément Delangue, Théo Matussière, Lysandre Debut, Stas Bekman, Pierric Cistac, Thibault Goehringer, Victor Mustar, François Lagunas, Alexander Rush, and Thomas Wolf. Datasets: A community library for natural language processing. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pp. 175-184, Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics. URL https://aclanthoot
org/ wwtpo//g..o...... 2021 emnlp-demo. 21]
Qingsong Ma, Ondřej Bojar, and Yvette Graham. Results of the wmt1 8 metrics shared task: Both characters and embeddings achieve good performance. Proceedings of the third conference on machine translation: shared task papers, pp. 671-688, 2018
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations, ICLR 2018, 2018.
Paul Michel, Xian Li, Graham Neubig, and Juan Pino. On evaluation of adversarial perturbations for sequence-tor coaenneetto-eqqeeee sequence models. In Proceedings of the 2019 Conference of the North Ameri- can Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume I (Long and Short Papers), pp. 3103-3114, 2019.

---

Under review as a conference paper at ICLR 2023
Seyed-Mohsen Moosavi-Dezfooll Alhussein Fawzi, and Pascal Frossard. Deepfool: simple and accurate method to fool deep neural networks. In Proceedings of the IEEE conference on com- puter vision and pattern recognition, pp. 2574-2582, 2016.
John Morris, Eli Lifland, Jack Lanchantin, Yangfeng Ji, and Yanjun Qi. Reevaluating adversarial examples in natural language. In Findings of the Association for Computational Linguisties: EMNLP 2020, pp. 1829-38394 2020.
Daniel Naber et al. A rule-based style and grammar checker. 2003.
Nathan Ng. Kyra Yee, Alexei Baevski, Myle Out, Michael Auli, and Sergey Edunov. Facebook fair's wmt19 news translation task submission. In Proceedings of the Fourth Conference on Machine Translation (Volume 2: Shared Task Papers, Day 1), pp. 314-319, 2019.
Guillermo Ortiz-Jiménez, Apostolos Modas, Seyed-Mohsen Moosavi-Dezfooli, and Pascal Frossard. Optimism in the face adversity: Understanding and improving deep learning through adversarial robustness. Proceedings of the IEEE, 10965)633--659, 2021.
Jungsoo Park, Mujeen Sung, Jinhyuk Lee, and Jaewoo Kang. Adversarial subword regularization for robust neural machine translation. In Findings of the Association for Computational Linguistics, ACL 2020: EMNLP 2020, pp. 1945-1953, Association for Computational Linguistics (ACL), 2020.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high- performance deep learning library. Advances in neural information processing systems, 32, 2019.
Maja Popović. chrf: character n-gram f-score for automatic mt evaluation. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pp. 392-395, 2015.
Matt Post. A call for clarity in reporting BLEU scores. In Proceedings of the Third Conference on Machine Translation: Research Papers, pp. 186-191, Belgium, Brussels, October 2018. As- sociation for Computational Linguistics. URL https: : wwwp://wwww/...cc wwwp://wwwwb...r//atttto..oooo
W18 6319
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, llya Sutskever, et al. Language models are unsupervised multitask learners. OpenAl blog, 1(8):9, 2019.
Shuhuai Ren, Yihe Deng, Kun He, and Wanxiang Che. Generating natural language adversarial examples through probability weighted word saliency. In Proceedings of the 57th annual meeting of the association for computational linguistics, pp. 1085-1097, 2019.
Sahar Sadrizadeh, Ljiljana Dolamic, and Pascal Frossard. Block-sparse adversarial attack to fool transformer-basee text classifiers. ICASSP 2022-2022 IFEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 7837-7841. IEEE, 2022.
Thibault Sellam, Amy Pu, Hyung Won Chung, Sebastian Gehrmann, Qijun Tan, Markus Freitag, Dipanjan Das, and Ankur Parikh. Learning to evaluate translation beyond english: Bleurt sub- missions to the wmt metrics 2020 shared task. In Proceedings of the Fifth Conference on Machine Translation, pp. 921 -279 2020.
Hassan S Shavarani and Anoop Sarkar. Better neural machine translation by extracting linguistic information from bert. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pp. 2772-2783, 2021.
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. In 2nd International Conference on Learning Representations, ICLR 2014, 2014.
Weiting Tan, Shuoyang Ding, Huda Khayrallah, and Philipp Koehn. Doubly-traincd adversarial data augmentation for neural machine translation. arXiv e-prints, pp. arXiv-21 2021.
12

---

Under review as a conference paper at ICLR 2023
Yuqing Tang, Chau Tran, Xian Li, Peng-Jen Chen, Naman Goyal, Vishrav Chaudhary, Jiatao Gu, and Angela Fan. Multilingual translation with extensible multilingual pretraining and finetuning. arXiv preprint arXiv:2008.00440 2020.
Ian Tenney, Patrick Xia, Berlin Chen, Alex Wang, Adam Poliak, R Thomas McCoy, Najoung Kim, Benjamin Van Durme, Samuel R Bowman, Dipanjan Das, al. What do you learn from con- text? probing for sentence structure in contextualized word representations. In 7th International Conference on Learning Representations, ICLR 2019, 2019.
Jörg Tiedemann. Parallel data, tools and interfaces in opus. In Eight International Conference on Language Resources and Evaluation, MAY 21-27, 2012, Istanbul, Turkey, pp. 2214-2218, 2012.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in neural information processing systems, pp.
2017.
Eric Milhhce.Micchlll Stern, and Dawn Song. Imitation attacks and defenses for black-box machine translation systems. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 5531 13346,,6, 2020.
Jun Wang, Chang Xu, Francisco Guzmán, Ahmed El-Kishky, Yuqing Tang, Benjamin Rubinstein, and Trevor Cohn. Putting words into the system's mouth: A targeted attack on neural machine translation using monolingual data poisoning. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pp. 1463-1473, 2021.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowiez, Joe Davison, Sam Shleifer, Patrick Von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gug- ger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush. Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pp. 38 45, Online, October 2020. As- sociation for Computational Linguistics. URL http::/:: //w::///wwwwwww www. ac www.wwwwbb....... org.orrg//aooooo. org/anthoooogy 2020. emnnp-demos....
Chang Xu, Jun Wang, Yuqing Tang, Francisco Guzmán, Benjamin IP Rubinstein, and Trevor Cohn. A targeted attack on black-box neural machine translation with parallel data poisoning. In Pro- ceedings of the Web Conference 2021. pp. 3650, 2021.
Yinfei Yang, Daniel Cer, Amin Ahmad, Mandy Guo, Jax Law, Noah Constant, Gustavo Hernandez Abrego, Steve Yuan, Chris Tar, Yun-Hsuan Sung, et al. Multilingual universal sentence encoder for semantic retrieval. In Proceedings of the 58th Anmual Meeting of the Association for Compu- tational Linguistics: System Demonstrations, pp. 87-94, 2020.
Heng Yu, Haoran Luo, Yuqi Yi, and Fan Cheng. A2r2: Robust unsupervised neural machine transla- tion with adversarial attack and regularization on representations. IEEE Access, 19999, 2021.
Yuan Zang, Fanchao Qi, Chenghao Yang, Zhiyuan Liu, Meng Zhang, Qun Liu, and Maosong Sun. Word-level textual adversarial attacking combinatorial optimization. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 6066-6080, 2020.
Biao Zhang, Philip Williams, Ivan Titov, and Rico Sennrich. Improving massively multilingual neural machine translation and shoo translation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 1628- 1639, 2020a.
Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi Bertscore: Evaluat- ing text generation with bert. International Conference on Learning Representations, 2019.
Wei Emma Zhang, Quan Z Sheng, Ahoud Alhazmi, and Chenliang Li. Adversarial attacks on deep-learning models in natural language processing: A survey. ACM Transactions on Intelligent Systems and Technology (TIST), 11(3):1-41, 2020b.
13

---

Under review as a conference paper at ICLR 2023
Xinze Zhang. Junzhe Zhang, Zhenhua Chen, and Kun He. Crafting adversarial examples for neural machine translation. In Proceedings of the 59th Anmual Meeting of the Association for Computa- tional Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume I: Long Papers), pp. 1967-1977, 2021.
14

---

Under review as a conference paper at ICLR 2023
Supplementary Material
TransFool: An Adversarial Attack against Neural Machine Translation Models
ABSTRACT
In this supplementary material, first provide datasets in Section A The ablation study of the hyperparameters TransFool is presented in Section We investigate the effect of the LM embedding representa- tion on TransFool and kNN in Section More results of the white-box attack are reported in D the results of other similarity metrics (Section D.Ip, performance over successful attacks (Section D.2), and some generated adversarial examples (Section D.4p. S more experiments on the black-box attack: the performance of attacking Google Translate (Section E.1 results of other similar- ity metries (Section E.2). and some generated adversarial examples (Section (E33). We discuss the effect of the back-translation model choice on WSLS in Section Finally, the license information and more details the assets (datasets, codes, and models) provided in Section G
statistics of the evaluation
A SOME STATISTICS OF THE DATASETS
Some statistics, including the number of sam- ples, the Average length of the sentences, and the translation quality of Marian NMT and mBART50, i.e., OPUSI00 100 (En-Zh) WMT14 (En-FR) and (En-De), are reported in table
Table 5: Some statistics of the evaluation datasets.
Dataset
Average
#Test Samples
Marian NMT
mBART50
Length
BLEU
chrF
BLEU
chrF
the evaluation datasets,
WMT14
En-Fr
27
3003
39.88
64.94
36.17
62.66
En-De
WMT14
26
3003
27.72
58.50
25.66
57.02
B ABLATION STUDY
OPUS-100
En-Zh
18
2000
33.11
50.98
29.27
41.92
In this Section, we analyze the effect of different hyperparameters (including the coefficients a and in our optimization problem (1). the step size of the gradient descent 7, and the relative BLEU score ratio in the stopping criteria Eq. ( on the white-box attack performance in terms of success rate, semantic similarity, and perplexity score.
In all the experiments, we consider English to French Marian NMT model and evaluate over the first 1000 sentences of the test of WMT14. The default values for the hyperparameters are as follows, except for the hyperparameter that varies in the different experiments, respectively: a = 20, 1.8, y 0.016, and À 0.4.
Effect of the similarity coefficient This hyperparameter determines the strength of the sim- ilarity term in the optimization problem ( Figure 2a shows the effect of a on the performance of attack. By increasing the similarity coefficient of the proposed optimization problem, are forcing our algorithm to find adversarial sentences that are more similar to the original sentence. Therefore, as shown in Figure 2a larger values of a result in higher semantic similarity. However, in this case, it is harder to fool the NMT model, i.c., lower attack success rate, RDBLEU, and RDchrF. Moreover, seems that, since the generated adversarial examples are more similar to the original sentence, they are more natural, and their perplexity score is lower.
Effect of the language model loss coefficient B. We analyze the impact of the hyperparameter 8, which controls the importance of the language model loss term in the proposed optimization
15

---

Under review as a conference paper at ICLR 2023
100
0.90 550
120
0..88 70-
Rate
0.85
70-
0.887
00.85-300
0.003
Semantic Similanly
Perp Serpeein Score
A5R
5im:
65
Sim.
70-
Sim.
0.88 -250
Adh
60-
Petp:
60
Perp
-085
150
Perp.
0.82 200
404
Perp
0.70
55
140
0.80
+0.80
1.25
1.50
1.75
2.00
1.2
1.6
2.0
0.2
0.4
0.6
0.6
Similarity Confficient
[M loss Confficient
Step Size
BI.FU Scone Ratio
(a)
(b)
(c)
(d)
0.57
BLEU
0.24
BLEU chrf
0.27
0.60
0.24
-0.30
0.22
0.50
0.25
0.2
0.50
RDRL
0.25
0.55
0.23
BLEU
0.20
0.21
0.40-
BLEU
0.18
0.45
chi
048
10.19
9.19
chrf
0.10
15
25
1.25
1.50
1.5
2,50
1.6
2.0
0.4
0.5
Similarity Confficient
LM loss Confficient
Step Size
x2e22
BLEU Score Ratio
(c)
(f)
(g)
(h)
Figure 2: Effect of different hyperparameters on the performance ToolsFool.
problem, in Figure 2b By increasing this coefficient, we weaken the effect of the similarity term, ie., the generated adversarial examples less similar to the original sentence. As result, the success rate and the effect on translation quality, i.e., RDBLEU and RDchrF, increase.
Effeet of the step size The step size of the gradient descent step of the algorithm can impact the performance of our attack, which is investigated Figure 2c Increasing the step size results in larger movement in the embedding space in each iteration of the algorithm. Hence, the generated adversarial examples are more aggressive, which results in lower semantic similarity and higher perplexity scores. However, we can find adversarial examples more easily and achieve higher attack success rate, RDBLEU, and RDchRE.
Effect of the BLEU score ratio This hyperparameter determines the stopping criteria of our iterative algorithm. Figure 2d studies the effects of this hyperparameter on the performance of attack. As this figure shows, a higher BLEU score ratio causes the algorithm to end in earlier iterations. Therefore, the changes applied to the sentence are less aggressive, and hence, we achieve higher semantic similarity and lower perplexity score. However, the attack success rate, RDBLEU, and RDchrF decrease since we make fewer changes to the sentences.
C EFFECT OF THE LM EMBEDDING REPRESENTATION
Table shows the results of Trans-
Table 6: Performance of white-box attack against Marian
Fool and kNN when we use LM em- beddings or NMT embeddings for
NMT (En-Fr) with/without language model embeddings.
measuring similarity between two tokens The LM embeddings re- sult in lower perplexity and higher semantic similarity for both meth- ods, which demonstrates the impor- tance of this component in generat- ing geeaiing-preserving fluent adversarial examples.
Method
ASR1
RDBLEU RDchrFf
Sim.1
Perp-j
TransFool w/ LM Emm..
69.48
0.56
0.23
0.85
177.20
TransFool w/ NMT Emb.
68.27
0.57
0.26
0.78
193.32
KNN LM Emb. KNN w/ NMT Emb.
32.13
0.32
0.15
0.85
246.52
36.65
0.35
0.16
0.82
375.84
'In order to have fair comparison, fine-tuned hyperparameters of Transfool, in the case when we do
LM embeddings, have similar attack success
16

---

Under review as a conference paper at ICLR 2023
D MORE RESULTS ON THE WHITE-BOX ATTACK
D.1 SEMANTIC SIMILARITY COMPUTED BY OTHER METRICS
To better assess the ability of adversarial attacks in maintaining semantic similarity, we can compute the similarity between the original and adversarial sentences using other metrics such as BERTScore (Zhag et al. 2019) and BLEURT-20 (Sellam et al., 2020). is shown in (Zhang et al. 2019) that BERTScore correlates well with human judgments. BLEURT-20 is also shown to correlates better with human judgment than traditional measures (Freitag et al. 2021). The results are reported in T These results indicate that the TransFool is indeed more capable of preserving the semantics of the input sentence. In the two cases where kNN has better similarity by using the Universal Sentence Encoder (USE) (Yang al. 2020), the performance of TransFool is better in terms of BERTScore and BLEURT-20.
Table 7: Similarity performance of white-box attacks.
Marian NMT
mBARTS0
Task
Method
USE
BERTScoret
BLEURT-20 f
USE
BERTScoref
BLEURT-20
TransFool
0.85
0.95
0.65
0.84
0.96
0.70
En-Fr
KNN
0.82
0.94
0.61
0.85
0.93
0.67
Seg2Sick
0.75
0.94
0.60
0.75
0.94
0.66
TransFool 0.84
0.96
0.67
0.83
0.95
0.69
En-De
kNN
0.82
0.94
0.61
0.86
0,93
0.67
Seg2Sick 0.67
0.93
0.52
0.66
0.92
0.58
TransFool 0.88
0.96
0.67
0,90
0.97
0.76
En -h-
kNN 0.86
0.95
0.66
0.90
0.95
0.72
Scq2Sick
0.73
0.94
0.54
0.78
0.95
0.67
D.2 PERFORMANCE OVER SUCCESSFUL ATTACKS
The evaluation metrics of the successful adversarial examples that strongly affect the translation quality also important, and they show the capability of the adversarial attack. Hence, evalu- ate TransFool, KNN, and Seq2Sick only over the successful adversarial examples [0 The results for the white-box setting are presented in Table 8 By comparing this Table and Table D which shows the results on the whole dataset, we can see that TransFool performance is consistent among suc- cessful and unsuccessful attacks. Moreover, successful adversarial examples generated by TransFool are still semantically similar to the original sentences, and their perplexity score is low. However, the successful adversarial examples generated by Seq2Sick and kNN do not preserve the semantic similarity and are not fluent sentences; hence, they are not valid adversarial sentences.
Table 8: Performance of white-box attack over successful adversarial examples.
Marian NMT
mBART50
Task
Method
ASR1
RDBLEU!
RDchrFi
Sim.1
Perp-J
TER.
ASR1
RDBLEU
RDchrF*
Sim.t
Perp-j
TER.
TransFool
69.38
0.66
0.26
0.83
229.75
15.33
60.68
0.66
0.27
0.82
164.52
12.56
En-Fr
KNN
36,53
0.70
0.30
0.76
746.89
24.52
30.84
0.72
0.28
0.77
691.64
28.05
Seq2Sick
27.01
0.72
0.40
0.56
648.92
25.28
25.53
0.74
0.41
0.53
556.61
25.16
TransFool KNN 39.22
0.72
0.25
0.83
191.51
14.54
62.87
0.73
169.76
12.66
En- De
0.75
0.29
0.77
675.01
23.07
35,99
0.75
0.23
0.81
574.68
25.75
Seq2Sick
35.60
0.78
0.40
0.53
659.90
25.67
35,59
0.78
0.40
0.52
612.22
26.67
TransFool 73.82
0.76
0.34
0.87
112.28
12.83
$7.50
0.73
0.31
0.88
99.08
9.86
En-Zh
KNN 31.12
0.72
0.29
0.80
355.25
22.55
27.25
295.53
23.58
Seq2Sick
28.76
0.72
0.46
0.58
437.49
26.84
24.25
0.79
0.44
0.60
292.55
25.59
D.3 TRADE-OFF BETWEEN SUCCESS RATE AND SIMILARITY/FLUENCS
The results in ablation study B show that there is trade-off between the quality of adversarial example, in terms of semantic-pre preeevicppreseraation and fluency, and the attack success rate. As studied in
the adversarial example is successful if the BLEUU score of its translation is less than half of the BLEU score of the original translation.
As defined in Section 5
17

---

Under review as a conference paper at ICLR 2023
(Moriis et al. 2020), can filter adversarial examples with low quality based on hard constraints semantic similarity and the number of added grammatical errors caused by adversarial perturbations.
We can analyze the trade-off between success rate and similartty/luency by setting different thresholds for filtering adversarial examples. If we evaluate the similarity by the sentence encoder sug- gested in (Morris et al., (220), the suc- cess rate with different threshold values
60-
TransFool
40-
TransFool
KNN
Attack Attack Sucess Rate
KNN
Attack Sucess
50-
Seq2Sick
30-
Seq2Sick
40-
30-
20
10-
for similarity in the case of Marian (En- Fr) depicted Figure 3b By consid- ering only the adversarial examples with similarity higher than a threshold, the
104
D+
0.5
1.0
2.0
0.80
0.85
0.90
0.55
Threshold for Grammar
Threshold for Similarity
(a)
(b)
success rate decreases as the threshold increases, and the quality of the adversar- ial examples increases. Similarly, we do the same analysis for fluency. As sug- gested in (Morris et al., 2020). we count the grammatical errors by Language (Naber al. 2003) for the original sen- tences and the adversarial examples. Figure Ba depicts the success rate for different thresholds of the number of added grammatical errors caused by adversarial perturbations.
Figure 3: Tradeoff between success rate and Similar- ity/fluency. The left figure shows the effect of acceptable number of added grammar errors by adversarial pertur- bation. The right figure shows the effect of similarity threshold.
These analyses show that with tighter constraints, we can generate better adversarial examples while the success rate decreases. All in all, according these results, TransFool outperforms the baselines for different thresholds of similarity and grammatical errors.
D.4 MORE ADVERSARIAL EXAMPLES
In this Section, we present more adversarial examples generated by TransFool, KNN, and Seq2Sick. In order show the effect of using LM embeddings the performance of TransFool, we also include the generated adversarial examples against English to French Marian NMT model when we do not LM embeddings. In all these tables, the tokens modified by TransFool in the original sentence, and the modified tokens by different adversarial attacks are written in red in their corresponding adversarial sentences. Moreover, the changes made by the adversarial attack to the translation that are not directly related to the modified tokens are written in orange, while the changes that are the direct result of modified tokens are written brown.
written in blue
As can be seen in the examples presented in Tables 9 and 10 TransFool makes smaller changes to the sentence. The generated adversarial example is a correct English sentence, and it is similar to the original sentence. However, KNN, Seq2Sick, and our method with the NMT embeddings make changes that are perceptible, and the adversarial sentences are not necessarily similar the original sentence. The higher semantic similarity of the adversarial sentences generated by TransFool is due to the integration of LM embeddings and the LM loss in the proposed optimization problem. We should highlight that Tool is able to make changes to the adversarial sentence translation that are not directly related to the modifications of the original sentence but are the result of the NMT model failure.
Other examples against different tasks and models are presented in Tables 11 to 16
18

---

Under review as a conference paper at ICLR 2023
Table 9: Adversarial examples against Marian NMT (En-Fr) by various methods (white-box).
Sentence
|BLEU
Text
Org.
The most eager is Oregon, which is enlisting 5,000 drivers in the country's 's biggest experiment.
Le plus déterminé est O'Oregon, qui a mobilisé 5 000 conducteurs pour mener I' l'expérience la plus impor- tante pays.
Ref. Trans.
Org. Trans.
21.66
Le plus avide est I'Oregon, qui recrute 000 pilotes dans la plus grande expérience du pays.
Adv. TransFool
The most eager isQuebee, which is enlisting 5,000 drivers in the country's biggest experiment.
Le Québec, qui partic de plus grande expérience du pays, compte 000 pilotes. (some parts not trunslated at all.)
Trans.
7.71
The most eager isCustom, which is enlisting Disk drivers in the country's 's editions Licensee.
Adv. w/ NMT Emb. Trans.
6.54
Le plus avide estCustom, qui recnute des pilotes de disque dans les éditions pays Licencié. Theve cager is Oregon, C aren enlisting 5,000 drivers in theau's 's biggest experiment.
Adv. KNN
Trans.
5.93
Theve avide est Oregon, C sont enrôlés 5 000 pilotes dans la plus grande expérience de Theau.
Adv. Seq2Sick
The buzz FREE, which chooseing Games commentt the country's great developer.
Trans.
10.31
Le plus buzz est GRATUIT, qui est de choisir Jeux commentaires dans le grand développeur du pays.
Table 10: Adversarial examples against Marian NMT (En-Fr) by various methods (white-box).
Sentence
|BLEU
Text
"They are in the process of abandoning and killing off emergency units that were reformed less than five years ago," he believes.
Org.
'IIs sont train de vider et d'asphyxier des urgences qui ont été rénovées y a moins de cinq ans'. estime-t-il.
Ref. Trans.
Ils sont train d' d'ahandonner de tuer des unités d' l'urgence qui été réformées y moins de cing ans s, croit-il.
Org. Trans.
37.53
'People are in the process of abandoning and killing off emergency units that been reformed less than five years ago," he believes.
Adv. TransFool
* Les gens ahandonnent et tuent les unités d' d'urgence réfirméés il y moins de cinq ans croit-il. (some parts are not translated.)
Trans.
23.83
Adv. w/ NMT Fimh.
"Manager are in the process of abundoning and killing off emergency units that were celebrating less than
five years ago," he believes.
Trans.
27.66
Le gestionnaire est en train d' d'ahandonner et de tuer des unités d' l'urgence qui céléhraient il y a moins de cing ans croit-il.
"They are in the process of ahandoning and killing aff emergency allotment that were reformedvoir88) five years ago," Stttes helieves.
Adv. KNN
Trans.
21.20
Ils sont
d' l'ahandonner tuer allocations d' l'urgence qui été réformées y cinq ans8 estime-t-il.
Adv. Seg2Sick
"They are in the prucess of abandoning shot off emergency units that were CSIS than five years ago," he believes.
IIs sont train d' d'ahundonner et de mer des unités d' d'urgence qui étaient SCRS il y mains de cing ans croit-il.
Trans.
33.58
Table 11: Adversarial examples against Marian NMT (En-Fr) by various methods (white-box).
Sentence
Text
Org.
Discovering neighlouthoods, our architecture, our eavironment are reference points.
Ref. Trans.
La découverte de quartiers, notre architecture, nos lieux sont des repères.
Org. Trans.
29.25
Décovrir les quattiers, notre architecture, notre environnement sont des points de référence.
Adv. TransFool
Discovering neighbourhoods, our architecture Whose our environment are reference points,
Trans.
7.9
Découverte des quartiers, de notre architecture dont notre environnement est des points de référence,
Adv. KNN
Discovering weighbourhoods these infrastructure, we envirenment is reference points.
Trans.
6.61
Découverte des quartiers, ces infrastructures, I'envionnement est un point référence,
Adv. Seq2Sick
awayingevaluations, our architecture, our energy are reference points,
Trans.
20.33
elimination des évaluations, notre architectrre, notre énergie sont des points de référence.
19

---

Under review as a conference paper at ICLR 2023
Table 12: Adversarial examples against Marian NMT (En-De) by various methods (white-box).
Sentence
BLEU Text
The devices, which track every mile motorist drives and transmit that information to bureaucrats, are at the center of controversial attempe in Washington and state planning offices to overhuul the outdated system for funding America's major roads.
Org.
Die Gerütte, die jeden gefahrenen Kilometer aufzcichnen und die Informationen an die Behörden melden. sind Kempunkt eines kontroversen Versuchs von Washington und den Planung der Bundesstaaten, Finanzierung US-amerikanischer Stralen zu überarbeiten.
Ref. Trans.
das veraltete System
Die Gerlite, die jede Mcile ein Autofahrer fährt und diese Informationen Burokraten weiterleitet, stchen im Zentrum eines umstrittenen Versuchs in Washington und in den stantlichen Planungshüres, das verallete System zur Finanzierung der grofien Straßen Amerikas zu überarbeiten.
Org. Trans.
23.65
The vehicles, which track every mile a motorist drives and transmit that information to bureaucrats, are at center of unjustified attempt in Washington and city planning offices to overhaul clearer system for funding America's 's major roads.
Adv. TransFool
Die Fahrzeuge, die jede Meile ein Autofahrer führt und diese Informationen Burokraten weiterleiter, stehee im Zentrum eines ungerechuferigten Versuchs in Washington und in den Stadpplanngsburro, das klarere System zur Finanzierung der aneriikaischeen HaupistraBen überarbeiten.
Trans.
9.36
The devices in which track every mile a motorist drives and transmit that M to bureaucrats, are 07:0 the center controversial attempt in Washington and state planning offices to overhaul the outdated Estate for funding America's 'ss major roads.
Adv. KNN
Die Vorrichtungen, in denen jede Meile ein Autofahrer fährt und diese M Bürokraten übeettägg, sind 07:0 das Zentrum cines umstrittenen Versuchs in Washington und staatlichen Planungsburos, das veraltete Estate für die Finanzierung der amirhhanischen Hauptstraßen überarbeiten.
Trans.
7.79
The devices. which road everyably a motorist drives and transmit that information to walnut socialisms. the center of Senate attempt in Washington and state planning offices toestablishment the outdated system for funding America's 's major paths.
Adv. Seq2Sick
Die Geräte, die allgegenwäriig ein Autofahrer antreibt und diese Informationen an Walaussoozalismen uberträgt, stchen im Zentrum cines Senatsversuchs in Washington und in den stuntlichen Planungsbüros. veraltete System zur Finanzierung der wichtigston Wege Amerikas einzurichten.
Trans.
22.48
Table 13: Adversarial examples against Marian NMT (En-Zh) by various methods (white-box).
Sentence
|ext
Org.
And what your busband said... Columbus had done it we'd all be Indians.
Ref. Trans.
# RWAA 3E 25 ff 18 St HI f2l
AI T
Org. Trans.
25.58
fixt KUBARA.. 11. 2F # (K59) J (8:1 [1B 2: fo 36 A
Adv. TransFool
And With your hushand said... if Columbus had RU of for (itt I 11 2: AT 32 A (some parts are not translated.)
it, we'd all he Indians.
Trans.
0.0
Adv. KNN
And what your husband said... if Columbus had60, we' Nineteen all it Indians. firt K.... fu tF t 6.60" 12.11199 1- A.
Trans.
24.45
Adv. Seq2Sick
And completing your penalties said... if timely had done it, we'd all be briefed. A60 W.... 301 Je SEet, He. (6 Mi
Trans.
22.09
Table 14: Adversarial examples against mBART50 (En-Fr) crafted by various methods (white-box).
Sentence
BLEU Text
Wearing wingsuit, he flew past over the famous Monserrate Sanctuary at 160km/h. The sanctuary is located at an altitude of over 3000 meters and numerous spectators had guthered there to watch his exploit.
Org.
Equipé d' In wingsuit, il est passé à 160 km/h au-dessus du célebre sanctuaire Monsemaae, situć à plus de 000 mètres d'altitude, où de nombreux badauds s' étaient rassembéés pour observer son exploit.
Ref. Trans.
Org. Trans.
27.33
f1 survolé à 160 km/h le célèhre sanctuaire de Monserrate, situé à une altitude de plus de 3000 mêtres, où de nombreux spectateurs se sont réunis pour assister son exploit.
Adv. TransFool
Wearing a wingsuit, he flew past over the famous Interesserrage Sanctuary at 160km/h. The sanctuary is located an altitude of over 3000 meters numerous spectators had gathered there to watch his exploit.
Trans.
6.16
Le canctuaire est situé à une altiude de plus de 3000 mêtres et de nombreux spectateurs se sont réunis pour assister à son exploit, (first part the sentence
tronslated at all.)
Wearing a wingsuit. he flow past over famous Monserrate Sanctuary at 160km/h. The sanctuary is located at anzu opinionstitude of over 8000 meters and numerous spectators had gathered there the watch exploit.
Adv. KNN
23.80 Il a survolé le célébre sanctuaire de Monserrate& à 160 km/h. Le sanctuaire est situé à une opiniontitude de plus de 8000 métres de nombreux spectateurs sont rassemblés pour observer son exploit.
Trans.
Wearing a wingsuit, he flew past over the famous Monserrate Sanctuary at 160km/h The sanctuary is located at altitude of over74 meters and numerous spectators had gathered there watch his exploil.
Adv. Seq2Sick
a survolé 160 km/h oélèbre sanctunire de Monserrate, situé à plus de 74 mêtres d'altitude, de nombreux spectateurs se sont réunis pour assister son exploit.
Trans.
32.52
20

---

Under review as a conference paper at ICLR 2023
Table 15: Adversarial examples against mBART50 (En-De) crafted by various methods (white-box).
Sentence
BLEU
Org.
In Oregon, planners are experimenting with giving drivers different choices.
Ref.
In Oregon experimentieren die Planer damit, Autofahrem eine Reihe Auswahlmmogichhkecennn geben.
Org. Trans.
22.38
Oregon experimentieren Planer damit, Fahrern verschiedene
zu gehen.
Adv. TransFool
In Oregon, planners were experimenting with giving drivers different choices.
Trans.
6.27
In Oregon experimentierten Planer mit der Bereitstellung unterschiedlehet Wihlmöglichkeiten für Fahrer.
Adv. KNN
In Oregon. planners
experimenting withkjer driver. different choices,
Trans.
3.4
in Oregon, Planer nemmeno experimentieren mitkjer Fahrer. versehiedene Wahlen. In acontece, planners studying with Kivakapis against decisions,
Adv. Seg2Sick
Trans.
3.22
In acontece studieren Planer mit Kivakapis gegen Entscheidungen.
Table 16: Adversarial examples against mBART50 (En-Zh) crafted by various methods (white-box).
Sentence
BBLEU Text
Org.
Delegations requested to suhmit
of representatives Secretary Preparatory
Committee, Ms. Vivian Pliee-JJseehh (room S S-2950E; fax: (212) 963-5935).
# & (C x 0 23 1
it 2
2 27
66
VivianPPiner Josephs (2S-29900
Ref. Trans.
(2:2119993
5935)
# fe (i) E
(1:212/963-593337
MS
Org.
61.63
Adv. TransFool
Delegations requested to submit the names of their representatives to the Secretary of the Preparatory Committee, Mr. Vivian Pliner Josephs (room C. C-2930F; fax: (211) 25.30935).
i # ft B1 22 MIRE 30 32 2 (x21119625-00935)
90-88 3e (Ex C-2930E:
Trans.
9.55
Delegations are requested to submit the names of their represeutatives that the Secretary of the Preparatory Committoe, VivianPline-- Joseph, (room S-2950 fox: (212) 963-5935).
Adv. KNN
Trans.
54.37
il $0 & 2: 2 R8 (Viiiin P SSe.eoospppSSS90O..... :19 (22222266-59335)
Adv. Seg2Sick
Delegations requested to submit the names of their representatives to the Secretary of the Preparatory Committee, Ms. jadan Pliner-Josephs froom S-2950E; 599; 212 96 2010,935.
i & ft z 16 3t FE de 05 NE 26 $1
13 St
Wr (S-
Trans.
13.40
9950E 299.:59999.22920009555
21

---

Under review as a conference paper at ICLR 2023
E MORE RESULTS ON THE BLACK-BOX ATTACK
E ATTACKING GOOGLE TRANSLATE
To evaluate the effect of different attacks in practice, we attack Google Translate" by TransFool, kNN, and Seq2Sick. Since querying Google Translate is limited per day, we were not able to attack with WSLS, which requires high number of queries. Table [7 presents the perfor- mance of the English to French transla- tion task. The results demonstrate that adversarial sentences crafted by Trans- Fool can degrade the translation quality more while preserving the semantics bet- ter. The perplexity score and word er- ror rate of TransFool compete with those metrics of Seq2Sick, but Seq2Sick is not meaning-rreserving and less effective.
Table 17:
Performance of black-box attack against Google Translate (En-Fr),
Method
ASR
RDBLEUf RDchrFt Sim.f
Perp.]
WERL
TransFool
67.83
0.55
0.23
0.85
184.35
20.85
KNN
37.22
0.35
0.17
0.82
389.45
30.24
Seq2Sick 23.49
0.20
0.15
0.75
174.88
20.34
Table 18: Performance of TransFool black-box attack against Google Translate (En-De), when the target lan- guage is different..
Task
ASR RDBLEU? RDehrFt
Sim.t
Perp.l
WER,
En-Fr En-De 67.42
0.65
0.26
0.85
198.56
20.78
We also performed the cross-lingual black-box attack. We consider Marian NMT (En-Fr) as the reference model and attack En-De
Google Translate. The results for TransFool are reported in Table 18
E.2 SEMANTIC SIMILARITY COMPUTED BY OTHER METRICS
Similar to the white-box attack, we compute the similarity between the adversarial and original sentences by
Table 19; Similarity performance of black-box attacks.
Task
Method
USE
BERTScoref
BLEURT-20
BERTScore and BLEURT-20. since they correlate well with human judgments. The similarity performance of Trans- Fool and WSL.$ in the black-box set- tings are demonstrated in Table [9 Ac- cording to Table 19] TransFool is bet- ter at maintaining semantic similarity. may be because used LM embeddings instead of the NMT ones in the similarity constraint.
TransFool
0.85
0.95
0.66
En-Fr
WSLS
0.84
0.93
0.58
TransFool
0.84
0.96
0.67
En-De
WSLS
0.86
0.94
0.61
TransFool 0.88
0.96
0.68
En-Zh
WSLS 0.83
0.93
0.56
E.3 SOME ADVERSARIAL EXAMPLES
We also present some adversarial examples generated by TransFool and WSLS, in the black-box setting, in Tables 20 to 22 these tables, the tokens modified by TransFool are written in blue in the original sentence, and the modified tokens by different adversarial attacks are written in red in their corresponding adversarial sentences. Moreover, the changes made by the adversarial attack to the translation that are not directly related to the modified tokens are written in orange, while the changes that are the direct result of modified tokens are written brown.
These examples show that modifications made by TransFool are less detectable, i.e., the generated adversarial examples are more natural and similar to the original sentence. Morcover, TransFool makes changes the translation that are not the direct result of the modified tokens of the adversarial sentence.
We should that since we do have tokenizer, we compute Word Error Rate (WER) instead of Token Error Rate (TER).
'The results of kNN and Seq2Sick are not reported since they transfer attacks, and their performance is already reported in Table
22

---

Under review as a conference paper at ICLR 2023
Table 20: Adversarial examples against mBART50 (En-Fr) crafted by various methods (black-box)
Sentence
|BLU Text
It therefore not surprising that he should be holding a mask in the promotional photography for L. I'Invitatoon au Voyage, by Louis Vuillon, of which he the new face.
Org.
Ref. Trans.
Rien d' étennant
qu 'll ait done un masque main dans la photographic la cumpagne L'Invitation Voyage. de la marque Louis Vuitton, dont incarne le nouveau visage.
Org. Trans.
31.15
n'est done pas surpeenant qu'il tienne
masque dans photographie promotionnelle de L L'Invitation
au Voyage, de Louis Vuitton, dost il est le nouveau visage.
It is therefore sans Voyage, by Louis Vuition, of which be is the new face.
surprising that he should be holding mask the promotional painting for L'Invitation
Adv. TransFool
n''st done pas surprenant ill lienne une masque dans la peinture promotionnelle de Louis Vuitton pour L' Invvitation sans Voyage, dont il est le nouveau visage.
Trans.
13.22
me is therefore not surprising that he should be holding a wig inthe the promos portraiture for L. L'Invitation au Voyage, by gary Vuitton. of which he is the fresh face.
Adv. WSLS
le ne suis done pas surpris qu' 'll porte une parache dans le portrait promotioneel de l'Inviatiion au Voyage, de Gary Vuitton, dont est le nouveau visage.
Trans.
21.21
It is therefore not surprising that he should be holding a mask in the promotional photography for L'In processedation Voyage, by Leais gooduition, which is associating new facel CoceConnction
Adv. KNN
Trans.
13.82
Il n' est done pas surprenant qu ill tienne une masque dans la photographie promotionnelle de I.In processe- dation Voyage, de Louis gooduition, dont il associe un neuveau visage Connection
Adv. Seq2Sick
It is therefore not pasture that he should be holding a hidden in the unclean goodneess for L'Invitation au face.
Voyage, Leuis Vuitton, of which he is the
s'agit donc pas påturage qu'll doit tenir caché dans la honté insaluhre pour L'Invitation Noyage, de Louis Vuitton, dont il est le nouveau visage.
Trans.
28.75
Table 21: Adversarial examples against mBART50 (En-De) crafted by various methods (black-box).
Sentence
BLEU Text
Org.
This really is must for our nation.
Ref. Trans.
Das ist wirklich ein Muss für unser Land,
Org. Trans.
61.05
Das wirklich ein Muss für unsere Nation.
Adv. TransFool
This really his must for our nation.
Trans.
20.16
Das ist wirklich seine Pflicht für unsere Nation.
Adv. WSLS
This really becomes
outfor our nation.
Trans.
33.03
Das wird wirklich ein Muss für unsere Nation,
Adv. KNN
This realities and a requisiteAstr our nation.
Trans.
4.77
Diese Realitäten und eine notwendige Astrologic unserer Nation.
Adv. Seq2Sick
This really must for our imperfect.
Trans.
61.05
Das ist wirklich ein Muss für unsere Unvollkommenheit.
Table 22: Adversarial examples against mBART50 (En-Zh) crafted by various methods (black-box)
Sentence
BLEU
Text
(c) To provide care and support by strengthening programming for orphans and vulnerable children in- fected/affected by AIDS and by expanding life skills training for young people.
Org.
to If aS BE IL aa io
Te A A.
Ref. Trans.
()) A0 A
A
Org. Trans.
21.07
Adv. TransFool
(c) To peovide care and support by strengthening programming for orphans and vulnerable children Dis- abled/ afflicted by AIDS and by expanding life skill training for young people.
[e)itfidd d 30 RURI A. (i do 83 JUR BE 3 31
Trans.
9.22
(c) To provide nursing and unstinted support by strengthening i Lifetv for orphans and susceptable children infected/affected by CPR mannequins and by broadening life skills training for young people.
Adv. WSLS
(e)iifiu in Sn JL CPR 28 Wol Haniih 8
Bo Rebi Liferv. 44 68 A (lE '2 mh
Trans.
12.57
H Lll 56 3P X Et
so) address provide care and support by strengthening prioritization for orphans and vulncrable children infected/affected AIDS and expanding life skills ismu for young people.
Adv. KNN
Hult, itf i2 t0 xf V. 38566 Myy th 1I 22 10 A
Trans.
5.63
31.RRa33 38 F0 3:
Adv. Seq2Sick
provide care support by strengthening digital for dress and harmful children Journal/ Letter by Region and by disappear Violence skills training for young people.
(e df At 10 Hil (2,5) 2e /H 39 4p A BE BEAFU Il, de K0F11
Trans.
14.99
23

---

Under review as a conference paper at ICLR 2023
F EFFECT OF PERFORMANCE
MODEL CHOICE ON WSLS
WSLS uses a tock-translttion model for crafting an adversarial example. In (Zhang et al. (021), the authors investigate the En-De task and use the winner model of the WMT19 De- En sub-track (g et al, 2019) for the back-translation model. However, they do not evaluate their method for
Table 23: Performance of WSLS (En-De) with two back- translation models.
Back- Transtatantti
ASR
RDBLEU
RDchrF
Sim.
Perp.
#Querles
Marian NMT
44.33
0.50
0.19
0.86
219.32
1262
eg al. 2019
51.68
0.58
0.21
0.81
241.96
1307
En-Fr and En-Zh tasks. To evaluate the performance of WSLS in Table We have used pre-trained Marian NMT models for all three back-translation models. In order to show the effect of our choice of
model, we compare the performance of WSLS for the En-De task when we use Marian NMT or (Ng et al. 2019) as the back- trecs-rranlatiin model in Table 23 As this Table shows, WSLS with Marian NMT as the nack-translation model results in even more semantic similarity and lower perplexity score. On the other hand, WSLS with (Ng et al. 2019) as the Pack-translation model has a slightly more success rate. These results show that our choice of fack-translation model does not highly affect the performance of WSLS.
G LICENSE INFORMATION AND DETAILS
In this Section, we provide some details about the datasets, codes, and models used in this paper. We should note that we used the models and datasets that are available in HuggingFace transformers (Wolf et al., 2020) and datasets (Lhoest et al. 2021) libraries 9 They are licensed under Apache License 2.0. Moreover, we used Py Torch for all experiments (Paszke et al. 2019), which is released under the BSD licens Im
G.1 DATASETS
WMT14 In the Ninth Workshop Statistical Machine Translation, WMT14 was introduced for four tasks. We used the En-De and En-Fr news translation tasks. There is no license available for this dataset.
OPUS-100 OPUS-100 100 is a multilingual translation corpus for 100 languages, which is randomly sampled from the OPUS collection (Tiedemann 2012). There is no license available for this dataset.
G.2 MODELS
Marian NMT Marian is a Neural Machine Translation framework, which is mainly developed by the Microsoft Translator team, and it released under MIT Licensa This model uses beam size of 4.
mBART50 mBART50 is multilingual machine translation model of 50 languages, which has been introduced by Facebook. This model is published in the Fairseq library, which is released under MIT License This model a beam size 5.
These two libraries are available at this Gitllub repostory http:///ttta..ooorrr//////////ar
24

---

Under review as a conference paper at ICLR 2023
G.3 CODES
kNN In order to compare our method with KNN (Michel et al. 2019), we used the code provided by the authors, which is released under the BSD 3-Clause "New" "Revised" License 13
Seq2Sick To compare our method with Seq2Sick (Cheng et al. 2020a), we used the code pub- lished by the authors There is no license available for their code.
WSLS We implemented and evaluated WSLS Zhang al. 2021 using the source code published
by the authors There is no license available for this GitHub repository.
H HUMAN EVALUATION
We conduct a preliminary human evaluation campaign of TransFool, KNN, and Seq2Sick attacks on Marian NMT (En-Frr in the white-box setting. We randomly choose 90 sentences from the test of the WMT14 (En-FR) dataset with the adversarial samples and their translations by the NMT model. We split 90 sentences into three different surveys to obtain a manageable size for each annotator. We recruited two annotators for each survey. For the English surveys, we ensure that the annotators are highly proficient English speakers. Similarly, for the French survey, we ensure that the annotators are highly proficient in French.
Before starting the rating task, we provided annotators with detailed guidelines similar to (Cer al.) 2017 Michel et al. 2019). The task is to rate the sentences for each criterion on continuous scale (0- 100) inspired by WMT18 practice (Ma et al. 2018) and Direct Assessment (Graham et al. 2013 For each sentence, we evaluate three aspects in three different surveys:
2011
We show the three adversarial sentences and the original sentence on the same page (in random order). We ask the annotators how much they agree with the "The sentence is fluent." statement for each sentence.
Fluency:
Semantic preservation: We show the original sentence on top and the three adversarial sentences afterwards (in random order). We ask the annotators how much they agree with the "The sentence is similar to the reference text." sttttement for each sentence.
Translation quality: Inspired by monolingual direct assessment (Ma et al. 2018 Graham et al.] (017) we evaluate the translation quality by showing the reference translation on top and the translations of three adversarial sentences afterwards (in random order). We ask the annotators how much they agree with the "The sentence is similar the reference text." statement for each translation.
We calculate 95% confidence intervals by using 15K bootstrap replications. The results are depicted in Figure 4 These results demonstrate that although the adversarial examples generated by Trans- and fluent than both baselines. According to the provided guide
Fool are more semantic-preservve to the annotators for semantic similarity, the score of 67.8 shows that the two sentences are roughly equivalent, but some details may differ. Moreover, a fluency of 66.4 demonstrates that although the generated adversarial examples by TransFool are more fluent than the baselines, there is still room to improve the performance in this regard.
We follow the direct assessment strategy measure the effectiveness of the adversarial attacks on translation quality. According to (Ma et al., 2018), since sufficient level of agreement of trans- lation quality is difficult to achieve with human evaluation, direct assessment simplifies the task to simpler monolingual assessment instead of a bilingual task. The similarity of the translations of the adversarial sentences with the reference translation is shown in Figure 4c The similarity of Seq2Sick is worse than other attacks. However, its similarity in the source language is worse. Therefore, we compute the decrease of similarity (between the original and adversarial sentences)
The source code is available at httpps////ttttttt.coooo hatp://wwwgggaaaaaaa////e//////r
and the license is avialable
http:///shhhhhhcc.oooaaaaaaaaaaa CENSE
source code is http://cccccccccaaaaaaooooo/aaaa wttpgh/hhhhnooiikk///////////eet
25

---

Under review as a conference paper at ICLR 2023
100
Orighsl Tranafool
c0-
Smilrrit barger Languigge (P))
50
Sirvlarity Durrose In
20-
MMN
Fivercy
60-
Sm tamarioo
60
10:
20-
10-
(a)
(b)
(c)
(d)
Figure 4: Human evaluation results for TransFool, kNN, and Seq2Sick attacks against Marian NMT
(En-Frr).
from the source language to the target language. The results in Figure 4d show that all attacks affect the translation quality and the effect of TransFool is more pronounced than that of both baselines.
Finally, we calculate Inter-Annotator Agreement (IAA). There are two hu- man judgments for each sentence. We average both scores to compute the final score for each sentence. To ensure that the two annotators agree, we only consider sentences where their corresponding scores less than 30. We compute IAA in terms of Pearson Correlation coefficient instead of the commonly used Cohen's K since scores in continuous scale. The results are presented in Table 24 Overall, we conclude that we achieve reasonable anter-annotator agreement for all sentence types and evaluation metrics.
Table 24; Inter-annotator agreement for human evaluation.
Sentence Type
Fluency Similarity in En Similairty in Fr
Original
0.68
TransFool
0.85
0.82
0.79
kNN
0.91
0.82
0.86
Seq2Sick
0.89
0.88
0.83
26


Reviews:
[{"review_id": "canPogon_3j", "reviewer": "Reviewer_8Cx6", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a white-box adversarial attack against neural machine translation model named TransFool. TransFool uses an optimization loss function with three terms: a) adversarial loss to maximize the loss of the target NMT model; b) a similarity term to ensure that the adversarial example is similar to the original sentence; c) and loss of a language model to generate fluent and natural adversarial examples. The authors compare their TransFool method with two NMT attack methods, kNN and seq2sick, with respect to the Attack Success Rate, Relative decrease of translation quality, Semantic Similarity, Perplexity score and Token Error Rate. The proposed method can outperform baseline methods on two NMT model, Marian NMT and mBART50 MNMT model. The authors also show that the adversarial examples they found can conduct transfer attack on black box NMT models, using the adversarial examples from Marian NMT to attack mBART50 MNMT model and google translation api.", "summary_of_the_review": "This paper proposes an adversarial attack method against neural machine translation model. The authors conduct experiments to show the superiority of their method. However, the novelty and the soundness of this paper is limited. So I tend to reject this paper.", "strength_and_weaknesses": "Pros:\n+ This paper is good written and easy to follow.\n+ The authors valid their method not only on research models but also on real world commercial product, like google translation.\n+ The transferability analysis section is enlightening. It is not widely studied in NLP field. The transferability between different language is an important issue in multilingual NLP filed.\n\n\nCons:\n- Novelty is limited. \na) First, the authors consider not only the attack success rate but also the fluency and semantic similarity. However, these two issues are widely studied in NLP attack field. For example, [1] has systematically reveal that the fluency and semantic similarity issue in textual adversarial attack and they empirically study the threshold of filtering the adversarial examples according to the fluency and semantic similarity. \nb) Second, this paper claims that they propose a new strategy to incorporate the embedding vectors of a language model. However, utilizing language model to attack NLP model is not a novel technique. For example, [2-3] utilize BERT to conduct word-level substitution.\n\n- Lack of soundness. This is not only the weakness of this paper but also the weakness of a large portion of textual adversarial attack works.\na) The soundness of their method is weak. This paper use language model due to it provide a meaningful representation of the tokens. However, based on the finding of previous work [1], the semantic similarity cannot be guaranteed with language model. A good example is that “I [like] eating apple.” and “I [hate] eating apple” have very high representation similarity according to the language model representation. But their semantic are totally opposite. So I am worried that the adversarial attack will change the semantic meaning of source sentence, leading to the over-estimation of attack success rate.\nb) The soundness of their evaluation is weak. The authors evaluate the semantic similarity by the universal sentence encoder and BERTScore. However, as I mentioned above, the soundness of such model-based automatic metrics is limited. The high attack success rate could be partially due to the changing of meaning in source sentence. Conducting a human annotation experiment will be better than just showing an example.\n\n[1] John Morris, Eli Lifland, Jack Lanchantin, Yangfeng Ji, Yanjun Qi ,Reevaluating Adversarial Examples in Natural Language, EMNLP 2020.\n[2] Linyang Li, Ruotian Ma, Qipeng Guo, Xiangyang Xue, Xipeng Qiu, BERT-ATTACK: Adversarial Attack Against BERT Using BERT, EMNLP 2020.\n[3] Siddhant Garg, Goutham Ramakrishnan, BAE: BERT-based Adversarial Examples for Text Classification, EMNLP 2020.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "SB5O_LGPxO", "reviewer": "Reviewer_6HVS", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper defines a new optimization objective function which combines the fluency, similarity and translation error to adversarially attack machine translation models. A gradient projection algorithm is applied to solve this optimization. Experiment results show the proposed method outperforms baselines. The transferability is also examined. \n", "summary_of_the_review": "The novelty of the proposed method is somewhat incremental. \n\nThe experiment part is strong in terms of datasets and models. The automatic evaluation metrics also show strong performance. \n\nI think adding a human evaluation would reveal lots of insights. ", "strength_and_weaknesses": "Strength\n\n- The proposed method is simple and straightforward. \n- The experiment results show superior performance in attack success rate and significant decrease in translation quality. \n- This work also demonstrates the capability and efficiency of blackbox attack.\n\n\nweaknesses\n\n- Existing adversarial attack on NMT aims at improving the robustness of these models. However, in this work, achieving a high attack success rate seems to be the goal. Therefore, I’m wondering if TransFool is as effective as baselines in improving robustness. Or what is the desired use case for this method?\n- Translation models use beam search or similar mechanisms to generate high-quality output. Is the proposed attack still effective when using these mechanisms?\n- Missing human validation. Although automatic metrics show significant decrease in translation quality, I’m not convinced that the algorithm triggers incorrect translation. Maybe the translation is correct but has a very low BLEU or chrF score (i.e., the attack method is attacking the automatic metrics instead of the NMT model, which could also be an interesting finding). \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "NwvnPv0fRsx", "reviewer": "Reviewer_Jg82", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes a new approach for adversarial attacks on NMT models.\nThe authors design a way to propagate the gradient for embeddings and propose a specific loss that targets reasonable criteria.\nThe propagation is based on the common idea of relaxation with a new additional FC part to create embeddings for comparison.\nThe metrics are good, and overall, the results are promising.", "summary_of_the_review": "The results seem promising, on the other side, the novelty is limited. From the paper, it is not clear, why this approach works in the NMT-attacks field, where other approaches failed.", "strength_and_weaknesses": "Strengths:\n- Quality metrics are better than that of competitors.\n- Experiments are interesting and numerous. They include human evaluation as well.\n- Nice adversarial example found in Table 2\n\nWeakness, methods:\n- The novelty is limited. The paper contains some engineering tricks to make everything work, but all ideas are similar to those previously discussed in the literature.\n- Not clear why we need the FC part in the approach. We can use BERT-like architectures to generate similar differentiable $v_i$-s as well. What would be the difference in this case?\n- Investigation on the distortion of the generated adversarial sequence $x'$ is limited and includes only \"Semantic Similarity\" from Yang et al. Can you include metrics like BERT score for a pair of $x$ and $x'$ as well?\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "Bk9krBqi4fR", "reviewer": "Reviewer_ZrSf", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This work proposes TransFool for generating non-targeted adversarial examples against neural machine translation models. One core idea is to utilize an autoregressive language model (GPT-2) to add a language model loss term, which helps generate fluent adversarial examples. They also add a similarity loss, which constrains the distances of embedding distances between the original input sentences and adversarial examples. They perform a comprehensive study to evaluate their approach on translating from English to different target languages, and show that TransFool achieves higher attack success rates compared to baselines, while the generated adversarial examples better preserve the semantic meaning and look more natural. Meanwhile, they show that the generated adversarial examples transfer to other models in the black-box setting, including Google Translate, and can also transfer to different target languages.", "summary_of_the_review": "This work proposes a well-designed adversarial attack for neural machine translation models, though it is not entirely novel. The paper presents a comprehensive study in different attack settings. In particular, the transferability of attacks between different target languages is interesting and new. Therefore, I lean towards accepting this work.\n\n-------------\nI thank the authors for explanation and adding more experiments, and I keep my original score.", "strength_and_weaknesses": "Strengths:\n1. Each term in the loss function of TransFool is well-motivated and properly designed.\n\n2. The empirical study is pretty thorough. To me, the most interesting finding is that the generated adversarial examples can transfer to different target languages. \n\nWeaknesses:\n\n1. The study of defense is lacking. For example, it is helpful to see how the attack works with existing defenses against adversarial examples for language models. Also, it is interesting to try adversarial training with TransFool adversarial examples.\n\n2. This is more of a question rather than a weakness, but have you tried TransFool for targeted attacks, and how does that work?\n\n3. It is good to investigate more into the transferability between different target languages. For example, have you done these experiments on Google Translate?", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject