Learn2Weight: Weights Transfer Defense against Similar-domain Adversarial Attacks
Authors: Siddhartha Datta
Venue:  (2021)
Keywords: adversarial attack
robustness
domain adaptation
privacy-preserving machine learning

Abstract: Recent work in black-box adversarial attacks for NLP systems has attracted attention. Prior black-box attacks assume that attackers can observe output labels from target models based on selected inputs. In this work, inspired by  adversarial transferability, we propose a new type of black-box NLP adversarial attack that an attacker can choose a similar domain and transfer the adversarial examples to the target domain and cause poor performance in target model. Based on domain adaptation theory, we then propose a defensive strategy, called Learn2Weight, which trains to predict the weight adjustments for target model in order to defense the attack of similar-domain adversarial examples. Using Amazon multi-domain sentiment classification dataset, we empirically show that Learn2Weight model is effective against the attack compared to standard black-box defense methods such as adversarial training and defense distillation. This work contributes to the growing literature on machine learning safety.

Under review as a conference paper at ICLR 2021
LEARN2WEIGHT: WEIGHTS TRANSFER DEFENSE AGAINST SIMILAR-DOMAIN ADVERSARIAL ATTACKS
Anonymous authors
Paper under double-blind review
ABSTRACT
Recent work in black-box adversarial attacks for NLP systems has attracted at- tention. Prior black-box attacks assume that attackers can observe output labels from target models based on selected inputs. this work, inspired by adversarial transferability, we propose new type of black-box NLP adversarial attack that an attacker can choose a similar domain and transfer the adversarial examples to the target domain and cause poor performance in target model. Based on domain adaptation theory, we then propose a defensive strategy, called 2earn2Weigth which trains to predict the weight adjustments for target model in order to defense the attack of similar-domann adversarial examples. Using Amazon multi-domain sentiment classification dataset, empirically show that Leaht model is effective against the attack compared to standard black-box defense methods such as adversarial training and defense distillation. This work contributes to the growing literature on machine learning safety.
INTRODUCTION
As machine learning models are applied to more and more real-world tasks, addressing machine learning safety is becoming an increasingly pressing issue. Deep learning algorithms have been shown be vulnerable to adversarial examples (Szegedy et al., 2013; Goodfellow et al. 2014 Papernot et al. 2016a). In particular, prior black-box adversarial attacks assume that the adversary is not aware the target model architecture, parameters or training data, but is capable of querying the target model with supplied inputs and obtaining the output predictions. The phenomenon that adversarial examples generated from one model may also be adversarial to another model is known as adversarial transferability (Szegedy al., 2013).
Motivated by adversarial transferability, we conjecture another black-box attack pipeline where the adversary does not even need to have access to the target model nor query labels from crafted inputs. Instead, as long as the adversary knows the task of the target, he can choose a similar domain, to build substitute model and then attack the target model with adversarial examples that generated from the attack domain.
Figure 1: Diagrammatic representation of the problem
Attack Domain
Target Domain
Adversarial samples labels
Oploid risk model (insurance company)
Attack Data
Target Data
Label:
Original patient history
HIGH
Adversarial
Attack Model
attock
Target Model
Painkller patient history data (attack domain)
Perturbed
algarithm
patient history
Opioid risk model (insurance company)
Misclassflle
classified sentences
Correctlyy
Predicted babels
Correctly-slassiiied sentences,
Surrogata model [physician)
Label: LOW
adversar
fiitered similarity
Fast Gradient Sign Method
symptoms original history
(a) Generalized architecture of similarity-based at- tacks.
(b) Flow of how an adversary physician can leverage similarity attack to fool opioid risk models.

---

Under review as a conference paper at ICLR 2021
The similar-domain adversarial attack may be more practical than prior blackbox attacks as label querying from target model is not needed. This attack can be illustrated with the following example (Figure [[)) in medical insurance fraud (Finlayson et al. 2019). Insurance companies may use hypothetical opioid risk models to classify the likelihood (high/low) of a patient to abuse the opioids to be prescribed, based on the patient's 's medical history as text input. Physicians
run the original patient history through the attack pipeline to generate an adversarial patient history, where the original more likely be rejected ("High" risk) and the adversarial is more likely to accepted ("Low" risk). Perturbations in patient history could be, for example, a slight perturbation from "alcohol abuse" to "alcohol dependence", and may successfully fool the insurance company's 's model.
Based on domain adaption theory features that cause the success the similar-domain attack. The adversarial examples with domain- variant features are likely to reside in the low-density regions (far away from decision boundary) of the empirical distribution of the target training data which could fool the target model (Zhang et al., 2019b). Literature indicates that worsened generalizability is tradeoff faced by existing defenses such as adversarial training (Raghunathan et al. 2019) and domain generalization techniques (Wang et al. 2019). In trying to increase robustness against adversarial inputs, model faces tradeoff of weakened accuracy towards clean inputs. Given that an adversarial training loss function is composed of loss against clean inputs and loss against adversarial inputs, improper optimization where the latter is highly-optimized and the former weakly-one -ootiioop does not improve general performance in the real-world. To curb this issue, methods have been proposed (Zhang et al.) 2019b Lamb et al. 2019 Schmidt et al. 2018), such as factoring in under-represented data points in training set.
et al. 2010), we conjecture that it is the domain- Nomain-variant
To defend against this similar-domain adversarial attack, we propose a weight transfer network approach, Learn2Weight, so that the target model's s decision boundary can adapt to the examples from low-density regions. Experiments confirm the effectiveness of our approach against the similar- domain attack over other baseline defense methods. Moreover, our approach is able to improve robustness accuracy without losing the target model's standard generalization accuracy.
Our contribution can be summarized as follows:
We are among the first to propose the iimilar-domain adversarial attack. This attack pipeline relaxes the previous black-box attack assumption that the adversary has access to the target model and can query the model with crafted examples.
We propose a defensive strategy for this attack based on domain adaptation theory. Experi- ment results show the effectiveness of our approach over existing defense methods, against the similar-domain attack.
2 RELATED WORK
Recent work in adversarial attack for NLP systems has attracted attention. See (Zhang et al. survey for an overview of the adversarial attack in NLP. Existing research proposes different attack methods for generating adversarial text examples (Moosavi-Dezfooll et al. 2016 Ebrahimi et al. 2018; Wallace et al. 2019) The crafted adversarial text examples have been shown to fool the tha NLP systems such as BERT (Jin et al. 2019). A large body of adversarial attack research focuses on black-box attack where the adversary builds a substitute model by querying the target model with supplied inputs and obtaining the output predictions. The key idea behind such black-box attack is that adversarial examples generated from one model may also be mis-classified by another model, which is known adversarial transferability (Szegedy al.) 2013 Cheng et al.) 2019). While prior work examines the transferability between different models trained over the same dataset, the transferability between the same different model trained over disjoint subsets of dataset, our work examines the adversarial transferability between different domains, which we call similar-domaio attack.
2020)
2

---

Under review as a conference paper at ICLR 2021
Attack domain: baby, Target domain: books
Original sentence (Actual label: Pos)
I purchased this toy for my son when he was 4 months old, At first, he seemed little intimidated by the toys.
Pos (0.712)
obtained this toys for my children when he was 4 weeks senior. At first, hoped a modest harassed by the toy.
Adversarial sentence
Neg (0.364)
It felt like a big commitment for me to have to run the program 2 times day, and near the end of my pregnancy I was annoyed with having anything strapped across my belly.
Original sentence (Actual lahel: Pos)
Pos (0.825)
It felt like a big committed for me to have to run the progrum 2 length a day. and near the end of my pregnancy I was annoyed with takes anything strapped across my belly.
Adversarial sentence
Neg (0.420)
Attack domain: dvd. Target domain: baby
Original sentence
Fast times at ridgemont high is clever, insightful, and wicked film! It is not just another teen movie.
Pos (0.614)
(Actual label: Pos)
Sooner days ridgemont high a sane, thoughtful, and wicked flick! It is not just another adolescent flick.
Adversarial sentence
Neg (0.335)
Original sentence
This dvd gives a very good 60 minute workout. As others have pointed out the cardio is very dancy. The first time I did it, I felt bit awkward with the steps. This dvd gives awfully okay 60 minute exercise. As others have pointed out the cardio very dancy. The first time did perceived bit awkward with the steps.
Pos (0.647)
(Actual label: Pos)
Adversarial sentence
Neg (0.258)
Table 1:
Comparison of attack domain sentences correctly classified when unperturbed by respective attack domain models and target domain models, then misclassified after perturbation by target models trained on books and baby domain. The perturbations are in blue, and prediction confidence in brackets.
3 SIMILAR-DOMAIN ADVERSARIAL ATTACK
3.1 ADVERSARIAL ATTACK BACKGROUND
Adversarial attacks modify inputs to cause errors in machine learning inference (Szegedy et al. 2013) ))
We utilize the basic gradient-based attack method Fast Gradient Sign Method (FGSM) (Goodfellow
let al. 2014) and its variants, RAND-
(Tramèr et al. (017)) and Basic Iterative Method (BIM) (Kurakin al., 2016a b) Xie al. 2018). Other NLP adversarial generation algorithms could also be used, such as DeepFool (Moosavi- Dezfooli et al. 2016). HotFlip (Ebrahimi et al., 2018), universal adversarial trigger (Wallace et al.] 2019). and TextFooler (Jin et al. (019). To perform gradient-bane based perturbations upon discrete space data, we follow (Yang al. 2018 to generate adversarial text.
Our proposed similar-domain adversarial attack is in-variant to adversarial algorithm, meaning that the adversarial algorithm used would not affect the attack performance. Without losing generality, we denote Adv[J,2) .)) as an NLP adversarial text generation method, defined as below.
Definition 1. NLP Adversarial Generation. Given a deep neural network model built on text data X, an NLP adversarial generation method produces one adversarial instances x' +- Adu(f, x) for x E X, x' ss x. The goal of the adversarial attack to deviate the label to incorrect one f(x') # f(x).
3.2 SIMILAR-DOMAN ADVERSARIAL ATTACK
We present the architecture of Aimilar-doman adversarial attack in Figure Th. The defender, the target of the attack, constructs target model trained on domain text data 7 (0). An attacker, only having rough idea about the target's task but lacking direct access to the target data or target model parameters, collects attack data from a similar domain S and trains attack model (1). He runs the attack model on the test data (2) to obtain correctly-classffeed instances (3). He chooses an adversarial attack algorithm and generates set of adversarial samples A (4). He exposes A to the target model, hoping A mislead the target model to produce an output of his choice (5). This type of attack works best as adversarial attack that compromises systems that base caaisiionmmainng
one-instance. Definition 2. Similar-domain Adversarial Attack. A target model f. built on target domain data T. a deep neural network model with parameter weights Wy that maps a text instance to a label: y < s(X, Wr). An adversary chooses source attack domain S, builds substitute model fs. and generates a set of adversarial examples from S using Ado(Js, S), so that during an attack f(A, Wr) fs(A).
3

---

Under review as a conference paper at ICLR 2021
Target Domain
book
magazine
baby
Original Accuracy
0.880
0.960
0.890
Intra-attack Accuracy
0.525
0.570
0.632
Attack Domain
magazine
baby
dvd
baby
dvd
book
dvd
book
magazinc
Unperturbed Accuracy
0.726
0.646
0.745
0.739
0.663
0.673
0.624
0.652
0.665
After-attack Accuracy
0.398
0.421
0.395
0.381
0.366
0.343
0.365
0.386
0,401
Shared Vocab
0.381
0.255
0,455
0.260
0.345
0.381
0.270
0.255
0.260
Transfer Loss
0.017
0.071
0.000
0.079
0.022
0.010
0.066
0.050
0.069
Table Similar-domain attack performance. Bold indicates the least successful attack domain (i.e. highest after-attack accuracy) for each target domain, as well as the corresponding transfer loss value.
3.3 DOMAIN SIMILARITY
Here, domain similarity refers to the similarity between attacker's chosen domain and defender's domain. Vocred measures the overlap of unique words, in each of the datasets; higher degree of overlapping vocabulary implies the two domains are more similar. We also use Transfer Loss, a standard metric for domain adaptation Blitzer et al. (2007): : Glorot et al. (20IT), to measure domain similarity; lower loss indicates higher similarity. The test error from a target model trained on target domain T' and evaluated on attack domain S returns transfer error e(S, 2). The baseline error e(T; T') term is the test error obtained from target model trained on target domain (train) data T and tested on target domain (evaluation) data T, This computes the transfer loss, (f T) = c(S. 7) e(T. T).
4
IS THE ATTACK EFFECTIVE?
4.1
SETUP
Dataset. We simulate the
adversarial attack using Amazon's multi-domain sentiment classification dataset (Blitzer et al., (007), a commonly-used dataset in cross-domain sentiment classificatior with 1,000 positive and 1,000 negative reviews for each of the 25 product categories.
Model. In practice, there could be unlimited choice for the attack model and target model, such as different deep learning architecture, different training parameters. To simplify the discussion, we choose Long Short-Term Memory (LSTM) network as suitable baseline sentiment classification model (Wang et al. 2018) for our target model and attack model. The architecture consists of 64 L.TTM cells, 80% dropout, using sigmoid activation function.
Metrics. We first report the accuracy of the target models on the target domain test samples before the attack the original accuracy. Then measure the accuracy of the target models against adversarial samples crafted from the attack domain samples, denoted as the after-atlack accuracy. Intra-attack accuracy denotes the after-attack accuracy where the attack domain is identical to the target domain. By comparing original and after-attack accuracy, we can evaluate the success of the attack. The greater the gap between the original and after-attack accuracy, the more successful the attack. Unperturbed accuracy measures the accuracy of the target model against the complete, unperturbed test set of the attack domain, to demonstrate that any drop in classification accuracy is not from domain shift alone but from adversarial transferability.
4.2 RESULTS
The
adversarial attack results are presented in Table 2 between original accuracy and after-attack accuracy indicating that this attack can impose valid threat to target NLP system. After the similar-domain adversarial attack, the accuracy drops dramatically by large margin. Take the book target domain for example, when the attack domain is magazine, the after-attack accuracy drops to 0.398, and when the attack domain is baby, the accuracy is 0.421. Moreover, we observe a positive correlation between transfer loss and after-attack accuracy, and negative correlation between shared vocab and after-attack accuracy.
We see a significant gap
Data is available from /www. http:///aa/////////////////////a
4

---

Under review as a conference paper at ICLR 2021
5 DEFENDING AGAINST SIMILAR-DOMAII ADVERSARIAL ATTACK
In order to defend against a similarity based adversarial attack, it is critical to block adversarial transferability. Adversarial training is the most intuitive yet effective defense strategy for adversarial attack (Goodfellow et al., 2014, Madry et al. 2017). However, this may not be effective for two reasons. First, there is no formal guidance for generating similar-domain adversarial examples because the defender has no idea what the attack data domain is. Second, simple feeding the target model with adversarial examples may even hurt the generalization of the target model (Raghunathan)
et al. 2019 Zhang et al. 2019a Su al. 2018), which is also confirmed in our experiments.
5.1
WEIGITT TRANSFER LEARNING
The use weight transfer networks (Ha al. 2016 Hu et al. 2018: Kuen al. 2019) concerned with adapting weights from one model into another, and generating/ generating/predicting the complete set of weights for a model given the input samples. In our context, distinctly different weights are produced for target models trained on inputs of different domains, and feature transferability (Yosinski et al. 2014) in the input space can be expected to translate to weight transferability the model weights space. Rather than completely regenerating classification weights, model robustification defense, Leight L2W predicts the perturbation to existing weights (w' W Aw) for each new instance.
5.2 LEARN2WEIGHT MODEL.
We conjecture that effective defense strategy is to perturb the target model weights depending on the feature distribution the input instance. L2W (Algorithm [ recalculates the target model weights depending the input. L2W (Algorithm 2) trains on sentences from different domains and weight differential for that domain (the weight adjustment required to tune the target model's 's weights to adapt the input's domain). We obtain the weight differential AW by finding the difference between the weights of f trained on sentence:label pairs from a specific domain Ws, and weights of trained sentence:label pairs from the target domain Wr. Other training models may be possible; here we trained a sequence-to-sequene network (Sutskever et al. 2014) on sentence: W pairs.
Algorithm 1:
Algorithm 2:
LeighWeight: Inference
Learn2Weight Training
inference (x.dv, L-W(-)) f' T)) Input :Passed arguments include
train (T,D)
Input :Target domain T = (T. 3 No: Set of M
the adversarial input X aadv the Learn2Weight model (-W(-). the target model f(Wr, T)
domains D = (S., =i with N sentences,
i indexing specific sentence tensor and j indexing specific domains
Output : Trained Leght model L2W(-)
Output: Target model with weighted updated by Leam2Weigght f(W*, T') is the expected oulput
Initialize empty X and to store sentences X; from each domain j with corresponding weight differential. X <- 0; Y +- 0;
Pass X as in input into the trained (-)((-) (-) function, and the weight differential required for the target model with weights Wr is AW. AW (X.
Compute weights of ff trained T. WT + f(T);
T:Y W Wr:
Train each domain S,, compute respective weights, append the differential AW to Y and each sentence in St, into X.
The returned function is the target model with updated weights Wy AW.
foreach domain S, E D do
Ws, f(S));
return f(Wr + AW,T);
AW Ws, Wy:
X S;, Y AW;
Train sequence model on (X Y) pairs, and return the model as Learn2Weight L2W. return L2W (X, Y);
5

---

Under review as a conference paper at ICLR 2021
Algorithm 3: tf-optimization
fOptimization (T, (mmas)
Input :Target domain T = (7, i Iizo to he used in synthesizing M similar domains; with N sentences, indexing specific sentence tensor; Nomr is the number of tf-optimization iterations
Set D containing domains
Initialize empty D to store synthesized domains Sj of index j.
D + 0:j= + 0;
while j < M do
Run each iteration until Amas.
for iter + 0 to Amaz do
Apply adversarial perturbations T.
Tade Adu(f.T);
Determine change to Ado(() or iter depending on computed transfer loss,
if ch T)) True then
If low enough, Tter can be added as synthetic domain into D.
D - Tters
break:
else
Adjust perturbation parameters. Adu(-) + adjjss((Ade())):
j j
Return set of Domains to be used for training by L2W. return D;
5.3 TRANSFER LOSS OPTIMIZATION
To generate synthetic domains of varying domain similarity so that defenders defend their model using only target domain data T, the following equation introduces transfer loss optimization (Algorithm The defender iteratively generates adversarial examples X X while maximizing the transfer loss Tunction iʃ:; this produces substitute attack domain corpora Sy. We iteratively adjust perturbation parameters Adv(-) and iteration count N minimize the transfer loss of our generated dataset.
3p.
arg min ifN (xx* :2) = - =eAAu((,)))))) - e (I,T)
N.Ade()
5.4 EXPLANATION: BLOCKING TRANSFERABILITY
To facilitate our explanation, we adapt from domain adaptation literature (Ben-David al. 2010 Liu al. 2019 Zhang al. 2019c):
e(A,T) S eTT,T) + dwan (A,T) +
where H. is the hypothesis space, J is hypothesis function that returns labels {0, 1}. and e(T, T) and e(A, T') are the generalization errors from passing target domain data T and adversarial data A through classifier trained on T. dæAH(AA (A, T') is the HAH-distance between T and A, and measures the divergence between the feature distributions of and T. e (h, h. and er h represents the probability that disagrees with h on the label of an input in the domain space A and T respectively.
dusu(A, T)
sup (ea(h, h') er
h.h h. EM
dAax(A,T) sup iew [x~s(((h<>) - n' (+)
Ex-r [[(((()) (+)1
Divergence dHAm measures the divergence between feature distributions and T. Higher duan indicates less shared features between 2 domains. The greater the intersection between feature distributions, the greater the proportion of domain-variant features; one approach to domain adaptation is learning domain-invariant features representations (Zhao al. 2019) to minimize dæAH.

---

Under review as a conference paper at ICLR 2021
Target Domain
magazine
baby
Attack Domain
baby
dvd
book
dvd
book
magazine
After-attack Accuracy
0.381
0.366
0.343
0.365
0.386
0.401
After-defense Accuracy
Adversarial training
0.6.9
0.559
0.657
0.358
0.377
0.661
Defensive distillation
0.549
0.561
0.597
0.588
0.629
0.577
Shared defense
0.628
0.653
0.631
0.664
0.668
0.621
Domain-adapted adversarial training
0.608
0.637
0.620
0.604
0.620
0.587
Learn2Weigth
0.796
0.842
0.843
0.774
0.751
0.737
Table 3: After-defense accuracy performance of different defensive methods.
Explaining Aimilarit-ddomen attacks, As demonstrated by empirical results, e(A, T') increases in a similarity-basee attack setting, and this would arise if dHAH increases correspondingly. dHaH computes inconsistent labels from inconsistent feature distributions, and attributes the success of the attack to domain-variant features.
FGSM and variants adjust the input data maximize the loss based on the backpropagated gradients of a model trained on S. As our pipeline used correctly-labelled sentences before adversarially perturbing them, we can infer that perturbations applied S were not class-dependent (i.e. the success of the attack is not based on the removal of class-specific features), but lass-independent features. It is already difficult for a model trained on to classify when there is insufficient class- dependent features (hence high (A, T)): in cross-domain setting, it must be even more difficult for a model trained on T to classify given a shortage of nomain-invarian class-dependent features.
duaN (A,,T) - e(T,T)-A
dusn S (t((ATT) A
Explaining Learn2Weight. L2W minimizes divergence by training (dxa*(5). AWa.s)) pairs, dS,n((S T) being reconstructed from the difference between 2 and xff' L2W is trained on (daan 10=0 (A 10= pairs, such that Aw = ( Intuitively the target model possesses decision boundary (Liu et al. 2019) to classify inputs based whether they cross the boundary or not; adversarial inputs have tendency of being near the boundary and fooling it. Weights transfer learning applies perturbations the decision boundary such that the boundary covers certain adversarial inputs otherwise misclassified, and in this way blocks transferability. The advantage of training on multiple domains {S is that the after-L2W divergence between A and T is smaller because L2W's ' weight perturbations render the decision boundary more precise in classifying inputs.
Explaining tf-optimization. We have attributed why adversarial sentences A are computed to be domain-dissimilat despite originating from S due to insufficient domain-invariant, class-dependent features resulting in low eAA, 7"), i.. low ff, T). To replicate this phenomenon in natural domains, we use tf-optimization to iteratively perturb T' to increase the proportion of class-independent features. This approximates the real-world similarity- features may be limited for inference. By generating the synthetic data, we are feeding L2W attack data with variations dHAR and class-independent feature distributions. This prepares L2W to robustify weights in ((T) when such feature distributions are encountered.
attack scenario where class- dependet
EXPERIMENTS
6.1 BASELINES
We consider two defense strategies that empirically effective and are widely used for general black-box -bx adversarial attacks: adversarial training (Goodfellow et al.) 2014 Madry et al., 2017)) and defensive distillation (Papernot al. 2016b 2017). In addition we consider two ablation lnnelines.
Defensive distillation: The high-level implementation of defensive distillation (Papernot et al.. 2016b; 2017) is to first train initial model against target domain inputs and labels, and retrieve the raw class probability scores. The predicted probability values would be used as the new labels for the same target sentences, and we would train model based this new label-sentence pair.
7

---

Under review as a conference paper at ICLR 2021
Target Domain
Original
Attack Domain
Attack Model
Stter-Defens Accuracy
Accuracy Target Model:
After-attack Accuracy
LSTM
BERT
LSTM
GRU
CNN
LogReg
BERT
LSTM
GRU
CNN
LogReg
dvd
0,342
0.413
0.477
0.335
0.440
0.786
0.847
0.804
0.816
0.782
book
0.880
kitchenware
0.350
0.372
0.325
0.353
0.425
0.765
0.826
0.795
0.742
0.767
electronics
0.400
0.389
0.416
0.315
0.460
0.792
0.812
0.784
0.770
0.725
book 0.326
0.434
0.479
0.383
0.490
0,816
0.795
0.824
0.804
0.794
dvd
0.920
kitchenware
0.355
0.370
0.379
0.359
0.490
0.728
0.796
0.755
0.735
0.695
electronics
0.387
0.377
0,332
0.348
0.455
0.825
0.836
0.812
0.834
0.796
book 0,425
0.394
0.473
0.358
0,474
0.775
0,821
0.795
0.782
0.712
electronics
0.910
dvd 0.342
0.395
0.452
0.368
0.493
0.784
0.845
0.855
0.842
0.792
kitchenware
0.390
0.384
0.464
0.329
0.432
0.730
0.824
0.753
0.724
0.678
Table 4: Learn2Weight comparison against different attack model architectures.
Adversarial training: It is shown that injecting adversarial examples throughout training increases the robustness of target neural network models. In this baseline, target model is trained with both original training data and adversarial examples generated from original training data. However, since the adversarial examples are still generated from the target domain, is unlikely that the method can defend similar-domain attack which is the result of domain-variant features.
Shared defense. Given that is the domain-varie variant features that cause the success of similar- domain attack, simple baseline is to remove those words that are not in the target domain's vocabulary. This tests whether the effect of perturbing target model weights w.r.t domain-variant features will yield incremental after-attack accuracy in the similar-domain attack setting.
Domain-adapted adversarial training. This ablation baseline tests for incremental performance baseline defense using domain-variant inputs. We adapt adversarial training to be trained on adversarial sentences from attack domain S, whereas the traditional adversarial training generates adversarial samples X xade,1' from its training data T, this adapted version uses adversarial samples X generated from S.
6.2 LEARN2WEIGHT PERFORMANCE
Defense performance. We present the results of different defense baselines in Table 3 see that Learn achieves the highest after-defense accuracy against the adversarial attack. Take the magazine as target domain for example, if the adversary chooses to use book data as the attack domain, it would reduce the target model accuracy to 0.343. However, the Learn2Weight method can improve the performance to 0.843, which is significant and substantial improvement against the attack. This improvements also exist across different target/attack domain pairs. Second, that all defense methods can improve the accuracy to some extent which indicates the importance and effectiveness of having robust training for machine learning models. Third, is interesting to note that two simple baselines Shared Vocab defense and Domain-adapten adversarial training yield overall better performance compared to adversarial training and defensive distillation.
First, we can
Attack model architectures. So far, all the results are conducted using the same LSTM as the target/attack model due to simplicity purpose. Here, we keep the target model unchanged, but vary the architecture of the attack model for the generation of adversarial examples. A variation of Recurrent Neural Network is Gated Recurrent Unit (GRU) network, with 512 GRU cells, 60% dropout and tanh activation function. We have also tested other attack model variants that are commonly-used sentiment classification, including Bidirectional Encoder Representations from Transformers (BERT) (Devlin et al. 2019). Convolutional Neural Network (CNN) (Kim 2014), and Logistic Regression (Maas et al.) 2011p. For both RNN and CNN, we use pre-computed Glove embeddings to encode words. All models are trained with enough epochs after ensuring the model achieved near state-of-the-art validation accuracy before proceeding to tests of adversarial attacks and defenses.
We present the results different attack model architectures in Table First, similar-domain attack model-agnostic and does not require the target and attack model to have identical architectures.
available httpaaammttttttee/o.o..cafrrrree
8

---

Under review as a conference paper at ICLR 2021
We can that all four attack model architectures are able to reduce the target model accuracy. Second, the results suggest that Leam2Weight is also model-agnostic as it can substantially improve the after-defense accuracy regardless which attack model is used.
CONCLUSION
In this domain to the target task, build a substitute model and produce adversarial examples to fool the target model. We also propose defense strategy, Leam2Weight, that learns adapt the target model's weight using crafted adversarial examples. Compared with other adversarial defense strategies, Leight can improve the target model robustness against the similar-domain attack. Our method demonstrates properties of good adversarial defense, such as adopting defense architectures that adapt situations/inputs rather than compromising standard error versus robustness error, to leverage class- class-iddppedent properties in domain-variant text, and factoring in domain similarity in adversarial robustness exercises.
impirically-effectii similar-domas attack, an adversary can choose similar
REFERENCES
Shai Ben-David, John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, and Jennifer Vaughan. A theory of learning from different domains. Machine Learning, 19:151-175, 2010. URL
http::/www..st www.nrrrinnnnnn...m http://www...ccccoaaaaaaa/a////1
John Blitzer, Mark Dredze, and Fernando Pereira. Biographies, bollywood, noom-boxes and blenders: Domain adaptation for sentiment classification. In ACI., pp. 440 447, 2007.
Shuyu Cheng, Yinpeng Dong, Tianyu Pang. Hang Su, and Jun Zhu. Improving black-box adversarial attacks with a transfer-based prior. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché- Buc, Fox, and R. Garnett (eds.), Advances in Neural Information Processing Systems 32. pp. 10934-10944 Curran Associates, Inc., 2019. http:/
wwtp:iicps....pp...eeeeeeepppeee - box- adversa ria attacks th- aacchha-ttraae trana based- odf
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter the Association for Computational Linguisties: Human Language Technologies, Volume (Long and Short Papers), pp. 4171-4186, Minneapolis, Minnesota, June 2019. Association for Computational Linguisties. doi: 10. 10.18653/v1/N19 1423. URL https
www. www.:alwwcb.crrcm wttp./////anntoooo//----11--
Javid Ebrahimi, Anyi Rao, Daniel Lowd, and Dejing Dou. Hotflip: White-box adversarial examples for text classification. In ACL, pp. 31-36, 2018
Samuel G. Finlayson, John D. Bowers, Joichi Ito, Jonathan L. Zittrain, Andrew L. Beam, and Isaac Kohane. Adversarial attacks medical machine learning- Science, (63164433) 2019. ISSN 0036-8075. doi: 10. 1..166/cciencc.uuw433333 URL htttp::/ www.:sciieencn.....e org/ content/36376433/125
Xavier Glorot, Antoine Bordes, and Yoshua Bengio. Domain adaptation for large-scale sentiment classification: A deep learning approach. In ICMI., pp. 513-520, 2011.
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572 2014.
David Ha, Andrew Dai, and Quoe V Le. Hypernetworks. arXiv preprint arXiv: /ww./............. 2016.
Ronghang Hu, Piotr Dollár, Kaiming He, Trevor Darrell, and Ross Girshick. Learning to segment every thing. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pp. 4233-4241, 2018
Di Jin, Zhijing Jin Jocy Tianyi Zhou, and Peter Szolovits. bert really robust? natural language attack text classification and entailment. arXiv preprint arXiv: /907. 1932, 2019.

---

Under review as a conference paper at ICLR 2021
Yoon Kim. Convolutional neural networks for sentence classification. In EMNLP, pp. 1746-1751, 2014.
Jason Kuen, Federico Perazzi, Zhe Lin, Jianming Zhang, and Yap-Peng Tan. Scaling object detection by transferring classification weights. In Proceedings of the IEEE International Conference Computer Vision, pp. 6044-6053 2019.
Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial machine learning at scale. arXiv preprint /61 1.01236, 2016a.
Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial examples in the physical world. arXiv preprint
2016b.
Alex Lamb, Vikas Verma, Juho Kannala, and Yoshua Bengio. Interpolated adversarial training: Achieving robust neural networks without sacrificing too much accuracy. In Proceedings of the 12th ACM Workshop on Artificial Intelligence and Security, AlSec' 19, pp. 103, New
York, NY, USA, 2019. Association for Computing Machinery. ISBN 9781450368339, doi: 10.1145/3338501 3335533333 .3357369. URL https::/dtii..oooo org// 0. 1.144133335000000 1453/3333001113333333 3357369
Hong Liu, Mingsheng Long, Jianmin Wang, and Michael Jordan. Transferable adversarial training: A general approach to adapting deep classifiers. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pp. 4013 4022, Long Beach, California. USA, 09-15 Jun 2019. PMLR. URL http://procceddnng mir www.s/pssssse.............s...e / /v liu19b html
Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts. Learning word vectors for sentiment analysis. ACL, pp. 142-150, 2011.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv: (tdv::170..060 06083, 2017.
Scyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, and Pascal Frossard. Deepfool: simple and accurate method to fool deep neural networks. In CVPR, pp.
2016.
Nicolas Papernot, Patrick McDaniel, and Ian Goodfellow. Transferability in machine learning: from phenomena to black-box attacks using adversarial samples. arXiv preprint arXiv: 1605.:105..077 07277. 2016a.
Nicolas Papernot, Patrick McDaniel, Xi Wu, Somesh Jha, and Ananthram Swami. Distillation as defense adversarial perturbations against deep neural networks. In 2016 IEEE Symposium on Security and Privacy (SP), pp. 582-597. IEEE, 2016b.
Nicolas Papernot, Patrick McDaniel, Ian Goodfellow, Somesh Jha, Z. Berkay Celik, and Ananthram Swami. Practical black-box attacks against machine learning. In Proceedings of the 2017 ACM on Asia Conference on Computer and Communications Security, ASIA CCS '17, pp. 506-519, New York, NY, USA. 2017. Association for Computing Machinery. ISBN 978 450344444 doi: 10.1145/3052973.305328
URL
3053009
Aditi Raghunathan, Sang Michael Xie, Fanny Yang. John C Duchi, and Percy Liang. Adversarial training can hurt generalization. arXiv preprint arXiv: 06032, 2019.
Ludwig Schmidt, Shibani Santurkar, Dimitris Tsipras, Kunal Talwar, and Aleksander Madry. Ad- versarially robust generalization requires more data. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, NIPS' 18, pp. 5019-5031, Red Hook, NY, USA, 2018. Curran Associates Inc.
Dong Su, Huan Zhang. Hongge Chen, Jinfeng Yi, Pin- Yu Chen, and Yupeng Gao. Is robustness the cost of accuracy?-a comprehensive study on the robustness of 18 deep image classification models. In Proceedings of ECCV, pp. 631-648, 2018.
Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. Sequence sequence learning with neural networks. In Proceedings of the 27th International Conference on Neural Information Processing Systems- Volume 2, NIPS'14 14, pp. 3104-3112, Cambridge, MA, USA, 2014. MIT Press.
10

---

Under review as a conference paper at ICLR 2021
Christian Szegedy, Wojeiech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, lan Goodfellow, and Rob Fergus. Intriguing properties neural networks. arXiv preprint arXiv:1312.6199 2013.
Florian Tramèr, Alexey Kurakin, Nicolas Papernot, Ian Goodfellow, Dan Boneh, and Patrick Mc- Daniel. Ensemble adversarial training: Attacks and defenses. arXiv preprint vrXiv:1705.072000 2017.
Eric Wallace, Shi Feng, Nikhil Kandpal, Matt Gardner, and Sameer Singh. Universal adversarial triggers for attacking and analyzing nip. EMNLP, pp. 2153-2162, 2019.
Haohan Wang, Songwei Ge, Zachary Lipton, and Eric Xing. Learning robust global representations by penalizing local predictive power. Neural Information Processing Systems, pp- 10506-10518, 2019.
Heur Wang, Ting- Wei Liu, Xiong Luo, and Long Wang. An LSTM approach to short text sentiment classification with word embeddings. In ROCLING, pp. 214-223, 2018.
Cihang Xie, Zhishuai Zhang, Yuyin Zhou, Song Bai, Jianyu Wang, Zhou Ren, and Alan Yuille. Improving transferability of adversarial examples with input diversity, 2018.
Puyudi Yang, Jianbo Chen, Cho-Jui Hsieh, Jane-Ling Wang, and Michael I. Jordan. Greedy attack and gumbel attack: Generating adversarial examples for discrete data, 2018
Jason Yosinski, Jeff Clune, Yoshua Bengio, and Hod Lipson. How transferable are features in deep neural networks? In Proceedings of the 27th International Conference on Neural Information Processing Systems Volume 2, NIPS' 14, pp. 3320-3328, Cambridge, MA, USA, 2014. MIT Press.
Hongyang Zhang. Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael Jordan. Theoretically principled trade-off between robustness and accuracy. In Proceedings of ICML, pp. 1472-7482,20111
Huan Zhang. Hongge Chen, Zhao Song, Duane Boning, Inderjit S. Dhillon, and Cho-Jui Hsieh. The limitations of adversarial training and the blind-spot attack, 2019b.
Wei Emma Zhang, Quan Z Sheng. Ahoud Alhazmi, and Chenliang Li. Adversarial attacks on deep-learning models in natural language processing: A survey, ACM Transactions on Intelligent Systems and Technology (TIST), 11(3):1-11, 2020.
Yuchen Zhang, Tianle Liu, Mingsheng Long, and Michael Jordan. Bridging theory and algorithm for domain adaptation. Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedings of the 36th International Conference Machine Learning, volume 97 of Proceedings of Machine Learning Research, pp. 7404-7413, Long Beach, California, USA, 09-15 Jun 2019c. PMLR. URL http: http::/prcceedings.niit www.ppres///vie zhanglanng9 _htm
Han Zhao, Remi Tachet Des Combes, Kun Zhang, and Geoffrey Gordon. On learning invariant representations for domain adaptation. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pp. 7523-7532, Long Beach, California, USA, 09-15 Jun 2019. 9, PMLR. URL httptttppppp/ooo http:://proceedingss ml www.://wwwwrrr.................. press v37 zhaol 9t.hlll


Reviews:
[{"review_id": "NucAUxOGgRN", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "In this paper, the authors propose a learn2weight framework to defend against similar-domain adversarial attacks. Experimental studies on Amazon dataset are done to verify the proposed learn2 weight. \n\nThe paper is not easy to follow. The presentation and organization should be further improved. Here are the detailed comments:\n\n(1)\tAdversarial attacks are widely used in various application domains, e.g., computer vision [ref1] and reinforcement learning [ref2]. It is necessary to discuss with these related works, and highlight the difference and importance of adversarial attack methods on NLP tasks.\n\n[ref1] Adversarial Examples that Fool both Computer Vision and Time-Limited Humans\n\n[ref2] Minimalistic Attacks: How Little it Takes to Fool Deep Reinforcement Learning Policies\n\n(2)\tThe authors highlight “domain adaptation theory” several times. Please give a clear description on what it is. \n\n(3)\tWhere is Table 1 used in the main content? \n\n(4)\tRegarding definition 2, the following two points are unclear: (1) is f_S(A) the true label of A. Based on the figure 1 (a), only correctly classified source samples are used while the definition does not show this. (2) why f(A,W_T) = f_S(A)? f is the target classifier, are you generating the domain-invariant samples? \n\n(5)\tThe rationale of similar domain adversarial attack is confused. It is more reasonable to use source data to help generate target adversarial samples X’ which confuse the classifier to deviate the label f(X) \\neq f(X’) where X is the original target sample. However, the paper generates source adversarial samples, which naturally may confuse the target classifier due to the domain divergence. It is unclear why and how these source adversarial samples can contribute to the robustness of the target classifier. \n\n(6)\tRegarding the accuracy drops in Table 2, it is highly possible caused by the data shift between different domains. How to differentiate the importance of the data shift and adversarial in the accuracy drops? \n\n(7)\tThe technical part is not easy to follow. The sections 5.1 to 5.3 are not linked well. It is necessary to give more contents on the motivation and flow of these algorithms instead of just putting them in algorithm charts. \n\n(8)\tWhy target data is used in Algorithm 2 and also transfer loss optimization? In the introduction, target domain information is assumed to be unavailable. Moreover, algorithm 2 is to reduce the domain divergence (if I understand correctly). I am quite curious how the proposed method differentiates from other transfer learning methods. \n\nUpdate: Thanks for the authors' response. After reading the response and the other reviewers' comments, I think the paper needs to be further improved, and thus I will keep my score.\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "HLdyLLMmK2o", "reviewer": "AnonReviewer5", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary:\n\nThe paper considers the adversarial attacks via a surrogate model constructed using data from a different domain. The authors propose a defense from such attacks by a special kind of adversarial training inspired by the idea of domain adaptation. The idea can be useful but raises a lot of questions, especially when looking at the evaluation of the proposed approach.\n\n########################################################################## \n\nReasons for score: I vote for a reject, as some findings are intriguing, while the experimental results are questionable.\n\nThe first major concern is, why do authors consider NLP models and attacks in the paper? It is much easier to work with Image datasets, and if the general idea is new, I suggest to start from this point to verify that the considered domain adaption works well in this scenario.\n\nAlso, the proposed attack is not new. It is just a surrogate model attack but using a surrogate model training on the data from a different domain (as the authors suggest due to the unavailability of the initial domain data). Also, for this new attack, the authors don't compare a surrogate model attack trained using the same domain data, which would be interesting to compare.\n\nThe authors use only one dataset, which is a bit strange for modern papers. For this dataset, they don't provide a full study, limiting the scope of experiments to particular pairs of source-target domains. From the paper, it is not clear how widely applicable are obtained results.\n\nThe comparison is not full. There are a lot of options to be tuned for alternative approaches like Adversarial training or other defenses. The hyperparameter selection for them has a crucial effect on their success. So, we can't say that the proposed approach works better than others.\n\n######################################################################### \n\nMajor concerns:\n\n* Only one dataset considered. I think that the inclusion of additional datasets (at least three) would improve the paper and make the conclusion by the authors more solid\n* Usage of surrogate models trained on other dataset is not new for general adversarial attacks [1 (mentioned in the paper), 2] and for adversarial attacks in NLP [3]\n* LSTM is not the state of the art model for the processing of NLP data\n* 4.2. what attack do you use? not explicitly specified. so the results can't be verified by replication of the described experiments\n* Table 2 will benefit from adding after-attack accuracy for the original domain. If it is similar to the presented accuracies, then why bother with a new method? \n* Table 3 comparison is not fair, as we have no details about training for each approach, e.g. we don't know how many additional examples we add during adversarial training. Also note, that the state-of-the-art for adversarial training is different from described in the paper. See [4, 5] \n* Table 4 After-Defense Accuracy for what model is presented? because it should be different for LSTM/GRU/CNN attack model\n* Tables 2,3,4 - I suggest to keep the list of pairs (target domain, substitute domain) similar for all tables to be sure that the presented examples are not cherry-picked (also, please consider running your approach on all pairs (target domain, substitute domain) and aggregating all these results)\n* Domain adaptation models, from my experience, are not easy to train. It is interesting to access the quality of the models for different runs of Learn2Weight (is it stable? etc.)\n\n\n1. Nicolas Papernot, Patrick McDaniel, and Ian Goodfellow. Transferability in machine learning: from\nphenomena to black-box attacks using adversarial samples. arXiv preprint arXiv:1605.07277,\n2016a.\n2. Cheng, S., Dong, Y., Pang, T., Su, H., & Zhu, J. (2019). Improving black-box adversarial attacks with a transfer-based prior. In Advances in Neural Information Processing Systems (pp. 10934-10944).\n3. Fursov, I., Zaytsev, A., Kluchnikov, N., Kravchenko, A., & Burnaev, E. (2020). Differentiable Language Model Adversarial Attacks on Categorical Sequence Classifiers. arXiv preprint arXiv:2006.11078.\n4. Shafahi, A., Najibi, M., Ghiasi, M. A., Xu, Z., Dickerson, J., Studer, C., ... & Goldstein, T. (2019). Adversarial training for free!. In Advances in Neural Information Processing Systems (pp. 3358-3369).\n5. Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu.\nTowards deep learning models resistant to adversarial attacks. ICLR, 2017.\n\n######################################################################### \n\nProposed minor improvements:\n\nTable 1: demonstrates one example that breaks the semantics of the attacked sentence. Can you provide good examples of why your approach work?\nDefinition 1: is not a definition, is X one instance or many instances? in this definition also not specified that X and X' should be similar\nEquation 1: why you avoid standard number of equations \\begin{equation} \\label{eq:sample_equation} sample text \\end{equation}?", "extra_scores": {}, "extra_text": {}}, {"review_id": "-HEWODswilw", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary:\nThis paper is about generating adversarial examples for some target model and protecting from such attacks.  Authors consider a setting when an adversary has access to some \"similar to target \" domain data, and can use this data to generate a surrogate model. Using this surrogate model an adversary can generate adversarial examples, that apparently also fool the target model. Then authors also propose a defense mechanism from this type of attack, Learn2Weight. This is a learnt network that, for a given example, returns perturbation of weights to the target model which will be applied to the target before inference. This model is trained by a defender on synthetic domains generated as perturbations to the target data\n\nOverall, this type of an attack is interesting. The paper is well organized and written, and easy to follow. Enough background is given for a reader to follow without the need to research around or going to appendix. Well done on clarity!\n I do have a problem understanding how effective this attack is (compared to other blackbox attacks) and how the proposed defense compares to standard domain generalization methods like learning domain invariant features. \n\n1) One concern I have is about practicality an availability of such \"similar\" domains. For testing authors used Amazon multi-domain sentitment classification, where domains are easily available. But how would you attack a pre-trained Imagenet for example? \n- What domains are similar?\n- and further more, how much data for these similar domains you need to have to train a good enough surrogate model?\n- Also you don't really have a way to calculate that your data is close to the actual target data.\n2) Definition 2: f(A, W_T) = f_S(A) requires an access to your model f, so I would not call this type of attack \"without access to the target model\"\n3) How does this attack compares to any other black box attack that uses target model? It really should be in Table 2. If other attacks are able to make target model performance worse than this type of attack, it is of less value to defend from a weaker attack\n4) Algo 3 - what are the adversarial perturbations you are talking about? \n5) I am not sure algorithm 2 is the best way of doing it? Why not to try any of domain generalization techniques (e.g. train on all domains with an adversarial head tries to distinguish between domains, or MMD or whatever). May be this way you won't need Learn2Weight model at all (since you are already learning domain invariant features only)\n\nMinor:\n- Table 2: What are u boldening ? I would expect the bolden result to be per source model (book) and the worse performance you get (so dvd attack gives the lowest after attack accuracy). You are boldening \"baby\", which is the weakest domain (on which your attack mode is trained) for an attack. \n- Algo 2 Compute weights of f trained on TY=W_T-W_T (just assign 0s?)\n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The submission considers a new attack model for adversarial perturbation in a framework where the attacker has neither access to the trained model nor the data used for training the model. The submission suggests a"domain adaptation inspired attack": learn a different model on a similar domain and generate the adversarial perturbations using that model. The authors then also develop a defense for this type of attack and provide some empirical evaluations of the resulting losses on a few NLP benchmark datasets.

The paper refers to the literature on domain adaptation theory to motivate their suggested defense, but this analysis remains on an intuitive (rather than a formally rigorous) level. Furthermore, the empirical evaluation does not compare to a variety of attacks and the defense is only evaluated with respected to the self-suggested attack. This is a very minimal bar for a defense to meet.

The reviewers have criticized the submission for the rather minimal extend of empirical evaluation. Given that the submission also doesn't provide a sound theoretical analysis for the  proposed attack and defense, I agree with the reviewers that the submission does not provide sufficient novel insight for publication at ICLR. 

In contrast to some of the reviewers, I do find it legitimate (and maybe recommendable even) to focus on one chosen application area such as NLP. I don't see a requirement to also present experiments on image data or re-inforcement learning applciations. However, I would recommend that the authors highlight more explicitly what general lessons a reader would learn from their study. This could be done through a more extensive and systematic set of experiments or a through analysis in a well defined theoretical framework.
Decision: Reject