Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 4: Ok but not good enough - rejection
review_content: Summary:

The paper considers the adversarial attacks via a surrogate model constructed using data from a different domain. The authors propose a defense from such attacks by a special kind of adversarial training inspired by the idea of domain adaptation. The idea can be useful but raises a lot of questions, especially when looking at the evaluation of the proposed approach.

########################################################################## 

Reasons for score: I vote for a reject, as some findings are intriguing, while the experimental results are questionable.

The first major concern is, why do authors consider NLP models and attacks in the paper? It is much easier to work with Image datasets, and if the general idea is new, I suggest to start from this point to verify that the considered domain adaption works well in this scenario.

Also, the proposed attack is not new. It is just a surrogate model attack but using a surrogate model training on the data from a different domain (as the authors suggest due to the unavailability of the initial domain data). Also, for this new attack, the authors don't compare a surrogate model attack trained using the same domain data, which would be interesting to compare.

The authors use only one dataset, which is a bit strange for modern papers. For this dataset, they don't provide a full study, limiting the scope of experiments to particular pairs of source-target domains. From the paper, it is not clear how widely applicable are obtained results.

The comparison is not full. There are a lot of options to be tuned for alternative approaches like Adversarial training or other defenses. The hyperparameter selection for them has a crucial effect on their success. So, we can't say that the proposed approach works better than others.

######################################################################### 

Major concerns:

* Only one dataset considered. I think that the inclusion of additional datasets (at least three) would improve the paper and make the conclusion by the authors more solid
* Usage of surrogate models trained on other dataset is not new for general adversarial attacks [1 (mentioned in the paper), 2] and for adversarial attacks in NLP [3]
* LSTM is not the state of the art model for the processing of NLP data
* 4.2. what attack do you use? not explicitly specified. so the results can't be verified by replication of the described experiments
* Table 2 will benefit from adding after-attack accuracy for the original domain. If it is similar to the presented accuracies, then why bother with a new method? 
* Table 3 comparison is not fair, as we have no details about training for each approach, e.g. we don't know how many additional examples we add during adversarial training. Also note, that the state-of-the-art for adversarial training is different from described in the paper. See [4, 5] 
* Table 4 After-Defense Accuracy for what model is presented? because it should be different for LSTM/GRU/CNN attack model
* Tables 2,3,4 - I suggest to keep the list of pairs (target domain, substitute domain) similar for all tables to be sure that the presented examples are not cherry-picked (also, please consider running your approach on all pairs (target domain, substitute domain) and aggregating all these results)
* Domain adaptation models, from my experience, are not easy to train. It is interesting to access the quality of the models for different runs of Learn2Weight (is it stable? etc.)


1. Nicolas Papernot, Patrick McDaniel, and Ian Goodfellow. Transferability in machine learning: from
phenomena to black-box attacks using adversarial samples. arXiv preprint arXiv:1605.07277,
2016a.
2. Cheng, S., Dong, Y., Pang, T., Su, H., & Zhu, J. (2019). Improving black-box adversarial attacks with a transfer-based prior. In Advances in Neural Information Processing Systems (pp. 10934-10944).
3. Fursov, I., Zaytsev, A., Kluchnikov, N., Kravchenko, A., & Burnaev, E. (2020). Differentiable Language Model Adversarial Attacks on Categorical Sequence Classifiers. arXiv preprint arXiv:2006.11078.
4. Shafahi, A., Najibi, M., Ghiasi, M. A., Xu, Z., Dickerson, J., Studer, C., ... & Goldstein, T. (2019). Adversarial training for free!. In Advances in Neural Information Processing Systems (pp. 3358-3369).
5. Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu.
Towards deep learning models resistant to adversarial attacks. ICLR, 2017.

######################################################################### 

Proposed minor improvements:

Table 1: demonstrates one example that breaks the semantics of the attacked sentence. Can you provide good examples of why your approach work?
Definition 1: is not a definition, is X one instance or many instances? in this definition also not specified that X and X' should be similar
Equation 1: why you avoid standard number of equations \begin{equation} \label{eq:sample_equation} sample text \end{equation}?
review_title: An idea is presented, but the meaningful evaluation is missing

final_score: rating: 5: Marginally below acceptance threshold
review_content: Summary:
This paper is about generating adversarial examples for some target model and protecting from such attacks.  Authors consider a setting when an adversary has access to some "similar to target " domain data, and can use this data to generate a surrogate model. Using this surrogate model an adversary can generate adversarial examples, that apparently also fool the target model. Then authors also propose a defense mechanism from this type of attack, Learn2Weight. This is a learnt network that, for a given example, returns perturbation of weights to the target model which will be applied to the target before inference. This model is trained by a defender on synthetic domains generated as perturbations to the target data

Overall, this type of an attack is interesting. The paper is well organized and written, and easy to follow. Enough background is given for a reader to follow without the need to research around or going to appendix. Well done on clarity!
 I do have a problem understanding how effective this attack is (compared to other blackbox attacks) and how the proposed defense compares to standard domain generalization methods like learning domain invariant features. 

1) One concern I have is about practicality an availability of such "similar" domains. For testing authors used Amazon multi-domain sentitment classification, where domains are easily available. But how would you attack a pre-trained Imagenet for example? 
- What domains are similar?
- and further more, how much data for these similar domains you need to have to train a good enough surrogate model?
- Also you don't really have a way to calculate that your data is close to the actual target data.
2) Definition 2: f(A, W_T) = f_S(A) requires an access to your model f, so I would not call this type of attack "without access to the target model"
3) How does this attack compares to any other black box attack that uses target model? It really should be in Table 2. If other attacks are able to make target model performance worse than this type of attack, it is of less value to defend from a weaker attack
4) Algo 3 - what are the adversarial perturbations you are talking about? 
5) I am not sure algorithm 2 is the best way of doing it? Why not to try any of domain generalization techniques (e.g. train on all domains with an adversarial head tries to distinguish between domains, or MMD or whatever). May be this way you won't need Learn2Weight model at all (since you are already learning domain invariant features only)

Minor:
- Table 2: What are u boldening ? I would expect the bolden result to be per source model (book) and the worse performance you get (so dvd attack gives the lowest after attack accuracy). You are boldening "baby", which is the weakest domain (on which your attack mode is trained) for an attack. 
- Algo 2 Compute weights of f trained on TY=W_T-W_T (just assign 0s?)

review_title: Learn2Weight and similar domains attacks

final_score: rating: 3: Clear rejection
review_content: In this paper, the authors propose a learn2weight framework to defend against similar-domain adversarial attacks. Experimental studies on Amazon dataset are done to verify the proposed learn2 weight. 

The paper is not easy to follow. The presentation and organization should be further improved. Here are the detailed comments:

(1)	Adversarial attacks are widely used in various application domains, e.g., computer vision [ref1] and reinforcement learning [ref2]. It is necessary to discuss with these related works, and highlight the difference and importance of adversarial attack methods on NLP tasks.

[ref1] Adversarial Examples that Fool both Computer Vision and Time-Limited Humans

[ref2] Minimalistic Attacks: How Little it Takes to Fool Deep Reinforcement Learning Policies

(2)	The authors highlight “domain adaptation theory” several times. Please give a clear description on what it is. 

(3)	Where is Table 1 used in the main content? 

(4)	Regarding definition 2, the following two points are unclear: (1) is f_S(A) the true label of A. Based on the figure 1 (a), only correctly classified source samples are used while the definition does not show this. (2) why f(A,W_T) = f_S(A)? f is the target classifier, are you generating the domain-invariant samples? 

(5)	The rationale of similar domain adversarial attack is confused. It is more reasonable to use source data to help generate target adversarial samples X’ which confuse the classifier to deviate the label f(X) \neq f(X’) where X is the original target sample. However, the paper generates source adversarial samples, which naturally may confuse the target classifier due to the domain divergence. It is unclear why and how these source adversarial samples can contribute to the robustness of the target classifier. 

(6)	Regarding the accuracy drops in Table 2, it is highly possible caused by the data shift between different domains. How to differentiate the importance of the data shift and adversarial in the accuracy drops? 

(7)	The technical part is not easy to follow. The sections 5.1 to 5.3 are not linked well. It is necessary to give more contents on the motivation and flow of these algorithms instead of just putting them in algorithm charts. 

(8)	Why target data is used in Algorithm 2 and also transfer loss optimization? In the introduction, target domain information is assumed to be unavailable. Moreover, algorithm 2 is to reduce the domain divergence (if I understand correctly). I am quite curious how the proposed method differentiates from other transfer learning methods. 

Update: Thanks for the authors' response. After reading the response and the other reviewers' comments, I think the paper needs to be further improved, and thus I will keep my score.

review_title: Official Blind Review #1

Meta-review: The submission considers a new attack model for adversarial perturbation in a framework where the attacker has neither access to the trained model nor the data used for training the model. The submission suggests a"domain adaptation inspired attack": learn a different model on a similar domain and generate the adversarial perturbations using that model. The authors then also develop a defense for this type of attack and provide some empirical evaluations of the resulting losses on a few NLP benchmark datasets.

The paper refers to the literature on domain adaptation theory to motivate their suggested defense, but this analysis remains on an intuitive (rather than a formally rigorous) level. Furthermore, the empirical evaluation does not compare to a variety of attacks and the defense is only evaluated with respected to the self-suggested attack. This is a very minimal bar for a defense to meet.

The reviewers have criticized the submission for the rather minimal extend of empirical evaluation. Given that the submission also doesn't provide a sound theoretical analysis for the  proposed attack and defense, I agree with the reviewers that the submission does not provide sufficient novel insight for publication at ICLR. 

In contrast to some of the reviewers, I do find it legitimate (and maybe recommendable even) to focus on one chosen application area such as NLP. I don't see a requirement to also present experiments on image data or re-inforcement learning applciations. However, I would recommend that the authors highlight more explicitly what general lessons a reader would learn from their study. This could be done through a more extensive and systematic set of experiments or a through analysis in a well defined theoretical framework.
Decision: Reject