Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 5: Marginally below acceptance threshold
review_content: ##########################################################################

Summary:

This paper study the fundamental relationship between adversarial transferability and knowledge transferability. Theoretical analysis is conducted, revealing that adversarial transferability can indicate knowledge transferability. In this procedure, two quantities are formally defined to measure adversarial transferability from different aspects. Furthermore, empirical evaluation in three different transfer learning scenarios on diverse datasets are carried out, showing a strong positive correlation between the adversarial transferability and knowledge transferability.


##########################################################################

Reasons for score:

Strengths:

- This work study the relationship between adversarial transferability and knowledge transferability, which is still under explored.

- Both adversarial transferability and knowledge transferability are defined quantitatively, which enables in-depth understanding of them.

Weaknesses:

- The novelty and contribution of this work are marginal. Although the problem is fundamental, this paper does not seem to be an adequate exploration of the relationship between adversarial transferability and knowledge transferability. The proof of a conclusion, which seems intuitive, may only bring us limited inspiration.

- The three experiments repeatedly prove the positive correlation between the adversarial transferability and knowledge transferability in three knowledge transfer scenarios, which seems to be repetitive. More ingenious experiments can be designed and conducted.

- The organization of this paper is somewhat confusing，e.g. Section 4.

##########################################################################

review_title: The discovery of this paper may not inspire the community enough as claimed.

final_score: rating: 5: Marginally below acceptance threshold
review_content: Summary and contributions
This paper studies the relationship between adversarial transferability and knowledge transferability. By defining two quantities to measure the adversarial transferability, it shows that adversarial transferability measured in this way indicates knowledge transferability both theoretically and empirically.

Strengths
This paper is the first work to theoretically focus on the correlation of two prevalent phenomena of DNN--adversarial transferability and knowledge transferability. Its backgrounds and theoretical results and proofs are clearly presented. Moreover, the experiments are complete: they include experiments for three types of knowledge transferability whose datasets, transfering methods and results are very clear. 

Weaknesses
First, this paper does not provide the intuition to use squared cosine value rather than the cosine value in the defination of $\tau_1$(although Proposition 2 theoretically shows the relationship between $\tau_1$ and Cross Adversarial Loss). Fig.1 in the paper also regards $\tau_1$ as the cosine value. For an example of the ill-defined parts of $\tau_1$, given $\bm \delta_{f_1}=\bm \delta_{f_2}=-\bm \delta_{f_3}$, one would claim that the adversarial transferabilities of $f_1$ to $f_2$ and $f_1$ to $f_3$ are same considering their same squared cosine values, which is counterintuitive to make $\tau_1$ an appropriate quantity to measure the similarity between two attacks. 

Second, in the definition of $\tau_2$, the authors provide the linear map $A$ and $\tau_2$ without literally stating their physical meanings. Besides, the demonstration of $\tau_2$ in Fig.1 is misleading since it seems that $\tau_2$ is a vector in Fig.1 which is not true. The authors also do not clarify the reasonability of comparing vectors of different dimensions which is necessary for most settings of this paper.

Third, this paper does not provide an indicator with both $\tau_1$ and $\tau_2$ for high adversarial transferability. Futhermore, it lacks the theoretical relation between $\tau_2$ and Cross Adversarial Loss to demonstrate its reasonability. 

If these concerns could be solved appropriately, I would consider to raise the score. 

Correctness
I have carefully checked the proofs of all theorems in this paper, and ensure that they are correct.

Reproducibility
Yes. The setting of their experiments is clear and complete. 

Clarity
The idea, structure and expression of this paper are easy to understand and follow. However, the explanation of Definition 3 is hard to understand. 

Relation to the prior work
Yes. Authors show they have a good understanding of prior work's contributions, especially the three types of knowledge transferability. 

Addition
Pro. Authors use PGD-attack adversarial transferring in Section 5.1 but virtual adversarial transferring in Section 5.2. Is there any difference between them, and if so, what's the difference?  
Pro. Authors claim that $g$ is a trainable function in Section 3 while use a specified $g$ instead in Theorem 3. Does a better $g$, such as $g = \arg \min \left\|g \circ f_{S}-y\right\|_{\mathcal{D}}^{2}$, make the bound in Theorem 3 tighter? 
review_title: Some concerns need to be solved

final_score: rating: 5: Marginally below acceptance threshold
review_content: This paper studies the relationship between adversarial and knowledge transferability. It develops two metrics to measure adversarial transferability and empirically shows that adversarial transferability correlates with knowledge transferability. 

Pros: 
The paper studies an interesting problem. Intuitively, both adversarial and knowledge transferability show how much the decision boundaries of the source and target classifiers are aligned. This paper is interesting in that it tries to quantify their correlation.  

Cons:
The paper provides an interesting observation but fails to further investigate how this observation can be used to gain better understanding of either fields of adversarial examples or knowledge transfer or how the insights can lead to better techniques. In the following, I provide a few comments and questions.  

On metrics tau1 and tau2:
-	Why do we need \tau1 and \tau2 as proxies for adversarial transferability? It seems from experimental results that the simple metric of adversarial loss can indeed be a simpler and more robust indicator of the knowledge transferability.  
-	\tau1 measures the correlation of the perturbations for the two models. The attack method used finds the perturbation that causes the largest change in output, which is the formulation for the misclassification attack. Different models may classify the perturbed example into different classes and so the output changes would be different. The paper says \tau1 is not enough and we need to look into the changes made to the output as well and whether an affine layer can map one into another. I feel if paper considered targeted attack, the attack could generate highly correlated output changes and so \tau2 wouldn’t be necessary. I wondered if authors tried targeted attack formulation and how the new tau1 would correlate with adversarial transferability. 
-	I’m a bit confused about Theorem 1 and how it relates to definition of \tau2. As far as I understand, \tau2 captures how much outputs of two models can be aligned using an affine transformation. Theorem 1 then says large \tau2 means an affine function g can be found that aligns the two models. Isn’t it just the way that \tau2 is defined?

Analysis and results:
-	Some parts of the paper seem a bit disconnected or not elaborated well enough. For example, what do theorems 2 and 3 tell us? Similarly for proposition 2. What are the takeaways?
-	Do experiments confirm the theoretical bounds and derivations? It might be good to define a simple synthetic task (maybe on a logistic regression or 1-layer neural net) to evaluate the claims and show the relationship between adversarial and knowledge transferability. 
-	The results need more investigation. For example, in figure 2 (right), \tau1 values are almost identical, which as paper mentions, shows \tau1 is not sufficient for determining transferability. However, although \tau2 values are correlated with transferability, they are almost zero. What does this imply? Do these results statistically support the claims? Similarly, in table 1, the relationship between \tau1 and \tau2 and adversarial-knowledge transfer does not seem to be statistically significant. 
-	Some statements in observations seem obvious. For example, section 5.1 reads “we show that the closer the source data distribution is to the target data distribution, the more adversarially transferable the source model to the reference model, thus we observe that the source model is more knowledge transferable to the target dataset.” This is a known concept that has been used in both designing better attacks and also better transfer learning methods. 


Paper edit suggestions:
- Writing and paper structure can be polished. I suggest moving background info and definitions into a section background and then laying out the structure for the rest of the paper. 
- The discussion of related work and the cited papers need a revision. Some papers are repeatedly cited while others seem to be discussed out of place. For example, page 8 refers to (Zamir et al., 2018) a total of nine times. 

review_title: Shows adversarial transferability is correlated with knowledge transferability but no clear takeaway

final_score: rating: 5: Marginally below acceptance threshold
review_content: ##########################################################################
Summary:
 
This paper demonstrates the relationship between adversarial transferability and knowledge transferability. The authors first formulate these two phenomena. Then under mild assumptions, they prove adversarial transferability indicates knowledge transferability. The authors further justify their theory with experiments.

##########################################################################
Reasons for score: 

Overall, I believe this paper studies an interesting problem. The relationship between adversarial transferability and knowledge transferability is of interest to both transfer learning and adversarial attack community. However, the paper does not provide how to further interpret their findings our apply them in practice. 

##########################################################################
Pros

The theoretical analysis seems sound and clearly organized. The authors provide a comprehensive analysis of their hypothesis both theoretically and empirically.

##########################################################################
Cons

1.	Although the problem studied is interesting, the results are not surprising. Adversarial transferability indicates the gradients of the two models are similar, which further indicates similar parameters and predictions. Models with similar parameters and predictions will naturally share knowledge. Even without reading the paper, one can expect the argument above will hold. 
2.	The authors did not elaborate on how to use their results in applications. For example, since adversarial transferability and knowledge transferability are related, will adversarial training help transfer learning? On the other hand, will transfer learning improve adversarial robustness? 


review_title: Interesting problem, solid analysis, but lack of interpretations and applications

Meta-review: This paper studies the relationship between adversarial transferability and knowledge transferability. It develops two metrics to measure adversarial transferability and a theoretical framework to justify the positive correlation between adversarial transferability and knowledge transferability. Synthetic experiments show that adversarial transferability measured by the proposed metrics indicates knowledge transferability.

While the paper studies an interesting and fundamental problem, with a sound theoretical analysis and a clear presentation, reviewers still have several reservations to directly accept it.
- Lack of interpretation. How this observation can be used to gain better understanding of either fields of adversarial examples or knowledge transfer?
- Lack of inspiration. How the insights can lead to better transfer techniques, apply to practical applications, and foster future research?
- Lack of justification. Why such definitions of metrics are the intrinsic ways of measuring adversarial transferability? How well do they correlate with the practical experience with advanced attack, defense, and transfer methods?

AC believes the endeavor made by this paper towards a fundamental problem is highly necessary to our field. But given the above reservations, AC would encourage the authors to further strengthen their work to make it more inspiring and useful.
Decision: Reject