Paper: Learning Transferable Robust Representations for Few-shot Learning via Multi-view Consistency (ICLR-2024)

Reviewer comment (Evaluation):
Even compared with existing models under traditional setting (training on base dataset), the performance of the proposed method is far lag behind them in clean setting. For example, for 5-shot tasks, FeLMi [c] achieves 89.47 on CIFAR-FS, 86.08 on miniImageNet and 77.61 on miniImageNet-->CUB, while this work only obtains 67.75, 47.56 and 53.70 respectively. Although the performance is improved in robust setting, the cost in clean setting is too expensive, which makes it meaningless.

Author response:
**Weakness 2**: The clean performance is poorer than existing clean meta-learning which costs more.
* While our clean performance is slightly lower than the clean meta-learning approaches (around -10%), the expense is justified by significant **gains in robustness (around +40%)**. Furthermore, existing clean meta-learning models could not withstand adversarial attacks (shows 0% performance). This demonstrates the importance and critical need for adversarial training in diverse tasks and domains.
* We appreciate your concerns regarding the trade-off between clean performance and robustness against adversarial attacks. However, as shown in Table 1 (also summarized in the following), our approach achieves the **highest robustness while having a minimal compromised clean performance** compared to previous adversarial meta-learning. This demonstrates that the enhanced model’s meta-generalizability stems from our novel multi-view learning framework as the Reviewer rEWc acknowledges.

|Type|Method|CIFAR-FS|(Unseen)|Mini-ImageNet|(Unseen)|
|-|:-|-|-|-|-|
| | |Avg. clean|Avg. rob.|Avg. clean|Avg. rob.|
|CML|MAML|53.83|4.24|59.31|7.23|
|AML|Ours|50.32(**-3.51%**)|28.20(**+23.96%**)|58.37(**-0.94%**)|27.96(**+20.73%**)|
|AML|AQ|41.96(-11.87%)|5.99(+1.75%)|51.18(-8.13%)|18.80(+11.57%)|
|AML|RMAML|32.49(-21.34%)|7.39(+3.15%)|35.20(-24.11%)|8.92(+1.69%)|
|AML|ADML|33.34(-20.49%)|7.10(+2.86%)|36.79(-22.52%)|9.46(+2.23%)|

Reviewer comment (Evaluation):
It is hard to believe that MetaOptNet performs worse than MAML in both clean and robust setting. In my experience, although MAML has a wider range of applications, such as reinforcement learning and so on, MAML is lag behind MetaOptNet in terms of few-shot classification. A direct evidence is the Table 6 and Table 12 in [d], where MetaOptNet achieves far better performance than MAML in both clean and robust setting. I think they are the reasonable results. Even we evaluate in the cross-domain settings, the results shouldn't be so different, since mini/Tiered-ImageNet or CARS are not so different with CIFAR-FS. MetaOptNet still should outperform MAML, as least in clean setting, although the difference could be smaller than in-domain.

Author response:
**Weakness 4**: MetaOptNet could not perform better than the MAML-based approach.
* There is a critical misunderstanding of our work. We propose a multi-view latent attack that enables MAML to outperform a mere combination of PGD with MetaOptNet.
* As indicated in Table 3, when we simply apply PGD attacks to both MAML and MetaOptNet, **MetaOptNet demonstrates superior performance across 5 different unseen domains**, with a margin of 12.2% in clean and 5.51% in robust average accuracy, respectively.
* Nevertheless, our approach surpasses the simple combination of PGD and MetaOptNet, which distinctly evidences the contributions of our method.
* Additionally, the MAML approach can outperform MetaOptNet in certain stable settings, as demonstrated in [1]. This could further support the superior performance of our MAML-based approach over the simple combination of MetaOptNet in adversarial meta-learning tasks.

[1] How to Train Your MAML to Excel in Few-Shot Classification, ICLR'22

Reviewer comment (Reproducibility):
According to the description in this paper, it seems that two encoders are used during training. Therefore, how to use the proposed model during test? It means that one encoder will be discarded during test? This part should be clear.

Author response:
**Weakness 2**: How to use two encoders in test time?
* There seems to be a misunderstanding regarding how our bootstrapped encoders are employed during the training. Our encoders operate as bootstrapped parameters during the inner-adaptation that are shared parameters. (page 5: algorithm last line/ page 5: green text)
* During the meta-optimization phase, we update only a single encoder. Consequently, a single encoder is employed at the test time. (page 6: green text)

Reviewer comment (Reproducibility):
Some details of the proposed framework are not clear. For example, which kind of FSL classifier is used in the proposed framework? Why select this kind of classifier?

Author response:
**Weakness 3**: What is FSL classifier?
* Our FSL classifier is a fully connected layer as follows the setting used in [1].
* To reflect your comments, we added experimental details in the paper (page 7: orange text)

[1] Li et al., Meta-SGD: Learning to Learn Quickly for Few-Shot Learning

Reviewer comment (Experiments):
In the experiments part, many comparison experiments are not fair. For example, why other AML methods are not compared in Table 2? In fact, there is a trade-off problem in adversarial training [1]. It is somewhat difficult to compare two adversarial meta-learning methods with two kinds of accuracies [2].

Author response:
**Weakness 4**: Experiment comparisons are not fair.
* Due to the page constraints, we compared our method only with the most robust adversarial baselines, namely AQ.
* As shown in the following table, other AML methods also demonstrate inferior transferability in non-RGB datasets. Based on your feedback, we also added these results to the main paper (page 7: orange)

|CIFAR-FS->|EuroSAT|-|ISIC|-|CropDisease|-|Avg|-|
|-|-|-|-|-|-|-|-|-|
|-|Clean|Rob.|Clean|Rob.|Clean|Rob.|Clean|Rob.|
|ADML|60.41|7.79|32.20|0.51|50.15|1.64|47.59 |3.31 |
|RMAML|52.21|6.56|29.23|2.46|55.08|5.43|45.51|4.82|
|AQ|46.05|4.62|31.90|0.62|47.38|0.51|41.78|1.92|
|Ours|59.39|19.90|30.77|5.23|57.85|27.28|**49.34**|**17.47**|

* We acknowledge the trade-off issue inherent in adversarial training. Nonetheless, as shown in Table 1 our method achieves superior robustness and the highest clean accuracy compared to other previous adversarial meta-learning approaches. Moreover, compared to the clean meta-learning method (MAML), our approach has an average of 3.51% and 0.94% less clean performance while achieving 23.96% and 20.73% robustness gains for CIFAR-FS and Mini-ImageNet, respectively. This suggests that our approach makes the least compromise on the clean data performance to attain adversarial robustness.

|Type|Method|CIFAR-FS|(Unseen)|Mini-ImageNet|(Unseen)|
|-|-|-|-|-|-|
| | |Clean Avg.|Rob. Avg.|Clean Avg.|Rob. Avg.|
|CML|MAML|53.83|4.24|59.31|7.23|
|AML|Ours|50.32(**-3.51%**)|28.20(**+23.96%**)|58.37(**-0.94%**)|27.96(**+20.73%**)|
|AML|AQ|41.96(-11.87%)|5.99(+1.75%)|51.18(-8.13%)|18.80(+11.57%)|
|AML|RMAML|32.49(-21.34%)|7.39(+3.15%)|35.20(-24.11%)|8.92(+1.69%)|
|AML|ADML|33.34(-20.49%)|7.10(+2.86%)|36.79(-22.52%)|9.46(+2.23%)|

We are glad our previous responses have resolved your original concerns regarding Weakness 2, 3, and 6. Thank you for your comments, and we also provide additional responses for the remaining concerns. 

***

**Weakness 4:** I cannot agree with the reason of the authors. This part should be supplemented with more results.
* What do you mean you do not agree with our results? We conduct cross domain experiment on non-RGB domain with baselines as you requested. 
* Could you specifically explain which additional results are needed to resolve your concern? We are willing to address your concerns thoroughly.

Reviewer comment (Novelty):
Some latest adversarial meta-learning methods are not reviewed, such as [2]. In addition, the work in [2] had already considered the unseen domain adversarial robustness transferability issue. Importantly, the work of [2] can achieve much better results than this work.

Author response:
**Weakness 5**: Recent work seems better, and needs comparison.
* Thank you for suggesting relevant baselines for our work. We will also add [1] to the main paper.
* However, it is important to note that the evaluation setting in [1] differs significantly from ours. Primarily, [1] evaluated the performance against FGSM attackers with a smaller size of epsilon of 0.01 which is weaker than our PGD-20 attacks with epsilon of 0.03.
* When we set the evaluating setting as [1] in CIFAR-FS, as shown in the following table, our approach achieves better performance than [1]. 
* Since [1] did not provide any codes for the model or dataloader, we will reproduce those results in our setting to add to Table 1. 
* Our approach achieves 15% higher clean accuracy and 4% better robustness than [1] in CIFAR-FS.

|Method|Clean|Robustness|
|-|-|-|
|DFSL [1]|55.51|48.88|
|Ours |**70.50**|**52.39**|


We are glad our previous responses have resolved your original concerns regarding Weakness 2, 3, and 6. Thank you for your comments, and we also provide additional responses for the remaining concerns. 

***

**Weakness 5:** I have just checked the paper of [1] and found that it indeed provides the code link in its abstract (https://github.com/WenbinLee/DefensiveFSL). Also, the code exists in the GitHub. In addition, note that the work of [1] has also provided the experiments of using PGD-10.
* As you can see in the code, the model and data loader codes are missing. But we found the model, and loader code from the authors’ other GitHub page.
* We train the model with official code and evaluate on the same setting with image size 32 and PGD-20 attackers (PGD-20, epsilon 8/255, alpha 8/2550).
* When we evaluate [1] against stronger attacks, [1] could not achieve better robustness than ours as shown in the following Table. 

|Method|Mini-ImageNet|(Seen)|CIFAR|(Unseen)|Cars|(Unseen)|CUB|(Unseen)|
|-|-|-|-|-|-|-|-|-|
|DFSL [1]|**53.00**|17.63|45.49|12.05|36.26|8.08|53.49|18.91|
|Ours|47.56|**18.18**|**65.45**|**36.51**|**42.25**|**14.42**|**53.70**|**20.64**|

Reviewer comment (Experiments):
Only one kind of adversarial attack is considered in this paper. More kinds of adversarial attacks are recommended to be added into this work [2], making the proposed method more convincing.

Author response:
**Weakness 6**: Evaluation against diverse attacks is recommended.**
* Thank you for the suggestions. We evaluated our models against SPSA [2], CW [3], and FGSM [4] attacks as shown in the following Table.  We will add these results to our manuscript.
* Our approach could consistently demonstrate robust results against diverse attacks compared to the best-performing AML baseline (AQ).

|Robustness|PGD-20| |FGSM| |CW| |SPSA| |
|:-|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|
| |AQ|Ours|AQ|Ours|AQ|Ours|AQ|Ours|AQ|Ours|
|Seen|42.82|**43.42**|67.86|**66.53**|48.31|**54.07**|67.06|**67.29**|
|Avg Unseen|5.99|**28.20**|40.91|**47.05**|28.92|**49.20**|41.08|**46.18**|

Reviewer comment (Experiments):
In the supplementary material, it seems that only 200 and 400 tasks are used to perform the meta-training and meta-test, respectively. I doubt the effectiveness of the experimental results, because the variance and randomness of the results will be very large, especially for adversarial training.

Author response:
**Weakness 7**: Variance for the experiments
* Thank you for the suggestion, we will add variances in our results.
* The variances of converged adversarial trained networks are not very large as shown in the following Table.

||CIFAR-FS||Mini-ImageNet||Tiered-ImageNet||CUB||Flower||Cars||
|-|-|-|-|-|-|-|-|-|-|-|-|-|
||Clean|Robustness|Clean|Robustness|Clean|Robustness|Clean|Robustness|Clean|Robustness|Clean|Robustness|
|Ours|64.69 $\pm$ 0.44|43.03 $\pm$0.57|41.26$\pm$0.33|20.08$\pm$0.11|44.25$\pm$0.29|25.72$\pm$0.20|48.88$\pm$0.12|24.82$\pm$0.04|66.74$\pm$0.61|41.87$\pm$0.51|39.26$\pm$0.64|19.75$\pm$0.25|


We are glad our previous responses have resolved your original concerns regarding Weakness 2, 3, and 6. Thank you for your comments, and we also provide additional responses for the remaining concerns. 

***

**Weakness 7:** This part is very important. Unfortunately, the current results cannot convince me.
* Could you explain more why our multiple seed results table is not convincing? We would be more than glad to address your concerns.

Reviewer comment (Novelty):
In my view, the multi-view latent attack enhances the model’s meta generalizability to some extent, therefore, it outperforms other Adversarial Meta-Learning methods in the cross-domain setting. And the multi-view training is quite similar to the task augmentation meta-learning methods [1,2,3]. Thus, I believe that a discussion or comparison in the related work section would enhance understanding of the multi-view’s effect.

Author response:
**Weakness 1**: A discussion or comparison with existing task-augmentation meta-learning methods is needed in a related section.
* Thank you for your suggestion. We have revised the related section to include suggested papers as shown in the paper. (page 3, Section 2: orange)
* For convenience, we briefly summarize the discussion in the following response.
* To achieve better generalization in tasks, previous works have proposed augmenting the tasks using rotation [1], task interpolation [2], and adversarial task up-sampling [3]. In a similar sense, our approach also implicitly brings the effect of task augmentation, enlarging the task distribution by introducing bootstrapped encoders, which naturally augment the tasks with the parameters of the encoders and could lead to generalized performance in unseen tasks.

Reviewer comment (Experiments):
During meta-testing, the evaluation was conducted on only 400 randomly selected tasks, which significantly impacts the final accuracy. What would the final performance be if the number of evaluated tasks during meta-testing were increased?

Author response:
**Weakness 2**: A larger number of evaluated tasks during the meta-testing.
* Thank you for your suggestion. We tested our framework under a larger number of evaluation tasks with 1000 tasks, and 5000 tasks. 
* The table below demonstrates that our framework **consistently delivers outperforming robust results** with an increasing number of evaluation tasks compared to the best AML baseline (AQ), highlighting our approach **generalizability to various types of unseen tasks**.


|Method|# Evaluation tasks|Mini-ImageNet||Tiered-ImageNet||CUB||Flower||Cars||CIFAR-FS|(Seen)|
|--|--|--|--|--|--|--|--|--|--|--|--|--|--|
|||Clean|Robustness|Clean|Robustness|Clean|Robustness|Clean|Robustness|Clean|Robustness|Clean|Robustness|
|Ours|400|**45.82**|**24.12**|**51.46**|**30.06**|**48.56**|**25.23**|**66.49**|**42.16**|**38.29**|**19.43**|67.75|**43.42**|
|AQ|400|33.09|3.32|37.41|5.05|38.37|4.10|60.14|11.03|36.83|4.47|**73.19**|42.82|
|Ours|1000|**44.15**|**20.65**|**51.55**|**23.51**|**50.01**|**25.31**|**68.83**|**41.36**|**39.28**|**19.71**|**67.44**|**44.01**|
|AQ|1000|33.24|3.20|36.65|4.92|38.23|4.02|60.09|11.03|37.52|4.64|65.75|23.52|
|Ours|5000|**43.85**|**25.98**|**45.87**|**23.73**|**50.08**|**25.51**|**69.03**|**41.45**|**39.17**|**19.52**|**67.15**|**43.88**|
|AQ|5000|33.12|3.20|36.91|5.09|38.47|4.02|60.27|11.02|37.14|4.36|65.83|23.52|

Reviewer comment (Experiments):
How does the model perform under different attacks, apart from the PGD-20 attack?

Author response:
**Weakness 3**: Results against different attacks, apart from PGD-20.
* Thank you for the suggestions. We will add these results to our manuscript. 
* We additionally evaluated our models under  CW [1], FGSM [2], and SPSA [3]  as shown in the following Table. Our approach **consistently demonstrates outperforming robustness** against diverse attacks compared to the best-performing AML baseline (AQ).

|Robustness|PGD-20| |FGSM| |CW| |SPSA| |
|--|--|--|--|--|--|--|--|--|
| |AQ|Ours|AQ|Ours|AQ|Ours|AQ|Ours|AQ|Ours|
|Seen|42.82|**43.42**|67.86|**66.53**|48.31|**54.07**|67.06|**67.29**|
|Avg Unseen|5.99|**28.20**|40.91|**47.05**|28.92|**49.20**|41.08|**46.18**|

[1] Towards Evaluating the Robustness of Neural Networks \
[2] Explaining and Harnessing Adversarial Examples \
[3] Adversarial Risk and the Dangers of Evaluating Against Weak Attacks

Reviewer comment (Novelty):
Although the paper successfully improves the performance of the model by introducing bootstrapping and contrastive learning, these contributions to robustness are already widely known outside of meta-learning and thus have little novelty [a,b]. In other words, the proposed method has little contribution other than introducing these technical components into the meta-learning problem setting, and it is unclear whether meta-learning is really necessary for generalizing adversarial robustness across domains. This is a problem of the adversarial meta-learning setting itself, and a baseline and comparison with non-meta-learning is needed to address this concern (-> Q1).

Author response:
**W1-1**. The proposed method has little contribution other than introducing already known technical components into the meta-learning problem setting.
* There seems to be a misunderstanding of our approach. The concept of each component of ours and the [a,b] is even a totally different concept.
* First of all, **our bootstrapped encoders are augmented encoders that diversify the task distribution and are different from the [a] ensemble models which are several individual classifiers**. There are none of the single similar concepts with [a].
* Furthermore, our multi-view latent attack is a latent adversarial attack that maximizes the discrepancy in the feature level while training the model in the **supervised meta-learning approach**. In contrast, [b] is a contrastive learning approach that tackles obtaining robustness without class information by introducing and maximizing the similarity of the individual instance latent. 
* This approach could seemingly be similar to ours but as we clearly demonstrate as shown in Section 3.3 and Table 3, **[b] could not leverage generalized robustness in meta-learning tasks because very few-shot instances are the bottleneck for contrastive learning that requires large batch size**.
* In particular, our approach is not simply introducing these technical components into the meta-learning approach as we clearly stated in Section 3.3, and also provides empirical results in Table 3. Therefore, it is quite unfair that our approach is simple technical combinations while we provide extensive experimental ablation and clear descriptions. We politely ask a reviewer to find the innovative ideas that we proposed to solve the generalized adversarial robustness in few-shot tasks.


**W1-2**. The reason why meta-learning is needed is unclear. Comparison with a baseline pre-trained with adversarial training should be provided. 
To the best of our knowledge, none of the adversarial self-supervised learning has shown generalizability on few-shot tasks. We tackle the problem of generalized adversarial robustness in the scope of the meta-learning approach.


Q1. Is meta-training really necessary?
* We understand your perspective that an adversarial self-supervised learning approach could be another research direction to tackle generalized robustness. However, this does not justify that meta-training is not a necessary research direction, especially since **none of the research has solved the generalized robustness problem**.
* Furthermore, **meta-training costs (18 hours with a single 2080 Ti GPU) way less than adversarial self-supervised learning (Takes 36 hours or more with two 2080 Ti GPUs)**.
The required data for **meta-training (only trained on 64 classes of CIFAR-100) is less than adversarial self-supervised learning (trained on 100 classes of CIFAR-100)**. Meta-learning is more applicable in the real world when the data is expensive or scarce such as clinical problems.
* From the experimental results, we believe we have shown the significance and necessity of meta-training on generalized adversarial robustness tasks.
|Method|CIFAR-FS|(Seen)|Mini-ImageNet||Tiered-ImageNet||CUB||Flower||Cars||Unseen Avg.||
|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|
||Clean|Rob.|Clean|Rob.|Clean|Rob.|Clean|Rob.|Clean|Rob.|Clean|Rob.|Clean|Rob.|
|TRADES|44.21|22.81|23.68|10.63|23.22|13.43|27.05|7.53|36.98|18.21|25.74|8.63|27.33|11.69|
|RoCL [b]|70.61|44.48|44.20|19.13|46.59|23.36|48.99|20.89|78.43|50.21|46.63|20.72|52.97|26.86|
|Ours|67.75|43.42|45.82|24.12|51.46|30.06|48.56|25.23|66.49|42.16|38.29|19.43|50.32|**28.20**|

* Thank you for your thoughtful review and for recognizing the clarity of our responses to W2 and Q2. We value your feedback and welcome this opportunity to further discuss the necessity and significance of our meta-learning approach (W1 and Q1).

* In addressing your concerns about our choice of meta-learning over self-supervised learning, we want to emphasize the novelty of our approach. Our research is one of the first to apply meta-learning in the context of transferable adversarial robustness, filling a gap in the current landscape of both adversarial meta-learning and adversarial self-supervised learning. This novel approach has not only demonstrated its effectiveness through our extensive experiments but also offers a fresh perspective to the field.
* Key findings from our experiments show that our meta-learning approach significantly improves the transferability of adversarial robustness in few-shot tasks, a result not yet achieved by current self-supervised methods. This underscores the unique contribution and necessity of our approach in advancing the understanding of adversarial robustness in machine learning.
* We acknowledge the merit of your suggestion that adversarial self-supervised learning approaches are better than ours. However, to clarify, let’s compare the cons of each paper to our approach:
   * [e] requires ImageNet1K (a larger dataset) to train CIFAR10 or CIFAR100, which **incurs larger datasets and computational costs even compared to RoCL** [2].
   * [f] requires **double the model capacity for teacher and student models** compared to ours.
   * [g] requires the same amount of time as RoCL, where RoCL takes 2m 30s/epoch and [g] takes 2m 34s/epoch on two 2080 Ti GPUs, which is **more costly than ours** (18 hours with a single 2080 Ti GPU).
* Compared to e, f, g, and RoCL, ours achieve transferable robustness with **fewer number of data instances** and **reduced training time** even **without additional networks**.
* We are not suggesting that these approaches are incorrect. However, there are always pros and cons in each research direction, and in line with this, our approach, which tackles the first transferable robustness in the few-shot learning problem, is still worth studying.
* Furthermore, we are considering the integration of pretrained checkpoints from SSL methods with ours to further enhance adversarial robustness, as referenced in [1]. Such integration would not only complement our current findings but also broaden the scope beyond simple adversarial self-supervised learning.
* Our goal is not to claim superiority over existing SSL methods but to highlight that our approach tackles an unexplored and critical problem in the field. Our work brings to the forefront the potential of meta-learning in adversarial contexts, supported by solid experimental evidence and thorough analysis.

We hope this response more directly addresses your concerns and illustrates the value of our meta-learning approach in contributing to broader research on adversarial transferable robustness. Thank you again for your valuable insights and for encouraging a deeper examination of our work.

[1] Rethinking Few-shot Image Classification: A Good Embedding is All You Need?\
[2] Adversarial self-supervised contrastive learning

Reviewer comment (Experiments):
The paper adds one more encoder for multi-view representation learning, increasing the capacity of models. Since there are no comparative experiments with baseline or ensemble methods with the same model capacity, it is not possible to distinguish the performance improvement by the proposed method from ones by increasing model capacity. Current baselines and evaluations are hard to say fair (-> Q2).

Author response:
**W2, Q2**. One more encoder for multi-view representation learning, increasing the capacity of models where comparison is not fair.
* There seems to be a misunderstanding regarding how our bootstrapped encoders are employed during the training. 
* **Our approach uses the same parameter capacity.** As noted in the algorithm, our encoders operate as bootstrapped parameters during the inner-adaptation which are shared parameters. (page 5: algorithm last line/ page 5: green text)
* Consequently, our approach has **the same capacity as previous works with the same backbone ResNet12.** (page 7: orange text) During the meta-optimization phase, we update only a single encoder. (page 6: green text)

Reviewer comment (Experiments):
Is meta-training really necessary? In previous studies [c,d], pre-trained models trained on meta-training datasets without meta-training were introduced as an important baseline for evaluating clean accuracy. In the case of this paper, a comparison with a baseline pre-trained with adversarial training (e.g., TRADES) would confirm the significance of meta-training.

Author response:
**W1-2**. The reason why meta-learning is needed is unclear. Comparison with a baseline pre-trained with adversarial training should be provided. 
To the best of our knowledge, none of the adversarial self-supervised learning has shown generalizability on few-shot tasks. We tackle the problem of generalized adversarial robustness in the scope of the meta-learning approach.


Q1. Is meta-training really necessary?
* We understand your perspective that an adversarial self-supervised learning approach could be another research direction to tackle generalized robustness. However, this does not justify that meta-training is not a necessary research direction, especially since **none of the research has solved the generalized robustness problem**.
* Furthermore, **meta-training costs (18 hours with a single 2080 Ti GPU) way less than adversarial self-supervised learning (Takes 36 hours or more with two 2080 Ti GPUs)**.
The required data for **meta-training (only trained on 64 classes of CIFAR-100) is less than adversarial self-supervised learning (trained on 100 classes of CIFAR-100)**. Meta-learning is more applicable in the real world when the data is expensive or scarce such as clinical problems.
* From the experimental results, we believe we have shown the significance and necessity of meta-training on generalized adversarial robustness tasks.
|Method|CIFAR-FS|(Seen)|Mini-ImageNet||Tiered-ImageNet||CUB||Flower||Cars||Unseen Avg.||
|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|
||Clean|Rob.|Clean|Rob.|Clean|Rob.|Clean|Rob.|Clean|Rob.|Clean|Rob.|Clean|Rob.|
|TRADES|44.21|22.81|23.68|10.63|23.22|13.43|27.05|7.53|36.98|18.21|25.74|8.63|27.33|11.69|
|RoCL [b]|70.61|44.48|44.20|19.13|46.59|23.36|48.99|20.89|78.43|50.21|46.63|20.72|52.97|26.86|
|Ours|67.75|43.42|45.82|24.12|51.46|30.06|48.56|25.23|66.49|42.16|38.29|19.43|50.32|**28.20**|

* Thank you for your thoughtful review and for recognizing the clarity of our responses to W2 and Q2. We value your feedback and welcome this opportunity to further discuss the necessity and significance of our meta-learning approach (W1 and Q1).

* In addressing your concerns about our choice of meta-learning over self-supervised learning, we want to emphasize the novelty of our approach. Our research is one of the first to apply meta-learning in the context of transferable adversarial robustness, filling a gap in the current landscape of both adversarial meta-learning and adversarial self-supervised learning. This novel approach has not only demonstrated its effectiveness through our extensive experiments but also offers a fresh perspective to the field.
* Key findings from our experiments show that our meta-learning approach significantly improves the transferability of adversarial robustness in few-shot tasks, a result not yet achieved by current self-supervised methods. This underscores the unique contribution and necessity of our approach in advancing the understanding of adversarial robustness in machine learning.
* We acknowledge the merit of your suggestion that adversarial self-supervised learning approaches are better than ours. However, to clarify, let’s compare the cons of each paper to our approach:
   * [e] requires ImageNet1K (a larger dataset) to train CIFAR10 or CIFAR100, which **incurs larger datasets and computational costs even compared to RoCL** [2].
   * [f] requires **double the model capacity for teacher and student models** compared to ours.
   * [g] requires the same amount of time as RoCL, where RoCL takes 2m 30s/epoch and [g] takes 2m 34s/epoch on two 2080 Ti GPUs, which is **more costly than ours** (18 hours with a single 2080 Ti GPU).
* Compared to e, f, g, and RoCL, ours achieve transferable robustness with **fewer number of data instances** and **reduced training time** even **without additional networks**.
* We are not suggesting that these approaches are incorrect. However, there are always pros and cons in each research direction, and in line with this, our approach, which tackles the first transferable robustness in the few-shot learning problem, is still worth studying.
* Furthermore, we are considering the integration of pretrained checkpoints from SSL methods with ours to further enhance adversarial robustness, as referenced in [1]. Such integration would not only complement our current findings but also broaden the scope beyond simple adversarial self-supervised learning.
* Our goal is not to claim superiority over existing SSL methods but to highlight that our approach tackles an unexplored and critical problem in the field. Our work brings to the forefront the potential of meta-learning in adversarial contexts, supported by solid experimental evidence and thorough analysis.

We hope this response more directly addresses your concerns and illustrates the value of our meta-learning approach in contributing to broader research on adversarial transferable robustness. Thank you again for your valuable insights and for encouraging a deeper examination of our work.

[1] Rethinking Few-shot Image Classification: A Good Embedding is All You Need?\
[2] Adversarial self-supervised contrastive learning