Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper presents a nice framework of test-time fine tuning through self-supervision for adversarially trained networks with the purpose of improving the robust accuracy on test data. A meta adversarial training strategy is also proposed that strengthens the correlation between the self-supervised and classification tasks to provide a good starting point for test-time fine tuning. The proposed method improved robustness performance against different attack methods including adaptive attacks where the attacker has knowledge of the fine tuning technique.
main_review: The test-time fine tuning method for better adversarial robustness performance is very interesting. Recently, the test-time training techniques have gained popularity mainly due to their improved performance and adaptability to different domains. Using this strategy for enhanced adversarial robustness is definitely a nice idea. While the authors presented the method with proper motivation followed by results that shows performance improvement, there are some points I would like to be clarified on as given below.

1)Basic methodology of the proposed work is mainly based on the papers [Hendrycks et al., 2019] (https://arxiv.org/pdf/1906.12340.pdf) and [Sun et al., 2020] (https://arxiv.org/pdf/1909.13231.pdf), the first one was not even mentioned anywhere in the introduction or the related work sections. While the improvements made over these mentioned papers are significant, it's very important to properly explain the previous efforts in this space and position the current work for a better understanding of the actual contribution.

2)The method analysis subsection (3.4) is not well explained in my view. It's not clear to me how a correlation mean, larger than robust accuracy of the adversarially trained network, implies the effectiveness of the proposed method.

3)While the results, presented for 3 different datasets, show good adversarial robustness performances against different attack methods, they were only compared with baselines that are other variations of the proposed method. It's very important to add results of other SOTA adversarial robustness methods that would improve the acceptance of the proposed method.

4)The only drawback of test-time training methods could be arguably the increased inference time. As adversarial training itself is a highly time-consuming method mainly for the inner maximization i.e. generating the adversarial images, it's important to show the time required for the proposed test-time fine tuning for adversarially robust methods.
summary_of_the_review: An interesting work that tried to improve adversarial robustness through test-time fine tuning using self-supervision techniques. Answering/commenting on the points, that I posted under the main review, can significantly improve the quality of this work.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: Fine-tuning the parameters of an adversarially-trained model at test-time with self-supervised tasks improves robustness to standard adversarial attacks. By incorporating meta-learning into the adversarial training stage, the test-time fine-tuning achieves marginally higher accuracy (though not reduced optimization time, or at least this is not reported). The test-time optimization approach requires the training data be kept available for joint updates on training batches and test data. The self-supervised tasks considered are rotation prediction, a common choice, and vertical flip recognition, which is a close relative of it. Results are reported on the standard adversarial defense benchmark of CIFAR-10, and additionally models are also evaluated on STL and Tiny ImageNet. Design choices are ablated and an adaptive attack is designed and experimented with, but there other checks against obfuscated gradients (for instance those suggested by Athalye et al. 2018) are not pursued, such as attacks in expectation, or decision-based attacks. No comment is made about the computational cost of test-time fine-tuning.
main_review: Strengths
- **Optimization for Defense is a Worthy Topic**. Attacks optimize against defenses so it is entirely sensible that research should explore defenses that optimize back. The proposed method in particular addresses not just the test-time fine-tuning to do so, but a training-time preparation to do so by meta-learning.
- **Standard Experimental Design**. The choice of attacks comprise the standard AutoAttack benchmark. There are standard choices of dataset (CIFAR-10) and architectures (wide resnet, standard resnet) that are common for work on adversarial defense. The baselines are sensible, and separate the effects of adversarial training, multi-task training with self-supervision, and the meta-learning variant of adversarial training.
- **Method Variants**. The experiments cover different scopes, like online or offline optimization, and ablations of the loss components (self-supervision and regularization) and meta adversarial training. The different parts of the proposed method all help. Furthermore, offline optimization on the full test set improves over online optimization on a portion of it, which could be a positive if the fine-tuned model transfers to more attacked data without further optimization.
- **Popular Choice of Self-Supervision**: rotation prediction is a common and effective choice for use and benchmarking (the Gidaris et al. 2018 paper on it is cited 1000+ times).

Weaknesses
- **There are Prior Test-Time Defenses**. There is existing work on test-time optimization of the model parameters (runtime masking and cleansing ICML'20), of model latents (Schott et al. ICLR'19), and of the input (SOAP/Online Adversarial Purification based on Self-Supervision ICLR'21). The last method is especially related as it optimizes self-supervised tasks at test-time to improve robustness. Apart from this recent wave, there is prior work on randomization at test-time (Raff et al., 2019; Cohen et al., 2019; Pang et al., 2020, Dhillon et al., 2018) and projecting inputs onto generative model distributions (Song et al., 2018; Samangouei et al., 2018; Hill et al., 2021). Although it could be argued this work differs from most in optimizing the model parameters, and not the input or latent variables, the point is somewhat moot because the optimization is episodic so parameter updates do not persist and affect inference on other data. None of these prior methods are included as baselines, but at least SOAP should be, as it also optimizes self-supervised tasks during testing.
- **Computational Cost**. Optimization during inference is more intensive than a mere forward pass of a static method. Even more expensive still is joint optimization on batches of training data and test data, as done by this method. Other test-time defenses only require test data (SOAP, https://arxiv.org/abs/2103.14347, https://arxiv.org/abs/2105.08714), or small amounts of auxiliary data (https://arxiv.org/abs/2103.14222). This expense could be justified by the amount of robustness gained, but the ratio here is not reported.
- **Main Attacks are for Static Defenses**. AutoAttack is a strong suite of attacks, but it is designed with _static_ defenses in mind, and the corresponding RobustBench benchmark only includes defenses that are deterministic and do not optimize during inference. With this in mind, improving robustness on this benchmark is necessary but not sufficient to show that a test-time defense is better. While one adaptive attack is provided, more should be done to heed well-established warnings about obfuscated gradients (Athalye et al. ICML 2018). Possibilities include a transfer attack from the static adversarial defense, an expectation attack on one test input with several samples of training and testing batches, and evaluation with a decision-based attack like boundary.
- **Narrow Look at Self-Supervision**. Rotation prediction is a popular task, but it is just one. Vertical flip prediction is so closely related that it is not so informative as a second task. A generic task, like reconstruction/auto-encoding, or a state-of-the-art task, like contrastive learning, would be more informative for the utility of self-supervision for defense.

For Rebuttal
- Please discuss the contributions of this work relative to the cited test-time defenses, and SOAP in particular https://openreview.net/forum?id=_i3ASPp12WS. Ideally this discussion could compare the results for optimizing the input, like SOAP, vs. optimizing the model parameters, like this work.
- Please comment on the computational requirements of this defense. How much more expensive is it in time and FLOPs than a single forward pass?
- Please discuss the obfuscated gradients checklist from Athalye et al. 2018 and cover how this work cannot be explained away as gradient obfuscation.
- Please experiment with a decision based attack, such as the boundary attack.

Miscellaneous Feedback
- [related work] Alongside the already published references mentioned above, there are a number of contemporary works published or posted to arxiv this year, which this work could discuss and compare against to strengthen its scope and evaluation: https://arxiv.org/abs/2103.14222, https://arxiv.org/abs/2103.14347, https://arxiv.org/abs/2105.08714, https://arxiv.org/abs/2106.04938, https://openreview.net/forum?id=RFGkzxMFqby, https://openreview.net/forum?id=36rU1ecTFvR, https://openreview.net/forum?id=3Uk9_JRVwiF
summary_of_the_review: The contributions are summarized as (1) introducing the framework of test-time fine-tuning for defense, (2) meta adversarial training, and (3) the adaptive attack against the proposed defense. However, the first and most novel contribution (1) needs qualification, as there is substantial prior work on test-time optimization for defense, and none are included as baselines. This lack of discussion and experiment makes the work incomplete. (2) is indeed a contribution. On the other hand (3) is more a requirement for proposing a defense than an independent contribution. As outlined in the main review, more could be done here to evaluate that the defense is more robust and not superficially interfering with attacks. As there is a wave of such defenses at present, it is important to verify improvement on what has already been done, and to work toward common evaluations techniques across them. As such, I encourage the authors to further test the proposed defense, but must recommend rejection of this edition of the work.

**Final Review** The rebuttal responded to the weaknesses of computational cost and the need for more attack types to assess gradient obfuscation but only partially addressed the prior and concurrent work on test-time optimization for defense. On the plus side, the potential issue of gradient masking is resolved by evaluation of a decision attack and reporting that robustness fails as the adversarial budget increases. However, on the minus side the computational cost of the method is significant and the transfer attack reduces the improvement in robustness from fine-tuning. More importantly, the response does not acknowledge the prior method of runtime masking and cleansing, which likewise updates model parameters. Furthermore, because the parameter updates in this work are _episodic_, it is perhaps not as different from purification methods as claimed. More work is needed to situate the contributions with respect to prior defenses, such as analysing when the model parameters (for this method) or input perturbation (for purification methods) is shared/unshared across the batch.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: Propose to improve the generalization and robust accuracy of adversarially-trained networks via self-supervised test-time fine-tuning. Introduce a meta adversarial training method to find a good starting point for test-time fine-tuning, which  incorporates the test-time fine-tuning procedure into the training phase. Observe consistent improvement under different attack strategies for both white-box and black-box attacks.
main_review: In summary, there isn't much to critique about this paper. The exposition was clear, barring a number of typos/grammatical errors which should be resolved in the revision. While the white-box version of the adaptive attack was unconvincing due to the gradient estimate used, the authors supplemented these results with a black-box attack that does not rely on local gradient information. Overall, a well executed paper. My only remaining concern is w.r.t. related work, while a number of papers are listed below, a comparison to existing work leveraging test-time training for adversarial robustness would be warranted. 

Test-time training applied to adversarial defense: https://arxiv.org/abs/2105.08714.  
Explored addition of unlabeled data for adversarial training: https://arxiv.org/abs/2010.03593.  
Applied self-supervised learning technique for robust learning: https://openreview.net/forum?id=bgQek2O63w

- Some typos, e.g. "As SGD is more efficient more large amount of data..."
- "...we use SGD to optimize $\theta$ when b is large (e.g. offline setting)." Please explicitly state what's done when b is small. 
- "... strengthens the correlation between the self-supervised and classification tasks" what evidence was provided that meta-adversarial training strengthens the correlation between the self-supervised and classification tasks? The analysis showed that if these tasks are correlated, self-supervised fine-tuning will help, but I do not see where this particular statement in the abstract is supported. 


summary_of_the_review: Paper is well-written and lacks any obvious concerns regarding the content, other than the lack of comparison against existing methods for leveraging test-time training for adversarial defence.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The paper introduces a meta adversarial training method to find a good starting point for test-time fine-tuning. It incorporates the test-time fine-tuning procedure into the training phase and strengthens the correlation between the self-supervised and classification tasks. Results on CIFAR-10, STL10 and Tiny-ImageNet show consistent robustness improvement.
main_review: Strengths:
========
Robust overfitting of adversarial training is a well known problem and thus require separate attention , in that context the paper is well motivated. The results are good in the sense that with Meta AT the accuracy improvement is consistent.

Weaknesses:
===========
1. I am not sure whether it is at all feasible to have the test samples even without labels before head to do self-supervised fine tuning. The assumption of fine tuning on test mini batch thus definitely limits the scope of applications of the proposed method.

2. Test time training (TTT) as a fine tuning for a full test set with the full model might be extremely in efficient and compute heavy that many inference platforms can't support. 

3. Though understand what the variable is,  please be specific about the definition of $\epsilon$, the size of budget is technically not correct. 

4. The difference that the authors mentioned from [1], looks like disadvantage to me, as the original paper did incremental fine tuning, whereas the current paper does fine tuning starting from same weights, for each image, The reasoning to take this different strategy is unclear. Also, if the batch size is >1, how does the author manage to start from same $\theta^0$ for each image of a batch? Please also mention what the $\mathcal{L}_{SS}$ loss is? For example, it is definitely not any cross entropy loss. 

5. Also, please explicitly mention  the $\theta_{E}$ and $\theta_{f}$  definitions that are used for the multi task formulation.

6. How is the author sure about that the $\theta^0$ yielded by meta learner gives best starting point. I think the author meant near optimal, hence please rephrase your sentences.

7. Eq. 4 uses test samples $\hat{B}_j$ for SS-loss where as Eq. 7 uses training sample batch for  SS-loss, why? 

8. Please do refer and compare with the paper [2] as they seem to solve similar issues of adversarial training.

9. Please provide a detailed analysis of the training and inference time and compute complexity over head due to the meta adversarial training and TTT.

10. How the authors use the same starting $\theta^0$ for batch size of $B$ > 1 is not clear to me. Please clarify.

11. The comparison with Regular AT and Regular AT + w/o FT in Table 4 is unfair to me, as in these cases the user has no chance to do any fine tuning with the test samples.


[1] Test-time training with self supervision for generalization under distribution shifts, ICML 2020.
[2] Robust Overfitting may be mitigated by properly learned smoothening, ICLR 2021.
summary_of_the_review: The authors proposes a costly training and slow fine tune based testing strategy to get rid of robust overfitting issues associated with AT. In this context the authors should justify the increased cost of training/inference with gain in performance trade-off. The authors should also compare with at least the one relevant paper I mentioned.  I can comment on the overall incremental novelty once I get to know the purpose of doing few things differently. For example the difference between TTT an proposed solution is less, and I am not convinced the $\theta^0$ initiation is necessary always. Please justify.
review_title: null

Meta-review: This paper proposes a method to improve the robust accuracy of classifiers using test-time training.  The reviewers all agree that the method is interesting, and many reviewers had a positive view of the method.  However, two main criticisms remain: (i) the method increases the runtime of inference, and (ii) comparisons to other related methods were lacking.  The authors responded to (i) by reporting runtimes for their method in the rebuttal.  Some reviewers were concerned that the runtime increase of the method is not acceptable, however I am not very concerned with this issue since I think the paper contains an interesting methodology even if it’s not ready for deployment at the industrial scale.  However, issue (ii) does not seem to have been adequately addressed.  The comparison to SOAP is a welcome addition the reviewers acknowledge, but a number of other methods, for example masking and cleansing, are closely related (but different) and so comparisons should be provided.
Decision: Reject