Paper: CosPGD: an efficient white-box adversarial attack for pixel-wise prediction tasks (ICLR-2024)

Reviewer comment (Novelty):
The core of the proposed method is very similar to SegPGD, as both aim to focus on the pixels where the attack has not been successful yet (e.g. pixels with large cosine similarity weight). Therefore, the novelty is limited.

Author response:
1. The proposed CosPGD is motivated by SegPGD and indeed the equations look similar. Yet, there are significant conceptual differences that provide a strong benefit of CosPGD over SegPGD, which we also discuss on page 3 in our submission. Please refer to our general reply for details on the conceptual differences between the two and the resulting benefit of CosPGD. Please note that no other existing attack scales the loss pixel-wise using **similarity between the posterior and target distributions**. All attacks focus on final model predictions. This makes CosPGD significantly novel. Does this answer the important concern?

Reviewer comment (Evaluation):
Ablation experiments lacking other metrics like cosine distance.

Author response:
2. To the best of our knowledge, cosine distance is not the most commonly evaluated metric for any of the tasks considered: semantic segmentation, optical flow estimation, and image restoration(we evaluate structural similarity for this task).
For all our considered models and tasks, we report all the metrics that are used by the authors to evaluate their proposed models and attacks.

Reviewer comment (Experiments):
Lack of performance comparison experiments with state-of-the-art methods [1] for semantic segmentation tasks.

Author response:
3. Our experiments focus on the wide applicability of CosPGD rather than on outperforming a particular, dedicated approach on a particular task. We will try to provide results comparing our method to [1] for the final version, which is however not trivial since [1] proposes a minimal attack and CosPGD is an epsilon-bounded attack. In any case, we now cite in the revised version and will discuss this paper. 
**Please note** that according to the ICLR guidelines, https://iclr.cc/Conferences/2024/ReviewerGuide, publications from recent conferences (“published [...] within the last four months”: “on or after May 28, 2023”) are assumed to be contemporaneous work and a comparison is not required - this is the case for [1].

Reviewer comment (Novelty):
Take the non-targeted attack as an example, the proposed loss function in Eq.(5) $L_{\mathrm{cos}}=\frac{1}{H \times W} \sum_{H \times W} \cos (\overrightarrow{\text { pred }}, \overrightarrow{\text { target }}) \cdot L\left(f_{\text {net }}\left(\boldsymbol{X}^{\text {adv } v}\right), \boldsymbol{Y}\right),$ is very similar with the loss function of SegPGD[1]  $L_{SegPGD} = \frac{1}{{H}\times{W}} \sum_{j\in P^T} L_j + \frac{1}{{H}\times{W}} \sum_{k\in P^F} L_k$. Thus, the novelty is limited.

Author response:
The proposed CosPGD is motivated by SegPGD and indeed the equations might look similar at first glance. Yet, there are significant conceptual differences that provide a strong benefit of CosPGD over SegPGD, which we also discuss on page 3 in our submission. Please refer to our general reply for details on the conceptual differences between the two and the resulting benefit of CosPGD. Please note, that no other existing attack scales the loss pixel-wise using **similarity between the posterior and target distributions**. All attacks focus on final model predictions. This makes CosPGD significantly novel. Does this answer this important concern?

Reviewer comment (Experiments):
In **Sec.4.2** the paper identifies their method performance in the optical flow task, but it only did experiments compared with PGD[2] in **Fig.5**. I wonder how SegPGD[1] performs in the optical flow task.

Author response:
SegPGD can not be used on Optical Flow for the reasons detailed in Answer 1 unless the method is modified by defining a threshold. Similar to the results we provide for SegPGD for restoration, we can also report for optical flow as detailed below:

Table 6 in Section C.1 in the appendix of the revised version (Table 4 in original submission). A Lower(↓) epe wrt to Target and a Higher( ↑) epe wrt to Initial signifies a stronger attack.

|Attack Iterations|KITTI2015| | | | | |MPI Sintel| | | | | | | | | | | |
|----:|----:|----:|----:|----:|----:|----:|----:|----:|----:|----:|----:|----:|----:|----:|----:|----:|----:|----:|
| | | | | | | |Clean| | | | | |Final| | | | | |
| |PGD| |CosPGD| |SegPGD| |PGD| |CosPGD| |SegPGD| |PGD| |CosPGD| |SegPGD| |
| |Target↓|Initial ↑|Target↓|Initial ↑|Target↓|Initial ↑|Target↓|Initial ↑|Target↓|Initial ↑|Target↓|Initial ↑|Target↓|Initial ↑|Target↓|Initial ↑|Target↓|Initial ↑|
|10|10.5|22.1|9|23.4|11.08|21.36|3.4|11.2|2.9|11.4|3.51|11.16|3.1|11.3|2.6|11.5|3.13|11.32|
|20|8.1|24.6|6.5|25.8|7.76|24.55|2.8|11.7|2|12.1|2.97|11.61|2.5|11.8|1.6|12.1|2.62|11.7|
|40|7.3|25|4.8|27.4|7.53|24.89|2.8|11.7|1.6|12.4|2.66|11.8|2.6|12.3|1.3|12.3|2.4|11.83|

However, please note again that SegPGD was not conceived for optical flow attacks and the low performance is to be expected since SegPGD is not designed for regression tasks.

Reviewer comment (Experiments):
In **Sec4.3** the authors said "We observe that at low number of attack iterations (3 attack iterations) it performs significantly worse than PGD, thus demonstrating its limitation on this task." However, the SegPGD[1] needs to adjust their balance factor during the attack iteration, and as far as I know, white box attacks usually don't compare attack performance at low iterations. So I do not think it is fair to compare with SegPGD[1] in 3 attack iterations.

Author response:
CosPGD is stronger when few iterations are considered and therefore provides reliable results at a lower compute budget. The purpose of adversarial attacks is to reveal model weaknesses with as few iterations as possible. Therefore, we understand that an attack that outperforms others when run for only a few attack iterations is particularly valuable. In particular, when considering adversarial training, high attack iterations are extremely expensive and 3 attack iterations would be a typical value.

Reviewer comment (Experiments):
Section 4.3's content warrants appropriate adjustment. This section primarily showcases the superior degradation effect of CosPGD on NAFNet in comparison to PGD and SegPGD (particularly at low attack iterations). However, this evidence alone may not adequately support the assertion that "CosPGD can efficiently enhance a new model's robustness." To convincingly substantiate this claim, the authors should present more compelling evidence within the main body of the paper, rather than relegating it to the appendix. It is particularly essential to include results from the denoising task (as presented in Appendix D2).

Author response:
Please refer again to our section 4.3. We never make the claim that  "CosPGD can efficiently enhance a new model's robustness". 
Instead, we propose CosPGD as an effective adversarial attack that can evaluate a model’s relative robustness correctly even with low attack iterations (iterations<=3).
Thus our claim is: “CosPGD can be very useful in **predicting** a new model’s robustness efficiently.” Would you agree that this claim is sufficiently substantiated by our experiments (including the new results we provide in the rebuttal), or is there anything else you would like to see in particular?

Reviewer comment (Experiments):
Although CosPGD exhibits substantial improvements over SegPGD in terms of attack efficacy and generality, it is worth noting that SegPGD also contributes significantly to enhancing model robustness through adversarial training. The absence of corresponding experiments makes it challenging to completely establish the effectiveness of this aspect.

Author response:
1. Adversarial attacks have a purpose beyond adversarial hardening: the evaluation of existing methods in terms of stability/robustness.  We therefore focused on evaluating CosPGD across diverse conditions. In the following, we also report a comparison in terms of adversarial training. Here the models were trained with the “Training Attack” with 3 attack iterations, $\alpha$=0.01 and $\epsilon\approx\frac{8}{255}$ with a 50%-50% minibatch split, meaning only 50% of the samples in a batch were adversarially perturbed. Then, the adversarially trained models were evaluated using “Testing Attack” with multiple attack iterations are shown in the following table (here we report only the mIoU(%) for better readability):

|Model|Training Attack|Testing Attack|3 attack iterations|5 attack iterations|10 attack iterations|20 attack iterations|40 attack iterations|100 attack iterations|
|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|
|UNet|None|PGD|23.18|14.58|8.21|5.57|4.14|3.6|
| |PGD|PGD|29.26|21.28|13.74|9.29|7.47|6.38|
| |SegPGD|PGD|31.77|22.77|14.86|11.03|8.98|7.45|
| |**CosPGD**|PGD|**47.35**|**43.75**|**38.1**|**34.33**|**32.28**|**30.55**|
| |None|SegPGD|12.38|7.75|4.46|2.98|2.2|1.55|
| |PGD|SegPGD|29.38|21.31|13.77|9.39|7.45|6.38|
| |SegPGD|SegPGD|31.69|22.47|14.82|10.9|9.09|7.33|
| |**CosPGD**|SegPGD|**47.16**|**43.85**|**37.64**|**33.99**|**31.91**|**30.48**|
| |None|CosPGD|9.67|3.71|0.61|0.06|0.03|0.01|
| |PGD|CosPGD|29.23|21.09|13.49|9.28|7.36|6.29|
| |SegPGD|CosPGD|31.53|22.46|14.81|10.86|9.2|7.28|
| |**CosPGD**|CosPGD|**47.07**|**43.95**|**37.64**|**34.01**|**32.0**|**30.55**|


We include this table in the revised version of the paper as Table 5 in Section B.4

Additionally, we include Figure 11 in Section B.4 in the revised version of the paper, in this figure we show the segmentation masks predicted by UNet after being adversarially trained. We observe that even after 100 attack iterations, the model adversarially trained using CosPGD is making reasonable predictions. 
However, the model trained with SegPGD is merely predicting a blob.

Reviewer comment (Experiments):
An inquiry arises regarding the rationale behind the author's choice of an optical flow experiment to evaluate the versatility of CosPGD. The choice of optical flow as a benchmark should be substantiated by explaining how the characteristics of this task effectively highlight the advantages of CosPGD. Furthermore, additional experiments should be incorporated to showcase CosPGD's performance in various image restoration tasks, such as single image deraining, to bolster its claims further.

Author response:
2. Could you please rephrase this question? We are not sure if we understand it correctly. Our rationale is that previous methods for attacks on pixel-wise prediction tasks always only focused on a single task, i.e. there has been SegPGD for semantic segmentation and PCFA for Optical Flow. CosPGD is more general than both and can therefore be applied to both. In Section 4.2 Optical Flow: we explain in detail how CosPGD is exposing model vulnerabilities that were unknown before. 
To further showcase the generality of the approach, we also evaluate it on image restoration for recent SotA methods. Does this address your question?
Regarding deraining, we attempted to evaluate CosPGD on the SotA [a]. Yet, the model is too large to fit on our hardware when access to model gradients is needed (which is the case for white-box attacks). The model [a] was trained on 32 NVIDIA Tesla V100. If you can refer us to a more light-weight model of your choice, we will gladly provide the evaluation. 
[a] Chen et al., Pre-Trained Image Processing Transformer (IPT), 2021.

Reviewer comment (Presentation):
It seems like the authors need to reorganize the contribution of the paper, since the core of the paper is actually a general improvement on adversarial training for pixelwise classification tasks.

Author response:
3. There seems to be a misconception: as specified in the contribution section, the contribution is not the improvement of adversarial training but proposing the first adversarial attack that provides a unified evaluation for diverse pixel-wise prediction tasks. Of course, adversarial attacks can be used for adversarial training (see our response to your Q1 for such results) and better adversarial attacks often result in better adversarial training. Our results above show that this is also the case for CosPGD.

Reviewer comment (Presentation):
The experimental results presented in Figures 14 and 15 make it challenging to discern the numerical values. Presenting the data in a tabular form would be more beneficial.

Author response:
4. Thank you for the suggestion. These values are indeed very similar to each other in many cases, and the plots are supposed to provide this information. When considering Figure 14 in the revised version (Figure 13 in the original submission, when considering targeted attacks as a whole), it is visible from the reported standard deviation that the difference is not always significant in this case. Yet, so far, we only have one run for the results in Figure 15 (target flow being 0 vector) and Figure 16(target flow being negative initial flow) in the revised version(Figures 14 and 15 in the original submission) and we do not observe any significant difference in numbers. We are currently running these experiments with multiple runs to be able to provide tabular results with standard deviations over multiple seeds for a better understanding.

Reviewer comment (Writing):
There is a lack of a detailed definition for $L$ in equations (1), (5), and (6).

Author response:
5. Indeed, we did not specify a particular loss $L$ in equations (1), (5) and (6). In all cases, it is supposed to be the loss of the model at hand, which will be a different loss term for semantic segmentation than for optical flow and again a different one for restoration. Since we propose CosPGD as an attack that can be applied across diverse pixel-wise prediction tasks, $L$ can take different forms. We now specify in the revised submission that $L$ is assumed to be a one-differentiable function of the model prediction and the target, that defines the loss the model aims to minimize.