Paper: Leveraging image representations for bounded adversarial attacks and robustness (ICLR-2024)

Reviewer comment (Evaluation):
Adversarial training using their attack introduces robustness to threat models which were not trained against originally. However, the robustness introduced against image corruptions via this method falls short of what dedicated methods attain (a limitation noted by the authors).

Author response:
The model trained to be robust under our proposed DWT subspace (that has an adversarial accuracy of 90.1% in this subspace) is also the most robust against image corruptions (average accuracy of 19 corruptions	is 86.21). So training for robustness under the DWT space transfers to robustness against unseen 19 image corruptions. You have worded this transferability differently: "Next, the authors show that adversarial training using their attack introduces robustness to threat models which were not trained against originally. This is a very significant problem and, from what I’ve seen, usually solved via some form of ensembling."

Reviewer comment (Experiments):
For the robustness-to-image-corruption finding in Point 3, there needs to be some control for the amount of image corruption added. Since adversarial examples produced by the authors are closer to the original image, it might be possible that their method only performs better because the amount of corruption is too low; adding more noise might make vanilla PGD perform better.

Author response:
* If you mean by "magnitude of image corruptions" the severity of image corruption types taken from the CIFAR-10-C image corruption benchmark: we have used the default severity 3 (the maximum is 5).
* If you mean by "magnitude of image corruptions" the difference between the clean images and the adversarial examples: 

    * For the attack evaluation: we have computed the $L^2$ and $L^\infty$ of the difference between clean images and adversarial examples produced by our attacks and those produces by the baseline attacks. We did so for 4 values of $\epsilon$ and reported the results in Table. 1.

    * For adversarial training: we have adopted a common magnitude: an $L^\infty$ box radius $\epsilon=0.05$. This magnitude is a common choice when PGD is used for adversarial training. We show that our method (selecting subspaces inside the box based on image transformations) improves the robustness on CIFAR-10-C compared to adversarial training on the full box.

Reviewer comment (Experiments):
There should be comparisons between attacks. If the claim is about the transferability of robustness, the authors should train the model with adversarial examples produced using their own attack and test them on adversarial examples produced via PGD (and vice versa). This would clarify the limits of the claimed transferability regardless of the outcome.

Author response:
Indeed this experiment is interesting. We did it and show the results below. In this table, rows correspond to classifiers trained under different regimes: 
* No adversarial training: Vanilla training,
* AT using TRADES loss in $L^\infty$ box: TRADES in $\mathbb{B}_\infty$,
* AT using TRADES loss in the proposed subspaces:
    * of DCT: $\mathbb{S}_\text{dct}$,
    * of DWT: $\mathbb{S}_\text{dwt}$,
    * of Glow: $\mathbb{S}_\text{glow}$.

Columns represent accuracies measured on:
* clean images: **Clean acc.**,
* average accuracy across all 19 corruption types: **Avg. acc. of 19 corruptions**,
* accuracy in adversarial examples produced by PGD laying the $L^\infty$ box: **Adv. acc. in $\mathbb{B}_\infty$**, 
* accuracy in adversarial examples produced by our barrier attacks operating in:
    * DCT subspace: **Adv. acc. in $\mathbb{S}_\text{dct}$**,
    * DWT subspace: **Adv. acc. in $\mathbb{S}_\text{dwt}$**,
    * and Glow subspace: **Adv. acc. in $\mathbb{S}_\text{glow}$**.

For example, the model trained using adversarial examples produced by our barrier attack in the DCT space (third row) achieves an accuracy of 90.3\% when tested on adversarial examples produced by the barrier attack operating in the Glow space (last column).


|                  | Clean acc. | Avg. acc. of 19 corruptions |Adv. acc. in $\mathbb{B}_\infty$ | Adv. acc. in $\mathbb{S}_\text{dct}$ | Adv. acc. in $\mathbb{S}_\text{dwt}$ | Adv. acc. in $\mathbb{S}_\text{glow}$
| :---                |          ---: |  ---: | --: | --: | --: | --:
| Vanilla training   | 95.46   |78.89| 0.01 | 53.1 | 32.38 | 92.09
| TRADES on $\mathbb{B}_\infty$ | 81.23 |73.93 | 36.8 | 71.13 | 67.54 | 76.3
| TRADES on $\mathbb{S}_\text{dct}$ (ours) | 94.84 |82.37| 0.11 | 93.1 | 64.6 | 90.3
| TRADES on $\mathbb{S}_\text{dwt}$ (ours)| 93.83 | **86.21** | 0.39 | 88.2 | **90.1** | 89.41
| TRADES on $\mathbb{S}_\text{glow}$ (ours)| 95.41 |79.04|  0.0 | 54.09 | 33.24 | 92.5
|

$$\def\B{\mathbb{B}_\infty}$$

$$\def\Sone{\mathbb{S}_\text{dct}}$$

$$\def\Stwo{\mathbb{S}_\text{dwt}}$$

$$\def\Sthree{\mathbb{S}__\text{glow}}$$


Training for robustness in $\B$  improves the robustness in this box (as expected) and further improves the robustness in tow of our proposed subspaces ($\Sone$ and $\Stwo$) but it decreases the robustness in the Glow space, on clean images and most importantly it decreases the average accuracy across the 19 corruption types. On the other hand, the models trained on images produced by our attacks (in particular the one operating the DWT subspace $\Stwo$) present low accuracy on images produced by PGD, but they maintain high accuracy on images produced by the corresponding attack (the one used for their training). Finally, we can conclude that the robustness in the proposed subspace (in particular $\Stwo$ of 90.1\%) transfers to robustness under unseen images corruptions.

Reviewer comment (Theory):
The barrier function was chosen to be of the form $b(a^{*}) = -\frac{1}{g(a^{*})}$. The reviewer asks what is the reason for this choice apart from it being continuous, and whether other choices were considered.

Author response:
There are two popular choices of the barrier functions in the literature, the negative inverse: $b(a) = -1/g(a)$, and the log: $b(a) = \log{g(a)}$. We have implemented both and we found that the former performs better.

Reviewer comment (Theory):
Question about why the sign of the gradient is taken in the SGD update instead of the value. Is there any reason beyond it working better in practice in this setting?

Author response:
There is no theoretical reason, only practical.

Reviewer comment (Evaluation):
The attack success rate of the proposed method is much lower compared to baseline methods, which may also weaken the results with smaller perturbation radius. The author could report avg. Lp distance of the same top% of adversarial examples from the baseline method.

Author response:
"...The author could report avg. Lp distance of the same top% of adversarial examples from the baseline method."

We have reported the avg. $L^p$ distances in Table 1. Would you please clarify what you mean by "same top%"?

Reviewer comment (Theory):
As the proposed method assumes that the image representation function $\sigma$ is invertible, do the DCT and DWT implemented in the paper invertible? In my understanding they produce a compressed version of the original image and are not invertible.

Author response:
"As the proposed method assume that the image representation function $\sigma$ is invertible, do the DCT and DWT implemented in the paper invertible? In my understanding they produce a compressed version of the original image and are not invertible."

The DCT and DWTs themselves are invertible, the former is even orthogonal (L2 norm preserving). Often, they are used in a pipeline with a subsequent lossy transformation (e.g., removing high frequencies or quantization). We did not include these.

Reviewer comment (Evaluation):
Comparison with baseline attacks using Linf boxes is not fair. The proposed method has smaller perturbations but also lower success rates. It is possible that the perturbation size of the baseline attacks may be reduced at the cost of the success rates. The proposed method and the baseline methods need to be compared by controlling for the success rate or the perturbation size.

Author response:
We have considered controlling the perturbation size for a fairer comparison. However, the subspace our attacks are operating on have different dimensions than the full space (the PGD's) $q<n$. This renders the "sizes" or the volumes incomparable. The success rate is unknown until running a particular attack over a set of images which prevents us from fixing the success rate a priori for comparison. As a result, we have compared based on the $L^\infty$ radius. We are open to suggestions.

Reviewer comment (Experiments):
The introduction says "In this paper we offer progress in the quest for corruption robustness by presenting a powerful novel adversarial attack and associated adversarial training". Thus, it sounds like improving the robustness against corruption is the main purpose of this paper. However, the experiments do not compare with any baseline specifically for robustness against corruption.

Author response:
We are approaching the problem of robustness from an adversarial training perspective unlike state-of-the-art corruption robust methods that are mainly based on data augmentation. We have discussed this in Section 4, the Limitations and discussion paragraph.

Reviewer comment (Evaluation):
The proposed adversarial training has improvement on Blur and Noise, but not really on Digital and Weather. Thus, the significance of the empirical improvement is limited.

Author response:
Not correct. From Table 2: under the Digital category, we have improved in Elastic from 85.04 to 88.42, in JPEG from 81.07 to 98.44, and in Pixelization from 79.23 to 92.09. In the Weather category, the improvement is in Snow from 84.33 to 88.89, in Fog from 89.15 to 90.81, and in Spatter from 87.99 to 89.78.

Reviewer comment (Novelty):
The concept of subspace adversarial training has earlier appeared in Li et al., 2022, which is missed in this work, although the techniques are not the same.  
Li, T., Wu, Y., Chen, S., Fang, K., & Huang, X. (2022). Subspace adversarial training. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 13409-13418).

Author response:
Yes, the subspaces we are proposing are induced by semantically relevant image transformations in contrast to those by Li et al., 2022 - we will add a reference.

Reviewer comment (Experiments):
After rebuttal, the authors stated they are approaching the problem of robustness from an adversarial training perspective unlike state-of-the-art corruption robust methods that are mainly based on data augmentation, and discussed this in Section 4. However, the problem and the goal are still the same, and thus a comparison is necessary.

Author response:
We are approaching the problem of robustness from an adversarial training perspective unlike state-of-the-art corruption robust methods that are mainly based on data augmentation. We have discussed this in Section 4, the Limitations and discussion paragraph.

Reviewer comment (Evaluation):
The method finds examples that are less perturbed than other methods compared but the attack success rate is not up to par with other methods. In fact, it is very low in some of the image representations used. The presented trade-off between distance from original image and success rate does not show it to be a very effective attack method. You can't really visually tell the difference between the adversarial images generated from this attack and PGD.

Author response:
> "The method finds examples that are less perturbed than other methods compared but the attack success rate is not upto par with other methods. In fact it is very low in some of the image representations used. The presented the trade-off between distance from original image and success rate does not show it to be very effective attack method."

True, there is a trade-off since our attacks operate in low dimensional spaces (we have more constraints to satisfy in addition to the $L^\infty$ box). We have discussed this trade-off in **Appendix B**.

> "You can't really visually tell the difference between the adversarial images generated from this attack and PGD."

We disagree. In Fig. 4 we plot the difference between clean and perturbed image for our method and the baselines. This shows a very significant difference of the adversarial examples found by our approach versus the prior ones, which is then also quantified by the LPIPS metric in Table 1.

Reviewer comment (Experiments):
The attack methods are only compared on CIFAR - ImageNet would be a good addition as a datapoint to the experiments.

Author response:
> "The attack methods are only compared on CIFAR - Imagenet would be a good addition as datapoint to the experiments."

Not true, we did evaluate our attacks on ImageNet, please refer to Fig 3 and Section 4.1.

Reviewer comment (Evaluation):
No mention of the cost of attack, number of steps to reach the adversarial example, and time taken by barrier method optimization.

Author response:
> "No mention of the cost of attack, number of steps to reach the adv example and time taken by barrier method optimization."

We did specify the number of steps $T = 10$ in Section 4.2. The cost depends on the chosen transformation. We have instantiated our attack for three transformations: DCT, DWT and Glow. For DCT and DWT, the additional cost (compared to PGD) is negligible since they are fast linear transforms. For the Glow-based attack, the additional cost is linear in the depth of Glow's neural network.

Reviewer comment (Experiments):
For adversarial training, comparison has only been made with PGD. Other methods should be used in the comparison.

Author response:
> "For adversarial training, comparison has only been made with PGD. I believe other methods should be used in the comparison."

We have used TRADES (Section 4.2, second paragraph) which remains the standard for adversarial training (despite being introduced in 2019).

Reviewer comment (Experiments):
ImageNet was not used for adversarial training, and it would be a good experiment to add.

Author response:
> "Again ImageNet not used for adversarial training, would be a good experiment to add."

The cost of a comprehensive adversarial training on ImageNet is beyond the computational resources we have access to. However, we are releasing the code under a GPLv2 license which allows any third-party to replicate the results on other datasets.

Reviewer comment (Experiments):
Overall, the experiments do not showcase the effectiveness of the method as an adversarial attack or for training robust models compared to other state-of-the-art methods.

Author response:
> "Overall the experiments do not showcase the effectiveness of method as adversarial attack or for training robust models compared to other state of the art methods."

We respectfully disagree, see Table 2 for evidence.

Reviewer comment (Writing):
Typo: Eqn 2: $R^n$ -> $R^q$ ?

Author response:
> "Typo:
> Eqn 2 : $\mathbb{R}^n$ ->  $\mathbb{R}^q$?"

Indeed, typo corrected.