Paper: Towards Reliable Evaluation and Fast Training of Robust Semantic Segmentation Models (ICLR-2024)

Reviewer comment (Presentation):
The organization of this paper is confusing. The introduction of some existing works like APGD should be put in the related works. An overview of the method's structure could be added to the beginning of section 2. As section 2 mentions PIR-AT many times, a short description about the PIR-AT model is also necessary.

Author response:
Thanks for the suggestions about the presentation. We agree with the Reviewer that the definition of PIR-AT comes late in the paper, we will add a short description earlier on in the revision. We will also add more detail of SEA at the start of Sec. 2, and discuss APGD in the related works.

Reviewer comment (Novelty):
As many related works and ablation studies are mixed in the method part, it is difficult to distinguish the contributions of this work from previous works.

Author response:
Thanks for the suggestions about the presentation. We agree with the Reviewer that the definition of PIR-AT comes late in the paper, we will add a short description earlier on in the revision. We will also add more detail of SEA at the start of Sec. 2, and discuss APGD in the related works.

Reviewer comment (Experiments):
Will AT obtain the same performance as PIR-AT if sufficient training time is given?

Author response:
The ablation study in Table 4 shows that, on Pascal-VOC, AT can reach the same level of robustness obtained by PIR-AT with 6x longer training (300 vs 50 epochs). However, on the more challenging ADE-20k, AT with 128 epochs cannot match the robustness of PIR-AT with 32 epochs, for both UPerNet and Segmenter architectures. This emphasizes the importance of using a robust backbone for initialization.

Reviewer comment (Reproducibility):
PIR-AT suggests using $L_{\infty}$-robust ImageNet model for initialization. How much computational resources are required to train this model compared to the normal ImageNet model with the same parameters?

Author response:
Adversarial training on ImageNet is typically carried out with 1 or 2 steps of PGD, which means 2-3x higher computational cost than clean training for the same number of epochs (Singh et al., 2023). However, we emphasize that we did **not** train the robust ImageNet classifiers, but took some of those open sourced by prior works. In fact, we want to stress that nowadays the cost of the initialization is virtually zero, since plenty of both clean and robust ImageNet models are publicly available. For example, there are already 18 robust models for ImageNet in [RobustBench](https://robustbench.github.io/#div_imagenet_Linf_heading) with various sizes and architectures.

Reviewer comment (Evaluation):
How the image-wise worst case over all losses in Table 2 is calculated? A short description is expected.

Author response:
Please refer to the paragraph “Metrics” in Sec. 2.1: for the worst-case over multiple attacks, we select the perturbation that gives the lowest average accuracy for each image and use it to compute the mIoU.

Reviewer comment (Experiments):
Does the conclusion in Figure 2 also generalize to clean models?

Author response:
We repeated the ablation study illustrated in Fig. 2 on the clean model ($\epsilon=0.5/255$ is used), and we observe results consistent with those obtained for the robust model. In particular, the red-$\epsilon$ scheme yields the lowest (best) robust accuracy for both the individual losses, except the Masked-Spherical loss, and worst-case evaluation. We will add this comparison in the revision of the paper.

Reviewer comment (Presentation):
What is the result of AT with 32 epoch in Figure 5?

Author response:
Fig. 5 includes models on Pascal-VOC, which are trained with either 50 or 300 epochs as reported in Table 4 (in particular, for Fig. 5 we use the clean model and the PIR-AT model trained with 5 steps for 300 epochs). Conversely, we use 32 epochs on ADE-20k (Table 4). Could the Reviewer please clarify which is the suggested experiment?

Reviewer comment (Reproducibility):
This paper needs to discuss more about PIR-AT. It is only mentioned that this paper proposes Pre-trained ImageNet Robust Models. What is this method in detail?

Author response:
**Q: This paper needs to discuss more about PIR-AT. It is only mentioned that this paper proposes Pre-trained ImageNet Robust Models. What is this method in detail?**

With PIR-AT (as described in Sec. 3.1) we propose to use a robust image classifier, i.e. obtained with adversarial training, on ImageNet as initialization of the backbone of the segmentation model. Such initialization allows adversarial training on the segmentation task to achieve higher robustness than existing methods, even with lower computational cost. As shown in Table 3 and Table 4, our PIR-AT models outperform DDC-AT (Xu et al., 2021), SegPGD-AT (Gu et al., 2022), and match or improve over AT from clean initialization with 4-6x fewer training epochs.

Reviewer comment (Theory):
What is the motivation to limit this paper to focus on l_{\limit} threat model?

Author response:
**Q: What is the motivation to limit this paper to focus on l_{\limit} threat model?**

We focus on the $\ell_\infty$-threat model since this is the most popular in the literature: in fact, for image classification, the large majority of defenses reported in [RobustBench](https://robustbench.github.io/#div_imagenet_Linf_heading) are for $\ell_\infty$, and all of those on ImageNet (which we use as initialization of the model backbone in PIR-AT). Similarly, the closest prior work for robust semantic segmentation, SegPGD (Gu et al., 2022), focuses on $\ell_\infty$-bounded attacks. However, we think that SEA might be easily extended to other $\ell_p$-threat models since APGD provides versions for such cases, which could be an interesting direction to explore for future work.

Reviewer comment (Experiments):
The paper only considers untargeted attacks, and it would be useful to extend the analysis to targeted attacks to showcase the strength of the proposed attack method.

Author response:
**Q: The paper only considers untargeted attacks, and it would be useful to extend the analysis to targeted attacks to showcase the strength of the proposed attack method.**

We mainly focus on untargeted attacks since these are the most popular threat model in the literature. However, our attacks can be readily extended to the targeted scenario by using the targeted versions of the various loss functions (e.g. one can minimize the cross-entropy loss of each pixel to a class). For example, we already used a targeted version of APGD on the JS divergence loss in Fig. 1, to have the entire image being classified as either “sky” or “grass” (this is also described in App. D). We would be happy to add more such examples in the revision of the paper, and, if possible before the end of the rebuttal, a quantitative evaluation which the Reviewer would consider relevant.

Finally, we note that the targeted attacks are a more restrictive threat model than untargeted attacks: this implies that models which are robust to untargeted attacks are also robust to targeted ones with the same perturbation radius.

Reviewer comment (Experiments):
The authors could conduct experiments to evaluate the transferability of the proposed attack to other models and compare it against the baseline PGD/CosPGD/SegPGD attacks.

Author response:
**Q: The authors could conduct experiments to evaluate the transferability of the proposed attack to other models and compare it against the baseline PGD/CosPGD/SegPGD attacks.**

As a preliminary experiment we compare the transferability of APGD on the Masked-CE loss and SegPGD on a few models on ADE-20k, and report the results in Table B. SegPGD is slightly better than APGD, especially at larger radii. We hypothesize that this is due to the fact that using a mask stops the optimization for misclassified pixels, but higher confidence misclassifications are expected to transfer better. In this regard, it might be interesting to explore how our different losses influence the transferability of the perturbations. However, we emphasize that transferability is **not** one of the goals of our attacks. In fact, the most effective transfer attacks typically rely on specific techniques to enhance transferability, e.g. [Wang \& He (2021)](https://arxiv.org/abs/2103.15571), [Wang et al. (2021)](https://arxiv.org/abs/2102.00436), [Zhang et al. (2023)](https://arxiv.org/abs/2303.15735), and do not necessarily correspond to the most successful white-box methods.

**Table B:** transferability of different attacks on ADE-20k.

|attack |source model | target model     | 4/255 |        | 8/255 |       | 12/255 |       |
|:----|:--------------|:-----------|-------:|--------:|-------:|-------:|--------:|-------:|
| | | |aAcc | (mIou)| aAcc | (mIou)| aAcc | (mIou)|aAcc | (mIoU)|
| SegPGD |Segmenter AT | Segmenter PIR-AT | 65.0 | (25.1) | 59.1 | (20.8)| 49.9 |(16.1) |
| APGD w/ Masked-CE |Segmenter AT | Segmenter PIR-AT | 69.1 | (28.7) | 66.3 | (26.0) | 63.1 | (23.8) |  57.4 | (19.9) |
| SegPGD |Segmenter AT | UPerNet PIR-AT | 67.7 | (28.7) | 63.4 |(25.2) | 55.6| (20.3) |
| APGD w/ Masked-CE |Segmenter AT | UPerNet PIR-AT | 68.6| (30.0) | 64.4 |(26.4) | 60.9 |(23.7) |

Reviewer comment (Experiments):
Why is AT in Section 3 performed with the PGD attack baseline? It would be interesting to use stronger attacks during the AT to develop even stronger robust models.

Author response:
**Q: Why is AT in Section 3 performed with the PGD attack baseline? It would be interesting to use stronger attacks during the AT to develop even stronger robust models**

We believe that it is an important contribution of this work to show how it is possible to train robust semantic segmentation models with standard PGD-based AT. As a result, our PIR-AT models significantly outperform previous, more sophisticated, methods such as SegPGD-AT and DDC-AT (see Table 3).

We agree with the Reviewer that future work might focus on optimizing the attack in AT. In fact, we briefly tested the effect of using alternative losses for PGD in AT, but this did not provide benefits (see the experimental setup paragraph at the beginning of Sec. 3): we hypothesize that, when using attacks with only a few iterations (2-5) as in adversarial training, optimizing the loss for all pixels without re-weighting or masking, as in the plain cross-entropy loss, is anyway the better choice.

Reviewer comment (Experiments):
How did you choose the budget scheme of 3:3:4 in the progressive reduction of epsilon approach?

Author response:
**Q: How did you choose the budget scheme of 3:3:4 in the progressive reduction of epsilon approach?**

We followed Croce & Hein (2021) for this split. Moreover, this scheme gives sufficient iterations for the attack to be effective at each radius, with a slight bias towards the final and target radius.