CAT: Collaborative Adversarial Training
Authors: xingbin liu
Huafeng Kuang
Xianming Lin
GUANNAN JIANG
YONGJIAN WU
Xi Wang
Rongrong Ji
Venue: Submitted to ICLR 2023 (2023)

Abstract: Adversarial training can improve the robustness of neural networks. Previous adversarial training methods focus on a single training strategy and do not consider the collaboration between different training strategies. In this paper, we find different adversarial training methods have distinct robustness for sample instances. For example, an instance can be correctly classified by a model trained using standard adversarial training (AT) but not by a model trained using TRADES, and vice versa.  Based on this phenomenon, we propose a collaborative adversarial training framework to improve the robustness of neural networks. Specifically, we simultaneously use different adversarial training methods to train two robust models from scratch. We input the adversarial examples generated by each network to the peer network and use the logit of the peer network to guide the training of its network. Collaborative Adversarial Training (CAT) can improve both robustness and accuracy. Finally, Extensive experiments on CIFAR-10 and CIFAR-100 validated the effectiveness of our method.
CAT achieved new state-of-the-art robustness without using any additional data on CIFAR-10 under the Auto-Attack benchmark.

Under review as a conference paper at ICLR 2023
CAT: COLLABORATIVE ADVERSARIAL TRAINING
Anonymous authors Paper under double-blind review
ABSTRACT
Adversarial training can improve the robustness of neural networks. Previous ad- versarial training methods focus on single training strategy and do not consider the collaboration between different training strategies. In this paper, we find dif- ferent adversarial training methods have distinct robustness for sample instances. For example, an instance can be correctly classified by a model trained using stan- dard adversarial training (AT) but not by a model trained using TRADES, and vice versa. Based on this phenomenon, we propose a collaborative adversarial training framework to improve the robustness of neural networks. Specifically, we simul- tancously use different adversarial training methods to train two robust models from scratch. We input the adversarial examples generated by each network to the peer network and use the peer network's logit to guide its network's 's training. Collaborative Adversarial Training (CAT) can improve both robustness and accu- racy. Finally, Extensive experiments effectiveness of our method. CAT achieved new state-of-the-an robustness with- out using any additional data on CIFAR-I0 10 under the Auto-Attack benchmark
CIFAR-10 and CIFAR-100 validated the
INTRODUCTION
With the development of deep learning. Deep Neural Networks (DNNs) have been applied to various fields, such as image classification (He et al. 2016), object detection (Redmon et al. 2016)) semantic segmentation (Pal & Pal 1993), etc. And state-of-the-art performance has been obtained. But recent research has found that DNNs are vulnerable to adversarial perturbations (Goodfellow et al. 2014). A finely crafted adversarial perturbation by malicious agent can easily fool the neural network. This raises security concerns about the deployment of neural networks in saccurty-criticoe areas such as Autonomous driving (Chen et al. 2019) and medical diagnostics (Kong et al. 2017).
To cope with the vulnerability of DNNs, different types of methods have been proposed to improve the robustness of neural networks, including adversarial training (Madry et al.] (017), defensive distillation (Papernot et al. 2016), feature denoising (Xie et al.. 2019) and model pruning (Madaan let al. 2020 Among them, Adversarial Training (AT)) is the most effective method improve adversarial robustness. AT can be regarded as type of data augmentation strategy that trains neural networks based on adversarial examples crafted from natural examples. AT is usually formulated as a min-maxinm mii-mmxxmiizatin problem, where the inner maximization generates adversarial examples, while the outer minimization optimizes the parameters of the model based on the adversarial examples generated by the inner maximization process.
The previous methods have focused on how improve the model's adversarial accuracy, focusing only on the numerical improvement, but not on the characteristics of the different methods. We ask: Do different adversarial training methods perform the same for different sample instances? We an- alyzed different adversarial training methods (taking AT (Madry et al., 2017) and TRADES (Zhang) cct al. 2019) as examples) and found that different methods behave difterently for sample instances, illustrated in Figure [ Specifically, for the same adversarial example, the network trained by AT can classify correctly, while the network trained by TRADES misclassifies. Similarly, some examples can be correctly classified by the network trained by TRADES, but not by the network trained by AT. That is, although AT and TRADES have the same numerical adversarial accuracy. they behave differently for sample instances. This raises the question:
Do two networks learn better if they collaborate?
attack

---

Under review as a conference paper at ICLR 2023
10000, classification result by AT
1
10000, classification result by TRADES
10000, Intersection result
Figure 1: Classification results of different adversarial training methods on sample instances. The first row is the classification result of the model trained by AT and the second row is the classifica- tion result of the model trained by TRADES. means correct classification and 0 means incorrect classification. 10000 is the size of the CIFAR-10 10 test set. The third row is the result of correct clas- sification by both AT and TRADES, and the result shown in red. can be seen that the models trained by different methods perform differently sample instances.
Based on this observation, we propose Collaborative Adversarial Training (CAT) framework to improve the robustness neural networks. Our framework is shown in Figure 2 simultaneously train two deep neural networks separately using different adversarial training methods. At the same time, the adversarial examples generated by this network are input to the peer network to obtain the corresponding logit. Then the logit obtained the peer network is used to guide the learning of this network together with its own adversarial training objective function. We expect improve the robustness of the neural network by allowing peers learn from each other in this collaborative learning way. Extensive experiments on different neural networks and different datasets demonstrate the effectiveness of our approach. CAT achieved new state-of-the-as robustness without any additional synthetic data or real data on CIFAR-10 under the Auto-Attack benchmark.
Specifically,
In summary, our contribution is threefold as follows.
We find that the models obtained using different adversarial training methods have different representations for individual sample instances.
We propose novel adversarial training framework: collaborative adversarial training. CAT simultaneously trains neural networks from scratch using different adversarial training methods and allows them to collaborate to improve the robustness of the model.
We have conducted extensive experiments on a variety of datasets and networks, and eval- uated them state-of-the-arr attacks. We demonstrate that CAT substantially improve the robustness of neural networks and obtain new state-of-the performance without any additional data.
RELATED WORK
2.1
ADVERSARIAL ATTACK
Since Goodfeloow al. (2144) discovered that DNNs vulnerable adversarial examples, large number of works have been proposed to craft adversarial examples. Based on the accessibility to the knowledge of the target model, it be divided into white-box attacks and black-box attacks. White-box attacks craft adversarial examples based on the knowledge of the target model, while black-box attacks agnostic the knowledge of the target model.
White-box Attack: Goodfellow et al. (2014) proposed FGSM to efficiently craft adversarial ex- amples, which be generated in just step. Later FGSM extended different iterative
2

---

Under review as a conference paper at ICLR 2023
attack methods. I-FGSM performs FGSM iteratively with small step size. Madry al. (2017) proposed PGD to generate adversarial examples, which is the most efficient way of using the first- order information of the network. Dong et al. (2018) combines momentum into the iterative process to help the model escape from local optimal points. And the adversarial examples generated by this method are also more transferable. Boundary-based attacks such as deepfool (Moosavi-Dez/oolli ett al., (216)) and CW (Carlini & Wagner, (017) also make the model more challenging. Recently, the ensemble approach of diverse attack methods (Auto-Attack,, consisting of APGD-CE (Croce & Hein 2020b), PGDD-DRR (Croce & Hein, 2020b), FAB (Croce & Hein 2020a) and Square Attack (Andriushchenko et al. 2020), became a benchmark for testing model robustness.
(20g))
Due to the similarity of the model structure, the adversarial examples generated on the surrogate model can be transferred to fool the target model. There are many ways to explore the transferability of adversarial examples for black-box attacks. (oong et al. (2018) combines mo- mentum with an iterative approach to obtain better transferability. Scale-invariance (Lin et al. 2019) boosts the transferability of adversarial examples by transforming the inputs on multiple scales.
Black-box Attack:
2.2 ADVERSARIAL ROBUSTNESS
Adversarial attacks present a significant threat to DNNs. For this reason, many methods have been proposed to defend against adversarial examples, including denoising (Xie al. 2019). adversarial training (Madry et al. (017), data aumentation (Rebuffi et al. (222).. and imput purification (Naseer et al. 2020). ANP (Madaan et al. 2020) finds the vulnerability of latent features and uses pruning to improve robustness. Madry uses PGD to generate adversarial examples for adversarial training, which is also the most effective way defend against adversarial examples. A large body of work uses new regularization or objective functions to improve the effectiveness of standard adversarial training. Kannan al. (2018) uses Adversarial logit pairing to improve robustness by encouraging the logits of normal and adversarial examples to be closer together. TRADES (Zhang et al. 2019) uses KL divergence to regularize the output of adversarial and pure examples.
2.3 KNOWLEDGE DISTILLATION
Knowledge distillation (KD) is commonly used for model compression and was first used by hin- ton (Hinton et al., 2015) to distill knowledge from well-trained teacher network student net- work. KD can significantly improve the accuracy of student models. There have been many later works to improve the effectiveness of KD (Romero et al. 2014). In recent years, KD has been ex- tended to other areas. Goldblum et al. (2020) analyzes the application of knowledge distillation to adversarial robustness and proposes ARD to transfer knowledge from a large teacher model with better robustness to a small student model. ARD produce student network with better robust- ness than training from scratch. In this paper, we propose a more effective collaborative training framework improve the robustness of the network.
PROPOSED METHOD
3.1 MOTIVATION
We investigated the performance of the robust models obtained by different training methods on sample instances. We found that different models perform differently on sample instances: for some samples, the model trained by correctly classifies AT (Madry et al. 2017). while the model trained by TRADES (Zhang et al., 2019) misclassifies. Similarly, for some samples, the network trained by TRADES can classify correctly, while the network trained by AT can misclassify. A straightforward conclusion can be drawn that the networks trained by different methods master different knowledge, although their accuracy values are about the same. So can we use the knowledge learned from these two networks to improve the robustness of neural networks? A simple idea is to let two networks that master different knowledge learn collaboratively. For this purpose, adversarial training. CAT improves the robustness of neural networks by making the knowledge of both networks interact during the training procedure. And the framework is illustrated in Figure 2
propose collaborative
3

---

Under review as a conference paper at ICLR 2023
fg
A Adversarial attack f
logits
knowledge interaction
adv
model f
clean
gr
Iu
A dvverarril attack
logits
adv
model g
Figure 2: The framework of CAT, performing adversarial training collaboratively. Given a batch of natural samples, the two networks and g are attacked separately to generate adversarial examples and v. Then and v are fed into both networks to obtain the corresponding logits. We then the logits obtained from the peer networks to guide the learning of its network, i.c., Iu + fa: fu + Dv-
3.2 COLLABORATIVE ADVERSARIAL TRAINING (CAT)
We take the methods of AT and TRADES as an example to introduce collaborative adversarial training. We first briefly introduce the training objective functions of AT and TRADES and then introduce CAT detail.
Adversarial training is defined min-maximization problem, and Madry et al. (0117) proposes to use PGD for adversarial training. That is, PGD used to generate adversarial examples for the internal maximization process, while external minimization uses the internal PGD-generated adver- sarial examples and the ground-truth label to optimize the model parameters. AT is formulated
as:
min E (xa)) (arg max L(f (x)dr) 8)).
ade EAT
8.
(2)
where Datta is the training data distribution, and y are training data and corresponding label samples from Danta- Jo is neural network parameterized by 0. is the standard cross-entropy loss the adversarial perturbations generated by PGD. Follwing
used in image classification tasks.
previous study, is bounded by loo-
Neural Networks trained by AT can obtain a certain level of robustness, with compromises on the accuracy of natural samples. For this purpose, TRADES uses a new training objective function for adversarial training. Formulated as:
min ((a.y))Daats L(gh
TRADES (I), + AD TRADES (x), 96'
TRADES
ade (ETRADES)),
(3)
where add the adversarial data corresponding to natural data r and y is the true label. L is the cross-entropy loss in classification task. DxL is the KI divergence to pushing natural logits and adversarial logits together. is a trade-off parameter.
CAT expects to improve robustness by letting neural networks trained by different methods change knowledge information, i.e., collaborative adversarial learning. As illustrated in Figure the logit of peer network to guide the learning of this network. Specifically, we input the adversarial data crafted by the network trained by AT into the network trained by TRADES to get the corresponding logit. Then use the logit obtained by the network trained by TRADES to guide the training of the network trained by AT. The formulation goes to:
La DKI((** (wodv) TRRDDS
(odd)..TRADES
(1
(4)
4

---

Under review as a conference paper at ICLR 2023
where DKz is KL divergence used to compute the relative entropy, the same in TRADES, fAT is the network trained with AT and g is the network trained with TRADES, 9 TRAADES represents that we take the logit obtained by network trained by TRADES as a constant. ædd is the adversarial data generated by f f with PGD function based on natural example x.
AT
Similarly, to make the two networks learn collaboratively. We need to feed the adversarial sam- ples generated by the TRADES network to the AT network to obtain the corresponding logit. And then the logit obtained by the peer network is used to guide the training of the network trained by TRADES. The loss is formulated as:
L DKL (g (1 (ETRADES), fAT (ETRADES))
(5)
ado T TRADES is the adversarial example crafted by network trained by TRADES use KL divergence function. fat (*TRADES) represents that we take the logit obtained by network trained by AT as
constant.
It is not enough to let the two networks learn from each other in this way. Real class labels are needed to guide them. For this purpose, we combine the respective training objective functions of the two networks and the mutual learning objective function to guide the learning of the networks together. Therefore, the training objective function for collaborative adversarial training based on AT and TRADES is:
Ltotal = aLTRDES + (1- + BLAT + (1 (1- L,,
(6)
where and 3 is the trade-off parameter to balance the guidance of peer logit and the original training objective function. LARADES is the training objective of TRADES defined Equation ( And LT is the training objective of AT defined in Equation ( The first two items in Equation ( are used to train model g and the last two items are used to train model f. due to we take the peer logit constant.
The decision boundaries learned by different adversarial training methods are different. Under the constraint of peer logit, i.e., Equation (4) and Equation (5). the two networks trained by different methods continuously optimize the classification decision boundaries in the process of collaborative learning. Finally, both networks learn better decision boundaries than learning alone obtain better robustness.
Our collaborative adversarial learning is a generalized adversarial training method that be used with any two adversarial methods. Generally, CAT can use any number of different adversarial training methods for collaborative learning. Results of CAT with three adversarial training methods are delayed to Appendix.
4 EXPERIMENT RESULTS
In this section, we conduct extensive experiments on popular benchmark datasets to demonstrate the effectiveness and performance of CAT. First, we briefly introduce the experiment setup and implementation details of CAT. Then, ablation studies are done to choose the best hyperparameters and CAT methods. Finally, according to the best CAT methods, we report the white-box and black- box adversarial robustness on two popular benchmark datasets. All images values are scaled into [0,1], and all our experiments are run on a single NVIDIA GeForee GTX 1080Ti.
We used two benchmark datasets, including 10 (Krizhevsky et al. 2009) and CIFAR-100 (Krizhevsky et al., 2012). CIFAR-10 has 10 classes. For each class, there are 5000 im- ages for training and 1000 images for test. And for CIFAR- 100, there are 100 classes, and similarly for each class, there are 500 images for training and 100 images for test. Both datasets are widely used for training and testing adversarial robustness. The image size in both datasets is 32 x 32.
Datasets:
Training setup: Our overall training parameters refer to Madaan et al. (020)). Specifically, we use SGD (momentum 0.9, batch size 128) to train ResNet18 for 200 epochs the CIFAR-10 dataset with weight decay 5e-4 and initial learning rate 0.1 which is divided by 10 at 100-th and 150-th epoch, respectively. RandomCrop and RandomHorizontalFiin the internal maximization process, we use PGD in adversarial attack to solve, with random start, step size 2.0/255, and perturbation size 8.0/255. We use X 100 in all experiments. The experimental
used data augmentation. For
5

---

Under review as a conference paper at ICLR 2023
M4.3
10:00
3441
10:10
00.0
14.10
09.03
18.00
RAD BLD
RUDE
0856 OUE 0U2 RDD
65
45
Figure 3: Adversarial robustness using different hyperparameters under the trades-at for collabora- tive adversarial training framework. From left to right, the results of Clean acc, FGSM acc, PGD acc, and AA acc are shown. model f and model represent the results of using TRADES and AT the CAT training framework, respectively.
Table 1: The white-box robustness results (accuracy (%)) of different CAT methods on CIFAR- 10. We report the results of best checkpoint and last checkpoint. The best results are marked using boldface. Two ResNet-18 networks are used in our CAT framework.
Best Checkpoint
Last Checkpoint
Method
Clean
FGSM
PGD20
CW
AA
Clean FGSM PGDz0
CW oc
AA
83.74
59.69
54.44
52.60
50.52
84.45
60.03
53.01
52.01
49.30
AT-ADESS
83.55
59.78
54.52
52.58
50.86
84.12
59.69
52.82
51.88
49.39
84.66
59.94
53.11
51.90
49.74
84.71
59.84
50.77
50.53
47.80
AT
85.21
60.21
53.02
52.13
49.96
85.27
59.75
51.10
50.69
47.91
83.91
59.76
54.44
52.56
51.02
84.67
59.85
52.51
51.43
49.31
CATTRADES
84.75
59.76
54.17
52.72
50.85
85.27
59.82
52.56
51.83
49.64
parameters of ResNet18 in CIFAR-100, WideResNett33-110 in CIFAR-10 and CIFAR-100 the same as described above.
We report the clean accuracy on natural examples and the adversarial accuracy on adversarial examples. For adversarial accuracy, we report both white-box and black-box. We follow the widely used protocols the adversarial research field. For the white-box attack, we consider three basic attack methods: FGSM (Goodfellow et al. 2014). PGD (Madry et al.) (017), and CW (Carlini & Wagner (017)) optimized by PGD20- and a stronger ensemble attack method named AutoAttack (AA) (Croce & Hein (0200b) For the black-box attacks, consider both transfer-basee attacks and query-based attacks.
Evaluation setup:
Due to the fact that CAT uses two methods for collaborative training, report the results for both networks. For example, the method collaborative adversarial training using TRADES and ALP can be denoted as CATTRADES-ALP.
4.1 ABLATION STUDY
4.1.1 HYPERPARAMETER:
CAT improves adversarial robustness through the learning collaboration, which requires both the knowledge of peer networks and the guidance of the ground truth label. The balance of original methods and the collaborative is a trade-off by a hyperparameter a and 3. For simplicity, we set equals 3 in method and experiment with different trades-off parameters. We test various a values varying from 1/50 to 1/5. The experiment results illustrated in Figure From the figure a is too high, i.e., little knowledge is drawn from the peer network, the effect is about the same training with AT and trades alone. If too low, i.e., overly focused the knowledge from the
experiment. We execute collaborative training by TRADES and AT the base
conclude that

---

Under review as a conference paper at ICLR 2023
peer network, The network is also not very robust. Since Auto-Attack is currently the most powerful integrated attack method, we choose hyperparameters a and 3 based primarily on the robustness of the network against AA. In the following experiments, set B 1/20.00 by default.
4.1.2 DIFFERENT CAT METHODS:
As described in Section 3.2 any two adversarial training methods can be incorporated into the CAT framework and learned collaboratively. Considering that different adversarial training methods have distinct properties, the performance of different CAT methods may also vary. For this reason, we consider three collaborative adversarial training methods, AT-TRADES, AT-ALP, and TRADES- ALP, respectively. Table shows the performance of the different CAT methods. All CAT training methods obtained good robustness against four attack methods. We again mainly consider the per- formance of auto-attack and choose TRADES-A ALP as the base method for our CAT.
4.2
ADVERSARIAL ROBUSTNESS
4.2.1 WHITE-BOX ROBUSTNESS
For FGSM, PGD, CW AA, the attack perturbations are all 8.0/255 and the step size for PGD, o are 2/25, with 20 iterations. According the way of reporting in previous papers, we report both the best checkpoint and the last checkpoint of the training phase. The best checkpoint result of the training phase selected based on the model's PGD defense results for the test dataset (attack step size 2.0/255, iteration number 10, perturbation size 8.0/255).
Table Table shows the adversarial accuracy of the networks trained by different methods on two datasets CIFAR-10 10 and CIFAR- 100 against the four attacks. We also report the accuracy of the model for natural examples. From the table, we can obtain the following conclusions: (1) our method obtains good robustness against all four attacks on both datasets. For example, for the strongest AA attack method, CAT can obtain 2% improvement. (2) our method obtains high adversarial robustness while ensuring accuracy for natural examples. Although there is still big gap compared 94.65 of the standard training strategy, there is a nearly 1% improvement in the accuracy of the natural examples compared to the other three methods. (3) The robustness of both networks is significantly improved in the CAT training framework. The robustness of both networks is higher than separately trained ones. (4) The difference in accuracy between the two networks trained in the CAT framework is smaller than separately trained ones, which demonstrates that the two networks do well in collab- orative learning. For example, for the CIFAR-IO 10 dataset, the difference in robustness between the two networks of TRADES-ALE against AA in the CAT training framework is 0.17%, while the difference 0.73% under separate training. The same conclusion can be drawn from CIFAR-100 datasets.
4.2.2 BL.ACK-BOX ROBUSTNESS
For black-box attacks, we consider both transfer- based attacks and query-based attacks. For the transfer-based attack, trained with the same parameters as described in Section 4 First, we perform the attack on the surrogate model generate adversarial examples and then transfer the adversarial examples the target network to get the robustness of the target network. Here, we consider four attacks: FGSM, PGD20, PGD and CW so. with the same attack parameters as Section 4.2.1 For query-based attacks, we consider Square attack, which is an efficient black-box query- based attack method. Table Table shows the results, and our method CAT achieves the best performance.
the standard adversarial training of ResNet-34 as the surrogate model,
4.3 COMPARISION TO SOTA
We use two WideResNet 34-10 10 (Zagoruyko & Komodakis 2016) networks for collaborative adver- sarial training. one using the TRADES (Zhang et al. 2019) training method and the other using the ALP (Kannan et al. 2018) training method. Table shows the accuracy of the different methods for natural examples and the robustness against Auto-Attack. For some methods, we also report the results of WideResNNt-3Ae 30- All results are from the original paper. From the table we can conclude that the robustness of both networks trained with CAT outperforms the previous methods, demonstrating the -fhh--r-- performance obtained by CAT.
7

---

Under review as a conference paper at ICLR 2023
Table The white-box robustness results (aceuracy (%) CAT on CIFAR-1O and CIFAR-I00, We report the results of best checkpoint and last checkpoint. The best results are marked using boldface. Two ResNet-18 networks are used in our CAT framework. TRA-ALP short for TRADES-ALP due to the limitation.
Best Checkpoint
Last Checkpoint
Dataset
Method
Clean
FGSM PGD20 CW
AA
Clean FGSM 2GD20 CW nc
AA
Natural
94.65
19.26
0.0
0.0
0.0 94.65 19.26
0.0
0.0
0.0
AT
82.82
57.57
51.76
50.05
47.55 84.53 53.90
43.56
44.19
41.57
CIFAR-10
TRADES
83.17
59.22
52.63
50.79
49.21
83.04
57.46
49.81
49.01
47.03
ALP
83.85
57.20
51.88
50.11
48.48
84.64
55.35
44.96
44.54
42.62
83.91
59.76
54.44
52.56
51.02
84.67
59.85
52.51
51.43
49.31
CATTRA-AR ALP
84.75
59.76
54.17
52.72
50.85
85.27
59.82
52.56
51.83
49.64
Natural
75.55
9.48
0.0
0.0
0.0 75.39
9.57
0.0
0.0
0.0
AT
57.42
31.90
28.78
27.27
24.88
57.34
26.77
21.24
21.50
19.59
CIFAR-100
TRADES
56.98
31.72
29.04
25.30
24.23
55.08
30.40
26.81
24.78
23.68
ALP
61.01
31.41
26.78
25.68
23.51
58.4
27.97
22.63
21.87
20.42
61.31
35.83
33.09
29.17
27.17
61.78
35.84
32.76
29.48
27.29
CATTRA-ALA
62.53
36.05
32.92
29.16
26.90
62.52
35.79
32.51
29.24
26.73
Table 3: The black-box robustness results (accuracy (%)) of CAT CIFAR-10 and CIFAR-100. We only report the results of the best checkpoint. The best results are marked using boldface. Two ResNet-18 networks are used in our CAT framework. TRA-ALP is short for TRADES-ALP due to the limitation.
CIFAR-10
CIFAR-100
Method
FGSM
PGD20
PGD40 CW oc Square
FGSM
PGD20
PGD40
CW nc: Square
AT TRADES
64.54
61.70
61.57
61.42
56.16
39.15
37.56
37.46
38.85
30.11
65.63
63.57
63.57
63.23
55.97
39.06
37.73
37.79
38.86
28.72
ALP
64.95
62.38
62.32
61.78
55.78
40.29
38.97
38.85
40.03
29.85
65.73
63.65
63.78
63.24
57.55
42.26
40.76
40.76
41.78
33.04
CATTRA- ALP
ALP
66.06
63.91
63.88
63.26
57.95
42.81
41.55
41.42
42.42
33.30
4.4 COMPARISION TO KD-AT
In general, the robustness of large models is higher than that of small models under the same training settings. For example, WideResNeet33-100 (Zagoruyko & Komodakis 2016) trained by TRADES can achieve 53.08% robustness against AA, while the accuracy of ResNet- is only 49.21% %. From this motivation, some authors have used knowledge distillation distill the robustness of large mod- els to small models and obtained good results. Considering that CAT also involves the collaborative training of two models, we compare CAT with the KD-AT method. To give fair comparison, unlike the previous experiments using two same-size networks for CAT, we use two different-size networks for CAT training and then report the accuracy of both the two networks, i.e., WideResNeet-4-111 and ResNet- 18. Note that, unlike the KD method where the teacher is trained in advance, our CAT is trained with both the large model and the small model simultaneously, so there is no concept of teacher and student. In another word, we extend previous off-line distillation an online way and achieve better performance.
Table shows the results of Knowledge Distillation-AT and CAT, where ARD (Goldblum et al. 2020). IAD (Zhu et al. (02I). and RSLAD (Zi et al., (02T)) are trained by KD-AT using TRDES trained Wide network teacher (second row in Table 5 CAT was collaboratively
8

---

Under review as a conference paper at ICLR 2023
Table 4: Quantitative comparison with the state-of-the-art adversarial traing methods. Two
WideResNott33-10 networks are used in our CAT framework.
Method
Architecture
Nat
AA
Bag of Tricks for AT (Pang et al. 2020a)
WideRessNet33410
86.28
53.84
HE* (Pang et al. 2020b
WideRessNet33-220
85.14
53.74
Overfitting in AT *h Rice et al. Overfitting in AT RRice et al.
2020
WideRessNet33-200
85.34
53,42
2020)
WideRResNet-34-10
85.18
53.14
Self-Adaptive Training (Huang et al.
2020
WideReesNet334100
83.48
53.34
FAT (Zhang al. 2020)
WideRessNet-3410
84.52
53.51
TRADES (Zhang et al. [2019)
WideResNNe134-10
84.92
53.08
LLR QQn
2019)
WideResNNet40-88
86.28
52.84
LBGAT+TRADES (a = U)*
(Cul et al.
202
WideResNNet33-200
88.70
53.57
LBGAT+TRADES (a- 0)
Cul et
2021
WideResNNet-44-11
88.22
52.86
LBGAT+TRADES (a 6)
Cul et al.
2021
WideResNett344110
81.98
53.14
LAS-AT (Jia et 2022
86.23
53.58
LAS-AWP (Ja a.
2022)
WideResNeet34-10
87.74
55.52
86.22
54.11
CATTRADES-ALP
WideResNNet-34-10
86.51
54.20
86.74
56.43
CATTRADES-ALP +AWP
WideResNNtt34-100
87.01
56.61
Table 5: Quantitative comparison with the state-of-the-art Knowledge-Distillatio AT methods. We use WideResNeet33-100 and ResNet-18 networks in the CAT framework for a fair comparison.
Method
Architecture
Distillation
Role
Clean
AA
TRADES (Zhang et al.
2019
WideResNet-34-10
teacher
84.92
53.08
ARD (Goldblum al.
2020
ResNet-18
/
student
83.93
49.19
IAD Zhu et al. 2021
ResNet-18
/
student
83.24
49.10
RSLAD Zi al.
ResNet-18
/
student
83.38
51.49
WideResNet-34-10
X
85.04
55.04
CÄTTRADES-ALA
ResNet-18 X
84.39
50.
trained using networks of different sizes. It be that method obtains good results and also obtains high clean accuracy.
5 CONCLUSIONS
In this paper, we first analyze the properties of different adversarial training methods and find that networks trained by different methods perform differently on sample instances. ie.. the network can correctly classify samples that are misclassified by other networks. Based on this observation, we propose collaborative adversarial training framework, which aims to use the knowledge learned by peer networks to guide its learning. Thus improving the robustness of both networks. Exten- sive experiments on different datasets and different networks demonstrate the effectiveness of our approach, and state-of-the-an performance is achieved. Broadly, CAT can be easily extended to mul- tiple networks for collaborative adversarial training, e.g, three peer networks. However, our method has some limitations the same the previous method that adversarial training consumes training resources, and CAT has to train two networks simultaneously. Work on how to accelerate CAT will be carried out in the future.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Maksym Andriushchenko, Francesco Croce, Nicolas Flammarion, and Matthias Hein. Square at- tack: query-efficient black-box adversarial attack via random search. In ECCV, 2020.
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 ieee symposium on security and privacy (sp), 2017.
Jianyu Chen, Bodi Yuan, and Masayoshi Tomizuka. Model-free deep reinforcement learning for urban autonomous driving. arXiv preprint 090::09944,0503,, 2019.
Francesco Croce and Matthias Hein. Minimally distorted adversarial examples with fast adaptive boundary attack. In JCML, 2020a.
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In ICML,, 2020b.
Jiequan Cui, Shu Liu, Liwei Wang, and Jiaya Jia. Learnable boundary guided adversarial training. In ICCV, 2021.
Yinpeng Dong, Fangzhou Liao, Tianyu Pang. Hang Su, Jun Zhu, Xiaolin Hu, and Jianguo Li. Boost- ing adversarial attacks with momentum. In CVPR, 2018.
Micah Goldblum, Liam Fowl, Soheil Feizi, and Tom Goldstein. Adversarially robust distillation. In AAAI, 2020.
lan J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572, 2014.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. CVPR, 2016.
Geoffrey Hinton, Oriol Vinyals, Jeff Dean, et al. Distilling the knowledge in a neural network. arXiv preprint arXiv: 1=X3.0/503.0253/ 2015.
Andrew Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam. Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprini 1:XXv:/704.044: 2017.
Lang Huang, Chao Zhang, and Hongyang Zhang. Self-adaptive training: beyond empirical risk minimization. NeurlPS, 2020.
Xiaojun Jia, Yong Zhang, Baoyuan Wu, Ke Ma, Jue Wang, and Xiaochun Cao. Las-at: Adversarial training with learnable attack strategy. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 13398-13408, 2022.
Harini Kannan, Alexey Kurakin, and Ian Goodfellow. Adversarial logit pairing. arXiv preprint arXiv:1803.000 1803.06373. 2018.
Bin Kong, Xin Wang, Zhongyu Li, Qi Song, and Shaoting Zhang. Cancer metastasis detection via spatially structured deep network. In IPMI, 2017.
Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton. Imagenet classification with deep convo- lutional neural networks. In NeurlPS, 2012.
Alex Krizhevsky et al. Learning multiple layers of features from tiny images. Tech Report, 2009.
Jiadong Lin, Chuanbiao Song, Kun He, Liwei Wang, and John E Hoperoft. Nesterov accelerated gradient and scale invariance for adversarial attacks. In ICLR, 2019.
Divyam Madaan, Jinwoo Shin, and Sung Hwang. Adversarial neural pruning with latent vulner- ability suppression. In ICML, 2020.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv: 1XX.:170060668. 2017.
10

---

Under review as a conference paper at ICLR 2023
Seyed-Mohsen Moosavi-Dezfoole Alhussein Fawzi, and Pascal Frossard. Deepfool: simple and accurate method to fool deep neural networks. In CVPR, 2016.
Muzammal Naseer, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, and Fatih Porikli. A self-supervised approach for adversarial robustness. In CVPR, 2020.
Nikhil Pal and Sankar K Pal. A review on image segmentation techniques. PR, 1993.
Tianyu Pang, Xiao Yang, Yinpeng Dong, Hang Su, and Jun Zhu. Bag tricks for adversarial training. In ICLR, 2020a.
Tianyu Pang, Xiao Yang, Yinpeng Dong, Kun Xu, Jun Zhu, and Hang Su. Boosting adversarial training with hypersphere embedding. NeurIPS, 2020b.
Nicolas Papernot, Patrick McDaniel, Xi Wu, Somesh Jha, and Ananthram Swami. Distillation as defense adversarial perturbations against deep neural networks. In 2016 IEEE symposium on security and privacy (SP), 2016.
Chongli Qin, James Martens, Sven Gowal, Dilip Krishnan, Krishnamurthy Dvijotham, Alhussein Fawzi, Soham De, Robert Stanforth, and Pushmeet Kohli. Adversarial robustness through local linearization. NewrIPS, 2019.
Sylvestre-Alvise Rebuffi, Sven Gowal, Dan Andrei Calian, Florian Stimberg, Olivia Wiles, and Timothy A Mann. Data augmentation can improve robustness. Advances in Neural Information Processing Systems, 2021.
Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi. You only look once: Unified, real-time object detection. In CVPR, 2016.
Leslie Rice, Eric Wong, and Zico Kolter. Overfitting in adversarially robust deep learning. In ICML, 2020.
Adriana Romero, Nicolas Ballas, Samira Ebrahimi Kahou, Antoine Chassang, Carlo Gatta, and Yoshua Bengio. Fitnets: Hints for thin deep nets. arXiv preprint arXiv:1412.6550, 2014.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint 1rXiv:1409.1556 2014.
Cihang Xie, Yuxin Wu, Laurens van der Maaten, Alan L Yuille, and Kaiming He. Feature denoising for improving adversarial robustness. In CVPR, 2019.
Sergey Zagoruyko and Nikos Komodakis.
Wide residual networks. arXiv preprint
arXiv:1605.07146, 2016.
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael Jordan. Theoretically principled trade-off between robustness and accuracy. In ICML, 2019.
Jingfeng Zhang, Xilie Xu, Bo Han, Gang Niu, Lizhen Cui, Masashi Sugiyama, and Mohan Kankan- halli. Attacks which do not kill training make adversarial learning stronger. In ICML, 2020.
Jianing Zhu, Jiangchao Yao, Bo Han, Jingfeng Zhang, Tongliang Liu, Gang Niu, Jingren Zhou, Jianliang Xu, and Hongxia Yang. Reliable adversarial distillation with unreliable teachers. arXiv preprint
2021.
Bojia Zi, Shihao Zhao, Xingjun Ma, and Yu-Gang Jiang. Revisiting adversarial robustness distilla- tion: Robust soft labels make student better. ICCV, 2021.
11

---

Under review as a conference paper at ICLR 2023
30
40
30
20
CAT2
CAT2 TRADES
TRADES
ALP
ALP
10
25
50
75
100
125
150
175
200
120
130
140
150
160
170
160
190
200
Figure 4: Test robust accuracy AT, ALP, TRADES, and CAT with ResNet-18 on CIFAR-10 datasets. CAT can alleviate the problem of overfitting.
confusion, matrix AT and ALP
contusian matri of AT and TRADES
confusion matrix of ALP and TRADES
prodictio
preeectioon
predection
prediction
TRADES prediction
TRADES prediction
Figure 5: Confusion matrices ALP-prediction and AT-prediction, TRADES -eeDds--redittonn and AT- prediction, ALP-prediction and TRADES-predictinn with ResNet-18 CIFAR-I0 datasets. Con- fusion exists in different defenses methods.
A ALLEVIATE OVERFITTING
Overfitting in adversarial training is first proposed by Rice et al. (2020), which shows the test ro- bustness decreases after peak robustness. And overfitting one of the most concerning problems in adversarial training. Here, we investigated the overfitting problem in CAT. Results are illustrated Fig. Our CAT can alleviate the overfitting probalem that widely occurs in previous adversarial methods. Moreover, the performance for CAT has not saturated, and high performance is expected with longer epoch training.
B CONFUSION MATRIX
To better present motivation for CAT, we show the confusion matrices of different methods. In Figure [5] three confusion matrices shown, i.c., ALP-prediction and AT-prediction, TRADES- prediction and AT-prediction, ALP-prediction and TRADES -rADDEtprreiition. Confusion exists in all three matrices, especially for blocks from class class 7. The conclusion is correspondence to Figure The prediction intersection is reported Tab.
C CORRELATION BETWEEN DISCREPANCY AND CAT PERFORMANCE
In this section, we analyze the correlation between the discrepancy of different adversarial training methods and their adversarial robustness after CAT. First, compute the prediction intersection
12

---

Under review as a conference paper at ICLR 2023
Table 6: The correlation between white-box robustness results (accuracy (%) and prediction dis- crepancy of different CAT methods on CIFAR-10. Two ResNet-18 networks are used in our CAT framework.
Method
PGD
Prediction intersection
CAT
53.11
81.02%
CATTRADES-ALP
54.44
78.95%
AT
54.52
77.46%
Table 7: The white-box robustness results (accuracy (%)) of CAT CIFAR-10. We report the results of the best checkpoint. The best results are marked using boldface. Two VGG-16 networks used in our CAT framework. TRA-ALP is short for TRADES-ALE due to the limitation.
Dataset
Method
Clean
FGSM
PGD
CW
AA
AT 78.31
53.11
48.39
46.32
43.69
TRADES
79.11
53.75
48.28
45.93
44.63
CIFAR-10
ALP
80.23
52.18
47.30
45.23
43.68
79.23
54.47
49.43
47.19
45.48
CATTRA-ALP
80.12
54.48
48.30
47.23
45.33
between different methods, formulated as:
interscction
Σ I( (x:), TRADES (x:))) ZEDD
where D is the datasets, and is indicator function, which is I when f (x:) g TRADES (I;) and 0 otherwise. The smaller this value is, the greater the discrepancy. Then, we report adver- sarial robustness of CAT trained by different settings. Results reported in Tab. 6 A conclusion can be drawn that the greater the discrepancy is, the higher the adversarial robustness after CAT.
D MORE EXPERIMENTAL RESULTS
D.1 VGG-16 RESULTS ON CIFAR-10
The white-box robustness of VGG-16 (Simonyan Zisserman, 2014) models trained using AT, ALP, TRADES, and CAT are reported in Tab. 7 The setting for VGG-16 is the same ResNet-18 models, i.e., a = 1.0/20 and 3 = 1.0/20. The improvement for CAT with VGG-16 models is as consistent with ResNet-18 models. CAT can boost model's robustness under AutoAttack with 2.0 points.
D.2 MOBILENET RESULTS ON CIFAR-10
Similar to the above VGG- 16 models, we report the while-box robustness of MobileNet (Howard et al. 2017) CIFAR-10 datasets under various attacks in Tab. The experiment set is the same as the previous setting. We can see that our CAT brings 1.0 improvement for MobileNet under AutoAttack, which is the most powerful adversarial attack method.
D.3 RESNET-18 RESULTS ON TINY-IMAGENET
For the large-scale ImageNet dataset, just all the baseline methods did not report the results, we are also unable to evaluate on ImageNet due to the very high training cost. To investigates the performance of our CAT in large datasets, conduct the experiment of white-box robustness of
13

---

Under review as a conference paper at ICLR 2023
Table The white-box robustness results (aceuracy (%)) CAT on CIFAR-10. We report the re- sults of the best checkpoint. The best results marked using boldface. Two MobileNet networks are used in our CAT framework. TRA-ALP short for TRADES-AUE ALP due to the limitation.
Dataset
Method
Clean
FGSM
PGD 20
CW nc
AA
AT 76.24
50.27
44.99
43.03
40.10
TRADES
75.84
49.65
45.26
42.04
41.08
CIFAR-10
ALP
79.46
50.14
43.95
42.08
40.01
80.14
51.25
46.38
44.24
42.20
CATTRA-ALP
79.86
51.28
46.22
44.05
42.16
Table 9: The white-box robustness results (accuracy (%) of CAT on Tiny-ImageNNe. We report the results of the best checkpoint. The best results are marked using boldface. Two ResNet- 18 networks are used in our CAT framework. TRA-ALP is short for TRADES-AI due to the limitation.
Dataset
Method
Clean
PGDs0
CW cc
AA
AT TRADES
43.98
19.98
17.60
13.78
39.16
15.74
12.92
12.32
Tiny-Imagenet
ALP
39.85
17.28
15.34
12.98
44.35
20.86
19.43
14.96
CATTRA-ALP
44.76
21.02
19.64
15.63
ResNet- 18 Tiny-ImageNet, which also is a widely used dataset in adversarial training. The results are shown in Tab. 9 Surprisingly, CAT shows impressive robustness on the large-scale dataset. The improvement as significant ResNet- in small datasets like CIFAR-10 and CIFAR-100.
D.4 CAT OF ONE MODEL WITH VARIOUS ATTACKS
The white-box robustness results (accuracy (%)) of CAT on CIFAR-1O. We report the results of the best checkpoint. The best results are marked using boldface. PGD-CW denotes one network trained by PGD and CW. TRA-ALP short for TRADES denoting two networks with TRADES and ALP. TRA-TRA is short for TRADES-TRADE TRADES and TRADES. AT-ALP-TRA short for ALP-ALL-TRADES.
Table 10:
denoting two networks with
Best Checkpoint
Dataset
Method
Clean
FGSM
PGD20
CW
AA
AT
82.82
57.57
51.76
50.05
47.55
CATPGD-OO cw
82.09
56.48
52.48
49.28
48.06
83.91
59.76
54.44
52.56
51.02
CATTRA-ALP
CIFAR-10
84.75
59.76
54.17
52.72
50.85
81.94
58.85
54.19
51.52
50.30
CATTRA-TRA
82.13
58.77
54.02
51.56
50.14
84.50
60.17
54.64
52.98
51.28
ATT
84.62
60.25
54.87
53.04
51.42
84.29
60.24
55.04
53.38
51.74
For CAT method, two networks and two different attack methods for each network to perform adversarial training. An interesting baseline is one network with two different attack methods. Therefore, use PGD and CW attack methods and one ResNet-18 as network.
14

---

Under review as a conference paper at ICLR 2023
The results reported in Tab. I0 (CATPGG--Ce entry). The improvement for this setting is not significant as the previous setting, but still, boosts the model's robustness against all four attacks.
D.5 CAT OF Two MODELS WITH SAME METHODS
Another interesting baseline is networks trained by the same adversarial training meth- ods, i.e., two ResNet-18 networks are both trained by TRADES. We denote this setting as CATTRADES-TRADEE
The results are reported in Tab. T0 The improvement for this setting is not significant the previous setting, but it still, boosts the model's robustness against all four attacks. However, the improvement is more significant than just using one network. conclusion can be drawn that two networks are important for CAT to achieve better adversarial robustness.
D.6 CAT OF THREE MODELS WITH THREE METHODS
Our CAT is a generalized method, which can use any number of different adversarial training meth- ods for collaborative learning. Here, we report the results of CAT trained with three adversarial training methods in Tab. AT AT--AL-TRADES The robustness improvement is more signif- icant than CAT trained with adversarial trained methods, which shows the generalizability of our CAT.
15


Reviews:
[{"review_id": "kFZXZfc4Wy", "reviewer": "Reviewer_df3y", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "Summary.\n\nThis paper is dedicated to developing improved adversarial training methods. The authors are inspired by the observations of prediction discrepancy from different adversarially trained models. Specifically, they introduce a collaborative adversarial training framework (called CAT) to improve the robustness. CAT inputs the adversarial example generated by each network to the peer network and uses the peer networks' logit to guide its training. Experiments are conducted on CIFAR-10 and CIFAR-100.", "summary_of_the_review": "Incremental ideas with insufficient studies.", "strength_and_weaknesses": "Pros.\n\n1. The paper is well-written and easy to follow.\n2. Robustness is evaluated by diverse adversarial attackers beyond PGD.\n\n\n\nCons. \n\n1. The collaborative training technique can be regarded as a \"soft data augmentation\" since it will use the input sample and associated predictions from the peer networks. Therefore, (a) the authors need to compare with data augmentation approaches in adversarial training like \"Data Augmentation Can Improve Robustness\"; (b) it is hard to say that the comparison in this paper is fair since CAT use at least 2 times resource cost than other approaches.\n2. Only small-scale datasets are considered like CIFAR-10 and CIFAR-100. It is insufficient to support the effectiveness of the proposed methods. Larger datasets like ImageNet are required.\n3. Meanwhile, the network backbones for evaluation are also limited. More architectures like VGG and MobileNet are needed.\n4. The provided method is motivated by the prediction discrepancy between different robustified networks. However, it is unconvincing between several important baselines are missing. For example, the prediction intersection results for the same AT method with different random seeds or different random start, are required. Whether the prediction difference between TRADES and AT is larger than the results between two AT runs?\n5. More ablations are needed. For instance, how about involving more attackers (e.g., FAT, CW, auto attack) in collaborative training? ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "dx9_BEzcIX", "reviewer": "Reviewer_JNu7", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes a Collaborative Adversarial Training (CAT) framework, which is based on the observation that robust models trained with different training methods have different performance on different instances, despite the models having similar accuracies. This gives rise to the proposed collaborative approach: Given the setup of two networks being trained with two different training methods, an adversarial sample generated by one network is fed to the other network to obtain the corresponding logit, which is then utilized to guide the learning of the first network. The evaluation is performed on two benchmark datasets CIFAR10 and CIFAR100 against a number of baselines to test the effectiveness of the approach in improving model robustness. The authors also compare their method to various knowledge distillation techniques which work on a similar principle.", "summary_of_the_review": "This work proposes a training regime to combine the benefits of different adversarial training algorithms, which is interesting and novel. However, there is little motivation on why this works, and how the trained models are different. Including some important ablations (mentioned above) can improve clarity on the impact of the proposed method. ", "strength_and_weaknesses": "Strengths-\n\n- The proposed approach is intuitive and simple to implement.\n- The work shows improved robustness compared to the baseline models trained non-collaboratively.\n- The paper is clear and well-written. \n\nWeaknesses/ Questions -\n\n- The proposed method involves higher complexity when compared to baselines - 2x computational cost, 2x memory overhead, more hyperparamaters. \n- Could the authors elaborate further on why and how CAT improves performance? How are these adversarial training methods different in the first place - and why does combining them result in improvements? \n- Could the authors share the confusion matrix of (correct and incorrect) predictions from TRADES and PGD-AT (or ALP)? This would better motivate the approach.\n- The best results on Robustbench are actually obtained by applying Adversarial Weight perturbation (AWP). Could the proposed method TRADES-ALP also be integrated with AWP? It would be useful to compare gains of AWP and no-AWP runs. \n- Could the authors share an ablation of TRADES-TRADES, where the same method (TRADES) is used for training both models?\n\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "YlNux05o_V", "reviewer": "Reviewer_a6Jb", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors observe a phenomenon that robust models adversarial trained by different methods (like AT and TRADES) have different reactions to the given input. Motivated by this, they proposed to perform collaboratively adversarial training to improve the robustness by training two neural networks from scratch.\n", "summary_of_the_review": "Pls see the above comments.", "strength_and_weaknesses": "Strength:\n\n1. The idea is straightforward, and the paper writing is easy to follow, which simply ensembles the knowledge from different adversarial training models.\n\n2. The experiments are extensive among a variety of datasets and networks, and the evaluation achieves state-of-the-art results.\n\n\nWeaknesses:\n\n1. The idea of the proposed method is quite lacks novelty. Similar ideas of collaboratively fusing knowledge have been proposed and studied for improving robustness [1][2][3].\n\n2. The paper does not theoretically analyse the benefits brought by fusing different adversarial training methods. The trivial experimental results do not clearly support their claims, the author seems to simply blend them together to get improvement without any further analysis.\n\n3. Is that necessary to train two different robust neural networks from scratch? Efficiency is the main challenge in adversarial training, and now the authors propose to scarify by doubling the time, making it worse. \n\n4. What if we train one neural network and fuse the knowledge by using different adversarial examples generated by different methods? This is a strong baseline that should be concluded for comparison.\n\n5. What is ALP? The abbreviation occurs suddenly in the experiment part without any reference or explanation.\n\n[1] Toward Learning Robust and Invariant Representations with Alignment Regularization and Data Augmentation\n\n[2] Improving adversarial robustness by learning shared information | Elsevier Enhanced Reader\n\n[3] Prior-Guided Adversarial Initialization for Fast Adversarial Training", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "MSliCqtyK3", "reviewer": "Reviewer_MzNC", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors made an observation that different adversarial defense strategies make different mistakes. Based on this observation, they proposed collaborative adversarial training, where simultaneously train two robust models. The objective of collaborative adversarial training is to minimize the symmetric KL-divergence between the logits of the first and second models. In the experiments, the authors showed improved robustness against AutoAttack on CIFAR-10 and CIFAR-100 datasets.", "summary_of_the_review": "The authors proposed a collaborative adversarial training method that combines two defense methods. The approach is interesting and marginally novel. It improves robustness upon TRADES and ALP baselines. However, the paper contains many typos, the empirical comparison does not include more recent SOTA methods, and some citations (such as ensemble adversarial training) are missing. ", "strength_and_weaknesses": "### Strengths\n\n- Proposed a framework for collaborative adversarial training, where two robust models are trained jointly.\n- Demonstrated promising experimental results on CIFAR-10 and CIFAR-100 benchmarks.\n\n### Weaknesses\n\n- The increased computational cost of training two robust models.\n- The lack of a theoretical analysis of the proposed method. It is not clear why the proposed objective will guide the training toward a more robust model compared to the single model, single attack, and single objective training.\n- The lack of more detailed experimental analysis and other baselines for comparison.\n\n### Questions\n- Can collaborative adversarial training be affected by catastrophic/robust overfitting?\n- Does collaborative adversarial training perform better than training one robust model with multiple attacks and/or multiple weighted objectives?\n- Can you include experiments with more than 2 defenses?", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject