Question: What is ICE’s advantage over the baseline methods when using a single model and dataset (i.e. the first set of rows in Table 2)?
Evidence: In this subsection, we perform GTA experiments on Cifar-100, which means we try to perturb Cifar-100 images to fool the target victim models. The models MobileNet-V3, VGG-16, ResNet-18, ResNet-34, SeResNet-26, and DenseNet-26 trained on Cifar-100 are used as victims to calculate the GTA success rate. All experimental results are reported in Table.
In the first experiment (the first row of Table), we use ResNet-18 and Cifar-10 as the source model and the source dataset, respectively, and conduct GTA on the testing images from Cifar-100. It is observed that among all baselines, FGSM performs the best. A possible underlying reason is that adversarial perturbations generated by multi-step gradient ascent tends to overfit the source model and source dataset. It can also be seen that the proposed ICE outperforms existing methods on the GTA problem. For instance, compared with FGSM, the average attack success rate on the six target models is improved by about 11.5%.
In the second experiment (the second row of Table), we consider the case where there are two source models -ResNet-18 and MobileNet-V1 trained on Cifar-10. Experimental results indicate that most of the baselines cannot leverage the additional source model MobileNet-V1 to boost their performances. In contrast, the proposed ICE can efficiently make use of the additional source model to improve its performances. For instance, by adding the MobileNet-V1 source model, the average attack success rate across 6 models of FGSM is decreased by about 0.7% while the success rate of ICE is improved by about 11.6%. In the third experiment, we use two ResNet-18 models trained on Cifar-10 and Tiered T 84 respectively as the source models, and use Cifar-10 and Tiered T 84 as the source datasets. It is interesting that the baselines' performances in this experiment are commonly worse than their performances in the first experiment. The possible reason for this result is that the resolution of the images from Tiered T 84 is 84×84, which differs greatly from the resolution of Cifar-100. As a comparison, ICE's performances in this experiment are much better than its performances in the first experiment, which indicates that ICE can efficiently make use of all the resources to improve the performance in spite of the difference among the source datasets.
In the fourth experiment, we use two ResNet-18 models respectively trained on Cifar-10 and Tiered T 56 as the source models. It is observed that most of the baselines' performances in this experiment are slightly better than their performances in the third experiment. For instance, compared with the performances in the third experiment, the performance of FGSM in this experiment is improved by about 1.7%. The possible reason for this result is that compared with the resolution of Tiered T 84 , the resolution of Tiered V 56 is more closer to the resolution of Cifar-100. In this experiment, the proposed ICE still outperforms all baselines with clear margins.
In the fifth experiment, we use three ResNet-18 models respectively trained on Cifar-10, Tiered T 84 and Tiered V 56 as the source models. It is clear that ICE's performances can be further improved by using more source datasets, while the baselines cannot efficiently utilize the additional source models to achieve better performances. AEG performs the best among all baselines with the average attack success rate of 58.3%. Compared with AEG, ICE promotes the average attack success rate by about 39.9%, which is a bigger margin than the margin in the first experiment. This experiment further indicates that ICE is more effective than baselines to leverage all the resource to solve the GTA problem.
4.4 GENERALIZED TRANSFERABLE ATTACK TO CIFAR-10, TIERED T 84 , AND TIERED V 56
We have performed GTA on Cifar-100 in the previous subsection. Now we show ICE still outperforms baselines when using other datasets as target images. Table reports the experimental results. There are four datasets in total (Cifar-10, Cifar-100, Tiered T 84 , and Tiered V 56 ), and each row (denoted as -target) shows the experiment when conducting GTA on the target dataset by using ResNet-18 trained on the other three datasets as source models. For example, the '-Cifar10' row denotes the experiment that utilizing the datasets Cifar-100, Tiered T 84 , and Tiered V 56 and the three respectively trained ResNet-18 models to conduct GTA to the images from Cifar-10. Table does not show the '-Cifar100' row because the corresponding results have been shown in the last row of Table. It is clear that given three datasets and the models trained on the three datasets, ICE performs the best to attack unknown images from other datasets.
Table 2: GTA success rates on Cifar-100
Answer: Table 2 shows that when given a single source model and dataset, ICE also outperforms the baselines. The possible reason is that ICE is trained to disturb source images without knowing label, which forces ICE to learn more generalizable perturbation pattern, and this ensures the advantage of ICE no matter the number of source models and datasets.