GREAT Score: Global Robustness Evaluation of Adversarial Perturbation using Generative Models

Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-14 17:04:12

Abstract: Current studies on adversarial robustness mainly focus on aggregating \textit{local} robustness results from a set of data samples to evaluate and rank different models. However, the local statistics may not well represent the true \textit{global} robustness of the underlying unknown data distribution. To address this challenge, this paper makes the first attempt to present a new framework, called \textit{GREAT Score}, for global robustness evaluation of adversarial perturbation using generative models. Formally, GREAT Score carries the physical meaning of a global statistic capturing a mean certified attack-proof perturbation level over all samples drawn from a generative model. For finite-sample evaluation, we also derive a probabilistic guarantee on the sample complexity and the difference between the sample mean and the true mean. GREAT Score has several advantages: (1) Robustness evaluations using GREAT Score are efficient and scalable to large models, by sparing the need of running adversarial attacks. In particular, we show high correlation and significantly reduced computation cost of GREAT Score when compared to the attack-based model ranking on RobustBench \cite{croce2021robustbench}. (2) The use of generative models facilitates the approximation of the unknown data distribution. In our ablation study with different generative adversarial networks (GANs), we observe consistency between global robustness evaluation and the quality of GANs. (3) GREAT Score can be used for remote auditing of privacy-sensitive black-box models, as demonstrated by our robustness evaluation on several online facial recognition services.

Main paper: 
 introduction: 
 Adversarial robustness is the study of model performance in the worst-case scenario, which is a key element in trustworthy machine learning. Without further remediation, state-of-the-art machine learning models, especially neural networks, are known to be overly sensitive to small humanimperceptible perturbations to data inputs (Goodfellow et al., 2014b) . Such a property of oversensitivity could be exploited by bad actors to craft adversarial perturbations leading to predictionevasive adversarial examples.Given a threat model specifying the knowledge of the target machine learning model (e.g., white-box or black-box model access) and the setting of plausible adversarial interventions (e.g., norm-bounded input perturbations), the methodology for adversarial robustness evaluation can be divided into two categories: attack-dependent and attack-independent. Attack-dependent approaches aim to devise the strongest possible attack and use it for performance assessment. A typical example is Auto-Attack (Croce & Hein, 2020) , a state-of-the-art attack based on an ensemble of advanced white-box and black-box adversarial perturbation methods. On the other hand, attack-independent approaches aim to develop a certified or estimated score for adversarial robustness, reflecting a quantifiable level of attack-proof certificate. Typical examples include neural network verification techniques (Wong & Kolter, 2018; Zhang et al., 2018) , certified defenses such as randomized smoothing (Cohen et al., 2019) , and local Lipschitz constant estimation (Weng et al., 2018) .Despite a plethora of adversarial robustness evaluation methods, current studies primarily focus on aggregating local robustness results from a set of data samples. However, the sampling process of these test samples could be biased and unrepresentative of the true global robustness of the underlying data distribution, resulting in the risk of incorrect or biased robustness benchmarks. For instance, we find that when assessing the ranking of Imagenet models through Robustbench (Croce et al., 2020) , using AutoAttack (Croce & Hein, 2020) with 10,000 randomly selected samples (the default choice) with 100 independent trials results in an unstable ranking coefficient of 0.907±0.0256 when compared to that of the entire 50,000 test samples. This outcome affirms that AutoAttack's model ranking has notable variations with an undersampled or underrepresented test dataset.An ideal situation is when the data distribution is transparent and one can draw an unlimited number of samples from the true distribution for reliable robustness evaluation. But in reality, the data distribution is unknown and difficult to characterize. In addition to lacking rigorous global robustness evaluation, many attack-independent methods are limited to the white-box setting, requiring detailed knowledge about the target model (e.g., model parameters and architecture) such as input gradients and internal data representations for robustness evaluation. Moreover, state-of-the-art attack-dependent and attack-independent methods often face the issue of scalability to large models and data volumes due to excessive complexity, such as the computational costs in iterative gradient computation and layer-wise interval bound propagation and relaxation (Katz et al., 2017; Gowal et al., 2019) .To address the aforementioned challenges including (i) lack of proper global adversarial robustness evaluation, (ii) limitation to white-box settings, and (iii) computational inefficiency, in this paper we present a novel attack-independent evaluation framework called GREAT Score, which is short for global robustness evaluation of adversarial perturbation using generative models. We tackle challenge (i) by using a generative model such as a generative adversarial network (GAN) (Goodfellow et al., 2014a; 2020) or a diffusion model (Ho et al., 2020) as a proxy of the true unknown data distribution. Formally, GREAT score is defined as the mean of a certified lower bound on minimal adversarial perturbation over the data sampling distribution of a generative model, which represents the global distribution-wise adversarial robustness with respect to the generative model in use. It entails a global statistic capturing the mean certified attack-proof perturbation level over all samples from a generative model. For finite-sample evaluation, we also derive a probabilistic guarantee quantifying the sample complexity and the difference between the sample mean and true mean.For challenge (ii), our derivation of GREAT score leads to a neat closed-form solution that only requires data forward-passing and accessing the model outputs, which applies to any black-box classifiers giving class prediction confidence scores as model output. Moreover, as a byproduct of using generative models, our adversarial robustness evaluation procedure is executed with only synthetically generated data instead of real data, which is particularly appealing to privacy-aware robustness assessment schemes, e.g., remote robustness evaluation or auditing by a third party with restricted access to data and model. We will present how GREAT score can be used to assess the robustness of online black-box facial recognition models. Finally, for challenge (iii), GREAT score is applicable to any off-the-self generative models so that we do not take the training cost of generative models into consideration. Furthermore, the computation of GREAT score is lightweight because it scales linearly with the number of data samples used for evaluation, and each data sample only requires one forward pass through the model to obtain the final predictions.We highlight our main contributions as follows:• We present GREAT Score as a novel framework for deriving a global statistic representative of the distribution-wise robustness to adversarial perturbation, based on an off-the-shelf generative model for approximating the data generation process.• Theoretically, we show that GREAT Score corresponds to a mean certified attack-proof level of L 2 -norm bounded input perturbation over the sampling distribution of a generative model (Theorem 1). We further develop a formal probabilistic guarantee on the quality of using the sample mean as GREAT Score with a finite number of samples from generative models (Theorem 2).• We evaluate the effectiveness of GREAT Score on all neural network models on RobustBench (Croce et al., 2020) (the largest adversarial robustness benchmark), with a total of 17 models on CIFAR-10 and 5 models on ImageNet. We show that the model ranking of GREAT score is highly aligned with that of the original ranking on RobustBench using AutoAttack (Croce & Hein, 2020) , while GREAT Score significantly reduces the computation time. Specifically, on CIFAR-10 the computation complexity can be reduced by up to 2,000 times. The results suggest that GREAT score is a competitive and computationally-efficient alternative for adversarial robustness evaluation.• As a demonstration of GREAT Score's capability for remote robustness evaluation of access-limited systems, we show how Great Score can audit several online black-box facial recognition APIs. (Szegedy et al., 2014; Carlini & Wagner, 2017; Chen et al., 2018) . The procedure of finding adversarial perturbation within a perturbation level is often formulated as a constrained optimization problem, which can be solved by algorithms such as projected gradient descent (PGD) (Madry et al., 2018) . The state-of-the-art adversarial attack is the Auto-Attack (Croce & Hein, 2020) , which uses an ensemble of white-box and black-box attacks. There are many methods (defenses) to improve adversarial robustness. A popular approach is adversarial training (Madry et al., 2018) , which generates adversarial perturbation during model training for improved robustness. One common evaluation metric for adversarial robustness is robust accuracy, which is defined as the accuracy of correct classification under adversarial attacks, evaluated on a set of data samples. RobustBench (Croce & Hein, 2020) is the largest-scale standardized benchmark that ranks the models using robust accuracy against Auto-Attack on test sets from image classification datasets such as CIFAR-10 . 
background and related works: 
 Generative Models. Statistically speaking, let X denote the observable variable and let Y denote the corresponding label, the learning objective for a generative model is to model the conditional probability distribution P (X | Y ). Among all the generative models, GANs have gained a lot of attention in recent years due to their capability to generate realistic high-quality images (Goodfellow et al., 2020) . The principle of training GANs is based on the formulation of a two-player zero-sum min-max game to learn the high-dimension data distribution. Eventually, these two players reach the Nash equilibrium that D is unable to further discriminate real data versus generated samples. This adversarial learning methodology aids in obtaining high-quality generative models. In practice, the generator G(•) takes a random vector z (i.e., a latent code) as input, which is generated from a zero-mean isotropic Gaussian distribution denoted as z ∼ N (0, I), where I means an identity matrix.Conditional GANs refer to the conditional generator G(•|Y ) given a class label Y . In addition to GAN, diffusion models (DMs) are also gaining popularity. DMs consist of two stages: the forward diffusion process and the reverse diffusion process. In the forward process, the input data is gradually perturbed by Gaussian Noises and becomes an isotropic Gaussian distribution eventually. In the reverse process, DMs reverse the forward process and implement a sampling process from Gaussian noises to reconstruct the true samples by solving a stochastic differential equation. In our proposed framework, we use off-the-shelf (conditional) GANs and DMs (e.g., DDPM (Ho et al., 2020) ) that are publicly available as our generative models.Formal Local Robustness Guarantee and Estimation. Given a data sample x, a formal local robustness guarantee refers to a certified range on its perturbation level such that within which the top-1 class prediction of a model will remain unchanged (Hein & Andriushchenko, 2017) . In L p -norm (p ≥ 1) bounded perturbations centered at x, such a guarantee is often called a certified radius r such that any perturbation δ to x within this radius (i.e., ∥δ∥ p ≤ r) will have the same top-1 class prediction as x. Therefore, the model is said to be provably locally robust (i.e., attack-proof) to any perturbations within the certified radius r. By definition, the certified radius of x is also a lower bound on the minimal perturbation required to flip the model prediction.Among all the related works on attack-independent local robustness evaluations, the CLEVER framework proposed in (Weng et al., 2018) is the closest to our study. The authors in (Weng et al., 2018) derived a closed-form of certified local radius involving the maximum local Lipschitz constant of the model output with respect to the data input around a neighborhood of a data sample x. They then proposed to use extreme value theory to estimate such a constant and use it to obtain a local robustness score, which is not a certified local radius. Our proposed GREAT score has major differences from (Weng et al., 2018) in that our focus is on global robustness evaluation, and our GREAT score is the mean of a certified radius over the sampling distribution of a generative model. In addition, for every generated sample, our local estimate gives a certified radius.Notations. All the main notations used in the paper are summarized in Appendix 6.1.3 GREAT SCORE: METHODOLOGY AND ALGORITHMS EQUATIONUnless the probability density function of P and every local minimal perturbation are known, the exact value of the true global robustness cannot be computed. An alternative is to estimate such a quantity. Extending Definition 1, let g(x) be a local robustness statistic. Then the corresponding global robustness estimate is defined asEQUATIONFurthermore, if one can prove that g(x) is a valid lower bound on ∆ min (x) such that g(x) ≤ ∆ min (x), ∀x, then the estimate Ω(f ) is said to be a certified lower bound on the true global robustness with respect to P , and larger Ω(f ) will imply better true global robustness. In what follows, we will formally introduce our proposed GREAT Score and show that it is a certified estimate of the lower bound on the true robustness with respect to the data-generating distribution learned by a generative model. 
using gms to evaluate global robustness: 
 Recall that a generative model (GM) takes a random vector z ∼ N (0, I) sampled from a zero-mean isotropic Gaussian distribution as input to generate a data sample G(z). In what follows, we present our first main theorem that establishes a certified lower bound Ω(f ) on the true global robustness of a classifier f measured by the data distribution given by G(•).Without loss of generality, we assume that all data inputs are confined in the scaled data range [0, 1] d , where d is the size of any flattened data input. The K-way classifier f : [0, 1] d → R K takes a data sample x as input and outputs a K-dimensional vector f (x) = [f 1 (x), . . . , f K (x)] indicating the likelihood of its prediction on x over K classes, where the top-1 class prediction is defined as ŷ = arg max k={1,...,K} f k (x). We further denote c as the groundtruth class of x. Therefore, if ŷ ̸ = c, then the classifier is said to make a wrong top-1 prediction. When considering the adversarial robustness on a wrongly classified sample x, we define the minimal perturbation for altering model prediction as ∆ min (x) = 0. The intuition is that an attacker does not need to take any action to make the sample x evade the correct prediction by f , and therefore the required minimal adversarial perturbation level is 0 (i.e., zero robustness).Given a generated data sample G(z), we now formally define a local robustness score function asEQUATIONThe scalar π/2 is a constant associated with the sampling Gaussian distribution of G, which will be apparent in later analysis. We further offer several insights into understanding the physical meaning of the considered local robustness score in ( 3): (i) The inner term f c (G(z)) -max k∈{1,...,K},k̸ =c f k (G(z)) represents the gap in the likelihood of model prediction between the correct class c and the most likely class other than c. A positive and larger value of this gap reflects higher confidence of the correct prediction and thus better robustness. (ii) Following (i), a negative gap means the model is making an incorrect prediction, and thus the outer term max{gap, 0} = 0, which corresponds to zero robustness.Next, we use the local robustness score g defined in (3) to formally state our theorem on establishing a certified lower bound on the true global robustness and the proof sketch.Theorem 1 (certified global robustness estimate). Let f : [0, 1] d → [0, 1] K be a K-way classifier and let f k (•) be the predicted likelihood of class k, with c denoting the groundtruth class. Given a generator G such that it generates a sample G(z) with z ∼ N (0, I). Defineg (G(z)) = π 2 • max{f c (G(z)) -max k∈{1,...,K},k̸ =c f k (G(z)), 0}.Then the global robustness estimate of f evaluated with L 2 -norm bounded perturbations, defined asΩ(f ) = E z∼N (0,I) [g(G(z))],is a certified lower bound of the true global robustness Ω(f ) with respect to G.The complete proof is given in Appendix 6.4.As defined in Theorem 1, the global robustness estimateΩ(f ) = E z∼N (0,I) [g(G(z))] is the mean of the local robustness score function introduced in ( 3) evaluated through a generator G and its sampling distribution. In practice, one can use a finite number of samples {G(z i |y i )} n i=1 generated from a conditional generator G(•|y) to estimate Ω(f ), where y denotes a class label and it is also an input parameter to the conditional generator. The simplest estimator of Ω(f ) is the sample mean, defined asEQUATIONIn what follows, we present our second main theorem to deliver a probabilistic guarantee on the sample complexity to achieve ϵ difference between the sample mean Ω S (f ) and the true mean Ω(f ).Theorem 2 (probabilistic guarantee on sample mean). Let f be a K-way classifier with its outputs bounded by [0, 1] K and let e denote the natural base. For any ϵ, δ > 0, if the sample size n ≥32e•log(2/δ) ϵ 2, then with probability at least 1 -δ, the sample meanΩ S (f ) is ϵ-close to the true mean Ω(f ). That is, | Ω S (f ) -Ω(f )| ≤ ϵ.The complete proof is given in Appendix 6.5. The proof is built on a concentration inequality in (Maurer & Pontil, 2021) . It is worth noting that the bounded output assumption of the classifier f in Theorem 2 can be easily satisfied by applying a normalization layer at the final model output, such as the softmax function or the element-wise sigmoid function.To conclude this section, the Algorithm 1 in Appendix 6.6 summarizes the procedure of computing GREAT Score using the sample mean estimator. It can be seen that the computation complexity of GREAT Score is linear in the number of generated samples N S , and for each sample, the computation of the statistic g defined in (3) only requires drawing a sample from the generator G and taking a forward pass to the classifier f to obtain the model predictions on each class. As a byproduct, GREAT Score applies to the setting when the classifier f is a black-box model, meaning only the model outputs are observable by an evaluator.In cases when one has additional knowledge of adversarial examples on a set of images from a generative model, e.g., successful adversarial perturbations (an upper bound on the minimal perturbation of each sample) returned by any norm-minimization adversarial attack method such as the CW attack (Carlini & Wagner, 2017) , The CW attack employs two loss terms, classification loss and distance metric, to generate adversarial examples. See Appendix 6.7 for details. we can further "calibrate" the Great Score with respect to the available perturbations. Moreover, since Theorem 1 informs some design choices on the model output layer, as long as the model output is a non-negative K-dimensional vector f ∈ [0, 1] K reflecting the prediction confidence over K classes, we will incorporate such flexibility in the calibration process. Specifically, we use calibration in the model ranking setup where there are M models {f (j) } M j=1 for evaluation, and each model (indexed by j) has a set of known perturbations {δ(j) i } N i=1 on a common set of N image-label pairs {x i , y i } Ni=1 from the same generative model. We further consider four different model output layer designs (that are attached to the model logits): (i) sigmoid(•|T 1 ): sigmoid with temperature T 1 , (ii) softmax(•|T 2 ): softmax with temperature T 2 , (iii) sigmoid(softmax(•|T 2 = 1)|T 1 ): sigmoid with temperature after softmax, and (iv) softmax(sigmoid(•|T 1 = 1)|T 2 ): softmax with temperature after sigmoid. Finally, let { Ω S (f (j) )} M j=1 denote the Great Score computed based on {x i , y i } N i=1 for each model. We calibrate Great Score by optimizing some rank statistics (e.g., the Spearman's rank correlation coefficient) over the temperature parameter by comparing the ranking consistency between { Ω S (f (j) )} M j=1 and {δ(j) i } N i=1 .In our experiments, we find that setting (iv) gives the best result and use it as the default setup for calibration, as detailed in Appendix 6.8. 
experiment setup: 
 Datasets and Models. We conduct our experiment on several datasets including CIFAR-10 ( Krizhevsky et al., 2009) , ImageNet-1K (Deng et al., 2009) and CelebA-HQ (Karras et al., 2018) /CelebA (Liu et al., 2015) . For neural network models, we use the available models on RobustBench (Croce et al., 2020) (see more details in the next paragraph), which includes 17/5 models on CIFAR-10/ImageNet, correspondingly. We also use several off-the-shelf GANs and diffusion models (DMs) trained on CIFAR-10 and ImageNet for computing GREAT Score in an ablation study (we defer the model details to later paragraphs).Summary of Classifiers on RobustBench. The RobustBench (RobustBench) is to-date the largest benchmark for robustness evaluation with publicly accessible neural network models submitted by contributors. RobustBench uses the default test dataset from several standard image classification tasks, such as CIFAR-10 and ImageNet-1K, to run Auto-Attack (Croce & Hein, 2020) and report the resulting accuracy with L 2 -norm and L ∞ -norm perturbations (i.e., the robust accuracy -RA) as a metric for adversarial robustness. Even under one perturbation type, it is not easy to make a direct and fair comparison among all submitted models on RobustBench because they often differ by the training scheme, network architecture, as well as the usage of additional real and/or synthetic data. To make a meaningful comparison with GREAT Score, we select all non-trivial models (having non-zero RA) submitted to the CIFAR-10 and ImageNet-1K benchmarks and evaluated with L 2 -norm perturbation with a fixed perturbation level of 0.5 using Auto-Attack. We list the model names in Table 1 and provide their descriptions in Appendix 6.9.GANs and DMs. We used off-the-shelf GAN models provided by StudioGAN (Minguk Kang & Park, 2021), a library containing released GAN models. StudioGAN also reports the Inception Score (IS) to rank the model quality. We use the GAN model with the highest IS value as our default GAN for GREAT Score, which are StyleGAN2 (Karras et al., 2020) / BigGAN (Brock et al., 2019) for CIFAR-10 /ImageNet with IS = 10.477/99.705, respectively. For the ablation study of using different generative models in GREAT Score (Section 4.4), we also use the following GAN/DM models: LSGAN (Mao et al., 2017) , GGAN (Lim & Ye, 2017) , SAGAN (Zhang et al., 2019) , SNGAN (Miyato et al., 2018) , DDPM (Ho et al., 2020) and StyleGAN2 (Karras et al., 2020) .GREAT Score implementation.The implementation follows Algorithm 1 in Appendix 6.6 with a sigmoid/softmax function on the logits of the CIFAR-10/ImageNet classifier to ensure the model output of each dimension is within [0, 1], as implied by Theorem 1. As ImageNet-1K has 1000 classes, applying sigmoid will make the robustness score function in (3) degenerate. We use softmax instead. 500 samples drawn from a generative model were used for computing GREAT Score. Comparative methods. We compare the effectiveness of GREAT Score in two objectives: robustness ranking (global robustness) and per-sample perturbation. For the former, we compare the RA reported in RobustBench on the test dataset (named RobustBench Accuracy) as well as the RA of Auto-Attack on the generated data samples (named AutoAttack Accuracy). For the latter, we report the RA of Auto-Attack in L 2 -norm with a fixed perturbation level of 0.5.Evaluation metrics. For robustness ranking, we report Spearman's rank correlation coefficient between two sets of model rankings (e.g., GREAT Score v.s. RobustBench Accuracy). A value closer to 1 means higher consistency. Robust accuracy refers to the fraction of correctly classified samples against adversarial perturbations. Calibration Method. We run L 2 -norm CW attack (Carlini & Wagner, 2017) (with learning rate 0.005 and 200 iterations) on each generated data sample to find the minimal adversarial perturbation. Then, we use grid search in the range [0,2] with an interval of 0.00001 to find temperature value maximizing the Spearmans' rank correlation coefficient between Great Score and CW attack distortion.Compute Resources. All our experiments were run on a GTX 2080 Ti GPU with 12GB RAM.Recall from Theorem 1 that the local robustness score proposed in (3) gives a certified perturbation level for generated samples from a generative model. To verify this claim, we randomly select 20 generated images on CIFAR-10 and compare their local certified perturbation level to the perturbation found by the CW attack (Carlini & Wagner, 2017 ) using the Rebuffi_extra model (Rebuffi et al., 2021) . Rebuffi et al. (Rebuffi et al., 2021) proposed a fixing data augmentation method such as using CutMix (Yun et al., 2019) and GANs to prevent over-fitting. Figure 1 shows the perturbation level of local GREAT Score in (3) and that of the corresponding CW attack per sample. We can see that the local GREAT Score is a lower bound of CW attack, as the CW attack finds a successful adversarial perturbation that is no smaller than the minimal perturbation ∆ min (i.e., an over-estimation). The true ∆ min value lies between these lower and upper bounds.In Figure 2 , we compare the cumulative robust accuracy (RA) of GREAT Score and Auto-Attack over 500 samples by sweeping the L 2 perturbation level from 0 to 1 with a 0.05 increment for Auto-Attack. The cumulative RA of GREAT Score at a perturbation level r means the fraction of samples with local GREAT scores greater than r, which gives an attack-proof guarantee that no attacks can achieve a lower RA at the same perturbation level. We see that the trend of attack-independent certified robustness (GREAT Score) is similar to that of empirical attacks (Auto-Attack). The gap between our certified curve and the empirical curve of AutoAttack does not necessarily indicate inferiority, it could mean that there exist undiscovered adversarial examples at higher perturbation radii.Table 1 compares the global robustness statistics of the 17 grouped CIFAR-10 models on RobustBench for uncalibrated and calibrated versions respectively, in terms of the GREAT Score and the average distortion of CW attack, which again verifies GREAT Score is a certified lower bound on the true global robustness (see its definition in Section 3.1), while any attack with 100% attack success rate only gives an upper bound on the true global robustness. We also observe that calibration can indeed enlarge the Great Score and tighten its gap to the distortion of CW attack. Following the experiment setup in Section 4.1, we compare the model ranking on CIFAR-10 using GREAT Score (evaluated with generated samples), RobustBench (evaluated with Auto-Attack on the test set), and Auto-Attack (evaluated with Auto-Attack on generated samples). Table 2 presents their mutual rank correlation (higher value means more aligned ranking) with calibrated and uncalibrated versions. We note that there is an innate discrepancy between Spearman's rank correlation coefficient (way below 1) of RobustBench v.s. Auto-Attack, which means Auto-Attack will give inconsistent model rankings when evaluated on different data samples. In addition, GREAT Score measures classification margin, while AutoAttack measures accuracy under a fixed perturbation budget ϵ.AutoAttack's ranking will change if we use different ϵ values. E.g., comparing the ranking of ϵ = 0.3 and ϵ = 0.7 on 10000 CIFAR-10 test images for AutoAttack, the Spearman's correlation is only 0.9485. Therefore, we note that GREAT Score and AutoAttack are complementary evaluation metrics and they don't need to match perfectly. Despite their discrepancy, before calibration, the correlation between GREAT Score and RobustBench yields a similar value. With calibration, there is a significant improvement in rank correlation between Great Score to Robustbench and Auto-Attack, respectively.Table 3 presents the global robustness statistics of these three methods on ImageNet. We observe almost perfect ranking alignment between GREAT Score and RobustBench, with their Spearman's rank correlation coefficient being 0.8, which is higher than that of Auto-Attack and RobustBench (0.6). These results suggest that GREAT Score is a useful metric for margin-based robustness evaluation.Ablation study on GANs and DMs. Evaluating on CIFAR-10, Figure 3 compares the inception score (IS) and the Spearman's rank correlation coefficient between GREAT Score and RobustBench on five GANs and DDPM. One can observe that models with higher IS attain better ranking consistency.Run-time analysis. Figure 4 compares the run-time efficiency of GREAT Score over Auto-Attack on the same 500 generated CIFAR-10 images. We show the ratio of their average per-sample run-time (wall clock time of GREAT Score/Auto-Attack is reported in Appendix 6.11) and observe around 800-2000 times improvement, validating the computational efficiency of GREAT Score.Sample Complexity and GREAT Score. In Appendix 6.12, we report the mean and variance of GREAT Score with a varying number of generated data samples. The results show that the statistics of GREAT Score are quite stable even with a small number of data samples (i.e., ≥500). We use an off-the-shelf face image generator InterFaceGAN (Shen et al., 2020) trained on CelebA-HQ dataset (Karras et al., 2018) , which can generate controllable high-quality face images with the choice of attributions such as eyeglasses, age, and expression. We generate four different groups (attributes) of face images for evaluation: Old, Young, With Eyeglasses, and Without Eyeglasses. For annotating the ground truth gender labels of the generated images, we use the gender predictions from the FAN classifier (He et al.) . In total, 500 gender-labeled face images are generated for each group. Appendix 6.14 shows some examples of the generated images for each group.We evaluate the GREAT Score on six online APIs for gender classification: BetaFace (BetaFace), Inferdo (Inferdo), Arsa-Technology (Arsa-Technology), DeepFace (Serengil & Ozpinar, 2021), Baidu (Baidu) and Luxand (Luxand) . These APIs are "black-box" models to end users or an external model auditor because the model details are not revealed and only the model inference results returned by APIs (prediction probabilities on Male/Female) are provided.Finally, we upload these images to the aforementioned online APIs and calculate the GREAT Score using the returned prediction results. Table 4 displays the group-level and overall GREAT Score results. Our evaluation reveals interesting observations. For instance, APIs such as BetaFace, Inferno, and DEEPFACE exhibit a large discrepancy for Old v.s. Young, while other APIs have comparable scores. For all APIs, the score of With Eyeglasses is consistently and significantly lower than that of Without Eyeglasses, which suggests that eyeglasses could be a common spurious feature that affects the group-level robustness in gender classification. The analysis demonstrates how Great Score can be used to study the group-level robustness of an access-limited model in a privacy-enhanced manner.To verify our evaluation, in Table 5 we compare GREAT Score to the black-box square attack (Andriushchenko et al., 2020) with ϵ=2 and # queries=100 on DEEPFACE. For both Age and Eyeglasses groups (Old v.s. Young and W/ v.s. W/O eyeglasses), we see consistently that a higher Great Score (second row) indicates better robust accuracy (%, first row) against square attack. 
conclusion: 
 In this paper, we presented GREAT Score, a novel and computation-efficient attack-independent metric for global robustness evaluation against adversarial perturbations. GREAT Score uses an off-the-shelf generative model such as GANs for evaluation and enjoys theoretical guarantees on its estimation of the true global robustness. Its computation is lightweight and scalable because it only requires accessing the model predictions on the generated data samples. Our extensive experimental results on CIFAR-10 and ImageNet also verified high consistency between GREAT Score and the attack-based model ranking on RobustBench, demonstrating that GREAT Score can be used as an efficient alternative for robustness benchmarks. We also demonstrated the novel use of Great Score for the robustness evaluation of online facial recognition APIs.Limitations. One limitation could be that our framework of global adversarial robustness evaluation using generative models is centered on L 2 -norm based perturbations. This limitation could be addressed if the Stein's Lemma can be extended for other L p norms. 3)Ω(f )/ Ω(f )true/estimated global robustness defined in Section 3.1 
more motivations on using generative models for robustness evaluation: 
 We emphasize the necessity of generative models using the points below.1. Global robustness assessment requires a GM. The major focus and novelty of our study are to evaluate the global robustness with respect to the underlying true data distribution, and we propose to use a GM as a proxy. We argue that such a proxy is necessary to evaluate global robustness unless the true data distribution is known.2. GAN can provably match data distribution. Recent works such as (Schreuder et al., 2021) and (Liang, 2021) have proved the convergence rate of approaching the true data distribution for a family of GANs under certain conditions. This will benefit global robustness evaluation (see Figure 3 for ablations on GAN variants).There are some works studying "global robustness", while their contexts and scopes are different than ours. In (Ruan et al., 2018) , the global robustness is defined as the expectation of the maximal certified radius of L 0 -norm over a test dataset. Ours is not limited to a test set, and we take the novel perspective of the entire data distribution and use a generative model to define and evaluate global robustness. The other line of works considers deriving and computing the global Lipschitz constant of the classifier as a global certificate of robustness guarantee, as it quantifies the maximal change of the classifier with respect to the entire input space (Leino et al., 2021) . The computation can be converted as a semidefinite program (SDP) (Fazlyab et al., 2019) . However, the computation of SDP is expensive and hard to scale to larger neural networks. Our method does not require computing the global Lipschitz constant, and our computation is as simple as data forward pass for model inference.In this section, we will give detailed proof for the certified global robustness estimate in Theorem 1. The proof contains three parts: (i) derive the local robustness certificate; (ii) derive the closed-form global Lipschitz constant; and (iii) prove the proposed global robustness estimate is a lower bound on the true global robustness.We provide a proof sketch below:1. We use the local robustness certificate developed in (Weng et al., 2018) , which shows an expression of a certified (attack-proof) L p -norm bounded perturbation for any p ≥ 1. The certificate is a function of the gap between the best and second-best class predictions, as well as a local Lipschitz constant associated with the gap function.2. We use Stein's Lemma (Stein, 1981) which states that the mean of a measurable function integrated over a zero-mean isotropic Gaussian distribution has a closed-form global Lipschitz constant in the L 2 -norm. This result helps avoid the computation of the local Lipschitz constant in Step 1 for global robustness evaluation using generative models.3. We use the results from Steps 1 and 2 to prove that the proposed global robustness estimate Ω(f ) is a lower bound on the true global robustness Ω(f ) with respect to G. To prove Theorem 2, we first define some notations as follows, with a slight abuse of the notation f as a generic function in this part. For a vector of independent random variables X = (X 1 ..., X n ), define X ′ = (X ′ 1 ..., X ′ n ) to be i.i.d. to X, x = (x 1 , ..., x n ) ∈ X, and the sub-exponential norms ∥•∥ ψ2 for any random variable Z as∥Z∥ ψ2 = sup p≥1 ∥Z∥ p √ p (15)Let f : X n → R. We further define the k-th centered conditional version of f as :EQUATIONLemma 5 (Concentration inequality from Theorem 3.1 in (Maurer & Pontil, 2021) ). Let f : X n → R and X = (X 1 , . . . , X n ) be a vector of independent random variables with values in a space X. Then for any t > 0 we haveEQUATIONRecall that we aim to derive a probabilistic guarantee on the sample mean of the local robustness score in (3) from a K-way classifier with its outputs bounded by [0, 1] K . Following the definition of g (for simplicity, ignoring the constant π/2), the sample mean f can be expressed as:EQUATIONwhere X i ∼ N (0, I).Following the definition of ( 16),EQUATIONDivide both side of ( 22) and multiply with -t 2 32e gives:EQUATIONTake exponential function over both side of (23) givesEQUATIONRecall Lemma 5, since this bound holds on both sides of the central mean, we rewrite it as:EQUATIONHence to ensure that given a statistical tolerance ϵ > 0 with δ as the maximum outage probability, i.e., Prob(|fEQUATIONFinally, (26) implies that the sample complexity to reach the (ϵ, δ) condition is n ≥ 32e•log(2/δ) ϵ 2. We provide a detailed comparison of the time complexity between GREAT Score and CW Attack.The time complexity of the GREAT Score algorithm is determined by the number of iterations (generated samples) in the loop, which is denoted as N S . Within each iteration, the algorithm performs operations such as random selection, sampling from a Gaussian distribution, generating samples, and predicting class labels using the classifier. We assume these operations have constant time complexity I and absorb them in the big O notation. Additionally, the algorithm computes the sample mean of the recorded statistics, which involves summing and dividing the values. As there are N S values to sum and divide, this step has a time complexity of O(N S ). Therefore, the overall time complexity of the algorithm can be approximated as O(N S • I).Using our nation, consider a K-way classifier f . Let x be a data sample and y be its top-1 classification label. Denote δ as the adversarial perturbation. The untargeted CW Attack (L 2 norm) solves the following optimization objective:EQUATIONwhere f k (•) is the prediction of the k-th class, and α > 0 is a hyperparameter.For CW attack, the optimization process iteratively finds the adversarial perturbation. The number of iterations required depends on factors such as the desired level of attack success and the convergence criteria. Each iteration involves computing gradients, updating variables, and evaluating the objective function. It also involves a hyperparameterα search stage to adjust the weighted loss function. (He et al., 2016a) . LIBRARY means using the package mentioned in Group of other models to train on ImageNet. Fast (Wong et al., 2020) means fast adversarial training. There is no L 2 -norm benchmark for ImageNet on RobustBench, so we use the L ∞ -norm benchmark. (Gowal et al., 2020) 0.034 59.586 Rebuffi_70_ddpm (Rebuffi et al., 2021) 0.034 61.3362 Rebuffi_28_ddpm (Rebuffi et al., 2021) 0.011 10.3828 Augustin_WRN_extra (Augustin et al., 2020) 0.013 10.096 Sehwag (Sehwag et al., 2021) 0.011 10.3662 Augustin_WRN (Augustin et al., 2020) 0.011 10.1056 Rade (Rade & Moosavi-Dezfooli, 2021) 0.007 4.4114 Rebuffi_R18(Rebuffi et al., 2021) 0.008 4.4644 Gowal (Gowal et al., 2020) 0.034 60.746 Sehwag_R18 (Sehwag et al., 2021) 0.007 3.8652 Wu2020Adversarial (Wu et al., 2020) 0.012 10.9826 Augustin2020Adversarial (Augustin et al., 2020) 0.014 6.9148 Engstrom2019Robustness (Engstrom et al., 2019) 0.012 6.6462 Rice2020Overfitting (Rice et al., 2020) 0.007 3.5776 Rony2019Decoupling (Rony et al., 2019) 0.010 8.5834 Ding2020MMA (Ding et al., 2018) 0.008 3.6194 6.10 APPROXIMATION ERROR AND SAMPLE COMPLEXITY Figure 6 presents the sample complexity as analyzed in Theorem 2 with varying approximation error (ϵ) and three confidence parameters (δ) for quantifying the difference between the sample mean and the true mean for global robustness estimation. As expected, smaller δ or smaller ϵ will lead to higher sample complexity.Figure 6 : The relationship between the approximation error (ϵ) and sample complexity in Theorem 2, with three different confidence levels: δ = {5, 15, 25}%.The complete run-time results of Figure 4 are given in Table 8: 6.12 SAMPLE COMPLEXITY AND GREAT SCORE Figure 7 reports the mean and variance of GREAT Score with a varying number of generated data samples using CIFAR-10 and the Rebuffi_extra model, ranging from 500 to 10000 with 500 increment. Figure 8 reports the mean and variance of GREAT Score ranging from 50 to 1000 with 50 increment. The results show that the statistics of GREAT Score are quite stable even with a small number of data samples.Besides evaluating the GREAT Score on the generated samples from GAN, we also run the evaluation process on 500 test samples of CIFAR-10. Table 9 shows the evaluated Great score. (Augustin et al., 2020) 78.79 86.20 0.525 0.525 Sehwag (Sehwag et al., 2021) 77.24 89.20 0.227 0.227 Augustin_WRN (Augustin et al., 2020) 76.25 86.40 0.583 0.489 Rade (Rade & Moosavi-Dezfooli, 2021) 76.15 86.60 0.413 0.331 Rebuffi_R18 (Rebuffi et al., 2021) 75.86 87.60 0.369 0.297 Gowal (Gowal et al., 2020) 74.50 86.40 0.124 0.109 Sehwag_R18 (Sehwag et al., 2021) 74.41 88.60 0.236 0.176 Wu2020Adversarial (Wu et al., 2020) 73.66 84.60 0.128 0.106 Augustin2020Adversarial (Augustin et al., 2020) 72.91 85.20 0.569 0.493 Engstrom2019Robustness (Engstrom et al., 2019) 69.24 82.20 0.160 0.127 Rice2020Overfitting (Rice et al., 2020) 67.68 81.80 0.152 0.120 Rony2019Decoupling (Rony et al., 2019) 66.44 79.20 0.275 0.221 Ding2020MMA (Ding et al., 2018) 66.09 77.60 0.112 0.08 
 
 Appendix: Privacy-sensitive remote model auditing. As shown in Sec 4.5, synthetic data from generative models can facilitate the robustness evaluation of privacy-sensitive models.EQUATIONwhere L q = max x∈S ∥∇f (x)∥ q : is the corresponding Lipschitz constant, and ∇f (x) = ( ∂f ∂x1 , ... ∂f ∂x d ) ⊤ is the gradient of the function f(x), and 1/q + 1/p = 1, p ≥ 1, q ≤ ∞.We say f is L q -continuous in L p norm if ( 5) is satisfied.Lemma 2 (Formal guarantee on lower bound for untargeted attack of Theorem 3.2 in (Weng et al., 2018) ). Let x 0 ∈ R d and f : R d → R K be a multi-class classifier, and f i be the i-th output of f . For untargeted attack, to ensure that the adversarial examples can not be found for each class, for all δ ∈ R d , the lower bound of minumum distortion can be expressed by:∥δ∥ p ≤ min i̸ =m f m (x 0 ) -f i (x 0 ) L i q (6)where m = arg max i∈{1,...,K} f i (x 0 ), 1/q + 1/p = 1, p ≥ 1, q ≤ ∞, and L i q is the Lipschitz constant for the function f m (x) -f i (x) in L q norm. A.2.2 Proof of closed-form global Lipschitz constant in the L 2 -norm over Gaussian distribution In this part, we present two lemmas towards developing the global Lipschitz constant of a function smoothed by a Gaussian distribution.Lemma 3 (Stein's lemma (Stein, 1981) ). Given a soft classifier F : R d → P, where P is the space of probability distributions over classes. The associated smooth classifier with parameter σ ≥ 0 is defined as:EQUATIONThen, F is differentiable, and moreover,EQUATIONIn a lecture note1 , Li used Stein's Lemma (Stein, 1981) to prove the following lemma:Lemma 4 (Proof of global Lipschitz constant). Let σ ≥ 0, let h : R d → [0, 1] be measurable, andlet H = h * N (0, σ 2 I). Then H is 2 πσ 2 -continuous in L 2 norm A.2.3Proof of the proposed global robustness estimate Ω(f ) is a lower bound on the true global robustness Ω(f ) with respect to G Recall that we assume a generative model G(•) generates a sample G(z) with z ∼ N (0, I). Following the form of Lemma 2 (but ignoring the local Lipschitz constant), letEQUATIONdenote the gap in the model likelihood of the correct class c and the most likely class other than c of a given classifier f , where the gap is defined to be 0 if the model makes an incorrect top-1 class prediction on G(z). Then, using Lemma 4 with g ′ , we defineEQUATIONand thus E z∼N (0,I) [g ′ (G(z))] has a Lipschitz constant 2 π in L 2 norm. This implies that for any input perturbation δ,|E z∼N (0,I) [g ′ (G(z) + δ)] -E z∼N (0,I) [g ′ (G(z))]| (11) ≤ 2 π • ∥δ∥ 2 (12)and thereforeEQUATIONNote that if the right-hand side of ( 13) is greater than zero, this will imply the classifier attains a nontrivial positive mean gap with respect to the generative model. This condition holds for any δsatisfying ∥δ∥ 2 < π 2 • E z∼N (0,I) [g ′ (G(z))].Note that by definition any minimum perturbation on G(z) will be no smaller than π 2• E z∼N (0,I) [g ′ (G(z))] as it will make g ′ (G(z)) = 0 almost surely.Therefore, by defining g = π 2• g ′ , we conclude that the global robustness estimate Ω(f ) in (2) using the proposed local robustness score g defined in ( 3) is a certified lower bound on the true global robustness Ω(f ) with respect to G.https://jerryzli.github.io/robust-ml-fall19/lec14.pdfhttps://github.com/MadryLab/robustness 


Decision: This paper is on evaluating adversarial robustness. There are three limitations of current evaluation methods:
- The evaluation is often limited to a fixed test dataset, which does not capture robustness on the global data distribution.
- The evaluation is often limited to white-box settings where input gradients are required.
- The evaluation may require lots of computation.

This paper proposes a “GREAT score” to address these three limitations. First, they use a generative model to approximate the unknown data distribution. Second, their evaluation method only requires forward passes (as shown in Eqn. (3)) and is useful when only black-box access is available. Third, the evaluation can use pretrained generative models, which reduces the computational burden. Furthermore, each data point for evaluation only requires one forward pass.

On the theory side, this paper shows that GREAT score corresponds to a lower bound on the robustness radius to L2 attacks. They also showed a formal probabilistic guarantee on the finite-sample performance of GREAT score. On the empirical side, the paper provides extensive evaluations using RobustBench and shows reductions in computation time, sometimes up to 2000 times.

The paper provides a novel approach to evaluate adversarial robustness effectively. It is fast, has theoretical guarantees, and is aligned with existing robustness evaluations based on the experiments on RobustBench. Several reviewers commented on the clear writing of the paper.

The authors were able to address most reviewers’ concerns. Unfortunately, one of the reviewers did not engage in the discussions. However I can see that most of their concerns are appropriately addressed. The reviewers gave several valuable suggestions to improve the current evaluations, such as ablating more hyperparameter search methods in the calibration process and discussing the quality of generative models and their effect on the evaluation of robustness.

Reviews: review: paper_summary: paper_summary: The paper addresses the important and under-explored problem of "global robustness evaluation" for neural networks. It proposes GREAT Score, a novel framework for assessing global robustness using generative models (GMs).  Besides, through Monte Carlo sampling from GMs and using Hoeffding's concentration bound, the algorithm can reach an epsilon probabilistic guarantee on the sample mean's closeness to the true mean. The paper then applies their proposed algorithm on various classifiers using GMs to measure global robustness scores.
strength_weakness: strength_weakness: 1) The reliance on GANs as a proxy for the true data distribution raises concerns about the method's accuracy. To the best of my knowledge, current GANs do not generate better coverage than the test set. GANs are a bad estimation of the underlying data distribution with known issues such as bias and model collapsing. Considering model collapse, the fixed test set is likely to have even better distribution coverage than the samples generated from GAN. It would be much more reliable and convincible by involving  the recent class generative models.
2) I also encourage the authors to include experiments with other local robustness estimators, further strengthening the submission.
How does the choice of generative model and local robustness estimator affect the reliability of the global measure computed by the paper?
3) Theoretically, while the authors provide a probabilistic guarantee on the obtained estimates derived from GMs and true estimate, there's a lack of theoretical bound on gap between the true estimate and models' global robustness arising from the distance of the  generative distribution and underlying data distribution. Otherwise, the significance and utility of the GREAT score is unclear for me, and this omission makes it unclear how the accuracy of the empirical distribution affects the overall error, beyond just sample complexity.
questions: questions: Please respond my questions in the weakness part.
limitations: limitations: Yes
paper_summary: paper_summary: The paper introduces a novel framework called GREAT Score (Global Robustness Evaluation of Adversarial Perturbation using Generative Models), aimed at evaluating the global robustness of machine learning models against adversarial perturbations. Unlike traditional methods that aggregate local robustness results from a finite set of data samples, GREAT Score leverages generative models to approximate the underlying data distribution, providing a more comprehensive global robustness assessment.
strength_weakness: strength_weakness: 1. The GREAT Score in the paper primarily focuses on adversarial perturbations under the L2 norm. While this is a common setting in adversarial attack research, it lacks ablation studies for other norms, such as the L∞ norm
2. The GREAT Score framework relies on generative models (such as GANs or diffusion models) to approximate the true data distribution. If the quality of the generative model is not high, the generated samples may not accurately represent the true data distribution, thus affecting the accuracy of robustness evaluation. Besides, the evaluation results of the GREAT Score also depend on the generated sample set. If the sample set is biased or fails to comprehensively cover the diversity of the data distribution, the evaluation results may be inaccurate or unrepresentative.
3. The evaluation of online facial recognition APIs using GREAT Score is innovative, but the paper could provide more detailed analysis and discussion on the specific challenges and insights derived from this application. For instance, exploring the variability in robustness scores among different groups (e.g., age, eyeglasses) in greater depth and providing potential reasons for these variations would add depth to the analysis.
4. The calibration process described in Section 3.5 appears somewhat ad-hoc, relying on grid search for optimizing temperature parameters. This could be perceived as lacking robustness and generalizability. A more systematic approach to calibration, possibly incorporating advanced optimization techniques or sensitivity analysis, would strengthen the framework. Discussing the stability and consistency of the calibration process across different models and datasets would also be beneficial.
5. Despite claiming computational efficiency, the paper does not provide a detailed analysis of the scalability of the GREAT Score framework, especially in the context of extremely large datasets and models. A thorough examination of how the computation time scales with increasing data size and model complexity would add significant value. This could include empirical results demonstrating the method's performance on larger datasets or theoretical analysis of its computational complexity.
questions: questions: 1. Have you considered extending the GREAT Score framework to other norm-based perturbations like L1 or L∞ norms? If so, what are the potential challenges or theoretical adjustments needed?
2. How does the quality of the generative model affect the GREAT Score evaluation? Have you conducted any experiments using generative models of varying quality to analyze this impact?
3. Can you provide more detailed information on the scalability of the GREAT Score framework with respect to extremely large datasets and models? How does the computation time scale with increasing data size and model complexity?
4. How stable and consistent is the calibration process described in Section 3.5 across different models and datasets? Have you explored any advanced optimization techniques for calibration?
limitations: limitations: Yes
paper_summary: paper_summary: The authors propose the GREAT score that uses conditional generative models to mimic the data generating distribution. Thereafter, the classification margin on a set of generated images can be used to obtain a global robustness score. For this, the authors make the connection between local and global robustness explicit and show that the classification margin yields a lower bound on the robustness w.r.t. L2 perturbations (convolution with Gaussian). The authors empirically validate their GREAT score on various benchmarks and highlight the strong correlation to other common robustness evaluations while reducing the computational cost for the robustness evaluation significantly.
strength_weakness: strength_weakness: 1. There are several assumptions on the generative model that are not sufficiently/prominently enough covered. The assumptions are: (a) the model generates an instance actually belonging to the conditioned class, (b) the true class is unambiguous (e.g., conversely, there might be cases where the "Bayes optimal" model cannot decide between two or more classes). (c) the generative model is a good approximation of the true data-generating distribution. The authors should highlight such limitations more and their implications for the guarantees/method.
1. Since the authors emphasize the guarantee on the average robustness of a model, the authors could elaborate more on the practical importance of such a guarantee
1. The derived Lipschitz constant might be a loose estimate since the Lipschitz constant also includes the generative model and not only the neural network. This is not accurately reflected in, e.g., Eq 10. Here it seems the model was convolved with the Gaussian ($g' * N(0,1))$), but it should actually be $((g' \circ G) * N(0,1))$.

Minor:
- The font size in figures and tables is very small
questions: questions: 1. Figure 2 shows a large gap between empirical attacks (upper bound) and the GREAT score (lower bound). To what extent do the authors expect this gap to be due to the looseness of the respective upper and lower bounds?
1. How is the class label (input of conditional generative model) distributed in the experiments for calculating the GREAT score?
1. Would it also be possible to derive guarantees for a subset of the data distribution's support? For example, obtaining class-specific average robustness guarantees?
limitations: limitations: The authors sufficiently addressed the limitations (except for the weaknesses stated above).
paper_summary: paper_summary: This paper presents an innovative adversarial robustness measure, which leverages a generative model to produce data samples, record the marginal confidence score as a local statistic and average them over the data distribution. The proposed measure is designed to be efficient, scalable, and potentially applicable to unknown data distributions. Empirical validation is conducted using local models and commercial inference APIs, demonstrating the utility of the robustness evaluation.

The concept introduced in this study is commendable for its originality, and the metric indeed offers valuable insights into model robustness. Nonetheless, it requires substantial revisions to enhance its clarity, presentation, and justification of claims before it can be accepted.
strength_weakness: strength_weakness: 1. Presentation issues that may lead to confusion include:  
   (1) The second paragraph of introduction lacks a precise definition of adversarial robustness evaluation, which could be problematic for less experienced readers.   
   (2) Putting the testing algorithm in the appendix hurts the coherence of the paper. It would be better to include it in the main text.     
   (3) Figure 2 requires additional clarification to elucidate how robust accuracy (RA) and the proposed metric are integrated into the same plot. Current discussion is insufficient. 

2. The generative model's training requires at least partial knowledge of the data distribution. So the claim that the proposed metric can scale to unknown data distribution needs justification.

3. The metric's performance is contingent on the generative model's capacity to produce benign samples, yet no guarantee is provided that the generative model's ability to do so.

4. The endeavor to train a generative model to produce benign samples should be considered in making it a cost-effective and scalable solution. Maybe this metric can consider online learning to update the generative model. Hope a discussion can be provided on this.

5. The claim regarding the limitation to white-box settings (Page 2, Line 56) is inaccurate, as adversarial accuracy can also be assessed in black-box scenarios, evidenced by the effectiveness of the Square Attack method.
questions: questions: 1. The paper omits discussion of several significant works that evaluate model robustness from different perspectives. The authors should consider addressing the following studies in the related works section:    
   [1] "How many perturbations break this model? evaluating robustness beyond adversarial accuracy." Olivier, Raphael, and Bhiksha Raj. International Conference on Machine Learning. PMLR, 2023.    
   [2] "Probabilistically robust learning: Balancing average and worst-case performance." Robey, Alexander, et al. International Conference on Machine Learning. PMLR, 2022.    
   [3] "Exploring the Adversarial Frontier: Quantifying Robustness via Adversarial Hypervolume." Guo, Ping, et al. arXiv preprint arXiv:2403.05100 (2024).

2. While the paper presents test results on the original test samples on CIFAR-10 in Table 9,  it does not provide results for the ImageNet dataset. The authors should explain this choice and consider including ImageNet results for a more comprehensive comparison.
limitations: limitations: See weakness and questions.

Comments: title: Response to rebuttal
comment: We thank the reviewer for the prompt response and for your support in championing our paper!
I'm grateful for your additional experiments. Based on the results, different search algorithms don't have a significant impact, which addresses my concern. From this perspective, whether to include this ablation study in the final paper is up to you. Anyway, I will improve my score and advocate for this paper to be accepted.
We greatly appreciate your insightful feedback and the suggestion for this valuable ablation study. We agree that demonstrating our framework's reliability across different search methods is crucial. We are also very pleased to learn that you are inclined to raise the score based on these additions.

In response to your recommendation, we have conducted a comprehensive ablation study comparing various search methods for calibration on the temperature, including grid search, binary search, simulated annealing, and genetic algorithm. This study was performed under the same experimental settings as the calibration process described in our paper. We will incorporate these findings into Section 4.4 of the final version, alongside the ablation study of generative models.






The results of our experiment are summarized in the table below:



| Method | GREAT Score vs. RobustBench Correlation | GREAT Score vs. AutoAttack Correlation | Best Temperature |
| ------ | -------------------------------------- | -------------------------------------- | ---------------- |
| Grid Search | 0.8971 | 0.6941 |0.00742 |
| Binary Search | 0.8971 | 0.6941 | 0.00781 |
| Simulated Annealing | 0.8971 | 0.6941 | 0.00748 |
| Genetic Algorithm | 0.8971 | 0.6941 | 0.00879 |


For the implementation of simulated annealing and genetic algorithm, we ensured fair comparison by maintaining consistent search space ([0, 2]), precision (minimum step size of 0.00001), and computational resources across all algorithms.

Simulated Annealing:
We implemented a standard simulated annealing algorithm with an exponential cooling schedule. The algorithm starts with an initial temperature of 1 (this temperature is not the same as the temperature in calibration) and employs an exponential cooling schedule with a decay rate of 0.95, continuing until the temperature reaches a minimum of 0.001 or other termination criteria are met. To accommodate time constraints, we reduced the number of iterations in the inner loop to 10, while maintaining the overall algorithm structure. The initial temperature was set high to allow for broad exploration, gradually decreasing to enable fine-tuning.

Genetic Algorithm:
Our genetic algorithm implementation used a population of 30 candidate solutions over 50 generations. We employed tournament selection, single-point crossover, and a mutation rate of 0.3. The population size and number of generations were adjusted to align with the computational time of the simulated annealing algorithm.

Both algorithms were adapted to optimize the temperature parameter in our GREAT Score calibration process, with the objective of maximizing the correlation scores. While the reduced iterations might affect the algorithms' ability to find the global optimum, this trade-off reflects real-world constraints and allows for a fair comparison of their performance under limited computational resources.


The ablation study demonstrates that while different search methods (grid search, binary search, simulated annealing, and genetic algorithm) yield slightly different optimal temperature values, they all converge to the same ranking results. This consistency across different search methods highlights the reliability and practical applicability of the GREAT Score calibration process.
Thanks for your clarification. If more search methods can be used as an ablation study, I will raise my score from 5 to 6.
We thank the reviewer again for the insightful discussion. We agree that a specialized generative model is needed when we are evaluating domain-specific image classifiers. One good example is Section 4.5, where we use InterFaceGAN (a domain-specific image generator) to evaluate the robustness of online facial recognition APIs. We will make sure this point and the reviewer's comment are reflected in the revised version of our paper.
Dear Authors,

Thank you for your esponse to my previous comments regarding the training data.

I would like to elaborate on my perspective concerning the training data for GANs. I have observed that your framework employs an off-the-shelf model, which has been pre-trained on a dataset comprising natural images. This approach aligns with the conventional focus of most research on generative models, which predominantly centers on the **natural image manifold**.

Nonetheless, I would like to draw attention to scenarios such as when the input and output images are not of natural scenes, for instance, medical X-ray images. In such cases, it may be necessary to train a specialized model. This consideration leads to the question of how one might acquire an appropriate dataset to ensure that the GAN is capable of generalizing across different models' evaluations.

While these concerns may not hold significant weight in discussions limited to natural images, I believe it is crucial to acknowledge this limitation. My intention is to highlight this aspect so that it may be addressed in future research endeavors, thereby examining the impact of data distribution on the performance of generative models.

Warm regards,
Dear Reviewer,


Thank you very much for your insightful and constructive feedback on our manuscript. We are pleased to learn that the reviewer has raised the score. We have carefully considered your comments and have made the following revisions to address your concerns:

- Presentation and Related Works: Thank you for your suggestion, especially for pointing out the 3 important related papers, we will update this in the final version. 
- Unknown Data Distribution: 
On the review comment "What would be the implications if the training data were not available for GAN training?". We are unsure what the review meant, could the reviewer elaborate on this point? We note that our framework uses an off-the-shelf generative model for robustness evaluation, which does not involve the training of a generative model. 
On the review comment "How would the use of only test data affect the training, and could this potentially lead to the collapse of GANs due to a shift in distribution?" We are also unsure about this question. Test data should not affect the training by any means. 
- GAN producing benign examples: We agree that the GAN Producing Benign Examples rely heavily on the generative quality to approximate the true distribution, as already justified by our results in Figure 3. We also want to emphasize that our framework is not limited to GAN. We also include diffusion models in our results.
- Online Learning: We agree with the reviewer that this is a good future direction to explore.
- Minor issues:
Black box setting: We thank the reviewer for pointing this out, we will acknowledge it in the manuscript.
Related Works: We will ensure we include sufficient discussion of related works.
Additional Results: We thank you for pointing out that we should also add the Imagenet experiment, we will include it in the appendix of the final version.

Thanks again for your efforts.

Sincerely,
Dear Authors,

Thank you for submitting your rebuttal in response to the concerns I previously raised. I appreciate the effort you have put into addressing these issues. Please find below my comments following your rebuttal:

## Overview
The manuscript presents an intriguing concept in the evaluation of adversarial robustness, which offers a novel perspective that diverges from the traditional attack-based methods. This is a commendable approach that contributes to the field.

Nevertheless, I observe that the proposed inference-based method bears resemblance to existing work in robustness evaluation, such as Adversarial Sparsity. While the incorporation of generative models mitigates some of the limitations inherent in prior methods, it also introduces new challenges associated with the characteristics of GANs. These issues are likely to be of interest to the readers.

Overall, these limitations do not hurt the significance of your contribution. Furthermore, your response has addressed most of my concerns, so I decide to raise my score. However, I have summarized your response and identified several additional points that may warrant further consideration.

 - **Presentation:** I apprecite your efforts in improving the readability, no further questions.
 - **Unknown Data Distribution:** The discussion surrounding the concept of a 'true unknown data distribution' is somewhat controversial. The use of training data for the model under evaluation in the training of generative models raises questions about the definition of unknown data distribution. This issue is not present in studies focusing solely on generative models.
   - What would be the implications if the training data were not available for GAN training?
   - How would the use of only test data affect the training, and could this potentially lead to the collapse of GANs due to a shift in distribution?
 - **GAN Producing Benign Examples:** This may be alleviated by considering the convergence properties of GANs. However, same issue of data dependence persists.
 - **Online Learning:** The topic of online learning is a potential avenue for future research by the authors and shares the aforementioned limitation regarding unknown data distribution. This is particularly relevant given the requirement for substantial data volumes.
 - **Minor Issues:**
   - **Black-box Setting:** The extension to a black-box-based accuracy appears to be a minor one, yet it should be acknowledged in the manuscript.
   - **Related Works:** I note that the authors intend to include additional related works in the revised manuscript. I believe that a more in-depth discussion of these works could provide valuable insights for future research in this area.
   - **Additional Results:** I am pleased with the results provided and have no further concerns in this regard.

I look forward to seeing the final version of the manuscript and wish to see your continued research in this important area.

Best regards,
Thank you for your valuable feedback. We are delighted to learn that the reviewer is inclined to raise the score. While we can't edit the submission at this point, we will include a detailed discussion of Weakness 1 in the main part of the revised paper, specifically in the experimental section, with a dedicated discussion on GANs. Additionally, we will explicitly add the dependency of the generator $G$ in the right-hand side of Equation 10 as suggested.
I thank for the detailed explanations. I will raise the score to 6 if the authors will include a detailed discussion concerning Weakness 1 in the main part of the revised paper and "correct" the RHS of Eq 10.