On the Scalability of Certified Adversarial Robustness with Generated Data

Authors: thomas  altstidl
david  dobre
arthur  kosmala
björn  eskofier
gauthier  gidel
leo  schwinn
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-15 13:35:31

Abstract: Certified defenses against adversarial attacks offer formal guarantees on the robustness of a model, making them more reliable than empirical methods such as adversarial training, whose effectiveness is often later reduced by unseen attacks. Still, the limited certified robustness that is currently achievable has been a bottleneck for their practical adoption. Gowal et al. and Wang et al. have shown that generating additional training data using state-of-the-art diffusion models can considerably improve the robustness of adversarial training. In this work, we demonstrate that a similar approach can substantially improve deterministic certified defenses but also reveal notable differences in the scaling behavior between certified and empirical methods. In addition, we provide a list of recommendations to scale the robustness of certified training approaches. Our approach achieves state-of-the-art deterministic robustness certificates on CIFAR-10 for the $\ell_2$ ($\epsilon = 36/255$) and $\ell_{\infty}$ ($\epsilon = 8/255$) threat models, outperforming the previous results by $+3.95$ and $+1.39$ percentage points, respectively. Furthermore, we report similar improvements for CIFAR-100.

Main paper: 
 introduction: 
 Deep learning models have been successfully applied for a variety of different applications. However, it is widely recognized that the vulnerability of neural networks to adversarial examples [1] remains an open problem and hinders their adoption in safety-critical domains. Prior research on improving the robustness of neural networks against adversarial examples can be broadly classified into empirical [2, 3] and certified approaches [4] .Adversarial training is currently the most prominent empirical robustification method [3] . Here, the training data of neural networks is augmented with adversarial examples, improving the robustness against attacks at inference time. Recent work has demonstrated that adversarial training can be considerably improved using synthetically generated data, even without training the generative model with external data [5, 6] . Nevertheless, empirical robustness has repeatedly been shown to be ineffective against more sophisticated attacks developed in subsequent work [7] .In contrast to empirical methods, certified approaches yield robustness guarantees given a predefined threat model, most often based on the ℓ 1 , ℓ 2 , or ℓ ∞ norm. As a result, these methods provide reliable protection against future attacks. Nevertheless, the robustness guarantees achieved by certification 38th Conference on Neural Information Processing Systems (NeurIPS 2024).&HUWLILHG$FFXUDF\ &OHDQ$FFXUDF\ &,)$5 ∞ , = 8/255 &HUWLILHG$FFXUDF\ &OHDQ$FFXUDF\ &,)$5 2 , = 36/255 2XUV methods are generally substantially lower than the robustness obtained by empirical defenses for the same threat model [3, 4, 8] .In this work, we aim to analyze how well certified robustness scales when utilizing additional data generated by diffusion models during the model training, a recipe that has previously proven successful for empirical robustness. Only Hu et al. [9] have already used generated data, and thus a broader in-depth review of the factors influencing scalability is to date missing. In our empirical study, we analyze models trained to be robust against ℓ ∞ and models trained to be robust against ℓ 2 norm attacks. The proposed approach improves robustness for both the (ℓ ∞ , ϵ = 8/255) and (ℓ 2 , ϵ = 36/255) threat models on CIFAR-10, improving upon the previous results in the literature by 3.95%p and 1.39%p (percentage points), respectively. In most experiments, the increase in certified accuracy is accompanied by an increase in accuracy on clean data, where we observe improvements by up to 4.83%p. Figure 1 summarizes the improvements compared to the previous state-of-the-art with respect to clean and certified accuracy on CIFAR-10. Further experiments show that the same approach considerably improves certified accuracy on CIFAR-100 as well.Moreover, we conduct ablations to evaluate the impact of different design choices, including regularization, the number of training epochs, the optimization schedule, and the optimal balance between real and generated data. We summarize the most important insights of this empirical study in a list of recommendations that can be followed to more accurately compare and improve the robustness of deterministic certified defenses. Lastly, we discern crucial differences in the scaling behavior between empirical and certified methods. All code used to produce the results and figures in this paper will be released on GitHub after publication. 
related work: 
 Empirical Robustness. Adversarial training was first introduced by Goodfellow et al. [2] . The authors employed the single-step Fast Gradient Sign Method (FGSM) to craft adversarial examples during training and thereby robustify the model against these attacks. Later research by Madry et al. [3] demonstrated that single-step adversarial training does not yield considerable robustness against multi-step attacks. They showed that using the multi-step Projected Gradient Descent (PGD) attack during training successfully improves the robustness of neural networks at test time, even against strong attacks. Subsequent work proposed improvements to the loss function, the adversarial attack used during training, and better trade-offs between clean and certified accuracy [11, 12] .Certified Robustness. Unlike empirical methods, certified methods yield robustness guarantees, thereby eliminating possible vulnerabilities to future attacks. Certification methods can be broadly classified into two methodologically distinct groups, namely probabilistic and deterministic methods. Probabilistic methods aim to approximate smooth classifiers using Monte Carlo sampling and noise injection [4] . A given sample is verified as robust with a certain probability depending on the noise magnitude and number of Monte Carlo samples. To obtain a tight verification bound, probabilistic methods need to perform a substantial amount of sampling procedures (forward passes)for each sample, considerably increasing the computational overhead in practice. Contrary to probabilistic methods, deterministic approaches do not entail considerable computational overhead during inference.Convex bound propagation [13, 14, 15, 16] is a group of deterministic methods that leverages interval arithmetic and linear programming to track how perturbations in the input space transform through each layer, effectively constructing an outer envelope that contains all possible network outputs for inputs within the specified perturbation region. Our initial results scaling the work by Palma et al. [16] , provided in Appendix H, show that this yields severely deteriorated performance when training with additional generated data and is thus a poor candidate for scaling certified robustness.Our main focus is hence devoted to deterministic approaches that bound the Lipschitz constant of each neural network layer to be small (generally smaller or equal to 1) for a predefined ℓ p norm [17, 18] . The Lipschitz constant of the whole network is bounded by the multiplication of the Lipschitz constants of the individual layers [19] . Given a network's upper bound of the Lipschitz constant, a robustness guarantee can be trivially obtained by computing the distance between the highest two logits in the output space.Diffusion Models. More recently, diffusion models have superseded generative adversarial networks (GANs) as the preferred method for image generation [20] . Denoising diffusion probabilistic models (DDPM) [21] can generate high-quality samples on CIFAR-10 [22] with an FID score of 3.17, a common measure of image quality. Since then, other variants have been proposed [23, 24] . By further analyzing the design space of these models [23] , elucidating diffusion models (EDMs) achieve a current state-of-the-art FID score of 1.79 on CIFAR-10. With additional discriminator guidance [24] , the quality of these EDM-generated images are reported to reach an FID score of 1.64, the best score reported in literature for CIFAR-10 at the time of writing.Improving Empirical Robustness with Auxiliary Data. Hendrycks et al. [25] showed that utilizing additional data from external datasets during adversarial training can improve empirical adversarial robustness. Gowal et al. [5] extended this approach to synthetically generated data from generative models only trained on the source dataset. Recently, Wang et al. [6] showed that leveraging the latest advances in diffusion models further improves empirical adversarial robustness.In this work, we investigate if leveraging data generated with state-of-the-art diffusion models can also improve certified robustness against adversarial attacks and analyze how certified training approaches can be scaled optimally.Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [NA] Justification: Limitations of the paper transfer from the models by Hu et al. [9] , Xu et al. [17] , Zhang et al. [18, 27] and the general approach of using generated data by Gowal et al.[5], Wang et al. [6] , and hence are not separately discussed Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 
experiment setup: 
 Given the recent improvements in adversarial training using additional data generated by diffusion models, we devise a set of experiments to investigate whether this also transfers to certified robustness. We focus on deterministic methods as probabilistic methods entail a tremendous computational overhead during inference time and do not achieve considerable robustness for the ℓ ∞ norm yet [10] .All our experiments are done on a single Nvidia A100 graphics card (40GB of VRAM) without distributed training.We perform experiments on CIFAR-10 and CIFAR-100 [22] , for which EDM-generated data is readily available and a wealth of previous robustness research exists [6] . Our experiments and ablation studies focus on CIFAR-10. We refrain from experiments on larger datasets like ImageNet [26] as robustness guarantees achieved by deterministic methods for these datasets are still comparatively low, and only [9] support it at the time of writing. We perform experiments on two common threat models, specifically (ℓ ∞ , ϵ = 8/255) and (ℓ 2 , ϵ = 36/255) adversaries. We do not consider the ℓ 1 threat model, as only smoothing-based approaches achieve considerable robustness for this threat model at the time of writing. For our experiments, we select the two best architectures from the popular certified robustness leaderboard introduced by Li et al. [10] for both the ℓ 2 (GloroNet [9] and LOT [17] ) and ℓ ∞ threat models (SortNet [18] and ℓ ∞ -dist Net [27] ). In total, we perform experiments on architectures from four different papers. To explore the effectiveness of augmenting the original CIFAR-10 and CIFAR-100 [22] datasets with generated data, we adjust the data loader of each model to use a fraction of generated data and original data in every epoch. We use the same generated data used by Wang et al. [6] , which was produced by an EDM trained only on the train set of CIFAR-10. In a preliminary experiment, we found the generated-to-real ratio to be optimal when 30% of training images are real and 70% are generated in every epoch during training, matching the ratio used by Wang et al. [6] . We performed experiments with 50, 000 (50k), 100, 000 (100k), 200, 000 (200k), 500, 000 (500k), 1 million (1m), 5 million (5m) and 10 million (10m) generated images. Wang et al. [6] sub-sampled the 1m images from 5m images choosing only the 20% most confidently classified images according to a pretrained WRN-28-10 model. In contrast, we naively sub-sample the 1m images from the 5m image dataset to avoid potential selection bias by the classifier used to select the data. Moreover, using the same selection process for all datasets should allow us to assess better the effect of the amount of generated data on the final robustness.With additional data, it is also expected that both model size and the number of training epochs can be further scaled to improve clean accuracy and robustness. We thus perform experiments on the influence of model depth and the number of epochs on clean and certified accuracy. For some models, we investigate further techniques that add learning capacity. Concretely, for SortNet [18] we also experiment with models that do not employ dropout, and for LOT [17] we adjust the learning rate scheduler to cosine annealing [28] .Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?Answer: [Yes] Justification: Datasets CIFAR-10 and CIFAR-100 are readily available, and link to code will be added upon acceptance Guidelines:• The answer NA means that paper does not include experiments requiring code. • Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted. 
results: 
 In the following, we first summarize the effect of using additional generated data on the achievable certified and clean accuracy. Furthermore, we ablate the effect of other design choices on the certified robustness, such as the number of training epochs, model size, the amount of additional synthetic data and other hyperparameters. Lastly, we summarize our findings and provide a list of recommendations to scale certified robustness effectively.Across all four reference models and all two threat models, we find that the inclusion of generated data can improve certified accuracy. In most cases, clean accuracy is considerably improved as well.An overview of our new state-of-the-art results in comparison with existing related work is given in Figure 1 . For the (ℓ ∞ , ϵ = 8/255) threat model on CIFAR-10 we can increase the robustness of the existing SortNet [18] to 41.78%, an improvement of 1.39 percentage points. For the (ℓ 2 , ϵ = 36/255) model we achieve a certified accuracy of 69.05% using LOT [17] , a substantial increase of 3.95 points compared to the best result previously reported in the literature [10] . In almost all cases this improvement coincides with an increase in clean accuracy. One exception is SortNet, where the clean accuracy slightly decreases from 54.84% to 54.75%.|D orig | + |D gen | &HUWLILHG$FFXUDF\ ∞ GLVW1HW 6RUW1HW ρ = . 85 ρ = . 00 |D orig | + |D gen | /27 ∞ GLVW1HW 6RUW1HW /Full results are given in Tab. 1 for both SortNet and ℓ ∞ -dist Net, as well as Tab. 2 for LOT and Tab. 3 for GloroNet. We find that by removing the dropout from SortNet we can improve certified accuracy from 41.32% to 41.78% when using auxiliary data. However, the same leads to a drop from 39.72% to 37.44% with only the original data, reinforcing the notion that the additional data acts as a good regularizer. Similarly, for LOT, cosine annealing is superior by a margin of up to 1.12 points compared to a multi-step scheduler, indicating that the model can make better use of its capacity when trained with auxiliary data. For full results using the multi-step scheduler, we refer to App. B, Tab. 2 -all results discussed in subsequent sections refer to those obtained with cosine annealing.Adding auxiliary data improves certified robustness on CIFAR-100 as well, as demonstrated in Tab. 4. We restrict our evaluation to the most effective models from the analysis on CIFAR-10 and do not further scale the number of training epochs. The most substantial improvements on CIFAR-100 are obtained for the SortNet model, where certified accuracy increases by 8.08 percentage points from 9.2% to 17.28%, and for GloroNet, where certified accuracy increases by 2.49 percentage points from 36.41% to 38.9%.Scaling the Amount of Auxiliary Data. The main goal of this work was to evaluate the influence of additional synthetic data during training on the achievable certified accuracy of deterministic certification methods. To this end, we analyze how different amounts of additional data affect the final certified robustness. We evaluate the saturation by training with with 50k, 100k, 200k, 500k, 1m, 5m, and 10m auxiliary generated images. All models are trained for at least twice the amount of epochs that were used in the original papers to ensure saturation in terms of training time. As seen in Fig. 2 , improvements in certified robustness beyond 1m are mostly negligible. This behavior is also largely independent of model size, which stands in contrast to prior results reported for adversarial training [5] .Scaling the Model Size. We performed experiments on several different model sizes to investigate possible correlations between the benefit of additional training data and the model capacity. The ℓ ∞ -based models are largely constructed out of fully connected layers. As a result, the computational effort when scaling these models increases quadratically. As experiments on the ℓ ∞ -based models proved to be computationally too expensive we refrain from scaling these models and focus instead on ℓ 2 -based models.Tables 1 to 3 demonstrate that scaling the model size can increase the certified robustness for both LOT and GloroNet. Shown are the best improvements when adding 1m, 5m, or 10m auxiliary data when compared to the same model trained without any auxiliary data -referred to as the base model from here on. The highest gains are for medium models with LOT and for large models with GloroNet. For GloroNet in particular we observe that model size becomes more important the longer the model is trained, with all models trained for the default 800 epochs showing similar gains.As larger models and additional training data may require longer model training to achieve optimal results we increased the number of training epochs compared to the original configurations for all tested models1 . For the ℓ ∞ -based models in Tab. 1 we see that by doubling the number of epochs we can further improve certified robustness when regularization is removed. A similar picture arises for the ℓ 2 -based models, presented in Tabs. 2 and 3. Here, we find that an increase in the number of epochs yields a significant improvement regardless of model size. Overall, this parameter had the strongest impact in combination with auxiliary data and all best certified accuracies are achieved at their respective maximum number of epochs, with the only exception being SortNet with dropout.The individual improvements in certified robustness vary considerably between different model and data configurations in our experiments. One possible explanation for these differences may be that the generalization gap of the respective models trained without auxiliary data -i.e., the base modelsare different, leading to different gains when closing this generalization gap. To investigate this, we ∆&HUWLILHG$FFXUDF\ correlate the difference in generalization gap with the improvement in certified accuracy obtained when adding auxiliary data. Here, generalization gap refers to the difference between the train and test accuracy on clean data for the best epoch. Figure 3a demonstrates a considerable correlation between the decrease in generalization gap compared to the base model trained with no auxiliary data and the improvement in certified robustness for models with auxiliary data. Moreover, we perform a line fit between the generalization gap and robustness improvement for all the analyzed models. Surprisingly the slope of the different lines is similar for most models, except ℓ ∞ -dist Net, indicating that robustness gains can be predicted once the offset of the line is known for unseen models. However, the offset of the different lines depends on the base model considered. Generalization gaps for each auxiliary dataset size |D gen | in Fig. 3b also correlate well with scaling curves in Fig. 2 .D 1 R Q H N N N N P P P |D gen | *HQHUDOL]DWLRQ*DS E ∞ GLVW1HW 6RUW1HW /27 *ORUR1HW *HQHUDWHGWR2ULJLQDO5DWLR $FFXUDF\ F FOHDQ FHUWLILHGThese results are in line with certified robustness gains achieved for SortNet with and without dropout shown in Tab. 1. Here, the certified robustness of the SortNet model trained without auxiliary data and for 3000 epochs decreases when using less regularization by removing dropout from 39.72% to 37.44%. At the same time, the generalization gap of the two models increases from 4.16% to 12.89%, respectively, as an effect of removing dropout. However, once additional synthetic data is used, removing dropout actually improves the certified robustness to up to 41.71% by nearly two points. Here, increasing the generalization gap by removing dropout had a positive effect on the final certified robustness, which fits observations in Fig. 3a .In every training epoch, we use a proportion of synthetic and real images and keep the total amount of images the same as the size of the original training set. The default configuration throughout our experiments, and the one also used by Wang et al. [6] , is to use 30% real images and 70% generated images in each batch. Figure 3c illustrates how using different proportions for generated and real data affects the certified robustness of the ℓ ∞ -dist Net and LOT-S architectures. We see that at ratios of 60% generated data the clean accuracy saturates, and with 70% the certified accuracy saturates.Notably, in all cases the accuracy when only training with generated images was higher than when only training with real images, indicating that it may be possible to fully train these models on only generated data in the future.To examine the underlying factors contributing to the observed increase in robustness when using auxiliary data, we conduct an analysis of the certification radius distribution for the SortNet (without dropout) and LOT-L architectures. best models trained with and without auxiliary data. Additionally, curves for correct and incorrect classifications are displayed separately.We observe no considerable differences between the distribution of certification radii for the LOT model obtained with or without auxiliary data. Nevertheless, models trained with auxiliary data show slightly higher robustness for correctly classified samples and lower robustness for misclassified samples on average. The SortNet architecture exhibits considerably higher robustness radii for both correct and incorrect classifications when using auxiliary data. Here, differences in certified robustness do not seem to come from a better generalization ability on clean data but from larger certification radii on unseen data. On the other hand, the LOT architecture shows similar certification radii but considerably better generalization on clean data. A more detailed analysis is given in App. G, Fig. 3 . Both models show considerable over-robustness for a considerable fraction of the test set, where the certification radius is well beyond the certification goal ϵ.Differences in certified robustness between distinct defense approaches are often marginal and even small improvements over prior work may be relevant. Here, we summarize the most important takeaways from the empirical study presented in this work on how to scale the robustness of deterministic certified models.• Scale the number of training epochs. Among all investigated hyperparameters, we found the number of training epochs had the most consistent effect on certified accuracy when training with auxiliary data. Based on our experiments, we expect the amount of auxiliary data to not matter as long as it is sufficient for closing the generalization gap.• Increase your model capacity. When using auxiliary data, a large generalization gap between training and testing accuracies is less of an issue and, based on our results, indicative of untapped performance improvements that can be leveraged (see Sec. 4.3) . This means model capacity can be scaled with little fear of overfitting. Our experiments show that reducing the amount of regularization, using better optimizers, and increasing the model size all improve certified accuracy when using auxiliary data without leading to large generalization gaps.• Compare with caution. Even small adjustments, such as a change to dropout, learning rate schedulers, or even different random seeds can improve certified accuracy by about 0.5 to 1 percentage points. This makes it difficult to assess the individual contribution of different architectures towards robustness -for example, contrary to results reported in the original papers, our results indicate that LOT may actually be superior to GloroNet.• Benchmark with generated auxiliary data. As the proposed approach does not entail a computational overhead for the same amount of training epochs, we recommend future work to compare their approaches using auxiliary data and ensure that models are trained till convergence. Differentiating between approaches that use auxiliary data and those that only utilize the original dataset may be helpful for future benchmarks. Similar approaches have been adopted in the empirical robustness domain [29] .• Decrease over-robustness. While not related to the usage of auxiliary data, our evaluation in Fig. 4 indicates that a considerable number of samples are noticeably more resistant to adversarial attacks than what was intended during training. Future research may consider using smaller certification objectives for samples that already demonstrate considerable robustness, an approach that has already been shown to be successful in adversarial training [12] . 
comparison to empirical robustness: 
 While we observe many of the same trends as Gowal et al. [5] and Wang et al. [6] , i.e., that larger models and more epochs generally help, we also note a few crucial differences.• Amount of training data. Scaling beyond one million CIFAR-10 generated images did not further improve certified accuracy, regardless of model size. This is different to scaling behavior reported previously for adversarial training, with, e.g., Gowal et al. [5] reporting an improvement of +2.65 and +2.52% when scaling their WRN-23-10 and WRN-70-16 models, respectively, from one million to 100 million generated images. One hypothesis would be that this is due to the consistency property of the Glivenko-Cantelli class, mentioned by Béthune et al. in Section 5.1 [30] , due to which Lipschitz-1 neural networks' training loss converges to the testing loss for increasingly large datasets. In contrast, adversarial training can be interpreted as a data-dependent operator norm regularization [31] and seems to require more data samples to close the generalization gap. More recent work suggests that the Bayes error may limit the achievable accuracy of both deterministic [32] and probabilistic Zhang and Sun [33] certified robustness, in line with our results.• Overfitting between best and last epoch. In adversarial training, prior work finds that the difference between the best and the last epoch becomes increasingly smaller with larger amounts of auxiliary data [6] . In contrast, we observe no such effect for certifiably robust models (see App. E, Fig. 1 ), highlighting another difference in scaling behavior between empirical and certified robustness.• Optimal generated-to-original ratio. Previous research regarding the optimal generatedto-original ratio for adversarial training observes a drop-off when using only generated data. Both Gowal et al. [5] (Fig. 5 ) and Wang et al. [6] (App. B, Fig. 3 ) suggest accuracy decreases again significantly beyond 70% generated data when using diffusion models. As evident in Fig. 3c this is not as pronounced for certifiably robust models.Together, these results suggest that for certifiably robust models it may be possible to determine a sufficient amount of generated data beforehand for any given dataset -contrary to what is the case for adversarial training, where more is always better. Moreover, it indicates that certified robustness is considerably harder to scale than empirical robustness, as once saturation with respect to data has been achieved, further gains can only be attained by better algorithms and increased model size. Future experiments should explore concrete scaling laws for certified and empirical adversarial robustness, which was out-of-scope of this paper. Deriving and verifying theoretical properties of certifiably robust models, such as those already known on graphs [34] , also remains an exciting topic for further research.  
limitations: 
 Despite our best efforts and extensive experiments, some limitations remain. Our focus is largely on deterministic Lipschitz-bound methods, which incidentally were used in the two best-performing models for the (ℓ ∞ , ϵ = 8/255) and (ℓ 2 , ϵ = 36/255) attacks on CIFAR-10 at the time the experiments were performed [10] . Other approaches for certified robustness, including probabilistic ones and other deterministic methods, stay largely unexplored except for our experiments on convex bound propagation in Appendix H. The inherently high computational complexity of the evaluated models [9, 17, 18, 27 ] also inhibited us from running each configuration multiple times, as even a single run already results in an overall compute time of around five thousand GPU hours. Finally, we made the conscious choice to not perform experiments on ImageNet as only one of the evaluated models, that by Hu et al. [9] , supported this dataset and thus no meaningful comparison could be made.Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?Answer: [Yes] Justification: Sec. 3 Experiment Setup describes in detail the configurations used for all experiments Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. • The full details can be provided either with the code, in appendix, or as supplemental material. 
conclusion: 
 We show that deterministic certified robustness can be improved by up to 5.28%p when additional generated data from a diffusion model is used during training. This is true across four different architectures and two different threat models, (ℓ ∞ , ϵ = 8/255) and (ℓ 2 , ϵ = 36/255), on CIFAR-10, where we report improved certified accuracies of 41.78% and 69.05%, respectively. For ℓ ∞ we are thus able to achieve a new state-of-the-art, while Hu et al. [9] report a slightly better accuracy of 70.1% for the ℓ 2 threat model using data generated by DDPM. In addition, we show the data scaling also improves certified accuracy on CIFAR-100 substantially.We find that the highest gains can be achieved for models where the generalization gap, i.e., the difference between training and testing accuracy, is high for the original model. When augmenting with generated data, the generalization gap is mostly eliminated across all models when a sufficient amount of additional images are used. As the generalization gap gets smaller, removing regularization techniques, such as dropout, and switching to learning rate schedulers aimed at better convergence yields additional improvements. We also note that increasing the number of epochs had the greatest impact when paired with generated data. Lastly, we observe that a considerable number of samples are noticeably more resistant to adversarial attacks than required by the ϵ-bound.Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?Answer: [No] Justification: With a single model run taking one to two days on a A100 graphics card it proved infeasible to perform multiple runs Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.• The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions). • The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors). • It should be clear whether the error bar is the standard deviation or the standard error of the mean. Justification: No specific concerns to address in terms of potential harmfulness Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [NA]Justification: Societal impact related to general impact of robustness research, i.e., robustness may improve safety of systems, but also cause emissions due to increased training times Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA] Justification: No specific risks to address Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes] Justification: Both datasets (CIFAR-10 and CIFAR-100) and models (ℓ ∞ -dist Net, SortNet, LOT, and GloroNet) are referenced Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.• If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. • If this information is not available online, the authors are encouraged to reach out to the asset's creators. 
new assets: 
 Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?Answer: [NA] Justification: No new assets provided Guidelines:• The answer NA means that the paper does not release new assets.• Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc. • The paper should discuss whether and how consent was obtained from people whose asset is used. • At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file. 
crowdsourcing and research with human subjects: 
 Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?Answer: [NA] Justification: No human subjects involved Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects. • Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper. • According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector. 
institutional review board (irb) approvals or equivalent for research with human subjects: 
 Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?Answer: [NA] Justification: No human subjects involved Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects. • Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.• We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix: Table 5 lists the different model configurations used. We limit our summary to the most important parameters -for the full parameters used during training we refer to the respective papers [9, 11, 17, 18] and repositories, as well as our own scripts provided in our code release. In Figure 5 we visualize the number of epochs between the epoch where the best-certified accuracy was achieved and the total amount of epochs trained on the x-axis. On the y-axis we plot the certified accuracy difference between the best and last epoch. Here, the auxiliary data used for the different configurations are visualized with unique colors and symbols. No clear connection between the amount of auxiliary data and the distance between the best and last epoch can be observed. However, the highest distance is observed for models using no auxiliary data. On average, the observed difference between the last and best epoch is small for all models and always below 0.5%. We conclude that robust overfitting is not an issue for the certified training approaches tested in our experiments.(SRFK'LIIHUHQFH ∆&HUWLILHG$FFXUDF\ 1RQH P P P Figure 6 visualizes the effects we also mention in our sensitivity analysis in Sec. Figure 7 illustrates the difference between the best base model (w/o auxiliary) and the best model trained with auxiliary data (w/ auxiliary). We investigate different combinations of correctness and certification for each image. An image may either be correctly or incorrectly classified, and either certified or not certified. If it is certified, this means that its certification radius is larger than ϵ. Our focus on Lipschitz-bound methods may raise questions regarding the applicability of data scaling to other certified robustness domains, such as convex bound propagation. Our additional experiments on MTL-IBP [16] , a summary of which is reproduced below, indicate vastly different scaling dynamics with no observed improvements even when further adjusting (being originally used by Palma et al. [16] for the larger ImageNet). A few remarks:• Verification runs were aborted after 24 hours to accommodate the discussion period. If aborted, reported numbers indicate percentage of certified/correctly classified samples on the subset of CIFAR-10 test images for which verification completed within 24 hours. • Non-completed runs included more hard-to-certify samples that required computationally expensive branch-and-bound verification. Extrapolated full runtimes for some configurations are up to about 2 weeks. • Runs that yielded 0% certified accuracy oftentimes completed quickly as the comparatively cheap PDG attack already found counterexamples, and thus a full verification was not needed.We conclude that MTL-IBP, a convex bound propagation approach, cannot benefit from additional generated data. This indicates that results with regards to scalability do not trivially transfer from one robustness method to another, and a detailed investigation of a single method, such as Lipschitz-bound approaches, seems justified. Justification: Analysis of data scaling behavior for certified robustness described in both abstract and introduction Guidelines:• The answer NA means that the abstract and introduction do not include the claims made in the paper. • The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers. • The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.During training of CIFAR-10 a model will see 0.7 • 50, 000 = 35, 000 generated images in each epoch. 


Decision: This paper proposes to use diffusion models for data augmentation to improve the certified robustness of image classification models. The paper is well-written and easy to follow, and the the proposed method achieves state-of-the-art results on CIFAR-10, demonstrating a significant improvement in robustness certificates for $\ell_\infinity$ and $\ell_2$ perturbations. I suggest the authors add the additional experimental results mentioned in the rebuttal into the final version. I recommend accept (poster).

Reviews: review: paper_summary: paper_summary: This work proposes using data augmentation with diffusion models to improve the certified robustness of image classification models. The authors analyze the training and certification behavior of different Lipschitz-bound-based machine learning models when the training data is supplemented with additional generated samples.
strength_weakness: strength_weakness: **Limited Novelty and Insights**

The idea of supplementing training data with generated data is not new and has been applied in various contexts. Specifically, it has already been used to improve Lipschitz certification by Hu et al. [22]. While this work offers a more thorough evaluation across different threat models and model architectures, the additional insights gained are limited. The paper mainly serves as an evaluation without providing new theoretical insights or methodological advancements.

**Missing Comparison to a Key Related Approach**

A very similar work by Hu et al. [22] also uses diffusion models (DDPM) to augment the training of Lipschitz-based certification methods. Although this work is listed in the references, it is neither discussed nor compared to, despite being the most related approach. This omission leads to several issues:
- The claim in L105 that certified guarantees on ImageNet are “close to random guessing” is inaccurate. Hu et al. report 35% CA for $\ell_2$-norm with $\epsilon = 36/255$, which is significantly higher than the 0.1% of random guessing.
- The claim in L36 that “[generated data] has not yet been combined with deterministic certified training methods” is incorrect, as Hu et al. have done this.
- The claimed improvement over SOTA methods (L13, L145) is not entirely accurate. For $\ell_2$-norm with $\epsilon = 36/255$ in CIFAR-10, Hu et al. report 70.1% CA, which surpasses the 69.05% reported in this work.

**Unclear Meaningfulness of Improvements**

The improvements over prior methods are minor, in the low single digits. As the authors note (L261), a change in hyper-parameters or different seeds can have similar effects on the model. Therefore, it is difficult to judge if the measured improvements are meaningful, especially considering the significant overhead of training a generator, generating the data, and additional model training on the larger dataset. This issue is compounded by the fact that all reported numbers are from single runs with one seed, without error bars or standard deviation. While the authors argue that the large computational cost makes multiple runs infeasible, it remains unclear how meaningful the improvements are.
questions: questions: L180: Do you really mean “1m, 1m, or 1m auxiliary data”?

**Remarks**

Tables and figures are sometimes far from where they are referenced (e.g., Table 1, Fig 2). This makes it difficult to read as the reader has to jump back and forth.

Some instances of % should be replaced by percentage points, e.g., in the abstract (L14), L42, etc.
limitations: limitations: The limitations are not adequately addressed. The paper should include a discussion of limitations, such as the fact that only four Lipschitz-bound-based certification methods are evaluated. It is unclear if these results generalize to other Lipschitz-based certification methods and are likely not applicable to certification methods from different families. Furthermore, only mid-scale datasets like CIFAR are considered; the generalization gap may not exist in larger-scale datasets. Additionally, the results are limited to image classification, and the resulting improvements are small.
paper_summary: paper_summary: The paper explores advancements in certified defenses against adversarial attacks in deep learning models by leveraging data from state-of-the-art diffusion models during training. It addresses the current challenges where empirical methods, such as adversarial training, augment data but face difficulties with new attacks, contrasting with certified approaches that provide robustness guarantees within predefined threat models but often exhibit lower overall robustness. By integrating additional data from diffusion models, the study achieves state-of-the-art robustness certifications on CIFAR-10 and CIFAR-100 under L infinite and L2 threat models. This approach not only enhances deterministic defenses but also improves accuracy on clean data. The paper also performs extensive ablation studies to examine the effects of various design choices on certified robustness.
strength_weakness: strength_weakness: 1. The paper evaluated four architectures selected from the certified robustness leaderboard [9]. However, the achieved clean accuracy and robustness accuracy on these neural networks are notably low. For instance, compared to architectures like WideResNet, which can attain at least 95% clean accuracy and over 80% robustness accuracy on CIFAR-10, the accuracies achieved by the approach in this paper remain insufficient.
2. While evaluating the impact of using generated data from diffusion models to bolster certified robustness is novel, the idea itself represents a somewhat incremental advance. Previous research has already explored the use of such data to enhance adversarial robustness  within empirical methods.
3. Despite the appeal of certified robustness due to its robustness guarantees, its lower accuracy compared to empirical methods diminishes its practicality. While this paper enhances certified robustness, its results have limited practical applicability.
questions: questions: 1. The clean accuracy and robust accuracy of the networks trained in this paper are still lower than those of other architectures. Could you explain why they are lower and provide some reasons for this?
2. Considering the relatively low accuracies, how practical is certified robustness? Could you list several practical scenarios where sacrificing accuracy for robustness to this extent would be justified?
limitations: limitations: 1. The results primarily focus on CIFAR-10 and CIFAR-100 datasets, with limited exploration of generalization to other datasets and real-world scenarios.
2. The results are based on four selected neural network architectures, and the generalization to a broader range of architectures is not thoroughly investigated.
paper_summary: paper_summary: The paper presents an empirical study on how synthetic data can help improve the robustness accuracy and clean accuracy of certified adversarial robustness. While existing studies have shown promising results in using synthetic data to improve empirical adversarial robustness, the effectiveness of synthetic data on certified robustness has never been explored. This paper bridges this gap by designing experiments on $l_2$ and $l_\infty$ robustness of multiple models over the CIFAR-10 and CIFAR-100 datasets. Experiment results show that synthetic data improves certified robustness but in a different way than their effects on the empirical one. The paper also provides ablation studies and guidance on different hyperparameters, such as dropout rate, epochs, model sizes, learning rate schedulers, quantity, and ratios between original and synthetic data.
strength_weakness: strength_weakness: 1. My biggest concern is that the paper misses a large chunk of related works in section 2 and its experiments.
    * In section 2, when discussing the deterministic approaches of certified robustness, the paper writes, "one deterministic approach consists of ..." There are other groups of deterministic approaches using convex bound propagation, such as IBP, SABR, TAPS [1, 2, 3]. The paper focuses only on approaches bounding the Lipschitz constant of each neural network layer. This narrow focus hurts the generalizability of the paper and should be justified. For example, the paper mentions that this group (bounding the Lipschitz constant) of approaches generates a robust guarantee by computing the distance between the highest two logits in the output space. This sentence justifies why not focus on the convex bound propagation approaches.
    * In experiments, the paper's generalizability can be enhanced by comparing against or combining state-of-the-art convex-bound-propagation approaches, such as SABR and TAPS, SOTA, to see how the synthetic data can help.

2. In section 4.3, the paper discusses the correlation between the generalization gap and certified accuracy. The reasoning in this section makes sense. However, the paper might neglect the fact that the model trained with dropout and synthetic data hurts the certified accuracy. This is a weird result. Could you provide the generalization gap of the model trained with $\rho=0.85$ and synthetic data to try to explain this wired result?

[1] On the Effectiveness of Interval Bound Propagation for Training Verifiably Robust Models

[2] Certified training: Small boxes are all you need

[3] Connecting Certified and Adversarial Training
questions: questions: 1. In section 3.2, the paper mentions that "we naively sub-sample the 1m images from the 5m image dataset to..." Are those images from generated data or original data? Also, the sizes of the generated data are different (50K~5m). Do you always perform a 20% sub-sampling, and where do the constants "1m" and "5m" come from? 
2. Does the diffusion model used to generate synthetic data see the robust classifier? Will it generate different synthetic data for different classifiers?
3. In section 5, the paper mentions "overfitting between best and last epoch." Does the paper perform experiments comparing one model trained by early stopping with the other without the early stopping?

Comment:

The paper points out a distinction between the scale of synthetic data on certified robustness and empirical robustness, i.e., scaling beyond one million CIFAR-10 generated images did not further improve certified accuracy, regardless of model size. This observation might be related to Bayes error [4,5].

[4] Certified Robust Accuracy of Neural Networks Are Bounded due to Bayes Errors

[5] How Does Bayes Error Limit Probabilistic Robust Accuracy
limitations: limitations: Although the paper mentions some limitations in their checklists, one significant limitation is whether synthetic data can help other deterministic training methods, such as convex bound propagation, as mentioned in the weakness.

Comments: comment: Thank you for encouraging us to perform additional experiments on convex bound propagation, for which we provide additional results in our overall rebuttal. In short, MTL-IBP performs significantly worse when successively adding auxiliary data and its scaling dynamics deviate significantly from those for Lipschitz-bound models. We'd kindly ask you to consider this in your final evaluation.
As mentioned by both the first and the third reviewer, our focus on Lipschitz-bound methods raises questions regarding the applicability of data scaling to other certified robustness domains, such as convex bound propagation. Our additional experiments on MTL-IBP [1], a summary of which is reproduced below, indicate vastly different scaling dynamics with no observed improvements even when further adjusting $\alpha$ ($0.05$ being originally used by [1] for the larger ImageNet). A few remarks:

* Verification runs were aborted after 24 hours to accommodate the discussion period. If aborted, reported numbers indicate percentage of certified/correctly classified samples on the subset of CIFAR-10 test images for which verification completed within 24 hours.
* Non-completed runs included more hard-to-certify samples that required computationally expensive branch-and-bound verification. Extrapolated full runtimes for some configurations are up to about 2 weeks.
* Runs that yielded 0% certified accuracy oftentimes completed quickly as the comparatively cheap PDG attack already found counterexamples, and thus a full verification was not needed.

We conclude that MTL-IBP, a convex bound propagation approach, cannot benefit from additional generated data. This indicates that results with regards to scalability do not trivially transfer from one robustness method to another, and a detailed investigation of a single method, such as Lipschitz-bound approaches, seems justified.

**Certified Accuracy**

| Model | Epochs | $\alpha$ | None | 50k | 100k | 200k | 500k | 1m | 5m | 10m |
| ----- | ------ | -------- | ---- | --- | ---- | ---- | ---- | -- | -- | --- |
| MTL-IBP | 260 | 0.5 | 35.08% | 34.14% | 32.04% | 21.07%* | 0.34%* | † | 0.00% | 0.00% |
| MTL-IBP | 520 | 0.5 | 35.95% | 34.74% | 34.57% | 33.05% | 17.57%* | 0.54%* | 0.00%* | 0.00% |
| MTL-IBP | 260 | 0.05 | 28.71%* | 26.67%* | 23.75%* | 8.12%* | 0.00%* | † | 0.00% | 0.00% |
| MTL-IBP | 520 | 0.05 | 27.76%* | 27.26%* | 26.68%* | 26.72%* | - | - | - | - |

**Clean Accuracy**

| Model | Epochs | $\alpha$ | None | 50k | 100k | 200k | 500k | 1m | 5m | 10m |
| ----- | ------ | -------- | ---- | --- | ---- | ---- | ---- | -- | -- | --- |
| MTL-IBP | 260 | 0.5 | 53.69% | 55.46% | 57.70% | 66.04%* | 79.31%* | † | 94.60% | 94.82% |
| MTL-IBP | 520 | 0.5 | 54.68% | 55.64% | 55.88% | 57.34% | 68.63%* | 79.10%* | 94.60%* | 95.09% |
| MTL-IBP | 260 | 0.05 | 64.58%* | 66.12%* | 67.67%* | 76.35%* | 84.92%* | † | 94.67% | 94.84% |
| MTL-IBP | 520 | 0.05 | 65.94%* | 66.08%* | 66.73%* | 67.77%* | - | - | - | - |

\* Verification run did not finish within 24 hours, preliminary result on test data subset shown

† Implementation by [Palma et al.](https://github.com/alessandrodepalma/expressive-losses) triggered an assertion error

[1] A. De Palma, R. Bunel, K. Dvijotham, M.P. Kumar, R. Stanforth, and A. Lomuscio, “Expressive Losses for Verified Robustness via Convex Combinations,” in Proc. Intl. Conf. Learn. Representations (ICLR), Vienna, AT, May 2024.
While we are unable to provide true error estimates based on different seeds within the discussion period, we can make an educated guess for GloroNet (L, 2400 epochs) based on a larger-scale sweep with 49 different auxiliary dataset sizes. When fitting a logarithmic curve to the obtained certified accuracies, this yields a standard deviation of 0.20 percentage points under a constant error model (i.e., the error is assumed to be independent of the amount of auxiliary data). This is well below the 2.91 percentage point difference with and without auxiliary data. Although this is certainly limited in methodology, we hope this may help further alleviate any concerns with regards to the significance of the differences observed.
Thanks for your clarification. My concerns about the incremental nature and the limited practical applicability of this work still remain. I believe these aspects are still critical. I will keep my score.
Thank you for your response. We agree that additional experiments with different seeds would help better judge the meaningfulness of the observed improvements and have scheduled four additional runs for the best configurations of each model with and without auxiliary data (e.g., the ones with 64.53% and 69.05% certified accuracies for LOT, Tab. 2). As we are currently running experiments on convex bound propagation we do not expect those results to be ready before the end of the discussion period, though.
Thank you for your response to my questions and comments. 

**Limited Novelty and Insights**

As stated in my original review, I agree that the evaluation is more thorough than Hu et al.'s and provides some additional insights. However, the novelty compared to prior work is still limited, as a very similar method has been proposed before.

**Missing Comparison to a Key Related Approach**

Thank you for expanding the discussion and comparison to Hu et al. and for correcting the related claims. The rebuttal alleviates this concern.

**Unclear Meaningfulness of Improvements**

I understand the reason for not running wide-scale additional experiments. However, the small improvements combined with single runs remain a weakness, as it is difficult to judge how meaningful the improvements are. Could it be possible to perform multiple runs with different seeds for some key configurations to limit the computational budget required and still show the mean and variance of results? I don't expect those results within the discussion period, but it would be good to add them to the final version.
Thank you for your response. I realized that I made a mistake when reading Table 1 regarding the second point of weakness. I will keep my score for now and I need to further discuss with other reviewers and ACs about the first point of weakness.