Towards Certification of Uncertainty Calibration under Adversarial Attacks
Authors: Cornelius Emde
Francesco Pinto
Thomas Lukasiewicz
Philip Torr
Adel Bibi
Venue: ICLR 2025 Poster (2025)
Keywords: Machine Learning
Adversarial Robustness
Certification
Adversarial Training
Uncertainty Quantification
Calibration
Deep Learning
Certified Calibration
Primary area: alignment, fairness, safety, privacy, and societal considerations
TL;DR: We introduce certified calibration, a worst case bound on probability calibration under adversarial attacks, provide a framework for approximating it and improving model training to tighten the bound.

Abstract: Since neural classifiers are known to be sensitive to adversarial perturbations that alter their accuracy, certification methods have been developed to provide provable guarantees on the insensitivity of their predictions to such perturbations. On the other hand, in safety-critical applications, the frequentist interpretation of the confidence of a classifier (also known as model calibration) can be of utmost importance. This property can be measured via the Brier score or the expected calibration error. We show that attacks can significantly harm calibration, and thus propose certified calibration providing worst-case bounds on calibration under adversarial perturbations. Specifically, we produce analytic bounds for the Brier score and approximate bounds via the solution of a mixed-integer program on the expected calibration error. Finally, we propose novel calibration attacks and demonstrate how they can improve model calibration through adversarial calibration training. The code will be publicly released upon acceptance.

Published as a conference paper at ICLR 2025
TOWARDS CERTIFICATION OF UNCERTAINTY CALIBRATION UNDER ADVERSARIAL ATTACKS
Cornelins Emde'; Francesco Pinto', Thomas Lukasiewicz*1, Philip Torr', Adel Bibi' 'University of Oxford VVienna University of Technology
ABSTRACT
Since neural classifiers are known to be sensitive to adversarial perturba- tions that alter their accuracy, cerlification methods have been developed to provide provable guarantees the insensitivity of their predictions to such perturbations. Furthermore, in safety-critical applications, the frequentist interpretation of the confidence of classifier (also known model cali- bration) can be of utmost importance, This property can be measured via the Brier score or the expected calibration error. We show that attacks significantly harm calibration, and thus propose certified calibration as worst- case bounds on calibration under adversarial perturbations. Specifically, we produce analytic bounds for the Bricr score and approximate bounds via the solution of mixed-integer program on the expected calibration error. Finally, we propose novel calibration attacks and demonstrate how they can improve model calibration through adversarial colibration training.
INTRODUCTION
The black-box nature of deep neural net- works and the unrcliability of their predic- tions under several forms of data shift hinder their deployment in safety-critical applica- tions (Abdar et al. 2021 Linardatos et al. 2021b. is well known that neural network classifiers are extremely sensitive to pertur- bations 7 on image x that are small enough to be imperceptible to the human eye, yet vields different prediction than Goodfellow et al. 2015 Szegedy et al. 20144] 20163. While a variety of methods have been proposed to improve the empirical robust- ness of neural networks, certification meth- ods have recently gained traction, since they provide provable guarantees the invari- ance of predictions under adversarial atlacks and thus provide guarantees on model accuracy (Wong and Kolter 2018)) (see Appendix A.1 for detailed introduction). Further, existing work provides bounds on the confidence scores (i.e., the top value of the softmax output vector) of certified models under adversarial inputs (Kumar et al. 2020). However, they do not inform us about the average mismatch between accuracy and contdence. The reliability of a classifier in these regards is generally quantified by the expected calibration error (ECE) (Naeini al. 2015) and the Brier score (BS) (Bieer] 1950 Bröcker 22009). Both describe the calibration of confidences across a set of predictions and play a strategic role in applications that leverage confidence scores in their decision-making process. For instance, utilize reliable confidence scores in medical diagnostics to decide whether to trust the machine learning classifier without further human intervention human export. However, we empirically show that it is possible to produce adversarics that severely impact the reliability of confidence scores while leaving the accuracy unchanged, even when the classifier is explicitly trained to be robust to adversarial attacks on the accuracy.
Lower bound Confidence Upper Bound
Coniiiied Calilmation Emor (CCF)
Sumples
Calibration Error
Confidence
This work proposes certificate (upper bound) on the calibration error of a classifier under adversaries. Each box on the left represents one prediction from a certified model: The range of each box represents a certificate on the confidence score under adversaries, the color whether the prediction is certifiably correct. We translate these certificates into certificate the calibration error (right), ie. worst-case under attack.
Figure 1:
might
whether to defer the decision to
"ooresponning Author cornelius. emde@cs. ac..c.

---

Published as a conference paper at ICLR 2025
For this reason, propose the notion of certified calibration to provide guarantees on the calibration of certified models under adversarial attacks. While certification of individual predictions enables guaranlees the accuracy of classifier under adversaries, the addition ional certificates on the confidences enable worst-case bounds on the reliability of confidence scores.
Our contributions are the following:
We identify and demonstrate attacks on model calibration that severely impact uncertainty estimation while remaining unnoticed when solely monitoring model accuracy.
We introduce certified calibration quantified through the certified Brier score (CBS) and the certified calibration error (CCE). For the former, present closed-form bound as certificate, while we show a mized-integer non-linear progrum reformulation for the latter with effective solver, dubbed approzimale certified colibration error (ACCE).
We introduce a new form of adversarial training, adversarial calibration training (ACT), that augments training with Brier and ACCE adversarics. We obtain ACCE adversaries by solving a mired inttegrr program and demonstrate that ACT can improve certified uncertainty calibration.
2 CONFIDENCE CALIBRATION
We first formally introduce the notion of calibration in Section 2.1 then, in Section 2.2 we show how attacks on confidence scores and calibration metries may be beneficial an attacker. Finally, in Section 2.2 we show that such attacks are not only theoretically possible but also realizable.
2.1 QUANTIFYING THE CONFIDENCE- ACCURNCCACCCAACC MISMATCHI
Let D = (Xn, Ja be a dataset of size N with x, E RD, E y (1, K)] :
K}. and RR + AK be a neural network, where Ax is a probability simplex over K classes. We
denote the k-th component of f as fk and define F RD + y to be the hand classifier predicting a class label obtained by Fxx) = arg max [x(x). This prediction is done with the confidence provided by the soft classifier z RD + [0, 1], obtained through z(x) == maxkey fa().. With slight abuse of notation, we refer to functions and their outputs simultaneously, e.g., z E [0, 1] is an output of z(z). The notion of reliability of uncertainty estimates is typically quantified through two different but related quantities that we introduce: the expected calibration error and the Brier score.
For classification tasks, calibration describes match between the model's confidence and its empirical performance (DeGroot and Fienberg I
Expected Calibration Error.
1983: Naeini et al., 2015). A well-calibrated model predicts with conlidence 2 when the Iraction of correct predictions is exactly 2, i... PPF = y =) z. This enables to interpret 2 as a probability the frequentist sense. The expected calibration error is the expected difference between confidence and accuracy over the distribution of confidence Z, ie., ECE = Ez [IP(F- =|==>) =l]. A common istimator is obtained by discretizing the empirical distribution over Z through binning (Guo et al. 2017). For each set of confidences Bmn in the m-th bin, the average confidence is compared with the accuracy:
M Bml N
EČE =
Σ
1
Σ G.
1
2m
(1)
Bm
B
m=1
mEBm
NEBm
where N is the number of samples, 2n the confidence of the n-th prediction Fa, and e, is the prediction correctness, i.e., On = if F2 = =nn and otherwise. Multiple variants of the calibration error and its estimators exist (Kumar et al. 2019 As commonly done in the litcrature, we focus on top label calibration that ignores the calibration of confidences of lower-ranked predictions. When using equal-width interval binning for the estimator in [1. we refer to as ECE, and when using an equal-count binning scheme, we refer to it as AdaECE (Nguyen and O'Connor 2015 Nixon et al. 2019
Brier Score. Accuracy and calibration represent different concepts, and one may not infer one from the her unambiguously (see Appendix B. The two concepts are unified under proper scoring rules, such the Brier Score (BS) (Brier 1950), which is commonly
2

---

Published as a conference paper at ICLR 2025
used in the calibration literature. It has been shown that this family of metrics can be decomposed into a calibration and a refinement termn (Murphy 1972 1973; Bröcker 2009). An optinal score can only be achieved by predicting accuralely and with appropriate confidence. The BS mathematically defined as the mean squared error between the confidence vector f and one-hot encoded label vector. Here, we focus on the top-label Brier score TLBS N- 211: which is the mean squared error between the N confidences z E [0, 1 and correctness of each prediction с {0, 13%
2.2 CALIBRATION UNDER ATTACK
Motivation. A wide range of adversarial attacks has been discussed in the literature, with label tacks being the most prevalent (Akhtar and Mian 2018). However, their immediate effects on the reliability of uncertainty estimation are olten overlooked: Label Lacks do not only impact the correctness of predictions, but also the confidence scores, both affecting uncertainty calibration. Beyond label attacks, techniques directly targeting confidence scores have been developed. Galii and El- Yaniv (202]) show that such attacks can cause significant harm in applications where confidence scores are used in decision processes at sample level, e.g., for risk estimation and credit pricing in credit lending. Yet, they evade detection by leaving predicted labels unchanged. We note that this impacts system-level uncertainty and conclude that both label and confidence attacks leave system calibration vulnerable.
Furthermore, machine learning systems incorporating uncertainty estimates in their deci- sion process are monitored regularly for both their predictive performance and colibration, particularly in safety-critical applications. When abnormalities in either monitored metric arise, the model may be pulled from deployment for further investigation, resulting in major operational costs. To this end, an allacker might produce alacks on the accuracy of a model to induce Denial-of-Serrice (DoS). While defenses against. label attacks effectively protect the model accuracy, the calibration is not sufficiently protected: An attacker can directly target system calibration, causing harm due to increased operational risk, and cause changes in monitored calibration metrics, thus forcing a DoS. Despite extensive countermcasures to adversarial attacks, existing technique addresses this vulnerability.
Extending the credit lending example, deploying a certified model provides provable pro- tections: First, the accuracy of predictions customer defaults is protected because of the provable invariance of predicted labels under allack (Cohen el al.] 2019). Second, the uncertainty per individual customer is protected by bounds on the contidence scores [Kumar et al. 2020). However, these protections do not inform the operational risk of the system, and the reliability of uncertainty estimation across the system remains vulnerable. Miscalibrated uncertainty can lead to underestimation of risks, resulting in unexpected customer defaults, or overestimation, rendering credit pricing uncompetitive and reducing revenue. It is, therefore, essential for the credit. lender to monitor calibration rigorously. An attacker might exploit this by directly targeting calibration with severe consequences for the credit lender.
Feasibility of Calibration Attacks. Af- ter discussing possible reasons for attacks on calibration, we now demonstrate that these are feasible. Galil and EI- Ya-nyy (202T) show that it is possible to produce allacks that leave the accuracy unchanged while degrading the Brier score. Expanding on their Attacks on Confidence Fstimation (ACE), we intro- duce a family of parameterised ACE attacks on the cross-entropy loss Lee that we call (1) w)-ACE attacks. These attacks solve the following objective:
Table 1: AdaECEJ (%) when performing our (n, ACAE attacks compared to the case in which the attack is not performed for PreAct- ResNet18 and ResNet50 on CIFAR-10 and Ima- geNet for models fitted through Standard Train- ing (ST) and Adversarial Training (AT).
CIFAR
ImageNet
Unattacked:
3.06
3.70
8/255
2/205
3/205
ST
8
2.49
1.08
10.62
18.79
47.23
46. 92
5.19
23.72
23.73
11.87
25.17
84.84
max (nItseeF(e+t)=F(e) (CCE((x + w),, (2)
Unattacked:
20.09
9.03
8/265
2/255
3/255
7.38
8.51
where n E (1,-1, E (v, 9 2 +7)+ is the prediction and y the true label. For = 1, this at tack decreases model confidence in label y or prediction 0, and for v
11.92
23.51
11.62
0.91
12.21
3.42
25.59
13.54
14.28
1, it
increases confidence, both without altering the label (i.e., F(x + +7 = F(x)). In Table 1
3

---

Published as a conference paper at ICLR 2025
show that all four possible configurations of our (n. -C-CCE can be effective at significantly allering the ECE of PreActResNel18 (He et al. 2016a) and ResNet50 (He el al. 2016b) on the validation set of CIFAR-10 (Krizhessky 2009] and ImageNet-1K3 Deng et al. 2009
respectively. The altacks can even be ellective on robustly trained models. For a Res Ne
on ImageNet, (1, attack with radius = 2/255 can increase the ECE from 3.70 to 47.23. Performing adversarial training can partly alleviate this issue, but a (1, A-AAC attack can still increase the ECE from 9.03 to 13.54. This clearly indicates that it is possible to significantly manipulate the calibration of a model while preserving its accuracy, In Appendix B.2, we provide reliability diagrams that illustrate the effect of each (NEE attack in addition to more details on the experimental setup.
3 CERTIFYING CALIBRATION
We propose to use cerlifioation as a framework protect calibration under adversarial attacks. Hence, in this section, we first introduce existing methods to certify predictions and then build upon them to introduce certified calibration.
Prerequisites. Certification methods establish mathematical guarantees on the behavior of model predictions under adversarial perturbations. Specifically, for given input X,, the certified model issues a set of guarantecs (such as the invariance of predictions) for all perturbations "nn up to adversarial budget of R, in some norm || ||: i.c., 11 R, Practitioners deploying a system with a designated threat model of attacking budget R receive guarantee each prediction with R, 2 R. This enables an estimate of the model's performance under adversaries. Most commonly discussed is the certified accuracy at R, the fraction of certifiably correct predictions. Our method assumes two certificates.
Assumption 3.1. Model predictions are invariant under all perturbations, vll%/ & R.: F(x, + +v) F(x.).
(C1)
Assumption 3.2. Prediction confidences are bounded under perturbations, vlh% S R, (=((x), RR) S =(xn + %x) S u(=(Xn), R.).
(C2)
We further, assume that Rn, 1(.) and u(-) can be computed for each sample x state- of-the-art method to obtain such l2 certificates on large models and datasets is Gaussian smoothing (Cohen et al. 2019). smooth classifier is constructed from a base classifier by augmenting model ipputs with Gaussian perturbations, 5 ~ ((, ( d²In), and aggregating predictions. We refer to F: -RD + y as the smoothed hard classifier, returning the predicted class and providing radius Rw, for which the prediction is invariant (see Certificate C1). The certificate on the prediction has been extended by [Salman et al. (2019) and Kumar et al. the confidence. We denote the smooth model returning confidence vector as f X Ak and denote the smoothed soft classifier as E: RD - [0, 11, obtained through 3(x) maxkey fx(x 8). E indicates the confidence of prediction F. Kumar al. (2020) provide two different bounds on ž as certificate C2 the STANDARD and The CDF cert ilicate. In Appendix A.I we provide more details on these, as well as a general introduction to Gaussian smoothing.
(2020), showing certificate
3.1 CERTIFYING BRIER SCORE
As discussed earlier, the Brier score is unified measure of model's discriminative per- formance and calibration, and widely used in the literature on model calibration. We introduce the cerlified Brier score (CBS) as the worst-case (i.e. largest) top label Brier score under any perturbation 1| S over a set of N samples. We assume model returning certified predictions and providing lower and upper bounds I, u R on the top confidence z E N given the certified radius R on the input perturbation, i.e. certificate C2 Based on this, we state the following upper bound on the Brier score.
Theorem 3.3. Let 1, be the bounds on z, and be the output of a certified classifier. Further, let e RN be the indicator that predictions are correct. The upper bound on the Brier score is given by:
max TLBS(Z, c) = - llc lc - 15
(3)
where the products are element-uiss Refer to Appendir 7 for the proof.
4

---

Published as a conference paper at ICLR 2025
This bound is tight and relies on the fact that shifting the confidences leaves each certified prediction F and thus с unchanged. Therefore, an adversary cannot flip the prediction to increase the confidence gap, the sample--eeel distance between correctness en and confidence Fn. The Brier score is maximized when the confidence gap is large, the opposite of a good classifier's s expected behavior.
3.2 CERTIFYING CALIBRATION ERROR
While both the Brier score and the expected calibration error capture some notion of calibration, the confidence scores bounding the Brier score (3) do not necessarily bound the ECE. We show that the ECE can be increased even further. Therefore, we introduce a definition for the certified calibration error and provide a method to approximate it.
Definition 3.4. The certified calibration error (CCE) on a dataset D = {x., Yn IN at radius R is defined as the maximum ECE that can be observed a result of perturbations on the imput within an lg ball of radius R. Let, Yn be the perturbation on input X,- Thus, the CCE is:
CCE
max EČE ((E(,, + 7m))
(4)
This is the largest estimated calibration error that is possible on datasel if every sample is perturbed by al most. R. Such bound on the calibration is fundamentally different from a bound on the confidence: A certificate on the confidence as described in [C2 bounds the confidence per sample. The CCE in (4) bounds the error between confidence and frequentist probability across a set of samples. Finding the bound in (4) is not trivial, as the estimator of the ECE in (1) is neither convex nor differentiable. Therefore, propose numerical method to estimate (4) and provide an empirical, approximate certificate, the approzimate certified calibration error (ACCE).
CCE AS MIXED-INTEGER PROGRAM 3.3
We show in this section that we may optimize (4 by reformulating the problem into a mixed-integer program (MIP). In the following, we Tirst introduce the intuition and notation to rewrite the objective and state equivalence of (5) to (4) Theorem 3.5 Subsequently, the constraints for equivalence are introduced in detail. We conclude by restating the problem in canonical form for clarity. Readers unfamiliar with MTP may refer to Appendix A.3 for a brief introduction. In Appendix E.22 we provide numerical example to illustrate the derivations presented in this section.
While the problem in (4) is defined over perturbations 17 S. R for each input X., we rely on certificate C2 and directly optimize over the confidences. Since the ECE estimator in [ uses bins to estimate average confidence and accuracy, maximizing the estimator requires solving a bin assignment problem, where N confidence scores 2n are assigned to M bins. We show how to jointly solve this assignment problem and find the values of 2n maximizing the calibration estimator across bins.
When perturbing the input x + % the bin assignment might change: 5(x) might belong to bin m, but ă(x 7) to m' * m While the assignment is determined by the confidence score, key to our reformulation to split. these into separate variables. The motivation is that very small changes in 2n might lead to shift in bin assignment and thus contribute differently to the calibration error. We define the binary assignment an.m of confidence zn NM NMM,, where N
to bin m and accordingly define a a= [a1.1..... M1,M, a2,1....; GN.M] T E {0, is the number of samples and M the number of bins. The confidence scores maximizing the calibration error might be different across bins, iee, it is possible that the worst-case for bin m is different than for bin m' * m: = * ", Therefore, we model the confidence independently for each bin, introducing bin-specific confidences 2n,m and with z = [.., 2L,M, 22, 2N.M] T E [0, 1 Further, let e, be the indicator whether prediction n is correct, i.e., Gn = T(F(xx))
yn) and let 6n,m = Gn Σn,m, the sample confidence gap. Note that G, is independent of the bin assignment as result of certification, Le., while the confidence may shift, the prediction will remain unchanged. Analog to a and 2, we define [ei,1. M, e,.1...., CN,M E NM Now let B be stack of N identity
In this section, we relax the notation of to 2, as we are assuming smoothed models everywhere.
5

---

Published as a conference paper at ICLR 2025
matrices of size M, ie.. B [M, .... Ias] T We define E(z) (el @ B R where @ is the Hadamard product and 1 is the 1-vector of size M. We can now rewrite the calibration error.
Theorem 3.5. Let and E be defined above, and be the output of certified classifier. The calibration error estimator in [ can be expressed as:
ECE = N
||(Z)) a||1-
Marimizing over (a. z) is equivalent solving 1 when subjecting a and z to the Unique A Assignment Constraint, Confidence Constraint, and Valid Assignment Constraint below. Refer Appendia E. for a proof.
Unique Assignment, Constraint. The assignment variable a has be constrained such that each data point is assigned to exactly one bin, ie., Σm On,m = 1 Vn. To this end, we define C = In @ IM E I NNM where @ is the Kronecker product. C sums up all assignments per data point, and hence the constraint C 1==1y- 1y.
Confidence Constraint. Let 1; S Zn.m I un be the lower and upper bound on the confidence provided by the certificate C2 In addition, any confidence assigned to bin m has to adhere to the boundaries of this bin, Le.,, 1B S Zn,m I uum We can combine these two conditions to unify the bounds: ma ( = In,m I 2n.m S Un,m = min(u, "m) With this, we define I 1= [4....... 1,,,,,2,11.. 12. IN.M]] and u [u1,1..... U1,M, U2,1,..., NNN,....UN,MM and state the full constraint: 1 S u
The bounds on the confidence per bin may not intersect with the certificates on the confidence, i.., for some n, m Ih u n [ " = 0. This expected for narrow certificates on or a large number of bins. For these instances, we set z to O and define I' and u' to be and u, respectively, with the same elements set to O. Thus, the feasible constraint set on the confidence per bin is defined as S₂ = {z Vn, m 1n.m S =n,m I "/
Valid Assignment Constraint. We have identified that some confidences can never be assigned to some bins when the confidence certificates and bin boundaries do not intersect. constrain an,m 0. Let kn,m I[(n,,
For those instances, bin m is inaccessible to data point n. We define matrix K to be a NM x N matrix summing all inaccessible bin assignments. Formally, letting k = [ KI,M kg.,,...,k.. kN.ar]" and K= k1 O (IN X 1M), the constraint is: KT ON
Un,m) be the indicator that
Formal Program Statement. We summarize the constraints above and restate the program introduced in Theorem 3.5 in its canonical form for clarity. The MIP over (a, z) is given by:
maximize
E(z) a/l1 N
subject to: a 10,1) a IN. K a
(6)
This problem is sparse in objectives and constraints as C, K and E contain at most NM non-zero elements. The reformulation of 4 into (6) provides us with a useful framework to run numerical solver.
3.4 ADMM SOLVER
We propose to use the ADMM algorithm [Boyd et al., 2011) to solve 6 While ADMM has proofs for convergence convex problems, it enjoys good convergence properties even convex problems |Wang et al. 2019). ADMM minimizes the augmented Lagrangian of the constrained problem by sequentially solving sub-problems, alternating between minimizing the primal variables and maximizing the dual variables. We introduce split variables to show that the problem can be reformulated into the standard ADMM objective (sce Appendix for more details). Due the sparsity of the problem, ADMM has lincar complexity in The number of bins and samples, O(NM). In our experiments, ADMM always converges in
"In Section 3.1 on the CBS, I, u E [0, 1 are the immediate bounds on z E [0, 1) provided by the certificate the confidences. Here, I, [0, M provide bounds the expanded e NM defined in this section, now combining binning and confidence certificates.

---

Published as a conference paper at ICLR 2025
under 3000 steps and runs within minutes. We provide more details implementation, run limes, and convergences in Appendix G.I
ADVERSARIAL CALIBRATION TRAINING
After introdueing the notion of certified calibration, which exlends the notion of certification to model calibration, & natural question is whether adversarial training techniques can improve certified calibration. First, we introduce preliminaries and then novel training method, adversarial calibralion training, that can improve certified calibration.
Adversarial Training. Adversarial training (AT) is commonly used to improve the adversarial robustness of neural networks (Kurakin et al., 2017 Madry et al. 2018). While standard training minimizes the empirical risk of the model under the data distribution, AT minimizes the risk under a data distribution perturbed by adversaries. [Salman et al. (2019) propose such an AT method specifically for randomized smoothing models: The SMOOTHADV method minimizes the cross-enlropy loss, LCE, of a smooth soft classifier So under adversary 7 [ We note that the SMOOTHÁDw objective can be seen constrained optimization problem, minimizing the model risk over some perturbation, conditioning upon that perturbation being Leg-adversary on fo. In this work, we have shown the impact of adversaries on calibration and thus propose to minimize the model risk of fo under calibration adversaries, giving rise to adversarial calibration training (ACT).
Adversarial Calibration Training. We propose to train models by minimizing the negative log-likelihood under calibration adversaries. Let LCalibration be loss the calibration, and let 8 E O be the model paramcters. We propose the following objective:
min cce (Fo (xn (n), vn) BEB
s.t.
Vn arg max CCalibration 11
(Ifo
(x; + =))) (7)
for adversarial calibration training (ACT) in two flavors: Brier-ACT, which uses the Brier and ACCE-ACT, which uses the ACCE to obtain Exccr-adversarics While the former is straight.orward extension of SMOOTHADV, the second requires more thought, as we describe now.
score to find adversaries (adrier-aveersariee
The Lacce-adversary can be found by extending the mixed-integer program in 6 The purpose of the mixed-integer program in (6) is to evaluate the ACCE, thus maximizing the calibration error over confidences z and binning a. Here, we seek the adversarial input to the model, x + to perform ACT and thus reformulate the mixed-integer program w.r.t. (a, 7) instead.
More precisely, we introduce a linear function h R - RMM that maps (f(x, + 7n) + z, where K is the number of classes, N the number samples, and M the number of bins. This function selects the top confidence from fo per data point n, then replicates that value M times across all bins and concatenates over N data points (the mapping is K + R per data point). We formally state the mixed-integer program over (a, (=)) with objective
V a" arg max |E (h (Ioo (x; + m))) all1 Y.a
subject to: A (0, Ca= IN, kTa= ON, lrill S.
Implementation Details. We follow Salman et al. (2019). who solve the objective using SGD and approximate the output of jo using Monte Carlo samples. As introduced in Section 3.4. use ADMM to approximately solve the problem in (8), giving rise to versary for ACCF-ACT. We obtain z through a forward pass and update 7. Afterwards, we update other primal variables and perform
the Ceccsradversary as function of updates dual ascent steps. Further, we obtain the constraint variable K using the STANDARD C2 certificate [Kumar et al.) 2020) based on very few Monte Carlo samples, as we certify without abstaining. The full algorithm is stated in Appendix F.2
3 As above, fo denotes the model returning K class confidences, whereas returns only the top confidence. As updating model parameters, we explicitly add 0 to the model notation in this section.
7

---

Published as a conference paper at ICLR 2025
- 0.25
0.5
1.0
0.75
0.50
Method
ADMM
dECE
0.80
0.6
0.00
0.5
1550
250
3.00
Radius
Radius
Radius
Figure 3: The ACCE returned by ADMM, dECF, and the Brier confidences are shown here for ImageNet. ADMM is the most effective method, as it uniformly yields the largest bounds.
Complexity, The complexity of ACCE-ACT in the data dimensionality, D, is determined by backprop to update the adversary, 7: and hence is equal to standard training methods. Further, the objective in (8) is sparse in C, K and E, enabling the same linear scaling in batch size as the MTP b The computational cost of ACCE attacks is comparable to standard SMOOTHADV (on average 3.4% slower on CIFAR-10 and 2.6% slower on ImageNet). While additional primal and dual updates are required, they are cheap in comparison the update of the adversary that is obtained by backpropagating through the network.
5 EXPERIMENTS
We empirically evaluate the methods introduced above. First, we evaluate the proposed metrics: In Section 5.1 we discuss the cortified Brier score, and in Scction 55.2 we compare various approaches to approximate the certified calibration error and demonstrate that solving the mixed-integer program using the ADMM solver works best. Second, in Section 5.3 we evaluate adversarial calibration training (ACT) and show it yields improved certified calibration.
5.1 CERTIFIED BRIER SCORE
Experimental Method. We rely on Gaussian smoothing classifiers obtain the certifiable radius based on the hard classifier CC (Cohen et al. 2019) and bound confidences with the STANDARD certificate C2). We lollow the work on certilying confidences (Kumar et al. 2020) in our experimental selup. We use a ResNet--110 model for CIFAR-10 experiments and ResNet-50 for ImageNet trained by Cohen et al. (2019). For ImageNet, we sample 500 images from the test set, following prior work. Gaussian smoothing is performed on 100, 000 samples, and we certify at = .001. We only certify calibration at R when the prediction can be certified at R. We compute certified metrics on R E (0, 0.05, 0.15, 0.2, 0.25, 0.5, 0.75, 1.00, 1.50, 2, 2.5} for CIFAR10 and additionally 3.0 for ImageNet.
Results. The certified Brier scores for CIFAR-10 are shown in Figure 16a (Appendix H. and for ImageNet in Figure 2 For both datasets, we observe that the CBSs increases with larger certified radii. Models with small o suffer from about 100% increase in Bricr score at = 0.25, while stronger smoothed models only increase by < 50%. As strongly smoothed models yield tighter certificates on confidences (see Figure g in Appendix C we find that those models are more robust. for larger radii at the cost of performance on smaller radii.
0.26
as
0.5
Bieer
0.9 8
6000 o2 00 Cortified Radius
1,, 175 200 200
Figure 2: Certified Brier scores on Ima- geNet. For small radii, small smoothing o outperforms larger ones, but as radii increase, large outperform smaller a.
5.2 ACCE
We compare the ACCE solution obtained by ADMM with two bascline and find that ADMM yields better approximations than both. First, we obtain the confidences bounding the Brier score (the "Brier confidences", sec (3) and compute the resulting ECE as bascline. Sccond, we utilize the differentiable calibration error (dECE) (Bohdal et al., (023) and perform gradient ascent to maximize it (sce Appendix M.1p. We compare these methods for a range of certified radii and different smoothing σ.
Experimental Method. Our experimental setup is that of Section 5.1 with the difference that focus subset of 2000 certified samples for CIFAR-10.
8

---

Published as a conference paper at ICLR 2025
Results. The ACCE returned by ADMM and baselines across certified radii is shown in Figure 15 in Appendix G.3 for CIFARIO and in Figure 3 for ImageNet. We may observe that large certified radii are associaled with worse calibration. We find that ADMM uniformly yields higher ACCE than the dECE with differences up lo approximately 0.2, which a strong qualitative difference in calibration. This result suggests that solving the proposed mixed- integer program with ADMM far more effective than the other methods in approximating the CCE. For larger-scale experiments, see Appendix G.3 Interestingly, we observe that all three methods return ACCE scores that are approximately equal at large radii. We believe this is not an insufficiency of ADMM or the dECE to approximate the CCE, but rather that the CCE and CBS solutions converge. For instance, for accuracies 0 and 1, and unbounded confidences, it is trivial to see that the CCE is achieved by the same confidences as the CBS. We conjecture that this is the case for accuracies in between as well.
5.3 ADVERSARIAL CALIBRATION TRAINING
We propose to apply ACT as introduced in Section 1 as fine-tuning method models that have been pre- vre-trained using SMOOTHADW (Salman al. 2019). We refer to SMOOTHADV as standard adversarial training (AT) to disambiguate it from adversarial calibration training (ACT). We demonstrate that our newly proposed ACT methods, Brier-ACT and ACTE-ACT are capable of improving model calibration.
Experimental Method. Across all exper- iments, we compare fine-tuning with Brier- ACT and ACCE-ACT to the AT baseline. For the majority of the experiments, we fo- cus on CIFAR-10 due to the lower compu- tational cost of SMOOTHADV training and randomized smoothing. For certified radii (0.05, 0.2, 0.5, 1.0), we select. to models from Salman el al. (2019) as baselines that provide a good balance between certified cal- ibration and certified accuracy (for baseline metrics and details on the selection proce- dure, see Appendix I.2p For each baseline, we perform Brier-ACTT and ACCE-ACT for 10 epochs using a set of 16 diverse hyperparame- ter combinations (more details in Appendix [.3p. Regarding data, model architecture, and Gaussian smoothing, we follow the same setup as in Section 5.2 In contrast to Section 5.2
Table 2: Comparison of calibration for CIFAR- 10 models achieving within 3% of the highest certi- fied accuracy (CA) per training method across certified radii (from 0.05 to 1.00). Metries are the approximate certified calibration error (ACCE and certified Brier score (CBS 4).
Metric
Method
0.05
0.50
1.00
s8j
[:4,
[56, 59]
[55,
ACCE ACCK
10.00
49:30
Beler-Ac7T
0.97
27.07
41.32
CB3
8.70
10:94
28.88
Beler-ACT
CBS
we utilize the CDF certificate on confidences (Kumar et al. 2020) instead of the STANDARD certificate as these are significantly tighter and thus will be used in practice (see Appendix
C a comparison and A. for a formal definition).
For each fine-tuned model, we compare the certified accuracy (CA), ACCE, and CBS During evaluation, the reported ACCE of any model is the maximum out of 16 runs with a diverse grid of hyperparameters (see Appendix [.1. We obtain the CBS in closed form using (3)
Results. We find that both flavors of ACT are capable of significantly reducing the cortified
calibration error while not harming or even improving certified accuracy. In Table 2 we
Radius 0.05
Radius 0.2
Radius 0.5
Radius 1.0
Training
ACCEACT втеraст At
0.0
-0.1
-02
0.00 Certileed Accurnty
-2.05
0.00
Certified Accuracy
Certiffied Accuraty
Certiied Accuracy
Figure 4: This figure shows the impact of fine-tuning a model via adversarial calibration training (ACT) on certified accuracy and approzimate certified calibration error (ACCE). Each figuree presents an adversarial training baseline ("AT") at the origin, certified at the given radius. Following multiple the-tunings using Brier-ACT and ACCE-ACT, changes in metrics are depicted. The ideal corner is indicated by *. ACCE-ACT significantly improves certified calibration at larger radii
9

---

Published as a conference paper at ICLR 2025
compare all models that come within 3% of the highest certified accuracy (CA) achieved by any model at a given radius, and evaluate the best certified calibration of these models. While the effect of ACT is very small on small radii (R = 0.05 to R 0.2), for larger radii, we observe an improved ACCE (down to 9933%) and CBS (down to - -11.4%), both of which are qualitatively strong differences. In particular, we find that certified calibration of ACCE-ACT at a radius of 1.0 is better than the certified calibration of AT at a radius of 0.5. Comparing both flavors of ACT, we find that ACCE-ACT outperforms Brier-ACT on R = 1.0 while performing similarly on smaller radii. Further, the effect of ACCE-ACT on the CBS is strong, which indicates that the calibration of models trained with ACCE-ACT generalizes across calibration metrics. We find that the rank correlation between the ACCE and the CBS across all models is = 0.98, indicating a strong agreement between metrics.
In this work, we have repeatedly argued for the importance of considering calibration besides accuracy and as a result, selecting the most appropriate model is now a multi-objective decision problem. To further illustrate this, we pick the best baseline model per certified radius and plot the changes in ACCE against certified accuracy for each fine-tuned model in F In these figures, the top left quadrant is strictly dominated by the pre-trained model, while models in the lower right corner improve upon accuracy and calibration (indicated by *). Using ACT, we are able to jointly improve upon certified accuracy and calibration. In line with the results in Table 2 improvements on small radii are limited. However, for a certified radius of 0.5, we observe decreases in ACCE of 7.5% while increasing the accuracy by 1.5 %. For a radius of 1.0, we observe an improvement of the ACCE of 12% without harming accuracy. This is a strong qualitative difference. In Appendix L.4 we replicate Figure for more baseline models and demonstrate that improvements are also possible on baselines with better certified calibration. Further, Appendix [.5] provides ablation studies showing that fine-tuning with AT cannot achieve the same calibration as ACT.
We replicate these experiments on FNshionMNIST, SVHN, CIFAR-100, and ImageNet, and present results in Appendix [J] We demonstrate consistent improvements across datasets with the exception of ImageNet, for which only marginal effects are visible (1-2%). This is in line with prior work optimizing calibration, which does not report ImageNet (Bohdal et al. 2023 Obadinma et al.) 2024) or struggles to find consistent effects (Karandikar et al.) 2021 See
Appendix K for discussion of trade-offs between calibration and robustness.
RELATED WORK
Only few papers discuss the confidence scores on cortified models. Jeoong et al. (2021 p propose a variant of micup (Zhang et al.] 2018a) for certified models to reduce overconfidence in rumner-up classes with the goal of increasing the certified radius, but do not examine confidence as uncertainty estimator. Others propose using conformal predictions on certified models to obtain certifiable prediction sets linchr wider body of literature relates ad adversarial robustness to calibration on non-cerlilied models. Grabinski et al.] (2022) show that robust models are better calibrated, while other work shows that poorly calibrated data points are easier lo allack (Qin et al.) 2021)) This is used by the latter to improve calibration through adversarial training. [Stutz et al. (2220), utilize confidence scores and calibration techniques to improve adversarial robustness. Few works investigate the calibration of uncertainty calibration under adversarial attack (Sensoy let al.] 2018; Tomani and Buettner 2021 Kopetzki et al. 2021), however, their work is not necessarily applicable here, as these do not study certified models.
al.) 2019 Gendler et al. 2022). A
While some works provide bounds on calibration in various contexts, none of them are applicable to our sctup (Kumar et al. 2019 Qiao and Valiant 2021 Wenger et al. 2020
Further, few works have proposed to directly optimize calibration. Kumar et al. (2018) propose a kernel- to effectively reduce calibration error. BBhddal et al. (2023) develop the dECE and use meta- learning to pick hyperparameters during Training Lo reduce the dECE on a validation set. A
measure of calibration that they use to augment cross-entropy training
similar metric has been proposed by Karandikar et al. 2021 as SB-ECE alongside S-AvUC that increases uncertainty on incorrect samples and decreases uncertainty on correct samples. Similarly, Obadinma et al. (2024) use a combination of surrogate losses to attack calibration. However, they do not directly train on calibration metrics.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS
This work is supported by UKRI grant Turing AI Fellowship (EP/ W002981/11 C. Emde is supported by the EPSRC Centre for Doctoral Training in Health Data Science SSP/S0248XX/1 and Cancer Rescarch UK (CRUK). A. Bibi has received an Amazon Research Award. F. Pinto's PhD funded by the European Space Agency (ESA). T. Lukasiewicz is supported by the AXA Research Fund. The authors also thank the Royal Academy of Engineering and FiveAI. In addition, the authors thank Mohsen Pourpounch for his support.
REFERENCES
Moloud Abdar, Farhad Pourpanah, Sadiq Hussain, Dana Rezazadegan, Li Liu, Mohammad Ghavamzadeh, Paul Fieguth, Xiaochun Cao, Abbas Khosravi, U. Rajendra Acharya, Vladimir Makarenkov, and Saeid Nahavandi. A review of uncertainty quantification in deep learning: Techniques, applications and challenges. Information Fusion, 76:243 297, 2021. ISSN 1566-2535
Naveed Akhtar and Ajmal Mian. Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey. IEEE Access, 6:14410-1440 2018.
Mislav Balunovic and Martin Vechev. Adversarial Training and Provable Defenses: Bridging the Gap. In International Conference on Learning Representations (ICLR), 2020.
Adel Bibi, A. Alqahtani, and B. Ghanem. Constrained Clustering: General Pairwise and Cardinality Constraints. TEEE Access, 11:5824 5836, 2023. ISSN 2169-3536.
Ondrej Bohdal, Yongxin Yang, and Timothy Hospedales. Meta- CictaCaaibrttionn Learning of Model Calibration Using Differentiable Expected Calibration Error. Tronsactions on Machine Leorning Research, 2023. ISSN 2835-8856.
Stephen Boyd, Neal Parikh, Eric Chu, Borja Peleato, and Jonathan Eckstein. Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers. Foundations and Trends in Machine Learning, 3(1):1-122 2011. ISSN 1935-8237.
Glenn W. Brier. Verification of Forecasts Expressed in Terms of Probability. Monthly Weather Review, 78(1):1, January 1950.
Jochen Bröcker. Reliability, sufficiency, and the decomposition of proper scores. Quarterly Journal of the Royal Meteorological Society, 151(619):1512-111,, 2009. ISSN 0035-9009. Place: Chichester, UK.
Jeremy Cohen, Elan Rosenfeld, and Zico Kolter. Certified Adversarial Robustness via. Randomized Smoothing. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 1310-1320. PMLR, June 2019.
Morris H. DeGroot and Stephen E. Fienberg. The Comparison and Evaluation of Forecasters. Journal of the Royal Statistical Society. Series D (The Statistician), (2(1)2):1 22, 1983. ISSN 00390526, 14679884. Publisher: [Royal Statistical Society, Wiley].
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. ImageNet: large- scale hierarchical image database. In Proceedings of the IEEE Conference Compuler Vision and Pattern Recognition (CVPR), 2009.
Ruediger Ehlers. Formal Verification of Piece- Wise Linear Feed-Forwal Neural Networks, 2017. URL atttps://arxijy org/ /ttbs///..... 01320
Ido Galil and Ran El-Yaniv. Disrupting Deep Uncertainty Estirnation Without Harming Accuracy. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P. S. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information. Processing Systems, volume 34, pages 21285-21 21296. Curran Associates, Inc., 2021.
11

---

Published as a conference paper at ICLR 2025
Asaf Gendler, Tsui-Wei Weng, Luca Daniel, and Yaniv Romano. Adversarially Robust Conformal Prediction. In International Conference on Learning Representations (ICLR), 2022.
lan Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and Harnessing Adversarial Examples. In International Conference on Learning Represenlations (ICLR), 2015. URL http://arxiv.. .rgg abs/ 4572
Sven Gowal, Krishnamurthy Dvijotham, Robert Stanforth, Rudy Bunel, Chongli Qin, Jonathan Ucsato, Relja Arandjelovic, Timothy Mann, and Pushmect Kohli. On the Effectiveness of Interval Bound Propagation for Training Verifiably Robust Models, 2019.
URL /tttps://axii org/ abs/ 1810. 12715.
Julia Grabinski, Paul Gavrikov, Janis Keuper, and Margret Keuper. Robust Models less Coee-Coni,dent. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors, Advances in Neural Information Processing Systems, 2022.
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. 34th International Conference on Machine Learning, ICML 2017, 3: 2130 2143, June 2017. ISSN 9781510855144. URL https: www.wworg//ttte wwtggabb/17700 04599
Kaiming He, X. Zhang, Shaoqing Ren, and Jian Sun. Identity Mappings in Deep Residual Networks. In European Conference on Computer Vision, 2016a.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep Residual Learning for Image Recognition. In Proceedings of the IEFE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016b.
Yotam Hechtlinger, Barnabás Póczos, and Larry Wasserman. Cautious Deep Learning, 2019. ab/0000..1156000
URL httts:///rxiiy
Like Hui and Mikhail Belkin. Evaluation of neural qrchitectures trained with square loss vs cross-entropy in classification tasks. In International Conference on Learning Representations (ICLR), 2021.
Jongheon Jeong, Sejun Park, Minkyu Kim, Tleun--hang Lee, Do-Guk Kim, and Jinwoo Shin. SmnothhMx: Training Confidence-calibrated Smoothed Classifiers for Certified Robustness. In M. Ranzato, A. Beygelzimer Y. Dauphin, P. S. Liang, and J. Wortman Vaughan, editors, Advances in Newral Information Processing Systems, volume 34, pages 30153 30168. Curran Associates, Inc., 2021.
Archit Karandikar, Nicholas Cain, Dustin Tran, Balaji Lakshminarayanann, Jonathon Shlens, Michael C Mozer, and Becca Roclofs. Soft Calibration Objectives for Neural Networks. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P. S. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems, volume 34, pages Associates, Inc., 2021.
Curran
Guy Katz, Derek A. Huang, Duligur Tbeling, Kyle Julian, Christopher Lazarus, Rachel Lim, Parth Shah, Shantanu Thakoor, Haoze Wu, Aleksandar Zeljić, David L. Dill, Mykel J. Kochenderfer, and Clark Barrett. The Marabou Framework for Verification and Analysis of Deep Neural Networks. In Isil Dillig and Serdar Tasiran, editors, Compuler Aided Verification, pages 443 443-452, Cham, 2019. Springer International Publishing. ISBN 978-3- 030-25540-4.
Guy Katz, Clark Barrett, David L. Dill, Kyle Julian, and Mykel J. Kochenderfer. Reluplex: a calculus for reasoning about deep neural networks. Formal Methods in System Design, 60(1):87-116, February 2022. ISSN 1572-8102.
inna-Kathrin Kopetzki, Bertrand Charpentier, Daniel Zügner, Sandhya Giri, and Stephan Günnemann. Evaluating Robustness of Predictive Uncertainly Estimation: Are Dirichlet- based Models Reliable? In Marina. Meila and Tong Zhang, editors, Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pages
PMIR, July 2021.
12

---

Published as a conference paper at ICLR 2025
Alex Krizhevsky. Learning Multiple Layers of Features from Tiny Images. 2009.
Ananya Kumar, Percy Liang, and Tengyu Ma. Verified Uncertainty Calibration. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d' texttquuoeei Alché-Bue, E. Fox, and R. Garnett, editors, Advances Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019.
Aounon Kumar, Alexander Levine, Soheil Fcizi, and Tom Goldstein. Certifying Confidence via Randomized Smoothing. H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 1165-5177. Curran Associates, Inc., 2020.
Aviral Kumar, Sunita Sarawagi, and Ujjwal Jain. Trainable Calibration Mcasures for Neural Networks from Kernel Mean Emboddings. In Jennifer Dy and Andreas Krause, editors, Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pages 2805-2814. PMLR, July 2018.
Alexcy Kurakin, Ian J. Goodfellow, and Samy Bengio. Adversarial Machine Learning at Scale. In International Conference Learning Representations (ICLR), 2017.
Yann LeCun, Lćon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-basod learning applicd to document recognition. Proc. IEEE, 86:2278-221 23224, 1998.
Pantelis Linardatos, Vasilis Papastefanoppulos, and Sotiris Kotsiantis. Explainable Al: A Review of Machine Learning Interpretability Methods. Entropy, 23(1), 2021. ISSN 1099-4300.
Changliu Liu, Tomer Arnon, Christopher Lazarus, Christopher Strong, Clark Barrett, and Mykel J. Kochenderfer. Algorithms for Verifying Deep Neural Networks. Foundations and Trends in Optimization, 4(3-4):244 404, 2021. ISSN 2167-3888.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards Deep Learning Models Resistant Adversarial Attacks. In International Conference on Learning Representations (ICLR), 2018.
Mark Huasong Meng, Guangdong Bai, Sin Gee Teo, Zhe Hou, Yan Xiao, Yun Lin, and Jin Song Dong. Adversarial Robustness of Deep Neural Networks: Survey from a Formal Verification Perspective. IEEE Transactions on Dependoble and Secure Computing, pages 1, 2022.
Matthew Mirman, Timon Gehr, and Martin T. Vechev. Differentiable Abstract Interpretation for Provably Robust Neural Networks. In International Conference on Machine Learning, 2018.
Jishnu Mukhoti, Viveka Kulharia, Amartya Sanyal, Stuart Golodetz, Philip HS Torr, and Puneet K Dokania. Calibrating Deep Neural Networks using Focal Loss. Advances in Neural Information Processing Systems, 2020.
Allan Murphy. Scalar and Vector Partitions of the Probability Score: Part II. N-State Situation. Journal of Applied Meteorology (1962-1982) 11(8):1183-1192 January 1972. Publisher: American Meteorological Society.
Allan H. Murphy. A New Vector Partition of the Probability Score. Journal of Applied Meteorology, 12(():595-60) 600, 1973.
Mark Niklas Müller, Franziska Eckert, Mare Fischer, and Martin Vechev. Certified Training: Small Boxes are All You Need. In International Conference on Learning Representations (ICLR), 2023.
Mahdi Pakdaman Naeini, Gregory F Cooper, and Milos Hauskrecht. Oblaining Well Calibrated Probabilities Using Bayesian Binning. Proceedings of the ... AAAT Conference on Arlificial Intelligence. AAAI Conference on Artificial Intelligence, 2015:2901 200-29977297 January 2015. ISSN 2159-5399.
13

---

Published as a conference paper at ICLR 2025
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y. Ng. Reading Digits in Natural Images with Unsupervised Fealure Learning. In NIPS Workshop on Deep Learning and Unsupervised Peature Learning 2011, 2011.
Khanh Nguyen and Brendan O'Connor. Posterior calibration and exploratory analysis for natural language processing models, 2015.
Jeremy Nixon, Michael W. Dusenberry, Linchuan Zhang, Ghassen Jerfel, and Dustin Tran. Measuring Calibration in Deep Learning. In Proceedings of the IEEE/ CVF Conference on Computer Vision and Pallern Recognilion (CVPR) Workshops, June 2019.
Stephen Obadinma, Xiaodan Zhu, and Hongyu Guo. Calibration Atlack: A Framework For Adversarial Atlckk Targeling Calibration; 2024.
Adain Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Yang. Zach DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. PyTorch: An Imperative Style, Hiigh-Performacce Deep Learning Library, 2019. URL https: org/abs. abs/ 1912.01703
Mingda Qiao and Gregory Valiant. Stronger Calibration Lower Bounds via Sidestepping, In Proceedings of the 53rd Annual ACM SIGACT Symposium on Theory of Computing, STOC 2021, pages 456-466, New York, NY, USA, 2021. Association for Computing Machinery. ISBN 978-1-4503-8053-9 event-place: Virtual, Italy.
Yao Qin, Xuezhi Wang, Alex Beutel, and Ed Chi. Improving Calibration through the Relationship with Adversarial Robustness. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P. S. Liang, and J. Wortman Vaughan, editors, Advances in Nearal Information Processing Systems, volume 34, pages 14358-14369. Curran Associates, Inc., 2021.
Hadi Salman, Jerry Li, Nlya Razenshteyn, Pengehuan Zhang, Huan Zhang, Sebastien Bubeck, and Greg Yang. Provably Robust Deep Learning via Adversarially Trained Smoothed Classifiers. In H. Wallach, H. Larochelle, A. Beygelzimer, F. Alché- E. Fox, and R. Garnett, editors, Advances in Newral Information Processing Systems, volume 32. Curran Associates, Inc., 2019.
Murat Sensoy, Lance Kaplan, and Melih Kandemir. Evidential deep learning to quantify classification uncertainty. In Advances Neural Information Processing Systems, volume 2018-Decem, pages 3179 3189, June 2018. URL
01768
Samuel Henrique Silva and Peyman Najafirad. Opportunities and Challenges Deep Learning Adversarial Robustness: A Survey, 2020. URL
wwtpaaawwwww..aaa//a/aaa//a////a 00753
Gagandeep Singh, Timon Gehr, Markus Püschel, and Martin Vechev. An abstract domain for certifying neural networks. Proc. ACM Progrum. Long., 3(PPLL), January 2019. Place: New York, NY, USA Publisher: Association for Computing Machinery.
David Stutz, Matthias Hein, and Bernt Schiclc. Confidence- Calibrated Confidence-Caiibratee Adversarial Training: Generalizing to Unscen Attacks. In Hal Daumé III and Aarti Singh, editors, Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pages 9155-9166. PMLR, July 2020.
Christian Szegedy, Wojeiech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. In International Conference on Learning Representations (ICLR), 2014.
Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jonathon Shlens, and Zbigniew Wojna Rethinking the Inception Architecture for Computer Vision. In Proceedings of the IEEE Computer Sociely Conference on Compuler Vision and Pollern Recognition, volume 6- 6DDeeem pages 2818 2826, December 2016. ISBN 978-1-46633-885-4 URL http: 7/arxiv. www. ta.bs//5120000
14

---

Published as a conference paper at ICLR 2025
Christian Tomani and Florian Buettner. Towards Trustworthy Predictions from Deep Neural Networks with Fast Adversarial Calibration. Proceedings of the AAAI Conference on Arlificial Intelligence, 35(11):9886 9896, May 2021.
Shiqi Wang, Kexin Pei, Justin Whitehouse, Junfeng Yang, and Suman Jana. Efficient Formal Safety Analysis of Neural Networks. In Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, Advances in Newral Information Processing Systems, volume 31. Curran Associates, Inc., 2018.
Shiqi Wang, Huan Zhang, Kaidi Xu, Xue Lin, Suman Jana, Cho-Jui Hsieh, and J Zico Kolter. Efficient bound propagation with per-neuron split constraints for complete and incomplete neural network verification. Advances Neural Information Processing Systems, 34, 2021.
Yu Wang, Wotao Yin, and Jinshan Zeng. Global convergence of ADMM in nonconvex nonsmooth optimization. Journal of Scientific Computing, 78:29-68 2019. Publisher:
Springer.
Jonathan Wenger, Hedvig Kjellström, and Rudolph Triebel). Non-Parametric Calibration for Classification. In Silvia Chiappa and Roberto Calandra, editors, Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics, volume 108 of Proceedings of Machine Learning Research, pages 178- 178-190. PMLR, August 2020.
Eric Wong and Zico Kolter. Provable Defenses against Adversarial Examples via the Convex Outer Adversarial Polytope. In Jennifer Dy and Andreas Krause, editors, Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pages
PMLR, July 2018.
Baoyuan Wu and Bernard Ghanem. eell p-Box ADMM: A Versatile Framework for Integer Programming. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41( 1695 1708, July 2019. ISSN 1939-3539.
Han Xiao, Kashif Rasul, and Roland Vollgraf. MNIS::: a Novel Image Dataset for Benchmarking Machine Learning Algorithms, August 2017. arXiv: cs. 170.0777.77777
Runtian Zhai, Chen Dan, Di He, Huan Zhang, Boqing Gong, Pradeep Ravikumar, Cho- Jui Hsieh, and Liwei Wang. MACER: Attack-free and Scalable Robust Training via Maximizing Certified Radius. In International Conference on Learning Representations (ICLR), 2020.
Hongyi Zhang, Moustapha Cisse, Yann N. Dauphin, and David Lopez-Paz. mixup: Beyond Empirical Risk Minimization. In Indernational Conference Learning Represenlations (ICLR), 2018a.
Huan Zhang, Tsui- Wei Weng, Pin-Yo Yu Chen, Cho-Jui Hsieh, and Luca Daniel. Efficient Neural Network Robustness Certification with General Activation Functions, 2018b. URL https://arrx wtt.aaiii...og/b/aaaa/aa/a www./oggg///booooooo... ab1///1../////../../11 0886.
15

---

Published as a conference paper at ICLR 2025
A PREREQUISITES
A.1 GAUSSIAN SMOOTIINGG
As some readers might not be familiar with certification and Gaussian smoothing, we provide brief introduction here. However, before getting into the details of Gaussian smoothing, we restate some notation introduced above and reiterate the certification assumptions from Section 2.2 for readability.
Certification. Let f: X. + AK be any model outputting K-dimensional probability sim- plex over labels y. We denote the k-th component of f as fx. We obtain the classifier indicat- ing the predicted label F; = X as the maximum the softmax: F(x) = arg maxkey fx)).. We refer to this as the hard classifier. We denote the function indicating the confidence of this prediction using z : X + [0, 1], which we obtain by =(x) = maxkey fx(x). We refer to this as the soft classifier.
Our method assumes two certificates. Assumption A.1. Model predictions are invariant under all perturbations, VI[ Rn: F(x, + +%, F(x.).
(C1)
Assumption A.2. Prediction confidences are bounded under perturbations, Vlhal & Rx: (=(xx), Rr) S xxx, + 7x)= S u(2(xn), Rn).
(C2)
We further, assume that R,, I(-) and u(.) can be computed for each sample x,-
Gaussian Smoothing. We achieve these guarantees using Gaussian smoothing (Cohen et al. 2019 Gaussian smoothing scales even to large datasets such as ImageNet and is agnostic towards model architecture. First, we obtain certificate C1 explain some nuances of Gaussian smoothing and then introduce certificate C2
Gaussian smoothing takes any arbitrary base model F and constructs a certified model CCohen et al. 2019). The smooth hard classifier, F, is obtained by:
F(x.) = arg max Po (F(x, + 5,,) = c). cEY where 8, is sampled from an isotropic Gaussian, iee. 8n (0,,02Ip) show the certificate C1 for F with a radius Rr given by
Cohen el. al. (2019)
σ ④ (p.)) R
2 - (p!))
(10)
2
where Tb donotes the Gaussian CDF, Pd denotes the probability of predicting class A, i.c., (x + 8n) = ca), and y denotes the probability of predicting the runner-up class: m (x + 5m)) = cn).
The true F, PA and p are unknown and have to be estimated. The authors propose to use a finite number of Ganssian samples to estimate F(x,), a lower bound p on p and a an upper bound P on p which are used to estimate certifiable radius At We choose a siguificance level
(0, 1) in estimating the model (usually a .001). With probability larger 1 1-a, the true model certifiably predicts the same class as the estimated model with a true radius of R. 2 Rr, the evidence is insufficient to certify a prediction threshold a, the model abstains.
This work has boen extended by Kumar et al. (2020) to certify confidences, i.c. obtaining certificate C2. We denote the smooth model as
Eslf + 8n.],
(11)
from which we obtain the smooth soft classifier as =(xx) maxtey (x,). Kumar el. al. 20200) propose two certificates that we apply on ž, which refer to as the STANDARD
16

---

Published as a conference paper at ICLR 2025
certificate and the CDF certificate mentioned above. The STANDARD certificate is very closely related to the certificale given by Cohen el al. (2019) and bounds the confidences as
Lo (
RD S 2(x, + 7n) S (2 (z(xn) ((xx))(x)) + R)
(12)
given a radius R where 2(xn) and ((x,) are lower and upper bound on a(x,) obtained using Hoeffding's inoquality, and 4 is the Gaussian CDF with standard deviation a. They further propose the CDF certificate. It utilises extra information to tighten the cortificate: the cCDF is estimated and the Dvoretzky-Kiefer- -iffeettwwtffeeolloowttta incquality is used to find cortificate bounds at threshold partition the interval [0, 1]. The bounds are given by
a. Let so = 0 and Sn+1 = 1 and let so < .... I < Snt1
Σ (8) 15-1) 2o Φ-- (pc,( 1t) S xxxn + 7n) 1=1
(8j+1 - 83) 13)
tb (01² (x=))xx)) + R), (13)
where Ps, is the lower bound on in the j-th bin of the eCDF, and Ps, the upper bound.
A.2 OTHER CERTIFICATION METHODS
This paper employs Gaussian smoothing (Appendix A.1 due to its scalability to large models and datasets, competitive certifiable sets, and its prominence in the literature. Nonetheless. a variety of alternative methods exist for obtaining certificates as defined in Equations C1 and C2 Certification in machine learning is closely related to the broader field of formal verification, which focuses on proving system correctness.
One significant branch is abstract interpretation, which includes techniques such as interval- bound propagation (IBP) (Mirman et al.) 2018 Gowal et al., 2019). CROWN (Zhang et al.
2018b its variant 3-CROWN (Wang al. 2021 and DeepPoly (Singh et al. 2019 Other
methods in this category leverage satishability modulo theories (SMT) solvers
Ehlers 2017
Katz et al. 2019 2022
Relaxation- techniques, such as Neurify (Wang al., 2018), approximate non-linear constraints with computationally efficient relaxations to enable certification. Certified training has also emerged as a crucial direction to improve model robustness by integrating certification constraints into the training process, resulting in models that are inherently casier to certify. For instance, [Gowal et al. (2019) enhance IBP-based certified training by improving bound tightness during training, building on the foundational work of DiffAl (Mirman et al. 2018), which introduced differentiable abstract interpretation for robustness.
Methods such as COLT (Balunovic and Vechev 2020) and SABR Müüller el al. 2023 focus
improving adversarial robustness while achieving strong certification pertormance. While these methods do not exclusively target certified training, they represent significant progress in combining adversarial robustness and certification.
This overview, while not exhaustive, highlights key approaches the field. For comprehensive reviews, readers are referred to Liu et al. (2021), which covers verification algorithms, Silva and Najafirad (202)). which highlights practical challenges in robustncss, and Meng et al. 2022), which provides insights into adversarial training and certification.
A.3 MIXED INTEGER PROGRAMS
For readers unfamiliar with mixed-integer programming we state the canonical form of a mixed-integer program here.
mixed-intedd mixee-integer program (MIP) is a constrained optimization problem over variables (a e N y E ZM) that can be written in the following form: min f(x v)
(14)
2.
17

---

Published as a conference paper at ICLR 2025
subject to:
for i = 1..,.... K h y) for j 1, 2, ..... L
(= v) S 0
In our case the constraints are linear, leading to constraint statements such as Ax = b Cy=d.
(15)
which are introduced in Section after the statement of Theorem
B MOTIVATION
B.1 CALIDRATION + ACCURACY
It is important to note that accuracy and calibration measure different concepts, and one may not infer model performance from calibration with cerlainly or vice versa. Consider the following examples, where we fix one quantity and construct datasels resulting in other quantity taking on opposing values. These are illustrated in Figure 5 First, we fix the calibration error to ECF = 0.5 and for Case 1, we construct N data points with label Yn L, confidence 2n 0.5 and thus prediction Dn = The calibration error here is 0.5 (represented by the horizontal line between (z = 0.5, accuracy is 1. For Case 2, our predictions remain the same, but change the labels to yn = 0 resulting in an accuracy of O while keeping the calibration error of 0.5. Next, we fix the accuracy to and construct examples with ECE 0.5 and ECE 0. The former is given by Case 1. The latter (Case 3) can be constructed using ys. = and 2n = 1. Thus, we can construct distribution over Z and Y such knowing the accuracy tells nothing about the calibration error and vice versa. Clearly, when evaluating the quality of predictions, it is insufficient to only assess the accuracy. Hence, argue that certifying accuracy in safety relevant applications insufficient and calibration should be considered.
1) and (z 1. 1)) and the
ECE=0.5
Accuracy=1
Case
Case 1
Case
Label (0)
Case
Accuracy
ECE
1.0
0.5
0.0
0.5
1.0 0.0
0.5
1.0
Confidence
Confidence
Figure This visualization shows that can fix either the accuracy the calibration and construct dataset to obtain the other quantity with opposite values. The example looks binary classification problem. The empty circle displays the point of perfect calibration and the full circle is the calibration on the data. The distance of the line in between is the calibration error.
B.2 EMPIRICAL ATTACKS
Training. We use publicly available chockpoints for the "standard training" model on ImageNet. In particular, we use the PyTorrh pretrained v1 model weights. For CIFAR-10 experiments we train PreActResNet18 model from scratch for 100 epochs. For the robust models we utilize standard adversarial training using PGD and train from scratch for CIFAR- 10 while fine-tuning for ImageNet. We use increase of adversarial budget over 25 epochs to 3/ 255 and train for 100 epochs for CIFAR-10. For ImageNet we use 10 epochs to warm-up the adversarial budget and train for 50 epochs.
18

---

Published as a conference paper at ICLR 2025
Attack. We perform attacks using the criterion described in 2 Using PGD enforce the constraint on || S e. Before applying a gradient update, we check the condition F(r 7)) = F(x) is still satisfied. If this is violated, we stop the oplimization. We use learning rate of 10 for 100 steps with automatic reduction when the loss plateaus. For further details please refer to the published code.
There are differences to Galil and El-Yaniv (2021] p in the theoretical objective and the practical setup. Their ACE attack directly optimises confidence scores to increase them when the model incorrect and decrease them when the model is correct. This what our (1, y)A attack does when setting objective x (in their notation) to be the softmax output. However, the loss is different. While Galil and El-Yaniv directly optimise the confidence score (i.e. the top softmax output), we optimise the entire softmax output using cross-cntropy. Despite these smaller differences, in essense, their attacks are special case of ours. Regarding the implementation there are various smaller differences e.g. in the learning rate scheduling and attack methods. For example, while Galil and El- El-Ya use FGSM, while use PGD. They manually reduce learning rates when the prediction invariance condition would no longer be satisfied by an update, whereas we automatic plateauing scheduling. For further details plase refer to Galil and El- Yaniv 2021
Reliability diagrams.
Callbrstio
Calbeator PM
Bratinn Hoc
Onme
144
(b)
Culbrettio Pot
Callbr
1.
3.2
02
33
34
30
044
(4)
(+)
Figure 6: Reliability diagrams a ResNet50 trained with expected risk minimization on ImageNet-
1K and attack radius = 2/255. (a) No altack, (b) (L,y)-ACE atlack, (c) (-1,))-AEE attack, (d)
(1. A)-ACE atlckk, (e) (-1, A)EECE attack. The histogram on the bottom represents the distribution of confidence scores.
19

---

Published as a conference paper at ICLR 2025
Callenad
Calosaie1
2309
6.
6-6
10
24
Cellaadie PAA
1001
[4]
Figure Reliability diagrams for a ResNet50 trained with adversarial risk minimization on ImageNet-
1K and attack radius = 2/255. (a) No attack, (b) (1,3)-ACE attack, (c) (-1,y)-ACB attack, (d) (1. AC)ACE attack, (e) (- ACE attack. The histogram on the bottom represents the distribution of confidence scores.
20

---

Published as a conference paper at ICLR 2025
C CONFIDENCE BOUNDS
o 0.12 R
0.12 R
o |R 0.25
o 0.12 0.5
0. 0.12
1.0
0.8
0.6
0.4
0.2
0.0
0= 0.25 88= 0.05
0.05
a= 0.25 RR
0 0.25 8 0.25
0= 025 1 0.5
0 0.25 8 11.00
1.0
STANDWRD sloumd
0.8
0.6
0.4
022
0.0
1.0
0-
0.5
R 0.05
o- 0.5 - 0.2
- 0.5 R 0.25
0 0.5 RR 0.5
8 1.0
STANDARD Souun
0.8
0.6
0.4
0- 1.0 |R 0.05
o- 1.0 | 0.1
1.0 R 0.25
o- 1.0 R---- 0.5
0- 1.0 R 1.0
0.8
0.2
6.15
100
225
6.75
8.0
025
0.75
100
6.00
9
6.9
190
8.78
1.00
CDF-Bound
CDFdd
Figure 8: Distance between upper and lower bound of the confidence certificates provided by Kunar al. 2020). Sub-sampled to 500 samples from the test set of CIFAR10.
Kumar el al. 20200 introduce bounds on the confidence score in order to issue a certificate the prediction, given the lower bound. Therefore, they do not investigate the upper bounds on the confidence. Here, we compute the interval of certified confidences and compare the two certificates on the confidence: The standard STANDARD bound given in (12 and the more advanced CDF bound. In Figure S the distance between the upper and lower bound plotted for a range of smoothing and certification radii R on random subset of 500 samples from the CIFAR-10 10 test set. The underlying models are trained by Cohen et al. 2019). We may observe that the CDF method yields uniformly tighter bounds in practice.
D BRIER BOUND
Here we provide a proof Theorem 3.3 stated above.
Proof. Let If-} be an clement-wer wise indicator function. Assume z' IIfc = ulfc = 0} is not the maximum and we want to change z' to maximise the TLBS. For some data point with en 1, the proposed confidence is the bound 2 Ln Reducing 2 is not possible houh leaving its leasible set. (4, < x,, < u,), and thus, the only way to find the maximum to increase it. However, increasing z would reduce G. =x which in turn reduces the error, as || || strictly increasing in 6, 2n. Thus, we have a contradiction for 6, = 1. The other case, C. 0 is analog and both true for all and thus, the maximum proven. n
21

---

Published as a conference paper at ICLR 2025
E CALIBRATION AS MIXED-INTEGER PROGRAM
E.1 RESTATING THE CALIBATIIO ERROR
Here, we provide a proof of Theorem 3.5
Proof. We plug ① into (4) and show equality to the maximumn over 55 Let E(z) and a be as defined above and denote dn 2n the gap between confidence and correctness. Further, let en denote the m-th column E.
max NEČE (lz(x, + 7o)l = max NEČE (z. Vall R
(16)
Isa5n
M
Σ |mml
max ISRCH m=1
Bm
Bm
DE Bi
WE B
(17)
max
Σ On
Zn
(18)
1825u
Bm
max
d
(19)
Bm
M
max
Cn Gn,m
(20)
Iszsu
m==1
M
max
e (z) T
(21)
Kzsu
m=
max 1s25u
E all
(22)
The equality in [6) is given we translate the constraint on [nm] into the confidence constraint, 1 < z < u. The equality assumes the certificate is tight. The equality between
19 and 20 is result of the unique assignment and valid assignment constraints.
Dividing both sides by N yields the result.
E.2 NUMERICAL EXAMPLE FOR MIP
In this section, we state a simple numerical example that illustrates the derivation of the Mixed Integer Program (MIP) as presented in Section 3.3 We first define the parameters of this example and then show what values cach variable defined in Scction 3.3 takes on.
Setup. We consider a problem of N = 2 samples predicting classes y {C Cr)-
Table 3: Example data for numerical example.
F(xx)
Un
I(E(x,,, R)
E(x,)
u(ā(x,,, R)
CA
CA
0.1
0.23
0.6
CA
CB
0.5
0.78
0.9
We regard the expected calibration error with M = 3 bins and equal-width binning. Hence, the bins are [0, 1/3), [1/3, 2/3), and [2/3, 1).
22

---

Published as a conference paper at ICLR 2025
MIP variables. We obtain the vector of correctness for each prediction: c= [1 0]" Next, we obtain the vector and u. To this end, we first obtain I and u from the table above: 1 = [0.1 0.5] and u = [0.6 0.9] We further obtain 1 and u from the binning
scheme of the ECE. 1 [0 1/3 2/3] and u =1/33 2/3
1] Using these we define
I and u.
[0.1 1/3 2/3
0.5
0.5 2/3] T
(23)
u 1/3 0.6 0.6 1/3 2/3 0.9]
It should become clear that bin 3 is inaccessible to sample n and bin m 1 is inaccessible to sample n Hence, [0 u':
0
0
0]
Using this we define I' and
1' 00.1 1/3
0.5
2/3]
(24)
u' 1/3
0.6
0
2/3
0.9
We define the vector of confidences z with inaccessible elements set to 0 (see "Confidence Constraint"). The vector of confidences is not uniquely defined through the problem above.
One example is z = [0223 0.45
0.6 0.78) together with a [1
0
0
0 0 1)_
We now obtain 6n,m Cn mn to define [0.77
0.55
0.6 0.78]. Finally,
obtain E by stacking e appropriately.
E1,1
0
0
0.77
0
0
122
0
0
0.55
0
11,3
0
0
1
E
(25)
E2,
0
0
6,
0
0
0.6
0
C,,3
0
0.78
The last variable that is value dependent is the matrix K, which can obtain from k as follows:
K1,1
0
0
f1.2
k1,3
0
K
(26)
12,1
0
12.2
0
k2,3
E.3 LAGRANGIAN
We follow Wu and Ghanemi 2209 and Bibi al. (0023) to reformulate the binary-constrainis Note that, E {0, NM NMM 22-sphere, both centered at We introduce auxiliary variables qi E S,, and qa S2 and qı and q2. Similarly, we replace the constraint z E S, by enforcing it on g and adding z = Updates on the primal variables (a, z, qı. q2, g) are performed via gradient descent.
S, S2, where Sp is the unit hypercube and S2 is the
add constraints
We formally state the augmented Lagrangian. The variables a, 2, qı, q2, and are described above, each of dimension NM.
23

---

Published as a conference paper at ICLR 2025
C(a, z, 91,2 A123455 = -E((z) (a(z)
(9) Sb) + {| aa qil
PI |a q1 11112
+ +lxx E 52 + A aa q2l + P2 a qS
(27)
+ + 1n] 2 C a 1N
A K T a P4 K aal²
1g E S + \o 2 g
gl
with dual variables A1,2, E ERAM, A E RR Here, Inc if the statement is true and oo otherwise. The values of ps > 0 are hyperparamelers to be luned.
E.4 ADMM UPDATES
We perform T ADMM steps. At each ADMM step cycle through the primal variables and z and perform gradient descent. For the variables qı, q2 and g, we obtain an analytic solution by equating the gradient to 0, solving the equation and projecting the variables into their feasible set. For qı the update Ug given by
A1 ( P1
q1 - clampjo.1
+a
(28)
for q2 the update Uua is given by
92 2
+
VNM
2
P2
a
2
(29)
2
and finally, the update Ug is given by
As
g - clamp p wn)
+
z
(30)
Ps
The updates on the dual variables X, are performed through a single gradient ascent step with slep size Pi-
With these definitions, we can formalise the algorithm for the ADMM updates.
Algorithm 1 The ADMM Updates
input ADMM parameters {oa, Oz2 01,3,443, primal variables { (a()) 2(0) ,q. .2 .8
(0)
and dual variables { output ACCE
for ( to T do
(1) + (t (t-1) 0a VaL
(t-11) (t (t-1), ,gggt (tt-)), (t-1) 21233.43)
प.. (t-1) g (t-1) (t-1) 1123111)
(0) t (111) 0x VaL
(1())
2
g (1) + Ug
(1)) 1 (t-1))
q Uqi
((1) 1 (1-1)
(1) Uap
(1 (1-1)
4
A2
(t) (t 1)
pi V2, L ((())(()
(0) (9 > (t-1)
99.. 123.455 for = 2,4,5
end for
24

---

Published as a conference paper at ICLR 2025
F ADVERSARIAL CALIBRATION TRAINING
F.1 LAGRANGIAN
We formally state the augmented Lagrangian. The variables a, 7: qı, q2 and are described above, each of dimension NM.
C(a,7 7, 91,2, 22333 = 7) aallM
+ l9 E {.) + AT Ja a- q-l +
P1
||a q l1
2
+ Imo 192 E 82) + X
a q2
+
P2
a q2ll
(31)
2
+ 1 [C a - 1vl +
Ps
C a
1N
2
+ x K + 2
K a 2
with dual variables 21,2 RMM A3.4 R.V. Here, T if the statement true and co otherwise. The values of p; > 0 are hyperparameters to be tuned.
Note, that this still constrained optimization problem in y.
F.2 ACT ALGORITHM
The algorithm here follows the general notation introduced above. Additionally, we define f to be the Monte Carlo approximation fo with V samples and FY the approximation to Fe. We further denole updates on the variables using 4. Updates are performed as in Appendix E.4 if not stated otherwise. Here, we use Uren to describe descent. In the algorithm below, the cossect and the inaccessibility matrix, K, are explicitly added, these require forward pass to be computed.
update by projected gradient
25

---

Published as a conference paper at ICLR 2025
Algorithm 2 Adversarial Calibration Training- One Batch
input Data (n InInn.n..,, InIn=l..... ADMM parameters {ay, az P123A), perturbation bound
and L = ACCE
output 0
Yn (0)
f 0 {Initialise Adversary)
(0) + [1/,,//M..... 1/M] {Iuitialise binning vector)
(0) (") (Initialise for Box Constraint]
qi
92 + a()) (Initialise for Sphere Constraint]
(0)
A + 0 for i= 1,,,2,3, (Initialise dual variables}
for to
T do
(t-1) - (J)'
Xn + + ++)))") n=I....N
if t= then K is created as described in section 3.3 for the ACCE constraint.)
Here, as function of 2()) {This is required c [T(u Pg (xn + (0)) n= (The correctness required for the ACCE loss.)
end if
(1) Upan (7((-)) a (tt-1),c,t) c..)
(2)
+ Ua
2
(((-1), ..
(1-1) (1-1)
-2
1,2,,44 K,
- Uqi
21 (1-1)
- Uq
a(t) A
((-1)
X + A (t-1)
V. 2
(t-1) (2) for = 1,2,3,4 A 123311
.9.2
Update Parameters: P1:4
end for
U (NV (x + x(11)) (Update model parameters)
G EXPERIMENTS ACCE
G.1 ADMM CONVERGENCE
In this section we present some insights and figures that closer examine the run time and convergence of ADMM on the mixed-integer program 5 Numbers reported in this section are based on CIFAR-10.
Run Time. In our experiments, ADMM always converges in under 3000 steps. Our implementation utiliscs the torch torch.sparse package in version 2.0 (Paszke al. 2019) and runs in less than 2 minutes for 7000 certified data points and 15 bins on a Nvidia RTX 3090. At convergence, we observe that all constraints are sufficiently met, and hence ADMM provides a feasible solution (see below). We validated a number of optimization hyperparameters and fix those for later experiments (see Appendix G.2 We recommend running ADMM on two different initializaatin of z, the observed adversary-free confidences, as well as those achieving the certified Brier score (3) and subsequently picking the larger ACCE. We find that the maximum ACCE is achieved well before ADMM has converged. Therefore, we recommend projecting copy of and into their feasible set after each step and calculating the ACCE.
Convergence Diagnostics. Figure shows the development of various components of the Lagrangian (see (27 in Appendix E.3 across steps. Figure 10 shows how well the constraints are met. Finally. Figure 11 plots parts of the assignment vector, a, across steps. All plots
are based on 2000 CIFAR-10 predictions of model trained by (Cohen et al. 2019) with
0.25.
More precisely, Figure plots seven metrics across 3000 steps of running ADMM. In the first. row, you may observe the Lagrangian (ignoring the Tox components). In the second row, you may observe the soft objective divided by N, which the relaxed ECE. Further, we project the solution to exactly satisfy the constraints, the projected ECE. The third and fourth plot
26

---

Published as a conference paper at ICLR 2025
Lagrangian
0
-2000
ECE ECE (Projected)
ECE (Relaxed)
0.6
0.4
0.2
0.0
500
Box Constraint
0
-500
Sphere Constraint
200
200
Unique Assignment
100
M00 Mmm
0
-100
1000
Constraint I S
0
-100
100
Feasible Assignment
1000
500
1000
1500
2000
2500
3000
Figure 9: Components of the Lagrangian for ADMM to find the ACCE on CIFAR-10 with e = 1.0. Labels explained in Appendix G.1 Observe, that all constraints are fully met.
are the Box and Sphere Constraints to a binary constraint a. The fifth row plots the constraint on the confidences I 1 z S u. The sixth row shows the Unique Assignment constraint and the seventh and final row shows the Valid Assignment constraint. You may observe, that all components converge well.
described above. Together they are equivalent
To closer describe Figure 10 we first introduce some notation: For some vector x and set S, we define (x, S) minyes ||p- Each subplot in Figure 10 tracks one constraint across the ADMM steps. For each constraints we provide distance from the constraint being met, i.c. the projection distance to the feasible set. We measure whether the confidence is bounded using dz, S2) where S {z 1 z < u). The Binary Assignment constraint is measured by dp A), where A is the sel of all binary vectors. Deviation from the Unique Assignment and Valid Assignment constraints are given by ||CT a 1|pp and K T a pp: respectively.
and oo norms for the
27

---

Published as a conference paper at ICLR 2025
10"
Norm
Confidence is bounded
10
107²
1
10-4
10
10a
Norm
Asignment Binary
Lop
10
10-2
10-3
10a
Norm
Unique Asignment
10-1
10-?
10-4
Norm
Impossible Asignment
100
10-1
10-2
10-3
500
1000
1500
2000
2500
3000
Step
Figure 10: The deviation of constraints of the mixed-integer program given in For precise delinitions of the y-axis please refer to Appendix G.I
Finally, we investigate the values of the assignment vector closer. In Figure 11 we plot the first 150 elements of a for steps O, 500 and 3000. These elements are the assignments of 10 data points to 15 bins. As you can the initialization of is uniform, while "preference" for some bins becomes apparent at 500 steps and finally, at 3000 steps the vector has converged to near-perfect binary vector.
G.2 ADMM HYPERPARAMETER SEARCH
We performed random hyperparameter search for the ADMM solver to find reasonable hyperparameters that are efficient across experiments.
The learning rate for 2, Ozz: We test. values from 1 x 10-5 to x 10-2 and find it has little influence.
28

---

Published as a conference paper at ICLR 2025
Step
Step 500
Step 3000
1.0
0.4
0.2
0,
m,
n.m
Figure 11: The first 150 elements of the assignment vector in the mixed-integer program across steps of the ADMM solver. The vector is initialized uniform but converges to a binary vector.
The learning rate for a, Oa: We test values from 5 10 to 5 10 for ImageNet and from x 10-5 to 1 x 10 for CIFARIO. Larger learning rates are preferred. In our experiments 0.001 to 0.05 worked best.
The Lagrangian smoothing variable pr: We tested starting values from 0.001 to 0.5. We find little influence but values around 0.01 to 0.05 works best. We test multiplicative schedules increase Pt over time with increases starting 1% to 4%. While larger values of improve constraint convergence, they can dominate gradients when the constraints have not sufficiently converged yet, and thus ADMM easily diverges. Schedules with factors around 1.02 to 1.05 per step are effective and stop increasing P at 10, which is completely sufficient to meet the constraints. We find, that Pi scheduling is important. Applying the same schedule described above for all Pı...., P5 works reasonably well.
We test performing 1 to 3 updates for and z per ADMM step and find that more steps do not aid results while slowing down ADMM. We thus, recommend step per ADMM step.
We test clipping values of a. As constrain q2, clip s.t. ||a 1/2 S 1.2 - 1/2 Note that when a converges, it will be significantly smaller than this constraint. We have never observed any decline in performance with this approach but seen that it stabilizes the optimization rare instances.
We clip the gradients of to 5 in infinity norm aiding stability.
The initialization of has a major effect on the performance of ADMM, more so than any other hyperparameter. While it is an obvious solution initialize ( such that it is valid assignment for (()), find that this is majorly outperformed by uniform initializations. We tested 0, 1/M and 1 and find that 1/M works best.
The initialization of z has a mild effect on ADMM performance given that sometimes, we might have prior knowledge how and in which direction the model currently is miscalibrated. We recommend initializing it with adversary-free confidences and with the confidences achicving the Brier bound and subscquently picking the larger onc.
G.3 ADMM vs. DECE VS. BRIER BOUND
To further assess the efficacy of the ADMM algorithm, we compare an alternative methods of approximating bounds: the differentiable calibration error (dECE) (Bohdal ct al. 2023) and the bounds obtained by the confidences that maximise the Brier score (Brier confidences). We find that ADMM outperforms both other techniques. We performed hyperparameter searches for ADMM and dECE with wide range, but carefully selected hyperparamelers. For ADMM we run between 259 and 1560 trials (depending on the runtime) and for dECE always 2000 trials. We explore subset of radii and smoothing a (as visible from our plots). Our first observation is, that the dECE is very sensitive to the initialization of confidence scores indicating that gradient ascent dECE (even with very large learning
29

---

Published as a conference paper at ICLR 2025
rates) does not sufficiently explore the loss surface. While differences also observable for ADMM, these are very small. Fig Figure 12 show these differences for ImageNet.
Method ADMM o= 0.25
Method ADMM ==
Method ADMM -= 1.0
0.6
0.5
0.3
0.2
Initialliaation
Method dECE| 0.25
Method dECE a 0.5
Method dECE 1.0
Adverary-free
confidence
0.6
0.5
0.2
0.1
0.05
0.2
0.5 1.0
2.0
0.05
0.2
0.5
1.0
2.0
0.05
0.2
0.5
1.0
2.0
Radius
Radius
Radius
The results of the hyperparameter search are shown here. For each combination of dalasel, a, radius, and inilializalion of z. the maximum achieved by the three methods, ADMM, dECE and Brier are shown here. ADMM performs well regardless of the initialization of The dECE is depends highly the initialization.
Figure 12:
Second, result of our hyperparameter search, we note that the best ADMM results outperform the best dECE by a significant margin. To demonstrate this, we compare the maxima achieved across trials by the dECE, the Brier bound and ADMM in Figure 13
Decase: Immagehars-ee IK| a 0.12
Dutaset Imagetett-ik 0.25
Dataset ImageNet-1K o 6.5
Decioe: Imagghher-1 IK| a 1.0
0.7
0.4
0.3
0.1
Mathod
Dessee:
GFAR-10
2.0
ader
XDMM
0.6
0.3
0.2
RadBuss
RarRus
Redius
Radius
Figure 13: The results the hyperparameter search are shown here. For each combination of dataset, a, and radius, the maximum achieved by the three methods, ADMM, dECE and Brier shown here. While dECE is able to be uniformlly better than the Brier confidences, ADMM outperforms both by significant margin.
As noted in the main paper, for large radii, the methods converge to each other. Beyond the comparison of maximum values above, we find that even single one-size-fie fett-fftllllle all set of hyperparameters for ADMM outperforms the maximum achieved by the dECE hyperparam- eter search in most cases shown in Figure [4] The hyperparameters used for ADMM are as follows: All p;s are initialised to 0.01 with increases every step by a factor of 0.4%. Both lcarning rates set to 0.001. The assignments initialised to 1/M, the confidence is
30

---

Published as a conference paper at ICLR 2025
initialised to adversary- and Brier confidence, mcasure benefiting the dECE much more than the ADMM. The gradients of the assignment variable are clipped to 1.
Dataset Imagelet-KK |K a 0.22
Dataset mageNett11 a 025
Dataset ImageMet-3K] or Dataset Imagetiet
0.7
0.4
0.2
Dataset CIFAR-10 0.12
Dataset CFAR-10 025
Dataset CIFAR-10 0.5
Dataset CFAA-10 1-1.0
Biter
0.5
0.3
0.2
0.5
6.0
0.5
030
0.5
6
1.00
0.59
020
0.50
1.00
280
Radius
Redis
Figure 14: We compare a single set of parameters for ADMM across all dataset, smoothing and radius combinations and find that it performs at least as good as the mazimum dECE from hyperparameter search of 2000 samples in 17/19 instances.
We use our results from the hyperparameter searches to run ADMM and dECE on a finer grid of certified confidences referenced in section 5.2 Here, present the results for the The results for ImageNet are shown in Figure
finer grid for CIFAR-10 (see Figure 15
2.12
025
U5
1.41
8 035
Method
Anted
Figure 15: The ACCE is shown here for ADMM, dECE and the Brier confidences. One set of carefully selected hyperparameter is used. For a wider overview see Figure 13
H EXPERIMENTS BRIER SCORE
Cerlittd Brier Score
0.25
0.25
0.8
0.5
0.6
Cortified Brier
0.4
0.4
0.2
0.2
0.00
8.10
6
60
25
50
775
2.00
2.50
0,06
6:10
6.15
15
1.
175
2000
Certified Radius
Radius
(a) Certified Brier scores (CBS) CIFAR-10 (b) Certified Brier scores (CBS) CIFAR-10 across range of certified radii for models with across range of certified radii for models with different smoothing Each CBS based all different smoothing a. Each data point is based certifiable samples that radius.
on the same set data points per model.
In Figure 16a we plot the certified Brier scores on CIFAR-10 for a range of certified radii. Observe that. the CBS increases significantly with certified radius and there is trade-off
31

---

Published as a conference paper at ICLR 2025
for CBS governed by d: Heavily smoothed models achieve worse CBS small radii but maintain it better for larger radii.
In 16b we plot the CBS on a constant subset of data that is certifiable at all shown radii, Hence, the certified accuracy constant this plot. We observe that these "most certifiable samples" maintain their CBS very well for small and medium radii. The CBS on these samples significantly lower than the CBS on all certifiable samples as shown in 16a
I EXPERIMENTS ADVERSARIAL CALIBRATION TRAINING
L1 ACCE EVALUATION PARAMETERS
In order to evaluate the ACCE during experiments on Adversarial Calibration Training, we report the maximum value over grid of 16 runs. We do this to prevent overfitting the ACCE on certain set of hyperparameters and thus ensure ACCE-ACT is not given an unfair advantage during evaluation. We run ADMM 16 times and pick the largest ACCE across them. The lyperparameters are given by the product of following parameters:
Initialization of the confidences using the observed, clean confidences, and the Brier confidences.
The learning rate for the confidences is 0.001 or 0.01.
The learning rate for the binning variable is 0.01 or 0.1.
The multiplicative factor for scheduling of the p.'s for the ADMM solver: 1.004 or 1.01.
1.2
BASELINE MODELS
This section provides more details on how models from Salman et al. 2019 were chosen for fine-tuning.
CIFAR-10. In Table 14 and 15 in Salman et (019)), the authors list the parameters of their adversarial training runs and the certilied accuracy per radius. For each certified radius we pick the best PGD result from each table (see Table 4 and perform certification on these checkpoints with 100,000 certification samples. We then use these cortificates to obtain the ACCE and CBS, and verify that our certified accuracy lies within +3% of [Salman et al. (2019'ss reported results. Finally, we pick one or multiple models per certified radius on (or very close to) the Pareto curve between certified accuracy and ACCE, i.e., for each of the models there exist no other model that simultaneously improves ACCE and certified accuracy (also Table 4 For all radii pick the model achieving the highest accuracy and for larger radii we also pick two more models that are already better calibrated (at the cost. of accuracy) to investigate whether our training can improve calibration on these models too. We use these models as baseline for fine-tuning.
ImageNett For ImageNet, the authors list their models in Table 6 in Salman al. (0119). For each certified radius we pick the best model. Analog to CIFAR10, we perform certification, verify that results close to Salman et al. (2019) and certify calibration. The results are shown in Table 5 We use each of these four models for fine-tuning.
32

---

Published as conference paper at ICLR 2025
Table 4: Certified metrics for the best models from Salman 2019
CIFAR-10. Resulls based 100,000 certification samples and the full CIFAR-10 test We report approximate certified calibration error (ACCE), certified accuracy (CA), the certified brier score (CBS) and whether that model chosen baseline for our fine-tuning experiments. The hyperparameters follow Salman et al. [2019] notation: T the number adversarial steps, Merain number of Monte Carlo samples, a the smoothing parameter and e is the L2 upper bound on the adversary.
T
Mirrii
ACCE
CA
CBS
Baseline for Radius
0.05
0.20
0.50
1.00
0.05
0.20
0.50
1.00
0.05
0.20
0.50
1.00
0.25
1.00
15.95
29.22
49.90
70.34
65.60
55.66
0.00
16.75
18.97
31.14
0.50
2.00
20.77
26.78
44.99
64.24
52.16
49.71
44.92
37.03
25.15
26.70
30.96
45.93
0.12
0.25
11.06
27.94
84.31
76.50
0.00
0.00
9.12
11.21
0.05, 0.20
0.25
0.50
17.85
21.39
42.89
76.00
70.16
55.71
0.00
12.64
13.88
21.21
0.50
1.00
2.00
24.60
26.36
36.52
50.66
44.93
42.44
38.00
30.87
28.32
29.55
32.39
37.46
0.12
0.50
12.09
30.43
83.44
76.67
0.00
0.00
9.54
13.56
10
0.12
0.50
17.14
35.30
77.76
71.34
0.00
0.00
12.84
18.71
335
10
0.50
0.50
18.26
31.00
38.13
53.38
62.31
58.07
48.17
32.64
18.50
20.41
23.34
32.17
10
0.50
1.00
15.98
31.13
44.16
59.81
57.27
54.32
47.06
35.10
22.04
23.93
28.16
41.36
10
0.50
2.00
19.93
24.94
45.73
64.35
48.48
46.08
41.42
33.89
25.74
27.11
31.18
46.18
10
0.50
0.50
17.98
30.26
33.59
50.97
64.11
59.08
49.40
32.62
17.06
18.61
20.94
28.67
0.50
10
2
0.50
1.00
16.21
30.61
39.51
56.36
61.02
57.43
49.50
36.10
19.66
21.40
24.97
36.25
10
0.50
2.00
22.34
29.03
45.76
64.59
52.59
50.16
44.88
37.20
25.92
27.38
31.53
46.52
1.00
10
1.00
2.00
22.70
24.18
35.11
46.71
43.10
41.01
36.90
30.40
27.53
28.62
31.05
36.13
1.00
10
0.12
0.25
11.32
28.50
83.41
75.96
0.00
0.00
9.63
12.06
10
0.25
1.00
15.21
27.25
49.30
72.13
67.24
56.98
0.00
15.45
17.52
28.96
10
0.50
0.25
19.29
27.93
29.87
44.96
66.23
60.10
47.07
26.89
15.04
16.40
16.79
22.00
10
0.50
1.00
16.28
29.62
37.69
52.78
63.46
58.87
49.54
34.30
18.97
20.57
23.51
32.90
1.00
10
0.12
0.25
10.90
27.83
84.00
75.78
0.00
0.00
8.70
10.94
10
0.25
0.50
17.20
20.53
41.34
76.78
70.45
55.20
0.00
12.01
12.94
19.51
10
0.25
1.00
15.21
28.68
49.65
73.43
68.58
57.62
0.00
15.22
17.25
28.88
0.50
10
1.00
2.00
26.14
28.13
41.08
50.96
46.36
44.03
39.32
31.35
28.36
29.52
32.34
37.40

---

Published as conference paper at ICLR 2025
Table 5: Certified metrics for the best four models from Salman et al. (2019 on ImageNet. Results are based on 100,000 certification samples and 500 test set samples. For notation refer to the caption of Table 4 For all models 7
and Merain = 1.
ACCE
CA
CBS
0.05
0.50
1.00
2.00
3.00
0.05
0.50
1.00
2.00
3.00
0.05
0.50
1.00
2.00
3.00
0.25
0.50
15.81
50.18
66.00
58.00
0.00
0.00
0.00
16.65
32.23
0.50
1.00
15.46
36.30
54.24
57.60
52.80
45.60
0.00
0.00
17.82
23.40
35.16
1.00
2.00
13.19
32.53
37.97
56.78
80.93
42.80
38.60
35.80
28.80
21.80
19.72
22.78
27.14
40.29
66.25
1.00
4.00
16.05
28.47
38.67
56.38
82.57
37.60
34.00
30.40
24.40
20.40
21.05
23.22
26.91
40.91
68.88

---

Published as a conference paper at ICLR 2025
I.3 FINE-TUNING PARAMETERS
experiments. We fine-tune various model checkpoints provided by (Salann et al. (2019) Generally, we adopt the number of Gaussian perturbations (Marain in |Salman et al. (2019 notation, V in ours), the attack boundary and the smoothing a from the model checkpoint
In this section, we provide more details on the hyperparameterrs for CIFAR-10
trained
before. We train using SGD with batch size 256 and weight decay of 0.0001. We use gradients obtained through backpropagation recommended by (Salman et al. 2019 opposed to Stein gradients. We further follow the authors in setting the learning rate for the attack as function of and T, i.e. the attack bound and the number of optimization steps. However, allow for some scaling and use a learning rate 20 with factor e as additional hyperparameter.
Brier-ACT. We fine-tune for 10 epochs with a linear warm-up schedule for that reaches full size at epoch 3. We decrease the learning rate for the model weights every epochs by a factor of 0.1. We train 16 models per baseline model that share these parameters, but form grid over the factor to scale attack learning rates E {0.01, 0.1, 1.0, 2.0}, weight learning rates f0.01, 0.05} and the number of steps used in the attack, T E {1.4].
ACCE-ACT. We fine-tune for 10 epochs with schedule and weight learning rate schedule as described above. As the calibration loss is a distributional measure over a set of samples, larger batch size is advisable and thus all of the runs are performed a batch size of 2048. We train 16 models per baseline with hyperparameters sampled from the Cartesian product of the following iyperparameters: The attack learning rate scaling factor E {0.01, 0.1, 1.0, 2.0}, number of steps T {1, 4, 10, 20}, weight learning rates {0.01, 0.05), and the factor in the multiplicative schedule of μ E {0.01, 0.05, 0.1, 0.5) in the Lagrangian that governs how early the constraints will be enforced.
I.4 CIFAR-10 10 FINE-TUNING EFFECTS
CIFARIO for all baseline models. Above, we compare fine-tuning methods on the baseline models with the highest. certified accuracy for each radius. However, due to the trade-off identified between certified accuracy and certified calibration, these models tend to have poor certified calibration. Therefore we also investigate whether, ACT is able to improve certified calibration on baselines that are naturally better calibratod. In Figure
17 you may observe that for each model that is certified for a radius R 2 0.5, significant improvements are possible.
[=0.12:644-4422
R- 9-0 0.50pa0.26::222mmmmm.
3.50 25555 8-T200
0.20
0.60
Faining
0.35
0.50
ACCE
0.55
0.35
0.30
0.45
IrtercCT
0.50
025
0.40
0.45
0.05
020
0.35
0.40
0.80
0.85
0.90
0.70
0.75
0.80
0.50
0.55
0.60
0.50
0.55
0.60
0.65
0.50 2:30
1.0000 |=1.0| (5000 255 4T
1.00 c 512 2T.00
R
1.000 510 2
0.45
0.75
0.35
0.70
8 0.35
0.55
0.50
0.65
0.45
0.30
0.60
0.25
0.43
0.55
0.40
0.45
0.55
0.30
0.35
0.40
0.30
0.40
0.45
0.30
0.35
Certified Accuracy
Cartified Accurary
Certified Arcurecy
Certified Accuracy
Figure 17: Each sub-figure depicts the effects of fine-tuning on the accuracy and ACCE for one pre-training model checkpoint. The pre-training model checkpoints ("AT") lie on the Pareto curve between ACCE and accuracy at the radius shown in the title. You may observe the ACCE-A ACT yields significant ACCE improvements for all models at larger radii. These are achieved without adverse effects on the accuracy for all but the last model 9R=1.00 | 1-1.00-e-512-m-----110)))
35

---

Published as a conference paper at ICLR 2025
L5 ABLATION STUDIES
To investigate the effectiveness of ACT beyond further fine-tuning using standard AT, we also fine-tune the baseline models with AT. We the same hyperparameters as for Brier-ACT as outlined in L.3 The results are shown in Table 6 (the sctup is equivalent to Table 2 You may observe that fine-tuning the model using AT yields similar results Brier-ACT but does not show the same effectivencss as ACCE-ACT, It is well-known that cross- cross-entropy training is equivalent to maximum likelihood estimation. It is further known that the log likelihood is proper scoring rule such as the Brier score. Hence, it is not surprising that careful utilisation for the cross-entropy loss is capable improving calibration. However, literalure suggests that the Brier score can be more elfective in training for calibration and cross-entropy lends to lead to overconfident predictions (Guo et al. 2017 Mukhoti et al. 2020 Hui and Belkin 2021 In Figure 18
we show that. line-tuning ACCE-ACT outpertorms AT on all tested
baselines,
Table Comparison of calibration for CIFAR-10 models achieving within 3% of the highest certified accuracy (CA) per training method across certified radii (from 0.05 1.00). Metrics are the approximate certified calibration error (ACCE, 4) and certified Brier score (CBS 4).
Metric
Method
0.05
0.20
0.50
1.00
CA
[83, 86]
[74, 78]
[56, 59]
[5, 38]
ACCE
AT
10.90
27.83
49.30
56.36
ACCE
Finetuned AT
9.99
27.15
42.72
51.71
ACCE
Brier-ACT
9.97
27.07
41.32
52.13
ACCE
ACCE-ACT
10.06
27.22
42.16
47.08
CBS
AT
8.70
10.94
28.88
36.25
CBS
Finetuned AT
8.09
10.12
20.86
31.89
CBS
Brier-ACT
8.10
10.20
19.38
32.54
CBS
ACCE-ACT
8.82
10.73
20.94
24.87
-0255-127--4322
0.00
Training
0.35
0.50
0.15
BrienACT
undd
0.50
0.25
0.40
0.35
0.40
0.00
0.80
0.75
0.50
0.65
0.75
0.60
0.55
0.85
0.50
0.45
0.25
0.45
0.55
0.40
Certiheed Acouracy
Certifies
Certified Acturacy
Certiidd Accuracy
Figure 18: Fine-tuning results for every model checkpoint including Fine-tuning using AT.
J EXPERIMENTS ADVERSARIAL CALIBRATION TRAINING FURTHER DATASETS
J.1 FASHIONMNIST RESULTS
We replicate the CIFAR-10 experiments on FashionMNIST (iao et. al. 2017 and find similar effects.
36

---

Published as a conference paper at ICLR 2025
J.1.1 EXPERIMENTAL SETUP
We replicate the CIFAR-10 experiments exactly, except for the following differences.
We train own baselines as Salman et al. 2019 do not discuss FashionMNIST. We use the hyperparameters they report for CIFAR-10, however, we use linear warm-up schedule for a over 8 epochs. Further, we set = 0 for epochs and then use their warm-up schedule for 8 epochs. These adaptions were required prevent models from diverging for 0.5.
We certify 500 samples on FashionMNIST rather than the full test-set on CIFAR-10 due to the cost. of randomized smoothing.
We additionally certify radius =
We follow Zhai et al. (2020), who apply LeNet (LeCun et al. 1998 MNIST and apply it to NIST.
J.1.2 RESULTS
Below we present Table for FashionMNISST, that replicates Table 2 that come within 3% of the model with the highest certified accuracy for each certified radius. We compare the best calibration achieved by any model. We observe that the Brier-ACT performs similar to AT (standard adversarial training). ACCE-ACT outperforms both other methods.
We compare models
Table 7: Comparison of calibration for FNSS models achieving within 3% of the highest certified accuracy (CA) per training method across certified radii (from 0.05 to 1.00). Metrics are the approximate certified calibration (ACCE 4) and certified Brier (CBS 4).
Metric Method
0.05
0.10
0.20
0.50
1.00
CA
[87, 90]
[8, 89]
[82, 85]
776, 79]
600, 63]
ACCE
AT
11.44
13.71
16.51
42.55
48.03
ACCE Brier-ACT
10.96
12.96
15.20
42.35
48.03
ACCE ACCE-ACT
10.32
11.26
13.19
39.62
43.78
CBS
AT
7.73
7.93
8.34
22.32
27.87
CBS Brier-ACT
7.05
7.37
7.88
21.95
27.34
CBS
ACCE-ACT
6.76
7.07
7.28
17.77
21.51
J.2
SVHN RESULTS
We replicate the CIFAR-10 experiments on the Street View House Number (SVHN) dataset Netzer et al. 2011 and find similar effects.
J.2.1
EXPERIMENTAL SETUP
We replicate the CIFAR-10 experiments exactly, except for the following differences.
We train our own basclines as Salman et al. (0099 do not discuss SVHN. We use the hyperparamoters they report for CIFAR- 10, however, use lincar warm-up schodule for a over epochs. Further, we sct c 0 for epochs and then use their warm-up schodule for epochs. These adaptions were required to prevent models from diverging for a 2 0.5. The basclines perform similar to (Zhai et al. 2020).
We certify 500 samples on SVHN rather than the full test-set on CIFAR-10 due to the cost of randomized smoothing.
We additionally certify radius R = 0.1.
J.2.2 RESULTS
Below we present. Table 8 for SVHN, that replicates Table 2 We compare models that come
within 3% of the model with the highest certified accuracy for each certified radius. We
37

---

Published as a conference paper at ICLR 2025
compare the best calibration achieved by any model. We observe that the Brier-ACT performs better than AT (standard adversarial training) on small radii. ACCE-ACT outperforms both other methods across all conditions.
Table 8: Comparison of calibration for SVHN models achieving within 3% of the highest certified accuracy (CA) per training method across certified radii (from 0.05 to 1.00). Metrics are the approximate certified calibration error (ACCE 4) and certified Brier score (CBS 4).
Metric
Method
0.05
0.10
0.20
0.50
1.00
CA
89, 92]
[87, 90]
[81, 84]
[61, 64]
[27, 30]
ACCE
AT
14.31
15.04
18.42
41.09
60.18
ACCE
Brier-ACT
11.55
12.63
15.09
44.37
61.98
ACCE
ACCE-ACT
11.40
12.15
14.20
39.29
49.68
CBS
AT
6.79
6.53
7.40
18.25
41.64
CBS
Brier-ACT
4.73
5.08
5.58
18.56
44.26
CBS
ACCE-ACT
4.49
4.51
5.45
16.21
26.08
J.3 CIFAR-100 RESULTS
We replicate the CIFAR-10 experiments on CIFAR-100 (Krizhevsky 2009) and find similar effects.
J.3.1 EXPERIMENTAL SETUP
We replicate the CIFAR-10 experiments exactly, except for the following differences.
We train our own baselines as Salman et al. (2019) do not discuss CIFAR-100. We use the hyperparameters they report for CIFAR-10, however, we use linear warm-up schedule for a over epochs. Further, we set = 0 for epochs and then use their warm-up schedule for 8 epochs. These adaptions were required to prevent. models from diverging for 0.5.
We certify 500 samples on CIFAR-100 rather than the full test-set on CIFAR-10 due the cost. of randomized smoothing.
We additionally certify radius R 0.1.
J.3.2 RESULTS
Below we present Table g for CIFAR-100, that replicates Table 2 We compare models that come within 3% of the model with the highest certified accuracy for each certified radius. We compare the best calibration achieved by any model. We obscrve that the Bricr-ACT performs best for small radii and ACCE-ACT for large radii. Both outperform the AT basclinc.
Table 9: Comparison of calibration for CIFAR-100 models achieving within 3% of the highest certified accuracy (CA) per training method across certified radií (from 0.05 1.00). Metrics are the approximate certified calibration error (ACCE J) and certified Brier score (CBS 4).
Metric
Method
0.05
0.10
0.20
0.50
1.00
CA
|5, 48]
[43, 46]
[38, 41]
[28, 31]
[15, 18]
ACCE
AT
29.00
32.23
35.52
55.77
60.46
ACCE Brier-ACT
25.14
31.54
32.91
56.89
55.88
ACCE
ACCE-ACT
27.21
30.70
30.95
51.85
53.11
CBS
AT
19.08
20.77
23.86
37.80
43.71
CBS
Brier-ACT
18.09
19.98
21.13
39.60
36.99
CBS
ACCE-ACT
18.78
20.33
22.16
34.35
32.04
38

---

Published as a conference paper at ICLR 2025
J.4 IMAGENET RESULTS
In this section, we provide details for the adversarial training experiments we conducted on ImageNet (Deng et al. 2009
J.4.1 EXPERIMENTAL SETUP
For ImageNet we pick the best. PGD model for each certified radius reported by Salman et al. (2019) their Table 6. These baselines and their certified metrics can be found in For these four models, we perform fine-tuning with Brier-ACC and ACCE-ACT, The experimental sctup similar to that for CIFAR-10 (see Section 5.3). We train with batch size 256 for Bricr and 1024 for ACCE, and fine-tune 6 models per bascline. The hyperparameters for these models are the product of weight learning rates {0.001, 0.005, 0.01] and attack learning rate factor of {1.0, 5.0} (see I.3 for an explanation). For all ACCE-ACT models we the same ADMM parameters (p 0.01). Due the cost of certification, only 10'000 Gaussian noise samples to obtain the smooth model.
J.4.2 RESULTS
In Table 10 we compare models that come within 3% of the model with the highest certified accuracy for each certified radius. We compare the best calibration achieved by any model per training method. The calibration ACT is consistently, but only marginally below that of AT. The best performing method here is "ACCE-ACT"" which is variant of ACCE-ACT in which we update model weights using loss nCcB (f(xt>"))) (CC)))(C (("(f(((+)))))) y). We pick between 0.8 and 0.99, the ACCE may be magnitudes larger than the CE. We think using the ACCE as part of the loss for updating the weights is a promising direction, but leave this for future, work.
Table 10: Comparison of calibration for ImageNet models achieving within 3% of the highest certified accuracy (CA) per training method across certified radii (from 0.05 1.00). Metrics are the approximate certified calibration error (ACCE J) and certified Brier score (CBS 4).
Metric
Method
0.05
0.50
1.00
2.00
CA
[66, 69]
[57, 60]
[44, 47]
[27, 30]
ACCE
AT
23.08
63.69
66.09
68.33
ACCE
Bricr-ACT
21.60
62.85
64.29
68.58
ACCE
ACCE-ACT
22.45
62.62
64.99
67.92
ACCE
ACCE-ACT*
21.67
61.96
64.23
67.56
CBS
AT
17.20
42.95
45.84
49.71
CBS
Brier-ACT
16.45
41.69
43.48
49.01
CBS
ACCE-ACT
16.79
41.73
44.41
48.17
CBS
ACCE-ACT*
16.28
40.84
43.75
47.91
K ROBUSTNESS: CALIBRATION-TRADE
Previous literature accuracy certification identifies robustness-accnne -cccuracyccccayyy trade-off governed by a smoothing parameter a: Strongly smoothed models achieve high certified accuracy on large radii, but tend to perform worse within small radii and vice versa (Cohen et al. 2019 Salman et al.) 2019). We observe a similar trade-off for certified calibration: Models with small provide tighter certified calibration small radii, while heavily smoothed models maintain certified calibration much better. This becomes apparent for both the ACCE (see Figure 15 and CBS (see Figure 16a
Further, we observe an association between calibration (ACCE and CBS), certified accuracy (CA) and average certified radius (ACR) (Zhai et al. 2020) that is heavily moderated by the certification radius. In Figure 19 the rank correlation between these metrics across all baseline and fine-tuned models are shown. For small radii, we observe that metrics harmonise: Large CA is associated with low ACCE and CBS. As the certification radius increases, these relationships invert, and we observe metrics competing: High CA is associated with high CBS and ACCE, i.e., poor calibration and low ACR is associated with poor calibration.
39

---

Published as a conference paper at ICLR 2025
These relationships imply that there is trade-off between calibration and accuracies at large certified radii. Certified accuracy and ACR both benefit strongly from large margins between the lop and the runner-up confidence, which can increase overconfidence and thus harm calibration. We leave closer invest igalion of the causes for future work.
Radius 0.05
Radius 1.00
-0,99
-0.82
-0.96
0.57
0.78
0.53
COS
-0.99
0.84
0.97
0.57
1
0.76
0.88
ACCE
.0.82
0.84
0.87
0.78
0.76
1
0.47
ACR
-0.96
0.97
0.87
0.53
0.88
0.47
1
CA
CB5
ACCE
ACR
CA
CBS
ACCE
ACR
Figure 19: Spearman rank correlation for certified accuracy (CA f), Certified Brier Score (CBS 1) Approzimate Certified Calibration Error (ACCE 1) and average certified radius (ARC t) for different certified radii. For small radii, optimal CBS, CA, and ACCE coincide. For larger radii there is a trade-off.
L FUTURE WORK
Our study offers an extensive analysis of certified calibration while highlighting avenues for future research. Initially, we note the capability of the MIP optimise adaptive binning ECE, which we do not explore due to limitations of the baseline against which we compare. Additionally, our ACCE method approximates an upper limit for the estimated ECE, paving the way for applying confidence intervals to better align this metric with the true ECE (e-g., Kumar et al. (2019)) Further, we suggest to perform adversarial training as inverse problem, where the targets are given by the worst-case confidences on the dataset. We also recognize calibration generalization gap in ImageNet, suggesting that meta-learning framework might yield improvements (Bohdal et al. 2023
M DIFFERENTIABLE CALIBRATION ERROR
M.1 DEFINITION
Bohdal et al. 2023 propose differentiable approximation to the ECE estimator. The authors note that the standard calibration error estimator is non-diffcrentiable in two operations: the accuracy and the hard binning. As are assuming certified predictions, their correctness en (and thus the accuracy per bin) is constant with respect to the confidence scores 2n and thus does not need to be differentiable. We therefore simplify their dECE and apply only one differentiable approximation. We restate the calibration error estimator from and simplify:
B
EČE
Σ ne Pm
(32)
N
Bml
Bml
m=1
M
an (cn zn)
(33)
m== ==1
where an,m E (0, 1} replaced with a soft indicator sn,m E [0, 1] with Σm 8n.m 1. Define matrix S E [0, 1] M
1} is the hard indicator whether data point n is assigned to bin m. This is
with elements sn,m and let e lc 21..... CN 2n We can write the differentiable calibration error dECE as:
dECE N ||S ella
(34)
40

---

Published as a conference paper at ICLR 2025
The soft assignment 8n,m is obtained the following way. For M equal width bins with cut-offs 31 ....
... < fM--, we define with elements b, = - m= Sm. Further, let w w=112 ..... M] we obtain the soft assignments S, through a tempered softmax function: Sn o ( +b/r). For - the vector Sn approximates a onc-hot encoded vector and (34 recovers the original ECE.
-m
M.2 DECE PARAMETERS
As for ADMM, we perform extensive hyperparameter search for the dECF. Our key insight to start the optimization with high values of T, i.e. 0.01 as this increases the smoothness of the objective function and slowly decrease to about x 10-6 at which point we usually observe a difference between the ECE and dECE less than 1 x 10-0.
We use the dECE as one method to approximate the CCE. As for ADMM, we test wide range of hyperparaneters for the dECE. The one most significant hyperparameter the initialization of z. We test the same initialization Brier confidences. In addition, we test random Gaussian and random uniform initialization. We do not find single optimal strategy. It is possible that the ad adversary-free ECE is higher than the ECE obtained by using the Brier confidences and hence the Brier confidences are not universally better values to initialize As dECE shows poor performance in exploring the loss surface, we recommend initialization to whatever initialization yields the largest error to begin with. All other hyperparameters are insignificant in comparison and usually over 95% of variation in results explained by the initialization of We test various learning rates, schedulers for T (as mentioned above), learning rate schedulers (Cosine Annealing, Constant, ReduceOnPleateen and optimizer momentum and find none of those hyperparameters to be significant.
for ADMM, i.e. the adversary-free and
N COMPUTE COST
We outline the computational cost required to replicate experiments. We mostly use A40 GPUs and equivalent older models.
The experiments in the motivation section require adversarial inference and possibly training. GPU hours 70 including training.
The experiments performing parameter search for ADMM and dECE to estimate the ACCE take around 400 GPU hours.
Training baselines that not provided by Salman al. (019 requires 600 GPU hours.
Performing randomized smoothing on bascline models takes 1000 GPU hours.
Performing randomized smoothing fine-tuned models takes 2200 GPU hours.
Fine-tuning models takes 350 GPU hours.
Running an ensemble of ADMMs to estimate the ACCE for the evaluation of baselines requires 320 GPU hours (baselines certified many radii).
Running ensemble of ADMMs to estimate the ACCE for fine-tuned models requires 90 GPU hours.
In total, we use around 5000 GPU hours for the experiments in this paper; additional exploratory experiments are not included.
41


Reviews:
[{"review_id": "qLpBzm56xw", "reviewer": "Reviewer_apBk", "summary": "This work addresses a critical vulnerability in neural network classifiers by examining how adversarial attacks can significantly harm model calibration (the reliability of confidence scores) while leaving accuracy unchanged - a particular concern for safety-critical applications. The authors introduce two new metrics for certified calibration: the Certified Brier Score (CBS) with a closed-form bound, and the Certified Calibration Error (CCE) approximated through mixed-integer programming. They also propose a novel training method called Adversarial Calibration Training (ACT) with two variants (Brier-ACT and ACCE-ACT) to improve certified calibration while maintaining accuracy.", "questions": "1. How does the certification process scale with the number of classes? Would the approach be practical for problems with hundreds or thousands of classes?\n2. Could the proposed methods be extended to other uncertainty estimation approaches beyond softmax confidence scores?\n3. How sensitive is the certification to the choice of binning strategy in the ECE computation?\n\nAdditional Advice:\nYou should follow the ICLR submission format.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Identifies an important security vulnerability (calibration under adversarial attacks) that wasn't previously well-studied\n2. Well-formulated mathematical foundations for certified calibration", "weaknesses": "1. The entire certification framework relies heavily on the binning strategy used in ECE computation\n2. Complex hyperparameter tuning required for ADMM solver", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "VTpoYeGR6f", "reviewer": "Reviewer_Xy4o", "summary": "In this paper, the authors show that it is possible to generate adversarial examples that don't change accuracy but impact confidence scores heavily. Thus, confidence scores can not really be trusted by introducing a parametrized attack that can raise and lower confidence scores without impacting the classification. Then they introduce *certified calibration*, a method to provide guarantees on the calibration of certified models under adversarial attacks. Then a training scheme is proposed to improve the certified uncertainty calibration.", "questions": "- For Table 1: could the authors (possibly via bootstrapping) estimate variances here? \n- What is the final loss that is optimized to solve equation 2? As in how did you encode it? \n- Is the certificate actually sound or just approximate (see Line 238)?\n- Is $a_{i,j}$ only for 1 $j$ allowed to be 1 while the others are 0?\n- What where the considerations for choosing ADMM? \n- Does the reviewer suppose correctly that the runtimes in G1 where reported for CIFAR-10?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The certified calibration is an important and currently not very developed problem. The authors do a good job in motivating the problem and proposing solutions. The paper is mostly easy to follow, though the reviewer feels that illustrations would sometimes have improved readability. The approach is novel and interesting.", "weaknesses": "- It is unclear how large the effect is in Table 2. The Brier Score although defined in simple terms is appears somewhat uninterpretable. \n- The authors did not report mean and std of their results over different seeds.  \n- The reviewer has the feeling that explanations could be made more concise from time to time while improving readability further. \n\nMinor:\n- 232/233 - the sentence here does not appear to be grammatically correct", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "bSEFpHbrlZ", "reviewer": "Reviewer_2ZV2", "summary": "This paper discusses the importance of model calibration and solutions for determining a model's certified calibration and training models to be robust against calibration attacks. The paper starts by highlighting the importance of considering model calibration along with model accuracy by showing attacks degrading calibration and discussing applications in which this could have a strong negative consequences. The paper generalizes existing attacks on confidence estimation to form their new $(\\eta, \\omega)$-ACE attacks. The authors go on to describe certified Brier scores and certified calibration error providing a closed form solution for the first and detailing how to approximate the later with a MIP and solving it using ADMM. The paper then explores a new method of adversarial training building off SmoothAdv which they call adversarial calibration training (ACT). Finally, through an extensive set of experiments they show the effectiveness of ACT.", "questions": "The main reasoning for my score is the 3 points above in the weaknesses section, I am happy to raise my score if these are addressed. The following list of comments I think could make the paper clearer but I don't think significantly impact my grading.\n\n**Clarity**\n\n1. (line 160) When $\\omega = \\hat{y}$ it seems that Equation 2 would read as maximizing $\\mathcal{L}_{CE}(f(x,\\gamma), f(x,\\gamma))$, I think the notation here is a bit unclear, I think you mean $\\hat{y} = f(x)$?\n2. (table 1) What metric is actually given here? The drop in AdaECE compared to the untracked version? I think at this point it is still slightly confusing how one should interpret these numbers, is higher better? worse? does it depend on $\\eta$ and/or $\\omega$?\n3. (line 168) Can you be a bit clearer about how this adversarial training is happening? The existing model is adversarially finetuned? A new model is trained with adversarial training? Standard PGD adversarial training? Does the accuracy of the adversarially trained model change? Or reference somewhere where this information can be found.\n4. (line 180) This is slightly confusing, a practitioner deploys a model with a threat model budget? I guess you are trying to say that the deployed model is certified for all $R \\leq R_n$? How are they receiving a guarantee? What is being guaranteed? This should be individual certificates on individual points, is this being generalized to a data distribution? I think there should be a clearer way to defined the certified accuracy here.\n5. (line 190) I think it is important to note that Gaussian smoothing gives probabilistic certificates. The certificate defined in C1,C2 are deterministic. There also exist methods for obtaining deterministic $l_p$ certificates through abstract interpretation/LP solvers/etc, so I think a distinction should be made here as these methods more accurately fit the certificates above.\n6. I think throughout the Section 3 it would make the argument easier to follow if there was a toy example of some sort.\n\n**Minor Comments**\n\n1. (line 34) This sentence reads strangely maybe \"...neural network classifiers **can be** extremely sensitive... human eye **for which** $x + \\gamma$ ...\"\n2. (line 42) \"and thus **provide guarantees on** model accuracy\"? otherwise the last clause it strange\n3. (line 45) \"average mismatch\"? undefined at this point\n4. (line 128) extra comma\n5. (line 137) There is no example being extended, it was mentioned in line 124 but has not been explained yet\n6. (line 166) This is an incomplete sentence, \"**For** a ResNet50 *model* on ImageNet, **a** $(1,y)$-ACE **attack**...$?\n7. (line 168) missing a word after the comma \"**but**\" or \"**however**\", also should it be \"a $(1,\\hat{y})$\" or \"an $(1,\\hat{y})$\"\n8. (line 184) I think the notation/wording is a bit confusing here. \"**for all** perturbations **s.t.**\"? I guess $\\forall ||\\gamma_n|| \\leq R_n$ seems slightly weird I understand what is trying to be said $\\forall \\gamma_n \\in \\[\\gamma \\text{ s.t. } ||\\gamma|| \\leq R_n\\]. F(x_n + y_n) = F(x_n)$. I guess you don't need to change this but it seems like a slight abuse of notation.\n9. (line 184) Why is this an assumption? Isn't it more like a definition?\n10. (line 208) Is this an assumption or are you saying that a model has these properties by definition, or clarify what the assumption is here i.e. is there some case where this doesn't hold?\n11. (line 214) Maybe place a dot between $l$ and $c$ otherwise it kind of seems like $lc$ is a new variable.\n12. (line 233) \"that **is** possible\"\n13. (line 244) is this backward? should it be (4) to (5)?\n14. (line 268) define $\\mathbf{1}_M^\\top$", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "I found this paper to very well written and argued. The paper is structured well and there is an appropriate level of formality and theorems which motivate the work. To the best of my knowledge they propose the novel metrics of certified brier score and certified calibration error and give algorithms to compute these. They also provide a novel algorithm, ACT, for training models to have better uncertainty calibration. I also found the paper to have extensive experiments showing the performance of ACT in various scenarios.", "weaknesses": "1. The main issue I have with this paper is that it does not at all address the extensive work in non-probabilistic verification/certification of Neural Networks (Abstract Interpretation: IBP, CROWN, DeepPoly, etc. SMT Solvers: Ehlers et al 2017, Marabou, etc. Relaxation: Neurify, etc. Certified Training: IBP, DiffAI, COLT, CROWN, SABR, etc.). The paper defines formal certificates (C1, C2) but does not note that the certificates it proves/approximates are based on randomized smoothing and thus probabilistic in nature (i.e. certifying a score above a certain probability threshold). I am also curious as to what the authors think about using non-probabilistic verification/certified training methods in the calibration certification case. Smoothed classifiers tend to have good performance at the cost of inference time (i.e. performing 10s-100s of inference passes to get a classification). Do the authors think that adapting abstract interpretation based approaches could work in this case for verification or training?\n2. This paper also lists training times in the 100s of GPU hours and verification times in the 1000s of GPU hours (Appendix N) using a total of 5000 GPU hours for the experiments in the paper (although admittedly there are many experiments). These finetuning times are approaching multibillion parameter LLM finetuning times. \n3. While the authors bring up valid motivations in Section 2.2 I find them a bit hard to see in practice. Currently accuracy still dominates as a metric for most practical applications, most SOTA defense methods protecting model accuracy against attackers actually sacrifice accuracy and thus even these have a hard time finding real-world use. The premise of this paper leans on these methods being used but not being protective enough of calibration (although the papers experiments do show that their method can be used without sacrificing much certified accuracy). This point along with (2) make me think this method seems a bit hard to justify in practice.\n\n**Minor Weaknesses**\n\n1. While I found the paper to be well written and easy to follow, I think it would greatly benefit from some overarching example which could help provide insight and make Section 3 a bit easier to follow.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "E2HLKBQBFO", "reviewer": "Reviewer_oWYq", "summary": "The authors consider the problem of constructing certifications for neural networks in parallel with the task of improving confidence. The authors achieve this through the development of a new approximate certification measure, and support their work with significant experimentation.", "questions": "Answers to the following questions from the authors would be appreciated:\n- Could you comment upon how you view the process of calculating/employing an approximate certification?\n- Which norm (see, for example, line 179-188) are you using? \n- Is there any sensitivity to the number of bins (M), or their partitioning?\n- Are there any uncertainties that could be calculated on Figure 3 or Table 2? For table 2 how did you settle upon selecting \"within 3% of the ihghest certified accuracy\"\n- In Figure 4, you claim \"ACCE-ACT significantly improves certified calibration at larger radii\" - why is this not present at smaller radii, and how much is this actually an improvement? The overlapping nature of the circles makes it very difficult to parse out the exact impact here, beyond tea-leaf reading.", "limitations": "", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "The authors investigate an interesting extension of the certification literature, which moves beyond more traditional (and incremental) advances in the field of image based certifications. \n\nIn doing this, the authors introduce new techniques and approaches, which are novel and interesting, and the mathematical presentation is well detailed (although I have issues regarding specificity and rigour in explanations, see below).", "weaknesses": "I hold a number of concerns regarding this paper. One of the primary issues relates to this technique relying on an approximation to construct ACCE. I'm intrinsically cautious about any result that presents itself as a certification while relying on breaking the guarantees of the certification in order to calculate results. Expectations around certifications guide their use, and violating the precepts of a certification while still labelling it as a certification is problematic to me. To me,  an approximation to a guarantee is only useful in rare circumstances, where other insights can be gained by the approximation process - something that I would argue is not present within this work. \n\nWhen it comes to the motivation for their work, the authors discuss possible reasons for an attack on calibration, however I would suggest that the current presentation of this is a bit sparse - it is sufficiently detailed for comparisons, but the ultimate applicability of this attack could require more justification. This perception is reinforced by statements being made regarding risk and processes that might support this work without citations - leaving this reader curious as to if the justifications actually exist. \n\nWhile the sentence-by-sentence writing is overall well done (although a touch clunky at times), I would argue that the ordering of content is somewhat jumbled, and it hinders readability and accessibility. For example, certifications are required knowledge to understand content long before they are formally introduced on Line 177. The document, as structured, has multiple examples of such structural issues that would make it difficult for a reader to parse content if they were unfamiliar with the space. \n\nThese issues with presentation extend to how some of the results are presented. Take Table 1 as an example - what  is the reader meant to get out of this? If I understand it correct the Unattacked rows correspond to the base values, and ideally the AdaECE should decrease in the case of a successful attack? But when there is the switching behaviour about $\\eta$, interpreting this table is nigh on impossible. Moreover, the compressed nature of the table, and its positioning within the document further hinder accessibility. \n\nGeneral comments regarding identified issues follow below: \n- L118 \"has been discussed\" -> \"have been discussed\". Moreover, in this sentence you use \"label attacks\" without explaining what these are, nor what an adversarial attack even is. \n- L119 \"their immediate effects\" - what is the subject of their: adversarial attacks? The literature? Label attacks? \n- L119 \"effects on the reliability of uncertainty estimation are often overlooked\" you can affect somethings reliability, but you cant affect the reliability. The reliability is a downstream consequence of the impact made. \n- L120 after the colon - why is this colon required? And why are these things overlooked if this is a statement that can be made without citation, suggesting it is so well known that such a justification is unnecessary. \n- L121 \"Beyond label attacks\" - the subject of this paragraph, to this point, has been label attacks. If a different framework is to be introduced, this should be a new paragraph. Is the point here not that it's not only that attacks can indirectly influence confidence, motivated attackers can attempt to directly manipulate the confidence scores as well?\n- L125 \"We note that this impacts system-level uncertainty and conclude that both label and confidence attacks leave system calibration vulnerable\" - you're noting the very things you've just stated in the preceding sentences, without stating why this is important. \n- Line 127 \"for both, their\" - comma is unnecessary\n- Line 128  \"when uncertainty estimates....\" - this sentence fragment is disconnected from its context. Is it not \"machine learning systems incorporating uncertainty estimates in their decision processes are monitored regularly for both their .... in safety-critical applications.\" ?\n- Line 127-129 - this is a statement of fact, are there any citations available for it? \n- Line 132 Denial of Service should be capitalised appropriately, as it is the name of a thing. \n- Line 120 and line 132 - incorrect use of colons. For one, they're unnecessary in these contexts, and for two they shouldn't have capitalisation after the colon (although there's some nuance here between British and American English, still, fundamentally, the colons are unnecessary). \n- Line 132: The defences are again a whole new concept, should this not be a new paragraph for readability? \n- Line 137: Discussion centred around certified models without properly defining what these are? Readers who are not familiar with the certification literature will have limited opportunities to catch up up here. \n- Table 1 - font too small, it's what, half the size of the standard maths font, and is really jammed in to the paper. \n- Line 195 \"The certificate on the prediction\" - is the first the necessary?. And what does \"showing a certificate on the confidence\" mean. \n- \"extended by Salman et al. (2019) and Kumar et al. (2020) showing a certificate on the confidence\" - this implies that both Salman AND Kumar certified confidence, rather than just Kumar.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)