Optimal and Provable Calibration in High-Dimensional Binary Classification: Angular Calibration and Platt Scaling
Authors: Yufan Li
Pragya Sur
Venue: NeurIPS 2025 spotlight ()
Keywords: High dimensional statistics
Primary area: theory
TL;DR: High dimensional theory for calibration of binary classification

Abstract: We study the fundamental problem of calibrating a linear binary classifier of the form \(\sigma(\hat{w}^\top x)\), where the feature vector \(x\) is Gaussian, \(\sigma\) is a link function, and \(\hat{w}\) is an estimator of the true linear weight $w^\star$. By interpolating with a noninformative \emph{chance classifier}, we construct a well-calibrated predictor whose interpolation weight depends on the angle \(\angle(\hat{w}, w_\star)\) between the estimator \(\hat{w}\) and the true linear weight \(w_\star\). We establish that this angular calibration approach is provably well-calibrated in a high-dimensional regime where the number of samples and features both diverge, at a comparable rate. The angle \(\angle(\hat{w}, w_\star)\)  can be consistently estimated. Furthermore, the resulting predictor is uniquely \emph{Bregman-optimal}, minimizing the Bregman divergence to the true label distribution within a suitable class of calibrated predictors.
Our work is the first to provide a calibration strategy that satisfies both calibration and optimality properties provably in high dimensions. Additionally, we identify conditions under which a classical Platt-scaling predictor converges to our Bregman-optimal calibrated solution. Thus, Platt-scaling also inherits these desirable properties provably in high dimensions.

Optimal and Provable Calibration in High-Dimensional Binary Classification: Angular Calibration and Platt Scaling
Yufan Li Department of Statistics Harvard University Cambrdige, MA 02138 yufan 1i@g. www.aaa.aaa..... edu
Pragya Sur Department of Statistics Harvard University Cambrdige, MA 02138 praqya harvard...addd.........
Abstract
We study the fundamental problem of calibrating a linear binary classifier of the form ( T xx). where the feature vector Gaussian, is an estimator of the true linear weight w² By interpolating with a noninformative chance classifier, we construct a well-calibrated predictor whose interpolation weight depends on the angle Z(w, w.) between the estimator ú and the true linear weight w.. We establish that this angular calibration approach provably well-calibrated in high-dimensional regime where the number of samples and features both diverge, at comparable rate. The angle (ww, w.) can consistently estimated. Furthermore, the resulting predictor is uniquely Bregman-optinen minimizing the Bregman divergence to the true label distribution within a suitable class of calibrated predictors. Our work is the first to provide a calibration strategy that satisfies both calibration and optimality properties provably in high dimensions. Additionally, identify conditions under which classical Platt-scaling predictor converges to our pegeman-optimal calibrated solution. Thus, Platt-scaling also inherits these desirable properties provably in high dimensions.
link function, and
Introduction
Calibration of predictive models is a fundamental problem in statistics and machine learning, es- pecially in applications that require reliable uncertainty quantification. A well-calibrated model ensures that predicted probabilities align closely with true event probabilities property essential
in fields such as medical decision-maie maaiinn 77] 41 meteorological forecasting 14 28, 23
65 64
self-driving systems (60), and natural language processing [66] 29]
Numerous algorithms have been proposed for calibrating the outputs of a trained model, including
classical methods such as Platt scaling 770 69 [T], histogram binning 89 77), isotonic regression
190
40
12
36 44 and more recent approaches such temperature scaling 29 47 ensemble-basee
methods
50
58
86
81, and Bayesian strategies 45| 21]. among others.
43 this literature primarily focuses traditional asymptotic theories finite-sample learning theoretic arguments. These approaches often overlook the impact of problem dimensionality, which particularly relevant for high-dimensional settings where the number of features may be substantial. Alternatively, separate line of research has explored calibration within a high-dimensional proportional asymptotic regime, where the sample size and the feature dimension d both diverge, comparable rate. This proportional scaling regime has gained significant traction in modern statistics and machine learning. In statistics, popularity stems from the fact that theories derived under this regime
While extensive prior work has studied calibration 48
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

capture -ieeediinnsional phenomena observed in moderate to large sized datasets unusually well 6
42
25
80
91
78
63
84
52
[3]. Consequently, this has spurred the creation of innovative
57 In machine learning, this regime has proven exceptionally valuable and effective analyzing the behavior of modern
methods displaying remarkable practical performance 27 9 54 76
neural networks and other interpolation learners under overparametrization [55] 34 56 59 2 75 67
For binary classification in this proportional regime, a substantial line of work [79 78 92 establishes that classical logistic regression yields seriously biased estimates; building upon these, f shows
that logistic regression tends to be inherently overconfident, while [22] discusses the impact of regularization under the same model. Finally, [II introduces expectation consistency and derives limiting calibration error formula as a function of the signal prior and other problem parameters.
Despite these udvancements, an approach that is provably calibrated in high dimensions, without knowledge of the true signal prior, missing. Moreover, there a lack of principled understanding regarding optimal calibration strategies from among the available options. Additionally, rigorous guarantees on the performance of classical calibration methods, such as Platt scaling, in modern high-dimensional scenarios is notably absent from the literature. In this paper, we address these gaps. We consider the challenge of calibrating a binary linear predictor a frequentist setting under Gaussian design. Our contributions are three-fold: (i) we introduce data-driven predictor that can provably calibrate in a broad class of high-ding nigh-dimensional binary classification problems; (ii) we show that our calibrated predictor is trregman-optinggg Bregman divergence relative to the true label- -ebel-generttion probability; (iii) we establish conditions under which classical Platt-scaled predictor converges to this Bregman-optimal calibrated solution, thereby formally showing that Platt scaling is both well-calibrated and Bregman optimal in our high-dimensional setting. Although we derive our theoretical results assuming Gaussian features, extensive recent universality results suggest that these should continue to hold for sufficiently light tailed distributions (see Section ( for discussion). We provide experiments that demonstrate this robustness to the Gaussian assumption (Section H.2 and 4).
meaning it uniquely minimizes any
We construct our calibrated predictor by interpolating with uninformative ("chance") predictor, where the interpolation weight is determined by the angle L(w, w.) between the estimated linear weight vi and the true weight w.. Our construction crucially leverages recent developments from the literature on observable estimation of unknown parameters in high dimensions. For instance, leveraging advances in estimable when n and d grow proportionally. To our knowledge, this is the first provable calibration method in high-dimensional setting, and it uncovers conceptual link between optimal calibration
38 g 9 10 18, 54, we can show that the angle (wi, w.) is consistently
and L(w, w.): the poorer the alignment of w with w., the greater the noise needed to be injected to prediction logits ensure calibration.
Setting
Suppose we observe i...d. data (v,, .x) satisfying
yi nd Bern (a (o]=:)), i= I,...,,,
(1)
where a: R + [0, 1] denotes the link function and the covariates E Id are drawn independently nd as x4 ~ (O, Σ). with E assumed to be known (say from a separate unlabeled dataset as in 17 18]
).
The true linear weight w. E IRd is an arbitrary deterministic vector, and we assume without loss of generality that w. Ew., = ||w,1| = 1. The training dataset is denoted as X = [.1,..., xn T E Raxd and y [yi,..., vn] IR..
To quantify the degree of miscalibration, we define the calibration error at level p for any predictor f as
Afal(f) = --EE Exro (o (w. [new))))) I.f (I = =p]
where Ex,e. denotes the expectation over Knew ~ NO, E). A predictor is said to be well-calibrated A (f) = 0 for all p in the range of f. Intuitively, this means that when the predictor assigns a probability p to label 1, the true probability of label 1 is indeed p.

---

We consider the regularized M-estimator
i arg min
71 Σ cv (w 2z) + g(w),
where g(-) is convex penalty and {(.) is convex loss function. In this setting, we consider
sequence of problem instances (y(d), (d). w. (d)]d2z) such that (d) E IRd and y(d) E (R)(d) generated from (T. well-known that the special case where {(+) equals the logistic loss and g(.)
zero, the corresponding predictor or (th" Fnew) is grossly mis-calibrated in the high- high-dimensional
regime - (0, +00) 79 78 [2], even where it is well-defined and unique [16].
In what follows, present, for the first time, predictor that is provably well-calibrated in this regime (for general convex losses and penalties beyond the special case mentioned above). We achieve this through angular calibration idea, and furthermore, establish that this is optimal in the sense that it minimizes any Bregman divergence to the true label distribution. We conclude showing an interesting connection-Plat scaling converges our angular predictor-and therefore is both provably well-calibrated and optimal in the aforementioned sense.
Introducing angular calibration
Most calibration strategies adjust a pre-trained predictor by learning a mapping F: + F(u) of the logits à Fnew- Platt scaling, for example, stipulates the parametric form F(u) o (Au + B), where A, B E are fit on holdout dataset. This raises a natural question:
Among all well-calibrated predictors of the form F(w" Inew), which one is "the best"?
In this section, we introduce a predictor with such an optimality property by interpolating between the prediction logits á² x and uninformative chance predictor. Specifically, show that if the interpolation weight is determined by the angle between the estimator and the true weight w.. given by
(W.,
0, = arccos
(2)
|
then the resulting interpolated predictor minimizes any Bregman divergence to the true label dis- tribution among all predictors of the form F(w Inew). that is both provably calibrated and optimal (in the aforementioned sense) has not been previously introduced for high high-dimensiooal problems.
F(w")new). To the best of our knowledge, predictor
Notably, our predictor uses the angle defined in ( thus to define data-driven predictor, we require a consistent estimate of this angle. Fortunately, recent advances in the high- -igh-dimensional literature (c.f., 38 g
10 18 allow us estimate the inner product (w, g. and therefore 9., when n. and d grow proportionally. We discuss the details of this estimation scheme later in Section 6 For now, we present our angular calibration idea assuming that we have access consistent estimator for 0,.
Definition 3.1. (Angular Predictor) Let
fang TuTTuw::
Ez (a
1 Inew
sin (0)
(0).2) )
where 0 a consistent estimator of 0, defined as in 13 and IEz denotes expectation with respect to the Gaussian noise Z N(0, 1). We will later refer to fang as the angular predictor for simplicity.
Theorem 3.2 below shows that the angular predictor is well-calibrated. We defer the proof to Section A
Theorem 3.2. Assume the link function is continuous. Then, the predictor fang defined in (3) is well-calibrated as d, n. + 00, n//d + (0, oo). That is, for any contained in the range of a, we have that
call (fams (:0) = =p Errnew [o (w. know) Sang (To Tnew: 0 = p) 0,
in probability where θ is consistent estimator for 0, (Cf. Proposition 6.20.
3

---

The above utilizes the result that when = 8, exactly, fang (+:02) is exactly well-calibrated (we state and prove this formally in Theorem A.1) and that i is consistent for 0.. We will later show that fang (:0) R fang (:;0.) is in fact optimal in the sense that it minimizes any Bregman divergence to the true label distribution. The construction (3) admits an intuitive interpretation. By the basic trigonometric identity cos? + (0.) 1,
that the logits, that is, the argument of a in is an interpolation between the informative component i Inew and the noninformative Gaussian noise Z. Notice that when T well aligned with w, (i.e., co.) 1), the angular predictor fang lies closer to the informative predictor a (w Inew). Conversely, when w. and are orthogonal (i.e., sin" (0.) 1), fang defaults to the non-informative chance predictor E[o [[ knew)] In other words,
The poorer the alignment between w, and L the greater the magnitude of noise Z required to maintain calibration.
We will show in the next section that this angular interpolation idea leads to uniquely Bregman optimal calibrated predictor. This provides the first calibration procedure that is calibrated and optimal in high dimensions, provably. Peusdocode for angular calibration, using angle estimator from Section 6 is included in Section
Main Result I: Calibrating Optimally using Angular Calibration
Before formally stating our results on optimality of angular calibration, we first define the following random probability vectors for label distribution.
F(w F(TTInnew) 4. :=:= (1- (w] know) ) OF = -1- a (w. (ul..on )
) (an 11-F(W) - (nnw))ew))))
(1 Sang (ū Inowi Õ) fang ( Inewi : 0)
where F - [0, 1] is any measurable function. Here, l. corresponds to the ground-truth probability distribution of the new label, OF the prediction probability distribution of an F-calibrated predictor, and gang the prediction probability distribution of our angular predictor.
Next, we define the Bregman loss function.
Definition 4.1 (Bregman Loss Functions). Let á IR2 + R be strictly convex differentiable function. Then, the Bregman loss function D3 R²² x R2 + defined as
Da(x,y) (bx) oly) (xc y. Volv))
The Bregman loss function class covers common losses such as the squared loss Do(r,
|| - y/12 - and Kullback-Liebler (KL) divergence Dz(2, v) = Σ} Iy log (x;/u;) between two probability vectors z, y.
Theorem 4.2 states that the prediction probability yang generated by the angular predictor uniquely minimizes any Bregman loss against the ground-truth probability vector d within the class of gr for any F. We defer the proof of the Theorem below to Section B
Theorem 4.2 (Optimality of angular predictor). Let d TR. + be any strictly convex differentiable function, and let D& be the corresponding Bregman loss function. Let E be finite. Then, the expected Bregman loss Ex,.. [Da(q., qr) admits a unique minimizer (up to a.s. equivalence) among all qF, FF E Further suppose that the link function a is continuous. We then have that as n, d - 0C, we have
{s: R. + [0, 1]]- Let this minimizer be F. arg minFer En Dalans qr)).
(aag (0) - QF. (w Inew)
0
in probability. That is, the label prediction probability vector from angular calibration converges to the optimal label prediction probability vector given by F..
We note that as i 0., jang precisely attains the optimal solution F. know). We defer the technical statement to Theorem B.2 Section

---

prediction prob.
Fang(U)
(arlu), Ano 100
(pidl)), Ono 200
(ordd)), Ono
0.4
faördU), Aoo 1000
Pianlu), neo 2500
(andl)), Pno 5000
(andu)) 10000
(airlu), noo 20000
Figure 1: Platt scaling of a logistic ridge predictor converges to angular calibration predictor, as holdout set size increases. The plot is generated with Gaussian data with covariance Σ - where Egi 0.5lk-1l, Vk, {i.... d), sigmoid link function in a data deficient setting where 1000, 2000. See more details in Section 7
Main Result II: Platt scaling is provably calibrated and Bregman-optimal
Platt scaling is arguably the most widely used calibration method in modern machine learning, yet its theoretical properties in high-dimensional settings remain unexplored. In this section, we identify conditions under which Platt scaling converges to our angular predictor, and therefore well-calibrated and dregman-optimal in high dimensions.
Platt scaling finds mapping F' of the prediction logits w xnew by minimizing the log- logllikeldoood on a holdout dataset. In this section, we specifically consider the setting where we have a holdout the and the negative log- likkliihoodd
dataset i i Vho, "
Rie
F))
Pbo,t log (F zho,s)) (1 -1-18 yho,;) log F(O Zho,s))
(5)
The Platt calibration procedure then searches for mapping F within some hypothesis class F Fplacs that minimizes the negative log-likelihood. Elementary asymptotic theory then shows that as Tho + oo (i.e. the holdout set is sufficiently large), [(()) in (5) converges to the population loss
E (F) = -Ezoo DKL (1-(0
ow! Inew) (1- knew))
11 F1
F(ü" Know
Inew) 2now)) ))]
(6)
almost surely up constant. This is exactly the argument of the Bregman loss Exaew [Do âr)l from Theorem 4.2 for ó specialized as the negative Shannon entropy. That is, such calibration procedures essentially trying to optimizing the Bregman loss but within the restricted hypothesis class.
This naturally raises the question of whether calibration procedures such as Platt scaling can achieve the optimal Bregman loss, say the limit sufficiently large holdout set. Theorem [so]ws that, if a is probit link function (or is closely approximated by one up to an affine transformation- for instance, sigmoid(z: RN D xx),, and if the negative log-likelihood (5) is minimized over the hypothesis class of the form o Au + B), A, E IR then the resulting Platt-scaled predictor converges Combining this connection with our results for the predictor 0, exactly. As mentioned previously
to our angular predictor 7ho fang (w; 0.) which is angular predictor fang (u: 0) with (see also Theorem A. and Theorem B.2 in Appendix), fang (u; 0.) exactly calibrated and Bregman- optimal, which shows that Platt scaling is both provably calibrated and Bregman optimal. This offers
5

---

guarantees of this kind for the widely well-known Platt scaling procedure. We defer the proof of the Theorem below Section
the first formal
Theorem 5.1. Consider the predictor pplatt (u) calibrated by the Platt scaling procedure, that is, plats. (w) knew) = o (A""i. Inew + B
with A - argmin l (u + aAAu + B)) (A,B)EH
(7)
for En (.) defined in (5. the link function a satisfies (x) = ((ax+b)) for for some a E (RR) R and the point (Au, B.) defined in ( is contained in compact subset H C RR, the angular predictor defined in (3) satisfies
fang(a; (u; 0.) = cAA. + B.) E Fplacs,
(8)
where
cos(O+)
B. b
A.
[ E V 1 + a² sin?((.)
V a² sin (f-)
and Fplatt = {u + (Au + B) : A, B E R). Moreover, as Tho - co, we have that Ámho
A., Tho + B. in probubility and
sup WER
platt
(u)- - fang( 0.) + +0
(10)
in probability. (Thoo
Here, in-probability convergence is with respect to the randomness of
Who (/11)
To be clear, the above theorem considers the asymptotics in the holdout set Tho for a fixed sample size and dimension n, d of the training dataset. We illustrate Theorem 5. in Figure (see Section 7 for detailed settings) where the solid red line plots our angular predictor u. laug (u) defined in
33 and the dashed lines plot the predictor u + platt (u) calibrated by Platt scaling on increasingly large holdout sets Tho- We observe that the Platt scaling predictors indeed converge to our angular predictor as the holdout set sizes nho increase.
6 Consistent angle estimation
Observe that the angular predictor fang depends on the unobserved quantity (w., W)x. Using recent advancements from Equation (IT) we are able to provide a consistent estimator for this quantity. For simplicity, we outline the estimation procedure here for twice-differentiable loss function e and strongly convex, twice-differemtiable penalty g. analogous results for unregularized M-estimation and other losses/penalties found in (8) We note that this result is part of long line of development in observable estimation of unknown quantities in high dimensions 38
9 10
18 54
data driven estimator for (w+, 3 proposed
is:
X
T X 9922)
a
(1I)
2
+ 20 TT
X + 12 X - rr0i12 - 1r2
where & E R. is the vector with components 2u
- (1. 0) 6= 1 Tr (D DX DX#XT D)
,9
Tr (XiXTD) for D diag (X))) and Á DX DX + + ( and م [| )
It can be shown that a3 (w., 3 0 in suitable high dighhdimmesional sense. We refer the technical
statement to Theorem D.I in Section D
To estimate (w., w we also need to estimate its sign.
We require reserving constant fraction
Tho = a n of the n training data for the sign estimator,
Sgn == sign
Σ
w
x
ho bo
y/
(12)

---

45' line
45' line Uncalibrated
0.90
Uncalibrated
0.90
Empirical Probability
Angular
Angular
Platt scaling (Nm 100)
scaling (Nha 20000)
0.70
0.70
0.50
0.50
0.30
0.30
0.10
0.10
0.10
0.30
0.90
0.30
0.50
0.70
Predicted Probability
Predicted Probability
Figure 2: Reliability plots for angular calibration and Platt scaling of a logistic ridge predictor. Left panel uses a small holdout set for Platt scaling with Tho = 100; Right panel uses a large holdout with Tho 2000. The plot is generated with Gaussian data with covariance Σ 4 where Σnl = 0.5lk -I, vk, I E {L...., d}, sigmoid link function in a data deficient setting where 1000, p 2000. See Section 7 for more details.
can be shown that the probability of wrong sign identification using sgn decreases exponentially with Tho- We defer the proof Section E
Proposition 6.1. Suppose that (zx) is well-defined and non-negative almost everywhere and (' > 0 on a set with non-zero Lebesgue measure. We then have for some absolute constant c > O,
Pho (sgn sign ((w,, w)+)) 2 1 exx (-crho (cos (0.) Eo' (2)))
where Z (0,01) and Pho is with respect the randomness in (yl xx (
Plugging and into have the following estimator 0 for 0, in
= arccos
(| sgn
Sgn
(13)
Theorem 6.2 below shows that is consistent.
Corollary 6.2. Under the assumption of Proposition 6.1 and Theorem D.1
as n. d ~ oo, we have
that 1 - 0.1 + 0 in probability where Tho = a n.for fixed constant a > 0.
7 Numerical experiments
7.1 Simulations
This section presents simple simulation to demonstrate results in Section 6 We generate i.idd. sam- ples ~ (00, E), = 1,... n where Σ 1 and Σn 0.5 k-1 Vk, E (1,....d) d);
also generate labels from ( with (u)) = sigmoid + 1) = 1/(1 + exp( (3u + 1))) and w. (O, (normalized to |l.lls
1). We consider the case of ridge logistic regres- sion with ly. (w) x) ys log(fw (x:)) (1 y:) log(1 (w:)) with P.. (z;) =+xxp(220))
||ll|2 with A 0.5. We assume that n 1000, p = 2000.
and g(w)
in data deficient setting where
The realizability plots in Figure 2 are generated from test set of size Phast 20000. To produce these plots, we bin the predicted prohabilities for label (on the x-axis) and then compute the average of the observed label within each bin (on the y-axis). Perfect calibration would align the binned
7

---

Table ECE on pretrained feature extractors. "Uneal." uncalibrated; "Angular" our method;
"Platt/Iso" use Tho E {100, 500} labeled hold-out points.
Model-Dataset
Uncal.
Angular
Platt 100
Iso 100
Platt 500
Iso 500
ResNet-34 34 CIFAR-10
0.1236
0.0199
0,0561
0,0484
0.0259
0.0298
MiniLM-20 Newsgroups
0.1392
0.0249
0.0931
0.1107
0.0679
0.0813
ChemBERTa-Toxs Tox21
0.1389
0.0132
0.0236
0.0497
0.0175
0.0293
points with the 45 line. In the left and right panels, Platt scaling is derived using holdout sets of Tho = 100 and 7ho = 20000, respectively, whereas both the uncalibrated predictor and the angular predictor remain unchanged across the two panels.
From the reliability plots, we see that the uncalibrated predictor (blue) is poorly calibrated, while, as expected, the angular predictor (green) shows good calibration. Here, we have used the angle estimator (TT) and the sign estimator (12) to estimate the value of (w., w) The estimated value for
(w., T)y; 0.4356 while the true value is 0,4526. We also ran 5000 Monte Carlo trials where found the probability of incorrect sign estimation to be 0.89% with a holdout set of size Tho = 100.
In contrast, the left panel of Figure 2 shows that Platt scaling (orange) with holdout set size of Tho 100 fails to properly calibrate. However, when the holdout set size increased to Tho 20000, Platt scaling also calibrates correctly. When Tho = 20000, the predictor calibrated from Platt scaling found to be (using scikit-learn package's 's CalibratedClassiffeCCC routine (68)
frino (wi ToTTae Fnew) (A7 W Tnew + Bru)), with Å "ho 0.3396, Bh -0.1521.
We check if A and The are indeed close to A., B. claimed in Theorem 5.1 Note that even though the link function is not strictly a probit function as required by Theorem 5.1 by approximating sigmoid(u) ~ ((v T/8- uu) we have (u) ~ 4 /8((uu 1)). We then obtain
A. 0. 0.2991, B, -..1597 from (9) setting a 3/ 3 4/88 VT/8, which are indeed quite close to A Pho- 0.3396, m 0..121.
7.2 Semi-real experiments
We assess angular calibration semi-real tasks that keep real-data covariates but simulate labels from the known generative model. We maintain settings in Section 7 but replace data covariates with: (i) final-layer logits of pretrained deep networks; and (ii) classic UCI benchmarks.
We found that II] plug-in estimator for (w* w) is unstable on these real datasets. This a known issue for estimators like (11) that are based on Wigner-type Modifying these estimator are an active research area 54 57
coam---marii-tttteeee assumptions. To isolate calibration effects, we
simulate w" and labels, using the true angle.
Each dataset is split into training set n, large unlabeled pool nlov > Merain for covariance estimation, and Tiest- We report Expected Calibration Error (ECE; lower is better) [29]. Post-hoc baselines use a labeled hold-out of size Tho {100,, 500) ("Platt 100/500" and "Iso 100/500").
Pretrained representations. We fit a lincar head on frozen embeddings and calibrate the resulting logits: ResNet-34 (ImageNet-IP 1K pretrain) CIFAR- 10 [35] 24, (6). MiniLM sentence embeddings
on 20 Newsgroups 885 72 5T, and ChemBERTa on Tox21 from MoleculeNet 200 87]. We
have
(300 x 512) (800
384)/(800 x 768), neav 30,000 3,000/3003/////00 and Plusst
10.000/110000 1,000/5000 '500 for CIFAR-10 -0 20NG / Tox21. The results are reported in Table | the reliability
plots deferred to Section H.2
UCI benchmarks. On Communities & Crime, Splice-junction Gene Sequences, and Madelon 1 we train linear predictors raw covariates with x = (200 100)/(100> 180)/(300x 500), Neav = 100022000990 and Nusst = 593/900 003/900/900 for Communities & Crime / Splice-junction / Madelon. The results are reported in Table 2 the reliability plots are deferred to Section H.2
8

---

Table 2: ECE on UCI datasets. Conventions as in Table
Dataset
Uncal.
Angular
Platt 100
Iso 100
Platt 500
Iso 500
Communities Crime
0.1700
0.0296
0.0620
0.0661
0.0262
0.0436
Splice-junction Gene Seq
0.1262
0.0208
0.0568
0.1041
0.0556
0.0879
Madelon
0.1473
0.0267
0.0721
0.0719
0.0299
0,0624
8 Extensions and future directions
We derived our theoretical results assuming that the covariates are Gaussian Gaussian-althoughh first pass this might appear stylistic, recent universality results [67 should continue hold as long as the covariates have sufficienlly light tails. We demonstrate this with further experiments. In Section H.2 and Figure 4 design matrices (iid Rademacher and uniform entries respectively) where we observe that our results continue to be accurate. Establishing such universality formally should be an interesting avenue for
( 33 26 49 62] demonstrate that these results
we reproduce [nd with non-Gaussian
future work-we include an informal discussion here. Consider general setting where 2; - 5//22, where 2; has iid entries with mean, unit variance and finite moments, and Σ. = p p-15 where E has bounded condition number. Denote w E1/20, W. = I//2=" and Tnew Anew to be observations test time with the same distribution as 24, 2, respectively. If we could apply the multivariate CLT we would obtain
P 1/2
W., noow,s: 1/2
Wy Fnowws
= (7:1, 72)
(14)
where (21, -2) ~ (0, L) for some positive definite covariance matrix L E R2x2 To apply the multivariate CLT, we require check the following moment condition (c.f.
p 3/2
nnew
( w:s
(1)
|
)
(w, w.)r ||..1 1|
Wasi Wi
1/2
min! (T. ( E
3
Σ
new
VP
We claim that the above holds almost surely for sufficiently large p, if we have constants Wi, W2 > 0 for which the following holds
T (w, ways
(0,, w. ||w.1l
) -L
P
Σ :=1
(W1, W2).
Recent universality results for either approximate message passing algorithms 19 or convex gaussian minmax theorems (CGMT) [33] allow one to prove this beyond Gaussian designs. Numerous works have already applied such arguments in the context of other high-dimensional problems
56 44). Using (14), the conditional distribution (15) in the proof of Theorem 3.2 can be extended to non-Gaussian, Wigner-type features preconditioned by some known D2/2 thus leading proof of Theorem 2 beyond Gaussian designs. In the interest of space, we defer formalizing this to future work.
37 62
Finally, consider binary classification in this work; it would be interesting extend our results to multi-index models, which includes multi-class classification, additive and interaction models, and two-layer neural networks {82
details in Section Multi-index model can be defined as follows: for K 2 and unobserved indices W. = [W.1. w E d K the true logits and model outputs are := W. Fnew
88 15
R π (Fnew = gGG) where y is a generalized link (vector- or
scalar-valued). We show in Section F that, given an estimator W = [ @xl of W., angular predictor in Definition 3.1 may be extended for the multi-indes model as follows,
Fang [
Knew := Ez [g (M.S L.Z)]
where the matrix quantity M. and L,, depends on cross-index angles (wxx; wz)x; E, E [K]. Though no estimators are given for these cross-index angles in literature as far as we know, recent theory for multi-index models [82] suggests that analogues of the single-index angle estimators [ are feasible. We leave this to future works.

---

Acknowledgments and Disclosure of Funding
P.S. was funded partially by NSF DMS Award No. 2113426 and NSF CAREER Award No. 2440824.
References
[1] Molecular biology (splice-junction gene sequences) [dataset]. UCI Machine Learning Reposi- tory, 1991.
[2] Ben Adlam and Jeffrey Pennington. The neural tangent kernel in high dimensions: Triple descent and multi-scale theory of generalization. International Conference on Machine Leurning, pages 74-84. PMLR, 2020.
[3] Yu Bai, Song Mei, Huan Wang, and Caiming Xiong. Don't In'ttjuu just blame over-parametrization for over-confidence: Theoretical analysis of calibration in binary classification. In International conference on machine learning, pages 566-576. PMLR, 2021.
[4] Arindam Banerjee, Xin Guo, and Hui Wang. On the optimality of conditional expectation bregman predictor. IEEE Transactions on Information Theory, 51(7): 2664-2699,,, 2005.
[5] Jean Barbier, Florent Krzakala, Nicolas Macris, Léo Miolane, and Lenka Zdeborová. Optimal errors and phase transitions in high- high-ddmensional generalized linear models. Proceedings of the National Academy of Sciences, 116(12)55551-444 2019.
[6] Derek Bean, Peter Bickel, Noureddine El Karoui, and Bin Yu. Optimal m-estimation in high-dimensional regression. Proceedings of the National Academy of Sciences, 14566)):145633 14568, 2013.
[7] Edmon Begoli, Tanmoy Bhattacharya, and Dimitri Kusnezov. The need for uncertainty quantifi- cation in machine-assisted medical decision making. Nature Machine Intelligence, 1(1):20-23, 2019.
[8] Pierre C Bellec. Observable adjustments in single-index models for regularized m-estimators. arXiv preprint arXiv:2204.06990, 2022.
[9] Pierre C Bellec and Cun-Hui Zhang. De-biasing the lasso with degrees of-oofffffooooo adjustment. Bernoulli, 28(2):713-743 2022.
[10] Pierre Bellec and Cun-Hui Zhang. Debiasing convex regularized estimators and interval estimation in linear models. The Annals of Statistics, 51(2):391 2023.
[11] Vidmantas Bentkus. A lyapunov-type bound in rd. Theory of Probability & Its Applications, 49(2):311-3)) 1-323,113332220 2005.
[12] Eugene Berta, Francis Bach, and Michael Jordan. Classifier calibration with roc-regularized isotonic regression. In International Conference on Artificial Intelligence and Statistics, pages 1972-1980. PMLR, 2024.
[13] Björ Böken. On the appropriateness of platt scaling in classifier calibration. Information Systems, 95:101641, 2021.
[14] Jochen Bröcker. Reliability, sufficiency, and the decomposition of proper scores. Quarterly Journal of the Royal Meteorological Society: A journal of the atmospheric sciences, applied meteorology and physical oceanography, 135(649):1512-1119, 2009.
[15] Joan Bruna and Daniel Hsu. Survey on algorithms for multi-index models. arXiv preprint 2.Xiv22504005222 2025.
[16] Emmanuel J Candès and Pragya Sur. The phase transition for the existence of the maximum likelihood estimate in high-dimensional logistic regression. The Annals of Statistics, 48(1):27- 42, 2020.
10

---

[17] Michael Celentano and Andrea Montanari. Correlation adjusted debiased lasso: debiasing the lasso with inaccurate covariate model. Journal of the Royal Statistical Society Series B: Statistical Methodology, page qkae039, 2024.
[18] Michael Celentano, Andrea Montanari, and Yuting Wei. The lasso with general gaussian designs with applications hypothesis testing. The Annals of Statisties, 51(5):2194 2220, 2023.
[19] Wei-Kuo Chen and Wai-Kit Lam. Universality of approximate message passing algorithms. Electron. J. Probab., 26:1-44, 44, 2021.
[20] Sowmya Chithrananda, Gabriel Grand, and Bharath Ramsundar. ChemBERTa: Large-scale self-supervised pretraining for molecular property prediction. arXiv preprint arXiv:2010.09885, 2020.
[21] Lucas Clarté, Bruno Loureiro, Florent Krzakala, and Lenka Zdeborová. Expectation consistency for calibration of neural networks. In Uncertainty in Artificial Intelligence, pages 443-453. PMLR, 2023.
[22] Lucas Clarté, Bruno Loureiro, Florent Krzakala, and Lenka Zdeborová. Theoretical characteri- zation uncertainty in high-dimensional linear classification. Machine Learning: Science and Technology, 4(2):025029, 2023.
[23] Morris H DeGroot and Stephen E Fienberg. The comparison and evaluation of forecasters. Journal of the Royal Statistical Society: Series (The Statistician), 32(1-2): 1983.
[24] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. ImageNet: A large-scale hierarchical image database. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition (CVPR), pages 245, 2009.
[25] David Donoho, Arian Maleki, and Andrea Montanari. Message-passing algorithms for compressed sensing. Proceedings of the National Acudemy of Sciences, 106(45):19914-18919 2009.
[26] Rishabh Dudeja, Subhabrata Sen, and Yue M Lu. Spectral universality in regularized linear regression with nearly deterministic sensing matrices. IEEE Trunsactions on Information Theory, 2024.
[27] Oliver Feng, Ramji Venkataramanan, Cynthia Rush, Richard Samworth, et al. unifying tutorial approximate message passing. Foundations and Trends@ in Machine Learning, 154)53335-33, 2022.
[28]
Tilmann Gneiting and Adrian E Raftery. Weather forecasting with ensemble methods. Science, 3 31065746)2248-25) 2005.
[29] Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. International conference on machine learning, pages 1321-1330. PMLR, 2017.
[30] Chirag Gupta, Aleksandr Podkopaev, and Aaditya Ramdas. Distribution-free binary classifi- cation: prediction sets, confidence intervals and calibration. Advances in Neural Information Processing Systems, 33:371 1-3723,, 2020.
[31] Chirag Gupta and Aaditya Ramdas. Online platt scaling with calibeating. In International Conference Machine Learning, pages 12182-12204. PMLR, 2023.
[32] Isabelle Guyon. Madelon [dataset]. UCI Machine Learning Repository, 2003.
[33] Qiyang Han and Yandi Shen. Universality of regularized regression estimators in high dimen- sions. The Annals of Statistics, 51(4): 2023.
[34] Trevor Hastie, Andrea Montanari, Saharon Rosset, and Ryan Tibshirani. Surprises in high- dimensional ridgeless least squares interpolation. Annals of statistics, 50(2):949, 2022.
[35] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for im- age recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 770-778, 2016.

---

[36] Alexander Henzi, Johanna F Ziegel, and Tilmann Gneiting Isotonic distributional regression.
Journal of the Royal Statistical Society Series B: Statistical Methodology, 83(5):963-993, 2021.
[37] Hong Hu and Yue M Lu. Universality laws for high-dimensional learning with random features. IEEE Transactions on Information Theory, 69(3):1932 1964, 2022.
[38] Adel Javanmard and Andrea Montanari. Debiasing the lasso: Optimal sample size for Gaussian designs. The Annals of Statistics, 46(6A):2593 - - 2018.
[39] Kuanhao Jiang, Rajarshi Mukherjee, Subhabrata Sen, and Pragya Sur. A new central limit theorem for the augmented ipw estimator: Variance inflation, cross-fit covariance and beyond. arXiv preprint arXiv: 101988 2022.
[40] Xiaoqian Jiang, Melanie Osl, Jihoon Kim, and Lucila Ohno-Machado. Smooth isotonic regression: a new method to calibrate predictive models. AMIA Summits on Translational Science Proceedings, 201 1: 1:16, 2011.
[41] Xiaoqian Jiang. Melanie Osl, Jihoon Kim, and Lucila Ohno-Machado. Calibrating predictive model estimates to support personalized medicine. Journal of the American Medical Informatics Association, 19(2):263-2) 2012.
[42] lain M Johnstone and D Michael Titterington. Statistical challenges of high-dimensiona. data, 2009.
[43] Christopher Jung, Changhwa Lee, Mallesh Pai, Aaron Roth, and Rakesh Vohra. Moment multicalibration for uncertainty estimation. In Conference on Learning Theory, pages 2634- 2678. PMLR, 2021.
[44] Adam Tauman Kalai and Ravi Sastry. The isotron algorithm: High-dimensional isotonic regression. In COLT, volume 1, page 9, 2009.
[45] Agustinus Kristiadi, Matthias Hein, and Philipp Hennig. Being bayesian, even just bit, fixes overconfidence in relu networks. In International conference on machine learning, pages 5436-5446. PMLR, 2020.
[46] Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical report, University of Toronto, 2009.
[47] Meelis Kull, Miquel Perello Nicto, Markus Kängsepp, Telmo Silva Filho, Hao Song, and Peter Flach. Beyond temperature scaling: Obtaining well-calibrated multi-class probabilities with dirichlet calibration. Advances in neural information processing systems, 32, 2019.
[48] Ananya Kumar. Percy Liang, and Tengyu Ma. Verified uncertainty calibration. Advances in Neural Information Processing Systems, 32, 2019.
[49] Samriddha Lahiry and Pragya Sur. Universality in block dependent linear models with applica- tions to nonlinear regression. JEEE Transactions on Information Theory, 2024.
[50] Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles. Advances in neural information processing systems, 30, 2017.
[51] Ken Lang. Newsweeder: Learning to filter netnews. In Proceedings of the Twelfth International Conference on Machine Learning (ICML), pages 331-339, 1995.
[52] Yufan Li, Zhou Fan, Subhabrata Sen, and Yihong Wu. Random linear estimation with rotationally-innariin designs: Asymptotics at high temperature. IEEE Transactions Infor- mation Theory, 70(3):2118-22 2153,, 2023.
[53] Yufan Li, Subhabrata Sen, and Ben Adlam. Understanding optimal feature transfer via fine-grained bias-variance analysis. arXiv preprint arXiv:2404.1 12481, 2024.
[54] Yufan Li and Pragya Sur. Spectrum-aware adjustment: new debiasing framework with applications to principal components regression. arXiv preprint arXiv:2309.07810, 2023.
12

---

[55] Tengyuan Liang and Alexander Rakhlin, Just interpolate: Kernel "ridgeless" regression can generalize. 2020.
[56] Tengyuan Liang and Pragya Sur. A precise high-dimensional asymptotic theory for boosting and minimum-ll-me -nomn interpolated classifiers. The Annals of Statistics, 5063):1669-1655 2022.
[57] Kevin Luo, Yufan Li, and Pragya Sur. Roti-gev: Generalized cross-validation for right- rotationally invariant data, arXiv preprini arXiv:2406. 1666, 2024.
[58] Andrey Malinin, Bruno Mlodozeniec, and Mark Gales. Ensemble distribution distillation. arXiv preprint arXiv: 1005.00000000 2019.
[59] Song Mei and Andrea Montanari. The generalization error random features regression: Precise asymptotics and the double descent curve. Communications on Pure and Applied Mathematics, 75(4))667-766, 2022.
[60] Rhiannon Michelmore, Marta Kwiatkowska, and Yarin Gal. Evaluating uncertainty quantifica- tion end-to-end autonomous driving control. arXiv preprint 1rI 2018
[61] Marco Mondelli and Ramji Venkataramanan. Approximate message passing with spectral initialization for generalized linear models. International Conference on Artificial Intelligence and Slatistics, pages 397-405. PMLR, 2021.
[62] Andrea Montanari and Basil N Saeed. Universality of empirical risk minimization. In Confer- ence on Learning Theory, pages 4310 0312.. PMLR, 2022.
[63] Andrea Montanari, Subhabrata Sen, al. friendly tutorial on mean-field spin glass techniques for non-physisssts. Foundations and Trends@ in Machine Learning, 17(1):1-173, 2024.
[64] Allan H Murphy, A new vector partition of the probability score. Journal of Applied Meteorology and Climatology, 12(4):595-600, 1973.
[65] Allan H Murphy and Robert L Winkler. Reliability of subjective probability forecasts of precipitation and temperature. Journal of the Royal Statistical Society Series C: Applied Statistics, 26(1):41-47) 1977.
[66] Khanh Nguyen and Brendan O'Connor. Posterior calibration and exploratory analysis for natural language processing models. arXiv preprint arXiv:150..051 2015.
[67] Pratik Patil, Jin-Hong Du, and Ryan J Tibshirani. Optimal ridge regularization for out-of- distribution prediction. arXiv preprint arXiv:2404.01233 2024.
[68] Pedregosa, G. Varoquaux, A. Gramfort, Michel, B. Thirion, Grisel, M. Blondel,
P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and Duchesnay, Scikit-learn: Machine learning in python. Journal of Machine Learning Research, 12:2825-2830, 2011.
[69] Nathan Phelps, Daniel Lizotte, and Douglas Woolford. Using platt's s scaling for calibration after undersampling -imitatiion and how to address them. arXiv preprint arXiv:2410.18144 2024.
[70] John Platt et al. Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. Advances large margin classifiers, 10(3):61-744 1999.
[71] Michael Redmond. Communities and crime [dataset]. UCI Machine Learning Repository, 2002.
[72] Nils Reimers and Iryna Gurevych. Bettence-BERT: Sentence embeddings using siamese BERT-networks. In Proceedings of EMNLP-IJCNLP, pages 3982-3992, 2019.
[73] Ralph Tyrell Rockafellar. Convex analysis. Princeton university press, 2015.
[74] Eliran Shabat, Lee Cohen, and Yishay Mansour. Sample complexity uniform convergence for multicalibration. Advances in Neural Information Processing Systems, 33:13331-11344 2020.
13

---

[75] Yanke Song, Sohom Bhattacharya, and Pragya Sur. Generalization enror of min-norm interpola- tors in transfer learning. arXiv preprint arXiv:2406.13944 2024.
[76] Yanke Song, Xihong Lin, and Pragya Sur. Hede: Heritability estimation high dimensions by ensembling debiased estimators. arXiv preprint urXiv:2406.1118% 2024.
[77] Zeyu Sun, Dogyoon Song, and Alfred Hero. Minimum-risk recalibration of classifiers. Advances in Neural Information Processing Systems, 36, 2024.
[78] Pragya Sur and Emmanuel Candès. A modern maximum-likelihood theory for high- dimensional logistic regression. Proceedings of the National Academy of Sciences, 116629::4516-14555, 2019.
[79] Pragya Sur, Yuxin Chen, and Emmanuel J Candès. The likelihood ratio test in high- -igh-dimensional logistic regression asymptotically rescaled chi-square. Probability theory and related fields, 175:487-558, 2019.
[80] Christos Thrampoulidis, Samet Oymak, and Babak Hassibi. The gaussian min min-max theorem in the presence of convexity. arXiv preprint wrXiv:1408.445 2014.
[81] Linh Tran, Bastiaan S Veeling, Kevin Roth, Jakub Swiatkowski, Joshua V Dillon, Jasper Snoek, Stephan Mandt, Tim Salimans. Sebastian Nowozin, and Rodolphe Jenatton. Hydra: Preserving ensemble diversity for model distillation. arXiv preprint arXiv:2001. 1.4694, 2020.
[82] Emanuele Troiani, Yatin Dandi, Leonardo Defilippis, Lenka Zdeborová, Bruno Loureiro, and Florent Krzakala. Fundamental computational limits of weak learnability in nigh-dimensional multi-index models. arXiv preprint arXiv:2405.15480, 2024.
[83] Aad W Van der Vaart. Asymptotic statistics, volume 3. Cambridge university press, 2000.
[84] Shuaiwen Wang, Haolei Weng, and Arian Malcki. Which bridge estimator is the best for variable selection? The Annals of Statistics, 48(5):2791 2823, 2020.
[85] Wenhui Wang, Furu Wei, Li Dong, Hangbo Bao, Nan Yang, and Ming Zhou. MiniLM: Deep self-attention distillation for task-agnostic compression of pre-trained transformers. arXiv preprint a0Xiv:2002. 10957, 2020.
[86] Yeming Wen, Dustin Tran, and Jimmy Ba. Batchensemble: an alternative approach to efficient ensemble and lifelong learning. arXiv preprint arXiv:2002.06715, 2020.
[87] Zhenqin Wu, Bharath Ramsundar, Evan N. Feinberg, Joseph Gomes, Caleb Geniesse, Aneesh Pappu, Karl Leswing, and Vijay Pande. MoleculeNet: A benchmark for molecular machine learning. Chemical Science, 92):513-530, 2018
[88] Zhuoran Yang, Krishnakumar Balasubramanian, Zhaoran Wang, and Han Liu. Estimating nigh-dimensional non-gaussian multiple index models via stein's lemma. Advances in Neural Information Processing Systems, 30, 2017.
[89] Bianca Zadrozny and Charles Elkan. Obtaining calibrated probability estimates from decision trees and naive bayesian classifiers. In Ieml, volume pages 609-616, 2001.
[90] Bianca Zadrozny and Charles Elkan. Transforming classifier scores into accurate multiclass probability estimates. In Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining, pages 694.699, 2002.
[91] Lenka Zdeborová and Florent Krzakala. Statistical physics of inference: Thresholds and algorithms. Advances in Physics, 65(5):453-552 2016.
[92] Qian Zhao, Pragya Sur, and Emmanuel J Candes. The asymptotic distribution of the mle in high-dimensional logistic models: Arbitrary covariance. Bernoulli, 28(3):1835-18611 2022.
14

---

A Proof of Theorem 3.2
Before proving Theorem 3.2 we first show that Sang (:;0.) is exactly calibrated.
Theorem A.1. The predictor Sang defined in is well-calibrated at all p E [0, 1] when. That is, for any p E [0, 1] and any d, 1
A cal (fang (::0)) = p-- Bxaeo [or (w, Fnew) faang (w Tnewi 0s) = =0.
Proof of TheoremA. A.1
Let us define the following event
A : {fans (w. Nnew 0.) PJ.
We have
E Exnow
(w_ knew) [A]
Brnow Bxnew [o (w, Inew)
2 2 w] [A]
cas (f.) x "n w + sin (0.) 2 2)] A
|
Bareow
Bz 1
Exnow
Sang (w] Inow; 0.) A
p.
where (i) follows from tower property of expectation and the fact that Sang (Know) depends Inew only through " W. (ii) follows from conditional expectation of multivariate Gaussian distribution
[w. Fnew
Fnew 20
cos (0.) 2 2now 2 + sin (0.) Z
(15)
for some Z ~ 10.
Proof of Theorem 3.2 probability, we have that
Using result from Theorem A.1 it suffices show that 10 00| 0.1 0 in
A cal (fang (::b))
Ap (fang (::0) 0.
(16)
Let us introduce the following notation for the of presentation:
V fang (it" Nuew: 0). Y. Sang (10 Iuew:
X = ( knew).
Then, can write LHS of [6
fx(()) I. (x (r. 10 I xfx.y. (x.
EX = E X Y, pl
where fv. fr. fx.Y, fx,y the distribution density functions of Ý Y. and joint density functions of (X, Y.) and (x, F). We now show that the RHS of the above converges to 0. Firstly, [y.(
because LÝ Y.l 0 in probability (and thus in distribution) by continuous mapping theorem. Secondly.
V æfx xfx. (x. [( æfx.Y, (I. p)dæ - 0
by bounded convergence theorem and the fact that (X, 8) converges (X, Y.). jointly. We conclude the proof.
D
15

---

B Proof of Theorem 4.2
We first state result from 4 for general random variables.
Proposition B.1 (Theorem 1, [... Let R R be strictly convex differentiable function, and let D& be the corresponding Bregman loss function. Let X be an arbitrary random variable taking values in IR for which both E and [[ are finite. Then, among all functions of Z. the conditional expectation is the unique minimizer (up equivalence) of the expected Bregman loss, i.e.,
arg min E (X) R[X 2).
YEA(Z)
Using the above results, we show that angular calibration with 0 = 8, minimizes Bregman divergence true label distribution among predictors of the form F knew).
Theorem B.2 (Optimality of angular predictor). Let Φ IR² + R be any strictly convex differentiable function, and let D& be the corresponding Bregman loss function. Let E be finite. Then, the expected Bregman loss E ge [D qr) admits unique minimizer (up equivalence) among all Qr, VF E F == (s R + [0, }].. Let this minimizer be F+ = arg min Fayeew (Dss (1x, (r)] We then have that almost surely
aang (0.) F. Inew)
where Jang(0, is the label prediction probability vector by angular calibration given with replaced by O..
Proof of Theorem
B.2 Firstly, we set X, Y, Theorem B.I as
(1
oww knew)
F Fnew)
F( Inow)) )
X
Y
Z Inow-
(1a((w, ow, knew)
The result then follows from Theorem
B. and the following
E
Z
E
(1
a (w, Inew) (w, knew)
))
x 10
(Jaew))
Sany (aang((uaaa (Freww
where we used
15 and
3 for the last equality.
Now are ready to state proof of Theorem 4.2
Proof. Using result from Theorem B.2 it suffices to show that
(ang (0) Gang (ang((.))
in probability. This an immediate consequence of the continuous mapping theorem under the assumption a is continuous.
C Proof of Theorem 5.1
Before proving Theorem 5.1 we first state two classic analysis results that we will later use.
Proposition C.1 (Theorem 5.7, g3 Let En be random functions on H. e be a fixed function on H. and 0* E H such that (i) uniform convergence of n to E holds:
sup DEN n
(0) (0)
inprob.
n 11000
(ii) the mode of Et well-separated, for all
sup 0EH:
(0) < 8" (0")
Then any sequence 0, maximizing e, converges in probability to 0*.
16

---

Proposition C.2 (Theorem 10.8, (73)). Let C be relatively open convex set, and let f1, f2...... be sequence of finite convex functions on C. Suppose that the sequence converges pointwise on dense subset of C, Le. that there exists subset C' of C such that its closure satisfies clC" 5 C and, for each x E C², the limit of (x)),, fa(x),.... exists and is finite. The limit then exists for every x E C. and the function f. where
f(z) = lim fi(x)
+-00
finite and convex on C. Moreover the sequence SI, J2. ...... converges to f uniformly on each closed bounded subset C.
Now we are ready to prove Theorem 5.1
Proof of Theorem 5.7 tion (-))
8
is obtained from applying the well-known identity below for probit func-
(V1+o)))
ED(p -))
N(0,
to
To prove that Ana - A., Buo - B. as Tho - oc, we would like to apply C.1 by setting
<- Tho x {A, B), 0" +- {A,, B.).
(0) + Ln (A, =
yho log( FA abo,))) - -(1 yyho,;) log(1 (1 - FA. (w) zho,)))
(17)
and
(0) - (A, E
σ (w,, Inew log (FA,B B (w Inew))
(18)
(1 o (w. Knew )) blooogg FA,B (T knew) ))
where we used the notation
FA, == (AAu + B) P(a(Au + B)+ b).
Here, RHS of 188) is up to an affine transform of the KI. divergence 6 and, therefore, it follows from and (??) that its minimizer is indeed A., B..
To verify condition (i) of Theorem C.I. we first note that (17) converges to (8) point-wise in probability following from law of large number theorem. Furthermore, note that the functions
[i(u) == - log(
fa(u) = - log(1 - +()))
are both strictly convex functions. To see this, one can show second derivatives are positive for all
IR by utilizing the two elementary inequalities: ub(uu) + 4'(u) > 0, ud(u)) + +'(w) > u.
then follows that E (A, B) is convex function of (A, B) on the domain IR.. Uniform convergence on H is then an application of Theorem
To verify condition (ii) of Theorem C.I
we only need to show that (A, B) is strictly convex function in (A, B). Let write the inside of the expectation of (T8) as
[(A, B) = ar (H)J H + aB + b) + - (H)) H aB + b)
where = w. Tnew (0, 1). Then, we have that
(o2H² a²H a²H a²
V*)(A (a(H)- S fi (aA H +a +(1 +11 -111-((H) (aA-H+aB+H)))
which is positive-definite almost surely when a * 0. Hence, we have that almost surely f (t(A1, B1) ) (1 (A,, < tf ((A1, + (1 t)f ((A1,
which implics that
Era. f (t(A,, B1) (((AI,BB) < ti of ((A), B1)) + (1 E)EE f ((A), B1)).
17

---

The claim that E (A, B) is strictly convex follows.
It then follows from Theorem C.I that A + A., Buh - B, in probability as Tho + cc. The uniform convergence plats (u) (un follows immediately.
D Inner product estimation
We restate the following results from Theorem 4.4, 8] We note that the quantity 6212 in the error bound is observable and typically of constant order in the proportional regime. See for details.
Theorem D.I. Suppose l continuously differentiable and g is strongly convex and twice differential S :/ for some 6 > 0, for arbitrarily large probability
penalty function. Assume also that 1 - 6, the following holds
C74
E à² (w., w)
1/2
0282
where C/ a constant depending only on g and 6.
E Sign estimation
Proof of Theurem 6. With respect randomness in validation dataset (that is, @ treated as deterministic), we have that ti arho who are iid across and satisfies that
bo Bern, ((((vr))))) TT Ewi
k) TT
@ ho ho w aie
x. + sin (0.) Z,
- || Bern, (a (cos (0.) U, sin (0.) Z,)) = H3
where Z; ~ NO, 1) and we have used w 2:o0 || |ا
U, for U, iid N(0, 1). It follows from
Gaussian integration by parts that
EH: (w,, w Eo' (cos (0.) U, + sin (0.) 7,) (w+, w) Ed Ro'(Z) (Z).
Meanwhile, H, is subGaussian with subGaussian norm
H 12 S L 22 11 < 3
By theorem assumption, Eo'(2) > 0 and
sign ((w,, 3) IE (Z)) = sign (ww,, w)
So the sign identification of sgn is correct if the following event holds
w T xi (w+, 10 Ro' (Z) < (w., 1 Ro' (Z))
7ho
1=1
By Hoeffding's inequality.
Tho
hoo
w
24 Wi (w,, B Eo' Eo'(Z)
> kw.,, 1) Eo' [z)(
nho
CTho (Eo'(Z)) (w., w)
cxp (-enho (cos (0.) Eo' (2))²)
< exx
The theorem statement follows.
18

---

Angular calibration for multi-index models
Let Neew 200,2 Rd. Fix K and let W,, [ W.kl RR Define
:= W new E RK <(Znew) = g(G),
where g a generalized link (vector- or scalar- valued). This setup covers:
Two-layer nets (frozen outer layer): g(u) = Dx an o(us)
Multi-class softmax: g(u) = softmax(()
Additive index model: g(u) = Σx (u()
Interaction index model: g(u) Σx fu(u)) Enee (ee uz)
The following extends angular calibration to mutli-index models. Note that to apply the angular predictor (9)). we must estimate cross-index angles (m+,,. We)xy similarly to the single-index case. Though no estimator given in literature far we know, recent theory for multi-index models [82 suggests that analogues of the single-index angle estimators are feasible. We leave derivation of these estimators to future works.
Theorem F. (Angular calibration for multi-index models). Ler W [......, oxl E BaxK be any estimator. Set
D diag( || (ls 10 l1s:);
S: D W Nnww E R
Then we have K x K covariance blocks
Cov(G) = W EW.,
:: Cov =
- D- EWD C: Cov(G, - EWD-1 D-1
where
REE = (wi, wehs
(Wok: We)s:
C&e
1e || w.
Then, assuming that invertible, we may define
M. = C E. E. -CRIG)
and we have that G S ~ N E.). For any factor L, with L, E,, and any 2 ~ N(0, Ix) independent of everything, define the multi-index angular predictor
Jang I T knew) EEz (M
(19)
Then for any p E K-1 and any d, n N+.
- E T (#now) Pang (W Inew) = = 0.
Proof. Let Fnew ~ N(, Σ) and define
: W. Nnww K
S D-W Inow k
with D) = diag(1 @1 I|,,..................
j 1x) and |auil = Vu Lv. (u, U) == u Dv. Set the 2K x d
linear map
WT
G S
T -=
Y =
= T Tnew-
Since Tnew is Gaussian and Y is a linear transform, Y is jointly Gaussian with mean 0 and covariance
EWW D- WW,
W. DWD-1 D D-D-1
Cov(Y) = TET
19

---

Thus,
Cov(G) W. EW, In particular, for k & E [K],
:: Cov(S) = D EWD- C: Cov(G, -.
(iiky wp)s
(Wek; wz)x
Rge
Oke
1 I "
| |I
| ||
Define the K x K matrices
E. - M. : CR
Consider the linear residual
U= G-
Because Y is Gaussian, U is Gaussian; further,
Cov(U, Cov(G, - CR C -
so U and S are independent (uncorrelated jointly Gaussian vectors are independent). Moreover,
Cov(U) Cov(G) - CC- E..
Hence we have the orthogonal decomposition
G M.S+UU U,
U 1L S, U~N(0,E,))
Equivalently, the conditional distribution is
G S (MSS, E.),
which is the standard multivariate normal conditioning formula via the Schur complement.
Finally, let L, be any matrix satisfying L,L. E, and let Z NO Ik) independent of (G, S).
Then U L, Z and
GlS - M.S + L.Z.
Therefore, for any measurable g (vector- or scalar-valued) with the requisite integrability,
Eig(G) S] Ez [MM,S + L,Z)],
which yields the stated multi-index angular predictor
Fang (W T Inew)
+ = Ez ( L.Z)].
To conclude, set X = T [Znow) = ((G) E K-1 and
Y Fang
W knew) = E[g(G)) SS)
Then Y is measurable and Y E[X S] almost surely. We claim that
X X y y
a.s.
Indeed, for any bounded measurable p A K-1 + R. E)) (X y) E El) (X Y) | ({) - (B[x $] [8-YY] Y)] 0,
so E[X YJ Y (coordinate-wise) by the defining property of conditional expectation.
By the existence of regular conditional expectations, this implies that for -lmostttt every p A K-1
E
π (Inew)
fang (W Inew) p E pl p.
i.e.,
E
n (Tnew) Bang W know) = p 0.
This establishes exact calibration of the multi-index angular predictor.
20

---

G
Pseudocode
Algorithm 1: Angular Calibration
Input :Training data {2 ys)] Ji=ii link convex loss & and penalty gi covariance Σ (known or estimated from unlabeled data);
holdout set {(a)") ")")) ) The for sign estimation;
Output: Calibrated predictor faug(z) that returns P [0, 1] for any
(A) Fit base linear model
@ arg min n >E= (woT x;) gww)
| <>
Σ 21/2
(B) Observable magnitude of (wi+; w)
v/ - Cy (2, w); S (wi,..... (in)T
D t (X H (x DX + n (()())))
Tr(D DXHXTD) D); TXXXXX D); f2 1//1//n
G 1382 L X 73222
11 Est. of (w., 0)
+ + X + L | 7/012 42
(C) Sign via holdout correlation
8gn
sign D (w x2!o) y! y!
(D) Angle & interpolation weights
sgn
e min max c}}
// numerical clip
0 arccos(c); a - cos -c,; c; 3 sin = -
(E) Define calibrated predictor fang For any new J:
4- W
u/ | E
draw 21,...,2M PM (O,, 1) and set faug(r) MM E/= gass B2,)
return fang( (z)
21

---

H
Additional plots
H.1
Universality
line
45 lie
Uncalbrared
0.90
Uncafieeced
Emplrical Probabliity
Angular
Empirical Probability
Angular
Flett scal oo
1000
scaling
70000
0.70
0.70
prediction
0.0
0.50
0.50
0.4
6.2
0.10
0.10
0.0
Predicted Probability
Predicted Probability
45 line
1.6
0.90
Uncalbmrted
0.90
Uncalreeed
Empirical Probability
Anguder
Felt
1000
Probability
Anguder
0.8
0.70
prediction
0.6
0.50
0.50
Empirical
0.30
0.30
0.30
0.30
0.50
0.70
0.90
0.30
0.30
0.50
0.70 0.90
2
Predicted Probability
Predicted Probability
Figure 3: Reproduce Figure I (in the third column) and (in first two columns) for Rademacher entries. Upper Row: rerun simulations in Section 7 but with subGaussian designs Wy where Wi sampled iid from Rademacher distribution, taking values +1, -1 with equal probability. Bottom
Row: we replace the sigmoid link function in Section 7 with a clipped relu link function o(x) =
clip(3z + 0.5) where clip(x) x, Vo [0, 1). clip(x) 0, Vr < 0 and clip(x)) 1, Vx 1.
22

---

45 tiee
45 line
1.c-
0.90
Uncatbnaced Anguder
0.90
Unsatbraced
Empirical Probability
190)
Empirical Probability
Angudar
Mattscaaig
200000
6.8
0.0
prediction
0.50
0.50
6.4
0.30
0.30
0.10
0.30
0.50
0.70
0.90
0.30
0.30
0.50
0.70
0.90
-2
Predicted Probability
Predicted Probability
0.50
0.90
Empirical Probability
Empirical Probability
np
0.70
0.70
prediction
0.30
0.30
0.10
0.10
0.0
Predicted Probability
Predicted Probability
Figure 4: Reproduce Figure | (the third column) and 2 (first two columns) for uniform entries.
Upper Row: rerun simulations in Section 7 but with non-Gaussian designs Wy where Wiy sampled iid from uniform distribution, taking values in interval [-v12/2, v12/2] uniformly at random. Bottom Row: we replace the sigmoid link function in Section with a clipped relu
link function (x) clip(3r + 0.5) where clip(x) Væ [0, 1], clip(x) 0, Vr < 0 and clip(r) = 1, Vx 1.
23

---

H.2 UCI benchmarks and pretrained embeddings
45" Fie Uneal
66" lnt
Uncalbrrtedd
Shoill ba araded
Empirical Probabiltty
Anpular
Empirical Probability
Fnn
Probability
Miziscalingg Sou 1901
scaling Vtum 2001
aod Mho
0.70
0.10
00.00
0.50
0.50
0.50
Emplrical
0.30
3.10
0.30
0.50
0,70
0.50
0.10
0.30
0.50
0.79
0.50
0.10
0.30
0.50
0.70
3.5
Predicted Probability
Predicted Probability
Predicted Probability
(a) CIFAR-10
(b) 20 Newsgroups
(c) Tox21
49 Ing
Unvel
incellemtedd
Uncal brated
Empirical Probability
Angolar
Pinteralinn Sie 1001
Empirical Probability
Aeg
scaliig Poe 2000
Probabiiity
Aeguir
Palt inc.. 1631
0.70
0.90
0.10
0.50
0.50
0.50
Impriical
2.10
0.50
0.50
0.70
0.90
0.30
0.30
0.50
0.73
0.90
0.10
0.30
0.30
0.70
3.90
Predicted Probability
Predicted Probability
Predicted Probability
(d) Communitics Crime
(c) DNA
(f) Madelon
24

---

I Simulation code and reproduction
The simulation code www.lacaaloo.iiyyy/.... ipynb, Semi_expppieee aaaii_eperrment..pppee included as supplemen- tary material. The experiment can be run on modern personal computers without needing special computing hardware. Detailed instructions is provided in the simulation code.
25

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We have clearly state the claims made, including the contributions made in the paper and important assumptions and limitations, abstract and introduction
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: Please see Limitations and future directions section.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
26

---

All assumptions are included either in section or in each theorem statement. Full proofs are included in appendices.
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We have fully disclose all the information needed to reproduce the main experimental results.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
27

---

Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We have provided source code for our simulations in supplementary file, along with detailed instructions.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please the NeurIPS code and data submission guidelines (https://nig) cc/
pub Mublic//e qttcc//giees//eeee CodeeSuu ss Peeioonollii ollyy) for more details.
While encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https //nips. cc/ pucl/pullii i c/ des CodeSubri ss i onPol icy for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
All simulation details are disclosed in Section 7 Simulations.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: See Section
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi-
dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for
example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
28

---

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: See Appendix
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make it into the paper).
9. Code ethics
Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Question:
Answer: [Yes]
Justification:
The paper conform, in every respect, with the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: Our work is purely theoretical, aimed at advancing the mathematical under- standing of calibration in an idealistic setting. It does not involve any personal, sensitive, or protected information. As such, there is no plausible way for our methods be used for discrimination, profiling, surveillance, or any other harmful purpose.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
29

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [NA]
Justification:
The paper does not use existing assets.
Guidelines:
The answer NA means that the paper does use existing assets. The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website). the copyright and terms of service of that source should be provided.
license, copyright information, and terms of use package should be provided. For popular datasets, baapeettsppprrssseeee paperssws thoode. com datasstt has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
assets are released,
30

---

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moddl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Bthies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, not include any information that would break anonymity (if applicable), such the institution conducting the review.
31

---

16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
Justification: LLM is only used for minor grammar editing.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer our LLM policy (https: : /ttp///eerpppppppe/e cc Cotp//ooeeeeeessss//../o 2025/ LLM) for what should or should not be described.
32


Reviews:
[{"review_id": "8U5j8JBxGA", "reviewer": "Reviewer_WYdS", "summary": "This paper addresses the fundamental problem of probabilistic calibration for binary classifiers in high-dimensional settings, where both the sample size $n$ and the feature dimension $d$ diverge at comparable rates. The authors propose a novel angular calibration method that achieves provable calibration by interpolating between the logit scores of a (given) linear predictor and an uninformative “chance” predictor (a purely random variable), with the interpolation weight determined by the angle between the estimated weight vector $\\hat{w}$ and the ground-truth weight $w_*$. A key contribution is the theoretical result that this predictor not only achieves calibration but also uniquely minimizes any Bregman divergence to the true label distribution, among a wide class of predictors (Section 4). Further, the authors show that classical Platt scaling converges to the angular calibration under certain conditions, thus inheriting both calibration and optimality guarantees in high dimensions. The paper also presents a data-driven estimator for the angle $\\angle(\\hat{w}, w_*)$ --- estimator of the squared inner product proposed in [7], plus a sign estimator --- and supports the theory with simulation results. The authors conclude the paper with a brief discussion and experimental results discussing universality beyond Gaussian design.", "questions": "**1. Scope of applicability beyond regularized M-estimators:** \nThe results in Sections 4 and 5 appear to apply to any fixed predictor $\\hat{w}$, even if it is not derived from a regularized M-estimation procedure. However, the angle estimation procedure in Section 6 critically relies on $\\hat{w}$ being a regularized M-estimator, to leverage observable quantities from recent debiasing literature. It would be helpful if the authors could clarify this distinction explicitly and discuss how the approach could (or could not) be extended to more general $\\hat{w}$ (not necessarily a regularized M-estimator). \n\n**2. Finite-sample behavior and out-of-sample generalization:**\nAs noted above, the theoretical results are asymptotic. While simulations indicate promising performance in specific scenarios, could the authors provide further guidance or empirical evaluation of how calibration quality and Bregman loss behave as functions of sample size, model misspecification, or holdout set size? In particular, how sensitive are the procedures to finite-sample noise in the angle estimator or to misspecification of the link function or data distribution? Addressing these questions, even heuristically or empirically, would help assess the method’s robustness in real-world applications.", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "### Strengths\n- **Significance:** The paper addresses an important problem of how to calibrate predictive probabilities in high dimensions, where traditional methods are known to fail. The angular calibration method is conceptually elegant, connects well with recent advances in debiasing techniques, and, to the best of my knowledge, offers the first provable calibration procedure in this regime with optimality guarantees.\n- **Theoretical quality:** The main results are mathematically rigorous and clearly stated. The proofs are well-grounded and provide meaningful insight into the behavior of calibration methods when the covariate dimension $d \\sim n$, the sample size. The analysis linking angular calibration and Platt scaling is particularly noteworthy and adds theoretical justification for a widely used heuristic.\n- **Clarity:** The paper is well-organized and clearly written. The paper provides useful intuition behind the angular predictor and its interpolation structure, making the main ideas accessible despite the technical depth.\n\n### Weaknesses\n- **Lack of finite-sample guarantees:** A notable limitation is the lack of finite-sample results. All theoretical guarantees are asymptotic in nature. While I understand this asymptotic result has its own values, it leaves open questions about the accuracy of calibration and optimality in practical settings. A discussion of the rate of convergence or any quantification of the asymptotic approximations’ accuracy in finite-sample setting would be useful to gauge practical applicability. Moreover, while simulations offer some insight, a more thorough empirical investigation into these finite-sample effects could help bridge the gap between theory and real-world use.\n- **Assumptions and practicality:** The results rely heavily on the Gaussian design assumption and knowledge (or estimability) of the covariance $\\Sigma$, Although the authors briefly discuss universality (Section 8) and show simulations under non-Gaussian designs, formal universality results are not yet developed and deferred to future work.\n- **Scope of empirical validation:** While the simulations are well-designed and support the theory, they are limited in scope. Empirical comparison against alternative calibration methods (e.g., temperature scaling, isotonic regression, histogram binning) beyond Platt scaling would further strengthen the case for the proposed method.", "was_revised": true, "final_justification": "The paper presents a clean and insightful optimal calibration method that theoretically justifies Platt scaling. The authors addressed my concerns and questions by providing additional theoretical results for multi-index models and experimental results to more realistic settings along with ablation studies. These additions strengthen both the practical relevance and theoretical scope of the work. Thus, I have raised my score in light of the authors’ detailed and satisfactory rebuttal.", "rebuttal": "We thank reviewer for their thoughtful review. \n\n## R1. Weakness\n\n1. Providing finite‑sample guarantees is an interesting direction. Theoretically, one may use Eq. (4.14) of \\[1], which shows estimator (11) of \\$\\langle w^\\star,\\hat w\\rangle\\$ converges at \\$n^{-1/2}\\$ rate. With additional smoothness on the link, we expect Theorem 3.2 to achieve the same \\$n^{-1/2}\\$ rate. Our simulations in **Section R4** suggest an empirical rate of \\$n^{-0.55}\\$\n2. We agree. Formalizing Section 8’s ideas and proving universality is important future work.\n3. See **Section R4** for experiments using angular calibration (v.s. isotonic regression and Platt scaling) on logits from ResNet‑34, MiniLM, and ChemBERTa over CIFAR‑10, 20 Newsgroups, and Tox21; we also include UCI datasets: Communities & Crime, Splice‑junction, Madelon, ISOLET\n\n## R2. Questions\n\n1. Correct. Theorem 3.2 applies to any estimator \\$\\hat w\\$ independent of the new observation \\$x\\_{\\text{new}}\\$. M‑estimation is only required to estimate \\$\\langle w^\\star,\\hat w\\rangle\\$. Theorem 3.2 currently treats binary linear classification, but it can be extended to nonlinear, multi‑index models, including 2‑layer neural networks, multi‑class classification, additive, and interaction models (see **Section R3** in our response to Reviewer **thu8**).\n2. We ran ablations addressing the reviewer’s questions:\n\n   * (i) **Section R4**: proportional \\$n,d\\$ (finite‑sample effect)\n   * (ii) **Section R5**: vary \\$n\\$ with fixed \\$d\\$ and vary \\$d\\$ with fixed \\$n\\$\n   * (iii) **Sections R6**: sensitivity to angle‑estimation noise\n   * (iv) **Sections R7**: sensitivity to  misspecified link functions\n\nWe report result in **Expected Calibration Error (ECE)**, a measure of confidence–accuracy gap \\[2].\n\n## R3. Semi‑real experiments\n\nFor Tables 1–2, each dataset is split into training (\\$n\\_{\\text{train}}\\$), covariance estimation (\\$n\\_{\\text{cov}}\\$), and test (\\$n\\_{\\text{test}}\\$). We set \\$n\\_{\\text{cov}}\\gg n\\_{\\text{train}}\\$ to reflect abundant unlabeled data, and report ECE for \\$n\\_{\\text{ho}}=100\\$ and \\$500\\$. \n\nWe found the plug‑in estimator in (11) for \\$\\langle w^\\star,\\hat w\\rangle\\$ unstable on real datasets—known for Wigner‑type random‑matrix–based estimators. Improving such estimators is an active area \\[3]. To isolate calibration effects, we simulate \\$w^\\star\\$ and labels using the *true* angle.\n\nTo extend calibration to nonlinear models, we apply it to pretrained logits:\n\n* **ResNet‑34 → CIFAR‑10**: 512‑dim penultimate layer (ImageNet‑pretrained)\n* **MiniLM → 20 Newsgroups**: mean‑pooled 384‑dim “all‑MiniLM‑L6‑v2” embeddings\n* **ChemBERTa → Tox21**: mean‑pooled 768‑dim hidden states from “seyonec/ChemBERTa‑zinc‑base‑v1”\n\n\\$n\\_{\\text{cov}}=30000,3000,3000\\$; \\$n\\_{\\text{test}}=10000,1000,500\\$\n\n*Table 1.* ECE for pretrained DL models:\n\n| Model–Dataset       | \\$n\\times d\\$ | Uncal  | Angular | Platt 100 | Iso 100 | Platt 500 | Iso 500 |\n| ------------------- | ------------- | ------ | ------- | --------- | ------- | --------- | ------- |\n| ResNet34–CIFAR10    | 300×512       | 0.1236 | 0.0199  | 0.0561    | 0.0484  | 0.0259    | 0.0298  |\n| MiniLM–20newsgroups | 800×384       | 0.1392 | 0.0249  | 0.0931    | 0.1107  | 0.0679    | 0.0813  |\n| ChemBERTa–Tox21     | 800×768       | 0.1389 | 0.0132  | 0.0236    | 0.0497  | 0.0175    | 0.0293  |\n\nFor UCI datasets:\n\n* Communities & Crime (200×100), Splice‑junction (100×180), Madelon (300×500), ISOLET (500×617)\n* \\$n\\_{\\text{cov}}=700,2000,900,5750\\$; \\$n\\_{\\text{test}}=593,900,900,1000\\$\n\n*Table 2.* ECE on UCI datasets:\n\n| UCI Dataset              | \\$n\\times d\\$ | Uncal  | Angular | Platt 100 | Iso 100 | Platt 500 | Iso 500 |\n| ------------------------ | ------------- | ------ | ------- | --------- | ------- | --------- | ------- |\n| Communities and Crime    | 200×100       | 0.1700 | 0.0296  | 0.0620    | 0.0661  | 0.0262    | 0.0436  |\n| Splice-junction Gene Seq | 100×180       | 0.1262 | 0.0208  | 0.0568    | 0.1041  | 0.0556    | 0.0879  |\n| Madelon                  | 300×500       | 0.1473 | 0.0267  | 0.0721    | 0.0719  | 0.0299    | 0.0624  |\n| ISOLET                   | 500×617       | 0.1256 | 0.0221  | 0.0438    | 0.0761  | 0.0219    | 0.0137  |\n\n# R4. Finite sample effect\n\nSame setting as paper Section 7. \n\n**Table 3.** ECE across increasing (n, d) proportionally\n\n| Method                  | 50×100          | 100×200         | 200×400         | 400×800         | 800×1600        | 1000×2000       | 1500×3000       |\n| ----------------------- | --------------- | --------------- | --------------- | --------------- | --------------- | --------------- | --------------- |\n| **Uncalibrated**        | 0.114 ± 0.0024  | 0.114 ± 0.0019  | 0.1145 ± 0.0021 | 0.1128 ± 0.0022 | 0.1129 ± 0.0020 | 0.1141 ± 0.0026 | 0.1134 ± 0.0027 |\n| **Angular**             | 0.0491 ± 0.0259 | 0.0387 ± 0.0277 | 0.0188 ± 0.0117 | 0.0184 ± 0.0154 | 0.0149 ± 0.0072 | 0.0080 ± 0.0031 | 0.0081 ± 0.0042 |\n| **Platt Scaling**       | 0.0305 ± 0.0196 | 0.0300 ± 0.0187 | 0.0366 ± 0.0148 | 0.0293 ± 0.0144 | 0.0395 ± 0.0244 | 0.0336 ± 0.0203 | 0.0309 ± 0.0141 |\n| **Isotonic Regression** | 0.0510 ± 0.0222 | 0.0479 ± 0.0266 | 0.0548 ± 0.0257 | 0.0432 ± 0.0189 | 0.0549 ± 0.0261 | 0.0484 ± 0.0210 | 0.0494 ± 0.0224 |\n\nWe fit a power law:\n$\\mathrm{ECE}\\approx 2.239\\,n^{-0.552},$\nso ECE improves at roughly the square‑root rate as \\$n\\$ increases.\n\n# R5. Vary \\$n,d\\$\n\nSame setting as paper Section 7, varying \\$n,d\\$. Each entry is mean ± std over 15 seeds.\n\n**Table 4.** ECE varying \\$n\\$ with \\$d=1000\\$ fixed\n\n| Method                  | \\$n=50\\$        | \\$n=100\\$       | \\$n=200\\$       | \\$n=500\\$       | \\$n=1000\\$      | \\$n=2000\\$      | \\$n=4000\\$      | \\$n=8000\\$      | \\$n=10000\\$     |\n| ----------------------- | --------------- | --------------- | --------------- | --------------- | --------------- | --------------- | --------------- | --------------- | --------------- |\n| **Uncalibrated**        | 0.1276 ± 0.0066 | 0.1267 ± 0.0055 | 0.1208 ± 0.0038 | 0.1128 ± 0.0021 | 0.1135 ± 0.0024 | 0.1304 ± 0.0022 | 0.1584 ± 0.0035 | 0.1739 ± 0.0024 | 0.1777 ± 0.0029 |\n| **Angular**             | 0.0276 ± 0.0196 | 0.0101 ± 0.0055 | 0.0349 ± 0.0351 | 0.0136 ± 0.0066 | 0.0125 ± 0.0068 | 0.0101 ± 0.0037 | 0.0068 ± 0.0024 | 0.0072 ± 0.0017 | 0.0066 ± 0.0021 |\n| **Platt Scaling**       | 0.0349 ± 0.0224 | 0.0372 ± 0.0273 | 0.0342 ± 0.0170 | 0.0339 ± 0.0236 | 0.0322 ± 0.0146 | 0.0305 ± 0.0131 | 0.0274 ± 0.0124 | 0.0243 ± 0.0084 | 0.0294 ± 0.0181 |\n| **Isotonic Regression** | 0.0580 ± 0.0246 | 0.0614 ± 0.0338 | 0.0489 ± 0.0240 | 0.0500 ± 0.0177 | 0.0432 ± 0.0183 | 0.0451 ± 0.0133 | 0.0424 ± 0.0160 | 0.0406 ± 0.0129 | 0.0337 ± 0.0118 |\n\n**Table 5.** ECE varying \\$d\\$ with \\$n\\$ fixed at \\$n=1000\\$\n\n| Method                  | \\$d=50\\$        | \\$d=100\\$       | \\$d=200\\$       | \\$d=500\\$       | \\$d=1000\\$      | \\$d=2000\\$      | \\$d=4000\\$      | \\$d=8000\\$      | \\$d=10000\\$     |\n| ----------------------- | --------------- | --------------- | --------------- | --------------- | --------------- | --------------- | --------------- | --------------- | --------------- |\n| **Uncalibrated**        | 0.1894 ± 0.0072 | 0.1780 ± 0.0058 | 0.1652 ± 0.0077 | 0.1315 ± 0.0029 | 0.1135 ± 0.0024 | 0.1127 ± 0.0025 | 0.1154 ± 0.0024 | 0.1204 ± 0.0023 | 0.1219 ± 0.0031 |\n| **Angular**             | 0.0145 ± 0.0064 | 0.0101 ± 0.0031 | 0.0109 ± 0.0053 | 0.0093 ± 0.0065 | 0.0125 ± 0.0068 | 0.0088 ± 0.0030 | 0.0274 ± 0.0086 | 0.0303 ± 0.0370 | 0.1105 ± 0.0425 |\n| **Platt Scaling**       | 0.0327 ± 0.0158 | 0.0254 ± 0.0152 | 0.0287 ± 0.0162 | 0.0284 ± 0.0127 | 0.0322 ± 0.0146 | 0.0335 ± 0.0144 | 0.0444 ± 0.0132 | 0.0315 ± 0.0147 | 0.0364 ± 0.0154 |\n| **Isotonic Regression** | 0.0487 ± 0.0156 | 0.0403 ± 0.0185 | 0.0483 ± 0.0163 | 0.0447 ± 0.0162 | 0.0432 ± 0.0183 | 0.0494 ± 0.0208 | 0.0561 ± 0.0202 | 0.0521 ± 0.0204 | 0.0612 ± 0.0210 |\n\n## R6. Inaccurate overlap estimation\n\nWe model inaccurate estimation of \\$a=\\langle w^\\star,\\hat w\\rangle\\$ as\n$\\hat a=a+\\varsigma\\,\\epsilon,\\qquad \\epsilon\\sim\\mathcal N(0,1).$\n\n**Table 6.** ECE versus estimation error\n\n| Estimation Error | 0               | 0.05            | 0.10            | 0.15            | 0.20            | 0.25            | 0.30            |\n| ---------------- | --------------- | --------------- | --------------- | --------------- | --------------- | --------------- | --------------- |\n| **ECE**          | 0.0073 ± 0.0000 | 0.0211 ± 0.0123 | 0.0411 ± 0.0263 | 0.0614 ± 0.0405 | 0.0683 ± 0.0412 | 0.0813 ± 0.0572 | 0.0974 ± 0.0792 |\n\nWe fit a power law:\n$\\mathrm{ECE}\\approx 0.246\\,\\varsigma^{0.779}.$\n\n## R7. Effect of inaccurate link function estimation\n\nWe model the estimated link as\n$\\hat\\sigma(u)=(1+\\varsigma\\,\\epsilon)\\,\\sigma(u),\\qquad \\epsilon\\sim\\mathcal N(0,1).$\n\nEach \\$\\varsigma\\$ uses 15 repetitions to report mean ± std ECE for angular calibration; other settings follow paper Section 7.\n\nUsing the data below, we fit:\n$\\mathrm{ECE}\\approx 0.183\\,\\varsigma^{0.981},$\nsuggesting near‑linear growth of ECE with link‑estimation error. At \\$\\varsigma=0.8\\$, mean ECE is already comparable to the uncalibrated predictor (ECE = 0.1133).\n\n**Table 7.** ECE vs. estimation noise \\$\\varsigma\\$\n\n| \\$\\varsigma\\$ | 0            | 0.0707          | 0.1000          | 0.1414          | 0.2000          | 0.2828          | 0.4000          | 0.5657          | 0.8000          |\n| ------------- | ------------ | --------------- | --------------- | --------------- | --------------- | --------------- | --------------- | --------------- | --------------- |\n| **ECE**       | 0.0077 ± 0.0 | 0.0081 ± 0.0008 | 0.0108 ± 0.0009 | 0.0182 ± 0.0010 | 0.0321 ± 0.0011 | 0.0560 ± 0.0013 | 0.0860 ± 0.0016 | 0.1164 ± 0.0018 | 0.1358 ± 0.0024 |\n\n# Reference\n\n\\[1] Bellec, 2022. *Observable adjustments in single‑index models*. arXiv:2204.06990.\n\n\\[2] Pavlovic. *Understanding Model Calibration ... expected calibration error (ECE)*. arXiv:2501.19047v2"}, {"review_id": "ElKd2yD9A8", "reviewer": "Reviewer_thu8", "summary": "This paper addresses the problem of calibration in binary classification, focusing on classifiers that are linear with respect to input features. It investigates the question: What is the best calibrated predictor achievable via a post-hoc parametric calibration function that maps the original prediction to a calibrated probability? To answer this, the paper introduces the notion of angular calibration, where it can be shown that calibration with this notion will achieve both calibration and Bregman-optimality simultaneously. Theoretical analysis and simulations further show that, given sufficient data, Platt scaling converges to the angular calibration solution.", "questions": "Questions:\n1. Extension to Multiclass Setting\nAre there any significant bottlenecks or theoretical challenges that make it difficult to extend the proposed method to the multiclass setting? Do you think algorithm for computing angular predictor will change significantly or very difficult to derive in this setting? If so, it would be valuable to include a discussion of these challenges in the paper.\n2. Calibration Metrics\nWhile reliability diagrams are useful, they can be difficult to interpret when curves overlap. It would be helpful to also report a summary metric such as Expected Calibration Error (ECE). This could be included in the figure caption or legend to save space, and would make it easier to compare methods quantitatively.\n3. Dimensionality of Simulated Data\nI may have missed it, but it would be helpful to clarify the dimensionality of the simulated data used in the experiments. Given the paper's focus on high-dimensional settings, this is an important detail.\n4. Effect of Varying Dimensions\nIt could be insightful to explore how calibration performance (e.g., using ECE) changes with different feature dimensions—such as low, moderate, and very high dimensions. This would provide a deeper understanding of how the proposed method behaves under varying regimes.\n5. Use of Benchmark Datasets\nThe assumption of Gaussian features is indeed quite restrictive, and I appreciate that this limitation is acknowledged in the paper. I also found Figures 3 and 4 helpful in demonstrating that the proposed method performs well even under non-Gaussian settings. That said, I believe the use of a standard benchmark dataset such as those from the UCI repository could provide additional insights and help better motivate the practical utility of the proposed method. This would also make it easier to compare with other methods and assess generalizability in more realistic scenarios. What are your thoughts on this?\n\nSuggestion:\nI suggest including a summary of the proposed algorithm in a LaTeX algorithm environment. This would help readers quickly grasp the procedure for computing the angular predictor and enhance reproducibility.", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 4, "strengths_and_weaknesses": "Strengths\n1. The paper is well-written and easy to follow.\n2. The theoretical results are clearly presented and, to the best of my knowledge, novel and innovative.\n3. The proposed algorithm is directly derived from the theoretical insights and is both simple and effective.\n\nWeaknesses\n1. The theoretical analysis is limited to binary classification.\n2. The assumption of a linear predictor restricts the applicability of the results to modern machine learning models, which often rely on more expressive architectures.\n3. The empirical evaluation is limited to toy datasets, raising concerns about practical relevance and generalizability.\n\nOverall Evaluation\nI thoroughly enjoyed reading this paper and found the theoretical contributions fascinating. However, the overall significance may be somewhat limited due to the restrictive assumptions and lack of empirical validation on real-world datasets. My current assessment, prior to the rebuttal, is borderline accept.", "was_revised": true, "final_justification": "After reading the rebuttal of the authors, I found that the reply was satisfactory. My all three concerns were addressed, which are the concern of the difficulty of extension to multiclass, extension to non-linear models, and the evaluation beyond a synthetic dataset. I increased the score to 5.", "rebuttal": "We are grateful for the reviewer’s thoughtful review, and ask that the reviewer consider raising their score if they found our response satisfactory. \n\n## R1. Weakness\n\n1. Theorems 1 and 8 naturally extend to **multi-index models**, including multi-class classification \\[1]; see **Theorem R1**.\n2. Our setting is the simplest analytically tractable regime for studying calibration under overparametrization. Even here, no existing method is provably calibrated. Additionally:\n\n   * (i) Our theory extends to multi-index models, which includes multi-class classification and covers more complex structures such as two-layer neural nets (with frozen outer layer), additive, and interaction index models (see **Section R3**).\n   * (ii) While derived for linear predictors, angular calibration can be applied to final-layer logits of deep nets—a standard practice. See **Section R4** for experiments using angular calibration (as well as isotonic regression and Platt scaling) on logits from **ResNet34**, **MiniLM**, and **ChemBERTa** over **CIFAR10**, **20newsgroups**, and **Tox21**.\n3. See **Sections R4** for additional experiments (UCI datasets: **Communities & Crime**, **Splice-junction**, **Madelon**, **ISOLET**) .\n\n## R2. Questions\n\n1. Yes, angular calibration extends to multi-class via the multi-index framework; see **Section R3**.\n2. Thank you for the suggestion. We have implemented this and will include in camera ready version. \n3. Section 7 specifies \\$n=1000\\$, \\$d=2000\\$. Additional experiments varying \\$n, d\\$ are in **Section R5**.\n4. See **Tables 3–4** in **Section R5**, comparing angular, Platt, and isotonic calibration across \\$n, d\\$. Angular calibration performs best when \\$d/n \\in \\[0.05, 8]\\$ and deteriorates outside this range.\n5. See **Section R4** for UCI experiments. While estimator (11) for \\$\\langle w^\\star, \\hat w \\rangle\\$ is unstable on real data, angular calibration remains robust when this angle is known accurately.\n\nWe will include the proposed algorithm in a LaTeX algorithm environment in camera-ready version. \n\n## R3. Multi-index models\n\nLet \\$x\\_{\\mathrm{new}}\\sim\\mathcal N(0,\\Sigma)\\$. Fix \\$K\\ge2\\$ and let \\$W\\_\\star=\\[w\\_{\\star 1},\\ldots,w\\_{\\star K}]\\in\\mathbb R^{d\\times K}\\$. Define the true logits and model output:\n\n$$\nG:=W_\\star^\\top x_{\\mathrm{new}}\\in\\mathbb R^K,\\quad \\pi(x_{\\mathrm{new}})=g(G)\n$$\n\nwhere \\$g\\$ is a generalized link (vector- or scalar-valued). This setup covers:\n\n* Two-layer neural nets with frozen outer layer: \\$g(u)=\\sum\\_k a\\_k \\sigma(u\\_k)\\$\n* Multi-class softmax: \\$g(u)=\\mathrm{softmax}(u)\\$\n* Additive model: \\$g(u)=\\sum\\_k f\\_k(u\\_k)\\$\n* Interaction model: \\$g(u)=\\sum\\_k f\\_k(u\\_k)+\\sum\\_{k<\\ell} h\\_{k\\ell}(u\\_k,u\\_\\ell)\\$\n\n**(Theorem R1. Angular calibration for multi-index models).**  Let $\\widehat W=[\\widehat w_1,\\ldots,\\widehat w_K]$ be any estimator. Set\n$$\nD:=\\operatorname{diag}(\\rVert \\widehat w_1 \\rVert_\\Sigma, \\ldots, \\rVert \\widehat w_K \\rVert_\\Sigma), \\quad\nS := D^{-1}\\widehat W^\\top x_{\\mathrm{new}} \\in \\mathbb{R}^K.\n$$\nThen we have\n$$\n\\operatorname{Cov}(G)=W_\\star^\\top\\Sigma W_\\star,\\quad\nR:=\\operatorname{Cov}(S)=D^{-1}\\widehat W^\\top\\Sigma \\widehat W D^{-1},\\quad\nC:=\\operatorname{Cov}(G,S)=W_\\star^\\top\\Sigma \\widehat W D^{-1}\n$$\nwhere \n\n$$R_{k\\ell}=\\frac{\\langle \\widehat w_k,\\widehat{w}_\\ell\\rangle _{\\Sigma}}{\\lVert\\widehat{w} _{\\ell}\\rVert _{\\Sigma} \\lVert\\widehat{w}_k \\rVert _{\\Sigma}}$$\n\n$$\nC_{k\\ell}=\\frac{\\langle w_{\\star k},\\widehat{w}_\\ell \\rangle _\\Sigma}{\\lVert\\widehat{w} _{\\ell}\\rVert _{\\Sigma} \\lVert w _{\\star k} \\rVert _{\\Sigma}}\n$$\n\n\nThen,\n\n$$\n\\ M_\\star:=C R^{-1},\\qquad\n\\Sigma_\\star:=\\operatorname{Cov}(G)-C R^{-1}C^\\top\\ \n$$\n\nsatisfy $G\\mid S\\sim\\mathcal N(M_\\star S,\\Sigma_\\star)$. For any factor $L_\\star$ with $L_\\star L_\\star^\\top=\\Sigma_\\star$ and any $Z\\sim\\mathcal N(0,I_K)$ independent of everything, define the multi-index angular predictor\n\n$$\n\\ \\widehat f_{\\mathrm{ang}}\\big(\\widehat W^\\top x_{\\mathrm{new}}\\big)\n:=\\mathbb E_{Z}\\left[g\\big(M_\\star S+L_\\star Z\\big)\\right]. \\ \n$$\n\nThen for any $p\\in\\Delta^{K-1}$ and any $d,n\\in\\mathbb N_+$,\n\n$$\np-\\mathbb E\\left[\\pi(x_{\\mathrm{new}})\\ \\middle|\\ \\widehat f_{\\mathrm{ang}}(\\widehat W^\\top x_{\\mathrm{new}})=p\\right]=0.\n$$\n\nTo apply the above, we must estimate \\$\\langle \\widehat w_k,\\widehat w_\\ell\\rangle _\\Sigma\\$, as in the single-index case. Though no estimator is given in literature as far as we know, recent theory for multi-index models \\[1] suggests that analogues of the single-index angle estimators \\[2] are feasible.\n\n## R4. Semi-real experiments\n\nFor Tables 1–2, we subsample each dataset into training (\\$n\\_{\\text{train}}\\$), covariance estimation (\\$n\\_{\\text{cov}}\\$), and test (\\$n\\_{\\text{test}}\\$) sets. We set \\$n\\_{\\text{cov}} \\gg n\\_{\\text{train}}\\$ to reflect abundant unlabeled data, and report ECE for \\$n\\_{\\text{ho}} = 100\\$ and \\$500\\$.\n\nWe found that Eq. (11)’s plug-in estimator for \\$\\langle w^\\star, \\hat{w} \\rangle\\$ is unstable on real datasets. This is a known issue for estimators like (11) that are based on Wigner-type random-matrix-theoretic assumptions. Modifying these estimator are an active research area \\[3]. To isolate calibration effects, we simulate \\$w^\\star\\$ and labels, using the *true* angle.\n\nTo extend calibration to nonlinear models, we apply it to the logits of pretrained models:\n\n* **ResNet-34 → CIFAR-10**: Use 512-dim penultimate layer of ResNet-34 (ImageNet pretrained)\n* **MiniLM → 20 Newsgroups**: Mean-pool 384-dim “all‑MiniLM‑L6‑v2” embeddings\n* **ChemBERTa → Tox21**: Mean-pool 768-dim hidden states from “seyonec/ChemBERTa‑zinc‑base‑v1”\n\n\\$n\\_{\\text{cov}} = 30000\\$, \\$3000\\$, \\$3000\\$; \\$n\\_{\\text{test}} = 10000\\$, \\$1000\\$, \\$500\\$\n\n*Table 1.* ECE for pretrained DL models:\n\n| Model–Dataset       | \\$n \\times d\\$ | Uncal  | Angular | Platt 100 | Iso 100 | Platt 500 | Iso 500 |\n| ------------------- | -------------- | ------ | ------- | --------- | ------- | --------- | ------- |\n| ResNet34–CIFAR10    | 300×512        | 0.1236 | 0.0199  | 0.0561    | 0.0484  | 0.0259    | 0.0298  |\n| MiniLM–20newsgroups | 800×384        | 0.1392 | 0.0249  | 0.0931    | 0.1107  | 0.0679    | 0.0813  |\n| ChemBERTa–Tox21     | 800×768        | 0.1389 | 0.0132  | 0.0236    | 0.0497  | 0.0175    | 0.0293  |\n\nFor UCI datasets:\n\n* Communities & Crime (200×100), Splice-junction (100×180), Madelon (300×500), ISOLET (500×617)\n* \\$n\\_{\\text{cov}} = 700\\$, \\$2000\\$, \\$900\\$, \\$5750\\$; \\$n\\_{\\text{test}} = 593\\$, \\$900\\$, \\$900\\$, \\$1000\\$\n\n*Table 2.* ECE on UCI datasets:\n\n| UCI Dataset              | \\$n \\times d\\$ | Uncal  | Angular | Platt 100 | Iso 100 | Platt 500 | Iso 500 |\n| ------------------------ | -------------- | ------ | ------- | --------- | ------- | --------- | ------- |\n| Communities and Crime    | 200×100        | 0.1700 | 0.0296  | 0.0620    | 0.0661  | 0.0262    | 0.0436  |\n| Splice-junction Gene Seq | 100×180        | 0.1262 | 0.0208  | 0.0568    | 0.1041  | 0.0556    | 0.0879  |\n| Madelon                  | 300×500        | 0.1473 | 0.0267  | 0.0721    | 0.0719  | 0.0299    | 0.0624  |\n| ISOLET                   | 500×617        | 0.1256 | 0.0221  | 0.0438    | 0.0761  | 0.0219    | 0.0137  |\n\n## R5. Vary $n, d$\nThe setting is the same as in paper Section 7, except we vary $n,d$ below. We repeat experiments with 15 seeds to obtain mean ± std.  \n\n**Table 3.** ECE varying $n$ with $d=1000$ fixed\n| Method                  | \\$n=50\\$        | \\$n=100\\$       | \\$n=200\\$       | \\$n=500\\$       | \\$n=1000\\$      | \\$n=2000\\$      | \\$n=4000\\$      | \\$n=8000\\$      | \\$n=10000\\$     |\n| ----------------------- | --------------- | --------------- | --------------- | --------------- | --------------- | --------------- | --------------- | --------------- | --------------- |\n| **Uncalibrated**        | 0.1276 ± 0.0066 | 0.1267 ± 0.0055 | 0.1208 ± 0.0038 | 0.1128 ± 0.0021 | 0.1135 ± 0.0024 | 0.1304 ± 0.0022 | 0.1584 ± 0.0035 | 0.1739 ± 0.0024 | 0.1777 ± 0.0029 |\n| **Angular**             | 0.0276 ± 0.0196 | 0.0101 ± 0.0055 | 0.0349 ± 0.0351 | 0.0136 ± 0.0066 | 0.0125 ± 0.0068 | 0.0101 ± 0.0037 | 0.0068 ± 0.0024 | 0.0072 ± 0.0017 | 0.0066 ± 0.0021 |\n| **Platt Scaling**       | 0.0349 ± 0.0224 | 0.0372 ± 0.0273 | 0.0342 ± 0.0170 | 0.0339 ± 0.0236 | 0.0322 ± 0.0146 | 0.0305 ± 0.0131 | 0.0274 ± 0.0124 | 0.0243 ± 0.0084 | 0.0294 ± 0.0181 |\n| **Isotonic Regression** | 0.0580 ± 0.0246 | 0.0614 ± 0.0338 | 0.0489 ± 0.0240 | 0.0500 ± 0.0177 | 0.0432 ± 0.0183 | 0.0451 ± 0.0133 | 0.0424 ± 0.0160 | 0.0406 ± 0.0129 | 0.0337 ± 0.0118 |\n\n\n**Table 4.** ECE varying $d$ with $n$ fixed at $n=1000$ \n| Method                  | \\$d=50\\$        | \\$d=100\\$       | \\$d=200\\$       | \\$d=500\\$       | \\$d=1000\\$      | \\$d=2000\\$      | \\$d=4000\\$      | \\$d=8000\\$      | \\$d=10000\\$     |\n| ----------------------- | --------------- | --------------- | --------------- | --------------- | --------------- | --------------- | --------------- | --------------- | --------------- |\n| **Uncalibrated**        | 0.1894 ± 0.0072 | 0.1780 ± 0.0058 | 0.1652 ± 0.0077 | 0.1315 ± 0.0029 | 0.1135 ± 0.0024 | 0.1127 ± 0.0025 | 0.1154 ± 0.0024 | 0.1204 ± 0.0023 | 0.1219 ± 0.0031 |\n| **Angular**             | 0.0145 ± 0.0064 | 0.0101 ± 0.0031 | 0.0109 ± 0.0053 | 0.0093 ± 0.0065 | 0.0125 ± 0.0068 | 0.0088 ± 0.0030 | 0.0274 ± 0.0086 | 0.0303 ± 0.0370 | 0.1105 ± 0.0425 |\n| **Platt Scaling**       | 0.0327 ± 0.0158 | 0.0254 ± 0.0152 | 0.0287 ± 0.0162 | 0.0284 ± 0.0127 | 0.0322 ± 0.0146 | 0.0335 ± 0.0144 | 0.0444 ± 0.0132 | 0.0315 ± 0.0147 | 0.0364 ± 0.0154 |\n| **Isotonic Regression** | 0.0487 ± 0.0156 | 0.0403 ± 0.0185 | 0.0483 ± 0.0163 | 0.0447 ± 0.0162 | 0.0432 ± 0.0183 | 0.0494 ± 0.0208 | 0.0561 ± 0.0202 | 0.0521 ± 0.0204 | 0.0612 ± 0.0210 |\n\n\n## References\n\\[1] Troiani et al., 2024. *Fundamental computational...multi-index models*. arXiv:2405.15480\n\n\\[2] Bellec, 2022. *Observable adjustments in single-index models*. arXiv:2204.06990\n\n\\[3] Li & Sur, 2023. *Spectrum-aware debiasing*. arXiv:2309.07810"}, {"review_id": "hDQf6Sp54A", "reviewer": "Reviewer_jieG", "summary": "This works concerns the problem of estimating class probability in binary classification in the regime where both # of features and sample diverge to infinity.\nThe focus is on an angle-with-the-ground-truth-based estimator (or as the authors call it angular estimator simply).", "questions": "question:\n- in theorem 3.2, is monotonicity of sigma not needed? it is quite surprising to me that continuity is all that is needed.\n- for the convex penalty g (from line 82), can it be arbitrary as long as it is convex?\n- what is the connection to \"The Isotron Algorithm: High-Dimensional Isotonic Regression\" by Kalai and Sastry? Since isotonic regression also targets the probability calibration, might be good to talk about similarity/differences of your work and Kalai and Sastry.\n\nsuggestion:\n- line 60 maybe qualify your usage of the word \"interpolating\" since its usage is a bit overloaded in the literature. perhaps a footnote or a parenthetical remark that interpolation in this context means a convex combination\n- it will be helpful to have an algorithm-style write up of the angular estimator. right now, it takes some work to chase it down. would be good to make it a bit more explicitly spelled out.", "limitations": "Yes", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 4, "originality": 4, "strengths_and_weaknesses": "Strength:\nThe result is quite beautiful and gives a surprising (but makes sense in hindsight) intuition about probability calibration on Line 128 - 129: \"when the estimator is poorly aligned, the estimation must be tempered with noise\"  (quote is this reviewer's paraphrasing).\nThe authors provide theory for platt scaling by showing (under certain parametric conditions) that platt scaling can be seen as an instance of the angle-based estimator.\n\nWeakness:\nThe theoretical results are for the gaussian features setting, which is not as broad as one would hope. But the gaussianity assumption is common in the random matrix-theoretic literature, and the phenomenon discovered often experimentally can be observed even without the Gaussianity assumption.\nLiterature review is pretty thorough, but I feel that \"The Isotron Algorithm: High-Dimensional Isotonic Regression\" by Kalai and Sastry is quite related.", "was_revised": true, "final_justification": "The authors provided a thorough rebuttal and additional discussion for the multiclass case which is exemplary.", "rebuttal": "We are grateful for the reviewer’s favorable review and encouraging comments.\n\nTo answer the reviewer's questions:\n\n1. Monotonicity of the link is not needed for Theorem 3.2. The result follows from the general, exact‑calibration statement in Theorem A.1 together with consistency of the angle estimator, using the tower property and the conditional law of jointly Gaussian variables. Continuity of the link is required only to pass the limit via the continuous‑mapping argument when the angle estimator in (11) converges.\n\n2. Not quite. The theory in \\[1] ensures consistency of the angle estimator only when (i) $g$ is strongly convex (Theorem 4.4 of \\[1]), or (ii) $g=0$ and $\\hat w$ exists (Theorem 5.2 of \\[1]). Outside these cases, the guarantee does not apply.\n\n3. We thank the reviewer for the reference and will add the citation. Kalai–Sastry’s Isotron alternates perceptron‑style updates with isotonic regression to learn a single‑index model; like our work, it exploits the single‑index structure by operating on the one‑dimensional score $w^\\top x$ and fitting a monotone map on that score. However, while its analysis yields dimension‑free bounds on cumulative squared error and PAC‑style learnability for Lipschitz SIMs, it provides neither population reliability guarantees nor optimality results in the proportional high‑dimensional regime ($n,d\\to\\infty$ with $n/d$ fixed). By contrast, we study post‑hoc calibration of a fixed linear predictor in this regime via angular calibration, proving asymptotic calibration and Bregman‑optimality, and showing that Platt scaling converges to the same limit under suitable conditions.\n\nWe thank the reviewer for suggestions on qualifying the \"interpolation\" and including an algorithm write-up. We will include them in later version of the draft. \n\n\\[1] Bellec, 2022. Observable adjustments in single-index models. arXiv:2204.06990"}, {"review_id": "4TN7CWnyFH", "reviewer": "Reviewer_Zw5n", "summary": "The paper studies calibration in the setting that the target labels are sampled from a single-index link-regression model under gaussian inputs. This work gives a clean method for transforming the predictions from a linear model such that the predictions converge to the optimal transformation with respect to Bregman divergence to the underlying distribution of the labels.  They demonstrate theoretically and empirically that Platt scaling converges to their angular calibration method with infinite holdout samples. The authors provide experimental validation for their algorithm and theoretical results.", "questions": "1.\tI am not sure I understand the optimality/consistency result. As the samples go to infinity, wouldn’t it be the case that any estimation method that has small error to the true predictor be perfectly calibrated given access to the link function as assumed for your angular predictor? Or is the consistency result saying that even if you have large error to the true w*, you will achieve optimal calibration provided n,d are infinite?\n2.\tYour calibration method will also requires data to estimate the angular distance of \\hat{w} to w*. Is the advantage then that you don’t require holdout data to compute this distance, whereas Platt scaling does?\n3.\tHow are calibration guarantees affected when the inputs to your calibration method is not exactly linear but instead the outputs of some nonlinear model? I am imagining transforming predictions with logistic regression as in Platt Calibration.", "limitations": "Yes.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "If my questions are addressed adequately, I will increase my score.\n\nStrengths:\\\n-\tThe problem tackled here is fundamental.\\\n-\tTheir approach is quite clean.\\\n-\tTheir approach justifies Platt Scaling which is a very common method for calibrating predictors of all types.\n\nWeaknesses:\\\n-\tThe calibration method proposed requires knowledge of the link function, which seems like a strong assumption, especially when you are applying this predictions to calibrate the outputs of some nonlinear predictor. \\\n-\tTheir method requires that the true function is linear with some link function that angular calibration method has access to. Their result might be made much stronger if the authors could address if and how their results extend to the nonlinear case.\\\n-\tExperiments are limited to the realizable case where the target is indeed some logistic regression function, and not in the standard calibration setting where your inputs are themselves predictions from some nonlinear method.", "was_revised": true, "final_justification": "As stated above, the paper introduces a clean optimal calibration method (assuming access to the known linear classifier). This method justifies Platt scaling which is a widely used method for calibration. I increased my score as they demonstrated experiments for their method on more realistic settings (logits of neural networks) and theoretical results for multi-index models.", "rebuttal": "We are grateful for the reviewer’s thoughtful review, and ask that the reviewer consider raising their score if they found our response satisfactory. \n\n## R1. Weakness \n\n1. We agree with the reviewer’s comment regarding the limitations of angular calibration—it indeed assumes knowledge of the true link function. However, recent developments in the literature \\[4] suggest that it is possible to consistently estimate the true link function in the proportional regime. One could, in principle, apply such a method to first estimate the link function and then use it in angular calibration. We also present a simulation in **Section R5**, showing that the expected calibration error of angular calibration increases approximately linearly with the estimation error of the true link function.\n\n2. Our setting is the simplest analytically tractable regime for studying calibration in high dimensions. Even here, no existing method is provably calibrated. Additionally, our theory extends to multi-index models, covering more complex structures such as two-layer neural nets (with frozen outer layer), additive, and interaction models. See **Section R3** below.\n \n3. While derived for linear predictors, angular calibration can be applied to final-layer logits of deep nets—a standard practice. See **Section R4** for experiments using angular calibration (as well as isotonic regression and Platt scaling) on logits from **ResNet34**, **MiniLM**, and **ChemBERTa** over **CIFAR10**, **20newsgroups**, and **Tox21**. There, we also include additional experiments on UCI benchmark datasets: **Communities & Crime**, **Splice-junction**, **Madelon**, **ISOLET**.\n\n\n## R2. Questions\n\n1. The latter is correct. In the proportional regime, the estimated signal $\\hat{w}$ does not converge to the true $w^\\star$ as both $n$ and $d$ grow proportionally. In fact, under our model assumptions, $\\langle \\hat{w}, w^\\star \\rangle / (\\|\\hat{w}\\|\\|w^\\star \\|)$ converges to a constant less than 1. Nevertheless, angular calibration achieves asymptotic optimality in the sense of minimizing among all calibrated predictor of the form $F(x^\\top_{new} \\hat{w})$ expected Bregman divergence\n\n2. Yes, exactly. Angular calibration does not require a validation set.\n\n3. Yes, one may apply angular calibration to the last layer logits output of modern deep learning models. One would essentially treat these logits as input data. See **Section R4** for our results. While we found that estimator (11) for $\\langle w^\\star, \\hat w \\rangle$ is unstable on real data, angular calibration remains robust when this angle is known accurately.\n\n## R3 Multi-index model\n\n**(Theorem R1. Angular calibration for multi-index models).**  Let $\\widehat W=[\\widehat w_1,\\ldots,\\widehat w_K]\\in\\mathbb R^{d\\times K}$ be any estimator. Set\n$$\nD:=\\operatorname{diag}(\\rVert \\widehat w_1 \\rVert_\\Sigma, \\ldots, \\rVert \\widehat w_K \\rVert_\\Sigma), \\quad\nS := D^{-1}\\widehat W^\\top x_{\\mathrm{new}} \\in \\mathbb{R}^K.\n$$\nThen we have $K\\times K$ covariance blocks\n$$\n\\operatorname{Cov}(G)=W_\\star^\\top\\Sigma W_\\star,\\quad\nR:=\\operatorname{Cov}(S)=D^{-1}\\widehat W^\\top\\Sigma \\widehat W D^{-1},\\quad\nC:=\\operatorname{Cov}(G,S)=W_\\star^\\top\\Sigma \\widehat W D^{-1}\n$$\nwhere \n\n$$R_{k\\ell}=\\frac{\\langle \\widehat w_k,\\widehat{w}_\\ell\\rangle _{\\Sigma}}{\\lVert\\widehat{w} _{\\ell}\\rVert _{\\Sigma} \\lVert\\widehat{w}_k \\rVert _{\\Sigma}}$$\n\n$$\nC_{k\\ell}=\\frac{\\langle w_{\\star k},\\widehat{w}_\\ell \\rangle _\\Sigma}{\\lVert\\widehat{w} _{\\ell}\\rVert _{\\Sigma} \\lVert w _{\\star k} \\rVert _{\\Sigma}}\n$$\n\n\nThen,\n\n$$\n\\ M_\\star:=C R^{-1},\\qquad\n\\Sigma_\\star:=\\operatorname{Cov}(G)-C R^{-1}C^\\top\\ \n$$\n\nsatisfy $G\\mid S\\sim\\mathcal N(M_\\star S,\\Sigma_\\star)$. For any factor $L_\\star$ with $L_\\star L_\\star^\\top=\\Sigma_\\star$ and any $Z\\sim\\mathcal N(0,I_K)$ independent of everything, define the multi-index angular predictor\n\n$$\n\\ \\widehat f_{\\mathrm{ang}}\\big(\\widehat W^\\top x_{\\mathrm{new}}\\big)\n:=\\mathbb E_{Z}\\left[g\\big(M_\\star S+L_\\star Z\\big)\\right]. \\ \n$$\n\nThen for any $p\\in\\Delta^{K-1}$ and any $d,n\\in\\mathbb N_+$,\n\n$$\np-\\mathbb E\\left[\\pi(x_{\\mathrm{new}})\\ \\middle|\\ \\widehat f_{\\mathrm{ang}}(\\widehat W^\\top x_{\\mathrm{new}})=p\\right]=0.\n$$\n\nTo apply the above, we must estimate \\$\\langle \\widehat w_k,\\widehat w_\\ell\\rangle _\\Sigma\\$, \\$\\forall k,\\ell\\in\\[K]\\$, as in the single-index case. Though no estimator is given in literature as far as we know, recent theory for multi-index models \\[1] suggests that analogues of the single-index angle estimators \\[2] are feasible.\n\n## R4. Semi-real experiments\n\nFor Tables 1–2, we subsample each dataset into training (\\$n\\_{\\text{train}}\\$), covariance estimation (\\$n\\_{\\text{cov}}\\$), and test (\\$n\\_{\\text{test}}\\$) sets. We set \\$n\\_{\\text{cov}} \\gg n\\_{\\text{train}}\\$ to reflect abundant unlabeled data, and report ECE for \\$n\\_{\\text{ho}} = 100\\$ and \\$500\\$. We report calibration quality using Expected Calibration Error (ECE), which measures the average gap between predicted confidence and actual accuracy across confidence levels, quantifying how well a model's probabilities are calibrated \\[5].\n\nWe found that Eq. (11)’s plug-in estimator for \\$\\langle w^\\star, \\hat{w} \\rangle\\$ is unstable on real datasets. This is a known issue for estimators like (11) that are based on Wigner-type random-matrix-theoretic assumptions. Modifying these estimator are an active research area \\[3]. To isolate calibration effects, we simulate \\$w^\\star\\$ and labels, using the *true* angle.\n\nTo extend calibration to nonlinear models, we apply it to the logits of pretrained models:\n\n* **ResNet-34 → CIFAR-10**: Use 512-dim penultimate layer of ResNet-34 (ImageNet pretrained)\n* **MiniLM → 20 Newsgroups**: Mean-pool 384-dim “all‑MiniLM‑L6‑v2” embeddings\n* **ChemBERTa → Tox21**: Mean-pool 768-dim hidden states from “seyonec/ChemBERTa‑zinc‑base‑v1”\n\n\\$n\\_{\\text{cov}} = 30000\\$, \\$3000\\$, \\$3000\\$; \\$n\\_{\\text{test}} = 10000\\$, \\$1000\\$, \\$500\\$\n\n*Table 1.* ECE for pretrained DL models:\n\n| Model–Dataset       | \\$n \\times d\\$ | Uncal  | Angular | Platt 100 | Iso 100 | Platt 500 | Iso 500 |\n| ------------------- | -------------- | ------ | ------- | --------- | ------- | --------- | ------- |\n| ResNet34–CIFAR10    | 300×512        | 0.1236 | 0.0199  | 0.0561    | 0.0484  | 0.0259    | 0.0298  |\n| MiniLM–20newsgroups | 800×384        | 0.1392 | 0.0249  | 0.0931    | 0.1107  | 0.0679    | 0.0813  |\n| ChemBERTa–Tox21     | 800×768        | 0.1389 | 0.0132  | 0.0236    | 0.0497  | 0.0175    | 0.0293  |\n\nFor UCI datasets:\n\n* Communities & Crime (200×100), Splice-junction (100×180), Madelon (300×500), ISOLET (500×617)\n* \\$n\\_{\\text{cov}} = 700\\$, \\$2000\\$, \\$900\\$, \\$5750\\$; \\$n\\_{\\text{test}} = 593\\$, \\$900\\$, \\$900\\$, \\$1000\\$\n\n*Table 2.* ECE on UCI datasets:\n\n| UCI Dataset              | \\$n \\times d\\$ | Uncal  | Angular | Platt 100 | Iso 100 | Platt 500 | Iso 500 |\n| ------------------------ | -------------- | ------ | ------- | --------- | ------- | --------- | ------- |\n| Communities and Crime    | 200×100        | 0.1700 | 0.0296  | 0.0620    | 0.0661  | 0.0262    | 0.0436  |\n| Splice-junction Gene Seq | 100×180        | 0.1262 | 0.0208  | 0.0568    | 0.1041  | 0.0556    | 0.0879  |\n| Madelon                  | 300×500        | 0.1473 | 0.0267  | 0.0721    | 0.0719  | 0.0299    | 0.0624  |\n| ISOLET                   | 500×617        | 0.1256 | 0.0221  | 0.0438    | 0.0761  | 0.0219    | 0.0137  |\n\n## R5. Effect of inaccurate link function estimation\n\nWe model the the estimated link function as follows: \n\n$$\\hat \\sigma (u)=(1+\\varsigma \\cdot \\epsilon)\\cdot \\sigma(u), \\qquad \\epsilon \\sim N(0,1)$$\n\nwhere $\\varsigma \\cdot \\epsilon$ models estimation noise. We run a sequence of experiments at different $\\varsigma$, each experiment repeated 15 times to produce mean ± std of Expected Calibration Error (ECE) of angular calibration. Otherwise, the setting is the same as Section 7 of the paper. \n\nUsing the data below, we may fit a power law relationship of how ECE relates to the noise size $\\varsigma$:\n\n$$\\mathrm{ECE} \\approx 0.183 \\cdot \\varsigma^{0.981}$$\n\nThis seems to suggest that the expected calibration error grows linearly with estimation error of the link function. \n\n\n**Table.** Expected Calibration Error (ECE) vs. Estimation Noise $\\varsigma$\n\n| Estimation noise $\\varsigma$ | 0        | 0.0707           | 0.1000           | 0.1414           | 0.2000           | 0.2828           | 0.4000           | 0.5657           | 0.8000           |\n|------------------------------|----------|------------------|------------------|------------------|------------------|------------------|------------------|------------------|------------------|\n| ECE                          | 0.0077 ± 0.0 | 0.0081 ± 0.0008 | 0.0108 ± 0.0009 | 0.0182 ± 0.0010 | 0.0321 ± 0.0011 | 0.0560 ± 0.0013 | 0.0860 ± 0.0016 | 0.1164 ± 0.0018 | 0.1358 ± 0.0024 |\n\n\n\n## References\n\\[1] Troiani et al., 2024. *Fundamental computational limits of weak learnability in high-dimensional multi-index models*. arXiv:2405.15480\n\n\\[2] Bellec, 2022. *Observable adjustments in single-index models*. arXiv:2204.06990\n\n\\[3] Li & Sur, 2023. *Spectrum-aware debiasing*. arXiv:2309.07810\n\n\\[4] Sawaya et al. *High-Dimensional Single-Index Models: Link Estimation and Marginal Inference*, arXiv:2404.17812\n\n\\[5] Pavlovic. *Understanding Model Calibration - A gentle introduction and visual exploration of calibration and the expected calibration error (ECE)* arXiv:2501.19047v2"}]

Author rebuttal:

Meta-review: The paper concerns the calibration of binary classifiers. The Authors study the problem in a high-dimensional regime, where both the number of samples and the number of features diverge at a comparable rate. They introduce a new angular calibration approach that not only achieves provable calibration but also uniquely minimizes any Bregman divergence to the true label distribution within a wide class of predictors. Interestingly, classical Platt scaling is shown to converge to angular calibration under certain conditions. The theoretical framework, however, is quite limited (e.g., restricted to linear models and Gaussian features), although the Authors provide a brief discussion and some experimental results extending beyond these constraints.

The Reviewers are very positive about the paper, emphasizing that it offers the first provable calibration procedure in the high-dimensional regime with optimality guarantees, provides a theoretical justification of Platt scaling, and presents mathematically rigorous and clearly stated results. They also point out weaknesses such as the limitations of the theoretical framework and the very constrained empirical studies. Nevertheless, all reviewers consistently agree that the paper is worth publishing.
Decision: Accept (spotlight)