Title: Optimal and Provable Calibration in High-Dimensional Binary Classification: Angular Calibration and Platt Scaling
Venue: NeurIPS 2025
Accepted: yes
Keywords: High dimensional statistics
Abstract: We study the fundamental problem of calibrating a linear binary classifier of the form \(\sigma(\hat{w}^\top x)\), where the feature vector \(x\) is Gaussian, \(\sigma\) is a link function, and \(\hat{w}\) is an estimator of the true linear weight $w^\star$. By interpolating with a noninformative \emph{chance classifier}, we construct a well-calibrated predictor whose interpolation weight depends on the angle \(\angle(\hat{w}, w_\star)\) between the estimator \(\hat{w}\) and the true linear weight \(w_\star\). We establish that this angular calibration approach is provably well-calibrated in a high-dimensional regime where the number of samples and features both diverge, at a comparable rate. The angle \(\angle(\hat{w}, w_\star)\)  can be consistently estimated. Furthermore, the resulting predictor is uniquely \emph{Bregman-optimal}, minimizing the Bregman divergence to the true label distribution within a suitable class of calibrated predictors.
Our work is the first to provide a calibration strategy that satisfies both calibration and optimality properties provably in high dimensions. Additionally, we identify conditions under which a classical Platt-scaling predictor converges to our Bregman-optimal calibrated solution. Thus, Platt-scaling also inherits these desirable properties provably in high dimensions.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper studies calibration in the setting that the target labels are sampled from a single-index link-regression model under gaussian inputs. This work gives a clean method for transforming the predictions from a linear model such that the predictions converge to the optimal transformation with respect to Bregman divergence to the underlying distribution of the labels.  They demonstrate theoretically and empirically that Platt scaling converges to their angular calibration method with infinite holdout samples. The authors provide experimental validation for their algorithm and theoretical results.\n\nSTRENGTHS AND WEAKNESSES:\nIf my questions are addressed adequately, I will increase my score.\n\nStrengths:\\\n-\tThe problem tackled here is fundamental.\\\n-\tTheir approach is quite clean.\\\n-\tTheir approach justifies Platt Scaling which is a very common method for calibrating predictors of all types.\n\nWeaknesses:\\\n-\tThe calibration method proposed requires knowledge of the link function, which seems like a strong assumption, especially when you are applying this predictions to calibrate the outputs of some nonlinear predictor. \\\n-\tTheir method requires that the true function is linear with some link function that angular calibration method has access to. Their result might be made much stronger if the authors could address if and how their results extend to the nonlinear case.\\\n-\tExperiments are limited to the realizable case where the target is indeed some logistic regression function, and not in the standard calibration setting where your inputs are themselves predictions from some nonlinear method.\n\nQUESTIONS:\n1.\tI am not sure I understand the optimality/consistency result. As the samples go to infinity, wouldn’t it be the case that any estimation method that has small error to the true predictor be perfectly calibrated given access to the link function as assumed for your angular predictor? Or is the consistency result saying that even if you have large error to the true w*, you will achieve optimal calibration provided n,d are infinite?\n2.\tYour calibration method will also requires data to estimate the angular distance of \\hat{w} to w*. Is the advantage then that you don’t require holdout data to compute this distance, whereas Platt scaling does?\n3.\tHow are calibration guarantees affected when the inputs to your calibration method is not exactly linear but instead the outputs of some nonlinear model? I am imagining transforming predictions with logistic regression as in Platt Calibration.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-21"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis works concerns the problem of estimating class probability in binary classification in the regime where both # of features and sample diverge to infinity.\nThe focus is on an angle-with-the-ground-truth-based estimator (or as the authors call it angular estimator simply).\n\nSTRENGTHS AND WEAKNESSES:\nStrength:\nThe result is quite beautiful and gives a surprising (but makes sense in hindsight) intuition about probability calibration on Line 128 - 129: \"when the estimator is poorly aligned, the estimation must be tempered with noise\"  (quote is this reviewer's paraphrasing).\nThe authors provide theory for platt scaling by showing (under certain parametric conditions) that platt scaling can be seen as an instance of the angle-based estimator.\n\nWeakness:\nThe theoretical results are for the gaussian features setting, which is not as broad as one would hope. But the gaussianity assumption is common in the random matrix-theoretic literature, and the phenomenon discovered often experimentally can be observed even without the Gaussianity assumption.\nLiterature review is pretty thorough, but I feel that \"The Isotron Algorithm: High-Dimensional Isotonic Regression\" by Kalai and Sastry is quite related.\n\nQUESTIONS:\nquestion:\n- in theorem 3.2, is monotonicity of sigma not needed? it is quite surprising to me that continuity is all that is needed.\n- for the convex penalty g (from line 82), can it be arbitrary as long as it is convex?\n- what is the connection to \"The Isotron Algorithm: High-Dimensional Isotonic Regression\" by Kalai and Sastry? Since isotonic regression also targets the probability calibration, might be good to talk about similarity/differences of your work and Kalai and Sastry.\n\nsuggestion:\n- line 60 maybe qualify your usage of the word \"interpolating\" since its usage is a bit overloaded in the literature. perhaps a footnote or a parenthetical remark that interpolation in this context means a convex combination\n- it will be helpful to have an algorithm-style write up of the angular estimator. right now, it takes some work to chase it down. would be good to make it a bit more explicitly spelled out.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 5, "CLARITY": 4, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-06-23"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses the problem of calibration in binary classification, focusing on classifiers that are linear with respect to input features. It investigates the question: What is the best calibrated predictor achievable via a post-hoc parametric calibration function that maps the original prediction to a calibrated probability? To answer this, the paper introduces the notion of angular calibration, where it can be shown that calibration with this notion will achieve both calibration and Bregman-optimality simultaneously. Theoretical analysis and simulations further show that, given sufficient data, Platt scaling converges to the angular calibration solution.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n1. The paper is well-written and easy to follow.\n2. The theoretical results are clearly presented and, to the best of my knowledge, novel and innovative.\n3. The proposed algorithm is directly derived from the theoretical insights and is both simple and effective.\n\nWeaknesses\n1. The theoretical analysis is limited to binary classification.\n2. The assumption of a linear predictor restricts the applicability of the results to modern machine learning models, which often rely on more expressive architectures.\n3. The empirical evaluation is limited to toy datasets, raising concerns about practical relevance and generalizability.\n\nOverall Evaluation\nI thoroughly enjoyed reading this paper and found the theoretical contributions fascinating. However, the overall significance may be somewhat limited due to the restrictive assumptions and lack of empirical validation on real-world datasets. My current assessment, prior to the rebuttal, is borderline accept.\n\nQUESTIONS:\nQuestions:\n1. Extension to Multiclass Setting\nAre there any significant bottlenecks or theoretical challenges that make it difficult to extend the proposed method to the multiclass setting? Do you think algorithm for computing angular predictor will change significantly or very difficult to derive in this setting? If so, it would be valuable to include a discussion of these challenges in the paper.\n2. Calibration Metrics\nWhile reliability diagrams are useful, they can be difficult to interpret when curves overlap. It would be helpful to also report a summary metric such as Expected Calibration Error (ECE). This could be included in the figure caption or legend to save space, and would make it easier to compare methods quantitatively.\n3. Dimensionality of Simulated Data\nI may have missed it, but it would be helpful to clarify the dimensionality of the simulated data used in the experiments. Given the paper's focus on high-dimensional settings, this is an important detail.\n4. Effect of Varying Dimensions\nIt could be insightful to explore how calibration performance (e.g., using ECE) changes with different feature dimensions—such as low, moderate, and very high dimensions. This would provide a deeper understanding of how the proposed method behaves under varying regimes.\n5. Use of Benchmark Datasets\nThe assumption of Gaussian features is indeed quite restrictive, and I appreciate that this limitation is acknowledged in the paper. I also found Figures 3 and 4 helpful in demonstrating that the proposed method performs well even under non-Gaussian settings. That said, I believe the use of a standard benchmark dataset such as those from the UCI repository could provide additional insights and help better motivate the practical utility of the proposed method. This would also make it easier to compare with other methods and assess generalizability in more realistic scenarios. What are your thoughts on this?\n\nSuggestion:\nI suggest including a summary of the proposed algorithm in a LaTeX algorithm environment. This would help readers quickly grasp the procedure for computing the angular predictor and enhance reproducibility.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses the fundamental problem of probabilistic calibration for binary classifiers in high-dimensional settings, where both the sample size $n$ and the feature dimension $d$ diverge at comparable rates. The authors propose a novel angular calibration method that achieves provable calibration by interpolating between the logit scores of a (given) linear predictor and an uninformative “chance” predictor (a purely random variable), with the interpolation weight determined by the angle between the estimated weight vector $\\hat{w}$ and the ground-truth weight $w_*$. A key contribution is the theoretical result that this predictor not only achieves calibration but also uniquely minimizes any Bregman divergence to the true label distribution, among a wide class of predictors (Section 4). Further, the authors show that classical Platt scaling converges to the angular calibration under certain conditions, thus inheriting both calibration and optimality guarantees in high dimensions. The paper also presents a data-driven estimator for the angle $\\angle(\\hat{w}, w_*)$ --- estimator of the squared inner product proposed in [7], plus a sign estimator --- and supports the theory with simulation results. The authors conclude the paper with a brief discussion and experimental results discussing universality beyond Gaussian design.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths\n- **Significance:** The paper addresses an important problem of how to calibrate predictive probabilities in high dimensions, where traditional methods are known to fail. The angular calibration method is conceptually elegant, connects well with recent advances in debiasing techniques, and, to the best of my knowledge, offers the first provable calibration procedure in this regime with optimality guarantees.\n- **Theoretical quality:** The main results are mathematically rigorous and clearly stated. The proofs are well-grounded and provide meaningful insight into the behavior of calibration methods when the covariate dimension $d \\sim n$, the sample size. The analysis linking angular calibration and Platt scaling is particularly noteworthy and adds theoretical justification for a widely used heuristic.\n- **Clarity:** The paper is well-organized and clearly written. The paper provides useful intuition behind the angular predictor and its interpolation structure, making the main ideas accessible despite the technical depth.\n\n### Weaknesses\n- **Lack of finite-sample guarantees:** A notable limitation is the lack of finite-sample results. All theoretical guarantees are asymptotic in nature. While I understand this asymptotic result has its own values, it leaves open questions about the accuracy of calibration and optimality in practical settings. A discussion of the rate of convergence or any quantification of the asymptotic approximations’ accuracy in finite-sample setting would be useful to gauge practical applicability. Moreover, while simulations offer some insight, a more thorough empirical investigation into these finite-sample effects could help bridge the gap between theory and real-world use.\n- **Assumptions and practicality:** The results rely heavily on the Gaussian design assumption and knowledge (or estimability) of the covariance $\\Sigma$, Although the authors briefly discuss universality (Section 8) and show simulations under non-Gaussian designs, formal universality results are not yet developed and deferred to future work.\n- **Scope of empirical validation:** While the simulations are well-designed and support the theory, they are limited in scope. Empirical comparison against alternative calibration methods (e.g., temperature scaling, isotonic regression, histogram binning) beyond Platt scaling would further strengthen the case for the proposed method.\n\nQUESTIONS:\n**1. Scope of applicability beyond regularized M-estimators:** \nThe results in Sections 4 and 5 appear to apply to any fixed predictor $\\hat{w}$, even if it is not derived from a regularized M-estimation procedure. However, the angle estimation procedure in Section 6 critically relies on $\\hat{w}$ being a regularized M-estimator, to leverage observable quantities from recent debiasing literature. It would be helpful if the authors could clarify this distinction explicitly and discuss how the approach could (or could not) be extended to more general $\\hat{w}$ (not necessarily a regularized M-estimator). \n\n**2. Finite-sample behavior and out-of-sample generalization:**\nAs noted above, the theoretical results are asymptotic. While simulations indicate promising performance in specific scenarios, could the authors provide further guidance or empirical evaluation of how calibration quality and Bregman loss behave as functions of sample size, model misspecification, or holdout set size? In particular, how sensitive are the procedures to finite-sample noise in the angle estimator or to misspecification of the link function or data distribution? Addressing these questions, even heuristically or empirically, would help assess the method’s robustness in real-world applications.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}