Title: Improved Bounds for Swap Multicalibration and Swap Omniprediction
Venue: NeurIPS 2025
Accepted: yes
Keywords: (Swap) Multicalibration

(Swap) Omniprediction

(Swap) Agnostic Learning

Contextual Swap Regret
Abstract: In this paper, we consider the related problems of multicalibration --- a multigroup fairness notion and omniprediction --- a simultaneous loss minimization paradigm, both in the distributional and online settings. The recent work of Garg et al. (2024) raised the open problem of whether it is possible to efficiently achieve $\tilde{\mathcal{O}}(\sqrt{T})$ $\ell_{2}$-multicalibration error against bounded linear functions. In this paper, we answer this question in a strongly affirmative sense. We propose an efficient algorithm that achieves $\tilde{\mathcal{O}}(T^{\frac{1}{3}})$ $\ell_{2}$-swap multicalibration error (both in high probability and expectation). On propagating this bound onward, we obtain significantly improved rates for $\ell_{1}$-swap multicalibration and swap omniprediction for a loss class  of convex Lipschitz functions. In particular, we show that our algorithm achieves $\tilde{\mathcal{O}}(T^{\frac{2}{3}})$ $\ell_{1}$-swap multicalibration and swap omniprediction errors, thereby improving upon the previous best-known bound of $\tilde{\mathcal{O}}(T^{\frac{7}{8}})$. As a consequence of our improved  online results, we further obtain several improved sample complexity rates in the distributional setting. In particular, we establish a $\tilde{\mathcal{O}}(\varepsilon ^ {-3})$ sample complexity of efficiently learning an  $\varepsilon$-swap omnipredictor for the class of convex and Lipschitz functions, $\tilde{\mathcal{O}}(\varepsilon ^{-2.5})$ sample complexity of efficiently learning an $\varepsilon$-swap agnostic learner for the squared loss, and $\tilde{\mathcal{O}}(\varepsilon ^ {-5}), \tilde{\mathcal{O}}(\varepsilon ^ {-2.5})$ sample complexities of learning $\ell_{1}, \ell_{2}$-swap multicalibrated predictors against linear functions, all of which significantly improve on the previous best-known bounds.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies multicalibration and omniprediction problems in distributional and online settings. In the online setting, they show an $O(T^{1/3})$ $\\ell_2$-swap multicalibration error, resolving an open question posed by Garg et al. The algorithm, which is based on Blum-Mansour reduction, is extended to significantly improve error and sample complexity bounds on several related notions of contextual swap regrets, multicalibration and omniprediction errors.\n\nSTRENGTHS AND WEAKNESSES:\nDisclaimer: I am not familiar with the topic of the paper.\n\n## Assessment: \nThis paper makes significant contribution in improving a large number of existing error and sample complexity bounds. In most cases, the improvements are orders of magnitude better than existing results (e.g. Theorem 1, Corollary 1 and Theorem 4).\n\nThe two new notions on pseudo-swap multicalibration and pseudo contextual swap regret, as well as the reduction from the former to the latter, seem fundamentally important as they ultimately lead to improved $\\ell_2$-swap multicalibration error. \n\nIn addition to novel ideas and improved bounds, it appears that the paper also introduce new analysis techniques as stated in line 134: \"By\nperforming a careful martingale analysis using Freedman’s inequality on a geometric partition of the interval [0, 1], we finally establish the desired concentration bound.\" However, this new martingale analysis is not made clear in the main text. \n\nIn terms of weaknesses, I do not find anything significant. For a theoretical paper with many upper bounds, it is beneficial to have a formal argument on the sort of lower bounds one can obtain. However, it is understandable that lower bound proofs might take an entirely different paper.\n\n## Questions:\n1. Can the authors clarify where the \"careful martingale analysis using Freedman’s inequality on a geometric partition of the interval [0, 1]\" is? I did look at Appendix D.1, but I cannot find any new technical lemma on this. Is this hidden in the proof of Lemma 9? In any cases, could you briefly describe the nature of this new analysis?\n2. I cannot find the discussion on the computational complexity of your algorithms. In particular, I am concerned whether or not your algorithms have to explicitly construct a cover in Proposition 1. \n3. How robust are your results with respect to the particular finite discretization of $[0,1]$ line 76? (This might be a stupid question as I am not familiar with this setting).\n4. While I suppose the paper is sufficiently clear for expert reviewers who are familiar with swap regret and multicalibration, some parts of it should be improved for better clarity so that even people outside of the domain can understand. For example, I am unable to grasp what $\\mathcal{P}_t$ in line 154 is. Where does it come from, for example is it something that the algorithm computes in time step $t$?\n5. Can you clarify what it means for $\\mathcal{P}_t$ to be deterministic?\n6. Similarly, Algorithm 1 is not self-contained as it is written. Are $p_{t,i}$ and $ALG_i$ input to Algorithm 1?\n\nQUESTIONS:\nPlease see the questions in the \"Strengths and Weaknesses\" section above.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 1, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-24"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper provides an algorithm and improved upper bounds for the problems of (swap) omniprediction and multicalibration in the online and (by an online-to-batch reduction) the offline batch settings. In particular, the authors employ an alternate proof strategy to solve an open problem proposed by Garg et al. on the possibility of achieving $O(\\sqrt{T})$ $\\ell_2$-multicalibration error against bounded linear functions. The authors' main technical result is an algorithm for achieving $\\ell_2$-swap multicalibration in $O(T^{1/3})$. This bound also implies significant rates for multicalibration, swap omniprediction for convex Lipschitz functions, and, accordingly, improved sample complexity in the offline analogues. The main argument, to my understanding, is as follows: \n\n1. The authors introduce two notions *pseudo swap multicalibration* and *pseudo swap contextual regret* that avoid dealing with averaging over the Learner's predictions in the online game, and a reduction exists from swap multicalibration to pseudo swap multicalibration (simply via a covering argument on the class of linear functions) \n2. The authors reduce pseudo swap multicalibration to pseudo contextual swap regret via properties of the linear function class\n3. **main technical contribution:** the authors develop an algorithm for pseudo contextual swap regret using a reduction to the classical external to swap regret reduction of Blum and Mansour and an improved rounding scheme. This reduction requires an external regret algorithm to exist, which the authors instantiate as simply Online Newton Step for a particular linear function class.\n\nThe upper bound obtained by this argument then implies improved results in online and offline (swap) omniprediction and $\\ell_1$-multicalibration through improvements of more standard arguments.\n\nSTRENGTHS AND WEAKNESSES:\nI am quite positive on this paper -- the results are clearly presented with an eye towards giving a nice proof sketch and overview of the entire argument for the reader, the techniques themselves are combined in a novel way, and the authors provide good contextualization with the rest of the literature on omniprediction and multicalibration. The only weakness that the authors might address is to possibly provide a bit more intuition on why these particular techniques result in the improved bounds -- although it seems clear to me that the improvements come from the reduction to the *pseudo* notions, a comparison to why the pseudo notions combined with the standard Blum/Mansour external to swap regret reduction and improved rounding scheme give a key algorithmic improvement to the prior work of Garg et al. I understand that the rates go down, but it would be very nice to also have the insight on *why* the pseudo notion is \"easier to optimize\" in this particular context. A couple sentences on that would improve an already very good paper.\n\n## Strengths \n\n**Originality:** To my knowledge, the authors' techniques in proving the main improved upper bound on $\\ell_2$-swap multicalibration is novel, and, in particular, the key introduction of *pseudo* swap multicalibration and *pseudo* contextual swap regret is novel. In fact, I am interested in seeing if similar notions have cropped up in the literature (see \"Questions\"), as the move from having to condition using the empirical conditional distribution of the $p_t$ to just dealing with the true distributions at each step certainly simplify the \"conditioning\" problem that arises when dealing with these multicalibration notions in the online setting, and I'd be interested in seeing if this kind of strategy has been employed before. To my knowledge, this seems like a new move. \n\n**Quality:** The results of the paper themselves improve bounds for several open questions, and the authors \"use the full force\" of their main derived bound to fully address the implications downstream for ominprediction and multicalibration by also working out the details for the offline setting, which I appreciated. It answers an existing open question about a specific faster rate for $\\tilde{O}(\\sqrt{T})$ for the specific problem of swap multicalibration in the affirmative, and the new algorithm that results in this upper bound uses the authors' pseudo notions effectively.\n\n**Clarity:** The paper itself is very well-written and the results are presented in a clear manner, with intuitive proof sketches and a logical flow to the sections. The provided diagram is helpful in understanding the big picture of the argument, and the authors do not waste space on more routine details. I had little trouble following along with the paper (though I am quite familiar with the multicalibration/omniprediction subfield, so others' mileage may vary).\n\n**Significance:** I believe that the results the authors provide are significant enough to warrant publication and answer an existing open question. The reduction to the pseudo notions also give us insight on a particular difficulty in multicalibration/related problems in regret minimization, particularly their swap variants. \n\n## Weaknesses\nThe weaknesses I see in the work are minor. \n\nThe first pertains to motivation: the authors might address is to possibly provide a bit more intuition on why these particular techniques result in the improved bounds -- although it seems clear to me that the improvements come from the reduction to the *pseudo* notions, a comparison to why the pseudo notions combined with their algorithm nets an improvement. \n\nAnother weakness one might argue that the results themselves are too narrow -- that the result for just the class of bounded linear functions doesn't achieve the scope of previous multicalibration results. I don't really see this as a big weakness, as the existing open question asked if a faster rate was possible even for bounded linear functions, and that was not known. Although it seems to me that more general function classes may need different techniques entirely (it is unclear to me how to extend dealing with the \"pseudo\" notions for more complex function classes), the technical contribution of the authors for bounded linear functions does add knowledge to the subfield, in my opinion.\n\nQUESTIONS:\n1. Have these pseudo notions previously appeared in the literature? It seems that the argument hinges around the construction of these notions, and they do point towards a major difficulty in multicalibration-style regret bounds -- the conditioning on the $p_t$s.\n\n2. Is the bounded linear functional necessary in this argument because of the covering and reduction to $P_t$ (pseudo notions)? The argument seems specialized once we go to pseudo notions because of the ease of constructing a cover. To this note, would more general function classes need new techniques entirely?\n\n3. I'm curious as to the efficiency compared to the previous algorithm of Garg et al? To my understanding, the Blum/Mansour reduction is efficient because it just employs $N$ external regret algorithms (where $N$ is from the rounding). Even a qualitative comparison of the \"efficiency\" compared to the previous algorithm would be interesting, just to note the algorithmic differences of the approaches.\n\nLIMITATIONS:\nYes (pure theory paper).", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper derives improved online rates and sample complexity rates for swap omniprediction.\n  Swap omniprediction is a strengthening of omniprediction (learn a predictor that is close enough to Bayes optimal that it can be post-processed to be near-optimal for any loss) that was introduced by GKR24, who motivated the concept by noting that while multicalibration > omniprediction, swap multicalibration = swap omniprediction.  \n  The authors obtain these improved rates by instead first deriving faster rates for a 'pseudo' variant of swap multicalibration that allows them to reduce to a more tractable problem of pseudo contextual swap regret.  \n\n The main proofs are for the online setting, with the distributional sample complexity rates following from an online-to-batch reduction. The main intuition of the paper (which is nicely summarized in Figure 1) is that instead of obtaining swap multicalibration rates by reducing to swap regret minimization, the authors instead reduce to pseudo swap regret minimization. One way to understand what this detour through pseudo calibration measures is doing is that its effectively separating the analysis of the variance caused by the forecaster's random forecasts with the actual learning component.\n\n  The interesting thing (at least to me, who has not followed the recent work here closely) is that reducing from swap multicalibration to pseudo-swap multicalibration costs such little overhead: $O(N)$ . This is done in Lemma 1 which shows SMCal_{ F, 2} leq O(N log (|F| N / delta)) + 6 PSMCal_{ F,2} with the overhead term being negligible for the types of discreitizations we normally consider, e.g. N=T^1/3. The analysis is an adaptation of Freedman's inequality, which is currently not really talked about in the main text.  \n\n  The pseudo-multicalibration rates are obtained by reducing to pseudo contextual swap regret. This happens in Lemma 3 (through lemma 4). Specifically: if there is pseudo-multicalibration error, then that error can be captured by a linear competitor meaning that there must be some pseudo contextual swap regret. So, pseudo swap contextual regret for the affine closure of functions in F always upper bounds pseudo multicalibration for F.  The other step of the reduction, going from swap multicalibration to swap omniprediction is standard.  \n\nThe other chunk of the paper is actually obtaining good rates for pseudo swap regret minimization. The algorithm follows the usual Blum-Mansour algorithm for minimizing swap regret where you instantiate multiple algorithms, one for each action, and then act according to a stationary distribution over the algorithms. In this case, we use the BM algorithm with actions corresponding to discretized level sets and online newton step for external regret minimizers (because squared loss is exp-concave). The algorithm design and analysis follow a pretty standard recipe, with the main interesting component being the observation that relaxing to \"pseudo\" swap regret means that instead of the BM reduction getting noisy feedback it gets deterministic feedback, and so one doesn't need to add in an error term for stochastic approximation.\n\nSTRENGTHS AND WEAKNESSES:\nThe paper is well-written with concrete improvements in theoretical results (improved rates for swap omniprediction) and a clean conceptual contribution: when trying to get rates for swap calibration/omniprediction/agnostic-learning, it's often easier to mentally separate out how one analyzes the stochastic approximation errors from forecast randomness—reducing to pseudo calibration and pseudo swap regret is a nice way to think about how to do this.\n\nI don't have any particular concerns with weaknesses in presentation or results. I did not check the proofs, which are deferred entirely to the appendix, carefully though the results and steps seem reasonable.\n\nQUESTIONS:\nI'm not very familiar with the swap omniprediction literature; is there a fundamental eps^-3-eps^-2 separation for swap vs normal omniprediction that's known?\n\nPrior offline swap omniprediction rates cited in Table 1 are attributed to [2, 13] rather than [5]: is there a challenge with online-to-batch adaptation of the rates of [1]?\n\nWhy do these detours into pseudo calibration measures arise with \"swap\" objectives whereas e.g. [6]'s optimal rates for omniprediction do not appear to need them?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-16"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper gives novel bounds on the multicalibration and ominprediction errors in the both the adversarial (online) and distributional settings. The proven error bounds in the adversarial setting are as follows:\n* swap multi-calibration regret: $T^{1/3}$ (improved from $T^{3/4}$)\n* swap omniprediction regret: $T^{2/3}$ (improved from $T^{7/8}$)\nSimilar improvements have also been established in the distributional setting, with an online-to-batch conversion. The key idea is to connect to a particular notion of a pseudo-regret (where some of the randomness is removed) via a martingale concentration argument.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n* This paper gives significantly improved bounds across the board over prior works on swap multi-calibration and, via established reductions, on swap omniprediction. Both of these are previously studied problems in the literature.\n* While the techniques and individual steps appear to be known or inspired from very similar ones, the improved bounds are obtained by carefully putting together various pieces.\n* The paper seems well-polished and I did not see any typos.\n\n**Weaknesses**:\n\nI found the paper nearly impossible to parse. As somebody who is familiar with learning theory but never previously heard about multi-calibration or omniprediction (or swap contextual regret, or similar terms), I expected to be able to at least understand the definitions and the significance of the results. While the paper reads like something that researchers knowledgeable in this (apparently niche) area would appreciate, I did not even understand the basic definitions. Further, it tries to pack a lot of math into a small number of pages, making the text incredibly dense as well.\n\nI especially did not understand the definition of multi-calibration. While this is an established notion and has a prior literature, I believe the onus is on the authors to provide enough intuition about the definitions and significance of the results (at least in the appendix). Similarly, I do not understand where the \"swap\" versions come from or why they are interesting.\n\nIt would have been nice to see more special cases/connections as to why these bounds are interesting, such as connections to textbook examples that a reader such as myself would appreciate. For example, the introduction mentions connections to fairness: that would be a good setting to expand upon.\n\nQUESTIONS:\n* What does the notation $\\mathbb{R}^X$ mean? Is this the set of functions $X \\to \\mathbb{R}$?\n* The connections between multi-calibration and fairness are unclear. It would be good to dedicate some space to that aspect to appeal to a broader audience.\n* Are any lower bounds known for any of these problems? What about simplifications to known textbook-type problems?\n* I'm unable to parse $\\rho_{p, f}$ and the meaning of the definition in eq. (1). This needs some interpretation.m\\\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 1, "CLARITY": 1, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-21"}