Title: A Unifying Perspective on Multi-Calibration: Game Dynamics for Multi-Objective Learning
Venue: NeurIPS 2023
Accepted: yes
Keywords: multicalibration

multi-objective learning

learning theory

calibration

fairness

games
Abstract: We provide a unifying framework for the design and analysis of multi-calibrated predictors. By placing the multi-calibration problem in the general setting of multi-objective learning---where learning guarantees must hold simultaneously over a set of distributions and loss functions---we exploit connections to game dynamics to achieve state-of-the-art guarantees for a diverse set of multi-calibration learning problems. In addition to shedding light on existing multi-calibration guarantees and greatly simplifying their analysis, our approach also yields improved guarantees, such as error tolerances that scale with the square-root of group size versus the constant tolerances guaranteed by prior works, and improving the complexity of $k$-class multi-calibration by an exponential factor of $k$ versus Gopalan et al.. Beyond multi-calibration, we use these game dynamics to address emerging considerations in the study of group fairness and multi-distribution learning.

A Unifying Perspective on Multicalibration: Game Dynamics for Multi-Objective Learning ∗

Abstract

We provide a unifying framework for the design and analysis of multicalibrated predictors. By placing the multicalibration problem in the general setting of multiobjective learning-where learning guarantees must hold simultaneously over a set of distributions and loss functions-we exploit connections to game dynamics to achieve state-of-the-art guarantees for a diverse set of multicalibration learning problems. In addition to shedding light on existing multicalibration guarantees and greatly simplifying their analysis, our approach also yields improved guarantees, such as error tolerances that scale with the square-root of group size versus the constant tolerances guaranteed by prior works, and improving the complexity of k -class multicalibration by an exponential factor of k versus Gopalan et al. [17]. Beyond multicalibration, we use these game dynamics to address emerging considerations in the study of group fairness and multi-distribution learning.

1 Introduction

Multicalibration has emerged as a powerful tool for addressing fairness considerations in machine learning. Based on calibrated forecasting [6, 12]-which requires among instances x a predictor h predicts the probability h ( x ) = v , a fraction v have a positive outcome-multicalibration yields more fine-grained guarantees by seeking calibration across large and possibly overlapping collections of sub-populations [21]. Multicalibration has been studied in numerous settings, including those with rich label sets (multi-class multicalibration [17]), adversarial rather than stochastic data (online multicalibration [18]), and problems where no Bayes classifier exists (agnostic multicalibration [38]). The concept of multicalibration has also been applied to estimate other quantities, such as higher moments [22], and been strengthened in various ways, such as providing conditional guarantees [2].

Multicalibration's versatility has led to the development of numerous specialized algorithms, each tailored to a unique multicalibration problem and requiring its own individualized analysis. Promising attempts to provide an overarching conceptual framework for multicalibration, such as outcome indistinguishability [7], have had limited success in unifying these various algorithms. In this paper, we tackle this challenge, developing a general-purpose algorithmic framework to guide the design of multicalibrated learning algorithms for a wide range of settings and considerations.

Our approach is a dynamical systems and game-theoretic approach [see, e.g., 13, 9]. We demonstrate that many multicalibration algorithms can be formulated as particular instances of two-player zerosum games where players independently either run no-regret algorithms or best-response algorithms. A wide range of multicalibration algorithms that exhibit varying trade-offs can be obtained by plugging in different no-regret and best-response algorithms. This unified framework both recovers existing guarantees and in many cases improves upon them.

Table 1: This table summarizes the sample complexity and agnostic learning oracle complexity rates we obtain for multicalibration (MC), compared with the previous state of the art. S denotes the set of groups one wants multicalibration on, d the VC dimension of S , k the number of label classes, and ε the error tolerance. (Det) and (Succinct) respectively denote when only deterministic or only succinct predictors are acceptable. 'Sqrt Err' refers to requiring an error tolerance of ε √ Pr( x ∈ S ) for each group rather than ε .

Although approaching multicalibration-a min-max optimization problem-with game dynamics seems like an obvious approach, no prior work has succeeded in using a game dynamics framing to unify multicalibration algorithms. The primary challenge is not reducing multicalibration to min-max optimization, which is straightforward (Facts 2.5,2.6), but rather solving the resulting equilibrium computation problem in a way that connects to practical algorithms. The needs of multicalibration (such as determinism, large and complex predictor spaces, etc.) differ significantly in this regard from earlier applications of general-purpose no-regret algorithms and game dynamics.

Our primary contributions can be categorized into three areas.

1) Unifying framework. In Section 3, we give an overview of game dynamics as a concise but general framework for obtaining multi-objective learning guarantees. To use these dynamics as a generic solution to various multicalibration problems, we introduce a powerful general-purpose no-regret algorithm (Theorem 3.7) and a distribution-free best-response algorithm (Theorem 3.8), for calibration-like objectives. This approach allows us to unify the diverse-and often, seemingly unrelated-algorithms that have been studied in the multicalibration literature, offering a general template for their derivation and analysis.

New guarantees. In Sections 4, we use our framework to improve guarantees for various multicalibration settings (see Table 1) by simply plugging in different no-regret and best-response algorithms. These improvements include an exponential (in k ) reduction in the complexity of k -class multicalibration over Gopalan et al. [17], a polynomial (in 1 /ε ) reduction in the complexity of learning a succinct multicalibrated predictor over Dwork et al. [7], the first conditional multicalibration results for the batch setting, the first agnostic multicalibration guarantees that beats uniform convergence, and a multicalibration algorithm that efficiently guarantees an error tolerance that scales with square-root of each group's probability mass. In Section 5, we demonstrate that our framework can be extended to analyze problems beyond multicalibration, such as multi-group learning [39].

Simplified analyses. We also demonstrate that our framework can recover the guarantees of various existing multicalibration algorithms, including online multicalibration [18] and moment multicalibration [22], while avoiding intricate and problem-specific arguments.

1.1 Related Work

The study of calibration originated in online (adversarial) forecasting [6, 20], with classical literature having also studied calibration across multiple sub-populations [11]. Multicalibration, on the other hand, is classically studied in the stochastic setting; i.e., where ( x i , y i ) ∼ D , in which case calibration is trivially satisfied by the predictor h ( x ) = E [ y ] . Due to this difference in formulation in the literature on calibration and that on multicalibration, the specific technical tools that have been developed in these areas have largely remained distinct. Our work can be viewed as bridging this gap by showing

that game-theoretic dynamics provides a unified foundation for studying multicalibration, just as no-regret learning underpins the study of calibration.

Motivated by fairness considerations, a formal definition of multicalibration was presented by [21], and has found a wide range of applications and conceptual connections to Bayes optimality, conformal predictions, and computational indistinguishability [21, 22, 16, 18, 7, 23]. Algorithms for multicalibration have largely developed along two lines: one studying oracle-efficient boostinglike algorithms [see, e.g., 21, 25, 17, 7] and another studying algorithms with flavors of online optimization [see, e.g., 18, 35]. Our work establishes that the contrast between these lines of work and the algorithms they develop are entirely attributable to different choices of game dynamics.

Multi-objective and multi-distribution learning are concepts that have found broad applications in addressing fairness, collaboration, and robustness challenges. Blum et al. [4] initiated the study of learning predictors with near-optimal accuracy across multiple populations (distributions), with later works attaining tight bounds [19, 34, 31]. Recently, multi-objective learning has been applied to settings where sub-populations are mutually compatible [37, 39, 3]. In Section 5, we use our framework to match and improve guarantees relative to this literature.

2 Preliminaries

We use X to denote a feature space and Y a label space, where Y = [ k ] in k -class classification. A data distribution D is a probability distribution supported on labeled datapoints X × Y . We use H to denote a set of hypotheses and G a set of objectives, where an objective-or equivalently, a loss-is a function glyph[lscript] : H× ( X × Y ) → [0 , 1] that takes a hypothesis and datapoint and returns a penalty value. We denote expected objective values by L D,glyph[lscript] ( h ) := E ( x,y ) ∼ D [ glyph[lscript] ( h, ( x, y ))] . For non-deterministic p ∈ ∆( H ) and q ∈ ∆( D×G ) , we overload notation to write L q ( p ) := E h ∼ p, ( D,glyph[lscript] ) ∼ q [ L D,glyph[lscript] ( h )] . We often use the shorthands x (1: T ) := x (1) , . . . , x ( T ) and { f ( x ( t ) ) } (1: T ) = f ( x (1) ) , . . . , f ( x ( T ) ) . We also write f ( a, · ) to denote the function x ↦→ f ( a, x ) or f ( · ) ( a ) to denote x ↦→ f x ( a ) . For y ∈ [ k ] , δ y ∈ { 0 , 1 } k denotes its one-hot delta function, while for y ∈ [ k ] , j ∈ [ k ] we write δ y,j = 1[ y = j ] .

2.1 Multicalibration

We use P = (∆ Y ) X to denote the set of all k -class predictors which are maps from features to label distributions. To differentiate between predictors and distributions over predictors, we refer to h ∈ P as a deterministic predictor, and p ∈ ∆( P ) , as a non-deterministic predictor. 2 Calibration is a property of predictors h ∈ P requiring, for example in binary classification, that among instances x assigned prediction probability h ( x ) = [1 -v, v ] a fraction v are truly labeled 1 . Multicalibration [21] is the finer-grained notion that requires calibration on subgroups of one's domain. This set of subgroups is typically finite or of finite VC dimension. In practice, we work with approximate notions of calibration/multicalibration and discretize the range of probability assignment. In k -class prediction, we partition the k -dimensional hypercube into λ k equal cubes V k λ , where V λ := { [0 , 1 /λ ) , [1 /λ, 2 /λ ) , . . . } . For any interval v ∈ V k λ , we use h ( x ) ∈ v to denote that prediction h ( x ) falls pointwise in the buckets of v , i.e., h ( x ) j ∈ v j for all j ∈ [ k ] . We next formally define multicalibration. 3

Definition 2.1. Fix ε > 0 , λ ∈ Z + , and a set of groups S ⊆ 2 X . A (possibly non-deterministic) k -class predictor p ∈ ∆( P ) is ( S , ε, λ ) -multicalibrated for some data distribution D if

That is, p is calibrated on every level set v ∈ V k λ of every group S ∈ S for every class j ∈ [ k ] . We are often specifically interested in a deterministic solution; that is, where p ∈ P .

The batch setting, where we want to find a multicalibrated predictor for some fixed data distribution D , is the most commonly studied. Multicalibration can also be defined for online settings where the data distribution changes adversarially over time.

Definition 2.2 (Online multicalibration) . Fix ε > 0 , λ ∈ Z + , and a set of groups S ⊆ 2 X . In online multicalibration, at every timestep t ∈ [ T ] , a learner chooses a k -class predictor p ( t ) ∈ ∆( P ) . Nature, which observes p (1: t ) , responds with any choice of data distribution D ( t ) . The learner's predictors p (1: T ) are ( S , ε, λ ) -online multicalibrated on D (1: T ) if

2.2 Multi-Objective Learning

We use multi-objective learning (a generalization of multi-distribution learning introduced by [19]) as a tool for studying multicalibration and other related problems. The goal of multi-objective learning is to find a hypothesis that simultaneously minimizes a set of objective values.

Definition 2.3 (Multi-objective learning) . A multi-objective learning problem ( D , G , H ) consists of a set of objectives G , a hypothesis class H , and a set of data distributions D . An ε -optimal solution to ( D , G , H ) is a (potentially non-deterministic) hypothesis p ∈ ∆( H ) where

We usually prefer a deterministic solution p ∈ H over a non-deterministic solution p ∈ ∆( H ) .

We mainly consider single-distribution multi-objective problems, where D = { D } , though multidistribution multi-objective learning arises in conditional multicalibration (Section E.1) and group fairness (Section 5). We can also consider multi-objective learning in online adversarial settings.

Definition 2.4 (Online multi-objective learning) . An online multi-objective learning problem ( D , G , H ) consists of a set of distributions D , objectives G and hypothesis class H . At each timestep t ∈ [ T ] , a learner first picks a hypothesis p ( t ) ∈ ∆( H ) . Nature, who sees p (1: t ) , responds with a data distribution D ( t ) ∈ D . We say the hypotheses p (1: T ) are ε -optimal on the distributions D (1: T ) if

In many problems we consider, like online multicalibration, Nature can pick from any data distribution; that is, D is unrestricted. Let us also note that the baseline in the right-hand-side of (2) differs from that of (1). This is intentional: when D is unrestricted, the min-max baseline of (1) may be large, since there may be no hypothesis that is simultaneously good for all distributions. Instead, the max-min baseline of (2) is the best hypothesis h ∗ for the most difficult distribution D ∗ .

Multicalibration as multi-objective learning. (Batch) Multicalibration is a single-distribution multi-objective learning problem, whose objectives penalize over-estimation and under-estimation of label probabilities on subsets of the domain.

Fact 2.5. Let D be a data distribution for some k -class prediction problem and fix ε > 0 , λ ∈ Z + , and a set of groups S ⊆ 2 X . For every direction i ∈ {± 1 } , level set v ∈ V k λ , group S ∈ S , and class j ∈ [ k ] , we define an objective glyph[lscript] i,j,S,v : P → [0 , 1] where

and G mc := { glyph[lscript] i,j,S,v } i,j,S,v is the set of these objectives. Predictor p ∈ ∆( P ) is a ε -optimal solution to the multi-objective problem ( { D } , G mc , P ) if and only if p is ( S , 2 ε, λ ) -multicalibrated for D .

Online multicalibration is similarly an online multi-objective learning problem.

Fact 2.6. Let D (1: T ) be data distributions for some k -class prediction problem, D be the set of all data distributions, and G mc be as defined in (3) . Fix ε > 0 , λ ∈ Z + , and a set of groups S ⊆ 2 X . A sequence of predictors p (1: T ) ∈ ∆( P ) is ε -optimal on D (1: T ) for the online multi-objective learning problem ( D , G mc , P ) if and only if p (1: T ) is ( S , 2 ε, λ ) -online multicalibrated on D (1: T ) .

3 Tools for Solving Multi-Objective Learning using Game Dynamics

A common approach to multi-objective learning is to imagine a game between a minimizing player who proposes hypotheses and a maximizing player who proposes objectives and data distributions. It is well-established (inspired by min-max equilibria, e.g., [14]) that a solution can be obtained when players use no-regret algorithms. However, considerations that arise in multicalibration motivate us to study a broader range of dynamics and their implications than has been commonly explored.

Online learning. In an online learning problem, at each timestep t ∈ [ T ] , a learner chooses an action a ( t ) ∈ A which an adversary observes and responds to with a cost function c ( t ) : A → [0 , 1] . We will usually assume costs to be linear maps. The learner's regret is defined as Reg( a (1: T ) , c (1: T ) ) := ∑ T t =1 c ( t ) ( a ( t ) ) -min a ∗ ∈A ∑ T t =1 c ( t ) ( a ∗ ) , which no-regret algorithms like Hedge [14] can bound.

Lemma 3.1 (Hedge Regret [24]) . In an online learning problem where the action set is the simplex A = ∆ k and costs are linear, the actions chosen by Hedge have a regret of at most 2 √ ln( k ) T .

We write no-regret algorithms as a function of a sequence of cost functions Alg : ([0 , 1] A ) ∗ →A . For example, when A = ∆ k , the output of the Hedge algorithm is defined as Hedge ( c (1) , . . . , c ( t ) ) = [ w 1 / ‖ w ‖ 1 , . . . , w k / ‖ w ‖ 1 ] where w i = exp( -η ∑ t τ =1 c ( t ) ( δ i )) , for a specific choice of η .

We sometimes define regret against a different baseline B ∈ R , with Reg B ( a (1: T ) , c (1: T ) ) := ∑ T t =1 c ( t ) ( a ( t ) ) -B . For example, we often consider the min-max baseline B weak := T · min a ∗ ∈A max c ∗ ∈C c ∗ ( a ∗ ) , where C is the set of cost functions that the adversary chooses from. We also often encounter stochastic cost functions , functions of form c : A× ( X × Y ) → [0 , 1] for which we want to minimize expected value L D,c := E ( x,y ) ∼ D [ c ( · , ( x, y ))] on some distribution D . A stochastic cost c is linear if, for every x ∈ X , y ∈ Y , c ( · , ( x, y )) is a linear map. The regret of online learning algorithms on stochastic costs concentrates quickly.

Lemma 3.2 (Stochastic Approximation [33], Lemma 3.1) . Consider an online learning problem on the simplex ∆ k with linear stochastic costs. Suppose, after each timestep t ∈ [ T ] -that is, after picking the action a ( t ) -we estimate the expected cost L D,c ( t ) with ̂ c ( t ) ( a ) := c ( t ) ( a, ( x ( t ) , y ( t ) )) , where ( x ( t ) , y ( t ) ) i.i.d. ∼ D . With probability at least 1 -δ , ∣ ∣ Reg( a (1: T ) , { L D,c ( t ) } (1: T ) ) -Reg( a (1: T ) , ̂ c (1: T ) ) ∣ ∣ ≤ O ( √ T ln( k/δ )) .

We say an action a is an ε -best response to a cost c if c ( a ) ≤ min a ∗ ∈A c ( a ∗ ) + ε . Similarly, an action a is a distribution-free ε -best response to a stochastic cost c if max x ∈X ,y ∈Y c ( a, ( x, y )) ≤ max x ∈X ,y ∈Y min a ∗ ∈A c ( a ∗ , ( x, y )) + ε . We sometimes express the sample complexity of an algorithm in terms of the number of calls made to an agnostic learning oracle , which can compute ε -best-responses to stochastic cost functions using few samples. See Appendix A for a discussion.

Game dynamics. We consider no-regret dynamics between a learner (minimizing player) who chooses hypotheses p ( t ) ∈ ∆( H ) and an adversary (maximizing player) who chooses data distributions and objectives q ( t ) ∈ ∆( D×G ) where the learner's loss is L q ( t ) ( p ( t ) ) . When both players are no-regret, the time-average actions they picked quickly converge to an approximate solution for the multi-objective learning problem ( D , G , H ) . This method of no-regret dynamics has long played a role in empirical convergence to notions of equilibria [14]. Here, we review these dynamics and their convergence guarantees. While the proofs of these lemmas are standard at a high level (and deferred to Appendix C.1), they differ in fundamental ways from past work. In particular, these lemmas consider weak regret , single timestep solutions (instead of time-averaged ones), and the consequences of having distribution-free best responses , all of which play important roles in multicalibration.

Lemma 3.3 (No-Regret vs. No-Regret (NRNR)) . Consider a multi-objective learning problem ( D , G , H ) , where a learner and adversary chose p (1: T ) ∈ ∆( H ) and q (1: T ) ∈ ∆( D×G ) . If both players are no-regret, Reg weak ( p (1: T ) , {L q ( t ) ( · ) } (1: T ) ) ≤ Tε and Reg( q (1: T ) , { -L ( · ) ( p ( t ) ) } (1: T ) ) ≤ Tε , then the non-deterministic hypothesis p = Uniform ( p (1: T ) ) is a 2 ε -optimal solution.

The next dynamic focuses on obtaining a solution from a single timestep, rather than time-averaged solutions. To obtain this, we consider a dynamics in which the learner goes first and is no-regret, the adversary observes the learner's action and then best responds.

Lemma 3.4 (No-Regret vs. Best-Response (NRBR)) . Consider a multi-objective learning problem ( D , G , H ) , where a learner chose p (1: T ) ∈ ∆( H ) and an adversary chose q (1: T ) ∈ ∆( D×G ) . If the learner is no-regret, Reg weak ( p (1: T ) , {L q ( t ) ( · ) } (1: T ) ) ≤ Tε , and the adversary ε -best-responded to the costs { -L ( · ) ( p ( t ) ) } (1: T ) using q (1: T ) , then there is a t ∈ [ T ] where p ( t ) is a 2 ε -optimal solution.

Once the existence of a single-round solution p ( t ) is established by Lemma 3.4, it is easy to find which time step corresponds to this solution by using a few samples and testing all p (1: T ) , as follows.

Lemma 3.5. Suppose a set of hypotheses p (1: T ) contains an ε -optimal solution. We can find a 5 ε -optimal solution p ( t ) ∈ p (1: T ) using O ( ε -2 |D| ln( |D| |G| T/δ )) samples with probability 1 -δ .

The next dynamic considers difficult distribution-free problems, such as online multi-objective learning. To enable learning in these scenarios, we consider a no-regret adversary that chooses objectives q (1: T ) ∈ ∆( G ) and a learner who first observes q ( t ) and plays a distribution-free bestresponse. The following lemma considers the consequences of these interactions.

Lemma 3.6 (Best-Response vs. No-Regret (BRNR)) . Consider an online multi-objective learning problem ( D , G , H ) , where a learner chose p (1: T ) ∈ ∆( H ) , an adversary chose q (1: T ) ∈ ∆( G ) , and D (1: T ) ∈ D is any sequence. Assume that the adversary is noregret, i.e., Reg( q (1: T ) , {-L D ( t ) , ( · ) ( p ( t ) ) } (1: T ) ) ≤ Tε , and the learner's actions p (1: T ) are distribution-free ε -best-responses to the stochastic costs q (1: T ) , i.e., max x,y L ( x,y ) ,q ( t ) ( p ( t ) ) ≤ max x,y min p ∗ L ( x,y ) ,q ( t ) ( p ∗ ) + ε . Then, the hypotheses p (1: T ) are 2 ε -optimal on D (1: T ) .

The question of which dynamic should be used and their implementation hinges on the type of solution desired and what online learning and best-response guarantees are possible for each player. NRNR dynamics often offer maximum sample efficiency, since calculating ε -best-responses may be more sample intensive, but produce a time-averaged solution. NRBR dynamics, though less sampleefficient due to the adversary's repeated best-response computation, provides a single timestep solution and is crucial for, e.g., deterministic multicalibration. BRNR dynamics, where learners follow (act after the adversary) and have greater ease being no-regret, are crucial for online settings.

3.1 No-regret and Best Response Computation in Multicalibration

In this section, we introduce algorithms for obtaining (weak) no-regret and best response guarantees in multicalibration, so that we can apply the previously discussed dynamics.

Since the adversary picks from the-usually, small-set of objectives G mc , it can be no-regret using standard algorithms like Hedge. However, the learner picks from the-very large-space of all predictors P ; if it used Hedge, its regret would grow linearly in domain size |X| . An important aspect of multicalibration is that its complexity must be independent of the domain size X (while it can depend on the complexity of the subgroups S ). We leverage structural properties of multicalibration objectives (see Appendix C.2 for formal treatment) to obtain generic no-regret and best response algorithms that give the learner domain-independent guarantees. Our first theorem gives a no-(weak)regret learning algorithm for the learner that provides three important properties simultaneously: 1) domain-independent regret-bound that is also logarithmic in k , 2) uses no samples (or knowledge of) the underlying distribution D , and 3) deterministically outputs a deterministic predictor per round. Properties 1-2 lead to fast convergence and low sample complexity in the aforementioned dynamics and property 3 is key for obtaining deterministic multicalibration guarantees (via NRBR).

Theorem 3.7. Consider P the set of k -class predictors and any adversarial sequence of stochastic costs q (1: T ) ∈ ∆( G mc ) , where G mc are the multicalibration objectives (3) . There is a no-regret algorithm that outputs (deterministic) predictors h (1: T ) ∈ P such that Reg weak ( h (1: T ) , { L D,q ( t ) } (1: T ) ) ≤ 2 √ ln( k ) T for every data distribution D . The algorithm does not need any samples from D .

Proof. Consider the following algorithm. At each feature x ∈ X , initialize a Hedge algorithm that picks an action h ( t ) ( x ) ∈ ∆( Y ) at each timestep t ∈ [ T ] . Aggregating each algorithm's action yields our learner's overall action h ( t ) ∈ P . For each x ∈ X , let h ( t +1) ( x ) be the outcome of Hedge at step

t +1 after observing linear loss functions f ( τ ) h ( τ ) ,x : R k → [0 , 1] for τ ∈ [ t ] :

where q ( τ ) i,j,S,v is the probability q ( τ ) assigns to loss glyph[lscript] i,j,S,v . Hedge gives ∑ T t =1 f ( t ) h ( t ) ,x ( h ( t ) ( x )) -min z ∗ ∈ ∆( Y ) ∑ T t =1 f ( t ) h ( t ) ,x ( z ∗ ) ≤ 2 √ ln( k ) T (Lemma 3.1). Since this inequality holds for all x ∈ X ,

by law of total expectation, where the last transition is by defining h ∗ such that h ∗ ( x ) = z ∗ for every x -dependent choice of z ∗ in (5). Add and subtract ∑ T t =1 E ( x,y ) ∼ D [ f ( t ) h ( t ) ,x ( δ y ) ] to obtain

where the last transition is by the fact h ∗ ( x ) = E [ δ y | x ] , ∑ T t =1 E [ f ( t ) h ( t ) ,x ( h ∗ ( x ) -δ y ) ] = 0 . 5 T . Next, we show the LHS of (6) is equivalent to Reg weak ( h (1: T ) , { L D,q ( t ) } (1: T ) ) . First recall from multicalibration objectives that glyph[lscript] i,j,S,v ( h, ( x, y )) = 0 . 5+0 . 5 · i · 1[ h ( x ) ∈ v, x ∈ S ] · ( h ( x ) j -δ y,j ) . Since q ( t ) i,j,S,v is the probability q ( t ) assigns to glyph[lscript] i,j,S,v ( · ) in (4), we have that q ( t ) ( h ( t ) , ( x, y )) = f ( t ) h ( t ) ,x ( h ( t ) ( x ) -δ y ) . Therefore, (6) implies that ∑ T t =1 L D,q ( t ) ( h ( t ) ) ≤ 0 . 5 T +2 √ ln( k ) T . It is left to establish that the min-max baseline of these losses is indeed at least 0 . 5 . This is implied by Fact 2.5 and its proof is deferred to Appendix C.2. Thus Reg weak ( h (1: T ) , { L D,q ( t ) } (1: T ) ) ≤ 2 √ ln( k ) T .

Our second theorem proves the existence of a distribution-free best-response algorithm for the learner, which is key for obtaining online multicalibration guarantees (via BRNR). Note that, as a distribution-free algorithm, it requires no samples to compute these best-responses.

Theorem 3.8. Consider the set of k -class predictors P . Fix an ε > 0 and let q ∈ ∆( G mc ) be a mixture of multicalibration objectives (3) . There always exists a (non-deterministic) predictor p ∈ ∆( P ) that is a distribution-free ε -best-response (7) to the stochastic cost function q ( · , ( x, y )) .

Proof Sketch. At a high level, this statement and proof are similar to the min-max proof of calibration [10, 20], with additional details in Appendix C.2. Let ˜ ∆( Y ) be a finite ε -covering of ∆( Y ) and φ x : ∆( ˜ ∆( Y )) × ∆( Y ) → [ -1 , 1] the bilinear function φ x ( a, b ) = E ( i,j,S,v ) ∼ q, ̂ a ∼ a [ i · 1[ ̂ a ∈ v, x ∈ s ] · ( ̂ a j -b j )] . Consider p ( x ) = arg min a ∈ ∆( ˜ ∆( Y )) max b ∈ ∆( Y ) φ x ( a, b ) . By the minmax theorem, max b ∈ ∆( Y ) φ x ( p ( x ) , b ) = max b ∈ ∆( Y ) min a ∈ ∆( Y ) φ x ( a, b ) ≤ max b ∈ ∆( Y ) min a ∈ ∆( Y ) φ x ( a, b ) + ε .

˜ Thus, max y ∈Y q ( p, ( x, y )) ≤ 0 . 5 · ε +max y ∈Y min h ∗ ∈P q ( h ∗ , ( x, y )) for all x ∈ X .

4 Multicalibration with Game Dynamics

We match and improve a broad set of previous results in multicalibration (See Table 1) that had received individualized and ad hoc treatments in the past. Our work establishes that not only is there a unified approach for obtaining these results but that it all comes back to game dynamics empowered by our no-regret and distribution-free-best response results for multicalibration-Theorems 3.7 and 3.8. Below we focus on three main results highlighting NRNR, NRBR, and BRNR dynamics. We defer the proofs, formal statement of algorithms, and additional results to Appendix D and E.

Multicalibration. Our first algorithm uses no-regret no-regret (NRNR) dynamics to find nondeterministic multicalibrated predictors. This algorithm matches the fastest known sample complexity rates for multicalibration [18, 35] of order O (ln( |S| λ k ) /ε 2 ) . It also improves upon existing fast-rate algorithms of Gupta et al. [18], Noarov et al. [35] by producing predictors with a succinct support and small circuit size. These properties were previously only known to be attained by the less sample efficient multicalibration algorithms of [21]. In this way, Algorithm 1 simultaneously attains the best aspects of the algorithms of [18] and [21].

Theorem 4.1. Fix ε > 0 , λ, k ∈ Z + , set of groups S ⊆ 2 X , and data distribution D . The below algorithm, with probability 1 -δ , returns a non-deterministic k -class predictor that is ( S , ε, λ ) -multicalibrated on D and takes no more than O ( ε -2 (ln( |S| /δ ) + k ln( λ )) ) samples from D .

Construct the problem ( { D } , G mc , P ) from Fact 2.5 and let T = Cε -2 ln( |S| λ k /δ ) for some universal constant C . Over T rounds, have an adversary choose q (1: T ) ∈ ∆( G mc ) by applying Hedge to the costs { 1 -glyph[lscript] ( · ) ( h ( t ) , ( x ( t ) , y ( t ) )) } (1: T ) where ( x ( t ) , y ( t ) ) i.i.d. ∼ D . In parallel, have a a learner choose predictors h (1: T ) ∈ P by applying the no-regret learning algorithm of Theorem 3.7 to the stochastic costs glyph[lscript] (1: T ) , where glyph[lscript] ( t ) i.i.d. ∼ q ( t ) . Return the predictor p = Uniform( h (1: T ) ) . This algorithm is written explicitly in Algorithm 1.

Theorem 4.1-along with all other results in this section-can be rewritten with VC( S ) log(1 /ε ) replacing ln( |S| ) ; this is done by taking a cover of S . We further note that Algorithm 1 can be instantiated with different choices of no-regret algorithms for the adversary and different versions of Theorem 3.7 for the learner. In Section 6, we empirically compare such variants of Algorithm 1.

Our second algorithm uses no-regret best-response (NRBR) dynamics to find deterministic multicalibrated predictors. This algorithm improves on Gopalan et al. [17]'s oracle complexity of O ( k/ε 2 ) with O ( ln( k ) /ε 2 ) , an exponential reduction of the dependence on the number of classes k . 4

Theorem 4.2. Fix ε > 0 , λ, k ∈ Z + , a set of groups S ⊆ 2 X , and a data distribution D . The following algorithm returns a deterministic k -class predictor that is ( S , ε, λ ) -multicalibrated on D and makes O (ln( k ) /ε 2 ) calls to an agnostic learning oracle. Moreover, with probability 1 -δ , the oracle calls can be implemented with ˜ O ( 1 ε 3 ( √ ln( k ) ln( k |S| /δ ) + k ln( λ )) ) samples from D .

Construct the problem ( { D } , G mc , P ) from Fact 2.5 and let T = Cε -2 ln( |S| λ k δ ) for some universal constant C . Over T rounds, have a learner choose predictors h (1: T ) ∈ P by applying the no-regret learning algorithm of Theorem 3.7 to the stochastic costs glyph[lscript] (1: T ) . Have an adversary choose glyph[lscript] (1: T ) by calling an agnostic learning oracle at each t ∈ [ T ] : glyph[lscript] ( t ) = A ε/ 8 (1 -L D, ( · ) ( h ( t ) )) . Using C ln( T/δ ) /ε 2 samples from D , return the predictor h ( t ∗ ) with the lowest empirical multicalibration error. This algorithm is written explicitly in Algorithm 2.

Online multicalibration. Our next algorithm uses best-response no-regret (BRNR) dynamics for online multicalibration, matching the best known regret bounds for online multicalibration [18, 35]. As with Theorem 4.1, our analysis simplifies that of [18, 35] by avoiding exponential potential arguments in favor of no-regret dynamics. We state the following result for binary classification, where Y = { 0 , 1 } but note it trivially extends to multi-class settings. For convenience, we will say that predictors P for binary classification output real-valued h ( x ) ∈ [0 , 1] , where h ( x ) is the predicted probability of class 1 and 1 -h ( x ) is the predicted probability of class 0.

Theorem 4.3. Fix ε > 0 , λ ∈ Z + , and a set of groups S ⊆ 2 X . The following algorithm guarantees ( S , ε, λ ) -online multicalibration with probability 1 -δ .

Construct the online multi-objective learning problem ( D , G mc , P ) in Fact 2.6 and let T = Cε -2 ln( |S| λδ ) for some universal constant C . Over T rounds, have an adversary choose q (1: T ) ∈ ∆( G mc ) by applying Hedge to the costs { 1 -glyph[lscript] ( · ) ( p ( t ) , ( x ( t ) , y ( t ) )) } (1: T ) , where

( x ( t ) , y ( t ) ) i.i.d. ∼ D ( t ) . Have a learner best-respond to each stochastic cost q ( t ) with the ( ε/ 2) -distribution-free best-response p ( t ) ∈ ∆( P ) of Theorem 3.8. This algorithm is written explicitly in Algorithm 3.

The high-probability condition of Theorem 4.3 can be removed if we assume nature presents datapoints rather than data distributions, as is assumed in prior works. Interestingly, Algorithm 3's use of best-response no-regret dynamics exactly recovers the online multicalibration algorithm of [18, 35]. The analysis of Theorem 4.3 is, however, significantly simpler because we make explicit the role of the no-regret dynamics, whereas [18, 35] use potential arguments that ultimately prove no-regret dynamics and the multiplicative weights algorithm from scratch.

Additional results. We can also use game dynamics to improve guarantees for various other multicalibration settings and considerations. We can use Algorithms 1 and 2 to achieve domainindependent sample complexity rates for agnostic multicalibration (Theorem E.9), improving upon Shabat et al. [38]'s uniform convergence guarantees. Plugging in an online learning algorithm with second-order regret bounds allows us to achieve the first non-trivial sample complexity guarantees for conditional multicalibration in the batch setting (Theorem E.3) and a multicalibration algorithm that-with a minor less than cube-root increase in sample complexity-guarantees an error tolerance for each group that scales with the square-root of the group's probability mass (Theorem 4.4). We highlight the latter result in the following theorem.

Theorem 4.4. Fix ε > 0 , λ, k ∈ Z + , two sets of groups S ⊆ 2 X , and a data distribution D . There is an algorithm that, with probability at least 1 -δ , takes O (ln( k ) · (ln( |S| /εδ )+ k ln( λ )) /ε 4 ) samples from D and returns a deterministic k -class predictor h satisfying

Plugging in an online learning algorithm with strongly adaptive regret bounds allows us to parallelize the moment multicalibration algorithm of Jung et al. [22] (Theorem E.13).

5 Other Fairness Notions

This general framework of approaching multi-objective learning with game dynamics can be extended beyond multicalibration to derive results on multi-group learning. Recall that, in agnostic multiobjective learning, the trade-off between objectives is arbitrated by the worst-off objective, which can be suboptimal when some objectives are inherently more difficult. An alternative is to, given a problem ( D , G , H ) , define a competitor class H ′ and try to learn a hypothesis h so that there is no objective that a competitor performs significantly better on, i.e. L ∗ H ′ ( h ) -min h ∗ ∈H L ∗ H ′ ( h ∗ ) ≤ ε where L ∗ H ′ ( h ) := max D ∈D max glyph[lscript] ∈G L D,glyph[lscript] ( h ) -min h ∗ ∈H ′ L D,glyph[lscript] ( h ∗ ) . We refer to such a solution h as being ε -competitive. A simple modification to multi-objective learning can find such a solution: replace the objectives G with new objectives G ′ := { 1 2 (1 + glyph[lscript] ( · ) -glyph[lscript] ( h ′ )) | glyph[lscript] ∈ G , h ′ ∈ H ′ } and solve as usual. Since the sample complexity of multi-objective learning is O ( ε -2 (ln( |H| ) + |D| ln( |D| |G| /δ ))) [19], we have the below sample complexity bound on finding ε -competitive solutions.

Theorem 5.1. Consider a multi-objective learning problem ( D , G , H ) and competitor class H ′ . There is an algorithm that takes O ( ε -2 (ln( |H| ) + |D| ln( |D| |H ′ | |G| /δ ))) samples and with probability 1 -δ returns a solution p ∈ ∆( H ) that is ε -competitive against H ′ .

Consider the multi-group learning problem, where we seek to simultaneously minimize a general loss function on different subsets of the domain [37].

Definition 5.2. Fix ε > 0 , a set of groups S ⊆ 2 X , a hypothesis class H and a loss glyph[lscript] : H × ( X × Y ) → [0 , 1] . An ε -optimal solution to the multi-group learning problem ( S , H ) is a randomized hypothesis p ∈ ∆( H ) that satisfies, for all S ∈ S , E [ glyph[lscript] ( p, ( x, y )) · 1[ x ∈ S ]] ≤ min h ∗ ∈H E [ glyph[lscript] ( h ∗ , ( x, y )) · 1[ x ∈ S ]] + ε . We always assume such a hypothesis p exists in class H .

A (near) optimal sample complexity for multi-group learning of O ( ln( |S| |H| ) /ε 2 ) was attained by [39] using a reduction to sleeping experts. [39] also asked whether there exists a simpler optimal algorithm that does not rely on sleeping experts. We answer this affirmatively by designing an optimal

algorithm that just runs two Hedge algorithms. The following is a direct implication of Lemma 3.3 and the observation that multi-group learning directly reduces to learning an ε -competitive solution.

Theorem 5.3. Fix a set of groups S ⊆ 2 X , loss glyph[lscript] : Y × Y → [0 , 1] , hypothesis class H : X → Y , and distribution D . There is a no-regret no-regret algorithm that takes 2 T = O ( ln( |S| |H| ) /ε 2 ) samples from D and returns an ε -optimal solution to the multi-group learning problem ( S , H ) .

6 Empirical Results

In this section, we study the empirical performance of multicalibration algorithms on the UCI Adult Income dataset [26], a real-world dataset for predicting individuals' incomes based on the US Census. We defer additional results, datasets, and methods to the Appendix G.

Experiment setup. This experiment, summarized in Table 2, aims to learn a predictor on the UCI dataset that predicts the 'income' label and is multicalibrated on the values of eight other labeled attributes including 'age'. We discretize with 0 . 1 -width bins ( λ = 10 ), and perform random 80-20 train/test splits of the dataset. This results in approximately 24000 training samples, 6000 test samples, and 130 groups. Each multicalibration algorithm is evaluated on 10 seeds, each after 50 iterations, with performance measured by their average iterate's and last iterate's multicalibration violations.

We study four multicalibration algorithms based on no-regret best-response dynamics, which use an empirical risk minimizer as the adversary and implements either Hedge [14], Prod [28], Optimistic Hedge (OptHedge) [36], or Gradient Descent (GD) as the learner. We study two algorithms based on no-regret no-regret dynamics, which plays against itself either Hedge (Hedge-Hedge) or Optimistic Hedge (OptHedge-OptHedge). Learning rate decay is tuned on the training set by sweeping over [0 . 8 , 0 . 85 , 0 . 9 , 0 . 95] for the learner and [0 . 9 , 0 . 95 , 0 . 98 , 0 . 99] for the adversary.

One's choice of no-regret algorithm matters. The original multicalibration algorithm of [21], which is based on gradient descent, consistently attains the worst multicalibration errors, both in terms of average-iterate and last-iterate. This is consistent with gradient descent being a theoretically less effective no-regret algorithm, due to instability near the boundaries of a probability simplex. Due to the superficial similarity between boosting and multicalibration, the field has already begun adopting multicalibration algorithms with Hedge's multiplicative updates rather than gradient descent's additive ones [25]. Our findings offer the first theoretical and empirical endorsement of this shift.

The last iterates of no-regret no-regret dynamics are surprisingly multicalibrated. The algorithms based on no-regret no-regret dynamics, namely Hedge-Hedge and OptHedge-OptHedge, consistently yield not only among the most multicalibrated randomized predictors (with their average iterate) but also the most multicalibrated deterministic predictors (with their last iterate). Note that these algorithms only enjoy a theoretical advantage over no-regret best-response algorithms in terms of average iterate guarantees. This does not appear to be an artifact of early stopping or learning rates (Figure 1), but may rather indicate that their more stable adversary updates provide regularization.

Acknowledgements. This work was supported in part by the National Science Foundation under grant CCF-2145898, a C3.AI Digital Transformation Institute grant, and the Mathematical Data Science program of the Office of Naval Research. This work was partially done while Haghtalab and Zhao were visitors at the Simons Institute for the Theory of Computing.

References

R. Bassily, K. Nissim, A. D. Smith, T. Steinke, U. Stemmer, and J. R. Ullman. Algorithmic stability for adaptive data analysis. Proceedings of the SIAM Journal on Computing , 50(3), 2021.

O. Bastani, V. Gupta, C. Jung, G. Noarov, R. Ramalingam, and A. Roth. Practical adversarial multivalid conformal prediction, 2022.

A. Blum and T. Lykouris. Advancing subgroup fairness via sleeping experts. In T. Vidick, editor, Proceedings of the ACM Conference on Innovations in Theoretical Computer Science Conference (ITCS) , 2020.

A. Blum, N. Haghtalab, A. D. Procaccia, and M. Qiao. Collaborative PAC learning. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems 30 , pages 2392-2401. Curran Associates, Inc., 2017.

A. Daniely, A. Gonen, and S. Shalev-Shwartz. Strongly adaptive online learning. In F. R. Bach and D. M. Blei, editors, Proceedings of the International Conference on Machine Learning (ICML) , volume 37 of Proceedings of the Journal of Machine Learning Research , pages 1405-1411. JMLR.org, 2015.

A. P. Dawid. The well-calibrated Bayesian. Journal of the American Statistical Association , 77 (379):605-610, 1982.

C. Dwork, M. P. Kim, O. Reingold, G. N. Rothblum, and G. Yona. Outcome indistinguishability. In Proceedings of the Annual ACM SIGACT Symposium on Theory of Computing (STOC) , pages 1095-1108. ACM, 2021.

C. Dwork, D. Lee, H. Lin, and P. Tankala. New insights into multicalibration, 2023.

D. Foster and R. Vohra. Calibrated learning and correlated equilibrium. Games and Economic Behavior , 21:40-55, 1997.

D. P. Foster. A proof of calibration via blackwell's approachability theorem. Games and

Economic Behavior , 29(1-2):73-78, 1999.

D. P. Foster and S. M. Kakade. Calibration via regression. In G. Seroussi and A. Viola, editors, Proceedings of 2006 IEEE Information Theory Workshop , pages 82-86. IEEE, 2006.

D. P. Foster and R. V. Vohra. Asymptotic calibration. Biometrika , 85(2):379-390, 1998.

Y. Freund and R. E. Schapire. Game theory, on-line prediction and boosting. In A. Blum and M. J. Kearns, editors, Proceedings of the Conference on Learning Theory (COLT) , pages 325-332. ACM, 1996.

Y. Freund and R. E. Schapire. A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences , 55(1):119-139, 1997.

P. Gaillard, G. Stoltz, and T. van Erven. A second-order bound with excess losses. In M. Balcan, V. Feldman, and C. Szepesvári, editors, Proceedings of the Conference on Learning Theory (COLT) , volume 35 of Proceedings of the Journal of Machine Learning Research , pages 176-196. JMLR.org, 2014.

P. Gopalan, A. T. Kalai, O. Reingold, V. Sharan, and U. Wieder. Omnipredictors. In M. Braverman, editor, Proceedings of the ACM Conference on Innovations in Theoretical Computer Science Conference (ITCS) , pages 79:1-79:21, 2022.

P. Gopalan, M. P. Kim, M. Singhal, and S. Zhao. Low-degree multicalibration. In P.-L. Loh and M. Raginsky, editors, Proceedings of the Conference on Learning Theory (COLT) , pages 3193-3234. PMLR, 2022.

V. Gupta, C. Jung, G. Noarov, M. M. Pai, and A. Roth. Online multivalid learning: Means, moments, and prediction intervals. In M. Braverman, editor, Proceedings of the ACM Conference on Innovations in Theoretical Computer Science Conference (ITCS) , pages 82:1-82:24. Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2022.

N. Haghtalab, M. I. Jordan, and E. Zhao. On-demand sampling: Learning optimally from multiple distributions. In S. Bengio, H. M. Wallach, H. Larochelle, K. Grauman, N. CesaBianchi, and R. Garnett, editors, Advances in Neural Information Processing Systems , 2022.

S. Hart. Calibrated forecasts: The minimax proof, 2022.

U. Hebert-Johnson, M. P. Kim, O. Reingold, and G. N. Rothblum. Multicalibration: Calibration for the (computationally-identifiable) masses. In J. G. Dy and A. Krause, editors, Proceedings of the International Conference on Machine Learning (ICML) , Proceedings of Machine Learning Research, pages 1944-1953. PMLR, 2018.

C. Jung, C. Lee, M. M. Pai, A. Roth, and R. Vohra. Moment multicalibration for uncertainty estimation. In M. Belkin and S. Kpotufe, editors, Proceedings of the Conference on Learning Theory (COLT) , Proceedings of Machine Learning Research, pages 2634-2678. PMLR, 2021.

C. Jung, G. Noarov, R. Ramalingam, and A. Roth. Batch multivalid conformal prediction, 2022.

S. Kale. Efficient Algorithms Using the Multiplicative Weights Update Method . Princeton University, 2007.

M. P. Kim, A. Ghorbani, and J. Y. Zou. Multiaccuracy: Black-box post-processing for fairness in classification. In V. Conitzer, G. K. Hadfield, and S. Vallor, editors, Proceedings of the AAAI Conference on Artificial Intelligence (AAAI) , pages 247-254. ACM, 2019.

R. Kohavi and B. Becker. UCI adult dataset. UCI Machine Learning Repository, May 1996.

M. Koklu and I. A. Ozkan. Multiclass classification of dry beans using computer vision and machine learning techniques. Computers and Electronics in Agriculture , 174:105507, 2020.

N. Littlestone. Learning quickly when irrelevant attributes abound: a new linear-threshold algorithm (extended abstract). In Proceedings of the Symposium on Foundations of Computer Science (FOCS) , pages 68-77. IEEE Computer Society, 1987.

S. Mannor and O. Shamir. From bandits to experts: on the value of side-observations. In J. Shawe-Taylor, R. S. Zemel, P. L. Bartlett, F. C. N. Pereira, and K. Q. Weinberger, editors, Advances in Neural Information Processing Systems , pages 684-692, 2011.

F. McSherry and K. Talwar. Mechanism design via differential privacy. In Proceedings of the Symposium on Foundations of Computer Science (FOCS) , pages 94-103. Proceedings of the IEEE Press, 2007.

M. Mohri, G. Sivek, and A. T. Suresh. Agnostic federated learning. In K. Chaudhuri and R. Salakhutdinov, editors, Proceedings of the International Conference on Machine Learning (ICML) , volume 97 of Proceedings of Machine Learning Research , pages 4615-4625. PMLR, 2019.

S. Moro, P. Cortez, and P. Rita. A data-driven approach to predict the success of bank telemarketing. Decision Support Systems , 62:22-31, June 2014.

A. Nemirovski, A. Juditsky, G. Lan, and A. Shapiro. Robust stochastic approximation approach to stochastic programming. SIAM Journal on Optimization , 19(4):1574-1609, 2009.

H. L. Nguyen and L. Zakynthinou. Improved algorithms for collaborative PAC learning. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, Advances in Neural Information Processing Systems 31 , pages 7642-7650. Curran Associates, Inc., 2018.

G. Noarov, M. M. Pai, and A. Roth. Online multiobjective minimax optimization and applications, 2021.

A. Rakhlin and K. Sridharan. Online learning with predictable sequences. In S. Shalev-Shwartz and I. Steinwart, editors, Proceedings of the Conference on Learning Theory (COLT) , volume 30 of Proceedings of the Journal of Machine Learning Research , pages 993-1019. JMLR.org, 2013.

G. N. Rothblum and G. Yona. Multi-group agnostic PAC learnability. In M. Meila and T. Zhang, editors, Proceedings of the International Conference on Machine Learning (ICML) , volume 139 of Proceedings of Machine Learning Research , pages 9107-9115. PMLR, 2021.

E. Shabat, L. Cohen, and Y. Mansour. Sample complexity of uniform convergence for multicalibration. In H. Larochelle, M. Ranzato, R. Hadsell, M.-F. Balcan, and H.-T. Lin, editors, Advances in Neural Information Processing Systems , 2020.

C. J. Tosh and D. Hsu. Simple and near-optimal algorithms for hidden stratification and multigroup learning. In K. Chaudhuri, S. Jegelka, L. Song, C. Szepesvari, G. Niu, and S. Sabato, editors, Proceedings of the International Conference on Machine Learning (ICML) , volume 162 of Proceedings of Machine Learning Research , pages 21633-21657. PMLR, 2022.

Contents

A Additional Background

In this section, we supplement our discussion of online learning with some additional notations and results.

No-regret algorithms. In the following lemma, we state the regret bound of the Hedge algorithm [14] for general choices of the learning rate η ∈ (0 , 1) .

Lemma A.1 (Hedge [24]) . Consider an online learning problem on the simplex ∆ k and any adversarial sequence of linear cost functions c (1: T ) . If Hedge, with learning rate η ∈ (0 , 1) , outputs a (1: T ) , then Reg( a (1: T ) , c (1: T ) ) ≤ Tη + ln( k ) η . If η = √ ln( |A| ) /T , Reg( a (1: T ) , c (1: T ) ) ≤ 2 √ ln( k ) T .

It is helpful to note that the learning rate of the Hedge algorithm also bounds how far its iterates move in the primal space.

Lemma A.2 (Hedge Iterate Stability) . Consider an online learning problem on the interval [0 , 1] and any sequence of linear costs. Let a (1: T ) ∈ [0 , 1] be the actions that get picked by Hedge instantiated with learning rate η ∈ (0 , 1) . Then, for every timestep t ∈ [ T ] , ∣ ∣ a (1) -a ( t ) ∣ ∣ ≤ 2 ηT .

Proof. By the triangle inequality, it suffices to prove that the learner's actions move by at most 2 η at each timestep. That is, ∣ ∣ a ( t +1) -a ( t ) ∣ ∣ ≤ 2 η at every timestep t ∈ [ T ] . By definition of the Hedge algorithm, the learner's action at timestep t is given by

We can rearrange this as a ( t +1) = a ( t ) · α · exp( -ηc ( t ) (1)) , where α is a ratio of normalization terms defined as

Note that α ≥ 1 . Let us write x 1 = exp( -η ∑ t -1 τ =1 c ( τ ) (0)) , x 2 = exp( -η ∑ t -1 τ =1 c ( τ ) (1)) , x 3 = exp( -ηc ( t ) (0)) and x 4 = exp( -ηc ( t ) (1)) , noting that the values x 1 , x 2 , x 3 , x 4 are all positive. Observe that x 1 + x 2 x 1 · x 3 + x 2 · x 4 ≤ max { 1 x 3 , 1 x 4 } . To see this, suppose without loss of generality that x 4 ≥ x 3 . Then x 2 ≤ x 2 · x 4 x 3 , which implies that x 1 + x 2 ≤ x 1 · x 3 + x 2 · x 4 x 3 , or equivalently, x 1 + x 2 x 1 · x 3 + x 2 · x 4 ≤ 1 x 3 . Substituting in our values for x 1 , x 2 , x 3 , x 4 , we therefore have

The second inequality is because costs are bounded in [0 , 1] .

This gives the iterate movement bound of

Here, the second inequality applies the fact that | e -x -1 | ≤ | x | for any x ≥ 0 to the right-hand value. Since η ∈ (0 , 1) , we can use the fact that | exp( η ) -1 | ≤ 2 η . Thus, we attain the desired bound ∣ ∣ a ( t +1) -a ( t ) ∣ ∣ ≤ 2 η .

Wecan modify the Hedge algorithm to obtain strongly adaptive regret bounds that provides guarantees for every contiguous interval.

Lemma A.3 (Strongly Adaptive Regret [5]) . Consider an online learning problem on the simplex ∆ k . There is a modified Hedge algorithm [5] that, for any adversarial sequence of linear costs c (1: T ) ,

outputs actions a (1: T ) such that, for every interval T 1 < T 2 ≤ T ,

There are also online learning algorithms that provide second-order regret bounds. We present the regret bound of one such algorithm, Prod [15].

Lemma A.4 (Second-Order Regret Bound of Prod [15]) . Consider an online learning problem on the simplex A = ∆ k and any adversarial sequence of linear costs c (1: T ) . If the Prod algorithm of [15] is used and outputs the actions a (1: T ) , then

We also know that there are adversarial bandit algorithms with sublinear regret guarantees.

Lemma A.5 (Semi-Bandit Regret Bounds [19]) . Consider an online learning problem on the simplex ∆ k , and any adversarial sequence of linear costs c (1: T ) . Let I be a partition of [ k ] into r groups. There is a high-probability variant of [29]'s ELP algorithm that, for any adversarial sequence of costs c (1: T ) , outputs a sequence of mixtures a (1: T ) ∈ ∆ k such that, with probability at least 1 -δ ,

Moreover, after each time the algorithm chooses a ( t ) , the algorithm samples an integer i ( t ) ∼ a ( t ) (unseen by the adversary). Let I ( i ( t ) ) is the group in I that i ( t ) belongs to. The algorithm will only ever observe the components of its cost vector corresponding to I ( i ( t ) ) : { c ( τ ) ( δ i ) | i ∈ I ( i ( t ) ) } ( t ) .

Best responses. An action a is an ε -best response to a cost function c : A → [0 , 1] if c ( a ) ≤ min a ∗ ∈A c ( a ∗ ) + ε . An agnostic learning oracle A ε : ( A → [0 , 1]) × 2 A →A is a function that, given a cost function c and subset of actions A ′ ⊆ A , computes an ε -best response A ε ( c, A ′ ) to c from A ′ . When A ′ = A , we may write A ε ( c ) without the second argument. Agnostic learning oracles are given this name because they are usually used to find best-responses to the expected values of stochastic cost functions using some number of samples. Through this paper, we design learning algorithms that only interact with a data distribution through querying an agnostic learning oracle; the oracle complexity of such an algorithm is defined as the number of agnostic oracle calls that the algorithm makes. Using standard sample complexity bounds, all our algorithms also have a corresponding sample complexity that circumvent the use of agnostic learning oracles.

Another concept we encounter in online settings is the distribution-free best response . An action a is a distribution-free ε -best response to a stochastic cost c if

Best-response algorithms. We can efficiently find an ε best-response to the expected value of a stochastic cost function using agnostic learning oracles. For example, computing a single ε bestresponse requires at most O (ln( |A| ) /ε 2 ) samples by uniform convergence. In game dynamics, we will often need to provide best-responses for a sequence of stochastic cost functions. These sequences are usually adaptive , in that which stochastic cost functions appear later in the sequence depending on how we responded to previous stochastic cost functions. Adaptive data analysis provides sample-efficient algorithms for these settings.

Lemma A.6 (Adaptive Data Analysis [1] Corollary 6.4) . There is an algorithm that, for any adaptive sequence of stochastic costs c (1: T ) , is guaranteed with probability 1 -δ to ε -best respond to each cost in { L D,c ( t ) } (1: T ) while drawing at most

samples from D . Here, ˜ O ( · ) suppresses ln(1 /ε ) and ln 1 / 2 (1 /δ ) factors.

Remark A.7. It often suffices, for our results, that a sequence of actions a (1: T ) is on average ε -best responding to a cost sequence c (1: T ) ; that is, ∑ T t =1 c ( t ) ( a ( t ) ) ≤ ∑ T t =1 min a ∗ ∈A c ( t ) ( a ∗ ) + Tε . Thus, it may be possible to use more efficient minimization oracles than Lemma A.6 in our algorithms.

B Proofs for Section 2

This section proves Fact 2.5 and Fact 2.6 from Section 2.

Fact 2.5. Let D be a data distribution for some k -class prediction problem and fix ε > 0 , λ ∈ Z + , and a set of groups S ⊆ 2 X . For every direction i ∈ {± 1 } , level set v ∈ V k λ , group S ∈ S , and class j ∈ [ k ] , we define an objective glyph[lscript] i,j,S,v : P → [0 , 1] where

and G mc := { glyph[lscript] i,j,S,v } i,j,S,v is the set of these objectives. Predictor p ∈ ∆( P ) is a ε -optimal solution to the multi-objective problem ( { D } , G mc , P ) if and only if p is ( S , 2 ε, λ ) -multicalibrated for D .

Proof. In the multi-objective learning problem ( D , G mc , P ) , the multi-objective value of a predictor p , L ∗ ( p ) := max D ∗ ∈D ,glyph[lscript] ∗ ∈G L D ∗ ,glyph[lscript] ∗ ( p ) , is exactly the (rescaled and shifted) magnitude of the predictor's multicalibration violation. Formally,

Since we can write the absolute value of an abstract value w as | w | = max i ∈{± 1 } i · w , we next observe that the optimal multi-objective value of the problem ( D , G mc , P ) is 0 . 5 .

This fact is because multicalibration objectives G mc are symmetric around 0 . 5 (where i = +1 and i = -1 penalize over and under estimation), leading to the worst loss to be at least 0 . 5 . Furthermore, the Bayes classifier h ∗ neither overestimates nor underestimates the label distribution and therefore achieves the loss of 0 . 5 exactly. Combining these inequalities, we have

Therefore L ∗ ( p ) -min h ∗ ∈P L ∗ ( h ∗ ) = ε if and only if our multicalibration violation is 2 ε .

Fact 2.6. Let D (1: T ) be data distributions for some k -class prediction problem, D be the set of all data distributions, and G mc be as defined in (3) . Fix ε > 0 , λ ∈ Z + , and a set of groups S ⊆ 2 X . A sequence of predictors p (1: T ) ∈ ∆( P ) is ε -optimal on D (1: T ) for the online multi-objective learning problem ( D , G mc , P ) if and only if p (1: T ) is ( S , 2 ε, λ ) -online multicalibrated on D (1: T ) .

Proof. Note that online multicalibration is an online multi-objective learning problem by construction. To analyze its optimality condition, we expand the definition of the objectives in G mc as

We again observe that the optimal value of the problem is 0 . 5 ; formally,

This is because choosing h ∗ to be the Bayes classifier for D ∗ achieves the value of 0 . 5 , which given the absolute value in the second term is the minimum achievable value. Thus, p (1: T ) is ε -optimal on D (1: T ) if and only if p (1: T ) is ( S , 2 ε, λ ) -optimal on D (1: T ) .

C Proofs for Section 3

We first recall our characterization of multi-objective learning as a two-player zero-sum game. In this game, a learner player chooses a non-deterministic hypothesis p ∈ ∆( H ) and an adversary player chooses a joint distribution over data distributions and objectives q ∈ ∆( D×G ) . The payoff of the game is the expected objective value L q ( p ) . In single-distribution multi-objective learning problems where the adversary only has one data distribution D to choose from, we sometimes write q ∈ ∆( G ) for simplicity. In online multi-objective learning, the adversary does not have control over which data distribution D is chosen by nature. In these cases, the adversary only chooses objectives q ∈ ∆( G ) and the game payoff function becomes L D,q ( p ) .

C.1 Game Dynamics

We now prove formalizations of Lemma 3.3, Lemma 3.4, Lemma 3.5, and Lemma 3.6 from Section 3.

The following is a formal restatement of Lemma 3.3.

LemmaC.1 (No-Regret vs. No-Regret) . Consider a multi-objective learning problem ( D , G , H ) , and two sequences p (1: T ) ∈ ∆( H ) and q (1: T ) ∈ ∆( D × G ) . Suppose Reg B ( p (1: T ) , {L q ( t ) ( · ) } (1: T ) ) ≤ Tε and Reg( q (1: T ) , { 1 -L ( · ) ( p ( t ) ) } (1: T ) ) ≤ Tε where B ∈ R . Then, the non-deterministic hypothesis p ∈ ∆( H ) defined as p := Uniform ( p (1: T ) ) satisfies L ∗ ( p ) ≤ B +2 ε . If the baseline B is the min-max baseline B weak, then p is a 2 ε -optimal solution for the problem ( D , G , H ) .

Proof. By connecting the adversary's regret bound and the learner's (weak) regret bounds of

we directly observe that max D ∗ ∈D ,glyph[lscript] ∗ ∈G 1 T ∑ T t =1 L D ∗ ,glyph[lscript] ∗ ( p ( t ) ) ≤ 2 ε + B . Linearity of expectation allows us to equate 1 T ∑ T t =1 L D ∗ ,glyph[lscript] ∗ ( p ( t ) ) = L D ∗ ,glyph[lscript] ∗ ( p )) , which yields our first claim that L ∗ ( p ) ≤ 2 ε + B . The second claim just plugs B weak into the previous inequality to obtain the definition of a 2 ε -optimal solution.

The following is a formal restatement of Lemma 3.4.

Lemma C.2 (No-Regret vs. Best-Response) . Consider a multi-objective learning problem ( D , G , H ) and two sequences p (1: T ) ∈ ∆( H ) and q (1: T ) ∈ ∆( D × G ) . Suppose Reg B ( p (1: T ) , {L q ( t ) ( · ) } (1: T ) ) ≤ Tε where B ∈ R . Further suppose q (1: T ) are, on average, ε best-responses to the cost functions { 1 - L ( · ) ( p ( t ) ) } (1: T ) . Then there exists a t ∈ [ T ] where L ∗ ( p ( t ) ) ≤ B +2 ε . If B = B weak, then p ( t ) is a 2 ε -optimal solution for the problem ( D , G , H ) .

Proof. Assume the contrary, namely that L ∗ ( p ( t ) ) > B +2 ε at all t ∈ [ T ] . Then

gives us a contradiction that T (2 ε -2 ε ) > 0 . This proves our first claim. The second claim follows by plugging B = B weak into our inequality L ∗ ( p ( t ) ) ≤ B +2 ε to obtain the definition of a 2 ε -optimal solution.

The following is a formal restatement of Lemma 3.5.

Lemma C.3. Consider a multi-objective learning problem ( D , G , H ) and a sequence p (1: T ) ∈ ∆( H ) of hypotheses where at least one hypothesis p ( t ) is ε -optimal. We can find a 3 ε -optimal solution p ( t ∗ ) ∈ p (1: T ) with probability at least 1 -δ by taking only O ( ε -2 ln( T |D| · |G| /δ )) samples from each distribution D ∈ D . If we further have access to a sequence q (1: T ) ∈ ∆( D × G ) of ε best-responses to the cost functions { 1 -L ( · ) ( p ( t ) ) } (1: T ) , then a 5 ε -optimal solution can be found with only O ( ε -2 ln( T/δ )) samples from each D ∈ D .

Proof. Let the ε -optimal solution be denoted p ( t good ) . This claim is a simple uniform convergence argument. Suppose that N i.i.d. samples x D are drawn from each distribution D ∈ D . We can then use ̂ L D,glyph[lscript] ( p ) to denote the empirical value of the loss function glyph[lscript] of hypothesis p on distribution D , as approximated by the samples x D . We similarly define ̂ L ∗ as the empirical analog of the multi-objective value L ∗ . Since the range of each objective is bounded in [0 , 1] , Chernoff's bound guarantees that for any q ∈ ∆( D × G ) and p ∈ ∆( H ) we have Pr x [∣ ∣ ∣ ̂ L q ( p ) -L q ( p ) ∣ ∣ ∣ ≥ ε ] ≤ 2 exp ( -2 Nε 2 ) . Taking a union bound over each D ∈ D , t ∈ [ T ] and glyph[lscript] ∈ G gives

Letting t ∗ = arg min t ∈ [ T ] ̂ L ∗ ( p ( t ) ) and N = O ( ε -2 ln( T |D| · |G| /δ ) ) guarantees

with probability at least 1 -δ . We can therefore return p ( t ∗ ) as our solution.

To prove our second claim, take instead a union bound over each t ∈ [ T ] so that

Letting t ∗ = arg min t ∈ [ T ] L q ( t ) ( p ( t ) ) and N = O ( ε -2 ln( T/δ ) ) guarantees with probability at least 1 -δ that we can return p ( t ∗ ) as our solution, as

The following is a formal restatement of Lemma 3.6.

Lemma C.4 (Best-Response vs. No-Regret) . Consider a online multi-objective learning problem ( D , G , H ) and the sequences p (1: T ) ∈ ∆( H ) , q (1: T ) ∈ ∆( G ) , and D (1: T ) ∈ D . Suppose Reg( q (1: T ) , { 1 - L D ( t ) , ( · ) ( p ( t ) ) } (1: T ) ) ≤ Tε and p (1: T ) ∈ ∆( H ) are distribution-free ε ′ bestresponses to q (1: T ) ( · , ( x, y )) . Then, p (1: T ) are ( ε + ε ′ ) -optimal on D (1: T ) .

Proof. Let D ∗ be the set of all data distributions over X × Y . By linearity of expectation, the maximum arg max D ∈D ∗ L D ( p ( t ) ) can always be attained on a degenerate distribution supported only on a single point ( x, y ) . Thus, the learner's distribution-free best-response guarantee provides the bound

We can obtain our desired claim by recovering (2) with a triangle inequality between the adversary's regret bound and a summation of the previous inequality over t ∈ [ T ] :

C.2 No-Regret and Best-Response Computation in Multicalibration

In this section, we prove generalizations of Theorem 3.7 and Theorem 3.8 from Section 3, which describe why multicalibration objectives are amenable to efficient online learning, to a more general class of multi-objective learning problems. We refer to such problems as having separable objectives .

Definition C.5. Consider a multi-objective learning problem ( D , G , H ) where H is the set of all functions of form h : X → W and W is some convex space (and not necessarily the label space Y ). We say the objectives G of such a problem are separable if every glyph[lscript] ∈ G is of form

where f glyph[lscript] : X × W → W and g glyph[lscript] : Y → W are arbitrary functions and c is some constant offset.

Wewill see that, like with multicalibration objectives, having a learner's cost functions all be separable objectives introduces two major advantages generally.

Advantage 1: There exist no-regret learning strategies that do not require one to randomize their actions, or even sample any data, yet still guarantee domain-independent regret bounds. The following theorem, which proves this advantage of separable objectives, is a generalization of Theorem 3.7.

Theorem C.6. Consider a multi-objective learning problem ( D , G , H ) where all objectives in G are separable and all distributions in D are absolutely continuous with respect to a common distribution D ∗ . Let A be an online algorithm that, for any linear costs c (1: T ) , outputs actions a (1: T ) ∈ W where Reg( a (1: T ) , c (1: T ) ) ≤ R ( T ) . If |D| > 1 , further assume Reg( a (1: T ) , c (1: T ) ) ≤ R ( T ) · max a ∗ ∈W √ 1 T ∑ T t =1 ( c ( t ) ( a ( t ) ) -c ( t ) ( a ∗ )) 2 . Then for any stochastic costs q (1: T ) ∈ ∆( G ) and distributions D (1: T ) ∈ D , the below algorithm outputs predictors h (1: T ) ∈ H where Reg B ∗ ( h (1: T ) , {L D ( t ) ,q ( t ) ( · ) } (1: T ) ) ≤ √ |D| R ( T ) and B ∗ is the baseline

At each timestep t , construct the predictor h ( t +1) ( x ) by setting, for all x ∈ X , h ( t +1) ( x ) := A ( c (1) x , . . . , c ( t ) x ) where c ( τ ) x ( w ) := 1 2 · dD ( τ ) ( x ) dD ∗ ( x ) · ( 1 + E glyph[lscript] ∼ q ( τ ) [ f glyph[lscript] ( x, h ( τ ) ( x )) ] · w ) , where dD ( τ ) ( x ) dD ∗ ( x ) is the Radon-Nikodym derivative of D ( τ ) with respect to D ∗ at x .

Proof. First, suppose that D = { D } , that is |D| = 1 . Then, for any x ∈ X , c (1: T ) x are linear costs and thus A guarantees Reg( { h ( t ) ( x ) } (1: T ) , c (1: T ) x ) ≤ R ( T ) . By the law of total expectation,

2 R ( T )

If |D| > 1 , A further guarantees

For simplicity, we assumed above that D (1: T ) each have discrete support. The law of total expectation gives

To bound the left-hand term, we can apply Cauchy-Schwartz to get

where T D is the number of timesteps t ∈ [ T ] where D ( t ) = D . Thus, whether |D| = 1 or |D| > 1 , we have

Adding and subtracting the term ∑ T t =1 E ( x,y ) ∼ D ( t ) ,glyph[lscript] ∼ q ( t ) [ f glyph[lscript] ( x, h ( t ) ( x ))( g glyph[lscript] ( y )) ] , we have

Adding the constant c to both sides, we have

We can directly prove Theorem 3.7 by instantiating the algorithm of Theorem C.6 with Hedge as the no-regret learning algorithm A .

Theorem 3.7. Consider P the set of k -class predictors and any adversarial sequence of stochastic costs q (1: T ) ∈ ∆( G mc ) , where G mc are the multicalibration objectives (3) . There is a no-regret algorithm that outputs (deterministic) predictors h (1: T ) ∈ P such that Reg weak ( h (1: T ) , { L D,q ( t ) } (1: T ) ) ≤ 2 √ ln( k ) T for every data distribution D . The algorithm does not need any samples from D .

Proof. First, we observe that every objective in multicalibration is separable. Specifically, for every i ∈ {± 1 } , j ∈ [ k ] , S ∈ S , v ∈ V k λ , the corresponding objective glyph[lscript] i,j,S,v ∈ G mc can be written in the form glyph[lscript] i,j,S,v ( h, ( x, y )) = c + f i,j,S,v ( x, h ( x )) · ( h ( x ) -g i,j,S,v ( y )) where c = 0 . 5 , W = ∆ k ∪-∆ k , g i,j,S,v ( y ) = δ y and f i,j,S,v ( x, h ( x )) = δ j · 0 . 5 · i · 1[ h ( x ) ∈ v, x ∈ S ] . Next, we recall that by Lemma A.1, the Hedge algorithm guarantees a regret bound of 4 √ ln( k ) T for any linear costs on W = ∆ k ∪ -∆ k (Lemma A.1). The algorithm of Theorem C.6, for any stochastic costs q (1: T ) ∈ ∆( G mc ) , outputs h (1: T ) where

if we choose h ∗ ( x ) = E ( x,y ) ∼ D [ g ( t ) ( y ) | x ] . Since the adversary is choosing from objectives symmetric around c , the min-max baseline B weak is at least c , meaning that Reg weak ( h (1: T ) , {L D,q ( t ) ( · ) } (1: T ) ) ≤ 8 √ ln( k ) T .

Advantage 2: There (almost) always exists distribution-free best responses. This advantage is what allows multicalibration to be achievable in online settings where data arrives adversarially. [10, 20] first showed that this advantage exists in calibrated online forecasting when Hart pointed out that Foster's asymptotic calibrated forecasting result could be obtained by appealing to Blackwell approachability. Moreover, their proof extends trivially to online multicalibration. Later on, [18, 35] independently rediscovered this same advantage and proof. In the following theorem, we provide a rigorous treatment-largely in line with the prior works-that generalizes to all multi-objective learning problems that have separable objectives and satisfy some weak compactness conditions.

Theorem C.7. Consider an online multi-objective learning problem ( D , G , H ) with separable objectives and where D is the unrestricted set of all data distributions. Further, consider any objective mixture q ∈ ∆( G ) with finite support and suppose that for every objective glyph[lscript] ∈ G , (1) there is a finite subset Y glyph[lscript] ⊆ Y s.t. { g glyph[lscript] ( y ) | y ∈ Y glyph[lscript] } is an ( ε/ 3) -net for { g glyph[lscript] ( y ) | y ∈ Y} , (2) the range of g glyph[lscript] is convex, compact and finite-dimensional, and (3) fixing any x, y pair, glyph[lscript] ( h, ( x, y )) is, in the argument h , a function of bounded variation. Then, for any ε > 0 , there is a non-deterministic hypothesis p ∈ ∆( H ) (given by (9) ) that is a distribution-free ε best-response for q .

Proof. Let G q = Support( q ) be the support of our objective mixture, noting that |G q | < ∞ by assumption. Let W g ⊆ ∏ glyph[lscript] ∈G q W denote the range of the vector-valued function g : Y → W g where g ( y ) := [ g glyph[lscript] ( y )] glyph[lscript] ∈G q . Fixing some x ∈ X , observe that we can rewrite our loss q ( h, ( x, y )) as the

function ˜ q x : W × W g → [0 , 1] where q ( h, ( x, y )) = ˜ q x ( h ( x ) , g ( y )) and we define ˜ q x ( w,v ) := c + E glyph[lscript] ∼ q [ f glyph[lscript] ( w,x )( w -v glyph[lscript] )] .

By assumption (3), ˜ q x ( w,v ) is a function of bounded variation in w for all x ∈ X , v ∈ W g . By Lemma C.9, for any x ∈ X , y ∈ Y , there must exist some finite subset W x,y ⊆ W such that { ˜ q x ( w, ( x, g ( y ))) } w ∈W x,y is an ( ε/ 3) -net of { ˜ q x ( w, ( x, g ( y ))) } w ∈W . Let us choose W x := ⋃ glyph[lscript] ∈G q ⋃ y ∈Y glyph[lscript] W x,y , which is a finite set as |Y glyph[lscript] | < ∞ by assumption (1) and we |G q | < ∞ by assumption. Moreover, by construction, for any y ∈ Y , { ˜ q x ( w, ( x, y )) } w ∈W x is an (2 ε/ 3) -net of { ˜ q x ( w, ( x, y )) } w ∈W .

We now define our best response p pointwise at each x ∈ X , letting

Because each objective glyph[lscript] is separable, ˜ q x ( w, ( x, g ( y ))) is linear in g ( y ) . By linearity of expectation, we also know that E w ∼ w ∗ [ ˜ q x ( w, ( x, g ( y )))] is linear in w ∗ . Thus, for any x ∈ X ,

In the above, we were able to apply the minimax theorem because of assumption (2). Recall that our construction of W x , guarantees, for any w ∈ W , y ∈ Y , there is a w ′ ∈ W x , y ′ ∈ Y ′ such that

We, therefore, have that

We can now directly prove Theorem 3.8.

Theorem 3.8. Consider the set of k -class predictors P . Fix an ε > 0 and let q ∈ ∆( G mc ) be a mixture of multicalibration objectives (3) . There always exists a (non-deterministic) predictor p ∈ ∆( P ) that is a distribution-free ε -best-response (7) to the stochastic cost function q ( · , ( x, y )) .

Proof. Let ( D ′ , G ′ mc , P ) be the relaxation of the problem ( D , G mc , P ) where, instead of assuming that nature can only sample discrete labels y ∈ [ k ] , nature too can sample mixed labels y ∈ ∆ k . By linearity of expectation, we have that max D ′ ∈D ′ L D ′ ,q ′ ( p ) = max D ∈D L D,q ( p ) for every predictor p ∈ ∆( P ) and objective q ∈ G mc , where q ′ ∈ G ′ mc is the relaxation of q . Thus, a distribution-free ε best-response (7) for the relaxed problem is a distribution-free ε best-response for our original problem ( D , G mc , P ) . We next observe that ( D ′ , G ′ mc , P ) satisfies the conditions of Theorem C.7.

Recall that every objective q ′ ∈ G ′ mc can be written in a separable format where, for every i ∈ {± 1 } , j ∈ [ k ] , S ∈ S , v ∈ V k λ , we can write glyph[lscript] i,j,S,v ( h, ( x, y )) = c + f i,j,S,v ( x, h ( x )) · ( h ( x ) -g i,j,S,v ( y )) where c = 0 . 5 , g i,j,S,v ( y ) = y and f i,j,S,v ( x, h ( x )) ∈ [0 , 1] k is δ j · 0 . 5 · i · 1[ h ( x ) ∈ v, x ∈ S ] . The domain and range of identity g i,j,S,v ( y ) = y is exactly ∆ k which is convex, compact, and k -dimensional; thus, it has a finite ε -covering. Finally, we observe that f i,j,S,v ( h ( x ) , ( x, y )) , and by extension glyph[lscript] i,j,S,v ( h, ( x, y )) , is always a piecewise constant function in h ( x ) with finite discontinuities. It follows that glyph[lscript] i,j,S,v must be of bounded total variation in h ( x ) as variation along any linear segment is at most 1 . Thus, Theorem C.7 states there exists a distribution-free ε best-response for ( D ′ , G ′ mc , P ) and by extension ( D , G mc , P ) .

Remark C.8 (Closed-form of distribution-free best-responses) . For calibration and multicalibration problems, the best-response implicitly defined in Theorem C.7 takes a clean closed-form: for any given x ∈ X , p ( x ) randomizes on two neighboring actions. This simple closed form was first derived by [10] and independently rediscovered by [18].

In Theorem C.7, we referenced the notion of a function being of bounded variation. This is a common notion in analysis which says that a function cannot go up and down too many times; thus reasonable objectives (including every finite loss function you can think of) should all be of bounded variation. For completeness, we prove below that bounded variation implies finite domain coverings.

LemmaC.9. Consider a function f : K → [0 , 1] where K is a convex compact subset of R n . Suppose that f has pathwise bounded variation on K : that is, there exists a finite constant M such that for any linear path γ : [0 , 1] → K , V γ ( f ) = sup 0= t 0 ≤ t 1 ≤···≤ t N =1 ∑ N i =1 ‖ f ( γ ( t i )) -f ( γ ( t i -1 )) ‖ ≤ M . Then for any ε > 0 , there exists a finite subset S ⊆ K such that for every y ∈ f ( K ) , there exists an x i ∈ S with ‖ f ( x i ) -y ‖ < ε .

Proof. Since K is compact, it is totally bounded. For any δ > 0 , there exists a finite set T such that K is covered by balls of radius δ centered at points in T . Let δ = ε 2 M , where M is the constant from the pathwise bounded variation condition. For each point t ∈ T , choose a point s t ∈ K such that ‖ s t -t ‖ < δ . Define S = { s t : t ∈ T } . Let y ∈ f ( K ) . Then there exists an x ∈ K with f ( x ) = y . Since K is covered by balls of radius δ centered at points in T , there exists a t ∈ T with ‖ x -t ‖ < δ . By the construction of S , we also have ‖ s t -t ‖ < δ . Then, by the triangle inequality, ‖ s t -x ‖ ≤ ‖ s t -t ‖ + ‖ t -x ‖ < 2 δ = ε M . Consider a linear continuous path γ : [0 , 1] → K with γ (0) = x and γ (1) = s t ; such a path must exist by the convexity of K . By the pathwise bounded variation condition, we have V γ ( f ) ≤ M . For the partition 0 = t 0 ≤ t 1 = 1 , we have ‖ f ( s t ) -y ‖ = ‖ f ( s t ) -f ( x ) ‖ ≤ V γ ( f ) · ‖ s t -x ‖ ≤ M · ε M = ε . Thus, for every y ∈ f ( K ) , there exists an s t ∈ S such that ‖ f ( s t ) -y ‖ < ε .

D Proofs and Algorithms for Section 4

In the algorithms throughout this section, given an objective glyph[lscript] ( t ) ∈ G mc , we write i ( t ) , j ( t ) , S ( t ) , v ( t ) such that glyph[lscript] ( t ) = glyph[lscript] i,j,S,v .

D.1 Batch Multicalibration

Multicalibration with non-deterministic predictors. Our first algorithm uses no-regret no-regret (NRNR) dynamics to find non-deterministic multicalibrated predictors. We now restate and prove its guarantees from Theorem 4.1.

Algorithm 1 Non-Deterministic Multicalibration Algorithm (Theorem 4.1)

1: Input: S ⊆ 2 X , ε ∈ (0 , 1) , k, λ, T ∈ Z + , and distribution D ;

2: Initialize Hedge iterate q (1) = Uniform ( G mc ) and Hedge iterate h (1) = [1 /k, . . . , 1 /k ] X ;

3: for t = 1 to T do

4: Sample objective glyph[lscript] ( t ) ∼ q ( t ) and datapoint ( x ( t ) , y ( t ) ) ∼ D ;

5: For every x ∈ X , let h ( t +1) ( x ) := Hedge( c (1: t ) x ) , where

6: Let q ( t +1) := Hedge( c adv (1: t ) ) , where c adv ( t ) ( glyph[lscript] ) := 1 -glyph[lscript] ( h ( t ) , ( x ( t ) , y ( t ) )) ;

7: end for

8: Return: p ∗ , a uniform distribution over h (1) , . . . , h ( T ) ;

Construct the problem ( { D } , G mc , P ) from Fact 2.5 and let T = Cε -2 ln( |S| λ k /δ ) for some universal constant C . Over T rounds, have an adversary choose q (1: T ) ∈ ∆( G mc ) by applying Hedge to the costs { 1 -glyph[lscript] ( · ) ( h ( t ) , ( x ( t ) , y ( t ) )) } (1: T ) where ( x ( t ) , y ( t ) ) i.i.d. ∼ D . In parallel, have a a learner choose predictors h (1: T ) ∈ P by applying the no-regret learning algorithm of Theorem 3.7 to the stochastic costs glyph[lscript] (1: T ) , where glyph[lscript] ( t ) i.i.d. ∼ q ( t ) . Return the predictor p = Uniform( h (1: T ) ) . This algorithm is written explicitly in Algorithm 1.

Proof. By Theorem 3.7, if T ≥ 64 ε -2 ln( k ) , the predictors h (1: T ) guarantee the learner a regret bound of Reg weak ( h (1: T ) , {L glyph[lscript] ( t ) } (1: T ) ) ≤ Tε/ 4 . Similarly, by Lemma 3.1, if T ≥ 576 ε -2 ln(2 kλ k |S| ) , the objective mixtures q (1: T ) guarantee the adversary a regret bound of Reg ( q (1: T ) , { 1 -glyph[lscript] ( · ) ( h ( t ) , ( x ( t ) , y ( t ) )) } (1: T ) ) ≤ T ε 12 .

We now argue that the adversary's regret with respect to the costs 1 -glyph[lscript] ( · ) ( h ( t ) , ( x ( t ) , y ( t ) )) approximates its regret with respect to the costs 1 -L D, ( · ) ( h ( t ) ) . Since E x ( t ) ,y ( t ) ∼ D [ glyph[lscript] ( · ) ( h ( t ) , ( x ( t ) , y ( t ) )) ] = L D, ( · ) ( h ( t ) ) , by Lemma 3.2, there is a universal constant C such that, if T ≥ Cε -2 ln(2 kλ k |S| /δ ) ,

with probability 1 -δ . Similarly, since E glyph[lscript] ( t ) ∼ q ( t ) [ L D,glyph[lscript] ( t ) ( h ( t ) ) ] = L D,q ( t ) ( h ( t ) ) , Lemma 3.2 guarantees

with probability 1 -δ . Taking a triangle inequality and union bound, we can see that for sufficiently large C , Reg ( glyph[lscript] (1: T ) , { 1 -L D, ( · ) ( h ( t ) ) } (1: T ) ) ≤ Tε/ 4 .

By Lemma 3.3, the ergodic iterate h ∗ is an ( ε/ 2) -optimal solution. By Fact 2.5, h ∗ is therefore a ( S , ε, λ ) -multicalibrated predictor. The sample complexity is exactly T since the algorithm only samples one datapoint at each iteration.

Multicalibration with deterministic predictors. We are often specifically interested in finding deterministic multicalibrated predictors. This is usually because non-deterministic predictors can be multicalibrated in a very weak sense, as we show in the following example.

Example D.1. Consider a data distribution D supported uniformly on X = { x 1 , x 2 } , where Y = [0 , 1] , Pr( Y = 1 | X = x 1 ) = 0 and Pr( Y = 1 | X = x 2 ) = 1 . The non-deterministic predictor that is supported uniformly on predictors h 1 and h 2 , where h 1 ( x 1 ) = 0 and h 1 ( x 2 ) = 0 . 5 and h 2 ( x 1 ) = 0 . 5 and h 2 ( x 2 ) = 1 , is technically multicalibrated. However, neither h 1 nor h 2 are calibrated.

Our second algorithm uses no-regret best-response (NRBR) dynamics to find deterministic multicalibrated predictors. We now restate and prove its guarantees from Theorem 4.2.

Algorithm 2 Deterministic Multicalibration Algorithm (Theorem 4.2)

1: Input: S ⊆ 2 X , ε ∈ (0 , 1) , k, λ, T, C ∈ Z + , distribution D , and agnostic learning oracle A ;

2: Initialize Hedge iterate h (1) = [1 /k, . . . , 1 /k ] X ;

3: for t = 1 to T do

4: Let glyph[lscript] ( t ) = A ε/ 8 ( c adv ( t ) , G mc ) where c adv ( t ) ( glyph[lscript] ) := 1 -L D,glyph[lscript] ( h ( t ) ) ;

5: For every x ∈ X , let h ( t +1) ( x ) := Hedge( c (1: t ) x ) , where

6: end for

7: Take C ln( T/δ ) /ε 2 samples x ∼ D and let t ∗ = arg min ∑ glyph[lscript] ( t ) ( h ( t ) , ( x, y )) ;

∗

t ∈ [ T ] ( x,y ) ∈ x

8: Return the predictor h ( t ) ;

Theorem 4.2. Fix ε > 0 , λ, k ∈ Z + , a set of groups S ⊆ 2 X , and a data distribution D . The following algorithm returns a deterministic k -class predictor that is ( S , ε, λ ) -multicalibrated on D and makes O (ln( k ) /ε 2 ) calls to an agnostic learning oracle. Moreover, with probability 1 -δ , the oracle calls can be implemented with ˜ O ( 1 ε 3 ( √ ln( k ) ln( k |S| /δ ) + k ln( λ )) ) samples from D .

Construct the problem ( { D } , G mc , P ) from Fact 2.5 and let T = Cε -2 ln( |S| λ k δ ) for some universal constant C . Over T rounds, have a learner choose predictors h (1: T ) ∈ P by applying the no-regret learning algorithm of Theorem 3.7 to the stochastic costs glyph[lscript] (1: T ) . Have an adversary choose glyph[lscript] (1: T ) by calling an agnostic learning oracle at each t ∈ [ T ] : glyph[lscript] ( t ) = A ε/ 8 (1 -L D, ( · ) ( h ( t ) )) . Using C ln( T/δ ) /ε 2 samples from D , return the predictor h ( t ∗ ) with the lowest empirical multicalibration error. This algorithm is written explicitly in Algorithm 2.

Proof. By Theorem 3.7, if T ≥ 256 ε -2 ln( k ) , the predictors h (1: T ) guarantee the learner a regret bound of Reg weak ( h (1: T ) , {L D,glyph[lscript] ( t ) } (1: T ) ) ≤ Tε/ 8 . Moreover, by construction, every glyph[lscript] ( t ) is an ( ε/ 8) best-response to the cost 1 -L D, ( · ) ( h ( t ) ) . By Lemma 3.4, there must exist a timestep where h ( t ) is ( ε/ 4) -optimal. By Lemma 3.5, the h ( t ∗ ) found by the algorithm is ( ε/ 2) -optimal with probability at least 1 -δ . By Fact 2.5, h ( t ∗ ) is a ( S , ε, λ ) -multicalibrated predictor. The oracle complexity is exactly T , while the sample complexity of oracle A is a standard adaptive data analysis result (Lemma A.6).

We note that our use of no-regret best-response dynamics recovers a multicalibration algorithm similar to the original multicalibration algorithm of [21] and which can also be found in [7, 25]. We also remark that the guarantees of Theorem 4.2 hold in weaker settings. In particular, Theorem 4.2 holds with the same analysis even if we only asked that our agnostic learning oracle A to be bestresponding with respect to a min-max baseline. Furthermore, the last step of Algorithm 2, which explicitly samples datapoints to find timestep t ∗ , can be removed if one assumes that our agnostic learning oracle A signals to us when it cannot find a greater than ε violation of multicalibration in the current predictor. This assumption would allow Algorithm 2 to terminate early and return the current predictor, and is assumed by prior multicalibration literature.

D.2 Online Multicalibration

Our next algorithm uses best-response no-regret (BRNR) dynamics for online multicalibration. We now restate and prove its guarantees from Theorem 4.3.

Algorithm 3 Online Multicalibration Algorithm

1: Input: S ⊆ 2 X , ε ∈ (0 , 1) , λ, T ∈ Z + ;

2: Initialize Hedge iterate q (1) = Uniform ( G mc ) ;

3: for t = 1 to T do

4: p ( t ) ( x ) := min p ∗ ( x ) ∈ ∆([0 ,ε/ 4 λ,..., 1]) max y ∈ [0 , 1] 1 2 E ̂ y ∼ p ∗ ( x ) [ 1 + E glyph[lscript] i,S,v ∼ q ( t ) [ i · 1[ x ∈ S, y ∈ v ] · ( ̂ y -y )] ] ;

5: Announce predictor p ( t ) to Nature and observe Nature's data distribution D ( t ) ;

6: Sample ( x ( t ) , y ( t ) ) ∼ D ( t ) and let q ( t +1) := Hedge( c adv (1: t ) ) where c adv ( t ) ( glyph[lscript] ) := 1 -glyph[lscript] ( h ( t ) , ( x ( t ) , y ( t ) )) ;

7: end for

Theorem 4.3. Fix ε > 0 , λ ∈ Z + , and a set of groups S ⊆ 2 X . The following algorithm guarantees ( S , ε, λ ) -online multicalibration with probability 1 -δ .

Construct the online multi-objective learning problem ( D , G mc , P ) in Fact 2.6 and let T = Cε -2 ln( |S| λδ ) for some universal constant C . Over T rounds, have an adversary choose q (1: T ) ∈ ∆( G mc ) by applying Hedge to the costs { 1 -glyph[lscript] ( · ) ( p ( t ) , ( x ( t ) , y ( t ) )) } (1: T ) , where ( x ( t ) , y ( t ) ) i.i.d. ∼ D ( t ) . Have a learner best-respond to each stochastic cost q ( t ) with the ( ε/ 2) -distribution-free best-response p ( t ) ∈ ∆( P ) of Theorem 3.8. This algorithm is written explicitly in Algorithm 3.

Proof. By Lemma 3.1, if T ≥ 576 ε -2 ln(2 kλ |S| ) , the objective mixtures q (1: T ) guarantee the adversary a regret bound of Reg ( q (1: T ) , { 1 -glyph[lscript] ( · ) ( p ( t ) , ( x ( t ) , y ( t ) )) } (1: T ) ) ≤ T ε 12 . By Lemma 3.2, there is a universal constant C such that, if T ≥ Cε -2 ln(2 kλ |S| /δ ) ,

with probability 1 -δ . Thus, Reg ( q (1: T ) , { 1 -L D ( t ) , ( · ) ( h ( t ) ) } (1: T ) ) ≤ Tε/ 4 with probability 1 -δ .

Since each p ( t ) is a ( ε/ 4) distribution-free best-response to q ( t ) , whose existence is proven by Theorem 3.8, by Lemma 3.6, the predictors p (1: T ) are ( ε/ 2) -optimal on D (1: T ) with probability 1 -δ . By Fact 2.6, p (1: T ) are also ( S , ε, λ ) -online multicalibrated on D (1: T ) .

The high-probability condition of Theorem 4.3 can be removed if we assume nature presents datapoints rather than data distributions, as is assumed in prior works. Interestingly, Algorithm 3's use of best-response no-regret dynamics exactly recovers the online multicalibration algorithm of [18, 35]. The analysis of Theorem 4.3 is, however, significantly simpler because we make explicit the role of the no-regret dynamics, whereas [18, 35] use potential arguments that ultimately prove no-regret dynamics and the multiplicative weights algorithm from scratch.

An online-to-batch reduction. The online multicalibration algorithm of Theorem 4.3 can also be used to obtain a non-deterministic multicalibrated predictor in batch settings through an online-tobatch reduction. This exactly recovers the non-deterministic multicalibration algorithm of [18, 35].

Theorem D.2 (Analysis of Algorithm 3 for batch multicalibration) . Fix ε > 0 , λ ∈ Z + , a set of groups S ⊆ 2 X , and a data distribution D . Simulate Algorithm 3 by having Nature choose D at every timestep and return a uniform distribution p over its outputs p (1: T ) . With probability 1 -δ , p is ( S , ε, λ ) -multicalibrated on D . This algorithm takes O (ln( |S| λ/δ ) /ε 2 ) samples from D .

Proof. By Theorem 4.3, max glyph[lscript] ∗ ∈G mc ∑ T t =1 L D,glyph[lscript] ∗ ( p ( t ) ) ≤ Tε . Thus, the non-deterministic predictor p given by taking a uniform distribution over p (1) , . . . , p ( T ) is a ( S , ε, λ ) -multicalibrated predictor.

Unlike the algorithm of Theorem 4.1, the predictor p output by Theorem D.2 is neither guaranteed to be succinct nor guaranteed to be of small circuit size. The lack of succinctness is because, even if p 's

predicted label distribution on any feature x ∈ X is succinct, p itself can require an exponentially large support over P . The large circuit size is because, at each timestep t , the learner is best-responding to a distribution q ( t ) over objectives G with a non-zero weight on every objective. In contrast, in the algorithm of Theorem 4.1, the learner only interacts with a single new objective glyph[lscript] ( t ) at each timestep t .

E Additional Multicalibration Considerations

In this section, we present additional results on multicalibration that all follow from the same game dynamics presented in Section 3.

E.1 Conditional Multicalibration

One shortcoming of existing definitions of multicalibration is that they measure violations marginally over the entire distribution D . That is, the amount that a predictor is allowed to violate calibration on a subgroup is inversely proportional to the probability mass of the subgroup, as reflected by the use of the indicator function in Definition 2.1. This can lead to predictors that are certifiably multicalibrated, but still poorly calibrated on underrepresented or minority groups. In contrast, the original definition of multicalibration proposed by [21], for which no non-trivial guarantee is known, is a conditional notion of multicalibration.

Below, we generalize this notion of conditional multicalibration so that setting S = S ′ recovers the original multicalibration definition of [21].

Definition E.1. Fix ε > 0 , λ ∈ Z + , and two sets of groups S , S ′ ⊆ 2 X . A k -class predictor p ∈ ∆( P ) is ( S , S ′ , ε, λ ) -conditionally multicalibrated for some data distribution D if

It is not possible to obtain conditional multicalibration generally, so we will assume sample access to the conditional distributions { D S ′ } S ′ ∈S ′ , where D S ′ := D | x ∈ S ′ . In practice, this assumption means that we are able to sample data from certain protected groups that may otherwise be underrepresented. For convenience, in this section, we will assume prior knowledge of { Pr D ( x ∈ S ′ ) } S ′ ∈S ′ but note these probabilities can be cheaply estimated beforehand.

To derive conditional multicalibration algorithms from game dynamics, we first write conditional multicalibration as a multi-distribution multi-objective learning problem.

Fact E.2. Let D be a data distribution for some k -class prediction problem and fix ε > 0 , λ ∈ Z + , and two sets of groups S , S ′ ⊆ 2 X . Let G mc be the set of objectives { glyph[lscript] i,j,S,v } as defined in Fact 2.5 and let D = { D ′ S } S ′ ∈S ′ . Predictor p ∈ ∆( P ) is a ε -optimal solution to the multi-objective learning problem ( D , G mc , P ) if and only if p is ( S , S ′ , 2 ε, λ ) -conditionally multicalibrated for D .

Proof. In the multi-objective learning problem ( { D ′ S } S ′ ∈S ′ , G mc , P ) , the multi-objective value of a predictor p , L ∗ ( p ) := max D ∗ ∈D ,glyph[lscript] ∗ ∈G L D ∗ ,glyph[lscript] ∗ ( p ) , is exactly the (rescaled and shifted) magnitude of the predictor's multicalibration violation. Formally,

The optimal multi-objective value of the problem ( D , G mc , P ) is 0 . 5 , as G mc are symmetric around 0 . 5 and the Bayes classifier still achieves a loss of 0 . 5 .

Thus, L ∗ ( p ) -min h ∗ ∈P L ∗ ( h ∗ ) = ε if and only if our conditional multicalibration violation is 2 ε .

Conditional multicalibration algorithms. The following Theorem E.3 is, to the best of our knowledge, the first non-trivial guarantee for conditional multicalibration.

Theorem E.3. Fix ε > 0 , λ, k ∈ Z + , two sets of groups S , S ′ ⊆ 2 X , and a data distribution D . The following algorithm returns a deterministic k -class predictor that is ( S , S ′ , ε, λ ) -conditionally multicalibrated on D and makes O (ln( k ) /ε 2 ) calls to an agnostic learning oracle. Moreover, with probability 1 -δ , the oracle calls can be implemented with ˜ O ( 1 ε 3 ( √ ln( k ) ln( k |S| |S ′ | /δ ) + k ln( λ )) ) samples from each conditional distribution { D ′ } ′ ′ .

S S ∈S

Construct the problem ( D , G mc , P ) from Fact E.2 and let T = Cε -2 |S ′ | ln( k ) for some universal constant C . Over T rounds, have a learner choose predictors h (1: T ) by applying the no-regret learning algorithm of Theorem C.6 to the stochastic costs glyph[lscript] (1: T ) and distributions { D S ′ ( t ) } (1: T ) . Have an adversary choose S ′ (1: T ) and glyph[lscript] (1: T ) by calling an agnostic learning oracle at each t ∈ [ T ] : S ′ ( t ) , glyph[lscript] ( t ) = A ε/ 8 (1 -L D ( · ) , ( · ) ( h ( t ) )) . Using C ln( T |S ′ | /δ ) /ε 2 samples from each distribution in { D ′ S } S ′ ∈S ′ , return the predictor h ( t ∗ ) with the lowest empirical multicalibration error. This algorithm is written explicitly in Algorithm 4.

Proof. In order to apply Theorem C.6, we first observe that every objective in G mc is separable and every distribution D ′ S is absolutely continuous with respect to D , By definition, for any S ′ ∈ S ′ , the Radon-Nikodym derivative of D ′ S with respect to D at x ∈ X is 1[ x ∈ S ′ ] / Pr D ( x ∈ S ′ ) . Thus, the sequence h (1: T ) corresponds to running the no-regret algorithm of Theorem C.6 when A is chosen to be Prod. By Lemma A.4, we know that choosing the Prod algorithm as A satisfies the conditions of Theorem C.6 with R ( T ) ∈ O ( √ ln( k ) T ) . Theorem C.6 therefore guarantees that Reg B ∗ ( h (1: T ) , {L D S ′ ( t ) ,glyph[lscript] ( t ) ( · ) } (1: T ) ) ≤ O ( √ |S ′ | ln( k ) T ) , where B ∗ is as defined in (8). Since we can lower bound the baseline B ∗ ≥ 0 . 5 by plugging the Bayes classifier h ∗ ( x ) = E ( x,y ) ∼ D [ y | x ] into (8), Reg weak ( h (1: T ) , {L D S ′ ( t ) ,glyph[lscript] ( t ) ( · ) } (1: T ) ) ≤ Reg B ∗ ( h (1: T ) , {L D S ′ ( t ) ,glyph[lscript] ( t ) ( · ) } (1: T ) ) ≤ O ( √ |S ′ | ln( k ) T ) . Thus, if T ≥ C |S ′ | ln( k ) /ε 2 , Reg weak ( h (1: T ) , {L D S ′ ( t ) ,glyph[lscript] ( t ) ( · ) } (1: T ) ) ≤ Tε/ 8 .

By construction, every pair S ′ ( t ) , glyph[lscript] ( t ) is an ( ε/ 8) best-response to the cost function 1 -L D ( · ) , ( · ) ( h ( t ) ) . By Lemma 3.4, since the learner has at most Tε/ 8 weak regret and the adversary is ε/ 8 bestresponding, there must exist a timestep t ∈ [ T ] where h ( t ) is ( ε/ 4) -optimal. By Lemma 3.5, the h ( t ∗ ) found by the algorithm is ( ε/ 2) -optimal with probability at least 1 -δ . By Fact E.2, h ( t ∗ ) is a ( S , S ′ , ε, λ ) -conditionally multicalibrated predictor. The oracle complexity is exactly T , while the sample complexity of oracle A is a standard adaptive data analysis result (Lemma A.6).

We can also implement faster non-deterministic conditional multicalibration algorithms.

Theorem E.4. Fix ε > 0 , λ, k ∈ Z + , two sets of groups S , S ′ ⊆ 2 X , and a data distribution D . The following algorithm returns a non-deterministic k -class predictor that is ( S , S ′ , ε, λ ) -conditionally multicalibrated on D and takes O ( |S ′ | (ln( |S| |S ′ | /δ ) + k ln( λ )) /ε 2 ) samples in total from the distributions in { D ′ S } S ′ ∈S ′ .

No-Regret vs No-Regret

Construct the problem ( D , G mc , P ) from Fact E.2 and let T = Cε -2 |S ′ | ln( |S| λ k δ ) for some universal constant C . Over T rounds, have a learner choose predictors h (1: T ) ∈ P by applying the no-regret learning algorithm of Theorem C.6 to the stochastic costs glyph[lscript] (1: T ) and distributions { D S ′ ( t ) } (1: T ) . In parallel, have an adversary choose S ′ (1: T ) , glyph[lscript] (1: T ) by applying ELP to costs { 1 - L D ( · ) , ( · ) ( h ( t ) ) } (1: T ) . Return the predictor p = Uniform ( h (1: T ) ) . This algorithm is written explicitly in Algorithm 5.

Proof. As we proved in Theorem E.3, Theorem C.6 guarantees that if T ≥ C |S ′ | ln( k ) /ε 2 , the learner's regret is bounded by Reg weak ( h (1: T ) , {L D S ′ ( t ) ,glyph[lscript] ( t ) ( · ) } (1: T ) ) ≤ Tε/ 4 . We now turn to the adversary.

We will define a sequence of costs ˜ c adv (1: T ) that mirrors the adversary's costs { 1 -L D ( · ) , ( · ) ( h ( t ) ) } (1: T ) . For all timesteps t ∈ [ T ] , rename ( x ( t ) S ′ ( t ) , y ( t ) S ′ ( t ) ) := ( x ( t ) , y ( t ) ) while for all other S ′ ∈ S ′ define the random samples ( x ( t ) S ′ , y ( t ) S ′ ) i.i.d. ∼ D S ′ as the datapoint that the adversary would have hypothetically sampled if it had chosen to sample from D ′ S instead of D S ′ ( t ) at timestep t . We now can define ˜ c adv ( t ) ( S ′ , ( i, j, S, v )) := (0 . 5 + 0 . 5 · i · 1[ h ( x ( t ) S ′ ) ∈ v, x ( t ) S ′ ∈ S ] · ( δ y ( t ) S ′ ,j -h ( x ( t ) S ′ ) j )) . Note that, since the ELP algorithm only observes values of c adv ( t ) (( i, j, S, S ′ , v )) where S ′ = S ′ ( t ) , Reg( { ( S ′ ( t ) , glyph[lscript] ( t ) ) } (1: T ) , c adv (1: T ) ) = Reg( { ( S ′ ( t ) , glyph[lscript] ( t ) ) } (1: T ) , ˜ c adv (1: T ) ) .

By Lemma A.5, the adversary satisfies Reg( { ( S ′ ( t ) , glyph[lscript] ( t ) ) } (1: T ) , c adv (1: T ) ) ≤ O ( √ |S ′ | ln( k/δ ) T ) . Thus, we also have that Reg( { ( S ′ ( t ) , glyph[lscript] ( t ) ) } (1: T ) , ˜ c adv (1: T ) ) ≤ O ( √ |S ′ | ln( k/δ ) T ) . By Lemma 3.2, there is a universal constant C such that, if T ≥ Cε -2 ln(2 kλ k |S| |S ′ | /δ ) ,

with probability 1 -δ . Taking a triangle inequality and union bound, we can see that for sufficiently large C , Reg( { ( S ′ ( t ) , glyph[lscript] ( t ) ) } (1: T ) , { 1 -L D ( · ) , ( · ) ( h ( t ) ) } (1: T ) ) ≤ Tε/ 4 .

By Lemma 3.3, the ergodic iterate h ∗ is an ( ε/ 2) -optimal solution. By Fact E.2, h ∗ is a ( S , S ′ , ε, λ ) -conditionally multicalibrated predictor. The sample complexity is exactly T since the algorithm only samples one datapoint at each iteration.

Stronger multicalibration guarantees for (almost) free. We can also drop the assumption of access to the conditional distributions { D S ′ } S ′ ∈S ′ and ask for a weaker guarantee than conditional multicalibration where a subgroup's error tolerance scales with √ Pr( x ∈ S ′ ) . For simplicity, we will fix S ′ = S here. Note that unlike Theorem E.3, Theorem 4.4 does not assume knowledge of the vector [Pr D ( x ∈ S )] S ∈S .

Theorem 4.4. Fix ε > 0 , λ, k ∈ Z + , two sets of groups S ⊆ 2 X , and a data distribution D . There is an algorithm that, with probability at least 1 -δ , takes O (ln( k ) · (ln( |S| /εδ )+ k ln( λ )) /ε 4 ) samples from D and returns a deterministic k -class predictor h satisfying

The sample complexity of Theorem 4.4 should be improvable to a sample complexity of ˜ O (( √ ln( k ) ln( |S| /δ ) + k ln( λ )) /ε 3 ) using adaptive data analysis [30, 1]. Even without using adaptive data analysis, Theorem E.3 guarantees a sample complexity that is only a 1 /ε factor greater (less than a cube-root increase) than the best known sample complexity for deterministic multicalibration. In fact, it matches the best sample complexity for deterministic multicalibration that is known to be possible without adaptive data analysis. This means that we can attain this strictly stronger multicalibration guarantee for free (compared to its non-adaptive data analysis counterpart) or with only a minor cube-root increase in sample complexity (compared to its adaptive data analysis counterpart). We defer a complete proof and algorithm statement to Section E.4.

E.2 Agnostic Multicalibration

An implicit assumption of multicalibration is that one has unlimited freedom to vary their predictions based on which subgroups a datapoint belongs to. This assumption arises because we assume that groups are defined as subsets of the domain, S ⊆ X , and predictors are allowed to condition arbitrarily on the domain. This is impractical in many settings including, for example, when subgroups correspond to protected demographics. This is an important concern for fairness applications of multicalibration.

One can extend multicalibration to a more general agnostic setting by assuming that group membership is passed to the predictor separately from the covariate. Note that, in this definition, different groups may not share a Bayes classifier and a fully multicalibrated predictor may not exist.

Definition E.5. Consider a data distribution D supported on X × X ′ ×Y and where the set P of k -class predictors that are functions of the visible covariates X but not the protected covariates X ′ .

Fix ε > 0 , λ ∈ Z + , and a set of groups S ⊆ 2 X ′ . A (possibly non-deterministic) k -class predictor p ∈ ∆( P ) is ( S , ε, λ ) -agnostic multicalibrated with respect to a baseline B if

In the original definition of multicalibration, it is implicitly assumed that one knows, a priori, the probability that a covariate x belongs to a certain group; this is necessary to attain sample complexity rates independent of domain size X . We will similarly allow agnostic multicalibration to take for granted knowledge of the group memberships of each covariate x ∈ X .

Choosing a baseline. One challenge with defining agnostic multicalibration is choosing a proper baseline B . An immediate option is to choose a min-max baseline:

However, achieving this baseline is intractable, with a sample and oracle complexity that dependspotentially linearly-on domain size. Moreover, the predictor attaining the min-max baseline is odd and undesirable: when two groups disagree on the label for a set of covariates, the predictor hedges its losses by spreading its predictions for those covariates into as many bins as possible.

Proposition E.6. Define X = [ m ] for some large m ∈ Z + and uniformly sample a random half of X as ˜ X ⊆ X where | ˜ X| = |X| / 2 . Let X ′ = { 1 , 2 } , λ = 3 and k = 2 , and define the marginal distributions Pr D ( x | x ′ = 1) and Pr D ( x | x ′ = 2) to be uniform on X . For x / ∈ ˜ X , let labels be random: Pr D ( y = 1 | x ′ = 1 , x ) = Pr D ( y = 1 | x ′ = 2 , x ) = 0 . 5 . For x ∈ ˜ X , let labels be deterministic: Pr D ( y = 1 | x ′ = 1 , x ) = 0 and Pr D ( y = 1 | x ′ = 2 , x ) = 1 . Finding a predictor that is ( S , ε, λ ) -agnostic multicalibrated with respect to the min-max baseline requires at least Θ( | x | ) samples must be taken from D for ε = 0 . 1 .

Proof. The min-max optimal multicalibated predictor divides ˜ X into 4 equal parts: ˜ X 1 , . . . , ˜ X 4 . It predicts h ( x ) = 0 . 5 for all x / ∈ ˜ X , x ∈ ˜ X 1 and x ∈ ˜ X 2 . It predicts h ( x ) = 1 / 3 for all x ∈ ˜ X 3 . It predicts h ( x ) = 2 / 3 for all x ∈ ˜ X 4 . This attains a multicalibration violation of 1 / 6 . Thus, achieving an ε -optimal predictor requires determining the membership (in ˜ X ) of at least a 3 / 4 -ε fraction of X .

To provide a more satisfactory definition of agnostic multicalibration, we ask that one's predictor assigns the same label probabilities to datapoints with the same group memberships and relaxes the baseline so that hedging is not necessary.

Definition E.7. We say that a k -predictor h is agnostic multicalibrated if the predictor h provides the same prediction probabilities h ( x 1 ) = h ( x 2 ) to datapoints x 1 , x 2 ∈ X that share the same group membership distributions ∀ S ∈ S : Pr( x ′ ∈ S | x = x 1 ) = Pr( x ′ ∈ S | x = x 2 ) ; and (2) is ( S , ε, λ ) -agnostic multicalibrated with respect to the multi-accuracy baseline :

This baseline is zero when the groups share a Bayes classifier, recovering Definition 2.1. With a slight tweak to Fact 2.5, we can write agnostic multicalibration as the following multi-objective learning problem.

Fact E.8. Let D be a data distribution for some k -class prediction problem and fix ε > 0 , λ ∈ Z + , and a set of groups S ⊆ 2 X ′ . For every protected feature x ′ ∈ ( X ′ ) X , direction i ∈ {± 1 } , level set v ∈ V k λ , group S ∈ S , and class j ∈ [ k ] , we define an objective glyph[lscript] x ′ ,i,j,S,v : P × ( X × Y ) → [0 , 1] where

and G ag = { glyph[lscript] x ′ ,i,j,S,v } x ′ ,i,j,S,v is the set of these objectives. We also define objectives without subscript x ′ , where glyph[lscript] i,j,S,v ( h, ( x, x ′ , y )) = 0 . 5 + 0 . 5 · i · 1[ h ( x ) ∈ v, x ′ ∈ S ] · ( h ( x ) j -δ y,j ) and

G ′ ag = { glyph[lscript] i,j,S,v } i,j,S,v . For every objective glyph[lscript] ∈ G ′ ag , let q glyph[lscript] ∈ ∆( G ag ) denote the objective mixture where E x ∼ D,glyph[lscript] ∼ q glyph[lscript] [ glyph[lscript] ( · )] = E x ′ ,x ∼ D [ glyph[lscript] ( · )] . If a predictor p ∈ ∆( P ) is a ε -optimal solution to the multi-objective learning problem ( { D } , G ′ ag , P ) with respect to the multi-accuracy baseline, then p is ( S , S ′ , 2 ε, λ ) -agnostic multicalibrated for D with respect to the multi-accuracy baseline.

A simple modification to our batch multicalibration algorithm suffices to achieve agnostic multicalibration.

Theorem E.9. Fix ε > 0 , λ, k ∈ Z + , a set of groups S ⊆ 2 X ′ , and a data distribution D . The following algorithm returns a deterministic k -class predictor that is ( S , ε, λ ) -agnostic multicalibrated on D with respect to the multi-accuracy baseline. It makes O (ln( k ) /ε 2 ) calls to an agnostic learning oracle and, with probability 1 -δ , these oracle calls can be implemented with ˜ O ( 1 ε 3 ( √ ln( k ) ln( k |S| /δ ) + k ln( λ )) ) samples from D .

Construct the problem ( { D } , G ag , P ) from Fact 2.5 and let T = Cε -2 ln( |S| λ k δ ) for some universal constant C . Over T rounds, have an adversary choose glyph[lscript] (1: T ) ∈ G ′ ag by calling an agnostic learning oracle at each t ∈ [ T ] : glyph[lscript] ( t ) = A ε/ 8 (1 - L D, ( · ) ( h ( t ) ) , G ′ ag ) . Have a learner choose predictors h (1: T ) ∈ P by applying the no-regret learning algorithm of Theorem C.6 to the stochastic costs { q glyph[lscript] ( t ) } (1: T ) ∈ ∆( G ag ) . Using C ln( T/δ ) /ε 2 samples from D , return the predictor h ( t ∗ ) with the lowest empirical multicalibration error. This algorithm is written explicitly in Algorithm 6.

Proof. By Theorem C.6, if T ≥ 256 ε -2 ln( k ) , the predictors h (1: T ) satisfy Reg B ∗ ( h (1: T ) , { q glyph[lscript] ( t ) } (1: T ) ) ≤ Tε/ 8 . Note that, since predictors h (1: T ) can only observe x and not x ′ , we applied the algorithm of Theorem C.6 to (mixtures of) objectives from G ag that depend only on ( x, y ) . Since E x ∼ D,glyph[lscript] x ′ ,i,j,S,v ∼ q glyph[lscript] ( t ) [ glyph[lscript] x ′ ,i,j,S,v ( · )] = E x ′ ,x ∼ D [ glyph[lscript] ( t ) ( · ) ] however, we can still bound regret with respect to the loss functions in G ′ ag , which do depend on x ′ , by Reg B ∗ ( h (1: T ) , glyph[lscript] (1: T ) ) = Reg B ∗ ( h (1: T ) , { q glyph[lscript] ( t ) } (1: T ) ) ≤ Tε/ 8 ; we define both of these regrets with respect to the set of all predictors from X → Y . Also note that the B ∗ baseline bounded by the multiaccuracy baseline (10):

Thus, we have regret Reg B multi-acc ( h (1: T ) , glyph[lscript] (1: T ) ) ≤ Reg B ∗ ( h (1: T ) , glyph[lscript] (1: T ) ) ≤ T ε 8 . By construction, every glyph[lscript] ( t ) is an ( ε/ 8) best-response to the cost 1 -L D, ( · ) ( h ( t ) ) . By Lemma 3.4, there must exist a timestep where h ( t ) is ( ε/ 4) -optimal with respect to B multiacc. By Lemma 3.5, the h ( t ∗ ) found by the algorithm is ( ε/ 2) -optimal with respect to B multiacc with probability at least 1 -δ . By Fact E.8, h ( t ∗ ) is a ( S , ε, λ ) -agnostic multicalibrated predictor. Lastly, we verify that datapoints with identical group membership probabilities must have the same label probabilities as the update rules for their predicted labels are identical.

The best existing sample complexity bounds for agnostic multicalibration come from uniform convergence [38], and scale with log( |H| ) ≈ |X| -that is, sample complexity scales with the size of one's domain. In contrast, Theorem E.9 provides a sample complexity that still depends only on the complexity of the groups on which one desires multicalibration. This can be an exponential (in |X| ) reduction in sample complexity.

Similarly to Algorithm 6, we can use the same modification to scale our non-deterministic multicalibration algorithm (Algorithm 1) to agnostic settings.

Theorem E.10. Fix ε > 0 , λ ∈ Z + and a set of groups S ⊆ 2 X . There exists an algorithm guarantees, with probability at least 1 -δ , the algorithm returns a randomized ( S , ε, λ ) -agnostic multicalibrated predictor while taking no more than O ( (ln( |S| /δ ) + k ln( λ )) /ε 2 ) samples from D .

E.3 Moment Multicalibration

Our treatment of multicalibration, and extensions of multicalibration, hold more generally for any problem that can be written as multi-objective learning with objectives that look separable. As an example, we demonstrate that we can use the same approach to devise algorithms for moment multicalibration , which concerns not only learning a multicalibrated label predictor but also a second predictor that estimates the higher-order moments of the label distribution [22]. As in online multicalibration, we will work with only binary classification problems in this section, and so will say that predictors output real-valued h µ ( x ) , h m ( x ) ∈ [0 , 1] .

Definition E.11. Let D be a data distribution for a binary classification problem and fix ε > 0 , λ, m ∈ Z + , and a set of groups S ⊆ 2 X . A pair of predictors p = ( p µ , p m ) where p µ , p m : X → [0 , 1] is ( S , ε, λ, m ) -mean-conditioned moment-multicalibrated 5 on D if, for all S ∈ S , v µ , v m ∈ V λ ,

We will write h = ( h µ , h m ) as shorthand, where h ( x ) = ( h µ ( x ) , h m ( x )) . Moment multicalibration can also be expressed as a multi-objective learning problem.

Fact E.12. Consider the set of all pairs of predictors H = { ( h µ , h m ) : h µ , h m : X → [0 , 1] } . Fix ε > 0 and λ, m ∈ Z + , and a set of groups S ⊆ 2 X . Define the objectives G µ mc := { glyph[lscript] µ,i,S,v } i ∈{± 1 } ,S ∈S ,v ∈ V 2 λ and G m mc := { glyph[lscript] m,i,S,v } i ∈{± 1 } ,S ∈S ,v ∈ V 2 λ where

A pair of predictors is an ε -optimal solution to the single-distribution multi-objective learning problem ( { D } , G m mc ∪G µ mc , P ) if and only if the pair is also ( S , 2 ε, λ, m ) mean-conditioned moment multicalibrated.

Thus, we can follow the same steps as before to derive a moment multicalibration algorithm. The algorithm we derive for Theorem E.13 recovers the sample complexity of [22] for moment multicalibration. Moreover, the algorithm is parallelized in that it learns the mean and moment estimators h µ , h m simultaneously, in contrast with [22]'s algorithm which uses nested optimization.

Theorem E.13. Fix ε > 0 , λ, r ∈ Z + , a set of groups S ⊆ 2 X , and a data distribution D . The following algorithm returns a pair of predictors that are ( S , ε, λ, m ) -mean-conditioned moment multicalibrated on D and makes O (1 /ε 2 ) calls to an agnostic learning oracle. Moreover, with probability 1 -δ , the oracle calls can be implemented with ˜ O ( m ln( |S| λ/δ ) /ε 4 ) samples from D .

Construct the problem ( { D } , G m mc ∪G µ mc , P ) from Fact E.12 and let T = Cε -2 ln( |S| λ k δ ) for some universal constant C . Over T rounds, have a learner choose predictors h (1: T ) m ∈ P by applying the no-regret learning algorithm of Theorem C.6 to the stochastic costs glyph[lscript] (1: T ) m , instantiating A to be Hedge with learning T -3 / 4 . In parallel, have a learner choose predictors h (1: T ) µ ∈ P by applying the no-regret learning algorithm of Theorem C.6 to the stochastic costs glyph[lscript] (1: T ) µ , instantiating A to be the strongly adaptive Hedge algorithm (Lemma A.3). Have an adversary choose glyph[lscript] ( t ) m , glyph[lscript] ( t ) m by calling an agnostic learning oracle at each t ∈ [ T ] : glyph[lscript] ( t ) m = A ε/ 8 (1 -glyph[lscript] ( · ) ( h ( t ) ) , G m mc ) and

glyph[lscript] ( t ) µ = A ε/ 8 (1 -glyph[lscript] ( · ) ( h ( t ) ) , G µ mc ) . Using C ln( T/δ ) /ε 2 samples from D , return the predictor h ( t ∗ ) with the lowest empirical multicalibration error. This algorithm is written explicitly in Algorithm 7.

First, we prove an intermediate result.

Lemma E.14. In the Algorithm 7, the sequence of hypotheses h (1: T ) satisfies the bounded regret condition Reg weak ( h (1: T ) , { glyph[lscript] m ( t ) + glyph[lscript] µ ( t ) } (1: T ) ) ∈ O ( √ mT 3 / 4 ) .

Proof. For simplicity, we'll round T to the next largest square. Recall that using Hedge to select T actions from the interval [0 , 1] with learning rate T -3 / 4 m -1 / 2 guarantees a regret bound of 2 ln(2) √ mT 3 / 4 (Lemma A.1). By Theorem C.6, since we chose A to be Hedge with learning rate T -3 / 4 when learning h ( t ) µ , ∑ T t =1 glyph[lscript] ( t ) µ ( h ( t ) ) ≤ 0 . 5 T +2ln(2) √ mT 3 / 4 . Similarly, by Theorem C.6, having chosen A to be strongly adaptive Hedge when learning h ( t ) m , we have for any T 1 ∈ [ T ] and h ∗ m ∈ H m ,

In the above equation, f ( t ) m and g ( t ) m are the separable components of the loss glyph[lscript] ( t ) m , as defined in Definition C.5. We now turn to bound the expectation term. By triangle inequality, f ( t ) m ( x, h ( t ) ( x ))( h ∗ m ( x ) -g ( t ) m ( y )) ≤ f ( t ) m ( x, h ( t ) ( x ))( h ∗ m ( x ) -g ( T 1 ) m ( y )) + ∣ ∣ ∣ g ( T 1 ) m ( y ) -g ( t ) m ( y ) ∣ ∣ ∣ . Using the inequality that | a m -b m | ≤ m | a -b | when a, b ∈ [0 , 1] , and the movement upper bound of Hedge (Lemma A.2), we have ∣ ∣ ∣ g ( T 1 ) m ( y ) -g ( t ) m ( y ) ∣ ∣ ∣ ≤ m ∣ ∣ ∣ h ( t ) µ ( x ) -h ( T 1 ) µ ( x ) ∣ ∣ ∣ ≤ mη µ √ T , where η µ = T -1 / 2 T -3 / 4 is the learning rate of the mean predictor h µ . Thus, for all T 1 , by choosing h ∗ m ( x ) = E [ g ( T 1 ) m ( y ) | x ] , we have that

We now return to proving Theorem E.13.

Proof of Theorem E.13. By Lemma E.14, the learner's regret is Reg weak ( h (1: T ) , { glyph[lscript] ( t ) m + glyph[lscript] ( t ) µ } (1: T ) ) ∈ O ( m 1 / 2 T 3 / 4 ) . When T = Cm 2 /ε 4 , we thus have Reg weak ( h (1: T ) , { max { glyph[lscript] ( t ) m , glyph[lscript] ( t ) µ }} (1: T ) ) ≤ Tε/ 8 . By construction, every max { glyph[lscript] ( t ) m , glyph[lscript] ( t ) µ } is an ( ε/ 8) best-response to the cost 1 - L D, ( · ) h ( t ) . By Lemma 3.4, there must exist a timestep where h ( t ) is ( ε/ 4) -optimal. By Lemma 3.5, the predictors h ( t ∗ ) found by the algorithm are ( ε/ 2) -optimal with probability at least 1 -δ . By Fact E.12, h ( t ∗ ) is a ( S , ε, λ, m ) -mean-conditioned moment multicalibrated predictor.

Algorithm 4 Conditional Multicalibration Algorithm (Theorem E.3)

1: Input: S , S ′ ⊆ 2 X , ε ∈ (0 , 1) , k, λ, T, C ∈ Z + , distributions { D ′ S } S ′ ∈S ′ , and agnostic learning oracle A ;

2: Initialize Prod iterate h (1) = [1 /k, . . . , 1 /k ] X ;

3: for t = 1 to T do

6: end for

7: Take C ln( T/δ ) /ε 2 samples x ( S ′ ) ∼ D S ′ for all S ′ ∈ S ′ , and t ∗ = arg min ∑ glyph[lscript] ( t ) ( h ( t ) , ( x, y )) ;

t ∈ [ T ] ( x,y ) ∈ x ( S ′ ( t ) 8: Return the predictor h ( t ) ;

let ) ∗

Algorithm 5 Non-Deterministic Conditional Multicalibration Algorithm (Theorem E.4)

1: Input: S , S ′ ⊆ 2 X , ε ∈ (0 , 1) , k, λ, T ∈ Z + , and distributions { D ′ S } S ′ ∈S ′ ;

2: Initialize Prod iterate h (1) = [1 /k, . . . , 1 /k ] X and ELP iterate ( S ′ (1) , glyph[lscript] (1) ) ∼ Uniform ( S ′ × G mc ) ;

3: for t = 1 to T do

4: Let h ( t +1) ( x ) = Prod( c (1: t ) x ) where

6: end for

7: Return h ∗ , a uniform distribution over h (1) , . . . , h ( T ) ;

Algorithm 6 Agnostic Deterministic Multicalibration Algorithm (Theorem E.9)

1: Input: S ⊆ 2 X , ε ∈ (0 , 1) , k, λ, T, C ∈ Z + , agnostic learning oracle A , and distribution D ;

2: Initialize Hedge iterate h (1) = [1 /k, . . . , 1 /k ] X ;

3: for t = 1 to T do

5: Let h ( t +1) ( x ) = Hedge( c (1: t ) x ) with

6: end for

7:

Take

C

ln(

T/δ

)

/ε

2

samples

8: Return the predictor h ( t ∗ ) ;

x

from

D

, and let

t

∗

= arg min

t

∈

[

T

]

∑

x,x

′

,y glyph[lscript]

(

t

)

(

h

(

t

)

,

(

x, x

(

)

∈

x

′

, y

))

;

Algorithm 7 Moment Multicalibration Algorithm (Theorem E.13)

1: Input: S ⊆ 2 X , ε ∈ (0 , 1) , k, λ, T, C ∈ Z + , agnostic learning oracle A , and distribution D ;

2: Initialize Hedge iterate h (1) m = 0 . 5 X and Hedge iterate h (1) µ = 0 . 5 X ;

3: for t = 1 to T do

6: end for

7: Take C ln( T/δ ) /ε 2 samples x from D and let t ∗ = arg min ∑ glyph[lscript] ( t ) m ( h ( t ) , ( x, y )) + glyph[lscript] ( t ) µ ( h ( t ) , ( x, y )) ;

8: Return the predictors h ( t ) m , h ( t ) µ ;

t

∈

[

T

]

(

x,y

)

∈

x

∗

∗

E.4 Square-root Multicalibration Guarantees

The following Theorem E.15 is a stronger restatement of Theorem 4.4.

Theorem E.15. Fix ε > 0 , λ, k ∈ Z + and sets of groups S ⊆ 2 X . Set T = C ln( k ) /ε 2 for some universal constant C . Algorithm 8, with probability at least 1 -δ , requires no more than ˜ O (ln( k ) · (ln( k |S| /εδ ) + k ln( λ )) /ε 4 ) samples 6 from D to find a deterministic k -class predictor h satisfying

for all S ∈ S , v ∈ V k λ , j ∈ [ k ] . That is, h is a deterministic predictor that is ( S , √ Pr( x ∈ S ) · ε, λ ) -multicalibrated, where the error tolerance √ Pr( x ∈ S ) · ε depends on the group mass.

Before proceeding to a proof, we first introduce some technical results which are simple variants of lemmas that the reader has seen previously in the manuscript. Note that, in the following multiobjective learning problem construction, we will allow negative objective values for simpler notation.

Fact E.16. Let D be a data distribution for some k -class prediction problem and fix ε > 0 , λ ∈ Z + , and a set of groups S ⊆ 2 X . We define the following set of multicalibration losses:

Predictor p ∈ ∆( P ) is a ε -optimal solution to the multi-objective learning problem ( { D } , G mc , P ) if and only if p is ( S , ε √ Pr D ( x ∈ S ) , λ ) -multicalibrated for D .

Proof. In the multi-objective learning problem ( { D } , G ′ mc , P ) , the multi-objective value of a predictor p , L ∗ ( p ) := max glyph[lscript] i,j,S,v ∈G L D,glyph[lscript] i,j,S,v ( p ) , is exactly the magnitude of the predictor's multicalibration violation. Formally,

The optimal value is 0 , as the objectives are symmetric around 0 and the Bayes clasifier achieves a loss of 0 . Thus, L ∗ ( p ) -min h ∗ ∈P L ∗ ( h ∗ ) ≥ ε if and only if our multicalibration violation is at least ε √ Pr( x ∈ S ) .

Lemma E.17. Let D be a data distribution for some k -class prediction problem, ε ∈ (0 , 0 . 6) , δ ∈ (0 , 1) and fix a set of groups S ⊆ 2 X where, for all S ∈ S , Pr D ( x ∈ S ) ≥ ε 2 . With only

O (ln( S /δ ) /ε 4 ) samples, one can find with probability at least 1 -δ a vector v ∈ R |S| where, for all S ∈ S ,

Proof. Fix some sufficiently large universal constant C . Sample N = C ln( S /δ ) /ε 4 datapoints X from D , and let v S = 1 | X | ∑ ( x,y ) ∈ X 1[ x ∈ S ] . Observe that E [ v ] = [Pr D ( x ∈ S )] S ∈S . The multiplicative Chernoff bound says that, fixing an S ∈ S ,

With our choice of N , taking a union bound over all S ∈ S , with probability at least 1 -δ , we have our first claim. Taking a square-root of both sides, we also have that for all S ∈ S , √ (1 -ε ) Pr D ( x ∈ S ) ≤ √ v S ≤ √ (1 + ε ) Pr D ( x ∈ S ) . The second claim then follows by observing that, for any ε ∈ (0 , 0 . 6) , 1 -1 √ 1+ ε ≤ ε , and 1 √ 1 -ε -1 ≤ ε . The third claim follows by observing that, for any ε ∈ (0 , 1) , √ 1 + ε -1 ≤ ε and 1 - √ 1 -ε ≤ ε .

The following lemma is a modification of Theorem C.6 that states that the learner has a no-regret strategy on the calibration objectives given in Fact E.16.

Lemma E.18. Consider the set of k -class predictors P , a data distribution D , and any adversarial sequence of stochastic costs glyph[lscript] (1: T ) ∈ G ′ mc , where G ′ mc are the multicalibration objectives defined in Lemma E.16. Suppose you are given v ∈ R satisfying (14) and (13) for all S ∈ S and that, for all S ∈ S , Pr D ( x ∈ S ) ≥ ε 2 . There is a no-regret algorithm that outputs (deterministic) predictors h (1: T ) ∈ P ′ such that Reg weak ( h (1: T ) , {L D,glyph[lscript] ( t ) } (1: T ) ) ≤ O ( √ ln( k ) T + εT ) . Moreover, the algorithm does not need any samples from D .

Proof. Consider the following algorithm. At each feature x ∈ X , initialize a Prod algorithm that picks an action h ( t ) ( x ) ∈ ∆( Y ) at each timestep t ∈ [ T ] . Aggregating each algorithm's action yields our learner's overall action h ( t ) ∈ P . For each x ∈ X , let h ( t +1) ( x ) be the outcome of Prod at step t +1 after observing linear loss functions f ( τ ) h ( τ ) ,x : R k → [0 , 1] for τ ∈ [ t ] :

Prod gives ∑ T t =1 f ( t ) h ( t ) ,x ( h ( t ) ( x )) -min z ∗ ∈ ∆( Y ) ∑ T t =1 f ( t ) h ( t ) ,x ( z ∗ ) ≤ C √ ln( k ) ∑ T t =1 v -1 S ( t ) (Lemma A.4) for some universal constant C . Since this inequality holds for all x ∈ X , applying the law of total expectation to Lemma A.4,

with the equality following because we allow arbitrary predictors. Expanding the definition of f ,

By definition of v ,

We also know by construction of the losses in G ′ mc that the weak baseline B weak = 0 . We can therefore bound

To bound the last inequality, let u ∈ R X , v ∈ R X be vectors defined as u x = √ Pr D ( x ) and v x = √ ∑ T t =1 Pr D S ( t ) ( x ) . Then, by Cauchy-Schwarz inequality, we have that u · v ≤ ‖ u ‖ 2 ‖ v ‖ 2 . In other words,

The following lemma states that the adversary can efficiently best-respond to the objectives from Fact E.16.

Lemma E.19. Consider a k -class predictor h ∈ P , a data distribution D , and set of multicalibration objectives G ′ mc . Suppose you are given v ∈ R satisfying (15) for all S ∈ S . There is an algorithm that, taking only ε -2 (ln( |S| /δ )+ k ln( λ )) samples for some universal constant C , returns a glyph[lscript] i,j,S,v ∈ G ′ mc such that, with probability at least 1 -δ , L D,glyph[lscript] i,j,S,v ( h ) + 2 ε ≥ max i ∗ ,j ∗ ,S ∗ ,v ∗ L D,glyph[lscript] i ∗ ,j ∗ ,S ∗ ,v ∗ ( h ) .

Proof. Consider the following algorithm. Draw N = Cε -2 (ln( |S| /δ ) + k ln( λ )) samples ( x, y ) (1: N ) from D . For all S ∈ S , initialize the empty buffer X S = {} . Then, for every value of r = 1 , . . . , N and for every group S ∈ S , if x ( r ) ∈ S , append ( x, y ) ( r ) to X S . Let glyph[lscript] i,j,S,v = max i,j,S,v √ v S Pr D ( x ∈ S ) 1 | X S | ∑ ( x,y ) ∈ X S glyph[lscript] i,j,S,v ( h, ( x, y )) be the multicalibration objective

that minimizes the empirical risk on its respective buffer X S . Note that finding glyph[lscript] i,j,S,v does not require knowledge of Pr D ( x ∈ S ) , as the explicit factor of √ 1 / Pr D ( x ∈ S ) is cancelled out by glyph[lscript] i,j,S,v .

We first observe that | X S | is a binomial random variable with parameters N and Pr( x ∈ S ) . By Chernoff's bound, with probability 1 -δ |S| -1 , | X S | ≥ Pr( x ∈ S ) · C ′ ε -2 (ln( |S| /δ ) + k ln( λ )) for some universal constant C ′ , where we choose C to be large enough so that C ′ ≥ C 2 . By union bound, with probability 1 -δ , for every S ∈ S , | X S | ≥ Pr( x ∈ S ) N/ 2 .

Condition on this event and fix a glyph[lscript] i,j,S,v ∈ G ′ mc . We observe that each ( x, y ) ∈ X S is an unbiased sample from D S , we have at least Pr( x ∈ S ) N/ 2 samples, and glyph[lscript] i,j,S,v ∈ ±{ 1 / √ Pr( x ∈ S ) } . Thus, by Chernoff's bound, with probability at least 1 -δ |G ′ mc | -1 ,

Thus with probability at least 1 -δ , for all glyph[lscript] i,j,S,v ∈ G ′ mc ,

Taking a union bound over all glyph[lscript] ∈ G ′ mc , by uniform convergence, with probability at least 1 -2 δ , our returned glyph[lscript] i,j,S,v is an ε best-response to the cost function glyph[lscript] i,j,S,v ↦→-√ v S Pr D ( x ∈ S ) L D,glyph[lscript] i,j,S,v ( h ( t ) ) . By (15), max glyph[lscript] i ∗ ,j ∗ ,S ∗ ,v ∗ L D,glyph[lscript] i ∗ ,j ∗ ,S ∗ ,v ∗ ( h ( t ) ) - L D,glyph[lscript] i,j,S,v ( h ( t ) ) ≤ ε . Thus, glyph[lscript] i,j,S,v is an 2 ε bestresponse to the cost function glyph[lscript] i,j,S,v ↦→-L D,glyph[lscript] i,j,S,v ( h ( t ) ) .

Proof of Theorem E.15. First, we will assume without loss of generality that, for every group S ∈ S , Pr D ( x ∈ S ) ≥ ε 2 / 32 . This is because for such S , our multicalibration constraint is trivially satisfied for tolerances of at least ε/ √ 32 . We can remove such S from our group set S by testing if Pr D ( x ∈ S ) ≤ ε 2 / 32 ; O ( ln( |S| /δ ) ε 4 ) samples suffices.

Next, we will sample v according to Lemma E.17, which also takes at most O ( ln( |S| /δ ) ε 4 ) samples. Lemma E.18 then guarantees that, since T ≥ C ln( k ) /ε 2 , Reg weak ( h (1: T ) , {L D,glyph[lscript] ( t ) ( · ) } (1: T ) ) ≤ Tε/ 8 . By Lemma E.19, glyph[lscript] ( t ) is an ( ε/ 8) best-response to the cost function -L D, ( · ) ( h ( t ) ) with probability at least 1 -δ/T at each timestep t . By Lemma 3.4, since the learner has at most Tε/ 8 weak regret and the adversary is ε/ 8 best-responding, there is a timestep t ∈ [ T ] where h ( t ) is ( ε/ 4) -optimal for the problem ( { D } , G ′ mc , P ) . By Lemma 3.5, the h ( t ) found by the algorithm is ( ε/ 2) -optimal with probability at least 1 -δ . By Fact E.16, this is an ( S , ε, λ ) -multicalibrated predictor.

F Other Fairness Notions

This general framework of approaching multi-objective learning problems with game dynamics can be extended beyond multicalibration. In this section, we use multi-objective learning to derive new guarantees for multi-distribution learning and multi-group learning.

Algorithm 8 Conditional Multicalibration Algorithm (Theorem E.15)

1: Input: S ⊆ 2 X , ε ∈ (0 , 1) , k, λ, T, C ∈ Z + , and distribution D ;

2: Initialize Prod iterate h (1) = [1 /k, . . . , 1 /k ] X ;

3: Sample C ln( S /δ ) /ε 4 datapoints X from D and, for each S ∈ S , remove S from S if 1 | X | ∑ ( x,y ) ∈ X 1[ x ∈ S ] ≤ ε 2 / 32 ;

4: Sample C ln( S /δ ) /ε 4 datapoints X from D , and let v S = 1 | X | ∑ ( x,y ) ∈ X 1[ x ∈ S ] for all S ∈ S ;

5: for t = 1 to T do

7: For all S ∈ S , let X t,S = { ( x, y ) i t | i ∈ [ N ] , x i ∈ S } ;

6: Draw N = Cε -2 (ln( |S| T/δ ) + k ln( λ )) samples ( x, y ) (1: N ) t from D ;

8: Let c adv ( t ) ( S, glyph[lscript] ) := √ v S | X t,S | -1 ∑ ( x,y ) ∈ X t,S glyph[lscript] ( h ( t ) , ( x, y )) ;

9: Let glyph[lscript] ( t ) = arg max glyph[lscript] ∈G ′ mc c adv ( t ) ( S, glyph[lscript] ) ;

10: If c adv ( t ) ( S ( t ) , glyph[lscript] ( t ) ) ≤ ε/ 8 terminate and return h ( t ) ;

( t +1) (1: t )

12: end for

13: Return p ∗ , a uniform distribution over h (1) , . . . , h ( T ) ;

Competing in multi-distribution learning. Usually, in agnostic multi-objective learning problems, the trade-off between objectives is arbitrated by the worst-off objectives max D ∗ ∈D ,glyph[lscript] ∗ ∈G L D ∗ ,glyph[lscript] ∗ ( · ) . However, this approach to negotiating trade-offs may be suboptimal when some objectives are inherently more difficult. In those cases, we can take into account the difficulty of individual objectives by asking for a predictor where there is no objective for which a competitor h ∈ H ∗ performs significantly better.

Definition F.1. For a multi-objective learning problem ( D , G , H ) , a solution that is ε -competitive with respect to a class H ′ is a hypothesis p ∈ ∆( H ) satisfying,

Only a simple modification is needed to provide ε -competitive guarantees: amplify your original objectives G into a new objective set G ′ := { 1 2 (1 + glyph[lscript] ( · ) -glyph[lscript] ( h ′ )) | glyph[lscript] ∈ G , h ′ ∈ H ′ } and solve as usual. The following fact, which holds by definition, formalizes this reduction.

Fact F.2. Consider a multi-objective learning problem ( D , G , H ) . For some choice of H ′ , let G ′ := { 0 . 5 + 0 . 5( glyph[lscript] ( · ) -glyph[lscript] ( h ′ )) | glyph[lscript] ∈ G , h ′ ∈ H ′ } . Any solution p that is ε -optimal for the multiobjective learning problem ( D , G ′ , H ) , is also 2 ε -competitive (Definition F.1) w.r.t. H ′ for the original problem ( D , G , H ) .

[19] showed that the sample complexity of finding an ε -optimal solution to a multi-objective learning problem ( D , G , H ) is O ( ε -2 (ln( |H| ) + |D| ln( |D| |G| /δ ))) . Thus, Fact F.2 immediately implies Theorem 5.1, our sample complexity bound for finding an ε -competitive solution.

Algorithm 9 Multi-Group Learning Algorithm (Theorem 5.3)

1: Input: S ⊆ 2 X , ε ∈ (0 , 1) , glyph[lscript] : Y ×Y → [0 , 1] , hypothesis class H : X → Y , and distribution D ;

2: Initialize Hedge iterate q (1) = Uniform ( S × H ) and p (1) = Uniform ( H ) ;

3: for t = 1 to T do

4: Sample ( x ( t ) q , y ( t ) q ) ∼ D and ( x ( t ) p , y ( t ) p ) ∼ D ;

5: Let q ( t +1) := Hedge( c (1: t ) q ) and p ( t +1) := Hedge( c (1: t ) p ) where

6: end for

7: Return: p ∗ , a uniform distribution over p (1) , . . . , p ( T ) ;

Multi-group learning. Consider the multi-group learning problem (Definition 5.2) where, rather than seeking simultaneously calibrated estimates of different subsets of the domain as in multicalibration, we seek to simultaneously minimize a general loss function on different subsets of the domain [37].

A (near) optimal sample complexity for multi-group learning of O ( ln( |S| |H| ) /ε 2 ) was attained by [39] using a reduction to sleeping experts. [39] also asked whether there exists a simpler optimal algorithm that does not rely on sleeping experts. We answer this affirmatively by designing an optimal algorithm that just runs two Hedge algorithms.

We can equate the multi-group learning problem (Definition 5.2) to finding an ε -competitive solution in a single-distribution multi-objective learning problem.

Fact F.3. Fix ε > 0 , a set of groups S ⊆ 2 X , a hypothesis class H , data distribution D , and a loss glyph[lscript] : H× ( X × Y ) → [0 , 1] . We define glyph[lscript] S ( h, ( x, y )) := glyph[lscript] ( h, ( x, y )) · 1[ x ∈ S ] . If a hypothesis p ∈ ∆( H ) is ε -competitive with respect to the class H for the single-distribution multi-objective learning problem ( { D } , { glyph[lscript] S } S ∈S , H ) , then p is also an ε -optimal solution to the multi-group learning problem ( S , H ) .

The following-a formalization of Theorem 5.3-is a direct consequence of running no-regret no-regret dynamics, as in Lemma 3.3.

Theorem F.4 (Multi-group learning) . Fix a set of groups S ⊆ 2 X , loss glyph[lscript] : Y×Y → [0 , 1] , hypothesis class H : X → Y , and distribution D ; For some universal constant C and T = C ln( |S| |H| ) /ε 2 , Algorithm 9 takes 2 T = O ( ln( |S| |H| ) /ε 2 ) samples from D and returns an ε -optimal solution to the multi-group learning problem ( S , H ) .

G Empirical Results

G.1 Experiment Setup

We conduct three sets of experiments to evaluate different batch multicalibration algorithms. The three sets of experiments we conduct correspond to three datasets: the UCI Adult Income dataset [26], a real-world dataset for predicting individuals' incomes based on the US Census, the UCI Bank Marketing dataset [32], a dataset for predicting whether an individual will subscribe to a bank's term deposit, and the Dry Bean Dataset [27], a dataset for predicting a dry bean's variety.

In every experiment, the performance of a multicalibration algorithm is measured based on the multicalibration violations of the average iterate and last iterate, where multicalibration violations are as defined in Definition 2.1. In every experiment, we empirically evaluate six multicalibration algorithms. Four algorithms are based on no-regret best-response dynamics, using an empirical risk minimizer as the adversary and implementing either Hedge [14] (Hedge-ERM), Prod [28] (Prod-ERM), Optimistic Hedge [36] (OptHedge-ERM), or Gradient Descent (GD-ERM) as the learner. Two algorithms are based on no-regret no-regret dynamics, using either Hedge (HedgeHedge) or Optimistic Hedge (OptHedge-OptHedge) as both the learner and adversary. We also note

that the most commonly used multicalibration algorithms are included in these comparisons. The original multicalibration algorithm of [21] is equivalent to GD-ERM. The revised, boosting-inspired, multicalibration algorithm of [25, 7] is equivalent to Hedge-ERM.

G.2 Results

The results of these experiments are summarized in Table 3, which reports the multicalibration errors of each algorithm's final iterate and average iterate. In Appendix G, Figure 1 plots the evolution of training and testing multicalibration errors over the duration of the training process. We identify two key trends that are statistically significant and hold consistently in all three experiments.

Figure 1 plots the evolution of training and testing multicalibration errors over the duration of the training process. These plots confirm that the relative performance of different multicalibration algorithms is fairly monotonic and regular, even across the duration of training and the learning rate schedule.

The last iterates of no-regret no-regret dynamics are surprisingly multicalibrated. On all datasets, the algorithms based on no-regret no-regret dynamics, namely Hedge-Hedge and OptHedgeOptHedge, consistently yield not only among the most multicalibrated randomized predictors (with their average iterate) but also the most multicalibrated deterministic predictors (with their last iterate). This is surprising because the last iterate of these algorithms is not guaranteed to be multicalibrated, and only enjoys a theoretical advantage over no-regret best-response algorithms in terms of average iterate guarantees. As corroborated by Figure 1, this trend does not appear to be an artifact of early stopping or learning rates, but may rather indicate that their more stable adversary updates provide regularization to these algorithms.

Multicalibration Errors On Adult Income Dataset

Calibration Error

0.100

0.1

Hedge

Hedge-Hedge

0.100

Calibration

OptHedge

0.010

OptHedge-OptHedge

Prod

6 × 10-2

Training

0.001

Hedge

Hedge-Hedge

Testing

4 × 10-2

OptHedge

0.000

OptHedge-OptHedge

3 × 10-2

Prod

0

10

20

30

40

50

0

10

20

30

40

50

Iterations

Iterations

Multicalibration Errors On Bank Market Dataset

g Calibration Error

0.100

Testing Calibration Error

Hedge

Hedge-Hedge

0.100

OptHedge-OptHedge

OptHedge

0.010

Prod

Training

Hedge

0.001

Hedge-Hedge

OptHedge

OptHedge-OptHedge

Prod

0.000

0

10

20

30

40

50

0

10

20

30

40

50

Iterations

Iterations

Multicalibration Errors On Dry Beans Dataset

0.100

Hedge

Calibration Error

0.100

Hedge

Hedge-Hedge

Hedge-Hedge

OptHedge

OptHedge

OptHedge-OptHedge

OptHedge-OptHedge

Prod

Prod

Testing

6 × 10-2

0.010

0

20

40

60

80

100

0

20

40

60

80

100

Iterations

Iterations

One's choice of no-regret algorithm matters. On all datasets, we see that the best multicalibration results are consistently achieved by algorithms that instantiate the Optimistic Hedge no-regret algorithm. This is consistent with the theoretical results of [36], which show that Optimistic Hedge converges faster than a standard Hedge in games. We also see that the original multicalibration algorithm of [21], based on gradient descent, consistently attains the worst multicalibration errors, both in terms of average-iterate and last-iterate. This is consistent with gradient descent being a theoretically less effective no-regret algorithm, as it is unstable near the boundaries of a probability simplex.

Due to the superficial similarity between boosting and multicalibration, the field has already begun adopting multicalibration algorithms with Hedge's multiplicative updates rather than gradient descent's additive ones, as suggested by [25]. Our findings offer the first theoretical and empirical endorsement of this shift. Moreover, our results suggest that practitioners should further explore the use of Optimistic Hedge-based algorithms and algorithms based on no-regret no-regret dynamics-even when one is only interested in deterministic predictors.

G.3 Experiment Details

The source code for these experiments is included in the repository https://github.com/ ericzhao28/multicalibration . Model checkpoints for replicating our results can be found at https://drive.google.com/drive/folders/1CVusrPZkB-15_55VVkoU3KrLXQGZzne4? usp=sharing . All experiments were performed on a 2021 MacBook Pro, with a M1 Pro chip. The total compute time for replicating the experiments in this section, including hyperparameter tuning, is approximately 20 hours.

UCI Adult Income Dataset. The UCI Adult Income dataset [26] is a dataset for predicting individuals' incomes based on the US Census. Our experiments use the dataset's binary 'income' attribute as the target label and form 129 protected groups using eight of the dataset's labeled attributes: 'age', 'workclass', 'education', 'marital-status', 'occupation', 'relationship', 'race', and 'sex'. We perform random 80-20 train/test splits of the dataset, resulting in approximately 24000 training samples, 6000 test samples, and 130 groups. We discretize the label space into 0 . 1 -width bins ( λ = 10 ). Experiments are repeated for 20 seeds, with multicalibration algorithms running for 50 iterations.

UCI Bank Market Dataset. The UCI Bank Marketing dataset [32] is a dataset for predicting individuals' subscriptions to a term deposit at a bank. Our experiments use the dataset's binary 'y' attribute as the target label and form 129 protected groups using eight of the dataset's labeled attributes: 'age', 'job', 'marital', 'education', 'default', 'housing', 'loan', and 'contact'. We round ages to the nearest age divisible by 5 since 'age' is a continuous attribute. We perform random 80-20 train/test splits of the dataset, resulting in approximately 36000 training samples, 9000 test samples, and 180 groups. We discretize the label space into 0 . 1 -width bins ( λ = 10 ). Experiments are repeated for 5 seeds, with multicalibration algorithms running for 50 iterations.

Dry Bean Dataset. The Dry Bean Dataset [27] is a dataset for predicting a dry bean's variety using its physical attributes. Our experiments use the dataset's bean 'type' attribute (which takes 7 possible values) as the target label and form 80 protected groups using eight of the dataset's labeled attributes: 'perimeter', 'major-axis-length', 'minor-axis-length', 'aspect-ratio', 'eccentricity', 'convex-area', and 'equivalent-diameter'. We discretize all numerical/continuously-valued attributes by evenly dividing the range of possible values into 10 segments. We perform random 80-20 train/test splits of the dataset, resulting in approximately 11000 training samples, 2700 test samples, and 80 groups. We discretize the label space into 0 . 25 -width bins ( λ = 4 ). Experiments are repeated for 5 seeds, with multicalibration algorithms run for 100 iterations.

Hyperparameter tuning. In each experiment, the learning rates of the algorithms are tuned on the training set using 10 seeds (Adult Income dataset), 5 seeds (Bank Market dataset), and 5 seeds (Dry Bean dataset). We sweep over the learning rate decay rates of η ∈ [0 . 8 , 0 . 85 , 0 . 9 , 0 . 95] for the learner and (if applicable) η ∈ [0 . 9 , 0 . 95 , 0 . 98 , 0 . 99] for the adversary, where the learning rate of the learner at the t th iteration is η t and the adversary is 100 · η t . In the Dry Bean dataset, learning rates are universally doubled to 2 η t and the adversary is 200 · η t . The selected learning rate decays are summarized below.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a novel approach to multicalibration by leveraging game dynamics and no-regret learning algorithms. The central idea is that multicalibration can be modeled as a multi-objective learning problem where an adversary and a learner play against each other, guided by no-regret dynamics.\n\nThe authors propose three types of dynamics: no-regret no-regret (NRNR), no-regret best-response (NRBR), and best-response no-regret (BRNR), each serving different purposes. These dynamics are used to establish the multicalibration algorithms that match or improve the fastest known sample complexity rates for multicalibration, provide deterministic multicalibrated predictors, and offer online multicalibration.\n\nThis work also extends the application of these dynamics to other fairness notions, specifically multi-group learning and multi-distribution learning. For the multi-group learning problem, the authors devise an optimal multi-group learning algorithm that relies on NRNR dynamics and is simpler than the existing approaches.\n\nThe authors validate their theoretical claims with empirical results on a few standard datasetst. The results highlight the importance of effective no-regret algorithms for better multicalibration, with the Optimistic Hedge outperforming others in the no-regret no-regret dynamics.\n\nOverall, this paper contributes to the field by providing a unified, game-theoretic framework for multicalibration that unites disparate existing results and offers more efficient solutions, both in theory and practice. Moreover, it highlights the broader applicability of game dynamics and no-regret algorithms to other fairness notions in machine learning.\n\nSTRENGTHS:\nThe paper introduces a novel perspective by approaching multicalibration and other fairness notions using game dynamics and no-regret learning algorithms. It presents multicalibration as a multi-objective learning problem within a game-theoretic framework. However, while the application to multicalibration is new, the use of no-regret algorithms and game-theoretic models is well-established in other areas of calibration literature.\n\nThe paper offers a technically sound approach with rigorous mathematical derivations. The theorems are well-proved, and the proposed algorithms are clearly detailed. However, the empirical section could be expanded upon to strengthen the validation of the theoretical claims with real-world data.\n\nThe authors have done a reasonable job in articulating complex concepts and methodologies, with the paper being generally well-structured. The use of tables aids understanding, though more insights can be provided for the technical results.\n\nThe work has the potential to unify and extend prior work on multicalibration and other fairness notions. However, the impact of the work may depend heavily on how effectively these theoretical insights can be translated into practical applications. Furthermore, while the findings from the empirical results could guide the selection of no-regret algorithms in multicalibration tasks, the experimental validation is relatively limited and does not fully exploit the range of situations where these algorithms could be applied.\n\nWEAKNESSES:\nThe paper provides a theoretically strong development of multicalibration algorithms using game dynamics. However, there seems to be a gap between the theoretical development and the empirical results. The authors could improve this aspect by designing more comprehensive experiments that test a wide range of scenarios to validate their theoretical claims. These could include different types of datasets, varying levels of complexity, and possibly real-world use cases.\n\nThe experimental evaluation is currently limited to only a few standard but simple dataset. This may not sufficiently test the robustness of the proposed algorithms under different conditions. More experiments with diverse datasets would provide a better understanding of the algorithms' performance and potential limitations. Additionally, it would be beneficial to compare the proposed methods with more baseline or state-of-the-art algorithms for multicalibration to understand the relative performance.\n\nAlso, the paper could be improved by providing more detailed descriptions of the proposed algorithms. While the authors do discuss the high-level ideas behind using game dynamics for multicalibration, it may be beneficial for readers to have more specific details about how these algorithms are implemented.\n\nAlthough the authors discuss several previous works, it is not entirely clear how their contributions improve upon or differ from these existing methods. Besides theoretical comparison, a clearer empirical comparison would make the authors' contributions more evident.\n\nThe paper could benefit from a more explicit discussion of the limitations of the proposed methods.\n\nQUESTIONS:\nThe paper provides an improved theoretical framework for multicalibration using game dynamics. However, it seems there's a disconnect between the theory and empirical results presented. Could the authors elaborate on why specific experiments were chosen and how they directly validate the theoretical developments?\n\nThe empirical evaluation mainly focuses on the standard datasets like UCI. How do the authors envision their proposed algorithms performing on other types of datasets, especially those with different characteristics, such as image datasets or high-dimensional problem? Could they provide any insights or theoretical guarantees on their algorithms' robustness across varied datasets?\n\nThe high-level descriptions of the proposed algorithms are appreciated, but having more concrete, step-by-step details of the algorithm implementations would be very helpful. Could the authors provide pseudocode or a more in-depth explanation of their proposed algorithms?\n\nThe paper could benefit from a more detailed comparison with previous works. How do the authors' contributions improve upon or differentiate from existing methods, specifically in terms of performance and applicability?\n\nThe paper presents a novel framework that generalizes previous works. How far can this generalization go? Are there any theoretical or practical constraints that limit its applications? Are there any lower bounds? Any insight into this would be beneficial for the understanding and application of the framework.\n\nLIMITATIONS:\nThe authors do not provide a clear discussion of the limitations of their work or potential negative societal impacts, which is a critical part of any comprehensive research paper. This is an area where the paper could be improved. Here are a few suggestions for potential limitations and societal impact considerations:\n\nDiscuss the computational complexity and scalability of the proposed methods. This is crucial for real-world applications, particularly when dealing with large and complex datasets.\n\nGiven that the paper deals with fairness notions in learning, the authors could consider discussing the implications of misapplication of these methods. While the goal is to improve fairness, any shortcomings or misuse could unintentionally lead to bias or unfair outcomes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper provides a two-player dynamics framework that seeks to unify many strands of recent work on multicalibration and multiobjective optimization. With three possible setups considered: No regret against No regret, Best response against No regret, and Best response against Best response, efficient algorithms are provided to demonstrate that these generic setups apply in particular to (multi-)calibration-like constraints. The framework is then applied to several existing and new settings in the literature with the goal of improving some convergence bounds or, respectively, simplifying and streamlining the analysis of specific algorithms. Some experiments are given that show what happens when proposed dynamics are run with specific well-known no regret algorithms in the driver's seat.\n\nSTRENGTHS:\nTo me, the main strength of this paper is the rather clean nature of the setup, which promises that by combining, in any combination, best-responding and no-regret agents, it is possible (though not without further work) to recover existing multigroup fairness results. Previous guarantees are (with a couple exceptions) improved somewhat marginally, or greater simplicity of the framework is claimed in some other cases, but overall it is the generality of this framework --- showcased by its adaptability and ability to recover various insights about calibration (not just regret bounds, but also questions of the simplicity of the output calibrated model, amount of randomness required, etc) --- that is the paper's main forte.\n\nAdditionally, the conditional multicalibration setup presented in the paper (and generalizing previous conditional results in the quantile setting) looks like an interesting addition to the literature.\n\nWEAKNESSES:\nNo particular weaknesses, other than the relatively well-studied nature of, and relatively small gains in, some of the applications (but no big deal), as well as the somewhat terse and cramped presentation.\n\nFor instance, there are a few \"intuitive-sounding\" claims made about where NRNR, BRNR, BRBR may be applied to the greatest utility, but the actual applications are then scattered throughout the paper rather than recalled immediately (there is also Table 1, but I would appreciate a more intuitive dive into which dynamic was used where and why, at the point where these dynamics were actually introduced). For another example, I would be interested in further comments on potential alternative definitions of conditional multicalibration and a slightly expanded treatment of the comparison to the existing 1/sqrt(# people in group) type of guarantee. \n\nStill, the paper is overall solid and well-done and I enjoyed reading it.\n\nQUESTIONS:\nSee the above section.\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-07"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work exploits connections to game dynamics to propose a unifying algorithmic framework to address the multicalibration problem which has been recently used for tackling fairness concerns in machine learning. More precisely, based on the classic game dynamics approach used in learning problems, it is shown that multicalibration results can be seen as learning dynamics for two-player zero-sum games relying on no-regret algorithms or best response dynamics. Using this unifying framework, the paper recovers some guarantees for existing multicalibration algorithms with simplified proofs and results. Multi-objective learning guarantees are shown and new guarantees are also established for several multicalibration settings including in particular an exponential reduction in the complexity of k-class multicalibration over prior work. Experiments were conducted to evaluate the empirical performance of multicalibration algorithms on some real-world datasets.\n\nSTRENGTHS:\n-  The paper proposes a unifying approach using game dynamics which recovers prior results with simpler proofs and establishes novel guarantees improving over prior work in multiple settings. \n- This paper is well-written, the presentation is very clear. \n- While I am not familiar with the calibration/multicalibration literature and I did not go through the long appendix supporting the main part in details, the results are sound, quite rigorously exposed and the proofs which leverage for instance prior online learning results seem solid to the best of my knowledge. \n- Several additional experimental results are also provided in the appendix to support the theoretical findings.\n\nWEAKNESSES:\nRegarding novelty, the related work section mentions in l. 55-56 that ‘no work has established a broad connection between no-regret learning and multicalibration’. No-regret learning has been priorly used for calibrated forecasting as acknowledged by the paper. While it is mentioned that multicalibration ‘has very different challenges than calibrated forecasting’, the discussion regarding these is reduced to a single line l. 57-58 in the related work section. Given that one of the main contributions of this work is the unifying framework based on the connection with game dynamics, I would expect a more detailed discussion to further clarify the novelty with respect to prior work and calibrated forecasting regarding this particular aspect. For instance, some proofs such as the one of Theorem 3.8 follow similar lines to the proof of calibration as also mentioned in l. 246-247. Moreover, it seems that this connection to game dynamics was known for multi-objective learning and multicalibration is linked to it via the simple facts 2.5 and 2.6.\n\nQUESTIONS:\n- The game dynamics approach to multi-objective learning is a ‘common approach’ as mentioned in l. 138. The paper connects multicalibration to multi-objective learning in Facts 2.5 and 2.6 which leads to a connection between multicalibration and the game dynamics approach. Are Facts 2.5 and 2.6 novel results? Is there a particular reason for using ‘Facts’ for these results instead of ‘Lemma’ for instance? Is this related to the simplicity of the proofs in Appendix B once the losses are defined as in the facts? \n\n\n- How do you justify the equality in l. 230-231?\n\n- Section 3.1 is about multicalibration but section 3 is about multi-objective learning. Section 4 is 'Multicalibration with Game Dynamics'. Would it be more appropriate to move section 3.1 to section 4? \n\n**Minor questions and suggestions:**\n\n- Why is the baseline of (2) not also used in (1) so that the formulations are unified?\n\n- The notations max and min are used for minimization over a hypothesis class or maximization over a class of loss functions and a set of data distributions. How are these guaranteed to be reached without further assumptions? Are the classes supposed to be finite or are the more general notations sup and inf more appropriate? For instance, cardinalities are used in Lemma 3.5 while the preliminaries do not seem to specify the nature of the classes. \n\n- Minor suggestion: in Definition 2.2., you could keep the same notation as in Definition 2.1 and only change the dependence on $t$ for $D$ and $p$ to ease the reading (with the same order for the quantities and a single indicator function). \n\n - l. 82: $\\Delta(\\mathcal{Y})^{\\mathcal{X}}$ with parenthesis.\n\nLIMITATIONS:\nLimitations of the results do not seem to be clearly discussed throughout the paper to the best of my knowledge.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-14"}

comments: SUMMARY:
The authors proposed a unified framework for multicalibration learning by exploiting its connection to the game dynamics in multi-objective learning. Strong theoretical guarantees were given and its extension to address group fairness was discussed.

STRENGTHS:
1. The analysis of the game dynamics in multi-objective learning was novel and strong.
2. Connection between multicalibration learning and game dynamics in multi-objective learning was well exploited, unified framework was given.
2. The author gave a clear presentation of the key ideas of the work despite substantial material.

WEAKNESSES:
1. The experiment parts seem to compare with algorithms within the proposed framework. Is there any comparison with existing baseline algorithms?
2. How does the multi-objective learning discussed in the paper related to the Pareto optimal one?
3. Given different learner choices shown in the empirical section, is there any learner who fits the proposed framework better?

QUESTIONS:
see weakness.

LIMITATIONS:
NA
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 4
DATE: 2023-07-18

comments: SUMMARY:
This paper provides a unifying framework for the algorithm design and performance analysis of multicalibrated predictors.
In this paper, the multicalibraion problems is placed in the setting of multi-objective learning.
Under this interpretation, approaches based on game dynamics is proposed and analyzed.
It is shown that this approach yields improved performance guarantees.

STRENGTHS:
- Bounds are improved for a variety of problems.
- Experimental results support the effectiveness of the proposed method.

WEAKNESSES:
- The approach based on game dynamics does not seem very surprising as it is a common approach to transform min-max optimization into a problem of finding an equilibrium solution by interpreting it as a zero-sum game.
- I have concerns about whether the definition is consistent with that of existing studies. (please refer to "Questions")


Minor comments:
- When citing existing results in Table 1, etc., it would be better to indicate the theorem number or the relevant section. I had a hard time checking the corresponding part.

QUESTIONS:
- In previous studies (e.g., [17,21,34]), multicalibrated predictors appear to be defined in terms of conditional expectation.
On the other hand, in Definition 2.1 of this paper, it appears to be defined in some sense by joint probabilities.
I believe this means that the errors bounded in 2.1 are smaller than those adopted as definitions in previous studies.
This raises concerns about whether comparisons with existing research bounds are valid.
I would appreciate an answer to this concern to see if there is a problem.
- Is there lower bounds that can be compared to the results obtained?

LIMITATIONS:
I have no concerns about the limitations and potential negative societal impact.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-26