SlowGuess/NeurIPS_2023_OCR huggingface dataset:
A Unifying Perspective on Multicalibration: Game Dynamics for Multi-Objective Learning*
Nika Haghtalab, Michael I. Jordan, and Eric Zhao
University of California, Berkeley {nika,jordan,eric.zh}@berkeley.edu
We provide a unifying framework for the design and analysis of multicalibrated predictors. By placing the multicalibration problem in the general setting of multi-objective learning—where learning guarantees must hold simultaneously over a set of distributions and loss functions—we exploit connections to game dynamics to achieve state-of-the-art guarantees for a diverse set of multicalibration learning problems. In addition to shedding light on existing multicalibration guarantees and greatly simplifying their analysis, our approach also yields improved guarantees, such as error tolerances that scale with the square-root of group size versus the constant tolerances guaranteed by prior works, and improving the complexity of  $k$ -class multicalibration by an exponential factor of  $k$  versus Gopalan et al. [17]. Beyond multicalibration, we use these game dynamics to address emerging considerations in the study of group fairness and multi-distribution learning.
Multicalibration has emerged as a powerful tool for addressing fairness considerations in machine learning. Based on calibrated forecasting [6, 12]—which requires among instances  $x$  a predictor  $h$  predicts the probability  $h(x) = v$ , a fraction  $v$  have a positive outcome—multicalibration yields more fine-grained guarantees by seeking calibration across large and possibly overlapping collections of sub-populations [21]. Multicalibration has been studied in numerous settings, including those with rich label sets (multi-class multicalibration [17]), adversarial rather than stochastic data (online multicalibration [18]), and problems where no Bayes classifier exists (agnostic multicalibration [38]). The concept of multicalibration has also been applied to estimate other quantities, such as higher moments [22], and been strengthened in various ways, such as providing conditional guarantees [2].
Multicalibration's versatility has led to the development of numerous specialized algorithms, each tailored to a unique multicalibration problem and requiring its own individualized analysis. Promising attempts to provide an overarching conceptual framework for multicalibration, such as outcome indistinguishability [7], have had limited success in unifying these various algorithms. In this paper, we tackle this challenge, developing a general-purpose algorithmic framework to guide the design of multicalibrated learning algorithms for a wide range of settings and considerations.
Our approach is a dynamical systems and game-theoretic approach [see, e.g., 13, 9]. We demonstrate that many multicalibration algorithms can be formulated as particular instances of two-player zero-sum games where players independently either run no-regret algorithms or best-response algorithms. A wide range of multicalibration algorithms that exhibit varying trade-offs can be obtained by plugging in different no-regret and best-response algorithms. This unified framework both recovers existing guarantees and in many cases improves upon them.
*Authors are ordered alphabetically. Correspondence to eric.zh@berkeley.edu.
Problem	Complexity	Dynamic	Previous Results	Our Results	Reference
MC (Det)	Oracle	NRBR	O(kε-2) [17]	O(ln(k)ε-2)	Thm 4.2
MC (Sqrt Err, Det)	Sample	NRBR	O(ε-6√k ln(k |S|))	O(ε-4 ln(k |S|))	Thm 4.4
Agnostic MC (Det)	Oracle	NRBR	O(|X| ε-2) [38]	O(ε-2)	Thm E.9
Agnostic MC	Sample	NRNR	O(|X| ε-2) [38]	O((d+k)ε-2)	Thm E.10
Cond. MC (Det)	Oracle	NRBR	O(|S|2 ε-2)	O(|S| ε-2)	Thm E.3
Cond. MC	Sample	NRNR	O(|S|2 (d+k)ε-2)	O(|S| (d+k)ε-2)	Thm E.4
MC (Succinct)	Sample	NRNR	O((d+k)ε-3) [17]	O((d+k)ε-2)	Thm 4.1
Online MC	Regret	BRNR	O(√log(|S|)T) [18]	(Matching)	Thm 4.3
rth Moment MC	Oracle	NRBR	O(rε-4) [22]	(Matching)	Thm E.13
Table 1: This table summarizes the sample complexity and agnostic learning oracle complexity rates we obtain for multicalibration (MC), compared with the previous state of the art.  $S$  denotes the set of groups one wants multicalibration on,  $d$  the VC dimension of  $S$ ,  $k$  the number of label classes, and  $\varepsilon$  the error tolerance. (Det) and (Succinct) respectively denote when only deterministic or only succinct predictors are acceptable. "Sqrt Err" refers to requiring an error tolerance of  $\varepsilon \sqrt{\operatorname*{Pr}(x\in S)}$  for each group rather than  $\varepsilon$ .
Although approaching multicalibration—a min-max optimization problem—with game dynamics seems like an obvious approach, no prior work has succeeded in using a game dynamics framing to unify multicalibration algorithms. The primary challenge is not reducing multicalibration to min-max optimization, which is straightforward (Facts 2.5,2.6), but rather solving the resulting equilibrium computation problem in a way that connects to practical algorithms. The needs of multicalibration (such as determinism, large and complex predictor spaces, etc.) differ significantly in this regard from earlier applications of general-purpose no-regret algorithms and game dynamics.
Our primary contributions can be categorized into three areas.
1) Unifying framework. In Section 3, we give an overview of game dynamics as a concise but general framework for obtaining multi-objective learning guarantees. To use these dynamics as a generic solution to various multicalibration problems, we introduce a powerful general-purpose no-regret algorithm (Theorem 3.7) and a distribution-free best-response algorithm (Theorem 3.8), for calibration-like objectives. This approach allows us to unify the diverse—and often, seemingly unrelated—algorithms that have been studied in the multicalibration literature, offering a general template for their derivation and analysis.
2) New guarantees. In Sections 4, we use our framework to improve guarantees for various multicalibration settings (see Table 1) by simply plugging in different no-regret and best-response algorithms. These improvements include an exponential (in  $k$ ) reduction in the complexity of  $k$ -class multicalibration over Gopalan et al. [17], a polynomial (in  $1 / \varepsilon$ ) reduction in the complexity of learning a succinct multicalibrated predictor over Dwork et al. [7], the first conditional multicalibration results for the batch setting, the first agnostic multicalibration guarantees that beats uniform convergence, and a multicalibration algorithm that efficiently guarantees an error tolerance that scales with square-root of each group's probability mass. In Section 5, we demonstrate that our framework can be extended to analyze problems beyond multicalibration, such as multi-group learning [39].
3) Simplified analyses. We also demonstrate that our framework can recover the guarantees of various existing multicalibration algorithms, including online multicalibration [18] and moment multicalibration [22], while avoiding intricate and problem-specific arguments.
The study of calibration originated in online (adversarial) forecasting [6, 20], with classical literature having also studied calibration across multiple sub-populations [11]. Multicalibration, on the other hand, is classically studied in the stochastic setting; i.e., where  $(x_{i},y_{i})\sim D$ , in which case calibration is trivially satisfied by the predictor  $h(x) = \mathbb{E}[y]$ . Due to this difference in formulation in the literature on calibration and that on multicalibration, the specific technical tools that have been developed in these areas have largely remained distinct. Our work can be viewed as bridging this gap by showing
that game-theoretic dynamics provides a unified foundation for studying multicalibration, just as no-regret learning underpins the study of calibration.
Motivated by fairness considerations, a formal definition of multicalibration was presented by [21], and has found a wide range of applications and conceptual connections to Bayes optimality, conformal predictions, and computational indistinguishability [21, 22, 16, 18, 7, 23]. Algorithms for multicalibration have largely developed along two lines: one studying oracle-efficient boosting-like algorithms [see, e.g., 21, 25, 17, 7] and another studying algorithms with flavors of online optimization [see, e.g., 18, 35]. Our work establishes that the contrast between these lines of work and the algorithms they develop are entirely attributable to different choices of game dynamics.
Multi-objective and multi-distribution learning are concepts that have found broad applications in addressing fairness, collaboration, and robustness challenges. Blum et al. [4] initiated the study of learning predictors with near-optimal accuracy across multiple populations (distributions), with later works attaining tight bounds [19, 34, 31]. Recently, multi-objective learning has been applied to settings where sub-populations are mutually compatible [37, 39, 3]. In Section 5, we use our framework to match and improve guarantees relative to this literature.
We use  $\mathcal{X}$  to denote a feature space and  $\mathcal{Y}$  a label space, where  $\mathcal{Y} = [k]$  in  $k$ -class classification. A data distribution  $D$  is a probability distribution supported on labeled datapoints  $\mathcal{X} \times \mathcal{Y}$ . We use  $\mathcal{H}$  to denote a set of hypotheses and  $\mathcal{G}$  a set of objectives, where an objective—or equivalently, a loss—is a function  $\ell: \mathcal{H} \times (\mathcal{X} \times \mathcal{Y}) \to [0,1]$  that takes a hypothesis and datapoint and returns a penalty value. We denote expected objective values by  $\mathcal{L}_{D,\ell}(h) := \mathbb{E}_{(x,y) \sim D}[\ell(h,(x,y))]$ . For non-deterministic  $p \in \Delta(\mathcal{H})$  and  $q \in \Delta(\mathcal{D} \times \mathcal{G})$ , we overload notation to write  $\mathcal{L}_q(p) := \mathbb{E}_{h \sim p,(D,\ell) \sim q}[\mathcal{L}_{D,\ell}(h)]$ . We often use the shorthands  $x^{(1:T)} := x^{(1)}, \ldots, x^{(T)}$  and  $\{f(x^{(t)})\}_{t=1}^{(1:T)} = f(x^{(1)}), \ldots, f(x^{(T)})$ . We also write  $f(a,\cdot)$  to denote the function  $x \mapsto f(a,x)$  or  $f_{(\cdot)}(a)$  to denote  $x \mapsto f_x(a)$ . For  $y \in [k]$ ,  $\delta_y \in \{0,1\}^k$  denotes its one-hot delta function, while for  $y \in [k]$ ,  $j \in [k]$  we write  $\delta_{y,j} = 1[y = j]$ .
2.1 Multicalibration
We use  $\mathcal{P} = (\Delta \mathcal{Y})^{\mathcal{X}}$  to denote the set of all  $k$ -class predictors which are maps from features to label distributions. To differentiate between predictors and distributions over predictors, we refer to  $h \in \mathcal{P}$  as a deterministic predictor, and  $p \in \Delta(\mathcal{P})$ , as a non-deterministic predictor. Calibration is a property of predictors  $h \in \mathcal{P}$  requiring, for example in binary classification, that among instances  $x$  assigned prediction probability  $h(x) = [1 - v, v]$  a fraction  $v$  are truly labeled 1. Multicalibration [21] is the finer-grained notion that requires calibration on subgroups of one's domain. This set of subgroups is typically finite or of finite VC dimension. In practice, we work with approximate notions of calibration/multicalibration and discretize the range of probability assignment. In  $k$ -class prediction, we partition the  $k$ -dimensional hypercube into  $\lambda^k$  equal cubes  $V_\lambda^k$ , where  $V_\lambda := \{[0, 1/\lambda), [1/\lambda, 2/\lambda), \ldots\}$ . For any interval  $v \in V_\lambda^k$ , we use  $h(x) \in v$  to denote that prediction  $h(x)$  falls pointwise in the buckets of  $v$ , i.e.,  $h(x)_j \in v_j$  for all  $j \in [k]$ . We next formally define multicalibration.
Definition 2.1. Fix  $\varepsilon >0$ ,  $\lambda \in \mathbb{Z}_{+}$ , and a set of groups  $S\subseteq 2^{\mathcal{X}}$ . A (possibly non-deterministic)  $k$ -class predictor  $p\in \Delta (\mathcal{P})$  is  $(S,\varepsilon ,\lambda)$ -multicalibrated for some data distribution  $D$  if
$$
\forall S \in \mathcal {S}, v \in V _ {\lambda} ^ {k}, j \in [ k ]: \left| \underset {(x, y) \sim D, h \sim p} {\mathbb {E}} \left[ (h (x) _ {j} - \delta_ {y, j}) \cdot 1 [ h (x) \in v, x \in S ] \right] \right| \leq \varepsilon .
$$
That is,  $p$  is calibrated on every level set  $v \in V_{\lambda}^{k}$  of every group  $S \in \mathcal{S}$  for every class  $j \in [k]$ . We are often specifically interested in a deterministic solution; that is, where  $p \in \mathcal{P}$ .
2Importantly, determinism of a predictor  $h$  does not imply that  $h\in \mathcal{V}^{\mathcal{X}}$  as opposed to  $\Delta (\mathcal{Y})^{\mathcal{X}}$
This discretized (binned) definition of multicalibration follows convention and can be readily translated into others [21]. We also follow convention in measuring in the  $\ell_{\infty}$  norm, the  $\ell_2$  norm is also commonly used in calibration. Some definitions of multicalibration [e.g., 21] appear as conditional expectations, but for constant probability domain subgroups, which makes them equivalent to our definition.
The batch setting, where we want to find a multicalibrated predictor for some fixed data distribution  $D$ , is the most commonly studied. Multicalibration can also be defined for online settings where the data distribution changes adversarially over time.
Definition 2.2 (Online multicalibration). Fix  $\varepsilon >0$ ,  $\lambda \in \mathbb{Z}_{+}$ , and a set of groups  $S\subseteq 2^{\mathcal{X}}$ . In online multicalibration, at every timestep  $t\in [T]$ , a learner chooses a  $k$ -class predictor  $p^{(t)}\in \Delta (\mathcal{P})$ . Nature, which observes  $p^{(1:t)}$ , responds with any choice of data distribution  $D^{(t)}$ . The learner's predictors  $p^{(1:T)}$  are  $(S,\varepsilon ,\lambda)$ -online multicalibrated on  $D^{(1:T)}$  if
$$
\forall S\in \mathcal{S},v\in V_{\lambda}^{k},j\in [k]:\left|\frac{1}{T}\sum_{t = 1}^{T}\underset { \begin{array}{c}h\sim p^{(t)}\\ (x,y)\sim D^{(t)} \end{array} }{\mathbb{E}}[1[x\in S]\cdot 1[h(x)\in v]\cdot (h(x)_{j} - \delta_{y,j})]\right|\leq \varepsilon .
$$
2.2 Multi-Objective Learning
We use multi-objective learning (a generalization of multi-distribution learning introduced by [19]) as a tool for studying multicalibration and other related problems. The goal of multi-objective learning is to find a hypothesis that simultaneously minimizes a set of objective values.
Definition 2.3 (Multi-objective learning). A multi-objective learning problem  $(\mathcal{D},\mathcal{G},\mathcal{H})$  consists of a set of objectives  $\mathcal{G}$ , a hypothesis class  $\mathcal{H}$ , and a set of data distributions  $\mathcal{D}$ . An  $\varepsilon$ -optimal solution to  $(\mathcal{D},\mathcal{G},\mathcal{H})$  is a (potentially non-deterministic) hypothesis  $p\in \Delta (\mathcal{H})$  where
$$
\max  _ {D \in \mathcal {D}, \ell \in \mathcal {G}} \mathcal {L} _ {D, \ell} (p) \leq \min  _ {h ^ {*} \in \mathcal {H}} \max  _ {D ^ {*} \in \mathcal {D}, \ell^ {*} \in \mathcal {G}} \mathcal {L} _ {D ^ {*}, \ell^ {*}} \left(h ^ {*}\right) + \varepsilon . \tag {1}
$$
We usually prefer a deterministic solution  $p \in \mathcal{H}$  over a non-deterministic solution  $p \in \Delta(\mathcal{H})$ .
We mainly consider single-distribution multi-objective problems, where  $\mathcal{D} = \{D\}$ , though multi-distribution multi-objective learning arises in conditional multicalibration (Section E.1) and group fairness (Section 5). We can also consider multi-objective learning in online adversarial settings.
Definition 2.4 (Online multi-objective learning). An online multi-objective learning problem  $(\mathcal{D},\mathcal{G},\mathcal{H})$  consists of a set of distributions  $\mathcal{D}$ , objectives  $\mathcal{G}$  and hypothesis class  $\mathcal{H}$ . At each timestep  $t\in [T]$ , a learner first picks a hypothesis  $p^{(t)}\in \Delta (\mathcal{H})$ . Nature, who sees  $p^{(1:t)}$ , responds with a data distribution  $D^{(t)}\in \mathcal{D}$ . We say the hypotheses  $p^{(1:T)}$  are  $\varepsilon$ -optimal on the distributions  $D^{(1:T)}$  if
$$
\frac {1}{T} \max  _ {\ell \in \mathcal {G}} \sum_ {t = 1} ^ {T} \mathcal {L} _ {D ^ {(t)}, \ell} (p ^ {(t)}) \leq \max  _ {D ^ {*} \in \mathcal {D}} \min  _ {h ^ {*} \in \mathcal {H}} \max  _ {\ell^ {*} \in \mathcal {G}} \mathcal {L} _ {D ^ {*}, \ell^ {*}} \left(h ^ {*}\right) + \varepsilon . \tag {2}
$$
In many problems we consider, like online multicalibration, Nature can pick from any data distribution; that is,  $\mathcal{D}$  is unrestricted. Let us also note that the baseline in the right-hand-side of (2) differs from that of (1). This is intentional: when  $\mathcal{D}$  is unrestricted, the min-max baseline of (1) may be large, since there may be no hypothesis that is simultaneously good for all distributions. Instead, the max-min baseline of (2) is the best hypothesis  $h^*$  for the most difficult distribution  $D^*$ .
Multicalibration as multi-objective learning. (Batch) Multicalibration is a single-distribution multi-objective learning problem, whose objectives penalize over-estimation and under-estimation of label probabilities on subsets of the domain.
Fact 2.5. Let  $D$  be a data distribution for some  $k$ -class prediction problem and fix  $\varepsilon > 0$ ,  $\lambda \in \mathbb{Z}_{+}$  and a set of groups  $S \subseteq 2^{\mathcal{X}}$ . For every direction  $i \in \{\pm 1\}$ , level set  $v \in V_{\lambda}^{k}$ , group  $S \in S$ , and class  $j \in [k]$ , we define an objective  $\ell_{i,j,S,v}: \mathcal{P} \to [0,1]$  where
$$
\ell_ {i, j, S, v} (h, (x, y)) = 0. 5 + 0. 5 \cdot i \cdot 1 [ h (x) \in v, x \in S ] \cdot (h (x) _ {j} - \delta_ {y, j}) \tag {3}
$$
and  $\mathcal{G}_{\mathrm{mc}} := \{\ell_{i,j,S,v}\}_{i,j,S,v}$  is the set of these objectives. Predictor  $p \in \Delta(\mathcal{P})$  is a  $\varepsilon$ -optimal solution to the multi-objective problem ( $\{D\}, \mathcal{G}_{\mathrm{mc}}, \mathcal{P}$ ) if and only if  $p$  is  $(S,2\varepsilon,\lambda)$ -multicalibrated for  $D$ .
Online multicalibration is similarly an online multi-objective learning problem.
Fact 2.6. Let  $D^{(1:T)}$  be data distributions for some  $k$ -class prediction problem,  $\mathcal{D}$  be the set of all data distributions, and  $\mathcal{G}_{\mathrm{mc}}$  be as defined in (3). Fix  $\varepsilon > 0$ ,  $\lambda \in \mathbb{Z}_+$ , and a set of groups  $S \subseteq 2^{\mathcal{X}}$ . A sequence of predictors  $p^{(1:T)} \in \Delta(\mathcal{P})$  is  $\varepsilon$ -optimal on  $D^{(1:T)}$  for the online multi-objective learning problem  $(\mathcal{D}, \mathcal{G}_{\mathrm{mc}}, \mathcal{P})$  if and only if  $p^{(1:T)}$  is  $(\mathcal{S}, 2\varepsilon, \lambda)$ -online multicalibrated on  $D^{(1:T)}$ .
3 Tools for Solving Multi-Objective Learning using Game Dynamics
A common approach to multi-objective learning is to imagine a game between a minimizing player who proposes hypotheses and a maximizing player who proposes objectives and data distributions. It is well-established (inspired by min-max equilibria, e.g., [14]) that a solution can be obtained when players use no-regret algorithms. However, considerations that arise in multicalibration motivate us to study a broader range of dynamics and their implications than has been commonly explored.
Online learning. In an online learning problem, at each timestep  $t \in [T]$ , a learner chooses an action  $a^{(t)} \in \mathcal{A}$  which an adversary observes and responds to with a cost function  $c^{(t)}: \mathcal{A} \to [0,1]$ . We will usually assume costs to be linear maps. The learner's regret is defined as  $\operatorname{Reg}(a^{(1:T)}, c^{(1:T)}) := \sum_{t=1}^{T} c^{(t)}(a^{(t)}) - \min_{a^* \in \mathcal{A}} \sum_{t=1}^{T} c^{(t)}(a^*)$ , which no-regret algorithms like Hedge [14] can bound.
Lemma 3.1 (Hedge Regret [24]). In an online learning problem where the action set is the simplex  $\mathcal{A} = \Delta_k$  and costs are linear, the actions chosen by Hedge have a regret of at most  $2\sqrt{\ln(k)T}$ .
We write no-regret algorithms as a function of a sequence of cost functions  $\mathrm{Alg}:([0,1]^{\mathcal{A}})^{*}\to \mathcal{A}$ . For example, when  $\mathcal{A} = \Delta_k$ , the output of the Hedge algorithm is defined as  $\mathrm{Hedge}(c^{(1)},\ldots ,c^{(t)}) = [w_1 / \| w\| _1,\dots ,w_k / \| w\| _1]$  where  $w_{i} = \exp (-\eta \sum_{\tau = 1}^{t}c^{(t)}(\delta_{i}))$ , for a specific choice of  $\eta$ .
We sometimes define regret against a different baseline  $B \in \mathbb{R}$ , with  $\mathrm{Reg}_B(a^{(1:T)}, c^{(1:T)}) := \sum_{t=1}^T c^{(t)}(a^{(t)}) - B$ . For example, we often consider the min-max baseline  $B_{\mathrm{weak}} := T \cdot \min_{a^* \in \mathcal{A}} \max_{c^* \in \mathcal{C}} c^*(a^*)$ , where  $\mathcal{C}$  is the set of cost functions that the adversary chooses from. We also often encounter stochastic cost functions, functions of form  $c: \mathcal{A} \times (\mathcal{X} \times \mathcal{Y}) \to [0,1]$  for which we want to minimize expected value  $\mathcal{L}_{D,c} := \mathbb{E}_{(x,y) \sim D}[c(\cdot, (x,y))]$  on some distribution  $D$ . A stochastic cost  $c$  is linear if, for every  $x \in \mathcal{X}, y \in \mathcal{Y}, c(\cdot, (x,y))$  is a linear map. The regret of online learning algorithms on stochastic costs concentrates quickly.
Lemma 3.2 (Stochastic Approximation [33], Lemma 3.1). Consider an online learning problem on the simplex  $\Delta_k$  with linear stochastic costs. Suppose, after each timestep  $t \in [T]$  that is, after picking the action  $a^{(t)}$  we estimate the expected cost  $\mathcal{L}_{D,c^{(t)}}$  with  $\widehat{c}^{(t)}(a) := c^{(t)}(a,(x^{(t)},y^{(t)}))$ , where  $(x^{(t)},y^{(t)}) \stackrel{\mathrm{i.i.d.}}{\sim} D$ . With probability at least  $1 - \delta$ ,  $\left|\operatorname{Reg}(a^{(1:T)},\{\mathcal{L}_{D,c^{(t)}}\}^{(1:T)}) - \operatorname{Reg}(a^{(1:T)},\widehat{c}^{(1:T)})\right| \leq O(\sqrt{T\ln(k/\delta)})$ .
We say an action  $a$  is an  $\varepsilon$ -best response to a cost  $c$  if  $c(a) \leq \min_{a^* \in \mathcal{A}} c(a^*) + \varepsilon$ . Similarly, an action  $a$  is a distribution-free  $\varepsilon$ -best response to a stochastic cost  $c$  if  $\max_{x \in \mathcal{X}, y \in \mathcal{Y}} c(a, (x, y)) \leq \max_{x \in \mathcal{X}, y \in \mathcal{Y}} \min_{a^* \in \mathcal{A}} c(a^*, (x, y)) + \varepsilon$ . We sometimes express the sample complexity of an algorithm in terms of the number of calls made to an agnostic learning oracle, which can compute  $\varepsilon$ -best-responses to stochastic cost functions using few samples. See Appendix A for a discussion.
Game dynamics. We consider no-regret dynamics between a learner (minimizing player) who chooses hypotheses  $p^{(t)} \in \Delta(\mathcal{H})$  and an adversary (maximizing player) who chooses data distributions and objectives  $q^{(t)} \in \Delta(\mathcal{D} \times \mathcal{G})$  where the learner's loss is  $\mathcal{L}_{q^{(t)}}(p^{(t)})$ . When both players are no-regret, the time-average actions they picked quickly converge to an approximate solution for the multi-objective learning problem  $(\mathcal{D}, \mathcal{G}, \mathcal{H})$ . This method of no-regret dynamics has long played a role in empirical convergence to notions of equilibria [14]. Here, we review these dynamics and their convergence guarantees. While the proofs of these lemmas are standard at a high level (and deferred to Appendix C.1), they differ in fundamental ways from past work. In particular, these lemmas consider weak regret, single timestep solutions (instead of time-averaged ones), and the consequences of having distribution-free best responses, all of which play important roles in multicalibration.
Lemma 3.3 (No-Regret vs. No-Regret (NRNR)). Consider a multi-objective learning problem  $(\mathcal{D},\mathcal{G},\mathcal{H})$ , where a learner and adversary chose  $p^{(1:T)} \in \Delta(\mathcal{H})$  and  $q^{(1:T)} \in \Delta(\mathcal{D} \times \mathcal{G})$ . If both players are no-regret,  $\operatorname{Reg}_{weak}\left(p^{(1:T)},\{\mathcal{L}_{q^{(t)}}(\cdot)\}^{(1:T)}\right) \leq T\varepsilon$  and  $\operatorname{Reg}(q^{(1:T)},\{-\mathcal{L}_{(\cdot)}(p^{(t)})\}^{(1:T)}) \leq T\varepsilon$ , then the non-deterministic hypothesis  $\overline{p} = \operatorname{Uniform}(p^{(1:T)})$  is a  $2\varepsilon$ -optimal solution.