SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Algorithm	Train Error	Test Error	Test Error (Ergodic)
Hedge-Hedge (NRNR)	2.0e-2 ± 2.0e-3	3.0e-2 ± 3.0e-3	2.3e-4 ± 2.7e-5
OptHedge-OptHedge (NRNR)	7.0e-3 ± 0.0	2.7e-2 ± 3.0e-3	2.6e-4 ± 2.8e-5
OptHedge-ERM (NRBR)	0.0 ± 0.0	4.7e-2 ± 1.0e-3	4.8e-4 ± 9.0e-6
Hedge-ERM (NRBR)	0.0 ± 0.0	6.4e-2 ± 1.0e-3	6.4e-4 ± 1.1e-5
Prod-ERM (NRBR)	0.0 ± 0.0	5.3e-2 ± 4.0e-3	5.3e-4 ± 4.4e-5
GD-ERM (NRBR)	5.3e-2 ± 1.1e-2	8.3e-2 ± 3.0e-3	9.5e-4 ± 6.5e-5
Table 2: Average (± standard error) of multicalibration violations on UCI Adult Dataset. Train Error and Test Error evaluate the last iterate (deterministic predictor) on training and test splits; Test Error (Ergodic) measures the average iterate (non-deterministic predictor) on test split. GD-ERM (NRBR) is worst and OptHedge-OptHedge (NRNR) is best on both deterministic and last iterates.
One's choice of no-regret algorithm matters. The original multicalibration algorithm of [21], which is based on gradient descent, consistently attains the worst multicalibration errors, both in terms of average-iterate and last-iterate. This is consistent with gradient descent being a theoretically less effective no-regret algorithm, due to instability near the boundaries of a probability simplex. Due to the superficial similarity between boosting and multicalibration, the field has already begun adopting multicalibration algorithms with Hedge's multiplicative updates rather than gradient descent's additive ones [25]. Our findings offer the first theoretical and empirical endorsement of this shift.
The last iterates of no-regret no-regret dynamics are surprisingly multicalibrated. The algorithms based on no-regret no-regret dynamics, namely Hedge-Hedge and OptHedge-OptHedge, consistently yield not only among the most multicalibrated randomized predictors (with their average iterate) but also the most multicalibrated deterministic predictors (with their last iterate). Note that these algorithms only enjoy a theoretical advantage over no-regret best-response algorithms in terms of average iterate guarantees. This does not appear to be an artifact of early stopping or learning rates (Figure 1), but may rather indicate that their more stable adversary updates provide regularization.
Acknowledgements. This work was supported in part by the National Science Foundation under grant CCF-2145898, a C3.AI Digital Transformation Institute grant, and the Mathematical Data Science program of the Office of Naval Research. This work was partially done while Haghtalab and Zhao were visitors at the Simons Institute for the Theory of Computing.
1.1 Related Work 2
2 Preliminaries 3
2.1 Multicalibration 3
2.2 Multi-Objective Learning 4
3 Tools for Solving Multi-Objective Learning using Game Dynamics 5
3.1 No-regret and Best Response Computation in Multicalibration 6
4 Multicalibration with Game Dynamics 7
5 Other Fairness Notions 9
6 Empirical Results 10
A Additional Background 14
B Proofs for Section 2 16
C Proofs for Section 3 17
C.1 Game Dynamics 17
C.2 No-Regret and Best-Response Computation in Multicalibration 19
D Proofs and Algorithms for Section 4 23
D.1 Batch Multicalibration 23
D.2 Online Multicalibration 25
E Additional Multicalibration Considerations 27
E.1 Conditional Multicalibration 27
E.2 Agnostic Multicalibration 29
E.3 Moment Multicalibration 32
E.4 Square-root Multicalibration Guarantees 35
F Other Fairness Notions 38
G Empirical Results 40
G.1 Experiment Setup 40
G.2 Results 41
G.3Experiment Details 43
In this section, we supplement our discussion of online learning with some additional notations and results.
No-regret algorithms. In the following lemma, we state the regret bound of the Hedge algorithm [14] for general choices of the learning rate  $\eta \in (0,1)$ .
Lemma A.1 (Hedge [24]). Consider an online learning problem on the simplex  $\Delta_k$  and any adversarial sequence of linear cost functions  $c^{(1:T)}$ . If Hedge, with learning rate  $\eta \in (0,1)$ , outputs  $a^{(1:T)}$ , then  $\mathrm{Reg}(a^{(1:T)}, c^{(1:T)}) \leq T\eta + \frac{\ln(k)}{\eta}$ . If  $\eta = \sqrt{\ln(|\mathcal{A}|)/T}$ ,  $\mathrm{Reg}(a^{(1:T)}, c^{(1:T)}) \leq 2\sqrt{\ln(k)T}$ .
It is helpful to note that the learning rate of the Hedge algorithm also bounds how far its iterates move in the primal space.
Lemma A.2 (Hedge Iterate Stability). Consider an online learning problem on the interval  $[0,1]$  and any sequence of linear costs. Let  $a^{(1:T)} \in [0,1]$  be the actions that get picked by Hedge instantiated with learning rate  $\eta \in (0,1)$ . Then, for every timestep  $t \in [T]$ ,  $|a^{(1)} - a^{(t)}| \leq 2\eta T$ .
Proof. By the triangle inequality, it suffices to prove that the learner's actions move by at most  $2\eta$  at each timestep. That is,  $\left|a^{(t + 1)} - a^{(t)}\right|\leq 2\eta$  at every timestep  $t\in [T]$ . By definition of the Hedge algorithm, the learner's action at timestep  $t$  is given by
$$
a ^ {(t + 1)} = \frac {\exp (- \eta \sum_ {\tau = 1} ^ {t} c ^ {(\tau)} (1))}{\exp (- \eta \sum_ {\tau = 1} ^ {t} c ^ {(\tau)} (0)) + \exp (- \eta \sum_ {\tau = 1} ^ {t} c ^ {(\tau)} (1))}.
$$
We can rearrange this as  $a^{(t + 1)} = a^{(t)} \cdot \alpha \cdot \exp (-\eta c^{(t)}(1))$ , where  $\alpha$  is a ratio of normalization terms defined as
$$
\alpha := \frac {\exp (- \eta \sum_ {\tau = 1} ^ {t - 1} c ^ {(\tau)} (0)) + \exp (- \eta \sum_ {\tau = 1} ^ {t - 1} c ^ {(\tau)} (1))}{\exp (- \eta \sum_ {\tau = 1} ^ {t - 1} c ^ {(\tau)} (0)) \cdot \exp (- \eta c ^ {(t)} (0)) + \exp (- \eta \sum_ {\tau = 1} ^ {t - 1} c ^ {(\tau)} (1)) \cdot \exp (- \eta c ^ {(t)} (1))}.
$$
Note that  $\alpha \geq 1$ . Let us write  $x_{1} = \exp (-\eta \sum_{\tau = 1}^{t - 1}c^{(\tau)}(0))$ ,  $x_{2} = \exp (-\eta \sum_{\tau = 1}^{t - 1}c^{(\tau)}(1))$ ,  $x_{3} = \exp (-\eta c^{(t)}(0))$  and  $x_{4} = \exp (-\eta c^{(t)}(1))$ , noting that the values  $x_{1}, x_{2}, x_{3}, x_{4}$  are all positive. Observe that  $\frac{x_1 + x_2}{x_1 \cdot x_3 + x_2 \cdot x_4} \leq \max \left\{\frac{1}{x_3}, \frac{1}{x_4}\right\}$ . To see this, suppose without loss of generality that  $x_{4} \geq x_{3}$ . Then  $x_{2} \leq \frac{x_{2} \cdot x_{4}}{x_{3}}$ , which implies that  $x_{1} + x_{2} \leq \frac{x_{1} \cdot x_{3} + x_{2} \cdot x_{4}}{x_{3}}$ , or equivalently,  $\frac{x_1 + x_2}{x_1 \cdot x_3 + x_2 \cdot x_4} \leq \frac{1}{x_3}$ . Substituting in our values for  $x_{1}, x_{2}, x_{3}, x_{4}$ , we therefore have
$$
1 \leq \alpha \leq \max \{\frac {1}{\exp (- \eta c ^ {(t)} (0))}, \frac {1}{\exp (- \eta c ^ {(t)} (1))} \} \leq \frac {1}{\exp (- \eta)}.
$$
The second inequality is because costs are bounded in  $[0,1]$ .
This gives the iterate movement bound of
$$
\begin{array}{l} \left| a ^ {(t + 1)} - a ^ {(t)} \right| \leq \max  \left\{\left| a ^ {(t)} \frac {\exp (- \eta c ^ {(t)} (1))}{\exp (- \eta)} - a ^ {(t)} \right|, \left| a ^ {(t)} \exp (- \eta c ^ {(t)} (1)) - a ^ {(t)} \right| \right\} \\ \leq \max  \left\{\left| a ^ {(t)} \exp (\eta) - a ^ {(t)} \right|, \left| \eta c ^ {(t)} (1) \right| \right\}. \\ \end{array}
$$
Here, the second inequality applies the fact that  $|e^{-x} - 1| \leq |x|$  for any  $x \geq 0$  to the right-hand value. Since  $\eta \in (0,1)$ , we can use the fact that  $|\exp(\eta) - 1| \leq 2\eta$ . Thus, we attain the desired bound  $|a^{(t + 1)} - a^{(t)}| \leq 2\eta$ .
We can modify the Hedge algorithm to obtain strongly adaptive regret bounds that provides guarantees for every contiguous interval.
Lemma A.3 (Strongly Adaptive Regret [5]). Consider an online learning problem on the simplex  $\Delta_{k}$ . There is a modified Hedge algorithm [5] that, for any adversarial sequence of linear costs  $c^{(1:T)}$ ,
outputs actions  $a^{(1:T)}$  such that, for every interval  $T_1 < T_2 \leq T$ ,
$$
\begin{array}{l} \operatorname {R e g} \left(a ^ {\left(T _ {1}: T _ {2}\right)}, c ^ {\left(T _ {1}: T _ {2}\right)}\right) := \sum_ {t = T _ {1}} ^ {T _ {2}} c ^ {(t)} \left(a ^ {(t)}\right) - \min  _ {a ^ {*} \in \mathcal {A}} \sum_ {t = T _ {1}} ^ {T _ {2}} c ^ {(t)} \left(a ^ {*}\right) \\ \leq O \left(\left(\sqrt {\ln (k)} + \ln (T)\right) \sqrt {T _ {2} - T _ {1}}\right). \\ \end{array}
$$
There are also online learning algorithms that provide second-order regret bounds. We present the regret bound of one such algorithm, Prod [15].
Lemma A.4 (Second-Order Regret Bound of Prod [15]). Consider an online learning problem on the simplex  $\mathcal{A} = \Delta_k$  and any adversarial sequence of linear costs  $c^{(1:T)}$ . If the Prod algorithm of [15] is used and outputs the actions  $a^{(1:T)}$ , then
$$
\operatorname {R e g} (a ^ {(1: T)}, c ^ {(1: T)}) \leq O \left(\max _ {a ^ {*} \in \mathcal {A}} \sqrt {\ln (k) \sum_ {t = 1} ^ {T} (c ^ {(t)} (a ^ {(t)}) - c ^ {(t)} (a ^ {*})) ^ {2}}\right).
$$
We also know that there are adversarial bandit algorithms with sublinear regret guarantees.
Lemma A.5 (Semi-Bandit Regret Bounds [19]). Consider an online learning problem on the simplex  $\Delta_k$ , and any adversarial sequence of linear costs  $c^{(1:T)}$ . Let  $\mathcal{I}$  be a partition of  $[k]$  into  $r$  groups. There is a high-probability variant of [29]'s ELP algorithm that, for any adversarial sequence of costs  $c^{(1:T)}$ , outputs a sequence of mixtures  $a^{(1:T)} \in \Delta_k$  such that, with probability at least  $1 - \delta$ ,
$$
\operatorname {R e g} \left(a ^ {(1: T)}, c ^ {(1: T)}\right) \leq O \left(\sqrt {r \ln (k / \delta) T}\right).
$$
Moreover, after each time the algorithm chooses  $a^{(t)}$ , the algorithm samples an integer  $i^{(t)} \sim a^{(t)}$  (unseen by the adversary). Let  $\mathcal{I}(i^{(t)})$  is the group in  $\mathcal{I}$  that  $i^{(t)}$  belongs to. The algorithm will only ever observe the components of its cost vector corresponding to  $\mathcal{I}(i^{(t)})$ :  $\{c^{(\tau)}(\delta_i) \mid i \in I(i^{(t)})\}^{(t)}$ .
Best responses. An action  $a$  is an  $\varepsilon$ -best response to a cost function  $c: \mathcal{A} \to [0,1]$  if  $c(a) \leq \min_{a^* \in \mathcal{A}} c(a^*) + \varepsilon$ . An agnostic learning oracle  $\mathcal{A}_{\varepsilon}: (\mathcal{A} \to [0,1]) \times 2^{\mathcal{A}} \to \mathcal{A}$  is a function that, given a cost function  $c$  and subset of actions  $\mathcal{A}' \subseteq \mathcal{A}$ , computes an  $\varepsilon$ -best response  $\mathcal{A}_{\varepsilon}(c, \mathcal{A}')$  to  $c$  from  $\mathcal{A}'$ . When  $\mathcal{A}' = \mathcal{A}$ , we may write  $\mathcal{A}_{\varepsilon}(c)$  without the second argument. Agnostic learning oracles are given this name because they are usually used to find best-responses to the expected values of stochastic cost functions using some number of samples. Through this paper, we design learning algorithms that only interact with a data distribution through querying an agnostic learning oracle; the oracle complexity of such an algorithm is defined as the number of agnostic oracle calls that the algorithm makes. Using standard sample complexity bounds, all our algorithms also have a corresponding sample complexity that circumvent the use of agnostic learning oracles.
Another concept we encounter in online settings is the distribution-free best response. An action  $a$  is a distribution-free  $\varepsilon$ -best response to a stochastic cost  $c$  if
$$
\max  _ {x \in \mathcal {X}, y \in \mathcal {Y}} c (a, (x, y)) \leq \max  _ {x \in \mathcal {X}, y \in \mathcal {Y}} \min  _ {a ^ {*} \in \mathcal {A}} c \left(a ^ {*}, (x, y)\right) + \varepsilon . \tag {7}
$$
Best-response algorithms. We can efficiently find an  $\varepsilon$  best-response to the expected value of a stochastic cost function using agnostic learning oracles. For example, computing a single  $\varepsilon$  best-response requires at most  $O(\ln(|\mathcal{A}|)/\varepsilon^2)$  samples by uniform convergence. In game dynamics, we will often need to provide best-responses for a sequence of stochastic cost functions. These sequences are usually adaptive, in that which stochastic cost functions appear later in the sequence depending on how we responded to previous stochastic cost functions. Adaptive data analysis provides sample-efficient algorithms for these settings.
Lemma A.6 (Adaptive Data Analysis [1] Corollary 6.4). There is an algorithm that, for any adaptive sequence of stochastic costs  $c^{(1:T)}$ , is guaranteed with probability  $1 - \delta$  to  $\varepsilon$ -best respond to each cost in  $\{\mathcal{L}_{D,c^{(t)}}\}^{(1:T)}$  while drawing at most
$$
O \left(\frac {\sqrt {T}}{\varepsilon^ {2}} \ln \left(\frac {| \mathcal {A} |}{\varepsilon}\right) \ln^ {3 / 2} \left(\frac {1}{\varepsilon \delta}\right)\right) \approx \widetilde {O} \left(\frac {\sqrt {T} \ln (| \mathcal {A} | / \delta)}{\varepsilon^ {2}}\right)
$$
samples from  $D$ . Here,  $\widetilde{O}(\cdot)$  suppresses  $\ln(1/\varepsilon)$  and  $\ln^{1/2}(1/\delta)$  factors.
Remark A.7. It often suffices, for our results, that a sequence of actions  $a^{(1:T)}$  is on average  $\varepsilon$ -best responding to a cost sequence  $c^{(1:T)}$ ; that is,  $\sum_{t=1}^{T} c^{(t)}(a^{(t)}) \leq \sum_{t=1}^{T} \min_{a^* \in \mathcal{A}} c^{(t)}(a^*) + T\varepsilon$ . Thus, it may be possible to use more efficient minimization oracles than Lemma A.6 in our algorithms.
B Proofs for Section 2
This section proves Fact 2.5 and Fact 2.6 from Section 2.
and  $\mathcal{G}_{\mathrm{mc}} \coloneqq \{\ell_{i,j,S,v}\}_{i,j,S,v}$  is the set of these objectives. Predictor  $p \in \Delta(\mathcal{P})$  is a  $\varepsilon$ -optimal solution to the multi-objective problem ( $\{D\}, \mathcal{G}_{\mathrm{mc}}, \mathcal{P}$ ) if and only if  $p$  is  $(S,2\varepsilon,\lambda)$ -multicalibrated for  $D$ .
Proof. In the multi-objective learning problem  $(\mathcal{D},\mathcal{G}_{\mathrm{mc}},\mathcal{P})$ , the multi-objective value of a predictor  $p$ ,  $\mathcal{L}^{*}(p)\coloneqq \max_{D^{*}\in \mathcal{D},\ell^{*}\in \mathcal{G}}\mathcal{L}_{D^{*},\ell^{*}}(p)$ , is exactly the (rescaled and shifted) magnitude of the predictor's multicalibration violation. Formally,
$$
\mathcal{L}^{*}(p) = \frac{1}{2} +\frac{1}{2}\max_{j\in [k],S\in \mathcal{S},v\in V_{\lambda}^{k}}\bigg|\underset {(x,y)\sim D,h\sim p}{\mathbb{E}}[(h(x)_{j} - \delta_{y,j})\cdot 1[h(x)\in v,x\in S]]\bigg|.
$$
Since we can write the absolute value of an abstract value  $w$  as  $|w| = \max_{i \in \{\pm 1\}} i \cdot w$ , we next observe that the optimal multi-objective value of the problem  $(\mathcal{D}, \mathcal{G}_{\mathrm{mc}}, \mathcal{P})$  is 0.5.
$$
\min_{h^{*}\in \mathcal{P}}\mathcal{L}^{*}(h^{*}) = \frac{1}{2} +\frac{1}{2}\max_{i\in \{\pm 1\}}i\cdot \left[ \max_{\substack{j\in [k],S\in \mathcal{S}\\ v\in V_{\lambda}^{k}}}\underset {h\sim p}{\mathbb{E}}[(h(x)_{j} - \delta_{y,j})\cdot 1[h(x)\in v,x\in S]]\right] = \frac{1}{2}.
$$
This fact is because multicalibration objectives  $\mathcal{G}_{\mathrm{mc}}$  are symmetric around 0.5 (where  $i = +1$  and  $i = -1$  penalize over and under estimation), leading to the worst loss to be at least 0.5. Furthermore, the Bayes classifier  $h^*$  neither overestimates nor underestimates the label distribution and therefore achieves the loss of 0.5 exactly. Combining these inequalities, we have
$$
\mathcal{L}^{*}(p) - \min_{h^{*}\in \mathcal{P}}\mathcal{L}^{*}(h^{*}) = \frac{1}{2}\max_{\substack{j\in [k],S\in \mathcal{S}\\ v\in V^{\prime}_{\lambda}^{k}}}\left|\underset {h\sim p}{\mathbb{E}}[(h(x)_{j} - \delta_{y,j})\cdot 1[h(x)\in v,x\in S]]\right|.
$$
Therefore  $\mathcal{L}^* (p) - \min_{h^* \in \mathcal{P}}\mathcal{L}^* (h^*) = \varepsilon$  if and only if our multicalibration violation is  $2\varepsilon$ .
Fact 2.6. Let  $D^{(1:T)}$  be data distributions for some  $k$ -class prediction problem,  $\mathcal{D}$  be the set of all data distributions, and  $\mathcal{G}_{\mathrm{mc}}$  be as defined in (3). Fix  $\varepsilon > 0$ ,  $\lambda \in \mathbb{Z}_+$ , and a set of groups  $S \subseteq 2^{\mathcal{X}}$ . A sequence of predictors  $p^{(1:T)} \in \Delta(\mathcal{P})$  is  $\varepsilon$ -optimal on  $D^{(1:T)}$  for the online multi-objective learning problem  $(\mathcal{D}, \mathcal{G}_{\mathrm{mc}}, \mathcal{P})$  if and only if  $p^{(1:T)}$  is  $(S, 2\varepsilon, \lambda)$ -online multicalibrated on  $D^{(1:T)}$ .
Proof. Note that online multicalibration is an online multi-objective learning problem by construction. To analyze its optimality condition, we expand the definition of the objectives in  $\mathcal{G}_{\mathrm{mc}}$  as
$$
\begin{array}{l} \max _ {\ell \in \mathcal {G}} \frac {1}{T} \sum_ {t = 1} ^ {T} \mathcal {L} _ {D ^ {(t)}, \ell} (p ^ {(t)}) \\ = \frac{1}{2} +\frac{1}{2}\max_{\substack{j\in [k],S\in \mathcal{S}\\ v\in V^{\prime}_{\lambda}}}  \left|\underset { \begin{array}{c}h\sim p^{(t)}\\ (x,y)\sim D^{(t)} \end{array} }{\mathbb{E}}\left[\frac{1}{T}\sum_{t = 1}^{T}1[x\in S]\cdot 1[h(x)\in v]\cdot \big(h(x)_{j} - \delta_{y,j}\big)\right]\right|. \\ \end{array}
$$
We again observe that the optimal value of the problem is 0.5; formally,
$$
\max  _ {D ^ {*} \in \mathcal {D}} \min  _ {h ^ {*} \in \mathcal {H}} \max  _ {\ell^ {*} \in \mathcal {G}} \mathcal {L} _ {D ^ {*}, \ell^ {*}} (h ^ {*}) = 0. 5.
$$
This is because choosing  $h^*$  to be the Bayes classifier for  $D^*$  achieves the value of 0.5, which given the absolute value in the second term is the minimum achievable value. Thus,  $p^{(1:T)}$  is  $\varepsilon$ -optimal on  $D^{(1:T)}$  if and only if  $p^{(1:T)}$  is  $(S,2\varepsilon,\lambda)$ -optimal on  $D^{(1:T)}$ .
C Proofs for Section 3
We first recall our characterization of multi-objective learning as a two-player zero-sum game. In this game, a learner player chooses a non-deterministic hypothesis  $p \in \Delta(\mathcal{H})$  and an adversary player chooses a joint distribution over data distributions and objectives  $q \in \Delta(\mathcal{D} \times \mathcal{G})$ . The payoff of the game is the expected objective value  $\mathcal{L}_q(p)$ . In single-distribution multi-objective learning problems where the adversary only has one data distribution  $D$  to choose from, we sometimes write  $q \in \Delta(\mathcal{G})$  for simplicity. In online multi-objective learning, the adversary does not have control over which data distribution  $D$  is chosen by nature. In these cases, the adversary only chooses objectives  $q \in \Delta(\mathcal{G})$  and the game payoff function becomes  $\mathcal{L}_{D,q}(p)$ .
C.1 Game Dynamics
We now prove formalizations of Lemma 3.3, Lemma 3.4, Lemma 3.5, and Lemma 3.6 from Section 3.
The following is a formal restatement of Lemma 3.3.
Lemma C.1 (No-Regret vs. No-Regret). Consider a multi-objective learning problem  $(\mathcal{D},\mathcal{G},\mathcal{H})$  and two sequences  $p^{(1:T)}\in \Delta (\mathcal{H})$  and  $q^{(1:T)}\in \Delta (\mathcal{D}\times \mathcal{G})$ . Suppose  $\operatorname{Reg}_B\left(p^{(1:T)},\{\mathcal{L}_{q^{(t)}}(\cdot)\}^{(1:T)}\right)\leq T\varepsilon$  and  $\operatorname{Reg}(q^{(1:T)},\{1 - \mathcal{L}_{(\cdot)}(p^{(t)})\}^{(1:T)})\leq T\varepsilon$  where  $B\in \mathbb{R}$ . Then, the non-deterministic hypothesis  $\overline{p}\in \Delta (\mathcal{H})$  defined as  $\overline{p} := \text{Uniform}(p^{(1:T)})$  satisfies  $\mathcal{L}^* (\overline{p})\leq B + 2\varepsilon$ . If the baseline  $B$  is the min-max baseline  $B_{weak}$ , then  $\overline{p}$  is a  $2\varepsilon$ -optimal solution for the problem  $(\mathcal{D},\mathcal{G},\mathcal{H})$ .
Proof. By connecting the adversary's regret bound and the learner's (weak) regret bounds of
$$
\max  _ {D ^ {*} \in \mathcal {D}, \ell^ {*} \in \mathcal {G}} \sum_ {t = 1} ^ {T} \mathcal {L} _ {D ^ {*}, \ell^ {*}} (p ^ {(t)}) - T \varepsilon \leq \sum_ {t = 1} ^ {T} \mathcal {L} _ {q ^ {(t)}} (p ^ {(t)}) \text {a n d} \sum_ {t = 1} ^ {T} \mathcal {L} _ {q ^ {(t)}} (p ^ {(t)}) \leq T (\varepsilon + B),
$$
we directly observe that  $\max_{D^{*}\in \mathcal{D},\ell^{*}\in \mathcal{G}}\frac{1}{T}\sum_{t = 1}^{T}\mathcal{L}_{D^{*},\ell^{*}}(p^{(t)})\leq 2\varepsilon +B$ . Linearity of expectation allows us to equate  $\frac{1}{T}\sum_{t = 1}^{T}\mathcal{L}_{D^{*},\ell^{*}}(p^{(t)}) = \mathcal{L}_{D^{*},\ell^{*}}(\overline{p})$ , which yields our first claim that  $\mathcal{L}^* (\overline{p})\leq 2\varepsilon +B$ . The second claim just plugs  $B_{\mathrm{weak}}$  into the previous inequality to obtain the definition of a  $2\varepsilon$ -optimal solution.
The following is a formal restatement of Lemma 3.4.
Lemma C.2 (No-Regret vs. Best-Response). Consider a multi-objective learning problem  $(\mathcal{D},\mathcal{G},\mathcal{H})$  and two sequences  $p^{(1:T)}\in \Delta (\mathcal{H})$  and  $q^{(1:T)}\in \Delta (\mathcal{D}\times \mathcal{G})$ . Suppose  $\operatorname{Reg}_B(p^{(1:T)},\{\mathcal{L}_{q^{(t)}}(\cdot)\}^{(1:T)})\leq T\varepsilon$  where  $B\in \mathbb{R}$ . Further suppose  $q^{(1:T)}$  are, on average,  $\varepsilon$  best-responses to the cost functions  $\{1 - \mathcal{L}_{(\cdot)}(p^{(t)})\}^{(1:T)}$ . Then there exists a  $t\in [T]$  where  $\mathcal{L}^* (p^{(t)})\leq B + 2\varepsilon$ . If  $B = B_{weak}$ , then  $p^{(t)}$  is a 2- $\varepsilon$ -optimal solution for the problem  $(\mathcal{D},\mathcal{G},\mathcal{H})$ .
Proof. Assume the contrary, namely that  $\mathcal{L}^*(p^{(t)}) > B + 2\varepsilon$  at all  $t \in [T]$ . Then
$$
\begin{array}{l} T \varepsilon \geq \sum_ {t = 1} ^ {T} \mathcal {L} _ {q ^ {(t)}} (p ^ {(t)}) - T B, \quad (\text {b o u n d e d r e g r e t w . r . t .} B) \\ \geq \sum_ {t = 1} ^ {T} \mathcal {L} ^ {*} (p ^ {(t)}) - T \varepsilon - T B, \quad (\text {o n a v e r a g e} \varepsilon \text {- b e s t - r e s p o n d i n g}) \\ > \sum_ {t = 1} ^ {T} (B + 2 \varepsilon - \varepsilon - B), \quad (\text {a s s u m p t i o n t o c o n t r a r y}) \\ \end{array}
$$
gives us a contradiction that  $T(2\varepsilon - 2\varepsilon) > 0$ . This proves our first claim. The second claim follows by plugging  $B = B_{\mathrm{weak}}$  into our inequality  $\mathcal{L}^{*}(p^{(t)}) \leq B + 2\varepsilon$  to obtain the definition of a  $2\varepsilon$ -optimal solution.
The following is a formal restatement of Lemma 3.5.
Lemma C.3. Consider a multi-objective learning problem  $(\mathcal{D},\mathcal{G},\mathcal{H})$  and a sequence  $p^{(1:T)}\in \Delta (\mathcal{H})$  of hypotheses where at least one hypothesis  $p^{(t)}$  is  $\varepsilon$ -optimal. We can find a  $3\varepsilon$ -optimal solution  $p^{(t^*)}\in p^{(1:T)}$  with probability at least  $1 - \delta$  by taking only  $O(\varepsilon^{-2}\ln (T|\mathcal{D}|\cdot |\mathcal{G}| / \delta))$  samples from each distribution  $D\in \mathcal{D}$ . If we further have access to a sequence  $q^{(1:T)}\in \Delta (\mathcal{D}\times \mathcal{G})$  of  $\varepsilon$  best-responses to the cost functions  $\{1 - \mathcal{L}_{(\cdot)}(p^{(t)})\}^{(1:T)}$ , then a  $5\varepsilon$ -optimal solution can be found with only  $O(\varepsilon^{-2}\ln (T / \delta))$  samples from each  $D\in \mathcal{D}$ .
Proof. Let the  $\varepsilon$ -optimal solution be denoted  $p^{(t_{\mathrm{good}})}$ . This claim is a simple uniform convergence argument. Suppose that  $N$  i.i.d. samples  $\mathbf{x}_D$  are drawn from each distribution  $D \in \mathcal{D}$ . We can then use  $\widehat{\mathcal{L}}_{D,\ell}(p)$  to denote the empirical value of the loss function  $\ell$  of hypothesis  $p$  on distribution  $D$ , as approximated by the samples  $\mathbf{x}_D$ . We similarly define  $\widehat{\mathcal{L}}^*$  as the empirical analog of the multi-objective value  $\mathcal{L}^*$ . Since the range of each objective is bounded in  $[0,1]$ , Chernoff's bound guarantees that for any  $q \in \Delta(\mathcal{D} \times \mathcal{G})$  and  $p \in \Delta(\mathcal{H})$  we have  $\operatorname*{Pr}_{\mathbf{x}}\left[\left|\widehat{\mathcal{L}}_q(p) - \mathcal{L}_q(p)\right| \geq \varepsilon\right] \leq 2\exp(-2N\varepsilon^2)$ . Taking a union bound over each  $D \in \mathcal{D}$ ,  $t \in [T]$  and  $\ell \in \mathcal{G}$  gives
$$
\Pr_ {\mathbf {x}} \left[ \exists D \in \mathcal {D}, \ell \in \mathcal {G}, t \in [ T ]: \left| \widehat {\mathcal {L}} _ {D, \ell} (p ^ {(t)}) - \mathcal {L} _ {D, \ell} (p ^ {(t)}) \right| \geq \varepsilon \right] \leq 2 | \mathcal {D} | | \mathcal {G} | T \exp \left(- 2 N \varepsilon^ {2}\right).
$$