SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Letting  $t^* = \arg \min_{t \in [T]} \widehat{\mathcal{L}}^*(p^{(t)})$  and  $N = O\left(\varepsilon^{-2} \ln (T|\mathcal{D}| \cdot |\mathcal{G}| / \delta)\right)$  guarantees
$$
\mathcal {L} _ {D ^ {*}, \ell^ {*}} ^ {*} (p ^ {(t ^ {*})}) - \varepsilon \leq \widehat {\mathcal {L}} ^ {*} (p ^ {(t ^ {*})}) \leq \widehat {\mathcal {L}} ^ {*} (p ^ {(t _ {\mathrm {g o o d}})}) \leq \mathcal {L} ^ {*} (p ^ {(t _ {\mathrm {g o o d}})}) + \varepsilon \leq \min  _ {h ^ {*} \in \mathcal {H}} \mathcal {L} ^ {*} (h ^ {*}) + 2 \varepsilon ,
$$
with probability at least  $1 - \delta$ . We can therefore return  $p^{(t^*)}$  as our solution.
To prove our second claim, take instead a union bound over each  $t \in [T]$  so that
$$
\Pr_ {\mathbf {x}} \left[ \exists t \in [ T ]: \left| \widehat {\mathcal {L}} _ {q ^ {(t)}} (p ^ {(t)}) - \mathcal {L} _ {q ^ {(t)}} (p ^ {(t)}) \right| \geq \varepsilon \right] \leq 2 T \exp \left(- 2 N \varepsilon^ {2}\right).
$$
Letting  $t^* = \arg \min_{t \in [T]} \mathcal{L}_{q^{(t)}}(p^{(t)})$  and  $N = O(\varepsilon^{-2} \ln (T / \delta))$  guarantees with probability at least  $1 - \delta$  that we can return  $p^{(t^*)}$  as our solution, as
$$
\begin{array}{l} \underbrace {\mathcal {L} ^ {*} \left(p ^ {\left(t ^ {*}\right)}\right) - 2 \varepsilon \leq \mathcal {L} _ {q ^ {\left(t ^ {*}\right)}} \left(p ^ {\left(t ^ {*}\right)}\right) - \varepsilon} _ {\text {(s i n c e q ^ {\left(t ^ {*}\right)} i s a n \varepsilon - b e s t r e s p o n s e .)}} \leq \widehat {\mathcal {L}} _ {q ^ {\left(t ^ {*}\right)}} \left(p ^ {\left(t ^ {*}\right)}\right) \leq \widehat {\mathcal {L}} _ {q ^ {\left(t _ {\text {g o o d}}\right)}} \left(p ^ {\left(t _ {\text {g o o d}}\right)}\right) \leq \mathcal {L} _ {q ^ {\left(t _ {\text {g o o d}}\right)}} \left(p ^ {\left(t _ {\text {g o o d}}\right)}\right) + \varepsilon \\ \leq \mathcal {L} ^ {*} (p ^ {(t _ {\text {g o o d}})}) + 2 \varepsilon \leq \min  _ {h ^ {*} \in \mathcal {H}} \mathcal {L} ^ {*} (h ^ {*}) + 3 \varepsilon . \\ \end{array}
$$
The following is a formal restatement of Lemma 3.6.
Lemma C.4 (Best-Response vs. No-Regret). Consider a online multi-objective learning problem  $(\mathcal{D},\mathcal{G},\mathcal{H})$  and the sequences  $p^{(1:T)}\in \Delta (\mathcal{H})$ ,  $q^{(1:T)}\in \Delta (\mathcal{G})$ , and  $\bar{D}^{(1:T)}\in \mathcal{D}$ . Suppose  $\operatorname {Reg}(q^{(1:T)},\{1 - \mathcal{L}_{D^{(t)},(\cdot)}(p^{(t)})\}^{(1:T)})\leq T\varepsilon$  and  $p^{(1:T)}\in \Delta (\mathcal{H})$  are distribution-free  $\varepsilon^{\prime}$  best-responses to  $q^{(1:T)}(\cdot ,(x,y))$ . Then,  $p^{(1:T)}$  are  $(\varepsilon +\varepsilon^{\prime})$ -optimal on  $D^{(1:T)}$ .
Proof. Let  $\mathcal{D}^*$  be the set of all data distributions over  $\mathcal{X}\times \mathcal{Y}$ . By linearity of expectation, the maximum  $\arg \max_{D\in \mathcal{D}^*}\mathcal{L}_D(p^{(t)})$  can always be attained on a degenerate distribution supported only on a single point  $(x,y)$ . Thus, the learner's distribution-free best-response guarantee provides the bound
$$
\mathcal {L} _ {D ^ {(t)}, \ell^ {(t)}} (p ^ {(t)}) - \varepsilon^ {\prime} \leq \max  _ {x \in \mathcal {X}, y \in \mathcal {Y}} q ^ {(t)} (p ^ {(t)}, (x, y)) - \varepsilon^ {\prime} \leq \max  _ {D ^ {*} \in \mathcal {D} ^ {*}} \min  _ {h ^ {*} \in \mathcal {H}} \mathcal {L} _ {D ^ {*}, \ell^ {(t)}} (h ^ {*}).
$$
We can obtain our desired claim by recovering (2) with a triangle inequality between the adversary's regret bound and a summation of the previous inequality over  $t \in [T]$ :
$$
\begin{array}{l} \max  _ {\ell^ {*} \in \mathcal {G}} \frac {1}{T} \sum_ {t = 1} ^ {T} \mathcal {L} _ {D ^ {(t)}, \ell^ {*}} (p ^ {(t)}) - T \varepsilon \leq \frac {1}{T} \sum_ {t = 1} ^ {T} \mathcal {L} _ {D ^ {(t)}, q ^ {(t)}} (p ^ {(t)}), \\ \frac {1}{T} \sum_ {t = 1} ^ {T} \mathcal {L} _ {D ^ {(t)}, \ell^ {(t)}} (p ^ {(t)}) - T \varepsilon^ {\prime} \leq \max  _ {D ^ {*} \in \mathcal {D} ^ {*}} \min  _ {h \in \mathcal {H}} \mathcal {L} _ {D ^ {*}, \ell^ {(t)}} (h) \leq \max  _ {D ^ {*} \in \mathcal {D} ^ {*}} \min  _ {h \in \mathcal {H}} \max  _ {\ell^ {*} \in \mathcal {G}} \mathcal {L} _ {D ^ {*}, \ell^ {*}} (h). \\ \end{array}
$$
C.2 No-Regret and Best-Response Computation in Multicalibration
In this section, we prove generalizations of Theorem 3.7 and Theorem 3.8 from Section 3, which describe why multicalibration objectives are amenable to efficient online learning, to a more general class of multi-objective learning problems. We refer to such problems as having separable objectives.
Definition C.5. Consider a multi-objective learning problem  $(\mathcal{D},\mathcal{G},\mathcal{H})$  where  $\mathcal{H}$  is the set of all functions of form  $h:\mathcal{X}\to \mathcal{W}$  and  $\mathcal{W}$  is some convex space (and not necessarily the label space  $\mathcal{V}$ ). We say the objectives  $\mathcal{G}$  of such a problem are separable if every  $\ell \in \mathcal{G}$  is of form
$$
\ell (h, (x, y)) = c + f _ {\ell} (x, h (x)) \cdot (h (x) - g _ {\ell} (y)),
$$
where  $f_{\ell}:\mathcal{X}\times \mathcal{W}\to \mathcal{W}$  and  $g_{\ell}:\mathcal{Y}\rightarrow \mathcal{W}$  are arbitrary functions and  $c$  is some constant offset.
We will see that, like with multicalibration objectives, having a learner's cost functions all be separable objectives introduces two major advantages generally.
Advantage 1: There exist no-regret learning strategies that do not require one to randomize their actions, or even sample any data, yet still guarantee domain-independent regret bounds. The following theorem, which proves this advantage of separable objectives, is a generalization of Theorem 3.7.
Theorem C.6. Consider a multi-objective learning problem  $(\mathcal{D},\mathcal{G},\mathcal{H})$  where all objectives in  $\mathcal{G}$  are separable and all distributions in  $\mathcal{D}$  are absolutely continuous with respect to a common distribution  $D^{*}$ . Let  $\mathcal{A}$  be an online algorithm that, for any linear costs  $c^{(1:T)}$ , outputs actions  $a^{(1:T)} \in \mathcal{W}$  where  $\mathrm{Reg}(a^{(1:T)}, c^{(1:T)}) \leq R(T)$ . If  $|\mathcal{D}| > 1$ , further assume  $\mathrm{Reg}(a^{(1:T)}, c^{(1:T)}) \leq R(T) \cdot \max_{a^{*} \in \mathcal{W}} \sqrt{\frac{1}{T} \sum_{t=1}^{T} (c^{(t)}(a^{(t)}) - c^{(t)}(a^{*}))^2}$ . Then for any stochastic costs  $q^{(1:T)} \in \Delta(\mathcal{G})$  and distributions  $D^{(1:T)} \in \mathcal{D}$ , the below algorithm outputs predictors  $h^{(1:T)} \in \mathcal{H}$  where  $\mathrm{Reg}_{B^{*}}(h^{(1:T)}, \{\mathcal{L}_{D^{(t)}, q^{(t)}}(\cdot)\}^{(1:T)}) \leq \sqrt{|\mathcal{D}|} R(T)$  and  $B^{*}$  is the baseline
$$
B ^ {*} := c + \min  _ {h ^ {*} \in \mathcal {H}} \frac {1}{T} \sum_ {t = 1} ^ {T} \max  _ { \begin{array}{c} (x, y) \sim D ^ {(t)} \\ \ell \sim q ^ {(t)} \end{array} } \mathbb {E} _ {\substack {(x, y) \sim D ^ {(t)} \\ \ell \sim q ^ {(t)}}} [ f _ {\ell} (x, h ^ {\prime} (x)) \cdot (h ^ {*} (x) - g _ {\ell} (y)) ]. \tag{8}
$$
At each timestep  $t$ , construct the predictor  $h^{(t + 1)}(x)$  by setting, for all  $x \in \mathcal{X}$ ,
$$
\left| h ^ {(t + 1)} (x) := \mathcal {A} \left(c _ {x} ^ {(1)}, \dots , c _ {x} ^ {(t)}\right) w h e r e c _ {x} ^ {(\tau)} (w) := \frac {1}{2} \cdot \frac {d D ^ {(\tau)} (x)}{d D ^ {*} (x)} \cdot \left(1 + \underset {\ell \sim q ^ {(\tau)}} {\mathbb {E}} \left[ f _ {\ell} (x, h ^ {(\tau)} (x)) \right] \cdot w\right), \right.
$$
where  $\frac{dD^{(\tau)}(x)}{dD^*(x)}$  is the Radon-Nikodym derivative of  $D^{(\tau)}$  with respect to  $D^*$  at  $x$ .
Proof. First, suppose that  $\mathcal{D} = \{D\}$ , that is  $|\mathcal{D}| = 1$ . Then, for any  $x \in \mathcal{X}$ ,  $c_x^{(1:T)}$  are linear costs and thus  $\mathcal{A}$  guarantees  $\mathrm{Reg}(\{h^{(t)}(x)\}^{(1:T)}, c_x^{(1:T)}) \leq R(T)$ . By the law of total expectation,
$$
\begin{array}{l} 2 R (T) \\ \geq \underset {x \sim D ^ {*}} {\mathbb {E}} \left[ \sum_ {t = 1} ^ {T} \underset {\ell \sim q ^ {(t)}} {\mathbb {E}} \left[ \frac {d D (x)}{d D ^ {*} (x)} f _ {\ell} (x, h ^ {(t)}) \right] \cdot h ^ {(t)} (x) - \min  _ {w ^ {*} \in \mathcal {W}} \sum_ {t = 1} ^ {T} \underset {\ell \sim q ^ {(t)}} {\mathbb {E}} \left[ \frac {d D (x)}{d D ^ {*} (x)} f _ {\ell} (x, h ^ {(t)}) \right] \cdot w ^ {*} \right] \\ = \underset {x \sim D} {\mathbb {E}} \left[ \sum_ {t = 1} ^ {T} \underset {\ell \sim q ^ {(t)}} {\mathbb {E}} \left[ f _ {\ell} (x, h ^ {(t)}) \right] \cdot h ^ {(t)} (x) \right] - \underset {x \sim D} {\mathbb {E}} \left[ \min  _ {w ^ {*} \in \mathcal {W}} \sum_ {t = 1} ^ {T} \underset {\ell \sim q ^ {(t)}} {\mathbb {E}} \left[ f _ {\ell} (x, h ^ {(t)}) \right] \cdot w ^ {*} \right] \\ = \sum_ {t = 1} ^ {T} \underset {x \sim D ^ {(t)}, \ell \sim q ^ {(t)}} {\mathbb {E}} \left[ f _ {\ell} (x, h ^ {(t)}) \cdot h ^ {(t)} (x) \right] - \min  _ {h ^ {*} \in \mathcal {H}} \sum_ {t = 1} ^ {T} \underset {x \sim D ^ {(t)}, \ell \sim q ^ {(t)}} {\mathbb {E}} \left[ f _ {\ell} (x, h ^ {(t)}) \cdot h ^ {*} (x) \right]. \\ \end{array}
$$
If  $|\mathcal{D}| > 1$ ,  $\mathcal{A}$  further guarantees
$$
\begin{array}{l} \operatorname * {P r} _ {D ^ {*}} (x) \cdot \operatorname {R e g} (\{h ^ {(t)} (x) \} ^ {(1: T)}, c _ {x} ^ {(1: T)}) \leq \operatorname * {P r} _ {D ^ {*}} (x) \cdot \max _ {w \in \mathcal {W}} R (T) \sqrt {\frac {1}{T} \sum_ {t = 1} ^ {T} (c _ {x} ^ {(t)} (h ^ {(t)} (x)) - c _ {x} ^ {(t)} (w)) ^ {2}} \\ \leq \operatorname * {P r} _ {D ^ {*}} (x) \cdot R (T) \sqrt {\frac {1}{2 T} \sum_ {t = 1} ^ {T} \left(\frac {d D ^ {(t)} (x)}{d D ^ {*} (x)}\right) ^ {2}} \\ \leq R (T) \cdot \sqrt {\frac {1}{2 T} \sum_ {t = 1} ^ {T} \operatorname * {P r} _ {D ^ {(t)}} (x) ^ {2}}. \\ \end{array}
$$
For simplicity, we assumed above that  $D^{(1:T)}$  each have discrete support. The law of total expectation gives
$$
\begin{array}{l} \sqrt {2} \cdot R (T) \cdot \sum_ {x \in \mathcal {X}} \sqrt {\frac {1}{2 T} \sum_ {t = 1} ^ {T} \operatorname * {P r} _ {D ^ {(t)}} (x) ^ {2}} \\ \geq \underset {x \sim D ^ {*}} {\mathbb {E}} \left[ \sum_ {t = 1} ^ {T} \underset {\ell \sim q ^ {(t)}} {\mathbb {E}} \left[ \frac {d D ^ {(t)} (x)}{d D ^ {*} (x)} f _ {\ell} (x, h ^ {(t)}) \right] \cdot h ^ {(t)} (x) - \min  _ {w ^ {*} \in \mathcal {W}} \sum_ {t = 1} ^ {T} \underset {\ell \sim q ^ {(t)}} {\mathbb {E}} \left[ \frac {d D ^ {(t)} (x)}{d D ^ {*} (x)} f _ {\ell} (x, h ^ {(t)}) \right] \cdot w ^ {*} \right] \\ = \sum_ {t = 1} ^ {T} \mathbb {E} _ {x \sim D ^ {(t)}, \ell \sim q ^ {(t)}} \left[ f _ {\ell} (x, h ^ {(t)}) \cdot h ^ {(t)} (x) \right] - \min  _ {h ^ {*} \in \mathcal {H}} \sum_ {t = 1} ^ {T} \mathbb {E} _ {x \sim D ^ {(t)}, \ell \sim q ^ {(t)}} \left[ f _ {\ell} (x, h ^ {(t)}) \cdot h ^ {*} (x) \right]. \\ \end{array}
$$
To bound the left-hand term, we can apply Cauchy-Schwartz to get
$$
\begin{array}{l} \sum_ {x \in \mathcal {X}} \sqrt {\frac {1}{T} \sum_ {t = 1} ^ {T} \operatorname * {P r} _ {D ^ {(t)}} (x) ^ {2}} = \sum_ {x \in \mathcal {X}} \sqrt {\sum_ {D \in \mathcal {D}} \frac {T _ {D}}{T} \operatorname * {P r} _ {D} (x) ^ {2}} \\ \leq \sum_ {x \in \mathcal {X}} \sum_ {D \in \mathcal {D}} \Pr_ {D} (x) \sqrt {\frac {T _ {D}}{T}} \\ = \sum_ {D \in \mathcal {D}} \sqrt {\frac {T _ {D}}{T}} \sum_ {x \in \mathcal {X}} \Pr_ {D} (x) \\ \leq \sqrt {| \mathcal {D} |}, \\ \end{array}
$$
where  $T_{D}$  is the number of timesteps  $t \in [T]$  where  $D^{(t)} = D$ . Thus, whether  $|\mathcal{D}| = 1$  or  $|\mathcal{D}| > 1$ , we have
$$
\sum_ {t = 1} ^ {T} \underset {x \sim D ^ {(t)}, \ell \sim q ^ {(t)}} {\mathbb {E}} \left[ f _ {\ell} (x, h ^ {(t)}) \cdot h ^ {(t)} (x) \right] - \min  _ {h ^ {*} \in \mathcal {H}} \sum_ {t = 1} ^ {T} \underset {x \sim D ^ {(t)}, \ell \sim q ^ {(t)}} {\mathbb {E}} \left[ f _ {\ell} (x, h ^ {(t)}) \cdot h ^ {*} (x) \right] \leq 2 \sqrt {| \mathcal {D} |} R (T).
$$
Adding and subtracting the term  $\sum_{t=1}^{T} \mathbb{E}_{(x,y) \sim D^{(t)}, \ell \sim q^{(t)}}\left[f_{\ell}(x,h^{(t)}(x))(g_{\ell}(y))\right]$ , we have
$$
\begin{array}{l} \sum_{t = 1}^{T}\underset { \begin{array}{c}(x,y)\sim D^{(t)}\\ \ell \sim q^{(t)} \end{array} }{\mathbb{E}}\left[f_{\ell}(x,h^{(t)}(x))(h^{(t)}(x) - g_{\ell}(y))\right] \\ \leq 2\sqrt{|\mathcal{D}|} R(T) + \min_{h^{*}\in \mathcal{H}}\sum_{t = 1}^{T}\underset { \begin{array}{c}(x,y)\sim D^{(t)}\\ \ell \sim q^{(t)} \end{array} }{\mathbb{E}}\left[f_{\ell}(x,h^{(t)}(x))(h^{*}(x) - g_{\ell}(y))\right]. \\ \end{array}
$$
Adding the constant  $c$  to both sides, we have
$$
\begin{array}{l} \sum_{t = 1}^{T}\mathcal{L}_{D^{(t)},q^{(t)}}(h^{(t)})\leq 2\sqrt{|\mathcal{D}|} R(T) + T\min_{h^{*}\in \mathcal{H}}\frac{1}{T}\sum_{t = 1}^{T}\underset { \begin{array}{c}(x,y)\sim D^{(t)}\\ \ell \sim q^{(t)} \end{array} }{\mathbb{E}}\left[c + f_{\ell}(x,h^{(t)}(x))(h^{*}(x) - g_{\ell}(y))\right] \\ \leq 2\sqrt{|\mathcal{D}|} R(T) + T\min_{h^{*}\in \mathcal{H}}\frac{1}{T}\max_{h^{\prime}\in \mathcal{H}}\sum_{t = 1}^{T}\underset { \begin{array}{c}(x,y)\sim D^{(t)}\\ \ell \sim q^{(t)} \end{array} }{\mathbb{E}}\left[c + f_{\ell}(x,h^{\prime}(x))(h^{*}(x) - g_{\ell}(y))\right]. \\ \end{array}
$$
We can directly prove Theorem 3.7 by instantiating the algorithm of Theorem C.6 with Hedge as the no-regret learning algorithm  $\mathcal{A}$ .
Proof. First, we observe that every objective in multicalibration is separable. Specifically, for every  $i \in \{\pm 1\}, j \in [k], S \in S, v \in V_{\lambda}^{k}$ , the corresponding objective  $\ell_{i,j,S,v} \in \mathcal{G}_{\mathrm{mc}}$  can be written in the form  $\ell_{i,j,S,v}(h,(x,y)) = c + f_{i,j,S,v}(x,h(x)) \cdot (h(x) - g_{i,j,S,v}(y))$  where  $c = 0.5$ ,  $\mathcal{W} = \Delta_k \cup -\Delta_k$ ,  $g_{i,j,S,v}(y) = \delta_y$  and  $f_{i,j,S,v}(x,h(x)) = \delta_j \cdot 0.5 \cdot i \cdot 1[h(x) \in v, x \in S]$ . Next, we recall that by Lemma A.1, the Hedge algorithm guarantees a regret bound of  $4\sqrt{\ln(k)T}$  for any linear costs on  $\mathcal{W} = \Delta_k \cup -\Delta_k$  (Lemma A.1). The algorithm of Theorem C.6, for any stochastic costs  $q^{(1:T)} \in \Delta(\mathcal{G}_{\mathrm{mc}})$ , outputs  $h^{(1:T)}$  where
$$
\begin{array}{l} \sum_ {t = 1} ^ {T} q ^ {(t)} \left(h ^ {(t)}\right) \leq 8 \sqrt {\ln (k) T} + T \cdot \min  _ {h ^ {*} \in \mathcal {H}} \max  _ {\ell^ {*} \in \mathcal {G}} \max  _ {h ^ {\prime} \in \mathcal {H}} \underset {(x, y) \sim D} {\mathbb {E}} \left[ c + f _ {\ell^ {*}} \left(x, h ^ {\prime} (x)\right) \cdot \left(h ^ {*} (x) - g (y)\right) \right] \\ \leq 8 \sqrt {\ln (k) T} + T \cdot \max _ {\ell^ {*} \in \mathcal {G}} \max _ {h ^ {\prime} \in \mathcal {H}} \underset {(x, y) \sim D} {\mathbb {E}} \left[ c + f _ {\ell^ {*}} (x, h ^ {\prime} (x)) \cdot (g (y) - g (y)) \right] \\ \leq 8 \sqrt {\ln (k) T} + T c, \\ \end{array}
$$
if we choose  $h^{*}(x) = \mathbb{E}_{(x,y)\sim D}\left[g^{(t)}(y)\mid x\right]$ . Since the adversary is choosing from objectives symmetric around  $c$ , the min-max baseline  $B_{\mathrm{weak}}$  is at least  $c$ , meaning that  $\operatorname{Reg}_{\mathrm{weak}}(h^{(1:T)},\{\mathcal{L}_{D,q^{(t)}}(\cdot)\}^{(1:T)}) \leq 8\sqrt{\ln(k)T}$ .
Advantage 2: There (almost) always exists distribution-free best responses. This advantage is what allows multicalibration to be achievable in online settings where data arrives adversarially. [10, 20] first showed that this advantage exists in calibrated online forecasting when Hart pointed out that Foster's asymptotic calibrated forecasting result could be obtained by appealing to Blackwell approachability. Moreover, their proof extends trivially to online multicalibration. Later on, [18, 35] independently rediscovered this same advantage and proof. In the following theorem, we provide a rigorous treatment—largely in line with the prior works—that generalizes to all multi-objective learning problems that have separable objectives and satisfy some weak compactness conditions.
Theorem C.7. Consider an online multi-objective learning problem  $(\mathcal{D},\mathcal{G},\mathcal{H})$  with separable objectives and where  $\mathcal{D}$  is the unrestricted set of all data distributions. Further, consider any objective mixture  $q\in \Delta (\mathcal{G})$  with finite support and suppose that for every objective  $\ell \in \mathcal{G}$ , (1) there is a finite subset  $\mathcal{V}_{\ell}\subseteq \mathcal{V}$  s.t.  $\{g_{\ell}(y)\mid y\in \mathcal{V}_{\ell}\}$  is an  $(\varepsilon /3)$ -net for  $\{g_{\ell}(y)\mid y\in \mathcal{V}\}$ , (2) the range of  $g_{\ell}$  is convex, compact and finite-dimensional, and (3) fixing any  $x,y$  pair,  $\ell (h,(x,y))$  is, in the argument  $h$ , a function of bounded variation. Then, for any  $\varepsilon >0$ , there is a non-deterministic hypothesis  $p\in \Delta (\mathcal{H})$  (given by (9)) that is a distribution-free  $\varepsilon$  best-response for  $q$ .
Proof. Let  $\mathcal{G}_q = \mathrm{Support}(q)$  be the support of our objective mixture, noting that  $|\mathcal{G}_q| < \infty$  by assumption. Let  $\mathcal{W}_g \subseteq \prod_{\ell \in \mathcal{G}_q} \mathcal{W}$  denote the range of the vector-valued function  $g: \mathcal{V} \to \mathcal{W}_g$  where  $g(y) \coloneqq [g_\ell(y)]_{\ell \in \mathcal{G}_q}$ . Fixing some  $x \in \mathcal{X}$ , observe that we can rewrite our loss  $q(h, (x, y))$  as the
function  $\widetilde{q}_x:\mathcal{W}\times \mathcal{W}_g\to [0,1]$  where  $q(h,(x,y)) = \widetilde{q}_x(h(x),g(y))$  and we define  $\widetilde{q}_x(w,v)\coloneqq c + \mathbb{E}_{\ell \sim q}[f_\ell (w,x)(w - v_\ell)]$
By assumption (3),  $\widetilde{q}_x(w,v)$  is a function of bounded variation in  $w$  for all  $x\in \mathcal{X},v\in \mathcal{W}_g$ . By Lemma C.9, for any  $x\in \mathcal{X},y\in \mathcal{Y}$ , there must exist some finite subset  $\mathcal{W}_{x,y}\subseteq \mathcal{W}$  such that  $\{\widetilde{q}_x(w,(x,g(y)))\}_{w\in \mathcal{W}_{x,y}}$  is an  $(\varepsilon /3)$ -net of  $\{\widetilde{q}_x(w,(x,g(y)))\}_{w\in \mathcal{W}}$ . Let us choose  $\mathcal{W}_x\coloneqq \bigcup_{\ell \in \mathcal{G}_q}\bigcup_{y\in \mathcal{Y}_\ell}\mathcal{W}_{x,y}$ , which is a finite set as  $|\mathcal{V}_{\ell}| < \infty$  by assumption (1) and we  $|\mathcal{G}_q| < \infty$  by assumption. Moreover, by construction, for any  $y\in \mathcal{V}$ ,  $\{\widetilde{q}_x(w,(x,y))\}_{w\in \mathcal{W}_x}$  is an  $(2\varepsilon /3)$ -net of  $\{\widetilde{q}_x(w,(x,y))\}_{w\in \mathcal{W}}$ .
We now define our best response  $p$  pointwise at each  $x \in \mathcal{X}$ , letting
$$
p (x) = \underset {w ^ {*} \in \Delta (\mathcal {W} _ {x})} {\arg \min } \max  _ {y \in \mathcal {Y}} \underset {w \sim w ^ {*}} {\mathbb {E}} [ \widetilde {q} _ {x} (w, (x, g (y))) ] = \underset {w ^ {*} \in \Delta (\mathcal {W} _ {x})} {\arg \min } \underset {g (y) \in \mathcal {W} _ {g}} {\max } \underset {w \sim w ^ {*}} {\mathbb {E}} [ \widetilde {q} _ {x} (w, (x, g (y))) ]. \tag {9}
$$
Because each objective  $\ell$  is separable,  $\widetilde{q}_x(w,(x,g(y)))$  is linear in  $g(y)$ . By linearity of expectation, we also know that  $\mathbb{E}_{w\sim w^*}[\widetilde{q}_x(w,(x,g(y)))]$  is linear in  $w^*$ . Thus, for any  $x\in \mathcal{X}$ ,
$$
\begin{array}{l} \max  _ {g (y) \in \mathcal {W} _ {g}} \widetilde {q} _ {x} (p (x), (x, g (y))) = \min  _ {w ^ {*} \in \Delta (\mathcal {W} _ {x})} \max  _ {g (y) \in \mathcal {W} _ {g}} \underset {w \sim w ^ {*}} {\mathbb {E}} [ \widetilde {q} _ {x} (w, (x, g (y))) ], (\text {c o n s t r u c t i o n o f} f) \\ = \max  _ {g (y) \in \mathcal {W} _ {g}} \min  _ {w ^ {*} \in \Delta (\mathcal {W} _ {x})} \mathbb {E} _ {w \sim w ^ {*}} \left[ \widetilde {q} _ {x} (w, (x, g (y))) \right], (\text {m i n i m a x t h e o r e m}) \\ \leq \max  _ {g (y) \in \mathcal {W} _ {g}} \min  _ {w ^ {*} \in \Delta (\mathcal {W})} \mathbb {E} _ {w \sim w ^ {*}} \left[ \widetilde {q} _ {x} (w, (x, g (y))) \right] + r \varepsilon , (\text {d i s c r e t i z a t i o n e r r o r}) \\ \leq \max  _ {g (y) \in \mathcal {W} _ {g}} \min  _ {p ^ {*} \in \Delta (\mathcal {H})} \widetilde {q} _ {x} \left(p ^ {*} (x), (x, g (y))\right) + r \varepsilon . \\ \end{array}
$$
In the above, we were able to apply the minimax theorem because of assumption (2). Recall that our construction of  $\mathcal{W}_x$ , guarantees, for any  $w \in \mathcal{W}, y \in \mathcal{Y}$ , there is a  $w' \in \mathcal{W}_x, y' \in \mathcal{Y}'$  such that
$$
\begin{array}{l} \left| \widetilde {q} _ {x} (w, (x, g (y))) - \widetilde {q} _ {x} (w, (x, g (y))) \right| \leq \varepsilon / 3, (y ^ {\prime} \in \mathcal {Y} ^ {\prime}) \\ \left| \widetilde {q} _ {x} \left(w, \left(x, g \left(y ^ {\prime}\right)\right)\right) - \widetilde {q} _ {x} \left(w ^ {\prime}, \left(x, g \left(y ^ {\prime}\right)\right)\right) \right| \leq \varepsilon / 3, \left(w ^ {\prime} \in \mathcal {W} _ {x, y ^ {\prime}}\right) \\ \left| \widetilde {q} _ {x} \left(w ^ {\prime}, (x, g (y ^ {\prime}))\right) - \widetilde {q} _ {x} \left(w ^ {\prime}, (x, g (y))\right) \right| \leq \varepsilon / 3, \left(y ^ {\prime} \in \mathcal {Y} ^ {\prime}\right) \\ \left| \widetilde {q} _ {x} (w, (x, g (y))) - \widetilde {q} _ {x} \left(w ^ {\prime}, (x, g (y))\right) \right| \leq \varepsilon . (\text {t r i a n g l e i n e q u a l i t y}) \\ \end{array}
$$
We, therefore, have that
$$
\begin{array}{l} \max  _ {D \in \mathcal {D}} \mathcal {L} _ {D, q} (p) = \max  _ {x \in \mathcal {X}, y \in \mathcal {Y}} \widetilde {q} _ {x} (p (x), (x, g (y))) \\ = \max  _ {x \in \mathcal {X}, g (y) \in \mathcal {W} _ {g}} \widetilde {q} _ {x} (p (x), (x, g (y))) \\ \leq \max  _ {x \in \mathcal {X}, g (y) \in \mathcal {W} _ {g}} \min  _ {p ^ {*} \in \Delta (\mathcal {H})} \widetilde {q} _ {x} \left(p ^ {*} (x), (x, g (y))\right) + r \varepsilon \\ = \max  _ {x \in \mathcal {X}, y \in \mathcal {Y}} \min  _ {p ^ {*} \in \Delta (\mathcal {H})} \widetilde {q} _ {x} (p ^ {*} (x), (x, g (y))) + r \varepsilon \\ = \max  _ {D \in \mathcal {D}} \min  _ {p ^ {*} \in \Delta (\mathcal {H})} \mathcal {L} _ {D, q} (p ^ {*}) + r \varepsilon . \\ \end{array}
$$
We can now directly prove Theorem 3.8.
Proof. Let  $(\mathcal{D}', \mathcal{G}_{\mathrm{mc}}, \mathcal{P})$  be the relaxation of the problem  $(\mathcal{D}, \mathcal{G}_{\mathrm{mc}}, \mathcal{P})$  where, instead of assuming that nature can only sample discrete labels  $y \in [k]$ , nature too can sample mixed labels  $y \in \Delta_k$ . By linearity of expectation, we have that  $\max_{D' \in \mathcal{D}'} \mathcal{L}_{D', q'}(p) = \max_{D \in \mathcal{D}} \mathcal{L}_{D,q}(p)$  for every predictor  $p \in \Delta(\mathcal{P})$  and objective  $q \in \mathcal{G}_{\mathrm{mc}}$ , where  $q' \in \mathcal{G}_{\mathrm{mc}}'$  is the relaxation of  $q$ . Thus, a distribution-free  $\varepsilon$  best-response (7) for the relaxed problem is a distribution-free  $\varepsilon$  best-response for our original problem  $(\mathcal{D}, \mathcal{G}_{\mathrm{mc}}, \mathcal{P})$ . We next observe that  $(\mathcal{D}', \mathcal{G}_{\mathrm{mc}}', \mathcal{P})$  satisfies the conditions of Theorem C.7.
Recall that every objective  $q' \in \mathcal{G}_{\mathrm{mc}}'$  can be written in a separable format where, for every  $i \in \{\pm 1\}, j \in [k], S \in \mathcal{S}, v \in V_{\lambda}^{k}$ , we can write  $\ell_{i,j,S,v}(h,(x,y)) = c + f_{i,j,S,v}(x,h(x)) \cdot (h(x) - g_{i,j,S,v}(y))$  where  $c = 0.5$ ,  $g_{i,j,S,v}(y) = y$  and  $f_{i,j,S,v}(x,h(x)) \in [0,1]^{k}$  is  $\delta_j \cdot 0.5 \cdot i \cdot 1[h(x) \in v, x \in S]$ . The domain and range of identity  $g_{i,j,S,v}(y) = y$  is exactly  $\Delta_k$  which is convex, compact, and  $k$ -dimensional; thus, it has a finite  $\varepsilon$ -covering. Finally, we observe that  $f_{i,j,S,v}(h(x),(x,y))$ , and by extension  $\ell_{i,j,S,v}(h,(x,y))$ , is always a piecewise constant function in  $h(x)$  with finite discontinuities. It follows that  $\ell_{i,j,S,v}$  must be of bounded total variation in  $h(x)$  as variation along any linear segment is at most 1. Thus, Theorem C.7 states there exists a distribution-free  $\varepsilon$  best-response for  $(\mathcal{D}',\mathcal{G}_{\mathrm{mc}}',\mathcal{P})$  and by extension  $(\mathcal{D},\mathcal{G}_{\mathrm{mc}},\mathcal{P})$ .