SlowGuess/NeurIPS_2023_OCR huggingface dataset:
This definition aligns with the definition of "pseudo-moment multicalibration" proposed by [22]. We adopt this particular definition as previous studies [22, 18] develop their algorithms and analyses based on this definition of moment multicalibration, with translations to other definitions occurring only retrospectively.
$\ell_{\mu}^{(t)} = \mathcal{A}_{\varepsilon /8}(1 - \ell_{(\cdot)}(h^{(t)}),\mathcal{G}_{\mathrm{mc}}^{\mu})$  Using  $C\ln (T / \delta) / \varepsilon^2$  samples from  $D$  , return the predictor  $h^{(t^{*})}$  with the lowest empirical multicalibration error. This algorithm is written explicitly in Algorithm 7.
First, we prove an intermediate result.
Lemma E.14. In the Algorithm 7, the sequence of hypotheses  $h^{(1:T)}$  satisfies the bounded regret condition  $\mathrm{Reg}_{weak}(h^{(1:T)},\{\ell_m^{(t)} + \ell_\mu^{(t)}\}^{(1:T)}) \in O(\sqrt{m} T^{3/4})$ .
Proof. For simplicity, we'll round  $T$  to the next largest square. Recall that using Hedge to select  $T$  actions from the interval  $[0,1]$  with learning rate  $T^{-3/4}m^{-1/2}$  guarantees a regret bound of  $2\ln(2)\sqrt{m}T^{3/4}$  (Lemma A.1). By Theorem C.6, since we chose  $\mathcal{A}$  to be Hedge with learning rate  $T^{-3/4}$  when learning  $h_{\mu}^{(t)}$ ,  $\sum_{t=1}^{T}\ell_{\mu}^{(t)}(h^{(t)}) \leq 0.5T + 2\ln(2)\sqrt{m}T^{3/4}$ . Similarly, by Theorem C.6, having chosen  $\mathcal{A}$  to be strongly adaptive Hedge when learning  $h_m^{(t)}$ , we have for any  $T_1 \in [T]$  and  $h_m^* \in \mathcal{H}_m$ ,
$$
\sum_ {t = T _ {1}} ^ {T _ {1} + \sqrt {T}} \left(\mathcal {L} _ {\ell_ {m} ^ {(t)}} (h ^ {(t)}) - 0. 5 - \underset {(x, y) \sim D} {\mathbb {E}} \left[ f _ {m} ^ {(t)} (x, h ^ {(t)} (x)) (h _ {m} ^ {*} (x) - g _ {m} ^ {(t)} (y)) \right]\right) \leq O \left(T ^ {1 / 4} \ln (T)\right).
$$
In the above equation,  $f_{m}^{(t)}$  and  $g_{m}^{(t)}$  are the separable components of the loss  $\ell_{m}^{(t)}$ , as defined in Definition C.5. We now turn to bound the expectation term. By triangle inequality,  $f_{m}^{(t)}(x,h^{(t)}(x))(h_{m}^{*}(x) - g_{m}^{(t)}(y))\leq f_{m}^{(t)}(x,h^{(t)}(x))(h_{m}^{*}(x) - g_{m}^{(T_{1})}(y)) + \left|g_{m}^{(T_{1})}(y) - g_{m}^{(t)}(y)\right|.$  Using the inequality that  $|a^m -b^m |\leq m|a - b|$  when  $a,b\in [0,1]$ , and the movement upper bound of Hedge (Lemma A.2), we have  $\left|g_m^{(T_1)}(y) - g_m^{(t)}(y)\right|\leq m\left|h_\mu^{(t)}(x) - h_\mu^{(T_1)}(x)\right|\leq m\eta_\mu \sqrt{T}$ , where  $\eta_{\mu} = T^{-1 / 2}T^{-3 / 4}$  is the learning rate of the mean predictor  $h_\mu$ . Thus, for all  $T_{1}$ , by choosing  $h_m^* (x) = \mathbb{E}\left[g_m^{(T_1)}(y)|x\right]$ , we have that
$$
\begin{array}{l} \sum_ {t = T _ {1}} ^ {T _ {1} + \sqrt {T}} \left(\mathcal {L} _ {\ell_ {m} ^ {(t)}} (h ^ {(t)}) - 0. 5\right) \\ \leq \min _ {h _ {m} ^ {*} \in \mathcal {H} _ {m}} \sum_ {t = T _ {1}} ^ {T _ {1} + \sqrt {T}} \underset {(x, y) \sim D} {\mathbb {E}} \left[ f _ {m} ^ {(t)} (x, h ^ {(t)} (x)) (h _ {m} ^ {*} (x) - g _ {m} ^ {(T _ {1})} (y)) \right] + m T \eta_ {\mu} + O \left(T ^ {3 / 4} \ln (T)\right) \\ \leq \sqrt {m} T ^ {3 / 4} + O \left(T ^ {3 / 4} \ln (T)\right). \\ \end{array}
$$
This gives  $\frac{1}{2}\sum_{t = 1}^{T}\mathcal{L}_{\ell_m^{(t)}}(h^{(t)}) + \mathcal{L}_{\ell_\mu^{(t)}}(h^{(t)}) - 1\leq O\left((\ln (T) + \sqrt{m})T^{3 / 4}\right)$  as desired.
We now return to proving Theorem E.13.
Proof of Theorem E.13. By Lemma E.14, the learner's regret is  $\mathrm{Reg}_{\mathrm{weak}}(h^{(1:T)},\{\ell_m^{(t)} + \ell_\mu^{(t)}\}^{(1:T)})\in O(m^{1 / 2}T^{3 / 4})$ . When  $T = Cm^2 /\varepsilon^4$ , we thus have  $\mathrm{Reg}_{\mathrm{weak}}(h^{(1:T)},\{\max \{\ell_m^{(t)},\ell_\mu^{(t)}\} \}^{(1:T)})\leq T\varepsilon /8$ . By construction, every  $\max \{\ell_m^{(t)},\ell_\mu^{(t)}\}$  is an  $(\varepsilon /8)$  best-response to the cost  $1 - \mathcal{L}_{D,(\cdot)}h^{(t)}$ . By Lemma 3.4, there must exist a timestep where  $h^{(t)}$  is  $(\varepsilon /4)$ -optimal. By Lemma 3.5, the predictors  $h^{(t^*)}$  found by the algorithm are  $(\varepsilon /2)$ -optimal with probability at least  $1 - \delta$ . By Fact E.12,  $h^{(t^*)}$  is a  $(S,\varepsilon ,\lambda ,m)$ -mean-conditioned moment multicalibrated predictor.
Algorithm 4 Conditional Multicalibration Algorithm (Theorem E.3)
Algorithm 5 Non-Deterministic Conditional Multicalibration Algorithm (Theorem E.4)
Algorithm 6 Agnostic Deterministic Multicalibration Algorithm (Theorem E.9)
1: Input:  $S, S' \subseteq 2^{\mathcal{X}}$ ,  $\varepsilon \in (0,1)$ ,  $k, \lambda, T, C \in \mathbb{Z}_{+}$ , distributions  $\{D_S'\}_{S' \in S'}$ , and agnostic learning oracle  $\mathcal{A}$ ;  
2: Initialize Prod iterate  $h^{(1)} = [1/k, \ldots, 1/k]^{\mathcal{X}}$ ;  
3: for  $t = 1$  to  $T$  do  
4: Let  $S'^{(t)}, \ell^{(t)} = \mathcal{A}_{\varepsilon/8}(c_{\mathrm{adv}}^{(t)}, S' \times \mathcal{G}_{\mathrm{mc}})$  where  $c_{\mathrm{adv}}^{(t)}(S', \ell) := 1 - \mathcal{L}_{D_{S'}, \ell}(h^{(t)})$ ;  
5: Let  $h^{(t+1)}(x) = \operatorname{Prod}(c_x^{(1:t)})$  where  $c_x^{(t)}(\widehat{y}) := \frac{1[x \in S'^{(t)}]}{2 \operatorname*{Pr}_D(x \in S'^{(t)})} (1 + 1[h(x) \in v^{(t)}, x \in S^{(t)}] \cdot i^{(t)} \cdot \widehat{y}_{j^{(t)}})$ ;  
6: end for  
7: Take  $C \ln(T/\delta)/\varepsilon^2$  samples  $\mathbf{x}(S') \sim D_{S'}$  for all  $S' \in S'$ , and let  $t^* = \arg \min \sum_{\ell^{(t)}} \ell^{(t)}(h^{(t)}, (x,y))$ ;  
8: Return the predicted  $h^{(t*)}$ ;
1: Input:  $S, S' \subseteq 2^{\mathcal{X}}, \varepsilon \in (0,1), k, \lambda, T \in \mathbb{Z}_{+}$ , and distributions  $\{D_S'\}_{S' \in S'}$ ;
2: Initialize Prod iterate  $h^{(1)} = [1 / k,\dots ,1 / k]^{\mathcal{X}}$  and ELP iterate  $(S^{\prime (1)},\ell^{(1)})\sim \mathrm{Uniform}(S^{\prime}\times \mathcal{G}_{\mathrm{mc}})$ ;
3: for  $t = 1$  to  $T$  do
4: Let  $h^{(t + 1)}(x) = \operatorname{Prod}(c_x^{(1:t)})$  where
$$
c _ {x} ^ {(t)} (\widehat {y}) := \frac {1 [ x \in S ^ {\prime (t)} ]}{2 \Pr_ {D} (x \in S ^ {\prime (t)})} (1 + 1 [ h (x) \in v ^ {(t)}, x \in S ^ {(t)} ] \cdot i ^ {(t)} \cdot \widehat {y} _ {j ^ {(t)}});
$$
5: Sample  $(x^{(t)},y^{(t)})\sim D_{S^{\prime}(t)}$  and let  $S^{\prime (t + 1)},\ell^{(t + 1)}\coloneqq \mathrm{ELP}(c_{\mathrm{adv}}^{(1:t)})$  with
$$
c _ {\mathrm {a d v}} ^ {(t)} \left(S ^ {\prime}, \ell_ {i, j, S, v}\right) := \frac {1}{2} 1 \left[ S ^ {\prime} = S ^ {\prime (t)} \right] \cdot \left(1 + i \cdot 1 \left[ h \left(x ^ {(t)}\right) \in v, x ^ {(t)} \in S \right] \cdot \left(\delta_ {y ^ {(t)}, j} - h \left(x ^ {(t)}\right) _ {j}\right)\right);
$$
6: end for
7: Return  $h^*$ , a uniform distribution over  $h^{(1)}, \ldots, h^{(T)}$ ;
1: Input:  $S \subseteq 2^{\mathcal{X}}, \varepsilon \in (0,1), k, \lambda, T, C \in \mathbb{Z}_{+}$ , agnostic learning oracle  $\mathcal{A}$ , and distribution  $D$ ;
2: Initialize Hedge iterate  $h^{(1)} = [1 / k, \dots, 1 / k]^{\mathcal{X}}$ ;
3: for  $t = 1$  to  $T$  do
4: Let  $\ell_{i,j,S,v} = \ell^{(t)} = \mathcal{A}_{\varepsilon /8}(c_{\mathrm{adv}}^{(t)},\mathcal{G}_{\mathrm{ag}}^{\prime})$  where  $c_{\mathrm{adv}}^{(t)}(\ell)\coloneqq 1 - \mathcal{L}_{D,\ell}(h^{(t)})$
5: Let  $h^{(t + 1)}(x) = \mathrm{Hedge}(c_x^{(1:t)})$  with
$$
c _ {x} ^ {(t)} (\widehat {y}) := \frac {1}{2} (1 + \sum_ {x ^ {\prime} \in \mathcal {X} ^ {\prime}} \operatorname * {P r} _ {D} (x ^ {\prime} \mid x) \cdot i ^ {(t)} \cdot 1 [ h (x) \in v ^ {(t)}, x ^ {\prime} \in S ^ {(t)} ] \cdot \widehat {y} _ {j ^ {(t)}};
$$
7: Take  $C\ln (T / \delta) / \varepsilon^2$  samples  $\mathbf{x}$  from  $D$ , and let  $t^{*} = \arg \min \sum \ell^{(t)}(h^{(t)},(x,x',y))$ ;
8: Return the predictor  $h(t^{*})$ ;  $t \in [T]$ $(x, x', y) \in \mathbf{x}$
Algorithm 7 Moment Multicalibration Algorithm (Theorem E.13)
1: Input:  $S \subseteq 2^{\mathcal{X}}, \varepsilon \in (0,1), k, \lambda, T, C \in \mathbb{Z}_{+}$ , agnostic learning oracle  $\mathcal{A}$ , and distribution  $D$ ;
2: Initialize Hedge iterate  $h_m^{(1)} = 0.5^{\mathcal{X}}$  and Hedge iterate  $h_{\mu}^{(1)} = 0.5^{\mathcal{X}}$ ;
3: for  $t = 1$  to  $T$  do
4: Let  $\ell_m^{(t)} = \mathcal{A}_{\varepsilon /8}(c_{\mathrm{adv}}^{(t)},\mathcal{G}_\mathrm{mc}^m)$  and  $\ell_{\mu}^{(t)} = \mathcal{A}_{\varepsilon /8}(c_{\mathrm{adv}}^{(t)},\mathcal{G}_\mathrm{mc}^\mu)$  where  $c_{\mathrm{adv}}^{(t)}(\ell)\coloneqq 1-$ $\mathcal{L}_{D,\ell}(h^{(t)})$
5: For  $x \in \mathcal{X}$ , let  $h_m^{(t+1)}(x) \coloneqq \operatorname{Hedge}(c_{m,x}^{(1:t)})$  and  $h_\mu^{(t+1)}(x) \coloneqq \operatorname{Hedge}(c_{\mu,x}^{(1:t)})$  where
$$
c _ {m, x} ^ {(t)} (\widehat {y}) := \frac {1}{2} (1 + i _ {m} ^ {(t)} \cdot 1 [ \widehat {y} \in v _ {m} ^ {(t)}, x \in S _ {m} ^ {(t)} ]), c _ {\mu , x} ^ {(t)} (\widehat {y}) := \frac {1}{2} (1 + i _ {\mu} ^ {(t)} \cdot 1 [ \widehat {y} \in v _ {\mu} ^ {(t)}, x \in S _ {\mu} ^ {(t)} ])
$$
6: end for
7: Take  $C\ln (T / \delta) / \varepsilon^2$  samples  $\mathbf{x}$  from  $D$  and let  $t^{*} = \arg \min$ $\sum \ell_{m}^{(t)}(h^{(t)},(x,y)) + \ell_{\mu}^{(t)}(h^{(t)},(x,y));$
8: Return the predictors  $h_{m}^{(t^{*})}, h_{\mu}^{(t^{*})}$ ;
E.4 Square-root Multicalibration Guarantees
The following Theorem E.15 is a stronger restatement of Theorem 4.4.
Theorem E.15. Fix  $\varepsilon > 0$ ,  $\lambda, k \in \mathbb{Z}_{+}$  and sets of groups  $S \subseteq 2^{\mathcal{X}}$ . Set  $T = C\ln(k)/\varepsilon^{2}$  for some universal constant  $C$ . Algorithm 8, with probability at least  $1 - \delta$ , requires no more than  $\widetilde{O}(\ln(k) \cdot (\ln(k|S|/\varepsilon\delta) + k\ln(\lambda))/\varepsilon^{4})$  samples from  $D$  to find a deterministic  $k$ -class predictor  $h$  satisfying
$$
\left| \mathbb {E} _ {(x, y) \sim D} \left[ (h (x) _ {j} - \delta_ {y, j}) \cdot 1 [ h (x) \in v, x \in S ] \right] \right| \leq \varepsilon \sqrt {\Pr (x \in S)},
$$
for all  $S \in \mathcal{S}, v \in V_{\lambda}^{k}, j \in [k]$ . That is,  $h$  is a deterministic predictor that is  $(S, \sqrt{\operatorname*{Pr}(x \in S)} \cdot \varepsilon, \lambda)$ -multicalibrated, where the error tolerance  $\sqrt{\operatorname*{Pr}(x \in S)} \cdot \varepsilon$  depends on the group mass.
Before proceeding to a proof, we first introduce some technical results which are simple variants of lemmas that the reader has seen previously in the manuscript. Note that, in the following multi-objective learning problem construction, we will allow negative objective values for simpler notation.
Fact E.16. Let  $D$  be a data distribution for some  $k$ -class prediction problem and fix  $\varepsilon > 0$ ,  $\lambda \in \mathbb{Z}_{+}$  and a set of groups  $S \subseteq 2^{\mathcal{X}}$ . We define the following set of multicalibration losses:
$$
\mathcal {G} _ {\mathrm {m c}} ^ {\prime} := \left\{\frac {1}{\sqrt {\operatorname* {P r} _ {D} (x \in S)}} \cdot i \cdot \left(h (x) _ {j} - \delta_ {y, j}\right) \cdot 1 [ h (x) \in v, x \in S ] \right\} _ {i \in \pm \{1 \}, j \in [ k ], S \in S, v \in V _ {\lambda} ^ {k}} \tag {12}
$$
Predictor  $p \in \Delta(\mathcal{P})$  is a  $\varepsilon$ -optimal solution to the multi-objective learning problem  $(\{D\}, \mathcal{G}_{\mathrm{mc}}, \mathcal{P})$  if and only if  $p$  is  $(S, \varepsilon \sqrt{\operatorname*{Pr}_D(x \in S)}, \lambda)$ -multicalibrated for  $D$ .
Proof. In the multi-objective learning problem  $(\{D\}, \mathcal{G}_{\mathrm{mc}}^{\prime}, \mathcal{P})$ , the multi-objective value of a predictor  $p$ ,  $\mathcal{L}^{*}(p) \coloneqq \max_{\ell_{i,j,S,v} \in \mathcal{G}} \mathcal{L}_{D,\ell_{i,j,S,v}}(p)$ , is exactly the magnitude of the predictor's multicalibration violation. Formally,
$$
\mathcal {L} ^ {*} (p) = \max  _ {j \in [ k ], S \in \mathcal {S}, v \in V _ {\lambda} ^ {k}} \frac {| \mathbb {E} _ {(x , y) \sim D , h \sim p} \left[ (h (x) _ {j} - \delta_ {y , j}) \cdot 1 [ h (x) \in v , x \in S ] \right] |}{\sqrt {\operatorname* {P r} _ {D} (x \in S)}}.
$$
The optimal value is 0, as the objectives are symmetric around 0 and the Bayes classifier achieves a loss of 0. Thus,  $\mathcal{L}^* (p) - \min_{h^*\in \mathcal{P}}\mathcal{L}^* (h^*)\geq \varepsilon$  if and only if our multicalibration violation is at least  $\varepsilon \sqrt{\operatorname*{Pr}(x\in S)}$
Lemma E.17. Let  $D$  be a data distribution for some  $k$ -class prediction problem,  $\varepsilon \in (0, 0.6)$ ,  $\delta \in (0, 1)$  and fix a set of groups  $S \subseteq 2^{\mathcal{X}}$  where, for all  $S \in S$ ,  $\operatorname{Pr}_D(x \in S) \geq \varepsilon^2$ . With only
Here, the tilde-O hides log-log factors.
$O(\ln (\mathcal{S} / \delta) / \varepsilon^4)$  samples, one can find with probability at least  $1 - \delta$  a vector  $v\in \mathbb{R}^{|\mathcal{S}|}$  where, for all  $S\in \mathcal{S}$ ,
$$
\left| v _ {S} - \Pr_ {D} (x \in S) \right| \leq \varepsilon \Pr_ {D} (x \in S), \tag {13}
$$
$$
\left| \frac {1}{\sqrt {v _ {S}}} - \frac {1}{\sqrt {\Pr_ {D} (x \in S)}} \right| \leq \frac {\varepsilon}{\sqrt {\Pr_ {D} (x \in S)}}, \tag {14}
$$
$$
\left| \sqrt {v _ {S}} - \sqrt {\operatorname* {P r} _ {D} (x \in S)} \right| \leq \varepsilon \sqrt {\operatorname* {P r} _ {D} (x \in S)}. \tag {15}
$$
Proof. Fix some sufficiently large universal constant  $C$ . Sample  $N = C\ln(\mathcal{S} / \delta) / \varepsilon^4$  datapoints  $X$  from  $D$ , and let  $v_{S} = \frac{1}{|X|}\sum_{(x,y)\in X}1[x\in S]$ . Observe that  $\mathbb{E}[v] = [\operatorname*{Pr}_D(x\in S)]_{S\in \mathcal{S}}$ . The multiplicative Chernoff bound says that, fixing an  $S\in \mathcal{S}$ ,
$$
\begin{array}{l} \Pr \left(\left| v _ {S} - \operatorname * {P r} _ {D} (x \in S) \right| \geq \varepsilon \operatorname * {P r} _ {D} (x \in S)\right) \leq 2 \exp \left(- \varepsilon^ {2} N \operatorname * {P r} _ {D} (x \in S) / 3\right) \\ \leq 2 \exp (- \varepsilon^ {4} N / 3). \\ \end{array}
$$
With our choice of  $N$ , taking a union bound over all  $S \in S$ , with probability at least  $1 - \delta$ , we have our first claim. Taking a square-root of both sides, we also have that for all  $S \in S$ ,  $\sqrt{(1 - \varepsilon)\operatorname{Pr}_D(x \in S)} \leq \sqrt{v_S} \leq \sqrt{(1 + \varepsilon)\operatorname{Pr}_D(x \in S)}$ . The second claim then follows by observing that, for any  $\varepsilon \in (0, 0.6)$ ,  $1 - \frac{1}{\sqrt{1 + \varepsilon}} \leq \varepsilon$ , and  $\frac{1}{\sqrt{1 - \varepsilon}} - 1 \leq \varepsilon$ . The third claim follows by observing that, for any  $\varepsilon \in (0, 1)$ ,  $\sqrt{1 + \varepsilon - 1} \leq \varepsilon$  and  $1 - \sqrt{1 - \varepsilon} \leq \varepsilon$ .
The following lemma is a modification of Theorem C.6 that states that the learner has a no-regret strategy on the calibration objectives given in Fact E.16.
Lemma E.18. Consider the set of  $k$ -class predictors  $\mathcal{P}$ , a data distribution  $D$ , and any adversarial sequence of stochastic costs  $\ell^{(1:T)} \in \mathcal{G}_{\mathrm{mc}}'$ , where  $\mathcal{G}_{\mathrm{mc}}'$  are the multicalibration objectives defined in Lemma E.16. Suppose you are given  $v \in \mathbb{R}$  satisfying (14) and (13) for all  $S \in S$  and that, for all  $S \in S$ ,  $\operatorname{Pr}_D(x \in S) \geq \varepsilon^2$ . There is a no-regret algorithm that outputs (deterministic) predictors  $h^{(1:T)} \in \mathcal{P}'$  such that  $\operatorname{Reg}_{weak}(h^{(1:T)}, \{\mathcal{L}_{D,\ell^{(t)}}\}^{(1:T)}) \leq O(\sqrt{\ln(k)T} + \varepsilon T)$ . Moreover, the algorithm does not need any samples from  $D$ .
Proof. Consider the following algorithm. At each feature  $x \in \mathcal{X}$ , initialize a Prod algorithm that picks an action  $h^{(t)}(x) \in \Delta(\mathcal{Y})$  at each timestep  $t \in [T]$ . Aggregating each algorithm's action yields our learner's overall action  $h^{(t)} \in \mathcal{P}$ . For each  $x \in \mathcal{X}$ , let  $h^{(t+1)}(x)$  be the outcome of Prod at step  $t + 1$  after observing linear loss functions  $f_{h^{(\tau)},x}^{(\tau)}: \mathbb{R}^k \to [0,1]$  for  $\tau \in [t]$ :
$$
f _ {h ^ {(\tau)}, x} ^ {(\tau)} (z) := \frac {1}{2 \sqrt {v _ {S ^ {(\tau)}}}} \left(1 + z _ {j ^ {(\tau)}} \cdot i ^ {(\tau)} \cdot 1 [ h ^ {(\tau)} (x) \in v ^ {(\tau)}, x \in S ^ {(\tau)} ]\right). \tag {16}
$$
Prod gives  $\sum_{t=1}^{T} f_{h^{(t)},x}^{(t)}(h^{(t)}(x)) - \min_{z^* \in \Delta(y)} \sum_{t=1}^{T} f_{h^{(t)},x}^{(t)}(z^*) \leq C \sqrt{\ln(k) \sum_{t=1}^{T} v_{S^{(t)}}^{-1}}$  (Lemma A.4) for some universal constant  $C$ . Since this inequality holds for all  $x \in \mathcal{X}$ , applying the law of total expectation to Lemma A.4,
$$
\begin{array}{l} \sum_ {x \in \mathcal {X}} C _ {D} ^ {\Pr} (x) \sqrt {\ln (k) \sum_ {t = 1} ^ {T} v _ {S ^ {(t)}} ^ {- 1}} \\ \geq \sum_ {x \in \mathcal {X}} \sum_ {t = 1} ^ {T} \operatorname * {P r} _ {D} (x) f _ {h ^ {(t)}, x} ^ {(t)} (h ^ {(t)} (x)) - \sum_ {x \in \mathcal {X}} \min  _ {z ^ {*} \in \Delta (\mathcal {Y})} \sum_ {t = 1} ^ {T} \operatorname * {P r} _ {D} (x) f _ {h ^ {(t)}, x} ^ {(t)} (z ^ {*}) \\ = \sum_ {x \in \mathcal {X}} \sum_ {t = 1} ^ {T} \Pr_ {D} (x) f _ {h ^ {(t)}, x} ^ {(t)} (h ^ {(t)} (x)) - \min  _ {h ^ {*} \in \mathcal {P}} \sum_ {x \in \mathcal {X}} \sum_ {t = 1} ^ {T} \Pr_ {D} (x) f _ {h ^ {(t)}, x} ^ {(t)} (h ^ {*} (x)) \\ = \max _ {h ^ {*} \in \mathcal {P}} \sum_ {x \in \mathcal {X}} \sum_ {t = 1} ^ {T} \operatorname * {P r} _ {D} (x) f _ {h ^ {(t)}, x} ^ {(t)} (h ^ {(t)} (x) - h ^ {*} (x)), \\ \end{array}
$$
with the equality following because we allow arbitrary predictors. Expanding the definition of  $f$ ,
$$
\begin{array}{l} \max  _ {h ^ {*} \in \mathcal {P}} \sum_ {x \in \mathcal {X}} \sum_ {t = 1} ^ {T} \Pr_ {D} (x) f _ {h ^ {(t)}, x} ^ {(t)} (h ^ {(t)} (x) - h ^ {*} (x)) \\ = \max  _ {h ^ {*} \in \mathcal {P}} \sum_ {x \in \mathcal {X}} \sum_ {t = 1} ^ {T} \Pr_ {D} (x) \frac {1}{2 \sqrt {v _ {S ^ {(t)}}}} \left(\left(h ^ {(t)} (x) - h ^ {*} (x)\right) \cdot i ^ {(t)} \cdot 1 [ h ^ {(t)} (x) \in v ^ {(t)}, x \in S ^ {(t)} ]\right) \\ \geq \sum_ {t = 1} ^ {T} \mathbb {E} _ {x \sim D} \left[ \frac {1}{2 \sqrt {v _ {S ^ {(t)}}}} \left(\left(h ^ {(t)} (x) - \mathbb {E} _ {y \sim D _ {S ^ {(t)}}} [ \delta_ {y} \mid x ]\right) \cdot i ^ {(t)} \cdot 1 \left[ h ^ {(t)} (x) \in v ^ {(t)}, x \in S ^ {(t)} \right]\right) \right] \\ = \sum_ {t = 1} ^ {T} \sqrt {\frac {\operatorname * {P r} _ {D} (x \in S ^ {(t)})}{v _ {S ^ {(t)}}}} \mathcal {L} _ {D, \ell^ {(t)}} (h ^ {(t)}). \\ \end{array}
$$
By definition of  $v$
$$
\sum_ {t = 1} ^ {T} \sqrt {\frac {\operatorname* {P r} _ {D} (x \in S ^ {(t)})}{v _ {S ^ {(t)}}}} \mathcal {L} _ {D, \ell^ {(t)}} (h ^ {(t)}) \geq \sum_ {t = 1} ^ {T} \sqrt {1 - \varepsilon} \mathcal {L} _ {D, \ell^ {(t)}} (h ^ {(t)}) \geq \sum_ {t = 1} ^ {T} \mathcal {L} _ {D, \ell^ {(t)}} (h ^ {(t)}) - T \varepsilon .
$$
We also know by construction of the losses in  $\mathcal{G}_{\mathrm{mc}}^{\prime}$  that the weak baseline  $B_{\mathrm{weak}} = 0$ . We can therefore bound
$$
\begin{array}{l} \operatorname {R e g} _ {\text {w e a k}} \left(h ^ {(1: T)}, \left\{\mathcal {L} _ {D, \ell^ {(t)}} \right\} ^ {(1: T)}\right) \leq T \varepsilon + \sum_ {x \in \mathcal {X}} C _ {D} \Pr (x) \sqrt {\ln (k) \sum_ {t = 1} ^ {T} \frac {1}{v _ {S ^ {(t)}}}} \\ \leq T \varepsilon + \sum_ {x \in \mathcal {X}} C _ {D} ^ {\Pr} (x) \sqrt {\ln (k) \sum_ {t = 1} ^ {T} (1 + \varepsilon) \frac {1}{\Pr_ {D} (x \in S ^ {(t)})}} \\ = T \varepsilon + C \sqrt {T \ln (k) (1 + \varepsilon)} \sum_ {x \in \mathcal {X}} \sqrt {\frac {\operatorname* {P r} _ {D} (x)}{T}} \sqrt {\frac {1}{T} \sum_ {t = 1} ^ {T} \operatorname* {P r} _ {D _ {S ^ {(t)}}} (x)}. \\ \end{array}
$$
To bound the last inequality, let  $u \in \mathbb{R}^{\mathcal{X}}$ ,  $v \in \mathbb{R}^{\mathcal{X}}$  be vectors defined as  $u_{x} = \sqrt{\operatorname*{Pr}_{D}(x)}$  and  $v_{x} = \sqrt{\sum_{t=1}^{T} \operatorname*{Pr}_{D_{S(t)}}(x)}$ . Then, by Cauchy-Schwarz inequality, we have that  $u \cdot v \leq \|u\|_2 \|v\|_2$ . In other words,
$$
\begin{array}{l} C \sqrt {T \ln (k)} \sum_ {x \in \mathcal {X}} \sqrt {\Pr_ {D} (x) \frac {1}{T} \sum_ {t = 1} ^ {T} \Pr_ {D _ {S ^ {(t)}}} (x)} \leq C \sqrt {T \ln (k)} \left(\sum_ {x \in \mathcal {X}} \Pr_ {D} (x)\right) \left(\sum_ {x \in \mathcal {X}} \frac {1}{T} \sum_ {t = 1} ^ {T} \Pr_ {D _ {S ^ {(t)}}} (x)\right) \\ \leq C \sqrt {T \ln (k)}. \\ \end{array}
$$
The following lemma states that the adversary can efficiently best-respond to the objectives from Fact E.16.
Lemma E.19. Consider a  $k$ -class predictor  $h \in \mathcal{P}$ , a data distribution  $D$ , and set of multicalibration objectives  $\mathcal{G}_{\mathrm{mc}}'$ . Suppose you are given  $v \in \mathbb{R}$  satisfying (15) for all  $S \in S$ . There is an algorithm that, taking only  $\varepsilon^{-2}(\ln(|S| / \delta) + k \ln(\lambda))$  samples for some universal constant  $C$ , returns a  $\ell_{i,j,S,v} \in \mathcal{G}_{\mathrm{mc}}'$  such that, with probability at least  $1 - \delta$ ,  $\mathcal{L}_{D,\ell_{i,j,S,v}}(h) + 2\varepsilon \geq \max_{i^*,j^*,S^*,v^*} \mathcal{L}_{D,\ell_{i^*,j^*,S^*,v^*}}(h)$ .
Proof. Consider the following algorithm. Draw  $N = C \varepsilon^{-2} (\ln(|S| / \delta) + k \ln(\lambda))$  samples  $(x, y)^{(1:N)}$  from  $D$ . For all  $S \in S$ , initialize the empty buffer  $X_S = \{\}$ . Then, for every value of  $r = 1, \ldots, N$  and for every group  $S \in S$ , if  $x^{(r)} \in S$ , append  $(x, y)^{(r)}$  to  $X_S$ . Let  $\ell_{i,j,S,v} = \max_{i,j,S,v} \sqrt{\frac{v_S}{\operatorname*{Pr}_D(x \in S)}} \frac{1}{|X_S|} \sum_{(x,y) \in X_S} \ell_{i,j,S,v}(h, (x,y))$  be the multicalibration objective
that minimizes the empirical risk on its respective buffer  $X_{S}$ . Note that finding  $\ell_{i,j,S,v}$  does not require knowledge of  $\operatorname{Pr}_D(x \in S)$ , as the explicit factor of  $\sqrt{1 / \operatorname{Pr}_D(x \in S)}$  is cancelled out by  $\ell_{i,j,S,v}$ .
We first observe that  $|X_S|$  is a binomial random variable with parameters  $N$  and  $\operatorname{Pr}(x \in S)$ . By Chernoff's bound, with probability  $1 - \delta |\mathcal{S}|^{-1}$ ,  $|X_S| \geq \operatorname{Pr}(x \in S) \cdot C' \varepsilon^{-2} (\ln (|\mathcal{S}| / \delta) + k \ln (\lambda))$  for some universal constant  $C'$ , where we choose  $C$  to be large enough so that  $C' \geq \frac{C}{2}$ . By union bound, with probability  $1 - \delta$ , for every  $S \in \mathcal{S}$ ,  $|X_S| \geq \operatorname{Pr}(x \in S) N / 2$ .
Condition on this event and fix a  $\ell_{i,j,S,v} \in \mathcal{G}_{\mathrm{mc}}^{\prime}$ . We observe that each  $(x,y) \in X_S$  is an unbiased sample from  $D_S$ , we have at least  $\operatorname*{Pr}(x \in S)N/2$  samples, and  $\ell_{i,j,S,v} \in \pm \{1/\sqrt{\operatorname*{Pr}(x \in S)}\}$ . Thus, by Chernoff's bound, with probability at least  $1 - \delta |\mathcal{G}_{\mathrm{mc}}^{\prime}|^{-1}$ ,
$$
\begin{array}{l} \left| | X _ {S} | ^ {- 1} \sum_ {(x, y) \in X _ {S}} \sqrt {\Pr (x \in S)} \ell_ {i, j, S, v} (h, (x, y)) - \sqrt {\Pr (x \in S)} \mathcal {L} _ {D _ {S}, \ell_ {i, j, S, v}} (h) \right| \\ \leq \varepsilon / (3 \sqrt {\Pr (x \in S)}). \\ \end{array}
$$
Thus with probability at least  $1 - \delta$ , for all  $\ell_{i,j,S,v} \in \mathcal{G}_{\mathrm{mc}}'$
$$
\begin{array}{l} \left| \left| X _ {S} \right| ^ {- 1} \sum_ {(x, y) \in X _ {S}} \Pr (x \in S) \ell_ {i, j, S, v} (h, (x, y)) - \Pr (x \in S) \mathcal {L} _ {D _ {S}, \ell_ {i, j, S, v}} (h) \right| \\ = \left| | X _ {S} | ^ {- 1} \sum_ {(x, y) \in X _ {S}} \Pr (x \in S) \ell_ {i, j, S, v} (h, (x, y)) - \mathcal {L} _ {D, \ell_ {i, j, S, v}} (h) \right| \\ \leq \varepsilon . \\ \end{array}
$$
Taking a union bound over all  $\ell \in \mathcal{G}_{\mathrm{mc}}^{\prime}$ , by uniform convergence, with probability at least  $1 - 2\delta$ , our returned  $\ell_{i,j,S,v}$  is an  $\varepsilon$  best-response to the cost function  $\ell_{i,j,S,v} \mapsto -\sqrt{\frac{v_S}{\operatorname*{Pr}_D(x \in S)}} \mathcal{L}_{D,\ell_{i,j,S,v}}(h^{(t)})$ . By (15),  $\max_{\ell_{i^*,j^*,S^*,v^*}} \mathcal{L}_{D,\ell_{i^*,j^*,S^*,v^*}}(h^{(t)}) - \mathcal{L}_{D,\ell_{i,j,S,v}}(h^{(t)}) \leq \varepsilon$ . Thus,  $\ell_{i,j,S,v}$  is an  $2\varepsilon$  best-response to the cost function  $\ell_{i,j,S,v} \mapsto -\mathcal{L}_{D,\ell_{i,j,S,v}}(h^{(t)})$ .