SlowGuess/NeurIPS_2023_OCR huggingface dataset:
This shows that  $\tilde{p} \in \mathrm{Lin}(\mathcal{C})$ , and moreover that it is the optimal approximation to  $p^*$  in  $\mathrm{Lin}(\mathcal{C})$ , as it is the projection of  $p^*$  onto  $\mathrm{Lin}(\mathcal{C})$ . This shows that  $\tilde{p}$  is an  $(\ell_2, \mathrm{Lin}(\mathcal{C}), 0)$ -agnostic learner.
It is easy to check that  $\tilde{p}$  is perfectly calibrated. It is  $(\mathcal{C},0)$ -multiaccurate, since it is the projection of  $p^*$  onto  $\operatorname{Lin}(\mathcal{C})$ , so  $\tilde{p} - p^*$  is orthogonal to  $\operatorname{Lin}(\mathcal{C})$ . Hence we can apply Lemma B.3 to conclude that it is  $(\mathcal{L}_{\mathrm{GLM}}, \operatorname{Lin}(C), 0)$ -loss OI, where  $\mathcal{L}_{\mathrm{GLM}}$  which contains the squared loss.
To show that  $\tilde{p}$  is not swap-agnostic, we observe that conditioning on the value of  $\tilde{p}(\mathbf{x}) = (4 + 3x_2)/8$  is equivalent to conditioning on  $x_2 \in \{\pm 1\}$ . For each value of  $x_2$ , the restriction of  $p^*$  which is now linear in  $x_1$  belongs to  $\operatorname{Lin}(\mathcal{C})$ . Indeed if we condition on  $\tilde{p}(x) = 1/8$  so that  $x_2 = -1$ , we have
$$
p ^ {*} (x) = \frac {1}{2} - \frac {3}{8} + \frac {1}{8} x _ {1} = \frac {1 + x _ {1}}{8}.
$$
Conditioned on  $\tilde{p}(x) = 7/8$  so that  $x_2 = 1$ , we have
$$
p ^ {*} (x) = \frac {1}{2} + \frac {3}{8} - \frac {1}{8} x _ {1} = \frac {7 - x _ {1}}{8}.
$$
$$
\underset {v \sim \mathcal {D} _ {\tilde {p}}} {\mathbf {E}} \left[ \left| \underset {h \in \operatorname {L i n} (\mathcal {C})} {\min } \mathbf {E} [ (\mathbf {y} - h (\mathbf {x})) ^ {2} | f (\mathbf {x}) = v ] \right| \right] = \mathbf {E} [ (y - p ^ {*} (x)) ^ {2} ] = \operatorname {V a r} [ \mathbf {y} ],
$$
whereas the variance decomposition of squared loss gives
$$
\begin{array}{l} \mathbf {E} [ (\mathbf {y} - \tilde {p} (\mathbf {x})) ^ {2} ] = \mathbf {E} [ (\mathbf {y} - p ^ {*} (\mathbf {x})) ^ {2} ] + \mathbf {E} [ (p ^ {*} (\mathbf {x}) - \tilde {p} (\mathbf {x})) ^ {2} ] \\ = \operatorname {V a r} [ \mathbf {y} ] + \frac {1}{6 4} \mathbf {E} \left[ \left(x _ {1} x _ {2}\right) ^ {2} \right] \\ = \operatorname {V a r} [ \mathbf {y} ] + \frac {1}{6 4}. \\ \end{array}
$$
Hence  $\tilde{p}$  is not a  $(\ell_2,\mathrm{Lin}(\mathcal{C}),\delta)$ -swap agnostic learner for  $\delta < 1 / 64$
To see that  $f$  is not multicalibrated for small  $\alpha$ , observe that conditioned on  $x_{2} \in \{\pm 1\}$ , the correlation between  $x_{1}$  and  $\tilde{p} - p^{*}$  is  $1/8$ .
Note that item (1) above separates swap omniprediction from omniprediction and agnostic learning. This separation can also be derived from [15, Theorem 7.5] which separated (standard) omniprediction from agnostic learning, since swap omniprediction implies standard omniprediction.
Comparing notions for GLM losses. When we restrict our attention to  $\mathcal{L}_{\mathrm{GLM}}$ , in fact, the notions of swap loss OI and swap omniprediction are equivalent. The key observation here is that  $\partial \mathcal{L}_{\mathrm{GLM}} \circ \mathcal{C} = \mathcal{C}$ , as shown in [14]. Paired with Theorem 3.3 and Theorem A.1 (proved next), we obtain the following collapse.
Claim B.5. The notions of  $(\mathcal{L}_{\mathrm{GLM}},\mathcal{C},\alpha_{1})$  -swap loss  $OI$  and  $(\mathcal{L}_{\mathrm{GLM}},\mathcal{C},\alpha_{2})$  -swap omniprediction are equivalent.
Proof. To see this, note that by Theorem A.1,  $(\mathcal{L}_{\mathrm{GLM}},\mathcal{C},\alpha_{1})$  -swap loss OI is equivalent to  $(\partial \mathcal{L}_{\mathrm{GLM}}\circ \mathcal{C},\alpha_1^{\prime})$  -swap multicalibration. We know from Theorem 3.3 that this is also equivalent to  $(\partial \mathcal{L}_{\mathrm{GLM}}\circ \mathcal{C},\alpha_{2})$  -swap omniprediction. So, by the fact that  $\partial \mathcal{L}_{\mathrm{GLM}}\circ \mathcal{C} = \mathcal{C}$ , we have the claimed equivalence.
Finally, we know that loss  $\mathrm{OI}$  implies omniprediction for  $\mathcal{L}_{\mathrm{GLM}}$ , since this holds true for all  $\mathcal{L}$ . We do not know if these notions are equivalent for  $\mathcal{L}_{\mathrm{GLM}}$ , since the construction in Lemma B.1 used the  $\ell_4$  loss which does not belong to  $\mathcal{L}_{\mathrm{GLM}}$ .
We define the multicalibration error of  $\tilde{p}$  wrt  $\mathcal{C}$  under  $\mathcal{D}$  as
$$
\mathsf {M C E} _ {\mathcal {D}} (f, \mathcal {C}) = \max  _ {c \in \mathcal {C}} \left. \mathbf {E} _ {\mathbf {v} \sim \mathcal {D} _ {\tilde {p}}} \left[ \left| \mathbf {E} _ {\mathcal {D} | _ {\mathbf {v}}} [ c (\mathbf {x}) (\mathbf {y} - \mathbf {v}) ] \right| \right]. \right.
$$
We define the swap multicalibration error of  $\tilde{p}$  wrt  $\mathcal{C}$  under  $\mathcal{D}$  as
$$
\mathsf {s M C E} _ {\mathcal {D}} (\tilde {p}, \mathcal {C}) = \underset {\mathbf {v} \sim \mathcal {D} _ {\tilde {p}}} {\mathbf {E}} \left[ \max  _ {c \in \mathcal {C}} \left| \underset {\mathcal {D} | _ {\mathbf {v}}} {\mathbf {E}} [ c (\mathbf {x}) (\mathbf {y} - \mathbf {v}) ] \right| \right]
$$
C.1 Properties of Swap Notions of Supervised Learning
Proof of Claim 2.4. We let  $\ell_v = \ell$  for all  $v \in \mathrm{Im}(\tilde{p})$ , so that  $k(v) = k_{\ell}(v)$ . We pick the hypothesis
$$
h_{v} = \operatorname *{arg  min}_{h\in \mathcal{H}}\mathbf{E}_{\mathcal{D}|_{v}}\left[\ell (\mathbf{y},h(\mathbf{x}))\right]
$$
The swap omniprediction guarantee reduces to
$$
\underset {\mathbf {v} \in \mathcal {D} _ {\tilde {p}}} {\mathbf {E}} \left[ \underset {\mathcal {D} | _ {\mathbf {v}}} {\mathbf {E}} \left[ \ell (\mathbf {y}, k _ {\ell} (\mathbf {v})) \right] = \underset {\mathcal {D}} {\mathbf {E}} \left[ \ell (\mathbf {y}, k _ {\ell} (\tilde {p} (\mathbf {x}))) \right] \right. \leq \underset {\mathbf {v} \sim \mathcal {D} _ {\tilde {p}}} {\mathbf {E}} \min  _ {h \in \mathcal {H}} \underset {\mathcal {D} | _ {\mathbf {v}}} {\mathbf {E}} \left[ \ell (\mathbf {y}, h (\mathbf {x})) \right] + \delta .
$$
This implies that  $f = k_{\ell} \circ \tilde{p}$  is a swap agnostic learner for every  $\ell \in \mathcal{L}$  since we allow the choice of  $h$  to depend on  $\tilde{p}(\mathbf{x})$  which is more informative than  $f(\mathbf{x}) = k_{\ell}(\tilde{p}(\mathbf{x}))$ .
Proof of Claim 2.7. We have
$$
\begin{array}{l} \mathsf {s M C E} _ {\mathcal {D}} (\tilde {p}, \mathcal {C}) = \left. \mathbf {E} _ {\mathbf {v} \sim \mathcal {D} _ {\bar {p}}} \left[ \max  _ {c \in \mathcal {C}} \left| \mathbf {E} _ {\mathcal {D} | _ {\mathbf {v}}} [ c (\mathbf {x}) (\mathbf {y} ^ {*} - \mathbf {v}) ] \right| \right] \right] \\ \geq \max  _ {c \in \mathcal {C}} \underset {\mathbf {v} \sim \mathcal {D} _ {\tilde {p}}} {\mathbf {E}} \left[ \left| \underset {\mathcal {D} | _ {\mathbf {v}}} {\mathbf {E}} [ c (\mathbf {x}) (\mathbf {y} - \mathbf {v}) ] \right| \right] = \operatorname {M C E} _ {\mathcal {D}} (\tilde {p}, \mathcal {C}) \\ \end{array}
$$
since the expectation of the max is higher than the max of expectations. Bounding the RHS by  $\alpha$  is equivalent to  $(\mathcal{C},\alpha)$ -multicalibration.
Proof of Claim 2.9. The  $\ell_{\infty}$  bound is immediate from the definition of  $\bar{p}$ . We bound the swap multicalibration error of  $tf$ . We have  $\bar{p} (\mathbf{x}) = j\delta$  iff  $\tilde{p} (\mathbf{x})\in B_j$ , so that  $|\tilde{p} (\mathbf{x}) - j\delta |\leq \delta$  holds conditioned on this event. So
$$
\begin{array}{l} \mathsf {s M C E} _ {\mathcal {D}} (\bar {p}, \mathcal {C}) = \sum_ {j \in [ m ]} \Pr [ \bar {p} (\mathbf {x}) = j \delta ] \max  _ {c \in \mathcal {C}} \left| \mathbf {E} [ c (\mathbf {x}) (\mathbf {y} - j \delta) | \bar {p} (\mathbf {x}) = j \delta ] \right| \\ = \sum_ {j \in [ m ]} \operatorname * {P r} [ \tilde {p} (\mathbf {x}) \in B _ {j} ] \max _ {c \in \mathcal {C}} \left| \mathbf {E} [ c (\mathbf {x}) (\mathbf {y} - j \delta) | \tilde {p} (\mathbf {x}) \in B _ {j} ] \right| \\ \leq \sum_ {j \in [ m ]} \Pr [ \tilde {p} (\mathbf {x}) \in B _ {j} ] \left(\delta + \max  _ {c \in \mathcal {C}} \left| \underset {\mathcal {D}} {\mathbf {E}} [ c (\mathbf {x}) (\mathbf {y} - \tilde {p} (\mathbf {x})) | \tilde {p} (\mathbf {x}) \in B _ {j} ] \right|\right) \\ \leq \delta + \sum_ {j \in [ m ]} \Pr [ \tilde {p} (\mathbf {x}) \in B _ {j} ] \max  _ {c \in \mathcal {C}} \left| \mathbf {E} [ c (\mathbf {x}) (\mathbf {y} - \tilde {p} (\mathbf {x})) | \tilde {p} (\mathbf {x}) \in B _ {j} ] \right| \tag {25} \\ \end{array}
$$
Let us fix a bucket  $B_{j}$  and a particular  $c\in \mathcal{C}$ . For  $\beta \geq \alpha$  to be specified later we have
$$
\begin{array}{l} | \mathbf {E} [ c (\mathbf {x}) (\mathbf {y} - \tilde {p} (\mathbf {x})) | \tilde {p} (\mathbf {x}) \in B _ {j} ] | \leq \Pr [ c (\mathbf {x}) (\mathbf {y} - \tilde {p} (\mathbf {x})) \geq \beta | \tilde {p} (\mathbf {x}) \in B _ {j} ] + \beta \Pr [ c (\mathbf {x}) (\mathbf {y} - f (x)) \leq \beta | \tilde {p} (\mathbf {x}) \in B _ {j} ] \\ \leq \frac {\Pr [ \tilde {p} (\mathbf {x}) \in \operatorname {B a d} _ {\beta} (c , f) \cap B _ {j} ]}{\Pr [ \tilde {p} (\mathbf {x}) \in B _ {j} ]} + \beta \\ \leq \frac {\Pr [ \tilde {p} (\mathbf {x}) \in \operatorname {B a d} _ {\beta} (c , f) ]}{\Pr [ \tilde {p} (\mathbf {x}) \in B _ {j} ]} + \beta \\ \leq \frac {\alpha / \beta}{\Pr [ \tilde {p} (\mathbf {x}) \in B _ {j} ]} + \beta . \\ \end{array}
$$
Since this bound holds for every  $c$ , it holds for the max over  $c \in \mathcal{C}$  conditioned on  $\tilde{p}(\mathbf{x}) \in B_j$ . Hence
$$
\begin{array}{l} \sum_ {j \in [ m ]} \Pr [ \tilde {p} (\mathbf {x}) \in B _ {j} ] \max  _ {c \in \mathcal {C}} \left| \underset {\mathcal {D}} {\mathbf {E}} [ c (\mathbf {x}) (\mathbf {y} - \tilde {p} (\mathbf {x})) | \tilde {p} (\mathbf {x}) \in B _ {j} ] \right| \leq \sum_ {j \in [ m ]} \Pr [ \tilde {p} (\mathbf {x}) \in B _ {j} ] \left(\frac {\alpha / \beta}{\Pr [ \tilde {p} (\mathbf {x}) \in B _ {j} ]} + \beta\right) \\ \leq \frac {\alpha}{\beta \delta} + \beta , \\ \end{array}
$$
where we use  $m = 1 / \delta$ . Plugging this back into Equation (25) gives
$$
\mathsf {s M C E} _ {\mathcal {D}} (\bar {p}, \mathcal {C}) = \underset {\mathbf {v} \sim \bar {p} _ {\mathcal {D}}} {\mathbf {E}} \left[ \max  _ {c \in \mathcal {C}} \left| \underset {\mathcal {D} | _ {\mathbf {v}}} {\mathbf {E}} [ c (\mathbf {x}) (\mathbf {y} - \mathbf {v}) ] \right| \right] \leq \frac {\alpha}{\beta \delta} + \beta + \delta .
$$
Taking  $\beta = \sqrt{\alpha / \delta}$  gives the desired claim.
C.2 Omitted Proofs from Main Result
Proof of Lemma 3.2. We will show that for  $p, p' \in [0,1]$  and  $t_0 \in I_\ell$ , we have
$$
\ell (p, t _ {0}) - \ell \left(p ^ {\prime}, t _ {0}\right) \leq | p - p ^ {\prime} | B.
$$
By the definition of  $\ell(p, t)$ , we have
$$
\begin{array}{l} \ell \left(p, t _ {0}\right) - \ell \left(p ^ {\prime}, t _ {0}\right) = \left(p - p ^ {\prime}\right) \ell \left(0, t _ {0}\right) + \left(1 - p - 1 + p ^ {\prime}\right) \ell \left(1, t _ {0}\right) \\ = (p - p ^ {\prime}) \left(\ell (0, t _ {0}) - \ell (1, t _ {0})\right) \\ \end{array}
$$
Taking absolute values and using the Boundedness property gives the desired claim.
Proof of Claim 3.4. Suppose that  $h \in \operatorname{Lin}(\mathcal{C}, W)$  of the form  $h(x) = \sum_{c \in \mathcal{C}} w_c \cdot c(x)$ . From Claim 2.7, we know that the multicalibration violation for  $c \in \mathcal{C}$  is bounded by  $\alpha(v)$  for every  $v \in \operatorname{Im}(\tilde{p})$ .
$$
\begin{array}{l} | \mathbf {E} [ h (\mathbf {x}) (\mathbf {y} - v) \mid \tilde {p} (\mathbf {x}) = v ] | = \left| \mathbf {E} \left[ \sum_ {c \in \mathcal {C}} w _ {c} \cdot c (\mathbf {x}) (\mathbf {y} - v) \mid \tilde {p} (\mathbf {x}) = v \right] \right| \\ \leq \left(\sum_ {c \in \mathcal {C}} | w _ {c} |\right) \cdot \max  _ {c \in \mathcal {C}} | \mathbf {E} [ c (\mathbf {x}) (\mathbf {y} - v) | \tilde {p} (\mathbf {x}) = v ] | \\ \leq W \cdot \alpha (v) \\ \end{array}
$$
The inequalities follow by Holder's inequality and the assumed bound on the weight of  $W$  for  $h \in \operatorname{Lin}(\mathcal{C}, W)$ .
Proof of Claim 3.5. Recall that  $\operatorname{Cov}[\mathbf{y},\mathbf{z}] = \mathbf{E}[\mathbf{y}\mathbf{z}] - \mathbf{E}[\mathbf{y}]\mathbf{E}[\mathbf{z}]$ . For any  $h\in \operatorname{Lin}(\mathcal{C},W)$  we have
$$
\begin{array}{l} | \operatorname {C o v} [ \mathbf {y}, h (\mathbf {x}) | \tilde {p} (\mathbf {x}) = v ] | = | \mathbf {E} [ h (\mathbf {x}) (\mathbf {y} - \mathbf {E} [ \mathbf {y} ]) | \tilde {p} (\mathbf {x}) = v ] | \\ = | \mathbf {E} [ h (\mathbf {x}) (\mathbf {y} - v) | \tilde {p} (x) = v ] | + | \mathbf {E} [ (v - \mathbf {y}) | \tilde {p} (\mathbf {x}) = v ] | \\ \leq (W + 1) \alpha (v) \\ \end{array}
$$
where we use the fact that  $h \in \operatorname{Lin}(\mathcal{C}, W)$  and  $1 \in \mathcal{C}$ . Since  $\mathbf{y} \in \{0, 1\}$ , this implies the claimed bounds by standard properties of covariance (see [15, Corollary 5.1]).
Proof of Lemma 3.6. For any  $y \in \{0,1\}$ ,
$$
\begin{array}{l} \mathbf {E} _ {\mathcal {D} | _ {v}} \left[ \ell (\mathbf {y}, h (\mathbf {x})) | (\tilde {p} (\mathbf {x}), \mathbf {y}) = (v, y) \right] = \mathbf {E} _ {\mathcal {D} | _ {v}} \left[ \ell (y, h (\mathbf {x})) | (\tilde {p} (\mathbf {x}), \mathbf {y}) = (v, y) \right] \\ \geq \ell (y, \mathbf {E} [ h (\mathbf {x}) | (\tilde {p} (\mathbf {x}), \mathbf {y}) = (v, y) ]) (26) \\ = \ell (y, \mu (h: v, y)) \\ \geq \ell (y, \Pi_ {\ell} (\mu (h: v, y))). (27) \\ \end{array}
$$
where Equation (26) uses Jensen's inequality, and Equation (27) uses the optimality of projection for nice loss functions. Further, by the 1-Lipschitzness of  $\ell$  on  $I_{\ell}$ , and of  $\Pi_{\ell}$  on  $\mathbb{R}$
$$
\begin{array}{l} \ell (y, \Pi_ {\ell} (\mu (h: v, y))) - \ell (y, \Pi_ {\ell} (\mu (h: v))) \leq | \Pi_ {\ell} (\mu (h: v, y)) - \Pi_ {\ell} (\mu (h: v)) | \\ \leq | \mu (h: v, y) - \mu (h: v) | \tag {28} \\ \end{array}
$$
$$
\begin{array}{l} \underset {\mathcal {D} | _ {v}} {\mathbf {E}} \left[ \ell (\mathbf {y}, \Pi_ {\ell} (\mu (h: v)) \right] - \underset {\mathcal {D} | _ {v}} {\mathbf {E}} \left[ \ell (\mathbf {y}, h (\mathbf {x})) \right] \\ = \sum_ {y \in \{0, 1 \}} \Pr [ \mathbf {y} = y | \tilde {p} (\mathbf {x}) = v ] \left(\ell (y, \Pi_ {\ell} (\mu (h: v))) - \mathbf {E} [ \ell (y, h (\mathbf {x})) | (\tilde {p} (\mathbf {x}), \mathbf {y}) = (v, y) ]\right) \\ \leq \sum_ {y \in \{0, 1 \}} \Pr [ \mathbf {y} = y | \tilde {p} (\mathbf {x}) = v ] \left(\ell \left(y, \Pi_ {\ell} \left(\mu (h: v)\right)\right) - \ell \left(y, \Pi_ {\ell} \left(\mu (h: v, y)\right)\right)\right) \quad (\text {B y E q u a t i o n (2 7)}) \\ \leq \sum_ {y \in \{0, 1 \}} \Pr [ \mathbf {y} = y | \tilde {p} (\mathbf {x}) = v ] \left| \mu (h: v, y) - \mu (h: v) \right| \tag {byEquation(28)} \\ \leq 2 (W + 1) \alpha (v). \quad (\text {B y}) \\ \end{array}
$$
D Details on Algorithm
Here, we give a high-level overview of the MCBoost algorithm of [20] and weak agnostic learning. Definition D.1 (Weak agnostic learning). Suppose  $\mathcal{D}$  is a data distribution supported on  $\mathcal{X} \times [-1, 1]$ . For a hypothesis class  $\mathcal{C}$ , a weak agnostic learner WAL solves the following promise problem: for some accuracy parameter  $\alpha > 0$ , if there exists some  $c \in \mathcal{C}$  such that
$$
\underset {(\mathbf {x}, \mathbf {z}) \sim \mathcal {D}} {\mathbf {E}} [ c (\mathbf {x}) \cdot \mathbf {z} ] \geq \alpha
$$
then  $\mathrm{WAL}_{\alpha}$  returns some  $h:\mathcal{X}\to \mathbb{R}$  such that
$$
\underset {(\mathbf {x}, \mathbf {z}) \sim \mathcal {D}} {\mathbf {E}} [ h (\mathbf {x}) \cdot \mathbf {z} ] \geq \operatorname {p o l y} (\alpha).
$$
For the sake of this presentation, we are informal about the polynomial factor in the guarantee of the weak agnostic learner. The smaller the exponent, the stronger the learning guarantee (i.e., we want  $\mathrm{WAL}_{\alpha}$  to return a hypothesis with correlation with  $\mathbf{z}$  as close to  $\Omega (\alpha)$  as possible). Standard arguments based on VC-dimension demonstrate that weak agnostic learning is statistically efficient.
D.1 MCBoost
The work introducing multicalibration [20] gives a boosting-style algorithm for learning multicalibrated predictors that has come to be known as MCBoost. The algorithm is an iterative procedure: starting with a trivial predictor, the MCBoost searches for a supported value  $v \in \mathrm{Im}(\tilde{p})$  and "subgroup"  $c_v \in \mathcal{C}$  that violate the multicalibration condition. Note that some care has to be taken to ensure that the predictor  $\tilde{p}$  stays supported on finitely many values, and that each of these values maintains significant measure in the data distribution  $\mathcal{D}_{\tilde{p}}$ . In this pseudocode, we ignore these issues; [20] handles them in full detail.
Importantly, the search over  $\mathcal{C}$  for condition (29) can be reduced to weak agnostic learning. Intuitively, we pass WAL samples drawn from the data distribution, but labeled according to  $\mathbf{z} = \mathbf{y} - v$  when  $\tilde{p} (\mathbf{x}) = v$ .
Lemma 3.8. The iteration complexity of MCBoost is directly (inverse quadratically) related to the size of the multicalibration violations we discover in (29). A standard potential argument can be found in [20].
By the termination condition, we can see that  $\tilde{p}$  must actually be  $(\mathcal{C},\alpha)$ -swap multicalibrated. In particular, when the algorithm terminates, then for all  $v\in \mathrm{Im}(\tilde{p})$ , we have that
$$
\max  _ {c _ {v} \in \mathcal {C}} \mathbf {E} \left[ c _ {v} (\mathbf {x}) \cdot (\mathbf {y} - v) \mid \tilde {p} (\mathbf {x}) = v \right] \leq \operatorname {p o l y} (\alpha) \leq \alpha .
$$
Therefore, averaging over  $\mathbf{v} \sim \mathcal{D}_{\tilde{p}}$ , we obtain the guarantee.
Corollary 3.9. By Lemma 3.8, we know that  $\tilde{p}$  returned by MCBoost is  $(\mathcal{C},\alpha)$ -swap multicalibrated. By Theorem 3.3,  $\tilde{p}$  is equivalently a  $(\mathcal{L}_{\mathrm{cvx}},\mathcal{C},\alpha')$ -swap omnipredictor for some polynomially-related  $\alpha'$ . In other words, by Claim 2.4, if we post-process  $\tilde{p}$  according to  $k_{\ell}$  for any nice convex loss function  $\ell$ , we obtain an  $(\ell,\mathcal{C},\varepsilon)$ -swap agnostic learner. Taking  $\alpha = \mathrm{poly}(\varepsilon)$  sufficiently small, we obtain the swap agnostic learning guarantee.
Algorithm 2 MCBoost
Parameters: hypothesis class  $\mathcal{C}$  and  $\alpha >0$
Initialize:  $\tilde{p} (x)\gets 1 / 2$
if  $\exists v\in \mathrm{Im}(\tilde{p})$  and  $c_{v}\in \mathcal{C}$  such that
$$
\mathbf {E} \left[ c _ {v} (\mathbf {x}) \cdot (\mathbf {y} - v) \mid \tilde {p} (\mathbf {x}) = v \right] > \operatorname {p o l y} (\alpha) \tag {29}
$$
update  $\tilde{p} (x)\gets \tilde{p} (x) + \eta c_v(x)\cdot \mathbf{1}[\tilde{p} (x) = v]$
Return:  $\tilde{p}$