SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} R_{W}(h) = \sup_{r:\mathcal{Z}\mapsto \mathbb{R}_{+},\int r(z)dP_{\text{data}}(z) = 1,r\in W}\mathbb{E}_{P_{\text{data}}} [r(z)\ell_{h}(z)] \\ \stackrel {(a)} {=} \sup  _ {r: \mathcal {Z} \mapsto \mathbb {R} _ {+}} \inf  _ {\lambda \geq \mathbf {0}, \tau} \mathbb {E} _ {P _ {\mathrm {d a t a}}} [ r (z) \ell_ {h} (z) ] + \tau \left(1 - \int r (z) d P _ {\mathrm {d a t a}} (z)\right) + \sum_ {i = 1} ^ {m} \lambda_ {i} \left(c _ {i} - \int g _ {i} (r (z)) d P _ {\mathrm {d a t a}} (z)\right) \\ \end{array}
$$
Since the above objective is concave in  $r$  and linear in  $\lambda, \tau$ , we rely on Lagrangian duality to rewrite it as
$$
R_{W}(h) = \inf_{\lambda \geq \mathbf{0},\tau}\sup_{r: \mathcal{Z}\mapsto \mathbb{R}_{+}}L(r,\lambda ,\tau),
$$
where  $L(r,\lambda ,\tau)$  is defined as:
$$
L (r, \lambda , \tau) = \mathbb {E} _ {P _ {\mathrm {d a t a}}} \left[ r (z) \left(\ell_ {h} (z) - \tau\right) - \sum_ {i = 1} ^ {m} \lambda_ {i} g _ {i} (r (z)) \right] + \sum_ {i = 1} ^ {m} \lambda_ {i} c _ {i} + \tau .
$$
Recall the interchangeability theorem:
$$
\inf  _ {r \in \mathcal {H}} \int F (r (z), z) p (z) d z = \int \left(\inf  _ {t \in \mathbb {R}} F (t, z)\right) p (z) d z,
$$
so long as the space  $\mathcal{H}$  is decomposable. Since in our case we are working with the set  $L_{1}(\mathcal{Z},P) = \{r:\mathcal{Z}\mapsto \mathbb{R}:\int r(z)p(z)dz = 1\}$ , which is decomposable, we can apply the interchangeability theorem to get:
$$
\begin{array}{l} \sup  _ {r: \mathcal {Z} \mapsto \mathbb {R} _ {+}} \mathbb {E} _ {P _ {\mathrm {d a t a}}} \left[ r (z) \left(\ell_ {h} (z) - \tau\right) - \sum_ {i = 1} ^ {m} \lambda_ {i} g _ {i} (r (z)) \right] = \mathbb {E} _ {P _ {\mathrm {d a t a}}} \sup  _ {t \geq 0} \left[ t \left(\ell_ {h} (z) - \tau\right) - \sum_ {i = 1} ^ {m} \lambda_ {i} g _ {i} (t) \right] \\ = \mathbb {E} _ {P _ {\text {d a t a}}} G _ {\lambda} ^ {*} (\ell_ {h} (z) - \tau), \\ \end{array}
$$
where  $G_{\lambda}(t) = \sum_{i=1}^{m} \lambda_i g_i(t)$ , and  $G_{\lambda}^*$  is its Fenchel conjugate. so that:
$$
R (\ell_ {h}) = \inf  _ {\lambda \geq \mathbf {0}, \tau} \sum_ {i = 1} ^ {m} \lambda_ {i} c _ {i} + \tau + \mathbb {E} _ {P _ {\mathrm {d a t a}}} G _ {\lambda} ^ {*} (\ell_ {h} (z) - \tau).
$$
We have the following properties of the Fenchel conjugate  $G_{\lambda}^{*}(t)$ . These follow from the properties of Fenchel conjugates described in Rockafellar [1970].
Lemma 20 Consider the setting of Proposition 19. The Fenchel conjugate  $G_{\lambda}^{*}$  is convex, differentiable and an increasing function that satisfies
$$
\frac {d G _ {\lambda} ^ {*} (x)}{d x} \geq 0, \forall x \in \mathbb {R}.
$$
Proof of Proposition 9 For the sake of clarity, we first state Proposition 9 below.
Proposition 21 (Bayes optimal classifier) Consider the problem of binary classification where  $\mathcal{Y} = \{-1, + 1\}$ . Suppose  $\ell (h(x),y) = \phi (yh(x))$  for some  $\phi :\mathbb{R}\to [0,\infty)$  which is either the  $0 / 1$  loss, or a convex loss function that is differentiable at 0 with  $\phi^{\prime}(0) < 0$ . Suppose the uncertainty set  $W$  is as specified in Equation (5). Moreover, suppose  $\{g_i\}_{i = 1\dots m}$  are convex and differentiable functions. Then, the vanilla Bayes optimal classifier is also a RAI-Bayes optimal classifier.
Proof. Following Proposition 19 it is easy to see that the RAI Bayes optimal classifier is the minimizer of the following problem
$$
\inf_{h}\inf_{\lambda \geq 0,\tau}\sum_{i = 1}^{m}\lambda_{i}c_{i} + \tau +\mathbb{E}_{P_{\mathrm{data}}}G^{*}_{\lambda}(\phi (yh(x)) - \tau),
$$
where the minimization over  $h$  is over the set of all classifiers. For any fixed  $(\lambda, \tau)$ , we now show that the classifier  $h$  that minimizes the above optimization problem is a vanilla Bayes optimal classifier. First note that the above optimization problem, for a fixed  $(\lambda, \tau)$ , can be rewritten as
$$
\inf  _ {h} \mathbb {E} _ {P _ {\mathrm {d a t a}}} G _ {\lambda} ^ {*} (\phi (y h (x)) - \tau).
$$
Using the interchangeability theorem, we can further rewrite this as
$$
\mathbb {E} _ {P _ {\mathrm {d a t a}} ^ {x}} \left[ \inf  _ {u \in \{- 1, + 1 \}} \mathbb {E} _ {P _ {\mathrm {d a t a}} (\cdot | x)} \left[ G _ {\lambda} ^ {*} (\phi (u y) - \tau) \right] \mid x \right].
$$
Here,  $P_{\mathrm{data}}^x$  is the marginal distribution of  $P_{\mathrm{data}}$  over  $x$ , and  $P_{\mathrm{data}}(\cdot | x)$  is the distribution of  $y$  conditioned on  $x$ .
Now suppose  $\phi$  is the  $0/1$  loss
$$
\phi (x) = \left\{ \begin{array}{l l} 0, & \text {i f} x > 0 \\ 1, & \text {o t h e r w i s e} \end{array} \right..
$$
Recall,  $G_{\lambda}^{*}$  is an increasing function (see Lemma 20). So  $G_{\lambda}^{*}(-\tau) \leq G_{\lambda}^{*}(1 - \tau)$ . Using this, it is easy to see that for any  $x$ , the following is a minimizer of  $\inf_{u \in \{-1,1\}} \mathbb{E}_{P_{\mathrm{data}}(\cdot | x)}[G_{\lambda}^{*}(\phi(uy) - \tau)]$
$$
u ^ {*} = \left\{ \begin{array}{l l} 1, & \text {i f} P _ {\mathrm {d a t a}} (y = 1 | x) \geq \frac {1}{2} \\ - 1, & \text {o t h e r w i s e} \end{array} \right..
$$
This shows that the vanilla Bayes optimal classifier is a minimizer of the population RAI risk.
Now suppose  $\phi : \mathbb{R} \to [0, \infty)$  is convex, differentiable at 0 with  $\phi'(0) < 0$ . Moreover, suppose  $h: \mathcal{X} \to \mathbb{R}$  is a real valued classifier. In this case, the RAI Bayes optimal classifier is a minimizer of the following objective
$$
\mathbb {E} _ {P _ {\text {d a t a}} ^ {x}} \left[ \inf  _ {u \in \mathbb {R}} \mathbb {E} _ {P _ {\text {d a t a}} (\cdot | x)} \left[ G _ {\lambda} ^ {*} (\phi (u y) - \tau) \right] \mid x \right].
$$
Let  $\iota(x) = G_{\lambda}^{*}(\phi(x) - \tau)$ . It is easy to see that  $\iota(x)$  is convex. This is because  $\iota'(x) = (G_{\lambda}^{*})'(\phi(x) - \tau)\phi'(x)$  is an increasing function; this follows from the fact that  $G_{\lambda}^{*}$  is convex with non-negative gradients. Moreover,  $\iota'(0) = (G_{\lambda}^{*})'(\phi(0) - \tau)\phi'(0) \leq 0$ . Then, Bartlett et al. [2006], Tewari and Bartlett [2007] show that for any  $x$ , the following  $u^{*}$  is a minimizer of the inner optimizatin problem:  $u^{*} > 0$  if  $P_{\mathrm{data}}(y = 1|x) \geq \frac{1}{2}$ ,  $u^{*} < 0$  otherwise. This shows that vanilla Bayes optimal classifier is minimizer of the population RAI risk.
G.2 Generalization Guarantees
G.2.1 Proof of Proposition 11
Proposition 11 directly follows from Proposition 19.
G.2.2 Proof of Theorem 12
We first present a key concentration result we use in the proof.
Lemma 22 (Hoeffding bound [Wainwright, 2019]) Suppose that the random variables  $\{X_i\}_{i=1}^n$  are independent with mean  $\mu_i$ , and bounded between  $[a, b]$ . Then for any  $t \geq 0$ , we have
$$
\mathbb {P} \left(\left| \sum_ {i = 1} ^ {n} X _ {i} - \mu_ {i} \right| \geq t\right) \leq 2 \exp \left(- \frac {2 t ^ {2}}{n (b - a) ^ {2}}\right).
$$
We now proceed to the proof of the Theorem. Following Proposition 11, we know that the population and empirical RAI risk of a classifier  $h$  can be written as
$$
R _ {W} (h) = \inf  _ {\lambda \geq \mathbf {0}, \tau} \sum_ {i = 1} ^ {m} \lambda_ {i} c _ {i} + \tau + \mathbb {E} _ {P _ {\mathrm {d a t a}}} G _ {\lambda} ^ {*} (\ell_ {h} (z) - \tau)
$$
$$
\widehat {R} _ {W _ {n}} (h) = \inf  _ {\lambda \geq \mathbf {0}, \tau} \sum_ {i = 1} ^ {m} \lambda_ {i} c _ {i} + \tau + \mathbb {E} _ {\widehat {P} _ {\mathrm {d a t a}}} G _ {\lambda} ^ {*} (\ell_ {h} (z) - \tau)
$$
Our goal here is to bound the following quantity for any given  $h$ :
$$
\left|R_{W}(h) - \widehat{R}_{W_{n}}(h)\right|\leq \sup_{(\lambda ,\tau)\in \mathcal{E},\lambda :\min_{i}\lambda_{i}\geq \Lambda}\left|\mathbb{E}_{P_{\mathrm{data}}}G^{*}_{\lambda}(\ell_{h}(z) - \tau) - \mathbb{E}_{\widehat{P}_{\mathrm{data}}}G^{*}_{\lambda}(\ell_{h}(z) - \tau)\right|.
$$
The rest of the proof focuses on bounding the RHS of the above equation. The overall idea is to first provide a high probability bound of the RHS for any given  $\lambda, \tau$ . Next, we take a union bound over all feasible  $(\lambda, \tau)$ 's by constructing an appropriate  $\epsilon$ -net.
Fixed  $\lambda, \tau$ . Observe that  $G_{\lambda}^{*}(\ell_{h}(z) - \tau)$  is bounded and satisfies
$$
G _ {\lambda} ^ {*} (- \tau) \leq G _ {\lambda} ^ {*} (\ell_ {h} (z) - \tau) \leq G _ {\lambda} ^ {*} (B - \tau).
$$
This follows from the fact that  $G_{\lambda}^{*}$  is an increasing function (see Proposition 20), and  $\ell_h$  is bounded between 0 and  $B$ . From Heoeffding bound we know that for any fixed  $h \in H$ , the following holds with probability at least  $1 - 2e^{-t}$
$$
\left| \mathbb {E} _ {P _ {\mathrm {d a t a}}} G _ {\lambda} ^ {*} (\ell_ {h} (z) - \tau) - \mathbb {E} _ {\widehat {P} _ {\mathrm {d a t a}}} G _ {\lambda} ^ {*} (\ell_ {h} (z) - \tau) \right| \leq G \sqrt {\frac {t}{n}}.
$$
Union bound over  $\lambda, \tau$ . Define set  $\mathcal{E}'$  as
$$
\mathcal {E} ^ {\prime} := \left\{\left(\lambda , \tau\right): \lambda \geq 0, \min  _ {i} \lambda_ {i} \geq \Lambda \right\} \cap \mathcal {E}. \tag {9}
$$
Let  $N(\mathcal{E}', \epsilon, \| \cdot \|_2)$  be the  $\epsilon$ -net over  $\mathcal{E}'$  in  $\| \cdot \|_2$  norm. It is well known that there exists such a set whose size is upper bounded by [Wainwright, 2019]
$$
| N (\mathcal {E} ^ {\prime}, \epsilon , \| \cdot \| _ {2}) | \leq O \left(\frac {\bar {\Lambda} + T}{\epsilon}\right) ^ {m + 1}
$$
. For any  $(\lambda ,\tau)\in \mathcal{E}^{\prime}$  , let  $(\lambda_{\epsilon},\tau_{\epsilon})$  be an element in  $N(\mathcal{E}',\epsilon ,\parallel \cdot \parallel_2)$  that is  $\epsilon$  -close to  $(\lambda ,\tau)$  . Now consider the following
$$
\begin{array}{l} \sup _ {(\lambda , \tau) \in \mathcal {E} ^ {\prime}} \left| \mathbb {E} _ {P _ {\mathrm {d a t a}}} G _ {\lambda} ^ {*} (\ell_ {h} (z) - \tau) - \mathbb {E} _ {\widehat {P} _ {\mathrm {d a t a}}} G _ {\lambda} ^ {*} (\ell_ {h} (z) - \tau) \right| \\ \leq \sup  _ {(\lambda , \tau) \in N (\mathcal {E} ^ {\prime}, \epsilon , \| \cdot \| _ {2})} \left| \mathbb {E} _ {P _ {\text {d a t a}}} G _ {\lambda} ^ {*} (\ell_ {h} (z) - \tau) - \mathbb {E} _ {\widehat {P} _ {\text {d a t a}}} G _ {\lambda} ^ {*} (\ell_ {h} (z) - \tau) \right| \\ + \sup  _ {(\lambda , \tau) \in \mathcal {E} ^ {\prime}} \left| \mathbb {E} _ {P _ {\mathrm {d a t a}}} G _ {\lambda} ^ {*} (\ell_ {h} (z) - \tau) - \mathbb {E} _ {P _ {\mathrm {d a t a}}} G _ {\lambda \epsilon} ^ {*} (\ell_ {h} (z) - \tau_ {\epsilon}) \right| \\ + \sup  _ {(\lambda , \tau) \in \mathcal {E} ^ {\prime}} \left| \mathbb {E} _ {\widehat {P} _ {\mathrm {d a t a}}} G _ {\lambda} ^ {*} (\ell_ {h} (z) - \tau) - \mathbb {E} _ {\widehat {P} _ {\mathrm {d a t a}}} G _ {\lambda \epsilon} ^ {*} (\ell_ {h} (z) - \tau_ {\epsilon}) \right| \\ \end{array}
$$
Since  $G_{\lambda}^{*}$  is  $L$ -Lipschitz, the last two terms in the RHS above can be upper bounded by  $L\epsilon$ . Substituting this in the above equation, we get
$$
\begin{array}{l} \sup  _ {(\lambda , \tau) \in \mathcal {E} ^ {\prime}} \left| \mathbb {E} _ {P _ {\text {d a t a}}} G _ {\lambda} ^ {*} (\ell_ {h} (z) - \tau) - \mathbb {E} _ {\widehat {P} _ {\text {d a t a}}} G _ {\lambda} ^ {*} (\ell_ {h} (z) - \tau) \right| \\ \leq \sup  _ {(\lambda , \tau) \in N (\mathcal {E} ^ {\prime}, \epsilon , \| \cdot \| _ {2})} \left| \mathbb {E} _ {P _ {\mathrm {d a t a}}} G _ {\lambda} ^ {*} (\ell_ {h} (z) - \tau) - \mathbb {E} _ {\widehat {P} _ {\mathrm {d a t a}}} G _ {\lambda} ^ {*} (\ell_ {h} (z) - \tau) \right| + 2 L \epsilon \\ \stackrel {(a)} {\leq} G \sqrt {\frac {t}{n}} + 2 L \epsilon , \\ \end{array}
$$
where  $(a)$  follows from Equation (9), and holds with probability at least  $1 - \left(\frac{\bar{\Lambda} + T}{\epsilon}\right)^{m + 1}e^{-t}$ . Choosing  $\epsilon = \frac{G}{L}\sqrt{\frac{t}{n}}$ , we get the desired result.
G.3 Proof of Corollary 13
The proof follows from a standard covering number argument. For any  $h \in H$ , let  $h_\epsilon$  be the point in the  $\epsilon$ -net that is closest to  $h$ . Then we have
$$
\begin{array}{l} \sup  _ {h \in H} | R _ {W} (h) - \widehat {R} _ {W _ {n}} (h) | \leq \sup  _ {h \in N (H, \epsilon_ {n}, \| \cdot \| _ {L ^ {\infty} (\mathcal {Z})})} | R _ {W} (h) - \widehat {R} _ {W _ {n}} (h) | \\ + \sup  _ {h \in H} | R _ {W} (h) - R _ {W} (h _ {\epsilon}) | + \sup  _ {h \in H} | \widehat {R} _ {W _ {n}} (h) - \widehat {R} _ {W _ {n}} (h _ {\epsilon}) | \\ \end{array}
$$
Observe that the last two terms above are bounded by  $\epsilon_{n}$ . Also observe that the first term in the RHS can be upper bounded by  $10n^{-1/2} G\left(\sqrt{t + m \log(nL)}\right)$  with probability at least  $1 - 2N(H, \epsilon_{n}, \| \cdot \|_{L^{\infty}(\mathcal{Z})})e^{-t}$ . Combining these two and substituting the value of  $\epsilon_{n}$  gives us the required result.
H Algorithms: Further Discussion
H.1 Equivalence Conditions for RAI Algorithms (1 and 2)
Proposition 23 Assume that we set  $\alpha_{t} = \frac{1}{t}$  and perform coordinate descent update in Algorithm 2 (FW) i.e.  $G^{t} = \operatorname*{argmin}_{h\in H}\left\langle h,\nabla_{Q}L_{\eta}(Q^{t - 1})\right\rangle$  then the update is equivalent to the update given by Algorithm 1 with  $\eta^t = \eta t$
Proof. From Equation 3, and using the fact that  $L_{\eta}(Q)$  can be written as a fenchel conjugate, we know that
$$
\begin{array}{l} \nabla_ {Q} L _ {\eta} (Q ^ {t - 1}) = \operatorname * {a r g m a x} _ {w \in W _ {n}} \mathbb {E} _ {h \sim Q ^ {t - 1}} \mathbb {E} _ {w} \ell (h (x), y) + \eta \operatorname {R e g} (w) \\ = \operatorname * {a r g m a x} _ {w \in W _ {n}} \sum_ {s = 1} ^ {t - 1} \mathbb {E} _ {w} \ell \left(h ^ {s} (x), y\right) + \eta t \operatorname {R e g} (w) \quad \left(\text {a s} \alpha^ {t} = \frac {1}{t}\right) \\ \end{array}
$$
This matches Equation 2 with  $\eta^{t - 1} = \eta t$ . Moreover,
$$
G ^ {t} = \underset {h \in H} {\operatorname {a r g m i n}} \mathbb {E} _ {h \in H} \mathbb {E} _ {w ^ {t}} l (h (x), y) \quad \text {w h e r e} \quad w ^ {t} = \nabla_ {Q} L _ {\eta} (Q ^ {t - 1})
$$
This corresponds to the update for  $h^t$  in Algorithm 1. Thus, we have our equivalence.
H.2 Weak Learning Conditions
For the well-known scenario of binary classification and zero-one loss, we recover the quasi-AdaBoost weak learning condition:
Proposition 24 Consider the scenario of binary classification and  $l$  as the zero-one loss. If the  $H$ -player only plays an approximate best response strategy i.e.  $h^t$  satisfies  $\mathbb{E}_{w^t}\ell (h^t (x),y)\leq 1 / 2 - \gamma$  for some  $\gamma >0$ , then  $\widehat{R}_{W_n}(h_{\text{det:}QT}) = 0$  for  $T > T_0$  for some large enough  $T_0$ .
Proof. Since the D-player uses regret optimal strategy, we have that:
$$
\frac {1}{T} \sum_ {t = 1} ^ {T} \mathbb {E} _ {w ^ {t}} \ell (h ^ {t} (x), y) \geq \max _ {w \in W _ {n}} \frac {1}{T} \sum_ {t = 1} ^ {T} \mathbb {E} _ {w} \ell (h ^ {t} (x), y) - \epsilon_ {T},
$$
while from the approximate-BR condition we have that:
$$
\frac {1}{T} \sum_ {t = 1} ^ {T} \mathbb {E} _ {w ^ {t}} \ell (h ^ {t} (x), y) \leq 1 / 2 - \gamma ,
$$
so that we have:
$$
\max  _ {w \in W _ {n}} \frac {1}{T} \sum_ {t = 1} ^ {T} \mathbb {E} _ {w} \ell \left(h ^ {t} (x), y\right) \leq 1 / 2 - \gamma + \epsilon_ {T},
$$
so that for  $T > T_0$  large enough so that  $\epsilon_T < \gamma / 2$ , we have that:
$$
\max  _ {w \in W _ {n}} \frac {1}{T} \sum_ {t = 1} ^ {T} \mathbb {E} _ {w} \ell \left(h ^ {t} (x), y\right) <   1 / 2 - \gamma / 2.
$$
As  $Q^T$  assigns mass  $1 / T$  to each of  $\{h^t\}_{t = 1}^T$ , we have:
$$
\begin{array}{l} \widehat {R} _ {W _ {n}} \left(h _ {\text {r a n d}; Q ^ {T}}\right) = \max  _ {w \in W _ {n}} \mathbb {E} _ {Q ^ {T}} \mathbb {E} _ {w} \ell \left(h ^ {t} (x), y\right) <   1 / 2 - \gamma / 2 \\ \Rightarrow \widehat {R} _ {W _ {n}} \left(h _ {\text {d e t}: Q ^ {T}}\right) = 0 \quad (\text {f r o m P r o p o s i t i o n 2}) \\ \end{array}
$$
Hence, we see that  $h_{det;Q^T}$  incurs zero error.
For the general setting, we have a slightly stronger weak learning condition, which follows from the analysis of Frank-Wolfe update [Freund and Grigas, 2014, Jaggi, 2013].
Proposition 25 Consider Algorithm 2 with the FW update and Reg(.) as a 1-strongly concave regularize w.r.t.  $\| .\| _1$  . If  $G^{t}$  satisfies  $G^{t}\leq \min_{Q}\left\langle Q,\nabla_{Q}L_{\eta}(Q^{t - 1})\right\rangle +\delta_{t}$  , where  $\{\delta_k\}$  is the sequence of approximation errors with  $\delta_t\geq 0$  , then:
1. If  $\delta_t \leq \frac{\epsilon}{\eta(t + 2)}$  i.e. decaying errors, then  $L_{\eta}(Q^T) \leq \min_Q L_{\eta}(Q) + \frac{2(1 + \epsilon)}{\eta(T + 2)}$
2. If  $\delta_t \leq \frac{\epsilon}{\eta}$  i.e. constant errors, then  $L_{\eta}(Q^T) \leq \min_Q L_{\eta}(Q) + \frac{2}{\eta(T+2)} + \frac{\epsilon}{\eta}$ .
Proof. Note that we are trying to minimize the objective  $L_{\eta}(Q)$  w.r.t  $Q$  by the FW update. Using properties of Fenchel conjugates, it is well known that  $L_{\eta}(Q)$  is  $\frac{1}{\eta}$  smooth w.r.t.  $\| . \|_1$ . Also, the diameter of the simplex  $\Delta_H$  w.r.t.  $\| . \|_1$  is  $\leq 1$ .
1. By [Jaggi, 2013] (Lemma 7, Theorem 1), we have  $C_f \leq \frac{1}{\eta}$ , and thus, we have:
$$
L _ {\eta} (Q ^ {T}) - \min  _ {Q} L _ {\eta} (Q) \leq \frac {2 (1 + \epsilon)}{\eta (T + 2)}
$$
2. By [Freund and Grigas, 2014] (Theorem 5.1), in case of approximation errors, the FW/optimality gap converges as before along with a convex combination of errors at each time step i.e. if we are able to solve the linear optimization problem within constant error  $\frac{\epsilon}{\eta}$ , then these errors do not accumulate. Moreover, the convex combination can be bound by the maximum error possible and we get,
$$
L _ {\eta} (Q ^ {T}) - \min _ {Q} L _ {\eta} (Q) \leq \frac {2}{\eta (T + 2)} + \frac {\epsilon}{\eta}
$$
RAI games constitute an optimization paradigm that goes beyond traditional approaches such as distributionally robust optimization, fairness, and worst-case performance. We have seen that for specific uncertainty sets  $W$ , RAI Games optimize over well-established robust optimization objectives. As such, the purpose of our experiments is to demonstrate the practicality and generality of our proposed strategies, rather than establishing state-of-the-art over baselines. Given a large number of possible  $W$ , we do not attempt an exhaustive empirical analysis. Instead, we underscore the plug-and-play nature of RAI Games.
I.1 Setup
Subpopulation Shift A prevalent scenario in machine learning involves subpopulation shift, necessitating a model that performs effectively on the data distribution of each subpopulation (or domain). We explore the following variations of this setting:
- Domain Oblivious (DO). Recent work [Hashimoto et al., 2018], [Lahoti et al., 2020], [Zhai et al., 2021a] studies the domain-oblivious setting, where the training algorithm lacks knowledge of the domain definition. In this case, approaches like  $\alpha$ -CVaR and  $\chi^2$ -DRO aim to maximize performance over a general notion of the worst-off subpopulation.
- Domain Aware (DA). Several prior works [Sagawa et al., 2019] have investigated the domain-aware setting, in which all domain definitions and memberships are known during training.
- Partially Domain-Aware (PDA). More realistically, in real-world applications, there usually exist multiple domain definitions. Moreover, some of these domain definitions may be known during training, while others remain unknown. The model must then perform well on instances from all domains, regardless of whether their definition is known. This setting is challenging as it necessitates the model to learn both domain-invariant and domain-specific features and to generalize well to new instances from unknown domains.
I.2 Further Details for Section 7
Base Learners We use linear classifiers, WRN-28-1, and WRN-28-5 [Zagoruyko and Komodakis, 2016] as base classifiers for COMPAS, CIFAR-10 and CIFAR-100 respectively. To get a sense of performance improvements, we also benchmark performance with larger models, namely a three-hidden-layer neural network for COMPAS and WRN-34-10 for CIFAR-10/100.
Proposed Methods This paper introduces two categories of algorithms. We elect not to present results for Algorithm 1, which we notice has similar performance to Algorithm 2. Conversely, we provide in-depth experimental analyses for both updates of Algorithm 2, which warrant some special attention due to due to their relation to AdaBoost. In this section, we refer to the FW and Gen-AdaBoost updates as RAI-FW and RAI-GA, respectively. Our implemented versions incorporate a few alterations: 1. We track the un-regularized objective value from Equation 1 for the validation set. If it increases at any round  $t$ , we increase the regularization factor  $\eta$  by a fixed multiple (specifically, 2). We notice that it leads to better generalization performance over the test set. 2. The same un-regularized objective w.r.t normalized  $Q^t$  is also used to perform a line search for the step size  $\alpha$ . For the FW update, our search space is a ball around  $\frac{1}{t}$  at round  $t$ , while for the GA update, we search within the range  $(0, 1)$ .
Training. We use SGD with momentum  $= 0.9$  for optimization. We first warm up the model with some predefined epochs of ERM (3 for COMPAS and 20 for CIFAR-10/100), followed by a maximum of  $T = 5$  base models trained from the warm-up model with sample weights provided by our algorithms. Each base model is trained for 500 iterations on COMPAS and 2000 iterations on CIFAR-10/100. Each experiment is run three times with different random seeds. For evaluation, we report the averaged expected and worst-case test loss from Equation 1.
Datasets We conduct our experiments on three real-world datasets:
- COMPAS [Angwin et al., 2016] pertains to recidivism prediction, with the target being whether an individual will re-offend within two years. This dataset is extensively used in fairness research. We randomly sample  $70\%$  of the instances for the training data (with a fixed random seed), and the remainder is used for validation/testing.
- CIFAR-10 and CIFAR-100 are widely used image datasets. For CIFAR-10, we consider two settings: the original set and an imbalanced split [Jin et al., 2021, Qi et al., 2021]. In the imbalanced split, we make worst-case performance more challenging by randomly sampling each category at different ratios. To be precise, we sample the  $ith$  class with a sampling ratio  $\rho_{i}$  where  $\rho = \{0.804, 0.543, 0.997, 0.593, 0.390, 0.285, 0.959, 0.806, 0.967, 0.660\}$ . For these datasets, we use the standard training and testing splits, reserving  $10\%$  of the training samples as validation data.
Hyperparameters For COMPAS, we warm up for 3 epochs and then train every base classifier for 500 iterations. For CIFAR-10 and CIFAR-100, we warm up the models for 20 epochs and train base classifiers for 2000 iterations. The mini-batch size is set to 128. It should be noted that the primary aim of our experiments is not hyperparameter tuning. The experiments in this paper are designed to demonstrate use cases and compare different algorithms. Hence, while we maintain consistency of hyperparameters across all experiments, we do not extensively tune them for optimal performance.
I.3 Interesting Observation: Boosting Robust Learners
Table 4: Mean and worst-case expected loss for RAI-FW + robust optimization algorithms.
Algorithm	COMPAS	CIFAR-10 (Imbalanced)	CIFAR10	CIFAR100
Average	Worst Group	Average	Worst Class	Average	Worst Class	Average	Worst Class
SGD (χ2)	32.0	33.7	13.3	31.7	11.3	24.7	27.4	65.9
RAI-FW + SGD (χ2)	30.9	32.2	13.6	31.0	11.2	23.8	27.6	63.8
Online GDRO	31.7	32.2	13.1	26.6	11.2	21.7	27.3	57.0
RAI-FW + Online GDRO	31.6	33.3	12.9	24.4	11.4	19.5	27.8	51.2
Boosting Robust Base Learners We conclude our results with one interesting observation. Until now, we have been comparing our ensembles with deterministic models. As such, we acknowledge