SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Abraham Wald. Generalization of a theorem by v. neumann concerning zero sum two person games. Annals of Mathematics, 46:281, 1945.
Stephen Simons. Minimax theorems and their proofs. 1995.
TES Raghavan. Zero-sum two-person games. Handbook of game theory with economic applications, 2:735-768, 1994.
Andrew Cotter, Maya Gupta, and Harikrishna Narasimhan. On making stochastic classifiers deterministic. Advances in Neural Information Processing Systems, 32, 2019.
Jimmy Wu, Yatong Chen, and Yang Liu. Metric-fair classifier derandomization. In International Conference on Machine Learning, pages 23999-24016. PMLR, 2022.
H Brendan McMahan. A survey of algorithms and analysis for adaptive online learning. The Journal of Machine Learning Research, 18(1):3117-3166, 2017.
Stephen Boyd and Lieven Vandenberghe. Convex Optimization. Cambridge University Press, 2004.
R Tyrrell Rockafellar. Convex analysis. Number 28. Princeton university press, 1970.
Peter L Bartlett, Michael I Jordan, and Jon D McAuliffe. Convexity, classification, and risk bounds. Journal of the American Statistical Association, 101(473):138-156, 2006.
Ambuj Tewari and Peter L Bartlett. On the consistency of multiclass classification methods. Journal of Machine Learning Research, 8(5), 2007.
Martin J Wainwright. High-dimensional statistics: A non-asymptotic viewpoint, volume 48. Cambridge university press, 2019.
Robert M. Freund and Paul Grigas. New analysis and results for the frank-wolfe method. Mathematical Programming, 155:199-230, 2014. doi: 10.1007/s10107-014-0841-6. URL http://link.springer.com/article/10.1007/s10107-014-0841-6/fulltext.html.
Sergey Zagoruyko and N. Komodakis. Wide residual networks. British Machine Vision Conference, 2016. doi: 10.5244/C.30.87.
A Broader Impact 18
B Limitations 18
C Terminology and Notation 19
C.1 Terminology 19
C.2 Notation 19
D Background 19
D.1 Two Player Zero-sum Games 19
D.2 Online Learning 20
D.3 General Minimax Theorems (Proof of Proposition 1) 21
E Ensemble RAI Games 21
E.1 Alternative Definitions: Deterministic Ensembles 21
E.2 Proofs 21
F Algorithms 22
F.1 Convergence Rates 22
F.2 Closed Form Updates for different uncertainty sets 23
F.3 Proof of Corollary 8 24
G Generalization 24
G.1 Population Risk 24
G.2 Generalization Guarantees 26
G.3 Proof of Corollary 13 27
H Algorithms: Further Discussion 28
H.1 Equivalence Conditions for RAI Algorithms (1 and 2) 28
H.2 Weak Learning Conditions 28
I Experiments 29
I.1 Setup 29
I.2 Further Details for Section 7 30
I.3 Interesting Observation: Boosting Robust Learners 30
I.4 Synthetic Datasets 31
Responsible AI has become an important topic as ML.AI systems increase in scale, and are being deployed in a variety of scenarios. Models not optimized for responsible facets can have disastrous consequences [Kalra and Paddock, 2016, Angwin et al., 2016, Fuster et al., 2018].
Our research promotes the principles of Responsible AI (RAI), encompassing ethics, fairness, and safety considerations. By providing a framework to consider these simultaneously, this research could help prevent scenarios where optimizing for one aspect unintentionally compromises another [Ma et al., 2022], mitigating the risk of creating biases or vulnerabilities in AI systems.
We also address the fragmentation in recent work around Responsible AI. The 'plug-and-play' nature of our approach allows for the easy adaptation of algorithms for different Responsible AI considerations. This adaptability could lead to more practical and user-friendly tools for implementing RAI across different applications.
Numerous ML models like Large language models, (GPT-3, GPT-4), are rapidly transforming numerous domains, including natural language processing, data analysis, content creation, and more. Given their remarkable ability to generate human-like text, they can be used to construct narratives, answer queries, or even automate customer service. However, with such capabilities come significant responsibilities, as these models can inadvertently perpetuate biases, misinformation, or harmful content if not correctly regulated. Therefore, ensuring the responsible behavior of these models is crucial [Chan, 2023, Bender et al., 2021]. Our research presents a general framework that can be pivotal in the responsible design of such large language models.
We now identify some limitations of our current work, along with corresponding future directions.
- To more concretely establish the empirical superiority of our optimization techniques, more experiments involving large over-parametrized models need to be conducted.
- The proposed generalization bounds are not tight for all risks. A more careful analysis would be needed for such a generalization.
- Our framework only handles uncertainty sets that are supported on the training data. It'd be interesting to generalize our framework further to support other uncertainty sets based on Wasserstein divergences that are not necessarily supported on the training data.
- Finally, the presence of outliers can often destabilize training of large models[Zhai et al., 2021b]. However, our current framework assumes the training data is un-corrupted. In future, we aim to extend our framework to support corruptions in the training data.
- We note that our framework can also be extended to the problem of adversarial test-time robustness where there is an adversary corrupting the inputs sent to the model during inference. Let  $\mathcal{A}(x)$  be the set of perturbations that the adversary can add to input  $x$ . The uncertainty set in this case contains distributions supported on  $\{(x', y') : \exists (x, y) \in \widehat{P}_{\mathrm{data}}$  such that  $x' \in x + \mathcal{A}(x), y' = y\}$ .
Our framework primarily covers RAI aspects which can written as minmax problems. Here we provide some other notions of RAI which our framework does not directly cover.
Fairness: Various notions of fairness have been studied by the ML community. While our framework captures minimax group fairness, it doesn't capture other notions of group fairness such as Demographic Parity [Louizos et al., 2015], Equality of Odds, Equality of Opportunity [Hardt et al., 2016b]. Our framework doesn't capture individual fairness notions.
Robustness: While our framework covers group robustness and certain forms of distributional robustness, it doesn't cover robustness to Wasserstein perturbations and other (un)structured distribution shifts often studied in the domain generalization community [Wang et al., 2018, Addepalli et al., 2022, Cha et al., 2022].
C Terminology and Notation
C.1 Terminology
Strongly Convex Sets A set  $A$  is  $\lambda$ -strongly convex w.r.t a norm  $\|\cdot\|$ , if, for any  $x, y \in A$ ,  $\gamma \in [0,1]$ , the  $\|\cdot\|$  norm ball with origin  $\gamma x + (1 - \gamma)y$  and radius  $\gamma(1 - \gamma)\lambda \|x - y\|^2/2$  lies in  $A$ .
$f$ -divergence For any two probability distributions  $P, Q$ ,  $f$ -divergence between  $P$  and  $Q$  is defined as  $D_{f}(Q||P) = \mathbb{E}_{P}[f(dQ / dP)]$ . Here  $f: \mathbb{R}_{+} \to \mathbb{R}$  is a convex function such that  $f(1) = 0$ .
C.2 Notation
Table 3: Notation
Symbol	Description
X	Input Random Variable
X	Input Domain
Y	Output Random Variable
Y	Output Domain
Z	Sample Random Variable (X,Y)
Z	Sample Domain X × Y
S	Sample Set
Pdata	Data Generating Distribution
Pdata	Empirical Distribution (Uniform) over S
H	Set of Hypothesis
Q	Distribution over Hypothesis from H
h	Any given hypothesis
hrand;Q	Randomized Ensemble given by Q
hdet;Q	De-randomized/Deterministic Classifier corresponding to hrand;Q
l	Loss Function
R(h)	Empirical Risk of h
Wn	Set of allowed sample weights (aka Uncertainty Set)
RWN(h)	Empirical RAI Risk of h
RW(h)	Population RAI Risk of h
Rrand;WN(Q)	Randomized Ensemble RAI Risk
KL(p||q)	KL-divergence Metric between p and q
Gi	Subpopulation/Domain i
Reg	Any given regularizer function
D Background
D.1 Two Player Zero-sum Games
Consider the following game between two players. One so-called "row player" playing actions  $h \in H$ , and the other "column player" playing actions  $z \in Z$ . Suppose that when the two players play actions  $h, z$  respectively, the row player incurs a loss of  $l(h, z) \in \mathbb{R}$ , while the column player incurs a loss of  $-l(h, z)$ . The sum of the losses for the two players can be seen to be equal to zero so such a game is known as a two-player zero-sum game. It is common in such settings to refer to the gain  $l(h, z)$  of the column player, rather than its loss of  $-l(h, z)$ . Both players try to maximize their gain/minimize their loss.
It is common in game theory to consider a linearized game in the space of probability measures, which is in general better behaved. To set up some notation, for any probability distributions  $P_{h}$  over  $H$ , and  $P_{z}$  over  $Z$ , define:
$$
l \left(P _ {h}, P _ {z}\right) = \mathbb {E} _ {P _ {h}, P _ {z}} l (h, z)
$$
Nash Equilibrium A Nash Equilibrium (NE) is a stable state of a game where no player can gain by unilaterally changing their strategy while the other players keep theirs unchanged. In a two-player zero-sum game, a Nash Equilibrium is a pair of mixed strategies  $(h^{*},z^{*})$  satisfying
$$
\sup  _ {z \in Z} l \left(h ^ {*}, z\right) \leq l \left(h ^ {*}, z ^ {*}\right) \leq \inf  _ {h \in H} l \left(h, z ^ {*}\right)
$$
Note that whenever a pure strategy NE exists, the minimax and maximin values of the game are equal to each other:
$$
\inf_{h\in H}\sup_{z\in Z}l(h,z) = l(h^{*},z^{*}) = \sup_{z\in Z}\inf_{h\in H}l(h,z)
$$
What often exists is a mixed strategy NE, which is precisely a pure strategy NE of the linearized game. That is,  $(P_h^*, P_z^*)$  is called a mixed strategy NE of the zero-sum game, if
$$
\sup  _ {P _ {z} \in \mathcal {P} _ {Z}} l \left(P _ {h} ^ {*}, P _ {z}\right) \leq l \left(P _ {h} ^ {*}, P _ {z} ^ {*}\right) \leq \inf  _ {P _ {h} \in \mathcal {P} _ {H}} l \left(P _ {h}, P _ {z} ^ {*}\right)
$$
For this paper,  $Q\equiv P_{h}$ $w\equiv P_z$ $\Delta_H\equiv \mathcal{P}_H$  and  $W_{n}\equiv \mathcal{P}_{Z}$
$\epsilon$ -Approximate Nash Equilibrium An  $\epsilon$ -approximate Nash Equilibrium is a relaxation of the Nash Equilibrium, where each player's strategy may not be the best response but is still within  $\epsilon$  of the best response. Formally, a pair of mixed strategies  $(P_h, P_z)$  is an  $\epsilon$ -approximate Nash Equilibrium if
$$
\inf  _ {P _ {h} \in \mathcal {P} _ {H}} l \left(P _ {h}, P _ {z}\right) + \epsilon \geq l \left(P _ {h}, P _ {z}\right) \geq \sup  _ {P _ {z} \in \mathcal {P} _ {Z}} l \left(P _ {h}, P _ {z}\right) - \epsilon \tag {7}
$$
No Regret Algorithms No-regret algorithms are a class of online algorithms used in repeated games. The regret of a player is defined as the difference between their cumulative payoff and the best cumulative payoff they could have achieved by consistently playing a single strategy. A no-regret algorithm guarantees that the average regret of a player goes to zero as the number of iterations (or rounds) tends to infinity. In the context of two-player zero-sum games, if both players follow no-regret algorithms, their average strategy profiles converge to the set of Nash Equilibria.
D.2 Online Learning
A popular and widely used approach for solving min-max games is to rely on online learning algorithms [Hazan, 2016, Cesa-Bianchi and Lugosi, 2006]. In this approach, the row (minimization) player and the column (maximization) player play a repeated game against each other. Both players rely on online learning algorithms to choose their actions in each round of the game, with the objective of minimizing their respective regret. The following proposition shows that this repeated gameplay converges to a NE.
Proposition 15 ([Gupta et al., 2020]) Consider a repeated game between the minimization and maximization players in the linearized game. Let  $(P_h^t, P_z^t)$  be the actions chosen by the players in iteration  $t$ . Suppose the actions are such that the regret of each player satisfies:
$$
\sum_ {t = 1} ^ {T} l \left(P _ {h} ^ {t}, P _ {z} ^ {t}\right) - \inf  _ {h \in H} \sum_ {t = 1} ^ {T} l \left(h, P _ {z} ^ {t}\right) \leq \epsilon_ {1} (T)
$$
$$
\sup  _ {z \in Z} \sum_ {t = 1} ^ {T} l \left(P _ {h} ^ {t}, z\right) - \sum_ {t = 1} ^ {T} l \left(P _ {h} ^ {t}, P _ {z} ^ {t}\right) \leq \epsilon_ {2} (T)
$$
Let  $P_{hAVG}, P_{zAVG}$  denote the mixture distributions  $\frac{1}{T}\sum_{i=1}^{T}P_h^i$  and  $\frac{1}{T}\sum_{i=1}^{T}P_z^i$ . Then  $(P_{hAVG}, P_{zAVG})$  is an  $\epsilon$ -approximate mixed NE of the game with:
$$
\epsilon = \frac {\epsilon_ {1} (T) + \epsilon_ {2} (T)}{T}
$$
There exist several algorithms such as FTRL, FTPL, and Best Response (BR), which guarantee sub-linear regret. It is important to choose these algorithms appropriately, given the domains  $H$ ,  $Z$  as our choices impact the rate of convergence to a NE and also the computational complexity of the resulting algorithm.
D.3 General Minimax Theorems (Proof of Proposition 1)
We first state the following convenient generalization of the original Von Neumann's minimax theorem.
Proposition 16 (Von Neumann-Fan minimax theorem, [Borwein, 2016]) Let  $X$  and  $Y$  be Banach spaces. Let  $C \subset X$  be nonempty and convex, and let  $D \subset Y$  be nonempty, weakly compact, and convex. Let  $g: X \times Y \to R$  be convex with respect to  $x \in C$  and concave and upper-semicontinuous with respect to  $y \in D$ , and weakly continuous in  $y$  when restricted to  $D$ . Then,
$$
\sup_{y\in D}\inf_{x\in C}g(x,y) = \inf_{x\in C}\sup_{y\in D}g(x,y)
$$
We now proceed to the proof of Proposition 1. Observe that a convex, compact  $W_{n}$  satisfies the conditions for  $D$  in the above proposition. Moreover, we have  $C = \Delta_H$  i.e. the set of probability measures on  $\Theta$ . It is indeed nonempty and convex. Also, our  $g$  is bilinear in  $Q$  and  $w$ , and thus is convex-concave. Thus, Proposition 1 directly follows from the above result.
Relaxations We can relax the assumption that  $h$  is parameterized by a finite dimensional vector  $\theta$ . For simpler  $H$ , the minimax result directly holds with mild assumptions.
- If  $H = \{h_1, h_2, \ldots, h_n\}$  i.e.  $H$  is finite. Then the original minimax theorem by Neumann holds for arbitrary functions  $l$ .
- If  $H = \{h_1, h_2, \ldots\}$  is denumerable, we further assume  $l$  is a bounded loss function. Then from Theorem 3.1 from [Wald, 1945] to compact and convex  $W$  over  $n$  (finite) datapoints, we can conclude the relation holds.
However, minimax theorems for more general  $H$  require other conditions like the continuity of loss, compactness in function space, etc. See [Simons, 1995] and [Raghavan, 1994].
E Ensemble RAI Games
E.1 Alternative Definitions: Deterministic Ensembles
Alternative definitions for deterministic ensembles could be considered. For example, one could consider  $h_{\mathrm{det};\mathrm{Q}}(x) = \arg \min_{y\in \mathcal{Y}}\mathbb{E}_{h\sim Q}\ell (h(x),y)$ . [Cotter et al., 2019, Wu et al., 2022] designed other more sophisticated strategies, but these are largely domain dependent. For reasons that will be explained later, we stick with Definition 3 in this work. For regression, a popular de-randomization strategy is to compute the expected prediction:  $h_{\mathrm{det};\mathrm{Q}}(x) = \mathbb{E}_{h\sim Q}[h(x)]$ .
E.2 Proofs
E.2.1 Proposition 2
$$
\begin{array}{l} \sup  _ {w \in \Delta_ {n}} \widehat {\mathbb {E}} _ {w} \mathbb {I} [ h _ {\det ; Q} (x) \neq y ] = \sup  _ {i \in [ n ]} \mathbb {I} [ y _ {i} \neq \arg \max  _ {y \in \mathcal {Y}} \mathbb {E} _ {Q} [ h (x _ {i}) = y ] ] \\ = \mathbb {I} [ \sup  _ {w \in \Delta_ {n}} \mathbb {E} _ {w} \mathbb {E} _ {Q} \mathbb {I} [ h (x) \neq y ] \geq 1 / 2 ] \\ = \mathbb {I} \left[ \widehat {R} _ {W _ {n}} \left(h _ {\text {r a n d}; Q}\right) \geq 1 / 2 \right] \\ \end{array}
$$
E.2.2 Proposition 3
Proof. Denote  $y_Q(x) = \arg \max_{y \in \mathcal{Y}} \mathbb{E}_Q(h(x) = y)$ . Then,
$$
\begin{array}{l} \widehat{R}_{W_{n}}(h_{\text{det};Q}) = \sup_{w\in W_{n}}\mathbb{E}_{w}\ell (y_{Q}(x),y) \\ \leq \sup  _ {w \in W _ {n}} \mathbb {E} _ {w} \ell (y _ {Q} (x), y) \frac {P _ {Q} (h (x) = y _ {Q} (x))}{1 / \gamma_ {Q}} \\ \leq \gamma_ {Q} \sup  _ {w \in W _ {n}} \mathbb {E} _ {w} \sum_ {y ^ {\prime} \in \mathcal {Y}} \ell (y ^ {\prime}, y) P _ {Q} (h (x) = y ^ {\prime}) \\ = \gamma_ {Q} \sup  _ {w \in W _ {n}} \mathbb {E} _ {w} \mathbb {E} _ {Q} \sum_ {y ^ {\prime} \in \mathcal {Y}} \ell (y ^ {\prime}, y) \mathbb {I} [ h (x) = y ^ {\prime} ] \\ = \gamma_ {Q} \sup  _ {w \in W _ {n}} \mathbb {E} _ {w} \mathbb {E} _ {Q} \ell (h (x), y) \\ = \gamma_ {Q} \widehat {R} _ {W _ {n}} (h _ {\text {r a n d}; Q}), \\ \end{array}
$$
F Algorithms
F.1 Convergence Rates
F.1.1 Gameplay
We begin with the following lemma adapted from [McMahan, 2017] (Theorem-1)
Lemma 17 Consider the setting of Algorithm 1, and further assume that  $\eta^t \geq \eta^{t-1} > 0$ ,  $Reg(w) \geq 0$ ,  $\eta^t Reg(w)$  is 1-strongly concave w.r.t. some norm  $\| \cdot \|_{(t)}$ . Then for any  $w^* \in W_n$  and any  $T > 0$ , we have:
$$
R e g r e t _ {D, T} \leq \eta^ {T - 1} R e g (w ^ {*}) + \frac {1}{2} \sum_ {t = 1} ^ {T} \| l ^ {t} \| _ {(t - 1), *} ^ {2} \quad (w h e r e l _ {i} ^ {t} = l (h ^ {t} (x _ {i}), y _ {i}))
$$
Consider  $\eta^t = \eta$  and  $\| \cdot \|_{(t)} = \sqrt{\eta}\| \cdot \| _1$ , then
$$
R e g r e t _ {D, T} \leq \eta \operatorname {R e g} \left(w ^ {*}\right) + \frac {1}{2 \eta} \sum_ {t = 1} ^ {T} \| l ^ {t} \| _ {*} ^ {2} \leq \eta D _ {R} ^ {2} + \frac {T}{2 \eta}
$$
Moreover, as H-player plays BR,
$$
R e g r e t _ {H, T} \leq 0
$$
Using Proposition 15, we achieve  $\epsilon$ -approximate NE with:
$$
\epsilon = \epsilon_ {T} \leq \frac {\operatorname {R e g r e t} _ {H , T} + \operatorname {R e g r e t} _ {D , T}}{T} = \frac {\eta D _ {R} ^ {2}}{T} + \mathcal {O} \left(\frac {1}{\eta}\right)
$$
Using definition in Equation 7 gives us the required result.
F.1.2 Greedy
FW Update Note that we are trying to minimize the objective  $L_{\eta}(Q)$  w.r.t  $Q$  by the FW update. Using properties of Fenchel conjugates, it is well known that  $L_{\eta}(Q)$  is  $\frac{1}{\eta}$  smooth w.r.t.  $\| \cdot \| _1$ . Also, the diameter of the simplex  $\Delta_H$  w.r.t.  $\| .\| _1$  is  $\leq 1$ . By [Jaggi, 2013] (Lemma 7), we have  $C_f\leq \frac{1}{\eta}$  and thus by [Jaggi, 2013] Theorem 1, we have:
$$
L _ {\eta} (Q ^ {T}) - \min  _ {Q} L _ {\eta} (Q) \leq \frac {2}{\eta (T + 2)} = \mathcal {O} \left(\frac {1}{\eta T}\right)
$$
Using the definition of  $L_{\eta}(Q)$
$$
L (Q ^ {T}) - \min  _ {Q} L (Q) \leq \eta D _ {R} ^ {2} + \mathcal {O} \left(\frac {1}{\eta T}\right)
$$
Gen-AdaBoost Update This update boils down to a standard coordinate descent update on convex and  $\frac{1}{\eta}$  smooth  $L_{\eta}(Q)$  (w.r.t  $\| .\| _1$ ). Following along the lines of analysis in [Boyd and Vandenberghe, 2004] (Section 9.4.3),
$$
L _ {\eta} (Q ^ {t}) \leq L _ {\eta} (Q ^ {t - 1}) - \frac {\eta}{2} \| \nabla_ {Q} L _ {\eta} (Q ^ {t - 1}) \| _ {\infty} ^ {2}
$$
Using this decent equation, we can follow standard gradient descent analysis to get:
$$
L _ {\eta} (Q ^ {T}) - \min  _ {Q} L _ {\eta} (Q) \leq \mathcal {O} \left(\frac {1}{\eta T}\right)
$$
The rest of the argument will go through as above.
F.2 Closed Form Updates for different uncertainty sets
In this section, we derive closed-form updates for Equation 2 for the entropic regularizer (also mentioned below). We consider common settings mentioned in Table 1.
$$
w ^ {t} \leftarrow \underset {w \in W _ {n}} {\operatorname {a r g m a x}} \sum_ {s = 1} ^ {t - 1} \mathbb {E} _ {w} \ell \left(h ^ {s} (x), y\right) - \eta^ {t - 1} \sum w \log (w)
$$
-  $W_{n} = \{\widehat{P}_{data}\}$  (Empirical Risk Minimization)
$$
w ^ {t} \gets \widehat {P} _ {d a t a}
$$
-  $W_{n} = \Delta_{n}$  (Worst Case Margin)
$$
w ^ {t} \gets \frac {u ^ {t}}{\| u ^ {t} \| _ {1}} \quad \mathrm {w h e r e} \quad u _ {i} ^ {t} \gets \exp \left(- \frac {\sum_ {s = 1} ^ {t - 1} l (h ^ {s} (x _ {i}) , y _ {i})}{\eta^ {t - 1}}\right)
$$
-  $W_{n} = \{w:w\in \Delta_{n},w\succeq \frac{1}{\alpha n}\}$  (α-CVaR)
$$
w _ {i} ^ {t} \leftarrow \min  \left(\frac {1}{\alpha n}, \exp \left(- \frac {\sum_ {s = 1} ^ {t - 1} l \left(h ^ {s} \left(x _ {i}\right) , y _ {i}\right)}{\eta^ {t - 1}} - \lambda\right)\right) \quad \text {f o r} \lambda \quad \text {S . T .} \sum_ {i} w _ {i} ^ {t} = 1
$$
Algorithm 3 describes a projection procedure to find such  $\lambda$  in  $\mathcal{O}(n\log n)$  time.
Algorithm 3 Projection for  $\alpha$ -CVaR set
Input:  $l,\eta ,\alpha$    
1:  $y_{i}\gets \exp \left(-\frac{\sum_{s = 1}^{t - 1}l(h^{s}(x_{i}),y_{i})}{\eta^{t - 1}}\right)$    
2:  $v\gets \frac{1}{\alpha n}$    
3: if  $\frac{y_i}{\|y_i\|_1}\leq v\forall i$  then   
4:  $w_{i}\gets \frac{y_{i}}{\|y_{i}\|_{1}}$    
5: return w   
6: else   
7:  $y_{(i)}\gets \mathrm{sort}\{y_i\}$  S.T.  $y_{(i)}\geq y_{(j)}\forall i\leq j$    
8: function CANDIDATE(m)   
9:  $c_{m}\gets \frac{v}{y(m)}$    
10:  $S_{m}\gets \sum_{i}v\mathbb{1}_{c_{m}y_{(i)}\geq v} + c_{m}y_{(i)}\mathbb{1}_{c_{m}y_{(i)} <   v}$    
11: return Sm   
12: end function   
13: Let  $m^*\gets$  binary search for the smallest m such that CANDIDATE  $(m)\leq 1$    
14:  $c_{m^*}\gets \frac{1 - \sum_{i\leq m^*}v}{\sum_{i > m^*}y(i)}$    
15:  $w_{i}\gets \min (c_{m^*}y_i,v)$    
16: return w   
17: end if
-  $W_{n} = \{w:D(w||\widehat{P}_{\mathbf{data}})\leq \rho_{n}\}$  (DRO) For general  $f$  -divergences, there do not exist closed form updates for  $w^{t}$ . However, they can still be empirically solved using FW-like updates.
-  $W_{n} = \{\widehat{P}_{\mathbf{data}}(G_{1}),\widehat{P}_{\mathbf{data}}(G_{2}),\ldots \widehat{P}_{\mathbf{data}}(G_{K})\}$  (Group DRO)
$$
w ^ {t} \leftarrow \frac {u ^ {t}}{\| u ^ {t} \| _ {1}} \quad \text {w h e r e} \quad u _ {i} ^ {t} \leftarrow \exp \left(- \frac {\sum_ {s = 1} ^ {t - 1} \sum_ {i \in G _ {k}} l \left(h ^ {s} \left(x _ {i}\right) , y _ {i}\right)}{\eta^ {t - 1} s _ {k}}\right) \text {f o r} i \in G _ {k}, s _ {k} = | G _ {k} |
$$
F.3 Proof of Corollary 8
Proof. Note that Reg is 1-strongly concave w.r.t  $\| .\| _1$  and  $\| .\| _2$ . A conservative upper bound for  $D_R^2\leq \log (n)$  for all  $W_{n}(\subseteq \Delta_{n})$ . Thus, we can thus take appropriate values of  $\eta$  to get:
$$
L (Q ^ {T}) \leq \min  _ {Q} L (Q) + \mathcal {O} \left(\sqrt {\frac {\log (n)}{T}}\right)
$$
Thus, we have  $\epsilon \sim \mathcal{O}\left(\sqrt{\frac{\log(n)}{T}}\right)$  from Proposition 15.
G Generalization
G.1 Population Risk
We first present a proposition which gives an equivalent characterization of the RAI population risk Proposition 18 The following are equivalent characterizations of the population RAI risk
$$
R_{W}(h) = \sup_{P:P\ll P_{data},\frac{dP}{dP_{data}}\in W}\mathbb{E}_{P}[\ell (h(x),y)].
$$
$$
R_{W}(h) = \sup_{r:\mathcal{Z}\mapsto \mathbb{R}_{+},\int r(z)dP_{data}(z) = 1,r\in W}\mathbb{E}_{P_{data}}[r(z)\ell_{h}(z)].
$$
Proof. The equivalence between (1) and (2) follows by reparameterizing  $P$  in (1) as follows
$$
d P (z) = r (z) d P _ {\mathrm {d a t a}} (z),
$$
for some  $r(z)\geq 0$
Now suppose the uncertainty set  $W$  is as defined in Equation (5). The next proposition uses duality to derive an equivalent characterization of the RAI risk in this setting.
Proposition 19 Suppose the uncertainty set  $W$  is as specified in Equation (5). Then for any hypothesis  $h$ , the population RAI risk can be equivalently written as
$$
R _ {W} (h) = \inf  _ {\lambda \geq \mathbf {0}, \tau} \mathbb {E} _ {P _ {d a t a}} G _ {\lambda} ^ {*} (\ell_ {h} (z) - \tau) + \sum_ {i = 1} ^ {m} \lambda_ {i} c _ {i} + \tau , \tag {8}
$$
where  $G_{\lambda}^{*}$  is the Fenchel conjugate of  $G_{\lambda}(t) = \sum_{i=1}^{m} \lambda_{i} g_{i}(t)$ .
Proof. We rely on duality to prove the proposition. First observe that the population RAI risk can be rewritten as