Title: Responsible AI (RAI) Games and Ensembles
Venue: NeurIPS 2023
Accepted: yes
Keywords: Responsible AI

fairness

DRO

robustness
Abstract: Several recent works have studied the societal effects of AI; these include issues such as fairness, robustness, and safety.  In many of these objectives, a learner seeks to minimize its worst-case loss over a set of predefined distributions (known as uncertainty sets), with usual examples being perturbed versions of the empirical distribution. In other words, the aforementioned problems can be written as min-max problems over these uncertainty sets. In this work, we provide a general framework for studying these problems, which we refer to as Responsible AI (RAI) games. We provide two classes of algorithms for solving these games:  (a) game-play based algorithms, and (b) greedy stagewise estimation algorithms. The former class is motivated by online learning and game theory, whereas the latter class is motivated by the classical statistical literature on boosting, and regression. We empirically demonstrate the applicability and competitive performance of our techniques for solving several RAI problems, particularly around subpopulation shift.

Responsible AI (RAI) Games and Ensembles

Abstract

Several recent works have studied the societal effects of AI; these include issues such as fairness, robustness, and safety. In many of these objectives, a learner seeks to minimize its worst-case loss over a set of predefined distributions (known as uncertainty sets), with usual examples being perturbed versions of the empirical distribution. In other words, aforementioned problems can be written as min-max problems over these uncertainty sets. In this work, we provide a general framework for studying these problems, which we refer to as Responsible AI (RAI) games . We provide two classes of algorithms for solving these games: (a) game-play based algorithms, and (b) greedy stagewise estimation algorithms. The former class is motivated by online learning and game theory, whereas the latter class is motivated by the classical statistical literature on boosting, and regression. We empirically demonstrate the applicability and competitive performance of our techniques for solving several RAI problems, particularly around subpopulation shift.

1 Introduction

In recent years, AI is increasingly being used in high-stakes decision-making contexts such as hiring, criminal justice, and healthcare. Given the impact these decisions can have on people's lives, it is important to ensure these AI systems have beneficial social effects. An emerging line of work has attempted to formalize such desiderata ranging over ethics, fairness, train-time robustness, test-time or adversarial robustness, and safety, among others. Each of these forms rich sub-fields with disparate desiderata, which are sometimes collated under the umbrella of 'responsible AI'. Many organizations are increasingly advocating the use of responsible AI models [Microsoft, 2021, Google, 2020].

But how do we do so when the majority of recent work around these problems is fragmented and usually focuses on optimizing one of these aspects at a time (DRO [Namkoong and Duchi, 2017, Duchi and Namkoong, 2018], GDRO [Sagawa et al., 2019], CVaR [Zhai et al., 2021a], Distribution Shift [Hashimoto et al., 2018, Zhai et al., 2021b])? Indeed optimizing for just one of these aspects has even been shown to exhibit adverse effects on the other aspects [Roh et al., 2020]. To address this, we study a general framework that is broadly applicable across many of the settings above, and which we refer to as Responsible AI (RAI) games. Our starting point is the recent understanding of a unifying theme in many of these disparate problems, that a learner seeks to minimize its worst-case loss over a set of predefined distributions. For example, in fairness, we seek to perform well on all sub-groups in the data. In robustness, we aim to design models that are robust to perturbations of the training data or the test distribution. This allows us to set up a zero-sum game between a learner that aims to learn a responsible model and an adversary that aims to prevent the learner from doing so. In

the general RAI game setting, this is a computationally intractable game that need not even have a Nash equilibrium. To address this computational issue, we study a relaxation of the single predictor RAI game, which we term the ensemble RAI game, which can also be motivated as a linearization of the original RAI game.

We note that our framework encompasses not only the responsible AI settings but also the setting of classical boosting. Drawing upon the insights from boosting, we provide boosting-based algorithms for solving responsible AI games. We provide convergence guarantees of our algorithms by relying on the connections between boosting and online convex optimization, two-player gameplay [Arora et al., 2012, McMahan, 2011, Bubeck, 2011]. We also conduct empirical analyses to demonstrate the convergence and utility of our proposed algorithms. Interestingly, the algorithms allow for plug-andplay convenience, with changes in the RAI settings requiring only simple changes to the algorithms. More importantly, we could consider intersections of different responsible AI considerations, which in turn can simply be incorporated into our algorithms. Finally, we also study the population risks of our algorithms in certain important settings. We show a surprising result that for the case of binary classification with the 0 / 1 loss, the optimal predictor for a large class of RAI games is the same as the Bayes optimal predictor, thus generalizing an emerging line of results demonstrating this for certain specific games [Hu et al., 2018]. Under such settings, solving the RAI game could nonetheless be helpful in finite sample settings (as also demonstrated in our experiments) since the RAI game serves to encode desiderata satisfied by the Bayes optimal classifier.

2 Problem Setup and Background

We consider the standard supervised prediction setting, with input random variable X ∈ X ⊆ R d , output random variable Y ∈ Y , and samples S = { ( x i , y i ) } n i =1 drawn from a distribution P data over X × Y . Let ̂ P data denote the empirical distribution over the samples. We also have a set H of hypothesis functions h : X ↦→ Y from which we wish to learn the best predictor. We evaluate the goodness of a predictor via a loss function ℓ : Y × Y ↦→ R , which yields the empirical risk: ̂ R ( h ) = E ̂ P data ℓ ( h ( x ) , y ) where E ̂ P data ( f ( x, y )) = 1 n ∑ n i =1 f ( x i , y i ) . Apart from having low expected risk, most settings require h to have certain properties, for example, robustness to distribution shift, fairness w.r.t subpopulations, superior tail performance, resistance to adversarial attacks, robustness in the presence of outliers, etc. We cast all these subproblems into an umbrella term 'Responsible AI'. Each of these properties has been studied extensively in recent works, albeit individually. In this work, we attempt to provide a general framework to study these problems.

2.1 Related Work

We draw our unified framework from seminal works over the past decade by responsible AI researchers on devising non-expected risk objectives, particularly min-max problems , to ensure ML models are responsible. These have resulted in a multitude of different objectives (even for a single responsible AI desideratum such as fairness), and also multiple different sub-communities (so that fairness and multiple disparate robustness communities are relatively fractured), many (if not all) of which we combine within a single umbrella. There is emerging work on relating worst-case performance to invariance [Bühlmann, 2018]; in other words, we might be able to get approximate group invariance via minimizing an appropriately constructed worst-group risk and vice-versa.

RAI aspects as constraints. Many prior works have enforced robustness as a constrained optimization [Shafieezadeh-Abadeh et al., 2015, Gao and Kleywegt, 2022, Namkoong and Duchi, 2016, Ben-Tal et al., 2011]. There have also been few prior works enforcing fairness constraints [Mandal et al., 2020]. To the best of our knowledge, there exists minimal prior work focusing on multiple desiderata at once in this regard.

Multi Objective Optimization. Several works have considered a multi-objective view of ensuring fairness in classifiers [Martinez et al., 2020, Oneto et al., 2018]. If used for multiple RAI objectives, there is usually overhead in choosing a model that achieves a good trade-off between various losses. Also, it is difficult to guarantee that the solution is robust to any of the involved aspects. Our framework guarantees a certain level of performance on each of the RAI aspects under consideration.

Distribution shift. [Koh et al., 2021] classifies distribution shift problems into two categories: Domain generalization, and subpopulation shift. In this work, we focus on the subpopulation shift problem, where the target distribution is absolutely continuous to the source distribution. It has two

main applications: fairness [Hashimoto et al., 2018, Hu et al., 2018, Sagawa et al., 2019, Zhai et al., 2021b] and long-tail learning ( i.e. learning on class-imbalanced datasets) [Cao et al., 2019, Menon et al., 2021, Kini et al., 2021].

Distributionally Robust Optimization (DRO). In DRO one aims to study classifiers that are robust to deviations of the data distribution. DRO has been studied under various uncertainty sets including f -divergence based uncertainty sets [Namkoong and Duchi, 2017, Duchi and Namkoong, 2018, Sagawa et al., 2019], Wasserstein uncertainty sets [Sinha et al., 2017, Gao et al., 2022], Maximum Mean Discrepancy uncertainty sets [Staib and Jegelka, 2019], more general uncertainty sets in the RKHS space [Zhu et al., 2020]. [Li et al., 2021a] evaluate model performance under worst-case subpopulations. Owing to its importance, several recent works have provided efficient algorithms for solving the DRO objective [Namkoong and Duchi, 2016, Qi et al., 2020, Kumar et al., 2023, Jin et al., 2021]. However, a lot of these techniques are specific to particular perturbation sets and are not directly applicable to the more general framework we consider in our work. Furthermore, in our work, we aim to learn an ensemble of models instead of a single model.

Boosting. Classical boosting aims to improve the performance of a weak learner by combining multiple weak classifiers to produce a strong classifier [Breiman, 1999, Friedman et al., 2000, Friedman, 2001, Freund and Schapire, 1995, Freund et al., 1996, Mason et al., 2000]. Over the years, a number of practical algorithms have been introduced such as AdaBoost [Schapire, 1999], LPBoost [Demiriz et al., 2002], gradient boosting [Mason et al., 1999], XGBoost [Chen and Guestrin, 2016], boosting for adversarial robustness [Zhang et al., 2022], [Meunier et al., 2021], [Balcan et al., 2023], and holistic robustness [Bennouna and Parys, 2022]. The algorithms we develop for RAI games are inspired by these algorithms.

Fairness. There are a number of fairness notions for algorithmic fairness, ranging from individual fairness [Dwork et al., 2012, Zemel et al., 2013], group fairness [Hardt et al., 2016a, Zafar et al., 2017], counterfactual fairness [Kusner et al., 2017], Rawlsian max-min fairness [Rawls, 2020, Hashimoto et al., 2018] and others [Barocas et al., 2017, Chouldechova and Roth, 2018, Mehrabi et al., 2021]. Our framework includes the popular notion of minimax group fairness. It doesn't capture other notions of group fairness such as Demographic Parity, Equality of Odds, Equality of Opportunity.

Population RAI Risks. Several recent works have studied properties of the population risks arising in various responsible AI scenarios. Hu et al. [2018] showed that the minimizer of population DRO risk (under general f -divergences) is the classical Bayes optimal classifier. Li et al. [2021b], Duchi and Namkoong [2018], Sinha et al. [2017] provided generalization guarantees for DRO risk under various divergence families ranging from f -divergences to Wasserstein perturbations.

3 RAI Games

In many cases, we do not wish to compute an unweighted average over training samples; due to reasons of noise, tail risk, robustness, and fairness, among many other 'responsible AI' considerations.

Definition 1 (RAI Risks) Given a set of samples { ( x i , y i ) } n i =1 , we define the class of empirical RAI risks (for Responsible AI risks) as: ̂ R W n ( h ) = sup w ∈ W n E w ( h ( x ) , y ) , where W n ⊆ ∆ n , is some set of sample weights (a.k.a uncertainty set), and E w ( f ( x, y )) = ∑ n i =1 w i f ( x i , y i ) .

Various choices of W n give rise to various RAI risks. Table 1 presents examples of RAI risks that are popular in ML. Interestingly, classical problems such as boosting are special cases of RAI risks. In this work, we rely on this connection to design boosting-inspired algorithms for minimizing RAI risks. More choices for W n can be obtained by combining the one's specified in Table 1 using union, intersection, convex-combination operations. For example, if one wants models that are fair to certain pre-specified groups, and at the same-time achieve good tail-risk, then one could choose W n to be the intersection of Group-DRO and α -CVaR uncertainty sets.

Given the empirical RAI risk ̂ R W n ( h ) of a hypothesis, and set of hypotheses H , we naturally wish to obtain the hypothesis that minimizes the empirical RAI risk: min h ∈ H ̂ R W n ( h ) . This can be seen as solving a zero-sum game.

Definition 2 (RAI Games) Given a set of hypothesis H , and a RAI sample weight set W n , the class of RAI games is given as: min h ∈ H max w ∈ W n E w ( h ( x ) , y ) .

We thus study RAI Games for the special cases above and for an arbitrary constraint set W n .

4 Ensemble RAI Games

In this section, we begin our discussion about ensembles. In general, a statistical caveat with Definition 2 is that good worst-case performance over the sample weight set W n is generally harder, and for a simpler set of hypotheses H , there may not exist h ∈ H that can achieve such good worst-case performance. Thus it is natural to consider deterministic ensemble models over H , which effectively gives us more powerful hypothesis classes. Let us first define RAI risk for such classifiers.

Definition 3 (Deterministic Ensemble) Consider the problem of classification, where Y is a discrete set. Given a hypothesis class H , a deterministic ensemble is specified by some distribution Q ∈ ∆ H , and is given by: h det;Q ( x ) = arg max y ∈Y E h ∼ Q I [ h ( x ) = y ] . Correspondingly, we can write the deterministic ensemble RAI risk as ̂ R W n ( h det;Q ( x )) = max w ∈ W n E w ℓ ( h det;Q ( x ) , y ) .

We discuss alternative definitions of deterministic ensembles in the Appendix. This admits a class of deterministic RAI games:

Definition 4 (Deterministic Ensemble RAI Games) Given a set of hypothesis H , a RAI sample weight set W n , the class of RAI games for deterministic ensembles over H is given as:

However, the aforementioned game is computationally less amenable because of the non-smooth nature of de-randomized predictions. Moreover, there are some broader challenges with RAI games given by Definitions 2 and 4. Firstly, they need not have a Nash Equilibrium (NE), and in general, their min-max and max-min game values need not coincide. This poses challenges in solving the games efficiently. Next, in some cases, directly optimizing over the worst-case performance might not even be useful. For instance, [Hu et al., 2016, Zhai et al., 2021a] show the pessimistic result that for classification tasks where when models are evaluated by the zero-one loss, ERM achieves the lowest possible DRO loss defined by some f -divergence or the α -CVaR loss, given that the model is deterministic. To this end, we consider the following randomized ensemble:

Definition 5 (Randomized Ensemble) Given a hypothesis class H , a randomized ensemble is specified by some distribution Q ∈ ∆ H , and is given by: P [ h rand;Q ( x ) = y ] = E h ∼ Q I [ h ( x ) = y ] . Similarly, we can define its corresponding randomized ensemble RAI risk: ̂ R rand ; W n ( Q ) = max w ∈ W n E h ∼ Q E w ℓ ( h ( x ) , y ) .

We can then also define the class of ensemble RAI games:

Definition 6 (Randomized Ensemble RAI Games) Given a set of hypothesis H , a RAI sample weight set W n , the class of mixed RAI games is given as:

This is a much better class of zero-sum games: it is linear in both the hypothesis distribution P , as well as the sample weights w , and if the sample weight set W n is convex, is a convex-concave game. As shown below, under some mild conditions, this game has a Nash equilibrium which can be well approximated via efficient algorithms.

Proposition 1 Let H be parameterized by θ ∈ Θ ⊆ R p , for convex, compact set Θ and let W n be a convex, compact set. Then min Q ∈ ∆ H max w ∈ W n E h ∼ Q E w ℓ ( h ( x ) , y ) = max w ∈ W n min Q ∈ ∆ H E h ∼ Q E w ℓ ( h ( x ) , y )

The proposition follows as a direct consequence of well known minimax theorems (Appendix D.3).

4.1 Going from Deterministic to Randomized Ensembles

To begin, we point out that what we want is a deterministic ensemble rather than a randomized ensemble. In fact, it can be seen that the deterministic ensemble in Definition 3 is a specific derandomization of the randomized ensemble. It is such deterministic ensembles that we usually simply refer to as ensemble predictors. But the RAI risk for the ensemble predictor is NOT equal to the ensemble RAI risk minimized by our desired game in Equation 1 above for randomized ensembles. Thus, the ensemble RAI game might not in general capture the ideal deterministic ensemble. In this section, we study why and when might solving for a random ensemble is meaningful.

Binary Classification. Interestingly, for the very specific case of binary classification, we can provide simple relationships between the risks of the randomized and deterministic ensemble.

See Appendix E.2 for a simple proof. In this case, we can also relate the existence of a perfect deterministic ensemble ('boostability') to a weak learning condition on the set of hypotheses. Specifically, suppose H is boostable iff there exists Q ∈ ∆ H s.t. ̂ R W n ( h det;Q ) = 0 . From the above proposition this is equivalent to requiring that ̂ R W n ( h rand;Q ) < 1 / 2 . We thus obtain:

̸

where the equivalence follows from the min-max theorem and the linearity of the objective in P . The last statement says that for any sample weights w ∈ W n , there exists a hypothesis h ∈ H that has w -weighted loss at most 1 / 2 . We can state this as a 'weak-learning' condition on individual hypotheses in H . The above thus shows that for the specific case of Y = {-1 , 1 } , the zero-one loss ℓ ( y, y ′ ) = I [ y = y ′ ] , and W n = ∆ n , we can relate boostability of H to a weak learning condition on hypothesis within H .

General Classification But in general, we do not have simple connections between ̂ R W n ( h det;Q ) and ̂ R W n ( h rand;Q ) . All we can guarantee is the following upper bound:

Corollary 4 For binary classification, we have γ P ≤ 2 and thus, we recover the well known bound ̂ R W n ( h det;Q ) ≤ 2 ̂ R W n ( h rand;Q )

Remark 5 These bounds might be loose in practice. Specifically, for the binary case, if ̂ R W n ( h rand;Q ) ≤ 1 2 then we have ̂ R W n ( h det;Q ) = 0 . To this end, prior work [Lacasse et al., 2006, Germain et al., 2015, Masegosa et al., 2020] have developed tighter bounds using second-order inequalities. We leave the analyses of these second-order RAI games to future work.

As such, we can cast minimizing randomized RAI risk as minimizing an upper bound on the deterministic ensemble RAI risk. Thus, the corresponding randomized RAI game can be cast as a relaxation of the deterministic RAI game. In the sequel, we thus focus on this randomized ensemble RAI game, which we will then use to obtain a deterministic ensemble. Following the bounds above, the corresponding deterministic ensemble risk will be bounded by the randomized ensemble RAI risk

5 Algorithms

In this section, we present two algorithms for solving the RAI game in Equation (1). Our first algorithm is motivated from online learning algorithms and the second algorithm is motivated from

greedy stepwise algorithms that have been popular for solving many statistical problems such as regression. For simplicity of presentation, we assume H is a finite set. However, our results in the section extend to uncountable sets.

5.1 Methods

Game-play. In game play based algorithms, both the min and the max players are engaged in a repeated game against each other. Both players rely on no-regret algorithms to decide their next action. It is well known that such a procedure converges to a mixed NE of the game Cesa-Bianchi and Lugosi [2006]. In this work, we follow a similar strategy to solve the game in Equation (1) (see Algorithm 1 for the pseudocode). In the t th round of our algorithm, the following distribution w t ∈ W is computed over the training data points

This update is called the Follow-The-Regularized-Leader (FTRL) update. Here, Reg ( · ) is a strongly concave regularizer and η t -1 is the regularization strength. One popular choice for Reg ( · ) is the negative entropy which is given by -∑ i w i log w i . This regularizer is also used by AdaBoost, which is a popular boosting algorithm. In Appendix F.2, we provide analytical expressions for w t for various choices of W n , Reg ( · ) . We note that the regularizer in the FTRL update ensures the stability of the updates; i.e., it ensures consecutive iterates do not vary too much. This stability is naturally guaranteed when W n is a strongly convex set (an example of a strongly convex set is the level set of a strongly convex function. See Appendix for a formal definition and more details). Consequently, the regularization strength η t -1 could be set to 0 in this case, and the algorithm still converges to a NE [Huang et al., 2017].

Once we have w t , a new classifier h t is computed to minimize the weighted loss relative to w t , and added to the ensemble. This update is called the Best Response (BR) update. Learning h t in this way helps us fix past classifiers' mistakes, eventually leading to an ensemble with good performance.

Algorithm 1 Game play algorithm for solving Equation (1)

Greedy. We now take an optimization theoretic viewpoint to design algorithms for Equation (1). Let L ( Q ) denote the inner maximization problem of (1): L ( Q ) := max w ∈ W n E h ∼ Q E w ℓ ( h ( x ) , y ) . When L ( Q ) is smooth (this is the case when W n is a strongly convex set), one could use Frank-Wolfe (FW) to minimize it. The updates of this algorithm are given by

Here, ∇ Q L ( Q t -1 ) = argmax w ∈ W n E h ∼ Q t -1 E w ℓ ( h ( x ) , y ) . This algorithm is known to converge to a minimizer of L ( Q ) at O (1 /t ) rate [Jaggi, 2013]. When L ( Q ) is non-smooth, we first need to smooth the objective before performing FW. In this work we perform Moreau smoothing [Parikh et al., 2014], which is given by

Here Reg ( · ) is a strongly concave regularizer. If Reg ( · ) is 1 -strongly concave, it is well known that L η ( Q ) is O (1 /η ) smooth. Once we have the smoothed objective, we perform FW to find its optimizer (see Algorithm 2 for pseudocode).

Relaxing the simplex constraint. We now derive a slightly different algorithm by relaxing the simplex constraint on Q . Using Lagrangian duality we can rewrite min Q ∈ ∆ H L η ( Q ) as the following problem for some λ ∈ R

One interesting observation is that when W n is the entire simplex and when λ = -1 / 2 , we recover the AdaBoost algorithm. Given the practical success of AdaBoost, we extend it to general W n . In particular, we set λ = -1 / 2 and solve the resulting objective using greedy coordinate-descent. The updates of this algorithm are given in Algorithm 2.

Remark 6 Algorithm 2 takes the step sizes { α t } T t =1 as input. In practice, one could use line search to figure out the optimal step-sizes, for better performance.

Algorithm 2 Greedy algorithms for solving Equation (1)

We provide convergence rates for the algorithms below:

Proposition 7 (Convergence Rates) Let l ( h ( x ) , y ) ∈ [0 , 1] ∀ h ∈ H, ( x, y ) ∈ D and Reg : ∆ n → R be a 1 -strongly concave function w.r.t norm ∥ . ∥ 1 . Let Q T be the output returned from running Algorithm 1 or 2 for T iterations. Let D R be a constant S.T. D 2 R = max x,y ∈ W n | Reg ( x ) -Reg ( y ) | .

(Gameplay) If η t = η , then Q T satisfies L ( Q T ) ≤ min Q L ( Q ) + ηD 2 R T + O ( 1 η ) .

(Greedy) If line-search is performed for α t , then Q T (FW or the Gen-AdaBoost update) satisfies L ( Q T ) ≤ min Q L ( Q ) + ηD 2 R + O ( 1 ηT ) .

We refer the reader to Appendix F.1 for a simple proof using existing theory on online convex optimization [McMahan, 2011, Jaggi, 2013]. Another useful insight is that Algorithms 1 and 2 are related to each other under special settings as shown by Appendix H.1.

n

Weak Learning Conditions It might not be practical for H -player to play BR (Step 3: Algorithm 1) or correspondingly, to find the best possible classifier at every round (Step 2: Algorithm 2). Under weak learning conditions, we can indeed achieve (approximate) convergence when we only solve these problems approximately. See Appendix H.2 for more details.

6 Generalization Guarantees

In this section, we study the population RAI risk and present generalization bounds which quantify the rates at which empirical RAI risk converges to its population counterpart.

6.1 Population RAI Games

Recall, the empirical RAI risk optimizes over all sample re-weightings w ∈ W n that lie within the probability simplex ∆ n . Thus it's population counterpart optimizes over distributions P that are absolutely continuous with respect to the data distribution P data :

Following [Shapiro et al., 2021], we can rewrite this as follows. Suppose we use Z = ( X,Y ) ∈ Z := X × Y , so that P, P data are distributions over Z . We then define ℓ h : Z ↦→ R as ℓ h ( z ) = ℓ ( h ( x ) , y ) . We can then write the population RAI risk as (see Appendix G for a proof):

For classification, we define the RAI-Bayes optimal classifier as: Q ∗ W = arg min Q R W ( Q ) . Here, the minimum is w.r.t the set of all measurable classifiers (both deterministic and random). This is the 'target' classifier we wish to learn given finite samples. Note that this might not be the same as the vanilla Bayes optimal classifier: Q ∗ = arg min Q E [ ̂ R ( Q )] , which only minimizes the expected loss, and hence may not satisfactorily address RAI considerations.

We now try to characterize the RAI-Bayes optimal classifier. However, doing this requires a bit more structure on W . So, in the sequel, we consider constraint sets of the following form:

where we assume that g i : R + ↦→ R , i ∈ [ m ] are convex. Note that this choice of W encompasses a broad range of RAI games including DRO with f -divergence, CVaR, soft-margin uncertainty sets. Perhaps surprisingly, the following proposition shows that the minimizer of population RAI risk is nothing but the vanilla Bayes optimal classifier.

Proposition 9 (Bayes optimal classifier) Consider the problem of binary classification where Y = {-1 , +1 } . Suppose ℓ ( h ( x ) , y ) = ϕ ( yh ( x )) for some ϕ : R → [0 , ∞ ) which is either the 0 / 1 loss, or a convex loss function that is differentiable at 0 with ϕ ′ (0) < 0 . Suppose the uncertainty set W is as specified in Equation (5) . Moreover, suppose { g i } i =1 ...m are convex and differentiable functions. Then, the vanilla Bayes optimal classifier is also a RAI-Bayes optimal classifier.

Remark 10 In the special case of m = 1 in Equation (5) , we recover the result of [Hu et al., 2018]. However, our proof is much more elegant than the proof of [Hu et al., 2018], and relies on the dual representation of the population RAI risk.

One perspective of the above result is that the vanilla Bayes optimal classifier is also 'responsible' as specified by the RAI game. This is actually reasonable in many practical prediction problems where the label annotations are actually derived from humans, who presumably are also responsible. Why then might we be interested in the RAI risk? One advantage of the RAI risks is in finite sample settings where the equivalence no longer holds, and the RAI risk could be construed as encoding prior knowledge about properties of the Bayes optimal classifier. We also note that the above equivalence is specific for binary classification.

6.2 Generalization Guarantees

Our generalization bounds rely on the following dual characterization of the RAI population risk.

Proposition 11 Suppose the uncertainty set W is as specified in Equation (5) . Then for any hypothesis h , the population RAI risk can be equivalently written as

where G ∗ λ is the Fenchel conjugate of G λ ( t ) = ∑ m i =1 λ i g i ( t ) .

We utilize the above expression for R W ( h ) to derive the following deviation bound for ̂ R W n ( h ) .

Theorem 12 Consider the setting of Proposition 11. Suppose { g i } i =1 ...m are convex and differentiable functions. Suppose ℓ h ( z ) ∈ [0 , B ] for all h ∈ H , z ∈ Z . Suppose, for any distribution P data , the minimizers ( λ ∗ , τ ∗ ) of Equation (6) lie in the following set: E = { ( λ, τ ) : ∥ λ ∗ ∥ ∞ ≤ ¯ Λ , | τ ∗ | ≤ T } . Moreover, let's suppose the optimal λ ∗ for P data is bounded away from 0 and satisfies min i λ ∗ i ≥ Λ ¯ .

Let G,L , be the range and Lipschitz constants of G ∗ λ :

For any fixed h ∈ H , with probability at least 1 -2 e -t

Given Theorem 12, one can take a union bound over the hypothesis class H to derive the following uniform convergence bounds.

Corollary 13 Let N ( H,ϵ, ∥ · ∥ L ∞ ( Z ) ) be the covering number of H in the sup-norm which is defined as ∥ h ∥ L ∞ ( Z ) = sup z ∈Z | h ( z ) | . Then with probability at least 1 -N ( H,ϵ n , ∥ · ∥ L ∞ ( Z ) ) e -t , the following holds for any h ∈ H : | R W ( h ) -̂ R W n ( h ) | ≤ 30 n -1 / 2 G ( √ t + m log( nL )) . Here ϵ n = n -1 / 2 G √ t + m log( nL ) .

The above bound depends on parameters ( λ ∗ , τ ∗ , G, L ) which specific to the constraint set W . To instantiate it for any W one needs to bound these parameters. We note that our generalization guarantees become sub-optimal as Λ ¯ → 0 . This is because the Lipschitz constant L could potentially get larger as λ approaches the boundary. Improving these bounds is an interesting future direction.

Remark 14 We note that aforementioned results results follow from relatively stringent assumptions. Exploring the impact of relaxing these assumptions is an interesting direction for future works.

7 Experiments

In this section, we demonstrate the generality of proposed RAI methods by studying one of the most well-studied problems in RAI i.e. the case of subpopulation shift. Given a large number of possible W , we acknowledge that this is not a complete analysis, even with respect to the problems that live within the minimax framework. Instead, we aim to display convergence, plug-and-play generality, and superior performance over some seminal baselines of this task. We conduct experiments on both synthetic and real-world datasets. Please refer to Appendix for details on synthetic experiments. We consider a number of responsible AI settings, including subpopulation shift, in the domain-oblivious (DO) setting where we do not know the sub-populations [Hashimoto et al., 2018, Lahoti et al., 2020, Zhai et al., 2021a], the domain-aware (DA) setting where we do [Sagawa et al., 2019], and the partially domain-aware (PDA) setting where only some might be known.

Datasets & Domain Definition. We use the following datasets: COMPAS [Angwin et al., 2016], CIFAR-10 (original, and with a class imbalanced split [Jin et al., 2021, Qi et al., 2021]) and CIFAR100. See the Appendix for more details on our datasets. For COMPAS, we consider race (White vs Other) and biological gender (Male vs Female) as our sensitive attributes. This forms four disjoint subgroups defined by these attributes. In the PDA setting, we partition only across the attribute race while training, but still run tests for all four subgroups. On CIFAR-10, class labels define our 10 subpopulations. Similarly as above, for the PDA setting, we make 5 super-groups of two classes each. On CIFAR-100, class labels define our 100 subpopulations. For the PDA setting, we make 20 super-groups, each consisting of five classes.

Baselines. We compare our method against the following baselines: (a) Deterministic classifiers trained on empirical risk (ERM) and DRO risks, particularly the quasi-online algorithm for Group DRO [Sagawa et al., 2019] (Online GDRO), and an ITLM-inspired SGD algorithm [Zhai et al., 2021b, Shen and Sanghavi, 2018] for χ 2 DRO (SGD ( χ 2 )) (b) Ensemble models AdaBoost [Schapire, 1999]. Note that the purpose of our experiments is to show that we can match baselines for a specific single desideratum (e.g. worst-case sub-population) while allowing for learning models that can solve multiple responsible AI desiderata at the same time, for which we have no existing baselines.

Proposed Methods. We focus on Algorithm 2 and refer to FW and Gen-AdaBoost updates as RAI-FW and RAI-GA, respectively. Moreover, our implementations include the following alterations: · We track the unregularized objective value from Equation 1 for the validation set, and whenever it increases we double the regularization factor η , which we find can improve generalization. · We also use this objective w.r.t the normalized Q t to perform a line search for the step size α . For the FW update, our search space is a ball around 1 t at round t , while for GA, we search within (0 , 1) .

Base Learners & Training. Training time scales linearly with the number of base learners. Inference, though, can be parallelized if need be. We usually find training on 3-5 learners is good enough on all scenarios explored in the paper. We defer further details of our base learners and hyperparameter choices to the Appendix.

Constraint sets W n . For RAI algorithms, we use the following constraint sets: · Domain Oblivious (DO): We use the χ 2 -DRO constraint set to control for worst-case subpopulations. · Domain Aware (DA): We use the Group DRO constraint set as the domain definitions are known. · Partially Domain-Aware (PDA): We use a novel set W n which is the intersection over Group DRO constraints over the known domains and χ 2 constraints to control for unknown group performance. For baselines, we use AdaBoost and SGD( χ 2 ) for the DO setting. Online GDRO serves as our baseline for both DA and PDA settings, where the algorithm uses whatever domain definitions are available.

Results and Discussion. We run our methods and baselines under the settings described above and report the results in Table 2. As such, we can make the following observations:

RAI-FW and RAI-GA methods significantly improve the worst-case performance with only a few base learners across all datasets in all three settings, while maintaining average case performance. Moreover, For seemingly harder tasks i.e. a large gap between average and worst-case performance, the algorithms are able to improve significantly over the baselines. For example, we observe a 5% improvement in performance in the case of CIFAR-100.

The plug-and-play framework allows for several different W n to enhance various responsible AI qualities at once. We demonstrate this with the partial domain aware setting (PDA), where the performance lead widens, indicating that RAI is able to jointly optimize effectively for both known and unknown subpopulations while Online GDRO suffers from some of the group information being unknown. In practice, one can construct many more novel sets W n .

Although bigger (complex) models exhibit stronger performance than RAI ensembles, there are several caveats to this observation. Firstly, these models are ∼ 10-15 times larger than our base models. This limits their use w.r.t both training & inference compute required. However, RAI ensembles utilize a small number of much smaller models which can be individually trained quite easily. Even with these large models as base learners, constructing ensembles exhibits a performance boost, indicating that our framework is able to 'boost' models of varying complexities.

8 Conclusion

Under the umbrella of 'responsible AI', an emerging line of work has attempted to formalize desiderata ranging over ethics, fairness, robustness, and safety, among others. Many of these settings (Table 1) can be written as min-max problems involving optimizing some worst-case loss under a set of predefined distributions. For all the problems that can be framed as above, we introduce and study a general framework, which we refer to as Responsible AI (RAI) games. Our framework extends to classical boosting scenarios, offering boosting-based algorithms for RAI games alongside proven convergence guarantees. We propose practical algorithms to solve these games, as well as statistical analyses of solutions of these games. We find that RAI can guarantee multiple responsible AI aspects under appropriate choices of uncertainty sets.

References

Microsoft. Microsoft. https://www.microsoft.com/en-us/ai/responsible-ai , 2021. Accessed: Date Accessed.

Google. Google. https://ai.google/responsibility/responsible-ai-practices/ , 2020. Accessed: Date Accessed.

Hongseok Namkoong and John C Duchi. Variance-based regularization with convex objectives. Advances in neural information processing systems , 30, 2017.

John Duchi and Hongseok Namkoong. Learning models with uniform performance via distributionally robust optimization. arXiv preprint arXiv:1810.08750 , 2018.

Shiori Sagawa, Pang Wei Koh, Tatsunori B Hashimoto, and Percy Liang. Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization. arXiv preprint arXiv:1911.08731 , 2019.

Runtian Zhai, Chen Dan, Arun Suggala, J Zico Kolter, and Pradeep Ravikumar. Boosted cvar classification. Advances in Neural Information Processing Systems , 34:21860-21871, 2021a.

Tatsunori B. Hashimoto, Megha Srivastava, Hongseok Namkoong, and Percy Liang. Fairness without demographics in repeated loss minimization. International Conference On Machine Learning , 2018.

Runtian Zhai, Chen Dan, Zico Kolter, and Pradeep Ravikumar. Doro: Distributional and outlier robust optimization. In Marina Meila and Tong Zhang, editors, Proceedings of the 38th International Conference on Machine Learning , volume 139 of Proceedings of Machine Learning Research , pages 12345-12355. PMLR, 18-24 Jul 2021b. URL https://proceedings.mlr.press/v139/ zhai21a.html .

Yuji Roh, Kangwook Lee, Steven Euijong Whang, and Changho Suh. Fr-train: A mutual informationbased approach to fair and robust training. arXiv preprint arXiv: Arxiv-2002.10234 , 2020.

Sanjeev Arora, Elad Hazan, and Satyen Kale. The multiplicative weights update method: a metaalgorithm and applications. Theory of computing , 8(1):121-164, 2012.

Brendan McMahan. Follow-the-regularized-leader and mirror descent: Equivalence theorems and l1 regularization. In Geoffrey Gordon, David Dunson, and Miroslav Dudík, editors, Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics , volume 15 of Proceedings of Machine Learning Research , pages 525-533, Fort Lauderdale, FL, USA, 11-13 Apr 2011. PMLR.

Sébastien Bubeck. Introduction to online optimization. Lecture notes , 2:1-86, 2011.

Weihua Hu, Gang Niu, Issei Sato, and Masashi Sugiyama. Does distributionally robust supervised learning give robust classifiers? In International Conference on Machine Learning , pages 20292037. PMLR, 2018.

Peter Bühlmann. Invariance, causality and robustness. arXiv preprint arXiv: 1812.08233 , 2018.

Soroosh Shafieezadeh-Abadeh, Peyman Mohajerin Esfahani, and D. Kuhn. Distributionally robust logistic regression. Neural Information Processing Systems , 2015.

Rui Gao and Anton Kleywegt. Distributionally robust stochastic optimization with wasserstein distance. Mathematics of Operations Research , 2022.

Hongseok Namkoong and John C. Duchi. Stochastic gradient methods for distributionally robust optimization with f-divergences. In Neural Information Processing Systems , 2016. URL https: //api.semanticscholar.org/CorpusID:7481496 .

Aharon Ben-Tal, Dick den Hertog, Anja De Waegenaere, Bertrand Melenberg, and Gijs Rennen. Robust solutions of optimization problems affected by uncertain probabilities. Advanced Risk & Portfolio Management® Research Paper Series , 2011. URL https://api.semanticscholar. org/CorpusID:761793 .

Debmalya Mandal, Samuel Deng, Suman Jana, Jeannette Wing, and Daniel J Hsu. Ensuring fairness beyond the training data. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems , volume 33, pages 18445-18456. Curran Associates, Inc., 2020. URL https://proceedings.neurips.cc/paper_files/paper/ 2020/file/d6539d3b57159babf6a72e106beb45bd-Paper.pdf .

Natalia Martinez, Martin Bertran, and Guillermo Sapiro. Minimax pareto fairness: A multi objective perspective. In Hal Daumé III and Aarti Singh, editors, Proceedings of the 37th International Conference on Machine Learning , volume 119 of Proceedings of Machine Learning Research , pages 6755-6764. PMLR, 13-18 Jul 2020. URL https://proceedings.mlr.press/v119/ martinez20a.html .

Oneto, Michele Donini, Amon Elders, and M. Pontil. Taking advantage of multitask learning for fair classification. AAAI/ACM Conference on AI, Ethics, and Society , 2018. doi: 10.1145/3306618. 3314255.

Pang Wei Koh, Shiori Sagawa, Henrik Marklund, Sang Michael Xie, Marvin Zhang, Akshay Balsubramani, Weihua Hu, Michihiro Yasunaga, Richard Lanas Phillips, Irena Gao, et al. Wilds: A benchmark of in-the-wild distribution shifts. In International Conference on Machine Learning , pages 5637-5664. PMLR, 2021.

Kaidi Cao, Colin Wei, Adrien Gaidon, Nikos Arechiga, and Tengyu Ma. Learning imbalanced datasets with label-distribution-aware margin loss. Advances in Neural Information Processing Systems , 32:1567-1578, 2019.

Aditya Krishna Menon, Sadeep Jayasumana, Ankit Singh Rawat, Himanshu Jain, Andreas Veit, and Sanjiv Kumar. Long-tail learning via logit adjustment. In International Conference on Learning Representations , 2021.

Ganesh Ramachandra Kini, Orestis Paraskevas, Samet Oymak, and Christos Thrampoulidis. Labelimbalanced and group-sensitive classification under overparameterization. In Thirty-Fifth Conference on Neural Information Processing Systems , 2021.

Aman Sinha, Hongseok Namkoong, Riccardo Volpi, and John Duchi. Certifying some distributional robustness with principled adversarial training. arXiv preprint arXiv:1710.10571 , 2017.

Rui Gao, Xi Chen, and Anton J Kleywegt. Wasserstein distributionally robust optimization and variation regularization. Operations Research , 2022.

Matthew Staib and Stefanie Jegelka. Distributionally robust optimization and generalization in kernel methods. Advances in Neural Information Processing Systems , 32, 2019.

Jia-Jie Zhu, Wittawat Jitkrittum, Moritz Diehl, and Bernhard Schölkopf. Kernel distributionally robust optimization. arXiv preprint arXiv:2006.06981 , 2020.

Mike Li, Hongseok Namkoong, and Shangzhou Xia. Evaluating model performance under worst-case subpopulations. In A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems , 2021a. URL https://openreview.net/ forum?id=nehzxAdyJxF .

Qi Qi, Yi Xu, Rong Jin, Wotao Yin, and Tianbao Yang. Attentional biased stochastic gradient for imbalanced classification. arXiv preprint arXiv:2012.06951 , 2020.

Ramnath Kumar, Kushal Majmundar, Dheeraj Nagaraj, and Arun Sai Suggala. Stochastic re-weighted gradient descent via distributionally robust optimization. arXiv preprint arXiv:2306.09222 , 2023.

Jikai Jin, Bohang Zhang, Haiyang Wang, and Liwei Wang. Non-convex distributionally robust optimization: Non-asymptotic analysis. In A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems , 2021. URL https: //openreview.net/forum?id=gZLhHMyxa-.

Leo Breiman. Prediction games and arcing algorithms. Neural computation , 11(7):1493-1517, 1999.

Jerome Friedman, Trevor Hastie, Robert Tibshirani, et al. Additive logistic regression: a statistical view of boosting (with discussion and a rejoinder by the authors). The annals of statistics , 28(2): 337-407, 2000.

Jerome H Friedman. Greedy function approximation: a gradient boosting machine. Annals of statistics , pages 1189-1232, 2001.

Yoav Freund and Robert E Schapire. A desicion-theoretic generalization of on-line learning and an application to boosting. In European conference on computational learning theory , pages 23-37. Springer, 1995.

Yoav Freund, Robert E Schapire, et al. Experiments with a new boosting algorithm. In icml , volume 96, pages 148-156. Citeseer, 1996.

Llew Mason, Jonathan Baxter, Peter L Bartlett, and Marcus R Frean. Boosting algorithms as gradient descent. In Advances in neural information processing systems , pages 512-518, 2000.

Robert E Schapire. The boosting approach to machine learning: An overview. In MSRI workshop on Nonlinear Estimation and Classification , 1999.

Ayhan Demiriz, Kristin P Bennett, and John Shawe-Taylor. Lpboost: A boosting algorithm with linear programming. In International Conference on Machine Learning , pages 143-150. AAAI Press, 2002.

Llew Mason, Jonathan Baxter, Peter Bartlett, and Marcus Frean. Boosting algorithms as gradient descent. In S. Solla, T. Leen, and K. Müller, editors, Advances in Neural Information Processing Systems , volume 12. MIT Press, 1999. URL https://proceedings.neurips.cc/paper_ files/paper/1999/file/96a93ba89a5b5c6c226e49b88973f46e-Paper.pdf .

Tianqi Chen and Carlos Guestrin. Xgboost: A scalable tree boosting system. arXiv preprint arXiv: Arxiv-1603.02754 , 2016.

Dinghuai Zhang, Hongyang Zhang, Aaron Courville, Yoshua Bengio, Pradeep Ravikumar, and Arun Sai Suggala. Building robust ensembles via margin boosting. In International Conference on Machine Learning , pages 26669-26692. PMLR, 2022.

Laurent Meunier, Meyer Scetbon, Rafael B Pinot, Jamal Atif, and Yann Chevaleyre. Mixed nash equilibria in the adversarial examples game. In Marina Meila and Tong Zhang, editors, Proceedings of the 38th International Conference on Machine Learning , volume 139 of Proceedings of Machine Learning Research , pages 7677-7687. PMLR, 18-24 Jul 2021. URL https://proceedings. mlr.press/v139/meunier21a.html .

Maria-Florina Balcan, Rattana Pukdee, Pradeep Ravikumar, and Hongyang Zhang. Nash equilibria and pitfalls of adversarial training in adversarial robustness games. In International Conference on Artificial Intelligence and Statistics , pages 9607-9636. PMLR, 2023.

A. Bennouna and Bart P. G. Van Parys. Holistic robust data-driven decisions. ARXIV.ORG , 2022. doi: 10.48550/arXiv.2207.09560.

Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. Fairness through awareness. In Proceedings of the 3rd innovations in theoretical computer science conference , pages 214-226, 2012.

Rich Zemel, Yu Wu, Kevin Swersky, Toni Pitassi, and Cynthia Dwork. Learning fair representations. In International conference on machine learning , pages 325-333. PMLR, 2013.

Moritz Hardt, Eric Price, and Nati Srebro. Equality of opportunity in supervised learning. Advances in neural information processing systems , 29, 2016a.

Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rodriguez, and Krishna P Gummadi. Fairness beyond disparate treatment & disparate impact: Learning classification without disparate mistreatment. In Proceedings of the 26th international conference on world wide web , pages 1171-1180, 2017.

Matt J Kusner, Joshua Loftus, Chris Russell, and Ricardo Silva. Counterfactual fairness. Advances in neural information processing systems , 30, 2017.

John Rawls. A theory of justice: Revised edition . Harvard university press, 2020.

Solon Barocas, Moritz Hardt, and Arvind Narayanan. Fairness in machine learning. Nips tutorial , 1: 2017, 2017.

Alexandra Chouldechova and Aaron Roth. The frontiers of fairness in machine learning. arXiv preprint arXiv:1810.08810 , 2018.

Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan. A survey on bias and fairness in machine learning. ACM Computing Surveys (CSUR) , 54(6):1-35, 2021.

Mike Li, Hongseok Namkoong, and Shangzhou Xia. Evaluating model performance under worst-case subpopulations. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems , volume 34, pages 17325-17334. Curran Associates, Inc., 2021b. URL https://proceedings.neurips.cc/paper_files/ paper/2021/file/908075ea2c025c335f4865f7db427062-Paper.pdf .

Manfred K Warmuth, Jun Liao, and Gunnar Rätsch. Totally corrective boosting algorithms that maximize the margin. In Proceedings of the 23rd international conference on Machine learning , pages 1001-1008, 2006.

Peter Bartlett, Yoav Freund, Wee Sun Lee, and Robert E Schapire. Boosting the margin: A new explanation for the effectiveness of voting methods. The annals of statistics , 26(5):1651-1686, 1998.

Mehryar Mohri, Gary Sivek, and Ananda Theertha Suresh. Agnostic federated learning. In International Conference on Machine Learning , pages 4615-4625. PMLR, 2019.

Weihua Hu, Gang Niu, Issei Sato, and Masashi Sugiyama. Does distributionally robust supervised learning give robust classifiers? International Conference On Machine Learning , 2016.

Alexandre Lacasse, François Laviolette, Mario Marchand, Pascal Germain, and Nicolas Usunier. Pac-bayes bounds for the risk of the majority vote and the variance of the gibbs classifier. Advances in Neural information processing systems , 19, 2006.

Pascal Germain, Alexandre Lacasse, Francois Laviolette, Mario Marchand, and Jean-Francis Roy. Risk bounds for the majority vote: From a pac-bayesian analysis to a learning algorithm. arXiv preprint arXiv:1503.08329 , 2015.

Andrés Masegosa, Stephan Lorenzen, Christian Igel, and Yevgeny Seldin. Second order pac-bayesian bounds for the weighted majority vote. Advances in Neural Information Processing Systems , 33: 5263-5273, 2020.

Nicolo Cesa-Bianchi and Gabor Lugosi. Prediction, learning, and games . Cambridge university press, 2006.

Ruitong Huang, Tor Lattimore, András György, and Csaba Szepesvári. Following the leader and fast rates in online linear prediction: Curved constraint sets and other regularities. The Journal of Machine Learning Research , 18(1):5325-5355, 2017.

Martin Jaggi. Revisiting frank-wolfe: Projection-free sparse convex optimization. In International conference on machine learning , pages 427-435. PMLR, 2013.

Neal Parikh, Stephen Boyd, et al. Proximal algorithms. Foundations and trends® in Optimization , 1 (3):127-239, 2014.

Alexander Shapiro, Darinka Dentcheva, and Andrzej Ruszczynski. Lectures on stochastic programming: modeling and theory . SIAM, 2021.

Preethi Lahoti, Alex Beutel, Jilin Chen, Kang Lee, Flavien Prost, Nithum Thain, Xuezhi Wang, and Ed Chi. Fairness without demographics through adversarially reweighted learning. Advances in neural information processing systems , 33:728-740, 2020.

Julia Angwin, Jeff Larson, Surya Mattu, and Lauren Kirchner. Machine bias: There's software used across the country to predict future criminals. and its biased against blacks. ProPublica , May 2016. URL https://www.propublica.org/article/ machine-bias-risk-assessments-in-criminal-sentencing .

Qi Qi, Zhishuai Guo, Yi Xu, Rong Jin, and Tianbao Yang. An online method for a class of distributionally robust optimization with non-convex objectives. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems , volume 34, pages 10067-10080. Curran Associates, Inc., 2021. URL https://proceedings.neurips.cc/paper_files/paper/2021/file/ 533fa796b43291fc61a9e812a50c3fb6-Paper.pdf .

Yanyao Shen and Sujay Sanghavi. Learning with bad training data via iterative trimmed loss minimization. International Conference On Machine Learning , 2018.

Nidhi Kalra and Susan M. Paddock. Driving to safety: How many miles of driving would it take to demonstrate autonomous vehicle reliability? Transportation Research Part A: Policy and Practice , 94:182-193, 2016.

Andreas Fuster, Paul Goldsmith-Pinkham, Tarun Ramadorai, and Alexander Walther. Predictably unequal? the effects of machine learning on credit markets. Social Science Research Network , (3072038), 2018.

Xinsong Ma, Zekai Wang, and Weiwei Liu. On the tradeoff between robustness and fairness. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems , volume 35, pages 26230-26241. Curran Associates, Inc., 2022. URL https://proceedings.neurips.cc/paper_files/paper/2022/file/ a80ebbb4ec9e9b39789318a0a61e2e43-Paper-Conference.pdf .

Anastasia Chan. Gpt-3 and instructgpt: technological dystopianism, utopianism, and 'contextual' perspectives in ai ethics and industry. AI and Ethics , 3(1):53-64, 2023.

Emily M Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. On the dangers of stochastic parrots: Can language models be too big. In Proceedings of the 2021 ACM conference on fairness, accountability, and transparency , pages 610-623, 2021.

Christos Louizos, Kevin Swersky, Yujia Li, M. Welling, and R. Zemel. The variational fair autoencoder. International Conference on Learning Representations , 2015.

Moritz Hardt, Eric Price, Eric Price, and Nati Srebro. Equality of opportunity in supervised learning. In D. Lee, M. Sugiyama, U. Luxburg, I. Guyon, and R. Garnett, editors, Advances in Neural Information Processing Systems , volume 29. Curran Associates, Inc., 2016b. URL https://proceedings.neurips.cc/paper_files/paper/2016/file/ 9d2682367c3935defcb1f9e247a97c0d-Paper.pdf .

Haohan Wang, Zexue He, Zachary Chase Lipton, and E. Xing. Learning robust representations by projecting superficial statistics out. International Conference on Learning Representations , 2018.

Sravanti Addepalli, Anshul Nasery, R. Venkatesh Babu, Praneeth Netrapalli, and Prateek Jain. Learning an invertible output mapping can mitigate simplicity bias in neural networks. arXiv preprint arXiv: 2210.01360 , 2022.

Junbum Cha, Kyungjae Lee, Sungrae Park, and Sanghyuk Chun. Domain generalization by mutualinformation regularization with pre-trained models. European Conference on Computer Vision , 2022. doi: 10.48550/arXiv.2203.10789.

Elad Hazan. Introduction to online convex optimization. Found. Trends Optim. , 2016. doi: 10.1561/ 2400000013.

Kartik Gupta, Arun Sai Suggala, Adarsh Prasad, Praneeth Netrapalli, and Pradeep Ravikumar. Learning minimax estimators via online learning. ARXIV.ORG , 2020.

Jonathan Michael Borwein. A very complicated proof of the minimax theorem. 2016.

Abraham Wald. Generalization of a theorem by v. neumann concerning zero sum two person games. Annals of Mathematics , 46:281, 1945.

Stephen Simons. Minimax theorems and their proofs. 1995.

TES Raghavan. Zero-sum two-person games. Handbook of game theory with economic applications , 2:735-768, 1994.

Andrew Cotter, Maya Gupta, and Harikrishna Narasimhan. On making stochastic classifiers deterministic. Advances in Neural Information Processing Systems , 32, 2019.

Jimmy Wu, Yatong Chen, and Yang Liu. Metric-fair classifier derandomization. In International Conference on Machine Learning , pages 23999-24016. PMLR, 2022.

H Brendan McMahan. A survey of algorithms and analysis for adaptive online learning. The Journal of Machine Learning Research , 18(1):3117-3166, 2017.

Stephen Boyd and Lieven Vandenberghe. Convex Optimization . Cambridge University Press, 2004.

R Tyrrell Rockafellar. Convex analysis . Number 28. Princeton university press, 1970.

Peter L Bartlett, Michael I Jordan, and Jon D McAuliffe. Convexity, classification, and risk bounds. Journal of the American Statistical Association , 101(473):138-156, 2006.

Ambuj Tewari and Peter L Bartlett. On the consistency of multiclass classification methods. Journal of Machine Learning Research , 8(5), 2007.

Martin J Wainwright. High-dimensional statistics: A non-asymptotic viewpoint , volume 48. Cambridge university press, 2019.

Robert M. Freund and Paul Grigas. New analysis and results for the frank-wolfe method. Mathematical Programming , 155:199-230, 2014. doi: 10.1007/s10107-014-0841-6. URL http://link.springer.com/article/10.1007/s10107-014-0841-6/fulltext.html .

Sergey Zagoruyko and N. Komodakis. Wide residual networks. British Machine Vision Conference , 2016. doi: 10.5244/C.30.87.

A Broader Impact

Responsible AI has become an important topic as ML/AI systems increase in scale, and are being deployed in a variety of scenarios. Models not optimized for responsible facets can have disastrous consequences [Kalra and Paddock, 2016, Angwin et al., 2016, Fuster et al., 2018].

Our research promotes the principles of Responsible AI (RAI), encompassing ethics, fairness, and safety considerations. By providing a framework to consider these simultaneously, this research could help prevent scenarios where optimizing for one aspect unintentionally compromises another [Ma et al., 2022], mitigating the risk of creating biases or vulnerabilities in AI systems.

We also address the fragmentation in recent work around Responsible AI. The 'plug-and-play' nature of our approach allows for the easy adaptation of algorithms for different Responsible AI considerations. This adaptability could lead to more practical and user-friendly tools for implementing RAI across different applications.

Numerous ML models like Large language models, (GPT-3, GPT-4), are rapidly transforming numerous domains, including natural language processing, data analysis, content creation, and more. Given their remarkable ability to generate human-like text, they can be used to construct narratives, answer queries, or even automate customer service. However, with such capabilities come significant responsibilities, as these models can inadvertently perpetuate biases, misinformation, or harmful content if not correctly regulated. Therefore, ensuring the responsible behavior of these models is crucial [Chan, 2023, Bender et al., 2021]. Our research presents a general framework that can be pivotal in the responsible design of such large language models.

B Limitations

We now identify some limitations of our current work, along with corresponding future directions.

To more concretely establish the empirical superiority of our optimization techniques, more experiments involving large over-parametrized models need to be conducted.

The proposed generalization bounds are not tight for all risks. A more careful analysis would be needed for such a generalization.

Our framework only handles uncertainty sets that are supported on the training data. It'd be interesting to generalize our framework further to support other uncertainty sets based on Wasserstein divergences that are not necessarily supported on the training data.

Finally, the presence of outliers can often destabilize training of large models[Zhai et al., 2021b]. However, our current framework assumes the training data is un-corrupted. In future, we aim to extend our framework to support corruptions in the training data.

We note that our framework can also be extended to the problem of adversarial test-time robustness where there is an adversary corrupting the inputs sent to the model during inference. Let A ( x ) be the set of perturbations that the adversary can add to input x . The uncertainty set in this case contains distributions supported on { ( x ′ , y ′ ) : ∃ ( x, y ) ∈ ̂ P data such that x ′ ∈ x + A ( x ) , y ′ = y } .

Our framework primarily covers RAI aspects which can written as minmax problems. Here we provide some other notions of RAI which our framework does not directly cover.

Fairness: Various notions of fairness have been studied by the ML community. While our framework captures minimax group fairness, it doesn't capture other notions of group fairness such as Demographic Parity [Louizos et al., 2015], Equality of Odds, Equality of Opportunity [Hardt et al., 2016b]. Our framework doesn't capture individual fairness notions.

Robustness: While our framework covers group robustness and certain forms of distributional robustness, it doesn't cover robustness to Wasserstein perturbations and other (un)structured distribution shifts often studied in the domain generalization community [Wang et al., 2018, Addepalli et al., 2022, Cha et al., 2022].

C Terminology and Notation

C.1 Terminology

Strongly Convex Sets Aset A is λ -strongly convex w.r.t a norm ∥·∥ , if, for any x, y ∈ A , γ ∈ [0 , 1] , the ∥ · ∥ norm ball with origin γx +(1 -γ ) y and radius γ (1 -γ ) λ ∥ x -y ∥ 2 / 2 lies in A .

f -divergence For any two probability distributions P, Q , f -divergence between P and Q is defined as D f ( Q || P ) = E P [ f ( dQ/dP )] . Here f : R + → R is a convex function such that f (1) = 0 .

C.2 Notation

D Background

D.1 Two Player Zero-sum Games

Consider the following game between two players. One so-called 'row player' playing actions h ∈ H , and the other 'column player' playing actions z ∈ Z . Suppose that when the two players play actions h, z respectively, the row player incurs a loss of l ( h, z ) ∈ R , while the column player incurs a loss of -l ( h, z ) . The sum of the losses for the two players can be seen to be equal to zero so such a game is known as a two-player zero-sum game. It is common in such settings to refer to the gain l ( h, z ) of the column player, rather than its loss of -l ( h, z ) . Both players try to maximize their gain/minimize their loss.

It is common in game theory to consider a linearized game in the space of probability measures, which is in general better behaved. To set up some notation, for any probability distributions P h over H , and P z over Z , define:

Nash Equilibrium A Nash Equilibrium (NE) is a stable state of a game where no player can gain by unilaterally changing their strategy while the other players keep theirs unchanged. In a two-player zero-sum game, a Nash Equilibrium is a pair of mixed strategies ( h ∗ , z ∗ ) satisfying

Note that whenever a pure strategy NE exists, the minimax and maximin values of the game are equal to each other:

What often exists is a mixed strategy NE, which is precisely a pure strategy NE of the linearized game. That is, ( P ∗ h , P ∗ z ) is called a mixed strategy NE of the zero-sum game, if

For this paper, Q ≡ P h , w ≡ P z , ∆ H ≡ P H and W n ≡ P Z .

ϵ -Approximate Nash Equilibrium An ϵ -approximate Nash Equilibrium is a relaxation of the Nash Equilibrium, where each player's strategy may not be the best response but is still within ϵ of the best response. Formally, a pair of mixed strategies ( P h , P z ) is an ϵ -approximate Nash Equilibrium if

No Regret Algorithms No-regret algorithms are a class of online algorithms used in repeated games. The regret of a player is defined as the difference between their cumulative payoff and the best cumulative payoff they could have achieved by consistently playing a single strategy. A no-regret algorithm guarantees that the average regret of a player goes to zero as the number of iterations (or rounds) tends to infinity. In the context of two-player zero-sum games, if both players follow no-regret algorithms, their average strategy profiles converge to the set of Nash Equilibria.

D.2 Online Learning

A popular and widely used approach for solving min-max games is to rely on online learning algorithms [Hazan, 2016, Cesa-Bianchi and Lugosi, 2006]. In this approach, the row (minimization) player and the column (maximization) player play a repeated game against each other. Both players rely on online learning algorithms to choose their actions in each round of the game, with the objective of minimizing their respective regret. The following proposition shows that this repeated gameplay converges to a NE.

Proposition 15 ([Gupta et al., 2020]) Consider a repeated game between the minimization and maximization players in the linearized game. Let ( P t h , P t z ) be the actions chosen by the players in iteration t . Suppose the actions are such that the regret of each player satisfies:

Let P hAV G , P zAV G denote the mixture distributions 1 T ∑ T i =1 P i h and 1 T ∑ T i =1 P i z . Then ( P hAV G , P zAV G ) is an ϵ -approximate mixed NE of the game with:

There exist several algorithms such as FTRL, FTPL, and Best Response (BR), which guarantee sub-linear regret. It is important to choose these algorithms appropriately, given the domains H,Z as our choices impact the rate of convergence to a NE and also the computational complexity of the resulting algorithm.

D.3 General Minimax Theorems (Proof of Proposition 1)

We first state the following convenient generalization of the original Von Neumann's minimax theorem.

Proposition 16 (Von Neumann-Fan minimax theorem, [Borwein, 2016]) Let X and Y be Banach spaces. Let C ⊂ X be nonempty and convex, and let D ⊂ Y be nonempty, weakly compact, and convex. Let g : X × Y → R be convex with respect to x ∈ C and concave and upper-semicontinuous with respect to y ∈ D , and weakly continuous in y when restricted to D . Then,

We now proceed to the proof of Proposition 1. Observe that a convex, compact W n satisfies the conditions for D in the above proposition. Moreover, we have C = ∆ H i.e. the set of probability measures on Θ . It is indeed nonempty and convex. Also, our g is bilinear in Q and w , and thus is convex-concave. Thus, Proposition 1 directly follows from the above result.

Relaxations We can relax the assumption that h is parameterized by a finite dimensional vector θ . For simpler H , the minimax result directly holds with mild assumptions.

If H = { h 1 , h 2 , ...h n } i.e. H is finite. Then the original minimax theorem by Neumann holds for arbitrary functions l .

If H = { h 1 , h 1 , ... } i.e. H is denumerable. We further assume l is a bounded loss function. Then from Theorem 3.1 from [Wald, 1945] to compact and convex W over n (finite) datapoints, we can conclude the relation holds.

Hoever, minimax theorems for more general H require other conditions like the continuity of loss, compactness in function space, etc. See [Simons, 1995] and [Raghavan, 1994].

E Ensemble RAI Games

E.1 Alternative Definitions: Deterministic Ensembles

Alternative definitions for deterministic ensembles could be considered. For example, one could consider h det;Q ( x ) = arg min y ∈Y E h ∼ Q ℓ ( h ( x ) , y ) . [Cotter et al., 2019, Wu et al., 2022] designed other more sophisticated strategies, but these are largely domain dependent. For reasons that will be explained later, we stick with Definition 3 in this work. For regression, a popular de-randomization strategy is to compute the expected prediction: h det;Q ( x ) = E h ∼ Q [ h ( x )] .

E.2 Proofs

E.2.1 Proposition 2

Proof.

as required.

̸

̸

̸

E.2.2 Proposition 3

Proof. Denote y Q ( x ) = arg max y ∈Y E Q ( h ( x ) = y ) . Then,

R

̂

W

n

(

h

det;Q

) =

=

=

=

sup

w

n

W

∈

sup

w

w

E

E

w

n

W

∈

γ

Q

γ

γ

γ

Q

Q

Q

sup

w

n

W

∈

sup

w

n

W

∈

sup

w

n

W

∈

R

̂

W

n

(

h

ℓ

(

ℓ

(

E

y

y

w

E

w

(

x

)

Q

Q

(

x

)

∑

y

′

, y

, y

)

)

ℓ

(

P

y

E

∑

Q

E

w

Q

E

rand;Q

y

′

ℓ

)

(

,

h

(

x

Q

′

, y

ℓ

(

h

)

(

(

x

) =

1

P

y

)

y

/γ

Q

Q

′

, y

, y

)

(

)

h

(

I

[

h

x

) =

(

x

Q

(

x

))

y

) =

′

)

y

′

]

as required.

F Algorithms

F.1 Convergence Rates

F.1.1 Gameplay

We begin with the following lemma adapted from [McMahan, 2017] (Theorem-1)

Lemma17 Consider the setting of Algorithm 1, and further assume that η t ≥ η t -1 > 0 , Reg ( w ) ≥ 0 , η t Reg ( w ) is 1-strongly concave w.r.t. some norm ∥ . ∥ ( t ) . Then for any w ∗ ∈ W n and any T > 0 , we have:

Consider η t = η and ∥ . ∥ ( t ) = √ η ∥ . ∥ 1 , then

Moreover, as H-player plays BR,

Using Proposition 15, we achieve ϵ -approximate NE with:

Using definition in Equation 7 gives us the required result.

F.1.2 Greedy

FWUpdate Note that we are trying to minimize the objective L η ( Q ) w.r.t Q by the FW update. Using properties of Fenchel conjugates, it is well known that L η ( Q ) is 1 η smooth w.r.t. ∥ . ∥ 1 . Also, the diameter of the simplex ∆ H w.r.t. ∥ . ∥ 1 is ≤ 1. By [Jaggi, 2013] (Lemma 7), we have C f ≤ 1 η , and thus by [Jaggi, 2013] Theorem 1, we have:

Using the definition of L η ( Q ) ,

Gen-AdaBoost Update This update boils down to a standard coordinate descent update on convex and 1 η smooth L η ( Q ) (w.r.t ∥ . ∥ 1 ). Following along the lines of analysis in [Boyd and Vandenberghe, 2004] (Section 9.4.3),

Using this decent equation, we can follow standard gradient descent analysis to get:

The rest of the argument will go through as above.

F.2 Closed Form Updates for different uncertainty sets

In this section, we derive closed-form updates for Equation 2 for the entropic regularizer (also mentioned below). We consider common settings mentioned in Table 1.

W n = { ̂ P data } (Empirical Risk Minimization)

W n = ∆ n (Worst Case Margin)

W n = { w : w ∈ ∆ n , w ⪯ 1 αn } ( α -CVaR)

Algorithm 3 describes a projection procedure to find such λ in O ( n log n ) time.

Algorithm 3 Projection for α -CVaR set

W n = { w : D ( w || ̂ P data ) ≤ ρ n } (DRO) For general f -divergences, there do not exist closed form updates for w t . However, they can still be empirically solved using FW-like updates.

W n = { ̂ P data ( G 1 ) , ̂ P data ( G 2 ) , . . . ̂ P data ( G K ) } (Group DRO)

F.3 Proof of Corollary 8

Proof. Note that Reg is 1 -strongly concave w.r.t ∥ . ∥ 1 and ∥ . ∥ 2 . A conservative upper bound for D 2 R ≤ log( n ) for all W n ( ⊆ ∆ n ) . Thus, we can thus take appropriate values of η to get:

Thus, we have ϵ ∼ O (√ log( n ) T ) from Proposition 15.

G Generalization

G.1 Population Risk

We first present a proposition which gives an equivalent characterization of the RAI population risk Proposition 18 The following are equivalent characterizations of the population RAI risk

1.

2.

Proof. The equivalence between (1) and (2) follows by reparameterizing P in (1) as follows

for some r ( z ) ≥ 0

Now suppose the uncertainty set W is as defined in Equation (5). The next proposition uses duality to derive an equivalent characterization of the RAI risk in this setting.

Proposition 19 Suppose the uncertainty set W is as specified in Equation (5) . Then for any hypothesis h , the population RAI risk can be equivalently written as

where G ∗ λ is the Fenchel conjugate of G λ ( t ) = ∑ m i =1 λ i g i ( t ) .

Proof. We rely on duality to prove the proposition. First observe that the population RAI risk can be rewritten as

Since the above objective is concave in r and linear in λ, τ , we rely on Lagrangian duality to rewrite it as

where L ( r, λ, τ ) is defined as:

Recall the interchangeability theorem:

so long as the space H is decomposable. Since in our case we are working with the set L 1 ( Z , P ) = { r : Z ↦→ R : ∫ r ( z ) p ( z ) dz = 1 } , which is decomposable, we can apply the interchangeability theorem to get:

where G λ ( t ) = ∑ m i =1 λ i g i ( t ) , and G ∗ λ is its Fenchel conjugate. so that:

We have the following properties of the Fenchel conjugate G ∗ λ ( t ) . These follow from the properties of Fenchel conjugates described in Rockafellar [1970].

Lemma20 Consider the setting of Proposition 19. The Fenchel conjugate G ∗ λ is convex, differentiable and an increasing function that satisfies

Proof of Proposition 9 For the sake of clarity, we first state Proposition 9 below.

Proposition 21 (Bayes optimal classifier) Consider the problem of binary classification where Y = {-1 , +1 } . Suppose ℓ ( h ( x ) , y ) = ϕ ( yh ( x )) for some ϕ : R → [0 , ∞ ) which is either the 0 / 1 loss, or a convex loss function that is differentiable at 0 with ϕ ′ (0) < 0 . Suppose the uncertainty set W is as specified in Equation (5) . Moreover, suppose { g i } i =1 ...m are convex and differentiable functions. Then, the vanilla Bayes optimal classifier is also a RAI-Bayes optimal classifier.

Proof. Following Proposition 19 it is easy to see that the RAI Bayes optimal classifier is the minimizer of the following problem

where the minimization over h is over the set of all classifiers. For any fixed ( λ, τ ) , we now show that the classifier h that minimizes the above optimization problem is a vanilla Bayes optimal classifier. First note that the above optimization problem, for a fixed ( λ, τ ) , can be rewritten as

Using the interchangeability theorem, we can further rewrite this as

Here, P x data is the marginal distribution of P data over x , and P data ( ·| x ) is the distribution of y conditioned on x .

Now suppose ϕ is the 0 / 1 loss

Recall, G ∗ λ is an increasing function (see Lemma 20). So G ∗ λ ( -τ ) ≤ G ∗ λ (1 -τ ) . Using this, it is easy to see that for any x , the following is a minimizer of inf u ∈{-1 , 1 } E P data ( ·| x ) [ G ∗ λ ( ϕ ( uy ) -τ )]

This shows that the vanilla Bayes optimal classifier is a minimizer of the population RAI risk.

Now suppose ϕ : R → [0 , ∞ ) is convex, differentiable at 0 with ϕ ′ (0) < 0 . Moreover, suppose h : X → R is a real valued classifier. In this case, the RAI Bayes optimal classifier is a minimizer of the following objective

Let ι ( x ) = G ∗ λ ( ϕ ( x ) -τ ) . It is easy to see that ι ( x ) is convex. This is because ι ′ ( x ) = ( G ∗ λ ) ′ ( ϕ ( x ) -τ ) ϕ ′ ( x ) is an increasing function; this follows from the fact that G ∗ λ is convex with non-negative gradients. Moreover, ι ′ (0) = ( G ∗ λ ) ′ ( ϕ (0) -τ ) ϕ ′ (0) ≤ 0 . Then, Bartlett et al. [2006], Tewari and Bartlett [2007] show that for any x , the following u ∗ is a minimizer of the inner optimizatin problem: u ∗ > 0 if P data ( y = 1 | x ) ≥ 1 2 , u ∗ < 0 otherwise. This shows that vanilla Bayes optimal classifier is minimizer of the population RAI risk.

G.2 Generalization Guarantees

G.2.1 Proof of Proposition 11

Proposition 11 directly follows from Proposition 19.

G.2.2 Proof of Theorem 12

We first present a key concentration result we use in the proof.

Lemma 22 (Hoeffding bound [Wainwright, 2019]) Suppose that the random variables { X i } n i =1 are independent with mean µ i , and bounded between [ a, b ] . Then for any t ≥ 0 , we have

We now proceed to the proof of the Theorem. Following Proposition 11, we know that the population and empirical RAI risk of a classifier h can be written as

Our goal here is to bound the following quantity for any given h :

The rest of the proof focuses on bounding the RHS of the above equation. The overall idea is to first provide a high probability bound of the RHS for any given λ, τ . Next, we take a union bound over all feasible ( λ, τ ) 's by constructing an appropriate ϵ -net.

Fixed λ, τ . Observe that G ∗ λ ( ℓ h ( z ) -τ ) is bounded and satisfies

This follows from the fact that G ∗ λ is an increasing function (see Proposition 20), and ℓ h is bounded between 0 and B . From Heoffding bound we know that for any fixed h ∈ H , the following holds with probability at least 1 -2 e -t

Union bound over λ, τ . Define set E ′ as

Let N ( E ′ , ϵ, ∥ · ∥ 2 ) be the ϵ -net over E ′ in ∥ · ∥ 2 norm. It is well known that there exists such a set whose size is upper bounded by [Wainwright, 2019]

. For any ( λ, τ ) ∈ E ′ , let ( λ ϵ , τ ϵ ) be an element in N ( E ′ , ϵ, ∥ · ∥ 2 ) that is ϵ -close to ( λ, τ ) . Now consider the following

Since G ∗ λ is L -Lipschitz, the last two terms in the RHS above can be upper bounded by Lϵ . Substituting this in the above equation, we get

where ( a ) follows from Equation (9), and holds with probability at least 1 -( ¯ Λ+ T ϵ ) m +1 e -t . Choosing ϵ = G L √ t n , we get the desired result.

G.3 Proof of Corollary 13

The proof follows from a standard covering number argument. For any h ∈ H , let h ϵ be the point in the ϵ -net that is closest to h . Then we have

Observe that the last two terms above are bounded by ϵ n . Also observe that the first term in the RHS can be upper bounded by 10 n -1 / 2 G ( √ t + m log( nL )) with probability at least 1 -2 N ( H,ϵ n , ∥ · ∥ L ∞ ( Z ) ) e -t . Combining these two and substituting the value of ϵ n gives us the required result.

H Algorithms: Further Discussion

H.1 Equivalence Conditions for RAI Algorithms (1 and 2)

Proposition 23 Assume that we set α t = 1 t and perform coordinate descent update in Algorithm 2 (FW) i.e. G t = argmin h ∈ H 〈 h, ∇ Q L η ( Q t -1 ) 〉 , then the update is equivalent to the update given by Algorithm 1 with η t = ηt .

Proof. From Equation 3, and using the fact that L η ( Q ) can be written as a fenchel conjugate, we know that

This matches Equation 2 with η t -1 = ηt . Moreover,

This corresponds to the update for h t in Algorithm 1. Thus, we have our equivalence.

H.2 Weak Learning Conditions

For the well-known scenario of binary classification and zero-one loss, we recover the quasi-AdaBoost weak learning condition:

Proposition 24 Consider the scenario of binary classification and l as the zero-one loss. If the H-player only plays an approximate best response strategy i.e. h t satisfies E w t ℓ ( h t ( x ) , y ) ≤ 1 / 2 -γ for some γ > 0 , then ̂ R W n ( h det : Q T ) = 0 for T > T 0 for some large enough T 0 .

Proof. Since the D-player uses regret optimal strategy, we have that:

while from the approximate-BR condition we have that:

so that for T > T 0 large enough so that ϵ T < γ/ 2 , we have that:

As Q T assigns mass 1 /T to each of { h t } T t =1 , we have:

Hence, we see that h det ; Q T incurs zero error.

For the general setting, we have a slightly stronger weak learning condition, which follows from the analysis of Frank-Wolfe update [Freund and Grigas, 2014, Jaggi, 2013].

so that we have:

Proposition 25 Consider Algorithm 2 with the FW update and Reg ( . ) as a 1 -strongly concave regularize w.r.t. ∥ . ∥ 1 . If G t satisfies G t ≤ min Q 〈 Q, ∇ Q L η ( Q t -1 ) 〉 + δ t , where { δ k } is the sequence of approximation errors with δ t ≥ 0 , then:

If δ t ≤ ϵ η ( t +2) i.e. decaying errors, then L η ( Q T ) ≤ min Q L η ( Q ) + 2(1+ ϵ ) η ( T +2)

If δ t ≤ ϵ η i.e. constant errors, then L η ( Q T ) ≤ min Q L η ( Q ) + 2 η ( T +2) + ϵ η

Proof. Note that we are trying to minimize the objective L η ( Q ) w.r.t Q by the FW update. Using properties of Fenchel conjugates, it is well known that L η ( Q ) is 1 η smooth w.r.t. ∥ . ∥ 1 . Also, the diameter of the simplex ∆ H w.r.t. ∥ . ∥ 1 is ≤ 1.

By [Jaggi, 2013] (Lemma 7, Theorem 1), we have C f ≤ 1 η , and thus, we have:

By [Freund and Grigas, 2014] (Theorem 5.1), in case of approximation errors, the FW/optimality gap converges as before along with a convex combination of errors at each time step i.e. if we are able to solve the linear optimization problem within constant error ϵ η , then these errors do not accumulate. Moreover, the convex combination can be bound by the maximum error possible and we get,

I Experiments

RAI games constitute an optimization paradigm that goes beyond traditional approaches such as distributionally robust optimization, fairness, and worst-case performance. We have seen that for specific uncertainty sets W , RAI Games optimize over well-established robust optimization objectives. As such, the purpose of our experiments is to demonstrate the practicality and generality of our proposed strategies, rather than establishing state-of-the-art over baselines. Given a large number of possible W , we do not attempt an exhaustive empirical analysis. Instead, we underscore the plug-and-play nature of RAI Games.

I.1 Setup

Subpopulation Shift A prevalent scenario in machine learning involves subpopulation shift, necessitating a model that performs effectively on the data distribution of each subpopulation (or domain). We explore the following variations of this setting:

Domain Oblivious (DO). Recent work [Hashimoto et al., 2018], [Lahoti et al., 2020], [Zhai et al., 2021a] studies the domain-oblivious setting, where the training algorithm lacks knowledge of the domain definition. In this case, approaches like α -CVaR and χ 2 -DRO aim to maximize performance over a general notion of the worst-off subpopulation.

Domain Aware (DA). Several prior works [Sagawa et al., 2019] have investigated the domain-aware setting, in which all domain definitions and memberships are known during training.

Partially Domain-Aware (PDA). More realistically, in real-world applications, there usually exist multiple domain definitions. Moreover, some of these domain definitions may be known during training, while others remain unknown. The model must then perform well on instances from all domains, regardless of whether their definition is known. This setting is challenging as it necessitates the model to learn both domain-invariant and domain-specific features and to generalize well to new instances from unknown domains.

I.2 Further Details for Section 7

Base Learners We use linear classifiers, WRN-28-1, and WRN-28-5 [Zagoruyko and Komodakis, 2016] as base classifiers for COMPAS, CIFAR-10 and CIFAR-100 respectively. To get a sense of performance improvements, we also benchmark performance with larger models, namely a threehidden-layer neural network for COMPAS and WRN-34-10 for CIFAR-10/100.

Proposed Methods This paper introduces two categories of algorithms. We elect not to present results for Algorithm 1, which we notice has similar performance to Algorithm 2. Conversely, we provide in-depth experimental analyses for both updates of Algorithm 2, which warrant some special attention due to due to their relation to AdaBoost. In this section, we refer to the FW and Gen-AdaBoost updates as RAI-FW and RAI-GA, respectively. Our implemented versions incorporate a few alterations: 1. We track the un-regularized objective value from Equation 1 for the validation set. If it increases at any round t , we increase the regularization factor η by a fixed multiple (specifically, 2 ). We notice that it leads to better generalization performance over the test set. 2. The same un-regularized objective w.r.t normalized Q t is also used to perform a line search for the step size α . For the FW update, our search space is a ball around 1 t at round t , while for the GA update, we search within the range (0 , 1) .

Training. We use SGD with momentum = 0 . 9 for optimization. We first warm up the model with some predefined epochs of ERM (3 for COMPAS and 20 for CIFAR-10/100), followed by a maximum of T = 5 base models trained from the warm-up model with sample weights provided by our algorithms. Each base model is trained for 500 iterations on COMPAS and 2000 iterations on CIFAR-10/100. Each experiment is run three times with different random seeds. For evaluation, we report the averaged expected and worst-case test loss from Equation 1.

Datasets We conduct our experiments on three real-world datasets:

COMPAS [Angwin et al., 2016] pertains to recidivism prediction, with the target being whether an individual will re-offend within two years. This dataset is extensively used in fairness research. We randomly sample 70% of the instances for the training data (with a fixed random seed), and the remainder is used for validation/testing.

CIFAR-10 and CIFAR-100 are widely used image datasets. For CIFAR-10, we consider two settings: the original set and an imbalanced split [Jin et al., 2021, Qi et al., 2021]. In the imbalanced split, we make worst-case performance more challenging by randomly sampling each category at different ratios. To be precise, we sample the ith class with a sampling ratio ρ i where ρ = { 0 . 804 , 0 . 543 , 0 . 997 , 0 . 593 , 0 . 390 , 0 . 285 , 0 . 959 , 0 . 806 , 0 . 967 , 0 . 660 } . For these datasets, we use the standard training and testing splits, reserving 10% of the training samples as validation data.

Hyperparameters For COMPAS, we warm up for 3 epochs and then train every base classifier for 500 iterations. For CIFAR-10 and CIFAR-100, we warm up the models for 20 epochs and train base classifiers for 2000 iterations. The mini-batch size is set to 128. It should be noted that the primary aim of our experiments is not hyperparameter tuning. The experiments in this paper are designed to demonstrate use cases and compare different algorithms. Hence, while we maintain consistency of hyperparameters across all experiments, we do not extensively tune them for optimal performance.

I.3 Interesting Observation: Boosting Robust Learners

Boosting Robust Base Learners We conclude our results with one interesting observation. Until now, we have been comparing our ensembles with deterministic models. As such, we acknowledge

that given the inherent differences between the two, making a fair comparison is challenging. However, we find that our setup can "boost" not only ERM but also other robust base learners i.e. if we use these robust optimization methods to find our base learners under analogous RAI constraints, we are able to further enhance the robust performance of these algorithms. The results are shown in Table 4. We hypothesize that individually robust base learners are able to help the ensemble generalize well, allowing our approach to further optimize through ensembles.

I.4 Synthetic Datasets

In this section, we use synthetic datasets to illustrate how our RAI algorithms converge, and how different constraints on W translate into performance across various responsible metrics. We use the following distributions to construct the datasets, and use class labels as the group labels.

Dataset-I: P ( X | Y = 0) = N ((0 , 0) , I ) , P ( X | Y = 1) = 1 3 N (( -3 , 1) , I )) + 1 3 N ((3 , 0) , I )) + 1 3 N ((0 , -3) , I )) , P ( Y = 0) = 0 . 7 , P ( Y = 1) = 0 . 3 .

Dataset-II: P ( X | Y = 0) = 5 12 N (( -2 , -2) , 0 . 5 I ) + 2 12 N (( -2 , -2) , 0 . 5 I ) + 5 12 N ((2 , 2) , 0 . 5 I ) , P ( X | Y = 1) = 2 5 N (( -3 , 0) , 0 . 3 I )) + 3 5 N ((3 , 0) , 0 . 3 I )) , P ( Y = 0) = 0 . 7 , P ( Y = 1) = 0 . 3

We sample 1000 points each for both training and testing from both distributions. Note that these datasets deliberately exhibit: 1. Class imbalance (particularly in Dataset-I) 2. Multiple minority sub-populations (within and between classes) 3. Varying noise levels in the sub-populations (predominantly in Dataset-II). Such characteristics are frequently encountered in real-world scenarios and demand responsible classifiers.

Models For base learners, we use linear classifiers for Dataset-I and neural network classifiers with a single hidden layer of size 4 and ReLU activations for Dataset-II. We find that base learners can be models with varying complexity.

Hyperparameters Due to the limited size of the datasets, we forgo the warm-up stage. At every round, we run 1000 iterations with a mini-batch size of 32 . We run α -LPBoost with the default value η = 1 . For α -CVaR experiments, we take α = 0 . 7 across all experiments. For lower values of α , we observe similar results and comparisons, albeit with a substantial reduction in average metrics. Consequently, we opt for conservative values to standardize average performance across all models and subsequently compare worst-case performance in responsible settings.

I.4.1 Results and Discussion

Domain Oblivious (DO) To begin, we run ERM, AdaBoost, α -LPBoost, and RAI games on Dataset-I. For RAI-GA and RAI-FW games, we use α -CVaR uncertainty set as W . Given the class imbalance, Y = 0 and Y = 1 represent good candidates for subpopulations of interest. The results are reported in Figure 1. We immediately observe the following:

Both proposed methods RAI-FW and RAI-GA effectively decrease the objective value and achieve lower worst-class classification loss, as compared to both ERM and AdaBoost.

They closely follow the α -LPBoost iterates. Intuitively, our quasi-boosting updates resemble α -LPBoost for the CVaR objective, and that is reflected in similar objective values.

Domain Aware (DA) For this setting, we run ERM, AdaBoost, Online GDRO, RAI-GA, and RAI-FW on Dataset-II. We use Group DRO over the five gaussian groups as the uncertainty set W . Although Dataset-II was selected due to the presence of more pronounced subpopulation behavior, we get similar results for Dataset-I as well. The results are reported in Figure 2 and Table 5.

Partially Domain-Aware (PDA) For this setting, we run our algorithms for Dataset-II. Similar to gaussian memberships, the class labels Y provide another secondary definition of implicit grouping in the dataset. We report the results in Table 5. A critical observation from the DA setting results is that Online GDRO, and RAI (Group) all exhibit inferior performance according to the secondary class definition i.e. although they optimize for the known groups (gaussian), they fail to optimize for unknown groups (class labels). Thus, a natural solution is to run RAI updates over the intersection

Algorithm

Synthetic Datset-I

Average Loss

Worst Class Loss

ERM

27.3

82.6

AdaBoost

26.5

27.7

α

-LPBoost

23.5

23.7

RAI-GA

23.9

24.4

RAI-FW

23.9

24.2

0

1

2

3

4

5

T (number of rounds)

0.30

0.32

0.34

0.36

0.38

0.40

CVaR Objective

AdaBoost (train)

AdaBoost (test)

RAIGame-FW (train)

RAIGame-FW (test)

LPBoost (train)

LPBoost (test)

RAIGame-GA (train)

RAIGame-GA (test)

ERM (test)

ERM (train)

6

4

2

0

2

4

6

4

3

2

1

0

1

2

3

4

x2

| ERM

| AdaBoost

| Online GDRO

| RAI-GA

| RAI-FW

class 0

class 1

1

2

3

4

5

T (Number of rounds)

0.08

0.10

0.12

0.14

0.16

0.18

0.20

Group DRO Objective

ERM (test)

AdaBoost (test)

Online GDRO (test)

RAI-GA (test)

RAI-FW (test)

x1

of χ 2 (for unknown groups) and Group (for known groups) constraints. As seen in the Table, we see that both RAI-GA and RAI-FW achieve a middle ground by significantly improving worst-case performance for both known and unknown groups.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper aims to provide a general framework that is broadly applicable across responsible AI (such as ethics, fairness, train-time robustness, test-time or adversarial robustness etc). Motivated by this, the authors proposed Responsible AI (RAI) games, where a learner seeks to minimize its worst-case loss over a set of predefined distributions. This is modeled by a zero-sum game between a learner that aims to learn a responsible model, and an adversary that aims to prevent the learner from doing so. To solve this problem, the authors first go from a single predictor to a deterministic ensemble, namely a distribution over predictors that can then be de-randomized via a majority vote. Then, the authors study a relaxation of this deterministic ensemble RAI game, which they term the randomized ensemble RAI game, which can also be motivated as a linearization of the original RAI game. Finally, the authors conduct empirical analyses to demonstrate the convergence and performance of their proposed algorithms.\n\nSTRENGTHS:\n- The topic of responsible AI is an interesting and important area of research.\n- The theoretical results are backed by proofs included in the appendix.\n\nWEAKNESSES:\n- Despite the appealing motivation of building a general framework that is broadly applicable across responsible AI, I find the proposed framework to be overly simplistic, as well as the experiments. This makes it difficult for me to connect the proposed theoretical framework with the motivation described in the introduction.\n- the paper is very difficult to follow and needs to be clarified for the audiences from the general AI community.\n\nQUESTIONS:\nCan the authors add some more details to clarify the experiments? For example, the research questions to be tested in support of the motivation of the paper, the hypothesis, and clarify the results analysis according to these questions.\n\nLIMITATIONS:\nYes the authors addressed limitation in Appendix B. I do not see any negative societal impact.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-06-24"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this paper the authors formulate a framework of responsible AI (RAI) games, which represents a variety of societal effects of AI as minimizing worst-case loss over a set of predefined distributions. The authors cover their framework and present a number of related definitions and propositions. They then present two algorithms for attempting to solve these games. They end by briefly covering results comparing their approach with a variety of baselines across a variety of common tasks/datasets. \n\nEdit: Based on the author's rebuttal and our discussion I have increased my initial review to a weak accept.\n\nSTRENGTHS:\nSection 2-4 are definitely the highlights of the paper. They give a well-written argument for the utility and implications of the RAI framework. Outside of these, the clarity of the introduction and coverage of related work are also both strong.\n\nWEAKNESSES:\nThere are three primary weaknesses in the current draft of the paper. First, the authors chose to use minimizing worst-case loss as a sole target for all societal effects of AI. However, this is not the only computational representation of these effects. For example, fairness is often defined in terms of symmetry across classes, which is not necessarily identical [1]. In general, this represents only one particular perspective on a subset of these societal effects. The paper would be improved with acknowledgement of this and direct discussion of what definitions are being used and why.\n\nThe second weakness is the relatively weak results. The performance of the RAI approaches appears to be very close to the baselines in all cases. This doesn't match the claim of \"solving responsible AI games\". The earlier text in the paper generally suggests an expectation of much stronger results. There's also an odd claim that larger models are limiting, when these are very commonly used models. Relatedly, for the third weakness, there is a lack of discussion around the results, just three bullet points and a two sentence conclusion. Reducing Sections 2-4 could allow for a better contextualization of the results to clarify if they are stronger than they appear. Otherwise, the proposed algorithms do not seem to be good fits for the RAI framework. \n\n1. van den Brink, René. \"An axiomatization of the Shapley value using a fairness property.\" International Journal of Game Theory 30 (2002): 309-319.\n\nQUESTIONS:\n1. What was the inclusion/exclusion criteria for the representations of AI societal effects not covered by the author's framework?\n2. What is the reason that the performance of the RAI approaches are so similar to the baselines?\n\nLIMITATIONS:\nThe paper does not appear to have any negative societal impacts, except perhaps overstating its contributions leading to overuse.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-05"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper demonstrates originality in its approach to proposing a general framework for Responsible AI (RAI) games. The authors introduce the concept of RAI risks and present algorithms based on game-play and greedy strategies to solve the RAI games. The connection between deterministic and randomized ensembles is also explored. Overall, the paper offers novel perspectives on addressing the challenges of responsible AI.\n\nSTRENGTHS:\nThe paper's contribution mainly lies in providing a comprehensive framework for studying responsible AI problems and proposing algorithms for solving RAI games. By addressing various aspects such as fairness, robustness, and safety within a unified framework, the paper offers practical solutions for building responsible AI systems.\n\nWEAKNESSES:\nIn general, the paper is well-structured but the overall presentation quality requires improvements, and there are instances where the language and explanations could be further improved for better clarity and comprehension.\n\nQUESTIONS:\nAbstract\n1. “In these problems, a learner seeks to minimize its worst-case loss over a set of predefined distributions.” This sentence could be rephrased for better clarity. “In these problems” reads a bit odd, the meaning of “predefined distributions” is not very clear.\n2. “In this work, we provide a general framework for studying these problems…” It is unclear what the authors mean by “these problems”. Are they referring to the aforementioned societal issues of AI such fairness, robustness, and safety? Are all RAI issues studied in this paper, or just the mentioned ones (fairness, robustness, and safety)?\n3. The abstract introduces the Responsible AI (RAI) games framework but does not provide a clear definition or description of what RAI games entail. It would be helpful to briefly outline the key components and principles of RAI games to give readers a better understanding of the framework being proposed.\n4. The phrase \"Empirically we demonstrate the generality and superiority of our techniques for solving several RAI problems around subpopulation shift\" could be rephrased to sound less self-promotional and more objective. Additionally, \"several RAI problems\" is too vague as RAI encompasses a broad range of issues. Furthermore, \"subpopulation shift\" is not mentioned or explained.\n\nSection I: Introduction\n5. The mention of fragmented work and optimization of different aspects is valuable, but it would be beneficial to provide references or briefly summarize the existing literature to give readers an understanding of the current state of research. This will help establish the context and importance of the proposed unified framework for responsible AI. \n6. The authors introduce RAI games as a general framework but does not clearly define what RAI games entail. It is essential to provide a concise definition or description of RAI games to ensure readers have a clear understanding of the framework being proposed from the outset.\n7. Please consider rephrasing “this is a computationally difficult game that need not even have a Nash equilibrium” for better clarity.\n8. The term \"population risks\" is introduced without a clear context or explanation.\n\nSection II: Problem Setup and Background\n9. Appendix C is mentioned as a location for related work. But it would be beneficial to include at least a brief summary or mention of some of the related works in this section itself. This will allow readers to gain insights into the existing literature without having to refer to the appendix separately.\n\nLIMITATIONS:\nThe authors identify the broader impact and limitations of this work in the Appendix. The limitations include inadequate evaluation, limited generalization, and the need for handling corruptions in training data.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-07"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper looks at different notions of Responsible AI (RAI) and formulates aspects such as fairness and tail-risk as weights on the samples.  The paper addresses the fact that work on RAI often only focuses on improving one of these aspects, which has been shown to worsen other aspects in some cases.  To simultaneously address these multiple aspects of RAI, the paper formulates the problem as a min-max problem, where the minimization is over an ensemble of models and the maximization is over different weight sets, with each weight set corresponding to a different aspect of RAI (e.g. subgroup performance).  The paper proposes two solutions to the min-max problem, one through game playing and alternating between maximizing the weights and minimizing over the models, and the other through a greedy descent algorithm.  The paper derives convergence bounds and contains empirical analyses showing that their method is superior to baseline methods such as Group DRO and AdaBoos for various settings, such as domain-oblivious, domain-aware, and partially domain aware.\n\nSTRENGTHS:\nThe paper presents an original formulation to solve the problem of simultaneously addressing different aspects of responsible AI such as fairness and robustness.  This problem is interesting and relevant to the general field of algorithmic fairness and responsible AI.  Concrete examples such as those found in Table 1 help make the formulation easier to understand, and the paper does a good job of balancing mathematical proofs with empirical results to support the usefulness of the proposed solution.\n\nWEAKNESSES:\nIn section 4, the introduction of deterministic and randomized ensembles could benefit from an example to help make the section easier to understand.  This section is pretty central to the problem formulation and so more wording could be devoted to this section.\n\nFor the experiments section, it did not seem that multiple RAI objectives were addressed, just subpopulation shift with various levels of awareness of the subpopulation.  It would be nice to see how the algorithm performs when more objectives are introduced.\n\nThe results table, Table 2, lacks confidence intervals, so it is hard to assess the significance of the improvements.\n\nQUESTIONS:\nDid the authors consider how the min-max formulation compares to treading the different RAI aspects as a multi-objective function in the loss?\n\nHow does the min-max approach compare to the baselines in terms of compute resources/time to solving the problem?\n\nLIMITATIONS:\nThe authors discuss some of the theoretical limitations of relaxing the problem from a deterministic ensemble to a randomized ensemble, as well as the potential for improving the bounds on the convergence rate.\n\nThere did not seem to be any discussions on the practical limitations of implementing this approach.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-27"}