On-Demand Sampling: Learning Optimally from Multiple Distributions
Anonymous Author(s)
Abstract
Social and real-world considerations such as robustness, fairness, social welfareand multi-agent tradeoffs have given rise to multi-distribution learning paradigms,such as collaborative [4], group distributionally robust [22], and fair federatedlearning [15].
In each of these settings, a learner seeks to minimize its worstcase loss over a set of n predefined distributions, while using as few samples aspossible.
In this paper, we establish the optimal sample complexity of these learningparadigms and give algorithms that meet this sample complexity.
Importantly, oursample complexity bounds exceed that of the sample complexity of learning asingle distribution only by an additive factor of n log( n ) ε.
These improve upon thebest known sample complexity of agnostic federated learning by Mohri et al.
[15]by a multiplicative factor of n , the sample complexity of collaborative learning by
Nguyen and Zakynthinou [17] by a multiplicative factor log n ε 3 , and give the firstsample complexity bounds for the group DRO objective of Sagawa et al.
Toachieve optimal sample complexity, our algorithms learn to sample and learn fromdistributions on demand.
Our algorithm design and analysis is enabled by ourextensions of stochastic optimization techniques for solving stochastic zero-sumgames.
In particular, we contribute variants of Stochastic Mirror Descent that cantrade off between players’ access to cheap one-off samples or more expensivereusable ones.
Introduction
Pervasive needs for robustness, fairness and multi-agent collaboration in learning have given rise tomulti-distribution learning paradigms (e.g., [4, 22, 15, 9]).
In these settings, we seek to learn a modelthat performs well on any distribution in a pre-defined set of interest.
For fairness considerations,these distributions may represent heterogeneous populations of different protected or socio-economicattributes; in robustness applications, they may capture a learner’s uncertainty regarding the trueunderlying task; and in muti-agent collaborative or federated applications, they may represent agentspecific learning tasks.
In these applications, the performance and optimality of a model is measuredby its worst test-time performance on a distribution in the set.
We are concerned with this fundamentalproblem of designing sample-efficient multi-distribution learning algorithms.
The sample complexity of multi-distribution learning differs from that of learning a single distributionin several ways.
On one hand, learning tasks of varying difficulty require different numbers ofsamples.
On the other hand, similarity or overlap among learning tasks may obviate the need tosample from some distributions.
This makes the use of a fixed per-distribution sample budgethighly inefficient and suggests that optimal multi-distribution learning algorithms should sampleon demand .
That is, algorithms should take additional samples whenever they need them andfrom whichever distribution they want them.
On-demand sampling is especially appropriate when
[Table]
some population data may be scarce to start with (as in fairness mechanisms in which samples areamended [19]); when the designer can actively perturb datasets towards rare or atypical instances(such as in robustness applications [13, 25]); or when sample sets represent agents’ contributions to aninteractive multi-agent system [15, 5].
Blum et al.
[4] demonstrated the benefit of on-demand samplingin the collaborative learning setting, where all data distributions are realizable with respect to thesame target classifier.
This line of work established that learning n distributions on-demand takes(cid:101) O (log( n )) times the sample complexity of learning a single realizable distribution [4, 6, 17], whereasrelying on batched uniform convergence takes (cid:101) Ω ( n ) times that of learning a single distribution [4].
However, beyond the realizable setting, the best known multi-distribution learning results fall shortof this promise: existing on-demand sample complexity bounds for agnostic collaborative learninghave highly suboptimal dependence on ε , requiring (cid:101) O (cid:0) log( n ) /ε 3 (cid:1) times the sample complexityof agnostically learning a single distribution [17].
On the other hand, agnostic federated learningbounds [15] have been studied only on algorithms that sample in one large batch and thus require(cid:101) Ω ( n ) times the sample complexity of learning a single task.
Moreover, the test-time performance ofsome key multi-distribution methods, such as group distributionally robust optimization [22], havenot been studied from a theoretical perspective before.
In this paper, we give a general framework for obtaining optimal and on-demand sample complexityfor three multi-distribution learning settings.
Table 1 summarizes our results.
All three settingsconsider a set D of n distributions and a model class H .
They evaluate the performance of a model h(or a distribution over models) by its worst-case performance, max D ∈D loss D ( h ) .
As a benchmark,they consider the worst-case loss of the best model, i.e., OPT = min h ∗ ∈H max D ∈D loss D ( h ∗ ) .
Importantly, all of our sample complexity upper bounds demonstrate only an additive increase ofε − 2 n log( n/δ ) over the sample complexity of learning a single task , compared to the multiplicativefactor increase required by existing works. - Collaborative learning of Blum et al.
[4]: For agnostic collaborative learning, our Theorem 4.1 gives a randomized and a deterministic model that achieve performance guaranteesof OPT + ε and 2 OPT + ε , respectively.
Our algorithms have an optimal sample complexity of
O (cid:0)
ε 2 (log( | H | ) + n log( nδ )) (cid:1) .
This improves upon the work of Nguyen and Zakynthinou [17]in two ways.
First, it provides error bounds of OPT + ε for randomized classifiers, where only2 OPT + ε was previously established.
Second, it improves the upper bound of Nguyen and
Zakynthinou [17] by a multiplicative factor of log( n ) /ε 3 .
In Theorem 4.3, we give a matchinglower bound on this sample complexity, thereby establishing the optimality of our algorithms.- Group distributionally robust learning (group DRO) of Sagawa et al.
[22]: For group DRO, weconsider a convex and compact model space H .
Our Theorem 5.1 studies a model that achievesan OPT + ε guarantee on the worst-case test-time performance of the model with an on-demandsample complexity of O (cid:0) [-
-]1 ε 2 ( D H + n log( nδ ) (cid:1) .
Our results also imply a high-probability bound for the convergence of group DRO training error that improves upon the (expected) convergenceguarantees of Sagawa et al.
[22] by a factor of n .-
Agnostic federated learning of [15]: For agnostic federated learning, we consider a finite classof hypotheses.
Our Theorems 4.1 and 5.1 show that on-demand sampling can accelerate thegeneralization of agnostic federated learning by a factor of n compared to batch results establishedby Mohri et al.
Our results also imply matching high-probability bounds to Mohri et al.
[15]on the convergence of the training error in the batched setting.
To achieve these results, we contribute new insights and techniques for solving stochastic zero-sumgames with sources of randomization that differ in both cost and quality.
We frame the multidistribution learning problems as a stochastic zero-sum game with uncertain payoffs and utilizestochastic mirror descent and a variational perspective to solve the game.
In this case, the maximizingplayer can be interpreted as a weight vector for distributions D , specifying from which distributionsfuture on-demand samples should be taken.
These on-demand samples form a stochastic gradient forthe players.
However, the quality of these estimators, the number of samples needed for them, andwhether they can be reused later on, differs between the two players.
We extend the Stochastic Mirror
Descent framework to optimally trade off these asymmetric needs for samples.
In Section 3 we givean overview of this approach and its technical challenges and contributions.
1.1 Related Work
Learning models.
Three independent lines of work study multi-distribution learning, with differentmotivating applications.
Collaborative learning interprets multiple distributions as players that eachseek to learn a model with low error on their data distributions [4, 17, 6].
Agnostic federated learninginterprets these distributions as clients in a federated learning system [15].
Group distributionallyrobust optimization interprets these distributions as data attributes or sources that a learner shouldavoid linking spuriously to labels [12, 22, 23].
Formally, these learning objectives are all equivalentbut have been studied from different points of view and with different technical tools.
Existing work on group DRO has assumed that data is pre-collected and has studied the convergenceof multi-distribution training error.
The agnostic federated learning literature has studied a singlebatch approach and derived data-dependent generalization bounds that suggest how much of the batchshould be collected from each distribution.
Finally, the collaborative learning literature has studiedan on-demand framework for collecting data from each distribution.
This approach also relates to aline of work on multi-source learning and domain adaptation [3, 14].
Stochastic game equilibria.
Our approach relates to a line of research on using online algorithmsto find min-max equilibria by playing no-regret algorithms against one another [21, 10, 18, 7, 8].
Onesuch method, online mirror descent (OMD), can also approximate minima of convex functions withhigh probability using noisy first-order information [20, 16, 2].
This allows OMD to efficiently findmin-max equilibria even in stochastic convex-concave zero-sum games [11].
We bring these onlinelearning tools to bear on the problem of finding equilibria in robust optimization formulations.
Preliminaries
Let X be an instance space, Y a label space, and Z = X ×Y a space of datapoints.
A data distribution
D is a joint probability distribution over Z .
We consider a hypothesis class H of a subset of functionsmapping X to Y .
We work with loss functions ℓ : H ×Z → [0 , 1] that measure the loss of hypothesish on data point z ∈ Z .
When Y = { 0 , 1 } , ℓ is the misclassification error.
We denote the expectedloss, i.e. risk, of a hypothesis h ∈ H under a data distribution D ∈ D by:loss D ( h ) :=
E ( x,y ) ∼ D [ ℓ ( h, ( x, y ))] .
For a distribution over the hypothesis class, p ∈ ∆ H , and a distribution over data distributions,q ∈ ∆ D , we refer to their expected loss by loss q ( p ) :
=
E D ∼ q [ E h ∼ p [loss D ( h )]] .
Collaborative Learning.
We will use the collaborative PAC learning model of Blum et al.
[4]and its agnostic extensions by Nguyen and Zakynthinou [17].
The overall goal of this setting is toguarantee low loss for every distribution in a collection of distributions.
Formally, we consider a set of data distributions D :=
{ D 1 , . . .
, D n } .
The goal of the learner is to learn a hypothesis h such that,with probability 1 − δ ,max D ∈D loss D ( h ) ≤ OPT + ε, where OPT :=
min
h ∈H max D ∈D loss D ( h ) .
[Equation]
Group Distribution Robustness.
We will also study the closely related setting of group distributionally robust optimization (Group DRO) of Sagawa et al.
Formally, the group DRO settingconsiders a model set Θ that is a convex compact subset of the Euclidean space and a convex lossfunction ℓ : Θ × Z → [0 , 1] that is assumed to be differentiable over Θ .
Given a set of datadistributions D := { D 1 , . . .
, D n } , the learner seeks a model θ ∈ Θ , such that, with probability 1 − δ ,
[Equation]
OPT + ε,
[Equation]
R-OPT :=
min θ ∈ Θ
[Equation]
D
[Equation]
There is a close relationship between the Group DRO setting and collaborative learning.
In particular,when Θ = ∆( H ) and H is finite, the two goals are analogous but with two exceptions: first, the Group
DRO could return a distribution over functions while collaborative learning requires the solution tobe a deterministic function, and second, allowing for randomized hypothesis leads to R-OPT beingpotentially more competitive than OPT.
We note that the group DRO setting is equivalent to theagnostic federated learning framework of [15], thus our results for DRO extend to that setting as well.
Sample complexity.
We are interested in the design of algorithms that achieve the above goalswhile using smallest number of samples from distributions D 1 , . . .
, D n .
We formalize the samplecomplexity by the total number of calls made to example oracles EX ( D i ) .
Each call EX ( D ) producesan i.i.d.
sample from D .
We note that these example oracles also allow us to sample from any mixturedistribution q ∈ ∆ D , e.g., by first selecting a D i according to the mixture and then calling EX ( D i ) .
2.1 Technical Background
We will use tools and definitions from the literature on zero-sum games and no-regret learningthroughout the paper.
This section provides a brief overview of these concepts.
Zero-Sum Games.
A finite two-player zero-sum game is described by the tuple ( A - , A + , ϕ ) where
A - = { 1 , . . . , n } and A
+ = { 1 , . . .
, m } are finite sets of actions and where ϕ : A - × A + → [0 , C ] .
In this game, the players choose mixed strategies over actions sets.
These are distributions that aredenoted by a vector of probabilities p
∈ ∆ A - and q ∈ ∆ A + .
The expected payoff of mixed strategiesis denoted by ϕ ( p, q ) =
E i ∼ p,j ∼ q [ ϕ ( i, j )] .
The goal of the minimizing player is to minimize thisexpected payoff and the maximizer seeks to maximize the expected payoff; that is, to solve
[Equation]
A pair ( p, q ) that solves this optimization problem is called a min-max equilibrium .
Similarly, asolution is called an ε -min-max equilibrium if neither player can unilaterally improve their objectiveby more than ε .
Formally, ( p, q ) is an ε -min-max equilibrium if both players’ regrets are at most ε ,i.e., Reg-Min ( p, q ) :
= ϕ ( p, q ) − min i ∗ ∈
A - ϕ ( i ∗ , q ) ≤ ε and Reg-Max ( p, q ) :
= max j ∗ ∈
A + ϕ ( p, j ∗ ) −ϕ ( p, q ) ≤ ε.
We will next describe methods that find ε -min-max equilibria by finding solutions ( p, q )for which Reg-Min ( p, q ) + Reg-Max ( p, q ) is at most ε .
We describe a more general formulation forconvex-concave zero-sum games in Appendix A.1 which we will use for the Group DRO problem.
No-Regret Learning.
We consider an online setting where an arbitrary set of operators ,g (1) , . . .
g ( T ) ∈
E ∗ , is revealed sequentially to a learner who must choose a matching sequenceof actions, w (1) , . . .
w ( T ) , from a convex compact set Z ⊆ E .
Here, E and E ∗ respectively refer toan arbitrary Euclidean space and its dual.
We focus on a setting where an online learner commitsto action w ( t ) ∈
Z before seeing g ( t ) , g ( t +1) , . . .
and aims to achieve vanishing variational error
Errv ( w (1: T ) ) defined by
[Equation]
We will denote no-regret algorithms by their update rule Q : { Z × E ∗ } → Z , where { Z × E ∗ }denotes the space of arbitrary length sequences of action-operator pairs.
Given a history sequencew (1) , . . .
, w ( t )
∈
Z and operator sequence g (1) , . . .
, g ( t )
∈
E ∗ , the algorithm returns w ( t +1) =
Q (cid:0)(cid:8) w (1) , g (1) (cid:9) , . .
, (cid:8) w ( t ) , g ( t ) (cid:9)(cid:1) .
When the history is clear from context, we write w ( t +1) =
Q (cid:0) w ( t ) , g ( t ) (cid:1) as shorthand.
For the particular case where Z = ∆ n is a probability simplex, one suchalgorithm is Exponential Gradient Descent (also known as Hedge):
[Equation]
where η is a user-defined step size, and w 1 is a user-defined initial iterate.
By default, we takew 1 = (cid:2) 1 n (cid:3) n .
The following lemma is a classical result on the variational error of exponential gradient descent.
Lemma 2.1 ([24]) .
Let g (1) , . .
, g ( T ) ∈
R n and Z
= ∆ n .
Further assume (cid:13)(cid:13) g
( t ) (cid:13)(cid:13) ∞ ≤ C for alltimesteps t = 1 , . .
, T .
Choosing η = log n/ √ T , after T iterations of exponential gradient descent,the output { w } Tt =1 satisfies,
[Equation]
Technical Overview of Our Approach
In this section, we provide an overview of our technical approach for addressing the sample complexityof collaborative learning and group DRO problems.
In later sections, we will refer to the approachoutlined in this section to sketch proofs and design algorithms.
We will focus our exposition oncollaborative learning and briefly indicate how the same approach applies to the group DRO setting.
At a high level, we first frame collaborative learning as a zero-sum game with uncertain payoffsand aim to use a variational perspective to learn its minmax equilibrium.
We specifically choosethe variational perspective (instead of an arbitrary online learning approach), since it allows us tolinearize the effect of uncertain payoffs on the resulting error.
We then use stochastic gradients to solvethe variational problem.
Our stochastic gradients will rely on i.i.d.
samples from the distributionsto estimate gradients both with respect to distributions over H and mixtures over D but with anasymmetric bound on the bias and variance of the estimates.
Along the way, we develop toolsand formalisms that handle the asymmetric cost of stochastic gradients and obtain optimal samplecomplexity results.
We now address these steps in more detail.
Collaborative Learning as Zero-Sum Games.
When the hypothesis class H is finite, the collaborative learning problem with distribution set D corresponds to a zero-sum game ( A - , A + , ϕ ) with
A - = H , A + = D , ϕ ( i, j ) = loss j ( i ) , such that the value of the min-max solution is equivalent to
R-OPT.
It is not hard to see that any ε -min-max equilibrium ( p, q ) of this game corresponds to a 2 εcollaborative learning solution, i.e.,
[Equation]
This enables us to use tools that have been developed for solving zero-sum games in order to addresscollaborative learning and group DRO settings.
We will use a similar construction when hypothesisclass H has finite VC dimension, where A - will instead refer to an appropriate ε -cover of H .
Using VI to deal with Payoff Uncertainty.
A sufficient condition for minimizing regret, and thusfinding ε -min-max equilibrium, is minimizing the variational error (Equation 3).
In particular, forany finite zero-sum game ( A - , A + , ϕ ) , defining Z = [∆ A - , ∆ A + ] and operators
[Equation]
ensures that variational error provides an upper bound on regret: Errv ( w (1: T ) ) ≥ Reg-Min ( p, q ) +
Reg-Max ( p, q ) , where w = ( p, q ) (see Lemma B.1).
In collaborative learning, when p ( t ) is the min-player’s distribution over hypotheses and q ( t ) is max-player’s distribution over the mixtures, thegradient vectors refer to
[Equation]
In the collaborative learning setting, we can only create noisy estimates (cid:98) g for these gradients fromsamples.
This is where no-regret algorithms that minimize variational error become advantageous.
By linearizing the effect of noise, ε ( t ) := g ( t ) − (cid:98) g ( t ) , they decompose the variational error into thetraining and generalization error as follows
[Equation]
In contrast, generic no-regret algorithms that do not solve the variational inequality (e.g., whenone player plays Hedge and another plays clairvoyant best-response as used in existing work incollaborative learning due to Blum et al.
[4], Nguyen and Zakynthinou [17], Chen et al.
[6]) nest thegeneralization and training errors which leads to a multiplicative increase in sample complexity.
Leveraging Noisy Stochastic Gradients.
We will work with stochastic estimators of g .
These arefunctions (cid:98) g : ξ × ∆ A - × ∆ A + of some external source of randomness, ξ ∈ ξ , and a strategy profileof interest.
For collaborative learning, the randomness source ξ is an i.i.d.-sampled data point froman appropriate mixture of distributions and the estimator (cid:98) g is then the empirical loss on this sample,which is an unbiased and bounded estimator in the range of the loss function, i.e., [0 , 1] .
Interestingly, estimators of these stochastic gradients have an asymmetric need for data.
As seen in
Equation 7, the min-player’s gradient g - ( p, q ) includes the loss of every hypothesis h ∈ H for thesame data distribution q .
Therefore, an unbiased estimator (cid:98) g - ( p, q ) can be constructed from a singlecall to an example oracle EX ( q ) .
We call this source of randomness ξ q and say that its cost is r − = 1 .
While ξ q costs 1 unit, the randomness it provides is specialized to the point of inquiry, that is, itcannot be used for estimating other (cid:98) g - ( p, q ′ ) .
We call this source of randomness and its associatedunbiased estimation a locally unbiased estimator.
On the other hand, the max-player’s gradient g + ( p, q ) includes the loss of the same hypothesis p onevery distribution D ∈ D .
Therefore, an unbiased estimator (cid:98) g
+ ( p, q ) requires n samples, i.e., a callto every example
oracle EX ( D i ) .
We call this source of randomness producing n samples ξ p andsay that its cost is r + = n .
Importantly, while ξ p costs n unit, the randomness it provides can bereused for estimating other gradients, that is, it can provide an unbiased and bounded estimators forall (cid:98) g
+ ( p ′ , q ′ ) .
We call this source of randomness and its associated unbiased estimator a globallyunbiased estimator.
To emphasize the fact that this source of randomness is agnostic to ( p, q ) werefer to it by ξ ⊥ hereafter.
We refer the reader to Appendix A.2 for a more formal definition anddescription of these asymmetries.
Minimizing Regret with Asymmetric Cost.
With the goal of minimizing sample complexity inmind, it is essential that we reuse randomness ξ ⊥ across n time steps of variational algorithms.
Todo this, we introduce a stochastic variational approach in Algorithm 1 that accommodates differentsampling frequencies for the minimizing and maximizing players.
This will decouple the samplecomplexity of the minimizing agent (who requires a time horizon of at least log( A - ) ≈ log( H ) )
andthe maximizing agent.
This decoupling will lead to additive n + log( H ) sample complexity insteadof the multiplicative n log( H ) .
Algorithm 1 uses the same randomness ξ ⊥ ( a ) of cost r for estimating g + ( p t , q t ) for all t ∈ [ ar +1 , . .
. , a ( r + 1)] .
On the other hand, the algorithm uses fresh randomness ξ ( t ) of cost 1 to estimateg - ( p t , q t ) for every time step t .
We note that the total randomness cost of this algorithm is 2 t becauseiteration of the outer loop incurs 2 r cost.
Lemma 3.1.
Let ( A - , A + , ϕ ) be a finite zero-sum game.
Assume there exists ξ q ( t ) of cost 1 providinglocally unbiased estimates (cid:98) g - ( · ) and there exists ξ ⊥ ( a ) of cost r providing globally unbiased estimates(cid:98) g + ( · ) .
With probability 1 − δ , Algorithm 1 returns an ε -min-max equilibrium of the game, so long as
T [Equation] (cid:18) max (cid:26) [Equation]
|
[Equation] (cid:18)
[Equation]
Moreover, the total cost of randomness incurred by the algorithm is at most 2 t .
Output: Mixed strategy profile ( p, q )
∈
∆ A - × ∆ A + ;
Input do
Realize ξ at cost r ; //
Sample datapoints from for t ar +
1 − r ar do
Realize ξ q ( t ) at cost 1 ; //
Sample from adversary-selected distribution.
Estimate gradients: (cid:98) g ( t ) + = (cid:98) g
+ (cid:16) ξ ⊥
( a ) , p ( t ) , q ( t ) (cid:17) , (cid:98) g ( t ) - = (cid:98) g - (cid:16) ξ q ( t ) , p ( t ) , q ( t ) (cid:17) ;
Run Hedge updates: p ( t +1) =
Q hedge (cid:16) p ( t ) , (cid:98) g ( t ) + (cid:17) , q ( t +1) =
Q hedge (cid:16) q ( t ) , (cid:98) g ( t ) + (cid:17) ; end for end for
Return the uniformly mixed strategies p = 1 T (cid:80) Tt =1 p ( t ) and q = 1 T (cid:80)
Tt =
1 q ( t ) ;
Proof sketch.
Our approach uses Equation 8 to decompose the variational error into training errorand generalization error.
Since exponential gradient descent is known to bound the training error(as shown in Lemma B.5), it only remains to bound the generalization error (the second term in
Equation 3).
We note that in expectation each summand (cid:10) ε ( t ) , w ( t ) − w ∗ (cid:11) is zero.
This is becauseε ( t ) =
g ( t ) − (cid:98) g ( t ) and (cid:98) g ( t ) are unbiased estimators.
Therefore, the sum of these terms has an intuitivemartingale interpretation and could be bounded by the Azuma-Hoeffding inequality.
There is a subtlety here, however.
When we reuse the maximizing player’s randomness over rrounds, we create correlations between these terms in the generalization error that cannot be directlyaccommodated by a martingale.
The trick here is to note that these correlations are entirely containedin r -length periods.
So, we can partition our sequence to r martingales and bound each one.
Thiscompletes the proof.
See Appendix B.1 for detailed proof of this lemma.
Derandomization.
The ε -min-max equilibria ( p, q ) returned by Exponentiated Gradient Descentgives a probability distribution p over the hypothesis class that achieves the collaborative learningbound.
To obtain a deterministic hypothesis, we can instead work with h Majp whose predictionsare p -weighted majority votes over the hypotheses.
As stated below, the error of this deterministicclassifier is approximately bounded by the expected error of p .
Lemma 3.2.
For any p ∈ ∆ H , max D ∈D loss D ( h Majp ) ≤ 2 max D ∈D loss D ( p ) .
This lemma in particular implies that for any ε -min-max equilibria ( p, q ) , we have
[Equation]
Collaborative Learning Bounds
In this section, we characterize the sample complexity of collaborative learning by providing tightupper and lower bounds for this problem.
4.1 Sample Complexity Upper Bounds
We are now prepared to describe our collaborative learning algorithm and guarantees, using the toolswe developed in Section 3.
Algorithm 2 is a direct application of Algorithm 1 to a zero-sum gamewith action sets A - = H , A + = D and payoff ϕ ( h, D ) =
loss D ( h ) .
Here, ξ q ( t ) makes one call to
EX ( q ( t ) ) and ξ ⊥ ( a ) makes one call to EX ( D ) for each D ∈ D .
In other words, Algorithm 2 constructsdistributions p ( t ) ∈
∆ H and q ( t ) ∈ ∆ D by running the Hedge update.
The gradient estimatorsused by Hedge are the empirical losses on a set of independent random variables.
In particular, theminimizing player uses gradients ℓ ( h, z ( t ) ) for all h ∈ H for a single sample z ( t ) ∼ EX ( q ( t ) )
andthe maximizing player uses gradients ℓ ( p ( t ) , z aD ) for all distributions D ∈ D where a single samplez aD ∼ EX ( D ) is drawn per distribution and is reused for all time steps t ∈
[( a − 1) n + 1 , . .
. , an ] .
Our main result in this section bounds the sample complexity of Algorithm 2.
Theorem 4.1.
For any finite hypothesis class H and unknown set of distributions D , with probability1 − δ , Algorithm 2 returns a distribution
p ∈
∆ H such that
[Equation]
using a number of samples that is O (cid:16) log |H| + n log( n/δ ) ε(cid:17) .
Proof sketch.
By construction, Lemma 3.1 guarantees that with probability at least 1 − δ , the pair( p, q ) is an ε/ 2 -min-max equilibrium for the corresponding zero-sum game.
As shown by Equation 5,p is a randomized classifier that meets the collaborative learning objective, i.e., its expected worst-caseerror is OPT + ε .
By Lemma 3.2, the corresponding deterministic classifier h Majp has worst-caseerror of 2 OPT + ε .
This bounds the error of the resulting classifier.
To bound the sample complexity, Lemma 3.1 shows that the randomness cost of Algorithm 1 isat most 2 t .
Since the cost of randomness is exactly the total number of samples we take from ourexample oracles, the total sample complexity of Algorithm 2 is 2 t ∈ O (cid:16) log |H| + n log( n/δ ) ε(cid:17) .
A similar result holds for the case of infinite hypothesis classes of bounded VC dimension.
In thiscase, one can instead run Algorithm 2 with a hypothesis class H ′ that is an ε -net with respect toevery distribution in D .
We note that such ε -nets of size nε − 2VCD( H ) necessarily exist (see, e.g.,[1]); for example, the union of ε -nets with respect to each distribution D ∈ D .
When such H ′ isknown in advance, we may run Algorithm 2 with H ′ and incur a sample complexity that now replaceslog( |H ′ | )
=
O ( d log(1 /ε )) in the sample complexity of Theorem 4.1.
We remark that it is not strictly necessary to know an ε -net in advance.
Instead, one can compute anet from samples or from other information about distributions in D .
In Appendix D, we explorea range of assumptions that allow us to compute such an ε -net from samples, without incurring asignificant increase in the sample complexity of Theorem 4.1.
As an example, here we mention twosuch assumptions.
Assumption 1: we know the marginal distribution for all D ∈ D , or a weaker
Assumption 2: we have access to n marginal distributions P 1 , . .
, P n such that for all x ∈ X ,d i ( A ) ≤ p i ( A )
poly(1 /ε, VCD( H ) , n ) for all A ⊆ X , where p i and d i are the densities of P i and
D i , respectively.
These assumptions allow one to construct ε -nets of small size, e.g., by projecting Hon a sufficiently large set of random feature vectors generated from distributions P i .
We refer thereader to Appendix D for more detail on how these assumptions can be used to construct ε -nets.
Theorem 4.2.
For any H of VC dimension d and unknown set of distributions D for which Assumption1 or 2 is met, there is an algorithm that, with probability 1 − δ , returns a distribution
p ∈
∆ H with,
[Equation]
and [Equation] using a number of samples that is O (cid:16)
[Equation]
We end this subsection with two remarks about our sample complexity upper bound.
Remark 4.1.
Theorem 4.1 improves over the best-known sample complexity for agnostic collaborativelearning by Nguyen and Zakynthinou [17] in two ways.
First, it provides OPT + ε for randomizedclassifiers whereas Nguyen and Zakynthinou [17] gave a 2 OPT + ε bound.
Second, it improvestheir sample complexity of O (cid:0) 1 ε 5 (cid:0) log( n ) log( |H| ) log (cid:0) 1 ε (cid:1) + n log (cid:0) nδ (cid:1)(cid:1)(cid:1) by a multiplicative factorof 1 ε 3 log ( n ) log (cid:0) 1 ε (cid:1) .
Remark 4.2.
For constants ε and δ , our sample complexity of O (log( |H| ) + n log n ) appears toviolate the lower bound of Ω (log( |H| ) log n + n log log |H| ) due to Chen, Zhang, and Zhou [6].
This discrepancy is due to a small error in the proof of that lower bound, which we have verified inprivate communications with the authors.
In the next subsection, we give lower bounds on the samplecomplexity of collaborative learning that match our upper bounds.
4.2 Sample Complexity Lower Bound
We now provide matching lower bounds for agnostic collaborative learning.
Our lower boundshold for collaborative learning algorithms obtaining error of R-OPT + ε , using a randomized ordeterministic hypothesis.
We call an algorithm an ( ε, δ ) -collaborative learning algorithm if for anycollaborative instances it attains an error of R-OPT + ε with probability at least 1 − δ .
Theorem 4.3.
Take any n, d ∈ Z
+ , ε, δ ∈ (0 , 1 / 8) , and ( ε, δ ) -collaborative learning algorithm A .
There exists a collaborative learning problem ( H , D ) with |D| = n and | h | = 2 d , on which A takesat least Ω (cid:0) {+
+} ε 2 (log |H| + |D| log(min {|D| , log |H|} /δ ))
(cid:1) samples in expectation.
Proof sketch.
We defer the formal proof of this theorem to Appendix B.2 and sketch the main ideashere.
We use X = { 1 , . . .
, d } , Y = { + , −} , and let H be the set of all functions X → Y .
Ourconstruction combines two types of hard distributions.
We describe the ideas for the case of n = d .
First, we use a hard construction for agnostic learning of hypothesis classes with VC dimension das the distribution of one of the agents.
This give us the Ω (cid:0) log( |H| ) /ε
2 (cid:1) part of the lower bound.
Second, we construct n hard instances each of VC dimension 1 on n independent points.
Since thelearning algorithms has to solve each problem it has to incur a loss of n log( n/δ ) /ε 2 .
Group DRO and Agnostic Federated Learning
The results we describe in the collaborative learning setting can be generalized to the group DROsetting, and equivalently, agnostic federated learning.
Theorem 5.1.
Consider any group distributionally robust problem (Θ , D ) with a convex compactparameter space Θ of Bregman radius D Θ = min x ∈ Θ max
y ∈ Θ D ( y || x ) , and convex loss functionℓ : Θ × Z → [0 , C ] .
A variant of Algorithm 2 (in particular Algorithm 5 in Appendix B.3),returns θ ∈ Θ such that max D ∈D
E z ∼ D
[ ℓ ( θ, z )] ≤ R-OPT + ε, using a number of samples that is
[Equation]
The proof of this lemma is deferred to Appendix B.3 and is similar to the proof of Theorem 4.1except that it uses a generalization of Lemma 3.1 for general convex-concave games.
This theoremestablishes a generalization bound for the problem of group distributionally robust optimization [22]and improves, by a factor of n , existing sample complexity bounds for agnostic federated learning[15].
This improvement is attained by sampling data on-demand, whereas [15] only chooses a fixeddistribution over groups/clients to sample from; this highlights the importance of adapting one’ssampling strategy on-the-fly when learning robust models.
Another important question is how fast the training error of stochastic gradient descent convergesfor the group DRO/AFL settings and was considered by Sagawa et al.
We can transfer ourgeneralization guarantees for on-demand settings into batch settings and achieve the followingcorollary, which improves on the convergence guarantees of Sagawa et al.
[22] by a factor of n .
Corollary 5.2.
Under the same assumptions of Theorem 5.1, we give a procedure (see Appendix B.3)that minimizes GDRO/AFL training error within ε of R-OPT with probability at least 1 − δ in fewersamples than O (cid:16) D Θ C 2 + nC 2 log( n/δ ) ε(cid:17) . Checklist For all authors...( a
) Do the main claims made in the abstract and introduction accurately reflect the paper’scontributions and scope?
[Yes](
b
) Did you describe the limitations of your work?
[Yes]
(c
) Did you discuss any potential negative societal impacts of your work
[N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms tothem
[Yes
If you are including
theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes]
(b) Did you include complete proofs of all theoretical results?
[Yes]
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi
-mental
results (either in the supplemental
material
or as a URL)?
[Yes]
(b) Did you specify all the training details (e
g., data splits, hyperparameters,
how they
[Table]
were chosen)?
[Yes]
[Table]
(c) Did you report error bars (e
g., with respect to the random seed after running experi-
[Table]
ments multiple times)?
[Yes]
(d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
[Yes]
If you are using existing assets (e
g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes]
(b) Did you mention the license of the assets?
[Yes]
(c) Did you include any new assets either in the supplemental material or as a URL?
[Yes]
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[Yes]
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[Yes]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if
applicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount
spent on participant compensation?
[N/A]