Multi-agent Performative Prediction with Greedy Deployment and Consensus Seeking Agents
Anonymous Author(s)
AffiliationAddress email
Abstract
We consider a scenario where multiple agents are learning a common decisionvector from data which can be influenced by the agents’ decisions.
This leads tothe problem of multi-agent performative prediction (Multi-PfD).
In this paper, weformulate Multi-PfD as a decentralized optimization problem that minimizes a sumof loss functions, where each loss function is based on a distribution influencedby the local decision vector.
We first prove the necessary and sufficient conditionfor the Multi-PfD problem to admit a unique multi-agent performative stable(Multi-PS) solution.
We show that enforcing consensus leads to a laxer conditionfor existence of Multi-PS solution with respect to the distributions’ sensitivities,compared to the single agent case.
Then, we study a decentralized extension tothe greedy deployment scheme [Mendler-Dünner et al., 2020], called the DSGD-GDscheme.
We show that DSGD-GD converges to the Multi-PS solution and analyze itsnon-asymptotic convergence rate.
Numerical results validate our analysis.
Introduction
Traditional learning/prediction problems are often formulated with the assumption that data follows astatic distribution, from which a decision vector is sought by solving a corresponding optimizationproblem.
While this assumption holds for ‘stationary’ tasks such as image classification, many realworld tasks are dynamical, involving data that could be influenced by decisions.
In the latter case, thepredictions are said to be performative .
Example scenarios include when users are strategic [Hardtet al., 2016, Dong et al., 2018, Kleinberg and Raghavan, 2020] such as in training an E-mail spamclassifier, where users (including spammers) can adapt to the classifier’s rules to evade detection.
A common way to model performative prediction is via incorporating decision-dependent distributionin formulating the learning problem [Quiñonero-Candela et al., 2008].
Since the pioneering work by
Perdomo et al.
[2020], there is a growing literature in analyzing the performative prediction problem,which studied the convergence of learning algorithms to stable point of performative prediction[Mendler-Dünner et al., 2020, Drusvyatskiy and Xiao, 2020, Brown et al., 2022, Li and Wai, 2022,
Wood et al., 2021], algorithms to find stationary solution of performative risk [Izzo et al., 2021, 2022,
Miller et al., 2021, Ray et al., 2022], update timescales of agent [Zrnic et al., 2021], etc.
Most of the existing works are focused on the single agent (learner) setting.
In this paper, weconcentrate on the multi-agent performative prediction (Multi-PfD) problem.
Here, the agents areconsensus-seeking who seek a common decision vector that minimizes the sum of loss functionsthrough communications on a graph/network.
Such setup arises naturally where each agent acquiresdata from different subset/population of users, and they desire to seek a common decision vector formaximal generalization performance.
For the example of training an E-mail spam classifier, eachagent represents a regional server providing services to a subset of users.
These users are in generaldifferent and may react to their serving agents differently, leading to heterogeneous and decision dependent data distributions.
Notably, our setup differs from the recent works in [Narang et al., 2022,
Piliouras and Yu, 2022] which consider a game theoretical setup of Multi-PfD; see Fig. 1 and §2.
For the algorithmic model, we study a decentralized extension of the greedy deployment scheme[Mendler-Dünner et al., 2020], particularly we concentrate on the decentralized stochastic gradient(DSGD)
[Lian et al., 2017] algorithm with greedy deployment ( DSGD-GD ).
Overall, the DSGD-GDscheme emulates a scenario where agents apply DSGD, a standard decentralized learning algorithm,while being agnostic to the performative nature of the prediction problem.
In particular, upon eachiteration, the non-converged local decisions will be deployed directly.
Then, agents optimize theirdecision using stochastic gradient constructed from data with distribution shifted by local decision.
Tackling Multi-PfD with the DSGD-GD scheme yields a practical scenario of performative predictionin the multi-agent setting.
We inquire the open questions: When will the Multi-PfD problem admit astable and consensual solution?
If so, how fast does it take for DSGD-GD to converge to such solution?
Does the limited communication in DSGD-GD impair its convergence rate?
We provide affirmativeanswers to the above.
Our idea hinges on adopting the concept of performative stable solution[Perdomo et al., 2020], which is a fixed point solution resulted from the interplay between agent(s)and data that react to the agent(s)’ decisions.
In particular, we focus on analyzing the multi-agentperformative stable (Multi-PS) solution in our problem.
To our best knowledge, this paper provides the first study and analysis of Multi-PfD with consensusseeking agents via a practical DSGD-GD scheme.
We highlight the following key contributions:
• We provide a necessary and sufficient condition on the sensitivity of decision dependent data
distributions for the existence and uniqueness of the Multi-PS solution.
An interesting finding is
that the Multi-PS solution exists even if some of the performative prediction problems of individual
agents are unstable.
The consensus seeking behavior in Multi-PfD tends to stabilize the problem.
• We study the DSGD-GD scheme and analyze its convergence towards the Multi-PS solution.
We first
show that the scheme is convergent under the same sufficient condition for existence of Multi-PSsolution.
With an appropriate step size rule, in expectation, the squared distance between theMulti-PS solution and the iterates decays as O (1 /t ) , and the squared consensus distance decays asO (1 /t 2 ) , where t is the iteration number.
Our fine-grained analysis also reveals that heterogeneoususers, poorly connected graph may slow down the convergence of DSGD-GD , but only so in thetransient, and the asymptotic rate has a linear speedup property.
• To validate our analysis, we conduct numerical experiments on synthetic data and real data.
Particularly, we consider the Gaussian mean estimation on synthetic data to validate our theory,
and the logistics regression problem on the spambase dataset.
The paper is organized as follows.
§2 introduces the Multi-PfD problem and DSGD-GD scheme.
§3presents the theoretical results on Multi-PfD, DSGD-GD together with a proof outline.
Finally, §4shows numerical experiments to validate our analysis.
All proofs can be found in the appendix.
Related Works.
As mentioned, the main ingredient of DSGD-GD scheme is the classical DSGDalgorithm.
The latter was introduced in [Ram et al., 2010, Bianchi and Jakubowicz, 2012, Sayed,2014] and is recently popularized for decentralized learning.
Of relevance to our work are [Lian et al.
,2017] which demonstrated a linear speed up against centralized SGD, and [Pu et al., 2021] with arefined analysis, also see [Kong et al., 2021, Bars et al., 2022] on the effects of consensus steps andtopology.
We also mention recent advances to improve the efficiencies of DSGD in [Tang et al., 2018,
Lan et al., 2020, Koloskova et al., 2019].
Our analysis entails fresh challenges as the sought Multi-PSsolution cannot be described explicitly as an optimal solution to the Multi-PfD problem.
Another line of relevant works pertains to (multi-agent) reinforcement learning whose formulationalso entails a decision-dependent distribution.
To this end, policy gradient algorithms are analyzed in[Zhang et al., 2020, Karimi et al., 2019], and they have been recently extended to the multi-agentsetting [Zhang et al., 2018, Chen et al., 2021]; also see the survey [Zhang et al., 2021].
Most of theabove works showed convergence to a stationary point which may not be unique.
This is in contrastto our analysis of DSGD-GD which converges to the unique Multi-PS solution.
Problem Setup
Consider a scenario where there are n agents connected on an undirected and connected graph
G =
( V, E ) such that V
= { 1 , . . . , n } , E ⊆ V × V .
Note that we include self-loops such that( i, i ) ∈ E for any i ∈ V .
Each agent i ∈
V draws samples from the i th population of users.
The latter is characterized by the distribution
D i ( θ i ) supported on Z ⊆ R p , p ∈ N and parameterizedby the agent’s decision vector θ i ∈ R d , d ∈ N .
In other words, the i th population of users are onlyinfluenced by the i th agent’s decision vector.
Note that the populations of users can be heterogeneoussuch that D i ( θ ) ̸
= D j ( θ ′ ) , i ̸
= j , even if θ = θ ′ .
The agents aim to find a common decision vector θ ∈ R d in a collaborative fashion that minimizesthe average of local losses.
Consider the multi-agent performative prediction (Multi-PfD) problem :
[Equation]
Since G is connected, the constraint θ i
= θ j for ( i,
j ) ∈
E enforces the decision to be in consensusacross the n agents.
In the above, ℓ ( θ i ; Z i ) is the loss function of the fitness of the decision vector θ iwith respect to (w.r.t.)
the sample Z i .
The expected value E Z
i
∼D
i
( θ i ) [ ℓ ( θ i ; Z i )] corresponds to theloss at agent i w.r.t.
the samples from the i th population of users.
Example 1.
We describe a strategic binary classification problem with linear utility for users toillustrate the application of (1) .
The sample Z i is defined by a tuple Z i
=
( X i , Y i ) ∈
R d × { 0 , 1 } offeature and binary label, and the loss function is taken as the logistic regression function:
[Equation]
where β > 0 is a regularization parameter.
The above loss function quantifies the mismatchesbetween the classifier θ and the given data tuple Z i =
( X i , Y i ) .
On the other hand, the distribution D i ( θ i ) is controlled by the i th population of users.
The distributionis decision dependent such that the sample
Z i ∼ D
i ( θ i ) depends on the i th decision θ i .
Observingthe
i th decision θ i , users of the i th population provides samples that are modified via a linear utilitysuch that Z i =
( X i , Y i ) ∼ D
i
( θ i ) is given by
[Equation]
for some ϵ i > 0 , where D o i is a base data distribution of the i th population.
Note that in the above,we have the closed form solution X i =
X
+ ϵ
i θ i .
Tackling (1) leads to a common classifier θ whichtakes the effects of the heterogeneous decision dependent distributions into account.
The Multi-PfD problem (1) comprises of a stochastic objective function with a decision dependentdistribution.
In particular, for each agent i , the distribution D i ( θ i ) captures a feedback mechanismwhere users of the i th population react to the decision made by the i th agent.
See Fig.
1 (left) for anillustration.
Due to non-convexity, the performative risk in (1) can be difficult to minimize.
In thispaper, we are interested in the multi-agent performative stable (Multi-PS) solution:
[Equation]
which approximately solves (1).
Notice that θ PS is defined to be a fixed point of the map M : R d → 118 R d .
The existence and uniqueness of θ PS will be shown under mild condition in Proposition 1.
Comparison to Existing Works.
Our setup differs from recent works in [Piliouras and Yu, 2022,
Narang et al., 2022].
Unlike ours, both works considered a game theoretical setting where the agents do not directly communicate their decisions.
Instead, agents are coupled through the data distributionswhich depend on the global decision profile ϑ :=
( θ 1 , . .
, θ n ) .
In [Narang et al., 2022], agent/player i seeks to solve min θ i E Z
i ∼D i ( ϑ ) [ ℓ i ( ϑ ; Z i )] where the localdistribution D
i depends on the global decision ϑ [Fig.
(middle)]; while Piliouras and Yu [2022] studya slightly different setting where the goal of agent/player i is to solve min θ
i E Z
i ∼D ( ϑ ) [ ℓ ( θ i ; Z i )]
[Fig.
(right)].
Both works showed convergence to a stable equilibria via (stochastic) gradienttype updates.
In contrast, our work considers cooperative agents who are willing to share theirdecisions with other agents.
Under this setting, we show that a natural decentralized scheme achievesconvergence to a consensual decision that is performatively stable.
Decentralized Scheme.
To find an approximate solution to (1) in a cooperative fashion, the naturalapproach is to deploy a decentralized scheme where agents communicate limited amount of information with neighbors that are directly connected on G .
Furthermore, we focus on a scenario whereagents are agnostic to that their users may react to the agents’ decisions.
The above motivates us tostudy the following variant of the standard DSGD algorithm
[Lian et al., 2017].
Let us define a mixing matrix W ∈ R n × n + on G such that W ij =
W ji > 0 if ( i, j ) ∈ E , otherwise
W ij =
if ( i, j ) / ∈ E , satisfying (cid:80) ni
=1 W ij = 1 for any i = 1 , . . . , n .
We consider the scheme:
DSGD with Greedy Deployment ( DSGD-GD ) Scheme
At iteration t = 0 , 1 , . .
, for any i ∈ V , agent i updates his/her decision ( θ ti ) by the recursion consisting of two phases (
Phase 1 )
Z t +
i
[Equation]
where γ t +1 > 0 is a step size.
Note that ∇ ℓ ( θ ti ; Z t +1 i ) denotes the gradient taken w.r.t.
the first argument θ ti , and the samples Z t +1 i at each agent are independent of each other.
The above combines the DSGD algorithm
[Lian et al., 2017] with greedy deployment [MendlerDünner et al., 2020].
For iteration t ≥ 0 , the scheme can be described with two phases:
In Phase 1 , agent i deploys θ ti at the i th population, whose users react to the decision and reveal asample Z t +1 i
∼ D
i ( θ ti ) to the agent.
Note that this is a greedy deployment scheme since the deployeddecision θ ti may not be stable w.r.t.
In Phase 2 , agent i receives the current decisions θ tj fromhis/her neighbors j ∈ N i and update θ t +1 i according to the ‘consensus’ + ‘stochastic gradient’ step.
Here, the ‘stochastic gradient’ step is based on the local decision θ ti and sample Z t
+1 i taken in phaseone with greedy deployment.
We remark that throughout the DSGD-GD scheme, the agents remainunaware of the performative behavior of their users.
Notice that the DSGD-GD scheme can be interpreted as a DSGD algorithm deploying biased stochasticgradient updates.
Denote E t [ · ] as the conditional expectation up to iteration t , we observe
[Equation]
The unbiased gradient on the r.h.s.
also involves derivative w.r.t.
the decision in the distribution.
Analyzing the convergence of DSGD-GD therefore requires different techniques.
Main Results
This section studies the convergence of the DSGD-GD scheme and demonstrates that the latter canapproximately solve the Multi-PfD problem (1).
To facilitate our discussions, we first define:
[Equation]
Note that the first arguments in f i , f denote the agent’s decision and the second argument is thedeployed decision known to the population of users.
For the rest of this paper, unless otherwisespecified, ∇ ℓ ( θ ; Z ) , ∇ f i ( θ ; θ ′ ) , ∇ f
( θ ; θ ′ )
denote the gradients taken w.r.t.
the first argument θ .
Using the above notations, (1) is equivalent to min θ f
( θ ; θ ) .
We consider the set of assumptions:
A1.
Fix any ¯ θ ∈ R d , the function f
( θ ; ¯ θ ) is µ -strongly convex in θ such that
[Equation]
A2.
For any i = 1 , . . . , n , the loss function ℓ ( θ ; z ) is L -smooth such that
[Equation]
A3.
For any i = 1 , . . . , n , there exists a constant ϵ i > 0 such that
[Equation]
where W 1 ( D , D ′ ) denotes the Wasserstein-1 distance between the distributions D , D ′ .
A1, A2 require the loss functions to be strongly convex and smooth, while A3 states that the amountof distribution shift caused by the reaction of i th population to the agent’s decision grows linearlywith difference in decision.
These assumptions are standard in the literature, e.g., [Perdomo et al.
,2020, Mendler-Dünner et al., 2020, Drusvyatskiy and Xiao, 2020].
To simplify notations, we definethe average, maximum sensitivity as ϵ avg := (cid:80) ni
=1 ϵ
i
/n , ϵ max := max i =1 ,...,n ϵ i , respectively.
Our first result establishes the existence of the Multi-PS solution θ PS satisfying ∇ f ( θ PS ; θ PS ) = 0 :
Proposition 1 ( Existence and Uniqueness of θ PS ).
Under A1–A3.
Define the map M : R d → R d
M
[Equation]
=
[Equation]
n (cid:80)
[Equation]
If ϵ avg < µ/L , then the map M ( θ ) is a contraction with the unique fixed point θ PS = M ( θ PS ) .
Ifϵ avg ≥ µ/L , then there exists an instance of (11) where lim T →∞ (cid:13)(cid:13)
M T ( θ ) (cid:13)(cid:13) = ∞ .
See §A for the proof.
Note the inverse condition number µ/L yields a tight threshold on the sensitivityof the population for the stability of Multi-PfD.
Our result can be viewed as the multi-agent extensionto [Perdomo et al., 2020, Prop. 4.1].
Notice that while θ PS may not solve (1), it yields an approximatesolution to the latter, depending on the magnitude of ϵ avg ; see [Perdomo et al., 2020].
Our result shows that Multi-PfD has a relaxed requirement for the existence of Multi-PS solution asit only depends on the average sensitivity ϵ avg .
Consider when ϵ i exceeds µ/L for the populationserved by i th agent, now there may not exist a performative stable solution for the individual agent[Perdomo et al., 2020].
Meanwhile, by Proposition 1, the Multi-PS solution still exists as long asϵ avg < µ/L , e.g., when there are agents in the network with less sensitive users.
We also compare Proposition 1 to [Narang et al., 2022].
Note that the latter considers decisiondependent distributions D i ( ϑ ) with ϑ :=
( θ 1 , . .
, θ n ) involving all decisions, which is differentfrom our setting.
Nevertheless, under similar conditions to A1–A3, Narang et al.
[2022, Theorem1] shows the performative stable equilibria exists if (cid:80) ni
=
ϵ 2 i < µ 2 /L 2 .
When ϵ i =
ϵ avg , it yieldsϵ avg < µ/ ( √ nL ) which is more restrictive than the requirement in Proposition 1.
The relaxed condition in Proposition 1 can be attributed to the consensus seeking nature of (1), whereagents with less sensitive users serve as mediators that prevent divergence of the map (11).
Convergence Analysis of DSGD-GD Scheme.
Next, we focus on analyzing the convergence of the
DSGD-GD scheme.
We require the following assumptions:
A4.
The non-negative matrix W is doubly stochastic, i.e., W 1 = W ⊤ 1 =
There exists a constantρ ∈ (0 , 1] such that (cid:13)(cid:13) W − (1 /n )
⊤ (cid:13)(cid:13) 2 ≤ 1 − ρ .
A5.
For any i = 1 , . . . , n and fixed θ ∈ R d , there exists σ ≥ 0 such that
[Equation]
A4 is a standard assumption on the mixing matrix, for example if G is a connected graph, then Wsatisfying the condition can be constructed [Boyd et al., 2004].
Meanwhile, A5 bounds the variance ofthe ‘stochastic gradient’ ∇ ℓ ( θ ; Z i ) as the expected value to the latter yields
E Z i ∼D i ( θ ) [ ∇ ℓ ( θ ; Z i )]
=∇ f
( θ ; θ ) , where the gradients are taken w.r.t.
the first argument θ .
Moreover, we assume that:
A6.
For any i = 1 , . . . , n , there exists ς ≥ 0 such that
[Equation]
It bounds the heterogeneity of the locally observed samples.
As ∇ f i ( θ ; θ ) =
E Z i ∼D i
( θ ) [ ∇ ℓ ( θ ; Z i )] ,if D i ( θ ) =
D j ( θ ) , the constant will be ς = 0 when the population associated with each agent produces identically distributed samples with the same deployed decision vector.
Otherwise, ς > 0measures degree of heterogeneity across populations.
We remark that even when D i ( θ ) =
D j ( θ ) ,the samples Z t +1 i , Z t +1 j may still not be identically distributed during the DSGD-GD iterations sincethe decision vectors θ ti , θ tj may not be in consensus when t < ∞ .
A6 also implies max i =1 ,...,n ∥∇ f i ( θ PS ; θ PS ) ∥ 2 ≤ ς 2 .
In fact, as we show in (52) of the appendix, itsuffices to prove the convergence of DSGD-GD without A6 as in [Pu et al., 2021, Yuan and Alghunaim,2021].
We proceed our analysis with A6 to extract a tighter bound especially when ς is small.
In both A5, A6, we allowed the upper bounds to grow with O (1 + ∥
θ − θ PS ∥ 2 ) , e.g. A5 issimilar to [Pu et al., 2021, Assumption 1].
This is a weaker condition than the commonly assumeduniform bound of O (1) , e.g., in [Lian et al., 2017].
For example, it covers situations when thequadratic components in ℓ ( · ) depends on both Z and θ .
Note that this relaxation implies that thevariance/heterogeneity can be unbounded, leading to additional challenges in our analysis.
Define the following quantities and constants: fix any δ > 0 ,
[Equation]
Note that (cid:101) θ t is the distance between the t th averaged iterate θ t and θ PS , while Θ to is a d × n matrixof the t th consensus error .
The following theorem establishes the convergence rate of DSGD-GD :
Theorem 1.
Under A1–A6 and the condition on average sensitivity that ϵ avg < µ (1+ δ ) L .
Suppose the step sizes { γ t } t ≥ 1 satisfy γ t +1 ≤ γ t for any t ≥ 1 ,
[Equation]
and the condition γ t γ t +1 ≤ min { (cid:113) 1 + ( (cid:101) µ/ 4) γ 2 t +1 , 3 (cid:113)
+ ( (cid:101) µ/ 4) γ 3 t
+1 , 1 + ρ/ (4 − 2 ρ ) } for any t ≥ 1 .
Then, the iterates generated by DSGD-GD admit the following bound for any t ≥ 0 ,
[Equation]
where D := ∥ (cid:101) θ
∥ 2 + γ 1 8 c 1 nρ (cid:13)(cid:13)
Θ 0 o (cid:13)(cid:13) 2 denotes the initial error, ∆ :
= D + 32 + 8 σ 2 c 2 n , and we recall the definitions of the constants
c 1 , c 2 , c 3 from (14) .
The free parameter δ > 0 in (14) can be chosen arbitrarily.
Thus, according to Theorem 1, DSGD-GDconverges when the average sensitivity ϵ avg is strictly below the threshold µ/L in Proposition 1, i.e.,the same sufficient and necessary condition that guarantees the existence of θ PS .
Further, our resultholds for general step size rules such as constant step size and diminishing step size.
To yield E [ ∥ (cid:101) θ t ∥ 2 ] → 0 , a common option is γ t = a 0 a 1 + t for some a 0 , a 1 > 0 .
In the latter case asγ t = O (1 /t ) , we observe from (16), (17) that the distance to performative stable solution ∥ (cid:101) θ t ∥ 2converges to zero as O (1 /t ) , while the consensus error ∥
Θ to ∥ 2 F converges to zero as O (1 /t 2 ) .
Weremark that our analysis also applies to the case of time varying graph; see §G for a proof sketch.
On the other hand, the bound (16) can be simplified to
[Equation]
As seen, the error is controlled by three terms.
The first term is a transient term that consists ofthe product (cid:81) ti =1 (1 − (cid:101) µγ i ) .
It decays sub-geometrically and is scaled by the initial error D .
The second term is a transient term and is affected by the spectral gap of mixing matrix ρ , the degree ofheterogeneity ς , etc.
It decays as O ( γ 2 t ) .
Finally, the last term is a fluctuation term that only dependson the averaged noise variance O ( σ 2 /n ) .
It decays at the slowest rate as O ( γ t ) .
Effects of Network Topology and Heterogeneity.
An interesting observation from (18) is on how thenetwork topology ( ρ ) and heterogeneity across local sample distribution ( ς ) affects the convergencebehavior of DSGD-GD .
First, we note that the last term of O ( γ t σ 2 / ( (cid:101) µn )) is similar to the fluctuationterm in SGD using the batch size of n under a centralized setting, e.g., [Moulines and Bach, 2011].
Second, the constants ρ, ς only affect the term of rate O ( γ 2 t ) .
In the case with diminishing step size γ t = a 0 a 1 + t , the second term of (18) will vanish at a faster ratethan the last term.
Particularly, if for some C > 0 ,
[Equation]
(19) then (18) will be dominated by O ( γ t σ 2 / ( (cid:101) µn )) .
In other words, the effects of the network topologyand heterogeneous population vanish asymptotically as θ t → θ PS .
As such, the ‘linear speedup’ behavior of DSGD in [Lian et al., 2017, Pu et al., 2021] can be extendedto Multi-PfD with the DSGD-GD scheme.
Again, we highlight that the consensus seeking behavior ofagents has led to such speedup in convergence towards the Multi-PS solution.
3.1 Proof of Theorem
We outline the main steps in proving Theorem 1.
As a preparatory step, we borrow the followinglemma on the Lipschitzness of ∇ f i ( θ , θ ) from [Drusvyatskiy and Xiao, 2020, Lemma 2.1]:
Lemma 2 ( Continuity of ∇ f i ).
Under A2, A3.
For any θ 0 , θ 1 , θ , θ ′ ∈ R d , it holds:
[Equation]
By A4, the update recursion for the average iterate of DSGD-GD can be expressed as
[Equation]
Using the above recursion, we show the following lemma for the one-step progress of DSGD-GD :
Lemma 3.
(Descent Lemma)
Fix any δ > 0 and let ϵ avg ≤ µ (1+ δ ) L .
Under A1, A2, A3, A5 and letthe step sizes satisfy sup t ≥
γ t +1 ≤
[Equation]
µc 2 , the following bound holds
[Equation]
for any t ≥ 0 , where E t [ · ] is the expectation operator conditioned on the iterates up to the t thiteration, and we recall the definitions of c 1 , c 2 , (cid:101) µ from (14) .
The proof is in §B.
We highlight that proving the upper bound (22) requires the smoothness property
Lemma 2 for handling the difference (cid:80) ni
=1 ∇ f i ( θ ti , θ ti ) − ∇ f i ( θ PS , θ PS ) as proportional to theerror against the Multi-PS solution (cid:101) θ t and the consensus error ∥
Θ t o ∥ 2 F .
Lemma 3 prompts us to studythe consensus error ∥
Θ to ∥ 2 F and a key observation is:
Lemma 4.
(Consensus Error Bound)
Under A2–A6 and let the step sizes satisfy sup t ≥
γ t +1 ≤ρ/ √ 2 c 3 , then it holds
[Equation] [Equation]
for any t ≥ 0 , where we recall that c 3 := 12 σ 2 + 18 L 2 (1 + ϵ max ) 2 .
The proof is in §C. In (52) of the appendix, we provide an alternative consensus error bound withoutusing A6.
Note that despite the decision dependent distributions due to the performative nature of
Multi-PfD, the above bound shows a similar trend as in [Bars et al., 2022, Pu et al., 2021, Kong et al.
,2021, Koloskova et al., 2019].
However, unlike [Bars et al., 2022, Lemma 2], the r.h.s.
of (23) contains a O ([ σ 2 + ς 2 ] γ 2 t +1 ∥ (cid:101)
θ t ∥ 2 )term which arises from A5, A6 with the growth condition.
This introduces new challenges to
Our plan is to consider Lemmas 3 and 4 simultaneously in order to control E ∥ (cid:101) θ t ∥ 2 , E ∥ Θ to ∥ 2 F .
Definethe following sequence of non-negative numbers: for any t ≥ 0 ,
[Equation]
We obtain the following lemma:
Lemma 5 ( Convergence of L t ).
Under A1–A6.
Suppose that the step sizes satisfy sup t ≥
γ t +1 ≤min
[Equation]
For any t ≥ 0 , it holds
[Equation]
Further, if the step sizes satisfy
[Equation] [Equation]
µ
[Equation] [Equation]
} for any t ≥ 1 , then
[Equation] [Equation]
where we recall that D := ∥ (cid:101) θ
∥ 2 + γ 1 8 c 1 ρn (cid:13)(cid:13)
Θ 0 o (cid:13)(cid:13) 2 F .
The proof is in §D, where (25) is based on a careful combination of Lemmas 3 and 4, and (26) iscomputed from the non-asymptotic analysis for the recursion (25).
Proof of Theorem 1.
Lemma 5 immediately leads to (16) of the theorem as the l.h.s.
of (26) is lowerbounded by E ∥ (cid:101) θ t +1 ∥ 2 .
To obtain (17), we observe from the simplifying the r.h.s.
of (26) that
[Equation]
Note that ∥ (cid:101) θ 0 ∥ 2 ≤ ∆ as well.
Substituting the above into Lemma 4 yields
[Equation]
Applying Lemma 7 in the appendix together with the step size condition γ t /γ t +1 ≤ 1 + ρ/ (4 − 2 ρ ) ,t ≥ 1 , leads to (17) of the theorem.
This concludes our proof.
□
Numerical Experiments
We consider two examples of performative prediction problems to verify our theories.
All experimentsare conducted with Python on a server using 80 threads of an Intel Xeon 6318 CPU.
Multi-agent Gaussian Mean Estimation.
We aim to illustrate Proposition 1, Theorem 1 via a scalar
Gaussian mean estimation problem on synthetic data.
We consider n = 25 agents connected on aring graph, and the Multi-PfD problem (1) is specified with ℓ ( θ i ; Z i ) =
( θ i − Z i ) 2 / 2 .
The localdistributions are given by D i ( θ i ) ≡ N (¯ z i
+ ϵ
i θ i , σ 2 ) , where ¯ z i is the mean value to be estimated.
For this problem, we have µ = 1 , L = 1 , as such if 0 <
¯ ϵ =
ϵ avg < 1 , the Multi-PS solution can becomputed in closed form as θ PS = (cid:80) ni
=
¯ z i / [ n (1 − ϵ avg )] ; while θ PS does not exist if ϵ avg ≥ 1 .
In our experiments, we set ¯ z
i
= 10 , σ 2 = 50 and step size for DSGD-GD as γ t = a 0 / ( a 1 + t ) witha
= 50 , a 1 = 10 4 .
In Fig.
2, we compare the gap ∥ θ t − θ PS ∥ 2 , consensus error ∥
Θ to ∥ 2 , expected performative risk f
( θ t ; θ t ) of (1), against the iteration number t .
We examine the behavior of DSGD-GDwhen the Multi-PfD problem has an averaged sensitivity parameter of ϵ avg ∈ { 0 . 9 , 1 . 01 , 1 . 05 , 1 . 1 }and under a heterogeneous, decision-dependent distribution environment where ϵ i are distinct.
We first observe from Fig.
2 (left) and (middle) that when ϵ avg = 0 .
9 < 1 , the gap ∥ θ t − θ PS ∥ 2decays at O (1 /t ) as t → ∞ , while the consensus error ∥
Θ to ∥ 2 decays at O (1 /t 2 ) .
This coincidesexactly with Theorem 1.
Furthermore, in Fig.
2 (right), we simulate a setting when one of the agentswith ϵ i = 1 .
01 is always disconnected from the network and perform the greedy deployment scheme
[Figure] [Figure]
individually .
We observe from the figure that its performative risk f
i ( θ ti ; θ ti ) diverges as t → ∞ .
This indicates that consensus can help stabilize the system.
Lastly, from Fig.
2 (middle) and (right), we observe that whenever ϵ avg > 1 , the consensus error andperformative risk diverge.
Again, this corroborates with our Proposition 1.
Email Spam Classification.
We evaluate the performance of DSGD-GD by simulating the performativeeffects on a real dataset.
This example is a multi-agent spam classification task based on spambase , adataset [Hopkins, 1999] with m = 4601 samples, d = 48 features.
We adopt Example 1 and simulatea scenario with 25 regional servers on a ring graph.
Each server has access to training data fromm i = 138 samples from spambase modeling the different set of users; the rest of m train = 1150samples are taken as testing data.
The servers aim to find a common spam filter classifier via (2)with β = 10 − 4 .
To model the strategic behavior of users, their features X i are adapted to θ i throughmaximizing a linear utility function, resulting in the shifted distribution D i ( θ i ) specified in (3).
Thesensitivity parameters are set as ϵ i ∈ { 0 .
4 ϵ avg , 0 .
45 ϵ avg , . .
, 1 . 6 ϵ avg } with ¯ ϵ =
ϵ avg ∈ { 0 . 01 , 0 .
Our results are shown in Fig.
3 as we compare the gradient ∥∇ f
( θ t ; θ t ) ∥ 2 evaluated on the trainingdataset, the consensus error ∥
Θ to ∥ 2 F , and the accuracy on the testing dataset, against the iterationnumber t .
From Fig.
3 (left) and (middle), we observe that the DSGD-GD scheme converges to the
Multi-PS solution and reaches consensus under various settings of ϵ avg , at the rates O (1 /t ) , O (1 /t 2 ) ,respectively.
In Fig.
3 (right), we evaluate the performance of the trained classifier θ ti on the testingdataset with shifted distribution due to θ ti .
We compare with a non-performatively trained solutionobtained by solving θ ⋆ = arg min θ (cid:80) ni
=
f i ( θ ; 0 ) ( DSGD in the legend), i.e., without any shift indistributions, but evaluate the performance on distribution shifted by θ ⋆ .
As observed, the testaccuracy decreases as sensitivity ϵ avg increases, and DSGD-GD achieves better accuracy than DSGD .
Conclusions & Limitations.
In this paper, we studied the Multi-PfD problem, and analyzed itsstability when a DSGD-GD scheme is applied.
Our results indicate that when agents are consensusseeking , Multi-PfD admits a performative stable solution with laxer condition and DSGD-GD achieveslinear speedup.
Limitations to our current results include the requirement of synchronous updatesamong agents, strongly convex loss [cf. A1], etc., which shall be explored in future extensions.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’scontributions and scope?
[Yes]
(b) Did you describe the limitations of your work?
[Yes]
(c) Did you discuss any potential negative societal impacts of your work?
[N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms tothem?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes]
(b) Did you include complete proofs of all theoretical results?
[Yes]
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)?
[Yes]
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how theywere chosen)?
[Yes]
(c) Did you report error bars (e
g., with respect to the random seed after running experiments multiple times)?
[Yes]
(d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
[Yes]
If you are using existing assets (e
g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes]
(b) Did you mention the license of the assets?
[Yes]
(c) Did you include any new assets either in the supplemental material or as a URL?
[N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or
offensive content
[N
A
(a) Did
you include the
full text
of instructions given to participants and screenshots
, if
applicable?
[N/A
(b) Did you describe any potential participant
risks
, with links to Institutional Review
Board
(IRB) approvals, if applicable?
[N/A
(c) Did you include the estimated hourly wage paid to participants and the total amount
spent on participant compensation?
[N/A