Multi-agent Performative Prediction with Greedy Deployment and Consensus Seeking Agents
Abstract
We consider a scenario where multiple agents are learning a common decision vector from data which can be influenced by the agents’ decisions.
This leads to the problem of multi-agent performative prediction (Multi-PfD).
In this paper, we formulate Multi-PfD as a decentralized optimization problem that minimizes a sum of loss functions, where each loss function is based on a distribution influenced by the local decision vector.
We first prove the necessary and sufficient condition for the Multi-PfD problem to admit a unique multi-agent performative stable (Multi-PS) solution.
We show that enforcing consensus leads to a laxer condition for existence of Multi-PS solution with respect to the distributions’ sensitivities, compared to the single agent case.
Then, we study a decentralized extension to the greedy deployment scheme [Mendler-Dünner et al., 2020], called the DSGD-GD scheme.
We show that DSGD-GD converges to the Multi-PS solution and analyze its non-asymptotic convergence rate.
Numerical results validate our analysis.
Introduction
Traditional learning/prediction problems are often formulated with the assumption that data follows a static distribution, from which a decision vector is sought by solving a corresponding optimization problem.
While this assumption holds for ‘stationary’ tasks such as image classification, many real world tasks are dynamical involving data that could be influenced by decisions.
In the latter case, the predictions are said to be performative .
Example scenarios include when users are strategic [Hardt et al., 2016, Dong et al., 2018, Kleinberg and Raghavan, 2020] such as in training an E-mail spam classifier, where users (including spammers) can adapt to the classifier’s rules to evade detection.
A common way to model performative prediction is via incorporating decision-dependent distribution in formulating the learning problem [Quiñonero-Candela et al., 2008].
Since the pioneering work by
Perdomo et al.
[2020], there is a growing literature in analyzing the performative prediction problem, which studied the convergence of learning algorithms to stable point of performative prediction [Mendler-Dünner et al., 2020, Drusvyatskiy and Xiao, 2020, Brown et al., 2022, Li and Wai, 2022,
Wood et al., 2021], algorithms to find stationary solution of performative risk [Izzo et al., 2021, 2022,
Miller et al., 2021, Ray et al., 2022], update timescales of agent [Zrnic et al., 2021], etc.
Most of the existing works are focused on the single agent (learner) setting.
In this paper, we concentrate on the multi-agent performative prediction (Multi-PfD) problem.
Here, the agents are consensus-seeking who seek a common decision vector that minimizes the sum of loss functions through communications on a graph/network.
Our setup pertains to the wide-spread applications of consensus-type decentralized algorithms to machine learning which leverages the large amount of data at cooperating agents, such as clinical data [Warnat-Herresthal et al., 2021], financial data [Li and
Wang, 2019], etc. In specific, adopting a consensus design can bring benefits such as better out-of- sample (i.e., generalization) performance, the ability to utilize data observed at all agents, improved robustness to agent failure. As pointed out by [Perdomo et al., 2020], real world applications may involve user data that can be influenced by the optimization solution leading to distribution shifts. For example, in clinical data, the agents wish to train a model for treating some medical conditions.
Upon the deployment of a model, the future patients may react to the just deployed model by overstating their symptoms accordingly, which increase their chances of receiving better treatment at the hospital.
Another example is the training of an E-mail spam classifier, where each agent represents a regional server providing services to a subset of users.
These users are in general different and may react to their serving agents differently, leading to heterogeneous and decision dependent data distributions.
Notably, our setup differs from the recent works in [Narang et al., 2022, Piliouras and Yu, 2022] which consider a game theoretical setup of Multi-PfD; see Fig. 1 and §2.
For the algorithmic model, we study a decentralized extension of the greedy deployment scheme [Mendler-Dünner et al., 2020], particularly we concentrate on the decentralized stochastic gradient (DSGD)
[Lian et al., 2017] algorithm with greedy deployment ( DSGD-GD ).
Overall, the DSGD-GD scheme emulates a scenario where agents apply DSGD, a standard decentralized learning algorithm, while being agnostic to the performative nature of the prediction problem.
In particular, upon each iteration, the non-converged local decisions will be deployed directly.
Then, agents optimize their decision using stochastic gradient constructed from data with distribution shifted by local decision.
Tackling Multi-PfD with the DSGD-GD scheme yields a practical scenario of performative prediction in the multi-agent setting.
We inquire the open questions: When will the Multi-PfD problem admit a stable and consensual solution?
If so, how fast does it take for DSGD-GD to converge to such solution?
Does the limited communication in DSGD-GD impair its convergence rate?
We provide affirmative answers to the above.
Our idea hinges on adopting the concept of performative stable solution [Perdomo et al., 2020], which is a fixed point solution resulted from the interplay between agent(s) and data that react to the agent(s)’ decisions.
In particular, we focus on analyzing the multi-agent performative stable (Multi-PS) solution in our problem.
To our best knowledge, this paper provides the first study and analysis of Multi-PfD with consensus seeking agents via a practical DSGD-GD scheme.
We highlight the following key contributions:
• We provide a necessary and sufficient condition on the sensitivity of decision dependent data distributions for the existence and uniqueness of the Multi-PS solution.
An interesting finding is that the Multi-PS solution exists even if some of the performative prediction problems of individual agents are unstable.
The consensus seeking behavior in Multi-PfD tends to stabilize the problem.
• We study the DSGD-GD scheme and analyze its convergence towards the Multi-PS solution.
We first show that the scheme is convergent under the same sufficient condition for existence of Multi-PS solution.
With an appropriate step size rule, in expectation, the squared distance between the Multi-PS solution and the iterates decays as O (1 /t ) , and the squared consensus distance decays as O (1 /t 2 ) , where t is the iteration number.
Our fine-grained analysis also reveals that heterogeneous users, poorly connected graph may slow down the convergence of DSGD-GD , but only so in the transient, and the asymptotic rate has a linear speedup property.
• To validate our analysis, we conduct numerical experiments on synthetic data and real data.
Particularly, we consider the Gaussian mean estimation on synthetic data to validate our theory, and the logistics regression problem on the spambase dataset.
The paper is organized as follows.
§2 introduces the Multi-PfD problem and DSGD-GD scheme.
§ presents the theoretical results on Multi-PfD, DSGD-GD together with a proof outline.
Finally, § shows numerical experiments to validate our analysis.
All proofs can be found in the appendix.
Related Works.
As mentioned, the main ingredient of DSGD-GD scheme is the classical DSGD algorithm.
The latter was introduced in [Ram et al., 2010, Bianchi and Jakubowicz, 2012, Sayed, 2014] and is recently popularized for decentralized learning.
Of relevance to our work are [Lian et al., 2017] which demonstrated a linear speed up against centralized SGD, and [Pu et al., 2021] with a refined analysis, also see [Kong et al., 2021, Bars et al., 2022] on the effects of consensus steps and topology.
We also mention recent advances to improve the efficiencies of DSGD in [Tang et al., 2018,
Lan et al., 2020, Koloskova et al., 2019].
Our analysis entails fresh challenges as the sought Multi-PS solution cannot be described explicitly as an optimal solution to the Multi-PfD problem.
Another line of relevant works pertains to (multi-agent) reinforcement learning whose formulation also entails a decision-dependent distribution.
To this end, policy gradient algorithms are analyzed in [Zhang et al., 2020, Karimi et al., 2019], and they have been recently extended to the multi-agent setting [Zhang et al., 2018, Chen et al., 2021]; also see the survey [Zhang et al., 2021].
Most of the above works showed convergence to a stationary point which may not be unique.
This is in contrast to our analysis of DSGD-GD which converges to the unique Multi-PS solution.
Problem Setup
Consider a scenario where there are n agents connected on an undirected and connected graph
G =
( V, E ) such that V
= { 1 , . . . , n } , E ⊆ V × V .
Note that we include self-loops such that ( i, i ) ∈ E for any i ∈ V .
Each agent i ∈
V draws samples from the i th population of users.
The latter is characterized by the distribution
D i ( θ i ) supported on Z ⊆ R p , p ∈ N and parameterized by the agent’s decision vector θ i ∈ R d , d ∈ N .
In other words, the i th population of users are only influenced by the i th agent’s decision vector.
Note that the populations of users can be heterogeneous such that D i ( θ ) ̸
= D j ( θ ′ ) , i ̸
= j , even if θ = θ ′ .
The agents aim to find a common decision vector θ ∈ R d in a collaborative fashion that minimizes the average of local losses.
Consider the multi-agent performative prediction (Multi-PfD) problem :
[Equation]
Since G is connected, the constraint θ i
= θ j for ( i,
j ) ∈
E enforces the decision to be in consensus across the n agents.
In the above, ℓ ( θ i ; Z i ) is the loss function of the fitness of the decision vector θ i with respect to (w.r.t.)
the sample Z i .
The expected value E Z
i
∼D
i
( θ i ) [ ℓ ( θ i ; Z i )] corresponds to the loss at agent i w.r.t.
the samples from the i th population of users.
Example 1.
We describe a strategic binary classification problem with linear utility for users to illustrate the application of (1) .
The sample Z i is defined by a tuple Z i
=
( X i , Y i ) ∈
R d × { 0 , 1 } of feature and binary label, and the loss function is taken as the logistic regression function:
[Equation]
where β > 0 is a regularization parameter.
The above loss function quantifies the mismatches between the classifier θ and the given data tuple Z i =
( X i , Y i ) .
On the other hand, the distribution D i ( θ i ) is controlled by the i th population of users.
The distribution is decision dependent such that the sample
Z i ∼ D
i ( θ i ) depends on the i th decision θ i .
Observing the i th decision θ i , users of the i th population provides samples that are modified via a linear utility such that Z i =
( X i , Y i ) ∼ D
i
( θ i ) is given by
[Equation]
for some ϵ i > 0 , where D o i is a base data distribution of the i th population.
Note that in the above, we have the closed form solution X i =
X
+ ϵ
i θ i .
Tackling (1) leads to a common classifier θ which takes the effects of the heterogeneous decision dependent distributions into account.
The Multi-PfD problem (1) comprises of a stochastic objective function with a decision dependent distribution.
In particular, for each agent i , the distribution D i ( θ i ) captures a feedback mechanism where users of the i th population react to the decision made by the i th agent.
See Fig.
1 (left) for an illustration.
Due to non-convexity, the performative risk in (1) can be difficult to minimize.
In this paper, we are interested in the multi-agent performative stable (Multi-PS) solution:
[Equation]
which approximately solves (1).
Notice that θ PS is defined to be a fixed point of the map M : R d → R d .
The existence and uniqueness of θ PS will be shown under mild condition in Proposition 1.
Comparison to Existing Works.
Our setup differs from recent works in [Piliouras and Yu, 2022,
Narang et al., 2022].
Unlike ours, both works considered a game theoretical setting where the agents do not directly communicate their decisions.
Instead, agents are coupled through the data distributions which depend on the global decision profile ϑ :=
( θ 1 , . .
, θ n ) .
In [Narang et al., 2022], agent/player i seeks to solve min θ i E Z
i ∼D i ( ϑ ) [ ℓ i ( ϑ ; Z i )] where the local distribution D i depends on the global decision ϑ [Fig.
1 (middle)]; while Piliouras and Yu [2022] study a slightly different setting where the goal of agent/player i is to solve min θ i E
Z
i ∼D ( ϑ ) [ ℓ ( θ i ; Z i )]
[Fig. [-
-]1 (right)].
Both works showed convergence to a stable equilibria via (stochastic) gradienttype updates.
In contrast, our work considers cooperative agents who are willing to share their decisions with other agents.
Under this setting, we show
Decentralized Scheme.
To find an approximate solution to (1) in a cooperative fashion, the natural approach is to deploy a decentralized scheme where agents communicate limited amount of information with neighbors that are directly connected on G .
Furthermore, we focus on a scenario where agents are agnostic to that their users may react to the agents’ decisions.
The above motivates us to study the following variant of the standard DSGD algorithm
[Lian et al., 2017].
Let us define a mixing matrix W ∈ R n × n + on G such that W ij =
W ji > 0 if ( i, j ) ∈ E , otherwise
W ij =
if ( i, j ) / ∈ E , satisfying (cid:80) ni
=1 W ij = 1 for any i = 1 , . . . , n .
We consider the scheme:
DSGD with Greedy Deployment ( DSGD-GD ) Scheme
At iteration t = 0 , 1 , . .
, for any i ∈ V , agent i updates his/her decision ( θ ti ) by the recursion consisting of two phases
[Equation]
where γ t +1 > 0 is a step size.
Note that ∇ ℓ ( θ ti ; Z t +1 i ) denotes the gradient taken w.r.t.
the first argument θ ti , and the samples Z t +1 i at each agent are independent of each other.
The above combines the DSGD algorithm
[Lian et al., 2017] with greedy deployment [MendlerDünner et al., 2020].
For iteration t ≥ 0 , the scheme can be described with two phases:
In Phase 1 , agent i deploys θ ti at the i th population, whose users react to the decision and reveal a sample Z t +1 i
∼ D
i ( θ ti ) to the agent.
Note that this is a greedy deployment scheme since the deployed decision θ ti may not be stable w.r.t.
In Phase 2 , agent i receives the current decisions θ tj from his/her neighbors j ∈ N i and update θ t +1 i according to the ‘consensus’ + ‘stochastic gradient’ step.
Here, the ‘stochastic gradient’ step is based on the local decision θ ti and sample Z t
+1 i taken in phase one with greedy deployment.
We remark that throughout the DSGD-GD scheme, the agents remain unaware of the performative behavior of their users.
Notice that the DSGD-GD scheme can be interpreted as a DSGD algorithm deploying biased stochastic gradient updates.
Denote E t [ · ] as the conditional expectation up to iteration t , we observe
[Equation]
The unbiased gradient on the r.h.s.
also involves derivative w.r.t.
the decision in the distribution.
Analyzing the convergence of DSGD-GD therefore requires different techniques.
Main Results
This section studies the convergence of the DSGD-GD scheme and demonstrates that the latter can approximately solve the Multi-PfD problem (1).
To facilitate our discussions, we first define:
[Equation]
Note that the first arguments in f i , f denote the agent’s decision and the second argument is the deployed decision known to the population of users.
For the rest of this paper, unless otherwise specified, ∇ ℓ ( θ ; Z ) , ∇ f i ( θ ; θ ′ ) , ∇ f ( θ ; θ ′ )
denote the gradients taken w.r
[Equation]
t. the first argument θ .
Using the above notations, (1) is equivalent to min θ f
( θ ; θ ) .
We consider the set of assumptions:
A1.
Fix any ¯ θ ∈ R d , the function f
( θ ; ¯ θ ) is µ -strongly convex in θ such that
[Equation]
A2.
For any i = 1 , . . . , n , the loss function ℓ ( θ ; z ) is L -smooth such that
[Equation]
A3.
For any i = 1 , . . . , n , there exists a constant ϵ i > 0 such that
[Equation]
where W 1 ( D , D ′ ) denotes the Wasserstein-1 distance between the distributions D , D ′ .
A1, A2 require the loss functions to be strongly convex and smooth, while A3 states that the amount of distribution shift caused by the reaction of i th population to the agent’s decision grows linearly with difference in decision.
These assumptions are standard in the literature, e.g., [Perdomo et al., 2020, Mendler-Dünner et al., 2020, Drusvyatskiy and Xiao, 2020].
To simplify notations, we define the average, maximum sensitivity as ϵ avg := (cid:80) ni
=1 ϵ
i
/n , ϵ max := max i =1 ,...,n ϵ i , respectively.
Our first result establishes the existence of the Multi-PS solution θ PS satisfying ∇ f ( θ PS ; θ PS ) = 0 :
Proposition 1 ( Existence and Uniqueness of θ PS ).
Under A1–A3.
Define the map M : R d → R d
[Equation]
If ϵ avg < µ/L , then the map M ( θ ) is a contraction with the unique fixed point θ PS = M ( θ PS ) .
If ϵ avg ≥ µ/L , then there exists an instance of (11) where lim T →∞ (cid:13)(cid:13)
M T ( θ ) (cid:13)(cid:13) = ∞ .
See §A for the proof.
Note the inverse condition number µ/L yields a tight threshold on the sensitivity of the population for the stability of Multi-PfD.
Our result can be viewed as the multi-agent extension to [Perdomo et al., 2020, Prop. 4.1].
Notice that while θ PS may not solve (1), it yields an approximate solution to the latter, depending on the magnitude of ϵ avg ; see [Perdomo et al., 2020].
Our result shows that Multi-PfD has a relaxed requirement for the existence of Multi-PS solution as it only depends on the average sensitivity ϵ avg .
Consider when ϵ i exceeds µ/L for the population served by i th agent, now there may not exist a performative stable solution for the individual agent [Perdomo et al., 2020].
Meanwhile, by Proposition 1, the Multi-PS solution still exists as long as ϵ avg < µ/L , e.g., when there are agents in the network with less sensitive users.
We also compare Proposition 1 to [Narang et al., 2022].
Note that the latter considers decisiondependent distributions D i ( ϑ ) with ϑ :=
( θ 1 , . .
, θ n ) involving all decisions, which is different from our setting.
Nevertheless, under similar conditions to A1–A3, Narang et al.
[2022, Theorem 1] shows the performative stable equilibria exists if (cid:80) ni
=
ϵ 2 i < µ 2 /L 2 .
When ϵ i =
ϵ avg , it yields ϵ avg < µ/ ( √ nL ) which is more restrictive than the requirement in Proposition 1.
The relaxed condition in Proposition 1 can be attributed to the consensus seeking nature of (1), where agents with less sensitive users serve as mediators that prevent divergence of the map (11).
Convergence Analysis of DSGD-GD Scheme.
Next, we focus on analyzing the convergence of the
DSGD-GD scheme.
We require the following assumptions:
A4.
The non-negative matrix W is doubly stochastic, i e., W 1 = W ⊤ 1 = 1 .
There exists a constant ρ ∈ (0 , 1] such that (cid:13)(cid:13) W − (1 /n )
⊤ (cid:13)(cid:13) 2 ≤ 1 − ρ .
A5.
For any i = 1 , . . . , n and fixed θ ∈ R d , there exists σ ≥ 0 such that
[Equation]
[Equation] A4 is a standard assumption on the mixing matrix, for example if G is a connected graph, then W satisfying the condition can be constructed [Boyd et al., 2004].
Meanwhile, A5 bounds the variance of the ‘stochastic gradient’ ∇ ℓ ( θ ; Z i ) as the expected value to the latter yields E Z i
∼D i ( θ ) [ ∇ ℓ ( θ ; Z i )]
= ∇ f
( θ ; θ ) , where the gradients are taken w.r.t.
the first argument θ .
Moreover, we assume that:
A6.
For any i = 1 , . . . , n , there exists ς ≥ 0 such that
[Equation]
It bounds the heterogeneity of the locally observed samples.
As ∇ f i ( θ ; θ ) =
E Z i ∼D i
( θ ) [ ∇ ℓ ( θ ; Z i )] , if D i ( θ ) =
D j ( θ ) , the constant will be ς = 0 when the population associated with each agent produces identically distributed samples with the same deployed decision vector.
Otherwise, ς >measures degree of heterogeneity across populations.
We remark that even when D i ( θ ) =
D j ( θ ) , the samples Z t +1 i , Z t +1 j may still not be identically distributed during the DSGD-GD iterations since the decision vectors θ ti , θ tj may not be in consensus when t < ∞ .
A6 also implies max i =1 ,...,n ∥∇ f i ( θ PS ; θ PS ) ∥ 2 ≤ ς 2 .
In fact, as we show in (52) of the appendix, it suffices to prove the convergence of DSGD-GD without A6 as in [Pu et al., 2021, Yuan and Alghunaim, 2021].
We proceed our analysis with A6 to extract a tighter bound especially when ς is small.
In both A5, A6, we allowed the upper bounds to grow with O (1 + ∥
θ − θ PS ∥ 2 ) , e.g. A5 is similar to [Pu et al., 2021, Assumption 1].
This is a weaker condition than the commonly assumed uniform bound of O (1) , e.g., in [Lian et al., 2017].
For example, it covers situations when the quadratic components in ℓ ( · ) depends on both Z and θ .
Note that this relaxation implies that the variance/heterogeneity can be unbounded, leading to additional challenges in our analysis.
Define the following quantities and constants: fix any δ > 0 ,
[Equation]
Note that (cid:101) θ t is the distance between the t th averaged iterate θ t and θ PS , while Θ to is a d × n matrix of the t th consensus error .
The following theorem establishes the convergence rate of DSGD-GD :
Theorem 1.
Under A1–A6 and the condition on average sensitivity that ϵ avg < µ (1+ δ ) L .
Suppose the step sizes { γ t } t ≥ 1 satisfy γ t +1 ≤ γ t for any t ≥ 1 ,
[Equation]
and the condition γ t γ t +1 ≤ min { (cid:113) 1 + ( (cid:101) µ/ 4) γ 2 t +1 , 3 (cid:113)
+ ( (cid:101) µ/ 4) γ 3 t
+1 , 1 + ρ/ (4 − 2 ρ ) } for any t ≥ 1 .
Then, the iterates generated by DSGD-GD admit the following bound for any t ≥ 0 ,
[Equation]
[Equation]
where D := ∥ (cid:101) θ
∥ 2 + γ 1 8 c 1 nρ (cid:13) (cid:13) Θ 0 o (cid:13)(cid:13) 2 denotes the initial error, ∆ :
= D + 32 + 8 σ 2 c 2 n , and we recall the definitions of the constants
c 1 , c 2 , c 3 from (14) .
The free parameter δ > 0 in (14) can be chosen arbitrarily.
Thus, according to Theorem 1, DSGD-GD converges when the average sensitivity ϵ avg is strictly below the threshold µ/L in Proposition 1, i.e., the same sufficient and necessary condition that guarantees the existence of θ PS .
Further, our result holds for general step size rules such as constant step size and diminishing step size.
To yield E [ ∥ (cid:101) θ t ∥ 2 ] → 0 , a common option is γ t = a 0 a 1 + t for some a 0 , a 1 > 0 .
In the latter case as γ t = O (1 /t ) , we observe from (16), (17) that the distance to performative stable solution ∥ (cid:101) θ t ∥converges to zero as O (1 /t ) , while the consensus error ∥
Θ to ∥ 2 F converges to zero as O (1 /t 2 ) .
We remark that our analysis also applies to the case of time varying graph; see §G for a proof sketch.
On the other hand, the bound (16) can be simplified to
[Equation]
As seen, the error is controlled by three terms.
The first term is a transient term that consists of the product (cid:81) ti =1 (1 − (cid:101) µγ i ) .
It decays sub-geometrically and is scaled by the initial error D .
The second term is a transient term and is affected by the spectral gap of mixing matrix ρ , the degree of heterogeneity ς , etc.
It decays as O ( γ 2 t ) .
Finally, the last term is a fluctuation term that only depends on the averaged noise variance O ( σ 2 /n ) .
It decays at the slowest rate as O ( γ t ) .
Effects of Network Topology and Heterogeneity.
An interesting observation from (18) is on how the network topology ( ρ ) and heterogeneity across local sample distribution ( ς ) affects the convergence behavior of DSGD-GD .
First, we note that the last term of O ( γ t σ 2 / ( (cid:101) µn )) is similar to the fluctuation term in SGD using the batch size of n under a centralized setting, e.g., [Moulines and Bach, 2011].
Second, the constants ρ, ς only affect the term of rate O ( γ 2 t ) .
In the case with diminishing step size γ t = a 0 a 1 + t , the second term of (18) will vanish at a faster rate than the last term.
Particularly, if for some C > 0 ,
[Equation]
(19) then (18) will be dominated by O ( γ t σ 2 / ( (cid:101) µn )) .
In other words, the effects of the network topology and heterogeneous population vanish asymptotically as θ t → θ PS .
As such, the ‘linear speedup’ behavior of DSGD in [Lian et al., 2017, Pu et al., 2021] can be extended to Multi-PfD with the DSGD-GD scheme.
Again, we highlight that the consensus seeking behavior of agents has led to such speedup in convergence towards the Multi-PS solution.
3.1 Proof of Theorem
We outline the main steps in proving Theorem 1.
As a preparatory step, we borrow the following lemma on the Lipschitzness of ∇ f i ( θ , θ ) from [Drusvyatskiy and Xiao, 2020, Lemma 2.1]:
Lemma 2 ( Continuity of ∇ f i ).
Under A2, A3.
For any θ 0 , θ 1 , θ , θ ′ ∈ R d , it holds:
[Equation]
By A4, the update recursion for the average iterate of DSGD-GD can be expressed as
[Equation]
[Equation]
Using the above recursion, we show the following lemma for the one-step progress of DSGD-GD :
Lemma 3.
(Descent Lemma)
Fix any δ > 0 and let ϵ avg ≤ µ (1+ δ ) L .
Under A1, A2, A3, A5 and let the step sizes satisfy sup t ≥ [-
-]0 γ t +1 ≤ (cid:101) µc 2 , the following bound holds
[Equation]
for any t ≥ 0 , where E t [ · ] is the expectation operator conditioned on the iterates up to the t th iteration, and we recall the definitions of c 1 , c 2 , (cid:101) µ from (14) .
The proof is in §B.
We highlight that proving the upper bound (22) requires the smoothness property
Lemma 2 for handling the difference (cid:80) ni
=1 ∇ f i ( θ ti , θ ti ) − ∇ f i ( θ PS , θ PS ) as proportional to the error against the Multi-PS solution (cid:101) θ t and the consensus error ∥
Θ to ∥ 2 F .
Lemma 3 prompts us to study the consensus error ∥
Θ to ∥ 2 F and a key observation is:
Lemma 4.
(Consensus Error Bound)
Under A2–A6 and let the step sizes satisfy sup t ≥
γ t +1 ≤ ρ/ √ 2 c 3 , then it holds
[Equation]
for any t ≥ 0 , where we recall that c 3 := 12 σ 2 + 18 L 2 (1 + ϵ max ) 2 .
The proof is in §C. In (52) of the appendix, we provide an alternative consensus error bound without using A6.
Note that despite the decision dependent distributions due to the performative nature of
Multi-PfD, the above bound shows a similar trend as in [Bars et al., 2022, Pu et al., 2021, Kong et al., 2021, Koloskova et al., 2019].
However, unlike [Bars et al., 2022, Lemma 2], the r.h.s.
of (23) contains a O ([ σ 2 + ς 2 ] γ 2 t +1 ∥ (cid:101)
θ t ∥ 2 ) term which arises from A5, A6 with the growth condition.
This introduces new challenges to analysis as it will be insufficient to conclude from (23) alone that DSGD-GD converges to a consensual solution.
Our plan is to consider Lemmas 3 and 4 simultaneously in order to control E ∥ (cid:101) θ t ∥ 2 , E ∥ Θ to ∥ 2 F .
Define the following sequence of non-negative numbers: for any t ≥ 0 ,
[Equation]
We obtain the following lemma:
Lemma 5 ( Convergence of L t ).
Under A1–A6.
Suppose that the step sizes satisfy sup t ≥
γ t +1 ≤ min (cid:110)
[Equation]
For any t ≥ 0 , it holds
[Equation]
Further, if the step sizes satisfy {+γ t − 1 γ t ≤ min { (cid:112) 1 + ( (cid:101) µ/ 4) γ 2 t , 3 (cid:112) 
+}
+ ( (cid:101) µ/ 4) γ 3 t } for any t ≥ 1 , then E
[Equation] [Equation]
where we recall that D := ∥ (cid:101) θ
∥ 2 + γ 1 8 c [Equation]
Θ [Equation] (cid:13)(cid:13) 2 F .
The proof is in §D, where (25) is based on a careful combination of Lemmas 3 and 4, and (26) is computed from the non-asymptotic analysis for the recursion (25).
Proof of Theorem 1.
Lemma 5 immediately leads to (16) of the theorem as the l.h.s.
of (26) is lower bounded by E ∥ (cid:101) θ t +1 ∥ 2 .
To obtain (17), we observe from the simplifying the r.h.s.
of (26) that
[Equation]
Note that ∥ (cid:101) θ 0 ∥ 2 ≤ ∆ as well.
Substituting the above into Lemma 4 yields
[Equation]
Applying Lemma 7 in the appendix together with the step size condition γ t /γ t +1 ≤ 1 + ρ/ (4 − 2 ρ ) , t ≥ 1 , leads to (17) of the theorem.
This concludes our proof.
□
Numerical Experiments
We consider two examples of performative prediction problems to verify our theories.
All experiments are conducted with Python on a server using 80 threads of an Intel Xeon 6318 CPU.
Multi-agent Gaussian Mean Estimation.
We aim to illustrate Proposition 1, Theorem 1 via a scalar
Gaussian mean estimation problem on synthetic data.
We consider n = 25 agents connected on a [Figure] [Figure] ring graph, and the Multi-PfD problem (1) is specified with ℓ ( θ i ; Z i ) =
( θ i − Z i ) 2 / 2 .
The local distributions are given by D i ( θ i ) ≡ N (¯ z i
+ ϵ
i θ i , σ 2 ) , where ¯ z i is the mean value to be estimated.
For this problem, we have µ = 1 , L = 1 , as such if 0 <
¯ ϵ =
ϵ avg < 1 , the Multi-PS solution can be computed in closed form as θ PS = (cid:80) ni
=
¯ z i / [ n (1 − ϵ avg )] ; while θ PS does not exist if ϵ avg ≥ 1 .
In our experiments, we set ¯ z
i
= 10 , σ 2 = 50 and step size for DSGD-GD as γ t = a 0 / ( a 1 + t ) [-witha 
-]with a 0 = 50 , a 1 = 10 4 .
In Fig.
2, we compare the gap ∥ θ t − θ PS ∥ 2 , consensus error ∥
Θ to ∥ 2 , expected performative risk f
( θ t ; θ t ) of (1), against the iteration number t .
We examine the behavior of DSGD-GD when the Multi-PfD problem has an averaged sensitivity parameter of ϵ avg ∈ { 0 . 9 , 1 . 01 , 1 . 05 , 1 . 1 } and under a heterogeneous, decision-dependent distribution environment where ϵ i are distinct.
We first observe from Fig.
2 (left) and (middle) that when ϵ avg = 0 .
9 < 1 , the gap ∥ θ t − θ PS ∥decays at O (1 /t ) as t → ∞ , while the consensus error ∥ Θ to ∥ 2 decays at O (1 /t 2 ) .
This coincides exactly with Theorem 1.
Furthermore, in Fig.
2 (dash-dotted), we simulate a setting where ϵ avg = 0 .but one of the agents with ϵ i = 1 .
01 is always disconnected from the network and perform the greedy deployment scheme individually .
We observe from the figure that its performative risk f
i ( θ ti ; θ ti ) diverges as t → ∞ .
This indicates that consensus can help stabilize the system.
Lastly, from Fig.
2 (middle) and (right), we observe that whenever ϵ avg > 1 , the consensus error and performative risk diverge.
Again, this corroborates with our Proposition 1.
Email Spam Classification.
We evaluate the performance of DSGD-GD by simulating the performative effects on a real dataset.
This example is a multi-agent spam classification task based on spambase , a dataset [Hopkins, 1999] with m = 4601 samples, d = 48 features.
We adopt Example 1 and simulate a scenario with 25 regional servers on a ring graph.
Each server has access to training data from m i = 138 samples from spambase modeling the different set of users; the rest of m train = samples are taken as testing data.
The servers aim to find a common spam filter classifier via (2) with β = 10 − 4 .
To model the strategic behavior of users, their features X i are adapted to θ i through maximizing a linear utility function, resulting in the shifted distribution D i ( θ i ) specified in (3).
The sensitivity parameters are set as ϵ i ∈ { 0 .
4 ϵ avg , 0 .
45 ϵ avg , . .
, 1 . 6 ϵ avg } with ¯ ϵ =
ϵ avg ∈ { 0 . 01 , 0 .
Our results are shown in Fig.
3 as we compare the gradient ∥∇ f
( θ t ; θ t ) ∥ 2 evaluated on the training dataset, the consensus error ∥
Θ to ∥ 2 F , and the accuracy on the testing dataset, against the iteration number t .
From Fig.
3 (left) and (middle), we observe that the DSGD-GD scheme converges to the
Multi-PS solution and reaches consensus under various settings of ϵ avg , at the rates O (1 /t ) , O (1 /t 2 ) , respectively.
In Fig.
3 (right), we evaluate the performance of the trained classifier θ ti on the testing dataset with shifted distribution due to θ ti .
We compare with a non-performatively trained solution obtained by solving θ ⋆ = arg min θ (cid:80) ni
=
f i ( θ ; 0 ) ( DSGD in the legend), i.e., without any shift in distributions, but evaluate the performance on distribution shifted by θ ⋆ .
As observed, the test accuracy decreases as sensitivity ϵ avg increases, and DSGD-GD achieves better accuracy than DSGD .
Conclusions & Limitations
In this paper, we studied the Multi-PfD problem, and analyzed its stability when a DSGD-GD scheme is applied.
Our results indicate that when agents are consensus seeking , Multi-PfD admits a performative stable solution with laxer condition and DSGD-GD achieves linear speedup.
Limitations to our current results include the requirement of synchronous updates among agents, strongly convex loss [cf. A1], etc., which shall be explored in future extensions.