Federated Granger Causality Learning For Interdependent Clients With State Space Representation
Authors: Ayush Mohanty
Nazal Mohamed
Paritosh Ramanan
Nagi Gebraeel
Venue: ICLR 2025 Poster (2025)
Keywords: State Space
Federated Learning
Granger Causality
Interdependencies
Primary area: learning on time series and dynamical systems
TL;DR: A federated learning based approach to learn granger causality encoded in the low-dimensional states of dynamic systems where the subsystems (clients) are operationally interdependent

Abstract: Advanced sensors and IoT devices have improved the monitoring and control of complex industrial enterprises. They have also created an interdependent fabric of geographically distributed process operations (clients) across these enterprises. Granger causality is an effective approach to detect and quantify interdependencies by examining how the state of one client affects the states of others over time. Understanding these interdependencies helps capture how localized events, such as faults and disruptions, can propagate throughout the system, potentially leading to widespread operational impacts. However, the large volume and complexity of industrial data present significant challenges in effectively modeling these interdependencies. This paper develops a federated approach to learning Granger causality. We utilize a linear state space system framework that leverages low-dimensional state estimates to analyze interdependencies. This helps address bandwidth limitations and the computational burden commonly associated with centralized data processing. We propose augmenting the client models with the Granger causality information learned by the server through a Machine
Learning (ML) function. We examine the co-dependence between the augmented client and server models and reformulate the framework as a standalone ML algorithm providing conditions for its sublinear and linear convergence rates. We also study the convergence of the framework to a centralized oracle model. Moreover, we include a differential privacy analysis to ensure data security while preserving causal insights. Using synthetic data, we conduct comprehensive experiments to demonstrate the robustness of our approach to perturbations in causality, the scalability to the size of communication, number of clients, and the dimensions of raw data. We also evaluate the performance on two real-world industrial control system datasets by reporting the volume of data saved by decentralization.

Published as a conference paper at ICLR 2025
FEDERATED GRANGER CAUSALITY LEARNING FOR INTERDEPENDENT CLIENTS WITH STATE SPACE REP- RESENTATION
Ayush Mohanty" Nazal Mohamed Paritoosh Ramanan', Nagi Gebraee! GGeorgia Institute of Technology, Atlanta, GA, USA: (amohanty42, naz, ngebraeee3Jegateeeeee edu Oklahooaa State University, Stillwater, OK, USA; paritosh www.aarekkttateee...m..mmma edu
ABSTRACT
Advanced sensors and IoT devices have improved the monitoring and control of complex industrial enterprises. They have also created an interdependent fabric of geographically distributed process operations (clients) across these enterprises. Granger causality is an effective approach to detect and quantify interdependen- cies by examining how the state of one client affects the states of others over time. Understanding these interdependencies helps capture how localized events, such faults and disruptions, can propagate throughout the system, potentially leading to widespread operational impacts. However, the large volume and com- plexity of industrial data present significant challenges in effectively modeling these interdependencies. This paper develops federated approach to learning Granger causality. We utilize linear state space system framework that leverages owwdimeensiona state estimates to analyze interdependencies. This helps address bandwidth limitations and the computational burden commonly associated with centralized data processing. We propose augmenting the client models with the Granger causality information learned by the server through Machine Learning (ML) function. We examine the co-dependence between the augmented client and server models and reformulate the framework as a standalone ML algorithm pro- viding conditions for its sublinear and linear convergence rates. We also study the convergence of the framework to a centralized oracle model. Moreover, we in- clude differential privacy analysis to ensure data security while preserving causal insights. Using synthetic data, we conduct comprehensive experiments to demon- strate the robustness of our approach perturbations in causality, the scalability the size of communication, number of clients, and the dimensions of raw data. We also evaluate the performance two real-world industrial control system datasets by reporting the volume of data saved by decentralization.
INTRODUCTION
The rapid growth of IoT devices and sensor networks has increased the interdependencies between process operations of decentralized systems, such as distributed manufacturing enterprises (Ok- wudire Madhyastha| (202T). Srai et al. (2020)) supply chains (Lee & Billington (1993), Bernstein & Federgruen (2005)), and power networks Singh
(Singh et al. (2018), Kekatos & Giannakis (2013)) These systems comprise geographically distributed assets (e.g., machines and processes) that rely on ad- vanced sensors and loT technologies for monitoring and control. These technologies often generate large volumes of high-dimensional time-series data that capture the operational state and reliabil- ity of various system components. Ensuring reliable system-wide operations is challenging due to operational interdependencies, which magnify the effects fault propagation Bian Gebraeel 2014) and cascading failures Fu al. 2023).
This paper focuses on systems with multiple geographically distributed entities- for example, man- ufacturing and utility plant sites, which refer clients-operraeee diients-operating in interconnected man- ner. We examine the operational interdependencies in these multi-client systems using state-space
clients
"qqual contribution

---

Published as a conference paper at ICLR 2025
modeling and causal analysis to better understand their cause-and-effedt relationships. Granger causality Granger (1969) is an effective approach to detect and quantify interdependencies by ex- amining how the state of one client affects the states of others over time. This approach captures how localized events, such as faults or disruptions, can propagate throughout the system, potentially leading to widespread operational impacts.
The decentralized nature of data, coupled with its large volume and high dimensionality, presents significant challenges in establishing causality through centralized data analysis. Aggregating data from multiple sources in a central server can become inefficient and impractical as the scale and complexity of the data increase. However, in many applications, it is possible to represent high- dimensional data using low- www-dimensional state. In the context of causality analysis, oww-dimensional state enable the identification of critical interdependencies without aggregating raw data.
In this work, we use linear time-invariant (LTT) state space representation for individual models of multi-client system. Clients operate independently using only their client-specific information. The measurements (i.e., raw data) at each client are assumed to be high-dimensional. Clients cannot share their measurements, but can only share their low-dimensional state with a central server. Our goal is to develop federated learning framework that allows a decentralized system of clients to collaboratively learn the off-diagonal blocks of the system's state matrix that represent the cross- client Granger causality-by sharing only their state with central server. To achieve this, pro- pose augmenting client models with the off-diagonal information of state matrix through a Machine Learning (ML) based function. To the best of our knowledge, this is the first study federated granger causality learning. Please refer to Appendix A. for preliminaries on state space modeling, Kalman filter, and Granger causality, along their brief mathematical representations.
Research Objective: Our objective is to develop a federated learning framework in which the augmented state gradually converges the centralized state, thus achieving parity between local and a centralized (oracle) model. Through this process, the decentralized system learns the off- diagonal blocks of the system's state matrix, which capture client interactions by sharing only their states with a central server rather than large volumes of high-dimensional measurements.
Main Contributions: Our key technical contributions can be delineated as follows:
1. We formulate a federated framework for multi-client state space system that operates via iterative optimization, where (1) the server learns cross-client Granger causality using low- dimensional states from all clients, and (2) client models, augmented with ML functions, implicitly capture these causality..
2. We prove convergence dependencies between server and client models, and reformulated the server-client iterative framework as a standalone ML algorithm with sublinear and lin- ear convergence rates its gradient descent.
3. We define centralized oracle benchmark and proved bounded differences between the ground-truth and learned Granger causality, with matrix bounds under specific conditions.
4. We performed a theoretical analysis to ensure that the communications (both client-to- server, and server-to-client) satisfy differential privacy.
5. Experiments on synthetic data highlight communication efficiency, robustness, and scala- bility. We also validate the framework on real-world ICS datasets, reporting the volume of data saved by decentralization without compromising the training loss.
2 RELATED WORK
Federated lcarning (FL) is a decentralized machine learning approach where model training occurs across multiple clients, sharing only model updates with a central server. Traditional FL. works (McMahan et al.] (2077). Yurochkin et al.] (2019)) with horizontally partitioned data (Yang et al.) (0019)), where each client has independent data sample but the same feature space. Our approach, however, aligns more with Vertical Federated Learning (VFL), where clients hold different features of the same sample. Since we use time-series data, in our case the features corresponds to the measurements and the samples refer to the time stamp. Unlike conventional VFL setting such as (Hu et al. (2019), Gu et al.] (202I), Chen et al. (2020). Ma et al.] (2023), Hardy et al.] (2017). Yang al. (2019a). Fang al. (2021) Wu al.
2020) which often involves sharing models to
2

---

Published as a conference paper at ICLR 2025
server and updating the client model, framework allows each client to maintain its model-based on client-specific observations, without centralizing data or models.
Split Learning and Multi Task Learning: Our method shares elements with both (1) split learning Veeaaoomaa et al. (2018), Poirot ct al. (2019), Thapa et al. (2022), Kim et al. (2017) where different parts of model trained separately, and (2) multi-task learning (Smith et al. (2017), Marfoq al..
) and Chen & Zhang (2022), ) where tasks share a common representation. However, unlike these methods, our approach maintains client models' autonomy.
Granger Causality: While Vector Auto Regressive models are widely applied for Granger causality (GC) learning such as Gong al. (2015), Geiger et al. (2015), |yvärinen et al. (20TO), Huang al.
2019), Chaudhry et al. 12 they struggle with systems involving hidden states. State-space (SS) representations offer more flexibility for such systems, but applications of GC in SS models such as
Elvira & Chouzenoux (2022), Józsa et al. (2019), Balashankar et al. (0233 are primarily centralized.
comprehensive review of GC can be found in Balashankar et al. (2023). Our framework extends
this by enabling federated GC learning in SS systems, where cross-client causality is inferred by estimating off-diagonal blocks of the state matrix A, assuming client-specific observations through a block diagonal output matrix C.
System Identification Traditional system identification literature ( [Keesman] (2011), Simpkins
(0012) Gibson & Ninness ) assumes centralized access to all data, violating the decentral-
ization premise of our framework. Recent methods such as Haberr & Verhaegen (2014), Stankovid et al. (2015), Mao & He 2022) address this through low-rank and sparse techniques, but still require centralized measurement aggregation or neighbor node knowledge. Our framework bypasses these requirements by estimating the A matrix using low-dimensional states, retaining the ability to infer causality without moving measurements.
Distributed Kalman Filter: Kalman filters estimate latent states from noisy data but face challenges in decentralized settings. Distributed Kalman Filtering such as the ones discussed in Zhang et al. (0022), Xin al. (2022), Cheng et al. (2021), Olfati-Saber & Shamma (2005), Olfati-Saber (2007), Farina & Carli 2018) allow for decentralized collaboration but typically assumes knowledge of the matrix or centralization after local filtering. In contrast, our approach estimates the A matrix without prior system knowledge or data movement.
PROBLEM SETTING
We assume server-client framework with M clients having operational interdependencies. Client m observes high dimensional time series measurements y E R Pm, utilizes client-specific state matrix Amm, and outputs low dimensional states (h'm), E Pm (Dm >> P..) via its client model fo(-), s.t., (he)) fo(y Amm). ). This model does not capture cross-client causality as it uses only Amm (and not using Ama + m). The framework then proceeds iteratively follows:
Client uses ML function f.I(.) augment the client model, producing (hi) where (hin), = -a((he) JMI. (w (m)) and fa is the augmentation model. The parameter Om encodes cross-client causality. Client m minimizes the loss (Lm)
| fo ((hm))aa) w.r.t fn, then communicates the tuple [(h() (hrm) ) to the server.
The server model fo(.) receives input (H'). = [(h)).... ..., (h)r))] to produce (H'), = Sa ((H')) [Amns Amm Ammn Vn * m]]. It optimizes the loss L, = (|HH),, (H²),,1 w.nt. parameters Amns where (H'), = [(h)).. (hir), The Amn are the learned cross- client Granger causality. The server then communicates the gradient of L, to the clients.
A discussion on the possible choices of fer fa, far along with the rationale behind our models, is provided in the Appendix A.2 A simplified pictorial description of the aforementioned problem setting is shown in figure I A pseudocode for our proposed framework is given in Appendix A.3 Readers can find the code of this paper and associated experiments in wtthtpp::: //ttt:::///gthbbbb com/ Federated- int interreeeeddency--ee learen eeeeeennnnnnneneeeeeeeee fed _ereed__raarrr___e Sawpnngerruuuaaeeeeee ity.git
3

---

Published as a conference paper at ICLR 2025
Augmented Model
Low Dimennsional States
Client
Client Model
ML function
Gredient of Server loss
Server Model
Clientn
Operatiónal Interdapendencies
manifested the measurements
Gradient Server loss
Client
Augmented Client Model
Clent m
Low Dimensional
Client m
Client Model
States
Cross-client Granger causelity the states
High Dimensional Messurement
ML function
Figure 1: Federated cross-client Granger causality learning framework
4 FEDERATED GRANGER CAUSALITY FRAMEWORK
Nomenclature: We define h as the "predicted state," i.e., the state predicted for time based on measurements y² also called the "prior state estimate", "one-step ahead prediction", or "predicted state estimate" in kalman filter literature. The variable ht is the "estimated state." based on y also known as the "posterior estimate", "updated state estimate", or "current state estimate" in literature. Assumption 4.1 (Client Model). The client model fe(-) is Kalman filter with access to client- specific measurements ym Vm E {1.....Mr M). .. It uses only the diagonal blocks of the state matrix A and output matrix C (given by Aum: and C.mm respectively). Equations in the first column of Table T define the client model, using only Amm. and Crm which are known apriori. If unknown, they can be estimated locally using Vm- The estimated and predicted states are (hmm)) e and (h with residual (r(.), and Kalman gain (.
Insufficiencc of Client Models: timation using client-specific measurements. However, they only utilize the diagonal blocks of the state matrix (Amm)), ignoring the off- off-diagonal blocks (Amn Vn o m). Consequently, the client models cannot capture the cross-client Granger causality.
The Kalman filter based client models provide optimal state es-
The centralized oracle is a Kalman filter that accesses mea- surements of from all M components. Unlike the client model, the oracle's state matrix has non-zero off-diagonal blocks representing cross-client causality. The third column of Table describe the oracle, where (h'), and (h'), are its estimated and predicted states. The matrix C' is assumed to be block diagonal. Residual (7*) ) and Kalman gain K, are similar to the client model.
Benchmark - A Centralized Oracle:
Table I: Equations for the Client Model, Augmented Client Model, and Centralized Oracle
Client Model
Augmented Client Model
Centralized Oracle
(hin)e Anm (him (rh)) = yn Com (Rim) h de = (he)) + (Km)rr (reea).
(hmm)a = Amm- (him-) (r l = y Com (Win) (he = (hin) + 6m yin
(h²), -A. -A-(h'1) (h²-l)
(r") = -C. Chh)))
(h²) = (h'), + Ko (r")
4.1 AUGMENTED CLIENT MODEL
To address the above insufficieny, we augment the client models with ML. function, enabling learn- ing of Granger causality within their "augmented states". The two salient characteristics of this ML function must be follows: (1) the parameter of that function must capture the Granger causality (which otherwise not captured by the client model), and (2) the function must only utilize the client-specific parameters Amm and Cmm,, and client-specifie measurements ym-
We assume a additive augmentation model s.t., Augmented Client Model = Client Model + ML function.. Furthermore, the underlying system is assumed to have LTT state space represen- tation, we make the following assumption on the ML function to facilitate mathematical insights:
Assumption 4.2. The MI. function (augmenting the client model) linear in you Vm {1, M)
4

---

Published as a conference paper at ICLR 2025
To draw analogies with the client model we state the augmented client model in the second col- umn of Table The estimated and the predicted augmented states are given by (km)a and (hi ) respectively. The augmentation is defined in the third equation (of Table I. where a linear MI. function given by mm is added the estimated state of the client model to provide the estimated augmented state. On is the parameter of the ML function.
Client Loss: Similar to the client model, the augmented model also uses client-specific state matrix (Amm)) and output matrix memm)). The second row of Table I defines the augmented client model's residual. The loss function of the augmented client model given by (Lm), || )
We make the following claim which is validated later in our theoretical analysis and experiments: Claim 4.3. The client model's parameter 9m captures the cross-client Granger causality informa- tion of state matrix's 's off off-diagonal blocks Amn Vn # m, and m, n E }
At training iteration k the learning of Om uses gradient descent algorithm shown in equation 1 There are two partial gradients involved in this step: one corresponding to the augmented client loss (L with learning rate of 1/1 and the other to the server model's loss L, with a learning rate of 172- Effectively, we are optimizing a weighted sum of (Lm,), and L,, where the weights are proportional to 7/ and 7/2- In equation 1] the second term, Vos (Im) can be computed locally at the client. Using the chain rule, we expand the third term of equation to derive equation 2 where V (hht., Lg is communicated from the server, and V (Kmm)a is computed locally client m.
@k+1 0% - 1/1 VOL (Lm)) - 1/2 Vok Lig
8 T7 Vas (Im) - 122 V (kt)). Lg ym T
(2)
Communication from Client to the Server: A tuple of the estimated states from the client, and the augmented client model i.e., (a . (hmm)al are communicated from the client m to the server.
4.2 SERVER MODEL.
Using the tuple of the estimates states communicated from all M clients, the objective of the server model to estimate the state matrix that encodes cross-client Granger causality in its off-diagonal blocks. We also make the following assumption about the diagonal blocks of that state matrix:
Assumption 4.4. The diagonal blocks of the state matrix given by Amm E R P.P x.. Vm E (1, M) are assumed to be known apriori at the server.
Assumption 4.4 is reasonable as the diagonal blocks are known (or estimated) apriori at the clients, and they need be communicated only once before the onset of the model training.
We use the augmented model's estimated state (him). and the diagonal blocks Anm to compute the predicted state (N (second column of Table 1). These (ha))a Vm are later used as training labels for the server model. On the other hand, a direct consequence of assumption 4.4 that only the off-diagonal blocks of the state matrix need to be estimated by the server model. We denote these estimated aff-diagonal blocks are Amn E R. xP",, n o m Vm, M...
Server Loss:
The server model inputs states (h%)) ). (from all M clients), predicts states (h%)), as output, and compares them against the true labels (hr)). These predictions and labels are concate- nated as (H'),- := [(h(),.... (hhr),]² and (H*) )₂- := [(hf) (h'r)al respectively. Then the loss of the server model is given by L,
-(H)), We now state a claim about the learning
of the server model and validate in later sections:
Claim 4.5. The estimated off-diagonal blocks Amn Vn + m encode the augmented client model's ss ML parameeetrrr On Ym {1,..., M}
Server Model Learning: The learning of Amn also uses gradient descent with 7 as the learning rate. At training iteration k. the gradient descent of Ayan given by:
Ak+1 = Ak
V AN L,
Communication from Server to Client: The gradient of the server's loss w.r.t. estimated state the augmented client model i.e., V (ht..1, Lg E Rm is communicated from the server to client m.
5

---

Published as a conference paper at ICLR 2025
Table States predicted by the server model and the centralized oracle
Server Model
Centralized Oracle
(mm))s Amm ( Σ mF
Amn (annhhg) (1 (hm)) Amm 1m + Σ Amm(h(1))
Comparison to the Centralized Oracle: The predicted states of the centralized oracle can be reformulated as shown in the second column of Table 2 which is analogous to the server model. It important to highlight that, while the oracle has access to the ground-truth state matrix, the server approximates this matrix using the states provided by the client model.
5 UNDERSTANDING DECENTRALIZATION THROUGH A CENTRALIZED LENS
In this section, we substitute the server model terms with high-dimensional data y's and replace the client model terms with the estimated off- diagonal blocks Amn's 's. This reformulation makes the framework appear "centralized"" as the s and Amn 's are available at one location. However, this is purely a theoretical tool for analysis, and in practice, models are trained without any centralization. Theorem 5.1 (Co-dependence), At the 1k iteration, the augmented client model's parameter i.e., +++1 depends on the kth iter. of the server model's ''s parameter i.e., A + m, and vice versa.
provides the following insights: (1) the augmented client model encodes the latest estimation of the state matrix during learning, and (2) the server model's estimated state matrix depends on the most recent client model.. Building these insights and theorem 5.1 we propose corollary 5.2 on the convergence of the ML parameters in both the client and server models.
Theorem 5.1
Corollary 5.2. On converges if and only if Amn n + m converges.
Next, we proposition 5.3 that gives the values of the optimal augmented client and server model parameters given by 0r and Ain. respectively. While the first condition gives the closed-fomm for 0% as function of the knowns; the second condition gives A as a function of 0% Proposition 5.3 (Optimal model parameters). unm Amm is full rank and v'n + Ovt, when augmented client and server model parameter converges to 8% and A n & m. respectively then:
1. IE W/ C'm (h + Amm O y =
2. E (Amm !t (h-=))e - Amp (h2 DT (
Vn
We now offer alternative perspective, representing the framework standalone ML algorithm. Theorem (.0eemmm5444mm 5.4junisses the iterative optimization of the server and client models into unified equation. Theorem 5.4 (Unified framework). For any client m. the federated framework effectively solves the following recurrent equation:
1++1 = -H.
(4)
where,
(
vec(At (m)
vec(At (mmm)))())))
voc(A vec(Ot))
Pu1
2216 60 T)
27 8 7(QT @ R)
H
2V I) 1/ (Qmi @0 RT)
PMM
7Q R (T-G&F)
(m(sm 3 RT)
J :==00 00 0 v with D = ATm C (r')) 1-17
and & is the Kronecker prod.
Pmm = (I (1 (ahh
((())) )a) 1 Qmn y 1h1 )o
R = 2Amm ; G = y 3m
1-17
F 1= (2. A Amm) ) + 1p2 (2AT Trm CT CimmAmen Vmn == (b) (((1)))))

---

Published as a conference paper at ICLR 2025
The augmented client's and server's loss functions ie., (Lm) and Lg convex in 8. and Amn. respectively, so their stationary points are global minima. Since 6n and Amn are elements of A in the recurrence equation 4 the stationary values can also be derived from its asymptotic behavior. Lemma 5.5 provides the asymptotic convergence condition of equation 4 and its stationary values.
Lemma 5.5 (Convergence of framework). The federated framework converges if and only if p(H) 1. Furthermore, the stationary value of A given by A -I-
Upon algebraic manipulation of equation 4 we can obtain the following recurrent linear equation:
A++1 A (I- H): A (I- = A (I H) [ A+l
(5)
Equation 5 is analogous gradient descent of the proposed federated framework parameterized by A. Let L, represent the loss function of the federated framework, whose explicit functiinnl form is unknown. Under special conditions on Ly we analyze the convergence rate of the gradient descent in the joint space Amn and 6n- Leveraging well established results on gradient descent provide theorems 5.6 andd discuss conditions for sub linear and linear convergence.
Theorem 5.6 (Sub linear conv.). If y is convex, and L-pischiitz smooth in the joint space of Amn and 0 with H chosen s.t., | H|| S 1, then convergence rate of Lg is (11/k)
Theorem 5.7 (Linear conv.). If F is C-Lipschitz smooth, and p-strongly convex in the joint space of Amn and Ow with H. chosen s.t., || H||
2C x+b
then convergence rate of L is O((1 )*
ASYMPTOTIC CONVERGENCE TO THE CENTRALIZED ORACLE
We assume that the centralized oracle convergent i.c., it has a zero steady state error. We first analyze the convergence of the states learned using our approach to the oracle. Theorem shows that the predicted states of the augmented client model converge in expectation to the oracle.
Theorem 6.1. Let (hhn)) )a = (hin). + y and (hir)) )a == Amm (hin)) )a (see table 1 Then, for full rank Crm we have the convergence: lim; (h(,,,) )a (hr) 0 Vm {I...., M}
Proposition 6.2 shows that the norm difference between the estimated states of centralized oracle and client model is bounded in expectation. We this bound to establish the subsequent results.
Proposition 6.2. If the client model satisfies p(Amm Amm (Km) C.Cm) then 36th such that the following bound holds: (hi, (2 8ma. Vm E [1,...,M]
Next, we analyze the error in estimating the state matrix. For any clients and n with + m, let A be the stationary point for the off-diagonal block of the estimated state matrix. Let Amn be the ground truth for those off-diagonal blocks. Then theorem 6.3 6.3]and and corollary 66.4 provide upper bound on the estimation error of the state matrix without apriori knowledge of its ground-truth
Theorem 6.3. If p(Ammm Amm (Km) Cmm)) 1 then, E
Enem AA:. Amn] (hh-2)))) s
Am 6t ll + 12 Amn Smaxll Vm E {I,...,M}
Corollary 6.4. 30 min s.t., Thiin == minngem (ht-")))) and the vectors [Ai.. - Amml-(hh are collinear Vn {1,..., M) and n o m then Vm E {1, M).
|| EE Amm]ll
(11Amm (1Ammmmazll Em Amn 8n 8man!
mein
7 PRIVACY ANALYSIS
In this section, we establish two theoretical results to ensure differential privacy of our framework.
Each client independently perturbs its client model's, and augmented client model's estimated states before sending them to the server as follows:
Client:
T (h%m)e + o Min (hi))a + ~N(00021)
7

---

Published as a conference paper at ICLR 2025
03111
2um
3-23
(a)
(b)
(c)
(d)
(e)
Figure 2: Loss functions at (a) client 1, (b) client and (c) server during the first mean-shift. (e) l norm diff. between states of centralized oracle, server, client, augmented client models of client (d) and evolution of Frobenius norm difference between estimation and ground-truth value of A21
Table 3: Cross-client Granger causality- - estimated (Å) vs ground truth (A)
How does client 1 Granger causes client 2?
How does client Granger causes client 1?
Estimated A21
Ground truth A21
Estimated A12
Ground truth A
0.27793 0.2441
0.25 0.25
0.0186 -0.0113
0 0
0.3351
0.3298
0.25
0.25
0.0102 0.0010
0
Server: The server computes the gradient V (lt,). L, and applies gradient clipping with clipping
threshold Cg and Gaussian noise addition as follows: F'm = Clip (v (V(hs.). Lus, Ln, Co + +00,0,
Theorem 7.1 (Client-to-Server Comm.). At each time step t, the mechanisms by which client m sends Rt and in... to the server satisfy (e, d)-differential privacy with respect to y provided that the noise standard deviations satisfy the following with E Ea and 5 = 8 + da:
Bx /2l.21.21////6 Ge 2
2By(BK + Bo) In 1.25/6o)
Oa
Ec
Ea
Theorem 7.2 (Server-to-Client Comm.). At each time step t, the mechanism by which the server sends I'n to client satisfies (E, diifferrent privacy with respect to any single client's data (states), provided that the noise standard deviation satisfies: 0g 2C, 2lm(111112///))
Please refer Appendix A.5 for a discussion on privacy, along with the meaning of By BR, R Ba.
EXPERIMENTS: SYNTHETIC DATASET
The synthetic data simulates multi-client linear state space system with "mean-shifts" representing an anomaly or change in operating condition. The absence of off-diagonal blocks of A matrix in client model affects the states only after mean- shift. This can be visualized in figure d2 whose details are explained later in this section. We use the same client and server models discussed in section 4 All models are regularized to ensure feasible solutions. Experiments began by checking convergence stability (ensuring p(H) < 1), adjusting hyperparameters if needed. Unless noted otherwise, experiments used two clients (M 2) with Dm = D = 8, Pm = P = 2 Vm. Exceptions apply to scalability studies.
Dataset Description & Experimental Settings:
Learning Granger Causality: We train the framework for a two-client system where the states of client 1 Granger-causes client 2 and not vice versa i.e, Afrain * 0 and Afraan = 0. The training losses are given figure (c))-c). The l2 norm differences between (1) the client model and the augmented client model, (2) the centralized oracle and the augmented client model, and (3) the centralized oracle and the server model are shown in figures d(... These plots validate claims 4.5 and theorem 6.1 Figure e track the Frobenius norm difference between the ground-truth and estimated state matrices, which decreases during training, further validating theorem 6.3 and corollary 6.4 The estimated and ground truth A matrices are mentioned in Table
4.3
Robustness to Perturbation in Causality: We introduce perturbations to all elements the off- diagonal blocks of A matrix to assess the framework's robustness. Specifically all elements of
8

---

Published as a conference paper at ICLR 2025
Table 4: Robustness to perturbations in causality and change in network topology
Perturbation =
5%
45% (L2)a Lg
85%
c= 125%
Framework
(L2)a
Ls
(L2)a L,
(L2) L,
No client aug.
10-5
10-6
10-5
10-5
No server model
0.22
0.58
0.88
1.135
Pre-trained client
0.22
0.007
0.58
0.015
0.88
0.022
1.135
0.028
Our method
0.39
0.003
0.57
0.007
0.76
0.010
0.93
.013
Net. Topology +
Preserving
Reversing
Eliminating
Bidirectional
Framework
(L2)a
Ls
(Lz)
Lg
(L2)
L,
(L2)
L,
No client aug.
10
10-3
10-3
10-5
No server model
0.182
0.24
0.279
0.127
Pre-trained client
0.182
0.006
0.24
0.065
0.279
0.012
0.127
0.014
Our method
0.37
0.003
0.35
0.033
0.40
0.006
0.34
0.008
training block matrix were perturbed to generate test data s.t., [A]rr] = [ASrain] where, 4 = {5, 45, 85, 125] % of [Atrain] Vi.
+ Vi E 1,... P
Robustness to Change in Network Topology: We trained the system with (A21 (Afrriin) + o.
(troin)
A (train) = 0). We now modify the test data topology under four conditions: (1) preserving causality
(A (Best) = All (ttrain) A (test) 0), (2) reversing causality (A (test) = 0, (test) * 0), (3) eliminating
causality (A = 0, A (test) 0), (4) using bidirectional causality (A (test) * 0, A (test) + 0).
12
Interpreting Robustness Results: When A matrix changes during testing, we expect L, and (L2) to be higher (than training). While a high L, refers to flag by the server model, a high (Lz), refers flag by the client (client 2's) model. We say a framework has learned causality if both server and client models flag with alteration in causality (i.e., either perturbation or change in topology).
For "our method", both (T2), and L, increase with increase in c, further validating the claims 4.3 and 4.55 Talso shows that with change in the topology, L, increases for "our method"'. The reverse causality shows the highest L. values, thereby inferring that the server model learns causality and flags with alterations in causality. Furthermore, the client model does not show a clear trend to change in network topology. Thus generate false alarms (that there is change in causality) if inferencing is done only based on (L2),, Further investigation need to analyze the reasons behind this observation.
The testing losses for the robustness studies are shown in Table 4
Baselines: We benchmark our framework against three other versions of our framework: (1) same framework without the client augmentation (this underscores the limitations of ignoring the effects of interdependencies with other clients), (2) same framework but without the server model (this highlights the importance of server model in improving the client augmentation), (3) pre-trained client models discussed in Ma et al. (003)) (this demonstrates the importance of the iterative optimization in estimating the true interdependencies). Given the constraints on space, we present the interpretation of our framework's performance relative these baselines in Appendix (A.8
Scalability Studies: We increase the dimensions of the measurements Dm, keeping the state di- mensions Pm, 2, and M 2. We trained and tested our framework with = {2, 4, 8, 16, 32). We also validated the scalability w.r.t. the number of clients, by scaling M to (2, (2, 8, 16, 32) by fixing Dm,
8 and P.. 5 While there is a trend observed for scalability w.r.t. D, none of the studies shows any drastic increase in the order of magnitude for La, thereby demonstrating that the framework scalable in both measurement dimension and number of clients.
Vm. The results for both studies are reported in Table
Table 5: Server loss I, by scaling measurement dim. D and number of clients M
Measurement Dim. (D)
No. of Clients (M)
D= 16
D= 32
D 64
D= 128
M= 2
M -==4 4
M = 8
M 16
0.0027
0.0061
0.0090
0.0084
0.0003
0.0001
0.0026
0.0004

---

Published as a conference paper at ICLR 2025
Table 6: Description of the real-world industrial control system (ICS) datasets
Dataset
M
>m 2===
Dm
Dataset Description
HAI
86
Steam turbine-power & pumped pumped-storage -torage hydropower generation Water treatment facility
SWaT
51
0.332
3.5
0.333
23.5
0.33
430
0.001
000
(a)
(b)
(c)
(d)
Figure 3: (a) Server loss and (b) Client 4's loss for HAI dataset, and (c) Sever loss and (d) Client 4's loss for SWaT dataset at randomly chosen time
EXPERIMENTS: REAL WORLD DATASETS
Datasets: We utilized two ICS datasets - (1) HAI: Hardware-the-loop Augmented Industrial control system Shin et al. (2023), and (2) SWaT: Secure Water Treatment Mathur & Tippenhauer (2016), For both of the datasets, clients in framework corresponds to the processes in the datasets. Details of the raw data are given in Table 6
Preprocessing: We first select the measurements with high (2 0.3) pairwise Pearson correlation with measurements from other clients. For each client m in the real-world dataset, client model was obtained follows: (1) Apply SVD to the measurement Ym and select the top P right singular vectors as the low-dimensional states, (2) Store Cum as the product of the left singular vectors and singular values up to dimensions, (3) Fit VAR model of the low-dimensional states compute Amm The framework was then trained on nominal data, free of attacks.
dimension for all M clients in either dataset. The server loss and the augmented client loss (at randomly chosen client) during training are provided in figure 3 We do not have a ground truth matrix for any of the real-world datasets. We first perform centralized estimation of A matrix and considered that as our ground-truth For P 2, the estimation error between federated and centralized method provided in Table 7 We also report the amount of data volume saved (in bytes) by utilizing our federated learning approach.
Granger Causality Learning: We used the same
Table 7: Comparison federated and centralized method for real-world datasets
Dataset HAI
| Al
Data saved per comm. round
0.8140
144 bytes
SWaT
3.0816
176 bytes
10 CONCLUSION AND LIMITATIONS
This paper introduces a federated framework for learning Granger causality in distributed systems, addressing high-dimensional data challenges. Using a linear state-space representation, cross-client Granger causality is modeled as off-diagonal terms in the state matrix. The framework augments client models with server-derived causal insights, improving accuracy. We provide theoretical guar- antees, demonstrate convergence rates, and include differential privacy analysis to ensure data security. Experiments on synthetic and real-world datasets validate the framework's robustness and scalability. Limitations and potential future extensions are discussed in Appendix A.9
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS
This effort is supported by NASA under grant number 8ONSSC19KI022 part of the NASA Space Technology Research Institute (STRI) Habitats Optimized for Missions of Exploration (HOME) 'SmartHab' Project. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Aeronautics and Space Administration.
REFERENCES
Martin Abadi, Andy Chu, Ian Goodfellow, H. Brendan McMahan, llya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, CCS '16, pp. 308-318, New York, NY, USA, 2016. Association for Computing Machinery. ISBN 9781450341394.
Ananth Balashankar, Srikanth Jagabathula, and Lakshmi Subramanian. Learning conditional granger causal temporal networks. In Proceedings of the Second Conference on Causal Learning and Reasoning, pp. 692-706. PMLR, August 2023.
Fernando Bernstein and Awi Federgruen. Decentralized supply chains with competing retailers under demand uncertainty. Management Science, 51(1):18-29, 2005.
Linkan Bian and Nagi Gebraeel. Stochastic modeling and real-time prognostics for multi-compoonnt systems with degradation rate interactions. IIE Transactions, 46(5):470-435 482, May 2014. ISSN 0740-817X.
Aditya Chaudhry, Pan Xu, and Quanquan Gu. Uncertainty assessment and false discovery rate control in high-dimensional granger causal inference. In Proceedings of the 34th International Conference on Machine Learning, pp. 684-693, PMLR, July 2017.
Jiayi Chen and Aidong Zhang. Fedmsplit: Correlation
federated multi-tass learning across multimodal split networks. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD '22, pp. 87-96, New York, NY, USA, August 2022. Associa- tion for Computing Machinery. ISBN 978-1-45003 1-0-1-1---09-----0
Tianyi Chen, Xiao Jin, Yuejiao Sun, and Wotao Yin. Vafi: a method of vertical asynchronous federated learning. (uXiiv2007.06081) Jul 2020. number: arXiv:2007.0608 arXiv:2007.066 .06081 [cs, math, stat].
Zhijian Cheng, Hongru Ren, Bin Zhang, and Renquan Lu. Distributed kalman filter for large-scale power systems with state inequality constraints. IEEE Transactions on Industrial Electronics, 68 (7):6238-6244) 2021. doi: 0001109/1E.2022229948777
Cynthia Dwork and Aaron Roth. The algorithmic foundations of differential privacy. Foundations and Trends@ in Theoretical Computer Science, 9(3-4):211-400 2014. ISSN 1551-305X.
Víctor Elvira and Émilie Chouzenoux. Graphical inference linear linear-gaussioe gaussian state-space models. IEEE Transactions Signal Processing, 70:4757-4771, 2022. ISSN 1941-0476. doi: 10.1 03P22022.220011
Wenjing Fang, Derun Zhao, Jin Tan, Chaochao Chen, Chaofan Yu, Li Wang. Lei Wang, Jun Zhou, and Benyu Zhang. Large-scale secure xgb for vertical federated learning. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management, CIKM '21. pp. 443-452, New York, NY, USA, October 2021. Association for Computing Machinery. ISBN 978-1-4503-8446699
Marcello Farina and Ruggero Carli. Partition-based distributed kalman filter with plug and play features. IEEE Transactions Control of Network Systems, 5(1):560-570, 2018.
Xiuwen Fu, Pasquale Pace, Gianluca Aloi, Antonio Guerrieri, Wenfeng Li, and Giancarlo Fortino. Tolerance analysis of syber-manufaattring systems to cascading failures. ACM Trans. Internet Technol., 23(4), November 2023. ISSN 1533-5399.

---

Published as a conference paper at ICLR 2025
Philipp Geiger, Kun Zhang, Bernhard Schoelkopf, Mingming Gong, and Dominik Janzing. Causal inference by identification of vector autoregressive processes with hidden components. In Pro- ceedings of the 32nd International Conference on Machine Learning, pp. 1917-1925. PMLR, June 2015.
Stuart Gibson and Brett Ninness. Robust maximun-likelihood estimation of multivariable dynamic systems, Automatica, 11610):166711682 October 2005. ISSN 0005-1098.
Mingming Gong, Kun Zhang, Bernhard Schoelkopf, Dacheng Tao, and Philipp Geiger. Discover- ing temporal causal relations from subsampled data. In Proceedings of the 32nd International Conference on Machine Learning, pp. 1898-1906. PMLR, June 2015.
C. W. Granger. Investigating causal relations by econometric models and cross-spectral methods. (annomettrica,37(3)///. 438, 1969. ISSN 00129682, 14680262. URL http://t://wwt jstor. org/stable/19122
Bin Gu, An Xu, Zhouyuan Huo, Cheng Deng, and Heng Huang. Privacy-preserve asynchronous vertical federated learning algorithms for multiparty collaborative learning. IEEE Transactions on Neural Networks and Learning Systems, pp. 1-13, 2021. ISSN 2162-2388.
Aleksandar Haber and Michel Verhaegen. Subspace identification of large-scale interconnected systems. IEEE Transactions Automatic Control, 59(10))2754-23)) 2014. doi: 10.1109/TAC. 2014.2310375.
Stephen Hardy, Wilko Henecka, Hamish Ivey-Law, Richard Nock, Giorgio Patrini, Guillaume Smith, and Brian Thorne. Private federated learning vertically partitioned data via entity resolution and additively homomorphic encryption, November 2017.
Yaochen Hu, Di Niu, Jiamming Yang, and Shengping Zhou. Fdml: A collaborative machine learn- ing framework for distributed features. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD '19, pp. USA, Jul 2019, Association for Computing Machinery. ISBN 978-1 45033-20666
New York, NY,
Biwei Huang, Kun Zhang, Mingming Gong, and Clark Glymour. Causal discovery and forecasting in nonstationary environments with state-space models. In Proceedings of the 36th International Conference on Machine Learning, pp. 2901-2910. 10. PMLR, May 2019.
Aapo Hyvärinen, Kun Zhang, Shohei Shimizu, and Patrik O. Hoyer. Estimation of a structural vector autoregression model using non-gaussianity. Journal of Machine Learning Research, 11 1563 2010. ISSN 1533-7928.
Mónika Józsa, Mihály Petreczky, and M. Kanat Camlibel. Relationship between granger noncausal- ity and network graph of state-space representations. IEEE Transactions on Automatic Control, 64(3):912-927, March 2019. ISSN 1558-2523.
K.J. Keesman. System Idlentffcattin: An Introduction. Advanced Textbooks in Control and Signal Processing. Springer London, 2011. ISBN 978-0-85729-522-44
Vassilis Kekatos and Georgios B. Giannakis. Distributed robust power system state estimation. IEEE Transactions on Power Systems, 28(2):1617-162 May 2013. ISSN 1558-0679.
Juyong Kim, Yookoon Park, Gunhee Kim, and Sung Ju Hwang. Splitnet: Learning to semantically split deep networks for parameter reduction and model parallelization. In Proceedings of the 34th International Conference Machine Learning, pp. 1866-1874 PMLR, July 2017.
Hau L. Lee and Corey Billington. Material management in decentralized supply chains. Operations Research, 41(5):835-847, October 1993, ISSN 0030-364X.
Tengfei Ma, Trong Nghia Hoang, and Jie Chen. Federated learning of models pre-trained on differ- ent features with consensus graphs. pp. 1336-1346. PMLR, July 2023.
Xiangyu Mao and Jianping He. Decentralized system identification method for large-scale networks. In 2022 American Control Conference (ACC), pp. 1173,,178, 2022. doi: 0023399/ACCC53348. 2022.9867511
12

---

Published as a conference paper at ICLR 2025
Othmane Marfoq, Giovanni Neglia, Aurélien Bellet, Laetitia Kameni, and Richard Vidal. Feder- ated multi-task learning under a mixture of distributions. In Advances Neural Information Processing Systems, volume 34, pp. 15434-15447. Curran Associates, Inc., 2021.
Aditya Mathur and Nils Ole Tippenhauer. Swat: a water treatment testbed for research and training ics security. In 2016 International Workshop on Cyber-physical Systems for Smart Water Networks (CySWater), pp. 31.31 31-36, 2016.
Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. Communication-effiienn learning of deep networks from decentralized data. In Proceedings of the 20th International Conference Artificial Intelligence and Statistics, pp. 1271-12-22222 1282. PMLR, Apr 2017.
Chinedum E. Okwudire and Harsha V. Madhyastha. Distributed manufacturing for and by the masses. Science, 372(6540):3311-322, 2021.
R. Olfati-Saher. Distributed kalman filtering for sensor networks. In 2007 46th IEEE Conference on Decision and Control, pp. 5492-5498, December 2007.
R. Olfati-Saber and J.S. Shamma. Consensus filters for sensor networks and distributed sensor fusion. In Proceedings of the 44th IEEE Conference on Decision and Control, pp. 6698-6703, December 2005.
Maarten G. Poirot, Praneeth Vepakomma, Ken Chang, Jayashree Salpathy-Cramee Rajiv Gupta, and Ramesh Raskar. Split learning for collaborative deep learning in healthcare, December 2019.
Hyeok-Ki Shin, Woomyo Lee, Seungoh Choi, Jeong-Han Yun, and Byung-Gi. Gi Min. Hai security datasets, 2023. URL htttt:://gtttitii http://caaacssatt//e/////a//
Alex Simpkins. System identification: Theory for the user, 2nd edition (/jung, 1;; 1999) [on the shelf]. IEEE Robotics Automation Magazine, 19(2):95-99)) June 2012. ISSN 1070-9932, doi: 11110099...002..21111
Rahul Singh, Kumar, and Le Xie. Decentralized control via dynamic stochastic prices: The independent system operator problem. IEEE Transactions on Automatic Control, 63(10): October 2018. ISSN 1558-2523.
Virginia Smith, Chao-Kai Chiang, Maziar Sanjabi, and Ameet S Talwalkar. Federated multi-task learning. Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017.
Jagjit Singh Srai, Gary Graham, Patrick Hennelly, Wendy Phillips, Dharm Kapletia, and Harri Lorentz. Distributed manufacturing: new form of localised production? International Jour- nal of Operations Production Management, 40(6):697-777, January 2020. ISSN 0144-3577.
Miloš S. Stankovic, Srdjan S. Stanković, and Dušan M. Stipanović. Consensus- decentralized real-time identification of large-scale systems. Automatica, 60:219-226, 2015. ISSN 0005-1098.
Chandra Thapa, Pathum Chamikara Mahawaga Arachchige, Seyit Camtepe, and Lichao Sun. Splitfed: When federated learning meets split learning. Proceedings of the AAAI Conference on Artificial Intelligence, 36(88):8485 8493, June 2022. ISSN 2374-3468.
Praneeth Vepakomma, Otkrist Gupta, Tristan Swedish, and Ramesh Raskar. Split learning for health: Distributed deep learning without sharing patient data, December 2018
Yuncheng Wu, Shaofeng Cai, Xiaokui Xiao, Gang Chen, and Beng Chin Ooi. Privacy preserving vertical federated learning for tree-based models. Proceedings the VLDB Endowment, 13(12): 2090-2103, August 2020. ISSN 2150-8097. arXiv:2008. 0006700 [cs].
Dong-Jin Xin, based on wasserstein average consensus. IEEE Transactions on Circuits and Systems II: Express Briefs, 69(4) 2022. doi: 10. 109971C3II.202221444111
Shi, and Xingkai Yu. Distributed kalman filter with faulty/reliable sensors
13

---

Published as a conference paper at ICLR 2025
Kai Yang, Tao Fan, Tianjian Chen, Yuanming Shi, and Qiang Yang. A quasi- method based vertical federated learning framework for logistic regression. arXiv:1912.00513 [cs, stat], (11Xiv:1912.00513). December 2019a.
Qiang Yang, Yang Liu, Tianjian Chen, and Yongxin Tong. Federated machine learning: Concept and applications. ACM Transactions on Intelligent Systems and Technology, 10(2):1:1-12:19,, Jan 2019b. ISSN 2157-6904.
Mikhail Yurochkin, Mayank Agarwal, Soumya Ghosh, Kristjan Greenewald, Nghia Hoang, and Yasaman Khazaeni. Bayesian nonparametric federated learning of neural networks. In Proceed- ings of the 36th International Conference on Machine Learning, pp. 7252-7261. PMLR, May 2019.
Yuchen Zhang, Bo Chen, Li Yu, and Daniel W. C. Ho. Distributed kalman filtering for intercon- nected dynamic systems. IEEE Transactions on Cybernetics, 12111). 2022. doi: 10.1 (..109/TCYBB.2221337211
14

---

Published as a conference paper at ICLR 2025
A
APPENDIX
CONTENTS
A.
Preliminaries
16
A.2
Defining Functions Jo M
16
A.3
Pseudocode
17
Proofs
17
A.4.1
Proof of Theorem
1..
17
A.4.2
Proof of Corollary 5.2
19
A.4.3
Proof of Proposition 5.3
20
A.4.4
Proof of Theorem 5.4
20
A.4.5
Proof of Lemma 5.5
21
A.4.6
Proof of Theorem
A.4.7
Proof of Theorem 5.7
22
A.8.8
Proof of Theorem
22
A.4.9
Proof of Proposition 6.2
22
10
Proof of Theorem
22
A.4.11 Proof of Corollary 6.4
23
A.112 Proof of Theorem
23
A.4.13 Proof of Theorem
24
A.5
Discussion on Privacy Analysis
................................
25
A.6
Addressing Non-IID Data Distributions
26
A.7
Interpretation of theoretical results
27
A.7.1
Understanding Decentralization through Centralized Lens
27
A.7.2
Asymptotic Convergence the Centralized Oracle
28
A.7.3
Privacy Analysis
28
A.7.4
Addressing IID Data Distributions
28
A.8
Performance against Baselines
28
Limitations and Future Work
29
15

---

Published as a conference paper at ICLR 2025
A.1 PRELIMINARIES
State-Space Model: A state-space model is mathematical framework used to represent the dynamics of physical system as a set of measurements y", states h', all related through a difference equation. It characterizes the system's 's evolution over time by describing how its state transitions and generates measurements, using two primary equations: the state transition equation (6) and the observation (measurement) equation 77)
h' Ah y = Ch' +v
(6)
Here, w and are the i.i.d Gaussian noise added to the states and measurements respectively. The matrices A and C are called the state transition and observation (measurement) matrices, re- spectively. The above representation linear, (2) A, C, distributions of w and are assumed to be stationary.
linear time-invariant (LTT) model, as (1) the equations are
In a system represented by state-space model, knowledge the state h' fundamental for under- standing the system dynamics. However in real world applications, one often observes only the measurements yf without direct access to the states h', Estimation of he using y' is typically done by Kalman filter, explained next.
Kalman Filter: A Kalman filter (KF) is an algorithm designed to provide optimal linear estimates of the states based measurement y". A brief walk-through of the steps involved in KF is explained in the next paragraph.
h' A.h h' x² yf C h
(8)
(9)
h' = h² + k
(10)
At time the KF predicts the state he using the previous estimate h (eq. 8 Upon receiving a new measurement y², computes the residual r', the difference between y² and the predicted measurement C h² (eq. 9). The state estimate h then updated by adding a correction term K rf to its predicted state h' (eq. 10), with Kalman gain K determining the weight of the residual.
KF operates under the assumption of complete knowledge of A matrix, and other parameters such C, noise covariances, etc. Without given A matrix, implementation of KF mandates its prior estimation. Granger causality is one of the techniques used to explicitly estimate the A matrix.
Granger Causality: time series h, said to "granger cause" another time series h2, if h can predict h2. In the context of state space models, the state matrix A characterizes the granger causality. For example, in a two state sytem, given in II. state hy is said to "granger cause" state hy if A12
An A21
A12 Ap2
W1
(11)
h
h
W2
Learning granger causality in a state space model involves estimation of A matrix. Estimating ma- trix often involves centralizing the measurements y' and performing tion. However, in systems with decentralized components (clients), centralizing the measurements y from each component can be a challenge. This especially true when y" high dimensional.
estima-
Decentralized learning of A matrix is the primary objective of our paper.
A.2 DEFINING FUNCTIONS
f.M.
1. Client Model fa(-):
The client model can be any machine learning model, such as neural networks, linear regression, state space model. For mathematical tractability, use state-space model.
One example of the client model can be anomaly detection model. In this paper, assume a client model fa(.) is trained independently using only client-specific measure- ments.
16

---

Published as a conference paper at ICLR 2025
2. ML Function (.):
The MI. function (-) machine learning model that specifically captures the effects interactions (granger causality) with other clients. It enhances the awareness of the local client model towards interdependencies with other clients.
In our implementation, use linear regression as fML- At client m, fML takes only the client-specific measurements yoe as input but encodes information from other clients during the gradient update process. This allows the model to benefit from collective insights while preserving data privacy.
3. Augmented Client Model fa(-):
The augmented client model fa(.) combines the client model fe and the ML function L to enhance the client model. In our case, this augmentation is achieved through simple addition:
fa(.) fe(-) + fML(())
However, more complex models like neural networks higher-order polynomials can also be used for augmentation. The output of fa(.) is designed to have the same dimension as fe(.), ensuring that it serves as direct enhancement the client model without altering its fundamental structure.
A.3 PSEUDOCODE
The pseudocode for running the client model given in algorithm I independent at each client without any federated approach. The output of the client model i.e., the estimated states (, Vm, i are utilized in the federated Granger causal learning, whose pseducodoe Ts given in algorithm T is the time series length, epoch is the maximum training epochs, tol is the stoppage tolerance for the server loss and k is the iteration index.
The client model is run
Algorithm Client Model
1: Inputs:T, Amms Cam
2: Choose at Client m: (Km)c
3: for = 1 to T do
4: (hin) t Amm (1 )
5: (mm) + ym - Cmm (hin)e
6:
(h +- (hmm)e (Km)e (r
7:
end for
A.4 PROOFS
A.4.1 PROOF OF THEOREM 5.1
We know that the server loss L, is given by,
L. [1' Hill
(12)
We know from the definition of augmented client states that,
(h1) )a
An 1g--1 )a
H
(13)
Anaad A ((i
We also know from the definition of server states H. that,
(lf)s
Au (hi-')) + +ppes Aip(hi (hi 1
H
(14)
Amen Eppnr A Abisp(hh (
17

---

Published as a conference paper at ICLR 2025
Algorithm 2 Federated Learning of Granger Causality
1: Inputs:T, Amm. Omm- (hin) Vm E {L...., 11,,,,,,, E {I.... T}
2: Choose: epoch, tol, & =
3: Initialize at Server: { mn Imzn
4: Choose at Server: y
5: Initialize at Client m: On
6: Choose at Client: 7)1, 1/2
7: while & < epoch T or Lg tol do
8: for = 1 to T do
9:
for each client do (hmm)a f Amm (h )a
10:
(h'n)a (k + 0 Vm
12: Send ((hm)) (bm)) to the server
13:
end for
At the server:
(H"), + [(11) (a and (H*) + [(ht) (d
(H'), computed using (kmm)s Amm (Nim Da E Am (hin )c
La f- [1(11) 2
Send V (bitr Lg the client
Ak+1 f- Åk
7. ViL Ls
mn
for each client do
16:
gk+1 + 8h 11 V (Lm)a - -12 V La
m end for
18: end for end while
19:
Therefore the server loss I., is given by,
An )a [An(h +Engt Al (hh
L.
(15)
AMMs (hir')) - (AM + Enfer A (h (ht=1)d))
(1) Update of Server Model Parameter:
We take derivative of Teg w.r.t. Ak obtain,
Ly = -2 Amm (aama(hir)) (hi=))a [Amm(h ( + Σ Ahh (11--1) (1-
(16)
mm
Substituting equation 16 in equation (i.e., gradient descent of Amm) we obtain,
Ak + 2 Amm (h=- )a- [Amm (ht (h-m + Σ (h )e] 1
(17)
know from the definition of augmented client model that, (h ) (Kim=)) ) + 0 1/-
(18)
Substituting equation 18 in equation 17 we obtain,
A k+1 Att + 27 AmmRl A y
At A (c) nytimm
Σ
(19)
18

---

Published as a conference paper at ICLR 2025
observe that the (A: + 1)th iteration of server model parameter i.., A is dependent on the ttth iteration of augmented client model parameter i.e., 8
From equation 19
(2) Update of Augmented Client Model Parameter:
At client m, the client loss (Lm) is given by,
(Lm)a = pm Cmm Anm ((hm + gh Um 1)
(20)
The analytical derivative of (Lm) w.n. t is given by,
V ol (m)) = 22 (Cmm Anm) (pin Cram Amm [(hmme Do 6mmn ()
2-17
Vm
(21)
Computing derivative of L, w.r.t 2 have,
(bim))) )a Tug 2. A
2.AT
(Amin (( (Nip="))
Σ n
A (1/z )c
(22)
The derivative of (h W.r.t. 0k is given by,
V (h 1/- Vm
1-12
(23)
Substituting equations 21 22 and 23 in equation 2 (i.e., gradient descent of 6% we obtain,
gh+11 0t + 1/2 (Crm Amm)" T (Vin Cmm Amm
[ + 0t vit 1) vmm
2 172 ATm
Amm ( (him))))
Do]
Σ metiin
Al (han (hi 20
(24)
From equation 24 we observe that the (I: + 1)th iteration of augmented client model parameter i.e., gh is dependent on the th iteration of server model parameter i.c., A m
A.4.2 PROOF OF COROLLARY 5.2
Assume that fm converges to 6% even when Amn diverges.
Therefore, using equation 24 we have.
lim gk+1 ) k-000
lim gh mm
80+00
+ 2 (Crm Ama)) (shn Cmm Amm
((itm )a +
lim oh
yin 2-12
k0000
2 1/2 AT ATm
AAm ( - (P )e
Σ no
lim
A/t (1i
y 1-
Since, Amn diverges, therefore limg-poo AB ma co. We also know that 6.. converges, thus leading
Tim 8% 8r Therefore, I.HS of RHS. Hence our assumption incorrect i.e., Am converges Amn converges.
We proceed similarly in the other direction, by assuming Amm converges even when 6n, diverges. We then leverage equation| t9 contradict the assumption.
Therefore, O.n converges if and only if Amm converges
19

---

Published as a conference paper at ICLR 2025
A.4.3 PROOF OF PROPOSITION 5.3
Condition (1):
At convergence of 0% m
o the gradient of the client loss Ven. (Tom) vanishes expectation.
From Equation 21
E V mm (mm)al= = 2E (Cmm (ymn Cmm Am (Aim f om '1) Umm
0.
(25)
Assuming Cmm is full rank and ym=1 + O, this simplities
B [ym] Com Amm [E [( + 0% [vm ())
(26)
This ensures the client's augmented model reconstructs B[y(n) unbiasedly (i.c., Bl((")) 0).
Condition (2): At convergence of A Amn. the gradient of the server loss V Amn L, vanishes in expectation. From Equation 16
Amn [ng] = -2 ((
E
Ama (f [Amm (h/m
Σ
Amp (ht
(
= 0.
(27)
Substitute (h (him t) + On (from Equation 18
-2
Σ
Ampp (hl
(h
=
(28)
pmtm
E
(Ammo )m
Σ
Amp
(1b
((
Vn * m.
(29)
pf
A.4.4 PROOF OF THEOREM 44
We rewrite the server model parameter and augmented client model parameter update equations as follows:
Ak+1 mn
A + 2 Amm Q "mm
Σ noftom
Alnn (c
(30)
gk+1 = 0 +2. 111 (Cmm Amm)² (Vi Cmm Amm ( + 0 vtr 1)
1-17T
Vm
2 ATm
Amm ( )a (Pe
Σ
Al Whtn (ht- )
2-1
(31)
To handle the matrix equations 30 and 31 we first linearize them using the vectorization technique described in definitions
Definition A.1. Vectorization of matrix linear transformation which converts the matrix into vector. Specifically, the vectorization of m x matrix Z, denoted vec(2 Z), is the mn x 1 column vector obtained by stacking the columns of the matrix Z top of another:
vec(Z) = [211.... 2m1, Z12
2m2 Zin. 2mn]
First, leverage definition A.I vectorize the matrix equation of update of Amn mentioned in equa- tion 19 to obtain:
vec
16+1 vec
A
-2yvec (Létinn
' "e+ Aran Pr
E
A (hg ))_ (
pfmm
(32)
20

---

Published as a conference paper at ICLR 2025
Similarly
we vectorize update equation of fm in equation 24 to obtain:
vec (0h+t1 ) vec 0mh.
21/2vec (AAhmm Amm 0% Umm
+ Σ
Afmm A (21
vm 1mm
re
2111 vec
[Arri Cum CammemmCaan Cmm Vin + (c) A
C Vim
(33)
We observe many terms in equations and 33 contain multiplication of two three matrices. To vectorize such terms we utilize definition Definition A.2. We express the multiplication of matrices as linear transformation i.e, for any three matrices X, Y and Z of compatible dimensions, vec(XY = - @ X)vee(Y)
Next, using definition A.2 and using Identity matrix of appropriate dimensions (whenever two ma- trices are multiplied) s.t., vec Z) (Z" [Y)ec(Y) obtain:
A k++ H A +J
(34)
where,
vec(A vec(Am(mt1)) voc(Atin) voc(oth) |"
A
vec (ec(A
vec(A (m-)))
Pn
2y12 11)
27(V 81)
7Q @ R)
H
(V MI 2
PMM ns mm(0mM 8 R
7Q @ R) (1- @ F)
mh (Qmi & R
J = [0 0 vec(D)) with D = ATm Caa (rm)) 1-1T and @ is the Kronecker prod.
Pmmm = (I - (h
1 ( (ht-)) T ) 1 I Qmn = vir (ht-1h ) R = 2Amm ; G -/m ym 1-1 F (2. Amm) 1p2 (2A CT AmmA)m))
Vmn = (ha (h
A.4.5 PROOF LEMMA 5.5
This is direct consequence of theorem 5.4 it is convergent if and only if p(H) 1.
Since all terms i.e., A, H ion equation are matrices,
Let the stationary value (or value at convergence) for A be denoted by A'. Then from equation obtain the results as follows:
A H A +J (I - H) A = A = (I H) -1-1.
A.4.6 PROOF OF THEOREM
For the loss function of the federated framework i.c., Ly we have the following gradient descent:
A++1 A -1 L,
(35)
Equation is analogous to gradient descent L, given equation 35 s.t.,
(I H) (A k A* = (A
(36)
Also, from definition of C-Lipschitz smoothness we know that,
| (A (A)) < A A+ll
(37)
Multiplying both sides of equation 37 with 77. and then substituting equation 36 we obtain:
[[T H) (A A nf|lA*
A
(38)
From well established theorems on gradient descent, we know that if T. is convex and L-Lipschitz smooth, the gradient descent converges with rate (1)/k) if n < 2. Substituting that condition in equation 38 we obtain:
T H|| < for O(1kk) rate of convergence
21

---

Published as a conference paper at ICLR 2025
A.4.7 PROOF OF THEOREM 5.7
We follow the same argument the last proof. However, we have the added condition of p- strong convexity. Using the convergence properties of gradient descent, a linear convergence rate is achieved when 1) S p+t
2 Substituting S P-C in inequality 38 we obtain:
2C for o (1=1)*) rate of convergence
|1- p+ C
A.4.8 PROOF THEOREM 6.
Definition A.3. If p(A CK,C) < then oracle's expected steady state error zero and we term such oracle convergent. For convergent oracle the expected residuals E[r.] 0.
From Table !
we know that,
(rm)' = v - Cym (kmm)o and, (rm)). = v Cmm (hm)a
(39)
(40)
Therefore, subtracting both the equations and taking expectation of their difference obtain:
E [((rmn) (r/m)ol E [Cmm ((hin) (Nmn)ao]
(41)
We know from definition A.3 that B(((()))) =.
We also know that (r v'n Cmm Amm ((im + 0 Oin Thus, from condition (1) of Proposition 5.3 we know that (E((")))))
Therefore, using equation 41 we know that, IE (e(Cm((M))) (Mm) = If Crm is of full-rank (given) then, B ([(()))) (1m)) =0
A.4.9 PROOF OF PROPOSITION 6.2
From Table I we know that,
(rm) vmn Cmm (hm)) and, (rm)*. = y'n Cmm (he)).
(42)
(43)
Subtracting both the equations and taking expectation of the la norm of their difference obtain:
E Bll((r - (rll) = [[I[Cmm ((hm)) - [l]]
(44)
B[I((r'no (rll) [[][Cmm (Amm (hin)o + Amm (ht)) Avem (h ).)l
(45)
B(I((rm) - (r'm) = [ [l[Cmm Amm (him) - (Ri)dd + Cmm Σ mytt
Amn (h
(46)
Given p(Amm - Ammm m < 1, the quantity (ri)(r)))
finite.
Therefore from inequality 46 we can say that Ell() - (el S 8m where, man is
function of B[I((")) (rm)) Cmmy Amm & En Amn (h()).
A.4.10 PROOF THEOREM 6.3
After convergence, we know that:
(r [1) = E ym Cmm Amm (hinde + Amm Ammmninn Uin )
(47)
22

---

Published as a conference paper at ICLR 2025
We also know from E
V Amn Ls]
0 the following:
I0 Amm Hm
Σ
Amn (hi))
=0
(48)
Substituting equation 48 in equation
we obtain:
(r( B ym
Cmm Ammm(hm))
Σ
Amm (
(49)
We know from the residuals equation of centralized oracle that the following is true:
(Amm (Rim)o + Σ Amn (hi)))) )
E( m0 =
Vm CCm
Subtracting equation 50 from 49 and using results from proposition 6.2 i.c., Elll( - (Aim)all $
8m
maz
we obtain:
E
Σ m
[Amn Amal (ha-t)))l blll < 11Ammoy Smzll +
Σ
Aina 5manmall
Vm E fl,..., M}
A.4.11 PROOF OF COROLLARY 6.4
If the vectors [Am. Amanl (h/-1) collinear Vm {L, M). and min ElI((h-11) ).1l) exists. Then the following is true
Σ m
Amn Amn (h 2hll
E
Azen
Amn |1-11(h )a
(51)
(
A
Amnn
min mom
'
(52)
Let o min
minnnmm Blll(ht- II]. then using results of theorem
6.3 and inequality
52 we can infer
that, Emytm (diin
Amn]]
S
(1AAmmm)) Maammmaal +
EΣ En
Ama
6 maa
A.4.12 PROOF OF THEOREM
Proof. First, we calculate the fz-sensitivities of (f(.)e and (a with respect to yin. From the definition of the estimated states of client model (see 1st column of Table 1 we know that,
( = = Km KmCCmii Dm) in + (Km).ym
Since hf, constant, the sensitivity is:
AS. max
|| Je - (Km) )
(53)
vin it
[l(mm))lll
max 11 siill1 wf
(54)
< (Kyll(Km))ll
(55)
2B,Br
(56)
From the definition of the estimated states of the augmented client model (see 2nd column of Table know that,
(1% (h'n)c Omyin-
23

---

Published as a conference paper at ICLR 2025
Thus, the associated sensitivity is:
ASa-
max
(himda (hm))
(57)
Vin Min-
max
(1 + mm (Km)athe - 0mbiill2
(58)
Min.t
= (m + 0mll2 12 max, ufn
11 - 3/all
(59)
S <2 + 0mll2
(60)
S 2ll (mm)l(((m)ll)) 10mll2)
(61)
(Bx + (Ba)
(62)
Therefore we obtain the conditions on 0, and o ensure (c., and (c): differential privacy
respectively. Finally using Definition A.7 we say that the mechanism satisfies (ec Eas 8, + 8, differential privacy.
A.4.13 PROOF THEOREM 7.2
Proof. We aim to compute the fz-sensitivity of the clipped gradient matrix Clip (V(RL) Los Cs) with respect to any single client's data (states).
Using Definition
A.8 we know that,
Clip (Viis.)a 1 CCg
(63)
Let D and D' be neighboring datasets differing only in the data (states) of a single client m. The server computes the gradient matrix with respect all clients' augmented estimated states. The change the clipped gradient matrix due to the change in client m''s s data is:
AG = G
where
G Clip
(((Rt) Ly, C.) + +... Clip V (h.,.). L,(D), Co) +... .... Clip (v Li La Co),
G'
Clip (Vett)) Las Co) +... Clip ( C,) +...+ Clip (V( Los Co)
All terms except for the m-th client's contribution remain the same in and G¹. Thus, the difference simplifies to:
AG Clip L.D)). -Cip (B L,()).CC))
Using the triangle inequality for the Frobenius norm:
|AG|
Clip
V Vihan (.t.)..(DD), LD), Co) Clip (v
< Clip V Viha.) L. (D), C,) + Clip (o (C)
< Cg + C, 2Cg.
Therefore, the fe-sensitivity (with respect to the Frobenius norm) of the clipped gradient matrix bounded by 2Cg-
To achieve (s, diifferrent privacy, we add Gaussian noise to each element of the gradient matrix. The standard deviation of the noise should be:
AS (1.5/0)) 2C, 2C, lm
Adding Gaussian noise with standard deviation o, to each element of the gradient matrix ensures that the mechanism satisfies (e,,))diiferenti privacy.
24

---

Published as a conference paper at ICLR 2025
A.5 DISCUSSION ON PRIVACY ANALYSIS
We discuss a comprehensive privacy analysis of our federated Granger causality learning framework within the context of differential privacy. We begin by introducing key definitions, which form the foundation for our analysis. Readers are encouraged to refer to Dwork & Roth (2014) for a detailed discussion on these definitions.
Definition A.4 (Differential Privacy). A randomized mechanism M satisfies (s, diifferrrni privacy if for all measurable subsets S of the output space and for any two neighboring datasets D and D' differing in at most one element,
Pr(M(D) E S EF Pr(M(D)) E S] +
Definition A.5 (&2 (Ey-Sensitivity). The se-sensityv AS of a function f D:D I8 is the maximum change in the output's f2-norm due to a change in a single data point:
AS
max lI( [(D)) D.D
where and D' are neighboring datasets.
Definition A.6 (Gaussian Mechanism). Given function f D R* with fz-sensitivity AS, the Gaussian mechanism M adds noise drawn from a Gaussian distribution to each output component:
M(D) f(D) + NN
where a 2 ASv ASv2In 21n.22)/ ensures that M satisfies (s, 5dddifferenttal privacy.
Definition A.7 (Sequential Composition). Let M1 and M2 be two randomized mechanisms such that, (1) MM1 satisfies (c1, 51 ddifferrent privacy, and (2) M2 satisfies (E2, d2)-different privacy.
When applied sequentially to the same dataset D, the combination of MM and M2 satisfies (ei + E2, 61 + 82) --))-differential privacy.
Definition A.8 (Clipping Function). Given a matrix G, the clipping function Clip(G, Cg) scales G' to ensure its Frobenius norm does not exceed Cg i.c.
Clip(G, Cg) = G min (1 (M
where ||G||| denotes the Frobenius norm of matrix G
To establish the main theoretical results related to privacy (theorems 7.1 and 7.2) rely the following assumptions:
Assumption A.9, The client's measurement data yfn is bounded i.e.. ll S By, Vm,
Assumption A.10. The Kalman gain matrix (Km), is bounded i.e., || (KK))lla S Вк, Vm
Assumption A.11. The aug. client model parameter 6, is bounded i.e.. |10 S Bo, Vm Assumption A.12, The predicted states of the client model i.e., (Nim). is constant w.r.t. vin Vm,,
Reasoning Behind Assumptions: The rationale behind the above assumptions are as follows:
1. Assumption A.9 is ensured by a stable state-space model. This means if the underlying dynamics the system is stable (called "Bounded Input Bounded Output" stable state space literature), then the l2 norm of measurements i.e., Ily'm 2 are bounded.
2. Assumption A. is again ensured at the client model. This is because, at client m the Kalman gain (Km)c is a tuning parameter that weights the correction term in Kalman filter algorithm (please refer to Appendix A.
3. Assumption A.II
direct consequence regularizing machine learning at the aug- mented client model. This is because 6,, is the ML parameter at client m, and regularizing it implies bounding the MI. parameter.
4. Assumption A.12 enforced by the definition of Kalman filter (please Appendix A.I This because hy is the predicted state at time t 1 when the system was observing y".. At the current time i.e., time 1,, hi is constant for the model.
Readers are encouraged to refer to Abadi et al. (016)) for details on the gradient clipping & pertur- bations explained in the context of deep neural networks.
25

---

Published as a conference paper at ICLR 2025
A.6 ADDRESSING JON-IID DATA DISTRIBUTIONS
In this section we prove using theorem A.14 that our framework inherently encompasses non-III data across clients. To simplify our case, will assume the process noise homogeneous across states i.e., it a i.i.d. zero mean gaussian with covariance q/.
Proposition A.13 below gives an equation for the steady-state covariance matrix of state-space model. This proposition will serve as a pre-requisite in proving our main result in theorem A.
Proposition A.13. For the two-client system defined above, at time let the covariance matrix be defined as M == h*). Then the steady-state covariance matrix satisfies, Do = A +ql
Proof. Using state-space equation, we can write the covariance of h as,
E +++7] [((AA* + w (An" w=²')
(64)
Expanding the right-hand side, we get:
E[++11++17 AE[ JAT E(w'r (au'w"" AB/h'' E(w
(65)
Since w is zero mean gaussian noise with covariance ql, we have,
Blw"u qT. The noise are independent from the states (since noise as i.i.d.) Therefore, E Www 0.
(66)
Eih'
(67)
We can substitute equations 67 and 66 in equation 63 we obtain,
E = AE 9l
(68)
Taking lim on both sides of equation 68 we obtain,
Ex AE 9l
For the ease of explanation, we assume two client system with state space representation s.t.,
h'
(69)
h
where, the state evolves according to the following dynamics (see appendix A.I for preliminaries):
h Ah tht-1
(70)
where:
An A21
A12 A22
with off-diagonal blocks A12 + O, and A21 +
w/ is zero mean i.i.d. Gaussian process with covariance matrix qT, i.e., w ~ N)),gT), where is the variance of the process noise and / is the identity matrix.
The corresponding measurement equation is: Ch v
(71)
where v is the is zero mean iid. Gaussian measurement with covariance matrix rl, i.c., w' (O, rf), where is the variance of the measurement noise and I is the identity matrix. measurement noise.
Note: We consider simpler of diagonal covariance matrix for process and measurement noise. If we can prove h and hJ are non-III for this case, will imply non-IID even for the general case (non diagonal noise covariance).
Without loss of generality, we will consider An # Azz, and A12 # 0, A21 # 0. Then we have the following theorem:
26

---

Published as a conference paper at ICLR 2025
Theorem A.14. For the two-client state-space model defined above, the h1 and h2 the state vector are (1) not identically distributed, and (2) are dependent in the steady state (as + oc)
Proof. (1) Non-Identical Distribution: The steady-state covariance matrix Ex of the state vector
h' satisfies the equation of proposition A.13 1.13as as follows:
Ex A + ql
where 20 0n
012
is the steady-state covariance matrix, with 01 and 022 representing the respectively.
112 022
variances of h and h
The system of equations derived from this Lyapunov equation for dn, and d22 is as follows:
01 = A 01 + A/2022 +
(72)
022 = A + 222 +
(73)
Solving this system shows that the variances 011 and 022 are generally different. Therefore, hf and hū not identically distributed.
(2) Dependence: The covariance 012 between hf and h2" is given by the off-diagonal of Eso s.t.,
012
A11 + A12 + 4A12A21
(74)
If A12 # Oor A21 # 0, then 12 * 0, implying that the components h and h are dependent.
Note: Theorem A.14 be extended to general M client state space model where terms such as A22022 will be replaced by Enzm Amn Enn Vm E { {1,... M}
A.7 INTERPRETATION OF THEORETICAL RESULTS
A.7.1 UNDERSTANDING DECENTRALIZATION THROUGH CENTRALIZED LENS
1. Theorem 5.1
The theorem highlights the co-dependence between client and server mod- els, showing that the update of the client model iteration (k + depends on the server model's parameters at iteration k, and vice versa. This interdependence facilitates coordi- nated learning. ensuring that both models continuously align and refine their understanding of cross-client causality during iterative optimization.
2. Corollary 5.2
This result establishes that the convergence of client model parameters is intrinsically linked to the convergence of server model parameters, and vice versa. It underscores the necessity for both models to stabilize simultaneously for the framework to achieve convergent solution, validating the iterative learning process.
3. Proposition 5.3; Proposition provides closed-form expressions for the optimal parameters of the client and server models after convergence. These expressions serve as theoretical benchmarks, helping to validate the framework's 's capability to achieve optimal solutions reflecting true interdependencies.
4. Theorem 5.4 This theorem formulates the decentralized learning process unified recurrent equation. demonstrates that the federated framework can be viewed as solving single recurrent system of linear equations, bridging client and server optimization steps into a cohesive framework.
5. Lemma 5.5: The lemma provides the convergence conditions for the federated framework, stating that convergence occurs if and only if the spectral radius of the recurrent system's 's transition matrix is less than 1. It also gives the stationary value of the combined parameter vector, which represents the framework's 's equilibrium state.
6. Theorem 5.6 the joint loss function is convex and Lipschitz smooth, the theorem guar- antees sub-linear convergence rate of O(1/k) under appropriate step-size conditions. This result ensures that the framework's iterative optimization process is computationally effi- cient and steadily progresses towards optimality.
7. Theorem 5.7 For strongly convex loss functions, this theorem establishes linear con- vergence rate of O((1 /1/ ())) where y is the strong convexity constant, and L is the Lipsehitz constant. This highlights the framework's 's efficiency in scenarios where the loss landscape is well-conditioned.
27

---

Published as a conference paper at ICLR 2025
A.7.2 ASYMPTOTIC CONVERGENCE TO THE CENTRALIZED ORACLE
Theorem 6.1 The theorem demonstrates that the augmented client model's predicted states converge. in expectation, to those of centralized oracle. This result provides theoretical guarantee that the framework approximates the oracle's performance despite operating in a decentralized manner.
2. Proposition 6.2 The proposition bounds the difference between the estimated states of the centralized oracle and the client model. It provides a measure of how well the decentralized framework can approximate the centralized oracle, emphasizing the quality of the learned interdependencies.
3. Theorem 6.3 This theorem provides upper bound on the enror in estimating the state matrix's s off off-diagonal blocks, which represent cross-client causality. The bound is given without requiring prior knowledge of the oracle's ground truth, demonstrating the robust- ness of the framework's 's causal inference.
4. Theorem 6.4;
The corollary refines the error bound by incorporating conditions on the ora- cle's state predictions, offering tighter guarantees on the accuracy of the estimated causality structure under specific statistical assumptions.
A.7.3 PRIVACY ANALYSIS
1. Theorem 7.1. The theorem formalizes the privacy guarantees for flient-to-served com- munication. It demonstrates how adding noise to client model updates ensures (E differential privacy, protecting individual clients' data while enabling accurate learning of interdependencies.
2. Theorem 7:2: This theorem provides differential privacy guarantees for server-to-client communication. ensures that server updates shared with clients do not reveal sensitive information about other clients' data (measurements), maintaining privacy while support- ing collaborative learning.
A.7.4 ADDRESSING NON-IID DATA DISTRIBUTIONS
The proposition establishes the steady-state covariance equation for state-space model, laying the groundwork for analyzing stability and convergence proper- ties of the federated framework under stochastic dynamics.
Proposition A.13
2. Theorem A.14 The theorem proves that the federated framework inherently assumes and handles non-llI data distributions across clients. It shows that the states of different clients are both dependent and non-identically distributed, reflecting real-world heterogeneity.
A.8 PERFORMANCE AGAINST BASELINES
Based on the results in Table 4 the following observations can be made when comparing the perfor- mance of our method against the three basclines:
1. No Client Augmentation: In this baseline, the clients do not augment their models with machine learning functions, thus ignoring the effects of interdependencies with other clients. As a result, the server model not expected learn cross-client causality. This is evident in the server loss L,, which remains constant at minimal value of 10-5 across all perturbations and topology changes. Without client augmentation, the framework nei- ther learns nor detects causality changes. This underscores the importance of incorporating interdependencies at the client level.
2. No Server Model: This baseline removes the server model entirely, leaving clients to operate without any coordination. Here, the client loss (L2), increases with higher levels of perturbation (c) and topology changes, thus raising a local (client-level) flag. However, due to the absence of server model, one cannot confirm that the increases in client loss are due changes in causality rather than local anomalies. This experiment highlights the crucial role of the server model in capturing and validating interdependencies.
3. Tre-Trained Clients: In this case, the client models are augmented with machine learn- ing functions but pre-trained independently without any iterative optimization with the
28

---

Published as a conference paper at ICLR 2025
server. Both the client loss (L2), and the server loss Lg increase with higher perturbations and topology changes. However, we observe larger Lg values compared to our method. This suggests that this baseline is over-sensitive to changes in causality, possibly due to overfitting at the client level in such pre-trained models. This observation requires further investigation.
A.9 LIMITATIONS AND FUTURE WORK
1. Scalability: While this paper focused on Granger causality using linear state-space model, it did not explore the scalability of the framework. Our current work derived the- oretical characteristies based on linear assumptions, which, although insightful, may limit applicability to more complex systems. The true potential of the framework emerges when we replace the linear state-space model at the server with more advanced machine learning models like Deep State Space Models, and Graph Neural Networks. These sophisticated models can capture more intricate interdependencies among larger number of clients while still preserving data privacy. Investigating our framework with these enhancements offers rich avenue for future research, enabling it to handle complex, high-dimensional data and providing deeper insights into the interconnected dynamics of large-scale systems.
2. Complex Interdependencies: The interdependencies in the current models are expressed in the elements of A matrices. 'he extension of this work stochastic interdependencies also represents valuable direction for future research and opens the door to incorporating probabilistic models such Dynamic Bayesian Networks. Additionally, the interdepen- dencies can also have time varying effects which are highly encountered in real world applications and therefore worthy of investigation.
3. Higher Order Temporal Dependencies: In this framework, we considered only constant one-step time lag that is uniform across all clients. However, real-world systems often ex- hibit more complex temporal dynamics, where dependencies can span multiple time steps and vary significantly between clients. Investigating the theoretical characteristics of the framework under higher-order temporal dependencies could provide deeper insights into its performance and applicability. This extension is particularly important for capturing more nuanced temporal patterns using Recurrent Neural Network, and Transformer making a promising avenue for future exploration.
4. Incorporating Decision Making: Both the server and client models in our framework function as data analytic models. A natural extension is to incorporate decision-making capabilities, where clients act based on local data while accounting for inferred interdepen- dencies. This aligns with work in Multi-Agent Reinforcement Learning which emphasizes centralized planning and decentralized execution. Investigating our framework within this context offers a compelling direction for future research, as it could enable more dynamic interactions between clients and the server while preserving data privacy.
5. Robustness: Our framework assumes ideal conditions, including synchronous updates and reliable client participation. However, real-world scenarios often involve client dropout, asynchronous updates, and even malicious clients. Addressing these challenges is essen- tial to improve the framework's robustness. Developing mechanisms to handle these issues, particularly in adversarial settings, would enhance the system's 's resilience and reliability. Exploring such enhancements represents an important direction for future research.
6. Privacy and Security: Our approach was motivated by the logistical challenges of han- dling high-dimensional measurements, rather than focusing primarily on privacy. To ad- dress privacy concerns, we provided a preliminary analysis on differential privacy (see Ap- pendix (A.5). showing how noise can be added to protect client data while maintaining the utility of learned interdependencies. This demonstrates the feasibility of privacy-preserving methods, but further exploration is needed. Incorporating advanced techniques like ho- momorphic encryption, zero-knowledge proofs, secure multiparty computation, or even more sophisticated differential privacy methods could offer stronger privacy guarantees, representing a fertile ground for further study.
29


Reviews:
[{"review_id": "6ANCoaBwre", "reviewer": "Reviewer_3GMe", "summary": "The paper presents a method of Granger causality computation in a federated setting, where clients with individual measurements do not share their data with the server directly, and a federated learning mechanism is applied to estimate the causality. The advantage of this method is that it can significantly reduce the amount of data transfer. Theoretical analysis shows that this approach converges to a baseline centralized oracle. Some results from experiments are also shown from both synthetic and real-world datasets.", "questions": "- The authors are strongly advised to provide some background knowledge on Granger causality and its connection to Kalman filters, which could be added to the appendix. \n- There should be a pseudocode with all the detailed steps of the proposed algorithm, for both the clients and the server. From the current description in the paper, it is quite hard to understand what exactly is being done. \n- Can the proposed method work beyond Kalman filter based models?\n- In Section 3, it would be helpful to clarify the relationship the state matrix and the low dimensional states. In particular, how are the functions $f_c$, $f_a$ and $f_{ML}$ usually defined?\n- In Figure 1 and also in the text description, what does a model mean in this context? It seems to be different from machine learning models (e.g., neural networks).\n- It would be best to define the centralized model and $C_{mm}$ before Table 1, since they are mentioned in Table 1 and $C_{mm}$ is also mentioned in the text below it, but they are defined much later in the paper. \n- In Equations (1) and (2), why are there two losses that need to be considered separately and have different learning rates? Wouldn't it be possible to define an overall objective to combine both losses, as commonly done in federated learning literature?\n- At line 250, `The server model is a ML tool whose predictions and labels are...` I don't understand what this sentence means. \n- At line 466, `Our method performed better than all three baselines mentioned in table 4.` I cannot tell from Table 4 why the proposed method performs better than the baselines. Which numbers in Table 4 should I look at and how should I compare the numbers?\n- Some theorems are stated too informally, such as Theorem 6.2. I would expect to at least have a condition of $t\\rightarrow \\infty$ for this condition to hold. Also, what is the difference between $t$ and $k$ (e.g., in (1) and (2))?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The application of federated learning to Granger causality is interesting and unique. \n- The paper has nice theoretical analysis with some insights discussed. \n- The problem studied appears to be important for practical applications in the IoT domain.", "weaknesses": "- As someone who is not an expert in Granger causality or Kalman filters, it is quite hard to follow the detailed steps of the proposed method and why it is designed in this way. \n- The proposed method seems to only work with Kalman filter based client models.\n- It is difficult to understand the key message from the experimental results. There doesn't seem to be much comparison with baseline methods, except for Tables 4 and 7, where it is unclear how the loss values in Table 4 should be interpreted, e.g., which one is better?\n- Related to federated learning, it is not very clear what is particularly unique compared to a standard federated learning algorithm that minimizes a generic finite-sum objective function.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "QX7BHjCoUV", "reviewer": "Reviewer_T1u8", "summary": "This paper proposes a federated approach to learning Granger causality for a multi-client state space system.The convergence of the framework to a centralized oracle model is proved. Comprehensive experiments are conducted to demonstrate the robustness and scalability of this method.", "questions": "1.\tTo the best of your knowledge, is your method the first federated approach to learning Granger causality？Is there any other federated approach that can be compared to your model? \n2.\tBaselines without client augmentation or the server model are mentioned in the experiments.I wonder that what parts of these baselines are similar to yours.For example, are there baseline methods without the server model but with client augmentation?If so, could you explain the parts of these methods that are relevant to your approach?\n3.\tCould you provide more related work on Distributed Kalman Filter in the last 5 years?\n4.\tI am not sure that whether this method is of value for further research.In other words,is it only effective in a particular scenario (Granger causality learning)?\n5.\tWhat are the possible limitations of this approach? Could you provide some ideas to address them in future work?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "This paper proposes a federated linear state space system framework with client augmentation and server model for clients to learn Granger causality information, facilitating a more accurate representation of cross-client causality. This method is innovative and inspirational. \n\nTheoretical guarantees are provided on the co-dependence of the augmented client and server model. Extensive experiments on both synthetic data and real-world datasets highlights the framework’s effectiveness in learning causality. \n\nI appreciate the author’s clear writing which is easy to follow.", "weaknesses": "1.\tThe authors could discuss the scalability of their method to larger and more complex datasets. It may help explain the further expansion of this approach.\n2.\tThe limitations of the proposed method don't seem to be mentioned in the paper, which I think is a necessary discussion.\n3.\tIt is not clear if the comparison is fair enough,i.e. more details about baselines need to be demonstrated.Authors can refer to question 2 lined below.\n4.\tThe two salient characteristics of this ML function mentioned in 4.1 need to be explained in more detail.\n5.\tThere should be more details about the proof in the paper.Listing more references or giving more steps for proof is helpful.For example,the proof of Proposition 5.3 can be discussed more.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "YJ1KbpUOWz", "reviewer": "Reviewer_bEsA", "summary": "The paper introduces a Federated Granger Causality Learning framework designed to analyze interdependencies among geographically distributed clients in industrial control systems. By leveraging a decentralized machine learning approach, the framework allows clients to maintain their own models while sharing only low-dimensional state information with a central server, overcoming the challenges of bandwidth limitations and computational burdens associated with centralized data processing. The authors establish convergence dependencies between server and client models with convergence rates. Empirical validation through experiments on synthetic and real-world datasets demonstrates the robustness, scalability, and communication efficiency of the approach, highlighting its ability to save data volume while effectively capturing operational interdependencies.", "questions": "1. How does the proposed method perform in non-IID data distributions among various clients?\n2. Can you provide a detailed privacy analysis of the proposed methods, and how do you plan to address this important aspect in the context of federated learning?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. The paper presents a novel approach to learning Granger causality in a federated setting, which is a relatively underexplored area in the context of decentralized systems. \n2. The authors provide a detailed convergence analysis and establish conditions for sublinear and linear convergence rates.\n3. The paper is well-structured and clearly written\n4. The significance of this research lies in its potential impact on various industrial applications where understanding interdependencies is crucial for operational efficiency and fault management.", "weaknesses": "The paper does not include any analysis or experimental results of federated learning in non-IID settings which is an important aspect of FL. \nA detailed privacy analysis of the proposed method is missing which is very important in a federated setting.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "9T7BcL4XIp", "reviewer": "Reviewer_GCuL", "summary": "This work presents a federated learning approach to detect Granger causality among geographically distributed industrial operations, addressing the challenges of large-scale, complex data. Using a linear state space system framework, it reduces bandwidth and computational demands by leveraging low-dimensional state estimates.", "questions": "Please refer to my comments above regarding suggestions to improve the paper.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "* The paper studied a fresh problem in the federated setup. \n\n*The presentation of the paper is solid and the results presented are of the interest to the community. \n\n*Although the paper studies a new problem that is unconventional to the people working on federated learning, the presentation of the paper makes it understandable to the general audience. \n\n*The results presented carry novelty. Also, the simulation section is solid covering various network settings.", "weaknesses": "The paper does not suffer from a clear weakness. Although there are some room for improvement:\n\n1) Please read the paper and correct the typos. For example, there is extra space before the comma in line 75.\n\n2) Please fix the typos in the paper. For example, line 329 refers to Theorem 5.5 but I think it should be Lemma 5.5.\n\n3) Without clear explanation of the results of the theorems and lemmas it is a bit hard to clearly think about the results and appreciate them. Given the space limitation in the paper, I suggest that you add clear discussions about the results obtained in the main text in the appendix. In particular, more discussions about the \"interpretation\" of the theoretical results can be added in the appendix to help the reader to appreciate the results.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)