Title: Streaming Federated Learning with Markovian Data
Venue: NeurIPS 2025
Accepted: yes
Keywords: Optimization - Non-convex

Federated Learning

Markovian Processes

Stochastic Approximation
Abstract: Federated learning (FL) is now recognized as a key framework for communication-efficient collaborative learning. Most theoretical and empirical studies, however, rely on the assumption that clients have access to pre-collected data sets, with limited investigation into scenarios where clients continuously collect data. In many real-world applications, particularly when data is generated by physical or biological processes, client data streams are often modeled by non-stationary Markov processes. Unlike standard i.i.d. sampling, the performance of FL with Markovian data streams remains poorly understood due to the statistical dependencies between client samples over time. In this paper, we investigate whether FL can still support collaborative learning with Markovian data streams. Specifically, we analyze the performance of Minibatch SGD, Local SGD, and a variant of Local SGD with momentum. We answer affirmatively under standard assumptions and smooth non-convex client objectives: the sample complexity is proportional to the inverse of the number of clients with a communication complexity comparable to the i.i.d. scenario. However, the sample complexity for Markovian data streams remains higher than for i.i.d. sampling. Our analysis is validated via experiments with real pollution monitoring time series data.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\n1. This paper establishes the upper bounds (of convergence rates) for Minibatch SGD and Local SGD (FL) on Markovian data.\n2. This paper establishes the upper bounds of the momentum-based variants of Local SGD.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**:\n1. The upper bounds of FL on Markovian data are novel (to the best of my knowledge) in the FL literature.\n2. The momentum-based variants are also new for FL on Markovian data.\n3. The amount of work in this paper is sufficient.\n4. The paper is well-organized and well-written.\n\n**Weaknesses**:\n\n1. My **main concern** is that there exists one non-vanishing term (as $T$ approaches infinity) in the upper bounds (e.g., Theorems 4.5 and 4.7). This is quite different from the classical results of Markov chain SGD [+1,+2,+3,+4] and FL [+5], whose upper bounds approach zero as $T$ approaches infinity if an appropriate step size is chosen.\n\n   [+1] Finite-Time Analysis of Markov Gradient Descent, TAC, 2022.\n\n   [+2] Adapting to Mixing Time in Stochastic Optimization with Markovian Data, ICML, 2022.\n\n   [+3] Stochastic Gradient Descent under Markovian Sampling Schemes, ICML, 2023.\n\n   [+4] First Order Methods with Markovian Noise: from Acceleration to Variational Inequalities, NeurIPS, 2023.\n\n   [+5] Tighter Theory for Local SGD on Identical and Heterogeneous Data, AISTATS, 2020.\n\n2. Could you discuss the different measures used for Makrov chain in this paper and previous works? After checking some works of Markov chain SGD, I note that the mixing time is common (e.g., [+3]). So could you elaborate on why we use $C_{\\infty}$ and $\\nu_{ps}$, rather than the mixing time (or other measures). I am curious about this.\n\nQUESTIONS:\n**Questions**:\n\nI am not familiar with Markov chain. So my questions are focused on the part related to Markov chain.\n\n1. Section 4.1, Assumptions, Lines 214--215: What does $\\tau$ mean?\n\n2. Proof of Lemma B.1, Lines 581--582. What does $x_{t-1}^{K-1} = x$ mean? Can you elaborate on the change of measures?\n\n3. Proof of Lemma B.1, Line 582. What does $|| \\cdot ||_{\\pi, \\infty}$ mean? Can you elaborate on how to use Holder's inequality? You can write the form of Holder's inequality used here and specify how to use it.\n\n4. Proof of Theorem B.3, Line 607. Can you elaborate on this equality?\n\n5. Proof of Lemma B.5, Line 626, the first inequality. Is it possible to do as follows?\n   \n    $$\n   \\begin{align*}\n   \\xi_t &= \\frac{1}{MK} \\sum_{m,k} \\eta^2  E_t \\\\| \\sum_{j=0}^{k-1}\\nabla f_m(w_t^{(m,j)};x_t^{(m,j)}) \\\\|^2 \\\\\\\\\n   &\\leq \\frac{1}{MK} \\sum_{m,k} \\eta^2  \\left( E_t \\\\| \\sum_{j=0}^{k-1}\\nabla f_m(w_t^{(m,j)};x_t^{(m,j)}) - \\sum_{j=0}^{k-1}\\nabla f_m(w_t;x_t^{(m,j)}) \\\\|^2 + E_t \\\\| \\sum_{j=0}^{k-1}\\nabla f_m(w_t;x_t^{(m,j)})- \\sum_{j=0}^{k-1}\\nabla F_m(w_t) \\\\|^2 + E_t \\\\|  \\sum_{j=0}^{k-1}\\nabla F_m(w_t) \\\\|^2 \\right)\n   \\end{align*}\n   $$\n\n   The above inequality has three terms. Can the second term be bounded by [41, Theorem 3.7] (as done in Lemma B.1)? If so, does this cause a different bound for $\\xi_t$? Does this cause different convergence rates for FL?\n\n**Suggestions**:\n\n1. Could you add the comparisons of upper bounds with Minibatch SGD and Local SGD with IID data for both without momentum and with momentum settings in a more straightforward way? One simple way is collect them in one table and highlight the main difference in different colors, just like the tables in [+6,+7]. This can help the readers see their differences more clearly.\n\n   [+6] Minibatch vs Local SGD for Heterogeneous Distributed Learning, NeurIPS, 2020.\n\n   [+7] Momentum Benefits Non-Iid Federated Learning Simply and Provably, ICLR, 2024.\n\n2. Could you add the comparisons with SGD on Markovian data in a more straightforward way? One simple way is collect them in one table and highlight the main difference in different colors, just like the tables in [+6,+7].\n\n3. Is it possible to remove the \"streaming\" in the title. Compared to the existing works focusing on FL with IID data [+5], this work study FL with Markovian data. Nevertheless, this work still considers the typical stochastic optimization, instead of online stochastic optimization. Thus, in my opinion, \"streaming\" may be unnecessary, and may misguide the readers into the works of online federated learning [+8,+9], which also claimed to address the problem of streaming data.\n\n   I know that this may cause some revisions of the introduction. I think that [+10] (see their introduction) will be helpful.\n\n   [+8] Asynchronous Online Federated Learning for Edge Devices with Non-IID Data, BigData, 2020.\n\n   [+9] Online Federated Learning, CDC, 2021.\n\n   [+10] On the Decentralized Stochastic Gradient Descent With Markov Chain Sampling, IEEE Transactions on Signal Processing, 2023.\n\n**Scores**:\n\nIf character count is limited, please focus on Weaknesses.\n\n1. I will raise my score to 4, as long as Weaknesses 1 and 2 are well explained. However, I cannot guarantee a higher score for now, as I am not an expert in Markov chains.\n2. I have checked the proofs of Theorems 4.5--4.8 (without momentum) and have the questions above. Explaining them clearly will make this work more convincing and accessible.\n3. The suggestions are not enforced. Nevertheless, I hope that the authors can consider them.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-17"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses the challenges of Federated Learning (FL) in the context of streaming, non-i.i.d. data, specifically focusing on Markovian data streams, which frequently arise in real-world applications such as health and environmental monitoring. Unlike standard FL setups where clients train on static datasets, the authors consider data generated by time-homogeneous Markov processes, which introduces temporal dependencies and non-stationarity. The authors conduct a rigorous theoretical analysis of three FL algorithms in this setting: (i) Minibatch SGD; (ii) Local SGD, and (iii) Local SGD with Momentum (Local SGD-M). In addition to the theoretical contributions, the paper includes experiments on a regression task using a real-world dataset to illustrate the behavior of the algorithms under Markovian sampling.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\n1) The paper tackles a realistic and underexplored setting in FL by modeling client data streams as non-stationary Markov processes, a framework that reflects the dynamics of many physical and biological systems.\n\n2) The theoretical analysis is rigorous and well-written, and characterizes the convergence behavior of Minibatch SGD and Local SGD under Markovian sampling. In particular, the results confirm that linear speedup (i.e., sample complexity scaling with $1/M$) is achievable under standard smoothness and bounded heterogeneity assumptions.\n\n3) An important contribution is the analysis of Local SGD with Momentum, which is shown to mitigate client drift due to data heterogeneity under Markovian streaming, achieving convergence guarantees without the need for gradient dissimilarity assumptions.\n\n**Weaknesses**\n\n1) *Main concern* : The variance term in the convergence bounds is independent of the step-size, which seems suboptimal. Intuitively, one would expect the variance term to decay as the step size decreases since smaller steps imply smaller parameter movement from initialization. This contradicts both empirical intuition and existing results in the literature (e.g., [1]), where the variance is weighted by the step-size. The authors should either provide theoretical justification for why such an \"incompressible\" variance term appears, or offer empirical evidence of its presence and persistence across different step-size regimes.\n\n2) In Section 4.3 (\"Impact of Markovian Data\"), the paper references [2] and asserts that the bias term in Theorem 2 of [2] is persistent and uncontrollable even with increased local computation. However, the bias in [2] originates from client heterogeneity, not from Markovian sampling. This remark is misleading and should be relocated to the section discussing heterogeneity.\n\n3)  In the experimental section, the choice of hyperparameters lacks consistency with the theoretical findings. For example, the theory suggests that the step size should scale as $\\mathcal{O}(K)$, yet the experiments use a fixed step size across different values of $K$. This makes it difficult to isolate whether stagnation in performance is due to the variance term, the heterogeneity term, or simply hyperparameter mismatch. I strongly suggest including either experiments where step-size is scaled according to theory, or to select the hyperparameters according to a grid search, to better support the empirical conclusions drawn in the paper.\n\n[1] Zhu,F. ,Mitra,A., and Heath, R.W. (2025) . Achieving tighter finite-time rates for heterogeneous federated stochastic approximation under markovian sampling.\n\n[2] Han Wang, Aritra Mitra, Hamed Hassani, George J. Pappas, and James Anderson. Federated TD learning with linear function approximation under environmental heterogeneity. Transactions on Machine Learning Research, 2024\n\nQUESTIONS:\n1) Can you clarify why the variance term in your convergence bounds is not controlled by the step-size, unlike in [1]? Can you either refine the analysis or show empirically that this term does not vanish with a smaller step-size?\n\n2) Would it be possible to rerun or supplement the experiments with step-size schedules that follow the theoretical scaling with respect to $K$ or a grid search?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-01"}

comments: SUMMARY:
The work provides methods for federated learning in the setting where clients hold a sliding window of K most-recent samples that evolve as a time-homogeneous Markov chain converging to a stationary distribution.

STRENGTHS AND WEAKNESSES:
Strengths: 
The convergence bounds provided are sharp. The method has a good communication efficiency.
The problem is well motivated as such temporal streams of data are quite common in the federated setting. 

It provides some mixing-time results to understand the convergence to a statioary target distribution. 

Main weakness:
1.) In order to guarantee convergence, they suggest that many local steps are needed. Can you share some more info about this?

QUESTIONS:
1.) Would it be preferred to have a small $\theta$ and $\delta$, in the context of the  bounded gradient dissimilarity (BGD) assumption? If so, how would you ensure that in your work?

2.) How would extending to mini-batch updates (multiple correlated samples) change the convergence results?

3.) What is local local SGD on line 212 and how is it different from local SGD? Is local local SGD a typo or refers to Local SGD with Momentum?

4.) What is the difference in the setup in proposition C.10 Vs. the equation between line 740 and 741 that give mixing time results?

LIMITATIONS:
These have been described in great detail.
TITLE: Official Review
CLARITY: 3
IMPACT: 4
ORIGINALITY: 3
DATE: 2025-07-02

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents a rigorous theoretical and empirical study of federated learning (FL) under Markovian streaming data, a realistic and understudied scenario in FL. The key contribution lies in providing convergence guarantees and communication/sample complexity bounds for Minibatch SGD, Local SGD, and Local SGD with Momentum (Local SGD-M), all under non-i.i.d., temporally correlated data generated by Markov chains. The work is well-motivated, theoretically sound, and empirically validated.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n- The paper is well-organized, beginning with a strong motivation and providing precise mathematical modeling.\n- The paper presents a rigorous theoretical analysis of FL under Markovian data streams, an underexplored yet realistic setting.\n- It provides tight convergence bounds for Minibatch SGD, Local SGD, and Local SGD-M, under well-stated assumptions.\n- Detailed proofs and assumptions are provided in the appendix; results are reproducible with shared code and setup.\n- Empirical validation on real-world, temporally dependent data strongly supports the theoretical findings.\n\n## Weaknesses\n- While the theoretical results are strong, some assumptions (e.g., bounded Radon-Nikodym derivative, fixed spectral gap) may not be easily verifiable in practice.\n- The experimental section is somewhat limited, only one real dataset is used, and synthetic settings are only briefly mentioned.\n- The practical implications for deployment in real FL systems are not deeply explored. For example, how to estimate spectral gap or choose $K$ in practice is left unaddressed.\n\nQUESTIONS:\n1. The paper evaluates performance on a single real-world dataset (air-quality time series). How generalizable are the findings across domains?\n2. You suggest that momentum-based methods (SGD-M) mitigate heterogeneity without requiring control variates (e.g., SCAFFOLD). Could you provide direct empirical comparison with control variate-based methods, such as SCAFFOLD or SCAFFLSA, under the Markovian setting?\n3. The theoretical results require careful tuning of parameters like step sizes $\\gamma$, $\\eta$, and momentum $\\beta$ based on unknown quantities. Is there an adaptive strategy or heuristic the authors can recommend for real-world deployment?\n4. All your evaluation metrics focus on gradient norm as the convergence criterion. Can you report test accuracy or loss (e.g., RMSE on PM2.5) to give a sense of practical performance?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}