Distributed Hierarchical Decomposition Framework for Multimodal Timeseries Prediction
Authors: Wei Ye
Prashant Khanduri
Jiangweizhi Peng
Feng Tian
Jun Gao
Jie Ding
Zhi-Li Zhang
Mingyi Hong
Venue: Accepted by TMLR ()

Abstract: We consider a distributed time series forecasting problem where multiple distributed nodes each observing a local time series (of potentially different modality) collaborate to make both local and global forecasts. This problem is particularly challenging because each node only observes time series generated from a subset of sources, making it challenging to utilize correlations among different streams for accurate forecasting; and the data streams observed at each node may represent different modalities, leading to heterogeneous computational requirements among nodes. To tackle these challenges, we propose a hierarchical learning framework, consisting of multiple local models and a global model, and provide a suite of efficient training algorithms to achieve high local and global forecasting accuracy. We theoretically establish the convergence of the proposed framework and demonstrate the effectiveness of the proposed approach using several time series forecasting tasks, with the (somewhat surprising) observation that the proposed distributed models can match, or even outperform centralized ones.

Published in Transactions on Machine Learning Research (11/2025)
Distributed Hierarchical Decomposition Framework for Multimodal Timeseries Prediction
Wei Yel, Prashant Khandurfi, Jiangweizhi Peng', Feng Tian', Jun Gao", Jie Ding', Zhi-Li Zhang', Mingyi Hong'
UUiivessity of Minnesota Twin Cities; Wayne State University; "Meta Platforms, Inc.
ye000094 @umn.edu, khanduri. dingf0umn. edu, zhshang
thanduri.proshent@@wee prwshantwwayyee edu,, (00g0e0000mmmedd. tianz399@umn.cde jungao Cmngaommet,.omm
Rcs, &cs, umn, edu, @ahon@@umn.cem edu
Reviewed OpenReview: https: // o/pereeveee net/ per//frmm? id= FFXWs9His LJ
Abstract
We consider a distributed time series forecasting problem where multiple distributed nodes each observing a local time series (of potentially different modalities) collaborate to make both local and global forecasts. This problem is particularly challenging because each node only observes time series generated from subset of sources, making challenging utilize correlations among different streams for accurate forecasting; and the data streams observed at cach node may represent different modalities, leading to heterogeneous computational requirements among nodes. To tackle these challenges, we propose hierurchical learning framework, consisting of multiple local models and a global model, and provides suite of efficient training algorithms to achieve high local and global forecasting accuracy. We theoretically establish the convergence of the proposed framework and demonstrate the effectiveness of the proposed approach using several time series forecasting tasks.
1 INTRODUCTION
Time series forecasting has diverse applications in various scientific disciplines, such as healthcare (Che et al., 2018), astronomy (Scargle, 1982), finance (Batres-Estrada, 2015), meteorology (Shi et al., 2015), and traffic engineering (Zhang et al., 2017). Historically, regression-based methods have enjoyed widespread popularity for time series forecasting (Box et al., 2015; Toda & Phillips, 1994; Frigola, 2015). However, the recent success of deep lcarning architectures for sequence learning tasks has led to the development of deep lcarning-basee time series prediction algorithiss that outperform the state-of-the-arl regression methods (Wu al., 2020b; Zheng et al., 2020; Lai et al., 2018; Shih et al., 2019; Beltagy et al., 2020).
Despite the success of deep learning architectures for time series forecasting, research on distributed time series forecasting has been scarce. With the advent of distributed data collection in contemporary settings, there is a pressing need for the development of efficient distributed implementations of time series prediction algorithms. However, a notable gap exists in the availability of distributed forecasting modules at a large scale, particularly for scenarios where individual nodes observe only a subset of the entire data stream. this work, we address a time series forecasting problem wherein multiple distributed nodes collaborate using their local data streams to generate both local (e.g., forecasting local traffic) and global predictions (e.g., predicting aggregated traffic). Only handful of works have addressed the problem, with the learning algorithms introduced being straightforward adaptations of centralized approaches (Nguyen et al., 2019; Perera et al., 2022; Wang et al., 2022). The existing distributed time series forccasting algorithms fall short of addressing requirements, such as safeguarding data privacy, capability to handle multimodal data, and providing model flexibility. Key challenges in developing a distributed implementation of time series forecasting systems are: Every node captures only fraction of the time series data, which introduces complexity in the lcarning process, as effective time series forecasting relies on harnessing the correlations within the data streams (Lai al., 2018; Wu et al., 2020b; Zheng al., 2020).

---

Published in Transactions on Machine Learning Research (11/2025)
@ The data streams observed at each node may represent a different modality, e.g., the modern Internet of Things (IoT) (Atzori et al., 2010). Making accurate predictions with multimodal data requires each node to choose an appropriate model for its modality, and importantly demands that the information distilled from these heterogencous models be properly fused.
@ Communication a major bottleneck in distributed learning (McMahan et al., 2017b), therefore, there is a need to develop algorithms that avoid high-dimensional parameter sharing and reduce privacy risks by avoiding raw data sharing for privacy-sensitive applications at the same time (Aouedi et al., 2022).
Current time series forecasting models lack well-established theoretical underpinnings. Analyzing distributed time series forecasting models presents a significant challenge due to the departure from the conventional
independent and identically distributed (i..d.) gradient assumptions in training, resulting in biased updates during the training process.
It is by no means clear how to design a system that addresses the above-mentioned challenges and provides accurate predictions while being able to: @ handle multi-modal data by leveraging current advances of the neural work-bb-ed time series models; @ utilize local data while sharing as little information among the nodes as possible while protecting data privacy; @ establish theoretical performance guarantees. In this work, we develop a novel framework, distributed vertical hicrarchical decompasition (DIVIDE), which addresses the anoove-mentioedd challenges while guaranteeing theoretical performance with non-ii.d. data samples. The applicability of the developed framework is much beyond time series prediction tasks and can be in fact used to solve general distributed learning problems. Specifically, DIVIDE subsumes a number of popular modern distributed learning systems wherein the feature space of a single data sample is distributed among the distributed nodes, such as the so-called vertical federated learning systems (Wei et al., 2022). For example, Webank uses such models for financial risk asscssment for their enterprise customers (Cheng et al., 2020). However, our work addresses siguificantly more challenging problem since individual data samples (i.e., pieces of the time series data) are correlated, and they may originate from potentially multimodal sources.
Contributions. In this work, we develop a novel distributed time series forecasting framework named DIVIDE. Central to our proposed framework is an innovative hierarchical structure, where several local models are constructed to handle data streams observed at local level, and concurrently, a global model is created to merge the embeddings generated by these local models. Further, suite of customized and efficient training algorithms has also been developed to ensure that the system generates high-quality forecasting solutions. Specifically, DIVIDE has the following major benefits:
Flexibility. It provides local computing nodes with the freedom to seleet their model architectures, i.c., each local node can choose from a range of available time series prediction models (such as those rooted in RNN or CNN), taking into account factors like their local data modality, available computing resources, and the intricacy of their specific prediction tasks.
Effective information sharing. A global model (either hosted at the server or at all the local nodes) is carefully designed to help fuse the (potentially multimodal) information generated from the local models, to best leverage the correlations among all the data streams. Importantly, the proposed algorithms avoid raw data sharing by exploring certain special structures of the loss function so that nodes only exchange low-dimensional embeddings.
Thcoretical guarantces. We represent the sampling for time scries training as a Markov chain (or scries) and demonstrate that the proposed algorithms, when applied to train the hierarchical time series forecasting model, converge towards the set of stationary solutions for the associated training problem.
2 Preliminaries
We address distributed learning problem with K nodes. Each node k E [K], observes a (potentially multimodal) time series x E Rdd, where n E N signifies the time index. We define the global time series, observed collectively across all nodes, x" Rd with d = Ek= de- The global time series x" is constructed ...., ...., )TF. We denote a sequence of multi-dimennsooal time series samples as:
as x : [(x]))
x= : [xx+1,x* x+2
xn+ RdD for E
(1)
2

---

Published in Transactions on Machine Learning Research (11/2025)
where D>
> 0 is the time window; Define the next samples
[x + D+1
x" D**]+ E Baxt for n E.N.
(2)
where T is the time horizon of the prediction. We note that for general prediction problems with streaming data, the labels y" can represent a general classification or regression task. The global task then is to predict a function of y using x", denoted as g(-) : Rdxx + R which transforms a vector of data points at a given time to a sunmary stalistics, e.g., for predicting the lotal traffic across all agents, we have
d
K du
g(y")
a+D +++++
[1]
Here we have utilized the notation (x";"* y ~ for n E N to denote the feature label pairs and where is the underlying distribution that generates the data.
Now, let's transition to the distributed scenario, where each node only access a subset of the dimensions of the global lime series x E 1R4 In this context, leveraging the local observations, can follow (1) (2) to define the tuple yx) as:
_ = +++ yk [x
X
PI
E RdxD xD
(3)
n+D+1
a+ xx
++ Rd
(4)
for E N. Furthermore, the specific objective at each local node, & E [K] is to forecast the local time series or address local inference problem up to the time horizon of T utilizing a time window D. As the local forocasting targets are denoted by ye s, we will refer to them as "labels" accessible at node thronghout. addition, we note that by using the notation of the local observations above, the tuple (x","" defined in (1)-(2) can be equivalently written as:
x" == [(x)) (x E RaxD y" = [(91)) (("")""..,,,$))))) (FR) E Rdxt
for n E N. This notation will be useful in the subsequent discussion. Overall, the goal of a forecasting algorithm is to learn a mapping from x" to g(5"), and/or mappings from each gg yk for all k, by utilizing set of training samples, so that some loss function is minimized.
3
The proposed framework
3.1 Problem formulation
In this subsection, we describe the hierarchical model adopted by the proposed DIVIDE framework, as well as its associated training problem.
Local models. We assume that each node & E [K], maintains an independent local machine learning model (e.g., LSTM (Hochreiter & Schmidhuber, 1997) or RNN (Mikolov et al., 2010)) characterized by its parameters 0% E 4 S PR. These models play a key role in conducting local predictive tasks. They take as input the data streams observed locally, and their output termed the "local embedding", denoted as fe(O;;; x for all n E N and all nodes [K]. Here, x defined as in equation (4). Importantly, it's worth noting that these embeddings often possess much lower dimensions compared to dę 7 for each node k in the set [K]. Each node is granted the flexibility to select its local model, depending its available computational resources and the modality of its local data streams, characterized as (x). V.). For the sake of brevity in our notation, we will subsequently represent (0x; x as simply (0x)
Global models. Apart from the individual local models, each node (or the server) also maintains a global machine learning model characterized by parameters 0g O S Po. This global model serves the purpose of capturing the correlations among different time series order improve the prediction accuracy. The inputs for the global model consist of the local embeddings, while its output, termed the "global embedding", is
3

---

Published in Transactions on Machine Learning Research (11/2025)
denoted as fo fi....,,fk) This global embedding shares the same dimension as the label to be predicted. For instance, in the context of time series forecasting, the dimension of the global embedding will be d x t since its design is centered around predicting the d-dimensional global time series for a time horizon of T (as detailed in equations (1) and (2). It's important to note that the global model is introduced to capture interdependeencies among diverse lime series data and, in doing so, aggregates the local embeddings from each node to learn and represent these correlations.
Problem. The overarching objective of the forecasting problem involves acquiring the optimal local parameters for each node while simultaneously optimizing the global model parameters. Defining 0 e O with == _ O S R and P = E P.. The goal of the forecasting algorithm is to jointly learn these parameters 0 E θ in a distributed manner to minimize a given loss function denoted as L: E.R + R. This is expressed as
0 := 10%
min [()) = E1x.))-I(((6(((6 EEE
(5)
Popular choices of L(-) for time series forecasting problems are the lz and & losses (Lai et al., 2018). In the subsequent section, will utilize the specific structure of these loss functions to design communicatonn-cffcteett distributed time scries forccasting algorithm. We also point out that problem (5) is in general non-convex since the loss function C(.) is non-convex w.r.l. 0 E θ for many problems of practical interest, e.g., when the local and global models are neural networks (Jain et al., 2017).
Remark 1 (Model Generality). We point out that although formulation (5) is designed for time series forecasting problems, is general enough to model standard distributed learning problems where each node has access to only partial feature originating from potentially disparate modality, i.c., when the label feature pairs do not originate from a time series. Specifically, our formulation encompasses a majority of distributed learning models, including vertical federated learning frameworks (Wei et al., 2022). However, note that the presence of both the local and the global models, combined with the fact that the data streams can be of potentially disparate modalities, makes our problem significantly more challenging than the ones considered in the past.
Next, we develop the algorithms to solve problem (5). We first describe a vanilla approach that relies on label sharing. Label sharing may be acceptable for some tasks (e.g., classification or regression), particularly when labels are low-dimensional and describe public or non-personal attributes (e.g., weather conditions or device operational states). However, for the time-series prediction problems in Section 2, the labels and features often share common support (sce (4)); morcover, in sensitive domains (c.g., medical or personal behavioral data), labels may encode privacy-critical information. Thus, label sharing may be undesirable. To miligate this issue, we exploit the separability of commonly used time-series loss functions: by Assumption 1, the loss is separable across dimensions, enabling a fully distributed implementation without any label sharing.
3.2 Prototype algorithm with label sharing
Our objective is to devise a combined optimization and communication strategy to minimize the global loss in equation (5). Specifically, each node strives to Iearn local model 0x, simultancously acquiring knowledge of global model, lo. which is shared across all nodes or centrally managed by the server. To commence, let's first calculate the stochastic gradients with respect to both the local and global models. Initially, we sample data point denoted as (X) ~ II, and compute the stochastic gradients (SG) as:
Local SG: Vo, L(B: ()) = Vo, [(oo, 01, 8e: (x)
V V f((a
Jo (6o.
(( fx(x)))))...,,()): 0o)),,
(6)
which follows from the application of the chain rule and the definition of the loss function in (5). Moreover, note that the local models implicitly depend on the local data partitions (see discussion after (5)). Similar to Local SG, we compute the Global SG as
Global SG: Va, (x, V C(0o, 0x. (x:
=
4

---

Published in Transactions on Machine Learning Research (11/2025)
Algorithm 1 DIVIDE with label sharing
Input: Rounds r= {0, :....; R 1}, local learning rates: k server learning rate: ni Initialize: Parameters, 108: 09. .... b
for r to R 12-11 do
4: Sample (%) ~ II (see (4)) Ak E [K]
5: Share (0) and yk. vk with server
6: Compute V - (x", ,"") at server via (7)
7: Compute V Fu f1... fk). (.). wk at server
8: Update: 0 0,+1 =6
Vo, oo00" (x", (Y))
Receive at each node V fo fi.,, fk), V Tof(ff(fi... (D):")) from server
10: Compute Vo, L(8" (x',,(x,,,)) y")) at each node using (6)
11: Update: 0%- =0 (6-mi mi V8 L(00; (x",, (")) vk E [K]
12: end for
which again follows from the application of the chain rule and the definition of the loss function in (5). Next, to implement an efficient algorithm, one would sample (x, in each round of training and implement an SG-type algorithu using the SG estims of (6) and (7). In the following, we first present the vanilla version of DIVIDE, which relies on label sharing among nodes.
A prototype algorithm. Algorithm deseribes the steps of prototype version of the proposed algorithm. At the beginning of each training round, the local nodes share their low-dimensional embeddings as well as their locally observed labels with the server (see Step 5). The server then utilizes the shared information from each node to update the global model using the Global SG constructed using (7) in Step 8. To construct the Local SG in (6), each node receives the relevant partial gradient from the server in Step 9. The local models are then updated Step 11. This process continues until convergence. Note that at the end of the training process, each node has access to the model parameters of while the server will have access 0f. To make the final predictions, each node forwards their local predictions to the server, which then completes the global prediction task.
We highlight key distinctions between DIVIDE and latent "master-node" architectures (Gilmer et al., 2017) as well as other graph-based based methods (Zheng et al., 2020; Wu et al., 2020b). While these methods all employ global coordination and message exchange, Gilmer et al. (2017); Zheng et al. (2020); Wu et al. (2020b) operate in a centralizod manner, embodding the nodes within a fully centralized graph model where message passing occurs directly between all node pairs. In contrast, DIVIDE operates in distributed setting: local nodes communicate only with global model that aggregates and redistributes information without peer-to-peer exchange. As result, DIVIDE transmits only low-dimensional embeddings, reducing communicatin overhead, preserving privacy, and improving scalability. Additionally, we note that privacy of DIVIDE can be enhanced with differential privacy (DP) (Dwork et al., 2014), local DP (Duchi et al., 2013), or secure aggregation (Bonawitz et al., 2017) to formally bound information leakage. Additional discussion of related works is provided in Appendix A.
Remark (Limitations). There are two major limitations of Algorithm 1 As briefly discussed earlier, the first limitation that the local nodes are leaking data. As noted in the implementation of Algorithm 1, the local labels are shared between the local nodes and the server in order to update the global model parameters by. This may be reasonable for some distributed learning tasks (like classification), but not for time series forecasting. e The ionselations between local nodes are not fully utilized, since the algorithm treats each local node independently and equally. However, there are rich correlations between data streams. For example, as observed in (Nguyen et al., 2019), the mobile edge devices (nodes) placed in the vicinity of each other may lead to correlated workload characteristics, thereby leading to correlated data streams. is well known the literature that the key to accurate prodictions time series to utilize correlation among the data streams (Nguyen et al. 2019; Lai et al., 2018; Wu al., 2020b; Zheng et al., 2020)-
Next, propose alternate implementation of DIVIDE that addresses the above two limitations.
5

---

Published in Transactions on Machine Learning Research (11/2025)
Algorithm DIVIDE without label sharing
Input: Rounds r r= {0.1..... I...., R 1}, local learning rates: ( server learning rate: mh Initialize: Parameters, 108: 09. .... b
for r to R do
4: Sample (%): ~ II (see (4)) Ak E [K]
5: Share: Local embeddings (o) ) wk with all nodes via the server
6: Compute: (5o(6 fx.....f))::.. fx):
at each node and share with all nodes via the server fx):))) at each node using Assumption 1 Compute: (=,o' = Σ 1k=1 (f
8: Compute: Ju 1o, f fK), and
0)) at each node ((6), (7)).
9: Update: 8) 0 - Vo, (x"", x"))
10: Update: 0%- = 0% - v Vo,4. ( ; (x", ,")) Ak E [K]
11:
end for
3.3 Algorithm without label sharing
In this section, we slightly modify the prototype implementation of DIVIDE that allows us to avoid any label sharing. Specifically, to tackle the first limitation, the global model is moved to the local nodes. During the training process, both the local and the global model parameters are updated locally while the server helps orchestrate the communication between the nodes. Note that the global parameters, even though available locally, are still global since they are kept identical at each node throughout the training process. Morcover, we make the following assumption the loss function.
Assumption 1. [(0) is dccomposable across dimensions, i.c., = (8,
We note that the above assumption holds for many loss functions of interest, including the lz- and fi-losses that are almost exclusively used for time series prediction problems (Tai et al., 2018). Now, using the above assumption, we implement the Local SG and the Global SG without requiring label sharing. The detailed procedure for solving (5) while operating under Assumption is provided in Algorithm 2. It is noteworthy that, following the initial sharing of local embeddings among the nodes (Step 5), each local node computes its partial gradient VZ(:))) based on its locally observed data in Step 6. Subsequently, these local partial gradients are shared among nodes to construct the complete partial gradient denoted as V using Assumption in Step 7. Moving forward to Step 8, the remaining partial gradients are calculated using the shared local embeddings to construct both the Local SG and the Global SG as defined in (6) and (7), rospectively. Ultimately, the global and local models updated concurrently using the computed stochastic gradients in Steps 9 and 10, respectively.
It's important to highlight that the DIVIDE framework offers the freedom to select both the global and local models. The local models are responsible for capturing the specifics of the individual time scries data observed locally, while the global models are designed to uncover the correlations between the distinet time series observed across different nodes. Next, we discuss the communication required by DIVIDE.
Remark 3 (Communication). Algorithm 1 requires information sharing in two steps, namely Steps 5 and Note that in Step 5, the local embeddings and labels are shared with the server, while in Step 9, the local (and global) intermediate partial gradients are shared. As discussed earlier in Section 3, the local embeddings are low-dimensional mappings (with dimensions much smaller than de for each k E [K]) while the global embeddings have dimension d x This implies that the total network-wide communication required by DIVIDE in each round of communication is only 4d7. Similarly, in Algorithm 2, the information sharing is conducted in Steps 5 and 6. In Step 5, the local embeddings are shared (back and forth) among all the nodes, while in Step 6, the decomposed local gradients (across dimensions) are shared among the nodes, which are then utilized to construct the full local and global SG. In Step 5, a total of 2dr real values are shared (back and forth) among nodes. Similarly, in Step 6, total of 2KdT real values are shared, making the total communication of 2dr + 2Kdr.
Compared to standard foderated learning methods, such as FedAvg (MeMahan et al., 2017a). FedProx (Li et al., 2020), FedDyn (Acar et al., 2021), FedNova (Wang et al., 2020), which synchronize full model

---

Published in Transactions on Machine Learning Research (11/2025)
parameters of size P each round at O(KP) cost, DIVIDE only exchanges low-dimensional embeddings, achieving significantly lower cost since d & P. While compression-uused federated learning variants, like QFedAvg (Li et al., 2019b), SignSGD (Bernstein et al., 2018), TernGrad (Wen et al., 2017), or sparsified federated lcarning tochniques (Aji & Heafield, 2017) reduce commuiiatiin cost, they still fundamentally scale with P due to increase in gradient variance (Shao et al., 2023; Han al., 2020).
4 Convergence guarantees
In this section, we provide the convergence analysis of the proposed approach. First, we note that the training samples (x", y") for n E utilized for solving (5) are non i.i.d. (see (1) and (2)). This follows from the fact that each pair of consecutive training samples x and x for E N share some common time series observations, L.e. x == [x++] ,...,, x shares the first D - observations with x-11== : x
This implies that the stochastic gradients computed using these non-ii.d. samples in Algorithms 1 and 2 will be biased and may cause the algorithms to diverge. However, note that the samples (x", follow the Markov property, meaning that we have P[x ...., x*1 = - which can be directly observed from the definition of x' (1). Specifically, the Markov property follows since conditioning on x the only new sample that is observed is x while the rest of the samples have already been observed in x Motivated by this observation, make the following assumptions about the data-generating process.
We assume that the data-generating process ""","")ee" h=20 follows Markov chain trajectory with M states. The Markov chain is time-honogeneous, irreducible, and aperiodic. The Markov chain has a transition matrix 7' E MxM and stationary distribution 11".
Assumption 2.
To the best of our knowledge, convergence analyses that explicitly exploit the Markov property of timo-scrics data in distributed iorecasting settings remain scarce and largely open. We also point out. that Assumption is also a practical assumption for other classes of problems since for many cases it is easy obtain samples from Markov chain trajectories rather than obtaining i.i.d. samples (Sun et al., 2018; Doan et al., 2020). Also, note that Assumption 2 assumes that the samples are generated from a finite state space. This assumption can easily be relaxed when the data samples (x", "",")) y J2 are generated from a Markov series rather than a Markov chain (Sun et al., 2018). Next, we make some assumptions about the loss function L(: ()).
htapffacattoo/regresooo/eeeeee /reinforccmmet learning)
Assumption 3. The Local SG and the Global SG derived in (6) and (7) are bounded, i.c., we have |Vo, (x, y) < G and [(0; (x)I S G. We also assume that the loss function I--Lipshiiz smooth, i.e., VeL(O, (x, is L-iisschitz.
Assumption is standard assumption in first-order algorithm analyses and has also been made in carlier works (Sun et al., 2018). Next, we state the convergence performance of DIVIDE.
Theorem 4.1. Suppose Assumptions 2 and 3 hold, and that the learning rates satisfy:
Σ nh = toc, Enn'r (nT) < too Vk E 10 K}.
Then we have 1im E| VoV(O) 0, where [(0) is desined in (5). Moreover, we have
((T)
E
min [ Isrsi
F min m K
k=0
where 0" := ==( ..... (0..,,(2)))T)) FTT and (T) is
(T) max { In(//>(T>)>
where X(T) = maxx (T))) E [0, 1) and Ax(T) E C is the tth largest eigenvalue of T.
2
We note that the above result matches the guarantees of a centralized Markov Chain Gradient Descent for K = 1 (Sun et al., 2018). Moreover, distributed Markov chain gradient descent under different settings have
7

---

Published in Transactions on Machine Learning Research (11/2025)
been considered in the past (Sun & Li, 2019; Wai, 2020). Specifically, the setting in Sun & Li (2019); Wai (2020) assumes that each node has access to complete data sample, however, in the DIVIDE, framework, each node only observes partial time series. Also, in Sun & Li (2019); Wai (2020) each node learns the same model, and therefore, has the same local stop sizes across the network. In contrast, in DIVIDE framework each node can flexibly choose the local model based on its local modality and learning task. Consequenlly, each node can perform local learning utilizing different step sizes. In addition, the DIVIDE framework allows each node to maintain a global model to learn the correlations among different time series. These key distinctions make the analysis of DIVIDE significantly challenging compared to prior works. Theorem 4.1 implies the following.
Corollary 1. If we choose the learning rates r7" = O(1/r9) with g (1/2, then DIVIDE achieves
(T) Rー-9
min {Il [II Igrsn n
It important to note that the non-convexity of the loss function [()) (defined in (5)) suggests that we cannot expect a gradient-based algorithm converge to the globally optimal solution. Theorem 4.1 and Corollary 1 establish the convergence of DIVIDE to stationary point, in expectation.
5 Numerical experiments
In this section, evaluate the performance of the proposed algorithm real-world datasets with number of time-scries forecasting basclines implemented in both centralized and distributed manners.
5.1 Experiment setup
We describe our numerical experiment setup below and refer the readers to Appendix B for more information about implementation details, hyperparameter settings, training and test schemes, etc.
Datascts. Our evaluation is conducted on number of diverse real-world datascts: [D1] Power demand dataset (POWR) (EIA, 2022), which contains the hourly power demand of thirteen major electricity grid service regions in the United States for 2022; [D2] NYC subway traffic dataset (SUBW) (EDDEN, 2021), which collects the number of people entering and exiting cach subway station per hour in NYC, 2017 2021; [D3] Meteorological dataset (WEAT) (BENIAGUEV, 2017), which reports weather data from 36 regions with different indicators from Oct. 2012 to Nov. 2017. [D4] DeepSense6G (DS6G) (Charan et al., 2022), which is a multinodal dataset containing sensory (e.g., camera, Radar, LiDAR) and radio information. Same four scenarios in experiments. Those datasets all inherently
with the ITU 2022 challenge (ITU, 2022), handle distributed scenarios as they are collected from independent sources.
Baseline models. We adopt five widely used deep learning baselines: LSTM (Hochreiter & Schmidhuber, 1997), TCN (Bai et al., 2018), LSTNet (Lai et al., 2018), Informer (Zhou et al., 2021), and DLinear (Zeng et al., 2023). These baselines are implemented in both centralized and distributed fashions. When incorporated as local models within our distributed framework. We denote them as "DIVIDE (local model)". For example, DIVIDE (LSTM) adopts LSTM in the local nodes. Additionally, we also consider the Prophet (Taylor & Letham, 2018b) as the baseline, which is widely recognized statistics-based algorithm. For the multivariate time-series forecasting task, we optimize the 'multi-prophet" library (Keča, 2020), where independent prophet models will model each data stream.
Evaluation metrics. For the multivariate time-series forecasting task, we utilize root mean square error (RMSE), which gauges the disparity within Euclidean space. For the multimodal time-scrics prediction with the DeepSense 6G dataset, we use the distance- bassedce-based accuracy (DBA) score; see the appendix for a detailed illustration.
8

---

Published in Transactions on Machine Learning Research (11/2025)
Table 1: Comparison of DIVIDE with other baselines for mid/long-term forecasting, The experiments are conducted for both centralized (i.c., LSTM, TCN, LSTNet, INFORMER, and DLincar) and distributed setups (i.e., DIVIDE(LSTM), DIVIDE(TON), DIVIDELLSTNet), (IVODEEL) and DIVIDE(DLinear) These methods utilize a 36-hour window to predict horizons (24h, 48h, 72h, 168h). Performance is evaluated
via RMSE, where smaller values indicate superior performance.
Multi- Prophet
DIVIDE (LSTM)
DIVIDE
DIVIDE
DIVIDE
DIVIDE
Models
LSTM
TON
(TCN)
LSTNet
(LSTNet)
INFORMER
(IFORMERR
DLinear
(DLinear)
WEAT SUBW POWR
24h
0.0764
0.0313
0.0303
0.0349
0.0321
0.0375
0.0374
0.0245
0.0268
0.0244
0.0249
48h
0.0787
0.0403
0.0411
0.0446
0.0439
0.0449
0.0427
0.0289
0.0303
0.0287
0.0312
72h
0.0791
0.0475
0.0447
0.0498
0.0479
0.0501
0.0468
0.0315
0.0334
0.0329
0.0325
168h
0,0820
0.0574
0.0575
0.0583
0
0.060S
O. 0.0588
0,0410
0.0402
0.0125
0,0437
24h
0.1242
0.0816
0.0765
0.0875
0.0941
0.0853
0.0850
0.0609
0.0613
0.0600
0.0623
48h
0.1361
0.0823
0,0804
0.0886
0,1040
0.0850
O. 0.0856
0.0657
0.0G08
0.0669
0.0689
72h
0.1440
0.0844
0,0850
0. 0.0865
0.1203
0. 0.0868
0.0865
0.0712
0.0720
0.0710
0.0726
168h
0.1452
0.0912
0.0892
0.0882
0.1068
0.0925
0.0919
0.0746
0.0785
0.0780
0.0799
24h
0.0204
0.0102
0.0119
0.0093
0.0118
0.0111
0.0142
0.0086
0.0092
0.0092
0.0089
48h
0,0211
0.0107
0.0103
0.0106
0.0126
0.0123
0.0164
0.0088
0,0089
0.0142
0,0137
72h
0.0215
0.0133
0.0130
0.0112
0.0113
0.0126
0.0137
0.0119
0.0108
0.0154
0.0140
168h
0.0246
0.0125
0.0136
0.0139
0.01-46
0.0222
0.0264
0.0124
0.0135
0.0159
0.0153
24-
Reaa
Pred
6.7
Real
Pred
17-
0.6
Real
Pred
5.2
24
0.11
(a) POWR.
(b) SUBW.
(c) WEAT.
Figure 1: Visualized forecast results of future 72-hrs using 36-hrs historical window via DIVIDE(TON).
5.2 Distributed time-series forecasting
Following the above experiment sctups, we now demonstrate the validity of DIVIDE by comparing it with baseline algorithns. Unless hehwwieee specified, we configure eight local nodes, some of which might include multiple data streams. The results are reported in Table 1.
Comparison with multi-Propheet We see that DIVIDE significantly outperforms Multi-Prophet, likely because DIVIDE employs neural networks and leverages correlations among data streams, thereby euhancing its learning capabilities in contrast to the cattisicallbbase Prophet, which independently models each data stream.
Comparison with centralized models. Despite each local node lacking direct access to all data streams under the distributed implementation setting, the results indicate that DIVIDE delivers competitive performance. In certain cases, we even observe that DIVIDE surpasses the centralized baseline. We attribute this performance gain to DIVIDE's ability to efficiently leverage cross-node correlations through structured aggregation, which also partially supported by the observation (see Sec. 5.3 and Appendix C.2) that models incorporating global component consistently outperform those without, implying that aggregation mitigates overfitting to local patterns. Moreover, discussed in Sec 3 and 4, DIVIDE is particularly advantageous under heterogeneous data distributions, where embeddings from different modalities provide complementary information that centralized models may overfit to and fail to fully exploit. Conversely, the dataset's 's spalio-temporal nature amplifies the training complexity of centralized models. For example, while a weather station in New Jersey may aid in predicting weather patterns in New York, its usefulness for predicting weather in California may be limited. In fact, such characteristics have motivated some centralizod time-series prediction models to adopt hierarchical or localized aggregation strategies (Wang et al., 2022; Li et al., 2017).
9

---

Published in Transactions on Machine Learning Research (11/2025)
5.3 In-depth discussion on distributed multivariate time-series forecasting
To further examine DIVIDE, conduct comprehensive studies under various settings and highlight results. We provide more details and additional experiments in Appendix C, including hierarchical prediction tasks, a complete set of ablation studies, and scalability of the proposed algorithms.
Flexibilitty of the local models. As noted before, DIVIDE enables individual nodes to independently select their local models. Table 1 presents the performance of DIVIDE utilizing different types of neural networks as the local model. It is evident that the various local models attain differing levels of performance different datasets, possibly attributed to the distinctive characteristics of the local model and the datasets. It is also worth noting that DIVIDE also supports the mixed types of local models.
Impact of the global model. We evaluate the performance of DIVIDE with and without the MLP global model. As illustrated in Fig. 2, the results clearly demonstrate that incorporating global models enhances forecasting performance by leveraging shared local node information. The numerical results reveal that adopting a global model can lead to about 4.4% performance improvement on average. Meanwhile, we also find that the inclusion of a global model enhances the scalability of DIVIDE, as its performance remains stable regardless of the number of nodes. Conversely, not having global model results in significantly poorer outcomes the number of nodes increases; see Appendix C.2 for details.
Hierarchical tasks. In addition to the forecasting task on each local node, the design DIVIDE also supports global tasks. In the Appendix C.2, we introduce two global tasks and refer to them as Sum that aims to capture an algebraic correlation (i.e., sum) among all local nodes data stream, and Spatial that aims to predict one of the local node information based on the others. From Table and Table 6, we see that achieving good performance is challenging for both Prophet and DIVIDE without the global model. It indicates the adaptability of DIVIDE to various tasks.
0.50
Reall
Pred
075-
Real
Pred
570
6.0-
a.n-
8:00
0.9
(a) Without global model.
(b) With global model.
Figure 2: 24-hrs forecast of DIVIDE(LSTNeE) on POWR datasel. with and without global model.
Asynchronous model updates. The local and global models in DIVIDE can be updated asynchronously; see the results attached in Appendix C.2, where we employ different learning rates. The results illustrate that our framework can effectively operale on various configurations and exhibits robust performance, corroborating the earlier findings.
5.4 Multimodal time-series prediction
We now utilize DIVIDE for multimodal time-scrics prediction. In this context, the dataset includes not only numerical time series but also diverse data modalities, such as RGB videos, point cloud streaming, etc. We conduct numerical experiments on the DeepSensetG dataset from the ITU 2022 challenges (Charan et al., 2022; ITU, 2022), aiming to predict the strongest signal power radio beam ID based on the environment sensoring information. To accommodate the different data modalities, we employ distinct local models, and the specific model architectures are outlined Appendix B.3. We highlight results in Table 2, with more details and discussions in Appendix C.3.
From Table 2, we observe that the global model is able to effectively fuse the information collected from the local model and improve performance by increasing the DBA score up to 15.9% by the single modality, What's more, DIVIDE surprisingly ranks in the Top 5, compared with other solutions on the leaderboard, which reports a lotal of 150+ submissions (ITU, 2022). It shows DIVIDE 's powerful learning capability in fusing the different modalities. It is worth noting that, since we have not targeted achieving the best result for this specific challenge, but rather to validate the effectiveness of the proposed algorithm in fusing different. time-series data modalities, we have not applied comprehensive data preprocessing as other submissions have done for the challenge.
10

---

Published in Transactions on Machine Learning Research (11/2025)
Table 2: The performance achieved by DIVIDE with different imput data modalities. Here, Sce. refers to the different scenarios used the ITU 2022 challenge (ITU, 2022). We use the DBA score as the metric, where larger values indicate better performance. DIVIDE ranks in the top 5 among 150+ submissions based on the overall score.
Sce. 31
Sce. 32
Sce. 33
Sce. 34
Overall
GPS-only
0.0256
0.6518
0.6332
0.6264
0.4843
Vision-only
0.1142
0.6163
0.6830
0.6619
0.5189
Loly
0.0653
0.6385
0.6120
0.5967
0.4781
DIVIDE
0.0715
0.7450
0.6895
0.7385
0.5611
6 Conclusion
In this work, we developed distributed time series forecasting framework that decomposes the entire time series prediction model into multiple local models, which are then composed construct the global model. This decomposition gives each local and global node the flexibility of the model construction so that they choose from a variety of existing time series prediction models depending on the local compute resources and the modality of local data. Also, the proposed framework is communication--ffiieeee and mitigates privacy risks by avoiding raw data sharing. We evaluated the proposed framework on number of lime-series forecasting problems and showed that the framework performs well on a majority of multivariate time-series forccasting tasks.
Acknowledgements
This research is partially funded by Meta Platforms, Inc. In addition, Hong acknowledges support from the USDA National Institute of Food and Agriculture (NIFA) and the National Science Foundation (NSF) National AI Research Institutes Competitive Award (No. 2023-67021-39829) as well as NSF Grants EPCN-231 1007 and CINS-2003000 Zhang's contribution is supported in part by NSF Grants CNS-2212318, LNS-2220286, and CNS-2220292. Ding also got supported by the Army Research Office through an Early Carcer Program Award (No. W911NF-23-10315).
References
An introduction to dynamic time warping.
http:://rtavenrrggtttttoo/// html## http:///w..2222o..o/.oo/oo///ao. 20alignments, ,, 2021. Accessod: 0023-10-22.
Multi modal beam prediction challenge 2022: Towards generalization. https://ww.deeppeesee. net/
challenge2022/, 2022. Accessed: 2023-10-14.
Durmus Alp Emre Acar, Yue Zhao, Ramon Matas Navarro, Matthew Mattina, Paul Whatmough, and Venkatesh Saligrama. Federated learning based on dynamic regularization. arXiv preprint arX.v:2111.0426. 2021.
Alham Fikri Aji and Kenneth Heafield. Sparse communication for distributed gradient descent. ar preprint wrXii:1704.0500 2017.
Ons Aouedi, Alessio Sacco, Kandaraj Piamrat, and Guido Marchetto. Handling privacy-sensitive medical data with lederated learning: Challenges and future directions. JEFE Journal of Biomedical and Health Informatics, pp. 1-14, 2022. doi: 10.1109/JBHT. 2022..0022.1186677
Luigi Atzori, Antonio lera, and Giacomo Morabito. The internet of things: A survey, Computer networks, 54 (15):2777-2805, 2010.
Shaojic Bai, J Zico Kolter, and Vladlen Koltun. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv preprint 1xXi::1803.013 2018.
Bilberto Batres-Fstrada. Deep learning for multivariate financial time series, 2015.
11

---

Published in Transactions on Machine Learning Research (11/2025)
Iz Beltagy, Matthew E Peters, and Arman Cohan. Longformer: The long-doeument transformer. arXiv preprinl ar urXiv:20040051500 2020.
DAVID BENIAGUEV. Historical hourly weather data 2012-2017. http:://www..gg//// wtw.gglee.om//attaeeeeteee sotiffhhheem/hitoootrooooooooood
2017. Accessed: 2023-10-10.
Jeremy Bernstein, Yu-Xiang Wang, Kamyar Adizadenesheli, and Animashree Anandkumar. signsgd: Compressed optimisation for non-convex problems. In International conference machine learning, pp. 560-569. PMLR, 2018.
Keith Bonawitz, Vladimir Tvanov, Ben Kreuter, Antonio Marcedone, H Brendan McMahan, Sarvar Patel, Daniel Ramage, Aaron Segal, and Karn Seth. Practical secure aggregation for privacy-preeerving machine learning. In proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, pp. 1191, 2017.
George EP Box, Gwilym M Jenkins, Gregory C Reinsel, and Greta M Ljung, Time series analysis: forecasling and control. John Wiley & Sons, 2015.
Gouranga Charan, Umut Demirhan, João Morais, Arash Behboodi, Hamed Pezeshki, and Ahmed Alkhateeb.
Multi-modal beam prediction challenge 2022: Towards generalization. arXiv preprint ar arXim:220900751 2022.
Zhengping Che, Sanjay Purushotham, Kyunghyun Cho, David Sontag, and Yan Liu. Recurrent neural networks for multivariate time scries with missing values. Scientific reports, 8(1):6085, 2018.
Yong Cheng, Yang Liu, Tianjian Chen, and Qiang Yang. Federated learning for srivacy-preeervigg ai. Communications of the ACM, 63(12):33 36, 2020.
Rewon Child, Scott Gray, Alec Radford, and Tlya Sutskever. Generating long sequences with sparse transformers. arviv preprint 1kic:1904.10500 2019.
Thinh T Doan, Lam M Nguyen, Nhan H Pham, and Justin Romberg. Convergence rates of accelerated markov gradient descent with applications in reinforcement learning. arXiv preprint arXiv:2002.08:: 02873, 2020.
2013 IEEE 54th annual symposium foundations of computer science, pp. 229. IEEE, 2013. John C Duchi, Michael I Jordan, and Martin J Wainwright. Local privacy and statistical minimax rates. In
Cynthia Dwork, Aaron Roth, al. The algorithmic foundations of differential privacy. Foundations and trends8 in theoretical computer science, 9(3 4):2114407, 2014.
EDDEN.
Nye subway traffic.
https://wwa https///wwww..ggl..o/ www.w//www......aaaaaatee//////e aor.e///cccc/tttsst//dedenngg 2021. Accessed: 2023-10-10.
EIA. Energy information administration electricity demand dataset. https://www. http://www........... gov/ 2022. Ac-
http://www..ccppaaeeeee/e//////a cessed: 2023-10-10.
Roger Frigola. Bayesion time series learning with Gaussian processes. PhD thesis, University of Cambridge, 2015.
Antonio Galicia, José Torres, Francisco Martinez-Alvare, and A Troncoso. novel spark-based multi-step forecasting algorithm for big data time series. Information Sciences, 467:800 818, 2018.
Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl. Neural message passing for quantum chemistry. In International conference on machine learning, pp. 1263-1272. Pmlr, 2017.
Penghhao Han, Shiqiang Wang, and Kin K. Leung. Adaptive gradient sparsification for efficient foderated learning: An online learning approach. In 2020 IEEE 40th International Conference on Distributed Computing Systems (ICDCS), pp. 1110-1120, 2020. doi: 10.100//CDDC33777.220000222
12

---

Published in Transactions on Machine Learning Research (11/2025)
Andrew C Harvey and Neil Shephard. Structural time series models. 1993.
Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memorr.. Neural Compul., 9(8):1335 17001737--080,, 1997. ISSN 0899-7667. doi: 8.1735.
URL
wwt///////...// 1997.9.
Prateck Jain, Purushottam Kar, et al. Non-convex optimization for machine learning. Foundations and Trendol in Muchine Learning, 10(3-4):142 -63, 2017.
Milan Keča. Multi prophet. https:// http//ggtttub... com/ coon//ommmmulii-poppheee 2020.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint orXiv:1412.000 69800 2014.
Guokun Lai, Wei-Cheng Chang, Yiming Yang, and Hanxiao Liu. Modeling long-and short-term temporal patterns with deep neural networks. In The 41st international ACM SIGIR conference on research & development in information retrieval, pp. 95 104, 2018.
Shiyang Li, Xiaoyong Jin, Yao Xuan, Xiyon Zhou, Wenhu Chen, Yu-Xiang Wang, and Xifeng Yan. Enhancing the locality and breaking the memory bottleneck of transformer on time series forecasting. Advances neural information processing systems, 32, 2019a.
Tian Li, Maziar Sanjabi, Ahmad Beirami, and Virginia Smith. Fair resource allocation in federated learning.
arXiv preprint arx: 1905. 10497, 2019b.
Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia. Smith. Federated optimization in heterogeneous networks. Proceedings of Machine learning and systems, 2:429-450, 2020.
Yaguang Li, Rose Yu, Cyrus Shahabi, and Yan Liu. Diffusion convolutional recurrent neural network:
Data-driven traffic forecasting. arXie preprint orXi:: 707.01920 2017.
Bryan Lim and Stefan Zohren. Time-series forecasting with deep learning: a survey. Philosophical Transactions of the Royal Society A, 379(2194):2020020) 2021.
Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. Commuicatiion-efiiient learning of deep networks from decentralized data. In Artificial intelligence and statisfics, pp. 1273 1282. PMLR, 2017a.
Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera Arcas. Communication-efficiie learning of deep networks from decentralized data. In Artificial intelligence and statistics, pp. 1273-1282. PMLR, 2017b.
Xiangrui Meng, Joseph Bradley, Burak Yavuz, Evan Sparks, Shivaram Venkataraman, Davies Liu, Jeremy Freeman, DB Tsai, Manish Amde, Sean Owen, al. Milib: Machine learning in apache spark. The Journal of Machine Learning Research, 17(1):1235-122) 1241, 2016.
Tomas Mikolov, Martin Karafiát, Lukas Burget, Jan Cernocký, and Sanjeev Khudanpur. Recurrent neural network based language model. In Interspeech, volume 2, pp. 1045 1048. Makuhari, 2010.
Chanh Nguyen, Cristian Klein, and Erik Elmroth. Multivariate based location-aware workload prediction for edge data centers. In 2019 19th JEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID), pp. 341-34110 350.. TEEE, 2019.
Maneesha Perera, Julian De Hoog, Kasun Bandara, and Saman Halgamuge. Multi-resolution, horizon distributed solar pv power forecasting with forecast combinations. Expert Systems with Applications, 205: 117690, 2022.
Larry I. Rockwood. Introduction to population ecology. John Wiley & Sons, 2015.
13

---

Published in Transactions on Machine Learning Research (11/2025)
Jeffrey D Scargle. Studies in astronomical time series analysis. ii-statistical aspects of spectral analysis of unevenly spaced data. Astrophysical Journal, Part 1, vol. 263, Dec. 15, 1982, p. 835-853., 263:835-855 1982.
Jiawei Shao, Zijian Li, Wenqiang Sun, Tailin Zhou, Yuchang Sun, Lumin Liu, Zehong Lin, Yuyi Mao, and Jun Zhang. A survey of what to share in federated learning: Perspectives model utility, privacy leakage, and communication efficiency, orXic preprinl or arXie:2307.106: 10655, 2023. Available al https: //axxiv.org/ab//2220 2006/
Xingjian Shi, Zhourong Chen, Hao Wang, Din-Yan Yeung, Wai-Kin Wong, and Wang-chun Woo. Convolutional lstm network: A machine learning approach for precipitation nowcasting. Advances in neural information processing systems, 28, 2015.
Shun-Yao Yao Shih, Fan-Keng Sun, and Hung-yi Lec. Temporal pattern attention for multivariate time scries forecasting. Machine Learning, 108(8):1421 1441, 2019.
Robert H Shumway, David S Stoffer, Robert H Shumway, and David S Stoffer. Arima models. Time Series Analysis and Its Applications: With R Ezamples, pp. 75-163, 2017.
Tao Sun and Dongsheng Li. Decentralized markov chain gradient descent. arXiv preprint 1rX9::10091008 2019.
Tao Sun, Yuejiao Sun, and Wotao Yin. On markov chain gradient descent. Advances in neural information processing systems, 31, 2018.
Sean J Taylor and Benjamin Letham. Forecasting at scale. The Americon Statistician, 72(1):37 2018a.
Sean J Taylor and Benjamin Letham. Forecasting at scale. The American Statistician, 72(1):37-45) 2018b.
Hiro Y Toda and Peter CB Phillips. Vector autoregression and causality: a theoretical overview and simulation study. Econometric reviews, 13(2):259-266 285, 1994.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
Hoi-To Wai. On the convergence of consensus algorithms with markovian noise and gradient bias. In 2020 59th IEEE Conference on Decision and Control (CDC), pp. 4897-4902. IEEE, 2020.
Jianyu Wang, Qinghua Liu, Hao Liang, Gauri Joshi, and H Vincent Poor. Tackling the objective inconsistency problem in heterogeneous federated optimization. Advances in neural information processing systems, 33: 7611-7623 2020.
Xiaoqian Wang, Yanfei Kang, Rob J Hyndman, and Feng Li. Distributed arima models for ultra-long time scrics. International Journal of Forecasting, 2022.
Kang Wei, Jun Li, Chuan Ma, Ming Ding, Sha Wei, Fan Wu, Guihai Chen, and Thilina Ranbaduge. Vertical federated learning: Challenges, mothodologics and experiments. arXiv preprint arXiv:2202. 04300, 2022.
Wei Wen, Cong Xu, Feng Yan, Chunpeng Wu, Yandan Wang, Yiran Chen, and Hai Li. Terngrad: Ternary gradients to reduce communication in distributed deep learning. Advunces in neural information processing systems, 30, 2017.
Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and S Yu Philip. A comprehensive survey on graph neural networks. IEFE transactions newral networks and learning systems, 22(1):4-24, 24, 2020a.
Zonghan Wu, Shirui Pan, Guodong Long, Jing Jiang, Xiaojun Chang, and Chongqi Zhang. Connecting the dots: Multivariate time series forecasting with graph neural networks. In Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining, pp- 2020b.
14

---

Published in Transactions on Machine Learning Research (11/2025)
Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. Are transformers effective for time series forecasting? In Proceedings of the AAAT conference on artificial intelligence, volume 37, pp. 11121-11128, 2023.
Junbo Zhang, Yu Zheng, and Dekang Qi. Deep spatio-temporal residual networks for citywide crowd flows prediction. In Thirty-first AAAI conference artificial intelligence, 2017.
Chuanpan Zheng, Xiaoliang Fan, Cheng Wang, and Jianzhong Qi. Gman:: A graph multi-attention network for traffic prediction. In Proceedings of the AAAT conference on artificial intelligence, volume 34, pp- 1234 1241, 2020.
Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient transformer for long sequence timo-scrics forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp. 11106 11115, 2021.
A Related work
Centralized models. Time scries forecasting has been a problem of interest for decades. Popular statistical approaches for time series prediction include vector autoregression (VAR) (Box et al., 2015), vector rego-regressive moving average model (VARMA) (Toda & Phillips, 1994), and Gaussian processes (GP) (Frigola, 2015). Moreover, the Prophet model developed in Taylor & Letham (2018a) proposes a modular regression model for time series forecasting that relies on decomposing the time series into three main model components, namely, trend, seasonality, and holidays, which are then learned through various statistical learning techniques (Harvey & Shephard, 1993; Rockwood, 2015). These statistical approaches learn interpretable models, but sometimes impose strong assumptions (such as stationarity and lincar dependence among variables) on the multiple time series models. To circumvent these issues, recently, deep learning-basse models have gained popularity for time series forecasting (Lim & Zohren, 2021). There are many neural network models one can utilize for time series forecasting, however, in practice, three types of neural network architectures, namely Recurrent Neural Networks (RNNs) (Mikolov et al., 2010), Graph neural networks (GNNs) (Wu et al., 2020a), and more recently, transformers (Vaswani et al., 2017) have gained popularity for solving such problems. The works (Lai et al., 2018) and (Shih et al., 2019) propose to utilize RNNS in conjunction with CNNs for time series prediction. The idea behind utilizing such a structure is that the CNNs will capture the spatial dependencies among variables while the RNNs will capture long-term temporal dependencies among the variables. More recently, GNN (Zheng al., 2020; Wu et al., 2020b) and Transformers- based architectures (Beltagy et al., 2020; Li et al., 2019a; Child et al., 2019; Zhou et al., 2021) have gained popularity for time series forecasting problems. In spite of a number of works on improving the time series forecasting capabilities of different models, there has been a lack of research on distributed implementations of time series forccasting algorithms.
Distributed time series forecasting. In the literature, distributed time series forecasting has received relatively less attention as compared its centralized counterpart, and this is partially because of several technical challenges. To conduct time series forecasting, practitioners usually have to adopt inadequate distributed platforms (Meng et al., 2016; Galicia et al., 2018) (please see discussion in (Wang et al., 2022, Section 1)). In Nguyen et al. (2019), the authors proposed a distributed workload forecasting algorithm for mobile edge devices wherein the authors leveraged correlation among workloads of edge devices in a close physical distance and applied LSTM to perform the prediction. In Perera et al. (2022), the authors proposed distributed forecasting algorithms for solar photovoltaic (PV) power generation, where a certain particle swarm optimization- approach is used to combine five state-of-the-art forecast models for solar PV power generation prediction. However, a major drawback of the algorithms proposed in Nguyen et al. (2019); Perera et al. (2022) is that they assume that the availability of the entire set of data streams is available at a central location, which effectively defcats the purpose of distributed processing. Recently, Wang et al. (2022) proposed a distributed ARIMA architecture for time series forecasting where the authors utilized a map-reduce architecture to perform predictions. In contrast to our model where each node observes a time series from a different source, in (Wang et al., 2022), the authors adopt a model where each node observes the same time scries but during separate time windows which is an impractical setting compared to the general setting considered in our work. In addition, instead of fixing local model, our framework endows each
15

---

Published in Transactions on Machine Learning Research (11/2025)
node with the flexibility of choosing local model best suited for its data modality and available computing resources, while Wang et al. (2022) constraints the individual nodes to utilize ARIMA models (Shumway et al., 2017).
B Implementation details
B.1 Datasets and tasks
Multivariate time-series forecasting, We evaluate the performance of DIVIDE on the three real-world datascts: the Power Demand dataset (POWR) (EIA, 2022). the NYC Subway Traffic dataset (SUBW) (EDDEN, 2021), and the Meteorological dataset (WEAT) (BENIAGUEV, 2017). In the following table, we provide a brief overview of the statistical information of these datasets:
Table 3: Stalistical information of the multivariate Lime-series forecasting datasets.
Name
# #imestamps
Cramularity
#Streams
Physical meanings
POWR
8737
1 hr
13
Each stream contains the electricity demand of the 13 major
US regions.
SUBW
9,911
hrs
469
Each stream contains the number of passengers eutering one subway
station New York City.
WEAT
45,252 1 hr
36
Each stream contains the temperature information of selected
cities; 30 are in North America, and 6 are in middle Asia.
Multimodal prediction. We also use the DeepSense6G (DS6G) (Charan et al., 2022) to evaluate the performance of DIVIDE for the multimodal prediction task, where we are supposed to utilize the environment information captured by sensors (e.g., camera, Radar, LiDAR) to predict the radio beam with the strongest signal strength. Specifically, of DeepSense6G to have a fair competition with the method the leaderboard. Each scenario is collected at different times and locations.
follow the ITU 2022 AT challenge (TTU, 2022) and focus scenario 31-34
B.2 Environment setups
Softwares assets. Our codes mainly rely on the following third-party libraries and all of them can be easily installed: Numpy', scikit-learn", Pytorch?, PyG*, multi-prophet', DTAIDDisancce Pandas", Matplothib".
Hardwares. Our workstation runs on the Unbuntu 18 system and is equipped with AMD Ryzen Threadripper PRO 3995WX CPU, 1TB memory, and three Nvidia RTX A6000 GPUs (48GB memory each).
Hyperparameters Settings. For all the experiments of neural network models, split the datasets into the training, validation, and test sets with the ratio of 0.5 0.2 : 0.3. The min-max scaler is applied for the data normalization. All models are trained using the Adam (Kingma & Ba, 2014) optimizer with a batch size of 128. If there is no additional description, we follow the baseline model's default settings but set 64 hidden cells for all positions. Lastly, we ensure that the training is long enough so that the models converge well; we repeat each of the experiments three times, and report the average results.
B.3 DIVIDE Architecture
Fig 3 highlights the design of DIVIDE, aligning with Algorithms 1 and 2 introduced in the main paper. Note that the proposed DIVIDE is hierarchical learning framework, comprising multiple local models and global
org/
http:////llll-llerooorttttaaaaa org/stable/
9httpe://pytorch.org
'nttps://xw..yyooorr//
"nttpe://pandas.yyee
hhttp:://natplorlib.orgg
16

---

Published in Transactions on Machine Learning Research (11/2025)
model. It benefits from simple collaboration, flexible model selection, and offers efficient training algorithms backed by theoretical guarantees to achieve superior local and global forecasting accuracy.
Model B.
Client
Model fb,
Client
Server
Wiw v MW WwV
Server
WW
Local Dateget
Modul fis
Client
Share local embeddings and grodieenes
Model
Client
Compute gradilenes
Local Dataset
(a) with label sharing.
(b) without label sharing.
Figure 3: Overview of DIVIDE design. Refer to Algorithms 1 and 2 for step-by-step illustrations.
Additionally, we modify the DIVIDE to accommodate the multimodal prediction task, as shown in Fig. 4. In this task, each node adopts different type of local model based on the modalities of data streaming, reflecting the flexibility of local model selections. Specifically, we employ CNNs for video streams, Point.et for point-cloud data modeling, and RNNs/LSTMs for processing time-series signals. It also worth noting that our theoretical guarantees still hold.
Camera
Gradients
videos
Server
PointNet fo
LiDAR
Gradients
update
Point cloud streaming
Gradients P
RNI f3
mmm M
GPS
Local Dataset
Figure 4: Modified DIVIDE for multimodal prediction task.
B.4 Training and test strategies
Neural network training and evaluation strategy. line with common time-series forecasting techniques, we employ a sliding window approach for training and assessing the ncural networks, illustratod in Fig. 5b. The model is initially trained on the segmented training dataset and subsequently evaluated. For the divided training and test datasets, data points within the historical window used to make predictions for the future window. Additionally, we configure the lcarning epoch to 300, which is relatively gencrous, considering the model often converges well before the initial epochs. Throughout the training phase, we continuously save the best-performing model on the validation set to ensure convergence for all models.
Prophet training and ovaluation strategics. Prophct was originally designed for the univariate timo-scrics forecasting task. We utilize the multi-prophet library" extend the Prophet for the multivariate time-series, where each time-series data stream will be used to fit Prophet independently. We optimize the library via multi-processing programming for speeding up. Note that Prophet instead of multi-Propher for simplicity in the paper.
hhttps://github. http:/hhhb..ccm//mmmmmmiiiiii com/vonum/multii tirppppeet
17

---

Published in Transactions on Machine Learning Research (11/2025)
We evaluate the prophet model in three schemes, shown in Fig. 5. We denote them as Entine, Everylter, and Incremental for simplicity. Specifically, the Entire scheme fits the Prophet on the whole training dataset and evaluates the test dataset in one shot. 'he Everylter only fits the model on the historical data within the sliding window (light blue) and prodicts the future (light green). Unlike the neural network, Prophet does not utilize the split training dataset (dark blue). other words, each sliding window entails the use of a new Prophet model. In contrast, the Incremental scheme will continuously update the training by adding past data samples to it (dark and light blue) as the time-domain sliding window moves forward and refit the Prophet model for prediction, instead of fitting fixed training set once and predicting the entire future. In this way, the prophet actually has an advantage over the neural network-based model, which is only trained on the fixed training set. However, it should be noted that both Everylter and Incremental result in increased computation lime compared to neural net norksr,, since they require refitting the model over and over again.
Model
Model
Train
Test
Train
(a)
Entire.
Model
Model
Train
Fubu
-oodel
Train
Test
3ld windo
t+2
Train
Hiz
Everylter.
(c)
Incremental.
Figure Training and Evaluation Strategies.
B.5
Evaluation Metrics
Root Mean Square Error (RMSE) a measure of the average magnitude of errors between predicted and obscrved values:
RMSE = Ei
2,)²
where ground truth observations, and T; are predicted values of the series. The lower values indicate better predictive performance,
Dynamic Time Warping (DTW) is a technique for measuring the similarity betweon two sequences while accounting for variations in the aligument and pacing of their data points:
DTW, (7,((,,'')
min TEA(a,))
Σ d(s,, 25)) (L.J)TT
where it is an alignment path, A(x, x') is the set of all admissible paths, and d(x;, x²)) is a specified distance metric (e.g. Euclidean); see met (2021) for more details. The lower values indicate better predictive performance.
Distance-Based Accuracy Score (DBA Score) is metric for assessing the accuracy of machine learning model's predictions by averaging the minimum differences between predicted labels and ground truth labels: DBA-Score Y/4 (Yi + Y/2 + Ys) where YK is defined as:
ynll
Yk
Σ min min
ISkSK
with Yo and F are the ground truth label and the kth predicted label respectively. The kth predicted label the kth most likely label predicted by the machine learning model. A is normalization factor, in our experiments, we used A = 5. The higher values indicate better predictive performance.
10https://facebbak github. http://ggiggioooopppaaaaa/////// httppppppae//cccc/aaaaaaaaa.a... html
18

---

Published in Transactions on Machine Learning Research (11/2025)
C Additional numerical results
C.1 Prophet
Table 4 shows the Prophet's performance when adopting different schemes. We can observe that the Incremental scheme achieves the best performance, aligning with the intuitive understanding that it visible to most data points. This is the scheme we selected and reported in the main paper.
Table 4: Performance of Prophet with different schemes. The lower value indicates better performance.
POWR
SUBW
WEAT
Prophet
24h
48h
72h
168h
24h
48h
72h
168h
24h
48h
72h
168h
RMSE
0.0858
0.0918
0.0949
0.1020
0.1517
0.1542
0.1734
0.1856
0.0263
0.0331
0.0398
0.0399
Entire
DTW
0.3621
0.4956
0.5564
0.8633
0.5382
0.7407
0.9251
0.0861
0.1398
0.1458
0.2545
RMSE
0.0765
0.0831
0.0879
0,0891
0.1322
0.1398
0.1523
0.1479
0.0249
0.0289
0.0354
0.0358
Everylter
DTW
0.3339
0.4487
0.5060
0,7587
0.4982
0.7043
0.8083
1.2661
0,0803
0.1205
0.1318
0.2285
RMSE
0.0764
0.0787
0.0791
0.0820
0,1242
0.1361
0.1440
0.1452
0.0204
0.0211
0.0215
0.0246
Iner.
DTW
0.3117
0.4315
0.4850
0.7017
0.4717
0.6832
0.7575
1.1922
0.0801
0.1179
0.1314
0.2210
C.2 Multivariate time-series forecasting
In this subsection, we provide additional numerical results on multivariate time-series forecasting, covering hierarchical tasks, mixed local models, the global model's impact, and asynchronous updates.
We now further introduce two global tasks referred to as Sum and Spatial for simplicity. In the Sum task, the global prediction aims to capture data streams that possess an algebraic correlation with all local nodes. For instance, it may involve predicting country's power demand as the sum of demands from each region. To address this, we modify the baseline methods and present the results in Table 5. In the case of Prophet and LDVIDE_NoC, where global models are absent and individual data streams are considered separately, we calculate the global prediction based on the aggregation of all local results. In the centralized model, an additional output dimension is introduced. For DIVIDE MLP, we employ the global prediction model.
Hicrarchical tasks:
Table The study of Sum task. We examine the DIVIDE without global model (DIVIDE NoG) and with MLP the global model (DIVIDE ___P_, well as Prophet and centralized LSTM model. Each model is supposed to utilize historical 36-hrs data to predict future 744,48,72,1683 168) -hrs.
Global Task
Model
Prophet
DIVIDE(LSTM) NoG
LSTM
DIVIDE(LSTM) MLP
Metrics
RMSE
DTW
RMSE
DTW
RMSE
DTW
RMSE
DTW
POWR
24h
0.094
0.109
0.071
0.093
0.067
0.090
0.060
0.095
Sum
48h
0.131
0.252
0.079
0.196
0.076
0.171
0.070
0.152
72h
0.167
0.426
0.120
0.236
0.108
0.192
0.106
0.178
168h
0.191
0.653
0.142
0.402
0.147
0.381
0.132
0.354
Methodology Local node predictions
Sum
Sum
One output dim in LSTM
Global model output
Sharing Info
Local node predictions
Raw data
Local node hidden states
Mcanwhile, we also consider the Spatial task, which is supposed to predict one of the local node information based on the others. For example, one region may lose monitoring, and we want to use the others to predict the missing value. We report the results Table In this task, both the Prophet and DIVIDE NoG have no way to do it due to the lack of global model. From those results, the hierarchical design of DIVIDE demonstrates its adaptability to various tasks and exhibits superior performance in global tasks.
Scalability: We vary the number of nodes in the system and study the impact of the global model on the scalability. Fig. 6 reports the results of applying (IVEIDELLSMM) on the POWR dataset with and wouh the global model. We can observe that (1) the DIVIDE with MLP consistently performs better than the no global model's version; (2) having MLP as the global model makes the performance insensitive to the number of nodes, while not having it leads to significantly worse results as the number of nodes increases. These observations consistent when applying the other local model on different datasets.
19

---

Published in Transactions on Machine Learning Research (11/2025)
Table 6: The study of Spatial task.
Global
Model
Prophet
DIVIDE NoG
LSTM
DIVIDE(LSTM) MLP
Task
Metries
RMSE
DTW
RMSE
DTW
RMSE
DTW
RMSE
DTW
Spatial
POWR
24h
X
X
X
X
0.107
0.180
0.096
0.161
48h
X
X
X
X
0.186
0.208
0.172
0.206
72h
X
X
X
X
0.207
0.497
0.203
0.442
168h
X
X
X
0.274
0.669
0.297
0.583
Methodology
way do
No way do
One oulpul dim LSTM
Global model outpul.
Sharing info
X
X
Raw data
Loral node hilden states
DIVIDE(LSTM) NoG DIVIDE(LSTM)
0.044
0.062
0.040
RMSF
0.030
0.006
0.03
0.632
0.50
Number nodes
Figure 6: Impact of the global model on the framework scalability. The experiments are conducted the POWR dataset with (4,6,1),32,64) nodes rumning the LSTM on the local nodes. The model is supposed to utilize historical 36-hour window to predict the future 24 hours.
Asynchronous model updates: We conduct experiments with different learning rates to the local and global updates to study the impact of asynchronous model updates. The results are presented Table 7. The results demonstrate that (1) our framework can flexibly operate on various configurations of global and local learning rates, and (2) it performs robustly across different learning rate choices, verifying the statement presented in the main paper above.
Table 7: The performance of DIVIDE(LSTM)) the POWR dataset with different global and local learning rates using RMSE. The model is supposed to utilize historical 36hrs window to predict future 24hrs. The to in global,,, represents the learning rate of the global model while in local denotes the local learning rate of the client nodes. The bold configuration is the one we used in the rest of the paper by default.
Learning Rate
Localo.001
Local 0.003
Localo.01
Localo.03
Local
Globalg.001
0.0330
0.0339
0.0298
0.0318
0.0345
Globalo.om
0.0288
0.0275
0.0293
0.0342
0.0338
Globalo
0.0295
0.0294
0.0303
0.0307
0.0291
Globalo.03
0.0315
0.0298
0.0309
0.0306
0.0326
Globaly.1 0.0301
0.0290
0.0324
0.0312
0.0308
C.3 Multimodal prediction
We follow the ITU Challenge setups (ITU, 2022) for the multimodal prediction task. In this setup, the training dataset consists of 80% the data from scenarios 32-34, and evaluation is conducted on the test datasot, which includes scenario and previously unseen data from scenarios 32~34. The DBA score is employed as the performance metric, and the overall DBA-Score calculated by averaging the results across these four scenarios, serving the metrie for the final ranking. Table 8 reports the numerical results achieved
20

---

Published in Transactions on Machine Learning Research (11/2025)
by the DIVIDE. It is worth noting that DIVIDE ranks in the top 5 among the reported 150+ submissions"! proving evidence of its capability to efficiently fuse and utilize different data modalities.
Meanwhile, we observe that DIVIDE performs worst in scenario 31 and speculate that this could be attributed to lack of data pre-processing. For example, the brightness has a significant impact on the video, and thus, the superior solutions adopt comprehensive data augmentation methods datasets across all the scenarios, but it is beyond this paper's scope as we emphasize the ability of modality fusion and flexibility of local model selection instead of simply beating the best solution in the competition via data pre- pre-processing. It's also worth noting that our model hasn't undergone fine-tuning using the adaptation dataset provided by ITU, which is another significant factor contributing to the poor performance in sccnario 31.
adjust and normalize the raw
Table 8: DIVIDE performance on multimodal prediction tasks evaluated on scenarios 31-34 of the DS6G datasets. The DBA score evaluates the performance, with the higher number indicating better performance. The last row reports the ranks based on the public leaderboard and is determined based on when all modalities are utilized.
Scenario
Scenario
Scenario
Scenario 34
Overall
GPS-only Vision-only
0.0256
0.6518
0.6332
0.6264
0.4843
0.1142
0.6163
0.6830
0.6619
0.5189
R-nly
0.0653
0.6385
0.6120
0.5967
0.4781
DIVIDE(AlI)
0.0715
0.7450
0.6895
0.7385
0.5611
Rank
13
2
10
6
5
D Proof of Theorem 4.1
Here, restate Theorem 4.1 for convenience.
Theorem D.1. Suppose Assumptions 2 and 9 hold, and that the learning rates satisfy:
Σ nh = +oc, =oc,
En²r (n)))² < too Ak E 10, 10,1....,K}
Then we have R-mo VoL(()) = Moreover, we have
[[Vo((8)) =0 =0 (
(T(T) DR min min
E
min ISPER
where or = [(05)T ,.... (0%) and (T) is
((T) max (1, (nT(//(T))) } with X(T) max([)2 An (t))
(0,1),
where (T) E is the tth largest eigenvalue of T.
To prove the Theorem D.1 we will utilize some intermediato results. First, let us set up some notations. Recall, that our goal is to solve the following problem
min 0E
{c(O)= 100
(8)
Let consider the (Markov chain) SGD-based update rule to solve (8) in Algorithms 1 and Note the overall (network-wied wide) stochastic gradient computed at round {0, 1, ... R R-1} -11 is
Va,"(0'; (x" (T),")) Ve, [(0"; (T)
Vo, (x",, (x"""))
Uhttp:://ww.deegsemeee htt.///laaakk-miil modallll-----------ooo-------- Meaaall-eea---e
peadaa--taan-aascctioo// prediction/
21

---

Published in Transactions on Machine Learning Research (11/2025)
Moreover, the Markov chain gradient descent update rule at each node and the server is
0--1 8: ni V [(0"; (x", ,")) for = {0. 10. K}
(9)
Next, rewrite problem (8) in finite-sum version. Let us suppose that the distribution IT supported set of M points ç ...., E then problem (8) above can be stated equivalently finite-sum problem
min BEeCR
{C(0)
M Σ fm (0)
(10)
where Cm (.) is defined as
L. (0) = M M- M-PMM y) = < L(So(00; ((()),,,, (1)
Note here that each state m [M] has uniform probability 1/M. The corresponding version of the Markov chain gradient descent for problem (10) can be written
8. 05 Th Vo Lm (0) for i= {0, .0..1 K]
(11)
where {m") )-20 is the trajectory of Markov chain on 21,2,.... M) that has uniform stationary distribution. Note that the Markov chain trajectories {(x", y²" }r)0 and {m" J.20 are two different but related Markov chains. A crucial difference in the update rules of (11) and (9) from a centralized setting is that the local updates at each node here are heterogeneous with potentially different learning rates (Sun et al., 2018). Next, we make the following assumption on the underlying Markov chain generating the trajectory hr20-
We assume that the data-generating process (m") follows Markov chain trajectory with M states. The Markov chain is time- homogeooogen,uus, irreducible, and aperiodic. The Markov chain has transition matrix T E IR Mx M and stationary distribution x' == [mi,..., Tel with Em T =.
Assumption 4.
Moreover, for analysis purposes, we make the following assumptions.
Assumption 5. The Local SG and the Global SG in (11) are bounded, i.c., we have VV Cm s G and VV (0)]l < G for each m E [M]. We also assume that the loss function is L-Lipschitz smooth, i.e., VaC. (0) is L-Lipschitz.
We note that for appropriately chosen and G Assumption 5 above and Assumption the main paper are equivalent. We define
2p/ In X(T) )) Kr [rr]rr)
(1))
T min max
In
where Cr is constant that depends on the Jordan canonical form of T and Kr is a constant that depends on X(T) and \²(T). Please see (Sun et al., 2018, Lemma 1). This choice of T, implies that we have from (Sun et al., 2018, Lemma 1)
1
1/r
T lis
<
for any i 21,
M
KG
Specifically, T. characterizes the mixing time of the Markov chain. Next, we define by F" the sigma-algebba generated by the sequence of iterates 0"
F' = (0' 02 .... 0" m ml :..., ,'....,mm" m..,,m"-).
where m² generated according to Assumption 4.
22

---

Published in Transactions on Machine Learning Research (11/2025)
Proof. Using the Lipschitz smoothness of [(-), have
-(0+t)) = (( -0) 2 110 - 07
= (VL(0- ),,, 0, 0-) + (VL(0") - L(0 L(00-T)),0 -0") + L
2 110 8712
0) < (VL(0--T.), ),),0 - + V2(00) - C(0 -o 11)11= L +
|1 - 11
2
2
- (VC((r-T.),00 ) 8*) + 2 1|0 0--1-112 + L +
L
1 1
where the first inequality follows from the Schwarz inequality and the final inequality results from the Lipschitz-smoothness of (()). Rearranging the terms, we get
I 1- T2) -0)) - (())) 1(0 + 100" 0
L+1
100
1-
2
2
Next, we consider the l.h.s. of the above equation. Taking expectation w.r.t. the sigma-algebra J we get
No 0o f. (0r) 7 Vo, Lm (Or)
-1)-1-)))-1--1 E
V cC(or-Tr).
FTーT
: 7 Vox (0m)
Ti V 60 Lm (0r- Vi Vo, f.m (rr
=E
Tik V 0x Lam (0 (T)
Vo Va, Go fom (0") Mi Vo, Lm (0r)
Vo Va, 0o L.n (0r- Ti Ve, Lm- (0 (0 To)
+ E
V T.rr
VKK Vore fare (0r)
Wikk Vox L.m (00-Tr)
1i Vo Fm (6 (TT
2 E ( v T.
17/ V6, Pm 2mm((r TTr
1/K Vog Lyp (0r }
K. G T. Σ
110
1
0 To 1|
1=0
Therefore, we get
Vh 80 Lm- -Br
M Vo, Lm 60-((r-TT) (or-
) |
E
v.(
S <E( -(0 F
1K V BNN Lm (0--To)
L"
+ + 1
E. [[|00 - [ To
Ee -
+
2
+ K. G. L
Σ
B 10r
or
Tr
[T-
i=0
23

---

Published in Transactions on Machine Learning Research (11/2025)
Taking expectations on both sides, get
To Ve, Lm (0--T-) Tr) Vi Va, mm (o--T=)
))
E
T.),
S [[(((T) [(ort))]
L
E2 [1 - 1-
2
Th Vox fm (or-T.))
K
I
+ K. G
Σ
b 10 - -r
+
E
[1 0 - 112
2
1=0
Next, we consider the Lh..sterm
To V 1o L. (0r-Tr)
7i V V. f.m" (T)
Emn
UKK Vox fm (0--To)
%o V 8o fm (0--T
M VL((r-T.),
T7 Vo, Lm TT)
Plmm" = ml mm
m=1
7K Vox Lune (f) T-)
To Vo Cm (0--Tn) To)) Oi Vo, Lm (0r-T)) Tr)
M
Σ
T.)
PImm" ml| /mmm To)
m=1
AK Hx Lmr (T--Ta) To 8o mm (0r Tr) 7i Vo, Lm (0r Tr)
Σ
VC(0r-TT)) T.).
TT mm" -T
1/ V Bg Law (0 T.)
Ti Ve ou Ci (8--Tr) "i Vo, Lane Tr)
M
V T.), Σ
m Aar-Toie T..m
T''k Vox Fm (0r T.
min [|(((--TT)) max (n/ ] i=
2r
Therefore, we finally get
min (.0 B[C(0')
c((r+1)) 1]]
L 1
0" -1110+t1-0'1?
B[110r+1
2
K + K
or -T- + 1,2
1- + max (1% x
G L
Σ
E
10
[[00
2
2r
k=0
Summing over r and using (Sun et al. 2018, Equations (6.58) (6.58).(6.59) and (6.60)), we get the following
Σ m BII )1) O max { (m((A//(7))) })
(12)
Next, using the Lipschitz-smmootnness of C(+), we have
< VL(O)) VL( T.), VL(00) + VC((--T))
< [V - Ve(Or-TT) |VL(0') + VC(00-7))
< 2K G. L 100- TT
24

---

Published in Transactions on Machine Learning Research (11/2025)
Multiplying both sides by min J we get
min(n VC((0)] (n [V
S 2K- G L. min m.min(77)) 110" ---
-00 - -r 112 K G. L. (min(n))*0)) )²
K. G L min 1
Again summing r, taking expectation, and using the discussion in (Sun et al., 2018, Equations (6.58).(6.59) and (6.60)), get the
Σ Σ( (min EIV min [EI T.)1)) Si + In(1/X(T))
C2
(13)
for some constants Ci, C2 > O. Finally, summing (12) and (13), we get the
L EI < o max im(a/erT))) 1)
Σ min(nf
Therefore, we have the proof of the main statement the Theorem. Moreover, the proof of lim V Equation (4.4)).
straightforward extension of the proof from (Sun et al., 2018, Theorem 2,
Hence, the theorem is proved.
25


Reviews:
[{"review_id": "XNNXLYbF6y", "reviewer": "Reviewer_gCmy", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper introduces DIVIDE, a distributed hierarchical framework for multimodal time series forecasting. Each local node trains its own model on its partial time series (possibly heterogeneous in modality, such as numerical, visual, or sensor data), while a global model aggregates low-dimensional embeddings to capture cross-node correlations and generate both local and global forecasts. The framework supports flexible model selection per node, avoids raw data sharing to preserve privacy, and reduces communication cost by only transmitting embeddings and gradients. The authors provide a theoretical convergence analysis under non-i.i.d. Markovian data assumptions and demonstrate performance gains over centralized and distributed baselines on real-world datasets. A notable empirical finding is that DIVIDE sometimes even surpasses centralized training. The main contributions are: (1) a novel hierarchical decomposition for distributed forecasting; (2) communication- and privacy-efficient training algorithms; (3) convergence guarantees with Markov-dependent samples.", "claims_and_evidence_yn": "Yes", "claims_explanation": "Yes. The claims are generally supported with solid theoretical and empirical evidence. The convergence proof is clearly derived under Markovian assumptions, and experiments show DIVIDE outperforming centralized and distributed baselines. However, the explanation of why DIVIDE occasionally surpasses centralized models (p. 8) remains anecdotal and would benefit from deeper analysis or ablation.", "audience_yn": "Yes", "audience_explanation": "Yes. TMLR’s audience would be interested in this work, as it addresses an underexplored but practically important problem: distributed multimodal time series forecasting. The hierarchical structure, theoretical guarantees, and multimodal support make the paper relevant to both machine learning theory and applied communities (federated learning, IoT, and time series analysis).", "requested_changes": "1. Clarify empirical observations: On p. 8, provide more concrete explanations or controlled experiments on why DIVIDE outperforms centralized models. For example, analyze whether hierarchical modeling reduces overfitting or whether data heterogeneity benefits distributed aggregation.\n\n2. Broader multimodal results): In Section 5.4, the multimodal evaluation with DeepSense6G is promising, but more detail on local model choices per modality should be given for clarity.\n\n3. Communication overhead analysis:The communication cost discussion is somewhat abstract. Providing quantitative comparisons would strengthen the claim of communication efficiency.", "broader_impact_concerns": "No major ethical concerns are identified. The framework respects privacy by avoiding raw data sharing and is well aligned with responsible AI practices.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "dyjQpNK54x", "reviewer": "Reviewer_BQdj", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper introduces DIVIDE, a distributed, hierarchical learning framework for time-series forecasting, focusing on settings where nodes observe only subsets of time-series data. DIVIDE decomposes forecasting models into local models (with flexibility for architecture and data modality) and a global model that fuses local information. The authors provide communication-efficient training algorithms, formal convergence guarantees, and empirical evaluations on few real-world datasets. \n\nStrengths:\n* Clear and detailed formulation of the method\n* Theoretical contributions\n* Communication efficient approach\n\nWeaknesses:\n* Privacy was claimed, however it was not supported by empirical or theoretical analysis\n* Weak baselines, no federated learning methods compared against\n* Certain statements are ''overclaimed\"", "claims_and_evidence_yn": "Yes", "claims_explanation": "Main claims of approach capabilities (other than privacy preservation) appear well supported.\n\nPrivacy guarantees are not that strong, as sharing embeddings and gradients could still leak information.\n\nI find some claims a bit of a stretch: \"Label sharing might be acceptable for some problems (like classification or regression), since the labels, in many cases, are low-dimensional and might not lead to significant privacy leakage.\" Even though low dimension - labels like presence of cancer or not in medical applications are quite a significant privacy concern.\n\n\" To the best of our knowledge, this is the first time the Markov property of the time series data has been utilized to guarantee convergence of the forecasting algorithms.\" Even Kalman filter work, from more than half a century ago, relies on Markov property for convergence guarantees.", "audience_yn": "Yes", "audience_explanation": "Many aspects of this work will be interesting to various sub-audiences of TMLR - distributed learning; time-series; theory.", "requested_changes": "Consider adding some stronger baselines like FEDformer, SplitFed, SplitNN.\n\nSoften some of the claims on novelty (e.g. first convergence guarantee using Markov property).\n\nAdd discussion on the privacy guarantees.", "broader_impact_concerns": "No ethical concerns.", "additional_comments": "Where is the letter V in the name of the approach 'DIVIDE' coming from?", "extra_scores": {}, "extra_text": {}}, {"review_id": "WDPqOFrfc4", "reviewer": "Reviewer_mogC", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "### Summary Of Contributions\nThis paper is concerned with how the hierachical learning which incorporate global and local model influences the distributed time series forecasting task. The authors propose DIVIDE and theoretical analysis demonstrating the framework's effectiveness and justifying the guarantees of convergence.\n### Strength And Weaknesses\nStrength:\n1. The paper is well motivated.\n2. The authors show theoretical and empirical contribution of using DIVIDE framework, which harness both global and local models in distributed settings.\n3. The results of ITU 2022 challenge show good performance on heterogeneous dataset.\n\nWeaknesses:\n1. The authors propose to use global models to capture the correlations among different local nodes. It seems to me that the idea is similar using a virtual latent master node which are present in [1]. I would like to ask the authors if they can relate to the proposed algorithm with it.\n2. The proposed mechanism where local nodes shares its embeddings $f_k(\\theta_k^r) \\ \\forall k$ with all nodes appears structurally similar to a fully connected GNN. It would be valuable if the authors could clarify how their approach differs from or improves upon fully connected GNN architectures, particularly in terms of the information flow patterns and computational mechanism.\n\n[1] Neural Message Passing for Quantum Chemistry", "claims_and_evidence_yn": "No", "claims_explanation": "While the authors present numerical results across multiple benchmark datasets, The distributed DIVIDE framework rarely demonstrates superior performance compared to centralized models across several datasets.", "audience_yn": "Yes", "audience_explanation": "The paper addresses the practically important problem of distributed time series forecasting, which is highly relevant to TMLR audience working on federated learning, and time series analysis applications.", "requested_changes": "Please see the questions in the weaknesses section in the review.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)