Original: Scalable Spatiotemporal Graph Neural Networks
Revised: Scalable Spatiotemporal Graph Neural Networks

Original: Anonymous Author(s)

Original: AffiliationAddress email

Original: Abstract
Revised: Abstract

Original: Neural forecasting of spatiotemporal time series drives both research and industrialinnovation in several relevant application domains.
Revised: Neural forecasting of spatiotemporal time series drives both research and industrial innovation in several relevant application domains.

Original: Graph neural networks (GNNs)are often the core component of the forecasting architecture.
Revised: Graph neural networks (GNNs) are often the core component of the forecasting architecture.

Original: However, in mostspatiotemporal GNNs, the computational complexity scales up to a quadraticfactor with the length of the sequence times the number of links in the graph,hence hindering the application of these models to large graphs and long temporalsequences.
Revised: However, in most spatiotemporal GNNs, the computational complexity scales up to a quadratic factor with the length of the sequence times the number of links in the graph, hence hindering the application of these models to large graphs and long temporal sequences.

Original: While methods to improve scalability have been proposed in the contextof static graphs, few research efforts have been devoted to the spatiotemporalcase.
Revised: While methods to improve scalability have been proposed in the context of static graphs, few research efforts have been devoted to the spatiotemporal case.

Original: To fill this gap, we propose a scalable architecture that exploits an efficientencoding of both temporal and spatial dynamics.
Revised: To fill this gap, we propose a scalable architecture that exploits an efficient encoding of both temporal and spatial dynamics.

Original: In particular, we use a randomizedrecurrent neural network to embed the history of the input time series into highdimensional state representations encompassing multi-scale temporal dynamics.
Revised: In particular, we use a randomized recurrent neural network to embed the history of the input time series into highdimensional state representations encompassing multi-scale temporal dynamics.

Original: Such representations are then propagated along the spatial dimension using differentpowers of the graph adjacency matrix to generate node embeddings characterizedby a rich pool of spatiotemporal features.
Revised: Such representations are then propagated along the spatial dimension using different powers of the graph adjacency matrix to generate node embeddings characterized by a rich pool of spatiotemporal features.

Original: The resulting node embeddings can beefficiently pre-computed in an unsupervised manner, before being fed to a feedforward decoder that learns to map the multi-scale spatiotemporal representations topredictions.
Revised: The resulting node embeddings can be efficiently pre-computed in an unsupervised manner, before being fed to a feedforward decoder that learns to map the multi-scale spatiotemporal representations to predictions.

Original: The training procedure can then be parallelized node-wise by samplingthe node embeddings without breaking any dependency, thus enabling scalabilityto large networks.
Revised: The training procedure can then be parallelized node-wise by sampling the node embeddings without breaking any dependency, thus enabling scalability to large networks.

Original: Empirical results on relevant datasets show that our approachachieves results competitive with the state of the art, while dramatically reducingthe computational burden.
Revised: Empirical results on relevant datasets show that our approach achieves results competitive with the state of the art, while dramatically reducing the computational burden.

Original:  Introduction
Revised:  Introduction

Original: As graph neural networks (GNNs; [1, 2]) are gaining more traction in many application fields, the needfor architectures scalable to large graphs – such as those associated with large sensor networks – isbecoming a pressing issue.
Revised: As graph neural networks (GNNs; [1, 2]) are gaining more traction in many application fields, the need for architectures scalable to large graphs – such as those associated with large sensor networks – is becoming a pressing issue.

Original: While research to improve the scalability of models for static graph signalshas been very prolific [3–6], little attention has been paid to the additional challenges encounteredwhen dealing with discrete-time dynamical graphs, i.e., spatiotemporal time series.
Revised: While research to improve the scalability of models for static graph signals has been very prolific [3–6], little attention has been paid to the additional challenges encountered when dealing with discrete-time dynamical graphs, i.e., spatiotemporal time series.

Original: Several ofthe existing scalable training techniques rely on subsampling graphs to reduce the computationalrequirements of the training procedure, e.g., [3, 5].
Revised: Several of the existing scalable training techniques rely on subsampling graphs to reduce the computational requirements of the training procedure, e.g., [3, 5].

Original: However, sampling the node-level observationsas if they were i.i.d. can break relational (spatial) dependencies in static graphs and it is even moreproblematic in the dynamic case, as dependencies occur also across the temporal dimension.
Revised: However, sampling the node-level observations as if they were i.i.d. can break relational (spatial) dependencies in static graphs and it is even more problematic in the dynamic case, as dependencies occur also across the temporal dimension.

Original: Indeed,complex temporal and spatial dynamics that emerge from the interactions across the whole graphover a long time horizon, can be easily disrupted by perturbing such spatiotemporal structure withsubsampling.
Revised: Indeed, complex temporal and spatial dynamics that emerge from the interactions across the whole graph over a long time horizon, can be easily disrupted by perturbing such spatiotemporal structure with subsampling.

Original: As an alternative, precomputing aggregated features over the graph allows for factoringout spatial propagation from the training phase in certain architetures [6].
Revised: As an alternative, precomputing aggregated features over the graph allows for factoring out spatial propagation from the training phase in certain architetures [6].

Original: However, similarly tothe subsampling approach, extending this method to the spatiotemporal case is not trivial as thepreprocessing step must account also for the temporal dependencies besides the graph topology.
Revised: However, similarly to the subsampling approach, extending this method to the spatiotemporal case is not trivial as the preprocessing step must account also for the temporal dependencies besides the graph topology.

Original: ESN
Revised: ESN

Original: ESN ESN
Revised: ESN ESN

Original: Temporal embedding:
Revised: Temporal embedding:

Original: Spatial embedding:
Revised: Spatial embedding:

Original: M L P ... i .
Revised: M L P ... i .

Original: i .
Revised: i .

Original: d .
Revised: d .

Original: s a m p l e s
Revised: s a m p l e s

Original: In this paper, we propose a novel scalableencoder-decoder architecture for processing spatiotemporal data.
Revised: In this paper, we propose a novel scalable encoder-decoder architecture for processing spatiotemporal data.

Original: Fig.
Revised: Fig.

Original: 1, shows a high-leveloverview of the architecture.
Revised: 1, shows a high-level overview of the architecture.

Original: The spatiotemporal encoding scheme is training-free: first,it exploits a deep randomized recurrent neuralnetwork [7, 8] to encode the history of each sequence in a high-dimensional vector embedding;then, it uses powers of the graph adjacency matrix to build informative node representations ofthe spatiotemporal dynamics at different scales.
Revised: The spatiotemporal encoding scheme is training-free: first, it exploits a deep randomized recurrent neural network [7, 8] to encode the history of each sequence in a high-dimensional vector embedding; then, it uses powers of the graph adjacency matrix to build informative node representations of the spatiotemporal dynamics at different scales.

Original: According to the downstream task at hand, the50 decoder maps the node representations into thedesired output, e.g., the future values of the timeseries associated with each node.
Revised: According to the downstream task at hand, the decoder maps the node representations into the desired output, e.g., the future values of the time series associated with each node.

Original: To improveefficiency, we exploit the structure of the extracted embedding to design the decoder to actas a collection of filters localized at differentspatiotemporal scales.
Revised: To improve efficiency, we exploit the structure of the extracted embedding to design the decoder to act as a collection of filters localized at different spatiotemporal scales.

Original: Since the spatiotemporal encoder requires neither training nor supervision, the representation of eachnode and time step can be computed in a preprocessing stage, without the constraints that come fromonline training on GPUs with limited memory.
Revised: Since the spatiotemporal encoder requires neither training nor supervision, the representation of each node and time step can be computed in a preprocessing stage, without the constraints that come from online training on GPUs with limited memory.

Original: The decoder is the only component of the architecturewith trainable parameters.
Revised: The decoder is the only component of the architecture with trainable parameters.

Original: However, since spatiotemporal relationships are already embedded in therepresentations, the embeddings can be processed independently from their spatiotemporal contextwith two consequent advantages.
Revised: However, since spatiotemporal relationships are already embedded in the representations, the embeddings can be processed independently from their spatiotemporal context with two consequent advantages.

Original: First, training can be done node-wise, allowing for samplingnode representations in mini-batches of a size proportional to the hardware capacity.
Revised: First, training can be done node-wise, allowing for sampling node representations in mini-batches of a size proportional to the hardware capacity.

Original: Second, thedecoder can be implemented similarly to a standard multilayer perceptron (MLP) readout, whichis fast and easy to train.
Revised: Second, the decoder can be implemented similarly to a standard multilayer perceptron (MLP) readout, which is fast and easy to train.

Original: Let T and E be the number of steps and the number of edges in theinput graph, respectively.
Revised: Let T and E be the number of steps and the number of edges in the input graph, respectively.

Original: The cost of training a standard spatiotemporal GNN on a mini-batch ofdata has a computational and memory cost that scales as O ( TE ) , or O ( T 2 E ) in attention-basedarchitectures [9].
Revised: The cost of training a standard spatiotemporal GNN on a mini-batch of data has a computational and memory cost that scales as O ( TE ) , or O ( T 2 E ) in attention-based architectures [9].

Original: Conversely, in our approach mini-batches can be sampled disregarding the lengthof the sequence and size of the graph, thus making scalability in training constant, i.e., O (1) , w.r.t.the spatiotemporal dimension of the problem.
Revised: Conversely, in our approach mini-batches can be sampled disregarding the length of the sequence and size of the graph, thus making scalability in training constant, i.e., O (1) , w.r.t. the spatiotemporal dimension of the problem.

Original: Our contributions can be summarized as follows.
Revised: Our contributions can be summarized as follows.

Original: • We propose a general scalable deep learning framework for spatiotemporal time series, whichexploits a novel encoding method based on randomized recurrent components and scalable GNNsarchitectures.•
Revised: • We propose a general scalable deep learning framework for spatiotemporal time series, which exploits a novel encoding method based on randomized recurrent components and scalable GNNs architectures.

Original:  We apply the proposed model to forecast multivariate time series, whose channels are subject tospatial relationships described by a graph.• We carry out a rigorous and extensive empirical evaluation of the proposed architecture andvariations thereof.
Revised: • We apply the proposed model to forecast multivariate time series, whose channels are subject to spatial relationships described by a graph. • We carry out a rigorous and extensive empirical evaluation of the proposed architecture and variations thereof.

Original: Notably, we introduce two benchmarks for scalable spatiotemporal forecastingarchitectures.
Revised: Notably, we introduce two benchmarks for scalable spatiotemporal forecasting architectures.

Original: Empirical results show that our approach performs on par with the state of the art while being easy toimplement, computationally efficient, and extremely scalable
Revised: Empirical results show that our approach performs on par with the state of the art while being easy to implement, computationally efficient, and extremely scalable.

Original: Given these considerations, we refer to our architecture as Scalable Graph Predictor (SGP) .
Revised: Given these considerations, we refer to our architecture as Scalable Graph Predictor (SGP) .

Original:  Preliminaries and Problem Definition
Revised:  Preliminaries and Problem Definition

Original: We consider discrete-time spatiotemporal graphs.
Revised: We consider discrete-time spatiotemporal graphs.

Original: In particular, given N interlinked sensors, weindicate with x i t ∈ R d x the d x -dimensional multivariate observation associated with the i -th sensor attime-step t , with X t ∈ R N × d x the node attribute matrix encompassing measurements graph-wise, andwith X t : t + T the sequence of T measurements collected in the time interval [ t, t + T ) at each sensor.
Revised: In particular, given N interlinked sensors, we indicate with x i t ∈ R d x the d x -dimensional multivariate observation associated with the i -th sensor at time-step t , with X t ∈ R N × d x the node attribute matrix encompassing measurements graph-wise, and with X t : t + T the sequence of T measurements collected in the time interval [ t, t + T ) at each sensor.

Original: Similarly, we indicate with U t ∈ R N × d u the matrix containing exogenous variables (e.g., weatherinformation related to a monitored area) associated with each sensor at the t -th time-step.
Revised: Similarly, we indicate with U t ∈ R N × d u the matrix containing exogenous variables (e.g., weather information related to a monitored area) associated with each sensor at the t -th time-step.

Original: Then,we indicate additional,optional, static node attributes as V ∈ R N × d v . The relational information
Revised: Then, we indicate additional,

Original: C O NC
Revised: C O NC

Original: A T E N A T
Revised: A T E N A T

Original: E
Revised: E

Original: [Figure] p ace time . .
Revised: Echo State Network s p ace time . .

Original: is encoded in a, potentially dynamic, weighted adjacency matrix A t ∈ R N × N .
Revised: is encoded in a, potentially dynamic, weighted adjacency matrix A t ∈ R N × N .

Original: We indicate withthe tuple G t = ⟨ X t , U t , V , A t ⟩ the graph signal at the t -th time-step.
Revised: We indicate with the tuple G t = ⟨ X t , U t , V , A t ⟩ the graph signal at the t -th time-step.

Original: Note that the number ofsensors in a network is here considered fixed only to ease the presentation; we only request nodesto be distinguishable across time steps.
Revised: Note that the number of sensors in a network is here considered fixed only to ease the presentation; we only request nodes to be distinguishable across time steps.

Original: The objective of spatiotemporal forecasting is to predictthe next H observations given a window of W past measurements.
Revised: The objective of spatiotemporal forecasting is to predict the next H observations given a window of W past measurements.

Original: In particular, we consider thefamily of forecasting models F θ ( · ) such that (cid:99)
Revised: In particular, we consider the family of forecasting models F θ ( · ) such that (cid:99)

Original: X t : t + H
Revised: X t : t + H

Original: =
Revised: =

Original: F
Revised: F

Original: θ
Revised: θ

Original: ( G t − W : t ) , where θ indicates the learnableparameters of the model and (cid:99)
Revised: ( G t − W : t ) , where θ indicates the learnable parameters of the model and (cid:99)

Original: X t : t + H the H -step ahead point forecast.
Revised: X t : t + H the H -step ahead point forecast.

Original: Echo-State Networks Echo state networks [7, 10] are a class of randomized architectures thatconsist of recurrent neural networks with random connections that encode the history of input signalsinto a high-dimensional state representation to be used as input to a (trainable) readout layer.
Revised: Echo-State Networks Echo state networks [7, 10] are a class of randomized architectures that consist of recurrent neural networks with random connections that encode the history of input signals into a high-dimensional state representation to be used as input to a (trainable) readout layer.

Original: Themain idea is to feed an input signal into a high-dimensional, randomized, and non-linear reservoir,whose internal state can be used as an embedding of the input dynamics.
Revised: The main idea is to feed an input signal into a high-dimensional, randomized, and non-linear reservoir, whose internal state can be used as an embedding of the input dynamics.

Original: An echo state networkis governed by the state update equation h t =
Revised: An echo state network is governed by the state update equation h t =

Original: σ ( W x x t + W h h t − 1 + b ) , where x t indicates ageneric input to the system, W x ∈ R d h × d x and W h ∈ R d h × d h are the random matrices defining theconnectivity pattern in the reservoir, b ∈ R d
Revised: σ ( W x x t + W h h t − 1 + b ) , where x t indicates a generic input to the system, W x ∈ R d h × d x and W h ∈ R d h × d h are the random matrices defining the connectivity pattern in the reservoir, b ∈ R d

Original: h is a randomly initialized bias, h t indicates the reservoirstate, and σ is a nonlinear activation function (usually tanh).
Revised: h is a randomly initialized bias, h t indicates the reservoir state, and σ is a nonlinear activation function (usually tanh).

Original: If the random matrices are definedproperly, the reservoir will extract a rich pool of dynamics characterizing the system underlying theinput time series x t and, thus, the reservoir states become informative embeddings of x t − T : t [10].
Revised: If the random matrices are defined properly, the reservoir will extract a rich pool of dynamics characterizing the system underlying the input time series x t and, thus, the reservoir states become informative embeddings of x t − T : t [10].

Original: Thanks to the non-linearity of the reservoir, the embeddings are commonly processed with a linearreadout that is optimized with a least squares procedure to perform classification, clustering, or timeseries forecasting [11].
Revised: Thanks to the non-linearity of the reservoir, the embeddings are commonly processed with a linear readout that is optimized with a least squares procedure to perform classification, clustering, or time series forecasting [11].

Original:  Scalable Spatiotemporal GNNs
Revised:  Scalable Spatiotemporal GNNs

Original: This section presents our approach to building scalable GNN architectures for time series forecasting.
Revised: This section presents our approach to building scalable GNN architectures for time series forecasting.

Original: Our method is based on a hybrid encoder-decoder architecture.
Revised: Our method is based on a hybrid encoder-decoder architecture.

Original: The encoder first constructs representations of the time series observed at each node by using a reservoir that accounts for dynamics atdifferent time scales.
Revised: The encoder first constructs representations of the time series observed at each node by using a reservoir that accounts for dynamics at different time scales.

Original: Representations are further processed to account for spatial dynamics describedby the graph structure.
Revised: Representations are further processed to account for spatial dynamics described by the graph structure.

Original: In particular, as shown on the right-hand side of Fig. 2, we use incrementalpowers of the graph adjacency matrix to propagate and aggregate information along the spatialdimension.
Revised: In particular, as shown on the right-hand side of Fig. 2, we use incremental powers of the graph adjacency matrix to propagate and aggregate information along the spatial dimension.

Original: Each power of the propagation matrix accounts for different scales of spatial dynamics.
Revised: Each power of the propagation matrix accounts for different scales of spatial dynamics.

Original: The final embedding is then built by concatenating representations obtained w.r.t.
Revised: The final embedding is then built by concatenating representations obtained w.r.t.

Original: each propagationstep, thus resulting in a rich encoding of both spatial and temporal features.
Revised: each propagation step, thus resulting in a rich encoding of both spatial and temporal features.

Original: The encoder does not need any training and, once computed, the embeddings can be uniformlysampled over time and space when training a nonlinear readout to perform H -step-ahead predictions.
Revised: The encoder does not need any training and, once computed, the embeddings can be uniformly sampled over time and space when training a nonlinear readout to perform H -step-ahead predictions.

Original: The straightforward choice for the decoder (i.e., readout) is to map the encodings to the outputs (i.e.,predictions) by using a linear transformation or a standard MLP.
Revised: The straightforward choice for the decoder (i.e., readout) is to map the encodings to the outputs (i.e., predictions) by using a linear transformation or a standard MLP.

Original: However, to further enhancescalability, our decoder exploits the structure of the embedding to reduce the number of parametersand learn filters that are localized in time and space.
Revised: However, to further enhance scalability, our decoder exploits the structure of the embedding to reduce the number of parameters and learn filters that are localized in time and space.

Original: As we will discuss, this is done by learningseparate weight matrices for each spatiotemporal scale.
Revised: As we will discuss, this is done by learning separate weight matrices for each spatiotemporal scale.

Original: In the following, we describe each componentof the architecture in detail.
Revised: In the following, we describe each component of the architecture in detail.

Original: Spatiotemporal Encoder We consider as temporal encoders deep echo state networks (DeepESN; Gallicchio et al.
Revised: Spatiotemporal Encoder We consider as temporal encoders deep echo state networks (DeepESN; Gallicchio et al.

Original: 8) with leaky integrator neurons [12].
Revised: 8) with leaky integrator neurons [12].

Original: In particular, we consider networkswhere the signal associated with each node is encoded first as h i, (0) t =(cid:2) x it ∥ u
Revised: In particular, we consider networks where the signal associated with each node is encoded first as h i, (0) t =

Original: it (cid:3) and then by a stackof L randomized recurrent layers s.t.

Original: [Equation]
Revised: [Equation]

Original: i, [Equation] l
Revised: x it ∥ u it (cid:3) and then by a stack of L randomized recurrent layers s.t.

Original: [Equation]
Revised: [Equation]

Original: where γ l ∈ (0 , 1] is a discount factor associated with l -th layer, W ( l )
Revised: where γ l ∈ (0 , 1] is a discount factor associated with l -th layer, W ( l )

Original: u ∈ R d hl × d hl − 1 , W h ∈ R
Revised: u ∈ R d hl × d hl − 1 , W h ∈ R

Original: d hl × d hl ,b ∈ R
Revised: d hl × d hl , b ∈ R

Original: d
Revised: d

Original: hl are random weight matrices, h i, ( l ) t indicates the hidden state of the system w.r.t.
Revised: hl are random weight matrices, h i, ( l ) t indicates the hidden state of the system w.r.t.

Original: the i -thnode at the l -th layer, and ∥ indicates node-wise concatenation.
Revised: the i -th node at the l -th layer, and ∥ indicates node-wise concatenation.

Original: As Eq.
Revised: As Eq.

Original: 1 shows, DeepESNs are ahierarchical stack of reservoir layers that, e.g., by changing the discount factor at each layer, extract arich pool of multi-scale temporal dynamics [8] 1 .
Revised: 1 shows, DeepESNs are a hierarchical stack of reservoir layers that, e.g., by changing the discount factor at each layer, extract a rich pool of multi-scale temporal dynamics [8] 2 .

Original: Given a DeepESN encoder, the input is representedby the concatenation of the states from each layer, i.e., we obtain node-level temporal encodings h itfor each node i and time-step t as h it = (cid:16) h i, (0)t
Revised: Given a DeepESN encoder, the input is represented by the concatenation of the states from each layer, i.e., we obtain node-level temporal encodings h it for each node i and time-step t as h it = (cid:16) h i, (0)

Original: ∥ h i, (1) t ∥ . . .
Revised: t [Equation]

Original: ∥h
Revised: ∥

Original: i, ( L ) t (cid:17) .
Revised: [Equation] (cid:17) .

Original: We indicate as H t the encodingfor the whole graph at time t .
Revised: We indicate as H t the encoding for the whole graph at time t .

Original: The extraction of the node-level temporal embeddings is depicted onthe left-end side of Fig.
Revised: The extraction of the node-level temporal embeddings is depicted on the left-end side of Fig.

Original: 2, where, to simplify the drawing, we depict an ESN with a single layer.
Revised: 2, where, to simplify the drawing, we depict an ESN with a single layer.

Original: The next step is to propagate information along the spatial dimension.
Revised: The next step is to propagate information along the spatial dimension.

Original: As discussed at the beginning ofthe section, we use powers of a graph shift operator (cid:101)
Revised: As discussed at the beginning of the section, we use powers of a graph shift operator (cid:101)

Original: A to propagate and aggregate node representationsat different scales.
Revised: A to propagate and aggregate node representations at different scales.

Original: We then obtain spatiotemporal encodings as
Revised: We then obtain spatiotemporal encodings as

Original: [Equation]
Revised: [Equation]

Original: with S t = (cid:16) S (0) t ∥ S (1) t ∥ . . .
Revised: with S t = (cid:16) S (0) t ∥ S (1) t ∥ . . .

Original: ∥ S ( K ) t (cid:17) and where (cid:101)
Revised: ∥ S ( K ) t (cid:17) and where (cid:101)

Original: A indicates a generic graph shift operator matchingthe sparsity pattern of the graph adjacency matrix.
Revised: A indicates a generic graph shift operator matching the sparsity pattern of the graph adjacency matrix.

Original: In practice, by indicating with D the graph degreematrix, we use (cid:101)
Revised: In practice, by indicating with D the graph degree matrix, we use (cid:101)

Original: A
Revised: A

Original: = D − 1 A in the case of a directed graph and the symmetrically normalized adjacency (cid:101)
Revised: = D − 1 A in the case of a directed graph and the symmetrically normalized adjacency (cid:101)

Original: A
Revised: A

Original: =
Revised: =

Original: D − 1 / 2 AD − 1 / 2 in the undirected case.
Revised: D − 1 / 2 AD − 1 / 2 in the undirected case.

Original: Furthermore, for directed graphs we optionallyincrease the number of representations to 2 K + 1 to account for bidirectional dynamics, i.e., werepeat the encoding process w.r.t.
Revised: Furthermore, for directed graphs we optionally increase the number of representations to 2 K + 1 to account for bidirectional dynamics, i.e., we repeat the encoding process w.r.t.

Original: the transpose adjacency matrix similarly to [14].
Revised: the transpose adjacency matrix similarly to [14].

Original: Intuitively, eachpropagation step (cid:101) AS ( k − 1) t propagates and aggregates – properly weighted – features between nodesconnected by paths of length k in the graph.
Revised: Intuitively, each propagation step (cid:101) AS ( k − 1) t propagates and aggregates – properly weighted – features between nodes connected by paths of length k in the graph.

Original: As shown in Eq. 2, features corresponding to each orderk can be computed recursively with K sparse matrix-matrix multiplications (Fig. 2).
Revised: As shown in Eq. 2, features corresponding to each order k can be computed recursively with K sparse matrix-matrix multiplications (Fig. 2).

Original: Alternatively,each matrix (cid:101)
Revised: Alternatively, each matrix (cid:101)

Original: A k can be precomputed and the computation of the different blocks of matrix S t canbe distributed in a parallel fashion.
Revised: A k can be precomputed and the computation of the different blocks of matrix S t can be distributed in a parallel fashion.

Original: Even in the case of extremely large graphs, features S t can becomputed offline by exploiting distributed computing as they do not need to be loaded on the GPUmemory.
Revised: Even in the case of extremely large graphs, features S t can be computed offline by exploiting distributed computing as they do not need to be loaded on the GPU memory.

Original: Multi-Scale Decoder
Revised: Multi-Scale Decoder

Original: The role of the decoder is that of selecting and weighing from the pool ofthe (possibly redundant) features extracted by the spatiotemporal encoder and mapping them to thedesired output.
Revised: The role of the decoder is that of selecting and weighing from the pool of the (possibly redundant) features extracted by the spatiotemporal encoder and mapping them to the desired output.

Original: Representations S t can be fed into an MLP that performs node-wise predictions.
Revised: Representations S t can be fed into an MLP that performs node-wise predictions.

Original: Sincethe representations are large vectors, a naïve implementation of the MLP results in many parametersthat hinder scalability.
Revised: Since the representations are large vectors, a naïve implementation of the MLP results in many parameters that hinder scalability.

Original: Therefore, we replace the first MLP layer with a more efficient implementationthat exploits the structure of the embeddings.
Revised: Therefore, we replace the first MLP layer with a more efficient implementation that exploits the structure of the embeddings.

Original: As we described in the previous paragraph, S t is theconcatenation of the representations corresponding to different spatial propagation steps which, inturn, are obtained from the concatenation of multi-scale temporal features.
Revised: As we described in the previous paragraph, S t is the concatenation of the representations corresponding to different spatial propagation steps which, in turn, are obtained from the concatenation of multi-scale temporal features.

Original: To exploit this structure,we design the first layer of the decoder with a sparse connectivity pattern to learn representations Z tsuch that
Revised: To exploit this structure, we design the first layer of the decoder with a sparse connectivity pattern to learn representations Z t such that

Original: [Equation]
Revised: [Equation]

Original: where Θ ( l ) k ∈ R d hl ×
Revised: where Θ ( l ) k ∈ R d hl ×

Original: d z are the learnable parameters and σ is an activation function.
Revised: d z are the learnable parameters and σ is an activation function.

Original: We indicatewith Z t the concatenated representations Z t = (cid:16) Z (0) t
Revised: We indicate with Z t the concatenated representations Z t = (cid:16) Z (0) t

Original: ∥ Z (1) t ∥ . . .
Revised: ∥ Z (1) t ∥ . . .

Original: ∥
Revised: ∥

Original: Z ( K ) t (cid:17) .
Revised: Z ( K ) t (cid:17) .

Original: In practice, Z t can beefficiently computed by exploiting grouped 1 -d convolutions (e.g., see Krizhevsky et al. 15) toparallelize computation on GPUs.
Revised: In practice, Z t can be efficiently computed by exploiting grouped 1 -d convolutions (e.g., see Krizhevsky et al. 15) to parallelize computation on GPUs.

Original: In particular, if we indicate the 1 -d grouped convolution operatorwith g groups and kernel size r as ⋆ r,g , and the collection of the decoder parameters Θ ( l ) k as Θ , weobtain Z t = σ (cid:0) Θ ⋆ 1 ,g S t (cid:1) , with g = L ( K +1) in the case of undirected graphs and g = L (2 K +1)for the directed case.
Revised: In particular, if we indicate the 1 -d grouped convolution operator with g groups and kernel size r as ⋆ r,g , and the collection of the decoder parameters Θ ( l ) k as Θ , we obtain Z t = σ (cid:0) Θ ⋆ 1 ,g S t (cid:1) , with g = L ( K +1) in the case of undirected graphs and g = L (2 K +1) for the directed case.

Original: Besides reducing the number of parameters by a factor of L ( K + 1) , thisarchitecture localizes filters Θ ( L ) k w.r.t.
Revised: Besides reducing the number of parameters by a factor of L ( K + 1) , this architecture localizes filters Θ ( L ) k w.r.t.

Original: the dynamics of spatial order k and temporal scale l .
Revised: the dynamics of spatial order k and temporal scale l .

Original: In fact,as highlighted in Eq. 3, representation Z t can be seen as a concatenation of the results of L ( K + 1)graph convolutions of different order.
Revised: In fact, as highlighted in Eq. 3, representation Z t can be seen as a concatenation of the results of L ( K + 1) graph convolutions of different order.

Original: Finally, the obtained representations are fed into an MLPthat predicts the H -step-ahead observations as (cid:98) x
Revised: Finally, the obtained representations are fed into an MLP that predicts the H -step-ahead observations as (cid:98) x

Original: it : t + H =
Revised: it : t + H =

Original: MLP (cid:0) z it , v i (cid:1) where the static node-levelattributes v i can also be augmented by concatenating a set of learnable parameters (i.e., a learnablepositional encoding).
Revised: MLP (cid:0) z it , v i (cid:1) where the static node-level attributes v i can also be augmented by concatenating a set of learnable parameters (i.e., a learnable positional encoding).

Original: Training and sampling The main improvement introduced by the proposed approach in terms ofscalability concerns the training procedure.
Revised: Training and sampling The main improvement introduced by the proposed approach in terms of scalability concerns the training procedure.

Original: Representations S t embed both the temporal and spatialrelationships among observations over the sensor network.
Revised: Representations S t embed both the temporal and spatial relationships among observations over the sensor network.

Original: Consequently, each sample s i t can beprocessed independently since no further spatiotemporal information needs to be collected.
Revised: Consequently, each sample s i t can be processed independently since no further spatiotemporal information needs to be collected.

Original: Thisallows for training the decoder with SGD by uniformly and independently sampling mini-batchesof data points s it .
Revised: This allows for training the decoder with SGD by uniformly and independently sampling mini-batches of data points s it .

Original: This is the key property that makes the training procedure extremely scalable anddrastically reduces the lower bound on the computational complexity required for the training w.r.t.standard spatiotemporal GNN architectures.
Revised: This is the key property that makes the training procedure extremely scalable and drastically reduces the lower bound on the computational complexity required for the training w.r.t. standard spatiotemporal GNN architectures.

Original:  Related works
Revised:  Related works

Original: Spatiotemporal GNNs are essentially based on the idea of integrating message-passing modulesin architectures to process sequential data.
Revised: Spatiotemporal GNNs are essentially based on the idea of integrating message-passing modules in architectures to process sequential data.

Original: Notably, Seo et al.
Revised: Notably, Seo et al.

Original: [16] and Li et al.
Revised: [16] and Li et al.

Original: [14] use messagepassing to implement gates of recurrent neural networks.
Revised: [14] use messagepassing to implement gates of recurrent neural networks.

Original: Yu et al.
Revised: Yu et al.

Original: [17] and Wu et al.
Revised: [17] and Wu et al.

Original: [18, 19]proposed architectures alternating temporal and spatial convolutions.
Revised: [18, 19] proposed architectures alternating temporal and spatial convolutions.

Original: Wu et al.
Revised: Wu et al.

Original: [9] and Marisca et al.[20], instead, exploit the attention mechanism to propagate information along both time and space.
Revised: [9] and Marisca et al. [20], instead, exploit the attention mechanism to propagate information along both time and space.

Original: Modern architectures often combine some type of relational inductive bias, with full Transformer-likeattention [21] along the spatial dimension [22–24], which, however, makes the computation scalequadratically with the number of nodes.
Revised: Modern architectures often combine some type of relational inductive bias, with full Transformer-like attention [21] along the spatial dimension [22–24], which, however, makes the computation scale quadratically with the number of nodes.

Original: SGP falls within the category of time-then-graph models ,i.e., models where the temporal information is encoded before being propagated along the spatialdimension.
Revised: SGP falls within the category of time-then-graph models , i.e., models where the temporal information is encoded before being propagated along the spatial dimension.

Original: Gao and Ribeiro [25] showed that such models can be more expressive than architecturesthat alternate temporal and spatial processing steps.
Revised: Gao and Ribeiro [25] showed that such models can be more expressive than architectures that alternate temporal and spatial processing steps.

Original: Research on scalable models for discrete-time dynamic graphs has been relatively limited.
Revised: Research on scalable models for discrete-time dynamic graphs has been relatively limited.

Original: Practitioners have mostly relied on methods developed in the context of static graphs which include node-centric,
Revised: Practitioners have mostly relied on methods developed in the context of static graphs which include node-centric,

Original: GraphSAGE-like, approaches [3] or subgraph sampling methods, such as ClusterGCN [4] or GraphSAINT [5].
Revised: GraphSAGE-like, approaches [3] or subgraph sampling methods, such as ClusterGCN [4] or GraphSAINT [5].

Original: Wu et al.
Revised: Wu et al.

Original: [19], Gandhi et al.
Revised: [19], Gandhi et al.

Original: [26], Wu et al.
Revised: [26], Wu et al.

Original: [27] are examples of such approaches.
Revised: [27] are examples of such approaches.

Original: Among scalable GNNs for static graphs, SIGN [6] is the approach most related to our method.
Revised: Among scalable GNNs for static graphs, SIGN [6] is the approach most related to our method.

Original: Likein our approach, SIGN performs spatial propagation as a preprocessing step by using different shiftoperators to aggregate across different graph neighborhoods, which are then fed to an MLP.
Revised: Like in our approach, SIGN performs spatial propagation as a preprocessing step by using different shift operators to aggregate across different graph neighborhoods, which are then fed to an MLP.

Original: However,
Revised: However,

Original: SIGN is limited to static graphs and propagates raw node-level attributes.
Revised: SIGN is limited to static graphs and propagates raw node-level attributes.

Original: Finally, similar to our work,
Revised: Finally, similar to our work,

Original: DynGESN [28] processes dynamical graphs with a recurrent randomized architecture.
Revised: DynGESN [28] processes dynamical graphs with a recurrent randomized architecture.

Original: However, thearchitecture in DynGESN is completely randomized, while ours is hybrid as it combines randomizedcomponents in the encoder with trainable parameters in the decoder.
Revised: However, the architecture in DynGESN is completely randomized, while ours is hybrid as it combines randomized components in the encoder with trainable parameters in the decoder.

Original:  Empirical evaluation
Revised:  Empirical evaluation

Original: We empirically evaluate our approach in 2 different scenarios.
Revised: We empirically evaluate our approach in 2 different scenarios.

Original: In the first, we compare the performanceof our forecasting architecture against state-of-the-art methods on popular, medium-scale, trafficforecasting benchmarks.
Revised: In the first, we compare the performance of our forecasting architecture against state-of-the-art methods on popular, medium-scale, traffic forecasting benchmarks.

Original: In the second, we evaluate the scalability of the proposed method on largescale spatiotemporal time series datasets by considering two novel benchmarks for load forecastingand PV production prediction.
Revised: In the second, we evaluate the scalability of the proposed method on largescale spatiotemporal time series datasets by considering two novel benchmarks for load forecasting and PV production prediction.

Original: Datasets In the first experiment we consider the METR-LA and PEMS-BAY datasets [14], whichare popular medium-sized benchmarks used in the spatiotemporal forecasting literature.
Revised: Datasets In the first experiment we consider the METR-LA and PEMS-BAY datasets [14], which are popular medium-sized benchmarks used in the spatiotemporal forecasting literature.

Original: In particular,
Revised: In particular,

Original: METR-LA consists of traffic speed measurements taken every 5 minutes by 207 detectors in the
Revised: METR-LA consists of traffic speed measurements taken every 5 minutes by 207 detectors in the

Original: Los Angeles County Highway, while PEMS-BAY includes analogous observations recorded by 325sensors in the San Francisco Bay Area.
Revised: Los Angeles County Highway, while PEMS-BAY includes analogous observations recorded bysensors in the San Francisco Bay Area.

Original: We use the same preprocessing steps of previous works toextract a graph and obtain train, validation and test data splits [18].
Revised: We use the same preprocessing steps of previous works to extract a graph and obtain train, validation and test data splits [18].

Original: For the second experiment, weintroduce two larger-scale datasets derived from energy analytics data.
Revised: For the second experiment, we introduce two larger-scale datasets derived from energy analytics data.

Original: The first dataset containsdata coming from the Irish Commission for Energy Regulation Smart Metering Project ( CERE ; Commission for Energy Regulation 29), which has been previously used for benchmarkingspatiotemporal imputation methods [30]; however, differently from previous works, we considerthe full sensor network consisting of 6435 smart meters measuring energy consumption every 30minutes at both residential and commercial/industrial premises.
Revised: The first dataset contains data coming from the Irish Commission for Energy Regulation Smart Metering Project ( CERE ; Commission for Energy Regulation 29), which has been previously used for benchmarking spatiotemporal imputation methods [30]; however, differently from previous works, we consider the full sensor network consisting of 6435 smart meters measuring energy consumption everyminutes at both residential and commercial/industrial premises.

Original: The second large-scale dataset isobtained from the synthetic PV-US 2 dataset [31], consisting of simulated energy production by[-
-]
Revised: The second large-scale dataset is obtained from the synthetic PV-US 3 dataset [31], consisting of simulated energy production by[-
-]

Original: PV farms scattered over the United States given historic weather data for the year 2006 , aggregated inhalf an hour intervals.
Revised: PV farms scattered over the United States given historic weather data for the year 2006 , aggregated in half an hour intervals.

Original: Since the model does not have access to weather information, PV productionat neighboring farms is instrumental for obtaining good predictions.
Revised: Since the model does not have access to weather information, PV production at neighboring farms is instrumental for obtaining good predictions.

Original: Notably, CER-E and PV-USdatasets are at least an order of magnitude larger than the datasets typically used for benchmarkingspatiotemporal time series forecasting models.
Revised: Notably, CER-E and PV-US datasets are at least an order of magnitude larger than the datasets typically used for benchmarking spatiotemporal time series forecasting models.

Original: Note that for both PV-US and CER-En the (weighted)adjacency is obtained by applying a thresholded Gaussian kernel to the similarity matrix obtainedby considering the geographic distance among the sensors and the correntropy [32] among the timeseries, respectively.
Revised: Note that for both PV-US and CER-En the (weighted) adjacency is obtained by applying a thresholded Gaussian kernel to the similarity matrix obtained by considering the geographic distance among the sensors and the correntropy [32] among the time series, respectively.

Original: We provide further details on the datasets in the supplemental material.
Revised: We provide further details on the datasets in the supplemental material.

Original: Baselines We consider the following baselines:
Revised: Baselines We consider the following baselines:

Original: LSTM : a single standard gated recurrent neural network [33] trained by sampling window ofobservations from each node-level time series by disregarding the spatial information; 
Revised: LSTM : a single standard gated recurrent neural network [33] trained by sampling window of observations from each node-level time series by disregarding the spatial information; 2.

Original: FC-LSTM : an LSTM processing input sequences as if they were a single high-dimensionalmultivariate time series; 
Revised: FC-LSTM : an LSTM processing input sequences as if they were a single high-dimensional multivariate time series; 3.

Original: DCRNN : a recurrent graph network presented in [14] – differently from the original model weuse a recurrent encoder followed by a linear readout (more details in the appendix);4.
Revised: DCRNN : a recurrent graph network presented in [14] – differently from the original model we use a recurrent encoder followed by a linear readout (more details in the appendix); 4.

Original: Graph WaveNet : a residual network that alternates temporal and graph convolutions over thegraph that is given as input and an adjacency matrix that is learned by the model [18]; 
Revised: Graph WaveNet : a residual network that alternates temporal and graph convolutions over the graph that is given as input and an adjacency matrix that is learned by the model [18]; 5.

Original: Gated-GN : a state-of-the-art time-than-graph [25] model introduced in [24] for which we considertwo different configurations.
Revised: Gated-GN : a state-of-the-art time-than-graph [25] model introduced in [24] for which we consider two different configurations.

Original: The first one – indicated as FC – uses attention over the full node setto perform spatial propagation, while the second one – indicated as UG – constrains the attentionto edges of the underlying graph.6.
Revised: The first one – indicated as FC – uses attention over the full node set to perform spatial propagation, while the second one – indicated as UG – constrains the attention to edges of the underlying graph.

Original: DynGESN : the echo state network for dynamical graphs proposed in [28].
Revised: DynGESN : the echo state network for dynamical graphs proposed in [28].

Original: For all the baselines, we use, whenever possible, the configuration found in the original papers orin their open-source implementation; in all the other cases we tune hyperparameters on the holdout validation 
Revised: For all the baselines, we use, whenever possible, the configuration found in the original papers or in their open-source implementation; in all the other cases we tune hyperparameters on the holdout validation set.

Original: set. Experimental setup For the traffic datasets, we replicate the setup used in previous works.
Revised: Experimental setup For the traffic datasets, we replicate the setup used in previous works.

Original: Inparticular, each model is trained to predict the 12 -step-ahead observations.
Revised: In particular, each model is trained to predict the 12 -step-ahead observations.

Original: In SGP, the input timeseries are first encoded by the spatiotemporal encoder, and then the decoder is trained by samplingmini-batches along the temporal dimension, i.e., by sampling B sequences G t − W : t of observations.
Revised: In SGP, the input time series are first encoded by the spatiotemporal encoder, and then the decoder is trained by sampling mini-batches along the temporal dimension, i.e., by sampling B sequences G t − W : t of observations.

Original: For the large-scale datasets, we focus on assessing the scalability of the different architectures ratherthan maximizing forecasting accuracy.
Revised: For the large-scale datasets, we focus on assessing the scalability of the different architectures rather than maximizing forecasting accuracy.

Original: In particular, for both datasets, we consider the first 6 monthsof data ( 4 for months for training, 1 month for validation, and 1 month for testing).
Revised: In particular, for both datasets, we consider the first 6 months of data ( 4 for months for training, 1 month for validation, and 1 month for testing).

Original: The models aretrained to predict the next { 00 : 30 , 07 : 30 , 11 : 00 } hours.
Revised: The models are trained to predict the next { 00 : 30 , 07 : 30 , 11 : 00 } hours.

Original: We repeat the experiment in two differentsettings to test the scalability of the different architectures w.r.t.
Revised: We repeat the experiment in two different settings to test the scalability of the different architectures w.r.t.

Original: the number of edges.
Revised: the number of edges.

Original: In the firstsetting, we extract the graph by sparsifying the graph adjacency matrix imposing a maximum of 100neighbors for each node, while in the second case we do not constrain the density of the adjacencymatrix.
Revised: In the first setting, we extract the graph by sparsifying the graph adjacency matrix imposing a maximum ofneighbors for each node, while in the second case we do not constrain the density of the adjacency matrix.

Original: [Table]
Revised: [Table]

Original: and memory consumption.
Revised: and memory consumption.

Original: ∗ indicates that subsampling was needed to comply with the memory constraints.
Revised: ∗ indicates that subsampling was needed to comply with the memory constraints.

Original: Bold numbers are within a standard deviation from the best reported average result.
Revised: Bold numbers are within a standard deviation from the best reported average result.

Original: [Table]
Revised: [Table]

Original: To assess the performance in terms of scalability, we fix a maximum GPU memory budget of[-
-]
Revised: To assess the performance in terms of scalability, we fix a maximum GPU memory budget of[-
-]

Original: GB and select the batch size accordingly; if a batch size of 1 does not fit in 12 GB, we uniformlysubsample edges of the graph to reduce the memory consumption.
Revised: GB and select the batch size accordingly; if a batch size of 1 does not fit in 12 GB, we uniformly subsample edges of the graph to reduce the memory consumption.

Original: Differently from the otherbaselines, in SGP we first preprocess the data to obtain spatiotemporal embeddings and then trainthe decoder by uniformly sampling the node representations.
Revised: Differently from the other baselines, in SGP we first preprocess the data to obtain spatiotemporal embeddings and then train the decoder by uniformly sampling the node representations.

Original: We train each model for 1 hour, thenrestore the weights corresponding to the minimum training error and evaluate the forecasts on thetest set.
Revised: We train each model for 1 hour, then restore the weights corresponding to the minimum training error and evaluate the forecasts on the test set.

Original: The choice of not running validation at each epoch was dictated by the fact that for some ofthe baselines running a validation epoch would take a large portion of the 1 hour budget.
Revised: The choice of not running validation at each epoch was dictated by the fact that for some of the baselines running a validation epoch would take a large portion of the 1 hour budget.

Original: The timerequired to encode the datasets with SGP’s encoder ranges from tens of seconds to ≈ 4 minutes on an
Revised: The time required to encode the datasets with SGP’s encoder ranges from tens of seconds to ≈ 4 minutes on an

Original: AMD EPYC 7513 processor with 32 parallel processes.
Revised: AMD EPYC 7513 processor with 32 parallel processes.

Original: To ensure reproducibility, the time constraintis not imposed as a hard time out; conversely, we measure the time required for the update step ofeach model on an NVIDIA RTX A5000 GPU and fix the maximum number of updates accordingly.
Revised: To ensure reproducibility, the time constraint is not imposed as a hard time out; conversely, we measure the time required for the update step of each model on an NVIDIA RTX A5000 GPU and fix the maximum number of updates accordingly.

Original: For SGP, the time required to compute node embeddings was considered as part of the trainingtime and the number of updates was appropriately reduced to make the comparison fair.
Revised: For SGP, the time required to compute node embeddings was considered as part of the training time and the number of updates was appropriately reduced to make the comparison fair.

Original: For all thebaselines, we keep the same architecture used in the traffic experiment.
Revised: For all the baselines, we keep the same architecture used in the traffic experiment.

Original: For SGP we use the samehyperparameters for the decoder, but we reduce the dimension of the embedding (the value of K ) sothat a preprocessed dataset can fit in a maximum of ≈ 80 GB of storage.
Revised: For SGP we use the same hyperparameters for the decoder, but we reduce the dimension of the embedding (the value of K ) so that a preprocessed dataset can fit in a maximum of ≈ 80 GB of storage.

Original: To account for the differenttemporal scales, we increase the window size for all baselines and increase the number of layers inthe ESN (while keeping the final size of H t similar).
Revised: To account for the different temporal scales, we increase the window size for all baselines and increase the number of layers in the ESN (while keeping the final size of H t similar).

Original: Additional details and the exact values of thehyperparameters are provided in the supplementary material.
Revised: Additional details and the exact values of the hyperparameters are provided in the supplementary material.

Original: 5.1 Results
Revised: 5.1 Results

Original: Results for the traffic benchmarks are reported in Tab. 1; while the outcomes of the scalabilityexperiments are shown in Tab. 
Revised: Results for the traffic benchmarks are reported in Tab. 1; while the outcomes of the scalability experiments are shown in Tab. 2.

Original: We consider mean absolute error (MAE), and mean absolutepercentage error (MAPE) as evaluation metrics.
Revised: We consider mean absolute error (MAE), and mean absolute percentage error (MAPE) as evaluation metrics.

Original: Traffic experiment The purpose of the first experiment is to demonstrate that the proposed methodachieves performance comparable to that of the state of the art.
Revised: Traffic experiment The purpose of the first experiment is to demonstrate that the proposed method achieves performance comparable to that of the state of the art.

Original: In this regard, results in Tab. 1show that in all the considered scenarios SGP is always among the best performing forecastingarchitectures.
Revised: In this regard, results in Tab.show that in all the considered scenarios SGP is always among the best performing forecasting architectures.

Original: The full-attention baseline is the strongest competitor which, however, has time andmemory complexities that scale quadratically with the number of nodes.
Revised: The full-attention baseline is the strongest competitor which, however, has time and memory complexities that scale quadratically with the number of nodes.

Original: Regarding the other baselines,
Revised: Regarding the other baselines,

Original: DCRNN underperforms compared to the other spatiotemporal GNN architectures.
Revised: DCRNN underperforms compared to the other spatiotemporal GNN architectures.

Original: DynGESN, thefully randomized architecture, despite being very fast to train, obtains reasonable performance inshort-range predictions but falls short over longer forecasting horizons in the considered scenarios.
Revised: DynGESN, the fully randomized architecture, despite being very fast to train, obtains reasonable performance in short-range predictions but falls short over longer forecasting horizons in the considered scenarios.

Original: In light of these results, it is worth commenting on the efficiency of SGP compared to the baselines.
Revised: In light of these results, it is worth commenting on the efficiency of SGP compared to the baselines.

Original: Approaches like DCRNN and Graph Wavenet, perform graph convolutions whose time and spaceof complexity is O ( LTE ) , being E the number of edges, L the number of layers ( 8 in Graph
Revised: Approaches like DCRNN and Graph Wavenet, perform graph convolutions whose time and space of complexity is O ( LTE ) , being E the number of edges, L the number of layers ( 8 in Graph

Original: Wavenet), and T the time steps.
Revised: Wavenet), and T the time steps.

Original: Such complexity is completely amortized by the preprocessingstep in our architecture.
Revised: Such complexity is completely amortized by the preprocessing step in our architecture.

Original: Similarly, Gated-GN, while being architecturally much simpler, propagatesspatial information by relying on the attention mechanism that is known to scale poorly with thedimensionality of the problem.
Revised: Similarly, Gated-GN, while being architecturally much simpler, propagates spatial information by relying on the attention mechanism that is known to scale poorly with the dimensionality of the problem.

Original: The bottom of Tab.
Revised: The bottom of Tab.

Original: 1 reports results for the ablation of key elementsof the proposed architecture: No-Space-Enc. indicates that the embeddings are built without thespatial propagation step; FC-Dec. consider the case where the structure of the embedding is ignoredin the readout and the sparse weight matrix in Eq.
Revised: 1 reports results for the ablation of key elements of the proposed architecture: No-Space-Enc. indicates that the embeddings are built without the spatial propagation step; FC-Dec. considers the case where the structure of the embedding is ignored in the readout and the sparse weight matrix in Eq.

Original: 3 is replaced by a fully-connected one; GC-Dec.indicates that the spatial propagation is limited to the neighbors of order K = 1 and, thus, the decoderbehaves similarly to a single-layer graph convolutional network.
Revised: 3 is replaced by a fully-connected one; GC-Dec. indicates that the spatial propagation is limited to the neighbors of order K = 1 and, thus, the decoder behaves similarly to a single-layer graph convolutional network.

Original: Results clearly show the optimalityof the proposed architectural design.
Revised: Results clearly show the optimality of the proposed architectural design.

Original:  Large-scale experiment Tab. 2 reports the results of the scalability experiment where we considered only the spatiotemporal GNNs trained by gradient descent.
Revised: [Figure] Large-scale experiment Tab. 2 reports the results of the scalability experiment where we considered only the spatiotemporal GNNs trained by gradient descent.

Original: We excluded the full-attentionbaseline (FC-Gated-GN) as its O ( N 2 ) complexity prevented scaling to the larger datasets; however,we considered the UG version where attention is restrained to each node’s neighborhood.
Revised: We excluded the full-attention baseline (FC-Gated-GN) as its O ( N 2 ) complexity prevented scaling to the larger datasets; however, we considered the UG version where attention is restrained to each node’s neighborhood.

Original: Thereare several comments that need to be made here.
Revised: There are several comments that need to be made here.

Original: First of all, batch size has a different meaning forour model and the other baselines.
Revised: First of all, batch size has a different meaning for our model and the other baselines.

Original: In our case, each sample corresponds to a single spatiotemporal(preprocessed) observation; for the other methods, a sample corresponds to a window of observations G t − W : t where edges of the graph are eventually subsampled if the memory constraints couldnot be met otherwise.
Revised: In our case, each sample corresponds to a single spatiotemporal (preprocessed) observation; for the other methods, a sample corresponds to a window of observations G t − W : t where edges of the graph are eventually subsampled if the memory constraints could not be met otherwise.

Original: In both cases, the loss is computed w.r.t.
Revised: In both cases, the loss is computed w.r.t.

Original: all the observations in the batch.
Revised: all the observations in the batch.

Original: [Figure] The results clearly show that SGP can be trainedefficiently also in resource-constrained settings,with contained GPU memory usage.
Revised: The results clearly show that SGP can be trained efficiently also in resource-constrained settings, with contained GPU memory usage.

Original: In particular, the update frequency (batch/s) is up to 2order of magnitude higher.
Revised: In particular, the update frequency (batch/s) is up to 2 order of magnitude higher.

Original: Notably, resourceutilization at training time remains constant (byconstruction) in the two considered scenarios,while almost all the baselines require edge subsampling in order to meet the resource constraints.
Revised: Notably, resource utilization at training time remains constant (by construction) in the two considered scenarios, while almost all the baselines require edge subsampling in order to meet the resource constraints.

Original: Fig.
Revised: Fig.

Original: 3 shows learning curves for the PV-US dataset, further highlighting the vastlysuperior efficiency, scalability, and learning stability of SGP.
Revised: 3 shows learning curves for the PV-US dataset, further highlighting the vastly superior efficiency, scalability, and learning stability of SGP.

Original: Finally, results concerning theforecasting accuracy show that performance iscompetitive with the state of the art in all theconsidered scenarios.
Revised: Finally, results concerning the forecasting accuracy show that performance is competitive with the state of the art in all the considered scenarios.

Original:  Conclusion
Revised:  Conclusion

Original: We proposed SGP, a scalable architecture for graph-based spatiotemporal time series forecasting.
Revised: We proposed SGP, a scalable architecture for graph-based spatiotemporal time series forecasting.

Original: Our approach can compete with the state of the art in popular medium-sized benchmark datasets
Revised: Our approach can compete with the state of the art in popular medium-sized benchmark datasets, while greatly improving the scalability in large sensor networks.

Original: while greatly the scalability large sensor networks constitutesan important  for future research on scalable spatiotemporal forecasting  has thepotential of being widely adopted by practitioners in both academia and industry.
Revised: We believe that SGP constitutes an important stepping stone for future research on scalable spatiotemporal forecasting and has the potential of being widely adopted by practitioners in both academia and industry.

Original: Checklist
Revised: Checklist

Original: For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
Revised: For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?

Original: [Yes](b) Did you describe the limitations of your work?
Revised: [Yes] (b) Did you describe the limitations of your work?

Original: [Yes] (c) Did you discuss any potential negative societal impacts of your work?
Revised: [Yes] (c) Did you discuss any potential negative societal impacts of your work?

Original: [No] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
Revised: [No] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?

Original: [Yes] 
Revised: [Yes] 2.

Original: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
Revised: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?

Original: [N/A](b) Did you include complete proofs of all theoretical results?
Revised: [N/A] (b) Did you include complete proofs of all theoretical results?

Original: [N/A]3.
Revised: [N/A] 3.

Original: If you ran experiments...
Revised: If you ran experiments...

Original: (a) Did you include the code, data, and instructions needed to reproduce the main experimental[Table] results (either in the supplemental material or as a URL)?[No]
Revised: (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?

Original: [Table]

Original:  (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were[Table] chosen)?[Yes]
Revised: [No] (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?

Original: [Table]

Original:  (c) Did you report error bars (e.g., with respect to the random seed after running experiments[Table] multiple times)?[Yes]
Revised: [Yes] (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?

Original: [Table] (d) Did you include the total amount of compute and the type of resources used (e.g., type of[Table] GPUs, internal cluster, or cloud provider)?
Revised: [Yes] (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?

Original: [Yes] [Table]
Revised: [Yes] 4.

Original: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...[Table] (a) If your work uses existing assets, did you cite the creators?[Yes]
Revised: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?

Original: [Table]

Original:  (b) Did you mention the license of the assets?[Yes]
Revised: [Yes] (b) Did you mention the license of the assets?

Original: [Table] (c) Did you include any new assets either in the supplemental material or as a URL?[No] [Table]
Revised: [Yes] (c) Did you include any new assets either in the supplemental material or as a URL?

Original:  (d) Did you discuss whether and how consent was obtained from people whose data you’re[Table] using/curating?
Revised: [No] (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?

Original: [Yes]

Original: [Table] (e) Did you discuss whether the data you are using/curating contains personally identifiable[Table] information or offensive content?
Revised: [Yes] (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?

Original: [No] [Table]
Revised: [No] 5.

Original: If you used crowdsourcing or conducted research with human subjects...[Table] (a) Did you include the full text of instructions given to participants and screenshots, if applicable?[Table]
Revised: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?

Original: [N/A]

Original:  (b) Did you describe any potential participant risks, with links to Institutional Review Board[Table] (IRB) approvals, if applicable?[N/A]
Revised: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?

Original: [Table] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on[Table] participant compensation?
Revised: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?

Original: [N/A]
Revised: [N/A]

Original: [Table]

Original: Appendix 
Revised: Appendix A Detailed experimental settings

Original: A Detailed experimental settings

Original: In this appendix, we provide additional details on the experimental settings for the results presentedin the paper.
Revised: In this appendix, we provide additional details on the experimental settings for the results presented in the paper.

Original: A.1 Software platform
Revised: A.1 Software platform

Original: The Python [34] code used to run all the computational experiments will be open-sourced in a futurerelease.
Revised: The Python [34] code used to run all the computational experiments will be open-sourced in a future release.

Original: We relied on the following open-source libraries:
Revised: We relied on the following open-source libraries:

Original: • PyTorch [35];• PyTorch Geometric [36];• Torch Spatiotemporal [37];• PyTorch Lightning [38]; • numpy [39].
Revised: • PyTorch [35]; • PyTorch Geometric [36]; • Torch Spatiotemporal [37]; • PyTorch Lightning [38]; • numpy [39].

Original: We relied on the Neptune 3 [40] DevOps infrastructure for the logging of the experiments.
Revised: We relied on the Neptune 4 [40] DevOps infrastructure for the logging of the experiments.

Original: For all thebaselines, we run all the experiments by relying on their open-source implementations.
Revised: For all the baselines, we run all the experiments by relying on their open-source implementations.

Original: A.2 Hardware platform
Revised: A.2 Hardware platform

Original: Experiments were run on a server equipped with two AMD EPYC 7513 processors and four NVIDIA
Revised: Experiments were run on a server equipped with two AMD EPYC 7513 processors and four NVIDIA

Original: RTX A5000.
Revised: RTX A5000.

Original: Reproducibility of the scalability experiments was ensured by taking timings for the update step of each model and setting the number of updates performed by each model accordingly (moredetails in Sec. A.5).
Revised: Reproducibility of the scalability experiments was ensured by taking timings for the update step of each model and setting the number of updates performed by each model accordingly (more details in Sec. A.5).

Original: [Table]
Revised: [Table]

Original: A.3 Datasets
Revised: A.3 Datasets

Original: [Table]

Original: All datasets used in our study are open-source or freely available for research purposes.
Revised: All datasets used in our study are open-source or freely available for research purposes.

Original: The inputgraphs are extracted by at first computing a weighted, dense adjacency matrix W from (side) spatialinformation, e.g., the geographic position of the sensors, or by computing a (dis)similarity metricamong the time series.
Revised: The input graphs are extracted by at first computing a weighted, dense adjacency matrix W from (side) spatial information, e.g., the geographic position of the sensors, or by computing a (dis)similarity metric among the time series.

Original: The adjacency is then sparsified to obtain A by zeroing out values under acertain threshold and, optionally, capping the maximum number of neighbors for each node.
Revised: The adjacency is then sparsified to obtain A by zeroing out values under a certain threshold and, optionally, capping the maximum number of neighbors for each node.

Original: For alldatasets, the only exogenous variable we consider is the encoding of the time of the day with twosinusoidal functions.
Revised: For all datasets, the only exogenous variable we consider is the encoding of the time of the day with two sinusoidal functions.

Original: Tab/ 3 shows additional details of the considered datasets.
Revised: Tab/ 3 shows additional details of the considered datasets.

Original: Traffic datasets Both METR-LA and PEMS-BAY are widely popular benchmarks.
Revised: Traffic datasets Both METR-LA and PEMS-BAY are widely popular benchmarks.

Original: We use thesame setup of previous works [18] for all the preprocessing steps.
Revised: We use the same setup of previous works [18] for all the preprocessing steps.

Original: As mentioned in Sec. 5, PEMS-BAYcontains 6 months of data from 325 traffic sensors in the San Francisco Bay Area, while METR-LAcontains 4 months of analogous readings acquired from 207 detectors in the Los Angeles County
Revised: As mentioned in Sec. 5, PEMS-BAY contains 6 months of data from 325 traffic sensors in the San Francisco Bay Area, while METR-LA contains 4 months of analogous readings acquired from 207 detectors in the Los Angeles County

Original: Highway [41].
Revised: Highway [41].

Original: In both datasets, observations are aggregated at a 5 minutes time scale.
Revised: In both datasets, observations are aggregated at a 5 minutes time scale.

Original: CER-En
Revised: CER-En

Original: The data from the Irish Commission for Energy Regulation Smart Metering Project [29]contains measurements of the energy consumption aggregated at a 30 minutes scale in households andsmall/medium enterprises.
Revised: The data from the Irish Commission for Energy Regulation Smart Metering Project [29] contains measurements of the energy consumption aggregated at a 30 minutes scale in households and small/medium enterprises.

Original: The full dataset consists of observations from 6435 smart meters measuringenergy consumption every 30 minutes.
Revised: The full dataset consists of observations from 6435 smart meters measuring energy consumption every 30 minutes.

Original: As mentioned in the paper, we use the same preprocessingof [30], and, in particular, an analogous strategy to extract a graph from the correntropy [32] amongtime series.
Revised: As mentioned in the paper, we use the same preprocessing of [30], and, in particular, an analogous strategy to extract a graph from the correntropy [32] among time series.

Original: Note that, differently from [30], we consider the full sensor network.
Revised: Note that, differently from [30], we consider the full sensor network.

Original: For all thespatiotemporal GNN baselines, we set the window size to 36 steps.
Revised: For all the spatiotemporal GNN baselines, we set the window size to 36 steps.

Original: Access to the dataset canbe obtained free of charge by following the information provided at https://www.ucd.ie/issda/data/commissionforenergyregulationcer .
Revised: Access to the dataset can be obtained free of charge by following the information provided at https://www.ucd.ie/ issda/data/commissionforenergyregulationcer .

Original: PV-US
Revised: PV-US

Original: The PV-US 4 dataset [31] instead consists in a collection of simulated energy productionby 5016 PV farms for the year 2006 .
Revised: The PV-US 5 dataset [31] instead consists in a collection of simulated energy production by 5016 PV farms for the year 2006 .

Original: In the raw datasets, samples are generated every 5 minute, weaggregate observations at 30 minutes intervals by taking their mean.
Revised: In the raw datasets, samples are generated every 5 minute, we aggregate observations at 30 minutes intervals by taking their mean.

Original: A (small) subset of this dataset(often referred to as “Solar Energy" 5 ) with only the 137 PV plants in Alabama state has been used asa multivariate time series forecasting benchmark [42].
Revised: A (small) subset of this dataset (often referred to as “Solar Energy" 6 ) with only the 137 PV plants in Alabama state has been used as a multivariate time series forecasting benchmark [42].

Original: To obtain an adjacency matrix, we considerthe virtual position of the farms in terms of geographic coordinates, and we apply a Gaussian kernelover the pairwise Haversine distances, as described at the beginning of this section.
Revised: To obtain an adjacency matrix, we consider the virtual position of the farms in terms of geographic coordinates, and we apply a Gaussian kernel over the pairwise Haversine distances, as described at the beginning of this section.

Original: Similarly to the
Revised: Similarly to the

Original: CER-En dataset
Revised: CER-En dataset

Original: , we set the window size of the baselines to 36 steps.
Revised: , we set the window size of the baselines to 36 steps.

Original: In the supplementary material,we provide the code to download and preprocess the data.
Revised: In the supplementary material, we provide the code to download and preprocess the data.

Original: A.4 Additional details on SGP architecture
Revised: A.4 Additional details on SGP architecture

Original: We implemented the DeepESN encoder following the design principles assessed in previous works [13,43].
Revised: We implemented the DeepESN encoder following the design principles assessed in previous works [13, 43].

Original: In particular, we decrease the discount factor λ progressively at each layer by subtracting 0 .{+
+}
Revised: In particular, we decrease the discount factor λ progressively at each layer by subtracting {+
+}

Original: 1from its initial value.
Revised: .from its initial value.

Original: We also randomly set 30% of the weights of the networks to 0 to obtain a sparsereservoir.
Revised: We also randomly set 30% of the weights of the networks to 0 to obtain a sparse reservoir.

Original: We use tanh as nonlinear activation function.
Revised: We use tanh as nonlinear activation function.

Original: The recurrent weights are normalized so thatthe spectral radius of the corresponding matrix is lower than one [
Revised: The recurrent weights are normalized so that the spectral radius of the corresponding matrix is lower than one [7].

Original: For the spatial encoding, we compute the embeddings at the different spatial scales iteratively.
Revised: For the spatial encoding, we compute the embeddings at the different spatial scales iteratively.

Original: Additionally, we also concatenate to the spatiotemporal embedding S t the graph-wise average of thetemporal embedding H t to act as a sort of global attribute [44].
Revised: Additionally, we also concatenate to the spatiotemporal embedding S t the graph-wise average of the temporal embedding H t to act as a sort of global attribute [44].

Original: The MLP decoder is implemented as standard feed-forward network with parametrized residual connections between layers [45], SiLU activation function [46] and optional Dropout [47] regularization.
Revised: The MLP decoder is implemented as standard feed-forward network with parametrized residual connections between layers [45], SiLU activation function [46] and optional Dropout [47] regularization.

Original: A.5 Training and evaluation procedure 
Revised: A.5 Training and evaluation procedure A.5.1 Traffic

Original: A.5.1 Traffic

Original: As previously mentioned, for the traffic datasets we used the same training settings of previous works.
Revised: As previously mentioned, for the traffic datasets we used the same training settings of previous works.

Original: For all the baselines we kept the same parameters of previous works whenever possible.
Revised: For all the baselines we kept the same parameters of previous works whenever possible.

Original: For SGP weselected the hyperparameters by performing an initial random search and then manually adjusting thehyperparameters of the reservoir and selecting the best performing configuration on the validationset.
Revised: For SGP we selected the hyperparameters by performing an initial random search and then manually adjusting the hyperparameters of the reservoir and selecting the best-performing configuration on the validation set.

Original: In particular, for METR-LA we used a DeepESN with 3 layers of 32 units each, an initial decayfactor of 0 . 9 , and a spectral radius of 0 .
Revised: In particular, for METR-LA we used a DeepESN with 3 layers of 32 units each, an initial decay factor of 0 . 9 , and a spectral radius of 0 .

Original: For PEMS-BAY, instead, we used an encoder with a singlelayer of 128 units, a decay rate of 0 . 8 , and a spectral radius of 0 .9 .
Revised: For PEMS-BAY, instead, we used an encoder with a single layer of 128 units, a decay rate of 0 . 8 , and a spectral radius of 0 .

Original: For both datasets, we set K = 4and used the bidirectional encoding scheme.
Revised: For both datasets, we set K =and used the bidirectional encoding scheme.

Original: In the decoder, for the first layer we used 32 units foreach group in METR-LA and 96 PEMS-BAY, followed by 2 fully connected layers of 256 units eachwith a dropout rate of 0 . 3 .
Revised: In the decoder, for the first layer we used 32 units for each group in METR-LA and 96 PEMS-BAY, followed by 2 fully connected layers of 256 units each with a dropout rate of 0 . 3 .

Original: The model is trained with early stopping for a maximum of 200 epochs of300 batch each with the Adam optimizer and a multi-step learning rate scheduler.
Revised: The model is trained with early stopping for a maximum of 200 epochs of 300 batch each with the Adam optimizer and a multi-step learning rate scheduler.

Original: A.5.2 Large-scale
Revised: A.5.2 Large-scale

Original: In Tab. 2 of the paper, we report the time required for a single model update (in terms of batchesper second) and GPU memory usage for every considered method.
Revised: In Tab. 2 of the paper, we report the time required for a single model update (in terms of batches per second) and GPU memory usage for every considered method.

Original: To ensure a fair assessment, werecord the time interval between the beginning of the inference step and the end weights’ update for150 batches and exclude the first 5 and last 5 measurements (that may have overheads).
Revised: To ensure a fair assessment, we record the time interval between the beginning of the inference step and the end weights’ update for 150 batches and exclude the first 5 and last 5 measurements (that may have overheads).

Original: We excludefrom the computation the overhead introduced – for every batch – by the edge subsampling strategyadopted for the scalability of the baselines.
Revised: We exclude from the computation the overhead introduced – for every batch – by the edge subsampling strategy adopted for the scalability of the baselines.

Revised: GPU memory and time constraints were selected to show scalability in resource-constrained scenarios.

Original: To measure the GPU memory required, we exploit NVIDIA System Management Interface 6 , whichprovides near real-time GPU usage monitoring.
Revised: To measure the GPU memory required, we exploit NVIDIA System Management Interface 7 , which provides near real-time GPU usage monitoring.

Original: All the experiments designed to measure time and memory requirements have been run on thesame machine on a dedicated reserved GPU.
Revised: All the experiments designed to measure time and memory requirements have been run on the same machine on a dedicated reserved GPU.

Original: We kept the models mostly unchanged w.r.t.
Revised: We kept the models mostly unchanged w.r.t.

Original: thetraffic experiment.
Revised: the traffic experiment.

Original: However, we increased the window size to 36 for the baselines and updated theconfiguration of the reservoir for SGP to account for the different time scales.
Revised: However, we increased the window size to 36 for the baselines and updated the configuration of the reservoir for SGP to account for the different time scales.

Original: In particular, weincreased the number of reservoir layers to 8 and 6 in PV-US and to in CER-En, respectively, andreduced the number of units accordingly.
Revised: In particular, we increased the number of reservoir layers to 8 and 6 in PV-US and to in CER-En, respectively, and reduced the number of units accordingly.

Original: The difference in the number of layers between the twodatasets is motivated by the choice of keeping the size of the preprocessed sequences similar.
Revised: The difference in the number of layers between the two datasets is motivated by the choice of keeping the size of the preprocessed sequences similar.

Original: Forthis reason, we also set K = 2 and use the unidirectional encoding to limit the amount of requiredstorage to a maximum ≈ 80 GB for each dataset.
Revised: For this reason, we also set K = 2 and use the unidirectional encoding to limit the amount of required storage to a maximum ≈ 80 GB for each dataset.

Original: A.5.3 Baselines
Revised: A.5.3 Baselines

Original: For LSTM and FC-LSTM we consider a single-layer LSTM with 128 units for the temporalembedding and an MLP with one hidden layer with 256 units and dropout rate of 0 . 1 .
Revised: For LSTM and FC-LSTM we consider a single-layer LSTM with 128 units for the temporal embedding and an MLP with one hidden layer with 256 units and dropout rate of 0 . 1 .

Original: For DCRNN ,as reported in [14], we set the number of units in the hidden state to 64 and the order of thediffusion convolution to K = 2 ; compared to the original mode, we use a feed-forward readoutinstead of a recurrent one to enable scalability on the larger benchmarks.
Revised: For DCRNN , as reported in [14], we set the number of units in the hidden state to 64 and the order of the diffusion convolution to K = 2 ; compared to the original mode, we use a feed-forward readout instead of a recurrent one to enable scalability on the larger benchmarks.

Original: For Graph WaveNet and
Revised: For Graph WaveNet and

Original: Gated-GN we use the same hyperparameters and learning rate schedulers reported in the relativepapers.
Revised: Gated-GN we use the same hyperparameters and learning rate schedulers reported in the relative papers.

Original: We implemented all the baselines in PyTorch and PyTorch Geometric (for graph-basedmethods) following the open-source implementations provided by the authors.
Revised: We implemented all the baselines in PyTorch and PyTorch Geometric (for graph-based methods) following the open-source implementations provided by the authors.

Original: To improve memoryand computation efficiency in message-passing layers, we use sparse matrix-matrix multiplicationsinstead of scatter-gather operations whenever possible.
Revised: To improve memory and computation efficiency in message-passing layers, we use sparse matrix-matrix multiplications instead of scatter-gather operations whenever possible.

Original: We fix the maximum number of trainingepochs to 300 to allow all the models to reach convergence, and stop the training if the MAE computed on the validation set does not decrease for 50 epochs.
Revised: We fix the maximum number of training epochs to 300 to allow all the models to reach convergence, and stop the training if the MAE computed on the validation set does not decrease for 50 epochs.

Original: We evaluate the models using the weightscorresponding to the minimum validation MAE.
Revised: We evaluate the models using the weights corresponding to the minimum validation MAE.

Original: For DynGESN we set the hyperparameters of the reservoir to the same ones used for SGP and increasethe number of units to approximately match the dimensions of the final embeddings extracted by ourmethod.
Revised: For DynGESN we set the hyperparameters of the reservoir to the same ones used for SGP and increase the number of units to approximately match the dimensions of the final embeddings extracted by our method.

Original: We trained the readout with Ridge regression by selecting the weight of the L2-regularizationterm on the validation set.
Revised: We trained the readout with Ridge regression by selecting the weight of the L2-regularization term on the validation set.