AZ-whiteness test: a test for signal uncorrelation on spatio-temporal graphs
Authors: Daniele Zambon
Cesare Alippi
Venue: NeurIPS 2022 Accept (2022)
Keywords: Whiteness test
spatiotemporal time series
residual analysis
uncorrelated signals
graph neural networks
white noise

Abstract: We present the first whiteness hypothesis test for graphs, i.e., a whiteness test for multivariate time series associated with the nodes of a dynamic graph; as such, the test represents an important model assessment tool for graph deep learning, e.g., in forecasting setups. The statistical test aims at detecting existing serial dependencies among close-in-time observations, as well as spatial dependencies among neighboring observations given the underlying graph. The proposed AZ-test can be intended as a spatio-temporal extension of traditional tests designed for system identification to graph signals. The AZ-test is versatile, allowing the underlying graph to be dynamic, changing in topology and set of nodes over time, and weighted, thus accounting for connections of different strength, as it is the case in many application scenarios like sensor and transportation networks. The asymptotic distribution of the designed test can be derived under the null hypothesis without assuming identically distributed data. We show the effectiveness of the test on both synthetic and real-world problems, and illustrate how it can be employed to assess the quality of spatio-temporal forecasting models by analyzing the prediction residuals appended to the graph stream.

AZ-whiteness test: a test for signal uncorrelation on spatio-temporal graphs
Daniele Zambon
Cesare Alippi
The Swiss AI Lab IDSIA
The Swiss AI Lab IDSIA Università della Svizzera italiana
Università della Svizzera italiana Lugano, Switzerland daniele. zambon@usi.ccommmmmm
Politecnico di Milano Lugano, Switzerland
cesare. al apippi weiipppisss... ch
Abstract
We present the first whiteness hypothesis test for graphs, i.e., whiteness test for multivariate time series associated with the nodes of a dynamic graph; as such, the test represents an important model assessment tool for graph deep learning, e.g., in forecasting setups. The statistical test aims at detecting existing serial dependencies among close-in-time observations, as well as spatial dependencies among neighboring observations given the underlying graph. The proposed AZ- test can be intended as a spatio-temporal extension of traditional tests designed for system identification to graph signals. The AZ-test is versatile, allowing the underlying graph to be dynamic, changing in topology and set of nodes over time, and weighted, thus accounting for connections of different strength, it is the case in many application scenarios like sensor and transportation networks. The asymptotic distribution of the designed test can be derived under the null hypothesis without assuming identically distributed data. We show the effectiveness of the test both synthetic and real-world problems, and illustrate how it be employed assess the quality of spatio-temporal forecasting models by analyzing the prediction residuals appended to the graph stream.
1 Introduction
In the past decade, machine learning methods based on graph-structured data have largely contributed to the field of multivariate time-series analysis with major achievements, e.g., in forecasting and missing values imputation ( 25) In this paper, we focus spatio-temporal time series x, [t]. 2...... positioned[aa at the vertices v E of a graph; the functional/structural dependency existing among nodes say, over space - is modeled by set E E V x V of relations. We denote E, X) the resulting (directed or undirected) graph whose nodes v are associated with = E,
with stochastic time series [,[t] X, and name X [t] Vu, t} a graph signal, to be intuitively intended as a set of measurements appended to the graph nodes. The framework can be naturally extended to deal with graphs seen realization of random variable. Graph G' can be static, meaning that vertex and edge sets are constant over time, or dynamic, hence modeling frameworks where topology and number of nodes can change. Dynamic graphs appear frequently in cyber-physical systems where sensors can be added or removed, or node data are missing either sensor readout faults. Another
pointwise for lapses of time, e.g., following communication example is provided by social networks where users expand their set of friends or change preferences and interests time. Indeed, we account for graphs with weighted edges encoding, for instance,
Known as time-vertex signals too, e.g., see
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

x
WI.-
We.a (8]
Xa ld)
Dynamic graph
Time
Mag
v
[u, v) E2(e)
(Sparial) edge present at time step
xs ld
(. C
Weight of edge (u,, v) at time step
x. il R2.,
x.(t) Pott 0033 F F.dmm. ggral of
time stop.
Snapshot graph
(v(c), oft +
(Temporal) adga nodo
A dynamic weighted graph and associated graph signal. The dynamic graph is defined over a set V of nodes; without requesting all nodes to be always available over time. The topology of the graph represented by solid lines. Dashed lines represent temporal edges connecting the same node at consecutive time steps. The graph signal at generic time step t and node v is multivariate. On the left-hand side, single snapshot of the temporal graph and graph signal are extracted.
Figure I:
the capacity or the strength the link. Figure provides visualization of a weighted dynamic graph G' with associated graph signal X.
In the presented setting we address whether the graph is white or not. We define white graph as a graph G = (V, E, X) where for all distinct x, [t.]. x.lt, E X we have that E [x, [t]]]] = 02 is the F-dimensional zero vector (with F dimension of the node signals) and E [x. [[,])] is the F x F zero matrix, in accordance with traditional white signals as will see, the graph structure fundamental asset designing a whiteness test. In turn, identifying white graphs entails us, for instance, to assess the optimality of a predictive model fo trained to solve forecasting task associated with stochastic graph signal X. In this prediction application, we consider a residual graph with graph signal R composed of residuals r. [t] x. [t x.. evaluated over the estimated values x, given by model So and measured ones x., [4] Indeed, if the residual graph is not white, there is further information in data not exploited by predictive model fo and the inference problem shows margins for improvement. While assessing whether X (or R.) is zero-mean can be carried out with traditional centrality tests for generic data samples [24), here we focus on the rather unexplored problem of detecting data dependencies within graph signals.
0F
In this manuscript, we propose a novel whiteness test to decide whether graph G can be considered white or not, i.e., there is prominent evidence of serial (time) and/or spatial correlation among data observations conditioned to the graph structure. In general, testing the assumption of independent signals is an ill- pooepo problem as requires studying unknown distributions Pe. (of the stochastic process associated with X) whose number is proportional to the cardinality of nodes and time steps, and for each of which we have a single observation only. We solve this ill-posed problem by integrating into the whiteness assessment method the relational inductive bias associated with the graph topology and the temporal coherence. The AZ-test we propose extends traditional methods for serial correlation [I4 [9] deal with the general case of spatio-temporal correlation and, with this alphabetic encompassing name, we emphasize its widespread applicability. To the best of our knowledge, this is the first paper proposing general whiteness test. The suggested test statistic is based on counting the positive and negative signs of the product between spatially and temporally adjacent observations, whose disproportion indicates either direct or inverse correlation between variables pair; indeed, authors can consider different statistics.
The contributions of the paper be summarized as
We propose the first statistical test to verify the whiteness hypothesis for a, possibly weighted and dynamic, graph G [Sections and
We derive the limit distribution of the test statistics under the null hypothesis [Theorem
2A scalar stationary time series is said to be white it is zero-mean and the autocorrelation function x(T) = [[]]|]][[t - r]] = 0 for all * 0 which, in turn, holds if and only if the power density spectrum S(w) constant [20]: the term "white" is in analogy the white light containing all frequencies in the visible spectrum.

---

Table Configurations in which the proposed whiteness test is applicable.
Topology
Temp, dimension
Edge weights
Node signals
Correlation
Static
=-1
Absent
Scalar (F = 1)
1-hop/lag K K-hop/lag (K > 1)
Dynamic
T>1
Present
Multivariate (F' > 1)
We present procedure based on the proposed test to assess whether a given forecasting model can be considered optimal with respect to the given data or not [Section 5.2
The proposed whiteness test computationally efficient for sparse graphs as the number of operations scales linearly with the number of edges and time steps. Moreover, the test is very general and allows us for incorporating very relevant application contexts and operational scenarios, e.g., those listed in Table In particular, the test is applicable when [x is uni- or multivariate, when G is static or dynamic, when the edges of G are weighted, and when the graph signal composed of set of time series or it is static, namely, X = {x. E I e V}. and there is no temporal dimension involved.
The remainder of the paper is structured follows. Section 2 reviews related work. Section presents the test in the simplified case without the temporal dimension. Section shows how the test designes ppatio-temporal signals on dynamic graphs. Section [5 reports empirical
in Section can be applied evidence of the statistical power of the test and shows how to assess the optimality of forecasting models by applying the test to prediction residuals. Finally, Section (action6]drane draws some conclusions and provides pointers to future research. Supplemental material accompanies the manuscript with proofs, extensions of the proposed test, and further experimental details.
2 Related work
Among the most renowned whiteness tests, there are the Durbin-Watson test 9 and the Ljung-Box test [19]. Both have been introduced to test serial dependency in univariate time series. More recently, Drouiche [8] proposed a test that operates on the spectral density of the signal. Several
whiteness tests have been proposed for multivariate data as well, e.g., see 2
16. [8]
To the best of our knowledge, no work has presented a whiteness test for spatio-temporal signals, and what proposed here is pioneering in this direction. A few tests consider graph structure to represent relationships among the given observations [1] but none of them is a whiteness test. The AZ-test we propose shares the fundamental idea of Geary's 's test [12 of counting sign changes between consecutive observations in a univariate time series, but introduces several advancements considering graphs,
that make it substantial and original contribution. Apart from the fact which is a major contribution per se, the most important result is the applicability of the test to spatio-temporal signals defined over graphs. Secondly, the designed test is general enough to operate on weighted and dynamic graphs with multivariate node signals.
3 Whiteness test for static graphs
Z
For the sake of clarity, we first present the test in simplified yet relevant - setting characterized by a static weighted graph and a static signal with single observation (scalar or vector) associated with each node; an example is given in Figure 2] The most gen- eral case where time is involved and the graph dynamic, in the scenario depicted in Figure [ is provided in next Section A as an ex- tension of what we present here by considering suitably constructed (multiplex) graph involving temporal edges alongside spatial edges.
3F,,3
20g
24
21455
Consider weighted graph G (V, E, W. X) defined over node set V. edge set E C Vx x without solfs and scalar weights
Figure 2: A static graph G with time-independent multi- variate node signals.
W = {Wa,e
R (u, v) E E}.
As comment below, and in Supplemental Material, we avoid self- simplify the notation only.
3

---

The graph topology and weights define the spatial structure underlying a graph signal
x= [x, E V
defined over the nodes of G. We assume that the edge weights are positive real numbers that encode the strength/capacity of the links; without loss of generality, we consider absent edges characterized by weight equal to zero. Node signals X, R can be scalars (F' 1) or vectors (F' 1), but they are static, meaning that no temporal information is associated with them, as shown in Figure 2 Essentially, we consider a single snapshot of the dynamic graph depicted in Figure D
The test The ultimate goal is to test if graph signal X is white or it displays dependencies among nodes. The statistical hypotheses of the test are
Ho: Xus X are uncorrelated for all u * v E V, Hi: Xus X.. are correlated for some + E V,
(1)
and the proposed AZ-whiteness test is
If[[))) > =
Reject the null hypothesis Ho:
(2)
and it based on threshold y > and statistic
C(G) VW
C(G)
~N(
|E| - oc,
(3)
(as we prove in Theorem D. where |Ei denotes the cardinality
that is, the number of
edges in G - while C(G) is defined over quantities
Ö(G)= = Σ
Wur sgn(x,), x,),
(4)
(NNDEE
W
Σ
Σ (u,o)EEe
(Wup Wv
(B,
with sgn(x) 1 if 0, sgn(x) = if < and sgn(xr) = 0 if O, and where Ee, accounts for multiple edges linking the same pair of nodes, that is E,,, = 0 for undirected graphs, otherwise E+, = ((u, v) E
v, (v,u) E E}.
The intuition behind (2) to test null hypothesis Ho against H1 is that, when the node signals in X are mutually independent and centered around zero, then random variables sgn(x]]xx x.)) for all u & v E V are centered in zero too as proven in Lemma (supplemental material). In particular, in the case of scalar signals, sgn(x,, x,) sgn(x.) sgn(x.) and we observe that large values of C(G) > 0 indicate the presence of few sign changes between observations of adjacent nodes, which in turn suggests correlation among variables. Similarly, negative correlation revealed by C(G) < 0. Therefore, [C(G) >> O is symptomatic of a dependent data set up, and justifies our test 2
Theorem D supports the soundness A7-whiteness test (2) and provides distribution-fan free criterion to select threshold 7 granting a user-defined significance level a = P(Reject Ho Ho). e.g., a 0.05.
Theorem 1. Consider weighted graph G (V, F. W, X.X) without self self-loops and stochastic graph
signal X = =xx E RRF. E V} on it, with x, + almost surely. Under assumptions
(A1) All {xo V E V} are mutually independent (hypothesis Ho in (T).
(A2) Ex. [sgn(x xx)] for all x E IR" {0} and E V,
(A3) Wu, (0, w. for all (u,w) E E, and W co as E co.
the distribution of C(G) in 3 converges weakly to standard Gaussian distribution N 1) as the number [E| of edges goes infinity.
In light of above Theorem threshold is selected to be the quantile 1 - -a/2 of the standard Gaussian distribution so that PP(CC(G) > 7 Ho) = a, thus ensuring to meet the user-defined significance level a.

---

Before sketching the proof, we comment that, although Assumption ( A22) is not always compatible with the null-mean hypothesis for X to be white noise, we need (A2)
) to test data correlation only. Moreover, in the scalar case with F = (A2) reduces to ask that the median m of all X, is zero and. when m * O, we can safely B and equivalently run the test on X' (x, m 2 E V). Notably, ( (A2) allows node signals to have different distributions. Assumption (A3) is technical and takes part in the limit case of E + oc; intuitively, it ensures that all edges bring a tangible contribution to the final statistics (G)). The same theorem can be proven under milder assumptions. Finally, the assumptions of no self-loops and (x, = 0) = 0 are made to simplify the notation only. More comments on the assumptions of Theorem can be found in Section B of the supplemental material.
Sketch of the proof. The proof of Theorem I is based on the fact that, under (A1) and (A2) random variables sx,(x., Xu), for all (u, E, are mutually independent, including edges that share one of the two ending nodes. It follows that statistic C(G) in is a weighted sum of independent Bernoulli random variables. Finally, we prove that, with bounded weights, the Lindeberg condition (Equation 13 in the supplemental material) holds for C(G) under (A3) and conclude by applying the central limit theorem [ for independent, but not equally distributed, terms of C(G). The detailed proof is given in Section A of the supplemental material.
Despite the apparently little amount of information required to perform the test (only the sign of the scalar products x, x, of all edges (u, v) E E), Geary 12 has shown that for scalar time-series (interpretable as path graph) the test performance was aligned with that of more sophisticated tests. Empirical evidence of the effectiveness of the AZ-whiteness test is given in Section 5
4 Whiteness test for spatio-temporal graphs
In this section, we show how to apply the test designed in Section to generic spatio-temporal signal associated with a possibly dynamic and weighted graph.
Consider time frame [1, 77 and a discrete- -imee dynamic graph
G V E. W. W. x) (IVo) (ED) (WOLL (
(x))
(6)
where the arrow "->" on top of the symbols recalls their temporal nature. For each time step t, (V,, E,, W., X.) is static graph like the one in Figure when we consider time the reference figure is that of Figure | Since we are dealing with node-level time series, we have a correspondence between the nodes of G at different time steps and, in general. V,, nV, *0 tf * t2. Note that the possibility of having variable node set is important to model scenarios such those related the integration of new intersections in street map, removal of a faulty device in a real syber-physicn system, missing data from smart meters in power grid. As edge weights denote with wn,vlf] E W, the weight associated with edge (u, v) present in the edge set E, at time t. In this new setting, each node u E U. V, is associated with a stochastic time series x. [4] available for all time steps / 1, 1,2, ... T, if v V2 for all or only for subset of (1, 2, ..,.. T). This setting is depicted in Figure
be dynamic too,
Indeed, we can model a spatio-temporal signal X associated with a static graph (V, E, W) simply by assuming in 6 that Vi = V, E, E, and W. W, for all t.
The test The test designed in the previous section is able to identify spatial dependencies existing among the different nodes; here, we extend it to deal also with temporal dependencies that might exist among observations at different time steps. The null and alternative hypotheses in (I) are extended to incorporate the temporal dimension as
Ho : All pairs x,](4], x [ty] E X, with (v, 4) o (g) are uncorrelated;
Hi At least pair x,, x,[;)] X, with (u,4) o ty), ) is correlated.
'Testing the correlation in X or X' is equivalent, since [x( (xx.))(x. [[xx]]]] = [x 173 E[x-- -m])].
E[xx m]] mx))xx
5

---

The proposed whiteness test for
graphs is
If |C( > = Reject null hypothesis Ho;
(8)
where )(G") is statistic CG)) in 3 but applied to different graph G* defined below: the sums in and ( are now over all edges of G instead of G. Graph G is conveniently chosen (static) representation given dynamic graph G so that the theory from Section 3 applies to C(G") too (e.g. to choose 7 according to Theorem to test the uncorrelation of graph signal X on dynamic graph G; accordingly, test (8) relies on the same assumption set of test (2) but, this time, related to graph G
representation of graph G* given in Figure where the nodes of & are replicated for all their occurrences across time while preserving their spatial connectivity (solid lines), and temporal edges (dashed lines) added by connecting corresponding nodes at subsequent time steps. In the remainder of the section, we detail the construction of graph G (V*, , E W ,X) from given graph (6).
Nodes of G
Define node set V* the disjoint union of Vi, for all that is,
v {o [t] () E V4, f 1.2....,
If node is not present in node set V,. for some t, then t) of V*. Therefore, the number of nodes |V* is E, [Vel; for a static graph G. |V| = IVI-T.
Edges of G Define edge set E as collection of spatial and temporal edges. The set of spatial edges is the disjoint union of sets E,, for all that is
Exp [u([t) eft)) (u, v) E,, t = 1.2,...,
Temporal edges, instead, connect consecutive occurrences of the same nodes:
Fum [(u[t), uft + 1)) E V2 n V t= 1,2,, T-
The edge set E is then given by E = Fim U Esp. Referring to Figure | set Eup collects all edges represented by solid lines, while edges in Fem are represented as dashed lines.
Weights of G* Define edge weights in W following the construction of E". Spatial edges (u[4), v[4) E Exp are associated with weight Wa [f] E W, corresponding to edge (u, v) E F,. The weight of the temporal edges be arbitrarily. In general, when all edge weights have similar values, the spatial correlation contributes more to the final statistic than the temporal one, because EEp T V d Fumll where d is the average node degree. In Section Cf of the supplemental material, we show how to assign a weight Wem to every temporal edge and yield balanced spatial and temporal contributions. In addition, Section Crroontssssttt straightforward way with which the user can decide how to trade off the impacts of the spatial and temporal edges.
Signal X
Arrange graph signal X in (6) over the node set V of G as follows
X = [x = x. [t] RR v[d] E V*, with [ x).
To conclude, we observe that when T 1 the statistic reduces the one of Section 3 and that by preprocessing G + G* we are able to apply the very same statistical test (2) in different scenarios, including that of a dynamic graph, did in the current section. In Section of the supplemental material, we discuss another preprocessing of G that allows for considering K-hop and K-lag correlations.
Experiments
We consider two experimental setups. In the first one, we study the ability the AZ-test detect data dependency of various amplitudes in a controlled environment. In the second one, apply the test to a prediction problem both on synthetic and real-world data. The code to reproduce the experiments is available at httttss: // githhb.... com cww htmm/dzambon///o wwizwhhtteessssseeeeeeeeeee

---

1,0
0.8
0.6
N-3.11 M13.3
(122133
L144.00 U15.3
0:313
CANE
0:321
0.503
C141
0.312
6112
5365
0.291
244
0351
011
0.2
2.20
1663
0
0272
8391
1000
E447
2
2011
631
0.0
0:22
0.500
1,000
3.003
0141
0.59
1,200
2.41
0.91
3,00
690
16
106
3.0
1090
Figure 3: Rate of rejected null hypotheses for significance level a = 0.05. Each block corresponds to different distribution P. Every block has different correlation parameters the rows and number of time steps T on the columns. The node feature dimension is F = 1. The graph underlying the graph signals drawn on the right-hand side.
5.1 Detection of correlated residuals
We consider the undirected unweighted graph G = (V, E) of Figure 3 and denote with A its adjacency matrix (A = 1 if (u, v) E E and A.,v = O, otherwise). In this set of experiments, we generate white-noise and correlated graphs by sampling the components of a matrix E VRIV i.i.d. from scalar probability distributions P with null median. Choices of P include unimodal symmetric distributions, as well as asymmetric and bimodal ones; in Figure we consider the standard Gaussian distribution (0, 1). the chi-squared distributions (2(d) with d 1, 5 degrees of freedom, and the following mixtures of distributions: (N-3, 1) + N (+3, 1). x2 (1) x2(5), and U[-4 0) + U[O,, 1).
where U' La b) uniform distribution in [a, b). In particular, X {xx[[]] Ze,e} is graph signal of
independent observations, meeting the null hypothesis Ho in 77.
Then, we generate correlated signals by propagating an independent signal Z [[([ across graph G and time as
X Z1>1 + Cem ZisT + Sup A Zi>1 - m,
(9)
where Zucish indicates the reduction of Z to the columns associated with time steps E (a, bj. The correlation between observations is controlled by the positive parameters Csp and Cem- Finally, scalar offset m subtracted to have a null-median process X, as requested by the test. We generate graph signals for variable number T of time steps, dimension F' of the node features, and correlation parameters cap and Com from Figure 5 in the supplemental material, it is evident that the higher the values of Cap and Com. the more correlated the signals appear on both the temporal and spatial axes.
All experiments in the current section are repeated 1000 times, each time generating different sequence and applying test (8) with threshold y set to meet a user-defined significance level a 0.05. The considered figure of merit is the rate of rejected null hypotheses, that is, the number of times the test statistic is greater than 7 over the total number of repetitions of the same experiment.
Power of the test Figure shows the ability of the test identify deviations from the null hypothesis for different distributions P. number of time steps T, and values of parameter with Cap = c, Cim d and where d is the average node degree. We observe that (i) the higher the value of the easier the AZ-whiteness test identifies the correlation, and (ii) the test becomes more powerful as the number T of time steps increases.
Rate of f'alse positives From Figure 3 we can also analyze whether selected test threshold y yields the desired rate of false positives, namely, significance level a 0.05. We see that when (independent signals) the rate of rejected null hypotheses is around the predefined significance level a, regardless of the data distribution P (symmetric/asymmetrii,, unimodal/bimodal), hence suggesting that 7 is chosen correctly.
Similar results are displayed in Figure 7 in the supplemental material, where we varied the node feature dimension F too.
Distributions (d) are shifted to have null median.
7

---

5.2 Optimality of forecasting models
As second of experiments, we train state-of-the-ar forecasting models and assess the whiteness of the residual graph signal; when it is not the case, i.e., the residual signal is colored, then predictors have margin to be improved.
5.2.1 Experimental setting
We consider three datasets.
GPVAR is a synthetic dataset generated from a graph polynomial vector autoregressive system model [5]. The model generates each observation x,[t] R time & and node u E V from Q e N observations the past and L-hop neighboring nodes, with L E N, as follows:
L Σ 1=
x(t] = tanh
S'xlt + z(t]
(10)
where x[[] RN concatenates all scalar node signals at time step t, S is graph shift operator, @ E R[(+I)xQ collects the model parameters, and z|(
RN is white noise generated form the standard Gaussian distribution [Section 5.1 Specifically, we consider the undirected and un- weighted graph shown in Figure 4 with the fol- lowing shift operator S = D D-1/2 A)D 1/2 where A is the adjacency matrix (with now self- loops), D is the diagonal degree matrix, and I the identity matrix. Model parameters θ
Figure 4: Graph of GPVAR dataset.
set
[5, [-, [-], and result in L = Q = 2. We generate T 30000 time steps.
PemsBay is traffic dataset collected by the California Transportation Agencies Performance Measurement System (PeMS) [7]] presents = 52128 scalar observations from N = 325 sensors in the Bay Area.
MetrLA is dataset containing traffic information from = 207 detectors along the Los Angeles County highway [17]. Observations are collected for 4 months and amount to T 34272 time steps.
The task solve is a I-step-ahead forecasting problem where, for every 6, we want to predict graph signal x[[] from window {x[t -] E R- = 1, 1,....,);; T): 7 is the size of the considered temporal window. In addition, for PemsBay and MetrLA, positional encoding of the day of observation is added as exogenous variable. On the above forecasting tasks, we trained Graph WaveNet (25) (GWNet), a Gated Graph Network [23] (GatedGN), and a Diffusion Convolutional RNN L7 (DCRNN). We also consider baseline model (FCRNN) composed of a 1-layer fully- connected encoder h[t] = [(x[t]; u(t)) processing graph signal x(1) each time step t and the associated exogenous variables u and 2-layer GRU decoder applied to the resulting window of representations [h[t- r), ... h(t 1]]. The size of the temporal window is set to 12 for all experiments. Train, validation and test sets contain 0.7, 0.1, and 0.2 of the original data. The models are trained until convergence with patience of 50 epochs. All models and datasets are available in supplemental material.
the porchSpatiotemporal library (6). except for GPVAR dataset. Further details are available as
5.2.2 Results
In Ta we report the results for all datasets. In particular, report the mean absolute deviation (MAE) achieved by the above models on the different datasets and the results of a statistical test on the median of the residuals; MAE and residual median are related in that the minimum of the MAE should produce residuals with null median. The test on the median is implemented as test on the parameter Bernoulli random variable sgn(x (x, [t]). Then, we report the results produced by the proposed AZ-whitenes, test. In addition to the AZ-test run on the given graph (referred to as "Spatio-temporal"" in TableEp. we consider two modified versions to assess the impact the temporal and spatial components alone, and named "Temporal", where only by setting zero the weights of all spatial edges, and "Spatial", where we consider the spatial edges only by setting the weights of all temporal edges to zero; note that the three test statistics are
operate on the temporal edges

---

Table 2: Analysis of observed residuals. The tests of null median report the estimated median while the AZ-tests report the statistic C(G"); associated p- value are subscripted. Bold results highlight results with p-value > 0.01. Values reported as 0.000 are intended as < 0.001.
Test
AZ-test Spatio-temporal
AZ-test Temporal
AZ-test
Dataset
Model
MAE
Median=0
Spatial
GPVAR
Optimal Pred.
0.319
0.001
0.083
-0.8 0.416
-0.9 0.355
0.233
GPVAR
FCRNN
0.385
0.003 0.010
5.0 0.000
8.9 0.000
-1.8 0.067
GPVAR
GWNET
0.324
0.004 0.000
0.3 0,709
0.3 0.706
0.1 0.881
GPVAR
GATEDGN
0.321
0.008 0.000
1.3 0,172
2.7 0.006
-0.8 0.414
GPVAR
DCRNN
0.328
0.013 0.000
-0.0 0.955
-0.6 0.534
0.5 0.587
PemsBay
FCRNN
2.016
0.032 0.000
1107.4 0,000
1035.1 0.000
531.0 0.000
PemsBay
GWNET
0.841
-0.003 0.000
422.7 0.000
7.1 0.000
590.7 0.000
PemsBay
GATEDGN
0.838
0.018 0.000
454.6 0.000
25.2 0.000
617.7 0.000
PemsBay
DCRNN
0.845
-0.004
433.0 0.000
14.2 0.000
598. 0.000
Metrl.
FCRNN
2.842
0.0166..0m
415.3 0.000
238.5 0.000
348. 8 0.000
Metrl.A
GWNET
2.115
0.014 0.000
162.6 0.000
-6.5 0.000
236.5 0.000
MetrLA
GATEDGN
2.151
0.010 0.000
200.1 0.000
2.2 0.022
280.7 0.000
MetrLA
DCRNN
2.141
-0.018
177.4 0.000
6.7 0.000
244. 0.000
comparable among each other because, shown in Section C of the supplemental material, they all asymptotically distributed as a standard Gaussian distribution NN 1).
We start by considering the synthetic dataset GPVAR. In this experiment, we know the data generating process (TO) and we are able to contrast the considered methods against the optimal predictor, i.e., the graph polynomial VAR filter with the same parameter θ that generated the data. From the first row of T we discover the value of the reference (optimal) MAE and observe that the residual median can be considered null. Then, we note that the outcomes of the AZ-test suggest uncorrelated residuals. Baseline model FCRNN produces a MAE substantially higher than the reference MAE, non-null median of the residual, and correlated residuals. From the Temporal and Spatial variations of the AZ-test, we see that the temporal correlation appears more prominent than the spatial one for FCRNN. In contrast, we can consider the training of GWNet, GatedGN, and DCRNN successful because they produce MAE close to the target value and relatively high p-values in the AZ-tests.
Regarding PemsBay and MetrLA datasets, we do not have a reference MAE, as the optimal predictor unknown. None of DCRNN, GatedGN, and GWNet produces uncorrelated residuals, however, looking at the test statistics, GWNet, GatedGN, and DCRNN display lower temporal correlation than spatial. We point out that the null hypothesis is not rejected as a consequence of Assumption (A2) being invalid. We verified it in the supplemental material, where we contrasted the outcome of the analysis conducted on the residuals of FCRNN, GWNet, GatedGN, and DCRNN (Table with that on the same residual, but with the empirical median subtracted (Table 3). For all models and datasets, we obtain almost identical results. We conclude that there is further information left in the residuals that model can, in principle, learn.
Conclusions
In this work, we propose the first whiteness test for spatio-temporal time series defined over the nodes of a graph. By exploiting the known structural and functional relations defined by the underlying graph, we are able to identify both temporal and spatial correlations that exist among data observations. We show that the test is asymptotically distribution- with respect to the number of time steps and graph edges and, therefore, we can apply to data coming from arbitrary distributions and, even, nonidentically distributed observations.
The designed test general, from which the definitive alphabetic encompassing name, and based on sign changes between observations that are close in time or with respect to the graph connectivity. The test is very versatile, too, as it can exploit edge weights encoding the strength of the link, allows the graph topology vary over time, and operate when nodes are inserted removed from the graph;

---

these scenarios are frequent when dealing with real-world graph signals, like those coming from sensor networks, e.g., transportation networks, and smart grids. Finally, the test computationally scalable for sparse graphs with complexity linear in the number of edges and time steps.
We empirically show that the proposed test is indeed capable of identifying dependencies among graph signals; when applied to analyze prediction residuals, the AZ-test can assess whether given forecasting models can assumed to optimal or not.
The proposed whiteness test pioneering in its nature, and allows for future extensions involving stochastic graphs and more sophisticated edge statistics.
Acknowledgements
This work was supported by the Swiss National Science Foundation project FNS 204061: HORD GNN: Higher-Orde Relations and Dynamics in Graph Neural Networks.
References
[1] P. Billingsley. Probability and measure. John Wiley & Sons: New York, 1995.
[2] A. Bose and W. Hachem. whiteness test based on the spectral measure large non-hermitian random matrices. In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 8768-8771, IEEE, 2020.
[3] G. E. Box, G. M. Jenkins, G. C. Reinsel, and G. M. Ljung. Time series analysis: forecasting and control. John Wiley & Sons, 2015.
[4] H. Chen and N. R. Zhang. Graph-based tests for two-sample comparisons of categorical data. Statistica Sinica, pages 1479-1503, 2013.
[5] R. V. Chitturi. Distribution of residual autocorrelations in multiple autoregressive schemes. Journal of the American Statistical Association, 69(348):928-9 9344,,4, 1974.
[6] A. Cini and I. Marisca. Torch Spatiotemporal. 2022.
URL https: /ttps://github.. com/
TorchSpatiotemporall/ss
[7] A. Cini, I. Marisca, and C. Alippi. Filling the g_ Multivariate time series imputation by graph neural networks. In International Conference on Learning Representations, 2021.
[8] K. Drouiche. A new test for whiteness. IEEE transactions signal processing, 48(7): 1864-1871, 2000.
[9] J. Durbin and G. S. Watson. Testing for serial correlation in least squares regression: I. Biometrika, 37(3/4):409442) 1950.
[10] M. Fey and J. E. Lenssen. Fast Graph Representation Learning with PyTorch Geometric, 2019. URL BRL https: https/////gthbbbb wwwpp/pgggeeaapppprrrcc..ceeeee
[11] H. Friedman and L. C. Rafsky. Multivariate generalizations of the wald-wolfowitz and smirnov two-sample tests. The Annals of Statistics, pages 697-717, 1979.
[12] R. Geary. Relative efficiency of count of sign changes for assessing residual autoregression in least squares regression. Biometrika, 57(1):123-127, 04 1970. ISSN 0006-3444. doi: 10. 10.099/iomee//7.1.122.
[13] Y. Hochberg. A sharper bonferroni procedure for multiple tests of significance. Biometrika, 75 (4):800-802, 1988.
[14] J. R. Hosking. The multivariate portmanteau statistic. Journal of the American Statistical Association, (5(371):602-600) 1980.
[15] E. Isufi, A. Loukas, N. Perraudin, and Leus. Forecasting time series with varma recursions on graphs. IEEE Transactions Signal Processing, 67(19):880--488 2019.
10

---

[16] W. Li and A. McLeod. Distribution of the residual autocorrelations in multivariate arma time series models. Journal of the Royal Statistical Sociery: Series B (Methodological), 43(2): 231-239, 1981.
[17] Y. Li, R. Yu, C. Shahabi, and Y. Liu. Diffusion convolutional recurrent neural network: Data- driven traffic forecasting. International Conference on Learning Representations, 2018. URL https://openreviet: net forun/idddniinGGii...
[18] Z. Li, C. Lam. Yao, and Q. Yao. On testing for high-dimensione white noise. The Annals of Statistics, (7(6):3382-3411 2019.
[19] G. M. Ljung and G. E. Box. On measure of lack of fit in time series models, Biometrika, 65 (2):297-303, 1978.
[20] Z. Peebles Jr. Probability, random variables, and random signal principles. McGraw-Hill, 2001.
[21] N. Perraudin and P. Vandergheynst. Stationary signal processing on graphs. IEEE Transactions on Signal Processing, 65(13).3462-3477 2017.
[22] P. R. Rosenbaum. An exact distribution-fres test comparing two multivariate distributions based on adjacency. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 67 (4):515-530, 2005.
[23] V. Satorras, S. S. Rangapuram, and Januschowski. Multivariate time series forecasting with latent graph inference. arXiv preprint arXiv:2203.0344 2022.
[24] A. W. Van der Vaart. Asymptotic statistics, volume 3. Cambridge university press, 2000.
[25] Z. Wu, S. Pan, G. Long, J. Jiang, and C. Zhang. Graph wavenet for deep spatial-temporal graph modeling. In The 28th International Joint Conference on Artificial Intelligence (IJCAT). International Joint Conferences on Artificial Intelligence Organization, 2019.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes] A summary of the contribution in the introduction
refers to the related parts of the papers.
(b) Did you describe the limitations of your work? [Yes] The considered set of assumptions reported in the theorem's statement, and extended comments about them provided in the supplementary material, Section B
(c) Did you discuss any potential negative societal impacts of your work? [N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full of assumptions of all theoretical results? [Yes] See the theorem's statement and comments in the supplemental material, Section B
(b) Did you include complete proofs of all theoretical results? [Yes] In the supplemental material, Section A
3. If you ran experiments..
(a) Did you include the code, data, and instructions needed reproduce the main exper- imental results (either in the supplemental material or as a URL)? [Yes] The code is provided as supplemental material and details how to replicate the results. Also, a link will be made available upon acceptance.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] Details can found in Section and supplementary material, Section

---

(c) Did you report error bars (e.g., with respect to the random seed after running ex- periments multiple times)? [No] FestretJestimates Bernoulli distributions, the data variability can be derived by the number of repetitions and the estimated parameter In 2 we are concerned about how the test can be applied analyze prediction residuals, is not meant at providing benchmark.
(d) Did you include the total amount of compute and the type of resources used (e.g.. type of GPUs, internal cluster, or cloud provider)? [Yes] See supplementary material, Section F
4. If you are using existing assets (e.g., code, data, models) or curating/releasngg new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes] We mention 6 10
(b) Did you mention the license of the assets?
[No] It is reported on their respective
repositories [6 [[]] and are both MIT licenses.
(c) Did you include any new assets either in the supplemental material or as URL? [Yes] Code for running the paper experiments and GPVAR synthetic dataset. Both will be made available online upon paper acceptance.
(d) Did you discuss whether and how consent was obtained from people whose data you're using/curating? [N/A]
(c) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [No]
5. If you used crowdsourcing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A]
(b) Did you describe any potential participant risks, with links Institutional Review Board (IRB) approvals. if applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
12


Reviews:
[{"review_id": "oZgwRcCSHNf", "reviewer": "Reviewer_Dqpj", "summary": "The authors propose a novel model assessment tool by testing the whiteness (of the residuals) for dynamic graphs. Asymptotic normal null distribution (theorem 1) is developed, which enables easy calibration of the test. ", "questions": "See above in weakness", "limitations": "No. Could we extend this test to graphs with negative weights? I am asking this since in practice the interactions between nodes could be both triggering effect or inhibiting effect.", "rating": 6, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength:\n\nThe author proposed a statically solid testing method. Most importantly, the asymptotic null distribution is developed. Extensive numerical results help support the effectiveness of the method. Implementation is also provided to help recover the results.\n\nWeakness:\n1. (minor) notations: What is $\\gamma$ in eq. (2)? I can see it is the testing threshold on line 122, but I think it should be clarified right after the equation; In theorem 1, I would prefer using A1 for Assumption 1 instead of Ass1.\n\n2. What is the power of this test? I would expect the relation between power and (sample size, dimensionality). (PS: I do see the numerical assessment, but since you have the asymptotic null distribution, it would be better if you also have the (asymptotic) power. Or what is the technical difficulty is preventing you to give the power?)\n\n3. (major) Although this method is novel in terms of the setting, I would expect numerical comparison to existing approaches for static graphs with the mentioned work in line 85.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "lFYOcjGrr1J", "reviewer": "Reviewer_LJnR", "summary": "This paper proposed a whiteness test to examine whether a given graph is \"white\" or not. The designed test extends classical methods so that it can deal with general case of spatio-temporal correlation. This method applies for uni or multivariate node signals, statistic or dynamic grah, (possibly) weighted edges. The author also show that this test can be used to access the optimality of a given forecating model, which is an interesting and important application. ", "questions": "1. The distribution of testing statistic is only characterized when the number of edges $|E|\\to\\infty$. Does it mean this test may not be powerful for small graph? Is there any possible idea to address this issue?\n2. In section 4, to perform whiteness test for dynamic and weighted graph, the author argued that we should pre-process it to make it static and then previous approach in Section 3 applies. Would this pre-processing procedure bring increase of type-I/type-II error for AZ-test? Is there any pre-processing procedure and which one is optimal? From my view point, this procedure seems too ad-hoc, and I hope to see some theoretical results behind this procedure.", "limitations": "1. Some notations and terminologies are introduced right after they first appeared, e.g., definition of $\\text{sng}, E_{\\leftrightarrow}, |E|, \\rightarrow$. At least the author should create a notation subsection part in Section 1 to introduce all special notations that will be frequently used. If possible, the author may create a preliminary section in main content or supplementary to talk about basics of static/dynamic graph in detail, instead of briefly touching the basics in Section 3/4.\n2. Assumptions (Ass1)-(Ass3) are important and repeatedly discussed in later section. The author are suggested to create a special \"Assumption\" environment to include them instead of putting them in Theorem 1. Also, comments of the assumption can be moved from supplementary to the main content if the paper gets accepted.", "rating": 6, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Originality**: This is the first whiteness test that applies for spatio-temporal signals on graph. The design of this test share some idea from Geary' test, but make some additional advancements. Moreover, the designed test is general enough to deal with weighted and dynamic graphs with multivariate node signals.\n\n**Quality/Significance**: From theoretical perspective, the test procedure together with the limiting distribution of testing statistic is derived. I have checked the proof of their theoretical result in detail, and I believe it is clear and correct. From practical perspective, the author examine the ability of AZ-test in various settings and present an interesting application that it can examine the optimality of forecating model. Both parts are excellent contributions.\n\n**Clarity**: The overall presentation is good. However, minor points can be updated to make it more reader-friendly. See some comments in the Limitation section.\n\n\n**Weakness**: \n1. The author pointed out this test share some insights from Geary' test but made some extra innovations. The detailed comparison is not clear to me. I would expect the author to clarify this point in the related work section. \n2. Some presentations can be improved (See Limitations section).", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "Pb5328epd9m", "reviewer": "Reviewer_8o7F", "summary": "The paper describes an asymptotic test of whiteness for graph signals that can be used to assess graph signal forecasting models. The main idea is that a whiteness test for graph signals is useful for evaluating forecasting models. Since the model is optimal (in a second order sense) if the residuals are white such that there is no bias and there is no more correlation in the error that the model could account for.  The test is general under the key assumptions including that the signals across space and time can be considered one large graph that is growing in nodes and edges. This generality allows for dynamic graph topology. The test is demonstrated on synthetic and real world datasets. \n\n\n", "questions": "Suggestions:\n\nThe idea of appending the signal to the graph and defining G as including the signal can lead to subsequent confusion.\n\nDescription of whiteness on line 36 is not precise. The quantities involved are vectors or matrices (assuming an outer product) but zero is a scalar. (From the figures row vectors are denoted which means that the second expectation in line 36 is a inner product, but this doesn't make much sense to me. Later on line 110 uses an inner product, so the reader can now be assured that a correlation matrix is intended on line 36.) Furthermore the precise nature of the stochastic time series is not made clear. The stated assumption in the footnote is that the signal is stationary. However, in line 36 it is not clear if the expectations are functions of time or only the time lag like in the footnote. In practice, it is unlikely to have assess to multiple realizations for estimation so assumptions on ergodicity/stationarity should be made, but this point isn't made until around line 52. Also in the footnote it is not clear that $\\mathbf{x}$ is multivariate or univariate. \n\nLine 102: Is does null equal zero for the weights? I'm trying to see if this is subtle variation or if a null is somehow distinct from zero. \n\nOn line 59, it is not clear what is meant by \"; indeed, authors can consider different statistics.\"\n\n\"Supplemental material accompanies the manuscript.\" It would be useful to detail what this is rather than the blanket statement as provided. \n\nLine 109: It would be better for continuity to use \", where\" or equivalent instead of ending the sentence \"Theorem 1\".\n\nLine 156: I think the G should have an arrow on top in equation 6. \n\n\nThe number of trials used to compute the power is a bit too small. The numbers for the null hypothesis vary. A safer choice of 1000 may enable the reader to better assess whether the false positive rate. \n\nOn lines 290 and 291 the comment about p-values being significantly different from zero should be altered to simply say that the null hypothesis cannot be rejected.\n\nReference list has some problems with format namely book titles should be in title case. [1] appears to be missing punctuation.  The publication type for [12] Geary's test is unclear. \n\nQuestions:\n\nIn Theorem 1, what is the relationship of the edge cardinality and the node cardinality? \nPart of my confusion is that Theorem 1 seems to rely on a distribution over random graphs. For the number of edges to go to infinity, the number of nodes would also have to. But this initially seems to involve a distribution over graph topologies not really a distribution over graph signals. Assumption 3 is important but it described initially as simply being 'technical'. (Line 139 mentions milder assumptions, but it is not clear if that is directly relating to Ass3 or something else.  ) Later on I understand this to be formed from linking nodes through time to have a growing graph. In Section 4, the reader can infer that the growth in edges/nodes required for Ass3 practically corresponds to product graphs that grow through time. \n\nIn Table 2 it is not clear why the optimal predictor has a higher MAE than the models.  Additionally the DCRNN model has a significantly non-zero median but shows uncorrelated residuals. Should a user always use both tests to assess modeling performance?  Finally, the description states that the test statistic is not valid since Ass2 is not held. Overall, these mixed results are not very encouraging on the practical usefulness of the method. \n\n\n\n\n\n", "limitations": "It is not clear if there is performance thresholds for large number of vertices and a few temporal snapshots.\n\nThe paper is perhaps missing baselines for tests of whiteness for signals on regular grids (time series or spatial-temporal forecasting in geospatial settings) where other tests can be readily applied. In these cases, higher resolution (more nodes) can be readily be obtained. ", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\nThe utility of a robust whiteness measure as the signature of correct modeling of graph signals has practical utility and significance. The paper introduces a general test that is scalable and has an asymptotic characterization. The use of only the sign of the inner product and weights is straightforward and yields a simple asymptotic distribution. The papers coverage of the development and application is thorough. Many more results are mentioned in the supplemental material. The language is easy to read and relations to prior work are stated. \n\nWeaknesses: A key critique is the occasional lack of precise definitions  (especially at the onset) and implicit assumptions that are only revealed after a careful reading. Thus the paper is not clear what is required and how the asymptotic nature is practically realized. \n \nAnother critique is that the practical application to real world data and models seems to have mixed results in terms of the model performance (MAE), median of residuals, and the proposed test. \n\nNot until the conclusion do we learn the alphabetic nature of the name. It wasn't apparent to me throughout...\n\n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "1pfCiCxI5kJ", "reviewer": "Reviewer_z1xh", "summary": "The paper presents a whiteness hypothesis test for spatio-temporal graphs. The proposed AZ-test is an extension of traditional tests designed for system identification within graph signals, to detect dependencies among temporal observations as well as spatial dependencies among graph neighborhoods. Evaluation is performed on both synthetic as well as real datasets on problems of forecasting in spatio-temporal graphs by analyzing the prediction of residuals within the graph stream.", "questions": "1. The abstract and the first paragraph of section 4 mentions that the proposed work is applicable in the case of dynamic graphs ( allowing for changing connectivity patterns over time). This is handled by converting to a larger graph (Fig. 1) with nodes as a disjoint union of all the temporal nodes, (weighted) edges as a union of spatial and temporal edges. In the case that a new node v (and corresponding signal Xv[t]) is not present at time t, but added at at time point t+1, what is the corresponding value of Xv[t] used for forming the larger graph?\n\n2. In the results in Table 2, the authors mention that \"From the Temporal and Spatial variations of the AZ-test, we see that the temporal correlation appears more prominent than the spatial one for FCRNN\". Is this based on the absolute values of C(G*) [without normalisation] in the two columns? If so, is this a fair comparison given that the number of spatial vs temporal edges can be vastly different in number?", "limitations": "There isn't a dedicated discussion on the limitations of the proposed framework. In this context, it would be good to consider and address the point mentioned in weaknesses about applicability and potential impact.", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "STRENGTHS:\n\nThe paper is easy to follow and the presentation. The technical details and supporting theoretical analysis are clearly presented. The theoretical setup of extending the A-Z whiteness hypothesis test to (spatio-temporal) graphs is interesting and novel in formulation.\n\nWEAKNESSES:\n\nThe applicability of the proposed framework beyond the considered forcasting setup is a bit unclear. If my understanding is correct, the null hypothesis test is constructed as a test for a zero mean and as well as spatially and temporally uncorrelated generating process. As such the test statistic C(G) does not distinguish between spatial and  temporal correlation patterns. Given that this is a rather strong assumption, I am unsure how such a test would be generally useful beyond examining residuals in the forecasting setting. \n ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept