Original: F ILLING THE G AP S : M ULTIVARIATE T IME S ERIES I MPUTATION BY G RAPH N EURAL N ETWORKS
Revised: F ILLING THE G AP S : M ULTIVARIATE T IME S ERIES I MPUTATION BY G RAPH N EURAL N ETWORKS

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Dealing with missing values and incomplete time series is a labor-intensive, tedious, inevitable task when handling data coming from real-world applications.
Revised: Dealing with missing values and incomplete time series is a labor-intensive, tedious, inevitable task when handling data coming from real-world applications.

Original: Effective spatio-temporal representations would allow imputation methods to reconstruct missing temporal data by exploiting information coming from sensors at different locations.
Revised: Effective spatio-temporal representations would allow imputation methods to reconstruct missing temporal data by exploiting information coming from sensors at different locations.

Original: However, standard methods fall short in capturing the nonlinear time and space dependencies existing within networks of interconnected sensors and do not take full advantage of the available – and often strong – relational information.
Revised: However, standard methods fall short in capturing the nonlinear time and space dependencies existing within networks of interconnected sensors and do not take full advantage of the available – and often strong – relational information.

Original: Notably, most state-of-the-art imputation methods based on deep learning do not explicitly model relational aspects and, in any case, do not exploit processing frameworks able to adequately represent structured spatio-temporal data.
Revised: Notably, most state-of-the-art imputation methods based on deep learning do not explicitly model relational aspects and, in any case, do not exploit processing frameworks able to adequately represent structured spatio-temporal data.

Original: Conversely, graph neural networks have recently surged in popularity as both expressive and scalable tools for processing sequential data with relational inductive biases.
Revised: Conversely, graph neural networks have recently surged in popularity as both expressive and scalable tools for processing sequential data with relational inductive biases.

Original: In this work, we present the ﬁrst assessment of graph neural networks in the context of multivariate time series imputation.
Revised: In this work, we present the ﬁrst assessment of graph neural networks in the context of multivariate time series imputation.

Original: In particular, we introduce a novel graph neural network architecture, named GRIN , which aims at reconstructing missing data in the different channels of a multivariate time series by learning spatio-temporal representations through message passing.
Revised: In particular, we introduce a novel graph neural network architecture, named GRIN , which aims at reconstructing missing data in the different channels of a multivariate time series by learning spatio-temporal representations through message passing.

Original: Empirical results show that our model outperforms state-of-the-art methods in the imputation task on relevant real-world benchmarks with mean absolute error improvements often higher than 20% .
Revised: Empirical results show that our model outperforms state-of-the-art methods in the imputation task on relevant real-world benchmarks with mean absolute error improvements often higher than 20% .

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Imputation of missing values is a prominent problem in multivariate time-series analysis (TSA) from both theoretical and practical perspectives (Little & Rubin, 2019).
Revised: Imputation of missing values is a prominent problem in multivariate time-series analysis (TSA) from both theoretical and practical perspectives (Little & Rubin, 2019).

Original: In fact, in a world of complex interconnected systems such as those characterizing sensor networks or the Internet of Things, faulty sensors and network failures are widespread phenomena that cause disruptions in the data acquisition process.
Revised: In fact, in a world of complex interconnected systems such as those characterizing sensor networks or the Internet of Things, faulty sensors and network failures are widespread phenomena that cause disruptions in the data acquisition process.

Original: Luckily, failures of these types are often sparse and localized at the single sensor level, i.e., they do not compromise the entire sensor network at once.
Revised: Luckily, failures of these types are often sparse and localized at the single sensor level, i.e., they do not compromise the entire sensor network at once.

Original: In other terms, it is often the case that, at a certain time step, missing data appear only at some of the channels of the resulting multivariate time series.
Revised: In other terms, it is often the case that, at a certain time step, missing data appear only at some of the channels of the resulting multivariate time series.

Original: In this context, spatio-temporal imputation methods (Yi et al., 2016;
Revised: In this context, spatio-temporal imputation methods (Yi et al., 2016;

Original: Yoon et al., 2018b) aim at reconstructing the missing parts of the signals by possibly exploiting both temporal and spatial dependencies.
Revised: Yoon et al., 2018b) aim at reconstructing the missing parts of the signals by possibly exploiting both temporal and spatial dependencies.

Original: In particular, effective spatio-temporal approaches would reconstruct missing values by taking into account past and future values, and the concurrent measurements of spatially close neighboring sensors too.
Revised: In particular, effective spatio-temporal approaches would reconstruct missing values by taking into account past and future values, and the concurrent measurements of spatially close neighboring sensors too.

Original: Here, spatial similarity does not necessarily mean physical (e.g., geographic) proximity, but rather indicates that considered sensors are related w.r.t.
Revised: Here, spatial similarity does not necessarily mean physical (e.g., geographic) proximity, but rather indicates that considered sensors are related w.r.t.

Original: a generic (quantiﬁable) functional dependency (e.g., Pearson correlation or Granger causality – Granger, 1969) and/or that are close in a certain latent space.
Revised: a generic (quantiﬁable) functional dependency (e.g., Pearson correlation or Granger causality – Granger, 1969) and/or that are close in a certain latent space.

Original: Relational information, then, can be interpreted as a set of constraints – linking the different time series – that allows replacing the malfunctioning sensors with virtual ones.
Revised: Relational information, then, can be interpreted as a set of constraints – linking the different time series – that allows replacing the malfunctioning sensors with virtual ones.

Original: Among different imputation methods, approaches based on deep learning (LeCun et al., 2015;
Revised: Among different imputation methods, approaches based on deep learning (LeCun et al., 2015;

Original: Schmidhuber, 2015; Goodfellow et al., 2016) have become increasingly popular (Yoon et al., 2018a;
Revised: Schmidhuber, 2015; Goodfellow et al., 2016) have become increasingly popular (Yoon et al., 2018a;

Original: Cao et al., 2018; Liu et al., 2019).
Revised: Cao et al., 2018; Liu et al., 2019).

Original: However, these methods often completely disregard available relational information or rely on rather simplistic modiﬁcations of standard neural architectures tailored for sequence processing (Hochreiter & Schmidhuber, 1997; Chung et al., 2014; Bai et al., 2018;
Revised: However, these methods often completely disregard available relational information or rely on rather simplistic modiﬁcations of standard neural architectures tailored for sequence processing (Hochreiter & Schmidhuber, 1997; Chung et al., 2014; Bai et al., 2018;

Original: Vaswani et al., 2017).
Revised: Vaswani et al., 2017).

Original: We argue that stronger, structural, inductive biases are needed to advance the state of the art in time series imputation and allow to build effective inference engines in the context of large and complex sensor networks as those found in real-world applications.
Revised: We argue that stronger, structural, inductive biases are needed to advance the state of the art in time series imputation and allow to build effective inference engines in the context of large and complex sensor networks as those found in real-world applications.

Original: In this work, we model input multivariate time series as sequences of graphs where edges represent relationships among different channels.
Revised: In this work, we model input multivariate time series as sequences of graphs where edges represent relationships among different channels.

Original: We propose graph neural networks (GNNs)
Revised: We propose graph neural networks (GNNs)

Original: (Scarselli et al., 2008; Bronstein et al., 2017; Battaglia et al., 2018) as the building block of a novel, bi-directional, recurrent neural network for multivariate time series imputation (MTSI).
Revised: (Scarselli et al., 2008; Bronstein et al., 2017; Battaglia et al., 2018) as the building block of a novel, bi-directional, recurrent neural network for multivariate timeseries imputation (MTSI).

Original: Our method, named Graph
Revised: Our method, named Graph

Original: Recurrent Imputation Network ( GRIN ), has at its core a recurrent neural network cell where gates are implemented by message-passing neural networks (MPNNs; Gilmer et al., 2017).
Revised: Recurrent Imputation Network ( GRIN ), has at its core a recurrent neural network cell where gates are implemented by message-passing neural networks (MPNNs; Gilmer et al., 2017).

Original: Two of these networks process the input multivariate time series in both forward and backward time directions at each node, while hidden states are processed by a message-passing imputation layer which is constrained to learn how to perform imputation by looking at neighboring nodes.
Revised: Two of these networks process the input multivariate time series in both forward and backward time directions at each node, while hidden states are processed by a message-passing imputation layer which is constrained to learn how to perform imputation by looking at neighboring nodes.

Revised: In fact, by considering each edge as a soft functional dependency that constraints the value observed at corresponding nodes, we argue that operating in the context of graphs introduces a positive inductive bias for MTSI.

Original: Our contributions are manifold: 1) we introduce a methodological framework to exploit graph neural networks in the context of MTSI, 2) we propose a novel, practical and effective implementation of a
Revised: Our contributions are manifold: 1) we introduce a methodological framework to exploit graph neural networks in the context of MTSI, 2) we propose a novel, practical and effective implementation of a

Original: GNN-based architecture for MTSI, and 3) we achieve state-of-the-art results on several and varied
Revised: GNN-based architecture for MTSI, and 3) we achieve state-of-the-art results on several and varied

Original: MTSI benchmarks.
Revised: MTSI benchmarks.

Original: Our method does not rely on any assumption on the distribution of the missing values (e.g., presence and duration of transient dynamics and/or length of missing sequences) other than stationarity of the underlying process.
Revised: Our method does not rely on any assumption on the distribution of the missing values (e.g., presence and duration of transient dynamics and/or length of missing sequences) other than stationarity of the underlying process.

Original: The rest of the paper is organized as follows.
Revised: The rest of the paper is organized as follows.

Original: In Sectionwe discuss the related works.
Revised: In Sectionwe discuss the related works.

Original: Then, in Section 3, we formally introduce the problem settings and the task of MTSI.
Revised: Then, in Section 3, we formally introduce the problem settings and the task of MTSI.

Original: We present our approach to MTSI in Section 4, by describing the novel framework to implement imputation architectures based on GNNs.
Revised: We present our approach to MTSI in Section 4, by describing the novel framework to implement imputation architectures based on GNNs.

Original: We proceed with an empirical evaluation of the presented method against state-of-the-art baselines in Section 5 and, ﬁnally, we draw our conclusions in Section 6.
Revised: We proceed with an empirical evaluation of the presented method against state-of-the-art baselines in Section 5 and, ﬁnally, we draw our conclusions in Section 6.

Original:  R ELATED WORKS
Revised:  R ELATED WORKS

Original: Time series imputation There exists a large literature addressing missing value imputation in time series.
Revised: Time series imputation There exists a large literature addressing missing value imputation in time series.

Original: Besides the simple and standard interpolation methods based onlinear and polynomial curve ﬁtting, popular approaches aim at ﬁlling up missing values by taking advantage of the similarities among time series.
Revised: Besides the simple and standard interpolation methods based on polynomial curve ﬁtting, popular approaches aim at ﬁlling up missing values by taking advantage of standard forecasting methods and similarities among time series.

Original: For example, several approaches rely on k-nearest neighbors (Troyanskaya et al., 2001; Beretta & Santaniello, 2016) or matrix factorization (Cichocki & Phan, 2009; Mei et al., 2017). Other standard methods exploit the expectation-maximization algorithm (Ghahramani & Jordan, 1994; Nelwamondo et al., 2007) or linear predictors and state-space models (Durbin & Koopman, 2012; Kihoro et al., 2013). 
Revised: For example, several approaches rely on k-nearest neighbors (Troyanskaya et al., 2001; Beretta & Santaniello, 2016), the expectation-maximization algorithm (Ghahramani & Jordan, 1994; Nelwamondo et al., 2007) or linear predictors and statespace models (Durbin & Koopman, 2012; Kihoro et al., 2013). Low-rank approximation methods, such as matrix factorization (Cichocki & Phan, 2009), are also popular alternatives which can also account for spatial (Cai et al., 2010;

Original: A more advanced method, namely STMVL (Yi et al., 2016), incorporates spatial and temporal information by combining temporalinterpolation and spatial imputation methods to ﬁll missing values in geographically tagged time series.
Revised: Rao et al., 2015) and temporal (Yu et al., 2016; Mei et al., 2017) information. Among linear methods, STMVL (Yi et al., 2016) combines temporal and spatial interpolation to ﬁll missing values in geographically tagged time series.

Original: More recently, several deep learning approaches have been proposed for MTSI.
Revised: More recently, several deep learning approaches have been proposed for MTSI.

Original: Among the others, deep autoregressive methods based on recurrent neural networks (RNNs) found widespread success (Lipton et al., 2016;
Revised: Among the others, deep autoregressive methods based on recurrent neural networks (RNNs) found widespread success (Lipton et al., 2016;

Original: Che et al., 2018; Luo et al., 2018; Yoon et al., 2018b; Cao et al., 2018).
Revised: Che et al., 2018; Luo et al., 2018; Yoon et al., 2018b; Cao et al., 2018).

Original: GRU-D (Che et al., 2018) learns how to process sequences with missing data by controlling the decay of the hidden states of a gated RNN.
Revised: GRU-D (Che et al., 2018) learns how to process sequences with missing data by controlling the decay of the hidden states of a gated RNN.

Original: Cao et al.
Revised: Cao et al.

Original: (2018) propose BRITS , a bidirectional
Revised: (2018) propose BRITS , a bidirectional

Original: GRUD-D-like RNN for multivariate time series imputation that takes into account correlation among different channels to perform spatial imputation.
Revised: GRU-D-like RNN for multivariate time series imputation that takes into account correlation among different channels to perform spatial imputation.

Original: Other successful strategies in the literature have been proposed that exploit the adversarial training framework to generate realistic reconstructed sequences (Yoon et al., 2018a; Fedus et al., 2018; Luo et al., 2018; 2019).
Revised: Other successful strategies in the literature have been proposed that exploit the adversarial training framework to generate realistic reconstructed sequences (Yoon et al., 2018a; Fedus et al., 2018; Luo et al., 2018; 2019).

Original: Notably, GAIN (Yoon et al., 2018a) uses GANs (Goodfellow et al., 2014) to learn models that perform imputation in the i.i.d.
Revised: Notably, GAIN (Yoon et al., 2018a) uses GANs (Goodfellow et al., 2014) to learn models that perform imputation in the i.i.d.

Original: settings.
Revised: settings.

Original: Luo et al.
Revised: Luo et al.

Original: (2018; 2019) aim, instead, at learning models that generate realistic synthetic sequences and exploit them to ﬁll missing values.
Revised: (2018; 2019) aim, instead, at learning models that generate realistic synthetic sequences and exploit them to ﬁll missing values.

Original: Miao et al.
Revised: Miao et al.

Original: (2021) use an approach similar to GAIN, but condition the generator on the predicted label for the target incomplete time series.
Revised: (2021) use an approach similar to GAIN, but condition the generator on the predicted label for the target incomplete time series.

Original: Concurrently to our work, Kuppannagari et al.
Revised: Concurrently to our work, Kuppannagari et al.

Original: (2021) developed a graph-based spatio-temporal denoising autoencoder for spatio-temporal data coming from smart grids with known topology.
Revised: (2021) developed a graph-based spatio-temporal denoising autoencoder for spatio-temporal data coming from smart grids with known topology.

Original: Liu et al.
Revised: Liu et al.

Original: (2019), instead, uses adversarial learning to train a multi-scale model that imputes highly sparse time series in a hierarchical fashion.
Revised: (2019), instead, uses adversarial learning to train a multi-scale model that imputes highly sparse time series in a hierarchical fashion.

Original: However, we argue that none of the above-cited methods can take full advantage of relational information and nonlinear spatio-temporal dependencies.
Revised: However, we argue that none of the above-cited methods can take full advantage of relational information and nonlinear spatio-temporal dependencies.

Original: Most importantly,the above methods do not fully exploit the ﬂexibility and expressiveness enabled by operating in the context of graph processing.
Revised: Most importantly,

Original: E X T R A
Revised: E X T R A

Original: C T
Revised: C T

Original: G R A P H               3 00 0 0 0 00 0 1 [-
-]
Revised: G R A P H               3 00 0 0 0 00 0 1 [-
-]1 the above methods do not fully exploit the ﬂexibility and expressiveness enabled by operating in the context of graph processing.

Original: Graph neural networks for TSA Graph neural networks have been exploited in TSA mostly to implement spatio-temporal forecasting methods.
Revised: Graph neural networks for TSA Graph neural networks have been exploited in TSA mostly in spatio-temporal forecasting methods.

Original: Thebasic idea behind most of the methods present in the literature is to modify standard neural network architectures for sequential data by relying on operators that work in the graph domain.
Revised: The idea behind most of the methods present in the literature is to modify standard neural network architectures for sequential data by relying on operators that work in the graph domain.

Original: For example, Seo et al.
Revised: For example, Seo et al.

Original: (2018) propose agated recurrent graph neural network where the standard gates used by a GRU cell (Chung et al., 2014) are implemented by spectral GNNs (Defferrard et al., 2016); Li et al.
Revised: (2018) propose a GRU cell where gates are implemented by spectral GNNs (Defferrard et al., 2016); Li et al.

Original: (2018) propose an analogous architecture replacing spectral GNNs with a diffusion-convolutional network (Atwood & Towsley, 2016).
Revised: (2018) propose an analogous architecture replacing spectral GNNs with a diffusion-convolutional network (Atwood & Towsley, 2016).

Revised: Note that these models are different w.r.t.

Revised: approaches that use recurrent networks to propagate information graph-wise (Scarselli et al., 2008; Li et al., 2016).

Original: Yu et al.
Revised: Yu et al.

Original: (2017) and Wu et al.
Revised: (2017) and Wu et al.

Original: (2019; 2020) propose, instead, spatio-temporal convolutional neural networks that alternate convolutions on temporal and spatial dimensions.
Revised: (2019; 2020b) propose, instead, spatio-temporal convolutional neural networks that alternate convolutions on temporal and spatial dimensions.

Original: Similar approaches have also been studied in the context of attention-basedsequence models (Vaswani et al., 2017) with  Transformer-like architecturesalternating spatial and temporal attention layers (Zhang et al., 2018;
Revised: Similar approaches have also been studied in the context of attention-based models (Vaswani et al., 2017) with spatio-temporal Transformer-like architectures (Zhang et al., 2018;

Original: Cai et al., 2020).
Revised: Cai et al., 2020).

Original: Another particularly interesting line of research is related to the problem of learning, end-to-end, thebest graph structure to process an input multivariate time series (Kipf et al., 2018; Wu et al., 2020; Shang et al., 2020).
Revised: Another particularly interesting line of research is related to the problem of learning the graph structure underlying an input multivariate time series (Kipf et al., 2018; Wu et al., 2020b; Shang et al., 2020).

Original: Whilethe previously mentioned approaches focus on multivariate time series prediction and use the graph structure only for representation learning, other methods aim at predicting changes inthe graph topology (Zambon et al., 2019; Paassen et al., 2020).
Revised: While previously mentioned approaches focus on multivariate time series prediction, other methods aim at predicting changes in graph topology (Zambon et al., 2019; Paassen et al., 2020).

Original: Finally, methods such as Temporal Graph Networks (Rossi et al., 2020) are tailored to learn node embeddings in dynamical graphs and adopt an event-based paradigm that is out of the scope of this work.
Revised: Conversely, methods such as Temporal Graph Networks (Rossi et al., 2020) are tailored to learn node embeddings in dynamical graphs.

Original: Finally, recent works have proposed GNNs for imputing missing features in the context of i.i.d. data.
Revised: Finally, recent works have proposed GNNs for imputing missing features in the context of i.i.d. data.

Original: Among the others, Spinelli et al.
Revised: Among the others, Spinelli et al.

Original: (2020) propose an adversarial framework to train GNNs on the data reconstruction task, while You et al.
Revised: (2020) propose an adversarial framework to train GNNs on the data reconstruction task, while You et al.

Original: (2020) propose a bipartite graph representation for feature imputation.
Revised: (2020) propose a bipartite graph representation for feature imputation.

Revised: Lately, GNNs have also been exploited for spatial interpolation (Appleby et al., 2020; Wu et al., 2020a) – sometimes referred to as kriging (Stein, 1999).

Original: To the best of our knowledge, no previous GNN-based method targeted missing value imputation in the context of generic multivariate time series.
Revised: To the best of our knowledge, no previous GNN-based method targeted missing value imputation for generic multivariate time series.

Original:  P RELIMINARIES
Revised:  P RELIMINARIES

Original: Sequences of graphs We consider sequences of weighted directed graphs, where we observe a graph G t with N t nodes at each time step t .
Revised: Sequences of graphs We consider sequences of weighted directed graphs, where we observe a graph G t with N t nodes at each time step t .

Original: A graph is a couple G t = (cid:104)
Revised: A graph is a couple G t = (cid:104)

Original: X t , W t (cid:105) , where X t ∈ R N t × d is the node-attribute matrix whose i -th row contains the d -dimensional node-attribute vector x it ∈ R d associated with the i -th node; entry w i,jt of the adjacency matrix W t ∈
Revised: X t , W t (cid:105) , where X t ∈ R N t × d is the node-attribute matrix whose i -th row contains the d -dimensional node-attribute vector x it ∈ R d associated with the i -th node; entry w i,jt of the adjacency matrix W t ∈

Original: R N t × N t denotes the scalar weight of the edge (if any) connecting the i -th and j -th node.
Revised: R N t × N t denotes the scalar weight of the edge (if any) connecting the i -th and j -th node.

Original: Fig.
Revised: Fig.

Original: 1 exempliﬁes this modelling framework.
Revised: 1 exempliﬁes this modelling framework.

Original: We assume nodes to be identiﬁed, i.e., to have a unique ID that enables time-wise consistent processing.
Revised: We assume nodes to be identiﬁed, i.e., to have a unique ID that enables time-wise consistent processing.

Original: This problem setting can be easily extended to more general classes of graphs with attributed edges and global attributes.
Revised: This problem setting can be easily extended to more general classes of graphs with attributed edges and global attributes.

Original: In this work, we mainly focus on problems where the topology of the graph is ﬁxed and does not change over time, i.e., at each time step W t = W and
Revised: In this work, we mainly focus on problems where the topology of the graph is ﬁxed and does not change over time, i.e., at each time step W t = W and

Original: N t = N .
Revised: N t = N .

Original: Any generic multivariate time series ﬁts the above framework by letting each channel of the sequence (i.e., each sensor) correspond to a node and using the available relation information to build an adjacency matrix.
Revised: Any generic multivariate time series ﬁts the above framework by letting each channel of the sequence (i.e., each sensor) correspond to a node and using the available relation information to build an adjacency matrix.

Original: If no relational information is available, one could use the identity matrix, but this would defeat the purpose of the formulation.
Revised: If no relational information is available, one could use the identity matrix, but this would defeat the purpose of the formulation.

Original: A more proper choice of W t can be made using any standard similarity score (e.g., Pearson correlation) or a (thresholded) kernel.
Revised: A more proper choice of W t can be made using any standard similarity score (e.g., Pearson correlation) or a (thresholded) kernel.

Original: A more advanced approach instead could aim at learning an adjacency directly from data by using, for instance, spatial attention scores or resorting to graph learning techniques, e.g., Kipf et al.
Revised: A more advanced approach instead could aim at learning an adjacency directly from data by using, for instance, spatial attention scores or resorting to graph learning techniques, e.g., Kipf et al.

Original: From now on we assume that input multivariate time series have homogeneous channels, i.e., sensors are of the same type.
Revised: From now on we assume that input multivariate time series have homogeneous channels, i.e., sensors are of the same type.

Original: Note that this assumption does not imply a loss in generality: it is always possible to standardize node features by adding sensor type attributes and additional dimensions to accommodate the different types of sensor readings.
Revised: Note that this assumption does not imply a loss in generality: it is always possible to standard- ize node features by adding sensor type attributes and additional dimensions to accommodate the different types of sensor readings.

Original: Alternatively, one might directly model the problem by exploiting heterogeneous graphs (Schlichtkrull et al., 2018).
Revised: Alternatively, one might directly model the problem by exploiting heterogeneous graphs (Schlichtkrull et al., 2018).

Original: Multivariate time series imputation To model the presence of missing values, we consider, at each step, a binary mask M t ∈ { 0 , 1 } N t × d where each row
Revised: Multivariate time series imputation To model the presence of missing values, we consider, at each step, a binary mask M t ∈ { 0 , 1 } N t × d where each row

Original: m it indicates which of the corresponding node attributes of x it are available in X t .
Revised: m it indicates which of the corresponding node attributes of x it are available in X t .

Original: It follows that, m i,jt = 0 implies x i,jt to be missing; conversely, if m i,jt = 1 , then x i,jt stores the actual sensor reading.
Revised: It follows that, m i,jt = 0 implies x i,jt to be missing; conversely, if m i,jt = 1 , then x i,jt stores the actual sensor reading.

Original: We denote by (cid:102)
Revised: We denote by (cid:102)

Original: X t the unknown ground truth node-attribute matrix, i.e., the complete node-attribute matrix without any missing data.
Revised: X t the unknown ground truth node-attribute matrix, i.e., the complete node-attribute matrix without any missing data.

Original: We assume stationarity of missing data distribution and, in experiments, we mostly focus on the missing at random (MAR) scenario
Revised: We assume stationarity of missing data distribution and, in experiments, we mostly focus on the missing at random (MAR) scenario

Original: (Rubin, 1976).
Revised: (Rubin, 1976).

Original: We neither make assumptions on the number of concurrent sensor failures, nor on the length of missing data blocks, i.e., multiple failures extended over time are accounted for.
Revised: We neither make assumptions on the number of concurrent sensor failures, nor on the length of missing data blocks, i.e., multiple failures extended over time are accounted for.

Original: Clearly, one should expect imputation performance to scale with the number of concurrent faults and the time length of missing data bursts.
Revised: Clearly, one should expect imputation performance to scale with the number of concurrent faults and the time length of missing data bursts.

Original: The objective of MTSI is to impute missing values in a sequence of input data.
Revised: The objective of MTSI is to impute missing values in a sequence of input data.

Original: More formally, given a graph sequence G [ t,t + T ] of length T , we can deﬁne the missing data reconstruction error as
Revised: More formally, given a graph sequence G [ t,t + T ] of length T , we can deﬁne the missing data reconstruction error as

Original: [Equation]
Revised: [Equation]

Original: where ˆ x ih is the reconstructed ˜ x ih ; M [ t,t + T ] and m ih are respectively the logical binary complement of M [ t,t + T ] and m ih , (cid:96) ( · , · ) is an element-wise error function (e.g., absolute or squared error) and (cid:104) · , · (cid:105) indicates the standard dot product.
Revised: where ˆ x ih is the reconstructed ˜ x ih ; M [ t,t + T ] and m ih are respectively the logical binary complement of M [ t,t + T ] and m ih , (cid:96) ( · , · ) is an element-wise error function (e.g., absolute or squared error) and (cid:104) · , · (cid:105) indicates the standard dot product.

Original: Note that, in practice, it is impossible to have access to (cid:102) X
Revised: Note that, in practice, it is impossible to have access to (cid:102) X

Original: [ t,t + T ] and, as a consequence, it is necessary to deﬁne a surrogate optimization objective by, for example, using a forecasting loss or generating synthetic missing values.
Revised: [ t,t + T ] and, as a consequence, it is necessary to deﬁne a surrogate optimization objective by, for example, using a forecasting loss or generating synthetic missing values.

Original: In the context of trainable, parametric, imputation methods we consider two different operational settings.
Revised: In the context of trainable, parametric, imputation methods we consider two different operational settings.

Original: In the ﬁrst one, named in-sample imputation , the model is trained to reconstruct missing values in a given ﬁxed input sequence X [ t,t + T ] , i.e., the model is trained on all the available data except those that are missing and those that have been removed from the sequence to emulate additional failures for evaluation.
Revised: In the ﬁrst one, named in-sample imputation , the model is trained to reconstruct missing values in a given ﬁxed input sequence X [ t,t + T ] , i.e., the model is trained on all the available data except those that are missing and those that have been removed from the sequence to emulate additional failures for evaluation.

Original: Differently, in the second one (referred to as out-of-sample imputation ), the model is trained and evaluated on disjoint sequences.
Revised: Differently, in the second one (referred to as out-of-sample imputation ), the model is trained and evaluated on disjoint sequences.

Original: Note that in both cases the model does not have access to the ground-truth data used for the ﬁnal evaluation.
Revised: Note that in both cases the model does not have access to the ground-truth data used for the ﬁnal evaluation.

Original: The ﬁrst operational setting simulates the case where a practitioner ﬁts the model directly on the sequence to ﬁll up its gaps.
Revised: The ﬁrst operational setting simulates the case where a practitioner ﬁts the model directly on the sequence to ﬁll up its gaps.

Original: The second, instead, simulates the case where one wishes to use a model ﬁtted on a set of historical data to impute missing values in an unseen target sequence.
Revised: The second, instead, simulates the case where one wishes to use a model ﬁtted on a set of historical data to impute missing values in an unseen target sequence.

Original:  G RAPH R ECURRENT I MPUTATION N ETWORK
Revised:  G RAPH R ECURRENT I MPUTATION N ETWORK

Original: In this section, we present our approach, the Graph Recurrent Imputation Network ( GRIN ), a graphbased, recurrent neural architecture for MTSI.
Revised: In this section, we present our approach, the Graph Recurrent Imputation Network ( GRIN ), a graphbased, recurrent neural architecture for MTSI.

Original: Given a multivariate time series X [ t,t + T ] with mask M [ t,t + T ] , our objective is to reconstruct missing values in the input sequence by combining the information coming from both the temporal and spatial dimensions.
Revised: Given a multivariate time series X [ t,t + T ] with mask M [ t,t + T ] , our objective is to reconstruct missing values in the input sequence by combining the information coming from both the temporal and spatial dimensions.

Original: To do so, we design a novel bi-directional graph recurrent neural network which progressively processes the input sequence both forward and backward in time by performing two stages of imputation for each direction.
Revised: To do so, we design a novel bi-directional graph recurrent neural network which progressively processes the input sequence both forward and backward in time by performing two stages of imputation for each direction.

Original: Then, a feed-forward network takes as input the representation learned by the forward and backward models and performs a ﬁnal – reﬁned – imputation for each node of the graph and step of the sequence.
Revised: Then, a feed-forward network takes as input the representation learned by the forward and backward models and performs a ﬁnal – reﬁned – imputation for each node of the graph and step of the sequence.

Original: More precisely, the ﬁnal imputation depends on the output of two GRIN modules whose learned representations are ﬁnally processed (space and time wise) by a last decoding multilayer perceptron (MLP).
Revised: More precisely, the ﬁnal imputation depends on the output of two GRIN modules whose learned representations are ﬁnally processed (space and time wise) by a last decoding multilayer perceptron (MLP).

Original: An overview of the complete architecture is given in Fig.
Revised: An overview of the complete architecture is given in Fig.

Original: As shown in the ﬁgure, the two modules impute missing values iteratively, using at each time step previously imputed values as input.
Revised: As shown in the ﬁgure, the two modules impute missing values iteratively, using at each time step previously imputed values as input.

Original: We proceed by ﬁrst describing in detail the unidirectional model, and then we provide the bidirectional extension.
Revised: We proceed by ﬁrst describing in detail the unidirectional model, and then we provide the bidirectional extension.

Original: Unidirectional model Each GRIN module is composed of two blocks, a spatio-temporal encoder and a spatial decoder, which process the input sequence of graphs in two stages.
Revised: Unidirectional model Each GRIN module is composed of two blocks, a spatio-temporal encoder and a spatial decoder, which process the input sequence of graphs in two stages.

Original: The spatio-temporal encoder maps the input sequence X
Revised: The spatio-temporal encoder maps the input sequence X

Original: [ t,t + T ] in a spatio-temporal representation H [ t,t + T
Revised: [ t,t + T ] in a spatio-temporal representation H [ t,t + T

Original: ] ∈
Revised: ] ∈

Original: R N t × l byexploiting an ad-hoc designed recurrent GNN. The spatial decoder, instead, takes advantage of the learned representations to perform two consecutive rounds of imputation. A ﬁrst-stage imputation is obtained from the representation by using a linear readout; the second one exploits available relational, spatial, information at time step t .
Revised: R N t × l by

Original: In particular, the decoder is implemented by an MPNN

Original: [Figure]
Revised: [Figure]

Revised: an ad-hoc designed recurrent GNN obtained from the representation by using a linear readout available relational, spatial

Original: Spatio-temporal Encoder
Revised: Spatio-temporal Encoder

Original: In the encoder, the input sequence X [ t,t + T ] and mask M [ t,t + T ] are processed sequentially one step at a time, by means of a recurrent neural network with gates implemented by message-passing layers.
Revised: In the encoder, the input sequence X [ t,t + T ] and mask M [ t,t + T ] are processed sequentially one step at a time, by means of a recurrent neural network with gates implemented by message-passing layers.

Original: Any message-passing operator could be used in principle.
Revised: Any message-passing operator could be used in principle.

Original: In particular, given z it,k − 1 , i.e., the node features vector at layer k − 1 , we consider the general class of
Revised: In particular, given z it,k − 1 , i.e., the node features vector at layer k − 1 , we consider the general class of

Original: MPNNs described as
Revised: MPNNs described as

Original: [Equation]
Revised: [Equation]

Original: where N ( i ) is the set of neighbors of the i -th node in G t , γ k and ρ k are generic, differentiable, update and message functions (e.g., MLPs), and Σ is a permutation invariant, differentiable aggregation function (e.g., sum or mean ).
Revised: where N ( i ) is the set of neighbors of the i -th node in G t , γ k and ρ k are generic, differentiable, update and message functions (e.g., MLPs), and Σ is a permutation invariant, differentiable aggregation function (e.g., sum or mean ).

Original: In particular, function Σ speciﬁes how to aggregate messages received from neighbors.

Original: Note that several deﬁnitions of neighborhood are possible, e.g., one might consider nodes connected by paths up to a certain length l .
Revised: Note that several deﬁnitions of neighborhood are possible, e.g., one might consider nodes connected by paths up to a certain length l .

Original: For the sake of simplicity, from now on we indicate with MPNN ( z it , W t ) the forward pass of a generic K -layered message-passing neural network.
Revised: For the sake of simplicity, from now on we indicate with MPNN ( z it , W t ) the forward pass of a generic K -layered message-passing neural network.

Original: In the following, we use MPNNs as the building blocks for our spatio-temporal feature extractors.
Revised: In the following, we use MPNNs as the building blocks for our spatio-temporal feature extractors.

Original: To learn the dynamics of the system, we leverage on gated recurrent units (GRUs; Cho et al., 2014).
Revised: To learn the dynamics of the system, we leverage on gated recurrent units (GRUs; Cho et al., 2014).

Original: As previously mentioned, similarly to Seo et al.
Revised: As previously mentioned, similarly to Seo et al.

Original: (2018) and Li et al.
Revised: (2018) and Li et al.

Original: (2018), we implement the GRU gates by relying on the message-passing layers deﬁned above.
Revised: (2018), we implement the GRU gates by relying on the message-passing layers deﬁned above.

Original: At the node level, the elements of the message-passing GRU (MPGRU) can be described as:
Revised: At the node level, the elements of the message-passing GRU (MPGRU) can be described as:

Original: [Equation]
Revised: [Equation]

Original: where r i t , u i t are the reset and update gates, respectively, h i t is the hidden representation of the i -th node at time t , and ˆ x i (2) t is the output of the decoding block at the previous time-step (see next paragraph).
Revised: where r i t , u i t are the reset and update gates, respectively, h i t is the hidden representation of the i -th node at time t , and ˆ x i (2) t is the output of the decoding block at the previous time-step (see next paragraph).

Original: The symbols (cid:12) and || denote the Hadamard product and the concatenation operator, re- spectively.
Revised: The symbols (cid:12) and || denote the Hadamard product and the concatenation operator, re- spectively.

Original: The initial representation H t − 1 can either be initialized as a constant or with a learnable embedding.
Revised: The initial representation H t − 1 can either be initialized as a constant or with a learnable embedding.

Original: Note that for the steps where input data are missing, the encoder is fed with predictions from the decoder block, as explained in the next subsection.
Revised: Note that for the steps where input data are missing, the encoder is fed with predictions from the decoder block, as explained in the next subsection.

Original: By carrying out the above computation time and node wise, we get the encoded sequence H [ t,t + T ] .
Revised: By carrying out the above computation time and node wise, we get the encoded sequence H [ t,t + T ] .

Original: Spatial Decoder As a ﬁrst decoding step, we generate one-step-ahead predictions from the hidden representations of the MPGRU by means of a linear readout
Revised: Spatial Decoder As a ﬁrst decoding step, we generate one-step-ahead predictions from the hidden representations of the MPGRU by means of a linear readout

Original: [Equation]
Revised: [Equation]

Original: where V h ∈ R l × d is a learnable weight matrix and b h ∈ R d is a learnable bias vector.
Revised: where V h ∈ R l × d is a learnable weight matrix and b h ∈ R d is a learnable bias vector.

Original: We then deﬁne the ﬁller operator as
Revised: We then deﬁne the ﬁller operator as

Original: [Equation]
Revised: [Equation]

Original: intuitively, the ﬁller operator replaces the missing values in the input X t with the values at the same positions in Y t .
Revised: intuitively, the ﬁller operator replaces the missing values in the input X t with the values at the same positions in Y t .

Original: By feeding (cid:98) Y (1) t to the ﬁller operator, we get the ﬁrst-stage imputation (cid:99) X (1) t such that the output is X t with missing values replaced by the one-step-ahead predictions (cid:98) Y (1) t .
Revised: By feeding (cid:98) Y (1) t to the ﬁller operator, we get the ﬁrst-stage imputation (cid:99) X (1) t such that the output is X t with missing values replaced by the one-step-ahead predictions (cid:98) Y (1) t .

Original: The resulting node-level predictions are then concatenated to the mask M t and the hidden representation
Revised: The resulting node-level predictions are then concatenated to the mask M t and the hidden representation

Original: H t − 1 , and processed by a ﬁnal one-layer MPNN which computes for each node an imputation representation s it as
Revised: H t − 1 , and processed by a ﬁnal one-layer MPNN which computes for each node an imputation representation s it as

Original: [Equation]
Revised: [Equation]

Original: Notice that, as previously highlighted, the imputation representations only depend on messages received from neighboring nodes and the representation at the previous step.
Revised: Notice that, as previously highlighted, the imputation representations only depend on messages received from neighboring nodes and the representation at the previous step.

Original: In fact, by aggregating only messages from the one-hop neighborhood, the representations s it are independent of the input features x it of the i -th node itself.
Revised: In fact, by aggregating only messages from the one-hop neighborhood, the representations s it are independent of the input features x it of the i -th node itself.

Original: This constraint forces the model to learn how to reconstruct a target input by taking into account spatial dependencies.
Revised: This constraint forces the model to learn how to reconstruct a target input by taking into account spatial dependencies: this has a regularizing effect since the model is constrained to focus on local information.

Original: Afterward, we concatenate imputation representation S t with hidden representation H t − 1 , and generate second-stage imputations by using a second linear readout and applying the ﬁller operator:
Revised: Afterward, we concatenate imputation representation S t with hidden representation H t − 1 , and generate second-stage imputations by using a second linear readout and applying the ﬁller operator:

Original: [Equation]
Revised: [Equation]

Original: Finally, we feed (cid:99) X (2) t as input to the MPGRU (Eq. 3 – 6) to update the hidden representation and proceed to process the next input graph G t +1 .
Revised: Finally, we feed (cid:99) X (2) t as input to the MPGRU (Eq. 3 – 6) to update the hidden representation and proceed to process the next input graph G t +1 .

Original: Bidirectional Model Extending GRIN to account for both forward and backward dynamics is straightforward and can be achieved by duplicating the architecture described in the two previous paragraphs.
Revised: Bidirectional Model Extending GRIN to account for both forward and backward dynamics is straightforward and can be achieved by duplicating the architecture described in the two previous paragraphs.

Original: The ﬁrst module will process the sequence in the forward direction (from the beginning of the sequence towards its end), while the second one in the other way around.
Revised: The ﬁrst module will process the sequence in the forward direction (from the beginning of the sequence towards its end), while the second one in the other way around.

Original: The ﬁnal imputation is then obtained with an MLP aggregating representations extracted by the two modules:
Revised: The ﬁnal imputation is then obtained with an MLP aggregating representations extracted by the two modules:

Original: [Equation]
Revised: [Equation]

Original: where fwd and bwd denote the forward and backward modules, respectively.
Revised: where fwd and bwd denote the forward and backward modules, respectively.

Original: The ﬁnal output can then be easily obtained as (cid:99) X [ t,t + T ] =
Revised: The ﬁnal output can then be easily obtained as (cid:99) X [ t,t + T ] =

Original: Φ( (cid:98) Y [ t,t + T ] ) .
Revised: Φ( (cid:98) Y [ t,t + T ] ) .

Original: Note that by construction our model can exploit all the available relevant spatio-temporal information, since the only value explicitly masked out for each node is x it .
Revised: Note that by construction our model can exploit all the available relevant spatio-temporal information, since the only value explicitly masked out for each node is x it .

Original: At the same time, it is important to realize that our model does not merely reconstruct the input as an autoencoder, but it is speciﬁcally tailored for the imputation task due to its inductive biases.
Revised: At the same time, it is important to realize that our model does not merely reconstruct the input as an autoencoder, but it is speciﬁcally tailored for the imputation task due to its inductive biases.

Original: The model is trained by minimizing the reconstruction error of all imputation stages in both directions (see Appendix A).
Revised: The model is trained by minimizing the reconstruction error of all imputation stages in both directions (see Appendix A).

Original:  E MPIRICAL EVALUATION
Revised:  E MPIRICAL EVALUATION

Original: In this section, we empirically evaluate our approach against state-of-the-art baselines on four datasets coming from three relevant application domains.
Revised: In this section, we empirically evaluate our approach against state-of-the-art baselines on four datasets coming from three relevant application domains.

Original: Our approach, remarkably, achieve state- of-the-art performance on all of them.
Revised: Our approach, remarkably, achieve stateof-the-art performance on all of them.

Original: • Air Quality (AQI) : dataset of recordings of several air quality indices from 437 monitoring stations spread across 43 Chinese cities.
Revised: • Air Quality (AQI) : dataset of recordings of several air quality indices from 437 monitoring stations spread across 43 Chinese cities.

Original: We consider only the PM2.5 pollutant.
Revised: We consider only the PM2.5 pollutant.

Original: Prior works on imputation (Yi et al., 2016; Cao et al., 2018) consider a reduced version of this dataset,including only 36 sensors ( AQI-36 in the following).
Revised: Prior works on imputation (Yi et al., 2016; Cao et al., 2018) consider a reduced version of this dataset,including only 36 sensors ( AQI-36 in the following).

Original: We evaluate our model on both datasets.
Revised: We evaluate our model on both datasets.

Original: We use as adjacency matrix a thresholded Gaussian kernel (Shuman et al., 2013) computed from pairwise geographic distances.
Revised: We use as adjacency matrix a thresholded Gaussian kernel (Shuman et al., 2013) computed from pairwise geographic distances.

Original: [Table]
Revised: [Table]

Original: • Trafﬁc : we consider the PEMS-BAY and METR-LA datasets from Li et al.
Revised: • Trafﬁc : we consider the PEMS-BAY and METR-LA datasets from Li et al.

Original: (2018), containing data from trafﬁc sensors from the San Francisco Bay Area and the Los Angeles County Highway.
Revised: (2018), containing data from trafﬁc sensors from the San Francisco Bay Area and the Los Angeles County Highway.

Original: We use the same approach of Li et al.
Revised: We use the same approach of Li et al.

Original: (2018) and Wu et al.
Revised: (2018) and Wu et al.

Original: (2019) to obtain an adjacency matrix.
Revised: (2019) to obtain an adjacency matrix.

Original: • Smart grids : we consider data from the Irish Commission for Energy Regulation Smart Metering Project ( CER-E ; Commission for Energy Regulation, 2016).
Revised: • Smart grids : we consider data from the Irish Commission for Energy Regulation Smart Metering Project ( CER-E ; Commission for Energy Regulation, 2016).

Original: We select only the subset of the available smart meters monitoring the energy consumption of small and medium-sized enterprises (SMEs), i
Revised: We select only the subset of the available smart meters monitoring the energy consumption of small and medium-sized enterprises (SMEs), i

Original: e., 485 time series with samples acquired everyminutes.
Revised: e., 485 time series with samples acquired everyminutes.

Original: We build an adjacency matrix by extracting a k-nearest neighbor graph (with k = 10 ) from the similarity matrix built by computing the correntropy (Liu et al., 2007) among the time series.
Revised: We build an adjacency matrix by extracting a k-nearest neighbor graph (with k = 10 ) from the similarity matrix built by computing the correntropy (Liu et al., 2007) among the time series.

Original: For the air quality datasets, we adopt the same evaluation protocol of previous works (Yi et al., 2016; Cao et al., 2018) and we show results for both the in-sample and out-of-sample settings.
Revised: For the air quality datasets, we adopt the same evaluation protocol of previous works (Yi et al., 2016; Cao et al., 2018) and we show results for both the in-sample and out-of-sample settings.

Original: For the trafﬁc and energy consumption datasets we consider only the out-of-sample scenario (except for MF which only works in-sample).
Revised: For the trafﬁc and energy consumption datasets we consider only the out-of-sample scenario (except for MF which only works in-sample).

Original: We simulate the presence of missing data by consideringdifferent settings: 1) Block missing , i.e, at each step, for each sensor, we randomly drop 5% of the available data and, in addition, we simulate a failure with probability p failure = 0 .
Revised: We simulate the presence of missing data by consideringdifferent settings: 1) Block missing , i.e, at each step, for each sensor, we randomly drop 5% of the available data and, in addition, we simulate a failure with probability p failure = 0 .

Original: 15% and sample its duration uniformly in the interval [ min steps, max steps ] , where min steps and max steps are the number of time steps corresponding respectively to 1 and 4 hours in the trafﬁc case andhours and 2 days for CER-E; 2) Point missing , i.e., we simply randomly mask out 25% of the available data.
Revised: 15% and sample its duration uniformly in the interval [ min steps, max steps ] , where min steps and max steps are the number of time steps corresponding respectively to 1 and 4 hours in the trafﬁc case andhours and 2 days for CER-E; 2) Point missing , i.e., we simply randomly mask out 25% of the available data.

Original: We split all the datasets into training/validation/test sets.
Revised: We split all the datasets into training/validation/test sets.

Original: We use as performance metrics the mean absolute error (MAE), mean squared error (MSE) and mean relative error (MRE; Cao et al., 2018) computed over the imputation window.
Revised: We use as performance metrics the mean absolute error (MAE), mean squared error (MSE) and mean relative error (MRE; Cao et al., 2018) computed over the imputation window.

Original: For all the experiments, we use as messagepassing operator the diffusion convolution introduced by Atwood & Towsley (2016).
Revised: For all the experiments, we use as messagepassing operator the diffusion convolution introduced by Atwood & Towsley (2016).

Original: We consider BRITS (Cao et al., 2018)
Revised: We consider BRITS (Cao et al., 2018)

Original: as the principal competing alternative among non-adversarial deep autoregressive approaches, as it shares architectural similarities with our methods.
Revised: as the principal competing alternative among non-adversarial deep autoregressive approaches, as it shares architectural similarities with our methods.

Original: As additional baselines we consider: 1) MEAN , i.e., imputation using the node-level average; 2) KNN , i.e., imputation by averaging values of the k = 10 neighboring nodes with the highest weight in the adjacency matrix W t ; 3) MICE (White et al., 2011), limiting the maximum number of iterations to 100 and the number of nearest features to 10 ; 4) Matrix Factorization ( MF ) with rank = 10 ; 5) VAR , i.e., a Vector
Revised: As additional baselines we consider: 1) MEAN , i.e., imputation using the node-level average; 2) KNN , i.e., imputation by averaging values of the k = 10 neighboring nodes with the highest weight in the adjacency matrix W t ; 3) MICE (White et al., 2011), limiting the maximum number of iterations to 100 and the number of nearest features to 10 ; 4) Matrix Factorization ( MF ) with rank = 10 ; 5) VAR , i.e., a Vector

Original: Autoregressive one-step-ahead predictor; 6) rGAIN , i.e., an unsupervised version of SSGAN (Miao et al., 2021) which can be seen as GAIN (Yoon et al., 2018a) with bi-directional recurrent encoder and decoder; 7) MPGRU , a one-step-ahead GNN-based predictor similar to DCRNN (Li et al., 2018).
Revised: Autoregressive one-step-ahead predictor; 6) rGAIN , i.e., an unsupervised version of SSGAN (Miao et al., 2021) which can be seen as GAIN (Yoon et al., 2018a) with bi-directional recurrent encoder and decoder; 7) MPGRU , a one-step-ahead GNN-based predictor similar to DCRNN (Li et al., 2018).

Original: We provide further comment and in depth details on baselines and datasets, together with additional experiments on synthetic data in the appendix.
Revised: We provide further comment and in depth details on baselines and datasets, together with additional experiments on synthetic data in the appendix.

Original: [Table]
Revised: [Table]

Original: 5. R ESULTS
Revised: 5. R ESULTS

Original: Empirical results show that GRIN can achieve large improvements in imputation performance on several scenarios, as well as increased ﬂexibility.
Revised: Empirical results show that GRIN can achieve large improvements in imputation performance on several scenarios, as well as increased ﬂexibility.

Original: In fact, differently from the other state-of-theart baselines
Revised: In fact, differently from the other state-of-theart baselines

Original: [Table]
Revised: [Table]

Original: GRIN can handle input with a variable number of dimensions.
Revised: GRIN can handle input with a variable number of dimensions.

Original: Tab. 1 shows the experimental results on the air quality datasets.
Revised: Tab. 1 shows the experimental results on the air quality datasets.

Original: In the in-sample settings, we compute metrics using as imputation the value obtained by averaging predictions over all the overlapping windows; in the out-of-sample settings, instead, we simply report results by averaging the error over windows.
Revised: In the in-sample settings, we compute metrics using as imputation the value obtained by averaging predictions over all the overlapping windows; in the out-of-sample settings, instead, we simply report results by averaging the error over windows.

Original: GRIN largely outperforms other baselines on bothin-sample and out-of-sample settings.
Revised: GRIN largely outperforms other baselines on both settings.

Original: In particular, in the latter case, GRIN decreases MAE w.r.t.
Revised: In particular, in the latter case, GRIN decreases MAE w.r.t.

Original: the closest baseline by more than 20% in AQI.
Revised: the closest baseline by more than 20% in AQI.

Original: Interestingly, GRIN consistently outperforms BRITS in imputing missing values also for sensors corresponding to isolated (disconnected) nodes, i.e., nodes corresponding to stations more thankm away from any other station (see B.1): this is empirical evidence of the positive regularization effect of the inductive biases encoded into GRIN .
Revised: Interestingly, GRIN consistently outperforms BRITS in imputing missing values also for sensors corresponding to isolated (disconnected) nodes, i.e., nodes corresponding to stations more than 40 km away from any other station (see B.1): this is empirical evidence of the positive regularizations encoded into GRIN .

Original: Our method achieves more accurate imputation also in the 36 -dimensional dataset, where we could expect the graph representation to have a lower impact.
Revised: Our method achieves more accurate imputation also in the 36 -dimensional dataset, where we could expect the graph representation to have a lower impact.

Original: Results for the trafﬁc and smart grids datasets are shown in Tab. 2.
Revised: Results for the trafﬁc and smart grids datasets are shown in Tab. 2.

Original: In the trafﬁc dataset, our method outperforms  BRITS (which is consistently the most competitive baseline) by a wide margin in all the considered settings while using a much lower number of parameters (see A).
Revised: In the trafﬁc dataset, our method outperforms both BRITS and rGAIN by a wide margin in all the considered settings while using a much lower number of parameters (see A).

Original: In the trafﬁc datasets, on average, GRIN reduces MAE by ≈ 29% w.r.t.
Revised: In the trafﬁc datasets, on average, GRIN reduces MAE by ≈ 29% w.r.t.

Original: BRITS and, in particular, in the Point missing setting of the PEMS-BAY dataset, the error ismore than halved.
Revised: BRITS and, in particular, in the Point missing setting of the PEMS-BAY dataset, the error is halved.

Original: In CER-E, GRIN consistently achieves the best performance, clearly outperforming other baselines.
Revised: In CER-E, GRIN consistently outperforms other baselines.

Original: Besides showing the effectiveness of our approach in a relevant application ﬁeld, this experiment also goes to show that GRIN can be exploited in settings where relational information is not readily available.
Revised: Besides show- ing the effectiveness of our approach in a relevant application ﬁeld, this experiment also goes to show that GRIN can be exploited in settings where relational information is not readily available.

Original: [Table]
Revised: [Table]

Original: Finally, Tab. 3 show results – in terms of MAE – of an ablation study on the out-of-sample scenario in AQI, METR-LA (in the Block Missing settings), and CER-E (Point Missing setting).
Revised: Finally, Tab. 3 show results – in terms of MAE – of an ablation study on the out-of-sample scenario in AQI, METR-LA (in the Block Missing settings), and CER-E (Point Missing setting).

Original: In particular, we compare GRIN against 3 baselines to assess the impact of the spatial decoder and of the bidirectional architecture.
Revised: In particular, we compare GRIN against 3 baselines to assess the impact of the spatial decoder and of the bidirectional architecture.

Original: The ﬁrst baseline is basically a bi-directional MPGRU where values are imputed by a ﬁnal MLP taking as inputs h fwdt − 1 and h bwdt +1 , while the second one has an analogous architecture, but uses hidden representation and time step t (for both directions) and, thus, behaves similarly to a denoising autoencoder.
Revised: The ﬁrst baseline is essentially a bi-directional MPGRU where values are imputed by a ﬁnal MLP taking as inputs h fwdt − 1 and h bwdt +1 , while the second one has an analogous architecture, but uses hidden representation and time step t (for both directions) and, thus, behaves similarly to a denoising autoencoder.

Original: As third baseline, we reportfor reference the results of the unidirectional MPGRU.
Revised: As reference, we report the results of the unidirectional MPGRU.

Original: Results show that the components we introduce do contribute to signiﬁcantly reduce imputation error.
Revised: Results show that the components we introduce do contribute to signiﬁcantly reduce imputation error.

Original: In particular, it is evident that spatial decoding and the bidirectional architecture are important to obtain accurate missing data reconstruction, especially in realistic settings with blocks of missing data.
Revised: It is clear that spatial decoding and the bidirectional architecture are important to obtain accurate missing data reconstruction, especially in realistic settings with blocks of missing data.

Original: Interestingly, the denoising model suffers in scenarios with blocks of missing data, while, as one might expect, works well in the Point Missing setting.
Revised: Interestingly, the denoising model suffers in the Block missing scenario, while, as one might expect, works well in the Point Missing setting.

Original: For an extension of the ablation study on more datasets, see C.2.
Revised: For additional results and discussion about scalability issues, we refer to the appendix of the paper.

Original: 5. V IRTUAL SENSING
Revised: 5. V IRTUAL SENSING

Original: [Figure]
Revised: [Figure]

Original: no
Revised: no

Revised: [Figure]

Revised: no

Original: [Figure]
Revised: [Figure]

Original: As a ﬁnal experiment, we provide a quantitative and qualitative assessment of the proposed method in virtual sensing.
Revised: As a ﬁnal experiment, we provide a quantitative and qualitative assessment of the proposed method in virtual sensing.

Original: The idea  is to simulate the presence of a sensor by adding a node with no available data and, then, let the model reconstruct the corresponding time series.
Revised: The idea (often studied in the context of kriging – see Section 2) is to simulate the presence of a sensor by adding a node with no available data and, then, let the model reconstruct the corresponding time series.

Original: Note that for the approach to work several assumptions are needed: 1) we have to assume that the physical quantity being monitored can be reconstructed from observations at neighboring sensors; 2) we should assume a high-degree of homogeneity of sensors (e.g., in the case of air quality stations we should assume that sensors are placed at the same height) or that the features characterizing each neighboring sensor (e.g., placement) are available to the model.
Revised: Note that for the approach to work several assumptions are needed: 1) we have to assume that the physical quantity being monitored can be reconstructed from observations at neighboring sensors; 2) we should assume a high-degree of homogeneity of sensors (e.g., in the case of air quality stations we should assume that sensors are placed at the same height) or that the features characterizing each neighboring sensor (e.g., placement) are available to the model.

Original: In this context, it is worth noting that, due to the inductive biases embedded in the model, GRIN performs reconstruction not only by minimizing reconstruction error at the single node, but by regularizing the reconstructed value for imputations at neighboring sensors.
Revised: In this context, it is worth noting that, due to the inductive biases embedded in the model, GRIN performs reconstruction not only by minimizing reconstruction error at the single node, but by regularizing the reconstructed value for imputation at neighboring sensors.

Original: We masked out observed values of the two nodes of AQI- 36 with highest (station no. 1014 ) and lowest (no. 1031 ) connectivity, and trained GRIN on the remaining part of the data as usual.
Revised: We masked out observed values of the two nodes of AQI- 36 with highest (station no. 1014 ) and lowest (no. 1031 ) connectivity, and trained GRIN on the remaining part of the data as usual.

Original: Results, in Fig.
Revised: Results, in Fig.

Original: 3, qualitatively show that GRIN can infer the trend and scale for unseen sensors.
Revised: 3, qualitatively show that GRIN can infer the trend and scale for unseen sensors.

Original: In terms of MAE, GRIN scored 11 .
Revised: In terms of MAE, GRIN scored 11 .

Original: 74 for sensor 1014 and 20 . 

Revised: 74 for sensor 1014 and 20 .

Original: for sensor 1031 (averages over 5 independent runs).
Revised: for sensor 1031 (averages over 5 independent runs).

Original:  C ONCLUSIONS
Revised:  C ONCLUSIONS

Original: We presented GRIN , a novel approach for MTSI exploiting modern graph neural networks.
Revised: We presented GRIN , a novel approach for MTSI exploiting modern graph neural networks.

Original: Our method imputes missing data by leveraging the relational information characterizing the underlying network of sensors and the functional dependencies among them.
Revised: Our method imputes missing data by leveraging the relational information characterizing the underlying network of sensors and the functional dependencies among them.

Original: Compared against state-of-the-art baselines, our framework offers higher ﬂexibility and achieves better reconstruction accuracy on all the considered scenarios.
Revised: Compared against state-of-the-art baselines, our framework offers higher ﬂexibility and achieves better reconstruction accuracy on all the considered scenarios.

Original: There are several possible directions for future works.
Revised: There are several possible directions for future works.

Original: From a theoretical perspective, it would be interesting to study the properties that would guarantee an accurate reconstruction.
Revised: From a theoretical perspective, it would be interesting to study the properties that would guarantee an accurate reconstruction.

Original: Furthermore, future work should study extensions able to deal with a non-stationary setting and further assess applications of GRIN in virtual and active sensing.
Revised: Furthermore, future work should study extensions able to deal with a non-stationary setting and further assess applications of GRIN in virtual and active sensing.

Original: R EPRODUCIBILITY S TATEMENT
Revised: R EPRODUCIBILITY S TATEMENT

Original: Code to reproduce experiments presented in the paper is provided as supplementary material together with conﬁguration ﬁles to replicate reported results.
Revised: Code to reproduce experiments presented in the paper is provided as supplementary material together with conﬁguration ﬁles to replicate reported results.

Original: All datasets, except CER-E, are open and downloading links are provided in the supplementary material.
Revised: All datasets, except CER-E, are open and downloading links are provided in the supplementary material.

Original: The CER-E dataset can be obtained free of charge for research purposes (see appendix).
Revised: The CER-E dataset can be obtained free of charge for research purposes (see appendix).

Original: For experiments where failures are emulated, we use random number generators with ﬁxed seed for missing data generation to ensure reproducibility and consistency among experiments and baselines
Revised: For experiments where failures are emulated, we use random number generators with ﬁxed seed for missing data generation to ensure reproducibility and consistency among experiments and baselines.

Original: A PPENDIX A D ETAILED EXPERIMENTAL SETTINGS
Revised: A PPENDIX A D ETAILED EXPERIMENTAL SETTINGS

Original: In this appendix, we give more details on the experimental settings used to evaluate our approach.
Revised: In this appendix, we give more details on the experimental settings used to evaluate our approach.

Original: We train all the models by sampling at random 160 batches of 32 elements for each epoch, we ﬁx the maximum number of epochs to 300 and we use early stopping on the validation set with a patience of 40 epochs.
Revised: We train all the models by sampling at random 160 batches of 32 elements for each epoch, we ﬁx the maximum number of epochs to 300 and we use early stopping on the validation set with a patience of 40 epochs.

Original: All methods are trained using a cosine learning rate scheduler with initial value of 0 .
Revised: All methods are trained using a cosine learning rate scheduler with initial value of 0 .

Original: 001 , decayed over the 300 training epochs.
Revised: 001 , decayed over the 300 training epochs.

Original: During training, we randomly mask out an additional 5% of the input data for each batch to foster robustness to noise and missing data.
Revised: During training, we randomly mask out an additional 5% of the input data for each batch to foster robustness to noise and missing data.

Original: For GRIN we minimize the following loss function is
Revised: For GRIN we minimize the following loss function is

Original: [Equation]
Revised: [Equation]

Original: where each L ( · , · , · ) is of the form of Eq.
Revised: where each L ( · , · , · ) is of the form of Eq.

Original: 1 and the element-wise error function is MAE.
Revised: 1 and the element-wise error function is MAE.

Original: Note that here we are using X [ t,t + T ] and M [ t,t + T ] instead of (cid:99) X
Revised: Note that here we are using X [ t,t + T ] and M [ t,t + T ] instead of (cid:99) X

Original: [ t,t + T ] and M [ t,t + T ] .
Revised: [ t,t + T ] and M [ t,t + T ] .

Original: For BRITS, we use the same network hyperparameters of Cao et al.
Revised: For BRITS, we use the same network hyperparameters of Cao et al.

Original: (2018) for the AQI-36 dataset.
Revised: (2018) for the AQI-36 dataset.

Original: To account for the larger input dimension, for the other datasets we increase the number of hidden
Revised: To account for the larger input dimension, for the other datasets we increase the number of hidden

Original: The number of neurons was tuned on the validation sets.
Revised: The number of neurons was tuned on the validation sets.

Original: For rGAIN we use the same number of units in the cells of the bi-directional RNN used by BRITS, but we concatenate a random vector (sampled from a uniform distribution) of dimension z
Revised: For rGAIN we use the same number of units in the cells of the bi-directional RNN used by BRITS, but we concatenate a random vector (sampled from a uniform distribution) of dimension z

Original: = 4 to the input vector in order to model the sampling of the data generating process.
Revised: = 4 to the input vector in order to model the sampling of the data generating process.

Original: To obtain predictions, we average out the outputs of k = 5 forward passes.
Revised: To obtain predictions, we average out the outputs of k = 5 forward passes.

Original: For VAR we used an order of 5 and trained the model with SGD.
Revised: For VAR we used an order of 5 and trained the model with SGD.

Original: Since the VAR model needs past 5 observations to predict the next step, we pad each sequence using the mean for each channel.
Revised: Since the VAR model needs past 5 observations to predict the next step, we pad each sequence using the mean for each channel.

Original: Here we used a batch size to 64 and and a learning rate of 0 . 0005 .
Revised: Here we used a batch size to 64 and and a learning rate of 0 . 0005 .

Original: The order was selected with a small search in the range [2 , 12] : we found out a window size of 5 to be ideal for all the considered datasets.
Revised: The order was selected with a small search in the range [2 , 12] : we found out a window size of 5 to be ideal for all the considered datasets.

Original: For GRIN we use the same hyperparameters in all the datasets: a hidden dimension of 64 neurons for both the spatio-temporal encoder and the spatial decoder and of 64 neurons for the
Revised: For GRIN we use the same hyperparameters in all the datasets: a hidden dimension of 64 neurons for both the spatio-temporal encoder and the spatial decoder and of 64 neurons for the

Original: MLP.
Revised: MLP.

Original: We use diffusion convolution as message-passing operation, with a diffusion step k = 2 in the spatio-temporal encoder and k = 1 in the temporal decoder.
Revised: We use diffusion convolution as message-passing operation, with a diffusion step k = 2 in the spatio-temporal encoder and k = 1 in the temporal decoder.

Original: Note that, due to the architectural differences, the other neural network baselines have a number of parameters that is far higher than GRIN (depending on the considered dataset, up to ≈ 4 M against ≈ 200 K ).
Revised: Note that, due to the architectural differences, the other neural network baselines have a number of parameters that is far higher than GRIN (depending on the considered dataset, up to ≈ 4 M against ≈ 200 K ).

Original: For MPGRU we use the same hyperparameters of GRIN ( 64 units for both the spatio-temporal encoder and the decoder).
Revised: For MPGRU we use the same hyperparameters of GRIN ( 64 units for both the spatio-temporal encoder and the decoder).

Original: For data processing we use the same steps of Li et al.
Revised: For data processing we use the same steps of Li et al.

Original: , data are normalized across the feature dimension (which means graph-wise for GRIN and node-wise for BRITS/rGAIN/VAR).
Revised: , data are normalized across the feature dimension (which means graph-wise for GRIN and node-wise for BRITS/rGAIN/VAR).

Original: Data masked out for evaluation are never used to train any model.
Revised: Data masked out for evaluation are never used to train any model.

Original: All the models were developed in Python (Van Rossum & Drake, 2009) using the following opensource libraries:
Revised: All the models were developed in Python (Van Rossum & Drake, 2009) using the following opensource libraries:

Original: • PyTorch (Paszke et al., 2019); • numpy (Harris et al., 2020); • Neptune 1 (neptune.ai, 2021); • scikit-learn (Pedregosa et al., 2011); • fancyimpute (Rubinsteyn & Feldman).
Revised: • PyTorch (Paszke et al., 2019); • numpy (Harris et al., 2020); • Neptune 1 (neptune.ai, 2021); • scikit-learn (Pedregosa et al., 2011); • fancyimpute (Rubinsteyn & Feldman).

Original: The implementation of the diffusion convolutional operator was adapted from the Graph-WaveNet codebase 2 .
Revised: The implementation of the diffusion convolutional operator was adapted from the Graph-WaveNet codebase 2 .

Original: For the implementation of BRITS, we used the code provided by the authors 3 .
Revised: For the implementation of BRITS, we used the code provided by the authors 3 .

Original: The code to reproduce the experiments of the paper is available as supplementary material.
Revised: The code to reproduce the experiments of the paper is available as supplementary material.

Original: G RAPH
Revised: G RAPH

Original: [Table]
Revised: [Table]

Original: B D ATASETS
Revised: B D ATASETS

Original: In this appendix
Revised: In this appendix

Original: [Table]
Revised: [Table]

Original: we provide more details on the datasets that we used to run experiments.
Revised: we provide more details on the datasets that we used to run experiments.

Original: Tab.shows detailed statistics for graph structure associated with each dataset
Revised: Tab.shows detailed statistics for graph structure associated with each dataset

Original: [Table]
Revised: [Table]

Original: while Fig.
Revised: while Fig.

Original: 4 shows the corresponding adjacency matrices.
Revised: 4 shows the corresponding adjacency matrices.

Original: Tab.
Revised: Tab.

Original: 5 shows missing data statistics.
Revised: 5 shows missing data statistics.

Original: In the following subsections, we go deeper into details for each dataset.
Revised: In the following subsections, we go deeper into details for each dataset.

Original: O RIGINAL DATA
Revised: O RIGINAL DATA

Original: I NJECTED FAULTS
Revised: I NJECTED FAULTS

Original: [Table]
Revised: [Table]

Original: B. A IR QUALITY
Revised: B. A IR QUALITY

Original: Air pollution is nowadays an ubiquitous problem.
Revised: Air pollution is nowadays an ubiquitous problem.

Original: The Urban Computing project (Zheng et al., 2014; 2015) published several datasets containing real measurements of different indices affecting human life in urban spaces.
Revised: The Urban Computing project (Zheng et al., 2014; 2015) published several datasets containing real measurements of different indices affecting human life in urban spaces.

Original: We consider as benchmark the dataset regarding the air quality index ( AQI ).
Revised: We consider as benchmark the dataset regarding the air quality index ( AQI ).

Original: The complete dataset contains hourly measurements of six pollutants from 437 air quality monitoring stations, spread over 43 cities in China, over a period of one year (from May 2014 to April 2015 ).
Revised: The complete dataset contains hourly measurements of six pollutants from 437 air quality monitoring stations, spread over 43 cities in China, over a period of one year (from May 2014 to April 2015 ).

Original: Prior works on imputation (Yi et al., 2016; Cao et al., 2018) considered a reduced version of this dataset, including only 36 sensors ( AQI-36 ).
Revised: Prior works on imputation (Yi et al., 2016; Cao et al., 2018) considered a reduced version of this dataset, including only 36 sensors ( AQI-36 ).

Original: This dataset is particularly interesting as a bench- mark for imputation due to the high rate of missing values ( 25 . 7% in AQI and 13 .
Revised: This dataset is particularly interesting as a bench- mark for imputation due to the high rate of missing values ( 25 . 7% in AQI and 13 .

Original: 2% in AQI-36).
Revised: 2% in AQI-36).

Original: Along with Yi et al.
Revised: Along with Yi et al.

Original: , we consider as the test set the months of March, June, September and
Revised: , we consider as the test set the months of March, June, September and

Original: December.
Revised: December.

Original: We consider both the in-sample and out-of-sample scenarios.
Revised: We consider both the in-sample and out-of-sample scenarios.

Original: In latter case, we do not consider windows overlapping with any of the test months.
Revised: In latter case, we do not consider windows overlapping with any of the test months.

Original: We use the same procedure of Yi et al.
Revised: We use the same procedure of Yi et al.

Original: (2016) to simulate the presence of missing data for evaluation.
Revised: (2016) to simulate the presence of missing data for evaluation.

Original: We select windows of data of length T = 24 for AQI and T = 36 for AQI-36 (in line with Cao et al.
Revised: We select windows of data of length T = 24 for AQI and T = 36 for AQI-36 (in line with Cao et al.

Original: To evaluate the imputation performances, we mask out from the test set and use as groundtruth the value x
Revised: To evaluate the imputation performances, we mask out from the test set and use as groundtruth the value x

Original: i,jt if: (1) the value is not missing ( m i,jt = 1 ) and (2) the value is missing at the same hour and day in the following month.
Revised: i,jt if: (1) the value is not missing ( m i,jt = 1 ) and (2) the value is missing at the same hour and day in the following month.

Original: Besides air quality readings, the dataset provides geographic coordinates of each monitoring station.
Revised: Besides air quality readings, the dataset provides geographic coordinates of each monitoring station.

Original: To obtain an adjacency matrix from the geographic distances between nodes, we use a thresholded Gaussian kernel (Shuman et al., 2013): the weight w i,jt = w i,j of the edge connecting i -th and j -th node is[Equation] otherwise
Revised: To obtain an adjacency matrix from the geographic distances between nodes, we use a thresholded Gaussian kernel (Shuman et al., 2013): the weight w i,jt = w i,j of the edge connecting i -th and j -th node is

Original: [Equation]
Revised: [Equation]

Original: where dist ( · , · ) is the geographical distance operator, γ controls the width of the kernel and δ is the threshold.
Revised: where dist ( · , · ) is the geographical distance operator, γ controls the width of the kernel and δ is the threshold.

Original: We set γ to the standard deviation of geographical distances in AQI-36 in both datasets.
Revised: We set γ to the standard deviation of geographical distances in AQI-36 in both datasets.

Original: We set δ so that it corresponds to a distance of ≈ 40 km.
Revised: We set δ so that it corresponds to a distance of ≈ 40 km.

Original: B. T RAFFIC
Revised: B. T RAFFIC

Original: The study of trafﬁc networks is key for the development of intelligent transportation systems and a relevant application ﬁeld for network science.
Revised: The study of trafﬁc networks is key for the development of intelligent transportation systems and a relevant application ﬁeld for network science.

Original: While previous works (Yu et al., 2017; Li et al., 2018; Wu et al., 2019; Shang et al., 2020) have assessed spatio-temporal deep learning methods for the trafﬁc forecasting task, we focus on reconstruction.
Revised: While previous works (Yu et al., 2017; Li et al., 2018; Wu et al., 2019; Shang et al., 2020) have assessed spatio-temporal deep learning methods for the trafﬁc forecasting task, we focus on reconstruction.

Original: We use as benchmark the PEMS-BAY and
Revised: We use as benchmark the PEMS-BAY and

Original: METR-LA datasets from Li et al.
Revised: METR-LA datasets from Li et al.

Original: PEMS-BAY contains 6 months of data from 325 trafﬁc sensors in San Francisco Bay Area, while METR-LA contains 4 months of sensor readings from 207 detectors in the Los Angeles County Highway (Jagadish et al., 2014); for both datasets, the sampling rate corresponds to 5 minutes.
Revised: PEMS-BAY contains 6 months of data from 325 trafﬁc sensors in San Francisco Bay Area, while METR-LA contains 4 months of sensor readings from 207 detectors in the Los Angeles County Highway (Jagadish et al., 2014); for both datasets, the sampling rate corresponds to 5 minutes.

Original: We use input sequences of 24 steps, which correspond to 2 hours of data.
Revised: We use input sequences of 24 steps, which correspond to 2 hours of data.

Original: For adjacency we use a thresholded Gaussian kernel applied to geographic distances following previous works Wu et al.
Revised: For adjacency we use a thresholded Gaussian kernel applied to geographic distances following previous works Wu et al.

Original: We split the data into three folds, using 70% of them for training and the remaining 10% and 20% for validation and testing, respectively.
Revised: We split the data into three folds, using 70% of them for training and the remaining 10% and 20% for validation and testing, respectively.

Original: B. S MART GRIDS
Revised: B. S MART GRIDS

Original: We consider data from the Irish Commission for Energy Regulation (CER) Smart Metering Project 4 .
Revised: We consider data from the Irish Commission for Energy Regulation (CER) Smart Metering Project 4 .

Original: We select only the subset of the available smart meters monitoring energy consumption of small and medium-sized enterprises (SMEs), i.e., 485 time series with samples acquired every 30 minutes.
Revised: We select only the subset of the available smart meters monitoring energy consumption of small and medium-sized enterprises (SMEs), i.e., 485 time series with samples acquired every 30 minutes.

Original: Note that access to dataset can be obtained free of charge for research purposes.
Revised: Note that access to dataset can be obtained free of charge for research purposes.

Original: We build an adjacency matrix by extracting a k-nearest neighbor graph (with k = 10 ) from the similarity matrix built by computing the week-wise correntropy (Liu et al., 2007) among time series.
Revised: We build an adjacency matrix by extracting a k-nearest neighbor graph (with k = 10 ) from the similarity matrix built by computing the week-wise correntropy (Liu et al., 2007) among time series.

Original: As in the trafﬁc case, we use a 70% / 10% / 20% split for training, validation and testing and use a window size of 24 steps.
Revised: As in the trafﬁc case, we use a 70% / 10% / 20% split for training, validation and testing and use a window size of 24 steps.

Original: Data were normalized using standard scaling as in the previous settings and we did not perform additional preprocessing steps.
Revised: Data were normalized using standard scaling as in the previous settings and we did not perform additional preprocessing steps.

Original: C A DDITIONAL RESULTS
Revised: C A DDITIONAL RESULTS

Original: In this appendix, we show an additional experiment in a controlled environment and additional ablation studies.
Revised: In this appendix, we show an additional experiment in a controlled environment, comparison against additional baselines, additional ablation studies, and sensitivity analyses.

Original: C. S YNTHETIC DATA
Revised: C. S YNTHETIC DATA

Original: In this experiment
Revised: In this experiment

Original: [Table]
Revised: [Table]

Original: we test our method on the simulated particle system introduced by Kipf et al.
Revised: we test our method on the simulated particle system introduced by Kipf et al.

Original: We simulate the trajectories of N = 10 particles in a (10 × 10) box with elastic collision.
Revised: We simulate the trajectories of N = 10 particles in a (10 × 10) box with elastic collision.

Original: Each particle carries a either positive or negative charge q ∈ { 1 , − 1 } .
Revised: Each particle carries a either positive or negative charge q ∈ { 1 , − 1 } .

Original: Two particles attract each other if they have opposite sign, otherwise they repel.
Revised: Two particles attract each other if they have opposite sign, otherwise they repel.

Original: Interaction forces between two particles are ruled by Coulomb’s law.
Revised: Interaction forces between two particles are ruled by Coulomb’s law.

Original: We collect two datasets, each containing 5000 independent simulations of T = 36 steps each.
Revised: We collect two datasets, each containing 5000 independent simulations of T = 36 steps each.

Original: In the ﬁrst dataset, particles have the same charge in every simulation.
Revised: In the ﬁrst dataset, particles have the same charge in every simulation.

Original: In the second one, we sample the charges uniformly at random at the beginning of every simulation.
Revised: In the second one, we sample the charges uniformly at random at the beginning of every simulation.

Original: In both scenarios, the initial location and velocity of the particles are drawn randomly.
Revised: In both scenarios, the initial location and velocity of the particles are drawn randomly.

Original: At each step, we randomly remove blocks of consecutive readings with a probability p failure = 2 .
Revised: At each step, we randomly remove blocks of consecutive readings with a probability p failure = 2 .

Original: 5% and a length sampled uniformly from the interval [4 , 9] .
Revised: 5% and a length sampled uniformly from the interval [4 , 9] .

Original: Here, a reading consists of the ( x, y ) coordinates of the particle’s position.
Revised: Here, a reading consists of the ( x, y ) coordinates of the particle’s position.

Original: We further mask out 2 .
Revised: We further mask out 2 .

Original: 5% of positions at random.
Revised: 5% of positions at random.

Original: The percentage of values not masked out is ≈ 74% .
Revised: The percentage of values not masked out is ≈ 74% .

Original: For evaluation purposes, we generate another mask using the same missing data distribution and use the masked values as ground-truth for evaluation.
Revised: For evaluation purposes, we generate another mask using the same missing data distribution and use the masked values as ground-truth for evaluation.

Original: We split dataset in training/validation/test folds using 70% / 10% / 20% splits, respectively.
Revised: We split dataset in training/validation/test folds using 70% / 10% / 20% splits, respectively.

Original: [Table]
Revised: [Table]

Original: We test our method ( GRIN ) and BRITS in both synthetic datasets.
Revised: We test our method ( GRIN ) and BRITS in both synthetic datasets.

Original: We use 32 units for the hidden layer of BRITS ( ≈ 25 K parameters) and 16 units for both the encoder and decoder of GRIN ( ≈ 10 K parameters).
Revised: We use 32 units for the hidden layer of BRITS ( ≈ 25 K parameters) and 16 units for both the encoder and decoder of GRIN ( ≈ 10 K parameters).

Original: Results are reported in Tab. 6.
Revised: Results are reported in Tab. 6.

Original: Both the methods take as input only the particles’ positions, with no information about the charges.
Revised: Both the methods take as input only the particles’ positions, with no information about the charges.

Original: As can be seen, consistently with what observed by Kipf et al.
Revised: As can be seen, consistently with what observed by Kipf et al.

Original: (2018), relational representations are impressively effective in this scenario.
Revised: (2018), relational representations are impressively effective in this scenario.

Original: Our method outperforms the baseline by more than an order of magnitude in terms of MSE.
Revised: Our method outperforms the baseline by more than an order of magnitude in terms of MSE.

Original: Surprisingly, BRITS is less accurate in the setting with varying charge.
Revised: Surprisingly, BRITS is less accurate in the setting with varying charge.

Original: Our hypothesis is that the added stochasticity acts as a regularization and forces BRITS to learn a more general model.
Revised: Our hypothesis is that the added stochasticity acts as a regularization and forces BRITS to learn a more general model.

Original: C. A BLATION STUDY
Revised: C. E MPIRICAL COMPARISON AGAINST MATRIX FACTORIZATION WITH SIDE INFORMATION

Revised: [Table] As mentioned in Section [Table]

Revised: several matrix factorization approaches – often studied in the context of recommender systems – can be regularized by considering priors on the spatio-temporal structure of the data.

Revised: Intuitively, spatial regularization is achieved by imposing soft constraints on the smoothness of the interpolated function w.r.t.

Revised: nodes of an underlying graph (Cai et al., 2010; Rao et al., 2015).

Revised: Temporal regularization can be obtained by imposing analogous constraints modelling temporal dependencies as – eventually weighted – edges of a graph.

Revised: In temporal regularized matrix factorization (TRMF; Yu et al., 2016), similarly, coefﬁcients of an autoregressive model are used as temporal regularizer.

Revised: Tab. 7 shows a comparison of different matrix factorization approaches on imputation in the air quality datasets (where we considered the in-sample setting in Section 5).

Revised: For TRMF we used an implementation adapted from the Transdim repository 6 , while for graph regularized matrix factorization (GMRF) we use a custom implementation of the method proposed by Cai et al. (2010).

Revised: We ﬁxed the rank to be equal to 10 (as the one used in all the experiments for standard MF) and tuned the regularization coefﬁcients on a validation set.

Revised: Results do show that introducing spatial and temporal regularization improve w.r.t.

Revised: vanilla MF; however, deep learning methods – and even linear VAR predictors – achieve far superior reconstruction accuracy here.

Revised: Arguably, low-rank approximation methods might instead have an edge in a low-data regime: this type of analysis is, however, out of the scope of this work.

Revised: C. S CALABILITY

Revised: With reference to a standard bi-directional GRU, using MPNNs to implement the cell’s gates increases the computational complexity by a factor that scales with the number of edges O ( E ) – if using an efﬁcient sparse implementation – or with the number of nodes squared O ( N 2 ) .

Revised: Luckily, this overhead can be amortized as most of the computation can be parallelized.

Revised: Research on scalable and memory efﬁcient GNNs is a very active ﬁeld (e.g., Hamilton et al., 2017; Frasca et al., 2020): depending on the task, the designer can opt for massage passing operators that meet the application requirements in terms of performance, time and space constraints.

Revised: C. A BLATION STUDY

Revised: Here we provide two different ablation studies, the ﬁrst one on the architecture of GRIN and the second one on the graph structure.

Revised: C.4. A RCHITECTURAL ABLATIONS

Original: Tab. 7 shows additional results for the ablation study presented in Section 5.
Revised: Tab. 8 shows additional results for the ablation study presented in Section 5.

Original: Consistently with what we already observed, the spatial decoder and bidirectional architecture improve performance and appear particularly relevant in settings with blocks of missing data.
Revised: Consistently with what we already observed, the spatial decoder and bidirectional architecture improve performance and appear particularly relevant in settings with blocks of missing data.

Original: [Table]
Revised: [Table]

Revised: C.4. G RAPH STRUCTURE ABLATIONS

Revised: Here we study how exploiting the relational structure of the problem affects the accuracy of the reconstruction.

Revised: In particular

Revised: [Table]

Revised: we run two additional experiments on the METR-LA dataset (Block missing settings)

Revised: [Table]

Revised: where instead of using as adjacency matrix the thresholded kernel in Eq.

Revised: [Table]

Revised: we use (1) a fully connected graph ( W = 1 ) and (2) a graph with no edges ( W = I ).

Revised: To provide node – i

Revised: [Table]

Revised: e., sensor – identiﬁcation, we use learnable embeddings as additional node features.

Revised: Results are shown in Tab. 9, performance for BRITS are reported as reference.

Revised: It is clear that the constraints posed by the graph structure do have an impact on the accuracy of missing data imputation and, at the same time, that spatial information is relevant for the task

Revised: [Table]

Revised: [Table]

Revised: S ENSITIVITY ANALYSIS

Revised: Finally

Revised: [Table]

Revised: in this subsection we carry out an assessment of performance degradation w.r.t.

Revised: the amount of missing data.

Revised: Before discussing results

Revised: [Table]

Revised: there a few remarks that are worth bringing up regarding imputation in highly sparse settings.

Revised: In the ﬁrst place, GRIN , as well as a large portion of the state-of-the-art baselines, is an autoregressive model, which means that it might be subject to error accumulation over long time horizons.

Revised: Furthermore, here, consistently with Section 5, we consider the out-of-sample setting which is particularly challenging in the sparse data regime.

Revised: That being said, GRIN achieves remarkable performance also in this benchmark.

Revised: We train one model each for GRIN and BRITS by randomly masking out 60% of input data for each batch during training, then, we run the models on the test set by using evaluation masks with increasing sparsity (note that this causes a distribution shift in evaluation).

Revised: For each level of sparsity, evaluation is repeated 5 times by sampling different evaluation masks.

Revised: Results are reported in Tab.and Fig.

Revised: 5 show that GRIN outperforms BRITS in all the considered scenarios

Revised: [Table]

Revised: [Table]

Revised: [Figure]