Original: M ULTIVARIATE T IME - SERIES I MPUTATION WITH D ISENTANGLED T EMPORAL R EPRESENTATIONS
Revised: M ULTIVARIATE T IME - SERIES I MPUTATION WITH D ISENTANGLED T EMPORAL R EPRESENTATIONS

Original: Anonymous authors

Original: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Multivariate time series often faces the problem of missing value.
Revised: Multivariate time series often faces the problem of missing value.

Original: Many time series imputation methods have been developed inthe literature.
Revised: Many time series imputation methods have been developed in literature.

Original: However, these methods all rely on an entangled representation to model dynamics of time series, which may fail to fully exploit the multiple factors (e.g., periodic patterns) contained in the time series.
Revised: However, they all rely on an entangled representation to model dynamics of time series, which may fail to fully exploit the multiple factors (e.g., periodic patterns) presented in the data.

Original: Moreover, the entangled representation usually has no semantic meaning, and thus they often lack interpretability.
Revised: Moreover, the entangled representations usually have no semantic meaning, and thus they often lack interpretability.

Original: In addition, many recent models are proposed to deal with the whole time series to identify temporal dynamics, but they are not scalable to long time-series datasets.
Revised: In addition, many recent models are proposed to deal with the whole time series to identify temporal dynamics, but they are not scalable to long time series.

Original: Different from existing approaches, we propose TIDER , a novel matrix factorization-based method with disentangled temporal representations that account for multiple factors, namely trend , seasonality , and local bias , to model complex dynamics.
Revised: Different from existing approaches, we propose TIDER , a novel matrix factorization-based method with disentangled temporal representations that account for multiple factors, namely trend , seasonality , and local bias , to model complex dynamics.

Original: The learned disentanglement makes the imputation process more reliable and offers explainability for imputation results.
Revised: The learned disentanglement makes the imputation process more reliable and offers explainability for imputation results.

Original: Moreover, TIDER is scalable to long time series.
Revised: Moreover, TIDER is scalable to long time series.

Original: Empirical results show that our method outperforms existing approaches on three typical real-world datasets, especially on long time series, with mean absolute error improvements up to 50 %.
Revised: Empirical results show that our method outperforms existing approaches on three typical real-world datasets, especially on long time series, reducing mean absolute error by up to 50%.

Original: It also scales well to long datasets on which existing deep learning based methods struggle.
Revised: It also scales well to long datasets on which existing deep learning based methods struggle.

Original: Disentanglement validation experiments further demonstrate the robustness  of our model in obtaining accurate and explainable disentangled components.  I NTRODUCTION
Revised: Disentanglement validation experiments further highlight the robustness and accuracy of our model.

Original: Multivariate time series analysis (e.g., forecasting (Zeng et al., 2021) and classification (Li et al., 2022)) has a wide spectrum of applications like traffic flow forecasting (Liu et al., 2020), electricity demand prediction (Kaur et al., 2021), motion detection (Laddha et al., 2021), health monitoring(Tonekaboni et al., 2021), etc.
Revised: Multivariate time series analysis (e.g., forecasting (Zeng et al., 2021) and classification (Li et al., 2022)) has a wide spectrum of applications like traffic flow forecasting (Liu et al., 2020), electricity demand prediction (Kaur et al., 2021), motion detection (Laddha et al., 2021), health monitoring(Tonekaboni et al., 2021), etc.

Original: Most of these multivariate time series analysis approaches typically assume intact input for building models.
Revised: Most of these multivariate time series analysis approaches typically assume intact input for building models.

Original: However, real-world multivariate time series tends to have missing values caused by factors like device malfunction, communication failure, or costly measurement, which will impair the performance of these approaches or even make them inapplicable.
Revised: However, real-world multivariate time series tends to have missing values caused by factors like device malfunction, communication failure, or costly measurement, leading to impaired performance of these approaches, or even rendering them inapplicable.

Original: In light of this, many time series imputation methods have been proposed to infer missing values from the observed ones.
Revised: In light of this, many time series imputation methods have been proposed to infer missing values from the observed ones.

Original: A multivariate time series, denoted as X ∈ R N × T , consists of N univariate time series (called channels) spanning over T time steps.
Revised: A multivariate time series, denoted as X ∈ R N × T , consists of N univariate time series (called channels) spanning over T time steps.

Original: Hence, it offers two perspectives for imputation: modeling cross-channel correlations and exploiting temporal dynamics.
Revised: Hence, it offers two perspectives for imputation: modeling cross-channel correlations and exploiting temporal dynamics.

Original: Earlier methods (Batista et al., 2002; Acuna & Rodriguez, 2004; Box et al., 2015) either aggregate observed entries across channels by estimating similarity betweentwo distinct channels, or solely exploit local smoothness or linear assumption in the same channel to fill in missing values.
Revised: Earlier methods (Batista et al., 2002; Acuna & Rodriguez, 2004; Box et al., 2015) either aggregate observed entries across channels by estimating similarity between distinct channels, or solely exploit local smoothness or linear assumption in the same channel to fill in missing values.

Original: Since these methods lack the ability of modeling nonlinear dynamics and complex correlations, they may not perform well in practice.
Revised: Since these methods lack the ability of modeling nonlinear dynamics and complex correlations, they may not perform well in practice.

Original: In recent years, deep learning based methods (Liu et al., 2019;
Revised: In recent years, deep learning based methods (Liu et al., 2019;

Original: Tashiro et al., 2021; Cini et al., 2022) were proposed for time series imputation.
Revised: Tashiro et al., 2021; Cini et al., 2022) were proposed for time series imputation.

Original: These methods typically employ Recurrent Neural
Revised: These methods typically employ Recurrent Neural

Original: Networks (RNNs) as the backbone to jointly model nonlinear dynamics along temporal dimension (by updating hidden states nonlinearly over time steps) and cross-channel correlations (by mapping x t ∈ R N from data space to hidden state space).
Revised: Networks (RNNs) as the backbone to jointly model nonlinear dynamics along temporal dimension (by updating hidden states nonlinearly over time steps) and cross-channel correlations (by mapping x t ∈ R N from data space to hidden state space).

Original: These RNN-based imputation models achieve better results by further combining with multi-task loss (Cao et al., 2018; Cini et al., 2022) or adversarial training (Liu et al., 2019; Miao et al., 2021).
Revised: These RNN-based imputation models achieve better results by further combining with multi-task loss (Cao et al., 2018; Cini et al., 2022) or adversarial training (Liu et al., 2019; Miao et al., 2021).

Original: Despite their success, these methods rely solely on a single entangled representation (hidden state) to model dynamics.
Revised: Despite their success, these methods rely solely on a single entangled representation (hidden state) to model dynamics.

Original: However, for many real-world multivariate time series, their dynamics are rich combinations of multiple independent factors like trend, seasonality (periodicity), and local idiosyncrasy (Woo et al., 2022).
Revised: However, for many real-world multivariate time series, their dynamics are rich combinations of multiple independent factors like trend, seasonality (periodicity), and local idiosyncrasy (Woo et al., 2022).

Original: Modeling combinations of these factors with a single entangled representation may not give good performance, as the entangled representation has to compromise itself to explain multiple orthogonal patterns (like local changes or exogenous interventions vs global patterns) together (Bengio et al., 2013).
Revised: Modeling combinations of these factors with a single entangled representation may not give good performance, as the entangled representation has to compromise itself to explain multiple orthogonal patterns (like local changes or exogenous interventions vs global patterns) together (Bengio et al., 2013).

Original: Furthermore, it becomes exacerbated when seasonal patterns dominate, as RNNs lack the inductive bias to initiatively capture periodicity (Hewamalage et al., 2021).
Revised: Furthermore, it becomes exacerbated when seasonal patterns dominate, as RNNs lack the inductive bias to initiatively capture periodicity (Hewamalage et al., 2021).

Original: In addition, the hidden states that these models learned are entangled and complex combination of various components.
Revised: In addition, the hidden states that these models learned are entangled and complex combination of various components.

Original: Thus it is difficult for them to provide interpretable information to explain imputation.
Revised: Thus it is difficult for them to provide interpretable information to explain imputation.

Original: Moreover,in these methods, the whole time seriesof length T is required to be fed into their models at each forward step to capture temporal dynamics.
Revised: Moreover, these methods require the entire time series to be fed in to their models at each forward step, to capture temporal dynamics.

Original: This is prohibitively costly for large T .
Revised: This is prohibitively costly for large T .

Original: Therefore, these methods are not applicable to long datasets.
Revised: Therefore, these methods are not applicable to long datasets.

Original: To address these limitations, in this paper, we propose a novel multivariate time series imputation method, Time-series Imputation with Disentangled tEmporal Representations ( TIDER ), in which we explicitly model the complex dynamics of multivariate time series with disentangled representations to account for different factors.
Revised: To address these limitations, in this paper, we propose a novel multivariate time series imputation method, Time-series Imputation with Disentangled tEmporal Representations ( TIDER ), in which we explicitly model the complex dynamics of multivariate time series with disentangled representations to account for different factors.

Original: We employ a low-rank matrix decomposition framework, and achieve the disentanglement by imposing different forms of constraints on different representations which compose the low-rank matrix.
Revised: We employ a low-rank matrix decomposition framework, and achieve the disentanglement by imposing different forms of constraints on different representations which compose the low-rank matrix.

Original: In particular, we introduce a neighboring-smoothness representation matrix to explain the trend, a Fourier series-based representation matrix to define periodic inductive bias, and a local bias representation matrix to capture idiosyncrasies specific to individual time step.
Revised: In particular, we introduce a neighboring-smoothness representation matrix to explain the trend, a Fourier series-based representation matrix to define periodic inductive bias, and a local bias representation matrix to capture idiosyncrasies specific to individual time step.

Original: These disentangled representations offer better flexibility and robustness to model the factors contributed to the dynamics in time series.
Revised: These disentangled representations offer greater flexibility and robustness to model the factors contributed to the dynamics in time series.

Original: Another notable benefit of these semantically meaningful disentangled representations is that they offer TIDER an interpretable perspective on imputation.
Revised: Another notable benefit of these semantically meaningful disentangled representations is that they offer TIDER an interpretable perspective on imputation.

Original: Moreover, TIDER is a scalable model.
Revised: Moreover, TIDER is a scalable model.

Original: It can be applied to time-series datasets with large T and achieve much better imputation performance.
Revised: It can be applied to time-series datasets with large T and achieve much better imputation performance.

Original: In summary, our contributions are as follows.
Revised: In summary, our contributions are as follows.

Original: • We propose  a new multivariate time series imputation model,TIDER , which is featured with effective and explainable disentangled representations to account for various factors that characterize the complex dynamics of time series.
Revised: • We propose TIDER , a new multivariate time series imputation model, which is featured with effective and explainable disentangled representations to account for various factors that characterize the complex dynamics of time series.

Original: To the best of our knowledge, TIDER is the first model to learn disentangled representations for multivariate time series imputation.
Revised: To the best of our knowledge, TIDER is the first model to learn disentangled representations for multivariate time series imputation.

Original: • TIDER is the first imputation model that introduces a learnable Fourier series-based representation to capture periodic patterns inherent in time series data.
Revised: • TIDER is the first imputation model that introduces a learnable Fourier series-based representation to capture periodic patterns inherent in time series data.

Original: • Extensive experiments show that our proposed method outperforms the baseline imputation methods in terms of effectiveness and scalability.
Revised: • Extensive experiments show that our proposed method outperforms the baseline imputation methods in terms of effectiveness and scalability.

Original: Especially, for imputation task in long time series, TIDER achieves more than 50% improvement in MAE, compared with the best imputation baseline approach.
Revised: Especially, for imputation task in long time series, TIDER achieves more than 50% improvement in MAE, compared with the best imputation baseline approach.

Original: Furthermore, TIDER is a scalable model.
Revised: Furthermore, TIDER is a scalable model.

Original: It can easily handle long multivariate time series while existing deep-learning methods struggle.
Revised: It can easily handle long multivariate time series while existing deep-learning methods struggle.

Original: We also demonstrate the explanability of the disentangled representations with several case studies
Revised: We also demonstrate the explanability of the disentangled representations with several case studies

Original:  R ELATED WORK
Revised:  R ELATED WORK

Original: Early time series imputation methods based on simple statistical strategies focus on exploiting local smoothness of temporal dimension as well as the similarity between different channels.
Revised: Early time series imputation methods based on simple statistical strategies focus on exploiting local smoothness of temporal dimension as well as the similarity between different channels.

Original: For example, SimpleMean / SimpleMedian (Fung, 2006) imputes missing values by averaging, and KNN (Batista et al., 2002) aggregates cross-channel observations to fill in missing slots with k-nearest neighbors.
Revised: For example, SimpleMean / SimpleMedian (Fung, 2006) imputes missing values by averaging, and KNN (Batista et al., 2002) aggregates cross-channel observations to fill in missing slots with k-nearest neighbors.

Original: Several approaches attempt to impute by building linear dynamics, including linear imputation and state-space models (Durbin & Koopman, 2012).
Revised: Linear dynamics-based imputation methods, including linear imputation and state-space models (Durbin & Koopman, 2012), have also been employed.

Original: MICE (Van Buuren & Groothuis-Oudshoorn, 2011) explores the idea of estimating missing slots with chain equations.
Revised: MICE (Van Buuren & Groothuis- Oudshoorn, 2011) explores the idea of estimating missing slots with chain equations.

Original: These methods typically lack the ability of exploiting nonlinear dynamics and complex correlation across channels.
Revised: These methods typically lack the ability of exploiting nonlinear dynamics and complex correlation across channels.

Original: A recent trendin imputation methods is to apply deep learning, particularly RNNs, for better capturing the nonlinearity of time series.
Revised: A recent trend is to apply deep learning, particularly RNNs, to better capture the nonlinearity.

Original: BRITS (Cao et al., 2018) treats missing values as variables which participate in bidirectional RNN back-propagation and are updated with hidden states.
Revised: BRITS (Cao et al., 2018) treats missing values as variables which participate in bidirectional RNN back-propagation and are updated with hidden states.

Original: GAIN (Yoon et al., 2018) combines Generative Adversarial Nets (GAN) and RNNs for imputation.
Revised: GAIN (Yoon et al., 2018) combines Generative Adversarial Nets (GAN) and RNNs for imputation.

Original: NAOMI (Liu et al., 2019) combines RNNs and adversarial training strategies to generate realistic-looking sequences.
Revised: NAOMI (Liu et al., 2019) merges RNNs and adversarial training strategies.

Original: SSGAN (Miao et al., 2021) is a semi-supervised GAN-based model which uses label information.
Revised: SSGAN (Miao et al., 2021) is a semi-supervised GAN-based model which uses label information.

Original: GRIN (Cini et al., 2022) fuses graph message passing intothe GRU structure to learn spatial-temporal patterns.
Revised: GRIN (Cini et al., 2022) fuses graph message passing into GRU structure to learn spatial-temporal patterns.

Original: All these approaches model dynamics based on a single entangled representation, which is insufficient to capture multiple factors underlying time series, especially when seasonality emerges, since RNNs lack the inductive bias to initiatively capture periodicity (Hewamalage et al., 2019).
Revised: All these methods model dynamics based on a single entangled representation, which is insufficient to capture multiple factors underlying time series, especially when seasonality emerges, since RNNs lack the inductive bias to initiatively capture periodicity (Hewamalage et al., 2019).

Original: Moreover, these approaches are not scalable to long datasets since they have to process time series of whole length T at each forward step to capture temporal dynamics.
Revised: Moreover, these approaches are not scalable to long datasets since they have to process time series of whole length T at each forward step to capture temporal dynamics.

Original: Thus, they do not perform well, or are even not applicable to long time-series data.
Revised: Thus, they do not perform well, or are even not applicable to long time-series data.

Original: Our proposedmodel TIDER is based on low-rank matrix factorization  (Yu et al., 2016; Bjorck et al., 2021).
Revised: Our proposed TIDER is based on low-rank matrix factorization (MF) (Yu et al., 2016; Bjorck et al., 2021).

Original: Vanilla matrix factorization based models impute missing entries by learningtwo latent factors U , V to exploit low-rank structure.
Revised: Vanilla MF-based models impute missing entries by learning latent factors U , V to exploit low-rank structure.

Original: However, it fails to consider the temporal continuity.
Revised: However, they overlook the temporal continuity.

Original: To address this, TRMF (Yu et al., 2016) imposes autoregressive constraints on temporal factor V .
Revised: To address this, TRMF (Yu et al., 2016) imposes autoregressive constraints on temporal factor V .

Original: However, similar to aforementioned methods, TRMF also relies onan entangled representationmatrix to account for all factors underlying the dynamics.
Revised: However, similar to aforementioned methods, TRMF also relies on entangled representation to account for all factors underlying the dynamics.

Original: In contrast,our proposed TIDER tackles this issue by introducing multiple disentangled representations.
Revised: In contrast, TIDER introduces multiple disentangled representations.

Original: We achieve the disentanglement by enforcing different forms of constraints into different representations.
Revised: It achieves the disentanglement by enforcing different forms of constraints into different representations.

Original:  P ROBLEM S TATEMENT
Revised:  P ROBLEM S TATEMENT

Original: Given N univariate time series data x 1 , x 2 , . . .
Revised: Given N univariate time series data x 1 , x 2 , . . .

Original: , x N ∈ R T collected over T time steps, we represent it with a multivariate time series matrix X ∈ R N × T , whose n -th row represents n -th univariate time series (channel) x n and t -th column denotes the observation of all time series at time step t .
Revised: , x N ∈ R T collected over T time steps, we represent it with a multivariate time series matrix X ∈ R N × T , whose n -th row represents n -th univariate time series (channel) x n and t -th column denotes the observation of all time series at time step t .

Original: The multivariate time series matrix X is incomplete and a fraction of entries are missed.
Revised: The multivariate time series matrix X is incomplete and a fraction of entries are missed.

Original: We aim to infer missing values from the observed ones, and we denote the mask matrix as M ∈ { 0 , 1 } N × T , where
Revised: We aim to infer missing values from the observed ones, and we denote the mask matrix as M ∈ { 0 , 1 } N × T , where

Original: M ij = (cid:26) 1 , if X ij is observed, 0 , otherwise.
Revised: M ij = (cid:26) 1 , if X ij is observed, 0 , otherwise.

Original:  M ETHODOLOGY 4. M ETHOD O VERVIEW
Revised:  M ETHODOLOGY 4. M ETHOD O VERVIEW

Original: The core idea of TIDER is to decompose multivariate time series X into two latent factors U and V , such that U only preserves features unique to each channel whereas V is determined by multiple disentangled representations that jointly capture temporal dynamics.
Revised: The core idea of TIDER is to decompose multivariate time series X into two latent factors U and V , such that U only preserves features unique to each channel whereas V is determined by multiple disentangled representations that jointly capture temporal dynamics.

Original: We employ such factorization for two main reasons: 1) univariate time series channels (rows of X ) are usually highly correlated; 2) observations at different time steps (columns of X ) exhibit strong temporal dynamics.
Revised: We employ such factorization for two main reasons: 1) univariate time series channels (rows of X ) are usually highly correlated; 2) observations at different time steps (columns of X ) exhibit strong temporal dynamics.

Original: The benefits of our design are twofold: 1) cross-channel correlations are decoupled since U only preserves channel-specific features 2) time-related information is isolated into V , and it enables us to model the potentially complex temporal dynamics with multiple explainable disentangled representations.
Revised: The benefits of our design are twofold: 1) cross-channel correlations are decoupled since U only preserves channel-specific features 2) time-related information is isolated into V , and it enables us to model the potentially complex temporal dynamics with multiple explainable disentangled representations.

Original: Figure 1 shows the architecture of TIDER .
Revised: Figure 1 shows the architecture of TIDER .

Original: We adopt a low-rank matrix decomposition framework to factorize multivariate time series matrix X into two latent factors U , V .
Revised: We adopt a low-rank matrix decomposition framework to factorize multivariate time series matrix X into two latent factors U , V .

Original: U is a correlationdecoupled matrix which accounts for channel-specific patterns, while V is a matrix for time-related information.
Revised: U is a correlationdecoupled matrix which accounts for channel-specific patterns, while V is a matrix for time-related information.

Original: Since temporal dynamics underlying real-world time series can be rich and complex combinations of multiple factors, e.g., trend and seasonality, modeling them merely through an entangled representation matrix will lead to model degradation.
Revised: Since temporal dynamics underlying real-world time series can be rich and complex combinations of multiple factors, e.g., trend and seasonality, modeling them merely through an entangled representation matrix will lead to model degradation.

Original: Woo et al.
Revised: Woo et al.

Original: (2022) proposes that under mild assumptions, seasonality and trend can be treated as independent factors in time series generating process, and Cohen (2013) suggests that independence can be used as a proxy criterion for disentanglement.
Revised: (2022) proposes that under mild assumptions, seasonality and trend can be treated as independent factors in time series generating process, and Cohen (2013) suggests that independence can be used as a proxy criterion for disentanglement.

Original: Inspired by them, we propose to model V with multiple disentangled representations, each accounting for one particular factor.
Revised: Inspired by them, we propose to model V with multiple disentangled representations, each accounting for one particular factor.

Original: We achieve the disentanglement by enforcing distinct forms of constraints on different representations, which introduce distinct inductive biases into these representations and make them more liable to capture specific semantically-independent patterns.
Revised: We achieve the disentanglement by enforcing distinct forms of constraints on different representations, which introduce distinct inductive biases into these representations and make them more liable to capture specific semantically-independent patterns.

Original: More specifically, we consider three important factors: trend , seasonality , and bias , which are specified by three representation matrix V t , V s , V b , respectively.
Revised: More specifically, we consider three important factors: trend , seasonality , and bias , which are specified by three representation matrix V t , V s , V b , respectively.

Original: Figure 2 illustrates an example of decomposing a time series into these three factors.
Revised: Figure 2 illustrates an example of decomposing a time series into these three factors.

Original: Trend representation matrix V t ∈ R D
Revised: Trend representation matrix V t ∈ R D

Original: × T captures the intrinsic trend which changes gradually and smoothly, and seasonality representation matrix V s
Revised: × T captures the intrinsic trend which changes gradually and smoothly, and seasonality representation matrix V s

Original: ∈ R D
Revised: ∈ R D

Original: × T illustrates the periodic patterns hidden in temporal dynamics.
Revised: × T illustrates the periodic patterns hidden in temporal dynamics.

Original: V t and V s jointly determine the dynamics driven by endogenous factors.
Revised: V t and V s jointly determine the dynamics driven by endogenous factors.

Original: Bias representation matrix V b ∈ R D
Revised: Bias representation matrix V b ∈ R D

Original: b × T characterizes variations specific to each time step.
Revised: b × T characterizes variations specific to each time step.

Original: Intuitively, V b explains the individual idiosyncratic behaviors of time steps, which are orthogonal to global dynamics but shared across channels at a given time step.
Revised: Intuitively, V b explains the individual idiosyncratic behaviors of time steps, which are orthogonal to global dynamics but shared across channels at a given time step.

Original: Hence, we treat it differently from V t , V s and interpret it as a residual term matrix, i.e., X − U
Revised: Hence, we treat it differently from V t , V s and interpret it as a residual term matrix, i.e., X − U

Original: ( V t + V s
Revised: ( V t + V s

Original: ≈ 1 N × D b V b .
Revised: ≈ 1 N × D b V b .

Original: Mathematically, we formulate the objective of TIDER as minimize
Revised: Mathematically, we formulate the objective of TIDER as minimize

Original: [Equation]
Revised: [Equation]

Original: where U a is the augmented matrix of latent factor U ∈ R N × D , f t and f b are corresponding inductive bias constraint functions imposed on V t and V b .
Revised: where U a is the augmented matrix of latent factor U ∈ R N × D , f t and f b are corresponding inductive bias constraint functions imposed on V t and V b .

Original: M is the mask matrix introduced in Sec 3.
Revised: M is the mask matrix introduced in Sec 3.

Original: η 1 ∥ U ∥ 2 , η 2 ∥ V ∥ 2 are used to regularize the magnitude of latent factors, and λ t , λ b , η 1 , η 2 are corresponding weights for each term.
Revised: η 1 ∥ U ∥ 2 , η 2 ∥ V ∥ 2 are used to regularize the magnitude of latent factors, and λ t , λ b , η 1 , η 2 are corresponding weights for each term.

Original: When training is completed, we use the learned U to get U a , and the learnt
Revised: When training is completed, we use the learned U to get U a , and the learnt

Original: V t , V s , and V b to form V .
Revised: V t , V s , and V b to form V .

Original: U a and V are then used to generate the imputed time series
Revised: U a and V are then used to generate the imputed time series

Original: ˆ X as
Revised: ˆ X as

Original: [Equation]
Revised: [Equation]

Original: We are now ready to elaborate on the details of the three disentangled representation matrices.
Revised: We are now ready to elaborate on the details of the three disentangled representation matrices.

Original: 4. T REND R EPRESENTATION M ATRIX
Revised: 4. T REND R EPRESENTATION M ATRIX

Original: Trend representation matrix V t characterizes the intrinsic trend of time series.
Revised: Trend representation matrix V t characterizes the intrinsic trend of time series.

Original: The evolution patterns dominated by V t are supposed to change gradually and smoothly in the absence of accidents or extreme events such as holidays (we will consider these external interventions and exogenous influences in V b ).
Revised: The evolution patterns dominated by V t are supposed to change gradually and smoothly in the absence of accidents or extreme events such as holidays (we will consider these external interventions and exogenous influences in V b ).

Original: Based on this, we impose a smoothness constraint on V t as
Revised: Based on this, we impose a smoothness constraint on V t as

Original: [Equation]
Revised: [Equation]

Original: where v jt is the j -th column of V t .
Revised: where v jt is the j -th column of V t .

Original: Equation 4 encourages close representations of two adjacent time steps in latent space, which will result in a smooth change in data space.
Revised: Equation 4 encourages close representations of two adjacent time steps in latent space, which will result in a smooth change in data space.

Original: We only impose constraints on two consecutive time steps to account for short-term patterns here whereas long-term patterns are explained by V s .
Revised: We only impose constraints on two consecutive time steps to account for short-term patterns here whereas long-term patterns are explained by V s .

Original: This is in contrast with TRMF (Yu et al., 2016) which uses one temporal matrix to account for both short-term and long-term patterns by imposing regression constraint.
Revised: This is in contrast with TRMF (Yu et al., 2016) which uses one temporal matrix to account for both short-term and long-term patterns by imposing regression constraint.

Original: 4. S EASONALITY R EPRESENTATION M ATRIX
Revised: 4. S EASONALITY R EPRESENTATION M ATRIX

Original: Real-world time series often demonstrate seasonal patterns.
Revised: Real-world time series often demonstrate seasonal patterns.

Original: For instance, traffic flow exhibits strong daily and weekly seasonality due to regularity of human activities, such as commuting patterns.
Revised: For instance, traffic flow exhibits strong daily and weekly seasonality due to regularity of human activities, such as commuting patterns.

Original: Solar power production presents clear periodic characteristics caused by climate seasonality and meteorological conditions.
Revised: Solar power production presents clear periodic characteristics caused by climate seasonality and meteorological conditions.

Original: Motivated by this and Fourier analysis in Section A.1.2, we propose to model the seasonality of time series by parameterizing representation matrix V s with Fourier basis.
Revised: Motivated by this and Fourier analysis in Section A.1.2, we propose to model the seasonality of time series by parameterizing representation matrix V s with Fourier basis.

Original: V s is a matrix with size D × T , we represent each row with a superposition of 2 K sinusoidal waves ( K ≪ T ) .
Revised: V s is a matrix with size D × T , we represent each row with a superposition of 2 K sinusoidal waves ( K ≪ T ) .

Original: More formally, let A ,
Revised: More formally, let A ,

Original: B ∈ R D
Revised: B ∈ R D

Original: × K be two learnable coefficient matrices, and ϕ sin , ϕ cos ∈ R T ×
Revised: × K be two learnable coefficient matrices, and ϕ sin , ϕ cos ∈ R T ×

Original: K be the corresponding Fourier basis matrices, which are defined as
Revised: K be the corresponding Fourier basis matrices, which are defined as

Original: [Equation]
Revised: [Equation]

Original: where t = [1 , . . .
Revised: where t = [1 , . . .

Original: , T ] ⊤ .
Revised: , T ] ⊤ .

Original: For one specific time-series with period P , ω is calculated as 2 π/P .
Revised: For one specific time-series with period P , ω is calculated as 2 π/P .

Original: Then the seasonality representation matrix V s can be defined as
Revised: Then the seasonality representation matrix V s can be defined as

Original: [Equation]
Revised: [Equation]

Original: In other words, V s is spanned by Fourier basis ϕ sin and ϕ cos .
Revised: In other words, V s is spanned by Fourier basis ϕ sin and ϕ cos .

Original: In particular, the d -th row of V s , denoted by ( v ds ) ⊤ , has the form
Revised: In particular, the d -th row of V s , denoted by ( v ds ) ⊤ , has the form

Original: [Equation]
Revised: [Equation]

Original: which is a truncated Fourier series with coefficients A d,k , B d,k .
Revised: which is a truncated Fourier series with coefficients A d,k , B d,k .

Original: This elaborate design of V s provides meaningful periodic inductive bias, which enables our model to capture seasonal patterns more accurately and effectively, by learning coefficient matrices A , B from data.
Revised: This elaborate design of V s provides meaningful periodic inductive bias, which enables our model to capture seasonal patterns more accurately and effectively, by learning coefficient matrices A , B from data.

Original: 4. B IAS T EMPORAL R EPRESENTATION M ATRIX
Revised: 4. B IAS T EMPORAL R EPRESENTATION M ATRIX

Original: The representation matrices V t , V s presented so far jointly determine the dynamics driven by endogenous factors of multivariate time series.
Revised: The representation matrices V t , V s presented so far jointly determine the dynamics driven by endogenous factors of multivariate time series.

Original: However, there are also various external factors (e.g., holidays, weekdays/weekends) that could affect real-world time series.
Revised: However, there are also various external factors (e.g., holidays, weekdays/weekends) that could affect real-world time series.

Original: These external factors usually occur at specific time points and yield local variations within short time period, thus they are independent of endogenous dynamics and cannot be captured by V t and V s .
Revised: These external factors usually occur at specific time points and yield local variations within short time period, thus they are independent of endogenous dynamics and cannot be captured by V t and V s .

Original: And also, these factors impact nearly equally on all channels.
Revised: And also, these factors impact nearly equally on all channels.

Original: To account for these local variations, inspired by the idea of user and item bias explored in collaborative filtering (Lü et al., 2012), we propose to learn another bias representation matrix V b ∈ R D b × T , where the representation of a specific time step is shared by all channels.
Revised: To account for these local variations, inspired by the idea of user and item bias explored in collaborative filtering (Lü et al., 2012), we propose to learn another bias representation matrix V b ∈ R D b × T , where the representation of a specific time step is shared by all channels.

Original: In addition, we impose an autoregressive constraint on V b in temporal dimension since the impact caused by local variation usually lasts for a short duration.
Revised: In addition, we impose an autoregressive constraint on V b in temporal dimension since the impact caused by local variation usually lasts for a short duration.

Original: Let v t be the t -th column of
Revised: Let v t be the t -th column of

Original: V b and L be the maximum time lag indicating duration, we define the constraint function as follows,
Revised: V b and L be the maximum time lag indicating duration, we define the constraint function as follows,

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original: where W = { W l ∈ R D
Revised: where W = { W l ∈ R D

Original: b × D b | l = 1 . . .
Revised: b × D b | l = 1 . . .

Original: L } is a group of learnable parameters.
Revised: L } is a group of learnable parameters.

Original: In our setting, D b and L are small numbers, thus group W only incurs very few extra parameters.
Revised: In our setting, D b and L are small numbers, thus group W only incurs very few extra parameters.

Original: 4. A DAPTIVE W EIGHT FOR T REND AND S EASONALITY
Revised: 4. A DAPTIVE W EIGHT FOR T REND AND S EASONALITY

Original: In Section 4.1, we use the additive form V t
Revised: In Section 4.1, we use the additive form V t

Original: + V s to characterize the influence of endogenous impacting factors.
Revised: + V s to characterize the influence of endogenous impacting factors.

Original: This implicitly assumes that trend and seasonality components contribute equally to endogenous dynamics.
Revised: This implicitly assumes that trend and seasonality components contribute equally to endogenous dynamics.

Original: But in practice the importance of trend and seasonality can vary drastically across data sources, which is illustrated in Figure 2.
Revised: But in practice the importance of trend and seasonality can vary drastically across data sources, which is illustrated in Figure 2.

Original: To address this, we adopt a learnable parameter α ∈ (0 , 1) to adaptively adjust the weight for these two components, which leads to a weighted additive form α V t + (1 − α ) V s .
Revised: To address this, we adopt a learnable parameter α ∈ (0 , 1) to adaptively adjust the weight for these two components, which leads to a weighted additive form α V t + (1 − α ) V s .

Original: The temporal matrix V in Equation 2 then becomes:
Revised: The temporal matrix V in Equation ?? then becomes:

Original: [Equation]
Revised: [Equation]

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: In this section, we evaluate the performance of TIDER by comparing with existing multivariate time series imputation methods, in terms of imputation accuracy and scalability.
Revised: In this section, we evaluate the performance of TIDER by comparing with existing multivariate time series imputation methods, in terms of imputation accuracy and scalability.

Original: We also show the explanability of TIDER with several case studies.
Revised: We also show the explanability of TIDER with several case studies.

Original: Hyperparameter sensitivity experiments are included as well to show that TIDER performs steadily under different hyperparameter settings.
Revised: Hyperparameter sensitivity experiments are included as well to show that TIDER performs steadily under different hyperparameter settings.

Original: The code of TIDER is available at https://anonymous.4open.science/r/TIDER-527C .
Revised: The code of TIDER is available at https://anonymous.4open.science/r/TIDER-527C .

Original: 5. E XPERIMENTAL S ETUP
Revised: 5. E XPERIMENTAL S ETUP

Original: Baseline Methods We compare our model with popular baselines used in the literature and recentlyproposed methods, including statistical models ( SimpleMean , KNN , MICE ), MF-based methods ( MF , MF+L2 , SoftImpute , TRMF ), and deep learning approaches ( BRITS , GAIN , NAOMI , SingleRes , SAITS , CSDI ).
Revised: Baseline Methods We compare our model with popular baselines used in the literature and recentlyproposed methods, including statistical models ( SimpleMean , KNN , MICE ), MF-based methods ( MF , MF+L2 , SoftImpute , TRMF ), and deep learning approaches ( BRITS , GAIN , NAOMI , SingleRes , SAITS , CSDI ).

Original: Details and settings of these baselines can be found in Section A.3.
Revised: Details and settings of these baselines can be found in Section A.3.

Original: In addition, we also include TIDER (no W), a variant of TIDER without the learnable parameter α , to verify the effectiveness of the adaptive weight introduced in Section 4.5.
Revised: In addition, we also include TIDER (no W), a variant of TIDER without the learnable parameter α , to verify the effectiveness of the adaptive weight introduced in Section 4.5.

Original: Datasets We use three typical real-world datasets.
Revised: Datasets We use three typical real-world datasets.

Original: Guangzhoudataset is arelatively small one where all methods can fit in.
Revised: Guangzhou is a small one where all methods can fit in.

Original: Solar-energy shares an extremely long time-span while Westminster is one with large N. Thesethree datasets represent three different types of multivariate time series (small, large T, large N), and these three types can cover most of the multivariate time series.
Revised: Solar-energy shares along time-span while Westminster is one with large N. These datasets represent three different types of multivariate time series (small, large T, large N), and these three types can cover most of the multivariate time series.

Original: For more details about these datasets, please refer to Section A.2.
Revised: For more details, please refer to Section A.2.

Original: Evaluation Metrics We adopt RMSE, MAE, and MAPE to evaluate the imputation accuracy of all compared methods.
Revised: Evaluation Metrics We adopt RMSE, MAE, and MAPE to evaluate the imputation accuracy of all compared methods.

Original: Details of these three metrics can be found in Section A.4.
Revised: Details of these three metrics can be found in Section A.4.

Original: Training Setup We randomly remove a subset of entries from X as validation and test datasets separately.
Revised: Training Setup We randomly remove a subset of entries from X as validation and test datasets separately.

Original: Let r be the missing rate variable, the ratio of training/validation/test is (0 .
Revised: Let r be the missing rate variable, the ratio of training/validation/test is (0 .

Original: 9 − r ) / 0 .
Revised: 9 − r ) / 0 .

Original: 1 /r .
Revised: 1 /r .

Original: For each model we run experiments 7 rounds on every dataset and report the imputed results averaged over these 7 runs.
Revised: For each model we run experiments 7 rounds on every dataset and report the imputed results averaged over these 7 runs.

Original: All experiments are conducted on a Linux workstation with a 32GB Tesla V
Revised: All experiments are conducted on a Linux workstation with a 32GB Tesla V

Original: GPU.
Revised: GPU.

Original: For more detailed hyperparameter settings, please refer to Table 7.
Revised: For more detailed hyperparameter settings, please refer to Table 7.

Original: 5. I MPUTATION A CCURACY C OMPARISON
Revised: 5. I MPUTATION A CCURACY C OMPARISON

Original: Table 1 and Table 2 show the imputation accuracy of all methods on three datasets with different missing rate r .
Revised: Table 1 and Table 2 show the imputation accuracy of all methods on three datasets with different missing rate r .

Original: OOM indicates out of memory.
Revised: OOM indicates out of memory.

Original: The meaning of asterisk and improvement are illustrated in Section A.3.
Revised: The meaning of asterisk and improvement are illustrated in Section A.3.

Original: Our proposed TIDER achieves the best performance in most cases in terms of the  three metrics.
Revised: Our proposed TIDER achieves the best performance in most cases in terms of the [Table] three metrics.

Original: In particular, the superiority of TIDER is much more significant in Solar-energy, the dataset withextremely large T .
Revised: The superiority of TIDER is much more significant in Solar-energy, the dataset with large T .

Original: In addition, we obtain similar improvements on other long time-series data.
Revised: In addition, we obtain similar improvements on other long time series.

Original: We reportcomparison experiments on another twolong time-series datasets in Section A.5.
Revised: We report experiments on another two datasets in Section A.5.

Original: In addition, we observe that the imputation accuracy of most methods drops as missing rate r increases, which is as expected since fewer patterns are available.
Revised: In addition, we observe that the imputation accuracy of most methods drops as missing rate r increases, which is as expected since fewer patterns are available.

Original: Among all baseline methods, CSDI has the best performance on most datasets, and deep-learning methods ( BRITS , GAIN , SAITS , CSDI ) usually perform better than other baselines.
Revised: Among all baseline methods, CSDI has the best performance on most datasets, and deep-learning methods ( BRITS , GAIN , SAITS , CSDI ) usually perform better than other baselines.

Original: However,
Revised: However,

Original: TIDER outperforms them on each dataset.
Revised: TIDER outperforms them on each dataset.

Original: Especially on the solar-energy dataset, where time span
Revised: Especially on the solar-energy dataset, where time span

Original: T is 52 , 560 and many deep learning methods run into OOM on a 32GB-memory GPU, TIDER is still applicable and achieves the best performance by large margins.
Revised: T is 52 , 560 and many deep learning methods run into OOM on a 32GB-memory GPU, TIDER is still applicable and achieves the best performance by large margins.

Original: This could be attributed to our model’s ability to model multiple factors in representation space in a disentangled way.[Table] [Table]
Revised: This could be attributed to our model’s ability to model multiple factors in representation space in a disentangled way.

Original: We observe that MF-based models can all work for long time series.
Revised: We observe that MF-based models can all work for long time series.

Original: The difference between these models lies in constraints on U and V .
Revised: The difference between these models lies in constraints on U and V .

Original: Compared with other MF-based baselines, TRMF outperforms others in most cases, since it intuitively captures auto-regressive dynamics.
Revised: Compared with other MF-based baselines, TRMF outperforms others in most cases, since it intuitively captures auto-regressive dynamics.

Original: However, since TRMF employs merely an entangled representation, it cannot model complex dynamics well.
Revised: However, since TRMF employs merely an entangled representation, it cannot model complex dynamics well.

Original: TIDER (no
Revised: TIDER (no

Original: W) and TIDER utilize disentangled features like trend, seasonality and local bias to rebuild patterns for time-series, which exploit explanatory temporal factors, thus achieving better imputation result.
Revised: W) and TIDER utilize disentangled features like trend, seasonality and local bias to rebuild patterns for time-series, which exploit explanatory temporal factors, thus achieving better imputation result.

Original: Furthermore, TIDER performs better than TIDER (no W) .
Revised: Furthermore, TIDER performs better than TIDER (no W) .

Original: This is consistent with our previous analysis that trend and seasonality components might not contribute equally to global dynamics.
Revised: This is consistent with our previous analysis that trend and seasonality components might not contribute equally to global dynamics.

Original: 5. S CALABILITY A NALYSIS
Revised: 5. S CALABILITY A NALYSIS

Original: We study the scalability of different methods in memory usage and training time.
Revised: We study the scalability of different methods in memory usage and training time.

Original: In particular, we compare our method with state-of-the-art models, BRITS , NAOMI , SingleRes , SAITS , and CSDI .
Revised: In particular, we compare our method with state-of-the-art models, BRITS , NAOMI , SingleRes , SAITS , and CSDI .

Original: Figure 3-(a) and Figure 3-(b) present the memory footprints of different methods against channel[Figure] [Table] number N and time span T .
Revised: Figure 3-(a) and Figure 3-(b) present the memory footprints of different methods against channel number N and time span T .

Original: It can be seen from Figure 3-(a) that memory usage of NAOMI and
Revised: It can be seen from Figure 3-(a) that memory usage of NAOMI and

Original: SingleRes grow rapidly with very steep slopes, and that of BRITS also grows quickly whereas
Revised: SingleRes grow rapidly with very steep slopes, and that of BRITS also grows quickly whereas

Original: TIDER needs much less memory.
Revised: TIDER needs much less memory.

Original: Similarly, Figure 3-(b) shows that the memory usage of NAOMI ,
Revised: Similarly, Figure 3-(b) shows that the memory usage of NAOMI ,

Original: SingleRes , and SAITS also grows fast as T increases.
Revised: SingleRes , and SAITS also grows fast as T increases.

Original: Again, TIDER needs the least amount of memory.
Revised: Again, TIDER needs the least amount of memory.

Original: There is also a gap between the curves of CSDI ( BRITS ) and TIDER in Figure 3, which is overwhelmed by the magnitude of NAOMI and SingleRes.
Revised: There is also a gap between the curves of CSDI ( BRITS ) and TIDER in Figure 3, which is overwhelmed by the magnitude of NAOMI and SingleRes.

Original: We present these gaps in
Revised: We present these gaps in

Original: Section A.9.
Revised: Section A.9.

Original: Lastly, Figure 3-(c) demonstrates the running time of different methods for 100 epochs when processing a 100 × 100 matrix.
Revised: Lastly, Figure 3-(c) demonstrates the running time of different methods for 100 epochs when processing a 100 × 100 matrix.

Original: TIDER runs much faster than RNNs-based methods.
Revised: TIDER runs much faster than RNNs-based methods.

Original: Notably,
Revised: Notably,

Original: TIDER outperforms BRITS by almost an order of magnitude.
Revised: TIDER outperforms BRITS by almost an order of magnitude.

Original: This is also true for the total time taken by entire training process.
Revised: This is also true for the total time taken by entire training process.

Original: Furthermore, the space complexity of BRITS is O ( N ( N + T ))
Revised: Furthermore, the space complexity of BRITS is O ( N ( N + T ))

Original: (Vaswani et al., 2017) while that of TIDER is O ( N + T ) .
Revised: (Vaswani et al., 2017) while that of TIDER is O ( N + T ) .

Original: The complexity of NAOMI was not established and is tricky to analyze due to its complex divide-and-conquer strategy, but its complicated procedure yields more intermediate variables and thus results in more memory consumption, as empirically validated by our experiments.
Revised: The complexity of NAOMI was not established and is tricky to analyze due to its complex divide-and-conquer strategy, but its complicated procedure yields [Table] CSDI 4. OOM

Revised: OOM

Revised: OOM 1.

Original: In conclusion, this experiment demonstrates the scalability of our model.
Revised: NAOMI 5. OOM OOM OOM

Revised: OOM

Revised: OOM

Revised: OOM

Revised: SingleRes 6.

Revised: OOM

Revised: OOM

Revised: OOM

Revised: OOM

Revised: OOM

Revised: OOM

Revised: TIDER (no W) 4.

Revised: TIDER 4.

Revised: Improvement (%) 2.

Revised: [Figure]

Revised: more intermediate variables and thus results in more memory consumption, as empirically validated by our experiments.

Revised: In conclusion, this experiment demonstrates the scalability of our model.

Original: 5. A BLATION A NALYSIS
Revised: 5. A BLATION A NALYSIS

Original: To verify the effectiveness of our proposed disentangled representation matrices, we conduct an ablation study on Guangzhou dataset by removing one of the trend, seasonality, and bias representation matrices while leaving the rest of the model unchanged.
Revised: To verify the effectiveness of our proposed disentangled representation matrices, we conduct an ablation study on Guangzhou dataset by removing one of the trend, seasonality, and bias representation matrices while leaving the rest of the model unchanged.

Original: The ablative results with missing rate r = 0 .
Revised: The ablative results with missing rate r = 0 .

Original: 2 (0 . 4) are presented in Table 3.
Revised: 2 (0 . 4) are presented in Table 3.

Original: We find that the model performance drops no matter which representation is removed, which validates that all of our proposed disentangled representations play an important role in imputation and jointly enhance the performance of the final model. 
Revised: We find that the model performance drops no matter which representation is removed, which validates that all of our proposed disentangled representations play an important role in imputation and jointly enhance the performance of the final model. [Table] [Figure]

Original: 5. C ASE S TUDY OF D ISENTANGLEMENT
Revised: 5. C ASE S TUDY OF D ISENTANGLEMENT

Original: This section is to demonstrate that the learned representations achieve disentanglement and offer explanability.
Revised: This section is to demonstrate that the learned representations achieve disentanglement and offer explanability.

Original: We first visualize a time-series from Guangzhou dataset and its learnt patterns, namely, trend patterns u ⊤ V t , seasonality patterns u ⊤ V s , and bias patterns 1 ⊤ V b , where u ⊤ is the time-series representation (a row of U ).
Revised: We first visualize a time-series from Guangzhou dataset and its learnt patterns, namely, trend patterns u ⊤ V t , seasonality patterns u ⊤ V s , and bias patterns 1 ⊤ V b , where u ⊤ is the time-series representation (a row of U ).

Original: Figure 4 shows that the learned representations generate semantically meaningful patterns.
Revised: Figure 4 shows that the learned representations generate semantically meaningful patterns.

Original: Figure 4-(b) shows a gradually changing curve that corresponds to trend patterns, whereas Figure 4-(c) exhibits clear periodicity, and Figure 4-(d) manifests more idiosyncratic behaviors.
Revised: Figure 4-(b) shows a gradually changing curve that corresponds to trend patterns, whereas Figure 4-(c) exhibits clear periodicity, and Figure 4-(d) manifests more idiosyncratic behaviors.

Original: Each of the displayed patterns captures information from a particular semantic perspective, which verifies that the learned representations V t , V s , V b exhibit independence and disentanglement.
Revised: Each of the displayed patterns captures information from a particular semantic perspective, which verifies that the learned representations V t , V s , V b exhibit independence and disentanglement.

Original: [Figure]

Original: In order to further show TIDER ’s interpretability, we conduct experiments on synthetic time-series where one pair of time-series are composed of the same trend but different seasonality, while another pair of time-series are composed of the same seasonality but a different trend.
Revised: In order to further show TIDER ’s interpretability, we conduct experiments on synthetic time-series where one pair of time-series are composed of the same trend but different seasonality, while another pair of time-series are composed of the same seasonality but a different trend.

Original: We visualize the ground truth trend and seasonality patterns, together with the learned ones.
Revised: We visualize the ground truth trend and seasonality patterns, together with the learned ones.

Original: Their results are depicted in
Revised: Their results are depicted in

Original: Figure 5 and Figure 6.
Revised: Figure 5 and Figure 6.

Original: We can clearly find that the learned disentangled components are very similar to the ground truth, while the disentangled patterns that are supposed to be close to each other indeed look similar (similar slopes on the trend components, together with similar amplitudes and periods on the seasonality components).
Revised: We can clearly find that the learned disentangled components are very similar to the ground truth, while the disentangled patterns that are supposed to be close to each other indeed look similar (similar slopes on the trend components, together with similar amplitudes and periods on the seasonality components).

Original: This further supports TIDER ’s explainability of disentanglement.
Revised: This further supports TIDER ’s explainability of disentanglement.

Original: 5. H YPERPARAMETER S ENSITIVITY
Revised: 5. H YPERPARAMETER S ENSITIVITY

Original: We present the performance ofTIDER varies against four key hyperparameters  D , K , D d , and P , in Figure 7, 8, 9, and 10, respectively.
Revised: We analyze the impact of four key hyperparameters ( D , K , D d , and P ) on performance of TIDER, and present the results in Figure 7, 8, 9, and 10, respectively.

Original: From these plots, we can observe that TIDER is relatively stable under different hyperparameter settings.
Revised: We observe that TIDER is relatively stable under different hyperparameter settings.

Original: Even for hyperparameter P (period), TIDER ’s imputation performance is also quite stable (although the interpretability might be damaged under undesirable ones).
Revised: Even for hyperparameter P (period), TIDER exhibits consistent imputation performance, although interpretability may be compromised.

Original: Therefore, ifour target is merely for accurate imputation, we might not need to pay much effort or spend much time on hyperparameter tuning, since TIDER is stable among different hyperparameter settings.
Revised: Therefore, if accurate imputation is the primary objective, extensive hyperparameter tuning may not be necessary, as TIDER is robust to variations in hyperparameters.

Original: If we target both accuracy and interpretability, then the only hyperparameter we need to figure out is the period of time series.
Revised: However, when both accuracy and interpretability are important, determining the appropriate time series period is critical.

Original: Thisoften can  be acquired by prior knowledge or other periodic detection models (Wen et al., 2021).
Revised: This can often be achieved through prior knowledge or other periodic detection models (Fan et al., 2022; Wang et al., 2022).

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: In this paper, we propose a scalable multivariate time series imputation method, TIDER , with multiple novel inductive biases under the framework of low-rank matrix factorization.
Revised: In this paper, we propose a scalable multivariate time series imputation method, TIDER , with multiple novel inductive biases under the framework of low-rank matrix factorization.

Original: In contrast to existing imputation approaches, TIDER adopts semantically meaningful disentangled representations to account for multiple factors of a time series.
Revised: In contrast to existing imputation approaches, TIDER adopts semantically meaningful disentangled representations to account for multiple factors of a time series.

Original: In particular, it enables capturing periodicity with a novel Fourier basis-based representation and allows to identify local time variation with a time bias representation.
Revised: In particular, it enables capturing periodicity with a novel Fourier basis-based representation and allows to identify local time variation with a time bias representation.

Original: TIDER ’s superiority is verified by experimental results.
Revised: TIDER ’s superiority is verified by experimental results.

Original: Moreover, it scales well to long time series where the existing methods struggle or even cannot fit.
Revised: Moreover, it scales well to long time series where the existing methods struggle or even cannot fit.

Original: In the future, it is interesting to investigate how to use the meaningful disentangled representations for forecasting tasks and how to design constraints on U when channel information is available.
Revised: In the future, it is interesting to investigate how to use the meaningful disentangled representations for forecasting tasks and how to design constraints on U when channel information is available.

Original: Furthermore, since now we use hyperparameter tuning techniques to decide P in V s , how to obtain P adaptively in a data-driven way is also worth further investigation.
Revised: Furthermore, since now we use hyperparameter tuning techniques to decide P in V s , how to obtain P adaptively in a data-driven way is also worth further investigation.

Original:  REPRODUCIBILITY STATEMENT
Revised:  REPRODUCIBILITY STATEMENT

Original: We provide an open-source implementation of our proposed model, TIDER , at https:// anonymous.4open.science/r/TIDER-527C .
Revised: We provide an open-source implementation of our proposed model, TIDER , at https:// anonymous.4open.science/r/TIDER-527C .

Original: Hyperparameter setting is shown in Section A.8.
Revised: Hyperparameter setting is shown in Section A.8.

Original: Users can download the code and run TIDER easily.
Revised: Users can download the code and run TIDER easily.

Original:  A A PPENDIX A. P RELIMINARY A.1. L OW - RANK M ATRIX F ACTORIZATION
Revised:  A CKNOWLEDGEMENT

Revised: This study is supported under the RIE2020 Industry Alignment Fund – Industry Collaboration Projects (IAF-ICP) Funding Initiative, as well as cash and in-kind contribution from Singapore

Original: Given a matrix X ∈ R N × T with rank k ≪ min { N
Revised: Telecommunications Limited (Singtel), through Singtel Cognitive and Artificial Intelligence Lab for Enterprises (SCALE@NTU). This research/project is also supported by the National Research Foundation, Singapore under its AI Singapore Programme (AISG Award No: AISG2-TC-2021-001).

Revised: Xiucheng Li is supported in part by the National Natural Science Foundation of China under Grant No.

Original:  T }  the low-rank matrix factorization aims to factorize X as the product of two low-rank matrices U ∈ R N × k and V ∈ R k × T ,
Revised: 62206074, Shenzhen College Stability Support Plan under Grant No. GXWD20220811173233001. The main part of his job is done when he is in Nanyang Technological University. A A PPENDIX A. P RELIMINARY A.1. L OW - RANK M ATRIX F ACTORIZATION Given a matrix X ∈ R N × T with rank k ≪ min { N, T } , the low-rank matrix factorization aims to factorize X as the product of two low-rank matrices U ∈ R N × k and V ∈ R k × T ,

Original: X = UV .
Revised: X = UV .

Revised: [Table]

Original: This low-rank matrix factorization can be extended to the circumstance where the true rank of X is larger than k .
Revised: This low-rank matrix factorization can be extended to the circumstance where the true rank of X is larger than k .

Original: In such a case, we use a metric ℓ to measure the discrepancy between X and the approximated product UV as
Revised: In such a case, we use a metric ℓ to measure the discrepancy between X and the approximated product UV as

Original: [Equation]
Revised: [Equation]

Original: where ∥ · ∥ denotes the Frobenius norm of a matrix.
Revised: where ∥ · ∥ denotes the Frobenius norm of a matrix.

Original: This low-rank approximation is fairly reasonable in practice, since many real-world structural observation matrices are low rank.
Revised: This low-rank approximation is fairly reasonable in practice, since many real-world structural observation matrices are low rank.

Original: This has been widely adopted and extensively studied in recommender systems (Takács et al., 2008; Chen et al., 2020).
Revised: This has been widely adopted and extensively studied in recommender systems (Takács et al., 2008; Chen et al., 2020).

Original: In the context of multivariate time series data, we propose to adopt low-rank matrix factorization for two reasons: 1) univariate time series channels (the rows of X ) are usually highly correlated; 2) observations at different time steps (the columns of X ) exhibit strong dependencies.
Revised: In the context of multivariate time series data, we propose to adopt low-rank matrix factorization for two reasons: 1) univariate time series channels (the rows of X ) are usually highly correlated; 2) observations at different time steps (the columns of X ) exhibit strong dependencies.

Original: A.1. F OURIER S ERIES
Revised: A.1. F OURIER S ERIES

Original: Fourier analysis(Stein & Shakarchi, 2011) states that any real-valued periodic function f ( x ) can be represented by an infinite series of sinusoidal functions, known as the Fourier series.
Revised: Fourier analysis(Stein & Shakarchi, 2011) states that any real-valued periodic function f ( x ) can be represented by an infinite series of sinusoidal functions, known as the Fourier series.

Original: These sinusoidal functions have their own coefficients and distinct frequencies.
Revised: These sinusoidal functions have their own coefficients and distinct frequencies.

Original: More formally, the Fourier series for a function f ( x ) with its period P can be described as
Revised: More formally, the Fourier series for a function f ( x ) with its period P can be described as

Original: [Equation]
Revised: [Equation]

Original: where ω = 2 π/P , and a n , b n ∈ R are the corresponding coefficients.
Revised: where ω = 2 π/P , and a n , b n ∈ R are the corresponding coefficients.

Original: Since sin( nωx ) and cos( nωx ) functions are orthogonal to each other and are able to span over the entire functional space, they are also called Fourier basis.[Table]
Revised: Since sin( nωx ) and cos( nωx ) functions are orthogonal to each other and are able to span over the entire functional space, they are also called Fourier basis.

Original: A. D ATASET D ETAILS
Revised: A. D ATASET D ETAILS

Original: • Guangzhou Traffic Data.
Revised: • Guangzhou Traffic Data.

Original: This dataset(Chen et al., 2018) contains traffic speed ofanonymous urban road segments for 5 days with a 10-minute sampling rate in Guangzhou, China.
Revised: This dataset(Chen et al., 2018) contains traffic speed ofanonymous urban road segments for 5 days with a 10-minute sampling rate in Guangzhou, China.

Original: It results in a 214 × 500 multivariate time series matrix.
Revised: It results in a 214 × 500 multivariate time series matrix.

Original: • Solar-Energy Production Data.
Revised: • Solar-Energy Production Data.

Original: This dataset consists of solar power production records of 137 PV plants in Alabama, USA sampled every 10 minutes.
Revised: This dataset consists of solar power production records of 137 PV plants in Alabama, USA sampled every 10 minutes.

Original: It results in a 137 × 52 ,data matrix.
Revised: It results in a 137 × 52 ,data matrix.

Original: • Westminster Uber Movement Data.
Revised: • Westminster Uber Movement Data.

Original: This dataset contains hourly averaged speed of road segments in Westminster, in Jan 2020, released by Uber.
Revised: This dataset contains hourly averaged speed of road segments in Westminster, in Jan 2020, released by Uber.

Original: Dimension of its data matrix is 7 , 489 × 744 .
Revised: Dimension of its data matrix is 7 , 489 × 744 .

Original: The dataset statistics are summarized in Table 4.
Revised: The dataset statistics are summarized in Table 4.

Original: A. D ETAILS OF B ASELINE M ODELS
Revised: A. D ETAILS OF B ASELINE M ODELS

Original: The details of baseline methods are briefly summarized as follows.
Revised: The details of baseline methods are briefly summarized as follows.

Original: For SimpleMean , KNN , and
Revised: For SimpleMean , KNN , and

Original: SoftImpute , we use their implementation provided by the package fancyimpute whereas for BRITS ,
Revised: SoftImpute , we use their implementation provided by the package fancyimpute whereas for BRITS ,

Original: GAIN , SAITS , CSDI , NAOMI and SingleRes , we use the source codes released by their authors.
Revised: GAIN , SAITS , CSDI , NAOMI and SingleRes , we use the source codes released by their authors.

Original: • SimpleMean (Acuna & Rodriguez, 2004) 

Revised: • SimpleMean (Acuna & Rodriguez, 2004)

Original: It imputes missing entries with mean values of corresponding columns.
Revised: It imputes missing entries with mean values of corresponding columns.

Original: • KNN (Batista et al., 2002) 

Revised: • KNN (Batista et al., 2002)

Original: It first finds k nearest rows with the highest similarity score to the target row, and then uses the weighted sum of these k rows for imputation.
Revised: It first finds k nearest rows with the highest similarity score to the target row, and then uses the weighted sum of these k rows for imputation.

Original: • SoftImpute (Mazumder et al., 2010) 6 .
Revised: • SoftImpute (Mazumder et al., 2010) 6 .

Original: It is a matrix completion approach based on iterative soft thresholding of Singular Value Decomposition (SVD).
Revised: It is a matrix completion approach based on iterative soft thresholding of Singular Value Decomposition (SVD).

Original: • MICE (Azur et al., 2011) 7 .
Revised: • MICE (Azur et al., 2011) 7 .

Original: It is the abbreviation for Multivariate Imputation by Chained Equations, a widely-used R package for imputation.
Revised: It is the abbreviation for Multivariate Imputation by Chained Equations, a widely-used R package for imputation.

Original: • MF (Takács et al., 2008).
Revised: • MF (Takács et al., 2008).

Original: It applies low-rank matrix factorization without any constraint on the latent factors.
Revised: It applies low-rank matrix factorization without any constraint on the latent factors.

Original: • MF+L2 (Takács et al., 2008).
Revised: • MF+L2 (Takács et al., 2008).

Original: It applies low-rank matrix factorization with L2 regularization on U and V .
Revised: It applies low-rank matrix factorization with L2 regularization on U and V .

Original: • TRMF
Revised: • TRMF

Original: (Yu et al., 2016).
Revised: (Yu et al., 2016).

Original: It applies low-rank matrix factorization with an autoregressive constraint imposed on temporal matrix V .
Revised: It applies low-rank matrix factorization with an autoregressive constraint imposed on temporal matrix V .

Original: • BRITS (Cao et al., 2018) 8 .
Revised: • BRITS (Cao et al., 2018) 8 .

Original: It is a time series imputation method based on Bidirectional Recurrent Neural Nets and a time decay mechanism.
Revised: It is a time series imputation method based on Bidirectional Recurrent Neural Nets and a time decay mechanism.

Original: • GAIN (Yoon et al., 2018) 9 .
Revised: • GAIN (Yoon et al., 2018) 9 .

Original: It employs Generative Adversarial Nets (GAN) for imputation.
Revised: It employs Generative Adversarial Nets (GAN) for imputation.

Original: • SAITS (Du et al., 2022) 10 .
Revised: • SAITS (Du et al., 2022) 10 .

Original: It imputes missing values based on self-attention mechanism.
Revised: It imputes missing values based on self-attention mechanism.

Original: [Figure]

Original: • CSDI (Tashiro et al., 2021) 11 .
Revised: CSDI (Tashiro et al., 2021) 11 .

Original: It utilizes score-based diffusion models to explore correlations between observed values and for imputation.
Revised: It utilizes score-based diffusion models to explore correlations between observed values and for imputation.

Original: • NAOMI
Revised: • NAOMI

Original: (Liu et al., 2019) 12 .
Revised: (Liu et al., 2019) 12 .

Original: It combines Bidirectional Recurrent Neural Nets with adversarial training to offer non-autoregressive style imputation.
Revised: It combines Bidirectional Recurrent Neural Nets with adversarial training to offer non-autoregressive style imputation.

Original: The divide-and-conquer strategy is adopted.
Revised: The divide-and-conquer strategy is adopted.

Original: • SingleRes (Liu et al., 2019) 13 .
Revised: • SingleRes (Liu et al., 2019) 13 .

Original: It is the autoregressive counterpart of NAOMI .
Revised: It is the autoregressive counterpart of NAOMI .

Original: In our experiments, we find that BRITS with the suggested batch size and RNN hidden size will result in OOM (Out of Memory) on the Westminster dataset.
Revised: In our experiments, we find that BRITS with the suggested batch size and RNN hidden size will result in OOM (Out of Memory) on the Westminster dataset.

Original: Thus we reduce the batch size to 1 and the dimension of RNN to 10 such that it could just fit the 32 GB GPU on that dataset, we use an asterisk symbol to indicate its results in Table 1 and Table 2, in which the Improvement is calculated as follows,
Revised: Thus we reduce the batch size to 1 and the dimension of RNN to 10 such that it could just fit the 32 GB GPU on that dataset, we use an asterisk symbol to indicate its results in Table 1 and Table 2, in which the Improvement is calculated as follows,

Original: [Equation]
Revised: [Equation]

Original: where best _ baseline represents the best performance among all the compared baseline models,
Revised: where best _ baseline represents the best performance among all the compared baseline models,

Original: TIDER ∗ stands for the better one between TIDER and TIDER (no W).
Revised: TIDER ∗ stands for the better one between TIDER and TIDER (no W).

Original: A. D ETAILS OF M ETRICS
Revised: A. D ETAILS OF M ETRICS

Original: Root Mean Square Error (RMSE), Mean Absolute Error (MAE), and Mean Absolute Percentage
Revised: Root Mean Square Error (RMSE), Mean Absolute Error (MAE), and Mean Absolute Percentage

Original: Error (MAPE) are adopted to evaluate the imputation accuracy of all compared methods.
Revised: Error (MAPE) are adopted to evaluate the imputation accuracy of all compared methods.

Original: These three 11 https://github.com/ermongroup/CSDIhttps://github.com/felixykliu/NAOMIhttps://github.com/felixykliu/NAOMI

Original: [Figure]

Original:  metrics are defined as
Revised: These three metrics are defined as

Original: [Equation]
Revised: [Equation]

Original: where X ij denotes the ground-truth values, ˆ X ij is the imputed values, and Ω is the index set of missing entries to be evaluated. 
Revised: where X ij denotes the ground-truth values, ˆ X ij is the imputed values, and Ω is the index set of missing entries to be evaluated. 5 https:

Revised: //github.com/iskandr/fancyimputehttps://github.com/iskandr/fancyimpute 7 https

Revised: //github.com/amices/mice 8 https://github.com/caow13/BRITShttps://github.com/jsyoon0823/GAINhttps://github.com/WenjieDu/SAITShttps://github.com/ermongroup/CSDIhttps://github.com/felixykliu/NAOMIhttps://github.com/felixykliu/NAOMI

Original:  A. A DDITIONAL E XPERIMENTS FOR I MPUTATION A CCURACY C OMPARISON ON L ONG T IME S ERIES
Revised: [Figure] [Figure] [Table] A. A DDITIONAL E XPERIMENTS FOR I MPUTATION A CCURACY C OMPARISON ON L ONG T IME S ERIES

Original: As Table 1 and Table 2 shows, TIDER outperforms the baseline models by large margins on long time-series dataset Solar-Energy.
Revised: As Table 1 and Table 2 shows, TIDER outperforms the baseline models by large margins on long time-series dataset Solar-Energy.

Original: In order to further verify its performance stability in imputing long multivariate time-series data, we conduct additional experiments on another two long time-series:
Revised: In order to further verify its performance stability in imputing long multivariate time-series data, we conduct additional experiments on another two long time-series:

Original: • HouseHold Power dataset 14 .
Revised: • HouseHold Power dataset 14 .

Original: This dataset contains every minute’s household electric consumption measurements gathered in a house located in Sceaux from December  to November 2010.
Revised: This dataset contains every minute’s household electric consumption measurements gathered in a house located in Sceaux from December 2006 to November 2010.

Original: The size of this long time-series matrix is 7 × 2075259 .
Revised: The size of this long time-series matrix is 7 × 2075259 .

Original: • Beijing Climate dataset 15 .
Revised: • Beijing Climate dataset 15 .

Original: This dataset records the PM2.5 concentration, dew point, temperature, pressure, combined wind direction, cumulative wind speed, cumulative hours of snow, and cumulative hours of rain in Beijing, from Jan 1st, 2010 to Dec 31st, 2014.
Revised: This dataset records the PM2.5 concentration, dew point, temperature, pressure, combined wind direction, cumulative wind speed, cumulative hours of snow, and cumulative hours of rain in Beijing, from Jan 1st, 2010 to Dec 31st, 2014.

Original: The size of this long time-series matrix is 7 × 43824 .
Revised: The size of this long time-series matrix is 7 × 43824 .

Original: The imputation accuracy of all models on these two long time series is shown in Table 5 and Table 6.
Revised: The imputation accuracy of all models on these two long time series is shown in Table 5 and Table 6.

Original: Similar to the observation made in Section 5.2, many deep-learning-based models that perform well in short time-series dataset Guangzhou (eg, CSDI, SAITS) run into out of memory on these two datasets, while our proposed method can easily handle them and achieve significantly better performance.[Table]
Revised: Similar to the observation made in Section 5.2, many deep-learning-based models that perform well in short time-series dataset Guangzhou (eg, CSDI, SAITS) run into out of memory on these two datasets, while our proposed method can easily handle them and achieve significantly better performance.

Original: [Table] These results further support the scalability of our model [Table] as well as its effectiveness in imputing long time-series datasets.
Revised: These results further support the scalability of our model, as well as its effectiveness in imputing long time-series datasets.

Original: A. D ISENTANGLEMENT V ALIDATION O N S YNTHETIC D ATA S ET
Revised: A. D ISENTANGLEMENT V ALIDATION O N S YNTHETIC D ATA S ET

Original: In order to further show TIDER ’s interpretability, we conduct experiments on synthetic time-series where one pair of time-series are composed of the same trend but different seasonality, while another pair of time-series are composed of the same seasonality but a different trend.
Revised: In order to further show TIDER ’s interpretability, we conduct experiments on synthetic time-series where one pair of time-series are composed of the same trend but different seasonality, while another pair of time-series are composed of the same seasonality but a different trend.

Original: We visualize the ground truth trend and seasonality patterns, together with the learned ones.
Revised: We visualize the ground truth trend and seasonality patterns, together with the learned ones.

Original: Their results are depicted in
Revised: Their results are depicted in

Original: Figure 5 and Figure 6.
Revised: Figure 5 and Figure 6.

Original: We can clearly find that the learned disentangled components are very similar to the ground truth, while the disentangled patterns that are supposed to be close to each other indeed look similar (similar slopes on the trend components, together with similar amplitudes and periods on the seasonality components).
Revised: We can clearly find that the learned disentangled components are very similar to the ground truth, while the disentangled patterns that are supposed to be close to each other indeed look similar (similar slopes on the trend components, together with similar amplitudes and periods on the seasonality components).

Original: This further supports TIDER ’s explainability of disentanglement.
Revised: This further supports TIDER ’s explainability of disentanglement.

Original: [Figure] [Figure]
Revised: [Table] [Figure]

Original: [Figure] [Figure] 
Revised: [Figure] [Figure] [Figure]

Original: [Table]
Revised: [Table]

Original: A. H YPERPARAMETER S ENSITIVITY
Revised: A. H YPERPARAMETER S ENSITIVITY

Original: In this section
Revised: In this section

Original: [Table]
Revised: [Table]

Original: we study the performance change of TIDER under different hyperparameter settings.
Revised: we study the performance change of TIDER under different hyperparameter settings.

Original: Figure 7 shows TIDER ’s hyperparameter sensitivity under different D .
Revised: Figure 7 shows TIDER ’s hyperparameter sensitivity under different D .

Original: Figure 8 draws the performance of TIDER changing against K , the number of sinusoidal waves in V s .
Revised: Figure 8 draws the performance of TIDER changing against K , the number of sinusoidal waves in V s .

Original: Figure 7 depicts the accuracy curves of TIDER varying over different D d .
Revised: Figure 7 depicts the accuracy curves of TIDER varying over different D d .

Original: Figure 10 is used to study the performance stability of TIDER under different periods in V s .
Revised: Figure 10 is used to study the performance stability of TIDER under different periods in V s .

Original: It can be seen that TIDER is rather stable under different hyperparameter settings.
Revised: It can be seen that TIDER is rather stable under different hyperparameter settings.

Original: Therefore, if our target is merely for accurate imputation, we might not need to pay much effort or spend much time on hyperparameter tuning, since TIDER is stable among different hyperparameter settings.
Revised: Therefore, if our target is merely for accurate imputation, we might not need to pay much effort or spend much time on hyperparameter tuning, since TIDER is stable among different hyperparameter settings.

Original: If we target both accuracy and interpretability, then the only hyperparameter we need to figure out is the period of time series.
Revised: If we target both accuracy and interpretability, then the only hyperparameter we need to figure out is the period of time series.

Original: This often can be acquired by our prior knowledge or by other periodic detection models Wen et al.
Revised: This often can be acquired by our prior knowledge or by other periodic detection models Wen et al.

Original: A. H YPERPARAMETER S ETTING
Revised: A. H YPERPARAMETER S ETTING

Original: We implement TIDER using Python 3.6 and Pytorch 1.9, and optimize the model parameters using
Revised: We implement TIDER using Python 3.6 and Pytorch 1.9, and optimize the model parameters using

Original: Adam (Kingma & Ba, 2014) with a learning rate 1 e − 3 .
Revised: Adam (Kingma & Ba, 2014) with a learning rate 1 e − 3 .

Original: We use gird search to select the optimal hyperparameters ( D , K , D d , P ) on the validation datasets.
Revised: We use gird search to select the optimal hyperparameters ( D , K , D d , P ) on the validation datasets.

Original: However, as we have observed in
Revised: However, as we have observed in

Original: Section 5.6 and Section A.8, the imputation performance of TIDER is relatively steady among different hyperparameter settings.
Revised: Section 5.6 and Section A.8, the imputation performance of TIDER is relatively steady among different hyperparameter settings.

Original: Thus other hyperparameter sets might also offer desirable results.
Revised: Thus other hyperparameter sets might also offer desirable results.

Revised: [Figure]

Original: The general principle and guidelines on parameter setting are as follows:
Revised: The general principle and guidelines on parameter setting are as follows:

Original: • ω :
Revised: • ω :

Original: This hyperparameter is involved in the seasonality representation matrix.
Revised: This hyperparameter is involved in the seasonality representation matrix.

Original: ω is in close relation with the time-series period P .
Revised: ω is in close relation with the time-series period P .

Original: At present, P is chosen by jointly using our prior knowledge of different time series and hyperparameter tuning techniques.
Revised: At present, P is chosen by jointly using our prior knowledge of different time series and hyperparameter tuning techniques.

Original: We first construct a candidate list of P based on our knowledge of these time series and select the best one according to performance on validation datasets.
Revised: We first construct a candidate list of P based on our knowledge of these time series and select the best one according to performance on validation datasets.

Original: We set P for Guangzhou, Solar-Energy, and Westminster datasets as 168, 168, and 24 respectively.
Revised: We set P for Guangzhou, Solar-Energy, and Westminster datasets as 168, 168, and 24 respectively.

Original: In other words, the optimal ω for these datasets are π 84 , π 84 and π 12 .
Revised: In other words, the optimal ω for these datasets are π 84 , π 84 and π 12 .

Original: • D : Dimension for matrix U and V t is also an important hyperparameter.
Revised: • D : Dimension for matrix U and V t is also an important hyperparameter.

Original: Small D will lack the ability to learn enough information while large D is prone to overfitting.
Revised: Small D will lack the ability to learn enough information while large D is prone to overfitting.

Original: [Figure] • λ t , λ b : These are the corresponding weights for the constraint functions of the trend representation matrix and bias representation matrix, respectively.
Revised: • λ t , λ b : These are the corresponding weights for the constraint functions of the trend representation matrix and bias representation matrix, respectively.

Original: We tune them by making a choice from a scale set { 0 . 01 , 0 . 1 , 0 . 2 , 0 .
Revised: We tune them by making a choice from a scale set { 0 . 01 , 0 . 1 , 0 . 2 , 0 .

Original: By following this principle, the optimal hyperparameters we obtained are listed in Table 7.
Revised: By following this principle, the optimal hyperparameters we obtained are listed in Table 7.

Original: A. S UPPLEMENTAL S CALABILITY A NALYSIS
Revised: A. S UPPLEMENTAL S CALABILITY A NALYSIS

Original: In Section 5.3, there is a little gap between memory usage of CSDI and TIDER in Figure 3(a), and little gap between memory usage of BRITS and TIDER in Figure 3(b).
Revised: In Section 5.3, there is a little gap between memory usage of CSDI and TIDER in Figure 3(a), and little gap between memory usage of BRITS and TIDER in Figure 3(b).

Original: In Figure 3-(a) (resp.
Revised: In Figure 3-(a) (resp.

Original: Figure 3-(b))
Revised: Figure 3-(b))

Original: the memory usage gap between CSDI (resp.
Revised: the memory usage gap between CSDI (resp.

Original: BRITS ) and TIDER is overwhelmed by the large magnitude of other methods.
Revised: BRITS ) and TIDER is overwhelmed by the large magnitude of other methods.

Original: Thus, we zoom in to give a comparison in this section.
Revised: Thus, we zoom in to give a comparison in this section.

Original: As depicted in Figure 11, there are still large memory usage gaps between TIDER and them, which demonstrates the scalability superiority of our proposed TIDER .
Revised: As depicted in Figure 11, there are still large memory usage gaps between TIDER and them, which demonstrates the scalability superiority of our proposed TIDER .