SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Abdullah Alomar
aalomar@mit.edu
Munther Dahleh
dahleh@mit.edu
Sean Mann
seanmann@mit.edu
The well-established practice of time series analysis involves estimating deterministic, non-stationary trend and seasonality components followed by learning the residual stochastic, stationary components. Recently, it has been shown that one can learn the deterministic non-stationary components accurately using multivariate Singular Spectrum Analysis (mSSA) in the absence of a correlated stationary component; meanwhile, in the absence of deterministic non-stationary components, the Autoregressive (AR) stationary component can also be learnt readily, e.g. via Ordinary Least Squares (OLS). However, a theoretical underpinning of multi-stage learning algorithms involving both deterministic and stationary components has been absent in the literature despite its pervasiveness. We resolve this open question by establishing desirable theoretical guarantees for a natural two-stage algorithm, where mSSA is first applied to estimate the non-stationary components despite the presence of a correlated stationary AR component, which is subsequently learned from the residual time series. We provide a finite-sample forecasting consistency bound for the proposed algorithm, SAMoSSA, which is data-driven and thus requires minimal parameter tuning. To establish theoretical guarantees, we overcome three hurdles: (i) we characterize the spectra of Page matrices of stable AR processes, thus extending the analysis of mSSA; (ii) we extend the analysis of AR process identification in the presence of arbitrary bounded perturbations; (iii) we characterize the out-of-sample or forecasting error, as opposed to solely considering model identification. Through representative empirical studies, we validate the superior performance of SAMoSSA compared to existing baselines. Notably, SAMoSSA's ability to account for AR noise structure yields improvements ranging from  $5\%$  to  $37\%$  across various benchmark datasets.
Background. Multivariate time series have often been modeled as mixtures of stationary stochastic processes (e.g. AR process) and deterministic non-stationary components (e.g. polynomial and harmonics). To handle such mixtures, classical time series forecasting algorithms first attempt to estimate and then remove the non-stationary components. For example, before fitting an Autoregressive Moving-average (ARMA) model, polynomial trend and seasonal components must be estimated and then removed from the time series. Once the non-stationary components have been eliminated<sup>1</sup>, the ARMA model is learned. This estimation procedure often relies on domain knowledge and/or fine-tuning, and theoretical analysis for such multiple-stage algorithms is limited in the literature.
Prior work presented a solution for estimating non-stationary deterministic components without domain knowledge or fine-tuning using mSSA [3]. This framework systematically models a wide
<sup>1</sup>Note that Autoregressive Integrated Moving-average (ARIMA) and Seasonal ARIMA models use differencing in conjunction with unit root tests to remove stochastic non-stationary components, and are not suited for the model we consider.
class of (deterministic) non-stationary multivariate time series as linear recurrent formulae (LRF), encompassing a wide class of spatio-temporal factor models that includes harmonics, exponentials, and polynomials. However, mSSA, both algorithmically and theoretically, does not handle additive correlated stationary noise, an important noise structure in time series analysis. Indeed, all theoretical results of mSSA are established in the noiseless setting [16] or under the assumption of independent and identically distributed (i.i.d.) noise [4, 3].
On the other hand, every stable stationary process can be approximated as a finite order AR process [8, 26]. The classical OLS procedure has been shown to accurately learn finite order AR processes [17]. However, the feasibility of identifying AR processes in the presence of a non-stationary deterministic component has not been addressed.
In summary, despite the pervasive practice of first estimating non-stationary deterministic components and then learning the stationary residual component, neither an elegant unified algorithm nor associated theoretical analyses have been put forward in the literature.
A step towards resolving these challenges is to answer the following questions: (i) Can mSSA consistently estimate non-stationary deterministic components in the presence of correlated stationary noise? (ii) Can the AR model be accurately identified using the residual time series, after removing the non-stationary deterministic components estimated by mSSA, potentially with errors? (iii) Can the out-of-sample or forecasting error of such a multi-stage algorithm be analyzed?
In this paper, we resolve all three questions in the affirmative: we present SAMoSSA, a two-stage procedure, where in the first stage, we apply mSSA on the observations to extract the non-stationary components; and in the second stage, the stationary AR process is learned using the residuals.
Setup. We consider the discrete time setting where we observe a multivariate time series  $Y(t) \coloneqq [y_{1}(t),\dots ,y_{N}(t)] \in \mathbb{R}^{N}$  at each time index  $t \in [T] \coloneqq \{1,\ldots ,T\}$  where  $T \geq N^2$ . For each  $n \in [N]$ , and each timestep  $t \in [T]$ , the observations take the form
$$
y _ {n} (t) = f _ {n} (t) + x _ {n} (t), \tag {1}
$$
where  $f_{n}:\mathbb{Z}^{+}\to \mathbb{R}$  denotes the non-stationary deterministic component, and  $x_{n}(t)$  is a stationary AR noise process of order  $p_n$  ( $\mathrm{AR}(p_n)$ ). Specifically, each  $x_{n}(t)$  is governed by
$$
x _ {n} (t) = \sum_ {i = 1} ^ {p _ {n}} \alpha_ {n i} x _ {n} (t - i) + \eta_ {n} (t), \tag {2}
$$
where  $\eta_{n}(t)$  refers to the per-step noise, modeled as mean-zero i.i.d. random variables, and  $\alpha_{ni} \in \mathbb{R} \forall i \in [p_n]$  are the parameters of the  $n$ -th  $\mathrm{AR}(p_n)$  process.
Goal. For each  $n \in [N]$ , our objective is threefold. The first is estimating  $f_{n}(t)$  from the noisy observations  $y_{n}(t)$  for all  $t \in [T]$ . The second is identifying the AR process' parameters  $\alpha_{ni} \forall i \in [p_n]$ . The third is out-of-sample forecasting of  $y_{n}(t)$  for  $t > T$ .
Contributions. The main contributions of this work is SAMoSSA, an elegant two-stage algorithm, which manages to learn both non-stationary deterministic and stationary stochastic components of the underlying time series. A detailed summary of the contributions is as follows.
(a) Estimating non-stationary component with mSSA under AR noise. The prior theoretical results for mSSA are established in the deterministic setting [16] or under the assumption of i.i.d. noise [4, 3]. In Theorem 4.1, we establish that the mean squared estimation error scales as  $\sim 1 / \sqrt{NT}$  under AR noise – the same rate was achieved by [3] under i.i.d. noise. Key to this result is establishing spectral properties of the “Page” matrix of AR processes, which may be of interest in its own right (see Lemma A.2).
(b) Estimating AR model parameters under bounded perturbation. We bound the estimation error for the OLS estimator of AR model parameters under arbitrary and bounded observation noise, which could be of independent interest. Our results build upon the recent work of [17] which derives similar results but without any observation noise. In that sense, ours can be viewed as a robust generalization of [17].
2if  $T <   N$  , we divide the  $N$  time series into  $\lceil N / T\rceil$  sets of time series where this condition will hold.
(c) Out-of-sample finite-sample consistency of SAMoSSA. We provide a finite-sample analysis for the forecasting error of SAMoSSA—such analysis of two-stage procedures in this setup is nascent despite its ubiquitous use in practice. Particularly, we establish in Theorem 4.4 that the out-of-sample forecasting error for SAMoSSA for the  $T$  time-steps ahead scales as  $\sim \frac{1}{T} + \frac{1}{\sqrt{NT}}$  with high probability.
(d) Empirical results. We demonstrate superior performance of SAMoSSA using both real-world and synthetic datasets. We show that accounting for the AR noise structure, as implemented in SAMoSSA, consistently enhances the forecasting performance compared to the baseline presented in [3]. Specifically, these enhancements range from  $5\%$  to  $37\%$  across benchmark datasets.
Related Work. Time series analysis is a well-developed field. We focus on two pertinent topics.
mSSA. Singular Spectrum Analysis (SSA), and its multivariate extension mSSA, are well-studied methods for time series analysis which have been used heavily in a wide array of problems including imputation [4, 3], forecasting [18, 2], and change point detection [22, 6]. Refer to [16, 15] for a good overview of the SSA literature. The classical SSA method consists of the following steps: (1) construct a Hankel matrix of the time series of interest; (2) apply Singular Value Decomposition (SVD) on the constructed Hankel matrix, (3) group the singular triplets to separate the different components of the time series; (4) learn a linear model for each component to forecast. mSSA is an extension of SSA which handles multiple time series simultaneously and attempts to exploit the shared structure between them [18]. The only difference between mSSA and SSA is in the first step, where Hankel matrices of individual series are "stacked" together to create a single stacked Hankel matrix. Despite its empirical success, mSSA's classical analysis has mostly focused on identifying which time series have a low-rank Hankel representation, and defining sufficient asymptotic conditions for signal extraction, i.e., when the various time series components are separable. All of the classical analysis mostly focus on the deterministic case, where no observation noise is present.
Recently, a variant of mSSA was introduced for the tasks of forecasting and imputation [3]. This variant, which we extend in this paper, uses the Page matrix representation instead of the Hankel matrix, which enables the authors to establish finite-sample bounds on the imputation and forecasting error. However, their work assumes the observation noise to be i.i.d., and does not accommodate correlated noise structure, which is often assumed in the time series literature [26]. Our work extends the analysis in Agarwal et al. [3] to observations under AR noise structure. We also extend the analysis of the forecasting error by studying how well we can learn (and forecast) the AR noise process with perturbed observations.
Estimating AR parameters. AR processes are ubiquitous and of interest in many fields, including time series analysis, control theory, and machine learning. In these fields, it is often the goal to estimate the parameters of an AR process from a sample trajectory. Estimation is often carried out through OLS, which is asymptotically optimal [11]. The asymptotic analysis of the OLS estimator is well established, and recently, given the newly developed statistical tools of high dimensional probability, cf. [31, 29], various recent works have tackled its finite-time analysis as well. For example, several works established results for the finite-time identification for general first order vector AR systems [20, 19, 27, 24, 21]. Further, González et al. [17] provides a finite-time bound on the deviation of the OLS estimate for general  $\mathrm{AR}(p)$  processes. To accommodate our setting, we extend the results of [17] in two ways. First, we extend the analysis to accommodate any sub-gaussian noise instead of assuming gaussianity; Second, and more importantly, we extend it to handle arbitrary bounded observation errors in the sampled trajectory, which can be of independent interest.
Deterministic Non-Stationary Component. We adopt the spatio-temporal factor model of [3, 6] described next. The spatio-temporal factor model holds when two assumptions are satisfied: the first assumption concerns the spatial structure, i.e., the structure across the  $N$  time series  $f_{1},\ldots ,f_{N}$ ; the second assumption pertains to the "temporal" structure. Before we describe the assumptions, we first define a key time series representation: the Page matrix.
Definition 2.1 (Page Matrix). Given a time series  $f: \mathbb{Z}^+ \to \mathbb{R}$ , and an initial time index  $t_0 > 0$ , the Page matrix representation over the  $T$  entries  $f(t_0), \ldots, f(t_0 + T - 1)$  with parameter  $1 \leq L \leq T$  is given by the matrix  $\mathbf{Z}(f, L, T, t_0) \in \mathbb{R}^{L \times [T / L]}$  with  $\mathbf{Z}(f, L, T, t_0)_{ij} = f(t_0 + i - 1 + (j - 1) \times L)$  for  $i \in [L]$ ,  $j \in [\lfloor T / L \rfloor]$ .
In words, to construct the  $L \times \lfloor T / L\rfloor$  Page matrix of the entries  $f(t_0),\ldots ,f(t_0 + T - 1)$ , partition them into  $\lfloor T / L\rfloor$  segments of  $L$  contiguous entries and concatenate these segments column-wise (see Figure 1). Now we introduce the main assumptions for  $f_{1},\dots,f_{N}$
Assumption 2.1 (Spatial structure). For each  $n \in [N]$ ,  $f_{n}(t) = \sum_{r=1}^{R} u_{nr} w_{r}(t)$  for some  $u_{nr} \in \mathbb{R}$  and  $w_{r}: \mathbb{Z}^{+} \to \mathbb{R}$ .
Assumption 2.1 posits that each time series  $f_{n}(\cdot) \forall n \in [N]$  can be described as a linear combination of  $R$  "fundamental" time series. The second assumption relates to the temporal structure of the fundamental time series  $w_{r}(\cdot) \forall r \in [R]$ , which we describe next.
Assumption 2.2 (Temporal structure). For each  $r \in [R]$  and for any  $T > 1$ ,  $1 \leq L \leq T, t_0 > 0$ ,  $\text{rank}(\mathbf{Z}(w_r, L, T, t_0)) \leq G$ .
Assumption 2.2 posits that the Page matrix of each "fundamental" time series  $w_{r}$  has finite rank. While this imposed temporal structure may seem restrictive at first, it has been shown that many standard functions that model time series dynamics satisfy this property [4, 3]. These include any finite sum of products of harmonics, low-degree polynomials, and exponential functions (refer to Proposition 2.1 in [3]).
Stochastic Stationary Component. We adopt the following two assumptions about the AR processes  $x_{n} \forall n \in [N]$ . We first assume that these AR processes are stationary (see Definition H.4 in Appendix H).
Assumption 2.3 (Stationarity and distinct roots).  $x_{n}(t)$  is a stationary  $\mathrm{AR}(p_n)$  process  $\forall n \in [N]$ . That is, let  $\lambda_{ni} \in \mathbb{C}$ ,  $i \in [p_n]$  denote the roots of  $g_{n}(z) \coloneqq z^{p_{n}} - \sum_{i = 1}^{p_{n}}\alpha_{ni}z^{p_{n} - i}$ . Then,  $|\lambda_{ni}| < 1$ $\forall i \in [p_n]$ . Further,  $\forall n \in [N]$ , the roots of  $g_{n}(z)$  are distinct.
Further, we assume the per-step noise  $\eta_{n}(t)$  are i.i.d. sub-gaussian random variables (see Definition H.1 in Appendix H).
Assumption 2.4 (Sub-gaussian noise). For  $n \in [N], t \in [T]$ ,  $\eta_n(t)$  are zero-mean i.i.d. sub-gaussian random variables with variance  $\sigma^2$ .
Model Implications. In this section, we state two important implications of the model stated above. The first is that the stacked Page matrix defined as
$$
\mathbf {Z} _ {f} (L, T, t _ {0}) = [ \mathbf {Z} (f _ {1}, L, T, t _ {0}) \quad \mathbf {Z} (f _ {2}, L, T, t _ {0}) \quad \dots \quad \mathbf {Z} (f _ {n}, L, T, t _ {0}) ],
$$
is low-rank. Precisely, we recall the following Proposition stated in [3].
Proposition 2.1 (Proposition 2 in [3]). Let Assumptions 2.1 and 2.2 hold. Then for any  $L \leq \lfloor \sqrt{T} \rfloor$  with any  $T \geq 1$ ,  $t_0 > 0$ , the rank of the Page matrix  $\mathbf{Z}(f_n, L, T, t_0)$  for  $n \in [N]$  is at most  $R \times G$ . Further, the rank of the stacked Page matrix  $\mathbf{Z}_f(L, T, t_0)$  is  $k \leq R \times G$ .
Throughout, we will use the shorthand  $\mathbf{Z}_f\coloneqq \mathbf{Z}_f(L,T,1)$  and  $\mathbf{Z}_f(t_0)\coloneqq \mathbf{Z}_f(L,T,t_0)^3$ . The second implication is that there exists a linear relationship between the last row of  $\mathbf{Z}_f$  and its top  $L - 1$  rows. The following proposition establishes this relationship. First, Let  $[\mathbf{Z}_f]_L$ . denote the  $L$ -th row of  $\mathbf{Z}_f$  and let  $\mathbf{Z}_f' \in \mathbb{R}^{(L-1) \times (N\lfloor T / L\rfloor)}$  denote the sub-matrix that consists of the top  $L - 1$  rows of  $\mathbf{Z}_f$ .
Proposition 2.2 (Proposition 3 in [3]). Let Assumptions 2.1 and 2.2 hold. Then, for  $L > RG$ , there exists  $\beta^{*} \in \mathbb{R}^{L-1}$  such that  $[\mathbf{Z}_f]_{L^{\cdot}}^{\top} = \mathbf{Z}_f^{\prime \top} \beta^{*}$ . Further,  $\|\beta^{*}\|_0 \leq RG$ .
The proposed algorithm provides two main functionalities. The first one is decomposing the observations  $y_{n}(t)$  into an estimate of the non-stationary and stationary components for  $t \leq T$ . The second is forecasting  $y_{n}(t)$  for  $t > T$ , which involves learning a forecasting model for both  $f_{n}(t)$  and  $x_{n}(t)$ .
Univariate Case. For ease of exposition, we will first describe the algorithm for the univariate case  $(N = 1)$ . The algorithm has the following parameters:  $1 < L \leq \sqrt{T}$ , and  $1 \leq \hat{k} \leq L$  (refer
3We will use the same shorthand for the stacked page matrices of  $y$  and  $x$ : namely,  $\mathbf{Z}_y$  and  $\mathbf{Z}_x$ .
to Appendix B.2 for how to choose these parameters). For clarity, we will assume, without loss of generality, that  $L$  is chosen such that  $T / L$  is an integer<sup>4</sup>. In the first step of the algorithm, we transform the observations  $y_{1}(t), t \in [T]$  into the  $L \times T / L$  Page matrix  $\mathbf{Z}(y_{1}, L, T, 1)$ . We will use the shorthand  $\mathbf{Z}_{y_{1}} := \mathbf{Z}(y_{1}, L, T, 1)$  henceforth.
Decomposition. We compute the SVD  $\mathbf{Z}_{y_1} = \sum_{\ell=1}^{L}s_\ell u_\ell v_\ell^\top$ , where  $s_1 \geq s_2 \dots \geq s_L \geq 0$  denote its ordered singular values, and  $u_\ell \in \mathbb{R}^L, v_\ell \in \mathbb{R}^{T/L}$  denote its left and right singular vectors, respectively, for  $\ell \in [L]$ . Then, we obtain  $\widehat{\mathbf{Z}}_{f_1}$  by retaining the top  $\hat{k}$  singular components of  $\mathbf{Z}_{y_1}$  (i.e., by applying Hard Singular Value Thresholding (HSVT) with threshold  $\hat{k}$ ). That is,  $\widehat{\mathbf{Z}}_{f_1} = \sum_{\ell=1}^{\hat{k}}s_\ell u_\ell v_\ell^\top$ .
We denote by  $\widehat{f}_1(t)$  and  $\widehat{x}_1(t)$  the estimates of  $f_1(t)$  and  $x_{1}(t)$  respectively. We read off the estimates  $\widehat{f}_1(t)$  directly from the matrix  $\widehat{\mathbf{Z}}_{f_1}$  using the entry that corresponds to  $t\in [T]$ . More precisely, for  $t\in [T]$ ,  $\widehat{f}_1(t)$  equals the entry of  $\widehat{\mathbf{Z}}_{f_1}$  in row  $(t - 1\bmod L) + 1$  and column  $\lceil t / L\rceil$ , while  $\widehat{x}_1(t) = y_1(t) - \widehat{f}_1(t)$ .
Forecasting. To forecast  $y_{1}(t)$  for  $t > T$ , we produce a forecast for both  $\widehat{f}_{1}(t)$  and  $\widehat{x}_{1}(t)$ . Both forecasts are performed through linear models  $(\widehat{\beta}$  for  $\widehat{f}_{1}(t)$  and  $\widehat{\alpha}_{1}$  for  $\widehat{x}_{1}(t)$ . For  $\widehat{f}_{1}(t)$ , we first learn a linear model  $\widehat{\beta}$  defined as
$$
\widehat {\beta} = \operatorname {a r g m i n} _ {\beta \in \mathbb {R} ^ {L - 1}} \sum_ {m = 1} ^ {T / L} \left(y _ {1} (L m) - \beta^ {\top} \widehat {F} _ {m}\right) ^ {2}, \tag {3}
$$
where  $\widehat{F}_m = [\widehat{f}_1(L(m - 1) + 1),\ldots ,\widehat{f}_1(L\times m - 1)]$  for  $m\in [T / L]^5$  We then use  $\widehat{\beta}$  and the  $L - 1$  lagged observations to produce  $\widehat{f}_1(t)$ . That is  $\widehat{f}_1(t) = \widehat{\beta}^\top Y_1(t - 1)$ , where  $Y_{1}(t - 1) = [y_{1}(t - 1),\dots ,y_{1}(t - L)]$ .
For  $\widehat{x}_1(t)$ , we first estimate the parameters for the  $\mathrm{AR}(p_1)$  process. Specifically, define the  $p_1$ -dimensional vectors  $\widehat{X}_1(t) \coloneqq [\widehat{x}_1(t), \ldots, \widehat{x}_1(t - p_1 + 1)]^6$ . Then, define the OLS estimate as,
$$
\widehat {\alpha} _ {1} = \operatorname {a r g m i n} _ {\alpha \in \mathbb {R} ^ {p _ {1}}} \quad \sum_ {t = p _ {1}} ^ {T - 1} \left(\widehat {x} _ {1} (t + 1) - \alpha^ {\top} \widehat {X} _ {1} (t)\right). \tag {4}
$$
Then, let  $\widetilde{x}_1(t') = y_1(t') - \widehat{f}_1(t')$  for any  $t' < t$ . We then use  $\widehat{\alpha}_1$  and the  $p_1$  lagged entries of  $\widetilde{x}_1(\cdot)$  to produce  $\widehat{x}_1(t)$ . That is  $\widehat{x}_1(t) = \widehat{\alpha}_1^\top \widetilde{X}_1(t - 1)$ , where  $\widetilde{X}_1(t - 1) = [\widetilde{x}_1(t - 1), \ldots, \widetilde{x}_1(t - p_1)]$ . Finally, produce the forecast  $\widehat{y}_1(t) = \widehat{f}_1(t) + \widehat{x}_1(t)$ .
Multivariate Case. The key change for the case  $N > 1$  is the use of the stacked Page matrix  $\mathbf{Z}_y$ , which is the column-wise concatenation of the Page matrices induced by individual time series. Specifically, consider the stacked Page matrix  $\mathbf{Z}_y \in \mathbb{R}^{L \times NT / L}$  defined as
$$
\mathbf {Z} _ {y} = \left[ \begin{array}{l l l l} \mathbf {Z} (y _ {1}, L, T, 1) & \mathbf {Z} (y _ {2}, L, T, 1) & \dots & \mathbf {Z} (y _ {n}, L, T, 1) \end{array} \right]. \tag {5}
$$
Decomposition. The procedure for learning the non-stationary component  $f_{1},\ldots ,f_{N}$  for  $t\leq T$  is similar to that of the univariate case. Specifically, we perform HSVT with threshold  $\hat{k}$  on  $\mathbf{Z}_y$  to produce  $\widehat{\mathbf{Z}}_y$ . Then, we read off the estimate of  $f_{n}(t)$  from  $\widehat{\mathbf{Z}}_y$ . Specifically, for  $t\in [T]$ , and  $n\in [N]$ , let  $\widehat{\mathbf{Z}}_{f_n}$  refer to sub-matrix of  $\widehat{\mathbf{Z}}_f$  induced by selecting only its  $[(n - 1)\times (T / L) + 1,\dots ,n\times T / L]$
4. Otherwise, one can apply this algorithm to the two ranges  $\{1,\dots ,L\times \lfloor T / L\rfloor \}$  and  $\{(T\bmod L) + 1,\ldots ,T\}$ .
In the forecasting algorithm, the estimates  $[\widehat{f}_1(L(m - 1) + 1),\dots,\widehat{f}_1(L\times m - 1)]$  are obtained by applying HSVT on a sub-matrix of  $\mathbf{Z}_{y_1}$  which consists of its first  $L - 1$  rows. This is done to establish the theoretical results as it helps us avoid dependencies in the noise between  $y_{1}(Lm)$  and  $\widehat{F}_m$  for  $m\in [T / L]$ .
Note that we assume knowledge of the true parameter  $p_1$  (and  $p_n$  in the multivariate case).
Note the subtle difference between  $\widehat{x}_1(t)$  and  $\widetilde{x}_1(t)$  - precisely, for  $t > T$ ,  $\widehat{x}_1(t)$  is an estimate of  $x_1(t)$  before observing  $y_1(t)$  (i.e., a forecast), whereas  $\widetilde{x}_1(t)$  is an estimate of  $x_1(t)$  after observing  $y_1(t)$ .
Figure 1: A visual depiction of SAMoSSA. The algorithm five steps are (i) transform the time series into its stacked Page matrix representation; (ii) decompose time series into non-stationary and stationary components (iii) estimate  $\widehat{\beta}$ ; (iv) estimates  $\hat{\alpha}_n \forall n \in [N]$ ; (v) produce the forecast  $\widehat{y}_n(t)$  for  $t > T$ .
columns. Then for  $t \in [T]$ ,  $\widehat{f}_n(t)$  equals the entry of  $\widehat{\mathbf{Z}}_{f_n}$  in row  $(t - 1 \mod L) + 1$  and column  $\lceil t / L \rceil$ . We then produce an estimate for  $x_n(t)$  as  $\widehat{x}_n(t) = y_n(t) - \widehat{f}_n(t)$ .
Forecasting. To forecast  $y_{n}(t)$  for  $t > T$ , as in the univariate case, we learn a linear model  $\widehat{\beta}$  defined as
$$
\widehat {\beta} = \operatorname {a r g m i n} _ {\beta \in \mathbb {R} ^ {L - 1}} \sum_ {m = 1} ^ {N \times T / L} \left(y _ {m} - \beta^ {\top} \widehat {F} _ {m}\right) ^ {2}, \tag {6}
$$