Dynamic Mode Decomposition for Multivariate Time Series Forecasting | by Xinyu Chen (陈新宇) | Towards Data Science

Dynamic mode decomposition (DMD) is a data-driven dimensionality reduction algorithm developed by Peter Schmid in 2008 (paper published in 2010, see [1, 2]), which is similar to matrix factorization and principle component analysis (PCA) algorithms. Given a multivariate time series data set, DMD computes a set of dynamic modes in which each mode is associated with a fixed oscillation frequency and decay/growth rate. Due to the intrinsic temporal behaviors underlying each dynamic mode, DMD indeed differs from commonly used dimensionality reduction algorithms like PCA. DMD allows one to interpret temporal behaviors of data with physically meaningful modes. One important feature is that DMD is capable of performing multivariate time series forecasting. Among the existing work, DMD models have broad applications in spatiotemporal fluid flow analysis (see Figure 1 for example).
In this post, we will introduce the preliminaries of DMD algorithm and discuss how to reproduce DMD in Python with Numpy. Then, to build a good understanding of DMD, we design a toy example experiment for performing multivariate time series forecasting. Have fun with the following content! :)
If you are interested in analyzing fluid dynamics as shown in Figure 1, there is also a blog post available at https://medium.com/p/d84065fead4d. This post introduces how to reconstruct fluid dynamics with tensor decomposition algorithm in Python.
DMD has a strong connect with vector autoregressive (VAR) model. VAR is a classical statistical model used to capture coevolution patterns of multivariate time series data, which has broad use in economics and finance. Given a time series data X of size N-by-T where N is the number of variables and T is the number of time steps, then in any time step t, the first-order VAR, or VAR(1), takes the form of
where xt denotes the snapshot vector in time t and the size is N-by-1. A is the coefficient matrix, and it is of size N-by-N.
In the VAR(1) model, the modeling goal is to find a well-behaved coefficient matrix and use it to represent temporal correlations. In the standard DMD model, we also take the form VAR(1). To compute the coefficients, the first impulse is to rewrite the above equation as follows,
where we define the following matrices in the equation:
Different from VAR(1), DMD does not explicitly compute the coefficient matrix. In DMD, there is a low-rank structure to approximate the coefficient matrix in VAR(1), it is the Koopman matrix.
In DMD, obtaining the Koopman matrix and implementing a forecasting task is not difficult. The most important tools in DMD are singular value decomposition and eigenvalue decomposition. To compute the Koopman matrix, there are only two steps that should follow:
Perform singular value decomposition on the data matrix X1:
where U consists of left singular vectors, V consists of right singular vectors, and Σ consists of singular values in its diagonal.
Implement a truncated singular value decomposition with a certain predefined rank r, and compute the Koopman matrix by
As mentioned above, DMD can explore interpretable temporal behaviors by dynamic modes. If one want to compute the dynamic mode, please follow these steps:
Perform eigenvalue decomposition on the Koopman matrix:
where Q consists of eigenvectors, and Φ consists of eigenvalues.
The DMD modes, also called dynamic modes, are the eigenvectors. Eigenvalues are always appearing as complex conjugate pairs, and they can be interpreted as DMD spectrum.
In above, we got the Koopman matrix, dynamic modes, DMD spectrum. One question would be how to perform multivariate time series forecasting. We can answer this question by adopting the following two equations:
Now, we have the coefficient matrix A as mentioned in VAR(1). Therefore, it is possible to implement a time series forecasting task. Here is the Python code for the DMD algorithm. We reproduce it by using Numpy, and it would be easy to follow.
import numpy as npdef DMD(data, r):    """Dynamic Mode Decomposition (DMD) algorithm."""        ## Build data matrices    X1 = data[:, : -1]    X2 = data[:, 1 :]    ## Perform singular value decomposition on X1    u, s, v = np.linalg.svd(X1, full_matrices = False)    ## Compute the Koopman matrix    A_tilde = u[:, : r].conj().T @ X2 @ v[: r, :].conj().T * np.reciprocal(s[: r])    ## Perform eigenvalue decomposition on A_tilde    Phi, Q = np.linalg.eig(A_tilde)    ## Compute the coefficient matrix    Psi = X2 @ v[: r, :].conj().T @ np.diag(np.reciprocal(s[: r])) @ Q    A = Psi @ np.diag(Phi) @ np.linalg.pinv(Psi)        return A_tilde, Phi, A
The outputs include the Koopman matrix, eigenvalues, and coefficient matrix.
Let us consider the following toy example.
import numpy as npdef DMD4cast(data, r, pred_step):    N, T = data.shape    _, _, A = DMD(data, r)    mat = np.append(data, np.zeros((N, pred_step)), axis = 1)    for s in range(pred_step):        mat[:, T + s] = (A @ mat[:, T + s - 1]).real    return mat[:, - pred_step :]
Write code here:
X = np.zeros((2, 10))X[0, :] = np.arange(1, 11)X[1, :] = np.arange(2, 12)pred_step = 2r = 2mat_hat = DMD4cast(X, r, pred_step)print(mat_hat)
Run it, and the output is
[[11. 12.] [12. 13.]]
As can be seen, the DMD forecasts are completely same as the ground truth data.
This is a simple post for introducing DMD algorithm and its application to multivariate time series forecasting tasks. From the formula, DMD is indeed a reduced-rank VAR model, which can address the over-parameterization issue in the high-dimensional time series data. There are also some other advantages of DMD over VAR. For example, DMD is of broad interest in spatiotemporal data analysis due to its interpretation. Alternatively, we can also use temporal matrix factorization model (components include both matrix factorization and VAR) to address the over-parameterization issue. There is a blog post (see https://medium.com/p/b1c59faf05ea) that discussed the fluid dynamic forecasting with temporal matrix factorization.
Thank you for reading this post! If you have any questions, please let me know!
[1] Peter Schmid (2010). Dynamic mode decomposition of numerical and experimental data. Journal of fluid mechanics, 656: 5–28.
[2] Dynamic mode decomposition on Wikipedia: https://en.wikipedia.org/wiki/Dynamic_mode_decomposition
[3] J. Nathan Kutz, Steven L. Brunton, Bingni W. Brunton, Joshua L. Proctor (2016). Dynamic mode decomposition: Data-driven modeling of complex systems. SIAM.