Discussion Paper No. 141
Deep learning for multivariate volatility forecastingin high-dimensional financial time series.

Rei Iwafuchi and Yasumasa Matsuda

May, 2024

Data Science and Service Research
Discussion Paper

Center for Data Science and Service Research
Graduate School of Economic and Management
Tohoku University27-1 Kawauchi, Aobaku

Sendai 980-8576, JAPANDeep learning for multivariate volatility
forecasting in high-dimensional ﬁnancial timeseries.
Rei Iwafuchi1 and Yasumasa Matsuda11Tohoku University
May 2024
Abstract

The market for investment trusts of large-scale portfolios, including index funds, continues to grow, and high-dimensional volatility estimation
is essential for assessing the risks of such portfolios. However, multivariate volatility models suitable for high-dimensional data have not been
extensively studied. This paper introduces a new framework based on theSpatial AR model, which provides fast and stable estimation, and demonstrates its application through simulations using historical data from theS&P 500.

1Introduction

In high-dimensional multivariate GARCH modeling, the paramount challenge isto construct a model that is both expressive and feasible in terms of the numberof parameters that can be eﬀectively estimated. Many models impose strong
constraints to facilitate estimation; however, these constraints do not necessarily
reﬂect the true nature and volatility characteristics of the actual data.
Traditionally, volatility matrix estimation has been attempted using two
main approaches. The ﬁrst approach involves a natural extension of ARCH
by Engle [1] and GARCH models by Bollerslev [2], including the VECH modelby Bollerslev, Engle, and Wooldridge [3], BEKK model by Engle and Kroner[4], and Constant Conditional Correlation (CCC) model by He and Teräsvirta[5]. The second approach attempts to simplify modeling by reducing the original data, with Factor-ARCH by Ding [6] and Orthogonal-GARCH models by
Alexander [7] being prime examples.
However, each approach has its problems when estimating the volatility ofhigh-dimensional data involving hundreds of dimensions. In models like theVECH and BEKK, the number of parameters increases exponentially with dimensionality, making estimation exceedingly diﬃcult. Even more parsimonious
1models, such as the CCC process, are highly restrictive. Its relaxed counterpart,the Dynamic Conditional Correlation (DCC) model by Engle [8], frequently involves the cumbersome computation of large matrix determinants. Additionally,models utilizing factors often struggle to adequately capture the variability inhigh-dimensional data with large sample sizes using only a few factors. Moreover, volatility matrices estimated using a factor model in high dimensions almost invariably lack positive deﬁniteness.
This paper introduces a robust and fast method for estimating high-dimensional
volatility matrices using a Spatial Autoregressive (SAR) model combined withLasso. Speciﬁcally, it focuses on the relationships between variables, removing
conditional correlations with the SAR model before estimating volatility with
the GARCH model. Furthermore, we present a ﬂexible extension to models
utilizing neural networks.

2SAR + Volatility Framework

2.1

NotationIn the ﬁeld of multivariate volatility modeling, we consider the volatility matrix
in a context where past information has been observed. Speciﬁcally, there isinterest in the conditional covariance matrix at time t, given the information
available up to time t − 1, which is typically denoted by Ft−1 . Let r t be a random vector representing stock returns at time t. Based on this understanding,multivariate volatility models are represented as follows:r t = μ + εtεt |Ft−1 ∼ N (0, Vt )In most cases, μ is assumed to be zero. The primary focus in this area is how
to model Vt
The famous univariate GARCH(p,q) model, which we are going to utilize
later, is described as:
r t = μ + εt ,σt2 = α0 +pεt |Ft−1 ∼ N (0, σt2 )αk ε2t−k +k=1q2β σt−=1This paper utilizes the GARCH model to estimate Vt , elaborating on how themodel accounts for the dynamic nature of volatility based on historical data andhow it can be adapted to capture the complexities of multivariate time series. Byemploying the GARCH model framework, we aim to provide a robust method
for estimating the conditional variances that are crucial for accurate ﬁnancial
forecasting and risk management.

22.2

SAR GARCH

The key assumption of the Spatial Autoregressive GARCH model is the following:
Assumption 1 The structure of conditional correlations among stocks can berepresented by the SAR model, and these relationships are sparse:
r t = W r t + ut ,Wii = 0(1)

It would be intuitively natural to assume that any given stock is related toa few others while unrelated to many. Under this assumption, we can estimate
volatility for each univariate separately, signiﬁcantly reducing the number ofparameters.
Assumption 2 The residuals of the SAR model ut are conditionally and crosssectionally uncorrelated, and follow a GARCH process.
ut |Ft ∼ N (0, St ),St,ii = α0 +p

St,ij = 0αk u2t−k,i +k=1i = j(2)β St−,ii(3)ifq
=1For estimating the volatility of residuals, the GARCH model is utilized. If
the volatility of the residuals is correctly estimated, the conditional covariancematrix of {r t } can be obtained as follows:
Vt = (I − W )−1 St [(I − W )−1 ]

2.3

Extension Using Deep Learning

Here, we introduce a method for extending the SAR+GARCH framework using
deep learning. In order to capture the complex dependencies of time series,LSTM(Hochreiter and Schmidhuber [9] Gers, Schmidhuber, and Cummins [10])is employed. Let N N be a neural network we employ. Then the formulation ofthe extended model using deep learning is as follows.
r t = W r t + ut , Wii = 0
ut |Ft−1 ∼ N (0, St ), St = diag(st ) snn
st = sgarchttsgarch= α0 + α1  ut−1 + α2  sgarch
tt−1
ut−1nnlog st = N N ( garch )st−1
The key idea here is that the important factors of volatility ﬂuctuations canbe captured by the GARCH model, and neural networks are used as an aid tocapture nonlinear dependencies. GARCH model is extremely eﬀective, and weutilize it to the fullest extent.
3

Figure 1: Brief Description of the Neural Network Structure
Figure 1 describe the structure of N N . In the neural network, the input isﬁrst compressed to an appropriate dimension using two Aﬃne Layers. Then,complex dependencies are captured using an LSTM Layer, followed by anotherAﬃne Layer to restore the dimensions to their original state.

3Estimation

In this study, we repeatedly predict the volatility of the following day using
one month’s worth of data. Then we minimize the negative log likelihood. For
simplicity, we omit the constant term included in the negative log likelihoodand double its value as bellow.
TLoss =1   −1u S ut − 2 log |I − W |T t=1 t t

Furthermore, we utilize mini-batch training to seek global minimum, where theoptimization method employed is Adam.

4Empirical Example

To evaluate the performance of the proposed model, we conduct the experiments
using price data of 331 stocks included in the S&P 500. The test period fromJune 22, 2018, to April 2, 2021, is divided into batches of 20 samples along
the timeline. Training is conducted using samples from T days immediately
preceding each test period, and losses of one to ﬁve periods ahead prediction
are calculated to compare multiple models. As benchmark models, we adopt theFactor Model + POET(See Fan, Fan, and Lv [11], Fan, Liao, and Mincheva [12]and Fan, Liao, and Mincheva [13] for the details.) and the univariate GARCH4model. To observe the diﬀerences caused by the sample size of the training data,experiments are conducted in three scenarios: T=250, T=1000, and T=3000.
Figure 2, Figure 3 and Figure 4 describe the mean loss of each test period
in each scenario. Here, SARNN refers to the neural network-based extended
model of the SAR GARCH model.
In the scenarios with T=250 and T=1000, both the SAR GARCH modeland the SARNN model outperform the two benchmark models. Under thesescenarios, the one-step-ahead forecasts of the two proposed models are almost
identical in performance. However, for the ﬁve-steps-ahead forecast, SARNN
shows exceptionally good performance. During shocks, sudden high-frequency
trading creates complex dependencies between stocks and past prices, making
multiple stepsd ahead predictions extremely challenging. However, the nonlinearity of SARNN is thought to have helped capture these relationships.
In the scenario with T=3000, the performance of the one- step-ahead forecast remains good for both models. However, for the ﬁve-steps-ahead forecast, SARNN’s performance signiﬁcantly deteriorated during the shock. This isthought to be due to the fact that there were no major shocks in the past 3000
samples, leading the highly expressive SARNN to overlearn the dependencies
during ordinary period and neglect the complex dependencies necessary for theﬁve-steps-ahead forecast.

5Conclusion

In this paper, we introduced the SAR + volatility framework as a new method
for high-dimensional volatility estimation. This framework is a fast and stablemodel that does not require frequent matrix inversion calculations during optimization iterations. Additionally, the utilization of Lasso and neural networks
allows for stable predictions even when the sample size is not suﬃciently large
compared to the dimension of the inputs.
The SAR GARCH model and its extension using neural networks, the SARNN
model, demonstrated superior performance over benchmark models in simulations. Furthermore, it was found that with proper setting of the training data,
the SARNN model signiﬁcantly outperforms others in multi-steps-ahead forecasting during ﬁnancial shocks. This is believed to be due to the nonlinearityof neural networks, which contributes to capturing the complex dependencies
between stocks or with past returns during shocks. We believe that the resultsof this research can be applied to various issues, including the risk assessmentof large-scale portfolios, and will contribute to the development of the ﬁeld ofﬁnance.

5(a) One step ahead forecast(b) Five steps ahead forecast

Figure 2: Mean loss of each test period with T=250(a) One step ahead forecast(b) Five steps ahead forecast

Figure 3: Mean loss of each test period with T=1000

6

(a) One step ahead forecast(b) Five steps ahead forecast

Figure 4: Mean loss of each test period with T=3000

References[1]

R. F. Engle, “Autoregressive conditional heteroscedasticity with estimatesof the variance of united kingdom inﬂation,” Econometrica, vol. 50, no. 4,
pp. 987–1007, 1982, issn: 00129682, 14680262.

[2]

T. Bollerslev, “Generalized autoregressive conditional heteroskedasticity,”
Journal of Econometrics, vol. 31, no. 3, pp. 307–327, 1986, issn: 0304-4076.

[3]

T. Bollerslev, R. F. Engle, and J. M. Wooldridge, “A capital asset pricing model with time-varying covariances,” Journal of Political Economy,
vol. 96, no. 1, pp. 116–131, 1988, issn: 00223808, 1537534X.

[4]

R. F. Engle and K. F. Kroner, “Multivariate simultaneous generalizedarch,” Econometric Theory, vol. 11, no. 1, pp. 122–150, 1995, issn: 02664666,14694360.

[5]

C. He and T. Teräsvirta, “An extended constant conditional correlation garch model and its fourth-moment structure,” Econometric Theory,
vol. 20, no. 5, pp. 904–926, 2004, issn: 02664666, 14694360.

[6]

Z. Ding, Time series analysis of speculative returns. UMI, 1994.

[7]

C. Alexander, “Orthogonal methods for generating large positive semideﬁnite covariance matrices,” University of Reading Working Paper No.
2000-06, 2000.

[8]

R. Engle, “Dynamic conditional correlation: A simple class of multivariate
generalized autoregressive conditional heteroskedasticity models,” Journalof Business & Economic Statistics, vol. 20, no. 3, pp. 339–350, 2002, issn:07350015.

[9]

S. Hochreiter and J. Schmidhuber, “Long Short-Term Memory,” NeuralComputation, vol. 9, no. 8, pp. 1735–1780, Nov. 1997, issn: 0899-7667.

[10]

F. Gers, J. Schmidhuber, and F. Cummins, “Learning to forget: Continual
prediction with lstm,” in 1999 Ninth International Conference on Artiﬁcial Neural Networks ICANN 99. (Conf. Publ. No. 470), vol. 2, 1999, 850–855 vol.2.
7

[11]

J. Fan, Y. Fan, and J. Lv, “High dimensional covariance matrix estimation
using a factor model,” Journal of Econometrics, vol. 147, no. 1, pp. 186–197, 2008, Econometric modelling in ﬁnance and risk management: Anoverview, issn: 0304-4076.

[12]

J. Fan, Y. Liao, and M. Mincheva, “High-dimensional covariance matrix estimation in approximate factor models,” The Annals of Statistics,
vol. 39, no. 6, pp. 3320–3356, 2011.

[13]

J. Fan, Y. Liao, and M. Mincheva, “Large covariance estimation by thresholding principal orthogonal complements,” Journal of the Royal StatisticalSociety. Series B (Statistical Methodology), vol. 75, no. 4, pp. 603–680,
2013, issn: 13697412, 14679868.

8