# Advantages of Combining Factorization Machine with Elman Neural Network for Volatility Forecasting of Stock Market

Authors: Fang Wang; Sai Tang; Menggang Li
Journal: Complexity
 (2021)

---

## Abstract

With a focus in the financial market, stock market dynamics forecasting has received much attention. Predicting stock market fluctuations is usually challenging due to the nonlinear and nonstationary time series of stock prices. The Elman recurrent network is renowned for its capability of dealing with dynamic information, which has made it a successful application to predicting. We developed a hybrid approach which combined Elman recurrent network with factorization machine (FM) technique, i.e., the FM-Elman neural network, to predict stock market volatility. In this paper, the Standard & Poor’s 500 Composite Stock Price (S&P 500) index, the Dow Jones industrial average (DJIA) index, the Shanghai Stock Exchange Composite (SSEC) index, and the Shenzhen Securities Component Index (SZI) were used to demonstrate the validity of our proposed FM-Elman model in time-series prediction. The results were compared with predictions obtained from the other two models which are basic BP neural network and the Elman neural network. Some experiments showed that the FM-Elman model outperforms others through different accuracy measures. Furthermore, the effects of volatility degree on prediction performance from different stock indexes were investigated. An interesting phenomenon had been found through some numerical experiments on the effects of different user-specified dimensions on the proposed FM-Elman neural network.


---

## Body

## 1. Introduction

In recent years, the fluctuation analysis of financial time series has received a lot of concerns. Stock market volatility prediction has become a significant topic in economic research. The study of stock market volatility forecasting can be helpful for policy makers to take appropriate decisions on asset allocation and risk management. Therefore, predicting the volatility of financial time series with a reasonable accuracy deserves much attention. However, stock market exhibits nonlinear and chaotic properties in nature [1, 2]. Statistical models then have some difficulties in dealing with nonlinear and nonstationary time series or deriving satisfactory forecasting performance under the statistical assumptions of normally distributed observations. The predicting becomes more challenging.Artificial neural network has the advantages on learning from sample data and capturing the nonlinear relations among interconnected neurons through training mode [3]. It is capable of dealing with nonlinear high-dimensional data and approximating any nonlinear functions with arbitrary precision [4–7]. Particularly, the simple recurrent network, i.e., Elman neural network (Elman NN) [8] has shown its stronger ability as it has the characteristic of time-varying. And the Elman NN is a kind of feedback network where the added layer connecting to the hidden layer can be regarded as a time delay operator capable of memorizing recent events. It is a time-varying predictive control system that has faster convergence and more accurate mapping ability.Elman NN has been utilized to financial prediction and applied to many other different types of time series. Most studies on Elman NN obtained higher accuracy. Zheng [9] used an Elman NN to forecast opening prices of the Shanghai Stock Exchange. Wu and Duan applied the Elman NN in predicting stock [10] and gold future markets [11], respectively. In the area of electricity prediction, Rani and Victoire [12] integrated the decomposition method and group search optimization algorithm into the Elman NN. It showed that the Elman NN outperformed other approaches.There are also other artificial neural networks like wavelet neural network and radial basis function neural network [13–16]. Some developed artificial intelligence techniques like expert systems [17, 18], support vector machines (SVMs) [19, 20], and hybrid methods [21, 22] are also applied in forecasting stock prices. Recently, some novel models have utilized random jump or random time effective function with different neural networks [23, 24] which have been proposed in forecasting financial market.Although the models which are based on artificial intelligent have achieved remarkable results, there are still limitations. There is a few technique in most models which pay attention to the nonlinear interactions among the inputs. For example, the nonlinearities in neural network models were handled by the activation functions. These models without consideration of interactions between features with different scales have been widely used in some applications such as image processing, mechanical translation, and speech recognition [25–27].FM is originally used for collaborative recommendations which were first introduced by Rendle [28]. FM is a supervised learning method that can model feature interactions with second-order even when the data have very high sparsity and high dimension. FMs show state-of-the-art performance as they have two main benefits. First, FMs are on a par with polynomial regression but can achieve empirical accuracy with smaller and faster evaluation results. Second, unlike the linear regression, FMs can infer the weights of feature interactions that were not observed in the training dataset. The weights of second-order feature interactions have the low-rank property which makes FMs become increasingly popular in the recommender system. Although FM is a general framework of matrix factorization, FM shows more flexibility as the matrix factorization method only models the relation between two entities [29]. FMs are general predictors like SVMs and have a lot of applications in industry. FMs are applicable to any variables with real feature and are not restricted to recommender systems. FM gives a promising direction for the prediction purpose in regression, classification, and ranking [30–33].As far as we know, real-world time series is rarely pure nontime-varying. And the linear regression is not always capable of deriving the interactions between features which however are more common in various applications. Hence, the problem of dealing with time-varying and nonlinear interactions can be solved by combining FM with Elman NN. Moreover, it is almost universally agreed in the forecasting literature that no single model is the best in every situation because a real-world problem is often complex. Using any single model may not be able to capture different patterns equally well [34]. Therefore, we propose a forecasting model combining FM technique with Elman NN for stock market volatility prediction in the present paper.In this paper, we apply the FM-Elman neural network to forecast the volatility degree’s behavior of the Standard & Poor’s 500 Composite Stock Price (S&P 500) index, the Dow Jones industrial average (DJIA) index, the Shanghai Stock Exchange Composite (SSEC) index, and the Shenzhen Securities Component index (SZI) from January 2nd, 2000, to December 31st, 2011. Different threshold values were introduced into our model, and the corresponding volatility prediction results were presented. To show the advantages of the proposed FM-Elman model, we compare the predicting results with two other neural network models including BP network and Elman recurrent network through three performance evaluation measures such as the mean absolute error (MAE), root mean square error (RMSE), and mean absolute percentage error (MAPE).The remainder of this paper is presented as following sections. In Section2, the Elman NN and FM are reviewed where they are prepared for our proposed model. Then, we give the prediction model FM-Elman neural network in Section 3. In this section, we first give the model description and in the same time introduce some needed ingredients of it. And the algorithm of the FM-Elman model is also given. Section 4 presents the main forecasting results of the FM-Elman model. This section gives predicting comparisons among our proposed model, BP neural network, and Elman neural network. It not only presents the effects of different parameters like volatility degree and user-specified dimension on the FM-Elman model’s performance but also considers other evaluation measures. And Section 5 highlights some necessary conclusions finally.


## 2. Elman Neural Network and Factorization Machine

### 2.1. Elman Neural Network (Elman NN)

Elman neural network was founded by Elman [8] in 1990 which is famous for its recurrent topology structure. Unlike the BP network, an Elman NN has a set of recurrent nodes. The so-called recurrent nodes in the buffer received message from the peered output nodes in the hidden layer and then transmitted message to the hidden layer. Every hidden node is connected to only one recurrent neuron, and the message will remain the same after transmitting. Hence, the number of recurrent layer nodes is the same as the number of hidden nodes, and the recurrent layer contains the state of input data from the hidden layer.Figure1 gives the structure of multi-input Elman NN. The Elman NN is composed of the input layer, the hidden layer, the output layer, and the recurrent layer. There are n nodes in the input layer, and both the hidden layer and the recurrent layer have m nodes. In the output layer, there exists only one unit neuron. The mathematical computation for the nonlinear state of the Elman NN is(1)u⟶t=fWAx⟶t−1+WBz⟶t,z⟶t=u⟶t−1,yt=gWCu⟶t,where u⟶t is the vector of output values in the hidden layer, yt is the final output of the network, and x⟶t−1=x1,t−1,x2,t−1,...,xn,t−1 denotes the input of the network at time t−1. The weight matrix WA connects the input layer node to the node in the hidden layer, WB connects the node in the recurrent layer to the hidden layer neuron, and WC is the matrix which connects the node in the hidden layer to the output node. Functions f⋅ and g⋅ are the activation functions where fx is the sigmoid function and gx is an identity function in this paper.Figure 1
Topology of the Elman neural network.From equation (1) and through deduction, we can obtain that(2)z⟶t=fWt−1Ax⟶t−2+Wt−1Bz⟶t−1,where z⟶t depends on the matrix Wt−1A and Wt−1B which comes from different time. Elman NN has the ability to adapt to time series varying.


### 2.2. Factorization Machine

FM has the same prediction ability as SVMs but also has capability of estimating reliable parameters under very sparse data. The feature of modelling all variable interactions is comparable to a polynomial kernel in SVM. The equation for a FM with second-order feature is defined as follows:(3)y^x=w0+∑i=1nwixi+∑i=1n∑j=i+1nvi,vjxixj,where the parameters w0∈ℝ, w∈ℝn, and V∈ℝn×k have to be determined. And ⋅,⋅ is the inner product of two vectors with size k. Then,(4)vi,vj=∑f=1kvi,f⋅vj,f,which models the interaction between the ith variable and the jth variable, where vi is the ith variable with k∈ℕ0+ dimension factors.Our intuition for the complexity of equation (3) is in Okn2 because all pairwise interactions have to be computed. As there is no parameter in a model depending on two variables directly, the pairwise interactions in equation (3) are reformulated as follows:(5)∑i=1n∑j=i+1nvi,vjxixj=12∑f=1k∑i=1nvifxi2−∑i=1nvif2xi2.And the equation only needs linear runtimeOkn to be computed after the reformulation. So, FMs are applicable from a computational point of view.


## 2.1. Elman Neural Network (Elman NN)

Elman neural network was founded by Elman [8] in 1990 which is famous for its recurrent topology structure. Unlike the BP network, an Elman NN has a set of recurrent nodes. The so-called recurrent nodes in the buffer received message from the peered output nodes in the hidden layer and then transmitted message to the hidden layer. Every hidden node is connected to only one recurrent neuron, and the message will remain the same after transmitting. Hence, the number of recurrent layer nodes is the same as the number of hidden nodes, and the recurrent layer contains the state of input data from the hidden layer.Figure1 gives the structure of multi-input Elman NN. The Elman NN is composed of the input layer, the hidden layer, the output layer, and the recurrent layer. There are n nodes in the input layer, and both the hidden layer and the recurrent layer have m nodes. In the output layer, there exists only one unit neuron. The mathematical computation for the nonlinear state of the Elman NN is(1)u⟶t=fWAx⟶t−1+WBz⟶t,z⟶t=u⟶t−1,yt=gWCu⟶t,where u⟶t is the vector of output values in the hidden layer, yt is the final output of the network, and x⟶t−1=x1,t−1,x2,t−1,...,xn,t−1 denotes the input of the network at time t−1. The weight matrix WA connects the input layer node to the node in the hidden layer, WB connects the node in the recurrent layer to the hidden layer neuron, and WC is the matrix which connects the node in the hidden layer to the output node. Functions f⋅ and g⋅ are the activation functions where fx is the sigmoid function and gx is an identity function in this paper.Figure 1
Topology of the Elman neural network.From equation (1) and through deduction, we can obtain that(2)z⟶t=fWt−1Ax⟶t−2+Wt−1Bz⟶t−1,where z⟶t depends on the matrix Wt−1A and Wt−1B which comes from different time. Elman NN has the ability to adapt to time series varying.


## 2.2. Factorization Machine

FM has the same prediction ability as SVMs but also has capability of estimating reliable parameters under very sparse data. The feature of modelling all variable interactions is comparable to a polynomial kernel in SVM. The equation for a FM with second-order feature is defined as follows:(3)y^x=w0+∑i=1nwixi+∑i=1n∑j=i+1nvi,vjxixj,where the parameters w0∈ℝ, w∈ℝn, and V∈ℝn×k have to be determined. And ⋅,⋅ is the inner product of two vectors with size k. Then,(4)vi,vj=∑f=1kvi,f⋅vj,f,which models the interaction between the ith variable and the jth variable, where vi is the ith variable with k∈ℕ0+ dimension factors.Our intuition for the complexity of equation (3) is in Okn2 because all pairwise interactions have to be computed. As there is no parameter in a model depending on two variables directly, the pairwise interactions in equation (3) are reformulated as follows:(5)∑i=1n∑j=i+1nvi,vjxixj=12∑f=1k∑i=1nvifxi2−∑i=1nvif2xi2.And the equation only needs linear runtimeOkn to be computed after the reformulation. So, FMs are applicable from a computational point of view.


## 3. Our Proposed Method

### 3.1. Modelling

We construct the Elman recurrent neural network with factorization machine, i.e., FM-Elman neural network, to predict the volatility of different stock indexes. The detailed topology of FM-Elman neural network is presented in Figure2.Figure 2
Structure of the Elman neural network with factorization machine.The layers of the FM-Elman neural network are analyzed as follows:(1)
Hidden Layer. The nodes in the hidden layer are partitioned into two parts. One part of them has normal nodes which show the linear relations among the input data. And the remaining nodes in the other part incorporate all interactions between each pair of features from the input data. The results are computed by(6)yjt=f∑i=1nvijxit+∑z=1muzjxzt,i=1,2,…,n,j=1,2,…,m−k,z=1,2,…,m,ym−k+jt=f12∑i=1nv^i,m−k+jxit2−∑i=1nv^i,m−k+j2xi2t+12∑i=1nu^z,m−k+jxzt2−∑i=1nu^z,m−k+j2xz2t,i=1,2,…,n,j=1,2,…,k,z=1,2,…,m,wherexi is the input value from input node i, yj denotes the value of the jth node in the hidden layer, vij is the undetermined weight which relates the ith input node to the jth normal node in the hidden layer, v^ij is the weight connecting the ith input node to the remaining node in the hidden layer which is also undetermined, uzj and u^zj have the same meaning with vij and v^ij except for the first two parameters which link the nodes in the recurrent layer to the hidden layer, t is the iteration number in the formulas, k is the user-specified dimension, and f is the activation function.(2)
Recurrent Layer. The number of nodes in the recurrent layer is the same as the number of hidden nodes. Each hidden node is connected to only one node in the recurrent layer, and the connected weight is a constant value one. So, the recurrent layer is also partitioned into two parts which are presented as follows:(7)xzt=yjt−1,j=1,2,…,m−k,z=1,2,…,m−k,xm−k+zt=ym−k+jt−1,j=1,2,…,k,z=1,2,…,k.(3)
Output Layer. The outputs are(8)Ojt=g∑i=1mwijyit,j=1,2,…,p,wherewij is the undermined weight, Oj is the output value of jth node in the output layer, and g is the activation function.There are different loss functions to calculate the error between the actual and the estimated values from the output layer. We consider the squared loss function which is given as follows:(9)Et=12Tt−Ot2,where Tt is the actual value in the tth iteration.So, the final output error of the FM-Elman model is computed by(10)l=1N∑t=1NEt=1N∑t=1N12Tt−Ot2.To optimize the FM-Elman model, we often use the stochastic gradient decent method to update the weights until it achieves convergence.


### 3.2. Algorithm of FM-Elman Model

The training process of the FM-Elman model is detailed as follows:(1)
The gradients of the weights and the updated rule in the output layer are computed as follows:(11)Δwj=−η∂l∂wj=ηTt−Otyjt,wj←wj−η∂l∂wj,wherej=1,2,…,m,η is the learning rate.(2)
The gradients of the weights and the updated rule in the hidden layer connected by the recurrent layer are calculated as the following two cases:(12)Δuzj=−η∂l∂uzj=ηTt−Otg′wjf′xzt,j=1,2,…,m−k,Δu^z,m−k+j=−η∂l∂u^z,m−k+j=ηTt−Otg′wm−k+jf′∑z=1mu^z,m−k+jxztxzt−∑z=1mu^z,m−k+jxz2t,j=1,2,…,k,uzj←uzj−η∂l∂uzj,j=1,2,…,m−k,u^z,m−k+j←u^z,m−k+j−η∂l∂u^z,m−k+j,j=1,2,…,k,wherez=1,2,…,m,η is the learning rate, k is the user-specified dimension, and g′ and f′ are corresponded derivative functions of g and f.(3)
The gradients of the weights and the updated rule in the hidden layer connected by the input layer are computed as the following two cases:(13)Δvij=−η∂l∂vij=ηTt−Otg′wjf′xit,j=1,2,…,m−k,Δv^i,m−k+j=−η∂l∂v^i,m−k+j=ηTt−Otg′wm−k+jf′∑i=1mv^i,m−k+jxitxzt−∑i=1mv^i,m−k+jxi2t,j=1,2,...,k,vij←vij−η∂l∂vij,j=1,2,…,m−k,v^i,m−k+j←v^i,m−k+j−η∂l∂v^i,m−k+j,j=1,2,…,k,where i=1,2,…,n,η is the learning rate, k is the user-specified dimension, and g′ and f′ are corresponded derivative functions of g and f.


## 3.1. Modelling

We construct the Elman recurrent neural network with factorization machine, i.e., FM-Elman neural network, to predict the volatility of different stock indexes. The detailed topology of FM-Elman neural network is presented in Figure2.Figure 2
Structure of the Elman neural network with factorization machine.The layers of the FM-Elman neural network are analyzed as follows:(1)
Hidden Layer. The nodes in the hidden layer are partitioned into two parts. One part of them has normal nodes which show the linear relations among the input data. And the remaining nodes in the other part incorporate all interactions between each pair of features from the input data. The results are computed by(6)yjt=f∑i=1nvijxit+∑z=1muzjxzt,i=1,2,…,n,j=1,2,…,m−k,z=1,2,…,m,ym−k+jt=f12∑i=1nv^i,m−k+jxit2−∑i=1nv^i,m−k+j2xi2t+12∑i=1nu^z,m−k+jxzt2−∑i=1nu^z,m−k+j2xz2t,i=1,2,…,n,j=1,2,…,k,z=1,2,…,m,wherexi is the input value from input node i, yj denotes the value of the jth node in the hidden layer, vij is the undetermined weight which relates the ith input node to the jth normal node in the hidden layer, v^ij is the weight connecting the ith input node to the remaining node in the hidden layer which is also undetermined, uzj and u^zj have the same meaning with vij and v^ij except for the first two parameters which link the nodes in the recurrent layer to the hidden layer, t is the iteration number in the formulas, k is the user-specified dimension, and f is the activation function.(2)
Recurrent Layer. The number of nodes in the recurrent layer is the same as the number of hidden nodes. Each hidden node is connected to only one node in the recurrent layer, and the connected weight is a constant value one. So, the recurrent layer is also partitioned into two parts which are presented as follows:(7)xzt=yjt−1,j=1,2,…,m−k,z=1,2,…,m−k,xm−k+zt=ym−k+jt−1,j=1,2,…,k,z=1,2,…,k.(3)
Output Layer. The outputs are(8)Ojt=g∑i=1mwijyit,j=1,2,…,p,wherewij is the undermined weight, Oj is the output value of jth node in the output layer, and g is the activation function.There are different loss functions to calculate the error between the actual and the estimated values from the output layer. We consider the squared loss function which is given as follows:(9)Et=12Tt−Ot2,where Tt is the actual value in the tth iteration.So, the final output error of the FM-Elman model is computed by(10)l=1N∑t=1NEt=1N∑t=1N12Tt−Ot2.To optimize the FM-Elman model, we often use the stochastic gradient decent method to update the weights until it achieves convergence.


## 3.2. Algorithm of FM-Elman Model

The training process of the FM-Elman model is detailed as follows:(1)
The gradients of the weights and the updated rule in the output layer are computed as follows:(11)Δwj=−η∂l∂wj=ηTt−Otyjt,wj←wj−η∂l∂wj,wherej=1,2,…,m,η is the learning rate.(2)
The gradients of the weights and the updated rule in the hidden layer connected by the recurrent layer are calculated as the following two cases:(12)Δuzj=−η∂l∂uzj=ηTt−Otg′wjf′xzt,j=1,2,…,m−k,Δu^z,m−k+j=−η∂l∂u^z,m−k+j=ηTt−Otg′wm−k+jf′∑z=1mu^z,m−k+jxztxzt−∑z=1mu^z,m−k+jxz2t,j=1,2,…,k,uzj←uzj−η∂l∂uzj,j=1,2,…,m−k,u^z,m−k+j←u^z,m−k+j−η∂l∂u^z,m−k+j,j=1,2,…,k,wherez=1,2,…,m,η is the learning rate, k is the user-specified dimension, and g′ and f′ are corresponded derivative functions of g and f.(3)
The gradients of the weights and the updated rule in the hidden layer connected by the input layer are computed as the following two cases:(13)Δvij=−η∂l∂vij=ηTt−Otg′wjf′xit,j=1,2,…,m−k,Δv^i,m−k+j=−η∂l∂v^i,m−k+j=ηTt−Otg′wm−k+jf′∑i=1mv^i,m−k+jxitxzt−∑i=1mv^i,m−k+jxi2t,j=1,2,...,k,vij←vij−η∂l∂vij,j=1,2,…,m−k,v^i,m−k+j←v^i,m−k+j−η∂l∂v^i,m−k+j,j=1,2,…,k,where i=1,2,…,n,η is the learning rate, k is the user-specified dimension, and g′ and f′ are corresponded derivative functions of g and f.


## 4. Forecasting Results

### 4.1. Data Selecting and Processing

Stock prices’ different changing behaviors and volatility predicting study have long been a focus in economic research. We use the logarithmic return to describe the statistical characteristic of a stock return volatility. The stock logarithmic return is defined as(14)rt=lnSt−lnSt−1,where St denotes the stock daily closing price at time t.The data (http://www.finance.yahoo.com) chosen for our experiment are the Standard & Poor’s 500 Composite Stock Price (S&P 500) index, the Dow Jones industrial average (DJIA) index, the Shanghai Stock Exchange Composite (SSEC) index, and the Shenzhen Securities Component index (SZI). All the data are collected from trading days ranging from January 2nd, 2000, to December 31st, 2011. So, the size of different time series is 3019, 3027, 2901, and 2902, respectively. We partition them into training sets (from January 2nd, 2000, to December 31st, 2007) and testing sets (from January 2nd, 2008, to December 31st, 2011). Table 1 describes the data’s statistical feature, where Nall and Ntest denote the sizes of the whole data and the testing samples, respectively.Table 1
The data and their descriptive statistical analysis.
NameNallNtestMeanStd.S&P 500301910091190.02184.26DJIA3027100910612.791397.02SSEC29019762226.36973.46SZI29029767028.884389.84In this paper, a threshold valueθ≥0 is introduced as the volatility degree. Let ℜθ denote the set in which the stock returns’ absolute values are greater than the valueθ, and the definition is given by ℜθ=rt‖rt|≥θ,t=1,2,…,T. Once the value θ is set, we can obtain the dataset including the satisfied stock daily closing price. Figure 3 gives an example of stock returns with different thresholds. For a fixed threshold value θ, the corresponding stock trading dates are determined. The trading dates are in the set where time t values satisfy rt≥θ,t=1,2,…,T. Newly formed series are arranged in a chronological order. Table 2 gives the numbers of data for stock indexes distributed in training data and testing data under different threshold values θ.Figure 3
Absolute returns of S&P 500 with different thresholds.Table 2
Numbers of data distributed in training data and testing data under different threshold valuesθ.
ThresholdθS&P 500DJIASSECSZITrainingTestingTrainingTestingTrainingTestingTrainingTesting02010100920181009192597619269760.00314157771426756148578515208240.006987585945544110564311906880.0096634556224318175209235620.012447350418327615411688462When the threshold valueθ equals 0, the averaged values of daily absolute returns for S&P 500, DJIA, SSEC, and SZI are 0.0094, 0.0089, 0.0117, and 0.0130, respectively. We set different volatility degrees 0.003, 0.006, 0.009, and 0.012 to see the data numbers distributed in the training and testing data set, respectively. In Table 2, as the threshold value θ increases, the quantity of data in both training dataset and testing dataset that exceeds the given threshold value gradually reduces. And it can be predicted that when θ is larger than 0.012, the corresponding numbers will be fewer.Four input variables including the daily opening prices, the daily highest prices, the daily lowest prices, and the daily closing prices are selected according to the newly formed dates. And we choose the next time daily closing price in the chronological ordered datasets as the output variable. In order to reduce the noise’s impact on the stock markets, all the input dataX are normalized as follows:(15)X′=X−minXmaxX−minX.Then, it is easily to obtain the actual prediction value throughX=X′maxX−minX+minX.


### 4.2. Performances of FM-Elman Model

In the proposed FM-Elman neural network, we choose the structure with4×10×1 where the number of input nodes is 4, the number of the hidden nodes is 10, and the number of the output nodes is 1. We set the maximum iterations number as 5000, η=0.02, and the predefined minimum training threshold is ε=5×10−5.To analyze and evaluate the predicting performance of the FM-Elman neural network model, we use the accuracy measures with the corresponding definitions as follows:(16)MAE=1N∑t=1NTt−Ot,RMSE=1N∑t=1NTt−Ot2,MAPE=100×1N∑t=1NTt−OtTt,where Tt and Ot are the actual value and the predictive value in the tth iteration and N is the sample size. When the values of these evaluation measures are smaller, the prediction performance is better.In this section, we derive the stock prices’ different fluctuation behaviors through the proposed FM-Elman neural network. First, the new proposed model is proved to be better compared with BPNN and Elman neural model through different evaluation measures. Then, the prediction performance of FM-Elman neural network is measured when threshold valueθ varies. And finally, we can see how the user-specified dimension k impacts the performance of the proposed model.When the threshold value equals 0, the training datasets and testing datasets are the original datasets. And the prediction results of S&P500 and SSEC by the FM-Elman neural model withk=3 are presented in Figures 4 and 5.Figure 4
Comparisons of prediction results and the daily closing prices of S&P 500 for testing datasets.Figure 5
Comparisons of prediction results and the daily closing prices of SSEC for testing datasets.We then give the performance comparisons among different prediction models forθ=0 in Table 3 where the MAPE (100) means the latest 100 days of MAPE in the testing data. Three different prediction models include BPNN, Elman neural network, and FM-Elman neural network with the user-specified dimension k=3. Table 3 shows that FM-Elman model’ evaluation errors are all smaller than those in the other two models. In addition, the MAPE (100) value is smaller than the corresponding stock index’ MAPE value. It shows that the short-term prediction outperforms the long-term prediction.Table 3
Comparisons of prediction performances among different models forθ=0.
Evaluation errorsS&P 500DJIABPNNElman NNFM-ElmanBPNNElman NNFM-ElmanMAE54.182246.843243.5088321.3915294.4472280.9011MAPE5.15664.52314.16923.31963.04732.9379RMSE60.212153.841449.7034382.7756354.6576348.8277MAPE (100)1.78131.55261.52731.19381.17331.1103Evaluation errorsSSECSZIBPNNElman NNFM-ElmanBPNNElman NNFM-ElmanMAE68.567253.627150.4433287.2003255.4868228.2216MAPE2.45981.93691.80082.63632.29332.0552RMSE85.452772.68269.7871355.0629324.7933298.6005MAPE (100)2.81512.59332.56183.05942.87072.7645


### 4.3. The Impact ofθ

Whenθ varies from 0.003 to 0.012, different prediction analysis of indexes S&P 500, DJIA, SSEC, and SZI can be performed by the FM-Elman neural network. Figures 6 and 7 are the prediction analysis of S&P 500 and SSEC by the FM-Elman neural model. The two figures also show the effectiveness of forecasting with different volatility degree values of θ. When θ is small, the performance of volatility prediction is revealed better through the empirical results. Like θ=0.03 and θ=0.06 in Figures 6(a) and 6(b), the predictive values are closer to the actual values than those in Figures 6(c) and 6(d). Figure 7 also indicates the similar results.Figure 6
Volatility degree prediction of S&P 500 with different thresholdsθ.
(a)(b)(c)(d)Figure 7
Volatility degree prediction of SSEC with different thresholdsθ.
(a)(b)(c)(d)We choose the often recommended criterion MAPE to measure the prediction performance for stock indexes S&P 500, DJIA, SSEC, and SZI under the FM-Elman neural model which is presented in Table4. When the value of θ increases, the value of MAPE increases gradually. And the numerical experiment results show that using the FM-Elman neural network model, the volatility degree forecasting is feasible.Table 4
Forecasting errors (MAPE) of the FM-Elman (k=3) model as threshold value θ varies.
MAPES&P 500DJIASSECSZIθ=0.0033.66572.90062.35932.5499θ=0.0064.43123.25542.36082.7811θ=0.0095.37263.29982.79773.1299θ=0.0126.33683.75712.94983.1579


### 4.4. The Impact ofk

In this subsection, we want to analyze how the user-specified dimensionk affects the prediction performance by the FM-Elman neural model through numerical experiment when θ=0. From the descriptions in Section 3, when k increases, the amount of red nodes in both hidden layer and recurrent layer becomes larger. That means more hidden and recurrent nodes in the proposed FM-Elman neural network will contain interaction information from the connected inputs. Then, the computation becomes complicated as k increases. It is interesting to see in Table 5 that the evaluation values of MAE, MAPE, and RMSE for indexes S&P 500, DJIA, SSEC, and SZI increase first and then decrease with the increasing of k. So, the low user-specified dimension or high user-specified dimension is a better choice. No matter which outperforms the predicting results of BPNN and Elman NN from the previous Table 3.Table 5
Forecasting errors of the FM-Elman model whenθ=0.
DimensionS&P 500DJIAMAEMAPERMSEMAEMAPERMSEk=140.32363.883446.6309263.6652.7759335.7891k=343.50884.169249.7034280.90112.9379348.8277k=639.50233.829346.4366254.5842.6649321.9889k=939.0413.759745.4894227.08232.3703290.28DimensionSSECSZIMAEMAPERMSEMAEMAPERMSEk=147.01321.654568.2408222.84882.032293.7208k=350.44331.800869.7871228.22162.0552298.6005k=650.85931.823370.5519241.77712.1825317.8106k=948.41281.709768.4333240.43622.1684316.2495


### 4.5. Further Predicting Performance Evaluation

We adopt three trend-type statistical methods, i.e., directional symmetry (DS), correct up-trend (CP), and correct down-trend (CD) [35], to check the practical stock movement. When the values of these three performance evaluation results become larger, the forecasting of change direction will be more precise. The definitions of these three performance evaluation methods are given as(17)DS=100N1∑t=1N1at,at=1,ifTt−Tt−1Ot−Ot−1≥0,0,otherwise,where N1 is the number of testing samples.(18)CP=100N2∑t=1N2at,at=1,ifTt−Tt−1>0andOt−Ot−1≥0,0,otherwise,where N2 is the number of testing samples which satisfy Tt−Tt−1>0.(19)CD=100N3∑t=1N3at,at=1,ifTt−Tt−1<0andOt−Ot−1≤0,0,otherwise,where N3 is the number of testing samples which satisfy Tt−Tt−1<0. Tt and Ot are the actual value and the predictive value in the tth iteration, respectively.In Table6, the trend-type measures DS, CP, and CD for stock indexes S&P 500, DJIA, SSEC, and SZI under varying volatility degrees are presented through some numerical experiments. When the value of θ changes, all the stock indexes change a little. And we can see that the direction forecasting results of SSEC and SZI show better performance than S&P 500 and DJIA since the DS, CP, and CD values in two indexes before all exceed 50. And the performance results of stock indexes S&P 500 and DJIA are more sensitive to θ with large value. For example, when θ changes from the value 0.009 to 0.012, the values of DS, CP, and CD for stock indexes S&P 500 and DJIA decrease sharply.Table 6
Trend-type forecasting evaluation of the FM-Elman (k=3) model as threshold value θ varies.
Evaluation errorsS&P 500DJIADSCPCDDSCPCDθ=048.66252.747343.844549.157652.398545.3961θ=0.00348.519952.132744.225449.206352.45145.4023θ=0.00651.111152.941249.103947.610348.936246.1832θ=0.00949.0115047.982147.099847.727346.4455θ=0.01243.714343.930643.502843.119344.642941.5094Evaluation errorsSSECSZIDSCPCDDSCPCDθ=05051.898750.922151.187953.216452.2541θ=0.00350.445950.64650.251351.33551.767750.9346θ=0.00651.010950.64151.359552.180252.43951.9444θ=0.00952.115452.509751.724152.84753.703752.0548θ=0.01251.338250.253852.336450.21153.777851.9481


## 4.1. Data Selecting and Processing

Stock prices’ different changing behaviors and volatility predicting study have long been a focus in economic research. We use the logarithmic return to describe the statistical characteristic of a stock return volatility. The stock logarithmic return is defined as(14)rt=lnSt−lnSt−1,where St denotes the stock daily closing price at time t.The data (http://www.finance.yahoo.com) chosen for our experiment are the Standard & Poor’s 500 Composite Stock Price (S&P 500) index, the Dow Jones industrial average (DJIA) index, the Shanghai Stock Exchange Composite (SSEC) index, and the Shenzhen Securities Component index (SZI). All the data are collected from trading days ranging from January 2nd, 2000, to December 31st, 2011. So, the size of different time series is 3019, 3027, 2901, and 2902, respectively. We partition them into training sets (from January 2nd, 2000, to December 31st, 2007) and testing sets (from January 2nd, 2008, to December 31st, 2011). Table 1 describes the data’s statistical feature, where Nall and Ntest denote the sizes of the whole data and the testing samples, respectively.Table 1
The data and their descriptive statistical analysis.
NameNallNtestMeanStd.S&P 500301910091190.02184.26DJIA3027100910612.791397.02SSEC29019762226.36973.46SZI29029767028.884389.84In this paper, a threshold valueθ≥0 is introduced as the volatility degree. Let ℜθ denote the set in which the stock returns’ absolute values are greater than the valueθ, and the definition is given by ℜθ=rt‖rt|≥θ,t=1,2,…,T. Once the value θ is set, we can obtain the dataset including the satisfied stock daily closing price. Figure 3 gives an example of stock returns with different thresholds. For a fixed threshold value θ, the corresponding stock trading dates are determined. The trading dates are in the set where time t values satisfy rt≥θ,t=1,2,…,T. Newly formed series are arranged in a chronological order. Table 2 gives the numbers of data for stock indexes distributed in training data and testing data under different threshold values θ.Figure 3
Absolute returns of S&P 500 with different thresholds.Table 2
Numbers of data distributed in training data and testing data under different threshold valuesθ.
ThresholdθS&P 500DJIASSECSZITrainingTestingTrainingTestingTrainingTestingTrainingTesting02010100920181009192597619269760.00314157771426756148578515208240.006987585945544110564311906880.0096634556224318175209235620.012447350418327615411688462When the threshold valueθ equals 0, the averaged values of daily absolute returns for S&P 500, DJIA, SSEC, and SZI are 0.0094, 0.0089, 0.0117, and 0.0130, respectively. We set different volatility degrees 0.003, 0.006, 0.009, and 0.012 to see the data numbers distributed in the training and testing data set, respectively. In Table 2, as the threshold value θ increases, the quantity of data in both training dataset and testing dataset that exceeds the given threshold value gradually reduces. And it can be predicted that when θ is larger than 0.012, the corresponding numbers will be fewer.Four input variables including the daily opening prices, the daily highest prices, the daily lowest prices, and the daily closing prices are selected according to the newly formed dates. And we choose the next time daily closing price in the chronological ordered datasets as the output variable. In order to reduce the noise’s impact on the stock markets, all the input dataX are normalized as follows:(15)X′=X−minXmaxX−minX.Then, it is easily to obtain the actual prediction value throughX=X′maxX−minX+minX.


## 4.2. Performances of FM-Elman Model

In the proposed FM-Elman neural network, we choose the structure with4×10×1 where the number of input nodes is 4, the number of the hidden nodes is 10, and the number of the output nodes is 1. We set the maximum iterations number as 5000, η=0.02, and the predefined minimum training threshold is ε=5×10−5.To analyze and evaluate the predicting performance of the FM-Elman neural network model, we use the accuracy measures with the corresponding definitions as follows:(16)MAE=1N∑t=1NTt−Ot,RMSE=1N∑t=1NTt−Ot2,MAPE=100×1N∑t=1NTt−OtTt,where Tt and Ot are the actual value and the predictive value in the tth iteration and N is the sample size. When the values of these evaluation measures are smaller, the prediction performance is better.In this section, we derive the stock prices’ different fluctuation behaviors through the proposed FM-Elman neural network. First, the new proposed model is proved to be better compared with BPNN and Elman neural model through different evaluation measures. Then, the prediction performance of FM-Elman neural network is measured when threshold valueθ varies. And finally, we can see how the user-specified dimension k impacts the performance of the proposed model.When the threshold value equals 0, the training datasets and testing datasets are the original datasets. And the prediction results of S&P500 and SSEC by the FM-Elman neural model withk=3 are presented in Figures 4 and 5.Figure 4
Comparisons of prediction results and the daily closing prices of S&P 500 for testing datasets.Figure 5
Comparisons of prediction results and the daily closing prices of SSEC for testing datasets.We then give the performance comparisons among different prediction models forθ=0 in Table 3 where the MAPE (100) means the latest 100 days of MAPE in the testing data. Three different prediction models include BPNN, Elman neural network, and FM-Elman neural network with the user-specified dimension k=3. Table 3 shows that FM-Elman model’ evaluation errors are all smaller than those in the other two models. In addition, the MAPE (100) value is smaller than the corresponding stock index’ MAPE value. It shows that the short-term prediction outperforms the long-term prediction.Table 3
Comparisons of prediction performances among different models forθ=0.
Evaluation errorsS&P 500DJIABPNNElman NNFM-ElmanBPNNElman NNFM-ElmanMAE54.182246.843243.5088321.3915294.4472280.9011MAPE5.15664.52314.16923.31963.04732.9379RMSE60.212153.841449.7034382.7756354.6576348.8277MAPE (100)1.78131.55261.52731.19381.17331.1103Evaluation errorsSSECSZIBPNNElman NNFM-ElmanBPNNElman NNFM-ElmanMAE68.567253.627150.4433287.2003255.4868228.2216MAPE2.45981.93691.80082.63632.29332.0552RMSE85.452772.68269.7871355.0629324.7933298.6005MAPE (100)2.81512.59332.56183.05942.87072.7645


## 4.3. The Impact ofθ

Whenθ varies from 0.003 to 0.012, different prediction analysis of indexes S&P 500, DJIA, SSEC, and SZI can be performed by the FM-Elman neural network. Figures 6 and 7 are the prediction analysis of S&P 500 and SSEC by the FM-Elman neural model. The two figures also show the effectiveness of forecasting with different volatility degree values of θ. When θ is small, the performance of volatility prediction is revealed better through the empirical results. Like θ=0.03 and θ=0.06 in Figures 6(a) and 6(b), the predictive values are closer to the actual values than those in Figures 6(c) and 6(d). Figure 7 also indicates the similar results.Figure 6
Volatility degree prediction of S&P 500 with different thresholdsθ.
(a)(b)(c)(d)Figure 7
Volatility degree prediction of SSEC with different thresholdsθ.
(a)(b)(c)(d)We choose the often recommended criterion MAPE to measure the prediction performance for stock indexes S&P 500, DJIA, SSEC, and SZI under the FM-Elman neural model which is presented in Table4. When the value of θ increases, the value of MAPE increases gradually. And the numerical experiment results show that using the FM-Elman neural network model, the volatility degree forecasting is feasible.Table 4
Forecasting errors (MAPE) of the FM-Elman (k=3) model as threshold value θ varies.
MAPES&P 500DJIASSECSZIθ=0.0033.66572.90062.35932.5499θ=0.0064.43123.25542.36082.7811θ=0.0095.37263.29982.79773.1299θ=0.0126.33683.75712.94983.1579


## 4.4. The Impact ofk

In this subsection, we want to analyze how the user-specified dimensionk affects the prediction performance by the FM-Elman neural model through numerical experiment when θ=0. From the descriptions in Section 3, when k increases, the amount of red nodes in both hidden layer and recurrent layer becomes larger. That means more hidden and recurrent nodes in the proposed FM-Elman neural network will contain interaction information from the connected inputs. Then, the computation becomes complicated as k increases. It is interesting to see in Table 5 that the evaluation values of MAE, MAPE, and RMSE for indexes S&P 500, DJIA, SSEC, and SZI increase first and then decrease with the increasing of k. So, the low user-specified dimension or high user-specified dimension is a better choice. No matter which outperforms the predicting results of BPNN and Elman NN from the previous Table 3.Table 5
Forecasting errors of the FM-Elman model whenθ=0.
DimensionS&P 500DJIAMAEMAPERMSEMAEMAPERMSEk=140.32363.883446.6309263.6652.7759335.7891k=343.50884.169249.7034280.90112.9379348.8277k=639.50233.829346.4366254.5842.6649321.9889k=939.0413.759745.4894227.08232.3703290.28DimensionSSECSZIMAEMAPERMSEMAEMAPERMSEk=147.01321.654568.2408222.84882.032293.7208k=350.44331.800869.7871228.22162.0552298.6005k=650.85931.823370.5519241.77712.1825317.8106k=948.41281.709768.4333240.43622.1684316.2495


## 4.5. Further Predicting Performance Evaluation

We adopt three trend-type statistical methods, i.e., directional symmetry (DS), correct up-trend (CP), and correct down-trend (CD) [35], to check the practical stock movement. When the values of these three performance evaluation results become larger, the forecasting of change direction will be more precise. The definitions of these three performance evaluation methods are given as(17)DS=100N1∑t=1N1at,at=1,ifTt−Tt−1Ot−Ot−1≥0,0,otherwise,where N1 is the number of testing samples.(18)CP=100N2∑t=1N2at,at=1,ifTt−Tt−1>0andOt−Ot−1≥0,0,otherwise,where N2 is the number of testing samples which satisfy Tt−Tt−1>0.(19)CD=100N3∑t=1N3at,at=1,ifTt−Tt−1<0andOt−Ot−1≤0,0,otherwise,where N3 is the number of testing samples which satisfy Tt−Tt−1<0. Tt and Ot are the actual value and the predictive value in the tth iteration, respectively.In Table6, the trend-type measures DS, CP, and CD for stock indexes S&P 500, DJIA, SSEC, and SZI under varying volatility degrees are presented through some numerical experiments. When the value of θ changes, all the stock indexes change a little. And we can see that the direction forecasting results of SSEC and SZI show better performance than S&P 500 and DJIA since the DS, CP, and CD values in two indexes before all exceed 50. And the performance results of stock indexes S&P 500 and DJIA are more sensitive to θ with large value. For example, when θ changes from the value 0.009 to 0.012, the values of DS, CP, and CD for stock indexes S&P 500 and DJIA decrease sharply.Table 6
Trend-type forecasting evaluation of the FM-Elman (k=3) model as threshold value θ varies.
Evaluation errorsS&P 500DJIADSCPCDDSCPCDθ=048.66252.747343.844549.157652.398545.3961θ=0.00348.519952.132744.225449.206352.45145.4023θ=0.00651.111152.941249.103947.610348.936246.1832θ=0.00949.0115047.982147.099847.727346.4455θ=0.01243.714343.930643.502843.119344.642941.5094Evaluation errorsSSECSZIDSCPCDDSCPCDθ=05051.898750.922151.187953.216452.2541θ=0.00350.445950.64650.251351.33551.767750.9346θ=0.00651.010950.64151.359552.180252.43951.9444θ=0.00952.115452.509751.724152.84753.703752.0548θ=0.01251.338250.253852.336450.21153.777851.9481


## 5. Conclusion

In this study, we developed an improved Elman recurrent neural network by introducing the factorization machine. Through extensive numerical experiments on the data from stock indexes S&P 500, DJIA, SSEC, and SZI, we demonstrated the effectiveness of the FM-Elman neural network. The prediction accuracy for all financial time series shows that our proposed FM-Elman model outperforms the BP neural network and the original Elman NN. We select training and testing datasets under different volatility degrees, i.e., the threshold valueθ varies, to predict. The prediction performance of the FM-Elman model will degrade as θ becomes larger. We also investigate the effect of the user-specified dimension on the prediction performance by the FM-Elman neural model.The contribution of this work includes the following two points: (1) a technique FM combined with Elman NN to form an FM-Elman neural model for nonstationary analysis which enjoys benefits from both FM and Elman NN and (2) we demonstrate the prediction accuracy in various metrics. The numerical experiments show significant improvements in prediction accuracy over the existing methods. However, the limitation of this research is that the proposed model is data dependent which does not guarantee excellent predictions on all datasets. And further study on high-order interactions among the inputs is also a challenging work.The power of combining FM with neural network to achieve better performance will likely exist for the area of classification and regression which will be useful for future studies. We believe that FM can be used in conjunction with other deep learning network such as LSTM to form the high quality predicting method. Various combinations in techniques and approaches can be investigated in the future to solve problems occurring in different applications.



---

*Source: 6641298-2021-05-24.xml*