# Predicting the Direction Movement of Financial Time Series Using Artificial Neural Network and Support Vector Machine

Authors: Muhammad Ali; Dost Muhammad Khan; Muhammad Aamir; Amjad Ali; Zubair Ahmad
Journal: Complexity
 (2021)

---

## Abstract

Prediction of financial time series such as stock and stock indexes has remained the main focus of researchers because of its composite nature and instability in almost all of the developing and advanced countries. The main objective of this research work is to predict the direction movement of the daily stock prices index using the artificial neural network (ANN) and support vector machine (SVM). The datasets utilized in this study are the KSE-100 index of the Pakistan stock exchange, Korea composite stock price index (KOSPI), Nikkei 225 index of the Tokyo stock exchange, and Shenzhen stock exchange (SZSE) composite index for the last ten years that is from 2011 to 2020. To build the architect of a single layer ANN and SVM model with linear, radial basis function (RBF), and polynomial kernels, different technical indicators derived from the daily stock trading, such as closing, opening, daily high, and daily low prices and used as input layers. Since both the ANN and SVM models were used as classifiers; therefore, accuracy andF-score were used as performance metrics calculated from the confusion matrix. It can be concluded from the results that ANN performs better than SVM model in terms of accuracy and F-score to predict the direction movement of the KSE-100 index, KOSPI index, Nikkei 225 index, and SZSE composite index daily closing price movement.


---

## Body

## 1. Introduction

Prediction of the stock market index and its direction has remained the most interesting and difficult task for researchers, data scientists, and econometricians because of its volatile nature. Although numerous studies have been conducted in the literature to predict the stock price indexes using different econometrics models’ meager studies are available on the prediction of its direction movement, this area of research got significant importance in the last few years especially after the emergence of the machine and deep learning methods. Predicting the direction of the stock price index movement accurately is essential for developing precise and valid market planning [1]. The financial time series such as the stock market has a nonlinear, complex, and chaotic structure; therefore, it is one of the most difficult tasks to predict [2]. For example, numerous financial variables such as demand and supply, interest rates, investments, dividends, economy, and political climate can affect the arbitrary movements and sometimes crash of the stock market [3]. Therefore, appropriate models and efficient preprocessing of the data are required for accurate and better predictions. Several classical time series models have been suggested in the literature and used to predict the stock market indexes. Similarly, the autoregressive conditional heteroskedasticity (ARCH) and generalized autoregressive conditional heteroskedasticity (GARCH) models proposed by Engle [4] and Bollerslev [5] have been used by numerous data scientists to forecast the financial time series data. The well-known ARMA (typically knows as Box & Jenkins models) proposed in [6] is the hybrid form of moving average (MA) and autoregressive (AR) terms and is used under stationary conditions to forecast the time series data. In contrast to this method, autoregressive integrated moving average (ARIMA) models were proposed for nonstationary time-series datasets. These classical techniques have been used by various researchers for many years to address the issue of nonlinearity and nonstationary of the stock market indexes.However, with the expansion of machine learning techniques, recently, many scientists have been using these models for the prediction of the nonlinear and complex nature of financial time series data. The most common of these models is ANN and SVM that are widely used to predict the direction of the stock market movement. But because of the noise and nonstationary behavior of the stock market indexes it becomes very difficult to train the ANN model which could produce efficient results [7–9]. Therefore, extreme care is needed while training models with appropriate tuning parameters like hidden layers, algorithm selection, nodes in each hidden layer, learning rate, and the activation function to minimize the forecast error. Recently, resilient back-propagation neural network (RBPNN) with and without weight backtracking was developed in [10] to train the feed-forward neural network (FFNN) which converges faster than the classical back-propagation neural network (BPNN). The novelty in this paper is that we have used this idea to build the architect for FFNN that predict the direction movement of the four well-known stock price indexes daily closing price movement by extracting various (15) technical indicators from the historical data and used them as input layers to ANN and SVMThe rest of the paper is categorized into the following sections. The related work is introduced in Section2; the preprocessing of the data, derivation of technical indicators used as inputs is described in Section 3; the proposed model is presented in Section 4. Similarly, results and discussion is presented in Section 5, followed by the conclusion of the paper in Section 6.


## 2. Literature Review

More recently, the prediction of the direction of the stock market indexes or stock price movement has become an attractive and fascinating field of research for both financial experts and theoretical scientists. Both classical time series prediction models, machine learning techniques particularly SVM, ANN, and hybrid models are widely used to predict the stock price indexes movements. Here, in this section, we will consider research studies conducted primarily with these soft computing models and their hybrid versions that are commonly used for predicting the stock market. However, many studies have been conducted in the literature using ANN and its hybrid versions some of which are discussed here.There is rich literature available on ANN and its hybrid versions to predict the random behavior of the stock market. Different procedures have been implemented to extract the important features/characteristics that are used as an input layer to construct the structure of the ANN model. Some of the related literature is discussed as follows.Abraham et al. [11] proposed a soft computing and automated method for the forecasting of the stock market and trend analysis for which two years’ daily data were taken from the Nasdaq-100 index and other companies listed in the same stock market. For analyzing the trend of the stock market, a one-day ahead forecasting was made based on the neural network and neuro-fuzzy system. The method of principal component analysis (PCA) was employed for preprocessing of the data. Performance measures that are root mean square (RMSE) were used to investigate the reliability of the proposed hybrid model where it was concluded that the forecasting results of the proposed hybrid model are promising.Lahmiri [12] has used resilient back-propagation neural network (RBNN) to predict the price level of five major international stock markets that is S&P500, Nikkei, STSE100, DAX, and CAC40. In this article, the author used different technical analysis measures, for example, indicators, oscillators, stochastics, and indexes along with the resilient back-propagation neural networks for the prediction of the price level. It is evident from the results of the study that technical analysis indicators can be used effectively in building the ANN architecture to estimate the stock price indexes.The direction movement of the Taiwan stock exchange was modeled by Chen et al. [13] with the help of the probabilistic neural network (PNN) to forecast the direction of the stock returns after it was trained by the historical data. The proposed method was compared with the generalized method of movement (GMM) with Kalman filter by using different performance measures. Empirical evidence showed that investment strategies based on the PNN model yield higher profits than by-and-hold strategies.Baba and Suto [14] constructed an intelligent decision support system for the prediction of the Tokyo stock market using a neural network (NN) and temporal difference learning (TDL) approach. The performance of the proposed system was simulated by utilizing the sixteen enterprises listed in the Tokyo stock exchange. The simulation results suggest the overall effectiveness of the combined system of the TDL and NN model. The total relative and maximum error suggests that the TD-learning method is a good forecasting tool for stock prices.A three-layer feed-forward neural network (FFNN) was used by [15] to predict the stock price moments of 367 different firms in the Shanghai stock exchange. The proposed method was compared with univariate and multivariate linear models, using statistical measures like mean absolute deviation (MAD), mean absolute percentage error (MAPE), mean square error (MSE), and standard deviation (SD). The results of these statistical measures suggest that the forecast reliability of the FFNN method is better than the other two competing models and hence recommended to predict the stock price moment of different firms listed on the Shanghai stock exchange.A composite time series adaptive network-based fuzzy inference system (ANFIS) approach was combined with the empirical mode decomposition (EMD) that is used to predict the stock prices of the Taiwan stock exchange [16]. Different classical and machine learning techniques including the well-known autoregressive (AR) and support vector machine (SVM) were compared with the proposed method to know its prediction reliability. It is evident from the performance measures of RMSE suggest that the hybrid ANFIS model is the best candidate among the other classical and hybrid models.Moreover, the direction movement of the stock and stock price index for the Indian stock exchange has been predicted by Patel et al. [17] with the help of implementing four different machine learning techniques such as ANN, SVM, random forest, and Naïve-Bayes with two different approaches for input to these models. The primary method uses ten different technical indicators like simple moving average, weighted moving average, momentum, stochastic %K, stochastic %D, relative strength index (RSI), moving average convergence divergence (MACD), Larry William R%, A/D (accumulation/distribution) oscillator, and commodity channel index (CCI). In the second method, the actual values of these technical parameters are transformed into discrete values which are called the trend deterministic data preparation layer. The performance of these techniques was checked by taking ten years of historical data of two Indian stocks. The experimental results show that when the actual values of the technical indicators are used as input layers, the performance of the random forest is better than the other models, and for the second approach, when the technical indicators are used as trend deterministic layers, the performance of all the four models are improved.To predict the daily Istanbul Stock Exchange 100- index, Kara et al. [18] applied ANN and SVM models as classifiers. The authors used ten technical indicators as input layers to build the architect of the mentioned two models. The choice of these technical parameters is based on the previous literature and expert opinion which confirms that these technical parameters influence and have the potential to bring significant changes in the stock market indexes. The empirical results confirm that the prediction reliability of the ANN model is 74.74% which is greater than the accuracy level of 71.52% of the SVM model.With the introduction of the support vector machine by Vapnik [19], it is widely used by numerous researchers along with its different versions for the prediction of different stock markets throughout the world. For example, the SVM model was used as a classifier to predict the short-term direction movement of the NIKKEI 225 index [20]. The United States of America (USA) imports one-third of its autos and their parts from Japan which confirms a close relationship between Japan and the USA economy. Therefore, the US S&P-500 index and exchange rate of US dollars against Japanese Yen (JPY) were used as inputs to the SVM model. The proposed model is then compared with Elman BPNN, linear discriminant analysis, and quadratic discriminant analysis to check the prediction performance of the SVM. It is evident from the results that the hit ratio of the SVM is 73% and integrating it with other classification methods gives an accuracy level of 75% which is higher than the other competing models.Similarly, an attempt was made to predict the daily Korean stock price index (KOSPI) by using the SVM method [8] where twelve technical indicators were used as the input layer to construct the SVM model. However, after the construction of the model, it was further applied to predict the direction moment of the KOSPI stock market index. Moreover, a comparison of the suggested SVM model was made with the BPNN and case-based reasoning (CBR) model where it was concluded that the SVM is the best forecasting tool for the prediction of the stock market index.The method of random forest and SVM was used by [21] to predict the S&P, CNX, and NIFTY stock market index of the national stock exchange which is one of the fastest-growing financial exchanges in the developing Asian countries. A comparison of these two methods was made with the neural network, discriminant analysis, and logit model. It is evident from the research work that the SVM method outperforms random forest, discriminant analysis, and logit model.Similarly, different artificial methods were hybridized by [22] to forecast the stock price index of seven major financial markets. In the preprocessing phase, the method of self-organizing map (SOM) was deployed to break down the data into different components based on the similarity of statistical distribution. After dividing the diversified dataset into different similar groups, the method of support vector regression (SVR) is applied to forecast the stock price index. The results indicate that the prediction accuracy can be increased by using the two-stage SVR algorithm rather than a single-stage classical method of SVR. For further details of the applicability of the supporting machine for the prediction of the stock markets, the readers are suggested to consult the review paper by [23].It is evident from the literature review that forecasting the direction movement of the stock price index is of paramount importance, but there are very limited studies conducted on the prediction of the KSE-100 index using support vector machine and artificial neural network with numerous technical indicators as input layers. Therefore, the main objective of this research work is to model the Pakistan KSE-100 index along with other well-established stock prices using state-of-the-art machine learning models with a higher degree of forecasting accuracy.


## 3. Materials and Methods

For this research study, we have used ten years of data ranging from January 1, 2011, to September 27, 2020, of four Asian countries, namely, the KSE-100 index of Pakistan, Korean stock price index (KOSPI), NIKKEI 225 of Japan, and Shanghai Component index of China. The total number of data points used in this study is sliced into two groups, i.e., one group consists of 80% of the data that is considered for training the model, and the second group having the remaining 20% which is utilized for testing the model. Since the main objective of this research work is to forecast the direction movement of the daily closing prices of four stock indexes with the help of ANN and SVM; therefore, fifteen technical indicators are used as input to these two well-known machine learning techniques. These technical indicators consist of the following 15 attributes/variables that are determined from an extensive literature review which are, namely, stochastic %K, stochastic %D, ROC, William Larry R%, Momentum, Disparity 5, Disparity 15, OSCP, CCI, RSI, Pivot point, S1, S2, R1, and R2 [7, 24]. The details of the indicators mentioned above, and their mathematical structure are presented in Table 1. The four stock price indices’ descriptive statistics, such as mean and standard deviation, are calculated and presented in Tables 2–5. To predict the direction movement of the stock market prices, the prediction problem is categorized as a classification problem by converting it into “0” and “1,” where “0” indicates that the daily closing stock price index for the next day is lower than today’s price and “1” means that it is higher. The percentage of increase and decrease cases in each year is also calculated and presented in Tables 6–9. Furthermore, the technical parameters are converted into a series of “0” and “1” by using min-max scaling criteria. The robustness of this approach makes it more useful than the classical normalization method which has zero breakdown point against any single outlying observation in the data. Mathematical composition of the min-max scaling is done by using the following mathematical equation:(1)YMin−Max Scaling=Y−YminYmax−Ymin,where Ymin and Ymax are the highest and lowest values in the data respectively.Table 1
The selected technical indicators and their mathematical expressions.
Indicator nameDescriptionFormulaStochastic %KIt is a stochastic oscillator that compares closed security price to its price range over a specified time frame. Its values always lie between 0% and 100%.%K=close−lowest low/highest high−lowest low×100%DMoving average of %K%D=∑i=0n−1%Ki−1/nPrice rate of change (ROC)Momentum-based technical indicator that measures the percentage change in current price and the price a certain period agoROC=cp−cp−n/cp−n×100William %RA momentum indicator that measures overbought and oversold levels. Its value moves between “0” and “100.”Williram%R=highest high−close/highest high−lowest lowMomentumMomentum measures the speed or velocity of price changes in a stock market, in simple words, it is the rate of rise or fall in stock prices.Ct − Ct−4Disparity 5A momentum-based indicator that measures the ratio of the current price and the moving average of 5 daysCt/MA5×100Disparity 14A momentum-based indicator that measures the ratio of the current price and the moving average of daysCt/MA14×10OSCPPrice oscillator, some combination of moving average. Its range depends upon the volatility of the stock.MA5−MA10/MA5CCIThe commodity channel index is a momentum-based oscillator used to determine when an investment vehicle is reaching condition of being overbought or oversoldCCI=typical price−MA/0.015×mean deviationwhere: typical price=∑i=1phigh+low+close/3MA=∑i=1ptypical price/Pmean deviation=∑i=1ptypical price−MA/PRSIRelative strength index is a momentum indicator that measures the magnitude of recent price change to evaluate overbought or oversold conditions in the price of a stock or other assetRSI=100−100/1+U/DWhere: D = average of downward price changeU = average of upward price changePivot pointTechnical indicator used to determine the overall trend of the market over different time frames. Since pivot point is not an oscillator, therefore they are static and remain at the same price throughout the day.PP=high+low+close/3S1Support 1: the first level of the pivot pointS1=PP×2−highS2Support 2: the second level of the pivot pointS2=PP−high−lowR1Resistance 1: third level of the pivot pointR1=PP×2−lowR2Resistance 2: fourth level of the pivot pointR2=PP+high−lowTable 2
Summary statistics of technical indicators for KSE-100 index.
IndicatorMin valueMax. valueMeanStandard deviation%K0.0099.4045.1130.72%D2.8696.8145.1229.04ROC93.99109.7499.891.48William %R0.59100.0054.8830.72Disparity 146.537.7007.110.16Disparity 594.87107.6799.881.16RSI2.3177.2744.2314.77Momentum2139.002919.00−26.34473.98OSCP1371.4610.45−1303.5234.24CCI−336.38258.24−24.93108.41Pivot point10.4552908.3328257.7012552.10S110.4552692.6728077.1812469.93S210.4552516.3327898.1912387.43R110.4553271.0028436.6912636.59R210.4553665.0028617.2112720.73Table 3
Summary statistics of technical indicators for KOSPI index.
IndicatorMin valueMax. valueMeanStandard deviation%K0.0010056.7531.506%D1.014100.0056.78529.343ROC87.16115.00100.021.477William %R0.00100.0043.2531.506Disparity 145.5917.7777.1460.1535Disparity 588.80108.99100.011.1707RSI12.1586.2551.8012.040Momentum−214.799222.3000.205729.245OSCP−1515−1237−130021.455CCI−309.162294.3908.929108.4Pivot point148625982068176.696S1138925882058177.400S2132125782047178.285R1151326082079176.264R2154426172089176.080Table 4
Summary statistics of technical indicators for Nikkei 225 index.
IndicatorMin valueMax. valueMeanStandard deviation%K0.0010059.7531.347%D0.218899.396759.74629.0587ROC83.92115.74100.091.877William %R0.00100.0040.2531.347Disparity 145.9487.8407.1600.1899Disparity 586.91111.31100.071.4681RSI4.16193.33553.35112.3618Momentum−1985.012658.8510.91313.129OSCP−1481−1219−129825.979CCI−387.11362.5617.04106.684Pivot point816524312168754828.938S1807524176167804812.77S2754424081166824795.813R1819524407169724848.784R2822924543170684867.672Table 5
Summary statistics of technical indicators for SZSE composite index.
IndicatorMin valueMax. valueMeanStandard deviation%K0.0010055.8332.222%D0.552699.732655.81830.044ROC85.68109.59100.042.34079William %R0.00100.0044.1732.222Disparity 145.6188.2837.1470.249Disparity 586.99107.67100.011.822RSI4.96792.24150.85912.723Momentum−2126.171385.1301.139257.862OSCP−1544−1183−130134.815CCI−386.890286.2546.046109.506Pivot point706118073100951765.377S1693517947100021739.134S268381779799051710.895R1712418224101921798.719R2723518404102861830.061Table 6
Percentage of increase and decrease cases of KSE-100.
YearIncrease%Decrease%Total201112851.6112048.38248201214257.0210742.97249201315462.349337.65247201413655.2811044.71246201512349.3912650.60249201614960.089939.91248201712349.3912650.06249201811647.3412952.65245201912550.6012249.4924720209954.098445.90183Table 7
Percentage of increase and decrease cases of KOSPI.
YearIncrease%Decrease%Total201112450.0012450.00248201212751.2012249.19249201312148.9812651.01247201412249.7912350.20245201512851.6112048.38248201613153.2511546.74246201713355.6410644.35239201812150.0012150.00242201913856.3210743.67245202010958.607741.39181Table 8
Percentage of increase and decrease cases of Nikkei 225.
YearIncrease%Decrease%Total201111646.7713253.22248201212851.6112048.38248201313454.6911145.30245201413053.2711446.72244201513153.6811346.31244201613856.3210743.67245201712650.0012650.00252201812450.4012249.49246201913254.7710945.2224120209452.228647.77180Table 9
Percentage of increase and decrease cases of SZSE composite.
YearIncrease%Decrease%Total201111847.5813052.41248201211244.9713755.02249201312148.9812651.01247201412651.2112048.78246201514257.0210742.97249201612349.5912550.40248201713253.0111746.98249201811145.3013454.69245201912550.6012249.39247202010758.467641.53183It is verified from Tables2–5 that the values of the stochastic oscillator %K and William %R vary between “0” and “100.” The four levels of pivot point, i.e., support 1 (S1), support 2 (S2), resistance 1 (R1), and resistance 2 (R2) have the same minimum value for the KSE-100 index, whereas for the other three indexes, the minimum values for these four levels are not the same as well as having and a slight variation amid its extreme values.It can be seen from the formulas of pivot point, S1, S2, R1, and R2 presented in Table1 that the previous day high, low, and closing stock prices are used to calculate its values. Pivot points are an intraday indicator for trading futures, commodities, and stocks. For example, traders know that if the price falls below the pivot point they will likely be shorting early in the session. Conversely, if the price is above the pivot point, they will be buying. The mathematical structure of pivot point and its four levels are probably same, that is why the minimum and maximum values are almost the same for the four stock price indexes.The mean and standard deviation of the pivot point and its four levels for all the four stock closing prices are approximately identical, demonstrating that they are strongly correlated which is theoretically sound as these four levels depend upon the pivot point.The percentage of the number of increase and decrease cases in each year is presented in Tables6–9. It is evident from these tables that there is nonlinear and disordered nature in the increase/decrease cases in each year for all four stock price indexes.During the pandemic period, Asian stock markets have shown greater resistance to the current pandemic of COVID-19. It can be seen from the investigational results presented in Tables7 and 9 that KOSPI and SZSE composite indices registered in 2020 showed the highest percentage of increase from all samples period as compared to the KSE-100 and Nikkei 225 indices.


## 4. Prediction Models

In this study, two different soft computing models have been used to predict the KSE-100 index, Nikkei 225 index, KOSPI index, and SZSE composite index daily closing price direction movement. A brief explanation of these models is outlined in the following segment of the paper.


### 4.1. ANN Model

Neural networks were first proposed in 1944 by Warren McCullough and Walter Pitts which are nonlinear mapping structures based on the function of the human brain. The main idea of ANN is derived from the neuron nodes interrelated similar to a net. The brain consists of hundreds of billions of neuron nodes that are responsible for processing information towards and from the brain. Just like the brain, ANN has also consisted of thousands of artificial neurons connected by nodes. The input components accept several methods and composition of data, built on an internal weighting structure, while NN tries to acquire information obtainable to yield one output information. ANNs correspondingly practice an established rule of directions named back-propagation as humans require instructions and directions to originate an outcome. A general architect of the “N” layer feed-forward neural network (FFNN) is presented in Figure 1. In this study, the resilient back-propagation learning procedure is proposed to train a single-layer feed-forward neural network. The benefit of resilient back-propagation to train the model over the traditional BPNN is that it takes minimum time to train the architect of the neural net and does not require any learning rate [25]. The threshold value of 0.04 with logistic as activation function is employed to predict the direction movement of the four different stock prices direction movement. The input layers for the network are 15 technical indicators (presented in Table 1), and the single output layer is known as the directional movement of the daily stock closing price taking the value of either “0” or “1.” If the output value is greater or equal to 0.5, the prediction of direction movement is considered to be upward, and if it is less than 0.5, it is considered to be downward. From the investigations of the literature review, a small threshold value of 0.04 is selected to train the FFNN model with a resilient back-propagation learning algorithm. Table 10 shows all the parameter settings for the proposed FFNN model. All the investigational results were found using the “neuralnet” package in Rstudio.Figure 1
The architect ofN hidden layers feed-forward neural network.Table 10
Tuning parameters for the feed-forward neural network model.
ParametersLevelsHidden layer1Neurons in hidden layer1Threshold0.04Activation functionLogisticStopping criteria (epoch)1591316AlgorithmResilient back-propagationAccuracy levelCross entropyError718.41


### 4.2. SVM Model

The method of SVM was developed by Vapnik [19], and until now, it is the most widely used supervised machine learning technique that is based on statistical theory and structured risk minimization rule. In practical problems, the method of SVM can be used both for classification as well as regression problems. However, it is mostly used in classification problems. The key concept of SVM is to contract the upper bound of generalization error which is a different approach to the other machine learning techniques like backpropagation network (BPN) that works on the principle of minimizing the empirical error [26].In the SVM algorithm, each data item is plotted as a point inn-dimensional space (where n is the number of features) with the value of each feature being the value of a particular coordinate. Then, classification is performed by finding the hyperplane that differentiates the two classes very well. Consider that yj∈Rc,j=1,2,…,n describe a set of input vectors with equivalent class labels denoted by xj∈+1,−1,j=1,2,…,N. The SVM algorithm has the potential to transform the input vectors yj∈Rc into a high-dimensional feature space φyj∈H. The SVM kernel is a function that takes low dimensional input space and transforms it to a higher dimensional space, i.e., it converts nonseparable problems to separable problems. It is mostly useful in nonlinear separation problems. To draw a hyperplane in a nonlinear separation problem, a kernel function K (xi, xj) is used that accomplishes the mapping θ⋅. The final limiting function of the hyperplane is expressed by the following mathematical structure:(2)fx=sgn∑i=1nyiαi.Kx,xi+b.The values ofαi in (2) can be found by solving the equation:(3)Maximize∑i=1nαi−12∑i=1n∑j=1nαiαj⋅xixj⋅Kyi,yj.Equation (3) holds only if 0≤αi≤c and ∑i=1nαixi=0,fori=1,2,…,n. There is a trade-off between the misclassification error and maximizing the margin that can be controlled by the parameter “C,” technically known as the regularization parameter. The most well-known and widely used kernels for SVM are linear, polynomial, and radial basis. We have considered the linear, RBF kernel, and polynomial kernels. It is evident from Table 5 that the precision of the SVM with radial bases kernel is maximum; therefore, is suggested to train the model. The mathematical structure of linear and radial bases kernel is defined in equations (4) and (5):(4)Linear Kernel:Kxi,xj=a+∑i=1nbi⋅x,xi,(5)RBF Kernel:Kxi,xj=exp−σxi−xj2,(6)Polynomial kernel:Kxi,xj=σxi⋅xj+Cd,where σ is the kernel parameter used in both RBF and polynomial kernels.For the linear kernel, the only hyperparameter that needs to be optimized is “C.” However, for the RBF kernel, both the regularization parameter C and the kernel parameter σ need to be optimized simultaneously with the help of grid search, whereas for the polynomial kernel we have three parameters to be tuned, i.e., C, d, and σ.


#### 4.2.1. Parameter Optimization Using a Grid Search

A grid search is a comprehensive method for the optimization of the model parameter based on the predefined set of the hyperparameter space using lower and upper bounds with the number of steps defined. Three different scales that are linear, quadratic, and logarithmic scales are used for this purpose, and their performance is evaluated using some performance metrics.Cross-validation (CV) can be used as a performance metric for the optimization of the SVM parameters (e.g.,C, degree, etc.) with the main objective is to identify a good combination of hyperparameter so that the classifier can make accurate predictions. Cross validation is a useful technique to avoid the problem of overfitting [27]. The flowchart of SVM parameter optimization using grid search is presented in Figure 2. However, to choose the best values of the parameters C and σ using k-fold CV, the data is splitted into “k” subsets (for example, k = 10) where “k − 1” subsets are used for training and the remaining one set as testing subset. Based on this splitting, the cross-validation error for the SVM classifier is computed for different combination of hyperparameters, and the best combination of hyperparameter values is selected based on highest accuracy or lower CV error for training of the SVM model. The results of the analyzed stock indices are presented in Figures 3–6. Moreover, there is one parameter C to be optimized in linear kernel, two parameters C and σ in RBF, and three parameters C, σ, and degree in polynomial. There can be additional parameters; however, a large number of parameters may result in a huge number of combinations. One of the limitations of the SVM parameters is that there are no exact ranges of C and σ values.Figure 2
Tuning of SVM parameters using a grid search.Figure 3
Optimized parameter selection using the grid search technique for KSE-100 index with (a) linear kernel, (b) RBF kernel, and (c) polynomial kernel.
(a)(b)(c)Figure 4
Optimized parameter selection using the grid search technique for Nikkei 225 index with (a) linear kernel, (b) RBF kernel, and (c) polynomial kernel.
(a)(b)(c)Figure 5
Optimized parameter selection using the grid search technique for KOSPI index with (a) linear kernel, (b) radial basis kernel, and (c) polynomial kernel of degree 2.
(a)(b)(c)Figure 6
Optimized parameter selection using the grid search technique for SZSE component index with (a) linear kernel, (b) radial basis kernel, and (c) polynomial kernel of degree 2.
(a)(b)(c)It is believed that the wider the parameter range is, the more possibilities the grid search method may have in finding the best combination of the parameters. Therefore, for this experiment, it was decided to choose the optimum range ofC and σ from 0.001 to 1000.There is a direct relationship between the values ofσ and C, for larger values, the effect of C became negligible if σ is small, C affects the SVM model with RBF kernel in the same way as it affects using a linear kernel. By using the grid search optimization technique, the maximum prediction accuracy for linear, RBF, and polynomial kernels using three different stock prices time series data reached its maximum and can be seen in Table 11. Furthermore, the investigational results suggest that the prediction accuracy of the SVM with linear, and RBF kernel for the four different stock prices that are optimized by the grid search method is maximum, and hence used in this study.Table 11
Tuning parameter selection for the SVM model using 10-fold cross-validation.
Index typeKernelAccuracyF-scoreTuning parametersKSE-100Linear0.85190.8395C = 964.7736Radial basis0.76880.7564C = 137.20, σ=0.0909Polynomial0.84380.8344C = 314.52, scale = 0.5554, degree = 2Nikkei 225Linear0.80220.7912C = 638.0629RBF0.76260.7354C = 1.596, σ=0.0059Polynomial0.78280.7532C = 314.52, scale = 0.5554, degree = 2KOSPILinear0.80330.7822C = 4RBF0.81800.7932C = 150, σ=0.00528Polynomial0.80330.7745C = 49.298, scale = 1.042, degree = 1SZSE compositeLinear0.89980.8790C = 324.72RBF0.87200.8412C = 464.666, σ=0.00180Polynomial0.89410.8620C = 110.17, scale = 0.822, degree = 2


## 4.1. ANN Model

Neural networks were first proposed in 1944 by Warren McCullough and Walter Pitts which are nonlinear mapping structures based on the function of the human brain. The main idea of ANN is derived from the neuron nodes interrelated similar to a net. The brain consists of hundreds of billions of neuron nodes that are responsible for processing information towards and from the brain. Just like the brain, ANN has also consisted of thousands of artificial neurons connected by nodes. The input components accept several methods and composition of data, built on an internal weighting structure, while NN tries to acquire information obtainable to yield one output information. ANNs correspondingly practice an established rule of directions named back-propagation as humans require instructions and directions to originate an outcome. A general architect of the “N” layer feed-forward neural network (FFNN) is presented in Figure 1. In this study, the resilient back-propagation learning procedure is proposed to train a single-layer feed-forward neural network. The benefit of resilient back-propagation to train the model over the traditional BPNN is that it takes minimum time to train the architect of the neural net and does not require any learning rate [25]. The threshold value of 0.04 with logistic as activation function is employed to predict the direction movement of the four different stock prices direction movement. The input layers for the network are 15 technical indicators (presented in Table 1), and the single output layer is known as the directional movement of the daily stock closing price taking the value of either “0” or “1.” If the output value is greater or equal to 0.5, the prediction of direction movement is considered to be upward, and if it is less than 0.5, it is considered to be downward. From the investigations of the literature review, a small threshold value of 0.04 is selected to train the FFNN model with a resilient back-propagation learning algorithm. Table 10 shows all the parameter settings for the proposed FFNN model. All the investigational results were found using the “neuralnet” package in Rstudio.Figure 1
The architect ofN hidden layers feed-forward neural network.Table 10
Tuning parameters for the feed-forward neural network model.
ParametersLevelsHidden layer1Neurons in hidden layer1Threshold0.04Activation functionLogisticStopping criteria (epoch)1591316AlgorithmResilient back-propagationAccuracy levelCross entropyError718.41


## 4.2. SVM Model

The method of SVM was developed by Vapnik [19], and until now, it is the most widely used supervised machine learning technique that is based on statistical theory and structured risk minimization rule. In practical problems, the method of SVM can be used both for classification as well as regression problems. However, it is mostly used in classification problems. The key concept of SVM is to contract the upper bound of generalization error which is a different approach to the other machine learning techniques like backpropagation network (BPN) that works on the principle of minimizing the empirical error [26].In the SVM algorithm, each data item is plotted as a point inn-dimensional space (where n is the number of features) with the value of each feature being the value of a particular coordinate. Then, classification is performed by finding the hyperplane that differentiates the two classes very well. Consider that yj∈Rc,j=1,2,…,n describe a set of input vectors with equivalent class labels denoted by xj∈+1,−1,j=1,2,…,N. The SVM algorithm has the potential to transform the input vectors yj∈Rc into a high-dimensional feature space φyj∈H. The SVM kernel is a function that takes low dimensional input space and transforms it to a higher dimensional space, i.e., it converts nonseparable problems to separable problems. It is mostly useful in nonlinear separation problems. To draw a hyperplane in a nonlinear separation problem, a kernel function K (xi, xj) is used that accomplishes the mapping θ⋅. The final limiting function of the hyperplane is expressed by the following mathematical structure:(2)fx=sgn∑i=1nyiαi.Kx,xi+b.The values ofαi in (2) can be found by solving the equation:(3)Maximize∑i=1nαi−12∑i=1n∑j=1nαiαj⋅xixj⋅Kyi,yj.Equation (3) holds only if 0≤αi≤c and ∑i=1nαixi=0,fori=1,2,…,n. There is a trade-off between the misclassification error and maximizing the margin that can be controlled by the parameter “C,” technically known as the regularization parameter. The most well-known and widely used kernels for SVM are linear, polynomial, and radial basis. We have considered the linear, RBF kernel, and polynomial kernels. It is evident from Table 5 that the precision of the SVM with radial bases kernel is maximum; therefore, is suggested to train the model. The mathematical structure of linear and radial bases kernel is defined in equations (4) and (5):(4)Linear Kernel:Kxi,xj=a+∑i=1nbi⋅x,xi,(5)RBF Kernel:Kxi,xj=exp−σxi−xj2,(6)Polynomial kernel:Kxi,xj=σxi⋅xj+Cd,where σ is the kernel parameter used in both RBF and polynomial kernels.For the linear kernel, the only hyperparameter that needs to be optimized is “C.” However, for the RBF kernel, both the regularization parameter C and the kernel parameter σ need to be optimized simultaneously with the help of grid search, whereas for the polynomial kernel we have three parameters to be tuned, i.e., C, d, and σ.


### 4.2.1. Parameter Optimization Using a Grid Search

A grid search is a comprehensive method for the optimization of the model parameter based on the predefined set of the hyperparameter space using lower and upper bounds with the number of steps defined. Three different scales that are linear, quadratic, and logarithmic scales are used for this purpose, and their performance is evaluated using some performance metrics.Cross-validation (CV) can be used as a performance metric for the optimization of the SVM parameters (e.g.,C, degree, etc.) with the main objective is to identify a good combination of hyperparameter so that the classifier can make accurate predictions. Cross validation is a useful technique to avoid the problem of overfitting [27]. The flowchart of SVM parameter optimization using grid search is presented in Figure 2. However, to choose the best values of the parameters C and σ using k-fold CV, the data is splitted into “k” subsets (for example, k = 10) where “k − 1” subsets are used for training and the remaining one set as testing subset. Based on this splitting, the cross-validation error for the SVM classifier is computed for different combination of hyperparameters, and the best combination of hyperparameter values is selected based on highest accuracy or lower CV error for training of the SVM model. The results of the analyzed stock indices are presented in Figures 3–6. Moreover, there is one parameter C to be optimized in linear kernel, two parameters C and σ in RBF, and three parameters C, σ, and degree in polynomial. There can be additional parameters; however, a large number of parameters may result in a huge number of combinations. One of the limitations of the SVM parameters is that there are no exact ranges of C and σ values.Figure 2
Tuning of SVM parameters using a grid search.Figure 3
Optimized parameter selection using the grid search technique for KSE-100 index with (a) linear kernel, (b) RBF kernel, and (c) polynomial kernel.
(a)(b)(c)Figure 4
Optimized parameter selection using the grid search technique for Nikkei 225 index with (a) linear kernel, (b) RBF kernel, and (c) polynomial kernel.
(a)(b)(c)Figure 5
Optimized parameter selection using the grid search technique for KOSPI index with (a) linear kernel, (b) radial basis kernel, and (c) polynomial kernel of degree 2.
(a)(b)(c)Figure 6
Optimized parameter selection using the grid search technique for SZSE component index with (a) linear kernel, (b) radial basis kernel, and (c) polynomial kernel of degree 2.
(a)(b)(c)It is believed that the wider the parameter range is, the more possibilities the grid search method may have in finding the best combination of the parameters. Therefore, for this experiment, it was decided to choose the optimum range ofC and σ from 0.001 to 1000.There is a direct relationship between the values ofσ and C, for larger values, the effect of C became negligible if σ is small, C affects the SVM model with RBF kernel in the same way as it affects using a linear kernel. By using the grid search optimization technique, the maximum prediction accuracy for linear, RBF, and polynomial kernels using three different stock prices time series data reached its maximum and can be seen in Table 11. Furthermore, the investigational results suggest that the prediction accuracy of the SVM with linear, and RBF kernel for the four different stock prices that are optimized by the grid search method is maximum, and hence used in this study.Table 11
Tuning parameter selection for the SVM model using 10-fold cross-validation.
Index typeKernelAccuracyF-scoreTuning parametersKSE-100Linear0.85190.8395C = 964.7736Radial basis0.76880.7564C = 137.20, σ=0.0909Polynomial0.84380.8344C = 314.52, scale = 0.5554, degree = 2Nikkei 225Linear0.80220.7912C = 638.0629RBF0.76260.7354C = 1.596, σ=0.0059Polynomial0.78280.7532C = 314.52, scale = 0.5554, degree = 2KOSPILinear0.80330.7822C = 4RBF0.81800.7932C = 150, σ=0.00528Polynomial0.80330.7745C = 49.298, scale = 1.042, degree = 1SZSE compositeLinear0.89980.8790C = 324.72RBF0.87200.8412C = 464.666, σ=0.00180Polynomial0.89410.8620C = 110.17, scale = 0.822, degree = 2


## 4.2.1. Parameter Optimization Using a Grid Search

A grid search is a comprehensive method for the optimization of the model parameter based on the predefined set of the hyperparameter space using lower and upper bounds with the number of steps defined. Three different scales that are linear, quadratic, and logarithmic scales are used for this purpose, and their performance is evaluated using some performance metrics.Cross-validation (CV) can be used as a performance metric for the optimization of the SVM parameters (e.g.,C, degree, etc.) with the main objective is to identify a good combination of hyperparameter so that the classifier can make accurate predictions. Cross validation is a useful technique to avoid the problem of overfitting [27]. The flowchart of SVM parameter optimization using grid search is presented in Figure 2. However, to choose the best values of the parameters C and σ using k-fold CV, the data is splitted into “k” subsets (for example, k = 10) where “k − 1” subsets are used for training and the remaining one set as testing subset. Based on this splitting, the cross-validation error for the SVM classifier is computed for different combination of hyperparameters, and the best combination of hyperparameter values is selected based on highest accuracy or lower CV error for training of the SVM model. The results of the analyzed stock indices are presented in Figures 3–6. Moreover, there is one parameter C to be optimized in linear kernel, two parameters C and σ in RBF, and three parameters C, σ, and degree in polynomial. There can be additional parameters; however, a large number of parameters may result in a huge number of combinations. One of the limitations of the SVM parameters is that there are no exact ranges of C and σ values.Figure 2
Tuning of SVM parameters using a grid search.Figure 3
Optimized parameter selection using the grid search technique for KSE-100 index with (a) linear kernel, (b) RBF kernel, and (c) polynomial kernel.
(a)(b)(c)Figure 4
Optimized parameter selection using the grid search technique for Nikkei 225 index with (a) linear kernel, (b) RBF kernel, and (c) polynomial kernel.
(a)(b)(c)Figure 5
Optimized parameter selection using the grid search technique for KOSPI index with (a) linear kernel, (b) radial basis kernel, and (c) polynomial kernel of degree 2.
(a)(b)(c)Figure 6
Optimized parameter selection using the grid search technique for SZSE component index with (a) linear kernel, (b) radial basis kernel, and (c) polynomial kernel of degree 2.
(a)(b)(c)It is believed that the wider the parameter range is, the more possibilities the grid search method may have in finding the best combination of the parameters. Therefore, for this experiment, it was decided to choose the optimum range ofC and σ from 0.001 to 1000.There is a direct relationship between the values ofσ and C, for larger values, the effect of C became negligible if σ is small, C affects the SVM model with RBF kernel in the same way as it affects using a linear kernel. By using the grid search optimization technique, the maximum prediction accuracy for linear, RBF, and polynomial kernels using three different stock prices time series data reached its maximum and can be seen in Table 11. Furthermore, the investigational results suggest that the prediction accuracy of the SVM with linear, and RBF kernel for the four different stock prices that are optimized by the grid search method is maximum, and hence used in this study.Table 11
Tuning parameter selection for the SVM model using 10-fold cross-validation.
Index typeKernelAccuracyF-scoreTuning parametersKSE-100Linear0.85190.8395C = 964.7736Radial basis0.76880.7564C = 137.20, σ=0.0909Polynomial0.84380.8344C = 314.52, scale = 0.5554, degree = 2Nikkei 225Linear0.80220.7912C = 638.0629RBF0.76260.7354C = 1.596, σ=0.0059Polynomial0.78280.7532C = 314.52, scale = 0.5554, degree = 2KOSPILinear0.80330.7822C = 4RBF0.81800.7932C = 150, σ=0.00528Polynomial0.80330.7745C = 49.298, scale = 1.042, degree = 1SZSE compositeLinear0.89980.8790C = 324.72RBF0.87200.8412C = 464.666, σ=0.00180Polynomial0.89410.8620C = 110.17, scale = 0.822, degree = 2


## 5. Results and Discussion

Here, the SVM and ANN models are used as classifiers; therefore, accuracy andF-score are implemented as precision metrics to check the efficiency of the suggested models. Before calculating accuracy and F-score, precision and recall are calculated by utilizing the confusion matrix, i.e., true positive (TP), false positive (FP), true negative (TN), and false negative (FN). The mathematical composition of these performance measures is defined in the following equations:(7)Precisionpositive=TPTP+FP,(8)Precisionnegative=TNTN+FN,(9)Recallpositive=TPTP+FN,(10)Recallnegative=TNTN+FP,(11)Accuracy=TP+TNTP+FP+TN+FN,(12)F−score=2×Precision×RecallPrecision+Recall,where precision is the ratio of true positive to all true and false positive. Since the main objective is the prediction of direction movement of the four stock indexes closing prices, true positive (TP) means that the direction movement is upward and the suggested model correctly predicted it. Similarly, FP means that the proposed model incorrectly predicted the direction movement as upward when it is downward. The true negative (TN) means that the proposed model correctly predicted the direction movement as downward, and the false negative (FN) means that the direction movement is actually upward and the suggested model predicted it incorrectly as downward. Both the SVM and ANN models are built with the best choices of hyperparameter selection as reported in Tables 11 and 12.Table 12
Prediction performance of ANN and SVM models.
IndexSingle-layer ANNSVMAccuracyF-scoreAccuracyF-scoreKernelKSE-1000.90110.88980.85190.8395LinearKOSPI0.85760.83140.81800.7932RBFNikkei 2250.84650.82770.80220.7912LinearSZSE composite0.95130.93450.89980.8790LinearThe empirical analysis consists of two steps that are training the models with optimization of hyperparameters using the grid search approach followed by checking its forecasting accuracy on the testing data. The optimization of the RBF kernel with two tuning parameters is conducted by usingk-fold cross-validation. The results of this cross-validation are presented in Figures 3–6, whereas the performance measures of the two models are presented in Table 12.Furthermore, it can be observed from Table11 that the accuracy and F-score for the KSE-100 index, Nikkei 225, and SZSE composite is maximum with a linear kernel with the single-tuned parameter using the grid search approach, i.e., the value of C = 964.77, 638.0629, and 324.72. The accuracy and F-score for these three indexes are (0.8519, 0.8395), (0.8022, 0.7912), and (0.8998, 0.8790), whereas for the KOSPI index, the maximum accuracy and F-score (0.8180, 0.7932) is achieved by using RBF kernel with tuned hyperparameters values of C = 150 and σ=0.00528. After the optimization of the hyperparameters of both the models, accuracy metrics defined in equations (7)–(12) are calculated for the testing dataset, and investigational results are presented in Table 12.Similarly, the feed-forward ANN model with a resilient back-propagation learning algorithm is used to train the model as per the input results presented in Table10. Here, a single hidden layer with one neuron is used to train the feed-forward ANN model that gives an accuracy level between 84 percent and 95 percent and therefore using more than one layer may probably lead to the problem of overfitting. After training the two proposed models with the best parameter selection using grid search, the next step is to compare them by using accuracy and F-score. These two-performance metrics are used for test data, and their results are presented in Table 12.It is evident from the results presented in Table12 that the performance of predicting the direction movement of closing prices of KSE-100, KOSPI, Nikkei 225, and SZSE composite indexes of ANN model is maximum than that of SVM model with linear and RBF kernel. The accuracy and F-score presented in the above table were found using equations (11) and (12). It can be observed from Table 12 that the accuracy of ANN model for KSE-100, KSOPI, Nikkei 225, and SZSE composite are 0.9011, 0.8576, 0.8456, and 0.9513, and the corresponding values of F-score are 0.8898, 0.8314, 0.8277, and 0.9345, respectively. These results are slightly higher than that of SVM model based on linear and RBF kernel with its accuracy 0.8519, 0.8180, 0.8022, and 0.8998 and its corresponding F-scores 0.8395, 0.7932, 0.7912, and 0.8790, respectively, suggesting it as a good competitor to the ANN model. The average prediction accuracy of the ANN model for the four different stock prices is 88.91% with an F-score of 87.08% suggesting that its performance is better than that of the SVM model with an average accuracy of 84.29% and an average F-score of 82.57%, respectively.


## 6. Conclusion

Since it is well-known fact that the behavior of stock market data is complex and nonlinear and that its prediction is not only challenging but a difficult task. The main purpose of this research study is to predict the direction movement of the closing price of the four different stock prices, namely, the KSE-100 index, KOSPI index, Nikkei 225 index, and SZSE composite using the two well-known machine learning techniques, i.e., ANN and SVM models. These two models are trained by using the ten years of historical data (i.e., from 2011 to 2020) of daily closing prices of these four stock price indexes from the yahoo finance official website. Fifteen various technical indicators have been extracted from the historical stock data and used as input layers to train the two models. For all the four stock indexes, the single-layer ANN model prediction accuracy and correspondingF-scores showed to be higher than the SVM model (with three different kernels, i.e., linear, RBF, and polynomials).However, given the accuracy and F-score values obtained for both the ANN and SVM models they are considered to be useful tools with the best possible input layers to predict the direction movement of the stocks and stock price indexes, which will minimize the risk of loss and will improve confidence level of the investors to invest in the stock markets such as the KSE-100 index, KOSPI index, Nikkei 225 index, and SZSE composite index. As evident from the results, the prediction accuracy of the ANN model is higher than that of SVM model which confirms that why many researchers in the previous studies (e.g., 7, 8, 20, 21) used these two models to predict the stock market indexes which outperforms other models utilized in the literature for the same purpose. Limitation of this study is the noninclusion of the impact of other macroeconomic variables such as foreign exchange rates, interest rates, and consumer price as input layers to the proposed models. Nonetheless, fifteen technical indicators used in this study proved to be very useful in predicting the direction movement of the KSE-100, KOSPI, Nikkei 255, and SZSE composite indexes using the ANN and SVM models.As future work, prediction of the stock prices by implementing an ensemble model based on a long-short-term memory model (LSTM) with different input layers is under consideration. Although most of the stock prices have a daily pattern as in this study, this can be different in other areas, such as there may be a seasonal pattern on some sales datasets. Therefore, an attempt will be made to use different lengths of historical data, different lengths of binary features, and other driving features such as exchange rates and other macroeconomic indicators.



---

*Source: 2906463-2021-12-02.xml*