International Journal of Computer Science and Telecommunications [Volume 4, Issue 12, December 2013]

17

A Machine Learning Model for Stock Market
Prediction
ISSN 2047-3338

1

2

3

Osman Hegazy , Omar S. Soliman and Mustafa Abdul Salam
1,2
Faculty of Computers and Informatics, Cairo University, Egypt
3
Higher Technological Institute (H.T.I), 10th of Ramadan City, Egypt
2
dr_omar_soliman@yahoo.com, 3mustafa.abdo@ymail.com

Abstract– Stock market prediction is the act of trying to
determine the future value of a company stock or other financial
instrument traded on a financial exchange. The successful
prediction of a stock's future price will maximize investor’s
gains. This paper proposes a machine learning model to predict
stock market price. The proposed algorithm integrates Particle
swarm optimization (PSO) and least square support vector
machine (LS-SVM). The PSO algorithm is employed to optimize
LS-SVM to predict the daily stock prices. Proposed model is
based on the study of stocks historical data and technical
indicators. PSO algorithm selects best free parameters
combination for LS-SVM to avoid over-fitting and local minima
problems and improve prediction accuracy. The proposed model
was applied and evaluated using thirteen benchmark financials
datasets and compared with artificial neural network with
Levenberg-Marquardt (LM) algorithm. The obtained results
showed that the proposed model has better prediction accuracy
and the potential of PSO algorithm in optimizing LS-SVM.
Index Terms– Least Square Support Vector Machine, Particle
Swarm Optimization, Technical Indicators and Stock Price
Prediction

I.

INTRODUCTION

S

TOCK price prediction has been at focus for years since it can
yield significant profits. Predicting the stock market is not a
simple task, mainly as a consequence of the close to randomwalk behavior of a stock time series. Fundamental and technical
analyses were the first two methods used to forecast stock prices.
Artificial Neural networks (ANNs) is the most commonly used
technique [1]. In most cases ANNs suffer from over-fitting problem
due to the large number of parameters to fix, and the little prior user
knowledge about the relevance of the inputs in the analyzed
problem [2].
Also, Support vector machines (SVMs) had been developed as an
alternative that avoids such limitations. Their practical successes can
be attributed to solid theoretical foundations based on VC-theory [3].
SVM compute globally optimal solutions, unlike those obtained with
ANNs, which tend to fall into local minima [4].
Least squares –support vector machines (LS-SVM) method was
presented in [5], which was reformulated the traditional SVM
algorithm. LS-SVM uses a regularized least squares function with
equality constraints, leading to a linear system which meets the
Karush-Kuhn-Tucker (KKT) conditions for obtaining an optimal
solution. Although LS-SVM simplifies the SVM procedure, the

Journal Homepage: www.ijcst.org

regularization parameter and the kernel parameters play an important
role in the regression system. Therefore, it is necessary to establish a
methodology for properly selecting the LS-SVM free parameters, in
such a way that the regression obtained by LS-SVM must be robust
against noisy conditions, and it does not need priori user knowledge
about the influence of the free parameters values in the problem
studied [6].
The perceived advantages of evolutionary strategies as
optimization methods motivated some researchers to consider such
stochastic methods in the context of optimizing SVM. A survey and
overview of evolutionary algorithms (EAs) found in [7]. Particle
swarm optimization (PSO) is one of the most used EAs. PSO is a
recently proposed algorithm by James Kennedy and Russell Eberhart
in 1995, motivated by social behavior of organisms such as bird
flocking and fish schooling [8]. The optimizer which is used in the
particle swarm optimization algorithm, while making adjustment
towards "local" and "global" best particles, is conceptually similar to
the crossover operation used by genetic algorithms [9]. As well
particle swarm optimization includes fitness function, which
measures the closeness of the corresponding solution to the
optimum. The main difference of particle swarm optimization
concept from the evolutionary computing is that flying potential
solutions through hyperspace are accelerating toward "better"
solutions, while in evolutionary computation schemes operate
directly on potential solutions which are represented as locations in
hyperspace [10]. SVM was used in stock market forecasting in [11].
Financial time series forecasting using SVM optimized by PSO was
presented in [12]. The Optimization of Share Price Prediction Model
Based on Support Vector Machine‫ ‏‬is presented in [14]. Financial
time series forecasting based on wavelet kernel support vector was
presented in [15]. Computational Intelligence Approaches for Stock
Price Forecasting was introduced in [16]. A hybrid approach by
integrating wavelet-based feature extraction with MARS and SVR
for stock index forecasting was presented in [17]. An interval type-2
Fuzzy Logic based system for modeling generation and
summarization of arbitrage opportunities in stock markets was
presented in [18]. Robust stock trading using fuzzy decision trees is
presented in [19]. Ensemble ANNs-PSO-GA Approach for Dayahead Stock E-exchange Prices Forecasting was presented in [20].
Index prediction with neuro-genetic hybrid network was proposed in
[21]. A hybrid fuzzy intelligent agent-based system for stock price
prediction was introduced in [22]. Improved Stock Market Prediction
by Combining Support Vector Machine and Empirical Mode
Decomposition was presented in [23]. Neural Network Ensemble
Model Using PPR and LS-SVR for Stock Market Forecasting was
proposed in [24]. Computational Intelligence Techniques for Risk
Management in Decision Making was introduced in [25]. Stock

Osman Hegazy et al.
market prediction algorithm using Hidden Markov Models was
proposed presented in [26]. Neural Networks and Wavelet DeNoising for Stock Trading and Prediction was introduced in [27].
The aim of this paper is to develop a machine learning model that
hybrids the PSO and LS-SVM model. The performance of LS-SVM
is based on the selection of free parameters C (cost penalty), ϵ
(insensitive-loss function) and γ (kernel parameter). PSO will be
used to find the best parameter combination for LS-SVM.
The paper is organized as follows: Section 2 presents the Least
square support vector machine algorithm; The Particle swarm
optimization algorithm is introduced in section 3; The proposed
model and its implementation stock prediction are discussed in
section 4 Section 5 introduces excremental results and discussions.
Finally, section 6 is devoted to conclusions of the proposed model.

II. LEAST SQUARE SUPPORT VECTOR MACHINE
Least squares support vector machines (LS-SVM) are least
squares versions of support vector machines (SVM), which
are a set of related supervised learning methods that analyze
data and recognize patterns, and which are used
for classification and regression analysis. In this version one
finds the solution by solving a set of linear equations instead
of a convex quadratic programming (QP) problem for
classical SVMs. Least squares SVM classifiers, were
proposed by Suykens and Vandewalle [28].

vector. Given the {xi , yi }

1

I

 Z T  W   0 

 1T  b   0 
  
 I  e   0 

0   a   y 

Z  [ ( x1),  ( x2),...,  ( xn )]T
From rows one and three in (3)

Then, by defining the kernel matrix K
1

0
1T  b   0 

    
1
K


I

 a   y 



Polynomial kernel of degree d:

vectors,
function (1) from a similar minimization problem found in the
SVM method [3]. However the main difference is that LSSVM involves equality constraints instead of inequalities, and
it is based on a least square cost function. Furthermore, the
LS-SVM method solves a linear problem while conventional
SVM solves a quadratic one. The optimization problem and
the equality constraints of LS-SVM are defined as follows:

1 T
1 T
min j(w,e,b)  w wC e
2
2
w, e, b

(2)

yi  wT  ( xi )  b  ei

(3)

n  1 error vector, 1 is a n  1 vector with all
C  R  is the tradeoff parameter between the

Where e is the

entries 1, and
solution size and training errors. From (2) a Lagranian is
formed, and differentiating with respect to w, b, e, a ( a is
Largrangian multipliers), we obtain;

K ( x, xi )  xiT x

(7)

K ( x, xi )  (1  xiT x / c) d




(8)

Radial basis function RBF kernel :

K ( x, xi )  exp(  x  xi

(1)

W and  (x) : R p  R n are n  1 column
and b  R . LS-SVM algorithm [5] computes the

(6)

Kernel function K types are as follows:

training data set, where

f ( x)  W  ( x )  b

 ZZ T , and the

parameter   C , the conditions for optimality lead to the
following overall solution

Linear kernel

T

(5)

w  Z T a and Ce  a



xi  R and yi  R , the LS-SVM goal is to construct the
function f ( x)  y , which represents the dependence of the
output y i on the input x i . This function is formulated as:

(4)

Where I represents the identity matrix and

p

Where

0 0
0 0
0 CI

n  1 output

Let X is n  p input data matrix and y is
n
i 1

I

0
0

 Z

18

2

/ 2 )

(9)

MLP kernel :

K ( x, xi )  tanh(kxiT x   )

(10)

In this work, MLP kernel is used.
III.

PARTICLE SWARM OPTIMIZATION
ALGORITHM

PSO is a relatively recent heuristic search method which is derived
from the behavior of social groups like bird flocks or fish swarms.
PSO moves from a set of points to another set of points in a single
iteration with likely improvement using a combination of
deterministic and probabilistic rules. The PSO has been popular in
academia and industry, mainly because of its intuitiveness, ease of
implementation, and the ability to effectively solve highly nonlinear,
mixed integer optimization problems that are typical of complex
engineering systems. Although the “survival of the fittest” principle
is not used in PSO, it is usually considered as an evolutionary
algorithm. Optimization is achieved by giving each individual in the
search space a memory for its previous successes, information about
successes of a social group and providing a way to incorporate this
knowledge into the movement of the individual.

International Journal of Computer Science and Telecommunications [Volume 4, Issue 12, December 2013]
Therefore, each individual (called particle) is characterized by its





position xi , its velocity vi , its personal best position
neighborhood best position


pg .


pi

Data Acquisition and Preprocessing (Stocks Histoical data)

and its
Feature Extraction and selection (Techincal Indicators)

The elements of the velocity vector for particle i are updated as:

ij  ij  c1q( xijpb  xij )  c2 ( x sbj  xij ), j  1,.., n
Where w is the inertia weight,

19

Optimizing and training LS-SVM with PSO algorithm

(11)

Testing LS-SVM-PSO model with new data

xipb is the best variable vector

Computing MSE for( LS-SVM, LS-SVM-PSO, and ANN)

sb

encountered so far by particle i , and x is the swarm best
vector, i.e. the best variable vector found by any particle in
the swarm, so far c1 and c 2 are constants, and q and r are
random numbers in the range (0, 1). Once the velocities have
been updated, the variable vector of particle i is modified
according to:

xij  xij  ij , j  1,..., n.

(12)

The cycle of evaluation followed by updates of velocities
and positions (and possible update of

xipb and x sb ) is then

repeated until a satisfactory solution has been found. PSO
algorithm is shown in Fig. 1.

Fig. 2: The Proposed Model

The proposed algorithm was tested for many companies
which cover all stock sectors in S&P 500 stock market. These
sectors are Information Technology (Adobe, Hp, and Oracle);
Financials (American Express and Bank of New York);
Health Care (Life Technologies, and Hospera); Energy
(Exxon-Mobile and Duck energy); Communications (AT&T);
Materials (FMC Corporation); Industrials (Honey Well).
Five technical indicators are calculated from the raw
datasets:


Relative Strength Index (RSI): A technical momentum
indicator that compares the magnitude of recent gains to
recent losses in an attempt to determine overbought and
oversold conditions of an asset. The formula for
computing the Relative Strength Index is as follows.
RSI = 100- [100 / (1+RS)]

(13)

Where RS = Avg. of x days’ up closes / Average of x
days’‫‏‬down closes.


Money Flow Index (MFI): This one measures the
strength of money in and out of a security. The formula
for MFI is as follows:
Money Flow (MF) = Typical Price * Volume.

(14)

Money Ratio (MR) = (Positive MF / Negative MF). (15)
MFI = 100 – (100/ (1+MR)).

Fig. 1: PSO Algorithm


IV.

THE PROPOSED MODEL

The proposed model is based on the study of historical data,
technical indicators and optimizing LS-SVM with PSO
algorithm to be used in the prediction of daily stock prices.
Levenberg-Marquardt (LM) algorithm is used as a benchmark
for comparison with LS-SVM and LS-SVM-PSO models. The
proposed model architecture contains six inputs vectors
represent the historical data and derived technical indicators
and one output represents next price.

(16)

Exponential Moving Average (EMA): This indicator
returns the exponential moving average of a field over a
given period of time. EMA formula is as follows.
EMA = [α *T Close] + [1-α* Y EMA].

(17)

Where T is Today’s close and Y is Yesterday’s close


Stochastic Oscillator (SO): The stochastic oscillator
defined as a measure of the difference between the
current closing price of a security and its lowest low
price, relative to its highest high price for a given period
of time. The formula for this computation is as follows:
%K = [(Close price – Lowest price) / (Highest Price – ‫‏‬
‫‏‏‏‏‏‏‏‏‏‏‏‬Lowest Price)] * 100
(18)



Moving Average Convergence/Divergence (MACD):
This function calculates difference between a short and a
long term moving average for a field. The formulas for
calculating MACD and its signal as follows.

Osman Hegazy et al.

20

MACD = [0.075*EMA of Closing prices] – ‫‏‏‏‏‏‏‏‏‏‏‏‏‏‏‏‏‏‏‏‬
(19)
‫[‏‏‏‏‏‏‏‏‏‏‏‏‏‬0.15*EMA of closing prices]
Signal Line = 0.2*EMA of MACD
V.

(20)

RESULTS AND DISCUSSION

LS-SVM-PSO, LS-SVM and ANN algorithms were trained
and tested with datasets form Jan 2009 to Jan 2012. All
datasets are available in [13]. All datasets are divided into
training part (70%) and testing
part (30%).
Fig. 3 to Fig. 14 outline the application of Proposed LSSVM-PSO model compared with LS-SVM and ANN-BP
algorithms at different data set with different sectors of the
market. In Fig. 3, Fig. 4, and Fig. 5, which present results of
three companies in information technology sector (Adobe,
Oracle and HP), results show that LS-SVM optimized with
PSO is the best one with lowest error value followed by LSSVM algorithm. Fig. 6 and Fig. 7 represent results of
financial sector (American Express, and Bank of New york),
we can remark that the predicted curve using the proposed
LS-SVM-PSO algorithm is most close to the real curve which
achieves best accuracy, followed by LS-SVM, while ANNBP is the worst one.
Fig. 8 represents results of using PSO-LS-SVM model in
Honeywell company which represent industrials stock sector,
proposed model still achieves best performance.
Fig. 9 and Fig. 10 outline the application of the proposed
algorithm to hospera and life technologies companies in
health stock sector. From figures one can remark the
enhancement in the error rate achieved by the proposed
model.
Fig. 11 and Fig. 12 outline the results of testing proposed
model on Exxon-mobile and duke energy companies which
represent energy stock sector. PSO-LS-SVM also the best
especially in fluctuation cases.
Fig. 13 represents results for FMC Corporation in materials
stock sector. The achievements of proposed model is very
promising compared with SVM and ANN
Fig. 14 outlines results for AT&T from communication
stock sector. We can notice from figure the role of proposed
model in reducing the error rate and overcoming local minima
problems which found in ANN results.
Table 1 outlines Mean Square Error (MSE) performance
function for proposed algorithm. It can be remarked that the
LS-SVM optimized with PSO always gives an advance over
LS-SVM and ANN trained with LM algorithms in all
performance functions and in all trends and sectors. Proposed
model performs better than other algorithms especially in
cases with fluctuations in the time series function.
Fig. 15 outlines comparison between PSO-LS-SVM, LSSVM and ANN algorithms according to MSE function.

Fig. 3: Results for Adobe Company

Fig. 4: Results for Oracle Company

Fig. 5: Results for HP Company

International Journal of Computer Science and Telecommunications [Volume 4, Issue 12, December 2013]

Fig. 6: Results for American Express Co

Fig. 7: Results for Bank of New York

Fig. 9: Results for Hospera Company

Fig. 10: Results for Life Technologies company

Fig. 11: Results for Exxon Mobile company
Fig. 8: Results for Honeywell company

21

Osman Hegazy et al.

22
Table 1: Mean Square Error (MSE) for proposed algorithm
‫‏‬Algorithm
‫‏‬

PSO-LS-SVM

LS-SVM

NN-BP

‫‏‬Company

Fig. 12: Results for Duke Energy company

Fig. 13: Results for FMC corporation

Adobe

‫‏‬0.5317

0.5703

0.8982

Oracle

‫‏‬0.6314

‫‏‬0.8829

‫‏‬0.9124

HP

‫‏‬0.7725

‫‏‬1.2537

‫‏‬1.9812

American Express

‫‏‬0.7905

‫‏‬1.0663

‫‏‬2.8436

Bank of New york

‫‏‬0.4839

‫‏‬1.2769

‫‏‬1.9438

Coca-Cola

‫‏‬0.6823

‫‏‬0.9762

‫‏‬1.7975

HoneyWell

‫‏‬0.9574

‫‏‬1.3371

‫‏‬2.1853

Hospera

‫‏‬0.8694

‫‏‬0.9320

‫‏‬1.4640

Life Tech.

‫‏‬0.7713

‫‏‬1.3221

‫‏‬1.3492

Exxon-Mobile

‫‏‬1.1000

‫‏‬1.6935

‫‏‬2.4891

AT & T

‫‏‬0.2911

‫‏‬0.4673

‫‏‬0.4055

FMC Corp.

‫‏‬1.5881

‫‏‬2.1034

‫‏‬3.5049

Duke Energy

‫‏‬0.1735

‫‏‬0.6097

‫‏‬0.6010

4
3.5
3
2.5
2
1.5
1
0.5
0

PSO-LSSVM

LSSVM

NN-BP

Fig. 15: MSE of proposed algorithm

VI.

Fig. 14: Results for AT &T Company

CONCLUSIONS

This paper, proposed a machine learning model that
integrates particle swarm optimization (PSO) algorithm and
LS-SVM for stock price prediction using financial technical
indicators. These indicators include relative strength index,
money flow index, exponential moving average, stochastic
oscillator and moving average convergence/divergence The
PSO is employed iteratively as global optimization algorithm
to optimize LS-SVM for stock price prediction. Also, PSO
algorithm used in selection of LS-SVM free parameters C
(cost penalty), ϵ (insensitive-loss function) and γ (kernel
parameter). The proposed LS-SVM-PSO model convergence

International Journal of Computer Science and Telecommunications [Volume 4, Issue 12, December 2013]

to the global minimum. Also, it is capable to overcome the
over-fitting problem which found in ANN, especially in case
of fluctuations in stock sector. PSO-LS-SVM algorithm
parameters can be tuned easily. The performance of the
proposed model is better than LS-SVM and compared
algorithms. LS-SVM-PSO achieves the lowest error value
followed by single LS-SVM, while ANN-BP algorithm is the
worst one.

[18]

[19]

[20]

REFERENCES
[1]
[2]

[3]
[4]

[5]

[6]

[7]

[8]

[9]

[10]

[11]

[12]

[13]
[14]

[15]

[16]

[17]

Olivier C., Blaise Pascal University: “Neural network
modeling for stock movement prediction, state of art”. 2007
Leng, X. and Miller, H.-G. : “Input dimension reduction for
load forecasting based on support vector machines”, IEEE
International Conference on Electric Utility Deregulation,
Restructuring and Power Technologies (DRPT2004), 2004.
Vapnik, V., “The nature of statistical learning”, second
edition, ©Springer, 1999.
Cherkassky, V. and Ma, Y., “Practical Selection of SVM
Parameters and Noise Estimation for SVM regression”.
Neural Networks, vol., 17, pp. 113-126, 2004.
Suykens, J. A. K., Gestel, V. T., Brabanter, J. D., Moor, B.D
and Vandewalle, J. “Least squares support vector machines”,
World Scientific, 2002.
ANDRÉS M., GENARO DAZA, S., CARLOS D., GERMÁN
C. : “Parameter Selection In Least Squares-Support Vector
Machines Regression Oriented, Using Generalized CrossValidation” , Dyna, year 79, Nro. 171, pp. 23-30. Medellin,
February, 2012.
Carlos A. Coello‫‏‬, Gary B. Lamont‫‏‬, David A. van Veldhuizen‫‏‬:
“Evolutionary Algorithms for Solving Multi-Objective
Problems”,Springer, 2007.
D. N. Wilke. “Analysis of the particle swarm optimization
algorithm”, Master's Dissertation, University of Pretoria,
2005.
Khalil A.S.: “An Investigation into Optimization Strategies of
Genetic Algorithms and Swarm Intelligence”. Artificial Life
(2001).
Kennedy J., Spears W.M.: “Matching Algorithms to
Problems”, An Experimental Test of the Particle Swarm and
Some Genetic Algorithms on the Multimodal Problem
Generator. Processes. Current as of December 15th, 2003.
Chun C, Qinghua M, Shuqiang L.: “Research on Support
Vector Regression in the Stock Market Forecasting”
©Springer, Advances in Intelligent and Soft Computing
Volume 148, , pp 607-612, 2012.
Guo Z., Wang H., Liu Q. : “Financial time series forecasting
using LPP and SVM optimized by PSO” © Springer, Soft
Computing Methodologies and Applications , December 2012.
http://finance.yahoo.com.
Xie, G.: “The Optimization of Share Price Prediction Model
Based on Support Vector Machine”, International Conference
on Control, Automation and Systems Engineering (CASE),
pp.1-4., 30-31 July 2011.
Huang C.; Huang L.; Han T.: “Financial time series
forecasting based on wavelet kernel support vector machine”
IEEE, Eighth International Conference on Natural
Computation (ICNC), 2012.
Jui Y.: “Computational Intelligence Approaches for Stock
Price Forecasting”, IEEE International Symposium on
Computer, Consumer and Control (IS3C), pp. 52 - 55, 2012.
Ling J., Chih C., Chi L, Chih H., “A hybrid approach by
integrating wavelet-based feature extraction with MARS and

[21]

[22]

[23]

[24]

[25]

[26]

[27]

[28]

23

SVR for stock index forecasting”, ELSEVIER, Decision
Support System, 2012.
Bernardo, D.:“An interval type-2 Fuzzy Logic based system
for model generation and summarization of arbitrage
opportunities in stock markets”, IEEE Computational
Intelligence (UKCI), 2012 12th UK Workshop, pp. 1-7., 2012.
Ochotorena, C.N. :“Robust stock trading using fuzzy decision
trees”, IEEE Conference on Computational Intelligence for
Financial Engineering & Economics (CIFEr), pp. 1-8, 2012.
Yi X., Jin X., Fengbin L. & Shouyang W. :“Ensemble ANNsPSO-GA Approach for Day-ahead Stock E-exchange Prices
Forecasting”, International Journal of Computational
Intelligence Systems, Volume 6, Issue 1, pp.96-114 2013.
Nayak, S.C. :“Index prediction with neuro-genetic hybrid
network: A comparative analysis of performance”, IEEE
International Conference on Computing, Communication and
Applications (ICCCA), pp. 1-6, 2012.
Fazel Z.,Esmaeil H., Turksen B.: “A hybrid fuzzy intelligent
agent-based system for stock price prediction”, International
Journal of Intelligent Systems, Volume 27, Issue 11, pages
947–969, 2012.
Honghai Y., Haifei L.: “Improved Stock Market Prediction by
Combining Support Vector Machine and Empirical Mode
Decomposition”, Fifth International Symposium on
Computational Intelligence and Design (ISCID), pp. 531 534 , 2012.
Lingzhi W., Jiansheng W.: “Neural Network Ensemble Model
Using PPR and LS-SVR for Stock Market Forecasting”,
Springer, Advanced Intelligent Computing Lecture Notes in
Computer Science Volume 6838, pp 1-8, 2012.
İhsan K., Cengiz K., Selçuk Ç. :“Computational Intelligence
Techniques for Risk Management in Decision Making”,
Springer, Handbook on Decision Making Intelligent Systems
Reference Library Volume 33, pp 9-38, 2012.
Gupta, A. :“Stock market prediction using Hidden Markov
Models”, IEEE Engineering and Systems (SCES), 2012
Students Conference on, pp.1-4, 2012.
Lipo W., Shekhar G.: “Neural Networks and Wavelet DeNoising for Stock Trading and Prediction”, Springer, Time
Series Analysis, Modeling and Applications Intelligent
Systems Reference Library Volume 47, pp 229-247, 2013.
Suykens, J.A.K.; Vandewalle, J. "Least squares support vector
machine classifiers", Neural Processing Letters, 9 (3), 293300., 1999.

