Adaptive Demand-Forecasting Approach based on
Principal Components Time-series（an application of data-mining technique to detection of market movement）
Toshio Sugihara

Abstract
In this study, an adaptive demand-forecasting approach adopting the data-mining
technique which detects the correlation between the target variable and other related
elements, is proposed. With the inclusion of the scheme of the time-series analysis
based on the state-space approach, this approach has two characteristic points. One is
the state-space which is formed by principal components composed of various market
variables. Another is the self-organization of the state-space using a neural network.
This approach is applied to the two cases of the demand-forecasting. Some comparisonsof forecasting accuracy (extrapolation test) with this approach and non-self-organizing
models (AR, etc) are used to evaluate the effectiveness of the proposed approach.
Consequently, we achieved significantly higher accuracy using this approach comparedto the other approaches.
Keywords: Adaptive demand-forecasting; Data-mining Approach, (Extended) Kalman
Filter; Principal Component Analysis; Neural Networks
1. Introduction
In the causes of unpredictable market movement in nowadays and logistics
requirements with JIT system, demand-forecasting, one of the core functions ofmarketing, has been required to be improved on the performances. This is shown by thefact that demand-forecasting is positioned as the center of SCP (Supply Chain
Planning) which is the planning function of SCM (supply Chain Management). In thearea of SCM which is the eminent innovation of management, SCP is the total control
program which transverses from production, logistics to marketing. Therefore, in the
published SCP packages, the performances of demand-forecasting are improved
distinctly[1]. The main requirements to these demand-forecasting are considered thenext two points.
･ Real-time response at any time･ Dynamic acquisition of the market movement relating the target product.
In trying to satisfy these requirements, demand-forecasting had been influenced by thevarious approaches of the economic time-series analysis[2, 3]. In nowadays, not only
linear models, but nonlinear models (for example, neural network) are applied[4, 5].
Almost of these approaches process the single time-series which is the sales orinventory, etc of the target variable. In the case of the approach using state-space, thestate-space is built mainly with past lag parameters of the single time-series[6]. The
market unpredictability is increasing according to the varieties of products andconsumer's behaviors. The approach of including the correlation with the target product1

and the related market movement is indispensable in demand-forecasting approach. In
recent days, the data-mining approach which based on the data warehouse, has madestartling progress[7, 8, 9]. In demand-forecasting including market movement, theapproach which takes in data-mining technique is considered to be effective approach.
Above stated, in this study, an adaptive and multi-variable prediction scheme which
the target product and the market movement are predicted simultaneously, is proposed.
The market movement is represented as the state-space which is built with the target
variable and the principal components time-series of the related variables. The
state-space is renewed at each observed time by the self-organized mechanism usingneural network. And for the aim at real-time response of this demand-forecasting, total
demand-forecasting scheme is based on the framework of the Kalman Filter.
2. Data-mining
Data-mining is not always defined as a confirmed and unified concept. However, it isdescribed generally as the "detection of valuable and nontrivial information from alarge-scale database" , and its concept is mainly based on the detection of knowledge [4,7]. However, data-mining is not the inspection of hypotheses or the verification of
established hypotheses, but should be "the automatic detection of new facts andrelations from a plain database". According to this definition, data-mining is considered
to be built on a large-scale database such as a data warehouse. The data-mining
approaches presented now and their applications are shown in Table 1.
［Table 1. Data-mining approaches and their applications [4]］Data-mining function
Algorithm
correlation
statistics, ensemble theory
classification

decision tree, neural networkmodeling

linear/nonlinear regression,

time-series prediction

curve fitting, neural network
statistical ARMA model,

time-series pattern

Box-Jenkins, neural network
statistics, ensemble theory

Applications
market basket
analysistarget marketing,quality control,risk evaluation
customer ordering,
price modeling,process control
sales-forecasting,rate-forecasting
inventory control
time-series market
basket analysis

In Table 1, as the tools for data-mining, some statistical methods, curve-fitting
methods, regression analysis, neural network and genetic algorithm are listed. The
contents in Table 1 appear to indicate that "data-mining is not the inspection of
statistical hypotheses or the verification of established hypotheses", but this means thatdata-mining is nothing but a tool which automatically detects new facts or relations
from data warehouses. Therefore, data-mining is a tool not of inspection but of
detection.

2

3. Basic aspects of the proposed demand-forecasting
3.1. Extraction of market movement and principal components time-series
The market movement is lied in data warehouse and it is considered that the timevarying of the stored data represents the market movement. Therefore, for the extractionof the market movement, not only single time-series of the target product but also multitime-series related with it should be processed as integrated. But these time-series are
considered strongly correlated with each other, and should not be handled individually.
For that reason, the principal components time-series which are integrated from theoriginal multi time-series by principal components analysis (PCA), can represent themarket movement[10]. The adopted principal components time-series are selected by
the proportion of eigenvalue. Consequently, the adoption of principal components
time-series may be considered as one of the Data-mining tools.
By applying principal component analysis, the time-series of N original variables areintegrated in the time-series of M (M < N) principal components. The most variations
based on the original market variables are represented as the movement of common
factors, namely market movement is represented by far fewer variables than in theoriginal method.
PCA of time-series composes principal components time-series from originaltime-series, and the time-series of the principal components are independent of eachother. At any time, the j-th component is presented as
pj = Σi=1N fjixi
(1)
Here,
index of the principal components : j = 1, 2, ......., M
index of the original variables : i= 1, 2, ......., NandΣi=1Nfji2 = 1(2)N
Σi=1 flifji = 0 （j≠l）.
(3)
The number of principal components adopted is determined by the proportion of theaccumulated eigenvalue, which occupies generally about 80-90 percent. The principal
components are selected in the order of high to low eigenvalue. fj is an eigenvector
obtained as the linear combination of the original variables. fj is determined at eachtime (with constant time period) , as shown by formula (1), the time-series of aprincipal component can be generated over the entire period.
The time-series of a principal component is symbolized as follows.
｛pkj : k = 1, 2, ....., n : j = 1, 2, ......, M｝
(4)
Here,
index of the principal components : j = 1, 2, ......., M
index of the time : k = 1, 2, ......., nGenerally, M is far smaller than N, and｛pj : j = 1, 2, ....., M｝is considered to present thecommon variance of the original variables. Considering the independence of eachcomponent from other components, the time-series of the principal component isassumed also to be independent. Therefore, the time-series of a principal component canbe the extracted market movement.
3.2. Total scheme of prediction
For the online prediction scheme, the Kalman Filter is adopted as the fundamental
framework of prediction in this work [11, 12]. In the field of linear system theory, the3Kalman Filter is the rational estimation / prediction approach to the sequential online
time-series. Also in the nonlinear field, the Kalman Filter is formulated as the Extended
Kalman Filter. Its concept is the multi input / output control system and multi variables
are estimated / predicted simultaneously.
The basic equations for the Kalman Filter are shown as equations (5) and (6) as
below. The Kalman Filter consists of the state vector and the measurement vector
involving multiple variables.
xk = Φk,k-1 xk-1 + Ck uk + vk
(5)
yk = Hk xk + wk
(6)
xk：state vector Φk,k-1：state transition matrix
Ck：control matrix uk：control vector vk：state noise vector
yk：measurement vector Hk：measurement matrix
wk：measurement noise vector
The state noise and the measurement noise are both assumed to be white at each timeand independent of each other. Fig.1 shows the total processing steps of the Kalmanfilter. The estimation is processed on the basis of maximum likelihood method and theprediction is gained as the output of the dynamical state-transition from input of its
estimation value. In the Kalman Filter, the chain of these process (from Step 1 to Step
7) is repeated. The predicted / estimated value is outputted in Step 3 / Step 6.
The transition matrix updates the state-space and the measurement matrix transforms
the state variables to the measurement variables. In this study, the state variables areconstructed with the target variable and the principal components. Therefore, the state
space is noted as below.
{pkj : k=1, 2,....., n : j=1, 2,....., M}{yk : k=1, 2,....., n }As showed below, the transition matrix is renewed by self organized mechanism usingneural network, the measurement variables are the same as the state space variables.
From these premise,(5) and (6) are formulated as below.
(7)
xk = Ф＊k,k-1xk-1 + vk
yk = Ixk + wk
(8)＊
Ф k,k-1 is self-organized transition matrix by neural network and I is unit matrix.

4

［Fig1．The chain of Kalman Filter processing steps］
step1 : initializationk = 0 (time initialization)x*k (initialization of estimated state vector)
xk,k-1 (initialization of predicted state vector)
Pk,k-1 (initialization of predicted error covariance matrix)
step2 : input of observed values
k=k+1
yk (input of observed values)
step3 : prediction
xk,k-1 = Φk,k-1 x*k-1
yk,k-1 = Hk xk,k-1
step4 : calculation of gain matrix
Bk = Pk,k-1tHk[HkPk,k-1tHk + Wk]-1(Wk is variance matrix of measurement noise)
step5 : calculation of estimated error covariance matrix
Pk,k = [I - BkHk]Pk,k-1t [I - BkHk] + BkWktBk= [P-1k,k-1 + tHkW-1kHk]-1 (I is unit matrix)
step6 : estimation
x*k = xk,k-1 + Bk[yk - yk,k-1]=xk,k-1 + Bk[yk - Hkxk,k-1]
yk=Hkx*k
step7 : calculation of predicted error covariance matrix
Pk,k-1 = Φk,k-1Pk,k-1tΦk,k-1 + Vk-1
(Vk is variance matrix of state noise)
go to step2
3.3. Acquisition of the structural market change to the prediction mechanism
As above mentioned, the structural market change is buried in stored data. In thisstudy, the prediction mechanism which intends to acquire the market movement is
considered to be driven by the effective Data-mining tool using neural network.
The essential points in forming the Kalman Filter are the determinations of the statetransition matrix and measurement matrix. In forming the Kalman Filter in physicalsystems, the dynamics of state and observation systems are represented as obvious
equations which give the expressions of them. In particular, the state transition matrix
presents the dynamics of system from a time to the next time, and therefore, plays themain role in the estimation of the system dynamics. Otherwise, in economic /management field, it appears to be impossible to elucidate the system dynamics
deductively, the components of the state transition matrix are obtained as thecoefficients of regression equations applied to the preceding time-series. In such a case,the elements are constants that cannot cope with the varying system. Considering thetheoretical conditions of the Kalman Filter, when the time variation of the transition
matrix is not guaranteed, this formulation cannot be applied to the non-stationary

5state[13].
To solve this problem, the self-organization of state space has been proposed [14].
Each element of Φk,k-1 is calculated based on self-organization at the time of update ofstate variables and its relations. In this paper, we apply a method which uses neuralnetwork processing of nonlinear relations[8, 9]. The applied neural network is
structured type, and is trained using the sets of neighboring time-series as training dataat constant interval. At each cycle of the Kalman Filter processing, the state transition
matrix is renewed by the above-mentioned self-organized process. Here, for thegeneration of the state transition matrix, training data are set as follows.
input set of training data {xj : j = k - T,...., k - 1} (T: training interval)
output set of training data {xj : j = k - T + 1,...., k} (T: training interval)
By this training process, the learned transition matrix Φ*k,k-1 is generated and theprediction vector {xk+1} is obtained by inputting the vector {xk} into the trained neuralnetwork. This procedure is shown as Fig 2.
［Fig 2.

training / prediction process of transition matrix］
training input / output( j = k-T, ….., k-1)
xj-1

training neural networkΦj,j-1

xj

x〜
xk-1 trained neural network
Φ＊k,k-1k

4. Scheme of proposed demand-forecasting approach
4.1. Total picture
Total picture of this demand-forecasting system is shown as Fig 3. The input
time-series are as follows.
{y}: target variable time-series{x1, x2, ......., xn}: relating variables time-series
These are seasonal adjusted and pre-processed by smoothing and normalization. The
state space is built with (M + 1) variables. The output variables are the estimation /prediction value of these variables.

6

［Fig 3.

Total picture］

Kalman - Filter
Smoothing /{x1,...,xn}

time-series
PCA

building
space statepredicted /estimatedvalue {y〜}normalizingsmoothing /self-organizing(neuralnetwork){y}normalizing

4.2. Procedure of the Kalman Filter
The procedure of the Kalman Filter which is the framework of this demand-forecasting
approach is built as follows. For the acquisition of market movement, the processing
time period is divided into two parts. The former period is named "Initial period" andthe latter is named "Prediction period". In "Initial period", the elements of transition
matrix are formed with regression coefficients corresponding to the update of eachvariable. In "Prediction period", they are formed with the ratio of output / input byneural network corresponding to each variable. As above mentioned, the state spacevariables are the same as the measurement variables.
＜measurement time-series＞
principal variable time-series：{pkj : k=1, 2,....., n : j=1, 2,....., M}
target variable time-series：{yk : k=1, 2,....., n}＜state space variables＞
t(pk1 , pk2 , ...... , pkM , yk)（ｔ shows transposed）( 9)＜measurement variables＞
the same as the state space variables(10)＜transition matrix＞[Initial period]
The principal variable is independent at each other and regression formula ofeachvariable is shown as below.
pkj = ajjpkj-1 + vkj (j = 1, 2, ......, M)(11)ajj : regression coefficient, vkj : regression error
The regression formula of target variable is formed as follow.
yk = Σj=1M+1aM+1jpkj + vkM+1(12)
aM+1j : regression coefficient , vkM+1 : regression error
Consequently, transition matrix is formed as below.

7Φk,k-1

=

 a11
 0

 0

 0
a M +11

0

0

a 22

0


0 
0 

0 
a M +1M +1 
0

0
a MM
a M +12(13)[Prediction period]
The elements of the transition matrix are formed with the ratio of output / input
by neural network corresponding to the each variable.
 p k~1 / p k −11
0
0
0 


~
0
0 
p k 2 / p k −12 0Φ*k,k-1 = 
(14)
0
0
0 


0
0
0 


0
0
0 y k~ / y k −1 ＜measurement matrix＞
The measurement matrix is unit matrix and its row is (M+1), column is (M+1).
＜state noise and measurement noise＞[state noise]State noise is generated at the update of state space and it is equivalent to theprediction
error. Namely, in "Initial period", the regression error is used and in"Prediction period", the convergence error of neural network is used.
[measurement noise]
Different from physical time-series, it is difficult to understand the meaning of
measurement noise of economic time-series. In this study, the measurement noise isgenerated as the difference between observed series and its curve-fitted series. This
operation is applied to both the principal components time-series and the target
variable time-series.
5. Case study
5.1. Case study(1)As the case, the demand-forecasting of Passenger New Car Sales (monthly
time-series) is adopted. Here, the time-series is seasonally adjusted, and the trend
leading series is the target series for demand-forecasting. The processing period is from1991 to 2000 and sample size is 120. The prediction period is 24 month(1999 - 2000).
Five time-series are related to the target series, and are listed as follows. They are allseasonally adjusted, and the period and sample size are the same as the target series.
y : Passenger New Car Sales
x1 : Private Final Consumption Expenditurex2 : Money Supply (M2 + CD)
x3 : Stock Averagex4 : Consumer Price Indicesx5 : Building Construction Starts.
After PCA of the 6 time-series above, each eigenvalue and accumulated proportion are
obtained as follows.

8first PC （p1 ）second PC（p2）third PC （p3）eigenvalue
0.027
0.005
0.004

accumulated proportion
0.722
0.860
0.955

We use the first PC, second PC, and third PC which make up 95 percents of theaccumulated proportion. The formation of the Kalman Filter in this case is as follows.
(a) state variables
first principal component --- third principal component :｛p1, p2, p3｝
passenger new car sales :｛y ｝
Three models are defined as combinations of the above-stated state variables.
model A :｛p1, y}model B :｛p1, p2, y｝
model C :｛p1, p2, p3, y｝(b) measurement variables
equal to state variables in each model(c) state transition matrix
Elements of state transition matrix are coefficients estimated by regression in theinitial period (1991〜1998, 8 years and 96 samples).
In the prediction period (1999〜2000), the predicted values are obtained as outputof the neural network, the input of which is the last observation. Here, the neuralnetwork is trained using samples of time-series for the preceding 2 years. The
degree of convergence in training is about 10-3.
(d) measurement matrix
The measurement matrix transforms the state vector to the measurement vector.
(e) state noise
In the initial period, each state noise is defined as the root mean square error of
regression equations according to models A, B and C.
In the prediction period, each state noise is defined as the mean square error of
network training. This is calculated and updated in the prediction cycle of Kalman
Filter processing.
(f) measurement noise
Measurement noises are defined as the root mean square error of the moving
average of {p1, p2, p3, y}. These are used both in the initial and prediction periods.
The evaluation of each model is shown in Table 2. The prediction error is calculated asa root mean square error (RMSE).
［Table 2 Prediction accuracy of each model］
model
prediction error

model A
0.0030model B
0.0036model C0.0032

To verify the efficiency of this approach, the prediction accuracy compared to those ofother approaches is evaluated (RMSE).

9［Table 3. Prediction accuracy of each model］
state variables{y,x1,x2,x3,x4,x5}{y,p1}

non self- organized

0.00410.0032

self – organized

0.00440.0030state space

AR Model (order 5) : 0.0040(Estimation)

5.2. Case study(2)Next the case, the demand-forecasting of Air Conditioners Sales (monthly
time-series) is adopted. Here, the time-series is seasonally adjusted, and the trend
leading series is the target series for demand-forecasting. The processing period is from1991 to 1999 and sample size is 108. The prediction period is 24 month(1998 - 1999).
Three time-series are related to the target series, and are listed as follows. They are allseasonally adjusted, and the period and sample size are the same as in the target series.
y : Air Conditioner Sales
x1 : Index of Industrial Production
x2 : Consumer Price Indicesx3 : Householders.
After PCA of the 4 time-series above, each eigenvalue and accumulated proportion are
obtained as follows.
first PC （p1 ）second PC（p2）third PC （p3）eigenvalue
0.024
0.006
0.002

accumulated proportion
0.747
0.945
0.992

To verify the efficiency of this approach, the prediction / estimation accuracy
compared to those of other approaches is evaluated (RMSE).
［Table 4. Prediction accuracy of each model］
state variables{y, x1, x2, x3}{y, p1}

non self- organized

0.00810.0152

self – organized

0.00740.0066state space

AR Model (order 3) : 0.0082(Estimation)10Based on Table 2, 3 and 4 the following is revealed.
・The effect of adopting self-organization of state-space on the prediction accuracy, isheavily weighted. The prediction accuracy of non self-organized state space which has
the transition matrix determined throughout the entire prediction period, is inferior tothose of self-organized cases.
・In the approach using principal components, the use of one components is optimal.
This indicates that increasing the number of principal components does not always
result in higher accuracy and the optimal the number of components representing
market variance exists.
6. Conclusions and discussions
In this study, we tried to define state-space formed by principal components. The
results presented in Section 5 indicated that better prediction accuracy than thatobtained other approach(AR and stationary Kalman Filter approach) was achieved, andit was considered that adopting of principal components was effective. The principal
components which were integrated from the original variables, therefore represent
independent market movement. It was considered that the formation of state-space
based on independent common market movement is more effective than that based on
original market variables.
Based on these results, the following are concluded.
・This approach is essentially a data-mining approach. For this reason, the adopted
common market movement can be integrated from any arbitrarily selected market
variable.
・The state-space which is formed with non-interactive market movement, is more
effective than that is formed with original market variables.
・The optimal number of principal components which form state-space, exists. Namely,
adopting too few or too many principal components reduces the accuracy.
With regard to this last point, further study on the optimal selection of principal
components is needed. The relative contribution of the variance of target time-series tothe total variance should be studied simultaneously.
On the premise of real-time prediction, this prediction approach is based on theframework of the Kalman Filter. Properly speaking, because of the convergenceprocessing of neural network, this approach is not strictly real-time processing.
However, the convergence cycle is about 20 - 30 iterations sufficiently up to theefficient convergence limit, and computer has sufficient CPU speed to process. Thisapproach can be nearly real-time processing. In addition to these points, a neuralnetwork which has a feedback mechanism is considered to be a valuable tool for
self-organization. a feedback neural network will be evaluated with respect to not only
the self-organization method of state-space transition but also a direct method for
demand-forecasting in real-time processing.

11References
1. T. Miyazaki, " SCM and OR", Communications of the Operations Research Societyof Japan, Operations Research, Vol. 44, No. 6, pp.300-303, 1999.
2. P. J. Brockwell, R. A. Davis "Time Series : Theory and Methods", 2nd Edition,
Springer-Verlag, New York, 1991.
3. C. W. J. Granger "Forecasting In Business And Economics", Academic Press, 1989.
4. J. P. Bigus, " Data Mining with Neural networks", McGraw-Hill, 1996.
5. H. Araki, I. Kimura, I. Arizono, H. Ohta " Demand Forecasting based on
Differences of Demands via Neural Networks" J. JIMA, No. 2, pp.59-68, 1996
6. G. Kitagawa, W. Gersch " A smoothness priors-state space modeling of time-series
with trend and seasonality", Journal of American Statistical Association, Vol. 79,
No. 386, pp.378-389. 1984
7. M.J.A. Berry, G. Linoff " Data Mining Techniques for Marketing, Sales andCustomer Support", John Wiley, 1997
8. C. G. Dasgupta, G. S. Disdensa, S. Ghose " Computing the predictive performanceof a neural network model with some traditional market response models",International Journal of Forecasting, vol.10, pp. 235-244, 1994
9. S. Dutta, S. Shekhar, W.Y. Wong " Decision support in non-conservative domains:
generalization with neural networks", Decision Support Systems, Vol.11, No.5,
pp.527-544, 1994
10. T. Kariya " The New Development of Econometric Analysis", Iwanami, 1994
11. M. Aoki " State Space Modeling of Time Series" Sprinder-Verlag,1987
12. T. Sugihara " Economic time-series analysis adopting adaptive models" Kougaku
Toshiyo, 1996
13. G. Kitagawa, W. Gersch " A smoothness priors time-varying AR coefficientmodeling of non-stationary covariance time series", IEEE Transactions on
Automatic Control, Vol. AC-30, No.1, pp.48-56, 1985
14. T. Sugihara, H. Tsubone " Demand Forecasting System adopting to market
movements for Supply Chain planning", Proceedings of 6-th. International
Symposium of Logistics, pp.113-116, 2001

12