沙漠研究 22-1, 183 -186 (2012 )"ICAL 1 / DT X"

Journal of Arid Land Studies－Refereed Paper－

Effect of Forgetting Factor on Gray Box Auto Regressive Exogenous Algorithm
for Short Term Diurnal Temperature Forecast
in Remote Area Regions
Saurabh KUMAR1), Satyendra Narayan SINGH1), Umesh Chandra RAY2) and Sanjay KUMAR*3)
Abstract: Remote arid areas are devoid of weather data which may be called fairly accurate. They have to rely upon predictions innearby cities, sometimes more than 300 kms away and that too in completely different demographic and physical structure. It is
important to establish alternative method of weather prediction. Model based weather prediction is presented in this paper for suchapplication. It is based upon the principle of dynamic modelling and parameter estimation. These parameters have no physicalmeaning but a change in their behaviour predicts the change in climatic condition. The selection of model based upon forgetting
factor is preferred after evaluating Akaike information criterion, Bayesian information criterion and, Deviance information criterion.
It can be interpreted accordingly to predict the climate at local scale. The methodology is based upon smaller data accumulation but
provides fairly accurate results.
Key Words: Arid area, Forgetting factor, Gray box modelling, Weather

1. Introduction
The processes and systems in climatological prediction get
more complex and difficult with our understanding of
non-linear processes involved. Here, however, high reliability
is not required. Short term weather predictions in remoteareas are extrapolated from weather data collected at nearby
cities, sometimes more than 300 km away with completelydifferent geographical (physical) and demographic conditions.
Remote sensing data are not cost-effective and reliable unlesssupported by ground observations, for such predictions. It isneither practical nor economically feasible to establish andmaintain weather station at remote area locations unless it isstrategically important. Forecast for short term period ranging
from 3 to 7 days is likely to have decisive impact on sustainable
developmental process and planning, especially when Very
Large Scale Photovoltaic Systems and renewable energy based
systems are being planned in remote areas and deserts. Overthe last decades, there has been considerable research activity inthe field of model based reasoning especially for information
poor processes. Model based analysis (MBA) is based upon
behavior descriptions and interconnections of the parameters
affecting the whole system. It can be understood as theinteraction of observed and predicted values. The basic ideaof dynamic modelling can be described as a simple comparison
between measured and modeled quantities. A straightforward model structure can be obtained if the characteristics ofeach component in the system are described by equations
derived from the basic laws of physics. The number ofparameters influencing climate is large and requires
supercomputer for computations, which renders it enviable due
to cost-effectiveness of requirement for remote areas.
Recursive Auto Regressive Exogenous (RARX) system
identification methodology with forgetting factor is developed
for such applications. The model rectifies itself on the basis ofnoises in the system and can be trained with small data sets.
The parameters of the model have no physical significance but
their deviation can be used to predict the changes in theclimatic condition. They are easier to set up and require muchless detailed information about the system to be modeled.
However, for the identification phase, models may need ratherrich data sets for a correct estimation of parameters, especially
if it is a dynamic model. Three criteria is used to evaluate themodel and its number of parameters, Akaike information
criterion, Bayesian information criterion and, Devianceinformation criterion. The RARX model with fairly small no.
of parameters with forgetting factor of 0.997 is found quite
efficient and useful in the present case. Forgetting factor
allows discounting past information and reducing
computational time and capacity for prediction of temperature
on diurnal basis.

* Corresponding Author: prof.kumars@gmail.com
P.O. Box - 5, Muzaffarpur-842001, Bihar, India
1) Centre for Renewable Energy and Environmental Research2) Dept. of Physics, Ranchi University, Jharkhand, India

3) National Institute of Technology, Patna, Bihar, India

2. Model Formulation
A typical recursive identification algorithm is given by,θ n′ = θ ′ (t − 1) + K (t ) (( y (t ) − y′(t ))(1)here, y’(t) is the parameter estimate at time t, and y(t) is theobserved output at time t. Moreover, y’(t) is a prediction ofthe value y(t) based on the observations up to time t-1 and also
based on the current model. The gain K(t) determines in what
way the current prediction error y(t)-y’(t) affects the update of
the parameter estimate. It is typically chosen as,K (t ) = Q(t ) ϕ (t )(2)where, φ(t) is (an approximation of) the gradient with respect toy of y’(t│θ). The latter symbol is the prediction of y(t)according to the model described by y(t). Q(t) is the matrix
which determines the adaptation gain and the way parameters
are updated. The model structure like AR and ARX that
correspond to linear regression can be written as,y (t ) = ϕ (t ) T θ (t ) + e(t )(3)where, the regression vector φ(t). contains old values ofobserved inputs and outputs, and e(t) is the noise source. The
most logical approach to the adaptation problem is to assume acertain model for how “true” parameter y’(t) changes. A
typical choice is to describe these parameters as a random walk,θ ' (t ) = θ ' (t − 1) + ω (t )(4)here, ω(t) is assumed to be white Gaussian noise withcovariance matrix,
E[ω(t) ωT(t)] = R1

(5)

where, R1 is variation of the variance. Considering,underlying description of the observation, a linear regression,an optimal choice of Qn in “eq. 1 and 2” can be computed fromthe Kalman filter. The above method is modified to discountold measurements so that the model adopts the changing
situation dynamically. An observation that is Γ samples oldcarries a weight that is R2τ of the weight of the most recent
observation. Now, the model can be constructed with several
parameters. To decide the number of parameters, three major
criterian are put forth in recent times. These can be used formodel selection.

3. Model Selection
3.1. Akaike information criterion
This criteria is used when relative measure of theinformation has lost while describing the real situation (Akaike,1974). It can be said to describe the tradeoff between bias andvariance in model construction. In other words, it can be used
for a trade off between accuracy and complexity of the model.
AIC also provides a means for comparison among models forselecting the better structure. However, it can tell nothing
about how well a model fits the data in an absolute sense. If
all the models are bad, this criteria fails. It is defined as,
AIC = 2k – 2 ln (L)(6)

here, k = the number of parameters in the model, and L = themaximized value of the likelihood function for the estimatedmodel. The preferred model is the one with the minimum
AIC value. AIC not only rewards goodness of fit, but also
includes a penalty that is an increasing function of the numberof estimated parameters. This penalty discourages overfitting.
Increasing the number of free parameters in the model
improves the goodness of the fit. AICc is AIC with acorrection for finite sample sizes,
2 k (k + 1)
(7 )AICc = AIC +n − k −1
where, k denotes the number of model parameters. Thus,
AICc is AIC with a greater penalty for extra parameters(Burnham and Anderson, 2002). If n is small or k is large it
gives better rsult. AICc converges to AIC as n gets large.
AIC increases the probability of selecting models that have too
many parameters, i.e., of overfitting when n is not many times
larger than k2. The time series models provides better result
when AICc criteria is used (Brockwell and Davis, 2009).
The underlying errors should be normally distributed andindependent. This leads to χ2 model fitting, 1 
L = ∏ 2 i =1  2pσ i n1/ 2

 n ( y − f ( x)) 2  (8)exp − ∑ i

2σ i2
 i =1

It reduces to,ln( L) = C − χ 2 / 2(9)where, C is a constant independent of the model used, anddepends only on the use of particular data points. i.e. it does
not change if the data do not change. The AIC is therefore
given by,

AIC = 2 k − 2 C + χ 2(10)

here, C can be ignored since only differences in AIC aremeaningful.
3.2. Bayesian information criterion
Bayesian information criterion (BIC) is also used for model
selection among a class with different numbers of parameters.
To optimize BIC is a form of regularization. When estimating
model parameters using maximum likelihood estimation, it ispossible to increase the likelihood by adding parameters, whichmay result in overfitting. The BIC resolves this problem byintroducing a penalty term for the number of parameters in themodel. This penalty is larger in the BIC than in the related
AIC (McQuarrie and Tsai, 1998).
Let, x = the observed data; n = the number of data points inx, the number of observations, or equivalently, the sample size;
k is the number of free parameters to be estimated, p(x|k) is theprobability of the observed data given the number ofparameters, L is the maximized value of the likelihood function
for the estimated model. Then, the BIC can be written as,− 2 ln p ( x k ) ≈ BIC = − 2 ln L + k ⋅ ln (n)(11)

here, it is assumed that the model errors or disturbances are
independent and identically distributed according to a normal
distribution and that the boundary condition that the derivativeof the log likelihood in respect to the true variance is zero, thisbecomes,

BIC = 2 n ⋅ ln(σ 2 ) + k ln (n)(12)

where σ2 is the error variance. The error variance in this caseis defined as,nσ 2 = 1 n ∑ ( xi − x ) 2(13)models. The expectation,

D = E θ [D(θ )](16)

is a measure of how well the model fits the data. A larger
value means worse fit. The effective number of parameters ofthe model is computed as,

p D = D − D (θ ′)(17)

where θ’ is the expectation of θ. Its larger value makes it easy
to model the data. The deviance information criterion iscalculated as,

DIC = p D + D(19)

The models with smaller DIC should be preferred tomodels with larger DIC. Models are penalized both by thevalue of D, which favors a good fit, but also by the effective
number of parameters pD. Since D will decrease as thenumber of parameters in a model increases, the pD termcompensates for this effect by favoring models with a smaller
number of parameters. The advantage of DIC over othercriteria in the case of Bayesian model selection is that the DIC
is easily calculated from the samples generated by Monte Carlosimulation.

i =1

Under the assumption of normality the following,

BIC = χ 2 + k ⋅ ln (n)(14)

In any estimated model, the model with the lower value ofBIC is more accurate. The BIC is an increasing function of σ2
and an increasing function of k. This implies unexplained
variation in the dependent variable and the number of
explanatory variables increases the value of BIC. Hence,lower BIC implies either fewer explanatory variables, better fit,or both. The BIC generally penalizes free parameters morestrongly than does the Akaike information criterion, though it
depends on the size of n and relative magnitude of n and k.
3.3. Deviance information criterion (DIC)
The deviance information criterion (DIC) is a hierarchicalmodeling generalization of the AIC and BIC. It is particularly
useful in model selections where the posterior distributions of
the models have been obtained by Markov chain Monte Carlo(MCMC) simulation. It is only valid when the posterior
distribution is approximately multivariate normal (Claeskensand Hjort, 2008). Let the deviance is defined as,D(θ ) = − 2 log ( p ( y θ )) + C(15)

where y are the data, θ are the unknown parameters of themodel and p(y│θ) is the likelihood function. C is a constantwhich cancels out in all calculations that compare different

4. Formulation and Forgetting Factor
To show the effect of forgetting factor and selection ofparameter order in metereological parameter prediction, amodel is formed to predict temperature profile in a typicalsummer day (April 21, 2010) at Muaffarpur, India. System
identification modelling are based upon identification ofimportant input parameter affecting the output to be predicted.
Present model is formulated with two input parameters,
humidity and air flow and one single output parameter,
temperature profile. The model is trained with daily datacollected of the three parameters over one month. These data
sets are used ten times with several forgetting factors.
Forgetting factor allows the model to remember its
previous data and its effect on its prediction. The modelaccordingly assigns weightage to the data remembered. The
forgetting factor R2 is also called as variance of the innovationse(t). A typical choice of R2 is in the range of 0.97 to 0.997,
which amounts to approximately remembering 33 to 333 last
observations, respectively. In the present case, a forgetting
factor of 0..995 was selected. The model, hence, remembered
last 200 data in predicting the output.
MATLAB software is used to calculate the predicted valueof the temperature profile. The time frame used for prediction
was one day, two days and three days. Since the error
function reintroduces error in prediction and changes

40

30

20

Series2
c

100

1
5
913

S3

1721

S2S1Fig. 1. Variation of temperature - (I) actual (S1), (II) predicted one day in advance (S2), (III) predicted two days in advance (S3).

dynamically the model itself, large error causes instability.
Hence, as we move from one day to two days and three days,
the model becomes more and more unstable and the error inprediction also increases. In Figure 1 S1 shows the actual
value of temperature profile. S2 and S3 shows predictedvalues for one day in advance and two days in advance. It canbe easily observed from the figure that S3 has more instability
as compared to actual data (S1) and one day prediction data(S2).
Result indicates that temperature can be predicted with
90% accuracy for up to two days in advance. It becomes
unstable with deviation in forgetting factor. The predicteddata also becomes slowly unstable for more than two days.
Nevertheless, sudden changes in climatological parametersneed to be studied further as these are indicated by largechanges in parameters. Such changes can be interpreted interms of impending dust storm or other harsh climatologicalchanges to issue warning. These predictions need to bequantified with large data accumulation and experience.
Mostly climate in arid areas do not change suddenly except incase of storms. Hence, at least five years would be preferredchoice even from nearby meteorological station for setting up
the initial model. With experience and accumulation offurther data, the model will refine itself continuously for better
prediction.

References
Akaike H. (1974): A new look at the statistical model
identification. IEEE Transactions on Automatic Control,
19(6): 716–723.
Burnham K.P., Anderson D.R. (2002): Model Selection andMultimodel Inference: A Practical Information-TheoreticApproach. 2nd ed. Springer-Verlag.
Brockwell P.J., Davis R.A. (2009): Time Series: Theory andMethods. 2nd ed. Springer.
McQuarrie A.D.R., Tsai C.L. (1998): Regression and TimeSeries Model Selection. World Scientific.
Claeskens G., Hjort N.L. (2008): Model Selection and Model
Averaging. Cambridge.