# Parameter Estimation in Mean Reversion Processes with Deterministic Long-Term Trend

Authors: Freddy H. Marín Sánchez; Verónica M. Gallego
Journal: Journal of Probability and Statistics
 (2016)

---

## Abstract

This paper describes a procedure based on maximum likelihood technique in two phases for estimating the parameters in mean reversion processes when the long-term trend is defined by a continued deterministic function. Closed formulas for the estimators that depend on observations of discrete paths and an estimation of the expected value of the process are obtained in the first phase. In the second phase, a reestimation scheme is proposed whena priori knowledge exists of the long-term trend. Some experimental results using simulated data sets are graphically illustrated.


---

## Body

## 1. Introduction

In the literature it is common to find studies and applications of mean reversion models, at both practical and theoretical levels, especially in fields such as energy markets, commodities, and interest rates. In these models there is a long-term trend which acts as an attractor making the process to oscillate around it and there is a random component that adds volatility to the movement. The specific characteristics of the models will depend on the structure of the trend and the volatility of the process.Some common models of mean reversion are those in which all the parameters in the process are constant, such as Ornstein-Uhlenbeck model, the CIR model proposed by Cox et al. [1], and in general the models obtained from the CKLS structure proposed by Chan et al. [2]. In addition to these models there are other methods not so common in the literature in which the reversion is determined not by a parameter but a deterministic function or another stochastic process [3, 4]. In general, the fit of the models to specific situations is done by assigning values to the parameters and functions, either froma priori knowledge of the problem or from parameter estimation techniques based on historical data series that help to detect unobservable information from the process.One of the first difficulties in the application of parameter estimation techniques is that it is necessary to have information with the same time resolution that is specified in the model. Since models of stochastic differential equations are formulated in continuous time and that the observed paths of the process can be obtained only in discrete time, it is necessary to discretize the EDE model. An approach frequently used for this purpose is given by Euler-Maruyama. With this methodology a new discrete-time process is obtained, with which it is possible to make inferences that are valid for continuous time model, as in the case of estimating the parameters.There are different procedures that can be implemented such as methods based on distributional moments [5], Kernel methods [6, 7], least squares, and maximum likelihood [8–10]. Parameter estimation becomes fundamental too for modeling control systems. In the literature it is possible to find different methodologies for those issues such as those developed for Meng and Ding [11], where the parameter estimation in an equation-error autoregressive (EEAR) system is done through a transformation of the model to an equivalent one removing the autoregressive term and to obtain the parameters of the original model the principle of equivalence is used. Another methodology is proposed for Cao and Liu [12] where the parameter estimation in a power system is carried out from a recursive procedure to estimate simultaneously all the parameters from techniques based on the hierarchical identification principle, using gradient algorithms and least squared algorithms. Complementarily, Ding et al. [13] used methods based on the gradient algorithms and least squares iterative algorithms for system identification in output error (OE) and output error moving average (OEMA) systems, where the parameters that depend on unknown variables are computed using estimates of these unknown variables from previously estimated parameters.In the area of multirate system identification, Ding et al. [14, 15] used the polynomial transformation technique to deal with the identification problem for dual-rate stochastic systems, while Sahebsara et al. [16] discussed the parameter estimation of multirate multi-input multioutput systems. Also, Ding and Chen [17] presented the combined parameter and state estimation algorithms of the lifted state-space models for general dual-rate systems based on the hierarchical identification method. Shi et al. [18] gave a crosstalk identification algorithm for multirate xDSL FIR systems.In general, the auxiliary model identification idea has been used to solve the identification problem of dual-rate single-input single-output systems as shown in [19].For more detailed information about multi-innovation stochastic gradient algorithm for multi-input multioutput systems and for multirate multi-input systems as well as auxiliary models based multi-innovation extended stochastic identification theory (see [20] and references therein). We will concentrate on one-factor stochastic model in which the parameters are estimated using maximum likelihood based on the discretized model when the long-term trend is given by a deterministic function. In this case we must estimate both the trend function and the parameters. To estimate the long-term trend function some convolution techniques and numerical differentiation are used, whereas the normality properties of residuals resulting from discretization are used for estimating parameters by maximum likelihood and in this way it is possible to obtain closed formulas for estimators based on the observations of a path of the process and the estimation of the long-term trend.In the estimation process there may be some bias in the estimates with respect to the actual values despite the fact that they are obtained by the method of maximum likelihood. When this situation occurs it is necessary to develop alternative methodologies to have a greater adjustment to the estimates based on the initial estimates [21].This paper is organized as follows. In Section2 a description of mean reversion processes is presented for cases when the long-term trend is given by a continued deterministic function. The first phase of the estimation technique is shown in Section 3. Section 4 presents some examples and preliminary results for the estimation technique and in Section 5 (second phase), a procedure for reestimating the parameters from additionala priori knowledge is described.


## 2. Mean Reversion Processes with Deterministic Long-Term Trend

Mean reversion processes of one factor with constant parameters and mean given by a deterministic function can be written as(1)
d
X
t
=
α
μ
t
-
X
t
d
t
+
σ
X
t
γ
d
B
t
;
t
∈
0
,
Twith initial condition X
0
=
x, where α
>
0, σ
>
0, and γ
∈
0
,
3
/
2 are constants, μ
(
t
) is a deterministic function, and B
t
t
≥
0 is a Unidimensional Standard Brownian Motion defined on a complete probability space Ω
,
F
,
P.α parameter is called reversion rate, μ
(
t
) is the mean reversion level or trend of long-run equilibrium, σ is the parameter associated with the volatility, and γ determines the sensitivity of the variance to the level of X
t.Model (1) is a generalization of the models CKLS, Chan et al. [2], where the mean reversion level is a deterministic function that captures the trend of the process. In a general way, μ
(
t
) plays the role of an attractor at each point t in the sense that, when X
t
>
μ
(
t
) the trend term α
μ
t
-
X
t
<
0 and therefore X
t decreases and when X
t
<
μ
(
t
) a similar argument establishes that X
t grows.To establish the relation between the mean reversion functionμ
(
t
) and the expected value of the process E
X
t, we can consider (1) written in integral form as(2)
X
t
-
X
0
=
α
∫
0
t
μ
s
-
X
s
d
s
+
σ
∫
0
t
X
s
γ
d
B
s
.Taking the expected value,E
[
X
t
]
-
E
[
X
0
]
=
α
∫
0
t
μ
s
-
E
X
s
d
s and thus obtain the ordinary differential equation:(3)
m
˙
t
=
α
μ
t
-
m
t
;
m
t
=
E
X
t
.The solution of this equation ism
(
t
)
=
m
0
e
-
α
t
+
e
-
α
t
∫
0
t
e
α
s
μ
s
d
s.To illustrate graphically the relation betweenμ
(
t
) and m
(
t
), we use μ
t
=
a
+
b
sin
⁡
2
π
t
/
P as an example. In this case the solution to (3) is given by(4)
m
t
=
a
+
m
0
-
a
+
α
β
b
α
2
+
β
2
e
-
α
t
+
α
b
α
2
+
β
2
α
sin
⁡
β
t
-
β
cos
⁡
β
t
,where β
=
2
π
/
P.Figure1 shows the dynamic behavior of μ
(
t
),  m
(
t
) and one path of X
t for some specific values in the parameters.Figure 1
Dynamic behavior form
(
t
), μ
(
t
), and X
t, where X
0
=
2, α
=
37, σ
=
0.7, γ
=
1, a
=
1.5, b
=
1, P
=
1, and β
=
2
π with Δ
t
=
1
/
250 for a total of 250 observations.From the Figure it can be seen that the expected value of the process mimics the long-term mean: that is just the way mean reversion works.


## 3. Phase 1: Parameter Estimation

Similar to the procedure established in Marín Sánchez and Palacio [10] and Huzurbazar [22] the main objective is to obtain closed formulas for the parameter estimators from discrete observations of a sample path of the process. This can be achieved from the discretized maximum likelihood function, which is obtained from the process that results from the Euler-Maruyama discretization scheme applied to (1) and using the normal and independent increments properties of the Brownian Motion over the residuals of the discrete process.Consider the differential equation (1) with the initial value X
0
=
x. α
μ
t
-
X
t and σ
X
t
γ are, respectively, the trend and diffusion functions that depend on t, X
t, and a vector of unknown parameters θ
=
α
,
σ
,
γ. Now suppose that the conditions that guarantee the existence and uniqueness of the solution Mao [23] are satisfied, so the expected value of X
t exists and μ
(
t
) can be defined using m
(
t
) and its derivative m
˙
(
t
) (see (3)). In this way,(5)
μ
t
=
m
t
+
m
˙
t
α
.Replacingμ
(
t
) in (1) results in(6)
d
X
t
=
α
m
t
+
m
˙
t
α
-
X
t
d
t
+
σ
X
t
γ
d
B
t
;
t
∈
0
,
T
.Now we can assume that we have the discrete observationsX
t
=
X
(
τ
t
) for the times τ
0
,
τ
1
,
…
,
τ
T, where Δ
t
=
τ
t
-
τ
t
-
1
≥
0, t
=
1,2
,
…
,
T. Using the numerical scheme of Euler-Maruyama on (6) it follows that(7)
X
τ
t
=
X
τ
t
-
1
+
α
m
τ
t
-
1
-
X
τ
t
-
1
+
m
˙
τ
t
-
1
Δ
t
+
X
γ
τ
t
-
1
ε
τ
t
,where ε
τ
t
≈
N
(
0
,
σ
2
Δ
t
).Define the new variableY
t:(8)
Y
t
=
X
t
-
X
t
-
1
-
α
m
t
-
1
-
X
t
-
1
+
m
˙
t
-
1
Δ
X
t
-
1
γ
with
Δ
constant.Note thatY
t,  i
=
1,2
,
3
,
…
,
T, are independent random variables distributed N
(
0
,
σ
2
Δ
).The variableY
t depends on the observations X
=
X
0
,
X
1
,
…
,
X
T
′ of the sample path of the process and on the unobservable quantities m
t and its derivative m
˙
t. Taking into account that the sample path resumes all the information that is known from the process, it is necessary to estimate the expected value m
(
t
) from those observations. As in Tifenbach [4], p. 31-32, we can assume that the expected value of the process can be approximated using a convolution1 over the sample path, so the variable m
(
t
) will depend only on the observations X
=
X
0
,
X
1
,
…
,
X
T
′.Thus, we assume that the expected value of the processX
t can be approximated by taking a convolution of the sample path X
=
X
0
,
X
1
,
…
,
X
T for some particular convolution function c
(
t
) given by(9)
m
i
=
∑
k
=
-
N
N
c
k
X
i
-
k
.The most common convolution is the moving average, where c
k constants are given by c
k
=
1
/
2
N
+
1 for all N, which is denoted by (
c
t
1
). Other convolutions can be defined varying the relative weight of each observation: for example, they can be defined from the coefficients of odd rows of Pascal’s triangle which is denoted by (
c
t
2
). In this case the central observations have a greater weight than more distant observations and for this reason the convolution is close to the path. Those specific cases of convolutions are presented in Figure 2.Figure 2
Dynamic behavior for processX
t, the expected value m
(
t
), and three types of convolution. (a) Moving average (
c
t
1
) with N
=
20. (b) Pascal’s weights (
c
t
2
) with N
=
20. (c) Hodrick-Prescott filter (
c
t
h
p
) with λ
h
p
=
40000. For all the cases Δ
t
=
1
/
250 for a total of 250 observations and the parameters of X
t are the same as in Figure 1.
(a)
(b)
(c)The convolution is used for the purpose of smoothing the sample path, eliminating the noise and capturing the trend. This objective can also be achieved using other techniques like filtering. In Figure2 the Hodrick-Prescott [24] filter denoted by (
c
t
h
p
) was used to obtain an estimation of m
(
t
) that are smoother than the convolutions (
c
t
1
) and (
c
t
2
).At this point we have that the expected valuem
(
t
) can be approximated from the sample path X
=
(
X
0
,
X
1
,
…
,
X
T
). Now, to get the derivative of m
(
t
) we can apply numerical derivatives techniques like Taylor’s theorem: for example, the derivative at the observation i can be defined using a three-point rule as(10)
m
˙
i
=
2
m
i
+
1
-
3
m
i
+
m
i
-
1
Δ
,for i
=
1,2
,
…
,
T
-
1. For i
=
0 the derivate is given by m
˙
0
=
m
1
-
m
0
/
Δ and for i
=
T by m
˙
T
=
m
T
-
m
T
-
1
/
Δ. This procedure can be done using a different number of points to calculate the derivate at each point.With the estimation ofm
(
t
) and m
˙
(
t
) we have one realization of Y
t and now we can define its normal density function:(11)
f
Y
i
:
θ
=
1
2
π
σ
2
Δ
1
/
2
exp
⁡
-
1
2
σ
2
Δ
X
i
-
X
i
-
1
-
α
m
i
-
1
-
X
i
-
1
+
m
˙
i
-
1
Δ
X
i
-
1
γ
2
.Since eachY
i is independent the joint density function can be expressed as the product of their marginal densities:(12)
f
Y
1
,
Y
2
,
…
,
Y
T
=
∏
k
=
1
T
f
Y
k
:
θ
.Consequently, the likelihood function is given by(13)
L
θ
∣
Y
t
=
1
2
π
σ
2
Δ
T
/
2
exp
⁡
-
1
2
σ
2
Δ
∑
i
=
1
T
X
i
-
X
i
-
1
-
α
m
i
-
1
-
X
i
-
1
+
m
˙
i
-
1
Δ
X
i
-
1
γ
2
.Taking the natural logarithm on both sides it follows that(14)
log
⁡
L
=
-
T
2
log
⁡
2
π
σ
2
Δ
-
1
2
σ
2
Δ
∑
i
=
1
T
X
i
-
X
i
-
1
-
α
m
i
-
1
-
X
i
-
1
+
m
˙
i
-
1
Δ
X
i
-
1
γ
2
.Now consider the problem of maximizing the likelihood function given by(15)
∂
log
⁡
L
∂
θ
=
0
→
,
ie
,
θ
^
=
arg
θ
⁡
max
⁡
log
⁡
L
.If we assumed that the value ofγ is knowna priori, the estimation process leads to the estimated parameter α given by(16)
α
^
=
∑
i
=
1
T
X
i
-
X
i
-
1
-
m
˙
i
-
1
Δ
m
i
-
1
-
X
i
-
1
/
X
i
-
1
2
γ
∑
i
=
1
T
m
i
-
1
-
X
i
-
1
/
X
i
-
1
γ
2
Δ
.The estimatorσ
^ is determined by the equation:(17)
σ
^
=
1
T
Δ
∑
i
=
1
T
X
i
-
X
i
-
1
-
α
^
m
i
-
1
-
X
i
-
1
+
m
˙
i
-
1
Δ
X
i
-
1
γ
2
.Finally, with the estimation ofα
^ we can now approximate μ
^
(
t
) with (5).


## 4. Preliminary Results

The aim of this section is to show the performance of the estimators defined in the previous section. To make this we will simulate some paths of a process satisfying (1) with a set of parameters α and σ and giving a deterministic form to the μ
(
t
) function and then find the estimators with the established procedure in order to determine—using statistical properties—if the estimates are approximate to actual parameters.We analyze the performance of the estimators for sinusoidal and parabolicμ
(
t
) trend functions and for a set of values of α and σ when γ
=
1. Each generated path consists of 250 observations with Δ
t
=
1
/
250. The statistics are calculated for 1000 different simulations. Considering that for the parameters estimation it is necessary to find the expected value of the process from each path, we compare two different convolution functions and the Hodrick-Prescott filter. For numerical derivation a five-point derivation rule is used.The values of the parameters andμ
(
t
) functions used in the simulations are presented in Table 1.Table 1
Simulation parameters for sinusoidal and parabolic trend functions.
Case  1
Case  2
α
30
30
σ
0.4
0.13
X
0
2
1.5
μ
(
t
)
1.5
+
sin
⁡
2
π
t
-
0.5
t
2
+
t
+
1.5In Table2 we present some basic statistics (mean θ
¯ and standard deviation s
θ) for the estimators obtained from the simulations. The convolution functions used in the simulations are the ones defined for the weights c
k
1 and c
k
3. c
k
1 weights are the same defined in Section 3 for the moving average and c
k
3 weights are given by c
k
,
N
3
=
2
N
-
k
/
N
(
3
N
+
1
) for k from -
N to N. In this convolution the central observation has a relative weight twice the most distant observation.Table 2
Results obtained with 1000 simulations with 250 observations each. The real values of the parameters are shown in Table1. For case 1 in the convolutions we use N
=
22 and for case 2 N
=
60. For both cases we set λ
=
62500 in c
k
h
p.
Case  1
Case  2
μ
1
(
t
)
c
k
1
c
k
3
c
k
h
p
μ
2
(
t
)
c
k
1
c
k
3
c
k
h
p
α
α
-
24.8540
31.3347
32.8771
α
α
-
31.1160
35.8249
51.1093
s
α
3.2218
3.9947
5.1406
s
α
5.8254
6.6984
10.1414
σ
σ
-
0.4002
0.3942
0.4023
σ
σ
-
0.1293
0.1281
0.1267
s
σ
0.0177
0.0174
0.0174
s
σ
0.0060
0.0059
0.0060From Table2 we can conclude that the estimation technique is working well, especially in the first example (sinusoidal trend). The relative error of the estimators is less than 2% in the case of the parameter σ but there is a bias in the parameter α that depends on the convolution function and at the same time on the parameter N used in such convolution.In Figure3 we can see the performance of the estimation of the deterministic function μ
(
t
). In that figure an estimation from a random path is shown instead of the average. The estimates are close to the theoretical function in both cases but having some differences at the beginning and the final of the observations. One form to measure the fit between the theoretical function and its estimation is using the Root Mean Square error (RMS). Table 3 presents that measure for the proposed estimation in cases 1 and 2.Table 3
Results of RMS error for each estimationμ
1
(
t
) and μ
2
(
t
).
RMS
Case  1
Case  2
c
k
1
0.0782
0.0266
c
k
3
0.0578
0.0238
c
k
h
p
0.0767
0.0140Figure 3
Estimatedμ
1
(
t
) and μ
2
(
t
). The left side correspond to Case 1 (sinusoidal) and the right side to Case 2 (parabolic). The (A) plots are obtained with the convolution c
k
1, (B) plots with (
c
k
3
) and (C) plots with (
c
k
h
p
). The parameters involved in this figure are presented in Tables 1 and 2.
(a)
(b)


## 5. Phase 2: Re-Estimation

From the simulations we can see that the estimate ofμ
(
t
) is not very accurate and that differences come from m
^
t and m
˙
^
t, and in turn this affects the estimation of other parameters.If it were possible to have all the paths of the process it would not be necessary for numerical approximation of the expected value or its derivative to capture information fromμ
(
t
), and it could be enough to perform an average of all paths at each time t
i and that could generate pretty accurate estimations. As this condition is not the usual but only has a path, the procedure detailed above is necessary and this involves the errors mentioned in the estimates of m
(
t
) and m
˙
(
t
).To correct this deficiency it is proposed to carry out a reestimate procedure trying to adjust a function to the initial estimate froma priori knowledge of the process. This recalculation allows a correction in the other estimators while providing a functional form that allows simulations of future paths of the process.The following procedure is proposed to perform the reestimate ofμ
(
t
):(1)
Get an estimate ofμ
^
(
t
) according to the procedure described in Section 4.
(2)
Define a functional structuref
(
Θ
,
t
) froma priori knowledge of the process.
(3)
Estimate the vector parametersΘ minimizing the error function defined by (18)
E
r
r
o
r
Θ
=
∑
t
=
1
T
f
Θ
,
t
-
μ
^
t
2
.As an example, suppose the functional form of the long-term mean is defined as(19)
f
Θ
,
t
=
a
t
2
+
b
t
+
c
,where Θ
=
a
,
b
,
c
′. In this case the error function would be given by(20)
E
r
r
o
r
Θ
=
∑
t
=
0
T
μ
^
t
-
a
t
2
+
b
t
+
c
2
.To estimate Θ it is necessary to solve the system of equations defined by ∇
E
r
r
o
r
(
Θ
)
=
0, which is equivalent to the system of linear equations M
Θ
=
d, where(21)
M
=
∑
t
4
∑
t
3
∑
t
2
∑
t
3
∑
t
2
∑
t
∑
t
2
∑
t
T
+
1
d
=
∑
t
2
μ
^
t
∑
t
μ
^
t
∑
μ
^
t
.A similar procedure can be implemented under the assumption that f
Θ
,
t
=
a
+
b
sin
⁡
2
π
/
P
t, to find Θ
=
a
,
b
′ if P is known. A comparison between μ
(
t
), μ
e
s
t
(
t
), and μ
r
e
s
t
(
t
)
=
f
(
Θ
^
,
t
) is shown in Figure 4 for both cases and for a random path of the process defined with the same set of parameters of Table 1.Figure 4
Comparison of dynamic behavior ofμ
(
t
), μ
e
s
t
(
t
), and μ
r
e
s
t
(
t
). In case (a) μ
1
(
t
)
=
1.5
+
sin
⁡
2
π
t and in case (b) μ
2
(
t
)
=
-
0.5
t
2
+
t
+
1.5.
(a)
(b)Having a best estimate ofμ
(
t
) allows a reestimation of the parameters α and σ. To achieve this we must perform a similar procedure to that proposed in Section 4, defining(22)
Y
t
=
X
t
-
X
t
-
1
-
α
μ
t
-
1
-
X
t
-
1
Δ
X
t
-
1
γ
Δ
constant
.So,(23)
α
^
^
=
∑
i
=
1
T
X
i
-
X
i
-
1
μ
^
^
i
-
1
-
X
i
-
1
Δ
/
X
i
-
1
2
γ
∑
i
=
1
T
μ
^
^
i
-
1
-
X
i
-
1
/
X
i
-
1
γ
2
Δ
,
σ
^
^
=
1
T
Δ
∑
i
=
1
T
X
i
-
X
i
-
1
-
α
^
^
μ
^
^
i
-
1
-
X
i
-
1
Δ
X
i
-
1
γ
2
.Table4 summarizes the results obtained from the first estimation and the reestimation procedure for the cases where μ
(
t
) has a sinusoidal form and a parabolic form and the other parameters are defined in Table 1.Table 4
Estimated and reestimated parameters. The estimations are calculated with the convolution(
c
k
h
p
). Results obtained with 1000 simulations with 250 observations each for μ
1 and μ
2 as defined in Table 1: case 1 α
=
30 and σ
=
0.4, case 2 α
=
30 and σ
=
0.13.
μ
1
(
t
)
α
^
α
^
^
μ
2
(
t
)
α
^
α
^
^
α
α
-
24.8540
29.6620
α
α
-
31.1160
30.4616
s
α
3.2218
3.4518
s
α
5.8254
5.6828
μ
1
(
t
)
σ
^
σ
^
^
μ
2
(
t
)
σ
^
σ
^
^
σ
σ
-
0.4002
0.4021
σ
σ
-
0.1293
0.1300
s
σ
0.0177
0.0187
s
σ
0.0060
0.0060From the results of the above example we can observe how the reestimation procedure reduces the bias in the initial estimation ofα for both sinusoidal and parabolic cases. The final bias in α is approximately 1.5% of the real value of the parameter, reducing almost 90% of the error in the first estimation in the case of sinusoidal function and near 60% for the parabolic function. The reestimations of σ do not significantly change from the original estimate. For the case of μ
(
t
) the RMS error is present in Table 5 for a random path of the process. In both cases the error in the reestimation is less than the 50% of the error present in the initial estimation.Table 5
Results of RMS error for the estimation and reestimation ofμ
(
t
): case 1 sinusoidal trend and case 2 parabolic trend.
RMS
Case  1
Case  2
μ
e
s
t
(
t
)
0.0961
0.0137
μ
r
e
s
t
(
t
)
0.0261
0.0058


## 6. Conclusions and Comments

This paper shows how starting from a path of a stochastic process is possible to find an approximation of the parameters that define the underlying dynamics It is important to highlight that, for modeling the dynamic behavior of prices of some commodities or interest rates, only one realization of the process in discrete time is available and it represents the only source of information to capture the nature of the process. Therefore, it becomes very important to obtain and interpret the invisible and visible information found on the path to make projections more adjusted to the future process behavior being studied.It is essential that a model that seeks to reproduce the behavior of a process that is generated in the real world incorporatesa priori information provided by an expert in the topic and that this additional information allows the model to better fit the process. This prior information is key in the development of the methodology proposed in this paper, because it allows us to adjust the initial estimates of the parameters making more reliable the results obtained with the adjusted models.On the other hand, in the development of the proposed methodology it can be seen that it allows us to obtain closed formulas for the maximum likelihood estimators in mean reversion process when the long-term trend is defined by a deterministic function, decreasing computational effort and allowing greater understanding of procedures.



---

*Source: 5191583-2016-08-22.xml*