# Extended Stochastic Gradient Identification Method for Hammerstein Model Based on Approximate Least Absolute Deviation

Authors: Bao-chang Xu; Zhong-hua Lin; Ying-Dan Zhang; Yu-yue Xiao
Journal: Mathematical Problems in Engineering
 (2016)

---

## Abstract

In order to identify the parameters of nonlinear Hammerstein model which are contaminated by colored noise and peak noise, the least absolute deviation (LAD) is selected as the objective function to solve the problem of large residual square when the identification data is disturbed by the impulse noise which obeys symmetrical alpha stable (S
α
S) distribution. However, LAD cannot meet the need of differentiability required by most algorithms. To improve robustness and to solve the nondifferentiable problem, an approximate least absolute deviation (ALAD) objective function is established by introducing a deterministic function to replace absolute value under certain situations. The proposed method is derived from ALAD criterion and extended stochastic gradient method. Due to the differentiability of the objective function, we can get a recursive identification algorithm which is simple and easy to calculate compared with LAD. The convergence of the proposed identification method is also proved by Lyapunov stability theory, and the simulation experiments show that the proposed method has higher accuracy and stronger robustness than the least square (LS) method in the identification of Hammerstein model with colored noise and impulse noise. The impact of impulse noise can be restrained effectively.


---

## Body

## 1. Introduction

In recent years, the Hammerstein model has drawn a lot of attention because of its block-oriented nonlinear (BONL) character. It can be used to describe a variety of nonlinear systems such as nonlinear filtering, actuator saturation, signal analysis, and chemistry in biology system. So far, there have been several kinds of methods for the identification of Hammerstein model with white noise such as the iterative method [1, 2], the overparameterization method [3], the support vector machine (SVM) method [4], the subspace method [5–7], the blind method [8], the frequency domain identification method [9], and the artificial intelligence method [10–12]. In industry process control, however, the noise always includes colored noise and peak noise. It is necessary to consider the identification method for Hammerstein model when data are contaminated by colored noise and peak noise. Chang and Luus proposed an iterative method for Hammerstein model with colored noise [13], but it cannot be used for online-identification. Ding and Chen proposed an extended stochastic gradient method based on least squares for the nonlinear Hammerstein ARMAX system and proved its convergence by the martingale convergence theorem [2]. In most researches, the LS criterion is taken as the objective function during the identification of Hammerstein model. The studies show that the LS method has better identification effect when the stochastic noise is normally distributed [14]. However, the LS method cannot work as well as the LAD method in certain conditions; for example, if the noise does not obey the normal distribution, the statistics performance of LAD estimation is better than LS estimation with irreplaceable advantages [15]. As to the square term in the objective function of LS method, a small change of the measured data will lead to a great influence on the identification results when there are outliers in the measured data [16].To compensate the effect of the impulse noise and outliers on the identification accuracy, the LAD criterion is chosen to be the objective function which replaces the square terms with absolute deviation. The LAD method decreases the sensitivity to impulse noise and outliers and greatly improves the robustness because the LAD criterion only does the first power computation of the deviation. The LAD objective function is not differentiable. It needs to solve a nonsmooth optimization problem [17], which complicates the computation. The proposed method replaces the absolute deviation in LAD with a certain differentiable function and rebuilds the ALAD objective function. This paper derives the identification algorithm for Hammerstein model from ALAD objective function and the extended stochastic gradient method. To improve the identification accuracy and convergence rates, we add an inertial term to the proposed method. The convergence of the algorithm is proved by Lyapunov stability theory at the same time. The simulation experiments show that the proposed method can effectively eliminate the influence of impulse noise and outliers. Compared with the LS method, the ALAD method has stronger robustness and higher identification accuracy, demonstrating the superiority of the proposed method.The rest of this paper is organized as follows. Section2 describes the identification problem of nonlinear Hammerstein systems with colored noise. Section 3 derives the proposed identification algorithm from ALAD criterion. The convergence of the proposed method is also discussed in Section 3. Section 4 offers an illustrative example and compares the proposed algorithm with existing LS methods. Some concluding remarks are provided in Section 5.


## 2. Hammerstein Model with Colored Noise

As is shown in Figure1, the structure of Hammerstein model is a series connection of a nonmemory static nonlinear block and a linear dynamic block. The static nonlinear block is connected with an input signal; the linear dynamic block is connected with an output signal.Figure 1
The structure of Hammerstein model.In Figure1, u
(
k
) is an input and x
(
k
) is the output of the nonlinear part and it is the input of the linear part as well. r
(
k
) is a real output. y
(
k
) is a measured output and the result of r
(
k
) which is disturbed by the colored noise e
(
k
). v
(
k
) is an additive white noise with zero mean. The colored noise e
(
k
) is the output of the linear link L
(
z
) that is driven by the white noise v
(
k
). L
(
z
) is the “noise model” which decides the property of e
(
k
), and G
(
z
) is the transfer function. Assuming the nonlinear part x
(
k
) could be represented by a polynomial about the input u
(
k
) with known order(1)
x
k
=
f
u
k
=
c
1
f
1
u
k
+
c
2
f
2
u
k
+
⋯
+
c
n
c
f
n
c
u
k
=
∑
j
=
1
n
c
c
j
f
j
u
kthe linear block of Hammerstein model is described by the model of CARMA/ARMAX (Controlled Autoregressive Moving Average/Autoregressive Moving Average eXogenous), the transfer function G
(
z
)
=
B
(
z
)
/
A
(
z
), and L
(
z
)
=
D
(
z
)
/
A
(
z
). Therefore, the relationship of input and output can be written as(2)
y
k
=
r
k
+
e
k
=
B
z
A
z
x
k
+
D
z
A
z
v
k
.From (1) and (2), it is obtained that(3)
A
z
y
k
=
B
z
∑
j
=
1
n
c
c
j
f
j
u
k
+
D
z
v
k
,where A
(
z
), B
(
z
), and D
(
z
) are the polynomials of the shift operator:(4)
A
z
=
1
+
a
1
z
-
1
+
⋯
+
a
n
a
z
-
n
a
;
B
z
=
b
1
z
-
1
+
⋯
+
b
n
b
z
-
n
b
;
D
z
=
1
+
d
1
z
-
1
+
⋯
+
d
n
d
z
-
n
d
,where a
i, b
j, c
k, and d
l (i
=
1
,
…
,
n
a
,
j
=
1
,
…
,
n
b
,
k
=
1
,
…
,
n
c
,
l
=
1
,
…
,
n
d) are the parameters to be identified.From (3), it is obtained that(5)
y
k
=
-
∑
i
=
1
n
a
a
i
y
k
-
i
+
∑
i
=
1
n
b
b
i
∑
j
=
1
n
c
c
j
f
j
u
k
-
i
+
∑
i
=
1
n
d
d
i
v
k
-
i
+
v
k
.Define(6)
θ
=
a
d
ζ
∈
R
n
,
n
=
n
a
+
n
b
×
n
c
+
n
d
,where(7)
ξ
=
b
⊗
c
=
b
×
c
1
b
×
c
2
⋮
b
×
c
n
c
∈
R
n
b
×
n
c
×
1
.
a, b, and d are the parameter vectors of the linear section and c is the parameter vector of the nonlinear section of the Hammerstein model:(8)
a
=
a
1
a
2
⋮
a
n
a
,
b
=
b
1
b
2
⋮
b
n
b
,
c
=
c
1
c
2
⋮
c
n
c
,
d
=
d
1
d
2
⋮
d
n
d
.We define(9)
φ
k
=
-
y
k
-
1
,
-
y
k
-
2
,
…
,
-
y
k
-
n
a
,
v
k
-
1
,
v
k
-
2
,
…
,
v
k
-
n
d
,
ψ
k
T
,
ψ
k
=
f
1
u
k
-
1
,
f
1
u
k
-
2
,
…
,
f
1
u
k
-
n
b
,
f
2
u
k
-
1
,
f
2
u
k
-
2
,
…
,
f
2
u
k
-
n
b
,
…
,
f
n
c
u
k
-
1
,
f
n
c
u
k
-
2
,
…
,
f
n
c
u
k
-
n
b
.
Then, (5) can be rewritten as(10)
y
k
=
φ
T
k
θ
+
v
k
.


## 3. The Identification Method Based on ALAD Criterion and Extended Stochastic Gradient Method

### 3.1. Approximate Least Absolute Deviation

As is known, the absolute deviation is not differentiable. To overcome this shortage, we choose a certain differentiable function to approximate the LAD criterion. This requirement can be met by the following logarithmic function:(11)
J
x
=
β
ln
⁡
cosh
⁡
x
β
.In (11), β is an adjustable parameter, and the nonlinear function J
(
∗
) is related to β. While β is small enough, nonlinear function cosh
⁡
(
x
/
β
)
≈
1
/
2
e
|
x
/
β
|, and β
ln
⁡
[
cosh
⁡
(
x
/
β
)
]
≈
|
x
|; that is, J
(
x
)
≈
|
x
|, approximating the absolute value function. The property curve of the logarithmic function when β
=
0.01 is shown in Figure 2.Figure 2
The curve of function property whenβ
=
0.01.The curve shows that whileβ is small enough, nonlinear function J
(
x
)
=
β
ln
⁡
[
cosh
⁡
(
x
/
β
)
] can approximate the absolute value function x effectively. Because J
(
x
) is differentiable, then the identification algorithm based on J
(
x
) can be solved by some optimization methods.


### 3.2. Algorithm Derivation

In (10), the input u
(
k
) and output y
(
k
) are measurement data. According to the known structure of the nonlinear function f
(
∗
), we can get the value of f
(
u
(
k
)
). But v
(
k
-
i
) in φ
(
k
) is white noise and immeasurable, which makes it impossible to solve the identification problem. Based on the idea that the noise can be replaced by its estimated value, we use v
^
(
k
-
i
), the noise residual in the ith step, to replace v
(
k
-
i
) in the i
+
1th step, so φ
(
k
) can be replaced by its estimated value φ
^
(
k
):(12)
φ
^
k
=
-
y
k
-
1
,
-
y
k
-
2
,
…
,
-
y
k
-
n
a
,
v
^
k
-
1
,
v
^
k
-
2
,
…
,
v
^
k
-
n
d
,
ψ
k
T
.Then, the identification of θ in (10) is to find θ that minimizes the following criterion function (ALAD) based on input data φ
^
(
k
) and output data y
(
k
):(13)
J
θ
θ
^
k
=
β
ln
⁡
cosh
⁡
y
k
-
φ
T
k
θ
β
θ
^
k
=
m
i
n
.For (13), it is a typical nonlinear function optimization. To reduce the computation complexity, we use the stochastic gradient method to derive the recursive equation of θ
^
k [18, 19]. The formula of the stochastic gradient method is(14)
θ
^
k
=
θ
^
k
-
1
-
μ
k
grad
⁡
J
θ
θ
^
k
-
1
,where grad
⁡
∗ represents gradient and μ
k is step size. The formula above shows that θ
^
k is corrected along the negative gradient direction until the extreme value of J
(
θ
) is obtained.Firstly, we get the gradient ofJ
(
θ
):(15)
grad
⁡
J
θ
θ
^
k
-
1
=
∂
J
∂
θ
θ
^
k
-
1
=
-
φ
^
k
tanh
⁡
y
k
-
φ
^
T
k
θ
^
k
-
1
β
;then, from (15) and (13), we obtain(16)
J
θ
θ
^
k
-
1
=
β
ln
⁡
cosh
⁡
y
k
-
φ
^
T
k
θ
^
k
-
1
+
μ
k
φ
^
k
tanh
⁡
y
k
-
φ
^
k
θ
^
k
-
1
/
β
β
=
β
ln
⁡
cosh
⁡
1
β
e
^
k
-
μ
k
φ
^
T
k
φ
^
k
tanh
⁡
e
^
k
β
,where e
^
(
k
)
=
y
(
k
)
-
φ
^
T
(
k
)
θ
^
(
k
-
1
).We can get the gradient ofJ
(
θ
) to μ
k from (16):(17)
∂
J
θ
∂
μ
k
=
-
K
k
∗
tanh
⁡
e
^
k
-
μ
k
K
k
β
,where(18)
K
k
=
φ
^
T
k
φ
^
k
tanh
⁡
e
^
k
β
.Equation (17) is obtained based on the exact line search algorithm and always leads to the two adjacent gradient search directions being orthogonal and usually results in zigzag phenomenon. When we use (17) to compute the optimal μ
k, the convergence rate is slow because of the linear convergence order, and the identification results of the parameters may be fluctuated. For this shortcoming, an improved stochastic gradient method is applied to get μ
k where μ
k is based on the information of gradient that is k times before the current iteration step; that is, (19)
K
k
=
∑
l
=
1
k
φ
^
T
l
φ
^
l
tanh
⁡
e
^
l
β
.From e
^
(
k
)
-
μ
k
K
k
=
0, it is obtained that(20)
μ
k
=
e
^
k
K
k
=
e
^
k
∑
l
=
1
k
φ
^
T
l
φ
^
l
tanh
⁡
e
^
l
/
β
.From (14), (15), and (19), it is obtained that (21)
θ
^
k
=
θ
^
k
-
1
+
e
^
k
∑
l
=
1
k
φ
^
T
l
φ
^
l
tanh
⁡
e
^
l
/
β
φ
^
k
tanh
⁡
e
^
k
β
;then(22)
θ
^
k
=
θ
^
k
-
1
+
φ
^
k
r
k
tanh
⁡
e
^
k
β
,where (23)
r
k
=
1
μ
k
=
∑
l
=
1
k
φ
^
T
k
φ
^
k
tanh
⁡
e
^
k
/
β
e
^
k
.r
(
k
) is computed by the following recursive formula:(24)
r
k
=
λ
k
r
k
-
1
+
1
e
^
k
φ
^
T
k
φ
^
k
tanh
⁡
e
^
k
β
,where λ
(
k
)
=
e
^
(
k
-
1
)
/
e
^
(
k
).Equation (24) is equivalent for adding an inertial term to the recursive equation of μ
(
k
). The weights of the inertial term are related to the iterative output error at k
-
1 and k step.In conclusion, we can obtain the extended stochastic gradient identification method based on approximate least absolute deviation (ALADESG):(25)
θ
^
k
=
θ
^
k
-
1
+
φ
^
k
r
k
tanh
⁡
e
^
k
β
,
r
k
=
λ
k
r
k
-
1
+
1
e
^
k
φ
^
T
k
φ
^
k
tanh
⁡
e
^
k
β
,
e
^
k
=
y
k
-
φ
^
T
k
θ
^
k
-
1
,
φ
^
k
=
-
y
k
-
1
,
-
y
k
-
2
,
…
,
-
y
k
-
n
a
,
v
^
k
-
1
,
v
^
k
-
2
,
…
,
v
^
k
-
n
d
,
ψ
k
T
,
ψ
k
=
f
1
u
k
-
1
,
f
1
u
k
-
2
,
…
,
f
1
u
k
-
n
b
,
f
2
u
k
-
1
,
f
2
u
k
-
2
,
…
,
f
2
u
k
-
n
b
,
…
,
f
n
c
u
k
-
1
,
f
n
c
u
k
-
2
,
…
,
f
n
c
u
k
-
n
b
.


### 3.3. Analysis of Convergence

Rewrite (22) as(26)
θ
^
k
=
θ
^
k
-
1
+
c
k
φ
^
k
e
^
k
,where(27)
c
k
=
tanh
⁡
e
^
k
/
β
∑
l
=
1
k
φ
^
T
l
φ
^
l
tanh
⁡
e
^
l
/
β
.If the truth values of process parameters are noted as θ
0, then y
(
k
)
=
φ
^
T
(
k
)
θ
0.Definingθ
~
(
k
)
=
θ
0
-
θ
^
(
k
), according to (26), we have(28)
θ
^
k
=
θ
^
k
-
1
+
c
k
φ
^
k
y
k
-
φ
^
T
k
θ
^
k
-
1
;subtracting θ
0 on both sides of (28)(29)
-
θ
~
k
=
-
θ
~
k
-
1
+
c
k
φ
^
k
φ
^
T
k
θ
~
k
-
1then(30)
θ
~
k
=
I
-
c
k
φ
^
k
φ
^
T
k
θ
~
k
-
1
.Constructing the Lyapunov function(31)
V
θ
~
k
,
k
=
θ
~
T
k
θ
~
k
>
0then(32)
Δ
V
θ
~
k
,
k
=
V
θ
~
k
,
k
-
V
θ
~
k
-
1
,
k
-
1
=
θ
~
T
k
-
1
G
k
θ
~
k
-
1
,where(33)
G
k
=
I
-
c
k
φ
^
k
φ
^
T
k
T
I
-
c
k
φ
^
k
φ
^
T
k
-
I
=
φ
^
T
k
φ
^
k
c
2
k
-
2
c
k
φ
^
k
φ
^
T
k
.
G
(
k
) is a real symmetric matrix, and the rank of r
G
=
1. Obviously, when the following inequality in (34) is fulfilled, the matrix G
(
k
) is negative and semidefinite, and then (28) will be asymptotically stable in a large scope at the balance point θ
~
(
k
)
=
0:(34)
c
2
k
φ
^
T
k
φ
^
k
-
2
c
k
≤
0
.According to (27), (35)
c
2
k
φ
^
T
k
φ
^
k
-
2
c
k
=
t
a
n
h
2
e
^
k
/
β
φ
^
T
k
φ
^
k
∑
l
=
1
k
φ
^
T
l
φ
^
l
tanh
⁡
e
^
l
/
β
2
-
2
tanh
⁡
e
^
k
/
β
∑
l
=
1
k
φ
^
T
l
φ
^
l
tanh
⁡
e
^
l
/
β
=
t
a
n
h
2
e
^
k
/
β
φ
^
T
k
φ
^
k
∑
l
=
1
k
φ
^
T
l
φ
^
l
tanh
⁡
e
^
l
/
β
2
-
2
tanh
⁡
e
^
k
/
β
∑
l
=
1
k
φ
^
T
l
φ
^
l
tanh
⁡
e
^
l
/
β
∑
l
=
1
k
φ
^
T
l
φ
^
l
tanh
⁡
e
^
l
/
β
2
=
-
t
a
n
h
2
e
^
k
/
β
∑
j
=
1
n
φ
j
2
k
-
2
tanh
⁡
e
^
k
/
β
∑
l
=
1
k
-
1
φ
^
T
l
φ
^
l
tanh
⁡
e
^
l
/
β
∑
l
=
1
k
φ
^
T
l
φ
^
l
tanh
⁡
e
^
l
/
β
2
.The second term of (35) can be written as(36)
-
2
tanh
⁡
e
^
k
/
β
r
k
-
1
e
k
-
1
r
2
k
e
2
k
=
-
2
tanh
⁡
e
^
k
/
β
e
k
1
r
2
k
e
k
-
1
r
k
-
1
e
k
,where (37)
tanh
⁡
e
^
k
/
β
e
k
>
0
,
1
r
2
k
>
0
.During the iterative process, if we could make sure that(38)
e
k
-
1
r
k
-
1
e
k
≥
0then we get (39)
c
2
k
φ
^
T
k
φ
^
k
-
2
c
k
≤
0
.Therefore, in order to guarantee the convergence of the proposed algorithm, we need to add (38) as a constraint when (24) is used for iteration.


### 3.4. Separation of Parameters by Average Method

It is noteworthy that in the description of the Hammerstein model (as shown in Figure1), for some nonzero and finite constant a, any a
f
(
u
) and G
(
z
)
/
a can match and yield identical input and output measurements; thus, the nonlinear function f
(
u
) and the transfer function G
(
z
) are not unique. In other words, any identification scheme will not be able to distinguish between (
f
(
u
)
,
G
(
z
)
) and (
a
f
(
u
)
,
G
(
z
)
/
a
). Therefore, without any loss of generality, one of the gains of f
(
u
) and G
(
z
) has to be fixed. There are several ways to normalize these gains [20, 21]. We adopt the following method.Assumption 1.
The first coefficient of the functionf
(
·
) equals 1 [22]; that is to say, in (1), c
1
=
1.
Based on the relationship between the parameters,c
^ can be computed using(40)
c
^
j
i
k
=
θ
^
n
a
+
n
b
+
i
k
b
^
i
k
,
j
=
2,3
,
…
,
n
c
,
i
=
1,2
,
…
,
n
b
.From (40), we can see that the number of c
^
j
i
(
k
) is n
b, and c
^
j
i
(
k
) is the estimated value of c
j
(
k
). In order to obtain the estimated value of the parameters with much higher accuracy, we use the average method below to estimate c
^:(41)
c
^
j
k
=
1
n
b
∑
i
=
1
n
b
c
^
j
i
k
=
1
n
b
∑
i
=
1
n
b
θ
^
n
a
+
n
b
+
i
k
b
^
i
k
,
j
=
2,3
,
…
,
n
c
.


## 3.1. Approximate Least Absolute Deviation

As is known, the absolute deviation is not differentiable. To overcome this shortage, we choose a certain differentiable function to approximate the LAD criterion. This requirement can be met by the following logarithmic function:(11)
J
x
=
β
ln
⁡
cosh
⁡
x
β
.In (11), β is an adjustable parameter, and the nonlinear function J
(
∗
) is related to β. While β is small enough, nonlinear function cosh
⁡
(
x
/
β
)
≈
1
/
2
e
|
x
/
β
|, and β
ln
⁡
[
cosh
⁡
(
x
/
β
)
]
≈
|
x
|; that is, J
(
x
)
≈
|
x
|, approximating the absolute value function. The property curve of the logarithmic function when β
=
0.01 is shown in Figure 2.Figure 2
The curve of function property whenβ
=
0.01.The curve shows that whileβ is small enough, nonlinear function J
(
x
)
=
β
ln
⁡
[
cosh
⁡
(
x
/
β
)
] can approximate the absolute value function x effectively. Because J
(
x
) is differentiable, then the identification algorithm based on J
(
x
) can be solved by some optimization methods.


## 3.2. Algorithm Derivation

In (10), the input u
(
k
) and output y
(
k
) are measurement data. According to the known structure of the nonlinear function f
(
∗
), we can get the value of f
(
u
(
k
)
). But v
(
k
-
i
) in φ
(
k
) is white noise and immeasurable, which makes it impossible to solve the identification problem. Based on the idea that the noise can be replaced by its estimated value, we use v
^
(
k
-
i
), the noise residual in the ith step, to replace v
(
k
-
i
) in the i
+
1th step, so φ
(
k
) can be replaced by its estimated value φ
^
(
k
):(12)
φ
^
k
=
-
y
k
-
1
,
-
y
k
-
2
,
…
,
-
y
k
-
n
a
,
v
^
k
-
1
,
v
^
k
-
2
,
…
,
v
^
k
-
n
d
,
ψ
k
T
.Then, the identification of θ in (10) is to find θ that minimizes the following criterion function (ALAD) based on input data φ
^
(
k
) and output data y
(
k
):(13)
J
θ
θ
^
k
=
β
ln
⁡
cosh
⁡
y
k
-
φ
T
k
θ
β
θ
^
k
=
m
i
n
.For (13), it is a typical nonlinear function optimization. To reduce the computation complexity, we use the stochastic gradient method to derive the recursive equation of θ
^
k [18, 19]. The formula of the stochastic gradient method is(14)
θ
^
k
=
θ
^
k
-
1
-
μ
k
grad
⁡
J
θ
θ
^
k
-
1
,where grad
⁡
∗ represents gradient and μ
k is step size. The formula above shows that θ
^
k is corrected along the negative gradient direction until the extreme value of J
(
θ
) is obtained.Firstly, we get the gradient ofJ
(
θ
):(15)
grad
⁡
J
θ
θ
^
k
-
1
=
∂
J
∂
θ
θ
^
k
-
1
=
-
φ
^
k
tanh
⁡
y
k
-
φ
^
T
k
θ
^
k
-
1
β
;then, from (15) and (13), we obtain(16)
J
θ
θ
^
k
-
1
=
β
ln
⁡
cosh
⁡
y
k
-
φ
^
T
k
θ
^
k
-
1
+
μ
k
φ
^
k
tanh
⁡
y
k
-
φ
^
k
θ
^
k
-
1
/
β
β
=
β
ln
⁡
cosh
⁡
1
β
e
^
k
-
μ
k
φ
^
T
k
φ
^
k
tanh
⁡
e
^
k
β
,where e
^
(
k
)
=
y
(
k
)
-
φ
^
T
(
k
)
θ
^
(
k
-
1
).We can get the gradient ofJ
(
θ
) to μ
k from (16):(17)
∂
J
θ
∂
μ
k
=
-
K
k
∗
tanh
⁡
e
^
k
-
μ
k
K
k
β
,where(18)
K
k
=
φ
^
T
k
φ
^
k
tanh
⁡
e
^
k
β
.Equation (17) is obtained based on the exact line search algorithm and always leads to the two adjacent gradient search directions being orthogonal and usually results in zigzag phenomenon. When we use (17) to compute the optimal μ
k, the convergence rate is slow because of the linear convergence order, and the identification results of the parameters may be fluctuated. For this shortcoming, an improved stochastic gradient method is applied to get μ
k where μ
k is based on the information of gradient that is k times before the current iteration step; that is, (19)
K
k
=
∑
l
=
1
k
φ
^
T
l
φ
^
l
tanh
⁡
e
^
l
β
.From e
^
(
k
)
-
μ
k
K
k
=
0, it is obtained that(20)
μ
k
=
e
^
k
K
k
=
e
^
k
∑
l
=
1
k
φ
^
T
l
φ
^
l
tanh
⁡
e
^
l
/
β
.From (14), (15), and (19), it is obtained that (21)
θ
^
k
=
θ
^
k
-
1
+
e
^
k
∑
l
=
1
k
φ
^
T
l
φ
^
l
tanh
⁡
e
^
l
/
β
φ
^
k
tanh
⁡
e
^
k
β
;then(22)
θ
^
k
=
θ
^
k
-
1
+
φ
^
k
r
k
tanh
⁡
e
^
k
β
,where (23)
r
k
=
1
μ
k
=
∑
l
=
1
k
φ
^
T
k
φ
^
k
tanh
⁡
e
^
k
/
β
e
^
k
.r
(
k
) is computed by the following recursive formula:(24)
r
k
=
λ
k
r
k
-
1
+
1
e
^
k
φ
^
T
k
φ
^
k
tanh
⁡
e
^
k
β
,where λ
(
k
)
=
e
^
(
k
-
1
)
/
e
^
(
k
).Equation (24) is equivalent for adding an inertial term to the recursive equation of μ
(
k
). The weights of the inertial term are related to the iterative output error at k
-
1 and k step.In conclusion, we can obtain the extended stochastic gradient identification method based on approximate least absolute deviation (ALADESG):(25)
θ
^
k
=
θ
^
k
-
1
+
φ
^
k
r
k
tanh
⁡
e
^
k
β
,
r
k
=
λ
k
r
k
-
1
+
1
e
^
k
φ
^
T
k
φ
^
k
tanh
⁡
e
^
k
β
,
e
^
k
=
y
k
-
φ
^
T
k
θ
^
k
-
1
,
φ
^
k
=
-
y
k
-
1
,
-
y
k
-
2
,
…
,
-
y
k
-
n
a
,
v
^
k
-
1
,
v
^
k
-
2
,
…
,
v
^
k
-
n
d
,
ψ
k
T
,
ψ
k
=
f
1
u
k
-
1
,
f
1
u
k
-
2
,
…
,
f
1
u
k
-
n
b
,
f
2
u
k
-
1
,
f
2
u
k
-
2
,
…
,
f
2
u
k
-
n
b
,
…
,
f
n
c
u
k
-
1
,
f
n
c
u
k
-
2
,
…
,
f
n
c
u
k
-
n
b
.


## 3.3. Analysis of Convergence

Rewrite (22) as(26)
θ
^
k
=
θ
^
k
-
1
+
c
k
φ
^
k
e
^
k
,where(27)
c
k
=
tanh
⁡
e
^
k
/
β
∑
l
=
1
k
φ
^
T
l
φ
^
l
tanh
⁡
e
^
l
/
β
.If the truth values of process parameters are noted as θ
0, then y
(
k
)
=
φ
^
T
(
k
)
θ
0.Definingθ
~
(
k
)
=
θ
0
-
θ
^
(
k
), according to (26), we have(28)
θ
^
k
=
θ
^
k
-
1
+
c
k
φ
^
k
y
k
-
φ
^
T
k
θ
^
k
-
1
;subtracting θ
0 on both sides of (28)(29)
-
θ
~
k
=
-
θ
~
k
-
1
+
c
k
φ
^
k
φ
^
T
k
θ
~
k
-
1then(30)
θ
~
k
=
I
-
c
k
φ
^
k
φ
^
T
k
θ
~
k
-
1
.Constructing the Lyapunov function(31)
V
θ
~
k
,
k
=
θ
~
T
k
θ
~
k
>
0then(32)
Δ
V
θ
~
k
,
k
=
V
θ
~
k
,
k
-
V
θ
~
k
-
1
,
k
-
1
=
θ
~
T
k
-
1
G
k
θ
~
k
-
1
,where(33)
G
k
=
I
-
c
k
φ
^
k
φ
^
T
k
T
I
-
c
k
φ
^
k
φ
^
T
k
-
I
=
φ
^
T
k
φ
^
k
c
2
k
-
2
c
k
φ
^
k
φ
^
T
k
.
G
(
k
) is a real symmetric matrix, and the rank of r
G
=
1. Obviously, when the following inequality in (34) is fulfilled, the matrix G
(
k
) is negative and semidefinite, and then (28) will be asymptotically stable in a large scope at the balance point θ
~
(
k
)
=
0:(34)
c
2
k
φ
^
T
k
φ
^
k
-
2
c
k
≤
0
.According to (27), (35)
c
2
k
φ
^
T
k
φ
^
k
-
2
c
k
=
t
a
n
h
2
e
^
k
/
β
φ
^
T
k
φ
^
k
∑
l
=
1
k
φ
^
T
l
φ
^
l
tanh
⁡
e
^
l
/
β
2
-
2
tanh
⁡
e
^
k
/
β
∑
l
=
1
k
φ
^
T
l
φ
^
l
tanh
⁡
e
^
l
/
β
=
t
a
n
h
2
e
^
k
/
β
φ
^
T
k
φ
^
k
∑
l
=
1
k
φ
^
T
l
φ
^
l
tanh
⁡
e
^
l
/
β
2
-
2
tanh
⁡
e
^
k
/
β
∑
l
=
1
k
φ
^
T
l
φ
^
l
tanh
⁡
e
^
l
/
β
∑
l
=
1
k
φ
^
T
l
φ
^
l
tanh
⁡
e
^
l
/
β
2
=
-
t
a
n
h
2
e
^
k
/
β
∑
j
=
1
n
φ
j
2
k
-
2
tanh
⁡
e
^
k
/
β
∑
l
=
1
k
-
1
φ
^
T
l
φ
^
l
tanh
⁡
e
^
l
/
β
∑
l
=
1
k
φ
^
T
l
φ
^
l
tanh
⁡
e
^
l
/
β
2
.The second term of (35) can be written as(36)
-
2
tanh
⁡
e
^
k
/
β
r
k
-
1
e
k
-
1
r
2
k
e
2
k
=
-
2
tanh
⁡
e
^
k
/
β
e
k
1
r
2
k
e
k
-
1
r
k
-
1
e
k
,where (37)
tanh
⁡
e
^
k
/
β
e
k
>
0
,
1
r
2
k
>
0
.During the iterative process, if we could make sure that(38)
e
k
-
1
r
k
-
1
e
k
≥
0then we get (39)
c
2
k
φ
^
T
k
φ
^
k
-
2
c
k
≤
0
.Therefore, in order to guarantee the convergence of the proposed algorithm, we need to add (38) as a constraint when (24) is used for iteration.


## 3.4. Separation of Parameters by Average Method

It is noteworthy that in the description of the Hammerstein model (as shown in Figure1), for some nonzero and finite constant a, any a
f
(
u
) and G
(
z
)
/
a can match and yield identical input and output measurements; thus, the nonlinear function f
(
u
) and the transfer function G
(
z
) are not unique. In other words, any identification scheme will not be able to distinguish between (
f
(
u
)
,
G
(
z
)
) and (
a
f
(
u
)
,
G
(
z
)
/
a
). Therefore, without any loss of generality, one of the gains of f
(
u
) and G
(
z
) has to be fixed. There are several ways to normalize these gains [20, 21]. We adopt the following method.Assumption 1.
The first coefficient of the functionf
(
·
) equals 1 [22]; that is to say, in (1), c
1
=
1.
Based on the relationship between the parameters,c
^ can be computed using(40)
c
^
j
i
k
=
θ
^
n
a
+
n
b
+
i
k
b
^
i
k
,
j
=
2,3
,
…
,
n
c
,
i
=
1,2
,
…
,
n
b
.From (40), we can see that the number of c
^
j
i
(
k
) is n
b, and c
^
j
i
(
k
) is the estimated value of c
j
(
k
). In order to obtain the estimated value of the parameters with much higher accuracy, we use the average method below to estimate c
^:(41)
c
^
j
k
=
1
n
b
∑
i
=
1
n
b
c
^
j
i
k
=
1
n
b
∑
i
=
1
n
b
θ
^
n
a
+
n
b
+
i
k
b
^
i
k
,
j
=
2,3
,
…
,
n
c
.


## 4. Simulation Results and Discussions

Consider the following Hammerstein model:(42)
A
z
y
k
=
B
z
x
k
+
D
z
v
k
,where B
(
z
-
1
)
=
b
1
z
-
1
+
b
2
z
-
2
=
0.85
z
-
1
+
0.65
z
-
2; D
(
z
-
1
)
=
1
+
d
1
z
-
1
=
1
-
0.64
z
-
1; A
(
z
-
1
)
=
1
+
a
1
z
-
1
+
a
2
z
-
2
=
1
-
1.60
z
-
1
+
0.80
z
-
2. The nonlinear static part can be described as the polynomial below:(43)
x
k
=
f
u
k
=
c
1
u
k
+
c
2
u
2
k
+
c
3
u
3
k
=
u
k
+
0.5
u
2
k
+
0.25
u
3
k
,where the input sequence u
(
k
) is a series of uncorrelated, continuous, and stochastic sequences with zero mean and unit variance σ
u
2
=
1. The noise sequence v
(
k
) is a group of stochastic white noise sequences with zero mean and variance σ
v
2
=
0.5. The input and output data for the identification procedure are obtained from this model. We then apply proposed algorithm (25) to estimate the parameters of this system at different situation. The identification process and result analysis are demonstrated in the following. The simulation experiments are performed in Matlab. The value of β is taken as 0.01.The relative error of the parameters estimation is used as the evaluation criteria of the algorithm which is defined as follows:(44)
δ
=
θ
^
k
-
θ
θ
.In order to verify the performance of the proposed algorithm, we designed two simulation cases.Case 1.
Only colored noise exists in the measured data. This case is used to prove the impact of inertial term on the identification results of ALADESG method.Case 2.
Not only colored noise but also impulse noise exists in the measured data. This case is used to compare the identification effects of ALADESG method with that of LSESG method when measured data are contaminated by impulse noise. The robustness of ALADESG method is also verified when impulse noise with different amplitude is added.The impulse noise in the experiment is subject toS
α
S distribution [23], and the probability density function of the standard S
α
S distribution is(45)
f
α
x
=
1
π
x
∑
k
=
1
∞
-
1
k
-
1
k
!
Γ
α
k
+
1
x
-
α
k
sin
⁡
k
α
π
2
,
0
<
α
<
1
,
1
π
x
∑
k
=
1
∞
-
1
k
2
k
!
Γ
2
k
+
1
α
x
2
k
,
1
<
α
≤
2
,where Γ
∗ is Γ gamma function and α is characteristics exponent. The less α is, the higher chance of large amplitude sample of the random variable which is subject to S
α
S distribution takes place, and the stronger pulse strength is.The way to get a random variable which obeys the standardS
α
S distribution follows the steps below [24]:(1)
Get a random variableU subject to uniform distribution on -
π
/
2
,
π
/
2.
(2)
Get a random variableE subject to exponential distribution and mean value is 1.
(3)
Get the variable subject to standardS
α
S distribution by the following formula:(46)
Y
=
sin
⁡
α
U
cos
⁡
U
1
/
α
cos
⁡
1
-
α
U
E
1
-
α
/
α
,where U and E are independent.


### 4.1. The Effect of Inertial Termλ
(
k
) on the Identification Results

(1) Letλ
(
k
)
=
0; namely, μ
k is computed by (17) with current gradient information only. The curve of the relative error is shown in Figure 3.Figure 3
Relative error curve whenλ
k
=
0.According to theoretical analysis, there are some drawbacks when (17) is used to compute the optimal search step size, such as linear convergence order, slow convergence rate, and fluctuant estimation of parameters. As shown in Figure 3, the curve of the relative error of the parameters of Hammerstein model fluctuates severely, and the estimated parameters cannot converge to the true value. That is to say, the method cannot provide a reliable parameter estimation for the identification of Hammerstein model when λ
(
k
)
=
0.(2) In order to improve the accuracy and convergence rate and to enhance the reliability of the identification results, we add an inertial termλ
(
k
)
=
e
^
(
k
-
1
)
/
e
^
(
k
) to the method, namely, applying (24) and (38) to parameter identification. The results are shown in Table 1.Table 1
Identification results whenλ
k
=
e
^
(
k
-
1
)
/
e
^
(
k
).
Parameters
a
1
a
2
b
1
b
2
c
2
c
3
d
1
δ (%)
Estimated value
−1.5937
0.7943
0.8173
0.6895
0.5062
0.2533
−0.5452
6.8363
True value
−1.600
0.800
0.850
0.650
0.500
0.250
−0.640
0Obviously, whenλ
(
k
)
=
e
^
(
k
-
1
)
/
e
^
(
k
), the deviations of the estimated values and true values of model parameters are small, and the identification accuracy is much better than that of λ
(
k
)
=
0. This shows that the inertial term coming from the improved stochastic gradient method enhances the identification accuracy effectively and insures the convergence of the ALADESG method. The relative error curve is shown in Figure 4.Figure 4
Relative error curve whenλ
k
=
e
^
(
k
-
1
)
/
e
^
(
k
).Comparing Figure3 with Figure 4 shows that the relative error curve in Figure 4 is much smoother, the estimated parameters converge to those true values steadily, and the relative error is smaller than the situation as shown in Figure 3. Simulation results in Figures 3 and 4 show that the inertial term can greatly improve the identification accuracy and significantly enhance the reliability of the parameter estimation.


### 4.2. Comparison of ALADESG and LSESG

#### 4.2.1. Considering the Measured Data with Colored Noise Only

According to the same idea in Section3.2, we can get the iterative equations of LSESG also containing a similar inertial term. Then the proposed ALADESG method was compared with the LSESG method when the measured data are contaminated with colored noise. The simulation result is shown in Figure 5.Figure 5
Relative error with colored noise.As shown in Figure5, the identification accuracies of the ALADESG method and the LSESG method are both acceptable when the measured data are only disturbed by colored noise, but the identification accuracy of the LSESG method is higher and the convergence rate is faster. So when the measured data is contaminated with colored noise only, the identification performance of the LSESG method is better than the ALADESG method.


#### 4.2.2. The Measured Data with Impulse Noise Subject toS
α
S Distribution

The measured data are contaminated with both colored noise and impulse noise. Let the characteristics exponent of the impulse noise be 1.5. The time-domain waveform of the impulse noise is shown in Figure6. Then the measured data are identified by the ALADESG method and the LSESG method, respectively. The relative errors are shown in Figure 7.Figure 6
The time-domain waveform of the impulse noise whenα
=
1.5.Figure 7
Relative error with impulse noise ofα
=
1.5.Figure7 shows that the identification accuracy of the ALADESG method is higher than that of the LSESG method when the measured data are disturbed by colored noise and impulse noise with α
=
1.5. From Figures 6 and 7, we notice that the relative error of the LSESG method will fluctuate severely at the moment when a large impulse is added, and the larger the amplitude of the impulse is, the severer the fluctuation is, which makes the parameter estimation unable to converge steadily. In the LSESG method, the objective function is the square of the error so the influence of impulse noise would be amplified especially when the impulse noise is large. In contrast, the ALADESG method is based on the ALAD criterion which restrains the influence of impulse noise and enhances the robustness, the parameter estimation converges to the true value quickly and steadily, and the identification accuracy is also improved. The simulation results indicate that the ALADESG method has a better identification performance than LSESG method when impulse noise is added.


#### 4.2.3. The Influence of Different Pulse Amplitude on the Identification Results of ALADESG Method

In this part, we conduct three simulations withα
=
1.5, α
=
1.2, and α
=
0.9. The identification results of the ALADESG method are shown in Table 2, and the relative errors are shown in Figure 8. In addition, the identification results of the ALADESG method are compared with those of LSESG method when α
=
1.2 and α
=
0.9. The relative errors curves when α
=
1.2 and α
=
0.9 are shown in Figures 9 and 10, respectively.Table 2
Identification results with differentα.
Parameters
a
1
a
2
b
1
b
2
c
2
c
3
d
1
δ (%)
α
=
1.5
−1.5711
0.7832
0.8461
0.7040
0.4769
0.2501
−0.7541
10.2411
α
=
1.2
−1.5177
0.7147
0.8189
0.7741
0.4764
0.2580
−0.7038
11.4612
α
=
0.9
−1.4660
0.6617
0.8396
0.7817
0.4818
0.2515
−0.6394
15.1781
True value
−1.600
0.800
0.850
0.650
0.500
0.250
−0.640
0Figure 8
Relative error with impulse noise of different amplitude.Figure 9
Relative error with impulse noise ofα
=
1.2.Figure 10
Relative error with impulse noise ofα
=
0.9.From Table2 and Figure 8, the identification accuracy of the ALADESG method becomes worse when α decreases, but the decrease of accuracy is small. In contrast, from Figures 7, 9, and 10, the identification accuracy of the LSESG method becomes worse when α is decreased and the decrease of accuracy is very obvious. Particularly when α
=
1.2 and α
=
0.9, the LSESG method cannot converge indeed, which suggests that the LSESG method cannot identify the parameters of Hammerstein model in this situation. But high accuracy and stable identification results still can be achieved by the ALADESG method. Therefore, it can be concluded that the method based on ALAD criterion overcomes the impact of impulse noise very well and has better robustness to impulse noise.


### 4.3. Identification Results Analysis

According to the identification results shown in Sections4.1 and 4.2, the following conclusions are summarized.(1)
When the measured data are contaminated by colored noise, the Hammerstein model can be identified effectively by the proposed method.
(2)
By applying inertial termλ
(
k
) which contains the gradient information of past time, the identification accuracy and convergence rates are improved significantly.
(3)
The identification performance of the LSESG method is better than that of the ALADESG method when only colored noise exists.
(4)
The ALADESG method has better accuracy and convergent rate than LSESG method when both colored noise and impulse noise exist in measured data. Particularly when the amplitude of impulse noise is very large (such as whenα
=
0.9), the ALADESG method still can get an acceptable identification result, which proves the adaptability of the method to more serious noise. The robustness of the method is also confirmed by simulation results.


## 4.1. The Effect of Inertial Termλ
(
k
) on the Identification Results

(1) Letλ
(
k
)
=
0; namely, μ
k is computed by (17) with current gradient information only. The curve of the relative error is shown in Figure 3.Figure 3
Relative error curve whenλ
k
=
0.According to theoretical analysis, there are some drawbacks when (17) is used to compute the optimal search step size, such as linear convergence order, slow convergence rate, and fluctuant estimation of parameters. As shown in Figure 3, the curve of the relative error of the parameters of Hammerstein model fluctuates severely, and the estimated parameters cannot converge to the true value. That is to say, the method cannot provide a reliable parameter estimation for the identification of Hammerstein model when λ
(
k
)
=
0.(2) In order to improve the accuracy and convergence rate and to enhance the reliability of the identification results, we add an inertial termλ
(
k
)
=
e
^
(
k
-
1
)
/
e
^
(
k
) to the method, namely, applying (24) and (38) to parameter identification. The results are shown in Table 1.Table 1
Identification results whenλ
k
=
e
^
(
k
-
1
)
/
e
^
(
k
).
Parameters
a
1
a
2
b
1
b
2
c
2
c
3
d
1
δ (%)
Estimated value
−1.5937
0.7943
0.8173
0.6895
0.5062
0.2533
−0.5452
6.8363
True value
−1.600
0.800
0.850
0.650
0.500
0.250
−0.640
0Obviously, whenλ
(
k
)
=
e
^
(
k
-
1
)
/
e
^
(
k
), the deviations of the estimated values and true values of model parameters are small, and the identification accuracy is much better than that of λ
(
k
)
=
0. This shows that the inertial term coming from the improved stochastic gradient method enhances the identification accuracy effectively and insures the convergence of the ALADESG method. The relative error curve is shown in Figure 4.Figure 4
Relative error curve whenλ
k
=
e
^
(
k
-
1
)
/
e
^
(
k
).Comparing Figure3 with Figure 4 shows that the relative error curve in Figure 4 is much smoother, the estimated parameters converge to those true values steadily, and the relative error is smaller than the situation as shown in Figure 3. Simulation results in Figures 3 and 4 show that the inertial term can greatly improve the identification accuracy and significantly enhance the reliability of the parameter estimation.


## 4.2. Comparison of ALADESG and LSESG

### 4.2.1. Considering the Measured Data with Colored Noise Only

According to the same idea in Section3.2, we can get the iterative equations of LSESG also containing a similar inertial term. Then the proposed ALADESG method was compared with the LSESG method when the measured data are contaminated with colored noise. The simulation result is shown in Figure 5.Figure 5
Relative error with colored noise.As shown in Figure5, the identification accuracies of the ALADESG method and the LSESG method are both acceptable when the measured data are only disturbed by colored noise, but the identification accuracy of the LSESG method is higher and the convergence rate is faster. So when the measured data is contaminated with colored noise only, the identification performance of the LSESG method is better than the ALADESG method.


### 4.2.2. The Measured Data with Impulse Noise Subject toS
α
S Distribution

The measured data are contaminated with both colored noise and impulse noise. Let the characteristics exponent of the impulse noise be 1.5. The time-domain waveform of the impulse noise is shown in Figure6. Then the measured data are identified by the ALADESG method and the LSESG method, respectively. The relative errors are shown in Figure 7.Figure 6
The time-domain waveform of the impulse noise whenα
=
1.5.Figure 7
Relative error with impulse noise ofα
=
1.5.Figure7 shows that the identification accuracy of the ALADESG method is higher than that of the LSESG method when the measured data are disturbed by colored noise and impulse noise with α
=
1.5. From Figures 6 and 7, we notice that the relative error of the LSESG method will fluctuate severely at the moment when a large impulse is added, and the larger the amplitude of the impulse is, the severer the fluctuation is, which makes the parameter estimation unable to converge steadily. In the LSESG method, the objective function is the square of the error so the influence of impulse noise would be amplified especially when the impulse noise is large. In contrast, the ALADESG method is based on the ALAD criterion which restrains the influence of impulse noise and enhances the robustness, the parameter estimation converges to the true value quickly and steadily, and the identification accuracy is also improved. The simulation results indicate that the ALADESG method has a better identification performance than LSESG method when impulse noise is added.


### 4.2.3. The Influence of Different Pulse Amplitude on the Identification Results of ALADESG Method

In this part, we conduct three simulations withα
=
1.5, α
=
1.2, and α
=
0.9. The identification results of the ALADESG method are shown in Table 2, and the relative errors are shown in Figure 8. In addition, the identification results of the ALADESG method are compared with those of LSESG method when α
=
1.2 and α
=
0.9. The relative errors curves when α
=
1.2 and α
=
0.9 are shown in Figures 9 and 10, respectively.Table 2
Identification results with differentα.
Parameters
a
1
a
2
b
1
b
2
c
2
c
3
d
1
δ (%)
α
=
1.5
−1.5711
0.7832
0.8461
0.7040
0.4769
0.2501
−0.7541
10.2411
α
=
1.2
−1.5177
0.7147
0.8189
0.7741
0.4764
0.2580
−0.7038
11.4612
α
=
0.9
−1.4660
0.6617
0.8396
0.7817
0.4818
0.2515
−0.6394
15.1781
True value
−1.600
0.800
0.850
0.650
0.500
0.250
−0.640
0Figure 8
Relative error with impulse noise of different amplitude.Figure 9
Relative error with impulse noise ofα
=
1.2.Figure 10
Relative error with impulse noise ofα
=
0.9.From Table2 and Figure 8, the identification accuracy of the ALADESG method becomes worse when α decreases, but the decrease of accuracy is small. In contrast, from Figures 7, 9, and 10, the identification accuracy of the LSESG method becomes worse when α is decreased and the decrease of accuracy is very obvious. Particularly when α
=
1.2 and α
=
0.9, the LSESG method cannot converge indeed, which suggests that the LSESG method cannot identify the parameters of Hammerstein model in this situation. But high accuracy and stable identification results still can be achieved by the ALADESG method. Therefore, it can be concluded that the method based on ALAD criterion overcomes the impact of impulse noise very well and has better robustness to impulse noise.


## 4.2.1. Considering the Measured Data with Colored Noise Only

According to the same idea in Section3.2, we can get the iterative equations of LSESG also containing a similar inertial term. Then the proposed ALADESG method was compared with the LSESG method when the measured data are contaminated with colored noise. The simulation result is shown in Figure 5.Figure 5
Relative error with colored noise.As shown in Figure5, the identification accuracies of the ALADESG method and the LSESG method are both acceptable when the measured data are only disturbed by colored noise, but the identification accuracy of the LSESG method is higher and the convergence rate is faster. So when the measured data is contaminated with colored noise only, the identification performance of the LSESG method is better than the ALADESG method.


## 4.2.2. The Measured Data with Impulse Noise Subject toS
α
S Distribution

The measured data are contaminated with both colored noise and impulse noise. Let the characteristics exponent of the impulse noise be 1.5. The time-domain waveform of the impulse noise is shown in Figure6. Then the measured data are identified by the ALADESG method and the LSESG method, respectively. The relative errors are shown in Figure 7.Figure 6
The time-domain waveform of the impulse noise whenα
=
1.5.Figure 7
Relative error with impulse noise ofα
=
1.5.Figure7 shows that the identification accuracy of the ALADESG method is higher than that of the LSESG method when the measured data are disturbed by colored noise and impulse noise with α
=
1.5. From Figures 6 and 7, we notice that the relative error of the LSESG method will fluctuate severely at the moment when a large impulse is added, and the larger the amplitude of the impulse is, the severer the fluctuation is, which makes the parameter estimation unable to converge steadily. In the LSESG method, the objective function is the square of the error so the influence of impulse noise would be amplified especially when the impulse noise is large. In contrast, the ALADESG method is based on the ALAD criterion which restrains the influence of impulse noise and enhances the robustness, the parameter estimation converges to the true value quickly and steadily, and the identification accuracy is also improved. The simulation results indicate that the ALADESG method has a better identification performance than LSESG method when impulse noise is added.


## 4.2.3. The Influence of Different Pulse Amplitude on the Identification Results of ALADESG Method

In this part, we conduct three simulations withα
=
1.5, α
=
1.2, and α
=
0.9. The identification results of the ALADESG method are shown in Table 2, and the relative errors are shown in Figure 8. In addition, the identification results of the ALADESG method are compared with those of LSESG method when α
=
1.2 and α
=
0.9. The relative errors curves when α
=
1.2 and α
=
0.9 are shown in Figures 9 and 10, respectively.Table 2
Identification results with differentα.
Parameters
a
1
a
2
b
1
b
2
c
2
c
3
d
1
δ (%)
α
=
1.5
−1.5711
0.7832
0.8461
0.7040
0.4769
0.2501
−0.7541
10.2411
α
=
1.2
−1.5177
0.7147
0.8189
0.7741
0.4764
0.2580
−0.7038
11.4612
α
=
0.9
−1.4660
0.6617
0.8396
0.7817
0.4818
0.2515
−0.6394
15.1781
True value
−1.600
0.800
0.850
0.650
0.500
0.250
−0.640
0Figure 8
Relative error with impulse noise of different amplitude.Figure 9
Relative error with impulse noise ofα
=
1.2.Figure 10
Relative error with impulse noise ofα
=
0.9.From Table2 and Figure 8, the identification accuracy of the ALADESG method becomes worse when α decreases, but the decrease of accuracy is small. In contrast, from Figures 7, 9, and 10, the identification accuracy of the LSESG method becomes worse when α is decreased and the decrease of accuracy is very obvious. Particularly when α
=
1.2 and α
=
0.9, the LSESG method cannot converge indeed, which suggests that the LSESG method cannot identify the parameters of Hammerstein model in this situation. But high accuracy and stable identification results still can be achieved by the ALADESG method. Therefore, it can be concluded that the method based on ALAD criterion overcomes the impact of impulse noise very well and has better robustness to impulse noise.


## 4.3. Identification Results Analysis

According to the identification results shown in Sections4.1 and 4.2, the following conclusions are summarized.(1)
When the measured data are contaminated by colored noise, the Hammerstein model can be identified effectively by the proposed method.
(2)
By applying inertial termλ
(
k
) which contains the gradient information of past time, the identification accuracy and convergence rates are improved significantly.
(3)
The identification performance of the LSESG method is better than that of the ALADESG method when only colored noise exists.
(4)
The ALADESG method has better accuracy and convergent rate than LSESG method when both colored noise and impulse noise exist in measured data. Particularly when the amplitude of impulse noise is very large (such as whenα
=
0.9), the ALADESG method still can get an acceptable identification result, which proves the adaptability of the method to more serious noise. The robustness of the method is also confirmed by simulation results.


## 5. Conclusions

In this paper, we expand the application of the LAD technique to the field of nonlinear identification. A new algorithm is proposed based on ALAD criterion and improved stochastic gradient search algorithm for the Hammerstein model, and the convergence is proven by Lyapunov stability theory. With the application of ALAD criterion, the proposed method overcomes the influence of impulse noise and colored noise on the identification results and improves the robustness to a large extent. The proposed algorithm is easy to implement. The simulation results show that the proposed algorithm yields better robustness against impulse noises and a faster convergence rate than that by the LSESG method.



---

*Source: 9548428-2016-06-13.xml*