# Least-Squares Parameter Estimation Algorithm for a Class of Input Nonlinear Systems

Authors: Weili Xiong; Wei Fan; Rui Ding
Journal: Journal of Applied Mathematics
 (2012)

---

## Abstract

This paper studies least-squares parameter estimation algorithms for input nonlinear systems, including the input nonlinear controlled autoregressive (IN-CAR) model and the input nonlinear controlled autoregressive autoregressive moving average (IN-CARARMA) model. The basic idea is to obtain linear-in-parameters models by overparameterizing such nonlinear systems and to use the least-squares algorithm to estimate the unknown parameter vectors. It is proved that the parameter estimates consistently converge to their true values under the persistent excitation condition. A simulation example is provided.


---

## Body

## 1. Introduction

Parameter estimation has received much attention in many areas such as linear and nonlinear system identification and signal processing [1–9]. Nonlinear systems can be simply divided into the input nonlinear systems, the output nonlinear systems, the feedback nonlinear systems, and the input and output nonlinear systems, and so forth. The Hammerstein models can describe a class of input nonlinear systems which consist of static nonlinear blocks followed by linear dynamical subsystems [10, 11].Nonlinear systems are common in industrial processes, for example, the dead-zone nonlinearities and the valve saturation nonlinearities. Many estimation methods have been developed to identify the parameters of nonlinear systems, especially for Hammerstein nonlinear systems [12, 13]. For example, Ding et al. presented a least-squares-based iterative algorithm and a recursive extended least squares algorithm for Hammerstein ARMAX systems [14] and an auxiliary model-based recursive least squares algorithm for Hammerstein output error systems [15]. Wang and Ding proposed an extended stochastic gradient identification algorithm for Hammerstein-Wiener ARMAX Systems [16].Recently, Wang et al. derived an auxiliary model-based recursive generalized least-squares parameter estimation algorithm for Hammerstein output error autoregressive systems and auxiliary model-based RELS and MI-ELS algorithms for Hammerstein output error moving average systems using the key term separation principle [17, 18]. Ding et al. presented a projection estimation algorithm and a stochastic gradient (SG) estimation algorithm for Hammerstein nonlinear systems by using the gradient search and further derived a Newton recursive estimation algorithm and a Newton iterative estimation algorithm by using the Newton method (Newton-Raphson method) [19]. Wang and Ding studied least-squares-based and gradient-based iterative identification methods for Wiener nonlinear systems [20].Fan et al. discussed the parameter estimation problem for Hammerstein nonlinear ARX models [21]. On the basis of the work in [14, 15, 21], this paper studies the identification problems and their convergence for input nonlinear controlled autoregressive (IN-CAR) models using the martingale convergence theorem and gives the recursive generalized extended least-squares algorithm for input nonlinear controlled autoregressive autoregressive moving average (IN-CARARMA) models.Briefly, the paper is organized as follows. Section2 derives a linear-in-parameters identification model and gives a recursive least squares identification algorithm for input nonlinear CAR systems and analyzes the properties of the proposed algorithm. Section 4 gives the recursive generalized extended least squares algorithm for input nonlinear CARARMA systems. Section 5 provides an illustrative example to show the effectiveness of the proposed algorithms. Finally, we offer some concluding remarks in Section 6.


## 2. The Input Nonlinear CAR Model and Estimation Algorithm

Let us introduce some notations first. The symbolI(In) stands for an identity matrix of appropriate sizes (n×n); the superscript T denotes the matrix transpose; 1n represents an n-dimensional column vector whose elements are 1; |X|=det[X] represents the determinant of the matrix X; the norm of a matrix X is defined by ∥X∥2=tr[XXT]; λmax[X] and λmin[X] represent the maximum and minimum eigenvalues of the square matrix X, respectively; f(t)=o(g(t)) represents f(t)/g(t)→0 as t→∞; for g(t)⩾0, we write f(t)=O(g(t)) if there exists a positive constant δ1 such that |f(t)|⩽δ1g(t).


### 2.1. The Input Nonlinear CAR Model

Consider the following input nonlinear controlled autoregressive (IN-CAR) systems [14, 21]:
(2.1)A(z)y(t)=B(z)u-(t)+v(t),
where y(t) is the system output, v(t) is a disturbance noise, the output of the nonlinear block u-(t) is a nonlinear function of a known basis (f1,f2,…,fm) of the system input u(t) [19],
(2.2)u-(t)=f(u(t))=c1f1(u(t))+c2f2(u(t))+⋯+cmfm(u(t)),A(z) and B(z) are polynomials in the unit backward shift operator z-1 [z-1y(t)=y(t-1)], defined as
(2.3)A(z):=1+a1z-1+a2z-2+⋯+anz-n,B(z):=b1z-1+b2z-2+b3z-3+⋯+bnz-n.
In order to obtain the identifiability of parameters bi and ci, without loss of generality, we suppose that c1=1 or b1=1 [14, 21].Define the parameter vectorϑ and information vector ψ(t) as
(2.4)ϑ:=[aT,c1bT,c2bT,…,cmbT]T∈Rn0,n0:=n+mn,a:=[a1,a2,…,an]T∈Rn,b≔[b1,b2,…,bn]T∈Rn,ψ(t):=[ψ0T(t),ψ1T(t),ψ2T(t),…,ψmT(t)]T∈Rn0,ψ0(t):=[-y(t-1),-y(t-2),…,-y(t-n)]T∈Rn,ψj(t):=[fj(u(t-1)),fj(u(t-2)),…,fj(u(t-n))]T∈Rn,j=1,2,…,m.
From (2.1), we have
(2.5)y(t)=[1-A(z)]y(t)+B(z)u-(t)+v(t)=-∑i=1n‍aiy(t-i)+∑i=1n‍bi∑j=1m‍cjfj(u(t-i))+v(t)=-∑i=1n‍aiy(t-i)+∑j=1m‍∑i=1n‍cjbifj(u(t-i))+v(t)(2.6)=-∑i=1n‍aiy(t-i)+c1b1f1(u(t-1))+c1b2f1(u(t-2))+⋯+c1bnf1(u(t-n))=+c2b1f2(u(t-1))+c2b2f2(u(t-2))+⋯+c2bnf2(u(t-n))+⋯=+cmb1fm(u(t-1))+cmb2fm(u(t-2))+⋯+cmbnfm(u(t-n))+v(t)=ψT(t)ϑ+v(t).
An alternative way is to define the parameter vector θ and information vector φ(t) as
(2.7)θ:=[aT,b1cT,b2cT,…T,bncT]T∈Rn0,a:=[a1,a2,…,an]T∈Rn,c:=[c1,c2,…,cm]T∈Rm,φ(t):=[φ0T(t),φ1T(t),φ2T(t),…,φnT(t)]T∈Rn0,φ0(t):=[-y(t-1),-y(t-2),…,-y(t-n)]T∈Rn,φj(t):=[f1(u(t-j)),f2(u(t-j)),…,fm(u(t-j))]T∈Rm,j=1,2,…,n.
Then (2.5) can be written as
(2.8)y(t)=-∑i=1n‍aiy(t-i)+∑i=1n‍∑j=1m‍bicjfj(u(t-i))=-∑i=1n‍aiy(t-i)+b1c1f1u(t-1)+b1c2f2u(t-1)+⋯+b1cmfmu(t-1)=+b2c1f1u(t-2)+b2c2f2u(t-2)+⋯+b2cmfmu(t-2)+⋯=+bnc1f1u(t-n)+bnc2f2u(t-n)+⋯+bncmfmu(t-n)+v(t)=φT(t)θ+v(t).
Equations (2.6) and (2.8) are both linear-in-parameters identification model for Hammerstein CAR systems by using parametrization.


### 2.2. The Recursive Least Squares Algorithm

Minimizing the cost function(2.9)J(θ):=∑j=1t‍[y(j)-φT(j)θ]2
gives the following recursive least squares algorithm for computing the estimate θ^(t) of θ in (2.8):
(2.10)θ^(t)=θ^(t-1)+P(t)φ(t)[y(t)-φ(t)θ^(t-1)],(2.11)P-1(t)=P-1(t-1)+φ(t)φT(t),P(0)=p0I.
Applying the matrix inversion formula [22]
(2.12)(A+BC)-1=A-1-A-1B(I+CA-1B)-1CA-1
to (2.11) and defining the gain vector L(t)=P(t)φ(t)∈ℝn0, the algorithm in (2.10)-(2.11) can be equivalently expressed as
(2.13)θ^(t)=θ^(t-1)+L(t)[y(t)-φ(t)θ^(t-1)],L(t)=P(t)φ(t)=P(t-1)φ(t)1+φT(t)P(t-1)φ(t),P(t)=P(t-1)-P(t)φ(t)φT(t)P(t)1+φT(t)P(t-1)φ(t)=[I-L(t)φT(t)]P(t-1),P(0)=p0I.
To initialize the algorithm, we take p0 to be a large positive real number, for example, p0=106, and θ^(0) to be some small real vector, for example, θ^(0)=10-61n0.


## 2.1. The Input Nonlinear CAR Model

Consider the following input nonlinear controlled autoregressive (IN-CAR) systems [14, 21]:
(2.1)A(z)y(t)=B(z)u-(t)+v(t),
where y(t) is the system output, v(t) is a disturbance noise, the output of the nonlinear block u-(t) is a nonlinear function of a known basis (f1,f2,…,fm) of the system input u(t) [19],
(2.2)u-(t)=f(u(t))=c1f1(u(t))+c2f2(u(t))+⋯+cmfm(u(t)),A(z) and B(z) are polynomials in the unit backward shift operator z-1 [z-1y(t)=y(t-1)], defined as
(2.3)A(z):=1+a1z-1+a2z-2+⋯+anz-n,B(z):=b1z-1+b2z-2+b3z-3+⋯+bnz-n.
In order to obtain the identifiability of parameters bi and ci, without loss of generality, we suppose that c1=1 or b1=1 [14, 21].Define the parameter vectorϑ and information vector ψ(t) as
(2.4)ϑ:=[aT,c1bT,c2bT,…,cmbT]T∈Rn0,n0:=n+mn,a:=[a1,a2,…,an]T∈Rn,b≔[b1,b2,…,bn]T∈Rn,ψ(t):=[ψ0T(t),ψ1T(t),ψ2T(t),…,ψmT(t)]T∈Rn0,ψ0(t):=[-y(t-1),-y(t-2),…,-y(t-n)]T∈Rn,ψj(t):=[fj(u(t-1)),fj(u(t-2)),…,fj(u(t-n))]T∈Rn,j=1,2,…,m.
From (2.1), we have
(2.5)y(t)=[1-A(z)]y(t)+B(z)u-(t)+v(t)=-∑i=1n‍aiy(t-i)+∑i=1n‍bi∑j=1m‍cjfj(u(t-i))+v(t)=-∑i=1n‍aiy(t-i)+∑j=1m‍∑i=1n‍cjbifj(u(t-i))+v(t)(2.6)=-∑i=1n‍aiy(t-i)+c1b1f1(u(t-1))+c1b2f1(u(t-2))+⋯+c1bnf1(u(t-n))=+c2b1f2(u(t-1))+c2b2f2(u(t-2))+⋯+c2bnf2(u(t-n))+⋯=+cmb1fm(u(t-1))+cmb2fm(u(t-2))+⋯+cmbnfm(u(t-n))+v(t)=ψT(t)ϑ+v(t).
An alternative way is to define the parameter vector θ and information vector φ(t) as
(2.7)θ:=[aT,b1cT,b2cT,…T,bncT]T∈Rn0,a:=[a1,a2,…,an]T∈Rn,c:=[c1,c2,…,cm]T∈Rm,φ(t):=[φ0T(t),φ1T(t),φ2T(t),…,φnT(t)]T∈Rn0,φ0(t):=[-y(t-1),-y(t-2),…,-y(t-n)]T∈Rn,φj(t):=[f1(u(t-j)),f2(u(t-j)),…,fm(u(t-j))]T∈Rm,j=1,2,…,n.
Then (2.5) can be written as
(2.8)y(t)=-∑i=1n‍aiy(t-i)+∑i=1n‍∑j=1m‍bicjfj(u(t-i))=-∑i=1n‍aiy(t-i)+b1c1f1u(t-1)+b1c2f2u(t-1)+⋯+b1cmfmu(t-1)=+b2c1f1u(t-2)+b2c2f2u(t-2)+⋯+b2cmfmu(t-2)+⋯=+bnc1f1u(t-n)+bnc2f2u(t-n)+⋯+bncmfmu(t-n)+v(t)=φT(t)θ+v(t).
Equations (2.6) and (2.8) are both linear-in-parameters identification model for Hammerstein CAR systems by using parametrization.


## 2.2. The Recursive Least Squares Algorithm

Minimizing the cost function(2.9)J(θ):=∑j=1t‍[y(j)-φT(j)θ]2
gives the following recursive least squares algorithm for computing the estimate θ^(t) of θ in (2.8):
(2.10)θ^(t)=θ^(t-1)+P(t)φ(t)[y(t)-φ(t)θ^(t-1)],(2.11)P-1(t)=P-1(t-1)+φ(t)φT(t),P(0)=p0I.
Applying the matrix inversion formula [22]
(2.12)(A+BC)-1=A-1-A-1B(I+CA-1B)-1CA-1
to (2.11) and defining the gain vector L(t)=P(t)φ(t)∈ℝn0, the algorithm in (2.10)-(2.11) can be equivalently expressed as
(2.13)θ^(t)=θ^(t-1)+L(t)[y(t)-φ(t)θ^(t-1)],L(t)=P(t)φ(t)=P(t-1)φ(t)1+φT(t)P(t-1)φ(t),P(t)=P(t-1)-P(t)φ(t)φT(t)P(t)1+φT(t)P(t-1)φ(t)=[I-L(t)φT(t)]P(t-1),P(0)=p0I.
To initialize the algorithm, we take p0 to be a large positive real number, for example, p0=106, and θ^(0) to be some small real vector, for example, θ^(0)=10-61n0.


## 3. The Main Convergence Theorem

The following lemmas are required to establish the main convergence results.Lemma 3.1 (Martingale convergence theorem: Lemma D.5.3 in [23, 24]).
IfTt, αt, βt are nonnegative random variables, measurable with respect to a nondecreasing sequence of σ algebra ℱt-1, and satisfy
(3.1)E[Tt∣Ft-1]⩽Tt-1+αt-βt,a.s.,
then when ∑t=1∞‍αt<∞, one has ∑t=1∞‍βt<∞,a.s.and Tt→T,a.s. (a.s.: almost surely) a finite nonnegative random variable.Lemma 3.2 (see [14, 21, 25]).
For the algorithm in (2.10)-(2.11), for any γ>1, the covariance matrix P(t) in (2.11) satisfies the following inequality:
(3.2)∑t=1∞‍φT(t)P(t)φ(t)[ln⁡|P-1(t)|]γ<∞,a.s.Theorem 3.3.
For the system in (2.8) and the algorithm in (2.10)-(2.11), assume that {v(t),ℱt} is a martingale difference sequence defined on a probability space {Ω,ℱ,P}, where {ℱt} is the σ algebra sequence generated by the observations {y(t),y(t-1),…,u(t),u(t-1),…} and the noise sequence {v(t)} satisfies E[v(t)∣ℱt-1]=0,and E[v2(t)∣ℱt-1]⩽σ2<∞,a.s [23], and [ln|P-1(t)|]γ=o(λmin[P-1(t)]), γ>1. Then the parameter estimation error θ~(t) converges to zero.Proof.
Define the parameter estimation error vectorθ~(t):=θ^(t)-θ and the stochastic Lyapunov function T(t):=θ~T(t)P-1(t)θ~(t). Let y~(t):=φT(t)θ^(t-1)-φT(t)θ=φT(t)θ~(t-1). According to the definitions of θ~(t) and T(t) and using (2.10) and (2.11), we have
(3.3)θ~(t)=θ~(t-1)+P(t)φ(t)[-y~(t)+v(t)],T(t)=T(t-1)-[1-φT(t)P(t)φ(t)]y~2(t)+φT(t)P(t)φ(t)v2(t)+2[1-φT(t)P(t)φ(t)]y~(t)v(t)⩽T(t-1)+φT(t)P(t)φ(t)v2(t)+2[1-φT(t)P(t)φ(t)]y~(t)v(t).
Here, we have used the inequality 1-φT(t)P(t)φ(t)=[1+φT(t)P(t-1)φ(t)]-1⩾0. Because y~(t) and φT(t)P(t)φ(t) are uncorrelated with v(t) and are ℱt-1 measurable, taking the conditional expectation with respect to ℱt-1, we have
(3.4)E[T(t)∣Ft-1]⩽T(t-1)+2φT(t)P(t)φ(t)σ2.
Since ln|P-1(t)| is nondecreasing, letting
(3.5)V(t):=T(t)[ln⁡|P-1(t)|]γ,γ>1,
we have
(3.6)E[V(t)∣Ft-1]⩽T(t-1)[ln⁡|P-1(t)|]γ+2φT(t)P(t)φ(t)[ln⁡|P-1(t)|]γσ2⩽V(t-1)+2φT(t)P(t)φ(t)[ln⁡|P-1(t)|]γσ2,a.s.
Using Lemma 3.2, the sum of the last term in the right-hand side for t from 1 to ∞ is finite. Applying Lemma 3.1 to the previous inequality, we conclude that V(t) converges a.s. to a finite random variable, say V0, that is:
(3.7)V(t)=T(t)[ln⁡|P-1(t)|]γ⟶V0<∞,a.s.,orT(t)=O([ln⁡|P-1(t)|]γ),a.s.
Thus, according to the definition of T(t), we have
(3.8)‖θ~(t)‖2⩽tr⁡[θ~T(t)P-1(t)θ~(t)]λmin⁡[P-1(t)]=O([ln⁡|P-1(t)|]γλmin⁡[P-1(t)])=O(o(λmin⁡[P-1(t)])λmin⁡[P-1(t)])⟶0,a.s.
This completes the proof of Theorem 3.3.According to the definition ofθ and the assumption b1=1, the estimates a^(t)=[a^1(t),a^2(t),…,a^n(t)]T and c^(t)=[c^1(t), c^2(t), …, c^m(t)]T of a and c can be read from the first n and second m entries of θ^, respectively. Let θ^i be the ith element of θ^. Referring to the definition of θ, the estimates b^j(t) of bj, j=2,3,…,n, may be computed by
(3.9)b^j(t)=θ^n+(j-1)m+i(t)c^i(t),j=2,3,…,n;i=1,2,…,m.
Notice that there is a large amount of redundancy about b^j(t) for each i=1,2,…,m. Since we do not need such m estimates b^j(t), one way is to take their average as the estimate of bj [14], that is:
(3.10)b^j(t)=1m∑i=1m‍θ^n+(j-1)m+i(t)c^i(t),j=2,3,…,n.


## 4. The Input Nonlinear CARARMA System and Estimation Algorithm

Consider the following input nonlinear controlled autoregressive autoregressive moving average (IN-CARARMA) systems:(4.1)A(z)y(t)=B(z)u-(t)+D(z)γ(z)v(t),(4.2)u-(t)=f(u(t))=c1f1(u(t))+c2f2(u(t))+⋯+cmfm(u(t)),γ(z):=1+γ1z-1+γ2z-2+⋯+γnγz-nγ,D(z):=1+d1z-1+d2z-2+⋯+dndz-nd.
Let
(4.3)w(t):=D(z)γ(z)v(t),
or
(4.4)w(t)=[1-γ(z)]w(t)+D(z)v(t)=-∑i=1nγ‍γiw(t-i)+∑i=1nd‍div(t-i)+v(t).
Define the parameter vector θ and information vector φ(t) as
(4.5)θ:=[θ1T,γ1,γ2,…,γnγ,d1,d2,…,dnd]T∈Rn+mn+nγ+nd,φ(t):=[φ1T(t),-w(t-1),-w(t-2)…,-w(t-nγ),v(t-1),v(t-2),…,v(t-nd)]T∈Rn+mn+nγ+nd,θ1:=[ab1cb2c⋮bnc]∈Rn+nm,φ1(t):=[φ0(t)φ1(t)φ2(t)⋮φn(t)]∈Rn+nm,a:=[a1a2⋮an]∈Rn,c:=[c1c2⋮cm]∈Rm,φ0(t):=[-y(t-1)-y(t-2)⋮-y(t-n)]∈Rn,φj(t):=[f1(u(t-j))f2(u(t-j))⋮fm(u(t-j))]∈Rm,j=1,2,…,n.
Then (4.1) can be written as
(4.6)y(t)=[1-A(z)]y(t)+B(z)u-(t)+w(t)=-∑i=1n‍aiy(t-i)+∑i=1n‍bi∑j=1m‍cjfj(u(t-i))+w(t)=-∑i=1n‍aiy(t-i)+∑i=1n‍∑j=1m‍bicjfj(u(t-i))+w(t)=φ1T(t)θ1+w(t)=φ1T(t)θ1-∑i=1nγ‍γiw(t-i)+∑i=1nd‍div(t-i)+v(t)=φT(t)θ+v(t).
This is a linear-in-parameter identification model for IN-CARARMA systems.The unknownw(t-i) and v(t-i) in the information vector φ(t) are replaced with their estimates w^(t-i) and v^(t-i), and then we can obtain the following recursive generalized extended least squares algorithm for estimating θ in (4.6):
(4.7)θ^(t)=θ^(t-1)+L(t)[y(t)-φ^T(t)θ^(t-1)],L(t)=P(t-1)φ^(t)[1+φ^T(t)P(t-1)φ^(t)]-1,P(t)=[I-L(t)φ^T(t)]P(t-1),P(0)=p0I,φ^(t)=[φ1T(t),-w^(t-1),-w^(t-2),…,-w^(t-nγ),v^(t-1),v^(t-2),…,v^(t-nd)]T,φ1(t)=[φ0(t)φ1(t)φ2(t)⋮φn(t)],φ0(t)=[-y(t-1)-y(t-2)⋮-y(t-n)],φj(t)=[f1(u(t-j))f2(u(t-j))⋮fm(u(t-j))],w^(t)=y(t)-φ^1T(t)θ^1(t),v^(t)=y(t)-φ^T(t)θ^(t),θ^(t)=[θ^1T(t),γ^1(t),γ^2(t),…,γ^nγ(t),d^1(t),d^2(t),d^nd(t)]T.This paper presents a recursive least squares algorithm for IN-CAR systems and a recursive generalized extended least squares algorithm for IN-CARARMA systems with ARMA noise disturbances, which differ not only from the input nonlinear controlled autoregressive moving average (IN-CARMA) systems in [14] but also from the input nonlinear output error systems in [15].


## 5. Example

Consider the following IN-CAR system:(5.1)A(z)y(t)=B(z)u-(t)+v(t),A(z)=1+a1z-1+a2z-2=1-1.35z-1+0.75z-2,B(z)=b1z-1+b2z-2=z-1+1.68z-2,u-(t)=f(u(t))=c1u(t)+c2u2(t)+c3u3(t)=u(t)+0.50u2(t)+0.20u3(t),θ=[θ1,θ2,θ3,θ4,θ5,θ6,θ7,θ8]T=[a1,a2,c1,c2,c3,b2c1,b2c2,b2c3]T=[-1.350,0.75,1.00,0.50,0.20,1.68,0.84,0.336]T,θs=[a1,a2,b2,c1,c2,c3]T=[-1.35,0.75,1.68,1.00,0.50,0.20]T.
In simulation, the input {u(t)} is taken as a persistent excitation signal sequence with zero mean and unit variance and {v(t)} as a white noise sequence with zero mean and constant variance σ2. Applying the proposed algorithm in (2.10)-(2.11) to estimate the parameters of this system, the parameter estimates θ and θs and their errors with different noise variances are shown in Tables 1, 2, 3, and 4, and the parameter estimation errors δ:=∥θ^(t)-θ∥/∥θ∥ and δs:=∥θ^s(t)-θ∥/∥θs∥ versus t are shown in Figures 1 and 2. When σ2=0.502 and σ2=1.502, the corresponding noise-to-signal ratios are δns=10.96% and δns=32.87%, respectively.Table 1
The parameter estimates (θ) (σ2=0.502, δns=10.96%).
t
a
1
a
2
c
1
c
2
c
3
b
2
c
1
b
2
c
2
b
2
c
3
δ (%)
100
−1.35989
0.76938
0.94139
0.49861
0.18862
1.69875
0.86164
0.32773
2.59527
200
−1.35622
0.76001
0.96720
0.50101
0.19076
1.67233
0.84941
0.34369
1.43552
500
−1.35239
0.75452
1.00256
0.50137
0.19363
1.66468
0.84394
0.34485
0.74281
1000
−1.35034
0.75193
1.00570
0.50128
0.19460
1.65482
0.85095
0.33765
1.06112
2000
−1.34844
0.74940
0.99224
0.50089
0.20143
1.69169
0.85148
0.33583
0.67584
3000
−1.34776
0.74847
0.99012
0.49943
0.20333
1.68675
0.85321
0.33416
0.68173
True values
−1.35000
0.75000
1.00000
0.50000
0.20000
1.68000
0.84000
0.33600Table 2
The parameter estimates (θs) (σ2 = 0.502, δns = 10.96%).
t
a
1
a
2
b
2
c
1
c
2
c
3
δ (%)
100
−1.35989
0.76938
1.75670
0.94139
0.49861
0.18862
3.90775
200
−1.35622
0.76001
1.74205
0.96720
0.50101
0.19076
2.81582
500
−1.35239
0.75452
1.70821
1.00256
0.50137
0.19363
1.15783
1000
−1.35034
0.75193
1.69268
1.00570
0.50128
0.19460
0.59233
2000
−1.34844
0.74940
1.69068
0.99224
0.50089
0.20143
0.52605
3000
−1.34776
0.74847
1.68512
0.99012
0.49943
0.20333
0.46851
True values
−1.35000
0.75000
1.68000
1.00000
0.50000
0.20000Table 3
The parameter estimates (θ) (σ2 = 1.502, δns = 32.87%).
t
a
1
a
2
c
1
c
2
c
3
b
2
c
1
b
2
c
2
b
2
c
3
δ (%)
100
−1.37143
0.79561
0.81731
0.49688
0.16665
1.75280
0.90056
0.30999
7.98804
200
−1.36256
0.77335
0.89403
0.50353
0.17365
1.65999
0.87041
0.36082
4.46419
500
−1.35374
0.76009
1.00710
0.50417
0.18108
1.63863
0.85315
0.36321
2.08028
1000
−1.35074
0.75537
1.01710
0.50372
0.18381
1.60488
0.87297
0.34101
3.17034
2000
−1.34587
0.74895
0.97678
0.50296
0.20424
1.71432
0.87455
0.33540
2.01031
3000
−1.34448
0.74649
0.97053
0.49834
0.20990
1.69896
0.87916
0.33025
2.00404
True values
−1.35000
0.75000
1.00000
0.50000
0.20000
1.68000
0.84000
0.33600Table 4
The parameter estimates (θs) (σ2 = 1.502, δns = 32.87%).
t
a
1
a
2
b
2
c
1
c
2
c
3
δ (%)
100
−1.37143
0.79561
1.93906
0.81731
0.49688
0.16665
12.66088
200
−1.36256
0.77335
1.88773
0.89403
0.50353
0.17365
9.26624
500
−1.35374
0.76009
1.77500
1.00710
0.50417
0.18108
3.83703
1000
−1.35074
0.75537
1.72205
1.01710
0.50372
0.18381
1.90836
2000
−1.34587
0.74895
1.71204
0.97678
0.50296
0.20424
1.57452
3000
−1.34448
0.74649
1.69603
0.97053
0.49834
0.20990
1.39744
True values
−1.35000
0.75000
1.68000
1.00000
0.50000
0.20000Figure 1
The parameter estimation errorsδ versus t.Figure 2
The parameter estimation errorsδs versus t.From Tables1–4 and Figures 1 and 2, we can draw the following conclusions.(i)
The larger the data length is, the smaller the parameter estimation errors become.(ii)
A lower noise level leads to smaller parameter estimation errors for the same data length.(iii)
The estimation errorsδ and δs become smaller (in general) as t increases. This confirms the proposed theorem.


## 6. Conclusions

The recursive least-squares identification is used to estimate the unknown parameters for input nonlinear CAR and CARARMA systems. The analysis using the martingale convergence theorem indicates that the proposed recursive least squares algorithm can give consistent parameter estimation. It is worth pointing out that the multi-innovation identification theory [26–33], the gradient-based or least-squares-based identification methods [34–41], and other identification methods [42–49] can be used to study identification problem of this class of nonlinear systems with colored noises.



---

*Source: 684074-2012-08-13.xml*