# Hierarchical Newton Iterative Parameter Estimation of a Class of Input Nonlinear Systems Based on the Key Term Separation Principle

Authors: Cheng Wang; Kaicheng Li; Shuai Su
Journal: Complexity
 (2018)

---

## Abstract

This paper investigates the identification problem for a class of input nonlinear systems whose disturbance is in the form of the moving average model. In order to improve the computation complexity, the key term separation principle is introduced to avoid the redundant parameter estimation. Based on the decomposition technique, a hierarchical Newton iterative identification method combining the key term separation principle is proposed for enhancing the estimation accuracy and handling the computational load with the presence of the high dimensional matrices. In the identification procedure, the unknown internal items or vectors are replaced with their iterative estimates. The effectiveness of the proposed identification methods is shown via a numerical simulation example.


---

## Body

## 1. Introduction

In the modern cyber-physical system, including robotics systems [1, 2], railway control systems [3, 4], etc., system identification plays an important role in establishing relationship between the virtual system and the real world by using the modeling technique [5, 6]. Generally, modeling techniques can be split into two groups: nonparametric modeling and parametric modeling. Nonparametric modeling, so-called black or grey box modeling, ignores the mechanism of the system and instead concentrates on studying the relationship between the system input and output [7, 8]. In contrast, parametric modeling focuses on estimating the parameters where the model structure is fixed on the basis of the first principle or others [9, 10].The parametric modeling or parameter estimation relies deeply on adaptive algorithm [11, 12]. The core idea of adaptive parameter estimation is to recursively adjust the parameters by using the residuals, which makes the estimates approximately approach the true value. Under this framework of adaptation, the recursive least squares algorithms [13, 14], the stochastic gradient algorithms [15], and the iterative algorithms [16, 17] are well developed and underpin several heuristic or bioinspired learning algorithms [18–20]. For example, we use the adaptive algorithms such as gradient descent algorithm for learning the weights in the neural networks or training the fitness in the genetic algorithms.Under the adaptive identification framework, the Newton iterative algorithm can produce high accuracy estimation with fast convergence property [21–23]. However, the identification process involves heavy matrix computation. In the industry field, the wireless embedded devices are so vulnerable to the complexity of the algorithm that high power-consumed computation needs be avoided. The decomposition technique, as an effective tool to improve the computation efficiency, is applied into many identification algorithms. For example, Ding et al. decomposed Hammerstein-controlled autoregressive systems into three subsystems and employed the auxiliary model identification idea for handling unknown parameters coupled in each subsystem [24]. Ma et al. proposed a decomposition-based recursive least squares identification methods for multivariate pseudolinear systems using the multi-innovation theory [25].Block-oriented identification of the nonlinear systems such as the Hammerstein models [26, 27], the Wiener models [28], or the Hammerstein-Wiener models has become the active topic in the research of nonlinear parametric modeling, and many identification methods have been developed [29–31]. Liu and Bai presented an iterative identification algorithm for Hammerstein systems and studied its convergence [32]. The iterative algorithms utilize the batch of data for parameter estimation and are used in various off-line applications. Ding et al. focused on on-line identification and proposed a recursive least squares parameter estimation algorithm for output nonlinear autoregressive systems [33]. Recently, Ma et al. used the variational Bayesian approach to identify the Hammerstein parameter varying systems for chemical dynamic processes, which often work under various conditions bringing the varying model parameters [34].The general Hammerstein model or input nonlinear model presented by a memoryless nonlinear block and a linear dynamic subsystem can be formulated.(1)yt=Gzu¯t+Nzvt,where yt∈ℝ and ut∈ℝ are the measured system output and input, vt∈ℝ is the stochastic white noise, Gz and Nz are the transfer functions of the system model and the noise model, the intermediate variable xt=Gzu¯t denotes the noise-free system output, and wt=Nzvt denotes the noise model output. The unmeasurable variable u¯t is the output of the nonlinear block and can be represented as the linear combination of the known parameter vectors γ1,γ2,⋯,γm and the known basis f=f1,f2,⋯,fm:
(2)u¯t=γ1f1ut+γ2f2ut+⋯+γmfmut=∑j=1mγjfjut=futγ,where the nonlinear block fut≔f1ut,f2ut,⋯,fmut∈ℝ1×m and γ≔γ1,γ2,⋯,γmT∈ℝm. The superscript T denotes the matrix/vector transpose.It is worth noting that the model in (1) can represent various input nonlinear systems, e.g., when Gz=Bz/Az, and the system is corrupted by the colored noise, i.e., Nz=Dz, system (1) denotes an input nonlinear equation-error moving average (IN-EEMA) system,
(3)Azyt=Bzu¯t+Dzvt,where
(4)Az≔1+a1z−1+a2z−2+⋯+anaz−na,Bz≔b0+b1z−1+b2z−2+⋯+bnbz−nb,Dz≔1+d1z−1+d2z−2+⋯+dndz−nd.Recently, a key variable separation-based (multi-innovation) Newton iterative algorithm has been proposed for nonlinear finite response moving average systems [22] and an identification model-based (multi-innovation) Newton iterative algorithm has been proposed for nonlinear finite response autoregressive moving average systems [23]. On the basis of the works in [22, 23], this paper studies several Newton iterative parameter estimation methods for IN-EEMA systems.The objective of this work is to develop algorithms for improving the computational efficiency and achieving accurate parameter estimation for system (3). We develop three extended Newton iterative parameter estimation algorithms using the key term separation principle and the decomposition technique. The simulation results show that the algorithms are effective for identifying the proposed systems.Briefly, this paper is organised as follows. Section2 derives the identification model and develops an extended Newton iterative identification algorithm. Section 3 proposes a key term separation-based extended Newton iterative algorithm. Section 4 presents a key term separation-based extended Newton iterative algorithm using the decomposition technique. Section 5 provides an illustrative example to show the effectiveness of the algorithms. Finally, concluding remarks are offered in Section 6.


## 2. The Extended Newton Iterative Identification Algorithm

For the identification of Hammerstein system, in order to reduce the sensitivity of the projection algorithm to noise and to improve convergence rates of the stochastic gradient algorithm, Ding et al. proposed a Newton recursive and a Newton iterative identification algorithms by using the Newton method (Newton-Raphson method) [35]. Based on the work in [35], this paper considers identification of input nonlinear systems with colored noise and the disturbance is an autoregressive moving average process. Consider system (3) and define the parameter vectors a, b, d, γ, and η as
(5)a≔a1,a2,⋯,anaT∈ℝna,b≔b0,b1,b2,⋯,bnbT∈ℝnb+1,d≔d1,d2,⋯,dndT∈ℝnd,γ≔γ1,γ2,⋯,γmT∈ℝm,η≔a1,a2,⋯,ana,d1,d2,⋯,dndT∈ℝna+nd,and the information vector φt and the information matrix Ft as
(6)φt≔−yt−1,−yt−2,⋯,−yt−na,vt−1,⋯,vt−ndT∈ℝna+nd,Ft≔fut,fut−1,⋯,fut−nbT∈ℝnb+1×m.The output of system (3) can be expressed as
(7)yt=1−Azyt+Bzu¯t+Dzvt=−∑i=1naaiyt−i+∑i=0nbbiu¯t−i+∑i=1nddivt−i+vt=φTtη+bTFtγ+vt.Assume that the input/output data isut,yt:t=1,2,⋯,L, where L denotes the data length. Define the stacked matrices:
(8)YL≔y1,y2,⋯,yLT∈ℝL,Φ0L≔φ1,φ2,⋯,φLT∈ℝL×na+nd,Φγ≔γTFT1γTFT2⋮γTFTL∈ℝL×nb+1,Ψb≔bTFT1bTFT2⋮bTFTL∈ℝL×m.Define the cost function:(9)Jθ=Jη,b,γ≔YL−Φ0Lη−Φγb2=YL−Φ0Lη−Ψbγ2.The Hessian matrix ofJθ is computed by
(10)HJη,b,γ≔Ψ0TLΦ0LΦ0TLΦγΦ0TLΨbΦTγΦ0LΦTγΦγMη,b,γΨTbΦ0LMTη,b,γΨTbΨb∈ℝn×n,where n≔na+nd+nb+1+m.Letk be an iterative variable,
(11)θ^k≔η^kb^kγ^kbe the iterative estimate of
(12)θ≔ηbγat the kth iteration. Using the similar method in [35], minimizing Jθ, we have
(13)θ^k=θ^k−1HJη^k−1,b^k−1,γ^k−1−1gradθJη^k−1,b^k−1,γ^k−1.Here, the question is that it is impossible to accomplish computingθ^k because the unmeasurable noise vkt−i,i=1,2,…,nd, contained in φt leads to an unknown Φ0 in HJη^k−1,b^k−1,γ^k−1. Inspired by the former work in [36, 37], these unknown issues can be dealt with by replacing the unknown items with their estimates. Let the iterative estimate of φt be φ^kt:
(14)φ^kt≔−yt−1,−yt−2,⋯,−yt−na,v^t−1,v^t−2,⋯,v^k−1t−ndT.According to (7), we have
(15)vt=yt−φTη−bTFtγ.Replacing the unknown parameters and vectors in the above equation with their iterative estimatesη^k,b^k,γ^, and φ^kt, we can compute v^kt−i through
(16)v^kt=yt−i−φ^Tkt−iη^k−b^kTFt−iγ^k.Then the extended Newton iterative (E-NI) algorithm for IN-EEMA systems is summarized as follows:(17)θ^k=θ^k−1+Ω^k−1Φ^0TLΦTγ^k−1ΨTb^k−1YL−Φ^0Lη^k−1−Φγ^k−1b^k−1,(18)Ω^k=Φ^0TLΦ^0LΦ^0TLΦγ^k−1Φ^0TLΨb^k−1Φ^Tγ^k−1Φ^0LΦTγ^k−1Φγ^k−1M^θ^k−1ΨTb^k−1Φ^0LM^Tθ^k−1ΨTb^k−1Ψb^k−1,(19)M^θ^k−1=∑t−1LFt−yt+φ^Ttη^k−1+b^k−1TFtγ^k−1+ΦTγ^k−1Ψb^k−1,(20)YL=y1,y2,⋯,yLT,(21)Φ^0L=φ^k1,φ^k2,⋯,φ^kLT,(22)Ψb^k−1=FT1b^k−1,FT2b^k−1,⋯,FTLb^k−1T,(23)Φγ^k−1=F1γ^k−1,F2γ^k−1,⋯,FLγ^k−1T,(24)φ^kt=−yt−1,−yt−2,⋯,−yt−na,v^t−1,v^t−2,⋯,v^k−1t−ndT,(25)Ft=f1utf2ut⋯fmutf1ut−1f2ut−1⋯fmut−1⋮⋮⋮f1ut−nbf2ut−nb⋯fmut−nb,(26)v^kt=yt−i−φ^kTt−iη^k−b^kTFt−iγ^k.Notice that the Hessian matrix in (18) requires more computational effort as the number of system parameters grows.


## 3. The Key Term Separation-Based Extended Newton Iterative Algorithm

In this section, the key term separation principle is employed to parameterize the IN-EEMA systems. The core idea of the key term separation technique is to express the system output as a linear combination of the system parameters [38, 39]. Therefore, the redundant parameter estimation can be avoided. Let the first parameter of Bz be 1, and the system in (7) can be rewrite as
(27)yt=−∑i=1naaiyt−i+∑i=1nbbiu¯t−i+∑i=1nddivt−i+vt.Substituting the key termut into (2) gives
(28)yt=−∑i=1naaiyt−i+∑j=1mγifiut+∑i=1nbbiu¯t−i+∑i=1nddivt−i+vt.Define the parameter vectorϑ and the information vectors φat, φdt, and ϕt as
(29)ϑ≔aT,γT,b1,b2,⋯,bnb,dTT∈ℝn,n=na+nb+m,φat≔−yt−1,−yt−2,⋯,yn−na∈ℝna,φdt≔vt−1,vt−2,⋯,vt−nd∈ℝnd,ϕt≔φaTt,fut,u¯t−1,u¯t−2,⋯,u¯t−nb,φdTtT∈ℝn.Then system (3) takes the form of the identification model as
(30)yt=ϕTtϑ+vt.ConsiderL sets of data and define the stacked vector YL and the stacked matrix ΦL as
(31)YL≔y1y2⋮yL∈ℝL,ΦL≔ϕ1ϕ2⋮ϕL∈ℝL×n.Define the cost function(32)Jϑ≔YL−ΦLϑ2.Letϑ^k be the iterative estimation of ϑ at the kth iteration, and using the Newton method to minimize Jϑ gives
(33)ϑ^k=ϑ^k−1−HJϑ^k−1−1gradJϑ^k−1=ϑ^k−1+ΦTLΦL−1ΦTLYL−ΦLϑ^k−1.Replace the unknown variablesu¯t−i at the kth iteration with the iterative estimates u^k−1t−i and define
(34)ϕ^kt≔φaTt,fut,u¯^k−1t−1,u¯^k−1t−2,⋯,u¯^k−1t−nb,φ^d,k−1TtT∈ℝn,ϑ^k≔akT,γkT,b^1,k,b^2,k,⋯,b^nb,k,d^kTT∈ℝn.Replacingγ^i,k in (2) with γi, the iterative estimates of u¯t−i can be computed through
(35)u¯^t−i=γ^1,kf1ut−i+γ^2,kf2ut−i+⋯+γ^m,kfmut−i=∑j=1mγ^j,kfjut−ifut−iγ^k.From (28), we have
(36)vt−i=yt−i−ϕTt−iϑ,Replacingϕt−i and ϑ with ϕ^kt−i and ϑ^k respectively, the iterative estimates of vt−i is computed by
(37)v^kt−i=yt−i−ϕ^kTt−iϑ^k.Substituting unknownϕt in the stacked matrix ΦL with its estimate, the iterative estimate of ΦL is given by
(38)Φ^L≔ϕ^k1,ϕ^k2,⋯,ϕ^kLT∈ℝL×n.Then we replace theΦL in (33) with Φ^L and summarize the key term separation-based extended Newton iterative (KT-NI) algorithm for the IN-EEMA model:
(39)ϑ^k=ϑ^k−1+Φ^kTLΦ^kL−1Φ^kTLYL−Φ^kLϑ^k−1,(40)YL=y1,y2,⋯,yLT,(41)Φ^kL=ϕ^k1,ϕ^k2,⋯,ϕ^kLT,(42)ϕ^kt=φaTt,fut,u¯^k−1t−1,u¯^k−1t−2,⋯,u¯^k−1t−nb,φ^d,k−1TtT,(43)φat=−yt−1,−yt−2,⋯,−yt−naT,(44)fut=f1ut,f2ut,⋯,fmut,(45)φ^d,kt=v^kt−1,v^kt−2,⋯,v^kt−ndT,(46)u¯^kt−i=fut−iγ^k,(47)v^kt−i=yt−i−ϕ^kTt−iϑ^k,(48)ϑ^kt=a^1,k,a^2,k,⋯,a^n,k,γ^kT,b^1,k,b^2,k,⋯,b^nb,k,d^1,k,d^2,k,⋯,d^nd,kT,(49)γ^k=γ^1,k,γ^2,k,⋯,γ^m,kT.


## 4. Extended Newton Iterative Algorithm Using the Decomposition Technique

Define the information vectorsφat and φdt as
(50)φat≔−yt−1,−yt−2,⋯,−yt−naT∈ℝna,φdt≔vt−1,vt−2,⋯,vt−ndT∈ℝnd.Rewrite the IN-EEMA system in (7) as
(51)yt=1−Azyt+Bzu¯t+Dzvt=−∑i=1naaiyt−i+∑i=0nbbiu¯t−i+∑i=1nddivt−i+vt=φaTta+bTFtγ+φdTtd+vt.By applying the decomposition technique to the IN-EEMA system, we divide model (51) into two subsystems. One subsystem contains the system parameter vector ϑ1≔a,bT, and the other subsystem contains the system parameter vector ϑ2≔γ,dT. Define two auxiliary outputs:
(52)y1t≔yt−φdTtd,(53)y2t≔yt−φaTta.Combining (51), (52), and (53), it gives
(54)S1:y1t=φaTt,γTFTtϑ1+vt,S2:y2t=bTFTt,φdTtϑ2+vt.Letϑ^1,k≔a^k,b^kT and ϑ^2,k≔γ^k,d^kT be the iterative estimates of ϑ1 and ϑ2 at the kth iteration, respectively. Define the cost functions:
(55)J1ϑ1≔Jϑ1,ϑ^2,k−1=∑t=1Ly1t−φaTt,γ^k−1TFtϑ12∑t=1Lyt−φaTta−bTFTtγ^k−1−φdTtd^k−12,J2ϑ2≔Jϑ^1,k,ϑ2=∑t=1Ly2t−bkTFt,φdTtϑ22∑t=1Lyt−φaTta−bkTFTtγ−φdTtd2.Notice that the unknown parameter vectorsϑ1 and ϑ2 in cost functions above are replaced by their iterative estimates. For the sake of brevity, define the stacked matrices:
(56)ΓL≔φaT1φaT2⋮φaTL∈ℝL×na,Φγ,L≔φaT1γTFT1φaT2γTFT2⋮⋮φaTLγTFTL∈ℝL×na+nb+1,ΩL≔φdT1φdT2⋮φdTL∈ℝL×nd,Ψb,L≔bTF1φdT1bTF2φdT2⋮⋮bTFLφdTL∈ℝL×na+nb+1.Then the cost functions can be rewritten as(57)J1ϑ1=YL−Φγ,Lϑ1−ΩLd2,J2ϑ2=YL−ΓLa−Ψb,Lϑ22.MinimizingJ1ϑ1 and J2ϑ2 by using the Newton method, it yields
(58)ϑ^1,k=ϑ^1,k−1−HJ1ϑ^1,k−1−1gradJ1ϑ^1,k−1=ϑ^1,k−1+ΦTγ,LΨγ,L−1ΨTγ,LYL−Φγ,Lϑ^1,k−1−ΩLd,ϑ^2,k=ϑ^2,k−1−HJ2ϑ^2,k−1−1gradJ2ϑ^2,k−1=ϑ^2,k−1+ΨTb,LΨb,L−1ΨTb,LYL−ΓLa−Ψb,Lϑ^2,k−1.The similar issue arises that the information vectorφdt contains the unknown items vt−i; hence, ϑ^1,k and ϑ^2,k in the two equations above cannot be directly calculated. Here, following the solution carried out in the last two sections and replacing the unknown noise items with their iterative estimates v^k−1t−i at iteration k−1, the iterative estimate of φdt can be represented as
(59)φ^d,kt≔v^k−1t−1,v^k−1t−2,⋯,v^k−1t−ndT∈ℝnd.From (51), we have
(60)vt=yt−φaTa−bTFtγ−φdTtd.Replacing unknown vectorsa, b, γ, and φdt with their iterative estimates a^k,b^k,γ^k, and φ^d,kt, the estimated noise v^kt can be calculated from
(61)v^kt=yt−φaTta^k−bkTFtγ^k−φ^d,kTtd^k.Substituting the estimated vectorsγ^k−1,d^k−1,a^k, and b^k for the unknown vectors γ, d, a, and b and substituting Ψ^Tb^k,L and Ω^L for ΨTb,L and ΩL, respectively, we can summarize the decomposition-based extended Newton iterative (D-KT-NI) identification algorithm for IN-EEMA systems i.e., the hierarchical extended Newton iterative algorithm
(62)ϑ^1,k=ϑ^1,k−1+Φ^Tγ^k−1,LΦ^Tγ^k−1,L−1Φ^Tγ^k−1,LYL−Φ^Tγ^k−1,Lϑ^1,k−1−Ω^kLd^k−1,(63)ϑ^2,k=ϑ^2,k−1+Ψ^kTb^k,LΨ^kb^k,L−1Ψ^kTb^k,LYL−ΓLa^k−Ψ^kb^k,Lϑ^2,k−1,(64)YL=y1,y2,⋯,yLT,(65)ΓL=φa1,φa2,⋯,φaLT,(66)Φγ^k,−1,L=φaT1γ^k−1TFT1φaT2γ^k−1TFT2⋮⋮φaTLγ^k−1TFTL,(67)Ψb^k,L=b^kTF1φ^d,kT1b^kTF2φ^d,kT2⋮⋮b^kTFLφ^d,kTL,(68)Ω^L=φ^d,kT1,φ^d,kT2,⋯,φ^d,kTLT,(69)φat=−yt−1,−yt−2,⋯,−yt−naT,(70)φ^d,kt=v^kt−1,v^kt−2,⋯,v^kt−nd,T,(71)v^kt=yt−φaTta^k−bkTFtγ^k−φ^d,kTtd^k,(72)Ft=f1utf2ut⋯fmutf1ut−1f2ut−1⋯fmut−1⋮⋮⋮f1ut−nbf2ut−nb…fmut−nb.The steps for computing the parameter estimatesϑ^1,k and ϑ^2,k in (62), (63), (64), (65), (66), (67), (68), (69), (70), (71), and (72) are as follows.
(1)
Set the data lengthL, collect input output data ut,yt:t=1,2,⋯,L, form YL, and preset a small threshold value ϵ>0
(2)
Letk=1, and set the initial values ϑ^1,0=1na+nb, ϑ^2,0=1m+nd, v^t−i=a random vector, i=1,2,…,nd,p0=106
(3)
Formφat using (69), ΓL using (65), and Φγ^k−1,L using (66)
(4)
Updateϑ1,k using (62)
(5)
FormΩ^L using (68) and Ψb^k,L using (67)
(6)
Updateϑ^2,k using (63)
(7)
Computevkt−i using (71)
(8)
Ifϑ^1,k−ϑ^1,k−1+ϑ^2,k−ϑ^2,k−1>ϵ, increase k by 1, go to step 4; otherwise, terminate the procedure and obtain k, ϑ^1,k, and ϑ^2,k


## 5. Example

Consider the following nonlinear simulation system:(73)Azyt=Bzu¯t+Dzvt,Az=1+a1z−1+a2z−2=1−1.10z−1+0.70z−2,Bz=b0+b1z−1+b2z−2=1+0.55z−1+0.90z−2,Dz=1+d1z−1=1−0.30z−1,u¯t=γ1ut+γ2u2t=0.45ut+0.60u2t,ϑ=a1,a2,b1,b2,γ1,γ2,d1T=−1.10,0.70,0.55,0.90,0.45,0.60,−0.30T.In this simulation, the input vectorut is taken as an uncorrelated persistent excitation signal sequence with zero mean and vt as a white noise sequence with zero mean and variances σ2=0.102 and σ2=0.502. Taking the data length L=1000 data, we apply the proposed E-NI algorithm in (17), (18), (19), (20), (21), (22), (23), (24), (25), and (26), the KT-NI algorithm in (39), (40), (41), (42), (43), (44), (45), (46), (47), (48), and (49), and the D-KT-NI algorithm in (62), (63), (64), (65), (66), (67), (68), (69), (70), (71), and (72) to estimate the parameters of the example system; the parameter estimates of each algorithm and their errors are shown in Tables 1–3, and their parameter estimation errors δ≔θ^t−θ/θ and δ≔ϑ^t−ϑ/ϑ versus t of each algorithm are illustrated in Figures 1–3.Table 1
The parameter estimates and errors of the E-NI algorithm.
σ2
k
a1
a2
b1
b2
γ1
γ2
d1
δ (%)
0.102
1
−1.08845
0.71303
0.74340
0.98758
0.58515
0.65293
−0.00146
18.66607
2
−1.11208
0.71639
0.61498
0.96651
0.49424
0.63639
0.11693
20.41973
3
−1.09042
0.70042
0.55489
0.90946
0.45489
0.60967
−0.13144
8.02220
4
−1.09966
0.69927
0.54109
0.89591
0.44933
0.60349
−0.28284
0.95099
5
−1.10337
0.70316
0.54118
0.89811
0.44974
0.60455
−0.30211
0.53527
10
−1.10207
0.70266
0.54220
0.89854
0.45006
0.60491
−0.30147
0.47475
15
−1.10207
0.70265
0.54220
0.89854
0.45006
0.60491
−0.30147
0.47471
20
−1.10207
0.70265
0.54220
0.89854
0.45006
0.60491
−0.30147
0.47471
0.502
1
−1.07028
0.70468
0.73066
0.99064
0.57906
0.65907
−0.01324
17.96686
2
−1.06241
0.69400
0.56781
0.90670
0.45678
0.62211
−0.19086
5.65096
3
−1.09391
0.69393
0.50848
0.87623
0.43431
0.60337
−0.29842
2.42364
4
−1.10633
0.70643
0.51112
0.89287
0.43789
0.60902
−0.31520
2.17016
5
−1.10194
0.70632
0.51319
0.89320
0.43879
0.60950
−0.31016
1.98764
10
−1.10163
0.70488
0.51303
0.89271
0.43855
0.60934
−0.30986
1.98664
15
−1.10165
0.70490
0.51301
0.89269
0.43856
0.60933
−0.30988
1.98745
20
−1.10165
0.70490
0.51301
0.89269
0.43856
0.60933
−0.30988
1.98744
True values
−1.10000
0.70000
0.55000
0.90000
0.45000
0.60000
−0.30000Table 2
The parameter estimates and errors of the KT-NI algorithm.
σ2
k
a1
a2
b1
b2
γ1
γ2
d1
δ (%)
0.102
1
−1.39971
0.79168
−0.00123
−0.00620
0.41797
0.80328
−0.00689
62.45484
2
−1.11611
0.71412
0.41512
0.72162
0.45094
0.56978
−0.01914
19.38946
3
−1.10182
0.69537
0.57541
0.93598
0.45429
0.60462
−0.07944
12.09068
4
−1.09800
0.70092
0.54331
0.88549
0.45409
0.59402
−0.26185
2.25872
5
−1.09991
0.69509
0.55565
0.90284
0.45388
0.59980
−0.29063
0.69445
10
−1.10056
0.69884
0.54936
0.89586
0.45383
0.59723
−0.30489
0.43470
15
−1.10040
0.69837
0.55037
0.89703
0.45382
0.59763
−0.30442
0.38602
20
−1.10043
0.69844
0.55022
0.89685
0.45382
0.59757
−0.30453
0.39353
0.502
1
−1.35007
0.75506
−0.01352
−0.00620
0.43528
0.80979
−0.00969
62.02069
2
−1.11307
0.71565
0.40087
0.75740
0.46839
0.56276
−0.13437
14.43024
3
−1.10170
0.68286
0.58704
0.95127
0.47074
0.60626
−0.26416
4.17881
4
−1.10217
0.70483
0.52615
0.85580
0.46942
0.57888
−0.30066
3.12057
5
−1.10021
0.68548
0.56763
0.90326
0.46933
0.59731
−0.29752
1.62874
10
−1.10403
0.69630
0.54549
0.87854
0.46914
0.58844
−0.30854
1.76861
15
−1.10348
0.69460
0.54940
0.88304
0.46914
0.59008
−0.30751
1.56660
20
−1.10359
0.69491
0.54869
0.88222
0.46914
0.58978
−0.30772
1.59993
True values
−1.10000
0.70000
0.55000
0.90000
0.45000
0.60000
−0.30000Table 3
The parameter estimates and errors of the D-KT-NI algorithm.
σ2
k
a1
a2
b1
b2
γ1
γ2
d1
δ (%)
0.102
1
−1.23155
0.79509
0.52286
0.89348
0.41594
0.56889
−0.00553
16.11602
2
−1.03367
0.65908
0.54663
0.90391
0.47140
0.62599
−0.26604
4.33402
3
−1.15708
0.72762
0.52849
0.89594
0.43727
0.57773
−0.34233
3.94531
4
−1.05106
0.67958
0.56540
0.91451
0.46594
0.62016
−0.27119
3.26187
5
−1.13731
0.71586
0.53839
0.89625
0.44084
0.58372
−0.24959
3.23833
10
−1.08228
0.69163
0.55344
0.90582
0.45739
0.60709
−0.28489
1.30713
15
−1.10542
0.70184
0.54666
0.90170
0.45054
0.59746
−0.29649
0.38449
20
−1.09554
0.69756
0.54952
0.90347
0.45350
0.60164
−0.29610
0.39095
0.502
1
−1.18817
0.76494
0.54458
0.93310
0.42779
0.57102
0.01268
15.85345
2
−1.00303
0.63606
0.57278
0.94755
0.47960
0.61914
−0.23259
7.02860
3
−1.13160
0.70802
0.53521
0.91149
0.45152
0.58110
−0.33064
2.46460
4
−1.05647
0.67685
0.55353
0.91567
0.47439
0.61744
−0.27791
3.02136
5
−1.12083
0.70421
0.53813
0.90975
0.45345
0.58761
−0.33396
2.11972
10
−1.08063
0.68732
0.54608
0.91189
0.46725
0.60713
−0.30157
1.52877
15
−1.09787
0.69440
0.54221
0.91065
0.46138
0.59886
−0.31698
1.18714
20
−1.09061
0.69140
0.54374
0.91111
0.46387
0.60236
−0.31068
1.19411
True values
−1.10000
0.70000
0.55000
0.90000
0.45000
0.60000
−0.30000Figure 1
The parameter estimation errorsδ versus k (E-NI).Figure 2
The parameter estimation errorsδ versus k (KT-NI).Figure 3
The parameter estimation errorsδ versus k (D-KT-NI).From the simulation results in Tables1–3 and Figures 1–3, we can conclude following markers.
(i)
For all three algorithms, the parameter estimation errors are getting smaller (in general) as the iterative stepsk increases
(ii)
Both algorithms can produce highly accurate parameter estimates under different noise variances
(iii)
When the sizes of Hessian matricesΦ^kTLΦ^kL and Ψ^kTLΨ^kL in E-NI algorithm and KT-NI algorithm expand, two algorithms cost massive computational loads. While using the decomposition technique in the D-KT-NI algorithm, the dimensions of two matrices are trimmed from na+nb+m+nd+1×na+nb+m+nd+1 and na+nb+m+nd×na+nb+m+nd to na+nb+1×na+nb+1 and m+nd×m+nd. The slimmer Hessian matrices bring better computational efficiency, which is favorable for certain real-time computational situation


## 6. Conclusions

In this work, we have presented extended Newton iterative algorithm, a key term separation-based extended Newton iterative algorithm, and a decomposition-based extended Newton iterative algorithm using the key term separation principle for a class of input nonlinear systems. The illustrative example shows that the decomposition-based extended Newton iterative algorithm using the key term separation principle can produce the high accurate estimates at a relatively lower computational expense. The proposed methods can be further extended to engineering systems [40–42] or other nonlinear scalar or multivariable systems [43–45].



---

*Source: 7234147-2018-10-24.xml*