# Two Identification Methods for a Nonlinear Membership Function

Authors: Yuejiang Ji; Lixin Lv
Journal: Complexity
 (2021)

---

## Abstract

This paper proposes two parameter identification methods for a nonlinear membership function. An equation converted method is introduced to turn the nonlinear function into a concise model. Then a stochastic gradient algorithm and a gradient-based iterative algorithm are provided to estimate the unknown parameters of the nonlinear function. The numerical example shows that the proposed algorithms are effective.


---

## Body

## 1. Introduction

Parameter estimation has many applications in system modelling and signal processing [1–5]. Recently, there exist many parameter estimation approaches, such as recursive least squares (RLS) methods [6–8], stochastic gradient (SG) methods [9–11], and iterative identification methods [12–14]. For example, Chen et al. proposed an iterative method for Hammerstein systems with saturation and dead-zone nonlinearities [15]. Li et al. presented a gradient iterative estimation algorithm and a Newton iterative estimation algorithm for a nonlinear function [16].In the past years, the approximation capability of fuzzy systems has received much attention especially for control purposes [17–19]. The approximation theory of fuzzy systems is often applied to approximate the unknown functions of control systems. It is shown that the identification of membership functions plays an essential role to ensure the system capability. There are many different types of membership functions in the approximation theory of fuzzy systems, such as the Gaussian membership functions [20–22] and other membership functions [23, 24].Since the membership functions usually have complex nonlinear structures, the traditional least squares may be infeasible for the identification of the functions because their derivative functions sometimes do not have analytical solutions [25–27]. Thanks to the gradient descent (GD) algorithm, which avoids solving an analytical function, thus it can be extended to complex nonlinear membership functions. The GD algorithm generates a sequence estimate using an iterative function, where the iterative function consists of two parts: one is the negative direction and the other is the step size [28, 29]. With a correct direction and an optimal step size, the GD algorithm can ensure the results converge to the true values.In this paper, we propose two methods to estimate the unknown parameters of a nonlinear membership function; these two methods are both based on the GD method. First, a gradient iterative algorithm is proposed, which can estimate the parameters based on all the collected data, and thus, it has heavy computational efforts. In order to reduce the computational efforts, we turn the model of the nonlinear function into a regression model and then use two identification algorithms to estimate the unknown parameters.The rest of the paper is organized as follows. Section2 introduces the nonlinear function and the gradient-based iterative algorithm. Section 3 develops the model transformation-based stochastic gradient method and iterative method. Section 4 provides an illustrative example. Finally, concluding remarks are given in Section 5.


## 2. The Nonlinear Membership Function and the Gradient-Based Iterative Algorithm

Let us introduce some notations first. The symbolI stands for an identity matrix of the appropriate sizes; the norm of a matrix X is defined as X:=XTX; the superscript T denotes the matrix transpose.Consider a nonlinear membership function in [30](1)yi=11+e−xi−a/b−11+e−xi−a/b+a,where xiandyi,i=1,2,3,…,nn≫3, are the measured data contaminated with noises and a and b are unknown parameters to be estimated. When the parameters a and b are already known, this nonlinear membership function is often used in a single-input and single-output fuzzy system.Define the cost function and the parameter vectorθ1 as(2)J1a,b≔12∑i=1nyi−11+e−xi−a/b+11+e−xi−a/b+a2,θ1≔a,bT.The gradient ofJ1θ1=J1a,b with respect to θ1 is(3)∇J1θ1=∂J1θ1∂a∂J1θ1∂b=∑i=1ne−xi+a/bb1+e−xi−a/b2−1−be−xi+a/b+ab1+e−xi−a/b+a2yi−11+e−xi−a/b+11+e−xi−a/b+a∑i=1n−xi−ae−xi+a/bb21+e−xi−a/b2−−xi−ae−xi+a/b+ab21+e−xi−a/b+a2yi−11+e−xi−a/b+11+e−xi−a/b+a.Letk be the iterative variable and θ^1k=a^kb^k be the estimates of θ1 at iteration k. We can get the following gradient-based iterative algorithm:(4)θ^1k+1=θ^1k−μ1k∇J1θ^1k,∇J1θ^1k=∑i=1ne−xi+a^k/b^kb^k1+e−xi−a^k/b^k2−1−b^ke−xi+a^k/b^k+ab^k1+e−xi−a^k/b^k+a2ɛ1ik∑i=1n−xi−a^ke−xi+a^k/b^kb^k21+e−xi−a^k/b^k2−−xi−a^ke−xi+a^k/b^k+ab^k21+e−xi−a^k/b^k+a^k2ɛ1ik,ɛ1ik=yi−11+e−xi−a^k/b^k+11+e−xi−a^k/b^k+a^k,P1k=−∇J1θ^1k,μ1k=argminμ1⩾0J1θ^1k+μ1P1k.There exist many methods which can compute the step sizeμ1k, e.g., the steepest descent method, the stochastic gradient method, and the projection method [31–35].Define(5)−1b=a1,−ab=a2,−ab+a=a3.Then, equation (1) can be simplified as(6)yt=11+ea1x+a2−11+ea1x+a3.Define the cost function and the parameter vectorθ2 as(7)J2a1,a2,a3≔12∑i=1nyi−11+e−xi−a/b+11+e−xi−a/b+a2,θ2≔a1,a2,a3T.The gradient ofJ2θ2=J2a1,a2,a3 with respect to θ2 is(8)∇J2θ2=∂J2θ2∂a1∂J2θ2∂a2∂J2θ2∂a3=∑i=1n−xiea1x+a21+ea1xi+a22−−xiea1xi+a31+ea1xi+a32yi−11+ea1xi+a2−11+ea1xi+a3∑i=1n−ea1xi+a21+ea1xi+a22yi−11+ea1xi+a2−11+ea1xi+a3∑i=1nea1xi+a31+ea1xi+a32yi−11+ea1xi+a2−11+ea1xi+a3.Letk be the iterative variable and θ^2k=a^1ka^2ka^3k be the estimates of θ2 at iteration k. We can get the following gradient-based iterative algorithm:(9)θ^2k+1=θ^2k−μ2k∇J2θ^2k,ɛ2ik=yi−11+ea^1kxi+a^2k−11+ea^1kxi+a^3k,∇J2θ^2k=∑i=1n−xiea^1kxi+a^2k1+ea^1kxi+a^2k2−−xiea^1kxi+a^3k1+ea^1kxi+a^3k2ɛik∑i=1n−ea^1kxi+a^2k1+ea^1kxi+a^2k2ɛik∑i=1nea^1kxi+a^3k1+ea^1kxi+a^3k2ɛik,P2k=−∇J2θ^2k,μ2k=argminμ2⩾0J2θ^2k+μ2P2k.The above iterative algorithms are difficult and have heavy computational burden because they update the parameters with all the collected data. In order to reduce the computational effort, we propose two modified methods in the next section.


## 3. The Model Transformation-Based Stochastic Gradient Method and Iterative Method

Convert the nonlinear function (6) to an identification model:(10)y=−ea2+ea3yea1x+ea3−ea2ea1x−ea2+a3ye2a1x.Then define the parameter vectorθ3 and the information vector φ3t as(11)θ3≔ea2+ea3ea3−ea2ea2+a3,φ3t≔−ytea1xtea1xt−yte2a1xt.Without loss of generality, a noise termvt with zero mean is introduced to the identification model in (10), and the identification model is put into a concise form(12)yt=φ3Ttθ3+vt.Letθ^3 (i) be the ith element of the vector θ^3; we have a^2=lnθ^31−θ^32/2, a^3=lnθ^31+θ^32/2, and a^=a^3−a^2. At last, with the estimated parameters a^ and a^2, we obtain b^=−a^/a^2.Letθ^3t and φ^3t be the estimate of θ3 and φ3, and θ^3t and φ^3t are defined as(13)θ^3t=ea^2t+ea^3tea^3t−ea^2tea^2t+a^3t,φ^3t=ytea^1t−1xtea^1t−1xtyte2a^1t−1xt.Using the gradient search and minimizing the quadratic criterion function(14)J3θ3≔yt−φ3Ttθ32lead to the model transformation-based stochastic gradient (MT-SG) algorithm:(15)θ^3t=θ^3t−1+φ^3tμ3tyt−φ^3Ttθ^3t−1,(16)a^1t−1=a^2t−1a^3t−1−a^2t−1,(17)φ^3t=ytea^1t−1xtea^1t−1xtyte2a^1t−1xt,(18)0<μ3t<2λmaxφ^3Ttφ^3t.The steps of computing the parameter estimation vectorθ^3t by the MT-SG algorithm in (15)–(18) are listed as follows:(1)
Collect the measured dataxt,yt:t=0,1,2,…,(2)
To initialize, lett=1 and θ^30=1/p0(3)
Computea^1t−1 according to (16)(4)
Computeφ^3t by (17)(5)
Chooseμ3t according to (18)(6)
Update the parameter estimation vectorθ^3t by (15), and compare θ^3t and θ^3t−1: if they are sufficiently close, or for some preset small ɛ, if θ^3t−θ^3t−1⩽ɛ, then terminate the procedure and obtain the estimate θ^3t; otherwise, increase t by 1 and go to step 3In general, the MT-SG algorithm is suitable for online identification and the iterative algorithm is used for offline identification. The iterative algorithm employs the idea of updating the estimateθ^3 using a fixed data batch with a finite length L and thus has a higher estimation accuracy than the SG algorithm. Next, we use finite measurement input-output data xi,yi,i=0,1,…,L and iterative with subscript k.Define the stacked output vectorYL and the stacked information matrix ΦL as(19)YL≔yL,yL−1,yL−2,…,y1T∈ℝL,ΦL≔φL,φL−1,φL−2,…,φ1T∈ℝL×3.Letθ^3k and φ^3k be the iterative estimate of θ3 and φ3 at iteration k=1,2,3,…, and θ^3k, φ^3k, and Φ3kL are defined as(20)θ^3k=ea^3k+ea^3k,ea^3k−ea^2k,ea^2k+a^3kT,φ^3kL=yLea^1k−1xL,ea^1k−1xL,yLe2a^1k−1xLT,Φ3kL=φ3kL,φ3kL−1,φ3kL−2,…,φ3k1T∈ℝL×3.Define a quadratic criterion function(21)J4θ3≔YL−Φ3Lθ32.MinimizingJ4θ3 and using the negative gradient search lead to the model transformation-based gradient iterative algorithm (MT-GI) of computing θ3:(22)θ^3k=θ^3k−1+μ3kΦ3kTLYL−Φ3kLθ^3k−1,(23)a^1k−1=a^2k−1a^3k−1−a^2k−1,(24)φ^3kL=yLea^1k−1xL,ea^1k−1xL,yLe2a^1k−1xLT,(25)Φ3kL=φ3kL,φ3kL−1,φ3kL−2,…,φ3k1T∈ℝL×3,(26)0<μ3k<2λmaxΦ^3kTLΦ^3kL.The steps of computing the parameter estimation matrixθ^3k by the MT-GI algorithm are listed as follows:(1)
Collect the input dataxt,yt:t=0,1,2,…,L(2)
To initialize, letk=1 and θ^30=1/p0, with 1 being a column vector whose entries are all unity and p0=106.(3)
Computea^1k−1 by (23)(4)
Formφ^3kt according to (24) and Φ^3kL according to (25)(5)
Choose a maximumμ3k by (26)(6)
Computeθ^3k by (22)(7)
Compareθ^3k and θ^3k−1: if they are sufficiently close, or for some preset small ɛ, if θ^3k−θ^3k−1⩽ɛ, then terminate the procedure and obtain the iterative times k, and estimate θ^3k; otherwise, increase k by 1 and go to step 3


## 4. Example

Consider a nonlinear function(27)y=11+e−x+a/b−11+e−x+a/b+a+v=11+e−x+0.6/1.2−11+e−x+0.6/1.2+0.6+v.where a1=−1/b=0.83,a2=−a/b=−0.5, and a3=−a/b+a=0.1, and then, we can conclude(28)θ3=ea2+ea3ea3−ea2ea2+a3=1.7120.4990.67.Define(29)c1c2c3c4≔ea2+ea3ea3−ea2ea2+a3a1=1.7120.4990.67−0.83.Assumext is the input and is taken as a persistent excitation signal with zero mean and unit variances and vt is taken as white noise with a zero mean and variance σ12=0.102. Applying the MT-SG algorithm and MT-GI algorithm to estimate the parameters of this system, the parameter estimates and their errors are shown in Tables 1 and 2, and the parameter estimation errors δ:=θ^3−θ3/θ3 versus t and k are shown in Figures 1 and 2.Table 1
The MT-SG estimates and errors.
tc1c2c3c4δ%100.714480.482370.95045−0.8329349.87382201.135110.482371.07447−0.8329333.91638301.203070.482370.83073−0.8329325.69778501.447680.482370.77121−0.8329313.645271001.675390.482370.68940−0.832932.153131501.705690.482370.67428−0.832930.891472001.711240.482370.67039−0.832930.813632501.711760.482370.67022−0.832930.812743001.711760.482370.67022−0.832930.81274True values1.712000.499000.67000−0.83000Table 2
The MT-GI estimates and errors withL=20.
kc1c2c3c4δ%101.547560.053910.49679−0.8329324.30909201.693110.084020.65245−0.8329320.00987301.713530.109390.67933−0.8329318.75623501.716810.157970.68337−0.8329316.426971001.715640.276160.68055−0.8329310.738341501.714410.390410.67768−0.832935.242012001.713220.500850.67491−0.832930.29525True values1.712000.499000.67000−0.83000Figure 1
The parameter estimation errorsδ versus t (MT-SG).Figure 2
The parameter estimation errorsδ versus k (MT-GI).From Tables1 and 2 and Figures 1 and 2, we can draw the following conclusions:(1)
The MT-GI algorithm has higher estimation accuracy than the MT-SG algorithm(2)
The parameter estimation errors by the MT-SG algorithm become smaller and smaller and go to zero witht increasing(3)
The parameter estimation errors by the MT-GI algorithm become smaller and smaller and go to zero with the subscriptk increasing


## 5. Conclusions

This paper presents two identification methods for a nonlinear membership function. An equation converted method is proposed to convert the nonlinear function to a concise model, and then an MT-SG algorithm and an MT-GI algorithm are provided to identify the nonlinear function. The simulation results verify the proposed algorithms.



---

*Source: 5515888-2021-05-03.xml*