# Robust Kernel Clustering Algorithm for Nonlinear System Identification

Authors: Mohamed Bouzbida; Lassad Hassine; Abdelkader Chaari
Journal: Mathematical Problems in Engineering
 (2017)

---

## Abstract

In engineering field, it is necessary to know the model of the real nonlinear systems to ensure its control and supervision; in this context, fuzzy modeling and especially the Takagi-Sugeno fuzzy model has drawn the attention of several researchers in recent decades owing to their potential to approximate nonlinear behavior. To identify the parameters of Takagi-Sugeno fuzzy model several clustering algorithms are developed such as the FuzzyC-Means (FCM) algorithm, Possibilistic C-Means (PCM) algorithm, and Possibilistic Fuzzy C-Means (PFCM) algorithm. This paper presents a new clustering algorithm for Takagi-Sugeno fuzzy model identification. Our proposed algorithm called Robust Kernel Possibilistic Fuzzy C-Means (RKPFCM) algorithm is an extension of the PFCM algorithm based on kernel method, where the Euclidean distance used the robust hyper tangent kernel function. The proposed algorithm can solve the nonlinear separable problems found by FCM, PCM, and PFCM algorithms. Then an optimization method using the Particle Swarm Optimization (PSO) method combined with the RKPFCM algorithm is presented to overcome the convergence to a local minimum of the objective function. Finally, validation results of examples are given to demonstrate the effectiveness, practicality, and robustness of our proposed algorithm in stochastic environment.


---

## Body

## 1. Introduction

Modeling and identification are significant steps in the design of the control system. Typical applications of these models are the simulation, the prediction, or the control system design. Generally, the modeling process consists of obtaining a parametric model with the same dynamic behavior of the real process. However, when the process is nonlinear and complex, it is very difficult to define the different mathematical or physical laws which describe its behavior [1, 2]. In this context, the modeling of nonlinear systems by the conventional methods is very difficult and occasionally ineffective [3, 4]. So, other nonconventional techniques based on fuzzy logic are used more often in modeling this kind of process due to excellent ability of describing its behavior [4–6].Among the best fuzzy modeling approaches developed in literature we mention the Takagi-Sugeno fuzzy model. In effect, this model is described by if-then rules. Each rule includes a fuzzy set antecedent and mathematical functions as consequent representing the process behavior in each region [2, 3, 7]. The identification problem consists of estimating the model parameters. In this context, to identify the parameters of Takagi-Sugeno fuzzy model, many techniques were developed such as the Adaptive schemes, heuristic approaches, nearest neighbor clustering, and support vector learning mechanisms. Besides fuzzy clustering algorithms are widely used in fuzzy modeling. Fuzzy C-Means (FCM), Gustafson-Kessel (G-K), Gath-Geva (G-G), Possibilistic c-Means (PCM), Fuzzy C-Regression Model (FCRM), Enhanced Fuzzy C-Regression Model (EFCRM), and Possibilistic Fuzzy c-Means (PFCM) are popular clustering algorithms used in structure identification part and LS, WRLS, and orthogonal least square (OLS) technique were applied in consequent parameter estimation. Among the clustering algorithms, Fuzzy c-Means (FCM) developed by Bezdek is well-known but this algorithm is sensitive to noise or outliers and susceptible to local minima [8, 9]. However, noise in the data sets can make the situation worse by creating many inauthentic minima. These are able to distort the global minimum solution found by FCM algorithm. This flaw has stimulated the researchers to overcome this inconvenience. To fight against the effects of outlying data, various approaches are considered such as the possibilistic clustering (PCM) proposed by Krishnapuram and Keller [10] and fuzzy noise clustering approach of Dave [11]. The possibilistic approach executed a possibilistic partition, in which a membership relation calculates the absolute degree of typicality of a point in a cluster. Although the PCM algorithm is robust against the noise points and allows identifying these outliers, it is very responsive to initializations and occasionally generates coincident clusters. To solve this deficiency of identical clusters, a Possibilistic Fuzzy c-Means (PFCM) algorithm was suggested by Wu and Zhou in 2006. Nonetheless, these algorithms are not efficient for unequal dimension clusters and cannot separate clusters that are nonlinearly separable in input space and their limits between two clusters are linear. In our work to overcome this shortcoming, kernel methods [12] are regarded as the way of dealing with this problem. We propose a new clustering algorithm called Robust Kernel Possibilistic Fuzzy C-Means (RKPFCM), which adopts a kernel induced metric in the data space to replace the original Euclidean norm metric. By changing the inner product with an appropriate hyper tangent kernel function, one can implicitly affect a nonlinear mapping to a high dimensional feature space where the data is more clearly separable. However, our proposed algorithm RKPFCM has an iterative nature that makes it sensitive to initialization and sensitive to converge to a local minimum. To overcome these problems, several solutions have been proposed in the literature. Among them is combining the clustering algorithm with a heuristic optimization technique. In this context, many researches have proposed the evolutionary computation technique based on Particle Swarm Optimization (PSO). They have been successfully applied to solve various optimization problems [13]. Thus, we introduce PSO into the RKPFCM algorithm to achieve global optimization. The efficacy of our algorithm compared to many algorithms is tested on noisy nonlinear systems defined by recurrent equations and an application to Box Jenkins system. This paper will be presented as follows. The second part of work is reserved for introducing the Takagi-Sugeno fuzzy model. The third part will be devoted to identifying the premise parameters of this model where we used the proposed RKPFCM algorithm and PSO algorithm is introduced. In the fourth part, we will focus on identification of consequent parameters. The simulations results and the model validity of RKPFCM and RKPFCM-PSO are presented in part five. Finally, we conclude this paper.


## 2. Takagi-Sugeno Fuzzy Model

Takagi-Sugeno fuzzy model (T-S) is one of the best techniques used for modeling a nonlinear system represented by the recurrent equationyk=gNL(xk). T-S model is constructed by a rule-based type “if … then” in which the consequent uses numeric variables rather than linguistic variables (case of Mamdani). The consequent can be expressed by a constant, a polynomial, or differential equation depending on the antecedent variables. The T-S fuzzy model allows approximating the nonlinear system into several locally linear subsystems [4, 17].In general, a Takagi-Sugeno fuzzy model is based on if… then rules of the form(1)Ri:ifx1kisAi1,…,xnkisAinthenyi=aiTxk+bi.The “if” rule function defines the premise part and “then” rule function constitutes the consequent part of the T-S fuzzy model.Ri: represents the ith rule;aiT=ai1,ai2,…,ain: is the parameters vector, such as ai∈Rn;bi: is a scalar;xk=x1k,x2k,…,xnkT: is observations vector;Ai1,Ai2,…,Ain: represents the fuzzy subsets,where i∈1,…,C.Here, the fuzzy sets are represented by the following membership function [7](2)Aijxjk=exp⁡-xjk-vij2σij2∈0,1i=1,2,…,C,j=1,2,…,n, where vi=[vi1,vi2,…,vij]T∈Rn are centers and σi=[σi1,σi2,…,σij]T∈Rn is the width of the membership function.The estimated output model is defined by the following equation [4]:(3)y^k=∑i=1Cβikyik.As(4)βik=∏j=1nAijxjk∑i=1C∏j=1nAijxjkk=1,2,…,N,so,(5)yk^=∑i=1cβikaiTxk+bi.


## 3. Identification Algorithm for Premise Parameters

To identify the premise parameters of a Takagi-Sugeno fuzzy model described by equation (1), we used the Possibilistic Fuzzy C-Means (PFCM) algorithm and our proposed algorithms (RKPFCM and RKPFCM-PSO).


### 3.1. Possibilistic FuzzyC-Means (PFCM) Algorithm

The Possibilistic FuzzyC-Means (PFCM) algorithm, which uses Euclidean distance, finds the partition of the collection X={x1,…,xN}⊂Rp of N measures, specified by k-dimensional vectors xi=x1k,x2k,…,xikT, into C fuzzy subsets by minimizing the following objective function [18]:(6)JPFCMU,T,V=∑i=1C∑k=1Naμikm+btikDik2+∑i=1Cηi∑k=1Ntiklog⁡tik-tik,where 1≤C≤N: the number of clusters;μik: the membership of xk in cluster i satisfying (7)0≤μik≤1,∑i=1Cμik=11≤k≤N;tik: the typicality of xk in classes i; (8)D2ik=xk-vi2;V: the set of cluster centers (vi∈Rp);ηi: the suitable positive numbers described by(9)ηi=K∑k=1Nμik,FCMmD2ik∑k=1Nμik,FCMm,K>0.Typically,K is chosen to be 1.μik,FCM are the terminal membership values of FCM.m is a weighting degree; this parameter has a significant impact on the form of clusters in data space.To minimize equation (6), we take its partial derivative of variables, μik,tik, and vi, equal to zero and obtain the following equations:(10)μik=∑j=1CDikDjk2/m-1-1,(11)tik=exp⁡-bDik2ηi,∀i,k,(12)vi=∑k=1Naμikm+btikxk∑k=1Naμikm+btik,∀i.PFCM Algorithm Steps. Given a set of observations X=x1,…,xNT.Initialization (l=0)Set the number of clustersC,1≤C≤N.Set the level of weightingm: 2≤m<4.Set the parametersa, b.Set the stopping criterionε: ε>0.Execute a FCM clustering algorithm to find initial fuzzy partition matrixU and cluster centers V.Initialize the typicality matrixT randomly.Computeηi by (9).Repeat for
l
=
1,2
,
….Step 1.
Compute the cluster centers by (12).Step 2.
Compute the membership matrixU=μik by (10).Step 3.
Compute the typicality matrixT=tik by (11).Until
U
l
-
U
l
-
1
<
ε; then stop. Otherwise, set l=l+1 and return to Step  1.


### 3.2. Proposed Robust Kernel Possibilistic FuzzyC-Means (RKPFCM) Algorithm

The PFCM can deal with noisy data better than FCM and PCM; nevertheless, these conventional clustering algorithms become more effective when applied on linearly separable data or with a reasonable quantity of errors. In reality, the linearly separable data are rare. Therefore, FCM, PCM, and PFCM share the same negative point in that they are unable to get good separation of data that are nonlinearly separable in input space. To correct the imperfections found in PFCM particularly the nonlinear separable problem, kernel [7] methods are regarded as the way of dealing with this problem. In this context, we proposed a new extension of Possibilistic Fuzzy C-Means algorithm based on kernel method (RKPFCM). The present work proposes a way of increasing the accuracy of the PFCM algorithm by exploiting hyper tangent kernel function to calculate the distance used in its objective function.The kernel function is defined as a generalization of the distance metric that measures the distance between two data points mapped into a future space in which the data are more clearly separable [12, 19–21].Define a nonlinear map asΦ:x→Φ(x)∈F, where x∈X,andF is the transformed feature space with higher or even infinite dimension. X denotes the data space mapped into F [20–22].The RKPFCM algorithm minimizes the following objective function:(13)JRKPFCMU,T,V=∑i=1C∑k=1Naμikm+btikϕxk-ϕvi2+∑i=1Cηi∑k=1Ntiklog⁡tik-tik.Then xk-vi is mapped into space F [22]:(14)xk-vi⟶ϕxk-ϕvi,where(15)ϕxk-ϕvi2=ϕxk-ϕvi·ϕxk-ϕvi=ϕxk·ϕxk+ϕvi·ϕvi-2ϕxk·ϕvi=Kxk,xk+Kvi,vi-2Kkk,vi.K(xk,vi)=ΦxkTΦ(vi) is an inner product kernel function.If we adopt the hyper tangent kernel function, that is,(16)Kxk,vi=1-tanh⁡-xk-vi2σ2whereσ>0,then K(xk,xk)=K(vi,vi)=1. Thus (15) can be written as(17)ϕxk-ϕvi2=21-Kxk,vi=2tanh⁡-xk-vi2σ2.Considering (17), the objective function (13) is transformed as follows: (18)JRKPFCMU,T,V=2∑i=1C∑k=1Naμikm+btiktanh⁡-xk-vi2σ2+∑i=1Cηi∑k=1Ntiklog⁡tik-tik.The derivation of the objective function (18) according to μik, ti, and vi, defines the relationship update of cluster centers and membership coefficients.(i) Derivative of
J
RKPFCM
(
U
,
T
,
V
) with respect to
v
i.(19)∂JRKPFCMU,T,V∂vi=2∑k=1Naμikm+btik1-tanh2⁡-xk-vi2σ2xk-viσ2=2∑k=1Naμikm+btik1-tanh⁡-xk-vi2σ21+tanh⁡-xk-vi2σ2xk-viσ2.So,(20)∂JRKPFCMU,T,V∂vi=2∑k=1Naμikm+btikKxk,vi1+tanh⁡-xk-vi2σ2xk-viσ2.Equating (20) to zero leads to(21)∂JRKPFCMU,T,V∂vi=0.Then, (22)vi=∑k=1Naμikm+btikKxk,vi1+tanh⁡-xk-vi2/σ2xk∑k=1Naμikm+btikKxk,vi1+tanh⁡-xk-vi2/σ2.(ii) Derivative of
J
RKPFCM
(
U
,
T
,
V
) with respect to
μ
i
k. In this part we used the Lagrange multiplier(23)JRKPFCMU,P,V=2∑i=1C∑k=1Naμikm+btiktanh⁡-xk-vi2σ2+∑i=1Cηi∑k=1Ntiklog⁡tik-tik-∑k=1Nλk·∑i=1Cμik-1,(24)∂JRKPFCMU,T,V∂λ=-∑i=1Cμik-1=0,(25)∂JRKPFCMU,T,V∂μik=2maμikm-1tanh⁡-xk-vi2σ2-λ=0.From expression (25), we can write μik in this form:(26)μik=λ2matanh⁡-xk-vi2/σ21/m-1.Substituting expression (26) in expression (24): (27)∂JRKPFCMU,T,V∂λ=∑j=1Cμjk=∑j=1Cλ2matanh⁡-xk-vj2/σ21/m-1=1.It is also(28)λk2ma1/m-1=1∑j=1C1/tanh⁡-xk-vj2/σ21/m-1.The two expressions (26) and (28) give the following expression:(29)μik=1∑j=1Ctanh⁡-xk-vi2/σ2/tanh⁡-xk-vj2/σ21/m-1.Therefore the updating relationship is(30)μik=∑j=1C1-Kxk,vi1-Kxk,vj1/m-1-1.(iii) Derivative of
J
RKPFCM
(
U
,
T
,
V
) with respect to
t
i
k.(31)∂JRKPFCMU,T,V∂tik=2btanh⁡-xk-vi2σ2+ηilog⁡tik=0,(32)tik=exp⁡-2btanh⁡-xk-vi2/σ2ηi,∀i,k.Therefore, the updating typicality matrix is(33)tik=exp⁡-2b1-Kxk,viηi,∀i,k.Similarly (9) is rewritten by(34)ηi=K∑k=1Nμik,FCMm21-Kxk,vi∑k=1Nμik,FCMm.RKPFCM Algorithm Steps. Given a set of observations X=x1,…,xNT.Initialization (l = 0)Set the number of clustersC,1≤C≤N.Set the level of weightingm: 2≤m<4.Set the parametersa, b, and σ.Set the stopping criterionε: ε>0.Execute a FCM clustering algorithm to find initial fuzzy partition matrixU and cluster centers V.Initialize the typicality matrixT randomly.Computeηi  by (34).Repeat for
l
=
1,2
,
….Step 1.
Compute the cluster centers by (22).Step 2.
Compute the membership matrixU=μik by (30).Step 3.
Compute the typicality matrixT=tik by (33).Until
U
l
-
U
l
-
1
<
ε; then stop. Otherwise, set l=l+1 and return to Step  1.


### 3.3. Robust Kernel Possibilistic FuzzyC-Means Algorithm Based on PSO (RKPFCM-PSO)

#### 3.3.1. PSO

The Particle Swarm Optimization is a heuristic search method proposed by Kennedy and Eberhart (1995). This technique uses random population solution particles to find an optimal solution to problems. Each particle moves in the search space with a dynamically adjusted position and velocity for the best solution. The particle is characterized by data structure that contains the coordinates of the current position in the search space, the best solution point visited so far, and the subset of other agents that are seen as neighbors. These adjustments are based on the historical behaviors of itself and other agents in the swarm. The change of speed (acceleration) and the position of each particle in the optimization landscape (search space) are iteratively [6, 12, 23](35)vkl+1=Kwvkl+ρ1pkl-xkl+ρ2pgl-xkl,xkl+1=xkl+vkl+1,where k=1,…,NP: size of particles;D: size of the landscape (search space);vk=(vk1,…,vkd,…,vkD): the speed of particle;xk=xk1,xk1,…,xkd,…,xkD: the position of particle;pk=pk1,pk1,…,pkd,…,pkD: the best previous position of particle;g: index represents the best particle among all particles in the group;K: the constriction factor described by the following relationship:(36)K=22-φ-φ2-4φ,whereφ=c1+c2,where
c1 and c2 are two positive constants satisfying the following relationship: (37)φ=c1+c2>4;ρ1 and ρ2: random variables defined as follows: (38)ρ1=r1×c1,ρ2=r2×c2,where
r1 and r2 are two random variables between 0 and 1;w: the weight of inertia according to this equation:(39)w=wmax-wmax-wminitermax×iter,wherewmax and wmin are the initial and final weight, itermax is the maximum iterations, and iter is the current iteration number.


#### 3.3.2. Fitness Function

The fitness function defines our optimization problem described by the following expression:(40)Fitness=GJRKPFCMU,T,V,where G is a positive constant.JRKPFCM(U,T,V) represents the objective function of the RKPFCM algorithm.RKPFCM-PSO Algorithm.Given a set of observations X=x1,x2,…,xNT, the RKPFCM-PSO algorithm is described by the following steps.Initialization (l=0)Select the number of clustersC, fuzzy degree m, the parametersaandb, the population size NP, the constants c1 and c2, the random variables r1 and r2, the weight of inertia wmax and wmin, the size of the search space D, the constant G, and the stopping criterion ε.Set the 1st particle generation clusters centers.Initialize the fitness function and speed of each particle.Computeηi  by (34).Repeat
l = l + 1Step 1.
Compute the fuzzy partition matrixU=μik by (30).Step 2.
Compute the typicality matrixT=tik by (33).Step 3.
Calculate the new value of fitness for each particle using (40).Step 4.
Compare the fitness of each particle withpbest, if the value is better than pbest and then set the pbest value.Step 5.
Compare the fitness value ofgbest with the following: if the value is better than gbest, gbest then is set equal to this value.Step 6.
Update position and speed of each particle by (35).So this algorithm is converged whenvk(l+1)-vk(l)<ε; that is to say, stop iteration and find the best solution in the last generation. If not, go back to Step1.


## 3.1. Possibilistic FuzzyC-Means (PFCM) Algorithm

The Possibilistic FuzzyC-Means (PFCM) algorithm, which uses Euclidean distance, finds the partition of the collection X={x1,…,xN}⊂Rp of N measures, specified by k-dimensional vectors xi=x1k,x2k,…,xikT, into C fuzzy subsets by minimizing the following objective function [18]:(6)JPFCMU,T,V=∑i=1C∑k=1Naμikm+btikDik2+∑i=1Cηi∑k=1Ntiklog⁡tik-tik,where 1≤C≤N: the number of clusters;μik: the membership of xk in cluster i satisfying (7)0≤μik≤1,∑i=1Cμik=11≤k≤N;tik: the typicality of xk in classes i; (8)D2ik=xk-vi2;V: the set of cluster centers (vi∈Rp);ηi: the suitable positive numbers described by(9)ηi=K∑k=1Nμik,FCMmD2ik∑k=1Nμik,FCMm,K>0.Typically,K is chosen to be 1.μik,FCM are the terminal membership values of FCM.m is a weighting degree; this parameter has a significant impact on the form of clusters in data space.To minimize equation (6), we take its partial derivative of variables, μik,tik, and vi, equal to zero and obtain the following equations:(10)μik=∑j=1CDikDjk2/m-1-1,(11)tik=exp⁡-bDik2ηi,∀i,k,(12)vi=∑k=1Naμikm+btikxk∑k=1Naμikm+btik,∀i.PFCM Algorithm Steps. Given a set of observations X=x1,…,xNT.Initialization (l=0)Set the number of clustersC,1≤C≤N.Set the level of weightingm: 2≤m<4.Set the parametersa, b.Set the stopping criterionε: ε>0.Execute a FCM clustering algorithm to find initial fuzzy partition matrixU and cluster centers V.Initialize the typicality matrixT randomly.Computeηi by (9).Repeat for
l
=
1,2
,
….Step 1.
Compute the cluster centers by (12).Step 2.
Compute the membership matrixU=μik by (10).Step 3.
Compute the typicality matrixT=tik by (11).Until
U
l
-
U
l
-
1
<
ε; then stop. Otherwise, set l=l+1 and return to Step  1.


## 3.2. Proposed Robust Kernel Possibilistic FuzzyC-Means (RKPFCM) Algorithm

The PFCM can deal with noisy data better than FCM and PCM; nevertheless, these conventional clustering algorithms become more effective when applied on linearly separable data or with a reasonable quantity of errors. In reality, the linearly separable data are rare. Therefore, FCM, PCM, and PFCM share the same negative point in that they are unable to get good separation of data that are nonlinearly separable in input space. To correct the imperfections found in PFCM particularly the nonlinear separable problem, kernel [7] methods are regarded as the way of dealing with this problem. In this context, we proposed a new extension of Possibilistic Fuzzy C-Means algorithm based on kernel method (RKPFCM). The present work proposes a way of increasing the accuracy of the PFCM algorithm by exploiting hyper tangent kernel function to calculate the distance used in its objective function.The kernel function is defined as a generalization of the distance metric that measures the distance between two data points mapped into a future space in which the data are more clearly separable [12, 19–21].Define a nonlinear map asΦ:x→Φ(x)∈F, where x∈X,andF is the transformed feature space with higher or even infinite dimension. X denotes the data space mapped into F [20–22].The RKPFCM algorithm minimizes the following objective function:(13)JRKPFCMU,T,V=∑i=1C∑k=1Naμikm+btikϕxk-ϕvi2+∑i=1Cηi∑k=1Ntiklog⁡tik-tik.Then xk-vi is mapped into space F [22]:(14)xk-vi⟶ϕxk-ϕvi,where(15)ϕxk-ϕvi2=ϕxk-ϕvi·ϕxk-ϕvi=ϕxk·ϕxk+ϕvi·ϕvi-2ϕxk·ϕvi=Kxk,xk+Kvi,vi-2Kkk,vi.K(xk,vi)=ΦxkTΦ(vi) is an inner product kernel function.If we adopt the hyper tangent kernel function, that is,(16)Kxk,vi=1-tanh⁡-xk-vi2σ2whereσ>0,then K(xk,xk)=K(vi,vi)=1. Thus (15) can be written as(17)ϕxk-ϕvi2=21-Kxk,vi=2tanh⁡-xk-vi2σ2.Considering (17), the objective function (13) is transformed as follows: (18)JRKPFCMU,T,V=2∑i=1C∑k=1Naμikm+btiktanh⁡-xk-vi2σ2+∑i=1Cηi∑k=1Ntiklog⁡tik-tik.The derivation of the objective function (18) according to μik, ti, and vi, defines the relationship update of cluster centers and membership coefficients.(i) Derivative of
J
RKPFCM
(
U
,
T
,
V
) with respect to
v
i.(19)∂JRKPFCMU,T,V∂vi=2∑k=1Naμikm+btik1-tanh2⁡-xk-vi2σ2xk-viσ2=2∑k=1Naμikm+btik1-tanh⁡-xk-vi2σ21+tanh⁡-xk-vi2σ2xk-viσ2.So,(20)∂JRKPFCMU,T,V∂vi=2∑k=1Naμikm+btikKxk,vi1+tanh⁡-xk-vi2σ2xk-viσ2.Equating (20) to zero leads to(21)∂JRKPFCMU,T,V∂vi=0.Then, (22)vi=∑k=1Naμikm+btikKxk,vi1+tanh⁡-xk-vi2/σ2xk∑k=1Naμikm+btikKxk,vi1+tanh⁡-xk-vi2/σ2.(ii) Derivative of
J
RKPFCM
(
U
,
T
,
V
) with respect to
μ
i
k. In this part we used the Lagrange multiplier(23)JRKPFCMU,P,V=2∑i=1C∑k=1Naμikm+btiktanh⁡-xk-vi2σ2+∑i=1Cηi∑k=1Ntiklog⁡tik-tik-∑k=1Nλk·∑i=1Cμik-1,(24)∂JRKPFCMU,T,V∂λ=-∑i=1Cμik-1=0,(25)∂JRKPFCMU,T,V∂μik=2maμikm-1tanh⁡-xk-vi2σ2-λ=0.From expression (25), we can write μik in this form:(26)μik=λ2matanh⁡-xk-vi2/σ21/m-1.Substituting expression (26) in expression (24): (27)∂JRKPFCMU,T,V∂λ=∑j=1Cμjk=∑j=1Cλ2matanh⁡-xk-vj2/σ21/m-1=1.It is also(28)λk2ma1/m-1=1∑j=1C1/tanh⁡-xk-vj2/σ21/m-1.The two expressions (26) and (28) give the following expression:(29)μik=1∑j=1Ctanh⁡-xk-vi2/σ2/tanh⁡-xk-vj2/σ21/m-1.Therefore the updating relationship is(30)μik=∑j=1C1-Kxk,vi1-Kxk,vj1/m-1-1.(iii) Derivative of
J
RKPFCM
(
U
,
T
,
V
) with respect to
t
i
k.(31)∂JRKPFCMU,T,V∂tik=2btanh⁡-xk-vi2σ2+ηilog⁡tik=0,(32)tik=exp⁡-2btanh⁡-xk-vi2/σ2ηi,∀i,k.Therefore, the updating typicality matrix is(33)tik=exp⁡-2b1-Kxk,viηi,∀i,k.Similarly (9) is rewritten by(34)ηi=K∑k=1Nμik,FCMm21-Kxk,vi∑k=1Nμik,FCMm.RKPFCM Algorithm Steps. Given a set of observations X=x1,…,xNT.Initialization (l = 0)Set the number of clustersC,1≤C≤N.Set the level of weightingm: 2≤m<4.Set the parametersa, b, and σ.Set the stopping criterionε: ε>0.Execute a FCM clustering algorithm to find initial fuzzy partition matrixU and cluster centers V.Initialize the typicality matrixT randomly.Computeηi  by (34).Repeat for
l
=
1,2
,
….Step 1.
Compute the cluster centers by (22).Step 2.
Compute the membership matrixU=μik by (30).Step 3.
Compute the typicality matrixT=tik by (33).Until
U
l
-
U
l
-
1
<
ε; then stop. Otherwise, set l=l+1 and return to Step  1.


## 3.3. Robust Kernel Possibilistic FuzzyC-Means Algorithm Based on PSO (RKPFCM-PSO)

### 3.3.1. PSO

The Particle Swarm Optimization is a heuristic search method proposed by Kennedy and Eberhart (1995). This technique uses random population solution particles to find an optimal solution to problems. Each particle moves in the search space with a dynamically adjusted position and velocity for the best solution. The particle is characterized by data structure that contains the coordinates of the current position in the search space, the best solution point visited so far, and the subset of other agents that are seen as neighbors. These adjustments are based on the historical behaviors of itself and other agents in the swarm. The change of speed (acceleration) and the position of each particle in the optimization landscape (search space) are iteratively [6, 12, 23](35)vkl+1=Kwvkl+ρ1pkl-xkl+ρ2pgl-xkl,xkl+1=xkl+vkl+1,where k=1,…,NP: size of particles;D: size of the landscape (search space);vk=(vk1,…,vkd,…,vkD): the speed of particle;xk=xk1,xk1,…,xkd,…,xkD: the position of particle;pk=pk1,pk1,…,pkd,…,pkD: the best previous position of particle;g: index represents the best particle among all particles in the group;K: the constriction factor described by the following relationship:(36)K=22-φ-φ2-4φ,whereφ=c1+c2,where
c1 and c2 are two positive constants satisfying the following relationship: (37)φ=c1+c2>4;ρ1 and ρ2: random variables defined as follows: (38)ρ1=r1×c1,ρ2=r2×c2,where
r1 and r2 are two random variables between 0 and 1;w: the weight of inertia according to this equation:(39)w=wmax-wmax-wminitermax×iter,wherewmax and wmin are the initial and final weight, itermax is the maximum iterations, and iter is the current iteration number.


### 3.3.2. Fitness Function

The fitness function defines our optimization problem described by the following expression:(40)Fitness=GJRKPFCMU,T,V,where G is a positive constant.JRKPFCM(U,T,V) represents the objective function of the RKPFCM algorithm.RKPFCM-PSO Algorithm.Given a set of observations X=x1,x2,…,xNT, the RKPFCM-PSO algorithm is described by the following steps.Initialization (l=0)Select the number of clustersC, fuzzy degree m, the parametersaandb, the population size NP, the constants c1 and c2, the random variables r1 and r2, the weight of inertia wmax and wmin, the size of the search space D, the constant G, and the stopping criterion ε.Set the 1st particle generation clusters centers.Initialize the fitness function and speed of each particle.Computeηi  by (34).Repeat
l = l + 1Step 1.
Compute the fuzzy partition matrixU=μik by (30).Step 2.
Compute the typicality matrixT=tik by (33).Step 3.
Calculate the new value of fitness for each particle using (40).Step 4.
Compare the fitness of each particle withpbest, if the value is better than pbest and then set the pbest value.Step 5.
Compare the fitness value ofgbest with the following: if the value is better than gbest, gbest then is set equal to this value.Step 6.
Update position and speed of each particle by (35).So this algorithm is converged whenvk(l+1)-vk(l)<ε; that is to say, stop iteration and find the best solution in the last generation. If not, go back to Step1.


## 3.3.1. PSO

The Particle Swarm Optimization is a heuristic search method proposed by Kennedy and Eberhart (1995). This technique uses random population solution particles to find an optimal solution to problems. Each particle moves in the search space with a dynamically adjusted position and velocity for the best solution. The particle is characterized by data structure that contains the coordinates of the current position in the search space, the best solution point visited so far, and the subset of other agents that are seen as neighbors. These adjustments are based on the historical behaviors of itself and other agents in the swarm. The change of speed (acceleration) and the position of each particle in the optimization landscape (search space) are iteratively [6, 12, 23](35)vkl+1=Kwvkl+ρ1pkl-xkl+ρ2pgl-xkl,xkl+1=xkl+vkl+1,where k=1,…,NP: size of particles;D: size of the landscape (search space);vk=(vk1,…,vkd,…,vkD): the speed of particle;xk=xk1,xk1,…,xkd,…,xkD: the position of particle;pk=pk1,pk1,…,pkd,…,pkD: the best previous position of particle;g: index represents the best particle among all particles in the group;K: the constriction factor described by the following relationship:(36)K=22-φ-φ2-4φ,whereφ=c1+c2,where
c1 and c2 are two positive constants satisfying the following relationship: (37)φ=c1+c2>4;ρ1 and ρ2: random variables defined as follows: (38)ρ1=r1×c1,ρ2=r2×c2,where
r1 and r2 are two random variables between 0 and 1;w: the weight of inertia according to this equation:(39)w=wmax-wmax-wminitermax×iter,wherewmax and wmin are the initial and final weight, itermax is the maximum iterations, and iter is the current iteration number.


## 3.3.2. Fitness Function

The fitness function defines our optimization problem described by the following expression:(40)Fitness=GJRKPFCMU,T,V,where G is a positive constant.JRKPFCM(U,T,V) represents the objective function of the RKPFCM algorithm.RKPFCM-PSO Algorithm.Given a set of observations X=x1,x2,…,xNT, the RKPFCM-PSO algorithm is described by the following steps.Initialization (l=0)Select the number of clustersC, fuzzy degree m, the parametersaandb, the population size NP, the constants c1 and c2, the random variables r1 and r2, the weight of inertia wmax and wmin, the size of the search space D, the constant G, and the stopping criterion ε.Set the 1st particle generation clusters centers.Initialize the fitness function and speed of each particle.Computeηi  by (34).Repeat
l = l + 1Step 1.
Compute the fuzzy partition matrixU=μik by (30).Step 2.
Compute the typicality matrixT=tik by (33).Step 3.
Calculate the new value of fitness for each particle using (40).Step 4.
Compare the fitness of each particle withpbest, if the value is better than pbest and then set the pbest value.Step 5.
Compare the fitness value ofgbest with the following: if the value is better than gbest, gbest then is set equal to this value.Step 6.
Update position and speed of each particle by (35).So this algorithm is converged whenvk(l+1)-vk(l)<ε; that is to say, stop iteration and find the best solution in the last generation. If not, go back to Step1.


## 4. Identification for Consequent Parameters

The defuzzification method, used in the Takagi-Sugeno fuzzy model, is linear with the consequent parametersθi=aiT,bi which can be obtained as a solution of a weighted least squares problem according to the following equation:(41)θi=XeTψiXe-1XeTψiY,where Xe=X;1 represents an extension of regression matrix;X=X1T,X2T,…,XNTT;Y=y1,y2,…,yNT is the output vector;ψi is a diagonal matrix of dimension (N × N) containing the coefficients μik of fuzzy memberships.The RKPFCM and RKPFCM-PSO clustering algorithm are used to find width of the membership functions by the following equation [24]:(42)σij=2∗∑k=1Nμikxjk-vij2∑k=1Nμik.


## 5. Simulation Results and Validation Model

### 5.1. Identical Data with Noise

In this example we have used 12 data which are composed of 10 models and two noises; this data set (X12) is presented in [25]. The FCM, PCM, PFCM, KPFCM, and our algorithms RKPFCM and RKPFCM-PSO were used in clustering the data set in tow groups (C=2).In this example the parameters settings area=1, b=2, m=2, G=10, NP=20, D=2, wmax = 0.9, wmin = 0.4, c1=2.05, c2=2.05,  σ = 20, itermax=1000, and  ε = 10−9.Figure1 shows the clustering results for our proposed method RKPFCM-PSO. The Ideal (true) centroids are(43)Videal=-3.3403.340.Table 1 shows the results of center clusters using the six algorithms. The error between the results prototypes and ideal center clusters is calculated by the next expression:(44)E∗=Videal-V∗2,where ∗ is the FCM, PCM, PFCM, KPFCM, RKPFCM, and RKPFCM-PSO.Table 1
Prototypes results of clusters centers.
Algorithms
CentersV∗
Error (E∗)
FCM [14]
-
2.98
0.54
2.98
0.54
0.414
PCM [14]
-
2.15
0.02
2.15
0.02
1.416
PFCM [14]
-
2.84
0.36
2.84
0.36
0.3796
KPFCM [14]
-
3.33
0.03
3.33
0.03
0.0005
RKPFCM
-
3.337
0.002
3.337
0.002
1.79
×
10
-
5
RKPFCM-PSO
-
3.3385
0.0001
3.3385
0.0001
4.23
×
10
-
6Figure 1
Clustering by RKPFCM-PSO algorithm.According to Table1, our proposed algorithm RKPFCM -PSO gives the best prototypes of centers.Figure1 shows the effectiveness of our approach as well.


### 5.2. Identification of T-S Fuzzy Model

After applying the identification algorithm, it is necessary to validate the Takagi-Sugeno fuzzy model. Several validation tests of the model are used. Among them, we cited the Mean Square Error (MSE) test, Root Mean Square Error (RMSE), and the Variance Accounting For (VAF) test.(45)MSE=1N∑k=1Nyk-yestk2,RMSE=1N∑k=1Nyk-yestk2,VAF=100%1-vary-yestvary, where “y” is the real output and “yest” is the estimated output.


#### 5.2.1. Example  1

Consider a nonlinear system described by the following difference equation [15]:(46)yk=y2k-3+y2k-2+y2k-1+tanh⁡uk1+y2k-1+y2k-2+y2k-3+ek,where y(k) and u(k) are the output and the input of the system, respectively.e
(
k
) is a noise. (47)uk=0.6sin⁡3πkTs+0.2sin⁡4πkTs+1.2sin⁡πkTs.Ts = 0.01.200 samples were generated by simulation and were used, where the selected input variables are chosen{y(k-1), y(k-2), y(k-3), u(k)}.The complete data set has been used to train the model. The noise influence is analyzed with different SNR levels (SNR = 10 dB and SNR = 5 dB).In this part, we have applied various algorithms and our proposed clustering RKPFCM and RKPFCM-PSO which approximate the nonlinear model (46).The used parameters area=1, b=3, m=2, G=10, NP = 30, D=4, wmax=0.9, wmin=0.4, c1=2.05, c2=2.05, σ = 20, itermax=1000, and ε = 10−9.The shape of the excitation signal used for identification is illustrated in Figure2.Figure 2
Input-output sequences.The simulation result given by the RKPFCM-PSO algorithm is illustrated in Figure3.Figure 3
Identification results for the RKPFCM-PSO algorithm with SNR = 10 DB.Table2 shows the various modeling performance results without noise obtained by different algorithms; this comparison results demonstrate that the best MSE and best VAF are obtained by the proposed methods (RKPFCM and RKPFCM-PSO).Table 2
Comparison results without noise.
Algorithms
Number of rules
MSE
VAF (%)
Cluster fuzzy [15]
6
1.9 × 10−3
—
Kalman cluster fuzzy [15]
2
2.2 × 10−5
—
FCM
2
2.1422 × 10−5
99.9581
GK
2
2.1142 × 10−5
99.9592
KFCM
2
2.1011 × 10−5
99.9601
PFCM
2
2.0152 × 10−5
99.9611
RKPFCM
2
1.8498 × 10−5
99.9723
RKPFCM-PSO
2
1.8412 × 10−5
99.9742Tables3 and 4 present the various modeling performance results with noise influence (SNR = 5 dB and 10 dB) obtained by the different algorithms. However, our proposed algorithm RKPFCM-PSO retained the best performance with a higher level of noise.Table 3
Comparison results with SNR = 10 dB.
Algorithms
Number of rules
MSE (10−4)
VAF (%)
FCM
2
9.8397
98.0345
GK
2
9.8831
98.0303
KFCM
2
9.9492
98.0289
PFCM
2
9.8313
98.0362
RKPFCM
2
8.5158
98.1628
RKPFCM-PSO
2
8.5149
98.1637Table 4
Comparison results with SNR = 5 dB.
Algorithms
Number of rules
MSE (10−3)
VAF (%)
FCM
2
1.7594
96.0655
GK
2
1.7552
96.0668
KFCM
2
1.7572
96.0659
PFCM
2
1.7534
96.0783
RKPFCM
2
1,7424
96.1462
RKPFCM-PSO
2
1,73310
96.1510The local linear models identified are given as follows:(48)y1k=0.3167yk-1+0.0263yk-2-0.0482yk-3+0.1183k+0.6916,y2k=0.4045yk-1+0.0665yk-2-0.0304yk-2+0.1519uk+0.5417.


#### 5.2.2. Example  2

Consider a highly complex modified nonlinear system described by the following difference equation [16]:(49)yk=yk-1yk-2+2yk-1+2.510+y2k-1+y2k-2+uk+ek,where y(k) is the model output and u(k) is the model input which is bounded between [−1+1]. The e(k) is a noise.The following input signal is expressed as(50)uk=sin2kπ250k≤5000.8∗sin⁡2kπ250+0.2∗sin⁡2kπ25otherwise.1500 samples were generated by simulation in which 1000 samples were used to train the model. Fuzzy model parameters have been identified once, testing of model was done by the remaining 500 samples, and {y(k-1), y(k-2), u(k), u(k-1)} are chosen as input variables. In this example the parameters settings are a=1, b=2, m=2, G=10, NP = 30, D=4, wmax = 0.9, wmin = 0.4, c1=2, c2=2.1, σ = 20, itermax=1000, and ε = 10−9.The noise influence is analyzed with different SNR levels (SNR = 20 dB, SNR = 10 dB, SNR =5 dB, and SNR = 1 dB).The obtained identification results by RKPFCM-PSO are, respectively, shown in Figures4 and 5.Figure 4
Input-output sequences.Figure 5
RKPFCM-PSO performances for test data with SNR = 10 dB.The evaluation performance index (RMSE-trn and RMSE-test) stands for training and testing data, respectively.Tables5–8 show the comparative performance of RKPFCM and RKPFCM-PSO with different existing algorithms such as FCM, G-K, Fuzzy Model Identification (FMI), FCRM, and MFCRM-NC. It is clearly seen from the results that our algorithm RKPFCM-PSO gives the best performance in noisy environments. The local linear models identified by RKPFCM-PSO are given as follows:(51)y1k=0.8562yk-1+0.0548yk-2+2.0967uk-1.6609uk-1+0.0814,y2k=0.8058yk-1-0.0924yk-2+1.3488uk-0.3718uk-1+0.8032,y3k=0.5673yk-1+0.1896yk-2+1.2330uk-0.8613uk-1+0.0520,y4k=0.8881yk-1-0.0571yk-2+1.5400uk-0.9195uk-1+0.4536.Table 5
20 dB Noise.
Algorithms
RMSE-trn
RMSE-test
Number of rules
FCM [16]
0.1393
0.3004
4
GK [16]
0.1326
0.1484
4
FMI [16]
0.1457
0.2378
4
FCRM [16]
0.1291
0.1495
4
MFCRM-NC [16]
0.1183
0.1395
4
RKPFCM
0.1167
0.1377
4
RKPFCM-PSO
0.1165
0.1370
4Table 6
10 dB Noise.
Algorithms
RMSE-trn
RMSE-test
Number of rules
FCM [16]
0.3453
0.7150
4
GK [16]
0.3330
0.3489
4
FMI [16]
0.3956
0.4416
4
FCRM [16]
0.3312
0.3694
4
MFCRM-NC [16]
0.3233
0.3465
4
RKPFCM
0.3199
0.3405
4
RKPFCM-PSO
0.3186
0.3384
4Table 7
5 dB Noise.
Algorithms
RMSE-trn
RMSE-test
Number of rules
FCM [16]
0.6170
1.0364
4
GK [16]
0.5855
0.6505
4
FMI [16]
0.6274
0.7021
4
FCRM [16]
0.5805
0.6485
4
MFCRM-NC [16]
0.5658
0.5998
4
RKPFCM
0.4958
0.5921
4
RKPFCM-PSO
0.4908
0.5908
4Table 8
1 dB Noise.
Algorithms
RMSE-trn
RMSE-test
Number of rules
FCM [16]
0.9330
1.3646
4
GK [16]
0.8932
1.0226
4
FMI [16]
0.8953
1.0320
4
FCRM [16]
0.8872
1.0095
4
MFCRM-NC [16]
0.8699
0.9947
4
RKPFCM
0.7600
0.8894
4
RKPFCM-PSO
0.7571
0.8809
4


#### 5.2.3. Example  3

We consider the Box Jenkins gas furnace data set which is used as a standard test for identification techniques. The data set is composed of 296 pairs of input-output measurements. The input “u” is the gas flow rate into a furnace and the output “y” is the CO2 concentration in the outlet gases. In order to take all the above-mentioned issues into account, we simulated the following experimental case [16]:all the 296 data pairs are used as training data and{y(k-1), u(k-4)} are selected as input variables to various algorithms, while we use two rules (C=2). In this example the used parameters are a=1, b=2, m=2, G=10, NP = 30, D=3, wmax = 0.9, wmin = 0.4, c1=1.95, c2 = 2.1,σ = 10, itermax=1000, and ε = 10−9.The simulation result given by the RKPFCM-PSO algorithm is illustrated in Figure 6.Figure 6
Identification results for the RKPFCM-PSO Algorithm (SNR = 10 dB).Based on the comparison presented in Table9, it is clear that the proposed algorithm RKPFCM-PSO is more robust to noise than the other algorithms found in literature.Table 9
Comparison results for Box Jenkins system.
Algorithms
Number of inputs
Number of rules
MSE
Tong (1980) [16]
2
19
0.469
Pedrycz (1984) [16]
2
81
0.320
Xu (1987) [16]
2
25
0.328
Sugeno (1991) [16]
2
2
0.359
Yoshinari (1993) [16]
2
6
0.299
Joo (1997) [16]
2
6
0.166
Zhang (2006) [16]
2
2
0.160
Glowaty (2008) [16]
2
2
0.391
Andri (2011) [16]
2
10
0.167
Tanmoy Dam (2015) [16]
Without noise
2
2
0.152
SNR = 30 dB
2
2
0.153
SNR = 10 dB
2
2
0.277
RKPFCM
Without noise
2
2
0.1458
SNR = 30 dB
2
2
0.1512
SNR = 10 dB
2
2
0.2342
SNR = 5 dB
2
2
0.2922
SNR = 2 dB
2
2
0.3014
RKPFCM-PSO
Without noise
2
2
0.1455
SNR = 30 dB
2
2
0.1503
SNR = 10 dB
2
2
0.2338
SNR = 5 dB
2
2
0.2902
SNR = 2 dB
2
2
0.3001When we use our algorithm RKPFCM-PSO the local linear models identified are given as follows:(52)y1k=0.6777yk-1-0.9593uk-4+17.3007,y2k=0.5560yk-1-1.3297uk-4+23.5880.


## 5.1. Identical Data with Noise

In this example we have used 12 data which are composed of 10 models and two noises; this data set (X12) is presented in [25]. The FCM, PCM, PFCM, KPFCM, and our algorithms RKPFCM and RKPFCM-PSO were used in clustering the data set in tow groups (C=2).In this example the parameters settings area=1, b=2, m=2, G=10, NP=20, D=2, wmax = 0.9, wmin = 0.4, c1=2.05, c2=2.05,  σ = 20, itermax=1000, and  ε = 10−9.Figure1 shows the clustering results for our proposed method RKPFCM-PSO. The Ideal (true) centroids are(43)Videal=-3.3403.340.Table 1 shows the results of center clusters using the six algorithms. The error between the results prototypes and ideal center clusters is calculated by the next expression:(44)E∗=Videal-V∗2,where ∗ is the FCM, PCM, PFCM, KPFCM, RKPFCM, and RKPFCM-PSO.Table 1
Prototypes results of clusters centers.
Algorithms
CentersV∗
Error (E∗)
FCM [14]
-
2.98
0.54
2.98
0.54
0.414
PCM [14]
-
2.15
0.02
2.15
0.02
1.416
PFCM [14]
-
2.84
0.36
2.84
0.36
0.3796
KPFCM [14]
-
3.33
0.03
3.33
0.03
0.0005
RKPFCM
-
3.337
0.002
3.337
0.002
1.79
×
10
-
5
RKPFCM-PSO
-
3.3385
0.0001
3.3385
0.0001
4.23
×
10
-
6Figure 1
Clustering by RKPFCM-PSO algorithm.According to Table1, our proposed algorithm RKPFCM -PSO gives the best prototypes of centers.Figure1 shows the effectiveness of our approach as well.


## 5.2. Identification of T-S Fuzzy Model

After applying the identification algorithm, it is necessary to validate the Takagi-Sugeno fuzzy model. Several validation tests of the model are used. Among them, we cited the Mean Square Error (MSE) test, Root Mean Square Error (RMSE), and the Variance Accounting For (VAF) test.(45)MSE=1N∑k=1Nyk-yestk2,RMSE=1N∑k=1Nyk-yestk2,VAF=100%1-vary-yestvary, where “y” is the real output and “yest” is the estimated output.


### 5.2.1. Example  1

Consider a nonlinear system described by the following difference equation [15]:(46)yk=y2k-3+y2k-2+y2k-1+tanh⁡uk1+y2k-1+y2k-2+y2k-3+ek,where y(k) and u(k) are the output and the input of the system, respectively.e
(
k
) is a noise. (47)uk=0.6sin⁡3πkTs+0.2sin⁡4πkTs+1.2sin⁡πkTs.Ts = 0.01.200 samples were generated by simulation and were used, where the selected input variables are chosen{y(k-1), y(k-2), y(k-3), u(k)}.The complete data set has been used to train the model. The noise influence is analyzed with different SNR levels (SNR = 10 dB and SNR = 5 dB).In this part, we have applied various algorithms and our proposed clustering RKPFCM and RKPFCM-PSO which approximate the nonlinear model (46).The used parameters area=1, b=3, m=2, G=10, NP = 30, D=4, wmax=0.9, wmin=0.4, c1=2.05, c2=2.05, σ = 20, itermax=1000, and ε = 10−9.The shape of the excitation signal used for identification is illustrated in Figure2.Figure 2
Input-output sequences.The simulation result given by the RKPFCM-PSO algorithm is illustrated in Figure3.Figure 3
Identification results for the RKPFCM-PSO algorithm with SNR = 10 DB.Table2 shows the various modeling performance results without noise obtained by different algorithms; this comparison results demonstrate that the best MSE and best VAF are obtained by the proposed methods (RKPFCM and RKPFCM-PSO).Table 2
Comparison results without noise.
Algorithms
Number of rules
MSE
VAF (%)
Cluster fuzzy [15]
6
1.9 × 10−3
—
Kalman cluster fuzzy [15]
2
2.2 × 10−5
—
FCM
2
2.1422 × 10−5
99.9581
GK
2
2.1142 × 10−5
99.9592
KFCM
2
2.1011 × 10−5
99.9601
PFCM
2
2.0152 × 10−5
99.9611
RKPFCM
2
1.8498 × 10−5
99.9723
RKPFCM-PSO
2
1.8412 × 10−5
99.9742Tables3 and 4 present the various modeling performance results with noise influence (SNR = 5 dB and 10 dB) obtained by the different algorithms. However, our proposed algorithm RKPFCM-PSO retained the best performance with a higher level of noise.Table 3
Comparison results with SNR = 10 dB.
Algorithms
Number of rules
MSE (10−4)
VAF (%)
FCM
2
9.8397
98.0345
GK
2
9.8831
98.0303
KFCM
2
9.9492
98.0289
PFCM
2
9.8313
98.0362
RKPFCM
2
8.5158
98.1628
RKPFCM-PSO
2
8.5149
98.1637Table 4
Comparison results with SNR = 5 dB.
Algorithms
Number of rules
MSE (10−3)
VAF (%)
FCM
2
1.7594
96.0655
GK
2
1.7552
96.0668
KFCM
2
1.7572
96.0659
PFCM
2
1.7534
96.0783
RKPFCM
2
1,7424
96.1462
RKPFCM-PSO
2
1,73310
96.1510The local linear models identified are given as follows:(48)y1k=0.3167yk-1+0.0263yk-2-0.0482yk-3+0.1183k+0.6916,y2k=0.4045yk-1+0.0665yk-2-0.0304yk-2+0.1519uk+0.5417.


### 5.2.2. Example  2

Consider a highly complex modified nonlinear system described by the following difference equation [16]:(49)yk=yk-1yk-2+2yk-1+2.510+y2k-1+y2k-2+uk+ek,where y(k) is the model output and u(k) is the model input which is bounded between [−1+1]. The e(k) is a noise.The following input signal is expressed as(50)uk=sin2kπ250k≤5000.8∗sin⁡2kπ250+0.2∗sin⁡2kπ25otherwise.1500 samples were generated by simulation in which 1000 samples were used to train the model. Fuzzy model parameters have been identified once, testing of model was done by the remaining 500 samples, and {y(k-1), y(k-2), u(k), u(k-1)} are chosen as input variables. In this example the parameters settings are a=1, b=2, m=2, G=10, NP = 30, D=4, wmax = 0.9, wmin = 0.4, c1=2, c2=2.1, σ = 20, itermax=1000, and ε = 10−9.The noise influence is analyzed with different SNR levels (SNR = 20 dB, SNR = 10 dB, SNR =5 dB, and SNR = 1 dB).The obtained identification results by RKPFCM-PSO are, respectively, shown in Figures4 and 5.Figure 4
Input-output sequences.Figure 5
RKPFCM-PSO performances for test data with SNR = 10 dB.The evaluation performance index (RMSE-trn and RMSE-test) stands for training and testing data, respectively.Tables5–8 show the comparative performance of RKPFCM and RKPFCM-PSO with different existing algorithms such as FCM, G-K, Fuzzy Model Identification (FMI), FCRM, and MFCRM-NC. It is clearly seen from the results that our algorithm RKPFCM-PSO gives the best performance in noisy environments. The local linear models identified by RKPFCM-PSO are given as follows:(51)y1k=0.8562yk-1+0.0548yk-2+2.0967uk-1.6609uk-1+0.0814,y2k=0.8058yk-1-0.0924yk-2+1.3488uk-0.3718uk-1+0.8032,y3k=0.5673yk-1+0.1896yk-2+1.2330uk-0.8613uk-1+0.0520,y4k=0.8881yk-1-0.0571yk-2+1.5400uk-0.9195uk-1+0.4536.Table 5
20 dB Noise.
Algorithms
RMSE-trn
RMSE-test
Number of rules
FCM [16]
0.1393
0.3004
4
GK [16]
0.1326
0.1484
4
FMI [16]
0.1457
0.2378
4
FCRM [16]
0.1291
0.1495
4
MFCRM-NC [16]
0.1183
0.1395
4
RKPFCM
0.1167
0.1377
4
RKPFCM-PSO
0.1165
0.1370
4Table 6
10 dB Noise.
Algorithms
RMSE-trn
RMSE-test
Number of rules
FCM [16]
0.3453
0.7150
4
GK [16]
0.3330
0.3489
4
FMI [16]
0.3956
0.4416
4
FCRM [16]
0.3312
0.3694
4
MFCRM-NC [16]
0.3233
0.3465
4
RKPFCM
0.3199
0.3405
4
RKPFCM-PSO
0.3186
0.3384
4Table 7
5 dB Noise.
Algorithms
RMSE-trn
RMSE-test
Number of rules
FCM [16]
0.6170
1.0364
4
GK [16]
0.5855
0.6505
4
FMI [16]
0.6274
0.7021
4
FCRM [16]
0.5805
0.6485
4
MFCRM-NC [16]
0.5658
0.5998
4
RKPFCM
0.4958
0.5921
4
RKPFCM-PSO
0.4908
0.5908
4Table 8
1 dB Noise.
Algorithms
RMSE-trn
RMSE-test
Number of rules
FCM [16]
0.9330
1.3646
4
GK [16]
0.8932
1.0226
4
FMI [16]
0.8953
1.0320
4
FCRM [16]
0.8872
1.0095
4
MFCRM-NC [16]
0.8699
0.9947
4
RKPFCM
0.7600
0.8894
4
RKPFCM-PSO
0.7571
0.8809
4


### 5.2.3. Example  3

We consider the Box Jenkins gas furnace data set which is used as a standard test for identification techniques. The data set is composed of 296 pairs of input-output measurements. The input “u” is the gas flow rate into a furnace and the output “y” is the CO2 concentration in the outlet gases. In order to take all the above-mentioned issues into account, we simulated the following experimental case [16]:all the 296 data pairs are used as training data and{y(k-1), u(k-4)} are selected as input variables to various algorithms, while we use two rules (C=2). In this example the used parameters are a=1, b=2, m=2, G=10, NP = 30, D=3, wmax = 0.9, wmin = 0.4, c1=1.95, c2 = 2.1,σ = 10, itermax=1000, and ε = 10−9.The simulation result given by the RKPFCM-PSO algorithm is illustrated in Figure 6.Figure 6
Identification results for the RKPFCM-PSO Algorithm (SNR = 10 dB).Based on the comparison presented in Table9, it is clear that the proposed algorithm RKPFCM-PSO is more robust to noise than the other algorithms found in literature.Table 9
Comparison results for Box Jenkins system.
Algorithms
Number of inputs
Number of rules
MSE
Tong (1980) [16]
2
19
0.469
Pedrycz (1984) [16]
2
81
0.320
Xu (1987) [16]
2
25
0.328
Sugeno (1991) [16]
2
2
0.359
Yoshinari (1993) [16]
2
6
0.299
Joo (1997) [16]
2
6
0.166
Zhang (2006) [16]
2
2
0.160
Glowaty (2008) [16]
2
2
0.391
Andri (2011) [16]
2
10
0.167
Tanmoy Dam (2015) [16]
Without noise
2
2
0.152
SNR = 30 dB
2
2
0.153
SNR = 10 dB
2
2
0.277
RKPFCM
Without noise
2
2
0.1458
SNR = 30 dB
2
2
0.1512
SNR = 10 dB
2
2
0.2342
SNR = 5 dB
2
2
0.2922
SNR = 2 dB
2
2
0.3014
RKPFCM-PSO
Without noise
2
2
0.1455
SNR = 30 dB
2
2
0.1503
SNR = 10 dB
2
2
0.2338
SNR = 5 dB
2
2
0.2902
SNR = 2 dB
2
2
0.3001When we use our algorithm RKPFCM-PSO the local linear models identified are given as follows:(52)y1k=0.6777yk-1-0.9593uk-4+17.3007,y2k=0.5560yk-1-1.3297uk-4+23.5880.


## 5.2.1. Example  1

Consider a nonlinear system described by the following difference equation [15]:(46)yk=y2k-3+y2k-2+y2k-1+tanh⁡uk1+y2k-1+y2k-2+y2k-3+ek,where y(k) and u(k) are the output and the input of the system, respectively.e
(
k
) is a noise. (47)uk=0.6sin⁡3πkTs+0.2sin⁡4πkTs+1.2sin⁡πkTs.Ts = 0.01.200 samples were generated by simulation and were used, where the selected input variables are chosen{y(k-1), y(k-2), y(k-3), u(k)}.The complete data set has been used to train the model. The noise influence is analyzed with different SNR levels (SNR = 10 dB and SNR = 5 dB).In this part, we have applied various algorithms and our proposed clustering RKPFCM and RKPFCM-PSO which approximate the nonlinear model (46).The used parameters area=1, b=3, m=2, G=10, NP = 30, D=4, wmax=0.9, wmin=0.4, c1=2.05, c2=2.05, σ = 20, itermax=1000, and ε = 10−9.The shape of the excitation signal used for identification is illustrated in Figure2.Figure 2
Input-output sequences.The simulation result given by the RKPFCM-PSO algorithm is illustrated in Figure3.Figure 3
Identification results for the RKPFCM-PSO algorithm with SNR = 10 DB.Table2 shows the various modeling performance results without noise obtained by different algorithms; this comparison results demonstrate that the best MSE and best VAF are obtained by the proposed methods (RKPFCM and RKPFCM-PSO).Table 2
Comparison results without noise.
Algorithms
Number of rules
MSE
VAF (%)
Cluster fuzzy [15]
6
1.9 × 10−3
—
Kalman cluster fuzzy [15]
2
2.2 × 10−5
—
FCM
2
2.1422 × 10−5
99.9581
GK
2
2.1142 × 10−5
99.9592
KFCM
2
2.1011 × 10−5
99.9601
PFCM
2
2.0152 × 10−5
99.9611
RKPFCM
2
1.8498 × 10−5
99.9723
RKPFCM-PSO
2
1.8412 × 10−5
99.9742Tables3 and 4 present the various modeling performance results with noise influence (SNR = 5 dB and 10 dB) obtained by the different algorithms. However, our proposed algorithm RKPFCM-PSO retained the best performance with a higher level of noise.Table 3
Comparison results with SNR = 10 dB.
Algorithms
Number of rules
MSE (10−4)
VAF (%)
FCM
2
9.8397
98.0345
GK
2
9.8831
98.0303
KFCM
2
9.9492
98.0289
PFCM
2
9.8313
98.0362
RKPFCM
2
8.5158
98.1628
RKPFCM-PSO
2
8.5149
98.1637Table 4
Comparison results with SNR = 5 dB.
Algorithms
Number of rules
MSE (10−3)
VAF (%)
FCM
2
1.7594
96.0655
GK
2
1.7552
96.0668
KFCM
2
1.7572
96.0659
PFCM
2
1.7534
96.0783
RKPFCM
2
1,7424
96.1462
RKPFCM-PSO
2
1,73310
96.1510The local linear models identified are given as follows:(48)y1k=0.3167yk-1+0.0263yk-2-0.0482yk-3+0.1183k+0.6916,y2k=0.4045yk-1+0.0665yk-2-0.0304yk-2+0.1519uk+0.5417.


## 5.2.2. Example  2

Consider a highly complex modified nonlinear system described by the following difference equation [16]:(49)yk=yk-1yk-2+2yk-1+2.510+y2k-1+y2k-2+uk+ek,where y(k) is the model output and u(k) is the model input which is bounded between [−1+1]. The e(k) is a noise.The following input signal is expressed as(50)uk=sin2kπ250k≤5000.8∗sin⁡2kπ250+0.2∗sin⁡2kπ25otherwise.1500 samples were generated by simulation in which 1000 samples were used to train the model. Fuzzy model parameters have been identified once, testing of model was done by the remaining 500 samples, and {y(k-1), y(k-2), u(k), u(k-1)} are chosen as input variables. In this example the parameters settings are a=1, b=2, m=2, G=10, NP = 30, D=4, wmax = 0.9, wmin = 0.4, c1=2, c2=2.1, σ = 20, itermax=1000, and ε = 10−9.The noise influence is analyzed with different SNR levels (SNR = 20 dB, SNR = 10 dB, SNR =5 dB, and SNR = 1 dB).The obtained identification results by RKPFCM-PSO are, respectively, shown in Figures4 and 5.Figure 4
Input-output sequences.Figure 5
RKPFCM-PSO performances for test data with SNR = 10 dB.The evaluation performance index (RMSE-trn and RMSE-test) stands for training and testing data, respectively.Tables5–8 show the comparative performance of RKPFCM and RKPFCM-PSO with different existing algorithms such as FCM, G-K, Fuzzy Model Identification (FMI), FCRM, and MFCRM-NC. It is clearly seen from the results that our algorithm RKPFCM-PSO gives the best performance in noisy environments. The local linear models identified by RKPFCM-PSO are given as follows:(51)y1k=0.8562yk-1+0.0548yk-2+2.0967uk-1.6609uk-1+0.0814,y2k=0.8058yk-1-0.0924yk-2+1.3488uk-0.3718uk-1+0.8032,y3k=0.5673yk-1+0.1896yk-2+1.2330uk-0.8613uk-1+0.0520,y4k=0.8881yk-1-0.0571yk-2+1.5400uk-0.9195uk-1+0.4536.Table 5
20 dB Noise.
Algorithms
RMSE-trn
RMSE-test
Number of rules
FCM [16]
0.1393
0.3004
4
GK [16]
0.1326
0.1484
4
FMI [16]
0.1457
0.2378
4
FCRM [16]
0.1291
0.1495
4
MFCRM-NC [16]
0.1183
0.1395
4
RKPFCM
0.1167
0.1377
4
RKPFCM-PSO
0.1165
0.1370
4Table 6
10 dB Noise.
Algorithms
RMSE-trn
RMSE-test
Number of rules
FCM [16]
0.3453
0.7150
4
GK [16]
0.3330
0.3489
4
FMI [16]
0.3956
0.4416
4
FCRM [16]
0.3312
0.3694
4
MFCRM-NC [16]
0.3233
0.3465
4
RKPFCM
0.3199
0.3405
4
RKPFCM-PSO
0.3186
0.3384
4Table 7
5 dB Noise.
Algorithms
RMSE-trn
RMSE-test
Number of rules
FCM [16]
0.6170
1.0364
4
GK [16]
0.5855
0.6505
4
FMI [16]
0.6274
0.7021
4
FCRM [16]
0.5805
0.6485
4
MFCRM-NC [16]
0.5658
0.5998
4
RKPFCM
0.4958
0.5921
4
RKPFCM-PSO
0.4908
0.5908
4Table 8
1 dB Noise.
Algorithms
RMSE-trn
RMSE-test
Number of rules
FCM [16]
0.9330
1.3646
4
GK [16]
0.8932
1.0226
4
FMI [16]
0.8953
1.0320
4
FCRM [16]
0.8872
1.0095
4
MFCRM-NC [16]
0.8699
0.9947
4
RKPFCM
0.7600
0.8894
4
RKPFCM-PSO
0.7571
0.8809
4


## 5.2.3. Example  3

We consider the Box Jenkins gas furnace data set which is used as a standard test for identification techniques. The data set is composed of 296 pairs of input-output measurements. The input “u” is the gas flow rate into a furnace and the output “y” is the CO2 concentration in the outlet gases. In order to take all the above-mentioned issues into account, we simulated the following experimental case [16]:all the 296 data pairs are used as training data and{y(k-1), u(k-4)} are selected as input variables to various algorithms, while we use two rules (C=2). In this example the used parameters are a=1, b=2, m=2, G=10, NP = 30, D=3, wmax = 0.9, wmin = 0.4, c1=1.95, c2 = 2.1,σ = 10, itermax=1000, and ε = 10−9.The simulation result given by the RKPFCM-PSO algorithm is illustrated in Figure 6.Figure 6
Identification results for the RKPFCM-PSO Algorithm (SNR = 10 dB).Based on the comparison presented in Table9, it is clear that the proposed algorithm RKPFCM-PSO is more robust to noise than the other algorithms found in literature.Table 9
Comparison results for Box Jenkins system.
Algorithms
Number of inputs
Number of rules
MSE
Tong (1980) [16]
2
19
0.469
Pedrycz (1984) [16]
2
81
0.320
Xu (1987) [16]
2
25
0.328
Sugeno (1991) [16]
2
2
0.359
Yoshinari (1993) [16]
2
6
0.299
Joo (1997) [16]
2
6
0.166
Zhang (2006) [16]
2
2
0.160
Glowaty (2008) [16]
2
2
0.391
Andri (2011) [16]
2
10
0.167
Tanmoy Dam (2015) [16]
Without noise
2
2
0.152
SNR = 30 dB
2
2
0.153
SNR = 10 dB
2
2
0.277
RKPFCM
Without noise
2
2
0.1458
SNR = 30 dB
2
2
0.1512
SNR = 10 dB
2
2
0.2342
SNR = 5 dB
2
2
0.2922
SNR = 2 dB
2
2
0.3014
RKPFCM-PSO
Without noise
2
2
0.1455
SNR = 30 dB
2
2
0.1503
SNR = 10 dB
2
2
0.2338
SNR = 5 dB
2
2
0.2902
SNR = 2 dB
2
2
0.3001When we use our algorithm RKPFCM-PSO the local linear models identified are given as follows:(52)y1k=0.6777yk-1-0.9593uk-4+17.3007,y2k=0.5560yk-1-1.3297uk-4+23.5880.


## 6. Conclusion

In literature, various clustering algorithms have been proposed for nonlinear systems identification. In this work, we developed a new clustering algorithm called RKPFCM-PSO for the nonlinear systems identification. Our algorithm is an improvement of the Possibilistic FuzzyC-Means Clustering (PFCM) where we used a hyper tangent kernel function to calculate the distance of data point from the cluster centers and a heuristic search algorithm PSO to reach the global minimum of the objective function. The proposed algorithm provides better results of fuzzy modeling of unknown nonlinear systems. The robustness and the quality of this proposed method are demonstrated by simulation results of noisy nonlinear systems described by recurrent equations and application to a Box Jenkins gas furnace system. Thus, the proposed methods show favorable results in noisy environments compared with the techniques mentioned in the literature.In the future, we will integrate other optimization methods such as the gravitational search algorithm to optimize our hybrid method and we will apply this algorithm for identification of some complex nonlinear real systems as the robotic or the mechatronic systems.



---

*Source: 2427309-2017-05-14.xml*