# Design of Polynomial Fuzzy Radial Basis Function Neural Networks Based on Nonsymmetric Fuzzy Clustering and Parallel Optimization

Authors: Wei Huang; Jinsong Wang
Journal: Mathematical Problems in Engineering
 (2013)

---

## Abstract

We first propose a Parallel Space Search Algorithm (PSSA) and then introduce a design of Polynomial Fuzzy Radial Basis Function Neural Networks (PFRBFNN) based on Nonsymmetric Fuzzy Clustering Method (NSFCM) and PSSA. The PSSA is a parallel optimization algorithm realized by using Hierarchical Fair Competition strategy. NSFCM is essentially an improved fuzzy clustering method, and the good performance in the design of “conventional” Radial Basis Function Neural Networks (RBFNN) has been proven. In the design of PFRBFNN, NSFCM is used to design the premise part of PFRBFNN, while the consequence part is realized by means of weighted least square (WLS) method. Furthermore, HFC-PSSA is exploited here to optimize the proposed neural network. Experimental results demonstrate that the proposed neural network leads to better performance in comparison to some existing neurofuzzy models encountered in the literature.


---

## Body

## 1. Introduction

With the learning and generalization abilities, Fuzzy Radial Basis Function Neural Networks (FRBFNN) have been utilized in numerous fields for engineering, medical engineering, and social science [1, 2]. They are developed by integrating the principles of Radial Basis Function Neural Networks (RBFNN) and invoking the mechanisms of information granulation [3]. In the design of classical FRBFNN, information granulation realized by using Fuzzy C-Means (FCM) is utilized to the premise part, while the consequence part (output) is treated as a linear combination of zero-order polynomials; with the use of FCM, the centers of the clusters are determined and the membership functions of the granules can be formed. The visible advantage of the FRBFNN is that it does not suffer from the curse of dimensionality which eminently appeared in other networks based on grid portioning [3]. When dealing with the FRBFNN, it is required to estimate the parameters based on vast amount of data. As powerful optimization tools in many science fields [4–6], various evolutionary algorithms have been also proposed to improve the accuracy of models.Recently, polynomial Fuzzy Radial Basis Function Neural Networks (PFRBFNN) [3] were proposed. The PFRBFNN adopts four polynomial types, which overcome the zero-order polynomials of FRBFNN. In the PFRBFNN, the output of the conventional FRBFNN is considered as a linear combination of four types of polynomials. In spite of successful construction in the design of PFRBFNN, there are still two following limitations: (1) the FCM selects the hidden node centers by partitioning the input space to an equal number of fuzzy sets for each input variable, while the modification of the original method in order to take into account nonsymmetric fuzzy partitions of the input space is not considered [7, 8] and (2) most of the optimization algorithms in the design of FRBFNN are not parallel; they lack parallel optimization algorithms.To alleviate the limitations, in this study we propose a parallel space search algorithm (PSSA) and introduce a design of PFRBFNN with the aid of HFC-PSSA and Nonsymmetric Fuzzy Clustering Method (NSFCM). On the one hand, with the use of nonsymmetric fuzzy partitions of the input space, information granulation [24–27] realized by NSFCM may overcome the first limitation. On the other hand, with the use of hierarchical fair competition strategy, the proposed PSSA may help to overcome the second limitation. The overall methodology of PFRBFNN is as follows: first, NSFCM is utilized to determine the premise part of PFRBFNN, while the coefficients of the consequence polynomials are estimated by Weighted Least Squares (WLS) method; second, HFC-PSSA is exploited to optimize the proposed PFRBFNN.The structure of the paper is organized as follows. Section2 presents the HFC-PSSA. Section 3 describes the architecture and learning methods applied to the PFRBFNN. Section 4 deals with the optimization of PFRBFNN. Section 4 reports on the experimental results. Finally, some conclusions are drawn in Section 5.


## 2. Hierarchical Fair Competition-Based Parallel Space Search Algorithm

First, let us recall the space search algorithm (SSA), an adaptive heuristic optimization algorithm whose search method comes with the analysis of the solution space [24]. The SSA generates new solutions from the old ones by using the so-called space search operators, which generate two successive steps: it first generates a new subspace (local area) and then realizes search in this new subspace. The search in the new space is realized by randomly generating a new solution (individual) located in this space, while the generation of the new space includes two cases [24]: (a) space search based on M selected solutions (denoted here as Case I) and (b) space search based on the current best solution (Case II).For convenience, we consider the following optimization problem(1)min(ormax)f(x1,x2,…,xn)st:xi∈[li,ui],i=1,2,3,…,n.Here a feasible solution is denoted in the following way(x1,x2,…,xn). Suppose that Xk=(x1k,x2k,…,xnk)is the kth solution, Li=minj=1Mxij, Ui=maxj=1Mxij.Two scenarios are considered as follows [24].(a)
Space search based onM selected solutions: in this case, M solutions are randomly selected from the current population. The role of this operator is to update the current solutions by new solutions approaching to the optimum. The adjacent space based on M solutions is given in the form
(2)V={∑k=1MaixikXnew∣xinew=∑k=1Maixik⋃Xnew∈S,where∑i=1Mai=1,-1≤ai≤2Xnew∣xinew=∑k=1Maixik∪Xnew∈S}.(b)
Space search based on the current best solution: in this case, the given solution is the best solution in the current population. The expression that generates a new solution is as follows:(3)V2={⋃xinew(x1new,x2new,…,xnnew)∣xjnew=xj(j≠i)⋃xinew∈[li,ui]}.In the HFC-PSSA, a migration mechanism that is executed in regular generation intervals is included in the evolutionary process. To explain the details of migration operation, let us consider the following sequence of steps [28–30].Step 1.
Normalize the fitness of individuals. We generate several subpopulations according to the idea of hierarchical fair competition, and then normalize the fitness of individuals in each subpopulation using the following expression:(4)nfj,i=fj,i-fminfmax-fmin,
wherefi,j is fitness of ith subpopulation and jth individual andfmaxandfminare maximum and minimum values of fitness, respectively.Step 2.
Calculate some admission threshold (ALi). The admission threshold for the ith subpopulation is determined by the average of normalized fitness. The expression of ALi is as follows:
(5)ALi=1ni∑j=1ninfj,i,
where ni is a size of ith subpopulation.Step 3.
Create admission buffer that is located at each admission threshold level.Step 4.
Migrate the qualified individuals. Here the individuals are migrated from the admission buffer to the corresponding subpopulation.
Algorithm1 summarizes the flow of computing the HFC-PSSA. The termination condition is such that all the solutions in the current population have the same fitness or terminate after a certain fixed number of generations.Algorithm 1:The flow of computing the HFC-PSSA.
BEGIN
Initialize the solution set (population)
Evaluate each solution in the solution set
Sort all current solutions in the solution set
While {the termination conditions are not met}
Implement migration operation and generate new solution setS′
Select the solutions fromS′
Search solution space (case I)
Search solution space (case II)
Sort all current solutions inS′
End while
Report the optimal solution
End


## 3. A New Design of the PFRBFNN

In the design of PFRBFNN, fuzzy clustering may be used to determine the number of RBFs, as well as a position of their centers and the values of the widths [31]. The gradient-method or the least square algorithm is used to realize parametric learning to deal with the conclusions of the rules [32, 33]. In contrast to PFRBFNN, the proposed PFRBFNN are designed as shown in Figure 1. There are mainly two new points in the design of PFRBFNN. First, in the construction of PFRBFNN, instead of FCM, NSFCM is used to realize radial basis functions. Second, in the optimization of PFRBFNN, we use the HFC-PSSA as an optimization vehicle.Figure 1
An overall design of PFRBFNN.


### 3.1. Realization of Radial Basis Functions Using Nonsymmetric Fuzzy Means

In the PFRBFNN, the receptive fields in Radial Basis Functions are formed by nonsymmetric fuzzy means (NSFM) [7, 8], where the main idea is as follows.Consider a system withN normalized input variablesxi, where i=1,2,…,N; the domain of each input variable is partitioned into a number of one-dimensional triangular fuzzy sets,c; then each fuzzy set can be written as
(6)Aij={aij,δα},i=1,…,N,j=1,…,c,
where aij is the center element of fuzzy set Aij andδαis half of the respective width; this partitioning technique creates a total of cN multidimensional fuzzy subspaces Al, where l=1,2,…,cN. Then one can define the center vector al and the side vector δα of each fuzzy subspace:
(7)Al={al,δα}={[a1j1l,a2j2l,…,aNjNl],[δa,δa,…,δa]},l=1,…,cN,
whereaij1lis the center element of the one-dimension fuzzy setAij1that has been assigned to inputi.In some senses, the conventional fuzzy means method can be regarded as “symmetric” fuzzy means [7], and the Euclideandl(x(k))relative distance betweenAland the input data vectorx(k)can be represented as
(8)dl(x(k))=∑i=1N(aij1l-xi(k))2Nδa.
In the nonsymmetric fuzzy means method, the Euclidean dl(x(k)) becomes
(9)dl(x(k))=∑i=1N(aij1l-xi(k))2N(δa)2.
To obtain the detail of NSFM, one can refer to the reference [8].


### 3.2. Construction of Consequence Polynomials Using Information Granulation

The PFRBFNN [3] based on information granulation [25–27] can be represented in form of “if-then” fuzzy rules
(10)Ri:IFxkisincludedinclusterAiTHENyki-Mi=fi(xk,vi),
whereRiis the ith fuzzy rule, i=1,…,n, n is the number of fuzzy rules (the number of clusters),fi(xk,vi)is the consequent polynomial of the ith fuzzy rule, that is, a local model representing input-output relationship of the ith subspace (local area);wikis the degree of membership (i.e., activation level) of the ith local model while vi=[vi1vi2…vil]T is the ith prototype.In the design of PFRBFNN, four types of polynomials are considered as the consequent part of fuzzy rules. One of the four types is selected for each subspace as the result of the optimization, which will be described later in this study. It is noted that the PFRBFNN using IG do not suffer from the curse of dimensionality (as all variables are considered en block) [3], more accurate and compact models with a small number of fuzzy rules by using high-order polynomials may be constructed. The four types of consequent polynomials are as follows [24–27].Type 1: zero-order polynomial (constant type)(11)fi(xk1,…,xkl,vi)=ai0.Type 2: first-order polynomial (linear type)(12)fi(xk1,…,xkl,vi)=ai0+ai1(xk1-vi1)+ai2(xk2-vi2)+⋯+ail(xkl-vil).Type 3: second-order polynomial (quadratic type)(13)fi(xk1,…,xkl,vi)=ai0+ai1(xk1-vi1)+a2(xk2-vi2)+⋯+ail(xkl-vil)+ai(l+1)(xk1-vi1)2+ai(l+2)(xk2-vi2)2+⋯+ai(2l)(xkl-vil)2+ai(2l+1)(xk1-vi1)(xk2-vi2)+⋯+ai((l+1)(l+2)/2)×(xk(l-1)-vi(l-1))(xkl-vil).Type 4: modified second-order polynomial (modified quadratic type)(14)fi(xk1,…,xkl,vi)=ai0+ai1(xk1-vi1)+ai2(xk2-vi2)+⋯+ail(xkl-vil)+ai(l+1)(xk1-vi1)(xk2-vi2)+⋯+ai(l(l+1)/2)×(xk(l-1)-vi(l-1))(xkl-vil).The determination of the numeric output of the model, based on the activation levels of the rules, is given in the form
(15)y^k=∑i=1nwikfi(xk1,…,xkl,vi).


### 3.3. Learning of the Consequent Part Using Weighted Learning Square

To determine the coefficients of the model, we use the weighted learning square (WLS) method. The minimization of the objective functionJLis as follows:
(16)JL=∑i=1n∑k=1mwik(yk-fi(xk-vi))2.It is clear thatJLcan be rearranged as [34]
(17)JL=∑i=1n(Y-Xiai)TWi(Y-Xiai)=∑i=1n(Wi1/2Y-Wi1/2Xiai)T(Wi1/2Y-Wi1/2Xiai),
where ai is the vector of coefficients of ith consequent polynomial (local model), Wi is the diagonal matrix (weighting factor matrix) which involves the activation levels; Xi is a matrix which includes input data shifted by the locations of the information granules (more specifically, centers of clusters). For example, if the consequent polynomial is type 2 (linear or a first-order polynomial), we have
(18)Wi=[wi10⋯00wi2⋯0⋮⋮⋱⋮00⋯wim]∈ℜm×m,Xi=[1(x11-vi1)⋯(xl1-vil)1(x12-vi1)⋯(xl1-vil)1⋮⋱⋮1(x1m-vi1)⋯(xlm-vil)],ai=[ai0ai1…ail].For the local learning algorithm [34], the objective function is defined as a linear combination of the squared error being the difference between the data and the corresponding output of each fuzzy rule, based on a weighting factor matrix. The weighting factor matrix, Wi, captures the activation levels of input data to ith subspace. In this sense the weighting factor matrix can be considered to form a discrete version of the fuzzy linguistic representation for the corresponding subspace.For theith fuzzy rule, the coefficients of polynomial in the consequent part can be written in a usual manner, namely,
(19)ai=(XiTWiXi)-1XiWiY.Notice that the coefficients of polynomial in the consequent part in each fuzzy rule are calculated independently by means of a certain subset of training data.


### 3.4. Optimization of the PFRBFNN Using HFC-PSSA

In the HFC-PSSA, a solution is represented as a vector comprising the fuzzification coefficient, the number of input variables, the input variable to be selected, the number of fuzzy rules, and the polynomial type. The length of solution vector corresponds to the maximal number of fuzzy rules to be considered in the optimization.Figure2 offers an interpretation of the content of the particle in case the upper bound of search space of the fuzzy rule is set to 6. As the number of rules and the polynomials orders (in consequent part) have to be integer number, we round off these values to the nearest integer. The fuzzification coefficient is equal to 1.03. The number of selected input variables is four, while the number of the rule is six. The first local model is of linear type while the other three local models are linear and quadratic.Figure 2
Solution composition of HFC-PSSA and its interpretation.Here we consider two performance indexes that is the standard root mean squared error (RMSE) and mean squared error (MSE) [24]
(20)PI(orE_PI)={1m∑i=1m(yi-yi*)2,(RMSE)1m∑i=1m(yi-yi*)2,(MSE),
where y* is the output of the fuzzy model, m is the total number of data, and i is the data index.The accuracy criterion MPI [24] includes both the training data and testing data and comes as a convex combination of the two components:
(21)MPI=θ×PI+(1-θ)×EPI,
where PI and EPI denote the performance index for the training data and testing data, respectively.θis a weighting factor that allows us to strike a sound balance between the performance of the model for the training and testing data.


## 3.1. Realization of Radial Basis Functions Using Nonsymmetric Fuzzy Means

In the PFRBFNN, the receptive fields in Radial Basis Functions are formed by nonsymmetric fuzzy means (NSFM) [7, 8], where the main idea is as follows.Consider a system withN normalized input variablesxi, where i=1,2,…,N; the domain of each input variable is partitioned into a number of one-dimensional triangular fuzzy sets,c; then each fuzzy set can be written as
(6)Aij={aij,δα},i=1,…,N,j=1,…,c,
where aij is the center element of fuzzy set Aij andδαis half of the respective width; this partitioning technique creates a total of cN multidimensional fuzzy subspaces Al, where l=1,2,…,cN. Then one can define the center vector al and the side vector δα of each fuzzy subspace:
(7)Al={al,δα}={[a1j1l,a2j2l,…,aNjNl],[δa,δa,…,δa]},l=1,…,cN,
whereaij1lis the center element of the one-dimension fuzzy setAij1that has been assigned to inputi.In some senses, the conventional fuzzy means method can be regarded as “symmetric” fuzzy means [7], and the Euclideandl(x(k))relative distance betweenAland the input data vectorx(k)can be represented as
(8)dl(x(k))=∑i=1N(aij1l-xi(k))2Nδa.
In the nonsymmetric fuzzy means method, the Euclidean dl(x(k)) becomes
(9)dl(x(k))=∑i=1N(aij1l-xi(k))2N(δa)2.
To obtain the detail of NSFM, one can refer to the reference [8].


## 3.2. Construction of Consequence Polynomials Using Information Granulation

The PFRBFNN [3] based on information granulation [25–27] can be represented in form of “if-then” fuzzy rules
(10)Ri:IFxkisincludedinclusterAiTHENyki-Mi=fi(xk,vi),
whereRiis the ith fuzzy rule, i=1,…,n, n is the number of fuzzy rules (the number of clusters),fi(xk,vi)is the consequent polynomial of the ith fuzzy rule, that is, a local model representing input-output relationship of the ith subspace (local area);wikis the degree of membership (i.e., activation level) of the ith local model while vi=[vi1vi2…vil]T is the ith prototype.In the design of PFRBFNN, four types of polynomials are considered as the consequent part of fuzzy rules. One of the four types is selected for each subspace as the result of the optimization, which will be described later in this study. It is noted that the PFRBFNN using IG do not suffer from the curse of dimensionality (as all variables are considered en block) [3], more accurate and compact models with a small number of fuzzy rules by using high-order polynomials may be constructed. The four types of consequent polynomials are as follows [24–27].Type 1: zero-order polynomial (constant type)(11)fi(xk1,…,xkl,vi)=ai0.Type 2: first-order polynomial (linear type)(12)fi(xk1,…,xkl,vi)=ai0+ai1(xk1-vi1)+ai2(xk2-vi2)+⋯+ail(xkl-vil).Type 3: second-order polynomial (quadratic type)(13)fi(xk1,…,xkl,vi)=ai0+ai1(xk1-vi1)+a2(xk2-vi2)+⋯+ail(xkl-vil)+ai(l+1)(xk1-vi1)2+ai(l+2)(xk2-vi2)2+⋯+ai(2l)(xkl-vil)2+ai(2l+1)(xk1-vi1)(xk2-vi2)+⋯+ai((l+1)(l+2)/2)×(xk(l-1)-vi(l-1))(xkl-vil).Type 4: modified second-order polynomial (modified quadratic type)(14)fi(xk1,…,xkl,vi)=ai0+ai1(xk1-vi1)+ai2(xk2-vi2)+⋯+ail(xkl-vil)+ai(l+1)(xk1-vi1)(xk2-vi2)+⋯+ai(l(l+1)/2)×(xk(l-1)-vi(l-1))(xkl-vil).The determination of the numeric output of the model, based on the activation levels of the rules, is given in the form
(15)y^k=∑i=1nwikfi(xk1,…,xkl,vi).


## 3.3. Learning of the Consequent Part Using Weighted Learning Square

To determine the coefficients of the model, we use the weighted learning square (WLS) method. The minimization of the objective functionJLis as follows:
(16)JL=∑i=1n∑k=1mwik(yk-fi(xk-vi))2.It is clear thatJLcan be rearranged as [34]
(17)JL=∑i=1n(Y-Xiai)TWi(Y-Xiai)=∑i=1n(Wi1/2Y-Wi1/2Xiai)T(Wi1/2Y-Wi1/2Xiai),
where ai is the vector of coefficients of ith consequent polynomial (local model), Wi is the diagonal matrix (weighting factor matrix) which involves the activation levels; Xi is a matrix which includes input data shifted by the locations of the information granules (more specifically, centers of clusters). For example, if the consequent polynomial is type 2 (linear or a first-order polynomial), we have
(18)Wi=[wi10⋯00wi2⋯0⋮⋮⋱⋮00⋯wim]∈ℜm×m,Xi=[1(x11-vi1)⋯(xl1-vil)1(x12-vi1)⋯(xl1-vil)1⋮⋱⋮1(x1m-vi1)⋯(xlm-vil)],ai=[ai0ai1…ail].For the local learning algorithm [34], the objective function is defined as a linear combination of the squared error being the difference between the data and the corresponding output of each fuzzy rule, based on a weighting factor matrix. The weighting factor matrix, Wi, captures the activation levels of input data to ith subspace. In this sense the weighting factor matrix can be considered to form a discrete version of the fuzzy linguistic representation for the corresponding subspace.For theith fuzzy rule, the coefficients of polynomial in the consequent part can be written in a usual manner, namely,
(19)ai=(XiTWiXi)-1XiWiY.Notice that the coefficients of polynomial in the consequent part in each fuzzy rule are calculated independently by means of a certain subset of training data.


## 3.4. Optimization of the PFRBFNN Using HFC-PSSA

In the HFC-PSSA, a solution is represented as a vector comprising the fuzzification coefficient, the number of input variables, the input variable to be selected, the number of fuzzy rules, and the polynomial type. The length of solution vector corresponds to the maximal number of fuzzy rules to be considered in the optimization.Figure2 offers an interpretation of the content of the particle in case the upper bound of search space of the fuzzy rule is set to 6. As the number of rules and the polynomials orders (in consequent part) have to be integer number, we round off these values to the nearest integer. The fuzzification coefficient is equal to 1.03. The number of selected input variables is four, while the number of the rule is six. The first local model is of linear type while the other three local models are linear and quadratic.Figure 2
Solution composition of HFC-PSSA and its interpretation.Here we consider two performance indexes that is the standard root mean squared error (RMSE) and mean squared error (MSE) [24]
(20)PI(orE_PI)={1m∑i=1m(yi-yi*)2,(RMSE)1m∑i=1m(yi-yi*)2,(MSE),
where y* is the output of the fuzzy model, m is the total number of data, and i is the data index.The accuracy criterion MPI [24] includes both the training data and testing data and comes as a convex combination of the two components:
(21)MPI=θ×PI+(1-θ)×EPI,
where PI and EPI denote the performance index for the training data and testing data, respectively.θis a weighting factor that allows us to strike a sound balance between the performance of the model for the training and testing data.


## 4. Experimental Study

To demonstrate the effectiveness of the proposed approach, we have done the experiments based on numerical examples. In all the experiments, we setθ=0.5. The proposed HFC-PSSA is carried out as an optimization vehicle in the design of PFRBFNN. Table 1 summarizes the list of parameters and boundaries of the HFC-PSSA.Table 1
List of the parameters of the HFC-PSSA and boundaries of search space.
HFC-PSSA parameters
Number of generations
200
Number of subpopulations
6
Number of subsolutions
[
40
,
40
,
40
,
40
,
40
,
40
]
Migration interval
25 generations
Number of solutions for search space (Case I)
8
Boundary of search space of decision variables
Fuzzification coefficient
1.01~5
Number of input variables*
2~10
Number of rules
2~20
Order of polynomials of each fuzzy rule
1~4 (type 1~type 4)
*If the number of input variables in dataset is smaller than 10, then we use the number of all input variables.


### 4.1. Sewage Treatment Process (STP)

The first well-known dataset comes from the sewage treatment system plant in Seoul, Republic of Korea. The proposed PFRBFNN are carried out on the sewage treatment process data [11], which consists of 52 input-output pairs and four input variables (MLSS, WSR, RRSP, and DOSP). This dataset has been intensively studied in the previous literature [9–14]. Here the gas furnace process is partitioned into two parts. The first 60% of dataset is selected as training data used for the construction of the fuzzy model. The remaining 40% dataset is considered as the testing dataset, which is used to quantify the predictive quality of the model. The performance index is specified as the RMSE given by (20).The optimal network that consists of four fuzzy rules with type 2 is obtained by using the 200 generations of PSSA. The polynomials in the consequence part of four rules are as follows:(22)R1:y=0.817017+77.17488(x1-1604.615)+14.82316(x2-2215),R2:y=-31.5631+0.007825(x1-1604.615)+0.010179(x2-1.152941),R3:y=0.017146+0.075889(x1-0.567143)-1.77867(x2-2215),R4:y=-20.7821+12.91189(x1-0.567143)+1.070568(x2-1.152941).Figure3 illustrates the resulting values of the performance index when running the PFRBFNN based on the HFC-PSSA. As it could have been expected, by increasing the total number of generations, the accuracy of PFRBFNN becomes better.Trace curves of the performance indexes produced by HFC-PSSA (STP).
(a)
Training error in successive optimization
(b)
Testing error in successive optimizationThe performance of the proposed model is compared with some other models available in the literature; refer to Table2. Local models of other models have same type of fuzzy rule such as constant or linear form. The proposed model can have different types of local models. In this comparison, the proposed model, having a small number of rules, shows better accuracy, while the model leads to disadvantage of having a large number of coefficients of local model in case of selecting the quadratic form.Table 2
Comparative analysis of selected models (STP).
Model
PI
E_PI
No. of rules
Performance index
Neural Network [9]
1.801
39.294
MSE
Fuzzy model [10]
14.11
16.56
6
MSE
Fuzzy model (min-max) [11]
12.49
32.65
6
MSE
Fuzzy model (HCM) [12]
12.65
21.03
6
MSE
SOPNN [13]
5.365
4.852
5
MSE
6.837
8.871
3
MSE
FRFNN [14]
10.584
12.108
MSE
Our model
Without the use of PSSA
2.14
4.52
6
RMSE
With the use of PSSA
2.07
4.20
6
RMSE


### 4.2. Medical Imaging System (MIS)

The second dataset is a medical imaging system dataset which involves 390 software modules written in Pascal and FORTRAN, and each module is described by 11 input variables [15–19]. Applying the proposed design methodology, the given dataset is randomly partitioned to produce two datasets [15–17]: the first 60% of dataset is used for training the models; the remaining 40% of dataset, the testing dataset, provides for quantifying the predictive quality (generalization ability) of the fitted models. We consider the RMSE (20) as the performance index.With the running 200 generations of PSSA, we obtain the optimal network that consists of four fuzzy rules with type 2. The polynomials standing in the consequence part of these four fuzzy rules are as follows:(23)R1:y=-0.01809-0.25241(x1-7.173333)+0.005256(x2-22.03571),R2:y=-0.13802+0.006942(x1-7.173333)-0.01327(x2-12.12128),R3:y=-0.04035-0.02267(x1-3.501149)+0.239457(x2-22.03571),R4:y=0.060663+0.278776(x1-3.501149)+0.099469(x2-12.12128).Figure4 illustrates the resulting values of the performance index when running the PFRBFNN based on the HFC-PSSA. The proposed model is also contrasted with some previously developed fuzzy models as shown in Table 3. It is easy to see that the performance of the proposed model is better in the sense of its approximation and prediction abilities.Table 3
Results of comparative analysis (MIS).
Model
PI
E_PI
Index
SONFN [15]
Simplified
40.375
17.898
MSE
Linear
35.745
17.807
MSE
FPNN [16]
SI = 2
32.195
18.642
MSE
SI = 3
32.251
19.622
MSE
GA-based FSONN [17]
PN-based FSONN
18.043
11.898
MSE
FPN-based FSONN
23.739
9.090
MSE
Incremental model [18]
Linear regression
5.877
6.570
RMSE
Incremental model
4.620
6.624
RMSE
GO-FPNN model [19]
SI = 2
2.2141
3.4630
RMSE
SI = 3
0.8852
3.4690
RMSE
SI = 4
0.7818
3.4211
RMSE
Our model
Without the use of PSSA
0.908
1.68
RMSE
With the use of PSSA
0.896
1.65
RMSETrace curves of the performance indexes produced by HFC-PSSA (MIS).
(a)
Training error in successive optimization
(b)
Testing error in successive optimization


### 4.3. Abalone Data (ABA)

Finally we experiment with the abalone dataset [20–23], which is a larger dataset consisting of 4,177 input-output pairs that concerns the age of abalone predicted on a basis of seven input variables (including length, weight, diameter, etc.). The dataset is split into two separate parts [20, 21]: the construction of the fuzzy model is completed for 2506 data points being regarded as a training set; and the rest of the dataset (i.e., 1671 data points) is retained for testing purposes. RMSE is considered as the performance index.After running 200 generations of PSSA, we obtain the optimal network with eight fuzzy rules with type 3, whose polynomials of the consequence part are as follows:(24)R1:y=7.9872288+11.45947664(x1-0.124376)+451.4966298(x2-0.630624)+⋯+143.9173838(x3-0.282961)-47.44803317(x1-0.124376)(x2-0.630624)+⋯-217.5141906(x1-0.124376)×(x3-0.282961)+162.4868597(x2-0.630624)(x3-0.282961)+⋯-1.667754581(x1-0.124376)2-9.677924915(x2-0.630624)2-134.1737445(x3-0.282961)2,R2:y=-1188.04-855.99(x1-0.124376)+464.2795(x2-0.630624)+904.6096(x3-0.446352)+⋯-1356.86(x1-0.124376)(x2-0.630624)+21.30605(x1-0.124376)(x3-0.446352)+⋯+21.12003(x2-0.630624)(x3-0.446352)+26.0111(x1-0.124376)2-778.713(x2-0.630624)2+⋯-95.3784(x3-0.446352)2,R3:y=-143.405-42.2197(x1-0.124376)+71.0844(x2-1.096818)+⋯+13.19766(x3-0.282961)-51.0632(x1-0.124376)(x2-1.096818)+⋯+1.11368(x1-0.124376)(x3-0.282961)-387.954(x2-1.096818)(x3-0.282961)+⋯-107.995(x1-0.124376)2+622.8024(x2-1.096818)2-95.3784(x3-0.282961)2,R4:y=-532.195-28.6716(x1-0.124376)+228.1629(x2-1.096818)+⋯+499.0509(x3-0.446352)+3659.62(x1-0.124376)(x2-1.096818)+⋯+803.6259(x1-0.124376)(x3-0.446352)+994.0846(x2-1.096818)(x3-0.446352)+⋯+2050.08(x1-0.124376)2+1659.251(x2-1.096818)2-6.39226(x3-0.446352)2,R5:y=-658.152+406.3837(x1-0.162877)+5785.213(x2-0.630624)+⋯-340.025(x3-0.282961)-3561.14(x1-0.162877)(x2-0.630624)+⋯-7102.14(x1-0.162877)(x3-0.282961)+2663.84(x2-0.630624)(x3-0.282961)+⋯-38.9337(x1-0.162877)2-291.782(x2-0.630624)2-108.948(x3-0.282961)2,R6:y=546.4625+55.4919(x1-0.162877)+50.1344(x2-0.630624)+⋯-744.188(x3-0.446352)+66.87478(x1-0.162877)(x2-0.630624)+⋯-28.5836(x1-0.162877)(x3-0.446352)+29.87009(x2-0.630624)(x3-0.446352)+⋯-1321.04(x1-0.162877)2-13672(x2-0.630624)2+744.3283(x3-0.446352)2,R7:y=-954.934+807.175(x1-0.162877)-3140.58(x2-1.096818)+⋯+16.91164(x3-0.282961)+47.5888(x1-0.162877)(x2-1.096818)+⋯-34.206(x1-0.162877)(x3-0.282961)+119.1346(x2-1.096818)(x3-0.282961)+⋯-14.2966(x1-0.162877)2+9.594292(x2-1.096818)2-1.58319(x3-0.282961)2,R8:y=-166.398+4.89712(x1-0.162877)+383.2878(x2-1.096818)+⋯-23.2903(x3-0.446352)-1369.65(x1-0.162877)(x2-1.096818)+⋯+69.9118(x1-0.162877)(x3-0.446352)-330.398(x2-1.096818)(x3-0.446352)+⋯+960.4564(x1-0.162877)2+82.85281(x2-1.096818)2+43.62129(x3-0.446352)2.Figure5 shows performance indexes generated by means of the HFC-PSSA in case of using training data and testing data. Table 4 summarizes the results of comparative analysis of the PFRBFNN using HFC-PSSA when being contrasted with other models. It is clear that the proposed model outperforms several previous fuzzy models reported in the literature.Table 4
Results of comparative analysis (ABA).
Model
PI
E_PI
Index
Linear regression [20]
14.15
17.22
MSE
RBFNN [20]
10.36
10.48
MSE
RBFNN (context-free clustering) [20]
10.54
10.58
MSE
Boosting of granular model [20]
8.39
8.68
MSE
RBFNN II [21]
6.710
6.894
MSE
GA-FIS [22]
2.393
2.454
MSE/2
SP-SVM [23]
7.116
6.123
MSE
Our model
Without the use of PSSA
1.942
2.247
RMSE
With the use of PSSA
1.910
2.243
RMSETrace curves of the performance indexes produced by HFC-PSSA (ABA).
(a)
Training error in successive optimization
(b)
Testing error in successive optimization


## 4.1. Sewage Treatment Process (STP)

The first well-known dataset comes from the sewage treatment system plant in Seoul, Republic of Korea. The proposed PFRBFNN are carried out on the sewage treatment process data [11], which consists of 52 input-output pairs and four input variables (MLSS, WSR, RRSP, and DOSP). This dataset has been intensively studied in the previous literature [9–14]. Here the gas furnace process is partitioned into two parts. The first 60% of dataset is selected as training data used for the construction of the fuzzy model. The remaining 40% dataset is considered as the testing dataset, which is used to quantify the predictive quality of the model. The performance index is specified as the RMSE given by (20).The optimal network that consists of four fuzzy rules with type 2 is obtained by using the 200 generations of PSSA. The polynomials in the consequence part of four rules are as follows:(22)R1:y=0.817017+77.17488(x1-1604.615)+14.82316(x2-2215),R2:y=-31.5631+0.007825(x1-1604.615)+0.010179(x2-1.152941),R3:y=0.017146+0.075889(x1-0.567143)-1.77867(x2-2215),R4:y=-20.7821+12.91189(x1-0.567143)+1.070568(x2-1.152941).Figure3 illustrates the resulting values of the performance index when running the PFRBFNN based on the HFC-PSSA. As it could have been expected, by increasing the total number of generations, the accuracy of PFRBFNN becomes better.Trace curves of the performance indexes produced by HFC-PSSA (STP).
(a)
Training error in successive optimization
(b)
Testing error in successive optimizationThe performance of the proposed model is compared with some other models available in the literature; refer to Table2. Local models of other models have same type of fuzzy rule such as constant or linear form. The proposed model can have different types of local models. In this comparison, the proposed model, having a small number of rules, shows better accuracy, while the model leads to disadvantage of having a large number of coefficients of local model in case of selecting the quadratic form.Table 2
Comparative analysis of selected models (STP).
Model
PI
E_PI
No. of rules
Performance index
Neural Network [9]
1.801
39.294
MSE
Fuzzy model [10]
14.11
16.56
6
MSE
Fuzzy model (min-max) [11]
12.49
32.65
6
MSE
Fuzzy model (HCM) [12]
12.65
21.03
6
MSE
SOPNN [13]
5.365
4.852
5
MSE
6.837
8.871
3
MSE
FRFNN [14]
10.584
12.108
MSE
Our model
Without the use of PSSA
2.14
4.52
6
RMSE
With the use of PSSA
2.07
4.20
6
RMSE


## 4.2. Medical Imaging System (MIS)

The second dataset is a medical imaging system dataset which involves 390 software modules written in Pascal and FORTRAN, and each module is described by 11 input variables [15–19]. Applying the proposed design methodology, the given dataset is randomly partitioned to produce two datasets [15–17]: the first 60% of dataset is used for training the models; the remaining 40% of dataset, the testing dataset, provides for quantifying the predictive quality (generalization ability) of the fitted models. We consider the RMSE (20) as the performance index.With the running 200 generations of PSSA, we obtain the optimal network that consists of four fuzzy rules with type 2. The polynomials standing in the consequence part of these four fuzzy rules are as follows:(23)R1:y=-0.01809-0.25241(x1-7.173333)+0.005256(x2-22.03571),R2:y=-0.13802+0.006942(x1-7.173333)-0.01327(x2-12.12128),R3:y=-0.04035-0.02267(x1-3.501149)+0.239457(x2-22.03571),R4:y=0.060663+0.278776(x1-3.501149)+0.099469(x2-12.12128).Figure4 illustrates the resulting values of the performance index when running the PFRBFNN based on the HFC-PSSA. The proposed model is also contrasted with some previously developed fuzzy models as shown in Table 3. It is easy to see that the performance of the proposed model is better in the sense of its approximation and prediction abilities.Table 3
Results of comparative analysis (MIS).
Model
PI
E_PI
Index
SONFN [15]
Simplified
40.375
17.898
MSE
Linear
35.745
17.807
MSE
FPNN [16]
SI = 2
32.195
18.642
MSE
SI = 3
32.251
19.622
MSE
GA-based FSONN [17]
PN-based FSONN
18.043
11.898
MSE
FPN-based FSONN
23.739
9.090
MSE
Incremental model [18]
Linear regression
5.877
6.570
RMSE
Incremental model
4.620
6.624
RMSE
GO-FPNN model [19]
SI = 2
2.2141
3.4630
RMSE
SI = 3
0.8852
3.4690
RMSE
SI = 4
0.7818
3.4211
RMSE
Our model
Without the use of PSSA
0.908
1.68
RMSE
With the use of PSSA
0.896
1.65
RMSETrace curves of the performance indexes produced by HFC-PSSA (MIS).
(a)
Training error in successive optimization
(b)
Testing error in successive optimization


## 4.3. Abalone Data (ABA)

Finally we experiment with the abalone dataset [20–23], which is a larger dataset consisting of 4,177 input-output pairs that concerns the age of abalone predicted on a basis of seven input variables (including length, weight, diameter, etc.). The dataset is split into two separate parts [20, 21]: the construction of the fuzzy model is completed for 2506 data points being regarded as a training set; and the rest of the dataset (i.e., 1671 data points) is retained for testing purposes. RMSE is considered as the performance index.After running 200 generations of PSSA, we obtain the optimal network with eight fuzzy rules with type 3, whose polynomials of the consequence part are as follows:(24)R1:y=7.9872288+11.45947664(x1-0.124376)+451.4966298(x2-0.630624)+⋯+143.9173838(x3-0.282961)-47.44803317(x1-0.124376)(x2-0.630624)+⋯-217.5141906(x1-0.124376)×(x3-0.282961)+162.4868597(x2-0.630624)(x3-0.282961)+⋯-1.667754581(x1-0.124376)2-9.677924915(x2-0.630624)2-134.1737445(x3-0.282961)2,R2:y=-1188.04-855.99(x1-0.124376)+464.2795(x2-0.630624)+904.6096(x3-0.446352)+⋯-1356.86(x1-0.124376)(x2-0.630624)+21.30605(x1-0.124376)(x3-0.446352)+⋯+21.12003(x2-0.630624)(x3-0.446352)+26.0111(x1-0.124376)2-778.713(x2-0.630624)2+⋯-95.3784(x3-0.446352)2,R3:y=-143.405-42.2197(x1-0.124376)+71.0844(x2-1.096818)+⋯+13.19766(x3-0.282961)-51.0632(x1-0.124376)(x2-1.096818)+⋯+1.11368(x1-0.124376)(x3-0.282961)-387.954(x2-1.096818)(x3-0.282961)+⋯-107.995(x1-0.124376)2+622.8024(x2-1.096818)2-95.3784(x3-0.282961)2,R4:y=-532.195-28.6716(x1-0.124376)+228.1629(x2-1.096818)+⋯+499.0509(x3-0.446352)+3659.62(x1-0.124376)(x2-1.096818)+⋯+803.6259(x1-0.124376)(x3-0.446352)+994.0846(x2-1.096818)(x3-0.446352)+⋯+2050.08(x1-0.124376)2+1659.251(x2-1.096818)2-6.39226(x3-0.446352)2,R5:y=-658.152+406.3837(x1-0.162877)+5785.213(x2-0.630624)+⋯-340.025(x3-0.282961)-3561.14(x1-0.162877)(x2-0.630624)+⋯-7102.14(x1-0.162877)(x3-0.282961)+2663.84(x2-0.630624)(x3-0.282961)+⋯-38.9337(x1-0.162877)2-291.782(x2-0.630624)2-108.948(x3-0.282961)2,R6:y=546.4625+55.4919(x1-0.162877)+50.1344(x2-0.630624)+⋯-744.188(x3-0.446352)+66.87478(x1-0.162877)(x2-0.630624)+⋯-28.5836(x1-0.162877)(x3-0.446352)+29.87009(x2-0.630624)(x3-0.446352)+⋯-1321.04(x1-0.162877)2-13672(x2-0.630624)2+744.3283(x3-0.446352)2,R7:y=-954.934+807.175(x1-0.162877)-3140.58(x2-1.096818)+⋯+16.91164(x3-0.282961)+47.5888(x1-0.162877)(x2-1.096818)+⋯-34.206(x1-0.162877)(x3-0.282961)+119.1346(x2-1.096818)(x3-0.282961)+⋯-14.2966(x1-0.162877)2+9.594292(x2-1.096818)2-1.58319(x3-0.282961)2,R8:y=-166.398+4.89712(x1-0.162877)+383.2878(x2-1.096818)+⋯-23.2903(x3-0.446352)-1369.65(x1-0.162877)(x2-1.096818)+⋯+69.9118(x1-0.162877)(x3-0.446352)-330.398(x2-1.096818)(x3-0.446352)+⋯+960.4564(x1-0.162877)2+82.85281(x2-1.096818)2+43.62129(x3-0.446352)2.Figure5 shows performance indexes generated by means of the HFC-PSSA in case of using training data and testing data. Table 4 summarizes the results of comparative analysis of the PFRBFNN using HFC-PSSA when being contrasted with other models. It is clear that the proposed model outperforms several previous fuzzy models reported in the literature.Table 4
Results of comparative analysis (ABA).
Model
PI
E_PI
Index
Linear regression [20]
14.15
17.22
MSE
RBFNN [20]
10.36
10.48
MSE
RBFNN (context-free clustering) [20]
10.54
10.58
MSE
Boosting of granular model [20]
8.39
8.68
MSE
RBFNN II [21]
6.710
6.894
MSE
GA-FIS [22]
2.393
2.454
MSE/2
SP-SVM [23]
7.116
6.123
MSE
Our model
Without the use of PSSA
1.942
2.247
RMSE
With the use of PSSA
1.910
2.243
RMSETrace curves of the performance indexes produced by HFC-PSSA (ABA).
(a)
Training error in successive optimization
(b)
Testing error in successive optimization


## 5. Conclusions

In this study, we have proposed an HFC-PSSA and have introduced a design of PFRBFNN based on the HFC-PSSA. A suite of comparative studies demonstrated that the proposed HFC-PSSA-based PFRBFNN achieve better performance in comparison with some other models reported in the literature.



---

*Source: 745314-2013-11-18.xml*