# An Interval Type-2 Fuzzy System with a Species-Based Hybrid Algorithm for Nonlinear System Control Design

Authors: Chung-Ta Li; Ching-Hung Lee; Feng-Yu Chang; Chih-Min Lin
Journal: Mathematical Problems in Engineering
 (2014)

---

## Abstract

We propose a species-based hybrid of the electromagnetism-like mechanism (EM) and back-propagation algorithms (SEMBP) for an interval type-2 fuzzy neural system with asymmetric membership functions (AIT2FNS) design. The interval type-2 asymmetric fuzzy membership functions (IT2 AFMFs) and the TSK-type consequent part are adopted to implement the network structure in AIT2FNS. In addition, the type reduction procedure is integrated into an adaptive network structure to reduce computational complexity. Hence, the AIT2FNS can enhance the approximation accuracy effectively by using less fuzzy rules. The AIT2FNS is trained by the SEMBP algorithm, which contains the steps of uniform initialization, species determination, local search, total force calculation, movement, and evaluation. It combines the advantages of EM and back-propagation (BP) algorithms to attain a faster convergence and a lower computational complexity. The proposed SEMBP algorithm adopts the uniform method (which evenly scatters solution agents over the feasible solution region) and the species technique to improve the algorithm’s ability to find the global optimum. Finally, two illustrative examples of nonlinear systems control are presented to demonstrate the performance and the effectiveness of the proposed AIT2FNS with the SEMBP algorithm.


---

## Body

## 1. Introduction

In recent years, fuzzy neural networks (FNNs) have been used successfully in many applications [1–20]. For FNNs, the asymmetric fuzzy membership functions (AFMFs) have been discussed and analyzed to improve the approximation accuracies and to effectively reduce the number of fuzzy rules [3, 12, 14, 16, 19, 21]. These AFMFs also provide better performance than traditional fuzzy systems do in the applications of function approximation, modeling, and control. Thus, the learning capability and flexibility can be upgraded. In addition, there has been a growing interest in type-2 fuzzy sets (T2FSs), also known as interval-valued fuzzy sets [1, 2, 10, 11, 20, 22–36]. T2FSs, which are extended from type-1 fuzzy sets, were first proposed by Zadeh [37]. Recently, Mendel and Karnik developed the complete theory of interval type-2 fuzzy logic systems (IT2 FLSs) [28–33, 38]. Many studies have shown that interval-valued fuzzy sets and interval type-2 fuzzy sets (IT2FSs) are the same [12, 28, 29, 32, 37, 39]. The resulting type-2 fuzzy neural network (T2FNN), which adopts IT2FSs as membership functions, provides better performance than those of type-1 do [10–12].In this paper, we propose an interval type-2 fuzzy neural system with AFMFs, referred to as an AIT2FNS, for nonlinear system controller design. The interval type-2 AFMFs (IT2 AFMFs) and the TSK-type consequent part are used to implement the network structure in AIT2FNS. It is well known that the major computational effort of an IT2FLS system is in type reduction, and the most commonly adopted method to accomplish this is the Karnik-Mendel (KM) procedure [32, 33]. The KM procedure computes the left and right endpoints needed to characterize type-2 fuzzy sets. As described in the literature [2], the type reduction is integrated into the network layers; that is, the KM procedure can be removed. This removal effectively reduces the required computational effort. In this paper, we use this integration technique to construct an AIT2FNS with reduced computational complexity.For training of the fuzzy neural systems, the back-propagation (BP) algorithm is a powerful and widely used training technique [11, 14, 17, 18, 31]. This method typically cannot find the global solution even if a local minimum is obtained rapidly. The optimization algorithms are less likely to become stuck in a local minimum than are gradient-based learning algorithms (e.g., genetic algorithms, GAs; particle swarm optimizations, PSOs; and electromagnetism-like mechanisms, EMs) [4–6, 8, 9, 15, 17, 38, 40, 41]. A novel hybrid algorithm, the improved electromagnetism-like mechanism using a BP technique (IEMBP), has been proposed to improve the EM algorithm performance [8, 9]. In IEMBP, the random neighborhood local search of the EM algorithm is replaced by a competitive selection and BP technique. However, a statistical analysis should be performed to obtain average performance results, and results are dependent on the population size.Here, we propose a species-based hybrid algorithm of the EM and BP algorithms (SEMBP) for the proposed AIT2FNS design. Several modifications are introduced to improve performance. One of these modifications is in the initialization step. The initial solution agents are generated using the uniform method [3, 35, 42], and these solution agents are evenly scattered over the feasible solution region. The uniform method does not require statistical analysis, which utilizes repetitive training to obtain average performance data. Additionally, the uniform method which has a lower probability of producing outliers can reduce the computational effort (only one trial should be performed) and can attain better performance. Another modification is the use of a species technique, wherein the population is dynamically divided into subpopulations (or subspecies) based on a similarity measurement, and multiple optima are located. By locating multiple optima, the possibility of finding the global optimum is increased. In these ways, the SEMBP algorithm combines the advantages of the uniform method, the species technique, and the BP local search by species seed. This algorithm has a faster convergence and a lower computational complexity; therefore, it is globally optimized. Finally, we use simulation results of nonlinear system control to illustrate the effectiveness of the AIT2FNS design with the SEMBP algorithm.The remainder of this paper is organized as follows. Section2 introduces the proposed AIT2FNS system. Section 3 introduces the hybrid SEMBP algorithm for AIT2FNS. Section 4 describes the simulation results and compares these results with other methods of nonlinear system control. Finally, Section 5 summarizes our conclusions.


## 2. An Interval Type-2 Fuzzy Neural System with Asymmetric Fuzzy Membership Functions (AIT2FNS)

### 2.1. Interval Type-2 Asymmetric Fuzzy Membership Functions

The literature indicates that using AFMFs can improve approximation accuracy and can effectively reduce the number of fuzzy rules [3, 12, 14, 16, 19, 21]. It also provides better performance than traditional fuzzy systems do in the applications of function approximation, modeling, and control. The learning capability and flexibility can be upgraded. Here, we introduce the IT2AFMFs. According to previous results [12], Gaussian functions are used to construct IT2AFMFs, as shown in Figure 1. Each IT2AFMF is constructed by parts of four Gaussian functions. The upper and lower membership functions (MFs) are constructed using two Gaussian MFs and one segment. The superscripts “l” and “r” denote the left and right curves of MF, respectively. The parameters of the lower and upper MFs are denoted by “-” and “-,” respectively. Accordingly, the upper AFMF is constructed as
(1)
μ
-
F
~
(
x
)
=
{
e
-
(
1
/
2
)
(
(
x
-
m
-
l
)
/
σ
-
l
)
2
,
for
x
≤
m
-
l
1
,
for
m
-
l
<
x
<
m
-
r
e
-
(
1
/
2
)
(
(
x
-
m
-
r
)
/
σ
-
r
)
2
,
for
m
-
r
≤
x
,
where m
-
l and m
-
r denote the means of two Gaussian MFs satisfying m
-
l
≤
m
-
r and σ
-
l and σ
-
r denote the deviations (width) of the two Gaussian MFs. Similarly, the lower AFMF is given as
(2)
μ
_
F
~
(
x
)
=
{
γ
_
·
e
-
(
1
/
2
)
(
(
x
-
m
_
l
)
/
σ
_
l
)
2
,
for
x
≤
m
_
l
γ
_
,
for
m
_
l
<
x
<
m
_
r
γ
_
·
e
-
(
1
/
2
)
(
(
x
-
m
_
r
)
/
σ
_
r
)
2
,
for
m
_
r
≤
x
,
where m
_
l
≤
m
_
r. To avoid a small activation value, γ
_ is chosen so that 0.5
≤
γ
_
≤
1. Thus, the restrictions m
-
l
≤
m
_
l
≤
m
_
r
≤
m
-
r, σ
_
l
≤
σ
-
l, σ
_
r
≤
σ
-
r, 0.5
≤
γ
_
≤
1 should be added to avoid unreasonable MFs.Figure 1
The interval type-2 AFMF [8, 12].


### 2.2. Fuzzy Reasoning of the Proposed AIT2FNS

Given the system input datax
i (i
=
1
,
2
,
…
,
n) and the fact that the AIT2FNS has M fuzzy rules, the jth rule of the proposed AIT2FNS can be expressed as follows.Rule j. If x
1 is F
~
1
j and ⋯ and x
n is F
~
n
j then
(3)
Y
j
=
C
j
0
+
C
j
1
x
1
+
C
j
2
x
2
+
⋯
+
C
j
n
x
n
,
where j
=
1
,
2
,
…
,
M; F
~
i
j is the antecedent fuzzy set that is formed by the IT2AFMFs shown in Figure 1; C
i
j denotes the consequent fuzzy set; Y
j is the output of jth rule. As described above, the antecedent part μ
F
~
i
j is an interval set; that is,
(4)
μ
F
~
i
j
(
x
i
j
)
=
⌊
μ
_
F
~
i
j
(
x
i
j
)
,
μ
-
F
~
i
j
(
x
i
j
)
⌋
.
Then, the firing set of jth rule computed using the product t-norm is
(5)
F
j
(
x
)
=
[
f
_
j
(
x
)
,
f
¯
j
(
x
)
]
,
where f
_
j
(
x
)
=
μ
_
F
~
1
j
(
x
1
j
)
*
⋯
*
μ
_
F
~
n
j
(
x
n
j
) and f
¯
j
(
x
)
=
μ
-
F
~
1
j
(
x
1
j
)
*
⋯
*
μ
-
F
~
n
j
(
x
n
j
). The consequent fuzzy set is also an interval set; that is,
(6)
C
j
i
=
[
c
j
i
-
s
j
i
,
c
j
i
+
s
j
i
]
,
where c
j
i denotes the center of C
j
i, and s
j
i denotes the spread of C
j
i. Hence, the consequent part of Rule j is also an interval set; that is, Y
j
=
[
y
j
l
,
y
j
r
], where
(7)
y
j
l
=
(
c
j
0
+
∑
i
=
1
n
c
j
i
x
i
j
)
-
(
s
j
0
+
∑
i
=
1
n
s
j
i
|
x
i
j
|
)
,
y
j
r
=
(
c
j
0
+
∑
i
=
1
n
c
j
i
x
i
j
)
+
(
s
j
0
+
∑
i
=
1
n
s
j
i
|
x
i
j
|
)
.
Hence, the output of the fuzzy logic system can be obtained using the extension principle and is given as
(8)
Y
TSK
(
u
)
=
[
y
l
,
y
r
]
=


∫
y
1
∈
[
y
1
l
y
1
r
]
⋯

∫
y
M
∈
[
y
M
l
y
M
r
]
∫
f
1
∈
[
f
_
1
f
¯
1
]
⋯
∫
f
M
∈
[
f
_
M
f
¯
M
]
1
∑
j
=
1
M
f
j
y
j
/
∑
j
=
1
M
f
j
.
Herein, we need to compute the left-end point y
l and right-end point y
r by utilizing the KM algorithm [32, 33]. In the KM algorithm, the left-end and right-end points are represented as
(9)
y
l
=
∑
i
=
1
L
f
¯
i
y
i
+
∑
L
+
1
M
f
_
i
y
i
∑
i
=
1
L
f
¯
i
+
∑
L
+
1
M
f
_
i
,
y
r
=
∑
i
=
1
R
f
_
i
y
i
+
∑
R
+
1
M
f
¯
i
y
i
∑
i
=
1
R
f
_
i
+
∑
R
+
1
M
f
¯
i
.
Then, we have to find the proper switch point values L and R by iterative procedure, where more details can be referred to [32, 33]. However, this iterative procedure for finding switch point values is time-wasted. In the proposed AIT2FNS, a simple weight-average method is used to replace the iterative procedure for finding L and R. Hence, we define the left-most and right-most points of firing strength by
(10)
f
j
l
=
ω
-
j
l
f
¯
j
+
ω
_
j
l
f
_
j
ω
-
j
l
+
ω
_
j
l
,
f
j
r
=
ω
-
j
r
f
¯
j
+
ω
_
j
r
f
_
j
ω
-
j
r
+
ω
_
j
r
,
where ω
-
j
l, ω
_
j
l, ω
-
j
r, and ω
_
j
r are adjustable weights. In (10), the uncertainty of the antecedent is reduced such that the requirement of the type-reduction is satisfied. In the sequel, we obtain the left-end point and right-end point of the output of interval type-2 fuzzy inference system as follows:
(11)
y
l
=
∑
j
=
1
M
f
j
l
y
j
l
∑
j
=
1
M
f
j
l
,
y
r
=
∑
j
=
1
M
f
j
r
y
j
r
∑
j
=
1
M
f
j
r
.
Finally, the defuzzified output of AIT2FNS is
(12)
y
(
x
)
=
y
l
+
y
r
2
.As above description, the iterative KM algorithm is replaced by several direct node operations such that the computational complexity of the proposed method is intuitively lower. Therefore, we analyze the computational cost between KM algorithm and left-most and right-most layers. For a two-input-single-output interval type-2 fuzzy system, we accumulate the used times of the addition and the multiplication operations in each method. The comparison results are shown in Table1. These results are performed using Matlab running on a computer with Intel i5-661 3.33 GHz and 3.24 GB of main memory. We can observe that the proposed simplified network structure does reduce the computational effort on computation time and operations.Table 1
Computation results between the KM algorithm and left-most and right-most layer.
Number  of rules
Operation
Computational effort
KM algorithm
Left-most and right-most layer
10
Addition
162.4
60
Multiplication
81.9
50
Computational time
0.000208 sec.
0.000120 sec.
20
Addition
325.6
120
Multiplication
163.3
100
Computational time
0.000278 sec.
0.000136 sec.
30
Addition
534.2
180
Multiplication
267.6
150
Computational time
0.000331 sec.
0.000171 sec.
40
Addition
715.8
240
Multiplication
358.4
200
Computational time
0.000493 sec.
0.000228 sec.


### 2.3. Network Structure of the AIT2FNS

Herein, we introduce the network structure of AIT2FNS. The multi-input-single-output case is considered here for convenience. An AIT2FNS withM fuzzy rules is implemented as the six-layer network shown in Figure 2. It can be noted that the operation of layers 4 and 5 is normalization, which will be introduced as follows. The signal propagation and operation functions of the nodes are indicated in each layer. In the following description, O
i
(
k
) denotes the ith output of a node in the kth layer.Figure 2
Network structure of the proposed AIT2FNS withM rules.Layer 1: Input Layer. For the ith node of layer 1, the net input and output are represented as
(13)
O
i
(
1
)
=
x
i
,
i
=
1
,
…
,
n
,
where x
i represents the ith input to the ith node of layer 1. The nodes in this layer only transmit input values to the next layer directly.Layer 2: Membership Layer. In this layer, each node performs an IT2 AFMF:
(14)
O
i
j
(
2
)
=
μ
F
~
i
j
(
O
i
(
1
)
)
=
[
O
_
i
j
(
2
)
,
O
-
i
j
(
2
)
]
=
[
μ
_
F
~
i
j
(
O
i
(
1
)
)
,
μ
-
F
~
i
j
(
O
i
(
1
)
)
]
,
h
l
i
=
1
,
…
,
n
,
j
=
1
,
…
,
M
,
where the subscript “i
j” indicates the jth term of the ith input.Layer 3: Rule Layer. The links in this layer are used to implement the antecedent matching. Using the productt-norm, the firing strength associated with the jth rule is
(15)
f
_
j
=
μ
_
F
~
1
j
(
O
1
(
1
)
)
*
⋯
*
μ
_
F
~
n
j
(
O
n
(
1
)
)
,
f
¯
j
=
μ
-
F
~
1
j
(
O
1
(
1
)
)
*
⋯
*
μ
-
F
~
n
j
(
O
n
(
1
)
)
,
where μ
_
F
~
i
j
(
·
) and μ
-
F
~
i
j
(
·
) are the lower and upper membership grades of μ
F
~
(
·
), respectively. Therefore, a simple product operation is used. Accordingly,
(16)
O
j
(
3
)
=
[
O
_
j
(
3
)
,
O
-
j
(
3
)
]
=
[
∏
i
=
1
n
O
_
i
j
(
2
)
,
∏
i
=
1
n
O
-
i
j
(
2
)
]
.Layer 4: Left-Most and Right-Most Layer. This layer evaluates the left-most and right-most firing points. Therefore, the output of layer 4 is
(17)
O
j
(
4
)
=
[
O
j
l
(
4
)
,
O
j
r
(
4
)
]
=
[
ω
-
j
l
O
-
j
(
3
)
+
ω
_
j
l
O
_
j
(
3
)
ω
-
j
l
+
ω
_
j
l
,
ω
-
j
r
O
-
j
(
3
)
+
ω
_
j
r
O
_
j
(
3
)
ω
-
j
r
+
ω
_
j
r
]
,
where O
j
l
(
4
) and O
j
r
(
4
) denote the left-most and right-most points of firing strength, respectively; ω
_
j
l, ω
-
j
l, ω
_
j
r, and ω
-
j
r are adjustable weights. According to the results presented in [2], the type reduction is integrated into the corresponding network layers. A simple weight-averaging method is used to allow the adaptive algorithm to evaluate the left-most and right-most firing points. Therefore, the traditional KM procedure, which is an iterative procedure for finding the right-most and left-most points of the centroid, is removed. Comparing with the use of a traditional KM algorithm for type reduction, there is a significant reduction in computational cost.Layer 5: TSK Layer. Because the IT2 FSs are used for the antecedents and the interval type-1 fuzzy sets are used for the consequent sets of the type-2 TSK rules, it is possible to state that the C
j
i terms are interval sets. In other words, C
j
i
=
[
c
j
i
-
s
j
i
,
c
j
i
+
s
j
i
], where i
=
1
,
…
,
n and j
=
1
,
…
,
M. In this expression, c
j
i denotes the center of C
j
i and s
j
i denotes the spread of C
j
i, where s
j
i
≥
0. Therefore, the consequent of rule j is
(18)
TSK
j
=
[
TSK
j
l
,
TSK
j
r
]
=
[
(
c
j
0
+
∑
i
=
1
n
c
j
i
x
i
)
-
(
s
j
0
+
∑
i
=
1
n
s
j
i
|
x
i
|
)
,
=
h
h
(
c
j
0
+
∑
i
=
1
n
c
j
i
x
i
)
+
(
s
j
0
+
∑
i
=
1
n
s
j
i
|
x
i
|
)
]
.
Accordingly, the output of layer 5 is
(19)
O
(
5
)
=
[
O
l
(
5
)
,
O
r
(
5
)
]
=
[
∑
j
=
1
M
O
j
l
(
4
)
TSK
j
l
∑
j
=
1
M
O
j
l
(
4
)
,
∑
j
=
1
M
O
j
r
(
4
)
TSK
j
r
∑
j
=
1
M
O
j
r
(
4
)
]
.Layer 6: Output Layer. Layer 6 is used to implement the defuzzification operation. The output is
(20)
O
(
6
)
=
O
l
(
5
)
+
O
r
(
5
)
2
.Based on the above interval type-2 fuzzy inference system, a connection structure based on thejth fuzzy rule can be illustrated as in Figure 3, where ∑ denotes a normalization-like operation used in Layers 4 and 5 (see (17) and (19)), and TSK
j is introduced in (18).Figure 3
The connection based on thejth rule construction of the proposed AIT2FNS.


## 2.1. Interval Type-2 Asymmetric Fuzzy Membership Functions

The literature indicates that using AFMFs can improve approximation accuracy and can effectively reduce the number of fuzzy rules [3, 12, 14, 16, 19, 21]. It also provides better performance than traditional fuzzy systems do in the applications of function approximation, modeling, and control. The learning capability and flexibility can be upgraded. Here, we introduce the IT2AFMFs. According to previous results [12], Gaussian functions are used to construct IT2AFMFs, as shown in Figure 1. Each IT2AFMF is constructed by parts of four Gaussian functions. The upper and lower membership functions (MFs) are constructed using two Gaussian MFs and one segment. The superscripts “l” and “r” denote the left and right curves of MF, respectively. The parameters of the lower and upper MFs are denoted by “-” and “-,” respectively. Accordingly, the upper AFMF is constructed as
(1)
μ
-
F
~
(
x
)
=
{
e
-
(
1
/
2
)
(
(
x
-
m
-
l
)
/
σ
-
l
)
2
,
for
x
≤
m
-
l
1
,
for
m
-
l
<
x
<
m
-
r
e
-
(
1
/
2
)
(
(
x
-
m
-
r
)
/
σ
-
r
)
2
,
for
m
-
r
≤
x
,
where m
-
l and m
-
r denote the means of two Gaussian MFs satisfying m
-
l
≤
m
-
r and σ
-
l and σ
-
r denote the deviations (width) of the two Gaussian MFs. Similarly, the lower AFMF is given as
(2)
μ
_
F
~
(
x
)
=
{
γ
_
·
e
-
(
1
/
2
)
(
(
x
-
m
_
l
)
/
σ
_
l
)
2
,
for
x
≤
m
_
l
γ
_
,
for
m
_
l
<
x
<
m
_
r
γ
_
·
e
-
(
1
/
2
)
(
(
x
-
m
_
r
)
/
σ
_
r
)
2
,
for
m
_
r
≤
x
,
where m
_
l
≤
m
_
r. To avoid a small activation value, γ
_ is chosen so that 0.5
≤
γ
_
≤
1. Thus, the restrictions m
-
l
≤
m
_
l
≤
m
_
r
≤
m
-
r, σ
_
l
≤
σ
-
l, σ
_
r
≤
σ
-
r, 0.5
≤
γ
_
≤
1 should be added to avoid unreasonable MFs.Figure 1
The interval type-2 AFMF [8, 12].


## 2.2. Fuzzy Reasoning of the Proposed AIT2FNS

Given the system input datax
i (i
=
1
,
2
,
…
,
n) and the fact that the AIT2FNS has M fuzzy rules, the jth rule of the proposed AIT2FNS can be expressed as follows.Rule j. If x
1 is F
~
1
j and ⋯ and x
n is F
~
n
j then
(3)
Y
j
=
C
j
0
+
C
j
1
x
1
+
C
j
2
x
2
+
⋯
+
C
j
n
x
n
,
where j
=
1
,
2
,
…
,
M; F
~
i
j is the antecedent fuzzy set that is formed by the IT2AFMFs shown in Figure 1; C
i
j denotes the consequent fuzzy set; Y
j is the output of jth rule. As described above, the antecedent part μ
F
~
i
j is an interval set; that is,
(4)
μ
F
~
i
j
(
x
i
j
)
=
⌊
μ
_
F
~
i
j
(
x
i
j
)
,
μ
-
F
~
i
j
(
x
i
j
)
⌋
.
Then, the firing set of jth rule computed using the product t-norm is
(5)
F
j
(
x
)
=
[
f
_
j
(
x
)
,
f
¯
j
(
x
)
]
,
where f
_
j
(
x
)
=
μ
_
F
~
1
j
(
x
1
j
)
*
⋯
*
μ
_
F
~
n
j
(
x
n
j
) and f
¯
j
(
x
)
=
μ
-
F
~
1
j
(
x
1
j
)
*
⋯
*
μ
-
F
~
n
j
(
x
n
j
). The consequent fuzzy set is also an interval set; that is,
(6)
C
j
i
=
[
c
j
i
-
s
j
i
,
c
j
i
+
s
j
i
]
,
where c
j
i denotes the center of C
j
i, and s
j
i denotes the spread of C
j
i. Hence, the consequent part of Rule j is also an interval set; that is, Y
j
=
[
y
j
l
,
y
j
r
], where
(7)
y
j
l
=
(
c
j
0
+
∑
i
=
1
n
c
j
i
x
i
j
)
-
(
s
j
0
+
∑
i
=
1
n
s
j
i
|
x
i
j
|
)
,
y
j
r
=
(
c
j
0
+
∑
i
=
1
n
c
j
i
x
i
j
)
+
(
s
j
0
+
∑
i
=
1
n
s
j
i
|
x
i
j
|
)
.
Hence, the output of the fuzzy logic system can be obtained using the extension principle and is given as
(8)
Y
TSK
(
u
)
=
[
y
l
,
y
r
]
=


∫
y
1
∈
[
y
1
l
y
1
r
]
⋯

∫
y
M
∈
[
y
M
l
y
M
r
]
∫
f
1
∈
[
f
_
1
f
¯
1
]
⋯
∫
f
M
∈
[
f
_
M
f
¯
M
]
1
∑
j
=
1
M
f
j
y
j
/
∑
j
=
1
M
f
j
.
Herein, we need to compute the left-end point y
l and right-end point y
r by utilizing the KM algorithm [32, 33]. In the KM algorithm, the left-end and right-end points are represented as
(9)
y
l
=
∑
i
=
1
L
f
¯
i
y
i
+
∑
L
+
1
M
f
_
i
y
i
∑
i
=
1
L
f
¯
i
+
∑
L
+
1
M
f
_
i
,
y
r
=
∑
i
=
1
R
f
_
i
y
i
+
∑
R
+
1
M
f
¯
i
y
i
∑
i
=
1
R
f
_
i
+
∑
R
+
1
M
f
¯
i
.
Then, we have to find the proper switch point values L and R by iterative procedure, where more details can be referred to [32, 33]. However, this iterative procedure for finding switch point values is time-wasted. In the proposed AIT2FNS, a simple weight-average method is used to replace the iterative procedure for finding L and R. Hence, we define the left-most and right-most points of firing strength by
(10)
f
j
l
=
ω
-
j
l
f
¯
j
+
ω
_
j
l
f
_
j
ω
-
j
l
+
ω
_
j
l
,
f
j
r
=
ω
-
j
r
f
¯
j
+
ω
_
j
r
f
_
j
ω
-
j
r
+
ω
_
j
r
,
where ω
-
j
l, ω
_
j
l, ω
-
j
r, and ω
_
j
r are adjustable weights. In (10), the uncertainty of the antecedent is reduced such that the requirement of the type-reduction is satisfied. In the sequel, we obtain the left-end point and right-end point of the output of interval type-2 fuzzy inference system as follows:
(11)
y
l
=
∑
j
=
1
M
f
j
l
y
j
l
∑
j
=
1
M
f
j
l
,
y
r
=
∑
j
=
1
M
f
j
r
y
j
r
∑
j
=
1
M
f
j
r
.
Finally, the defuzzified output of AIT2FNS is
(12)
y
(
x
)
=
y
l
+
y
r
2
.As above description, the iterative KM algorithm is replaced by several direct node operations such that the computational complexity of the proposed method is intuitively lower. Therefore, we analyze the computational cost between KM algorithm and left-most and right-most layers. For a two-input-single-output interval type-2 fuzzy system, we accumulate the used times of the addition and the multiplication operations in each method. The comparison results are shown in Table1. These results are performed using Matlab running on a computer with Intel i5-661 3.33 GHz and 3.24 GB of main memory. We can observe that the proposed simplified network structure does reduce the computational effort on computation time and operations.Table 1
Computation results between the KM algorithm and left-most and right-most layer.
Number  of rules
Operation
Computational effort
KM algorithm
Left-most and right-most layer
10
Addition
162.4
60
Multiplication
81.9
50
Computational time
0.000208 sec.
0.000120 sec.
20
Addition
325.6
120
Multiplication
163.3
100
Computational time
0.000278 sec.
0.000136 sec.
30
Addition
534.2
180
Multiplication
267.6
150
Computational time
0.000331 sec.
0.000171 sec.
40
Addition
715.8
240
Multiplication
358.4
200
Computational time
0.000493 sec.
0.000228 sec.


## 2.3. Network Structure of the AIT2FNS

Herein, we introduce the network structure of AIT2FNS. The multi-input-single-output case is considered here for convenience. An AIT2FNS withM fuzzy rules is implemented as the six-layer network shown in Figure 2. It can be noted that the operation of layers 4 and 5 is normalization, which will be introduced as follows. The signal propagation and operation functions of the nodes are indicated in each layer. In the following description, O
i
(
k
) denotes the ith output of a node in the kth layer.Figure 2
Network structure of the proposed AIT2FNS withM rules.Layer 1: Input Layer. For the ith node of layer 1, the net input and output are represented as
(13)
O
i
(
1
)
=
x
i
,
i
=
1
,
…
,
n
,
where x
i represents the ith input to the ith node of layer 1. The nodes in this layer only transmit input values to the next layer directly.Layer 2: Membership Layer. In this layer, each node performs an IT2 AFMF:
(14)
O
i
j
(
2
)
=
μ
F
~
i
j
(
O
i
(
1
)
)
=
[
O
_
i
j
(
2
)
,
O
-
i
j
(
2
)
]
=
[
μ
_
F
~
i
j
(
O
i
(
1
)
)
,
μ
-
F
~
i
j
(
O
i
(
1
)
)
]
,
h
l
i
=
1
,
…
,
n
,
j
=
1
,
…
,
M
,
where the subscript “i
j” indicates the jth term of the ith input.Layer 3: Rule Layer. The links in this layer are used to implement the antecedent matching. Using the productt-norm, the firing strength associated with the jth rule is
(15)
f
_
j
=
μ
_
F
~
1
j
(
O
1
(
1
)
)
*
⋯
*
μ
_
F
~
n
j
(
O
n
(
1
)
)
,
f
¯
j
=
μ
-
F
~
1
j
(
O
1
(
1
)
)
*
⋯
*
μ
-
F
~
n
j
(
O
n
(
1
)
)
,
where μ
_
F
~
i
j
(
·
) and μ
-
F
~
i
j
(
·
) are the lower and upper membership grades of μ
F
~
(
·
), respectively. Therefore, a simple product operation is used. Accordingly,
(16)
O
j
(
3
)
=
[
O
_
j
(
3
)
,
O
-
j
(
3
)
]
=
[
∏
i
=
1
n
O
_
i
j
(
2
)
,
∏
i
=
1
n
O
-
i
j
(
2
)
]
.Layer 4: Left-Most and Right-Most Layer. This layer evaluates the left-most and right-most firing points. Therefore, the output of layer 4 is
(17)
O
j
(
4
)
=
[
O
j
l
(
4
)
,
O
j
r
(
4
)
]
=
[
ω
-
j
l
O
-
j
(
3
)
+
ω
_
j
l
O
_
j
(
3
)
ω
-
j
l
+
ω
_
j
l
,
ω
-
j
r
O
-
j
(
3
)
+
ω
_
j
r
O
_
j
(
3
)
ω
-
j
r
+
ω
_
j
r
]
,
where O
j
l
(
4
) and O
j
r
(
4
) denote the left-most and right-most points of firing strength, respectively; ω
_
j
l, ω
-
j
l, ω
_
j
r, and ω
-
j
r are adjustable weights. According to the results presented in [2], the type reduction is integrated into the corresponding network layers. A simple weight-averaging method is used to allow the adaptive algorithm to evaluate the left-most and right-most firing points. Therefore, the traditional KM procedure, which is an iterative procedure for finding the right-most and left-most points of the centroid, is removed. Comparing with the use of a traditional KM algorithm for type reduction, there is a significant reduction in computational cost.Layer 5: TSK Layer. Because the IT2 FSs are used for the antecedents and the interval type-1 fuzzy sets are used for the consequent sets of the type-2 TSK rules, it is possible to state that the C
j
i terms are interval sets. In other words, C
j
i
=
[
c
j
i
-
s
j
i
,
c
j
i
+
s
j
i
], where i
=
1
,
…
,
n and j
=
1
,
…
,
M. In this expression, c
j
i denotes the center of C
j
i and s
j
i denotes the spread of C
j
i, where s
j
i
≥
0. Therefore, the consequent of rule j is
(18)
TSK
j
=
[
TSK
j
l
,
TSK
j
r
]
=
[
(
c
j
0
+
∑
i
=
1
n
c
j
i
x
i
)
-
(
s
j
0
+
∑
i
=
1
n
s
j
i
|
x
i
|
)
,
=
h
h
(
c
j
0
+
∑
i
=
1
n
c
j
i
x
i
)
+
(
s
j
0
+
∑
i
=
1
n
s
j
i
|
x
i
|
)
]
.
Accordingly, the output of layer 5 is
(19)
O
(
5
)
=
[
O
l
(
5
)
,
O
r
(
5
)
]
=
[
∑
j
=
1
M
O
j
l
(
4
)
TSK
j
l
∑
j
=
1
M
O
j
l
(
4
)
,
∑
j
=
1
M
O
j
r
(
4
)
TSK
j
r
∑
j
=
1
M
O
j
r
(
4
)
]
.Layer 6: Output Layer. Layer 6 is used to implement the defuzzification operation. The output is
(20)
O
(
6
)
=
O
l
(
5
)
+
O
r
(
5
)
2
.Based on the above interval type-2 fuzzy inference system, a connection structure based on thejth fuzzy rule can be illustrated as in Figure 3, where ∑ denotes a normalization-like operation used in Layers 4 and 5 (see (17) and (19)), and TSK
j is introduced in (18).Figure 3
The connection based on thejth rule construction of the proposed AIT2FNS.


## 3. Species-Based Hybrid Algorithm SEMBP for Training AIT2FNS

This section introduces the proposed species-based hybrid algorithm, SEMBP, for training of the AIT2FNS on nonlinear control applications. The SEMBP algorithm combines the advantages of the EM and BP algorithms with uniform initialization, the species technique, and the BP local search by species seed. This combination results in high-speed convergence, lower computational complexity, and global optimization. Figure4 shows the flow chart of the SEMBP algorithm. There are four phases in the SEMBP algorithm: “initialization,” “evaluation,” “species,” and “IEM operation.” Additionally, the modifications of the EM algorithm are the use of the uniform initialization method, the operation of the charge movement of each species, the local search using the best particle of each species, and the removal of the redundant particles.Figure 4
Description of the proposed SEMBP algorithm.The SEMBP for optimization problem is in the form of(21)
Minimize
f
(
x
)
subject
to
x
∈
S
,
S
=
{
x
∈
ℜ
n
∣
l
k
≤
x
k
≤
u
k
,
l
k
,
u
k
∈
ℜ
,
subhhhhje
.
ct
to
x
∈
S
,
k
=
1
,
…
,
n
}
,
where u
k and l
k are the corresponding upper and lower bounds and f
(
x
) is the function that is being minimized. Each particle x represents a solution where the charge depends on the fitness function f
(
x
).When considering the nonlinear control problem, our goal is to generate a proper control signalu
(
k
) such that the system output y
(
k
) follows the desired trajectory y
r
(
k
), where k is the discrete time index. The AIT2FNS with the SEMBP algorithm plays the role of controller for a nonlinear plant, and the SEMBP-based AIT2FNS control scheme is shown in Figure 5. The tracking error is e
(
k
)
=
y
r
(
k
)
-
y
(
k
). Next, we define the objective function as
(22)
E
(
·
)
=
1
2
∑
k
e
(
k
)
2
.
Our goal is to generate the control signal such that the tracking error approaches zero, that is, to minimize the objective function E
(
·
). Thus, the fitness function can be chosen as the mean-square-error of the tracking error; that is, MSE: ∑
k
=
1
T
e
2
(
k
)
/
T, where T is the data number.Figure 5
The SEMBP-based AIT2FNS control scheme for a nonlinear system.


### 3.1. Initialization Phase

By using the SEMBP algorithm for training the AIT2FNS, each particle denotes a weighting vector with dimensionD (Figure 6). Typically, the initial particles are chosen randomly from the feasible solution region. When using random initialization, the particles may be crowded in a region or the particle diversity may be lost. Therefore, a statistical analysis with repetitive training should be performed. To overcome this requirement, we adopted the uniform initialization method [3, 42, 43]. The initial particles are “uniformly distributed”; that is, all initial particles can be evenly distributed within the high-dimensional region. Furthermore, limited data can be used to achieve credible results, and the required simulation time is greatly reduced. Here, we used the good lattice point method, which is one of the uniform methods for constructing the uniform arrays that are used in the initialization phase. Using the uniform method, a statistical analysis is not necessary, and only one trial should be performed. This reduction in trials reduces the computational effort required for performance analysis. In addition, the uniform method has a lower probability of producing outliers that may deeply affect the results. Here, the good lattice point method of uniform initialization provides a series of uniform arrays, U
N
(
q
m
), for different values of m and q [42, 44]. Based on U
N
(
q
m
), N particles can be generated such that
(23)
x
i
=
[
x
l
1
+
2
u
i
1
-
1
2
q
(
x
u
1
-
x
l
1
)
⋯
x
l
m
+
2
u
i
m
-
1
2
q
(
x
u
m
-
x
l
m
)
]
,
where x
i represents the particles, u
i
k is the element of U
N
(
q
m
), i
=
1
,
…
,
N, and k
=
1
,
…
,
m. One can choose u
k
∈
U such that the lattice points are uniformly scattered in the feasible solution region [42, 44].Figure 6
Particle representation of the AIT2FNS.As suggested by the literature [9], the population size (denoted as PS) can be chosen as the half of problem dimension; that is, PS
=
D
/
2. However, because of the restrictions imposed by a uniform array, the population size is chosen to be approximately D
+
1. By using a prime number, P, the lattice points are uniformly scattered in feasible solution region [42]. If the selected prime number, P, is smaller than the dimension, D, of the problem, we use the peak value of the upper MF. After training, this parameter was not changed and was used to supply the difference of parameters, (
D
-
P
+
1
), in the initial array.


### 3.2. Evaluation Phase

This phase is used to calculate fitness values for the entire set of particles and to take the MSE ranking for each species. Particles that have improved MSEs between the particles of generationg and g
+
1 are retained, and this process can be used to attain better performance from the algorithm. Another task in this phase is to remove the redundant particles as determined by a similarity measurement. The conditions of the “particles combination” process are
(24)
∥
x
i
-
x
j
∥
2
<
r
s
10
,
|
MSE
(
x
i
)
-
MSE
(
x
j
)
MSE
(
x
i
)
|
<
μ
th
,
i
≠
j
,
where ∥
·
∥
2 is the Euclidean norm, r
s is the species radius, and μ
th is the threshold. In general, similar (or identical) particles may converge to the same (or similar) optima; our method retains the best of these similar particles. The other redundant particles do not contribute further to the improvement of convergence. In addition, all particles are retained when these particles are similar to the species seed (the best particle of species). Hence, the particles will be led in the best direction while the efficiency of the SEMBP algorithm is improved. Figure 7 describes the removal of redundant particles when particles are identified by similarity measurement. In the case of particles S
1 and Q
5, the particles satisfy the particles combination conditions, and particle Q
5 should be removed. The fitness values of particles Q
1 and Q
2 are almost equal, but the distance, dis
(
Q
1
,
Q
2
), is not smaller than 0.1
×
r
s. Although particles Q
3 and Q
4 have a similar location, they do not satisfy the other condition for particle removal.Figure 7
Illustrated example of removing redundant particles by similarity measurement in one dimension problem.


### 3.3. Species Determination Phase

The species technique aims to identify multiple speciesi population followed by identification of the best particle in each species. The dominant particle in each species is regarded as the “neighborhood best” and is deemed the “species seed.” All particles that fall within a given distance from the species seed are classified as being of the same species. This distance depends on the radius, r
s, which is the particle’s distance from the seed. Therefore, if r
s is small, many isolated species are created in each generation. These isolated particles tend to prematurely identify a local minimum. If there are not sufficient numbers of particles in each species, the species will stop evolving. However, if r
s is large, it is possible to cover the entire variable range with one species, and the species technique would have no effect.Next, the particles in the population are sorted by their fitness values in the MSE ranking step. Then, during species determination, the best performing particle is denoted as the species seed. The particles whose distance from the first species seed is smaller thanr
s are categorized into the first species. The remaining particles do not categorize to the first species seed, and the particle with the minimum MSE is selected as a new species seed. The remaining particles are then checked to determine if they belong to the new species. These steps are repeated until all of the particles have been categorized. Figure 8 illustrates this phase. In this example, the algorithm locates three species, and particles S
1, S
2, and S
3 are identified as species seeds. It can be noted that there is an overlap between the first species and the third species. Hence, the previously identified species (centered at S
1) dominates the overlap that belongs to the third species. In other words, particle Q should belong to the species led by S
1.Figure 8
Illustration of species determination for a one-dimensional problem.


### 3.4. IEM Operation Phase

There are three steps in the IEM operation phase: “local search for best particle by BP,” “total force calculation,” and “movement.” To improve the random process, the step length(
λ
) in movement is chosen to be one that is expected to accelerate the convergence speed. After species determination, each subspecies proceeds to the total force calculation and movement steps, which are the same as in IEMBPs [9]. Nevertheless, in contrast to that of the complete population, determining the electromagnetic charge of each particle in the subpopulations has lower computational complexity. In other words, the complete population should require PS
×
(
PS
-
1
)
×
⋯
×
1 charge computations during total force calculation, but the subpopulations should only require ∑
s
p
s
×
(
p
s
-
1
)
×
⋯
×
1 computations. In these expressions, s is the number of species and p
s is the number of particles in each of the subpopulation, respectively. The best particle of the subspecies then proceeds to the local search by BP step.


#### 3.4.1. Local Search of Species Seed by BP

The BP technique is adopted to derive a local search procedure in the SEMBP algorithm for AIT2FNS optimization. For clarification, we consider the single-output system and define the error cost function as (22); that is, E
(
g
)
=
(
1
/
2
)
∑
k
e
(
k
)
2, where g is the generation index. Using the BP technique, the updated parameters law is
(25)
W
(
g
+
1
)
=
W
(
g
)
+
Δ
W
(
g
)
=
W
(
g
)
+
η
(
-
∂
E
(
g
)
∂
W
)
,
where η is the learning rate. W
=
[
W
_
,
W
-
,
γ
,
W
ω
,
C
]
T denote the adjustable parameters, where C represents the parameters of the TSK layer, W
ω represents the consequent weights, W
_ represents the parameters of the lower MFs, W
- represents the upper MFs parameters, and γ
_ represents the column vectors; that is,
(26)
C
=
[
c
s
]
T
,
W
ω
=
[
ω
_
l
ω
_
r
ω
-
l
ω
-
r
]
T
,
W
_
=
[
m
_
l
m
_
r
σ
_
l
σ
_
r
]
T
,
W
-
=
[
m
-
l
m
-
r
σ
-
l
σ
-
r
]
T
.
For the nonlinear controller design problem, ∂
E
(
g
)
/
∂
W cannot be obtained directly [7, 9]. The update law should be multiplied by the system information term (i.e., the plant sensitivity, ∂
y
/
∂
u). Based on the results described in the literature [8, 9, 18], the gradient ofE should be replaced by the following:
(27)
∂
E
∂
W
=
∂
E
∂
e
·
∂
e
∂
y
·
∂
y
∂
u
·
∂
u
∂
W
=
-
∑
k
(
e
+
Δ
e
)
∂
u
∂
W
=
-
∑
k
(
e
+
Δ
e
)
∂
O
(
6
)
∂
W
,
where e
(
k
)
=
y
r
(
k
)
-
y
(
k
) and Δ
e
(
k
)
=
e
(
k
)
-
e
(
k
-
1
). Thus, (25) can be rewritten as
(28)
W
(
g
+
1
)
=
W
(
g
)
+
η
∑
k
(
e
+
Δ
e
)
∂
O
(
6
)
(
k
)
∂
W
.
The remaining steps require finding the corresponding partial derivative with respect to each parameter.The Derivation of the Update Law for  C. The update law for the AIT2FNS parameter C is
(29)
c
j
(
k
+
1
)
=
c
j
(
k
)
+
η
∑
k
(
e
+
Δ
e
)
∂
O
(
6
)
(
k
)
∂
c
j
,
s
j
(
k
+
1
)
=
s
j
(
k
)
+
η
∑
k
(
e
+
Δ
e
)
∂
O
(
6
)
(
k
)
∂
s
j
,
where
(30)
∂
O
(
6
)
(
k
)
∂
c
j
0
=
1
2
[
O
j
l
(
4
)
∑
j
=
1
M
O
j
l
(
4
)
+
O
j
r
(
4
)
∑
j
=
1
M
O
j
r
(
4
)
]
,
∂
O
(
6
)
(
k
)
∂
c
j
i
=
1
2
·
x
i
·
[
O
j
l
(
4
)
∑
j
=
1
M
O
j
l
(
4
)
+
O
j
r
(
4
)
∑
j
=
1
M
O
j
r
(
4
)
]
,
∂
O
(
6
)
(
k
)
∂
s
j
0
=
1
2
[
-
O
j
l
(
4
)
∑
j
=
1
M
O
j
l
(
4
)
+
O
j
r
(
4
)
∑
j
=
1
M
O
j
r
(
4
)
]
,
∂
O
(
6
)
(
k
)
∂
s
j
i
=
1
2
·
|
x
i
|
·
[
-
O
j
l
(
4
)
∑
j
=
1
M
O
j
l
(
4
)
+
O
j
r
(
4
)
∑
j
=
1
M
O
j
r
(
4
)
]
.The Derivation of the Update Law for  W
ω. The update law for the AIT2FNS parameter W
ω is
(31)
ω
_
j
(
k
+
1
)
=
ω
_
j
(
k
)
+
η
∑
k
(
e
+
Δ
e
)
∂
O
(
6
)
(
k
)
∂
ω
_
j
,
ω
-
j
(
k
+
1
)
=
ω
-
j
(
k
)
+
η
∑
k
(
e
+
Δ
e
)
∂
O
(
6
)
(
k
)
∂
ω
-
j
,
where
(32)
∂
O
(
6
)
(
k
)
∂
ω
_
j
l
=
1
2
[
TSK
j
l
-
O
l
(
5
)
∑
j
=
1
M
O
j
l
(
4
)
·
O
_
j
(
3
)
-
O
j
l
(
4
)
ω
_
j
l
+
ω
-
j
l
]
,
∂
O
(
6
)
(
k
)
∂
ω
_
j
r
=
1
2
[
TSK
j
r
-
O
r
(
5
)
∑
j
=
1
M
O
j
r
(
4
)
·
O
_
j
(
3
)
-
O
j
r
(
4
)
ω
_
j
r
+
ω
-
j
r
]
,
∂
O
(
6
)
(
k
)
∂
ω
-
j
l
=
1
2
[
TSK
j
l
-
O
l
(
5
)
∑
j
=
1
M
O
j
l
(
4
)
·
O
-
j
(
3
)
-
O
j
l
(
4
)
ω
_
j
l
+
ω
-
j
l
]
,
∂
O
(
6
)
(
k
)
∂
ω
-
j
r
=
1
2
[
TSK
j
r
-
O
r
(
5
)
∑
j
=
1
M
O
j
r
(
4
)
·
O
-
j
(
3
)
-
O
j
r
(
4
)
ω
_
j
r
+
ω
-
j
r
]
.The Derivation of the Update Law for  W
_. The update law for the AIT2FNS parameter W
_ is
(33)
m
_
i
j
(
k
+
1
)
=
m
_
i
j
(
k
)
+
η
∑
k
(
e
+
Δ
e
)
∂
O
(
6
)
(
k
)
∂
m
_
i
j
σ
_
i
j
(
k
+
1
)
=
σ
_
i
j
(
k
)
+
η
∑
k
(
e
+
Δ
e
)
∂
O
(
6
)
(
k
)
∂
σ
_
i
j
,
where
(34)
∂
O
(
6
)
(
k
)
∂
W
_
=
-
1
4
γ
[
TSK
j
l
-
O
l
(
5
)
∑
j
=
1
M
O
j
l
(
4
)
·
ω
_
j
l
·
O
_
j
(
3
)
ω
_
j
l
+
ω
-
j
l
hhhhhh
+
TSK
j
r
-
O
r
(
5
)
∑
j
=
1
M
O
j
r
(
4
)
·
ω
_
j
r
·
O
_
j
(
3
)
ω
_
j
r
+
ω
-
j
r
]
×
∂
(
(
x
-
m
_
)
/
σ
_
)
2
∂
W
_
,
∂
∂
m
_
i
j
l
(
x
i
-
m
_
i
j
l
σ
_
i
j
l
)
2
=
-
2
(
x
i
-
m
_
i
j
l
)
(
σ
_
i
j
l
)
2
,
∂
∂
m
_
i
j
r
(
x
i
-
m
_
i
j
r
σ
_
i
j
r
)
2
=
-
2
(
x
i
-
m
_
i
j
r
)
(
σ
_
i
j
r
)
2
,
∂
∂
σ
_
i
j
l
(
x
i
-
m
_
i
j
l
σ
_
i
j
l
)
2
=
-
2
(
x
i
-
m
_
i
j
l
)
2
(
σ
_
i
j
l
)
3
,
∂
∂
σ
_
i
j
r
(
x
i
-
m
_
i
j
r
σ
_
i
j
r
)
2
=
-
2
(
x
i
-
m
_
i
j
r
)
2
(
σ
_
i
j
r
)
3
.The Derivation of the Update Law for  W
-. The update law for the AIT2FNS parameter W
- is
(35)
m
-
i
j
(
k
+
1
)
=
m
-
i
j
(
k
)
+
η
∑
k
(
e
+
Δ
e
)
∂
O
(
6
)
(
k
)
∂
m
-
i
j
,
σ
-
i
j
(
k
+
1
)
=
σ
-
i
j
(
k
)
+
η
∑
k
(
e
+
Δ
e
)
∂
O
(
6
)
(
k
)
∂
σ
-
i
j
,
where
(36)
∂
O
(
6
)
(
k
)
∂
W
-
=
-
1
4
[
TSK
j
l
-
O
l
(
5
)
∑
j
=
1
M
O
j
l
(
4
)
·
ω
-
j
l
·
O
-
j
(
3
)
ω
_
j
l
+
ω
-
j
l
hhhhh
+
TSK
j
r
-
O
r
(
5
)
∑
j
=
1
M
O
j
r
(
4
)
·
ω
-
j
r
·
O
-
j
(
3
)
ω
_
j
r
+
ω
-
j
r
]
×
∂
(
(
x
-
m
-
)
/
σ
-
)
2
∂
W
-
,
∂
∂
m
-
i
j
l
(
x
i
-
m
-
i
j
l
σ
-
i
j
l
)
2
=
-
2
(
x
i
-
m
-
i
j
l
)
(
σ
-
i
j
l
)
2
,
∂
∂
m
-
i
j
r
(
x
i
-
m
-
i
j
r
σ
-
i
j
r
)
2
=
-
2
(
x
i
-
m
-
i
j
r
)
(
σ
-
i
j
r
)
2
,
∂
∂
σ
-
i
j
l
(
x
i
-
m
-
i
j
l
σ
-
i
j
l
)
2
=
-
2
(
x
i
-
m
-
i
j
l
)
2
(
σ
-
i
j
l
)
3
,
∂
∂
σ
-
i
j
r
(
x
i
-
m
-
i
j
r
σ
-
i
j
r
)
2
=
-
2
(
x
i
-
m
-
i
j
r
)
2
(
σ
-
i
j
r
)
3
.The Derivation of the Update Law for  γ
_. The update law for the AIT2FNS parameter γ
_ is
(37)
γ
_
i
j
(
k
+
1
)
=
γ
_
i
j
(
k
)
+
η
∑
k
(
e
+
Δ
e
)
∂
O
(
6
)
(
k
)
∂
γ
_
i
j
,
where
(38)
∂
O
(
6
)
(
k
)
∂
γ
_
i
j
=
1
2
[
TSK
j
l
-
O
l
(
5
)
∑
j
=
1
M
O
j
l
(
4
)
·
ω
_
j
l
ω
_
j
l
+
ω
-
j
l
hhh
+
TSK
j
r
-
O
r
(
5
)
∑
j
=
1
M
O
j
r
(
4
)
·
ω
_
j
r
ω
_
j
r
+
ω
-
j
r
]
O
_
j
(
3
)
γ
_
i
j
.


## 3.1. Initialization Phase

By using the SEMBP algorithm for training the AIT2FNS, each particle denotes a weighting vector with dimensionD (Figure 6). Typically, the initial particles are chosen randomly from the feasible solution region. When using random initialization, the particles may be crowded in a region or the particle diversity may be lost. Therefore, a statistical analysis with repetitive training should be performed. To overcome this requirement, we adopted the uniform initialization method [3, 42, 43]. The initial particles are “uniformly distributed”; that is, all initial particles can be evenly distributed within the high-dimensional region. Furthermore, limited data can be used to achieve credible results, and the required simulation time is greatly reduced. Here, we used the good lattice point method, which is one of the uniform methods for constructing the uniform arrays that are used in the initialization phase. Using the uniform method, a statistical analysis is not necessary, and only one trial should be performed. This reduction in trials reduces the computational effort required for performance analysis. In addition, the uniform method has a lower probability of producing outliers that may deeply affect the results. Here, the good lattice point method of uniform initialization provides a series of uniform arrays, U
N
(
q
m
), for different values of m and q [42, 44]. Based on U
N
(
q
m
), N particles can be generated such that
(23)
x
i
=
[
x
l
1
+
2
u
i
1
-
1
2
q
(
x
u
1
-
x
l
1
)
⋯
x
l
m
+
2
u
i
m
-
1
2
q
(
x
u
m
-
x
l
m
)
]
,
where x
i represents the particles, u
i
k is the element of U
N
(
q
m
), i
=
1
,
…
,
N, and k
=
1
,
…
,
m. One can choose u
k
∈
U such that the lattice points are uniformly scattered in the feasible solution region [42, 44].Figure 6
Particle representation of the AIT2FNS.As suggested by the literature [9], the population size (denoted as PS) can be chosen as the half of problem dimension; that is, PS
=
D
/
2. However, because of the restrictions imposed by a uniform array, the population size is chosen to be approximately D
+
1. By using a prime number, P, the lattice points are uniformly scattered in feasible solution region [42]. If the selected prime number, P, is smaller than the dimension, D, of the problem, we use the peak value of the upper MF. After training, this parameter was not changed and was used to supply the difference of parameters, (
D
-
P
+
1
), in the initial array.


## 3.2. Evaluation Phase

This phase is used to calculate fitness values for the entire set of particles and to take the MSE ranking for each species. Particles that have improved MSEs between the particles of generationg and g
+
1 are retained, and this process can be used to attain better performance from the algorithm. Another task in this phase is to remove the redundant particles as determined by a similarity measurement. The conditions of the “particles combination” process are
(24)
∥
x
i
-
x
j
∥
2
<
r
s
10
,
|
MSE
(
x
i
)
-
MSE
(
x
j
)
MSE
(
x
i
)
|
<
μ
th
,
i
≠
j
,
where ∥
·
∥
2 is the Euclidean norm, r
s is the species radius, and μ
th is the threshold. In general, similar (or identical) particles may converge to the same (or similar) optima; our method retains the best of these similar particles. The other redundant particles do not contribute further to the improvement of convergence. In addition, all particles are retained when these particles are similar to the species seed (the best particle of species). Hence, the particles will be led in the best direction while the efficiency of the SEMBP algorithm is improved. Figure 7 describes the removal of redundant particles when particles are identified by similarity measurement. In the case of particles S
1 and Q
5, the particles satisfy the particles combination conditions, and particle Q
5 should be removed. The fitness values of particles Q
1 and Q
2 are almost equal, but the distance, dis
(
Q
1
,
Q
2
), is not smaller than 0.1
×
r
s. Although particles Q
3 and Q
4 have a similar location, they do not satisfy the other condition for particle removal.Figure 7
Illustrated example of removing redundant particles by similarity measurement in one dimension problem.


## 3.3. Species Determination Phase

The species technique aims to identify multiple speciesi population followed by identification of the best particle in each species. The dominant particle in each species is regarded as the “neighborhood best” and is deemed the “species seed.” All particles that fall within a given distance from the species seed are classified as being of the same species. This distance depends on the radius, r
s, which is the particle’s distance from the seed. Therefore, if r
s is small, many isolated species are created in each generation. These isolated particles tend to prematurely identify a local minimum. If there are not sufficient numbers of particles in each species, the species will stop evolving. However, if r
s is large, it is possible to cover the entire variable range with one species, and the species technique would have no effect.Next, the particles in the population are sorted by their fitness values in the MSE ranking step. Then, during species determination, the best performing particle is denoted as the species seed. The particles whose distance from the first species seed is smaller thanr
s are categorized into the first species. The remaining particles do not categorize to the first species seed, and the particle with the minimum MSE is selected as a new species seed. The remaining particles are then checked to determine if they belong to the new species. These steps are repeated until all of the particles have been categorized. Figure 8 illustrates this phase. In this example, the algorithm locates three species, and particles S
1, S
2, and S
3 are identified as species seeds. It can be noted that there is an overlap between the first species and the third species. Hence, the previously identified species (centered at S
1) dominates the overlap that belongs to the third species. In other words, particle Q should belong to the species led by S
1.Figure 8
Illustration of species determination for a one-dimensional problem.


## 3.4. IEM Operation Phase

There are three steps in the IEM operation phase: “local search for best particle by BP,” “total force calculation,” and “movement.” To improve the random process, the step length(
λ
) in movement is chosen to be one that is expected to accelerate the convergence speed. After species determination, each subspecies proceeds to the total force calculation and movement steps, which are the same as in IEMBPs [9]. Nevertheless, in contrast to that of the complete population, determining the electromagnetic charge of each particle in the subpopulations has lower computational complexity. In other words, the complete population should require PS
×
(
PS
-
1
)
×
⋯
×
1 charge computations during total force calculation, but the subpopulations should only require ∑
s
p
s
×
(
p
s
-
1
)
×
⋯
×
1 computations. In these expressions, s is the number of species and p
s is the number of particles in each of the subpopulation, respectively. The best particle of the subspecies then proceeds to the local search by BP step.


### 3.4.1. Local Search of Species Seed by BP

The BP technique is adopted to derive a local search procedure in the SEMBP algorithm for AIT2FNS optimization. For clarification, we consider the single-output system and define the error cost function as (22); that is, E
(
g
)
=
(
1
/
2
)
∑
k
e
(
k
)
2, where g is the generation index. Using the BP technique, the updated parameters law is
(25)
W
(
g
+
1
)
=
W
(
g
)
+
Δ
W
(
g
)
=
W
(
g
)
+
η
(
-
∂
E
(
g
)
∂
W
)
,
where η is the learning rate. W
=
[
W
_
,
W
-
,
γ
,
W
ω
,
C
]
T denote the adjustable parameters, where C represents the parameters of the TSK layer, W
ω represents the consequent weights, W
_ represents the parameters of the lower MFs, W
- represents the upper MFs parameters, and γ
_ represents the column vectors; that is,
(26)
C
=
[
c
s
]
T
,
W
ω
=
[
ω
_
l
ω
_
r
ω
-
l
ω
-
r
]
T
,
W
_
=
[
m
_
l
m
_
r
σ
_
l
σ
_
r
]
T
,
W
-
=
[
m
-
l
m
-
r
σ
-
l
σ
-
r
]
T
.
For the nonlinear controller design problem, ∂
E
(
g
)
/
∂
W cannot be obtained directly [7, 9]. The update law should be multiplied by the system information term (i.e., the plant sensitivity, ∂
y
/
∂
u). Based on the results described in the literature [8, 9, 18], the gradient ofE should be replaced by the following:
(27)
∂
E
∂
W
=
∂
E
∂
e
·
∂
e
∂
y
·
∂
y
∂
u
·
∂
u
∂
W
=
-
∑
k
(
e
+
Δ
e
)
∂
u
∂
W
=
-
∑
k
(
e
+
Δ
e
)
∂
O
(
6
)
∂
W
,
where e
(
k
)
=
y
r
(
k
)
-
y
(
k
) and Δ
e
(
k
)
=
e
(
k
)
-
e
(
k
-
1
). Thus, (25) can be rewritten as
(28)
W
(
g
+
1
)
=
W
(
g
)
+
η
∑
k
(
e
+
Δ
e
)
∂
O
(
6
)
(
k
)
∂
W
.
The remaining steps require finding the corresponding partial derivative with respect to each parameter.The Derivation of the Update Law for  C. The update law for the AIT2FNS parameter C is
(29)
c
j
(
k
+
1
)
=
c
j
(
k
)
+
η
∑
k
(
e
+
Δ
e
)
∂
O
(
6
)
(
k
)
∂
c
j
,
s
j
(
k
+
1
)
=
s
j
(
k
)
+
η
∑
k
(
e
+
Δ
e
)
∂
O
(
6
)
(
k
)
∂
s
j
,
where
(30)
∂
O
(
6
)
(
k
)
∂
c
j
0
=
1
2
[
O
j
l
(
4
)
∑
j
=
1
M
O
j
l
(
4
)
+
O
j
r
(
4
)
∑
j
=
1
M
O
j
r
(
4
)
]
,
∂
O
(
6
)
(
k
)
∂
c
j
i
=
1
2
·
x
i
·
[
O
j
l
(
4
)
∑
j
=
1
M
O
j
l
(
4
)
+
O
j
r
(
4
)
∑
j
=
1
M
O
j
r
(
4
)
]
,
∂
O
(
6
)
(
k
)
∂
s
j
0
=
1
2
[
-
O
j
l
(
4
)
∑
j
=
1
M
O
j
l
(
4
)
+
O
j
r
(
4
)
∑
j
=
1
M
O
j
r
(
4
)
]
,
∂
O
(
6
)
(
k
)
∂
s
j
i
=
1
2
·
|
x
i
|
·
[
-
O
j
l
(
4
)
∑
j
=
1
M
O
j
l
(
4
)
+
O
j
r
(
4
)
∑
j
=
1
M
O
j
r
(
4
)
]
.The Derivation of the Update Law for  W
ω. The update law for the AIT2FNS parameter W
ω is
(31)
ω
_
j
(
k
+
1
)
=
ω
_
j
(
k
)
+
η
∑
k
(
e
+
Δ
e
)
∂
O
(
6
)
(
k
)
∂
ω
_
j
,
ω
-
j
(
k
+
1
)
=
ω
-
j
(
k
)
+
η
∑
k
(
e
+
Δ
e
)
∂
O
(
6
)
(
k
)
∂
ω
-
j
,
where
(32)
∂
O
(
6
)
(
k
)
∂
ω
_
j
l
=
1
2
[
TSK
j
l
-
O
l
(
5
)
∑
j
=
1
M
O
j
l
(
4
)
·
O
_
j
(
3
)
-
O
j
l
(
4
)
ω
_
j
l
+
ω
-
j
l
]
,
∂
O
(
6
)
(
k
)
∂
ω
_
j
r
=
1
2
[
TSK
j
r
-
O
r
(
5
)
∑
j
=
1
M
O
j
r
(
4
)
·
O
_
j
(
3
)
-
O
j
r
(
4
)
ω
_
j
r
+
ω
-
j
r
]
,
∂
O
(
6
)
(
k
)
∂
ω
-
j
l
=
1
2
[
TSK
j
l
-
O
l
(
5
)
∑
j
=
1
M
O
j
l
(
4
)
·
O
-
j
(
3
)
-
O
j
l
(
4
)
ω
_
j
l
+
ω
-
j
l
]
,
∂
O
(
6
)
(
k
)
∂
ω
-
j
r
=
1
2
[
TSK
j
r
-
O
r
(
5
)
∑
j
=
1
M
O
j
r
(
4
)
·
O
-
j
(
3
)
-
O
j
r
(
4
)
ω
_
j
r
+
ω
-
j
r
]
.The Derivation of the Update Law for  W
_. The update law for the AIT2FNS parameter W
_ is
(33)
m
_
i
j
(
k
+
1
)
=
m
_
i
j
(
k
)
+
η
∑
k
(
e
+
Δ
e
)
∂
O
(
6
)
(
k
)
∂
m
_
i
j
σ
_
i
j
(
k
+
1
)
=
σ
_
i
j
(
k
)
+
η
∑
k
(
e
+
Δ
e
)
∂
O
(
6
)
(
k
)
∂
σ
_
i
j
,
where
(34)
∂
O
(
6
)
(
k
)
∂
W
_
=
-
1
4
γ
[
TSK
j
l
-
O
l
(
5
)
∑
j
=
1
M
O
j
l
(
4
)
·
ω
_
j
l
·
O
_
j
(
3
)
ω
_
j
l
+
ω
-
j
l
hhhhhh
+
TSK
j
r
-
O
r
(
5
)
∑
j
=
1
M
O
j
r
(
4
)
·
ω
_
j
r
·
O
_
j
(
3
)
ω
_
j
r
+
ω
-
j
r
]
×
∂
(
(
x
-
m
_
)
/
σ
_
)
2
∂
W
_
,
∂
∂
m
_
i
j
l
(
x
i
-
m
_
i
j
l
σ
_
i
j
l
)
2
=
-
2
(
x
i
-
m
_
i
j
l
)
(
σ
_
i
j
l
)
2
,
∂
∂
m
_
i
j
r
(
x
i
-
m
_
i
j
r
σ
_
i
j
r
)
2
=
-
2
(
x
i
-
m
_
i
j
r
)
(
σ
_
i
j
r
)
2
,
∂
∂
σ
_
i
j
l
(
x
i
-
m
_
i
j
l
σ
_
i
j
l
)
2
=
-
2
(
x
i
-
m
_
i
j
l
)
2
(
σ
_
i
j
l
)
3
,
∂
∂
σ
_
i
j
r
(
x
i
-
m
_
i
j
r
σ
_
i
j
r
)
2
=
-
2
(
x
i
-
m
_
i
j
r
)
2
(
σ
_
i
j
r
)
3
.The Derivation of the Update Law for  W
-. The update law for the AIT2FNS parameter W
- is
(35)
m
-
i
j
(
k
+
1
)
=
m
-
i
j
(
k
)
+
η
∑
k
(
e
+
Δ
e
)
∂
O
(
6
)
(
k
)
∂
m
-
i
j
,
σ
-
i
j
(
k
+
1
)
=
σ
-
i
j
(
k
)
+
η
∑
k
(
e
+
Δ
e
)
∂
O
(
6
)
(
k
)
∂
σ
-
i
j
,
where
(36)
∂
O
(
6
)
(
k
)
∂
W
-
=
-
1
4
[
TSK
j
l
-
O
l
(
5
)
∑
j
=
1
M
O
j
l
(
4
)
·
ω
-
j
l
·
O
-
j
(
3
)
ω
_
j
l
+
ω
-
j
l
hhhhh
+
TSK
j
r
-
O
r
(
5
)
∑
j
=
1
M
O
j
r
(
4
)
·
ω
-
j
r
·
O
-
j
(
3
)
ω
_
j
r
+
ω
-
j
r
]
×
∂
(
(
x
-
m
-
)
/
σ
-
)
2
∂
W
-
,
∂
∂
m
-
i
j
l
(
x
i
-
m
-
i
j
l
σ
-
i
j
l
)
2
=
-
2
(
x
i
-
m
-
i
j
l
)
(
σ
-
i
j
l
)
2
,
∂
∂
m
-
i
j
r
(
x
i
-
m
-
i
j
r
σ
-
i
j
r
)
2
=
-
2
(
x
i
-
m
-
i
j
r
)
(
σ
-
i
j
r
)
2
,
∂
∂
σ
-
i
j
l
(
x
i
-
m
-
i
j
l
σ
-
i
j
l
)
2
=
-
2
(
x
i
-
m
-
i
j
l
)
2
(
σ
-
i
j
l
)
3
,
∂
∂
σ
-
i
j
r
(
x
i
-
m
-
i
j
r
σ
-
i
j
r
)
2
=
-
2
(
x
i
-
m
-
i
j
r
)
2
(
σ
-
i
j
r
)
3
.The Derivation of the Update Law for  γ
_. The update law for the AIT2FNS parameter γ
_ is
(37)
γ
_
i
j
(
k
+
1
)
=
γ
_
i
j
(
k
)
+
η
∑
k
(
e
+
Δ
e
)
∂
O
(
6
)
(
k
)
∂
γ
_
i
j
,
where
(38)
∂
O
(
6
)
(
k
)
∂
γ
_
i
j
=
1
2
[
TSK
j
l
-
O
l
(
5
)
∑
j
=
1
M
O
j
l
(
4
)
·
ω
_
j
l
ω
_
j
l
+
ω
-
j
l
hhh
+
TSK
j
r
-
O
r
(
5
)
∑
j
=
1
M
O
j
r
(
4
)
·
ω
_
j
r
ω
_
j
r
+
ω
-
j
r
]
O
_
j
(
3
)
γ
_
i
j
.


## 3.4.1. Local Search of Species Seed by BP

The BP technique is adopted to derive a local search procedure in the SEMBP algorithm for AIT2FNS optimization. For clarification, we consider the single-output system and define the error cost function as (22); that is, E
(
g
)
=
(
1
/
2
)
∑
k
e
(
k
)
2, where g is the generation index. Using the BP technique, the updated parameters law is
(25)
W
(
g
+
1
)
=
W
(
g
)
+
Δ
W
(
g
)
=
W
(
g
)
+
η
(
-
∂
E
(
g
)
∂
W
)
,
where η is the learning rate. W
=
[
W
_
,
W
-
,
γ
,
W
ω
,
C
]
T denote the adjustable parameters, where C represents the parameters of the TSK layer, W
ω represents the consequent weights, W
_ represents the parameters of the lower MFs, W
- represents the upper MFs parameters, and γ
_ represents the column vectors; that is,
(26)
C
=
[
c
s
]
T
,
W
ω
=
[
ω
_
l
ω
_
r
ω
-
l
ω
-
r
]
T
,
W
_
=
[
m
_
l
m
_
r
σ
_
l
σ
_
r
]
T
,
W
-
=
[
m
-
l
m
-
r
σ
-
l
σ
-
r
]
T
.
For the nonlinear controller design problem, ∂
E
(
g
)
/
∂
W cannot be obtained directly [7, 9]. The update law should be multiplied by the system information term (i.e., the plant sensitivity, ∂
y
/
∂
u). Based on the results described in the literature [8, 9, 18], the gradient ofE should be replaced by the following:
(27)
∂
E
∂
W
=
∂
E
∂
e
·
∂
e
∂
y
·
∂
y
∂
u
·
∂
u
∂
W
=
-
∑
k
(
e
+
Δ
e
)
∂
u
∂
W
=
-
∑
k
(
e
+
Δ
e
)
∂
O
(
6
)
∂
W
,
where e
(
k
)
=
y
r
(
k
)
-
y
(
k
) and Δ
e
(
k
)
=
e
(
k
)
-
e
(
k
-
1
). Thus, (25) can be rewritten as
(28)
W
(
g
+
1
)
=
W
(
g
)
+
η
∑
k
(
e
+
Δ
e
)
∂
O
(
6
)
(
k
)
∂
W
.
The remaining steps require finding the corresponding partial derivative with respect to each parameter.The Derivation of the Update Law for  C. The update law for the AIT2FNS parameter C is
(29)
c
j
(
k
+
1
)
=
c
j
(
k
)
+
η
∑
k
(
e
+
Δ
e
)
∂
O
(
6
)
(
k
)
∂
c
j
,
s
j
(
k
+
1
)
=
s
j
(
k
)
+
η
∑
k
(
e
+
Δ
e
)
∂
O
(
6
)
(
k
)
∂
s
j
,
where
(30)
∂
O
(
6
)
(
k
)
∂
c
j
0
=
1
2
[
O
j
l
(
4
)
∑
j
=
1
M
O
j
l
(
4
)
+
O
j
r
(
4
)
∑
j
=
1
M
O
j
r
(
4
)
]
,
∂
O
(
6
)
(
k
)
∂
c
j
i
=
1
2
·
x
i
·
[
O
j
l
(
4
)
∑
j
=
1
M
O
j
l
(
4
)
+
O
j
r
(
4
)
∑
j
=
1
M
O
j
r
(
4
)
]
,
∂
O
(
6
)
(
k
)
∂
s
j
0
=
1
2
[
-
O
j
l
(
4
)
∑
j
=
1
M
O
j
l
(
4
)
+
O
j
r
(
4
)
∑
j
=
1
M
O
j
r
(
4
)
]
,
∂
O
(
6
)
(
k
)
∂
s
j
i
=
1
2
·
|
x
i
|
·
[
-
O
j
l
(
4
)
∑
j
=
1
M
O
j
l
(
4
)
+
O
j
r
(
4
)
∑
j
=
1
M
O
j
r
(
4
)
]
.The Derivation of the Update Law for  W
ω. The update law for the AIT2FNS parameter W
ω is
(31)
ω
_
j
(
k
+
1
)
=
ω
_
j
(
k
)
+
η
∑
k
(
e
+
Δ
e
)
∂
O
(
6
)
(
k
)
∂
ω
_
j
,
ω
-
j
(
k
+
1
)
=
ω
-
j
(
k
)
+
η
∑
k
(
e
+
Δ
e
)
∂
O
(
6
)
(
k
)
∂
ω
-
j
,
where
(32)
∂
O
(
6
)
(
k
)
∂
ω
_
j
l
=
1
2
[
TSK
j
l
-
O
l
(
5
)
∑
j
=
1
M
O
j
l
(
4
)
·
O
_
j
(
3
)
-
O
j
l
(
4
)
ω
_
j
l
+
ω
-
j
l
]
,
∂
O
(
6
)
(
k
)
∂
ω
_
j
r
=
1
2
[
TSK
j
r
-
O
r
(
5
)
∑
j
=
1
M
O
j
r
(
4
)
·
O
_
j
(
3
)
-
O
j
r
(
4
)
ω
_
j
r
+
ω
-
j
r
]
,
∂
O
(
6
)
(
k
)
∂
ω
-
j
l
=
1
2
[
TSK
j
l
-
O
l
(
5
)
∑
j
=
1
M
O
j
l
(
4
)
·
O
-
j
(
3
)
-
O
j
l
(
4
)
ω
_
j
l
+
ω
-
j
l
]
,
∂
O
(
6
)
(
k
)
∂
ω
-
j
r
=
1
2
[
TSK
j
r
-
O
r
(
5
)
∑
j
=
1
M
O
j
r
(
4
)
·
O
-
j
(
3
)
-
O
j
r
(
4
)
ω
_
j
r
+
ω
-
j
r
]
.The Derivation of the Update Law for  W
_. The update law for the AIT2FNS parameter W
_ is
(33)
m
_
i
j
(
k
+
1
)
=
m
_
i
j
(
k
)
+
η
∑
k
(
e
+
Δ
e
)
∂
O
(
6
)
(
k
)
∂
m
_
i
j
σ
_
i
j
(
k
+
1
)
=
σ
_
i
j
(
k
)
+
η
∑
k
(
e
+
Δ
e
)
∂
O
(
6
)
(
k
)
∂
σ
_
i
j
,
where
(34)
∂
O
(
6
)
(
k
)
∂
W
_
=
-
1
4
γ
[
TSK
j
l
-
O
l
(
5
)
∑
j
=
1
M
O
j
l
(
4
)
·
ω
_
j
l
·
O
_
j
(
3
)
ω
_
j
l
+
ω
-
j
l
hhhhhh
+
TSK
j
r
-
O
r
(
5
)
∑
j
=
1
M
O
j
r
(
4
)
·
ω
_
j
r
·
O
_
j
(
3
)
ω
_
j
r
+
ω
-
j
r
]
×
∂
(
(
x
-
m
_
)
/
σ
_
)
2
∂
W
_
,
∂
∂
m
_
i
j
l
(
x
i
-
m
_
i
j
l
σ
_
i
j
l
)
2
=
-
2
(
x
i
-
m
_
i
j
l
)
(
σ
_
i
j
l
)
2
,
∂
∂
m
_
i
j
r
(
x
i
-
m
_
i
j
r
σ
_
i
j
r
)
2
=
-
2
(
x
i
-
m
_
i
j
r
)
(
σ
_
i
j
r
)
2
,
∂
∂
σ
_
i
j
l
(
x
i
-
m
_
i
j
l
σ
_
i
j
l
)
2
=
-
2
(
x
i
-
m
_
i
j
l
)
2
(
σ
_
i
j
l
)
3
,
∂
∂
σ
_
i
j
r
(
x
i
-
m
_
i
j
r
σ
_
i
j
r
)
2
=
-
2
(
x
i
-
m
_
i
j
r
)
2
(
σ
_
i
j
r
)
3
.The Derivation of the Update Law for  W
-. The update law for the AIT2FNS parameter W
- is
(35)
m
-
i
j
(
k
+
1
)
=
m
-
i
j
(
k
)
+
η
∑
k
(
e
+
Δ
e
)
∂
O
(
6
)
(
k
)
∂
m
-
i
j
,
σ
-
i
j
(
k
+
1
)
=
σ
-
i
j
(
k
)
+
η
∑
k
(
e
+
Δ
e
)
∂
O
(
6
)
(
k
)
∂
σ
-
i
j
,
where
(36)
∂
O
(
6
)
(
k
)
∂
W
-
=
-
1
4
[
TSK
j
l
-
O
l
(
5
)
∑
j
=
1
M
O
j
l
(
4
)
·
ω
-
j
l
·
O
-
j
(
3
)
ω
_
j
l
+
ω
-
j
l
hhhhh
+
TSK
j
r
-
O
r
(
5
)
∑
j
=
1
M
O
j
r
(
4
)
·
ω
-
j
r
·
O
-
j
(
3
)
ω
_
j
r
+
ω
-
j
r
]
×
∂
(
(
x
-
m
-
)
/
σ
-
)
2
∂
W
-
,
∂
∂
m
-
i
j
l
(
x
i
-
m
-
i
j
l
σ
-
i
j
l
)
2
=
-
2
(
x
i
-
m
-
i
j
l
)
(
σ
-
i
j
l
)
2
,
∂
∂
m
-
i
j
r
(
x
i
-
m
-
i
j
r
σ
-
i
j
r
)
2
=
-
2
(
x
i
-
m
-
i
j
r
)
(
σ
-
i
j
r
)
2
,
∂
∂
σ
-
i
j
l
(
x
i
-
m
-
i
j
l
σ
-
i
j
l
)
2
=
-
2
(
x
i
-
m
-
i
j
l
)
2
(
σ
-
i
j
l
)
3
,
∂
∂
σ
-
i
j
r
(
x
i
-
m
-
i
j
r
σ
-
i
j
r
)
2
=
-
2
(
x
i
-
m
-
i
j
r
)
2
(
σ
-
i
j
r
)
3
.The Derivation of the Update Law for  γ
_. The update law for the AIT2FNS parameter γ
_ is
(37)
γ
_
i
j
(
k
+
1
)
=
γ
_
i
j
(
k
)
+
η
∑
k
(
e
+
Δ
e
)
∂
O
(
6
)
(
k
)
∂
γ
_
i
j
,
where
(38)
∂
O
(
6
)
(
k
)
∂
γ
_
i
j
=
1
2
[
TSK
j
l
-
O
l
(
5
)
∑
j
=
1
M
O
j
l
(
4
)
·
ω
_
j
l
ω
_
j
l
+
ω
-
j
l
hhh
+
TSK
j
r
-
O
r
(
5
)
∑
j
=
1
M
O
j
r
(
4
)
·
ω
_
j
r
ω
_
j
r
+
ω
-
j
r
]
O
_
j
(
3
)
γ
_
i
j
.


## 4. Simulation Results

This section presents two examples of nonlinear system control to demonstrate the performance of the AIT2FNS with the SEMBP algorithm. The first example is nonlinear plant tracking control. In the second example, we focus on a more practical control problem. Hence, the control of temperature in a water bath system is presented. All simulations were performed using Matlab running on an Intel Pentium 4 computer with a clock rate of 3 GHz and 1.25 GB of main memory. The SEMBP algorithm is adopted to adjust the parameters of an AIT2FNS controller such that the tracking error approaches zero. The block diagram of the SEMBP-based AIT2FNS control scheme for the nonlinear system is shown in Figure5. We adopt MSE as the performance index, which is defined as
(39)
MSE
≡
E
(
g
)
=
1
T
∑
k
=
1
T
e
2
(
k
)
,
where T is the data number.Example 1 (nonlinear plant tracking control).
Consider the tracking control of the following nonlinear system [42]:
(40)
y
(
k
+
1
)
=
y
(
k
)
1
+
y
2
(
k
)
+
u
3
(
k
)
,
where y
(
k
) is the nonlinear system output and u
(
k
) is the control input of the plant. In designing the AIT2FNS controller, the reference trajectory y
r
(
k
) is
(41)
y
r
(
k
)
=
sin
(
π
k
50
)
cos
⁡
(
π
k
30
)
.
The inputs and output of AIT2FNS are y
r
(
k
), y
(
k
-
1
), and u
(
k
), respectively. The total number of time steps, N, is 250. The dimension, D, is 120, and the population size, PS, is 113. The initial parameters for AIT2FNS are generated uniformly between [
-
1.5
,
1.5
]. The threshold, μ
th, is selected to be 0.5, and the learning rate of BP is 0.01. The parameters of the SEMBP algorithm and the AIT2FNS are chosen in a manner described as follows:(i)
total number of rules (R): 4;
(ii)
network structure (layer 1~layer 6): (2-8-4-8-2-1);
(iii)
parameters number of AIT2FNS (D): 120;
(iv)
population size (PS): 113;
(v)
maximum generation (G): 20.The simulation results are shown in Figure9. Figure 9(a) shows the system trajectories after training. A comparison of MSEs between the SEMBP and other algorithms is shown in Figure 9(b). Obviously, the SEMBP algorithm exhibited a similar performance (in terms of MSEs) to the SEM and EM algorithms. However, the SEMBP algorithm spent less computational time and exhibited a better performance than the other algorithms. Table 2 summarizes the corresponding comparisons of simulation results and computational time when using Matlab. After training (20 generations), the MSE of the AIT2FNS was 0.0017631, which was less than the best results of other algorithms (Figure 9(b)). Thus, the AIT2FNS with the SEMBP algorithm spends less time than the other algorithms do to achieve higher accuracy and the optimal AIT2NFS. The final IT2 AFMFs are shown in Figures 10(a) and 10(b). The constructed fuzzy rules are given as follows.Table 2
Comparison of average performance in terms of MSEs and computational complexity for Example2 when using different algorithms (G
=
20; D
=
120; P
s
=
113).
Algorithm
SEMBP
SEM
EM
PSO
GA
Average time
528.4
14933.7
949.5
338.3
333.2
Average MSE
0.0017631
0.0031130
0.0086983
0.0139041
0.0108090
Best MSE
—
0.0022200
0.0029797
0.0089709
0.0066581
Worst MSE
—
0.0037332
0.0244482
0.0313254
0.0140582Simulation results for Example1: (a) output trajectory (solid line: actual output; dotted line: desired output) and (b) the best MSE in 20 generations.
(a)
(b)IT2 AFMFs after training for Example1: (a) MFs for x
1; (b) MFs for x
2.
(a)
(b)Rule 1.
IFx
1 is F
~
11 and x
2 is F
~
21
THEN(42)
Y
1
=
[
-
0.3638,0.0744
]
+
[
-
0.9677
,
-
0.3175
]
x
1
+
[
-
0.8312,0.0316
]
x
2
,where ω
-
=
[
-
1.0174
,
-
1.0077
] and ω
_
=
[
-
1.0100
,
-
1.0100
].Rule 2.
IFx
1 is F
~
12 and x
2 is F
~
22
THEN(43)
Y
2
=
[
-
0.2134,0.8584
]
+
[
-
0.9346,0.3284
]
x
1
+
[
0.0845,1.5459
]
x
2
,where ω
-
=
[
-
0.8089
,
-
0.8089
] and ω
_
=
[
-
0.8089
,
-
0.8089
].Rule 3.
IFx
1 is F
~
13 and x
2 is F
~
23
THEN(44)
Y
3
=
[
0.2733,1.9825
]
+
[
-
0.5960,1.3016
]
x
1
+
[
0.4477,2.5505
]
x
2
,where ω
-
=
[
-
0.6747
,
-
0.6746
] and ω
_
=
[
-
0.6747
,
-
0.6747
].Rule 4.
IFx
1 is F
~
14 and x
2 is F
~
24
THEN(45)
Y
4
=
[
-
2.6773
,
-
0.3213
]
+
[
-
0.2803,2.3337
]
x
1
+
[
-
2.0843,0.7631
]
x
2
,where ω
-
=
[
-
0.4633
,
-
0.3492
] and ω
_
=
[
-
0.4438
,
-
0.4435
].


### 4.1. Discussion of Algorithms

Table2 shows the comparison of MSEs for 20 training generations, and the learning process is repeated for 20 independent runs. Several multiagent based algorithms, SEM, EM, PSO, and GA, are tested. The SEMBP algorithm has a smaller MSE (0.0017631) than the other algorithms do(SEM: 0.0022200, EM: 0.0029797, PSO: 0.0089709, and GA: 0.0066581). Additionally, the SEMBP algorithm only requires 528.4 seconds, which indicates a reduction in the computational complexity. GA and PSO require less computational time, but they obtain worse results. Figure 11 shows a comparison of various algorithms in terms of MSE versus the number of evaluations. The SEMBP algorithm exhibits better performance in terms of MSE and convergence than the other algorithms.Figure 11
MSE versus number of evaluations for algorithms tested in Example1.


### 4.2. Discussion of Network Structure

The comparison results from Example1 using different fuzzy neural networks with SEMBP are shown in Figure 12 and Table 3. Figure 12(a) shows the comparison of different fuzzy neural networks using the same number of rules (R
=
4). Figure 12(b) shows the comparison of different fuzzy neural networks using a similar parameter number. Figure 12 and Table 3 show that the AIT2FNS has a better MSE and a higher convergence speed. Due to the TSK-type consequent part and the AFMFs, the AIT2FNS exhibits better approximation performance than the others networks.Table 3
Comparison of parameter number, computational complexity, and MSE of the SEMBP algorithm for Example1 using different networks and number of rules (G
=
20; P
s
=
113).
Network
Network structure
Number of rules
Parameter number
Time
MSE
AIT2FNS
2-8-4-8-2-1
4
120
528.4
0.0017631
IT2TFNN-A
2-8-4-1
4
104
678.1
0.0085178
2-10-5-1
5
130
1044.6
0.0077511
IT2FNN-A
2-8-4-1
4
88
630.3
0.0063496
IT2FNN
2-8-4-1
4
56
580.6
0.0085290
2-14-7-1
7
98
851.5
0.0062085
FNN
2-8-4-1
4
20
178.1
0.0107710
2-20-10-1
10
50
408.2
0.0039030
2-32-16-1
16
80
633.2
0.0036811
2-44-22-1
22
110
768.8
0.0019379
TSK-FNN
2-8-4-1
4
28
185.3
0.0107014
2-20-10-1
10
70
408.2
0.0036001
2-32-16-1
16
112
684.6
0.0034152Simulation results using different networks for Example1: (a) the same number of rules and (b) a similar parameter number.
(a)
(b)


### 4.3. Discussion of Type-Reduction Method

See Table3; the AIT2FNS and the IT2TFNN-A have similar structure except the type-reduction part. The AIT2FNS adopts the left-most and right-most layers while the IT2TFNN-A adopts the traditional KM algorithm for type-reduction. The result using same rule of AIT2FNS and IT2TFNN-A are 0.0017631 and 0.0085178. The computational costs of them are 528.4 and 678.1 second. As above description, we easily find that the AIT2FNS performs better than IT2TFNN-A with fewer computational effort. We further increased the number of rules to make the parameter number of IT2TFNN-A close to 120. The result improved but is still worse than the result of AIT2FNS.


### 4.4. Discussion of Maximum Generation (G)

The comparison of MSEs for AIT2FNS while using different algorithms for 100 generations is shown in Figure13. Most curves tend to smooth at a maximum generation (G
=
20). Therefore, twenty is suggested as a proper maximum generation for AIT2FNSs.Figure 13
Comparison of MSEs for Example1 in 100 generations.


### 4.5. Discussion of Species Radius(
r
s
)

Table4 shows the comparison results from Example 1 using various species radii. We used d
max
⁡
/
3 (8.439) as the species radius, r
s, which achieves the same MSE with lower computational complexity. In other words, for a species radius smaller than 8.439, the SEMBP algorithm may achieve a similar MSE result, but it requires more computational effort. Therefore, we suggest d
max
⁡
/
3 (8.439) to be a proper species radius for an AIT2FNS optimization using uniform initialization via the SEMBP algorithm.Table 4
Comparison of MSEs and computational complexity for Example1 when using different radii (G
=
20; D
=
120; P
s
=
113).
Species radius
16.879(d
max
⁡/1.5 )
12.659(d
max
⁡/2 )
10.127(d
max
⁡/2.5 )
8.439(d
max
⁡/3 )
7.234(d
max
⁡/3.5 )
Average time
364.3
374.4
419.0
528.4
1040.5
Average MSE
0.011802
0.008501
0.004335
0.0017631
0.0017631


### 4.6. Discussion of Threshold for Particle Recombination (μ
t
h)

Table5 illustrates the discussion of threshold in Example 1. The method of decreasing the threshold can effectively reduce the computational complexity in the SEMBP algorithm. Finally, we set the threshold value to 0.5.Table 5
Comparison of MSEs and computational complexity for Example1 when using different thresholds (G
=
20; D
=
120).
μ
th
1.5
1
0.5
0.1
0.01
Time
468.9
487.3
528.4
551.6
611.0
MSE
0.0023779
0.0019786
0.0017631
0.0017631
0.0017631
Redundant particle
14
12
9
6
0Example 2 (control of a water bath temperature system).
Herein, we present a more practical control problem. The control of the water bath temperature system is given by [5, 15, 45]
(46)
d
y
(
t
)
d
t
=
u
(
t
)
C
+
Y
0
-
y
(
t
)
R
C
,
where y
(
t
) is the system output temperature in °C, u
(
t
) is heating flowing inward the system, Y
0 is the room temperature, C is the equivalent thermal capacity, and R is the equivalent thermal resistance between the system borders and surroundings.Assuming thatR and C are constants, we can rewrite (46) into a discrete-time form with some reasonable approximation. Hence, the system is described as
(47)
y
(
k
+
1
)
=
e
-
α
T
s
y
(
k
)
+
(
β
/
α
)
(
1
-
e
-
α
T
s
)
1
+
e
0.5
y
(
k
)
-
40
u
(
k
)
+
[
1
-
e
-
α
T
s
]
Y
0
.
The system parameters used here are α
=
1.00151
×
10
-
4, β
=
8.67973
×
10
-
3, and Y
0
=
25 (°C), which were obtained from a real world water bath manufacturer [45]. The plant control input u
(
k
) should be positive and is limited between 0 V and 5 V. The reference water temperature y
r
(
k
) is
(48)
y
r
(
k
)
=
{
3
4
°
C
for
k
≤
30
,
(
34
+
0.5
(
k
-
30
)
)
°
C
for
30
<
k
≤
50
,
(
44
+
0.8
(
k
-
50
)
)
°
C
for
50
<
k
≤
70
,
(
60
+
0.5
(
k
-
70
)
)
°
C
for
70
<
k
≤
30
,
7
0
°
C
for
90
<
k
≤
120
.
It can be noted that the upper and lower bounds of the reference water temperature are 34 and 70. The actual water temperature is limited at 10°C~90°C. Thus, the inputs for AIT2FNS are the previous plant output, (
y
(
k
-
1
)
-
50
)
/
20, and the reference water temperature, (
y
r
(
k
-
1
)
-
50
)
/
20, which are scaled to the interval [
-
2
,
2
] by experience. The output of the AIT2FNS algorithm is the plant control input, u
(
k
). Each simulation is performed over 120 time steps. The AIT2FNS controller is used to generate the proper control input such that the system output follows the reference trajectory.In this simulation, the dimension size is 90 and the nearest prime number is 89. Therefore, a population size of 89 is chosen to construct the uniform array using the good lattice point method between[
-
2
,
2
]. The threshold, μ
th, is selected to be 0.5, and the learning rate of BP is set at 0.01. The parameters of the SEMBP algorithm and the AIT2FNS are chosen as follows:(i)
total number of rules (R): 3;
(ii)
network structure (layer 1~layer 6): (2-6-3-6-2-1);
(iii)
parameters number of AIT2FNS (D): 90;
(iv)
population size (PS): 89;
(v)
maximum generation (G): 20.The simulation results are shown in Figure14. Figure 14(a) shows the system trajectories after 20 training generations (solid line: plant output; dotted line: reference signal; blue dashed line: control signal). A comparison of the MSE results between the SEMBP and each algorithm’s best result (SEM, EM, PSO, and GA) is shown in Figure 14(b). The final IT2 AFMFs for x
1 and x
2 are shown in Figures 15(a) and 15(b), respectively. The constructed fuzzy rules are as follows.Simulation results of Example2: (a) output trajectories (solid line: plant output; dotted line: reference signal; blue dashed line: control signal) and (b) comparison results in MSE after 20 generations.
(a)
(b)Membership functions for Example2: (a) MFs for x
1 after training and (b) MFs for x
2 after training.
(a)
(b)Rule 1.
IFx
1 is F
~
11 and x
2 is F
~
21
THEN(49)
Y
1
=
[
-
1.1280
,
-
0.0510
]
+
[
-
1.5286,2.2266
]
x
1
+
[
-
1.8038,0.7194
]
x
2
,where ω
-
=
[
0.9664,1.0040
] and ω
_
=
[
1.0040,1.0040
].Rule 2.
IFx
1 is F
~
12 and x
2 is F
~
22
THEN(50)
Y
2
=
[
0.4594,1.6820
]
+
[
-
2.8075,0.9879
]
x
1
+
[
0.4894,3.0392
]
x
2
,where ω
-
=
[
-
0.3894
,
-
0.3894
] and ω
_
=
[
-
0.3894
,
-
0.3894
].Rule 3.
IFx
1 is F
~
13 and x
2 is F
~
23
THEN(51)
Y
3
=
[
-
1.8313
,
-
0.5693
]
+
[
-
0.2372,3.6584
]
x
1
+
[
-
0.7165,1.9071
]
x
2
,where ω
-
=
[
-
1.5306
,
-
1.5279
] and ω
_
=
[
-
1.5285
,
-
1.5283
].


### 4.7. Discussion of Algorithms

The curves in Figure14(b) seem to settle at the similar value since they are depicted by their best results. However, as shown in Figure 14(b), we can observe that the SEMBP algorithm has the fastest convergence speed than other algorithms. Table 6 summarizes the comparison of MSE results and computational effort for 20 training generations repeated for 20 independent runs (algorithms: SEM, EM, PSO, and GA). The MSE of the SEMBP algorithm (MSE: 1.9181) is smaller than the other average MSEs of other algorithms (SEM: 1.9279, EM: 2.2301, PSO: 2.1697, and GA: 2.2299). The SEM algorithm also exhibits better simulation results than the EM algorithm does. This result indicates that the species technique can enhance the approximation accuracy. For comparison of computational complexity, the SEMBP algorithm requires 276.1 seconds, which indicates a reduction in the computational effort. Additionally, the SEMBP algorithm produced better MSEs than the GA and PSO algorithms did, which had similar (or smaller) computational times. From the best MSE, the SEM and PSO algorithms exhibit similar performance to that of the SEMBP algorithm. However, they require more independent trials, where only one trial is needed for the SEMBP. Additionally, Figure 16 shows the comparison results in MSEs versus the number of evaluations for the best result of each algorithm. The SEMBP algorithm also achieves better MSEs for the same number of evaluations. As above description, we can conclude that the SEMBP demonstrates the better performance than other algorithms.Table 6
Comparison results of algorithms, in terms of MSE and computational complexity, for Example2 (G
=
20; D
=
90; P
s
=
89).
Algorithm
SEMBP
SEM
EM
PSO
GA
Average time
276.1
4981.5
465.2
232.4
236.0
Average MSE
1.9181
1.9279
2.2301
2.1697
2.2299
Best MSE
—
1.9188
1.9689
1.9191
1.9443
Worst MSE
—
1.9584
2.8230
2.5505
2.5018Figure 16
Comparison of MSE versus number of evaluations for algorithms examined in Example2.


### 4.8. Discussion of Network Structures

The comparative results of the network structure for Example2 are shown in Figure 17. Figure 17(a) shows the comparison of different fuzzy neural networks using the same number of rules (R
=
3). Figure 17(b) shows the comparison of different fuzzy neural networks using a similar parameter number. Figure 17 demonstrates that the AIT2FNS algorithm has a better initial MSE and a higher convergence speed. As seen in Table 7, the AIT2FNS performs better than the other algorithms do while using the same number of rules (R
=
3). Obviously, the AIT2FNS using TSK fuzzy rules and the AFMFs demonstrate better performance than the other networks do.Table 7
Comparison of parameter number, computational complexity, and MSEs with the SEMBP algorithm for Example2 while using different networks and number of rules (G
=
20).
Network
Network structure
Number of rules
Parameter number
Time
MSE
AIT2FNS
2-6-3-6-2-1
3
90
276.1
1.9181
IT2TFNN-A
2-6-3-1
3
78
367.7
2.0504
2-8-4-1
4
104
587.8
1.9371
IT2FNN-A
2-6-3-1
3
66
302.9
1.9844
IT2FNN
2-6-3-1
3
42
231.8
3.6466
2-12-6-1
6
84
418.5
1.9694
FNN
2-6-3-1
3
15
94.4
11.4574
2-12-6-1
6
30
146.4
4.9611
2-24-12-1
12
60
272.2
2.0214
2-34-17-1
17
85
344.3
1.9210
TSK-FNN
2-6-3-1
3
21
141.8
8.5124
2-12-6-1
6
42
187.8
3.8451
2-24-12-1
12
84
308.56
1.9332Simulation results comparing the different networks for Example2: (a) the same number of rules and (b) the similar parameter number.
(a)
(b)


### 4.9. Discussion of Type-Reduction Method

See Table7; the results using same rule of AIT2FNS and IT2TFNN-A are 01.9181 and 2.0504. The computational costs of them are 276.1 and 367.7 second. Obviously, the AIT2FNS performs better than IT2TFNN-A with fewer computational effort. We also increased the number of rule to make the parameter number of IT2TFNN-A similar to AIT2FNS case. The result in MSE is 1.9371 which is very close to the result of AIT2FNS. However, the computational cost is much huger.


### 4.10. Discussion of Maximum Generation (G)

The comparison of AIT2FNS, in terms of MSE, with regard to the use of different algorithms (whereD
=
90, P
s
=
89) for 100 generations is shown in Figure 18. These six algorithms achieve similar MSEs when the number of generations is sufficiently large (
G
>
40
). Also, the curves tend to smooth at approximately generation (
G
=
20
). Therefore, we suggest that 20 is a proper maximum generation for AIT2FNS optimization. This parameter can be modified to be the MSE specification, the maximum number of evaluations, or other stop criterion.Figure 18
Comparison of MSEs for 100 generations.


### 4.11. Discussion of Species Radius (r
s)

Table8 shows the comparison of results from Example 2 when using different species radii. The good lattice point method is used to construct the uniform initialization; therefore, the population size in Example 2 is chosen as PS
=
89, where D
=
90 is the dimension of the problem. We use d
max
⁡
/
3 (11.6859) as the species radius r
s, which achieves a similar MSE and requires less computational complexity. In other words, if the species radius is smaller than 11.6859, the SEMBP algorithm achieves a similar result in terms of MSE, but it requires more computational effort. Therefore, we propose d
max
⁡
/
3 (11.6859) to be a proper species radius for AIT2FNS optimization via the SEMBP algorithm.Table 8
Comparison of MSEs and computational complexity for Example2 while using different radii (G
=
20; D
=
90; P
s
=
89).
Species radius
19.4765(d
max
⁡/2 )
14.6074(d
max
⁡/2.5 )
11.6859 (d
max
⁡/3 )
9.7382(d
max
⁡/3.5 )
8.3471(d
max
⁡/4 )
Average time
230.0
240.2
276.1
473.6
558.9
Average MSE
1.9211
1.9211
1.9181
1.9181
1.9181


### 4.12. Discussion of Threshold for Particle Recombination (μ
t
h)

Table9 shows the comparative results from Example 2 while using different thresholds μ
th. The smaller threshold may have more redundant particles, which cause redundant computation. Alternately, a larger threshold, which requires less time, removes more redundant particles. Nevertheless, higher MSE values are obtained because too many particles are removed. As shown in the above discussion, a smaller threshold not only attains better results but also reduces computational complexity. We conclude that the method for decreasing the threshold can effectively reduce computational complexity in the SEMBP algorithm. Finally, we set the threshold, μ
th, to be 0.5.Table 9
Comparison of MSEs and computational complexity for Example2 when using different thresholds (G
=
20; D
=
90).
μ
th
0.8
0.5
0.1
0.01
0.001
Time
268.9
276.1
283.2
293.6
305.8
MSE
2.012
1.9181
1.9181
1.9181
1.9186
Redundant particles
9
7
4
1
0


## 4.1. Discussion of Algorithms

Table2 shows the comparison of MSEs for 20 training generations, and the learning process is repeated for 20 independent runs. Several multiagent based algorithms, SEM, EM, PSO, and GA, are tested. The SEMBP algorithm has a smaller MSE (0.0017631) than the other algorithms do(SEM: 0.0022200, EM: 0.0029797, PSO: 0.0089709, and GA: 0.0066581). Additionally, the SEMBP algorithm only requires 528.4 seconds, which indicates a reduction in the computational complexity. GA and PSO require less computational time, but they obtain worse results. Figure 11 shows a comparison of various algorithms in terms of MSE versus the number of evaluations. The SEMBP algorithm exhibits better performance in terms of MSE and convergence than the other algorithms.Figure 11
MSE versus number of evaluations for algorithms tested in Example1.


## 4.2. Discussion of Network Structure

The comparison results from Example1 using different fuzzy neural networks with SEMBP are shown in Figure 12 and Table 3. Figure 12(a) shows the comparison of different fuzzy neural networks using the same number of rules (R
=
4). Figure 12(b) shows the comparison of different fuzzy neural networks using a similar parameter number. Figure 12 and Table 3 show that the AIT2FNS has a better MSE and a higher convergence speed. Due to the TSK-type consequent part and the AFMFs, the AIT2FNS exhibits better approximation performance than the others networks.Table 3
Comparison of parameter number, computational complexity, and MSE of the SEMBP algorithm for Example1 using different networks and number of rules (G
=
20; P
s
=
113).
Network
Network structure
Number of rules
Parameter number
Time
MSE
AIT2FNS
2-8-4-8-2-1
4
120
528.4
0.0017631
IT2TFNN-A
2-8-4-1
4
104
678.1
0.0085178
2-10-5-1
5
130
1044.6
0.0077511
IT2FNN-A
2-8-4-1
4
88
630.3
0.0063496
IT2FNN
2-8-4-1
4
56
580.6
0.0085290
2-14-7-1
7
98
851.5
0.0062085
FNN
2-8-4-1
4
20
178.1
0.0107710
2-20-10-1
10
50
408.2
0.0039030
2-32-16-1
16
80
633.2
0.0036811
2-44-22-1
22
110
768.8
0.0019379
TSK-FNN
2-8-4-1
4
28
185.3
0.0107014
2-20-10-1
10
70
408.2
0.0036001
2-32-16-1
16
112
684.6
0.0034152Simulation results using different networks for Example1: (a) the same number of rules and (b) a similar parameter number.
(a)
(b)


## 4.3. Discussion of Type-Reduction Method

See Table3; the AIT2FNS and the IT2TFNN-A have similar structure except the type-reduction part. The AIT2FNS adopts the left-most and right-most layers while the IT2TFNN-A adopts the traditional KM algorithm for type-reduction. The result using same rule of AIT2FNS and IT2TFNN-A are 0.0017631 and 0.0085178. The computational costs of them are 528.4 and 678.1 second. As above description, we easily find that the AIT2FNS performs better than IT2TFNN-A with fewer computational effort. We further increased the number of rules to make the parameter number of IT2TFNN-A close to 120. The result improved but is still worse than the result of AIT2FNS.


## 4.4. Discussion of Maximum Generation (G)

The comparison of MSEs for AIT2FNS while using different algorithms for 100 generations is shown in Figure13. Most curves tend to smooth at a maximum generation (G
=
20). Therefore, twenty is suggested as a proper maximum generation for AIT2FNSs.Figure 13
Comparison of MSEs for Example1 in 100 generations.


## 4.5. Discussion of Species Radius(
r
s
)

Table4 shows the comparison results from Example 1 using various species radii. We used d
max
⁡
/
3 (8.439) as the species radius, r
s, which achieves the same MSE with lower computational complexity. In other words, for a species radius smaller than 8.439, the SEMBP algorithm may achieve a similar MSE result, but it requires more computational effort. Therefore, we suggest d
max
⁡
/
3 (8.439) to be a proper species radius for an AIT2FNS optimization using uniform initialization via the SEMBP algorithm.Table 4
Comparison of MSEs and computational complexity for Example1 when using different radii (G
=
20; D
=
120; P
s
=
113).
Species radius
16.879(d
max
⁡/1.5 )
12.659(d
max
⁡/2 )
10.127(d
max
⁡/2.5 )
8.439(d
max
⁡/3 )
7.234(d
max
⁡/3.5 )
Average time
364.3
374.4
419.0
528.4
1040.5
Average MSE
0.011802
0.008501
0.004335
0.0017631
0.0017631


## 4.6. Discussion of Threshold for Particle Recombination (μ
t
h)

Table5 illustrates the discussion of threshold in Example 1. The method of decreasing the threshold can effectively reduce the computational complexity in the SEMBP algorithm. Finally, we set the threshold value to 0.5.Table 5
Comparison of MSEs and computational complexity for Example1 when using different thresholds (G
=
20; D
=
120).
μ
th
1.5
1
0.5
0.1
0.01
Time
468.9
487.3
528.4
551.6
611.0
MSE
0.0023779
0.0019786
0.0017631
0.0017631
0.0017631
Redundant particle
14
12
9
6
0Example 2 (control of a water bath temperature system).
Herein, we present a more practical control problem. The control of the water bath temperature system is given by [5, 15, 45]
(46)
d
y
(
t
)
d
t
=
u
(
t
)
C
+
Y
0
-
y
(
t
)
R
C
,
where y
(
t
) is the system output temperature in °C, u
(
t
) is heating flowing inward the system, Y
0 is the room temperature, C is the equivalent thermal capacity, and R is the equivalent thermal resistance between the system borders and surroundings.Assuming thatR and C are constants, we can rewrite (46) into a discrete-time form with some reasonable approximation. Hence, the system is described as
(47)
y
(
k
+
1
)
=
e
-
α
T
s
y
(
k
)
+
(
β
/
α
)
(
1
-
e
-
α
T
s
)
1
+
e
0.5
y
(
k
)
-
40
u
(
k
)
+
[
1
-
e
-
α
T
s
]
Y
0
.
The system parameters used here are α
=
1.00151
×
10
-
4, β
=
8.67973
×
10
-
3, and Y
0
=
25 (°C), which were obtained from a real world water bath manufacturer [45]. The plant control input u
(
k
) should be positive and is limited between 0 V and 5 V. The reference water temperature y
r
(
k
) is
(48)
y
r
(
k
)
=
{
3
4
°
C
for
k
≤
30
,
(
34
+
0.5
(
k
-
30
)
)
°
C
for
30
<
k
≤
50
,
(
44
+
0.8
(
k
-
50
)
)
°
C
for
50
<
k
≤
70
,
(
60
+
0.5
(
k
-
70
)
)
°
C
for
70
<
k
≤
30
,
7
0
°
C
for
90
<
k
≤
120
.
It can be noted that the upper and lower bounds of the reference water temperature are 34 and 70. The actual water temperature is limited at 10°C~90°C. Thus, the inputs for AIT2FNS are the previous plant output, (
y
(
k
-
1
)
-
50
)
/
20, and the reference water temperature, (
y
r
(
k
-
1
)
-
50
)
/
20, which are scaled to the interval [
-
2
,
2
] by experience. The output of the AIT2FNS algorithm is the plant control input, u
(
k
). Each simulation is performed over 120 time steps. The AIT2FNS controller is used to generate the proper control input such that the system output follows the reference trajectory.In this simulation, the dimension size is 90 and the nearest prime number is 89. Therefore, a population size of 89 is chosen to construct the uniform array using the good lattice point method between[
-
2
,
2
]. The threshold, μ
th, is selected to be 0.5, and the learning rate of BP is set at 0.01. The parameters of the SEMBP algorithm and the AIT2FNS are chosen as follows:(i)
total number of rules (R): 3;
(ii)
network structure (layer 1~layer 6): (2-6-3-6-2-1);
(iii)
parameters number of AIT2FNS (D): 90;
(iv)
population size (PS): 89;
(v)
maximum generation (G): 20.The simulation results are shown in Figure14. Figure 14(a) shows the system trajectories after 20 training generations (solid line: plant output; dotted line: reference signal; blue dashed line: control signal). A comparison of the MSE results between the SEMBP and each algorithm’s best result (SEM, EM, PSO, and GA) is shown in Figure 14(b). The final IT2 AFMFs for x
1 and x
2 are shown in Figures 15(a) and 15(b), respectively. The constructed fuzzy rules are as follows.Simulation results of Example2: (a) output trajectories (solid line: plant output; dotted line: reference signal; blue dashed line: control signal) and (b) comparison results in MSE after 20 generations.
(a)
(b)Membership functions for Example2: (a) MFs for x
1 after training and (b) MFs for x
2 after training.
(a)
(b)Rule 1.
IFx
1 is F
~
11 and x
2 is F
~
21
THEN(49)
Y
1
=
[
-
1.1280
,
-
0.0510
]
+
[
-
1.5286,2.2266
]
x
1
+
[
-
1.8038,0.7194
]
x
2
,where ω
-
=
[
0.9664,1.0040
] and ω
_
=
[
1.0040,1.0040
].Rule 2.
IFx
1 is F
~
12 and x
2 is F
~
22
THEN(50)
Y
2
=
[
0.4594,1.6820
]
+
[
-
2.8075,0.9879
]
x
1
+
[
0.4894,3.0392
]
x
2
,where ω
-
=
[
-
0.3894
,
-
0.3894
] and ω
_
=
[
-
0.3894
,
-
0.3894
].Rule 3.
IFx
1 is F
~
13 and x
2 is F
~
23
THEN(51)
Y
3
=
[
-
1.8313
,
-
0.5693
]
+
[
-
0.2372,3.6584
]
x
1
+
[
-
0.7165,1.9071
]
x
2
,where ω
-
=
[
-
1.5306
,
-
1.5279
] and ω
_
=
[
-
1.5285
,
-
1.5283
].


## 4.7. Discussion of Algorithms

The curves in Figure14(b) seem to settle at the similar value since they are depicted by their best results. However, as shown in Figure 14(b), we can observe that the SEMBP algorithm has the fastest convergence speed than other algorithms. Table 6 summarizes the comparison of MSE results and computational effort for 20 training generations repeated for 20 independent runs (algorithms: SEM, EM, PSO, and GA). The MSE of the SEMBP algorithm (MSE: 1.9181) is smaller than the other average MSEs of other algorithms (SEM: 1.9279, EM: 2.2301, PSO: 2.1697, and GA: 2.2299). The SEM algorithm also exhibits better simulation results than the EM algorithm does. This result indicates that the species technique can enhance the approximation accuracy. For comparison of computational complexity, the SEMBP algorithm requires 276.1 seconds, which indicates a reduction in the computational effort. Additionally, the SEMBP algorithm produced better MSEs than the GA and PSO algorithms did, which had similar (or smaller) computational times. From the best MSE, the SEM and PSO algorithms exhibit similar performance to that of the SEMBP algorithm. However, they require more independent trials, where only one trial is needed for the SEMBP. Additionally, Figure 16 shows the comparison results in MSEs versus the number of evaluations for the best result of each algorithm. The SEMBP algorithm also achieves better MSEs for the same number of evaluations. As above description, we can conclude that the SEMBP demonstrates the better performance than other algorithms.Table 6
Comparison results of algorithms, in terms of MSE and computational complexity, for Example2 (G
=
20; D
=
90; P
s
=
89).
Algorithm
SEMBP
SEM
EM
PSO
GA
Average time
276.1
4981.5
465.2
232.4
236.0
Average MSE
1.9181
1.9279
2.2301
2.1697
2.2299
Best MSE
—
1.9188
1.9689
1.9191
1.9443
Worst MSE
—
1.9584
2.8230
2.5505
2.5018Figure 16
Comparison of MSE versus number of evaluations for algorithms examined in Example2.


## 4.8. Discussion of Network Structures

The comparative results of the network structure for Example2 are shown in Figure 17. Figure 17(a) shows the comparison of different fuzzy neural networks using the same number of rules (R
=
3). Figure 17(b) shows the comparison of different fuzzy neural networks using a similar parameter number. Figure 17 demonstrates that the AIT2FNS algorithm has a better initial MSE and a higher convergence speed. As seen in Table 7, the AIT2FNS performs better than the other algorithms do while using the same number of rules (R
=
3). Obviously, the AIT2FNS using TSK fuzzy rules and the AFMFs demonstrate better performance than the other networks do.Table 7
Comparison of parameter number, computational complexity, and MSEs with the SEMBP algorithm for Example2 while using different networks and number of rules (G
=
20).
Network
Network structure
Number of rules
Parameter number
Time
MSE
AIT2FNS
2-6-3-6-2-1
3
90
276.1
1.9181
IT2TFNN-A
2-6-3-1
3
78
367.7
2.0504
2-8-4-1
4
104
587.8
1.9371
IT2FNN-A
2-6-3-1
3
66
302.9
1.9844
IT2FNN
2-6-3-1
3
42
231.8
3.6466
2-12-6-1
6
84
418.5
1.9694
FNN
2-6-3-1
3
15
94.4
11.4574
2-12-6-1
6
30
146.4
4.9611
2-24-12-1
12
60
272.2
2.0214
2-34-17-1
17
85
344.3
1.9210
TSK-FNN
2-6-3-1
3
21
141.8
8.5124
2-12-6-1
6
42
187.8
3.8451
2-24-12-1
12
84
308.56
1.9332Simulation results comparing the different networks for Example2: (a) the same number of rules and (b) the similar parameter number.
(a)
(b)


## 4.9. Discussion of Type-Reduction Method

See Table7; the results using same rule of AIT2FNS and IT2TFNN-A are 01.9181 and 2.0504. The computational costs of them are 276.1 and 367.7 second. Obviously, the AIT2FNS performs better than IT2TFNN-A with fewer computational effort. We also increased the number of rule to make the parameter number of IT2TFNN-A similar to AIT2FNS case. The result in MSE is 1.9371 which is very close to the result of AIT2FNS. However, the computational cost is much huger.


## 4.10. Discussion of Maximum Generation (G)

The comparison of AIT2FNS, in terms of MSE, with regard to the use of different algorithms (whereD
=
90, P
s
=
89) for 100 generations is shown in Figure 18. These six algorithms achieve similar MSEs when the number of generations is sufficiently large (
G
>
40
). Also, the curves tend to smooth at approximately generation (
G
=
20
). Therefore, we suggest that 20 is a proper maximum generation for AIT2FNS optimization. This parameter can be modified to be the MSE specification, the maximum number of evaluations, or other stop criterion.Figure 18
Comparison of MSEs for 100 generations.


## 4.11. Discussion of Species Radius (r
s)

Table8 shows the comparison of results from Example 2 when using different species radii. The good lattice point method is used to construct the uniform initialization; therefore, the population size in Example 2 is chosen as PS
=
89, where D
=
90 is the dimension of the problem. We use d
max
⁡
/
3 (11.6859) as the species radius r
s, which achieves a similar MSE and requires less computational complexity. In other words, if the species radius is smaller than 11.6859, the SEMBP algorithm achieves a similar result in terms of MSE, but it requires more computational effort. Therefore, we propose d
max
⁡
/
3 (11.6859) to be a proper species radius for AIT2FNS optimization via the SEMBP algorithm.Table 8
Comparison of MSEs and computational complexity for Example2 while using different radii (G
=
20; D
=
90; P
s
=
89).
Species radius
19.4765(d
max
⁡/2 )
14.6074(d
max
⁡/2.5 )
11.6859 (d
max
⁡/3 )
9.7382(d
max
⁡/3.5 )
8.3471(d
max
⁡/4 )
Average time
230.0
240.2
276.1
473.6
558.9
Average MSE
1.9211
1.9211
1.9181
1.9181
1.9181


## 4.12. Discussion of Threshold for Particle Recombination (μ
t
h)

Table9 shows the comparative results from Example 2 while using different thresholds μ
th. The smaller threshold may have more redundant particles, which cause redundant computation. Alternately, a larger threshold, which requires less time, removes more redundant particles. Nevertheless, higher MSE values are obtained because too many particles are removed. As shown in the above discussion, a smaller threshold not only attains better results but also reduces computational complexity. We conclude that the method for decreasing the threshold can effectively reduce computational complexity in the SEMBP algorithm. Finally, we set the threshold, μ
th, to be 0.5.Table 9
Comparison of MSEs and computational complexity for Example2 when using different thresholds (G
=
20; D
=
90).
μ
th
0.8
0.5
0.1
0.01
0.001
Time
268.9
276.1
283.2
293.6
305.8
MSE
2.012
1.9181
1.9181
1.9181
1.9186
Redundant particles
9
7
4
1
0


## 5. Conclusion

In this paper, we propose a novel interval type-2 fuzzy neural system with the hybrid-learning algorithm SEMBP for nonlinear controller design. The proposed AIT2FNS method uses a type-2 AFMFS and a TSK-type consequent part to enhance the performance of the IT2FNN. In addition, the type reduction was integrated into the adaptive network layers, and this integration reduced the computational complexity. The update laws of the AIT2FNS for nonlinear system control are given by SEMBP. The SEMBP algorithm combines EM and BP with the species technique and uniform initialization to obtain faster convergence, reduced computational complexity, and global optimization. Simulation results including nonlinear system tracking control and water bath temperature control were presented to demonstrate the effectiveness and the performance of the proposed SEMBP and AIT2FNS methods.



---

*Source: 735310-2014-04-24.xml*