# Universal Approximation of a Class of Interval Type-2 Fuzzy Neural Networks in Nonlinear Identification

Authors: Oscar Castillo; Juan R. Castro; Patricia Melin; Antonio Rodriguez-Diaz
Journal: Advances in Fuzzy Systems
 (2013)

---

## Abstract

Neural networks (NNs), type-1 fuzzy logic systems (T1FLSs), and interval type-2 fuzzy logic systems (IT2FLSs) have been shown to be universal approximators, which means that they can approximate any nonlinear continuous function. Recent research shows that embedding an IT2FLS on an NN can be very effective for a wide number of nonlinear complex systems, especially when handling imperfect or incomplete information. In this paper we show, based on the Stone-Weierstrass theorem, that an interval type-2 fuzzy neural network (IT2FNN) is a universal approximator, which uses a set of rules and interval type-2 membership functions (IT2MFs) for this purpose. Simulation results of nonlinear function identification using the IT2FNN for one and three variables and for the Mackey-Glass chaotic time series prediction are presented to illustrate the concept of universal approximation.


---

## Body

## 1. Introduction

Several authors have contributed to universal approximation results. An overview can be found in [1–8]; further references to prime contributors in function approximations by neural networks are in [4, 9–12] and type-2 fuzzy logic modeling in [13–23]. It has been shown that a three-layer NN can approximate any real continuous function [24]. The same has been shown for a T1FLS [1, 25] using the Stone-Weierstrass theorem [3]. A similar analysis was made by Kosko [2, 9] using the concept of fuzzy regions. In [3, 26] Buckley shows that, with a Sugeno model [27], a T1FLS can be built with the ability to approximate any nonlinear continuous function. Also, combining the neural and fuzzy logic paradigms [28, 29], an effective tool can be created for approximating any nonlinear function [4]. In this sense, an expert can use a type-1 fuzzy neural network (T1FNN) [10–12, 30] or IT2FNN systems and find interpretable solutions [15–17, 31–34]. In general, Takagi-Sugeno-Kang (TSK) T1FLSs are able to approximate by the use of polynomial consequent rules [7, 27]. This paper uses the Levenberg-Marquardt backpropagation learning algorithm for adapting antecedent and consequent parameters for an adaptive IT2FNN, since its efficiency and soundness characteristics make them fit for these optimizing problems. An Adaptive IT2FNN is used as a universal approximator of any nonlinear functions. A set Ψ of IT2FNNs is universal if and only if (iff), given any process Ω, there is a system Φ∈Ψ such that the difference between the output from IT2FNN and output from Ω is less than a given ε.In this paper the main contribution is the proposed IT2FNNs architectures, which are shown to be universal approximators and are illustrated with several benchmark problems to verify their applicability for real world problems.


## 2. Interval Type-2 Fuzzy Neural Networks

An IT2FNN [15, 31, 35] combines a TSK interval type-2 fuzzy inference system (TSKIT2FIS) [13, 14, 33, 34] with an adaptive NN in order to take advantage of both models best characteristics. In general, when representing IT2FNN graphically, rectangles are used to represent adaptive nodes and circles to represent nonadaptive nodes. Output values of pair nodes (green color) and odd nodes (blue color) represent uncertainty intervals (Figures 1–4). In this kind of interval type-2 neurofuzzy adaptive networks, nodes represent processing units called neurons, which can be classified into crisp and fuzzy neurons.Figure 1
IT2FNN-1 architecture.Figure 2
IT2FNN-3 architecture.Figure 3
IT2FNN-0 architecture.Figure 4
IT2FNN-2 architecture.The IT2FNN-1 architecture has 5 layers (Figure1) [35] and consists of adaptive nodes with equivalent function to lower-upper membership in fuzzification layer (layer 1). Nonadaptive nodes in rules layer (layer 2) interconnect with the fuzzification layer (layer 1) in order to generate TSK IT2FIS rules antecedents. Adaptive nodes in the consequent layer (layer 3) are connected to the input layer (layer 0) to generate rules consequents. Nonadaptive nodes in type-reduction layer (layer 4) evaluate left-right values with the Karnik and Mendel (KM) [13, 14] algorithm. The nonadaptive nodes in the defuzzification layer (layer 5) average left-right values.The IT2FNN-3 architecture has 8 layers (Figure2) [35] and uses IT2FN for fuzzifying the inputs (layers 1-2). Nonadaptive nodes in rules layer (layer 3) interconnect with lower-upper linguistic values layer (layer 2) to generate TSK IT2FIS rules antecedents. Adaptive nodes in layer 4 adapt left-right firing strength, biasing rules lower-upper trigger forces with synaptic weights between layers 3 and 4. Layer 5’s nonadaptive nodes normalize rules lower-upper firing strength. Nonadaptive nodes I consequent layer (layer 6) interconnect with input layer (layer 0) to generate rules consequents. Nonadaptive nodes in type-reduction layer (layer 7) evaluate left-right values adding lower-upper product of lower-upper triggering forces normalized by rules consequent left-right values. Node in defuzzification layer is adaptive and its output y^ is defined as biased average of left-right values and parameter γ. Parameter γ (0.5 by default) adjusts uncertainty interval defined by left-right values [y^l,y^r].Architectures IT2FNN-0 and IT2FNN-2, which will be shown in Sections3.2 and 3.3, respectively, as universal approximators, are described with more details in Section 2.1.


### 2.1. IT2FNN-0 Architecture

An IT2FNN-0 is a seven-layer IT2FNN, which integrates a first order TSKIT2FIS (interval type-2 fuzzy antecedents and real consequents) with an adaptive NN. The IT2FNN-0 (Figure3) layers are described as follows.Layer 0. Inputs
(1)oi0=xi,i=1,…,n.Layer 1. Adaptive type-1 fuzzy neuron (T1FN)
(2)netk1=wk,i1,0xi+bk1∀i=1,…,n,k=1,…,ϑ,ok1=μ(netk1), where the transfer function μ is a membership function, netk1 is the weighted sum of inputs (xi) and the synaptic weights (wk,i1,0), and bk1 is the threshold for each neuron.Layer 2. Nonadaptive T1FN. This layer contains T-norm and S-norm fuzzy nodes
(3)o2k-12=o2k-11·o2k1∀k=1,…,ϑ,T-normfuzzynode,
where ϑ is the number of nodes in layers 1 and 2
(4)o2k2=o2k-11+o2k1-o2k-12,S-normfuzzynode,τ=ℓk,i for all k=1,…,M, and i=1,…,n, where ℓk,i is the table of indices of the antecedents of the rules π=∑j=1i-1vj+|τ|, where π is a vector of indices for each node of layer 2ifτ>0(5)μ_ik=oil(π)2,μ¯ik=oiu(π)2else(6)μ_k,i=null,μ¯k,i=nullend,whereμ_ik,μ¯ik are lower and upper membership function values, respectively. il(π) and iu(π) are vectors with even and odd indices of the nodes of layer 2.Layer 3. Lower-upper firing strength (w_k,w¯k). Having nonadaptive nodes for generating lower-upper firing strength of TSK IT2FIS rules (7),
(7)o2k-13=w_k,o2k3=w¯k,w_k=∏i=1nμ_F~ik(x),w¯k=∏i=1nμ¯F~ik(x),
where μF~ik(x)∈[μ_F~ik(x),μ¯F~ik(x)] is the Gaussian interval type-2 membership function, igaussmtype2 (x,[σik,1mik,2mik]), defined by
(8)1μF~ik(xi,[σik,1mik])=exp[-12(xi-1mikσik)2],(9)2μF~ik(xi,[σik,2mik])=exp[-12(xi-2mikσik)2],(10)μ_Fik(x)={2μF~ik(xi,[σik,2mik]),xi≤1mik,1μF~ik(xi,[σik,1mik]),xi>2mik,(11)μ¯Fik(x)={1μF~ik(xi,[σik,1mik]),xi<1mik,1,1mik≤xi≤2mik,2μF~ik(xi,[σik,2mik]),xi>2mik.Layer 4. Lower-upper firing strength rule normalization (ϕ_k,ϕ¯k). Nodes in this layer are nonadaptive and the output is defined as the ratio between the kth lower-upper firing strength rule (w_k,w¯k) and the sum of lower-upper firing strength of all rules (13) and (14):
(12)o2k-14=ϕ_k,o2k4=ϕ¯k.
If we view ϕ_k,ϕ¯k as fuzzy basis functions (FBF) (32) and (33) and yk(x) as linear function (16), then y^(x) can be viewed as a linear combination of the basis functions (20) and (21):
(13)ϕ_k=w_kDl,k=1,…,M,
where Dl=∑k=1Mw_k,
(14)ϕ¯k=w¯kDr,k=1,…,M,
where Dr=∑k=1Mw¯k.Layer 5. Rule consequents. Each node is adaptive and its parameters are {cik,c0k}. The node’s output corresponds to partial output of kth rule yk (16):
(15)o2k=15=yk,o2k5=yk,(16)yk=∑i=1ncikxi+c0k;k=1,…,M.Layer 6. Estimating left-right interval values [y^l,y^r] (18), nodes are nonadaptive with outputs y^l,y^r. Layer 6 output is defined by
(17)o16=y^l(x),o26=y^r(x),
where
(18)y^l(x)=∑k=1Mϕ_kyk,y^r(x)=∑k=1Mϕ¯kyk.Layer 7. Defuzzification. This layer’s node is adaptive, where the output y^, (20) and (21), is defined as weighted average of left-right values and parameter γ. Parameter γ (default value 0.5) adjusts the uncertainty interval defined by left-right values [y^l,y^r]:
(19)o17=y^(x),
where
(20)y^(x)=γy^l(x)+(1-γ)y^r(x),(21)y^(x)=∑k=1M[γϕ_k(x)+(1-γ)ϕ¯k(x)]yk(x).


### 2.2. IT2FNN-2 Architecture

An IT2FNN-2 [31] is a six-layer IT2FNN, which integrates a first order TSKIT2FIS (interval type-2 fuzzy antecedents and interval type-1 fuzzy consequents), with an adaptive NN. The IT2FNN-2 (Figure 4) layers are described in a similar way to the previous architectures.


## 2.1. IT2FNN-0 Architecture

An IT2FNN-0 is a seven-layer IT2FNN, which integrates a first order TSKIT2FIS (interval type-2 fuzzy antecedents and real consequents) with an adaptive NN. The IT2FNN-0 (Figure3) layers are described as follows.Layer 0. Inputs
(1)oi0=xi,i=1,…,n.Layer 1. Adaptive type-1 fuzzy neuron (T1FN)
(2)netk1=wk,i1,0xi+bk1∀i=1,…,n,k=1,…,ϑ,ok1=μ(netk1), where the transfer function μ is a membership function, netk1 is the weighted sum of inputs (xi) and the synaptic weights (wk,i1,0), and bk1 is the threshold for each neuron.Layer 2. Nonadaptive T1FN. This layer contains T-norm and S-norm fuzzy nodes
(3)o2k-12=o2k-11·o2k1∀k=1,…,ϑ,T-normfuzzynode,
where ϑ is the number of nodes in layers 1 and 2
(4)o2k2=o2k-11+o2k1-o2k-12,S-normfuzzynode,τ=ℓk,i for all k=1,…,M, and i=1,…,n, where ℓk,i is the table of indices of the antecedents of the rules π=∑j=1i-1vj+|τ|, where π is a vector of indices for each node of layer 2ifτ>0(5)μ_ik=oil(π)2,μ¯ik=oiu(π)2else(6)μ_k,i=null,μ¯k,i=nullend,whereμ_ik,μ¯ik are lower and upper membership function values, respectively. il(π) and iu(π) are vectors with even and odd indices of the nodes of layer 2.Layer 3. Lower-upper firing strength (w_k,w¯k). Having nonadaptive nodes for generating lower-upper firing strength of TSK IT2FIS rules (7),
(7)o2k-13=w_k,o2k3=w¯k,w_k=∏i=1nμ_F~ik(x),w¯k=∏i=1nμ¯F~ik(x),
where μF~ik(x)∈[μ_F~ik(x),μ¯F~ik(x)] is the Gaussian interval type-2 membership function, igaussmtype2 (x,[σik,1mik,2mik]), defined by
(8)1μF~ik(xi,[σik,1mik])=exp[-12(xi-1mikσik)2],(9)2μF~ik(xi,[σik,2mik])=exp[-12(xi-2mikσik)2],(10)μ_Fik(x)={2μF~ik(xi,[σik,2mik]),xi≤1mik,1μF~ik(xi,[σik,1mik]),xi>2mik,(11)μ¯Fik(x)={1μF~ik(xi,[σik,1mik]),xi<1mik,1,1mik≤xi≤2mik,2μF~ik(xi,[σik,2mik]),xi>2mik.Layer 4. Lower-upper firing strength rule normalization (ϕ_k,ϕ¯k). Nodes in this layer are nonadaptive and the output is defined as the ratio between the kth lower-upper firing strength rule (w_k,w¯k) and the sum of lower-upper firing strength of all rules (13) and (14):
(12)o2k-14=ϕ_k,o2k4=ϕ¯k.
If we view ϕ_k,ϕ¯k as fuzzy basis functions (FBF) (32) and (33) and yk(x) as linear function (16), then y^(x) can be viewed as a linear combination of the basis functions (20) and (21):
(13)ϕ_k=w_kDl,k=1,…,M,
where Dl=∑k=1Mw_k,
(14)ϕ¯k=w¯kDr,k=1,…,M,
where Dr=∑k=1Mw¯k.Layer 5. Rule consequents. Each node is adaptive and its parameters are {cik,c0k}. The node’s output corresponds to partial output of kth rule yk (16):
(15)o2k=15=yk,o2k5=yk,(16)yk=∑i=1ncikxi+c0k;k=1,…,M.Layer 6. Estimating left-right interval values [y^l,y^r] (18), nodes are nonadaptive with outputs y^l,y^r. Layer 6 output is defined by
(17)o16=y^l(x),o26=y^r(x),
where
(18)y^l(x)=∑k=1Mϕ_kyk,y^r(x)=∑k=1Mϕ¯kyk.Layer 7. Defuzzification. This layer’s node is adaptive, where the output y^, (20) and (21), is defined as weighted average of left-right values and parameter γ. Parameter γ (default value 0.5) adjusts the uncertainty interval defined by left-right values [y^l,y^r]:
(19)o17=y^(x),
where
(20)y^(x)=γy^l(x)+(1-γ)y^r(x),(21)y^(x)=∑k=1M[γϕ_k(x)+(1-γ)ϕ¯k(x)]yk(x).


## 2.2. IT2FNN-2 Architecture

An IT2FNN-2 [31] is a six-layer IT2FNN, which integrates a first order TSKIT2FIS (interval type-2 fuzzy antecedents and interval type-1 fuzzy consequents), with an adaptive NN. The IT2FNN-2 (Figure 4) layers are described in a similar way to the previous architectures.


## 3. IT2FNN as a Universal Approximator

Based on the description of the interval type-2 fuzzy neural networks, it is possible to prove that under certain conditions, the resulting IT2FIS has unlimited approximation power to match any nonlinear functions on a compact set [36, 37] using the Stone-Weierstrass theorem [5, 6, 10, 30].


### 3.1. Stone-Weierstrass Theorem

Theorem 1 (Stone-Weierstrass theorem).
LetZ be a set of real continuous functions on a compact set U. If (1)  Z is an algebra, that is, the set Z is closed under addition, multiplication, and scalar multiplication, (2)  Z separates points on U, that is, for every x,y∈U, x≠y, there exists f∈Z such that f(x)≠f(y), and (3)  Z vanishes at no point of U, that is, for each x∈U there exists f∈Z such that f(x)≠0, then the uniform closure of Z consists of all real continuous functions on U; that is, (Z,d∞) is dense in (C[U],d∞) [36–38].Theorem 2 (universal approximation theorem).
For any given real continuous functiong(u) on the compact set U⊂Rn and arbitrary ε>0, there exists f∈Y such that supx∈U(|g(x)-f(x)|)<ε.


### 3.2. Applying Stone-Weierstrass Theorem to the IT2FNN-0 Architecture

In the IT2FNN-0, the domain on which we operate is almost always compact. It is a standard result in real analysis that every closed and bounded set inℜn is compact. Now we shall apply the Stone-Weierstrass theorem to show the representational power of IT2FNN with simplified fuzzy if-then rules. We now consider a subset of the IT2FNN-0 on Figure 5. The set of IT2FNN-0 with singleton fuzzifier, product inference, center of sets type reduction, and Gaussian interval type-2 membership function consists of all FBF expansion functions of the form (38), (40). f:U⊂Rn→R, x=(x1,x2,…,xn)∈U; μF~ik(x)∈[μ_F~ik(x),μ¯F~ik(x)] is the Gaussian interval type-2 membership function, igaussmtype2 (x,[σik,1mik,2mik]), defined by (27) and (31). If we view ϕ_k(x),ϕ¯k(x) as fuzzy basis functions (32) and (33) and yk(x) are linear functions (34), then y^(x) of (38) and (40) can be viewed as a linear combination of the fuzzy basis functions, and then the IT2FNN-0 system is equivalent to an FBF expansion. Let Y be the set of all the FBF expansions (38) and (40) with ϕ_k(x),ϕ¯k(x) given by (13) and (38) and let d∞(f1,f2)=supx∈U(|f1(x)-f2(x)|) be the supmetric; then, (Y,d∞) is a metric space [38]. We use the following Stone-Weierstrass theorem to prove our result.An example of the IT2FNN-0 architecture.
(a)
Antecedent IT2MFs for fuzzy rules
(b)
Overall I/O curve for crisp rules
(c)
An example of the interval type-2 fuzzy basis functions
(d)
Overall I/O curve for IT2FNN-0Suppose we have two IT2FNN-0sf1,f2∈Y; the output of each system can be expressed as
(22)f1(x)=αy^l1(x)+(1-α)y^r1(x),
where
(23)y^l1(x)=∑k=1Mϕ_1k(x)z1k(x)=∑k=1M1w_1k(x)z1k(x)Dl1,y^r1(x)=∑k=1M1ϕ¯1k(x)z1k(x)=∑k=1M1w¯1k(x)z1k(x)Dr1,
where
(24)Dl1=∑k=1M1∏i=1nμ_1F~ik(x),Dr1=∑k1=1M1∏i=1nμ-1F~ik(x),w_1k=∏i=1nμ_1F~ik(x),w¯1k=∏i=1nμ-1F~ik(x),ϕ_1k(x)=w_1k(x)Dl1,ϕ¯1k(x)=w¯1kDr1,z1k(x)=∑i=1n1cikxi+1c0k,k=1,…,M,(25)f2(x)=γy^l2(x)+(1-γ)y^r2(x),
where
(26)y^l2(x)=∑k=1Mϕ_2k(x)z2k(x)=∑k=1M2w_2k(x)z2k(x)Dl2,(27)y^r2(x)=∑k=1M2ϕ¯2k(x)z2k(x)=∑k=1M1w¯2k(x)z2k(x)Dr2,
where
(28)w_2k=∏i=1nμ_2F~ik(x),w¯2k=∏i=1nμ-2F~ik(x),Dl2=∑k=1M2∏i=1nμ_2F~ik(x),Dr2=∑k=1M2∏i=1nμ-2F~ik(x),ϕ_2k(x)=w_2k(x)Dl2,ϕ¯2k(x)=w¯2k(x)Dr2,z2k(x)=∑i=1n2cikxi+2c0k,k=1,…,M.Lemma 3.
Y is closed under addition.Proof.
The proof of this lemma requires our IT2FNN-0 to be able to approximate sums of functions. Suppose we have two IT2FNN-0s,f1(x) and f2(x) with M1 and M2 rules, respectively. The output of each system can be expressed as
(29)f1(x)+f2(x)=∑k1=1M1∑k2=1M2[w_1k1w_2k2(αz1k1+γz2k2)]Dl1Dl2+∑k1=1M1∑k2=1M2[w¯1k1w¯2k2{(1-α)z1k1+(1-γ)z2k2}]Dr1Dr2
and that Φ_k1,k2=(w_1k1w_2k2)/(Dr1Dr2) and Φ¯k1,k2=(w¯1k1w¯2k2)/(Dr1Dr2), where the FBFs are known to be nonlinear. Therefore, an equivalent to IT2FNN-0 can be constructed under the addition of f1(x) and f2(x), where the consequents form an addition of αz1k1+γz2k2 and (1-α)z1k1+(1-γ)z2k2 multiplied by a respective FBFs expansion (Theorem 1), and there exists f∈Y such that supx∈U(|g(x)-f(x)|)<ε (Theorem 2). Since f(x) satisfies Lemma 3 and Y∈f(x)=f1(x)+f2(x) then we can conclude that Y is closed under addition. Note that z1k1 and z2k2 can be linear since the FBFs are a nonlinear basis interval and therefore the resultant function, f(x), is nonlinear interval (see Figure 5).Lemma 4.
Y is closed under multiplication.Proof.
Similar to Lemma3, we model the product of f1(x)f2(x) of two IT2FNN-0s. The product f1(x)f2(x) can be expressed as
(30)f1(x)f2(x)=1Dl1Dl2[∑k1=1M1∑k2=1M2w_1k1w_2k2αγz1k1z2k2]+1Dl1Dr2[∑k1=1M1∑k2=1M2w_1k1w¯2k2α(1-γ)z1k1z2k2]+1Dr1Dl2[∑k1=1M1∑k2=1M2w¯1k1w_2k2(1-α)γz1k1z2k2]+1Dr1Dr2[∑k1=1M1∑k2=1M2w¯1k1w¯2k2(1-α)(1-γ)z1k1z2k2].
Therefore, an equivalent to IT2FNN-0 can be constructed under the multiplication of f1(x) and f2(x), where the consequents form an addition of αγz1k1z2k2, α(1-γ)z1k1z2k2, (1-α)γz1k1z2k2, and (1-α)(1-γ)z1k1z2k2 multiplied by a respective FBFs (Φk1,k2l,l=(w_1k1w_2k2)/(Dl1Dl2), Φk1,k2l,r=(w_1k1w¯2k2)/(Dl1Dr2), Φk1,k2r,l=(w¯1k1w_2k2)/(Dr1Dl2), and Φk1,k2r,r=(w¯1k1w¯2k2)/(Dr1Dr2)) expansion (Theorem 1), and there exists f∈Y such that supx∈U(|g(x)-f(x)|)<ε (Theorem 2). Since f(x) satisfies Lemma 3 and Y∈f(x)=f1(x)f2(x) then we can conclude that Y is closed under multiplication. Note that z1k1 and z2k2 can be linear since the FBFs are a nonlinear basis interval and therefore the resultant function, f(x), is nonlinear interval. Also, even if z1k1 and z2k2 were linear, their product z1k1z2k2 is evidently polynomial (see Figure 5).Lemma 5.
Y is closed under scalar multiplication.Proof.
Let an arbitrary IT2FNN-0 bef(x) (20); the scalar multiplication of cf(x) can be expressed as
(31)cf(x)=αcy^l(x)+(1-α)cy^r(x)cf(x)=∑k=1M[αDrw_k(x)+(1-α)Dlw¯k(x)]czk(x)DlDr.
Therefore we can construct an IT2FNN-0 that computes czk(x) in the form of the proposed IT2FNN-0; Y is closed under scalar multiplication.Lemma 6.
For everyx0,y0∈U and x0≠y0, there exists f∈Y such that f(x0)≠f(y0); that is, Y separates points on U.Proof.
We prove that(Y,d∞) separates points on U. We prove this by constructing a required f(x) (20); that is, we specify f∈Y such that f(x0)≠f(y0) for arbitrarily given (x0,y0)∈U with x0≠y0. We choose two fuzzy rules in the form of (8) for the fuzzy rule base (i.e., M=2). Let x0=(x10,x20,…,xn0) and y0=(y10,y20,…,yn0). If xi0=(xli0+xri0)/2 and yi0=(yli0+yri0)/2 with xi0≠yi0, we define two interval type-2 fuzzy sets (F~i1,[μ_F~i1,μ¯F~i1]) and (F~i2,[μ_F~i2,μ¯F~i2]) with
(32)μ¯F~i1(xi)={exp[-12(xi-xli0)2],xi<xli0,1,xli0≤xi≤xri0,exp[-12(xi-xri0)2],xi>xli0,(33)μ_F~i1(xi)={exp[-12(xi-xri0)2],xi≤xli0,exp[-12(xi-xli0)2],xi>xri0,(34)μ¯F~i2(xi)={exp[-12(xi-yli0)2],xi<yli0,1,yli0≤xi≤yri0,exp[-12(xi-yri0)2],xi>yli0,(35)μ_F~ik(xi)={exp[-12(xi-yri0)2],xi≤yli0,exp[-12(xi-yli0)2],xi>yri0.
If xi0=yi0, then F~i1=F~i2 and μ_F~i1(xi0)=μ_F~i2(yi0), μ¯F~i1(xi0)=μ¯F~i2(yi0); that is, only one interval type-2 fuzzy set is defined. We define two real value sets z1 and z2 with zk(x)=∑i=1ncikxi+c0k, where k=1,2. Now we have specified all the design parameters except zk; that is, we have already obtained a function f which is in the form of (10) with M=2 and (F~i1,[μ_F~i1,μ¯F~i1]) given by (18), (20), and (21). With this f, we have
(36)f(x0)=α[ϕ_1(x0)z1(x0)+ϕ_2(x0)z2(x0)]+(1-α)×[ϕ¯1(x0)z1(x0)+(1-ϕ¯1(x0))z2(x0)],
where
(37)ϕ_1(x0)=∏i=1nμ_Fi1(xi0)∏i=1nμ_Fi1(xi0)+∏i=1nμ_Fi2(xi0),ϕ_2(x0)=∏i=1nμ_Fi2(xi0)∏i=1nμ_Fi1(xi0)+∏i=1nμ_Fi2(xi0),ϕ¯1(x0)=11+∏i=1nμ¯Fi2(xi0),(38)f(y0)=α[ϕ_1(y0)z1(y0)+ϕ_2(y0)z2(y0)]+(1-α)×[(1-ϕ¯2(y0))z1(y0)+ϕ¯2(y0)z2(y0)],
where
(39)ϕ_1(y0)=∏i=1nμ_Fi1(yi0)∏i=1nμ_Fi1(yi0)+∏i=1nμ_Fi2(yi0),ϕ_2(y0)=∏i=1nμ_Fi2(yi0)∏i=1nμ_Fi1(yi0)+∏i=1nμ_Fi2(yi0),ϕ¯2(y0)=1∏i=1nμ¯Fi1(yi0)+1.
Since x0≠y0, there must be some i such that xi0=yi0; hence, we have ∏i=1nμ_Fi1(xi0)≠1 and ∏i=1nμ¯Fi2(xi0)≠1. If we choose z1=0 and z2=1, then f(x0)=αϕ_2(x0)+(1-α)[1-ϕ¯1(x0)]≠αϕ_2(y0)+(1-α)ϕ¯2(y0)=f(y0). Separability is satisfied whenever an IT2FNN-0 can compute strictly monotonic functions of each input variable. This can easily be achieved by adjusting the membership functions of the premise part. Therefore, (Y,d∞) separates points on U.Lemma 7.
For eachx∈U, there exists f∈Y such that f(x)≠0; that is, Y vanishes at no point of  U.Finally, we prove that(Y,d∞) vanishes at no point of  U. By observing (8)–(11), (20), and (21), we simply choose all yk(x)>0 (k=1,2,…,M); that is, any f∈Y with yk(x)>0 serves as the required f.Proof of Theorem2.
From (20) and (21), it is evident that Y is a set of real continuous functions on U, which are established by using complete interval type-2 fuzzy sets in the IF parts of fuzzy rules. Using Lemmas 3, 4, and 5, Y is proved to be an algebra. By using the Stone-Weierstrass theorem together with Lemmas 6 and 7, we establish that the proposed IT2FNN-0 possesses the universal approximation capability.


### 3.3. Applying the Stone-Weierstrass Theorem to the IT2FNN-2 Architecture

We now consider a subset of the IT2FNN-2 on Figure2. The set of IT2FNN-2 with singleton fuzzifier, product inference, type-reduction defuzzifier (KM) [13, 14], and Gaussian interval type-2 membership function consists of all FBF expansion functions. f:U⊂Rn→R, x=(x1,x2,…,xn)∈U; μF~ik(x)∈[μ_F~ik(x),μ¯F~ik(x)] is the Gaussian interval type-2 membership function, igaussmtype2 (x,[σik,1mik,1mik]), defined by (8)–(11). If we view ϕ_lk(x), ϕ¯lk(x), ϕ_rk(x), ϕ¯rk(x) as basis functions (44), (46), (49), (50) and ylk,yrk are linear functions (41), then y^(x) can be viewed as a linear combination of the basis functions. Let Y be the set of all the FBF expansions with ϕ_lk(x), ϕ¯lk(x), ϕ_rk(x), ϕ¯rk(x) and let d∞(f1,f2)=supx∈U(|f1(x)-f2(x)|) be the supmetric; then, (Y,d∞) is a metric space [38]. The following theorem shows that (Y,d∞) is dense in (C[U],d∞), where C[U] is the set of all real continuous functions defined on U. We use the following Stone-Weierstrass theorem to prove the theorem.Suppose we have two IT2FNN-2sf1,f2∈Y; the output of each system can be expressed as
(40)f1(x)=γy^l1(x)+(1-γ)y^r1(x),
where
(41)y^l1(x)=∑k1=1L11ϕ¯lk1(x)1zlk1(x)+∑k1=L1+1M11ϕ_lk1(x)1zlk1(x)y^l1(x)=∑k1=1L1w¯1k1(x)1zlk1(x)+∑k=L1+1M1w_1k1(x)1zlk1(x)Dl1,y^r1(x)=∑k1=1R11ϕ_rk1(x)1zrk1(x)+∑k1=R1+1M11ϕ¯rk1(x)1zrk1(x)y^r1(x)=∑k1=1R1w_1k1(x)1zrk1(x)+∑k1=R1+1M1w¯1k1(x)1zrk1(x)Dr1,
where
(42)w_1k1=∏i=1nμ_1F~ik1(x),w¯1k1=∏i=1nμ-1F~ik1(x),Dl1=∑k1=1L1w¯1k1+∑k1=L1+1M1w_1k1,Dr1=∑k1=1R1w_1k1+∑k1=R1+1M1w¯1k1,1ϕ¯lk1(x)=w¯1k1Dl1∀k1=1,…,L1(x),1ϕ_lk1(x)=w_1k1Dl1∀k1=L1(x)+1,…,M1,1ϕ_rk1(x)=w_1k1Dr1∀k1=1,…,R1(x),1ϕ¯rk1(x)=w¯1k1Dr1∀k1=R1(x)+1,…,M1,1zlk1=∑i=1n1cik1xi+1c0k-∑i=1n1sik1|xi|-1s0k1,k1=1,…,M1,1zrk1=∑i=1n1cik1xi+1c0k1+∑i=1n1sik1|xi|+1s0k1,k1=1,…,M1,f2(x)=γy^l2(x)+(1-γ)y^r2(x),
where
(43)y^l2(x)=∑k2=1L22ϕ¯lk2(x)2zlk2(x)+∑k2=L2+1M22ϕ_lk2(x)2zlk2(x)=∑k2=1L2w¯2k2(x)2zlk2(x)+∑k2=L2+1M2w_2k2(x)2zlk2(x)Dl2,(44)y^r2(x)=∑k2=1R22ϕ_rk2(x)2zrk2(x)+∑k2=R2+1M22ϕ¯rk2(x)2zrk2(x)=∑k2=1R21w_2k2(x)2zrk2(x)+∑k2=R2+1M2w¯2k2(x)2zrk2(x)Dr1,
where
(45)w_2k2=∏i=1nμ_2F~ik2(x),w¯2k2=∏i=1nμ_2F~ik2(x),Dl2=∑k2=1L2w¯2k2+∑k2=L2+1M2w_2k2,Dr2=∑k2=1R2w_2k2+∑k2=R2+1M2w¯2k2,2ϕ¯lk2(x)=w¯2k2Dl2∀k2=1,…,L2(x),2ϕ_lk2(x)=w_2k2Dl2∀k2=L2(x)+1,…,M2,2ϕ_rk2(x)=w_2k2Dr2∀k2=1,…,R2(x),2ϕ¯rk2(x)=w¯2k2Dr2∀k2=R2(x)+1,…,M2,2zlk2=∑i=1n2cik2xi+2c0k2-∑i=1n2sik2|xi|-2s0k2,k2=1,…,M2,2zrk2=∑i=1n2cik2xi+2c0k2+∑i=1n2sik2|xi|+2s0k2,k2=1,…,M2.Lemma 8.
Y is closed under addition.Proof.
The proof of this lemma requires our IT2FNN-2 to be able to approximate sums of functions. Suppose we have two IT2FNN-2sf1(x) and f2(x) with rules M1 and M2, respectively. The output of each system can be expressed as
(46)f1(x)+f2(x)=((∑k1=1L1∑k2=1L2[αDl2w¯1k11zlk1(x)+γDl1w¯2k22zlk2(x)])×(Dl1Dl2)-1(∑k1=1L1∑k2=1L2[αDl2w¯1k11zlk1(x)+γDl1w¯2k22zlk2(x)]))+((∑k1=L1+1M1∑k2=L2+1M2[αDl2w_1k11zlk1(x)+γDl1w_2k22zlk2(x)])×(Dl1Dl2)-1(∑k1=1L1∑k2=1L2[αDl2w¯1k11zlk1(x)+γDl1w¯2k22zlk2(x)]))+((∑k1=1R1∑k2=1R2[(1-α)Dr2w_1k11zrk1(x)+(1-γ)Dr1w_2k22zrk2(x)]∑k1=1R1∑k2=1R2[(1-α)Dr2w_1k11zrk1(x))×(Dr1Dr2)-1(∑k1=1R1∑k2=1R2[(1-α)Dr2w_1k11zrk1(x)+(1-γ)Dr1w_2k22zrk2(x)]))+((∑k1=R1+1M1∑k2=R2+1M2[(1-α)Dr2w¯1k11zrk1(x)+(1-γ)Dr1w¯2k22zrk2(x)]∑k1=1R1∑k2=1R2[(1-α)Dr2w_1k11zrk1(x))×(Dr1Dr2)-1∑k1=R1+1M1∑k2=R2+1M2[(1-α)Dr2w¯1k11zrk1(x)).
Therefore, an equivalent to IT2FNN-2 can be constructed under the addition of f1(x) and f2(x), where the consequents form an addition of α1zlk1+γ2zlk2 and (1-α)1zrk1+(1-γ)2zrk2 multiplied by a respective FBFs expansion (Theorem 1), and there exists f∈Y such that supx∈U(|g(x)-f(x)|)<ε (Theorem 2). Since f(x) satisfies Lemma 3 and Y∈f(x)=f1(x)+f2(x) then we can conclude that Y is closed under addition. Note that z1k1 and z2k2 can be linear interval since the FBFs are a nonlinear basis and therefore the resultant function, f(x), is nonlinear interval (see Figure 6).An example of the IT2FNN-2 architecture.
(a)
Antecedent IT2MFs for fuzzy rules
(b)
Overall I/O curve for interval rules
(c)
An example of the interval type-2 fuzzy basis functions
(d)
Overall I/O curve for IT2FNN-2Lemma 9.
Y is closed under multiplication.Proof.
In a similar way to Lemma8, we model the product of f1(x)f2(x) of two IT2FNN-2s which is the last point we need to demonstrate before we can conclude that the Stone-Weierstrass theorem can be applied to the proposed reasoning mechanism. The product f1(x)f2(x) can be expressed as
(47)f1(x)f2(x)=αγDl1Dl2×[∑k1=1L1∑k2=1L2w¯1k1(x)w¯2k2(x)1zlk1(x)2zlk2(x)+∑k1=1L1∑k2=L2+1M2w¯1k1(x)w_2k2(x)1zlk1(x)2zlk2(x)+∑k1=L1+1M1∑k2=1L2w_1k1(x)w¯2k2(x)1zlk1(x)2zlk2(x)+∑k1=L1+1M1∑k2=L2+1M2w_1k1(x)w_2k2(x)1zlk1(x)2zlk2(x)]+α(1-γ)Dl1Dr2×[∑k1=1L1∑k2=1R2w¯1k1(x)w_2k2(x)1zlk1(x)2zrk2(x)+∑k1=1L1∑k2=R2+1M2w¯1k1(x)w¯2k2(x)1zlk1(x)2zrk2(x)+∑k1=L1+1M1∑k2=1R2w_1k1(x)w_2k2(x)1zlk1(x)2zrk2(x)+∑k1=L1+1M1∑k2=R2+1M2w_1k1(x)w¯2k2(x)1zlk1(x)2zrk2(x)]+(1-α)γDr1Dl2×[∑k1=1R1∑k2=1L2w_1k1(x)w¯2k2(x)1zrk1(x)2zlk2(x)+∑k1=1R1∑k2=L2+1M2w_1k1(x)w_2k2(x)1zrk1(x)2zlk2(x)+∑k1=R1+1M1∑k2=1L2w¯1k1(x)w¯2k2(x)1zrk1(x)2zlk2(x)+∑k1=R1+1M1∑k2=R2+1M2w¯1k1(x)w_2k2(x)1zrk1(x)2zlk2(x)]+(1-α)(1-γ)Dr1Dr2×[∑k1=1R1∑k2=1R2w_1k1(x)w_2k2(x)1zrk1(x)2zrk2(x)+∑k1=1R1∑k2=R2+1M2w_1k1(x)w¯2k2(x)1zrk1(x)2zrk2(x)+∑k1=R1+1M1∑k2=1R2w¯1k1(x)w_2k2(x)1zrk1(x)2zrk2(x)+∑k1=R1+1M1∑k2=R2+1M2w¯1k1(x)w¯2k2(x)1zrk1(x)2zrk2(x)].
Therefore, an equivalent to IT2FNN-2 can be constructed under the multiplication of f1(x) and f2(x), where the consequents form an addition of αγ1zlk12zlk2, α(1-γ)1zlk12zrk2, (1-α)γ1zrk12zlk2, and (1-α)(1-γ)1zrk12zrk2 multiplied by a respective FBFs expansion (Theorem 1), and there exists f∈Y such that supx∈U(|g(x)-f(x)|)<ε (Theorem 2). Since f(x) satisfies Lemma 3 and Y∈f(x)=f1(x)f2(x) then we can conclude that Y is closed under multiplication. Note that z1k1 and z2k2 can be linear intervals since the FBFs are a nonlinear basis interval and therefore the resultant function, f(x), is nonlinear interval. Also, even if z1k1 and z2k2 were linear, their product z1k1z2k2 is evidently polynomial interval (see Figure 10).Lemma 10.
Y is closed under scalar multiplication.Proof.
Let an arbitrary IT2FNN-2 bef(x) (51); the scalar multiplication of cf(x) can be expressed as
(48)cf1(x)=αcy^l1(x)+(1-α)cy^r1(x)=∑k1=1L1w¯1k1(x)αc1zlk1(x)+∑k=L1+1M1w_1k1(x)αc1zlk1(x)Dl1+∑k1=1R1w¯1k1(x)(1-α)c1zrk1(x)Dr1+∑k1=R1+1M1w_1k1(x)(1-α)c1zrk1(x)Dr1.
Therefore we can construct an IT2FNN-2 that computes all FBF expansion combinations with αc1zlk1(x) and (1-α)c1zrk1(x) in the form of the proposed IT2FNN-2, and Y is closed under scalar multiplication.Lemma 11.
For every(x0,y0)∈U and x0≠y0, there exists f∈Y such that f(x0)≠f(y0); that is, Y separates points on U.We prove this by constructing a requiredf; that is, we specify f∈Y such that f(x0)≠f(y0) for arbitrarily given x0,y0∈U with x0≠y0. We choose two fuzzy rules in the form of (8) for the fuzzy rule base (i.e., M=2). Let x0=(x10,x20,…,xn0) and y0=(y10,y20,…,yn0). If xi0=(xli0+xri0)/2 and yi0=(yli0+yri0)/2 with xi0≠yi0, we define two interval type-2 fuzzy sets (F~i1,[μ_F~i1,μ¯F~i1]) and (F~i2,[μ_F~i2,μ¯F~i2]) with
(49)μ¯F~i1(xi)={exp[-12(xi-xli0)2],xi<xli0,1,xli0≤xi≤xri0,exp[-12(xi-xri0)2],xi>xli0,(50)μ_F~i1(xi)={exp[-12(xi-xri0)2],xi≤xli0,exp[-12(xi-xli0)2],xi>xri0,(51)μ¯F~i2(xi)={exp[-12(xi-yli0)2],xi<yli0,1,yli0≤xi≤yri0,exp[-12(xi-yri0)2],xi>yli0,(52)μ_F~ik(xi)={exp[-12(xi-yri0)2],xi≤yli0,exp[-12(xi-yli0)2],xi>yri0.
If xi0=yi0, then F~i1=F~i2 and μ_F~i1(xi0)=μ_F~i2(yi0), μ¯F~i1(xi0)=μ¯F~i2(yi0); that is, only one interval type-2 fuzzy set is defined. We define two interval value real sets z1∈[zl1,zr1] and z2∈[zl2,zr2]. Now we have specified all the design parameters except [zlk,zrk]; that is, we have already obtained a function f which is in the form of (20), (21) with M=2 and (F~ik,[μ_F~ik,μ¯F~ik]) given by (8)–(11). With this f, we have
(53)f(x0)=α[ϕl1(x0)zl1(x0)+(1-ϕl1(x0))zl2(x0)]+(1-α)[ϕr1(x0)zr1(x0)+ϕr2(x0)zr2(x0)],
where
(54)ϕl1(x0)=11+∏i=1nμ_Fi2(xi0),ϕr1(x0)=∏i=1nμ_Fi1(xi0)∏i=1nμ_Fi1(xi0)+∏i=1nμ¯Fi2(xi0),ϕr2(x0)=∏i=1nμ¯Fi2(xi0)∏i=1nμ_Fi1(xi0)+∏i=1nμ¯Fi2(xi0),f(y0)=α[ϕl1(y0)zl1(y0)+ϕl2(y0)zl2(y0)]+(1-α)×[(1-ϕr2(y0))zr1(y0)+ϕr2(y0)zr2(y0)],
where
(55)ϕr2(y0)=1∏i=1nμ_Fi1(yi0)+1,ϕl1(y0)=∏i=1nμ¯Fi1(yi0)∏i=1nμ¯Fi1(yi0)+∏i=1nμ_Fi2(yi0),ϕl2(y0)=∏i=1nμ_Fi2(yi0)∏i=1nμ¯Fi1(yi0)+∏i=1nμ_Fi2(yi0).
Since x0≠y0, there must be some i such that xi0=yi0; hence, we have ∏i=1nμ_Fi1(xi0)≠1 and ∏i=1nμ¯Fi2(xi0)≠1. If we choose zlr1=0 and zlr2=1, then f(x0)=α(1-ϕl1(x0))+(1-α)ϕr2(x0)≠αϕl2(y0)+(1-α)ϕr2(y0)=f(y0). Therefore, (Y,d∞) separates point on U.Lemma 12.
For eachx∈U, there exists f∈Y such that f(x)≠0; that is, Y vanishes at no point of  U.Finally, we prove that(Y,d∞) vanishes at no point of  U. By observing (8)–(11), (20), and (21), we just choose all zk>0 (k=1,2,…,M); that is, any f∈Y with zk>0 serves as required f.Proof of Theorem2.
From (20) and (21), it is evident that Y is a set of real continuous functions on U, which are established by using complete interval type-2 fuzzy sets in the IF parts of fuzzy rules. Using Lemmas 8, 9, and 10, Y is proved to be an algebra. By using the Stone-Weierstrass theorem together with Lemmas 11 and 12, we establish that the proposed IT2FNN-2 possesses the universal approximation capability.
Therefore by choosing appropriate class of interval type-2 membership functions, we can conclude that the IT2FNN-0 and IT2FNN-2 with simplified fuzzy if-then rules satisfy the five criteria of the Stone-Weierstrass theorem.


## 3.1. Stone-Weierstrass Theorem

Theorem 1 (Stone-Weierstrass theorem).
LetZ be a set of real continuous functions on a compact set U. If (1)  Z is an algebra, that is, the set Z is closed under addition, multiplication, and scalar multiplication, (2)  Z separates points on U, that is, for every x,y∈U, x≠y, there exists f∈Z such that f(x)≠f(y), and (3)  Z vanishes at no point of U, that is, for each x∈U there exists f∈Z such that f(x)≠0, then the uniform closure of Z consists of all real continuous functions on U; that is, (Z,d∞) is dense in (C[U],d∞) [36–38].Theorem 2 (universal approximation theorem).
For any given real continuous functiong(u) on the compact set U⊂Rn and arbitrary ε>0, there exists f∈Y such that supx∈U(|g(x)-f(x)|)<ε.


## 3.2. Applying Stone-Weierstrass Theorem to the IT2FNN-0 Architecture

In the IT2FNN-0, the domain on which we operate is almost always compact. It is a standard result in real analysis that every closed and bounded set inℜn is compact. Now we shall apply the Stone-Weierstrass theorem to show the representational power of IT2FNN with simplified fuzzy if-then rules. We now consider a subset of the IT2FNN-0 on Figure 5. The set of IT2FNN-0 with singleton fuzzifier, product inference, center of sets type reduction, and Gaussian interval type-2 membership function consists of all FBF expansion functions of the form (38), (40). f:U⊂Rn→R, x=(x1,x2,…,xn)∈U; μF~ik(x)∈[μ_F~ik(x),μ¯F~ik(x)] is the Gaussian interval type-2 membership function, igaussmtype2 (x,[σik,1mik,2mik]), defined by (27) and (31). If we view ϕ_k(x),ϕ¯k(x) as fuzzy basis functions (32) and (33) and yk(x) are linear functions (34), then y^(x) of (38) and (40) can be viewed as a linear combination of the fuzzy basis functions, and then the IT2FNN-0 system is equivalent to an FBF expansion. Let Y be the set of all the FBF expansions (38) and (40) with ϕ_k(x),ϕ¯k(x) given by (13) and (38) and let d∞(f1,f2)=supx∈U(|f1(x)-f2(x)|) be the supmetric; then, (Y,d∞) is a metric space [38]. We use the following Stone-Weierstrass theorem to prove our result.An example of the IT2FNN-0 architecture.
(a)
Antecedent IT2MFs for fuzzy rules
(b)
Overall I/O curve for crisp rules
(c)
An example of the interval type-2 fuzzy basis functions
(d)
Overall I/O curve for IT2FNN-0Suppose we have two IT2FNN-0sf1,f2∈Y; the output of each system can be expressed as
(22)f1(x)=αy^l1(x)+(1-α)y^r1(x),
where
(23)y^l1(x)=∑k=1Mϕ_1k(x)z1k(x)=∑k=1M1w_1k(x)z1k(x)Dl1,y^r1(x)=∑k=1M1ϕ¯1k(x)z1k(x)=∑k=1M1w¯1k(x)z1k(x)Dr1,
where
(24)Dl1=∑k=1M1∏i=1nμ_1F~ik(x),Dr1=∑k1=1M1∏i=1nμ-1F~ik(x),w_1k=∏i=1nμ_1F~ik(x),w¯1k=∏i=1nμ-1F~ik(x),ϕ_1k(x)=w_1k(x)Dl1,ϕ¯1k(x)=w¯1kDr1,z1k(x)=∑i=1n1cikxi+1c0k,k=1,…,M,(25)f2(x)=γy^l2(x)+(1-γ)y^r2(x),
where
(26)y^l2(x)=∑k=1Mϕ_2k(x)z2k(x)=∑k=1M2w_2k(x)z2k(x)Dl2,(27)y^r2(x)=∑k=1M2ϕ¯2k(x)z2k(x)=∑k=1M1w¯2k(x)z2k(x)Dr2,
where
(28)w_2k=∏i=1nμ_2F~ik(x),w¯2k=∏i=1nμ-2F~ik(x),Dl2=∑k=1M2∏i=1nμ_2F~ik(x),Dr2=∑k=1M2∏i=1nμ-2F~ik(x),ϕ_2k(x)=w_2k(x)Dl2,ϕ¯2k(x)=w¯2k(x)Dr2,z2k(x)=∑i=1n2cikxi+2c0k,k=1,…,M.Lemma 3.
Y is closed under addition.Proof.
The proof of this lemma requires our IT2FNN-0 to be able to approximate sums of functions. Suppose we have two IT2FNN-0s,f1(x) and f2(x) with M1 and M2 rules, respectively. The output of each system can be expressed as
(29)f1(x)+f2(x)=∑k1=1M1∑k2=1M2[w_1k1w_2k2(αz1k1+γz2k2)]Dl1Dl2+∑k1=1M1∑k2=1M2[w¯1k1w¯2k2{(1-α)z1k1+(1-γ)z2k2}]Dr1Dr2
and that Φ_k1,k2=(w_1k1w_2k2)/(Dr1Dr2) and Φ¯k1,k2=(w¯1k1w¯2k2)/(Dr1Dr2), where the FBFs are known to be nonlinear. Therefore, an equivalent to IT2FNN-0 can be constructed under the addition of f1(x) and f2(x), where the consequents form an addition of αz1k1+γz2k2 and (1-α)z1k1+(1-γ)z2k2 multiplied by a respective FBFs expansion (Theorem 1), and there exists f∈Y such that supx∈U(|g(x)-f(x)|)<ε (Theorem 2). Since f(x) satisfies Lemma 3 and Y∈f(x)=f1(x)+f2(x) then we can conclude that Y is closed under addition. Note that z1k1 and z2k2 can be linear since the FBFs are a nonlinear basis interval and therefore the resultant function, f(x), is nonlinear interval (see Figure 5).Lemma 4.
Y is closed under multiplication.Proof.
Similar to Lemma3, we model the product of f1(x)f2(x) of two IT2FNN-0s. The product f1(x)f2(x) can be expressed as
(30)f1(x)f2(x)=1Dl1Dl2[∑k1=1M1∑k2=1M2w_1k1w_2k2αγz1k1z2k2]+1Dl1Dr2[∑k1=1M1∑k2=1M2w_1k1w¯2k2α(1-γ)z1k1z2k2]+1Dr1Dl2[∑k1=1M1∑k2=1M2w¯1k1w_2k2(1-α)γz1k1z2k2]+1Dr1Dr2[∑k1=1M1∑k2=1M2w¯1k1w¯2k2(1-α)(1-γ)z1k1z2k2].
Therefore, an equivalent to IT2FNN-0 can be constructed under the multiplication of f1(x) and f2(x), where the consequents form an addition of αγz1k1z2k2, α(1-γ)z1k1z2k2, (1-α)γz1k1z2k2, and (1-α)(1-γ)z1k1z2k2 multiplied by a respective FBFs (Φk1,k2l,l=(w_1k1w_2k2)/(Dl1Dl2), Φk1,k2l,r=(w_1k1w¯2k2)/(Dl1Dr2), Φk1,k2r,l=(w¯1k1w_2k2)/(Dr1Dl2), and Φk1,k2r,r=(w¯1k1w¯2k2)/(Dr1Dr2)) expansion (Theorem 1), and there exists f∈Y such that supx∈U(|g(x)-f(x)|)<ε (Theorem 2). Since f(x) satisfies Lemma 3 and Y∈f(x)=f1(x)f2(x) then we can conclude that Y is closed under multiplication. Note that z1k1 and z2k2 can be linear since the FBFs are a nonlinear basis interval and therefore the resultant function, f(x), is nonlinear interval. Also, even if z1k1 and z2k2 were linear, their product z1k1z2k2 is evidently polynomial (see Figure 5).Lemma 5.
Y is closed under scalar multiplication.Proof.
Let an arbitrary IT2FNN-0 bef(x) (20); the scalar multiplication of cf(x) can be expressed as
(31)cf(x)=αcy^l(x)+(1-α)cy^r(x)cf(x)=∑k=1M[αDrw_k(x)+(1-α)Dlw¯k(x)]czk(x)DlDr.
Therefore we can construct an IT2FNN-0 that computes czk(x) in the form of the proposed IT2FNN-0; Y is closed under scalar multiplication.Lemma 6.
For everyx0,y0∈U and x0≠y0, there exists f∈Y such that f(x0)≠f(y0); that is, Y separates points on U.Proof.
We prove that(Y,d∞) separates points on U. We prove this by constructing a required f(x) (20); that is, we specify f∈Y such that f(x0)≠f(y0) for arbitrarily given (x0,y0)∈U with x0≠y0. We choose two fuzzy rules in the form of (8) for the fuzzy rule base (i.e., M=2). Let x0=(x10,x20,…,xn0) and y0=(y10,y20,…,yn0). If xi0=(xli0+xri0)/2 and yi0=(yli0+yri0)/2 with xi0≠yi0, we define two interval type-2 fuzzy sets (F~i1,[μ_F~i1,μ¯F~i1]) and (F~i2,[μ_F~i2,μ¯F~i2]) with
(32)μ¯F~i1(xi)={exp[-12(xi-xli0)2],xi<xli0,1,xli0≤xi≤xri0,exp[-12(xi-xri0)2],xi>xli0,(33)μ_F~i1(xi)={exp[-12(xi-xri0)2],xi≤xli0,exp[-12(xi-xli0)2],xi>xri0,(34)μ¯F~i2(xi)={exp[-12(xi-yli0)2],xi<yli0,1,yli0≤xi≤yri0,exp[-12(xi-yri0)2],xi>yli0,(35)μ_F~ik(xi)={exp[-12(xi-yri0)2],xi≤yli0,exp[-12(xi-yli0)2],xi>yri0.
If xi0=yi0, then F~i1=F~i2 and μ_F~i1(xi0)=μ_F~i2(yi0), μ¯F~i1(xi0)=μ¯F~i2(yi0); that is, only one interval type-2 fuzzy set is defined. We define two real value sets z1 and z2 with zk(x)=∑i=1ncikxi+c0k, where k=1,2. Now we have specified all the design parameters except zk; that is, we have already obtained a function f which is in the form of (10) with M=2 and (F~i1,[μ_F~i1,μ¯F~i1]) given by (18), (20), and (21). With this f, we have
(36)f(x0)=α[ϕ_1(x0)z1(x0)+ϕ_2(x0)z2(x0)]+(1-α)×[ϕ¯1(x0)z1(x0)+(1-ϕ¯1(x0))z2(x0)],
where
(37)ϕ_1(x0)=∏i=1nμ_Fi1(xi0)∏i=1nμ_Fi1(xi0)+∏i=1nμ_Fi2(xi0),ϕ_2(x0)=∏i=1nμ_Fi2(xi0)∏i=1nμ_Fi1(xi0)+∏i=1nμ_Fi2(xi0),ϕ¯1(x0)=11+∏i=1nμ¯Fi2(xi0),(38)f(y0)=α[ϕ_1(y0)z1(y0)+ϕ_2(y0)z2(y0)]+(1-α)×[(1-ϕ¯2(y0))z1(y0)+ϕ¯2(y0)z2(y0)],
where
(39)ϕ_1(y0)=∏i=1nμ_Fi1(yi0)∏i=1nμ_Fi1(yi0)+∏i=1nμ_Fi2(yi0),ϕ_2(y0)=∏i=1nμ_Fi2(yi0)∏i=1nμ_Fi1(yi0)+∏i=1nμ_Fi2(yi0),ϕ¯2(y0)=1∏i=1nμ¯Fi1(yi0)+1.
Since x0≠y0, there must be some i such that xi0=yi0; hence, we have ∏i=1nμ_Fi1(xi0)≠1 and ∏i=1nμ¯Fi2(xi0)≠1. If we choose z1=0 and z2=1, then f(x0)=αϕ_2(x0)+(1-α)[1-ϕ¯1(x0)]≠αϕ_2(y0)+(1-α)ϕ¯2(y0)=f(y0). Separability is satisfied whenever an IT2FNN-0 can compute strictly monotonic functions of each input variable. This can easily be achieved by adjusting the membership functions of the premise part. Therefore, (Y,d∞) separates points on U.Lemma 7.
For eachx∈U, there exists f∈Y such that f(x)≠0; that is, Y vanishes at no point of  U.Finally, we prove that(Y,d∞) vanishes at no point of  U. By observing (8)–(11), (20), and (21), we simply choose all yk(x)>0 (k=1,2,…,M); that is, any f∈Y with yk(x)>0 serves as the required f.Proof of Theorem2.
From (20) and (21), it is evident that Y is a set of real continuous functions on U, which are established by using complete interval type-2 fuzzy sets in the IF parts of fuzzy rules. Using Lemmas 3, 4, and 5, Y is proved to be an algebra. By using the Stone-Weierstrass theorem together with Lemmas 6 and 7, we establish that the proposed IT2FNN-0 possesses the universal approximation capability.


## 3.3. Applying the Stone-Weierstrass Theorem to the IT2FNN-2 Architecture

We now consider a subset of the IT2FNN-2 on Figure2. The set of IT2FNN-2 with singleton fuzzifier, product inference, type-reduction defuzzifier (KM) [13, 14], and Gaussian interval type-2 membership function consists of all FBF expansion functions. f:U⊂Rn→R, x=(x1,x2,…,xn)∈U; μF~ik(x)∈[μ_F~ik(x),μ¯F~ik(x)] is the Gaussian interval type-2 membership function, igaussmtype2 (x,[σik,1mik,1mik]), defined by (8)–(11). If we view ϕ_lk(x), ϕ¯lk(x), ϕ_rk(x), ϕ¯rk(x) as basis functions (44), (46), (49), (50) and ylk,yrk are linear functions (41), then y^(x) can be viewed as a linear combination of the basis functions. Let Y be the set of all the FBF expansions with ϕ_lk(x), ϕ¯lk(x), ϕ_rk(x), ϕ¯rk(x) and let d∞(f1,f2)=supx∈U(|f1(x)-f2(x)|) be the supmetric; then, (Y,d∞) is a metric space [38]. The following theorem shows that (Y,d∞) is dense in (C[U],d∞), where C[U] is the set of all real continuous functions defined on U. We use the following Stone-Weierstrass theorem to prove the theorem.Suppose we have two IT2FNN-2sf1,f2∈Y; the output of each system can be expressed as
(40)f1(x)=γy^l1(x)+(1-γ)y^r1(x),
where
(41)y^l1(x)=∑k1=1L11ϕ¯lk1(x)1zlk1(x)+∑k1=L1+1M11ϕ_lk1(x)1zlk1(x)y^l1(x)=∑k1=1L1w¯1k1(x)1zlk1(x)+∑k=L1+1M1w_1k1(x)1zlk1(x)Dl1,y^r1(x)=∑k1=1R11ϕ_rk1(x)1zrk1(x)+∑k1=R1+1M11ϕ¯rk1(x)1zrk1(x)y^r1(x)=∑k1=1R1w_1k1(x)1zrk1(x)+∑k1=R1+1M1w¯1k1(x)1zrk1(x)Dr1,
where
(42)w_1k1=∏i=1nμ_1F~ik1(x),w¯1k1=∏i=1nμ-1F~ik1(x),Dl1=∑k1=1L1w¯1k1+∑k1=L1+1M1w_1k1,Dr1=∑k1=1R1w_1k1+∑k1=R1+1M1w¯1k1,1ϕ¯lk1(x)=w¯1k1Dl1∀k1=1,…,L1(x),1ϕ_lk1(x)=w_1k1Dl1∀k1=L1(x)+1,…,M1,1ϕ_rk1(x)=w_1k1Dr1∀k1=1,…,R1(x),1ϕ¯rk1(x)=w¯1k1Dr1∀k1=R1(x)+1,…,M1,1zlk1=∑i=1n1cik1xi+1c0k-∑i=1n1sik1|xi|-1s0k1,k1=1,…,M1,1zrk1=∑i=1n1cik1xi+1c0k1+∑i=1n1sik1|xi|+1s0k1,k1=1,…,M1,f2(x)=γy^l2(x)+(1-γ)y^r2(x),
where
(43)y^l2(x)=∑k2=1L22ϕ¯lk2(x)2zlk2(x)+∑k2=L2+1M22ϕ_lk2(x)2zlk2(x)=∑k2=1L2w¯2k2(x)2zlk2(x)+∑k2=L2+1M2w_2k2(x)2zlk2(x)Dl2,(44)y^r2(x)=∑k2=1R22ϕ_rk2(x)2zrk2(x)+∑k2=R2+1M22ϕ¯rk2(x)2zrk2(x)=∑k2=1R21w_2k2(x)2zrk2(x)+∑k2=R2+1M2w¯2k2(x)2zrk2(x)Dr1,
where
(45)w_2k2=∏i=1nμ_2F~ik2(x),w¯2k2=∏i=1nμ_2F~ik2(x),Dl2=∑k2=1L2w¯2k2+∑k2=L2+1M2w_2k2,Dr2=∑k2=1R2w_2k2+∑k2=R2+1M2w¯2k2,2ϕ¯lk2(x)=w¯2k2Dl2∀k2=1,…,L2(x),2ϕ_lk2(x)=w_2k2Dl2∀k2=L2(x)+1,…,M2,2ϕ_rk2(x)=w_2k2Dr2∀k2=1,…,R2(x),2ϕ¯rk2(x)=w¯2k2Dr2∀k2=R2(x)+1,…,M2,2zlk2=∑i=1n2cik2xi+2c0k2-∑i=1n2sik2|xi|-2s0k2,k2=1,…,M2,2zrk2=∑i=1n2cik2xi+2c0k2+∑i=1n2sik2|xi|+2s0k2,k2=1,…,M2.Lemma 8.
Y is closed under addition.Proof.
The proof of this lemma requires our IT2FNN-2 to be able to approximate sums of functions. Suppose we have two IT2FNN-2sf1(x) and f2(x) with rules M1 and M2, respectively. The output of each system can be expressed as
(46)f1(x)+f2(x)=((∑k1=1L1∑k2=1L2[αDl2w¯1k11zlk1(x)+γDl1w¯2k22zlk2(x)])×(Dl1Dl2)-1(∑k1=1L1∑k2=1L2[αDl2w¯1k11zlk1(x)+γDl1w¯2k22zlk2(x)]))+((∑k1=L1+1M1∑k2=L2+1M2[αDl2w_1k11zlk1(x)+γDl1w_2k22zlk2(x)])×(Dl1Dl2)-1(∑k1=1L1∑k2=1L2[αDl2w¯1k11zlk1(x)+γDl1w¯2k22zlk2(x)]))+((∑k1=1R1∑k2=1R2[(1-α)Dr2w_1k11zrk1(x)+(1-γ)Dr1w_2k22zrk2(x)]∑k1=1R1∑k2=1R2[(1-α)Dr2w_1k11zrk1(x))×(Dr1Dr2)-1(∑k1=1R1∑k2=1R2[(1-α)Dr2w_1k11zrk1(x)+(1-γ)Dr1w_2k22zrk2(x)]))+((∑k1=R1+1M1∑k2=R2+1M2[(1-α)Dr2w¯1k11zrk1(x)+(1-γ)Dr1w¯2k22zrk2(x)]∑k1=1R1∑k2=1R2[(1-α)Dr2w_1k11zrk1(x))×(Dr1Dr2)-1∑k1=R1+1M1∑k2=R2+1M2[(1-α)Dr2w¯1k11zrk1(x)).
Therefore, an equivalent to IT2FNN-2 can be constructed under the addition of f1(x) and f2(x), where the consequents form an addition of α1zlk1+γ2zlk2 and (1-α)1zrk1+(1-γ)2zrk2 multiplied by a respective FBFs expansion (Theorem 1), and there exists f∈Y such that supx∈U(|g(x)-f(x)|)<ε (Theorem 2). Since f(x) satisfies Lemma 3 and Y∈f(x)=f1(x)+f2(x) then we can conclude that Y is closed under addition. Note that z1k1 and z2k2 can be linear interval since the FBFs are a nonlinear basis and therefore the resultant function, f(x), is nonlinear interval (see Figure 6).An example of the IT2FNN-2 architecture.
(a)
Antecedent IT2MFs for fuzzy rules
(b)
Overall I/O curve for interval rules
(c)
An example of the interval type-2 fuzzy basis functions
(d)
Overall I/O curve for IT2FNN-2Lemma 9.
Y is closed under multiplication.Proof.
In a similar way to Lemma8, we model the product of f1(x)f2(x) of two IT2FNN-2s which is the last point we need to demonstrate before we can conclude that the Stone-Weierstrass theorem can be applied to the proposed reasoning mechanism. The product f1(x)f2(x) can be expressed as
(47)f1(x)f2(x)=αγDl1Dl2×[∑k1=1L1∑k2=1L2w¯1k1(x)w¯2k2(x)1zlk1(x)2zlk2(x)+∑k1=1L1∑k2=L2+1M2w¯1k1(x)w_2k2(x)1zlk1(x)2zlk2(x)+∑k1=L1+1M1∑k2=1L2w_1k1(x)w¯2k2(x)1zlk1(x)2zlk2(x)+∑k1=L1+1M1∑k2=L2+1M2w_1k1(x)w_2k2(x)1zlk1(x)2zlk2(x)]+α(1-γ)Dl1Dr2×[∑k1=1L1∑k2=1R2w¯1k1(x)w_2k2(x)1zlk1(x)2zrk2(x)+∑k1=1L1∑k2=R2+1M2w¯1k1(x)w¯2k2(x)1zlk1(x)2zrk2(x)+∑k1=L1+1M1∑k2=1R2w_1k1(x)w_2k2(x)1zlk1(x)2zrk2(x)+∑k1=L1+1M1∑k2=R2+1M2w_1k1(x)w¯2k2(x)1zlk1(x)2zrk2(x)]+(1-α)γDr1Dl2×[∑k1=1R1∑k2=1L2w_1k1(x)w¯2k2(x)1zrk1(x)2zlk2(x)+∑k1=1R1∑k2=L2+1M2w_1k1(x)w_2k2(x)1zrk1(x)2zlk2(x)+∑k1=R1+1M1∑k2=1L2w¯1k1(x)w¯2k2(x)1zrk1(x)2zlk2(x)+∑k1=R1+1M1∑k2=R2+1M2w¯1k1(x)w_2k2(x)1zrk1(x)2zlk2(x)]+(1-α)(1-γ)Dr1Dr2×[∑k1=1R1∑k2=1R2w_1k1(x)w_2k2(x)1zrk1(x)2zrk2(x)+∑k1=1R1∑k2=R2+1M2w_1k1(x)w¯2k2(x)1zrk1(x)2zrk2(x)+∑k1=R1+1M1∑k2=1R2w¯1k1(x)w_2k2(x)1zrk1(x)2zrk2(x)+∑k1=R1+1M1∑k2=R2+1M2w¯1k1(x)w¯2k2(x)1zrk1(x)2zrk2(x)].
Therefore, an equivalent to IT2FNN-2 can be constructed under the multiplication of f1(x) and f2(x), where the consequents form an addition of αγ1zlk12zlk2, α(1-γ)1zlk12zrk2, (1-α)γ1zrk12zlk2, and (1-α)(1-γ)1zrk12zrk2 multiplied by a respective FBFs expansion (Theorem 1), and there exists f∈Y such that supx∈U(|g(x)-f(x)|)<ε (Theorem 2). Since f(x) satisfies Lemma 3 and Y∈f(x)=f1(x)f2(x) then we can conclude that Y is closed under multiplication. Note that z1k1 and z2k2 can be linear intervals since the FBFs are a nonlinear basis interval and therefore the resultant function, f(x), is nonlinear interval. Also, even if z1k1 and z2k2 were linear, their product z1k1z2k2 is evidently polynomial interval (see Figure 10).Lemma 10.
Y is closed under scalar multiplication.Proof.
Let an arbitrary IT2FNN-2 bef(x) (51); the scalar multiplication of cf(x) can be expressed as
(48)cf1(x)=αcy^l1(x)+(1-α)cy^r1(x)=∑k1=1L1w¯1k1(x)αc1zlk1(x)+∑k=L1+1M1w_1k1(x)αc1zlk1(x)Dl1+∑k1=1R1w¯1k1(x)(1-α)c1zrk1(x)Dr1+∑k1=R1+1M1w_1k1(x)(1-α)c1zrk1(x)Dr1.
Therefore we can construct an IT2FNN-2 that computes all FBF expansion combinations with αc1zlk1(x) and (1-α)c1zrk1(x) in the form of the proposed IT2FNN-2, and Y is closed under scalar multiplication.Lemma 11.
For every(x0,y0)∈U and x0≠y0, there exists f∈Y such that f(x0)≠f(y0); that is, Y separates points on U.We prove this by constructing a requiredf; that is, we specify f∈Y such that f(x0)≠f(y0) for arbitrarily given x0,y0∈U with x0≠y0. We choose two fuzzy rules in the form of (8) for the fuzzy rule base (i.e., M=2). Let x0=(x10,x20,…,xn0) and y0=(y10,y20,…,yn0). If xi0=(xli0+xri0)/2 and yi0=(yli0+yri0)/2 with xi0≠yi0, we define two interval type-2 fuzzy sets (F~i1,[μ_F~i1,μ¯F~i1]) and (F~i2,[μ_F~i2,μ¯F~i2]) with
(49)μ¯F~i1(xi)={exp[-12(xi-xli0)2],xi<xli0,1,xli0≤xi≤xri0,exp[-12(xi-xri0)2],xi>xli0,(50)μ_F~i1(xi)={exp[-12(xi-xri0)2],xi≤xli0,exp[-12(xi-xli0)2],xi>xri0,(51)μ¯F~i2(xi)={exp[-12(xi-yli0)2],xi<yli0,1,yli0≤xi≤yri0,exp[-12(xi-yri0)2],xi>yli0,(52)μ_F~ik(xi)={exp[-12(xi-yri0)2],xi≤yli0,exp[-12(xi-yli0)2],xi>yri0.
If xi0=yi0, then F~i1=F~i2 and μ_F~i1(xi0)=μ_F~i2(yi0), μ¯F~i1(xi0)=μ¯F~i2(yi0); that is, only one interval type-2 fuzzy set is defined. We define two interval value real sets z1∈[zl1,zr1] and z2∈[zl2,zr2]. Now we have specified all the design parameters except [zlk,zrk]; that is, we have already obtained a function f which is in the form of (20), (21) with M=2 and (F~ik,[μ_F~ik,μ¯F~ik]) given by (8)–(11). With this f, we have
(53)f(x0)=α[ϕl1(x0)zl1(x0)+(1-ϕl1(x0))zl2(x0)]+(1-α)[ϕr1(x0)zr1(x0)+ϕr2(x0)zr2(x0)],
where
(54)ϕl1(x0)=11+∏i=1nμ_Fi2(xi0),ϕr1(x0)=∏i=1nμ_Fi1(xi0)∏i=1nμ_Fi1(xi0)+∏i=1nμ¯Fi2(xi0),ϕr2(x0)=∏i=1nμ¯Fi2(xi0)∏i=1nμ_Fi1(xi0)+∏i=1nμ¯Fi2(xi0),f(y0)=α[ϕl1(y0)zl1(y0)+ϕl2(y0)zl2(y0)]+(1-α)×[(1-ϕr2(y0))zr1(y0)+ϕr2(y0)zr2(y0)],
where
(55)ϕr2(y0)=1∏i=1nμ_Fi1(yi0)+1,ϕl1(y0)=∏i=1nμ¯Fi1(yi0)∏i=1nμ¯Fi1(yi0)+∏i=1nμ_Fi2(yi0),ϕl2(y0)=∏i=1nμ_Fi2(yi0)∏i=1nμ¯Fi1(yi0)+∏i=1nμ_Fi2(yi0).
Since x0≠y0, there must be some i such that xi0=yi0; hence, we have ∏i=1nμ_Fi1(xi0)≠1 and ∏i=1nμ¯Fi2(xi0)≠1. If we choose zlr1=0 and zlr2=1, then f(x0)=α(1-ϕl1(x0))+(1-α)ϕr2(x0)≠αϕl2(y0)+(1-α)ϕr2(y0)=f(y0). Therefore, (Y,d∞) separates point on U.Lemma 12.
For eachx∈U, there exists f∈Y such that f(x)≠0; that is, Y vanishes at no point of  U.Finally, we prove that(Y,d∞) vanishes at no point of  U. By observing (8)–(11), (20), and (21), we just choose all zk>0 (k=1,2,…,M); that is, any f∈Y with zk>0 serves as required f.Proof of Theorem2.
From (20) and (21), it is evident that Y is a set of real continuous functions on U, which are established by using complete interval type-2 fuzzy sets in the IF parts of fuzzy rules. Using Lemmas 8, 9, and 10, Y is proved to be an algebra. By using the Stone-Weierstrass theorem together with Lemmas 11 and 12, we establish that the proposed IT2FNN-2 possesses the universal approximation capability.
Therefore by choosing appropriate class of interval type-2 membership functions, we can conclude that the IT2FNN-0 and IT2FNN-2 with simplified fuzzy if-then rules satisfy the five criteria of the Stone-Weierstrass theorem.


## 4. Application Examples

In this section the results from simulations using ANFIS, IT2FNN-0, IT2FNN-1 [35], IT2FNN-2, and IT2FNN-3 [35] are presented for nonlinear system identification and forecasting the Mackey-Glass chaotic time series [39] with τ=60 with different signal noise ratio values, SNR(dB) = 0, 10, 20, 30, free as uncertainty source. These examples are used as benchmark problems to test the proposed ideas in the paper. We have to mention that the IT2FNN-1 and IT2FNN-3 architectures are very similar to I2FNN-0 and IT2FNN-2, respectively [35], and their results are presented for comparison purposes. The proposed IT2FNN architectures are validated using 10-fold cross-validation [40, 41] considering sum of square errors (SSE) or root mean square error (RMSE) in the training or test phase. We use cross-validation to measure the variability of the RMSE in the training and testing phases to compare network architectures IT2FNN. Cross-validation procedure evaluation is done using Matlab’s crossvalind function. Noise is added by Matlab’s awgn function.InK-fold cross-validation [40], the original sample is randomly partitioned into K subsamples. Of the K subsamples, a single subsample is retained as the validation data for testing the model, and the remaining K-1 subsamples are used as training data. The cross-validation process is then repeated K times (the folds), with each of the K subsamples used exactly once as the validation data. The K results from the folds then can be averaged (or otherwise combined) to produce a single estimation. The advantage of this method over repeated random subsampling is that all observations are used for both training and validation, and each observation is used for validation exactly once. 10-fold cross-validation is commonly used. Three application examples are used to illustrate proofs of universality, as follows.Experiment 1 (identification of a one variable nonlinear function).
In this experiment we approximate a nonlinear functionf:ℜ→ℜ:
(56)f(u)=0.6sin(πu)+0.3sin(3πu)+0.1sin(5πu)+η,
(where η is a uniform noise component) using a-one input one-output IT2FNN, 50 training data sets with 10-fold cross-validation with uniform noise levels, six IT2MFs type igaussmtype2, 6 rules, and 50 epochs. Once the ANFIS and IT2FNN models are identified a comparison was made, taking into account RMSE statistic values with 10-fold cross-validation. Table 1 and Figure 7 show the resulting RMSE (CHK) values for ANFIS and IT2FNN; it can be seen that IT2FNN architectures [31] perform better than ANFIS.Table 1
RMSE (CHK) values of ANFIS and IT2FNN with 10-fold cross-validation for identifying non-linearity of Experiment1.
SNR (dB)
ANFIS
IT2FNN-0
IT2FNN-1
IT2FNN-2
IT2FNN-3
0
0.6156
0.3532
0.2764
0.2197
0.1535
10
0.2375
0.1153
0.0986
0.0683
0.0453
20
0.0806
0.0435
0.0234
0.0127
0.0087
30
0.0225
0.0106
0.0079
0.0045
0.0028
Free
0.0015
0.0009
0.0004
0.0002
0.0001Figure 7
RMSE (CHK) values of ANFIS and IT2FNN using 10-fold cross-validation for identifying nonlinearity in Experiment1.Experiment 2 (identification of a three variable nonlinear function).
A three-input one-output IT2FNN is used to approximate nonlinear Sugeno [27] function f:ℜ3→ℜ:
(57)f(x1,x2,x3)=(1+x1+1x2+1x33)2+η.
216 training data sets are generated with 10-fold cross-validation and 125 for tests; 2 igaussmtype2 IT2MFs for each input, 8 rules, and 50 epochs. Once the ANFIS and IT2FNN models are identified, a comparison is made with RMSE statistic values and 10-fold cross-validation. Table 2 and Figure 8 show the resultant RMSE (CHK) values for ANFIS and IT2FNN. It can be seen that IT2FNN architectures [31] perform better than ANFIS.Table 2
Resulting RMSE (CHK) values in ANFIS and IT2FNN for non-linearity identification in Experiment2 with 10-fold cross-validation.
SNR (dB)
ANFIS
IT2FNN-0
IT2FNN-1
IT2FNN-2
IT2FNN-3
0
1.0432
0.7203
0.6523
0.5512
0.5267
10
0.3096
0.2798
0.2583
0.2464
0.2344
20
0.1703
0.1637
0.1592
0.1465
0.1387
30
0.1526
0.1408
0.1368
0.1323
0.1312
Free
0.1503
0.1390
0.1323
0.1304
0.1276Figure 8
Resulting RMSE (CHK) values obtained by ANFIS and IT2FNN for nonlinearity identification in Experiment2 with 10-fold cross-validation.Experiment 3.
Predicting the Mackey-Glass chaotic time series.
Mackey-Glass chaotic time series is a well-known benchmark [39] for systems modeling and is described as follows:
(58)x˙(t)=0.1x(t-τ)1+x10(t-τ)-0.1x(t).
1200 data sets are generated based on initial conditions x(0)=1.2 and τ=60, using fourth order Runge-Kutta method adding different levels of uniform noise. For comparing with other methods, an input-output vector is chosen for IT2FNN model with the following format:
(59)[x(t-18),x(t-12),x(t-6),x(t);x(t+6)].
Four-input and one-output IT2FNN model is used for Mackey-Glass chaotic time series prediction, choosing 500 data sets for training and 500 test data data sets with 10-fold cross-validation test, 2 IT2MFs for each input with membership function igaussmtype2, 16 rules, and 50 epochs. ANFIS and IT2FNN models are identified, comparing RMSE statistical values with 10-fold cross-validation. Table 3 and Figures 9 and 10 show the number of ς points out of uncertainty interval Y~(x)∈[y^l(x),y^r(x)] evaluated by IT2FNN model, RMSE training values (TRN) and test (CHK) obtained for ANFIS and IT2FNN models. It can be seen that IT2FNN model architectures predict better Mackey-Glass chaotic time series.Table 3
RMSE (TRN/CHK) andς values determined by ANFIS and IT2FNN models with 10 fold cross-validation for Mackey-Glass chaotic time series prediction with τ=60.
SNR (dB)τ=60
ANFIS
IT2FNN-0
IT2FNN-1
IT2FNN-2
IT2FNN-3
TRN
CHK
ς
TRN
CHK
ς
TRN
CHK
ς
TRN
CHK
ς
TRN
CHK
ς
0
0.3403
0.3714
NA
0.2388
0.2687
37
0.2027
0.2135
35
0.1495
0.1536
34
0.1244
0.1444
31
10
0.1544
0.1714
NA
0.1312
0.1456
33
0.1069
0.1119
30
0.0805
0.0972
28
0.0532
0.0629
25
20
0.0929
0.1007
NA
0.0708
0.0781
28
0.0566
0.0594
26
0.0424
0.0485
24
0.0342
0.0355
21
30
0.0788
0.0847
NA
0.0526
0.0582
19
0.0411
0.0468
18
0.0309
0.0332
16
0.0227
0.0316
14
Free
0.0749
0.0799
NA
0.0414
0.0477
10
0.0321
0.0353
9
0.0251
0.0319
6
0.0215
0.0293
4Figure 9
RMSE (TRN) values determined by ANFIS and IT2FNN models with 10-fold cross-validation for Mackey-Glass chaotic time series prediction withτ=60.Figure 10
RMSE (CHK) values determined by ANFIS and IT2FNN models with 10-fold cross-validation for Mackey-Glass chaotic time series prediction withτ=60.


## 5. Conclusions

In this paper we have shown that an interval type-2 fuzzy neural network (IT2FNN) is a universal approximator. Simulation results of nonlinear function identification using the IT2FNN for one and three variables and for the Mackey-Glass chaotic time series prediction have been presented to illustrate the theoretical result. In these experiments, the estimated RMSE values for nonlinear function identification with 10-fold cross-validation for the hybrid architectures (IT2FNN-2:A2C0 and IT2FNN-2:A2C1) illustrate the proof based on Stone-Weierstrass theorem, that they are universal approximators for efficient identification of nonlinear functions, complying with|g(x)-f(x)|<ε. Also, it can be seen that while increasing the Signal Noise Ratio (SNR), IT2FNN architectures handle uncertainty more efficiently. We have also illustrated the ideas presented in the paper with the benchmark problem of Mackey-Glass chaotic time series prediction.



---

*Source: 136214-2013-07-31.xml*