# Estimation of Fuzzy Measures Using Covariance Matrices in Gaussian Mixtures

Authors: Nishchal K. Verma
Journal: Applied Computational Intelligence and Soft Computing
 (2012)

---

## Abstract

This paper presents a novel computational approach for estimating fuzzy measures directly from Gaussian mixtures model (GMM). The mixture components of GMM provide the membership functions for the input-output fuzzy sets. By treating consequent part as a function of fuzzy measures, we derived its coefficients from the covariance matrices found directly from GMM and the defuzzified output constructed from both the premise and consequent parts of the nonadditive fuzzy rules that takes the form of Choquet integral. The computational burden involved with the solution ofλ-measure is minimized using Q-measure. The fuzzy model whose fuzzy measures were computed using covariance matrices found in GMM has been successfully applied on two benchmark problems and one real-time electric load data of Indian utility. The performance of the resulting model for many experimental studies including the above-mentioned application is found to be better and comparable to recent available fuzzy models. The main contribution of this paper is the estimation of fuzzy measures efficiently and directly from covariance matrices found in GMM, avoiding the computational burden greatly while learning them iteratively and solving polynomial equations of order of the number of input-output variables.


---

## Body

## 1. Introduction

Generalized fuzzy model (GFM) [1–3] is the backbone of this work that employs two norms for computing the strength of a rule: the multiplicative T-norm operator for determining the strength of a rule [4–6] and the additive S-norm operator for combining the outputs of all the rules. The effect of input fuzzy sets is taken into the defuzzified output in the form of rule strengths or weights. Gan et al. [7] have simplified the formulation of GFM by setting both the inputs and the output to be jointly Gaussian and proved that the input-output relation is as an expectation using the Bayesian framework. On simplification, this turns out to be the coveted Gaussian mixture model (GMM), which is linked by an additive function between the inputs and the output. The GMM, also known as cluster-weighted modeling (CWM) [8–12], is advocated by many researchers as means of statistical modeling of input-output systems [10, 13–15]. After the establishment of equivalence between GMM and GFM in [7], the difficult-to-compute index of fuzziness of GFM has become easy-to-compute prior probability [16–23] in GMM. Further simplification of GFM will be explored when it is converted into nonadditive case.The use of GMM in GFM has provided a generalized framework for additive fuzzy systems. A few existing additive fuzzy systems urging our attention are due to fuzzy models given in Kosko [24] and Wang and Mendel [25]. Using the framework of GMM, Abonyi et al. [26] have made a semblance of generalizing TS model [4, 27, 28] by attaching weight to each of its rules and subsequently treating it as the strength of the rule. However, nowhere they touched upon the concept of GFM that seeks to unify both Mamdani and TS models [4, 5].Incorporation of the nonadditive property into the fuzzy sets is done so that the corresponding output of a non-additive fuzzy system is explored here. Let us throw some light on the additive and non-additive fuzzy systems [29–31]. Conventional evaluation with multiple attributes which are independent is based on the concept of additive fuzzy systems where importance of each attribute is given a weight.Some important applications of fuzzy measures are now mentioned. A random generation of fuzzy measures have been introduced and some subfamilies of fuzzy measures tackled in [32]. A set of isometric fuzzy measures for any isometric transformation in each family of fuzzy measures (e.g., convex) is also studied. In [33] fuzzy measure is shown to be a unifying structure for modeling knowledge about an uncertain variable. Kim Le in [34] presents an expression to evaluate a fuzzy measure from set of aggregated evidences. Hierarchical autoregression model is presented in [34] using nonmonotonic fuzzy measures and the Choquet integral. Narukawa et al. in [35] discuss a space of fuzzy measures bearing the topology introduced by Choquet integral and the space of nonnegative fuzzy measure is shown to be locally convex. Fuzzy measures and integrals (e.g., Sugeno and Choquet) are used in decision making and modeling auctions [36, 37]. Fuzzy measures also find an application in [38] to track a moving object from a dynamic image sequence.Fascinated by the ever growing importance of the fuzzy measure theory [39–45], the additive function of the GFM model in the consequent part is replaced by a nonadditive function satisfying certain axioms of the fuzzy measures. The defuzzified output of the resultant non-additive GFM is shown to be in the form of Choquet integral [41]. This formulation is intended for real-life applications in which the information from different sources needs not to be additive; hence our efforts in this work will go a long way in evolving different types of non-additive fuzzy systems, like dynamic, adaptive, and so forth, but here our attempt is only on a simple non-additive fuzzy system. To take account of the overlapping information in the fuzzy sets, Sugeno [40] has introduced the concept of λ-measure such that an appropriate value of λ can account for the interaction. In the proposed formulation, Q-measure takes the role of this λ-measure for reducing the computational complexity of the former.Fuzzy integrals have come into vogue for the information fusion as they aggregate information from several sources. Out of all fuzzy integrals [40–44], the Choquet fuzzy integral [39, 41, 45] and Sugeno integral [46] are widely used. A handful of applications of fuzzy integrals include: handwriting recognition [47], landmine detection [48], pattern recognition [49], and decision making [37]. In the present study, the Choquet fuzzy integral is favored for it employs the non-additive property of fuzzy measures. The Choquet integral has not lost its luster even now; instead it is gaining prominence day by day and making inroads into new applications.The main theme of the present work is the estimation of fuzzy densities, and hence fuzzy measures for the non-additive fuzzy model are derived from GFM; thus the fuzzy densities can be calculated straightaway from the covariance matrices used in GMM [50]. As the large number of input variables is increased for the purpose of fusion, the computational complexity grows exponentially [31, 51] with the λ-measure. To overcome this problem a new fuzzy measure known as Q-Measure is introduced in [52]. This new measure is just normalization of λ-measure, and with this modification, the fuzzy densities are decorrelated. In this λ is taken as a variable to be learned using the training data set. The proposed modeling technique considerably reduces the computational complexity in two steps: firstly finding fuzzy densities without learning them through iterative process and secondly using Q-measure that eliminates the burden of solving the exponentially increasing polynomials.This paper is organized as follows. Section2 briefly describes the fuzzy measures. In Section 3, the formulation for the non-additive GFM is presented. An algorithm for non-additive fuzzy modeling based on Q-measure is given in Section 4. Two benchmark applications and one real-time electric load demand prediction application are provided in Section 5 to illustrate the proposed methodology. Finally conclusions are drawn in Section 6.


## 2. Fuzzy Measures

The fuzzy measure or fuzzy capacity is a subjective evaluation introduced by Choquet in 1953 and defined by Sugeno in 1974 for fuzzy integrals. Fuzzy measure includes a number of special class of measures like Sugeno’sλ-measure and Q-measure. In this work, we make use of Fuzzy measures in quantifying the influence of the interactive inputs on the output function using the fuzzy measures. The resulting model is non-additive in the sense that the information from the interactive inputs is computed non-additively. For converting GFM into a non-additive fuzzy system, the relevant properties of the fuzzy measures [46].Definition 1.
Let us denoteX={x1,x2,…,xD} as a finite set with its elements sorted from x and let χ be the power set of X, that is, set of all subsets of X. Then a fuzzy measure over a set X is a function given by
(1)ζ:χ⟶[0,1].
Satisfying the two properties(1)
ζ(ϕ)=0,ζ(X)=1.(2)
IfXi, Xj⊂χ and Xi⊂Xj, then ζ(Xi)≤ζ(Xj), which is the monotonicity relation in fuzzy measure. Next we invoke the third property, namely, the so-called λ-measure due to Sugeno [40] stated as follows:(3)
for allXi, Xj⊂χ with Xi∩Xj=ϕ(2)ζ(Xi∪Xj)=ζ(Xi)+ζ(Xj)+λζ(Xi)ζ(Xj).Example 2.
LetXd={xd,xd+1,…,xD} be the input sets satisfying the properties 1 and 2; then considering g as a fuzzy density satisfying the property 3, the values of fuzzy measure, ζ(Xd) can be computed recursively as
(3)ζ(Xd)=ζ({xd}∪Xd+1)=gd+ζ(Xd+1)+λgdζ(Xd+1),for1≤d<D,
starting with the one-input set to find the first measure as
(4)ζ(XD)=ζ({xD})=gD.
It is then combined with the next input set {xD-1} to compute their λ-measure since these two input sets are disjoint, that is, {xD}∩{xD-1}=ϕ, but their fuzzy sets AD and AD-1 are dependent, that is, AD∩AD-1≠ϕ. Equation (3) indicates that we must consider every time the fuzzy measure of two disjoint input sets, {xd} and Xd+1, which consists of inputs xd+1,…,xD. It may be noted that a single element fuzzy measure is the fuzzy density itself. The following polynomial which arises from the condition ζ(X)=1 (see the appendix) needs to be solved for λ:
(5)1+λ=∏d=1D(1+λgd).
The solution of (5) is computationally intensive. We will now discuss the Q-measure to address this problem.Q-Measure
Theλ-measure is burdened with the problem of finding the roots of the polynomial equation in λ. As the number of dimensions, D increases, the complexity of the equation increases enormously. In order to surmount this problem, Q-measure is proposed in [52]. It derives its name from the word “Quotient.” The Q-measure follows the same properties of λ-measure. For the finite X the Q-measure is defined as the ratio of two fuzzy measures:
(6)Q(Xd)=ζ(Xd)ζ(X)=ζ(Xd)ζ(X1),
where ζ(X)=ζ(X1) is a function of fuzzy densities and is a polynomial in λ. The Q-measure [52] over a set X is similar to ζ(X) in (1):
(7)Q:χ⟶[0,1],
with the two properties as follows:(1)
(8)Q(ϕ)=0,Q(X)=1.(2)
IfXi, Xj⊂χ and Xi⊂Xj, then Q(Xi)≤Q(Xj), which is the monotonicity relation.The third property is as follows:(3)
for allXi, Xj⊂χ with Xi∩Xj=ϕ,
(9)Q(Xi∪Xj)=Q(Xi)+Q(Xj)+λQ(Xi)Q(Xj)forλ>-1.
Following (9), Q(Xd) is evaluated from
(10)Q(Xd)=Q({xd}∪Xd+1)=gd+Q(Xd+1)+λ⋅gd⋅Q(Xd+1)for1≤d<D.
The first measure is computed from(11)Q(XD)=Q({xD})=gD.Convergence Behavior of Q-Measure
Let{x1}, {x2,x3,…,xD}⊂X Such that, {x1}∪{x2,x3,…,xD}=X, and {x1}∩{x2,x3,…,xD}=ϕ, that is, null. Supposing G1=ζ({x1}) and
(12)G2=ζ({x2,x3,…,xD})=ζ(X2)∀λ≥-1,G1≥0,G2≥0.
Then the λ-measure of G1 and G2 is
(13)ζ(X1)=G1+G2+λ⋅G1⋅G2.
Using (10) the Q-measure is written as
(14)Q(XD-1)=gD-1+Q(XD)+λ⋅gD-1⋅Q(XD),
or
(15)Q(XD-1)=gD-1+gD+λ⋅gD-1⋅gD.
For gD-1, gD∈[0,1] and λ≥-1, it is always assured that
(16)gD-1+gD>λ⋅gD-1⋅gD.
Hence, Q(XD-1)>gD-1 and also Q(XD-1)>gD. Next, we have(17)Q(XD-2)=gD-2+Q(XD-1)+λ⋅gD-2⋅Q(XD-1)=Q(XD-1)+gD-2(1+λ⋅Q(XD-1)),
where λQ(XD-1)<1. Therefore gD-2(1+λ·Q(XD-1))>0 and the values lie in between 0 and 1. Hence we have
(18)Q(XD-2)>Q(XD-1).
This will continue until D=3 when we will have
(19)Q(X1)>Q(X2).
But Q(X1)=1; this means that the Q-measure converges to unity.


## 3. From GMM to Nonadditive GFM withQ-Measure

The GFM [7] includes both the Mamdani and TS models. The Mamdani model (CRI-model) inhibits the property of fuzziness around the fixed centroid of the consequent part while the TS model gives a varying singleton for the consequent part in each fuzzy rule. The output of GFM is a fuzzy set, which is the output of Mamdani model with an index of fuzziness and with a varying centroid as the output of the TS model. To combine both of these properties, Azeem et al. [1, 53] have introduced a GFM fuzzy rule of the form(20)Rk:IFxkisAkTHENykisBk(fk(x),vk).The fuzzy setBk(fk(x),vk) may also be in the linguistic form with the output function fk(x) which may be linear or nonlinear regression of inputs and the index of fuzziness vk. The fuzzy output of the kth rule is yk. The defuzzified output obtained by applying additive S-norm to the consequent parts of GFM rules is(21)ŷ=∑k=1Kμk(x).vk∑l=1Kμl(x)⋅vlfk(x),
with fk(x)=b0k+b1kx1+⋯+bDkxD. As noted in Section 1 on introduction, GFM represents the additive fuzzy system. Using the framework of GMM, we intend to generalize to the case when the input fuzzy sets have overlapping information, that is, Aim∩Ai+1m≠ϕ. So, we modify the above fuzzy rule to the nonadditive GFM as
(22)Rm:ifxmisAmsuchthatAim∩Ai+1m≠ϕ.ThenymisBm(fm′(X),υm)withX⊆x.
To take account of the above constraint, we form the input set X from which the fuzzy measures of subsets Xd=({xd}∪Xd) are found recursively using (10) and (11). Using the fuzzy measures as the coefficients the output function is expressed as [31](23)fm′(X)=β0m+β1mx1+⋯+βDmxD.
Equation (23) is an additive function of fuzzy measures as explained in the sequel. As the input fuzzy sets in (22) are overlapping, we take disjoint input sets from the power set χ and compute their fuzzy measure. The function fm′(X) combines the fuzzy measures from subsequent fuzzy sets, and the resulting system arising from the fuzzy rules is called nonadditive fuzzy system unlike the output function in TS model where each coefficient corresponds to its own fuzzy set and there is no concept of fuzzy measure in this model. We now impose three constraints for the model to be called non-additive:(i)
the input fuzzy setsAi associated with the power set χ are overlapping,(ii)
the input setsXi are disjoint,(iii)
the output function is additive in fuzzy measures.We will now show that the above conditions are necessary to represent the defuzzified output of the nonadditive GFM in the Choquet integral form as follows:(24)ŷj=∑m=1M(μm(xj)⋅p(cm)∑l=1Mμl(xj)⋅p(cl))⋅fm′(X)=∑m=1M(μm(xj)⋅p(cm)∑l=1Mμl(xj)⋅p(cl))⋅∑d=0Dβdm⋅xjd,
where for jth set of the data we have(25)μm(xj)=∏d=1Dμm(xjd)=∏d=1DN1(xjdm;x̅dm,∑m,d),N1(xjdm;x̅dm,∑m,d)=exp⁡(-(xjdm-x̅dm)22∑m,d),with∑m,d being the diagonal element. The firing strength of each rule is obtained by taking the multiplicative T-norm of the membership functions (by suppressing the subscripts “j”) of the premise parts of the rules as μm(x)=μm(x1)Λμm(x2)Λ⋯Λμm(xD). We intend to compute the coefficients of the output function by reformulating (21) in the Choquet fuzzy integral form. The role of GMM is to facilitate the computation of M,μm(x) and p(cm).


### 3.1. Choquet Fuzzy Integral and Q-Measure

When a fuzzy measure is available on a finite setX, we can use a fuzzy integral as a computational scheme to integrate all values from χ, that is, the subsets Xi of X nonlinearly. In other words, a fuzzy integral relies on the concept of a fuzzy measure. A general definition is that given a class of functions F⊆{h:χ→ℜ+} and a class of Q-measures Q(·), we have a functional f:F×Q=h×Q→ℜ, which is a fuzzy integral [42]. The Choquet integral is a particular fuzzy integral [41] that serves as an alternative computational scheme for aggregating information.Definition 3.
LetQ-measure over a set X be a function whose elements (discrete) are x1,x2,…,xD, then the Choquet integral of a positive function h with respect to Q is defined by [39]
(26)fg(h)=∑d=1D((h(xd)-h(xd-1))Q(Xd)),
or
(27)fg(h)=∑d=1Dh(xd)(Q(Xd)-Q(Xd+1)),
where h(x1)≤h(x2)≤⋯≤h(xD). We assume here that Q(XD+1)=0 and h(x0)=0. To convert (24) into the Choquet fuzzy integral form, we shall represent coefficients βdm in terms of Q-measures, Q(Xd) as
(28)βdm=Qm(Xd)-Qm(Xd+1)=gdm+λmgdmQm(Xd+1),ford=1,2,3…,DandQm(XD+1)=0.
Interchanging the summations in (24) leads to
(29)ŷj=∑d=0D(∑m=1M(μm(xj)⋅p(cm)⋅xjd∑l=1Mμl(xj)⋅p(cl)))⋅(Qm(Xd)-Qm(Xd+1)).
Next separating out the d=0 term yields,
(30)ŷj=a0+∑d=1D(∑m=1M(μm(xj)⋅p(cm)⋅xjd∑l=1Mμl(xj)⋅p(cl)))⋅(Qm(Xd)-Qm(Xd+1)),
where a0 is a constant. Letting
(31)hm(xjd)=(μm(xj)⋅p(cm)⋅xjd∑l=1Mμl(xj)⋅p(cl)).Equation (30) is simplified as
(32)ŷj=a0+∑d=1D(∑m=1Mhm(xjd)⋅(Qm(Xd)-Qm(Xd+1))).
Denoting [h(xjd)]=[h1(xjd)h2(xjd)⋯hM(xjd)] and
(33)[(Q(Xd)-Q(Xd+1))]=[(Q1(Xd)-Q1(Xd+1))(Q2(Xd)-Q2(Xd+1))⋮(QM(Xd)-QM(Xd+1))]
allows us to write (32) compactly for D input variables as
(34)ŷj=a0+∑d=1D[h(xjd)]⋅[(Q(Xd)-Q(Xd+1))],
or
(35)ŷj=a0+∑d=1D[h(xjd)-h(xj(d-1))]⋅[Q(Xd)],
where hm(xj1)≤hm(xj2)≤⋯≤hm(xjD) and hm(xj0)=0 for m=1,2,…,M, is the monotonicity condition. Note that hm(xj1),hm(xj2),…,hm(xjD) are the evidences provided by the input sources x1,x2,…,xD, respectively, and Qm(X1),Qm(X2),…,Qm(XD) are the Q-measures formth cluster. Here, we take Qm(XD+1) to be zero. Equations (34) and (35) similar to (26) and (27) are the two forms of Choquet fuzzy integral [42]. We have now proved that the defuzzified output of the non-additive GFM fuzzy rules is in the form of Choquet fuzzy integral. The underlying model is nonadditive GFM because both h and Q are nonlinear functions of x. Thus the Choquet integral provides the functionality of non-additive GFM. The functionality of the Choquet integral for GMM will be discussed later. We will now estimate the unknown Q-measures from fuzzy densities determined using (9). We then evaluate the performance of the fuzzy model with a measure, which is a function of mean square error [54], chosen as
(36)E=∑j=1nej2n⋅yr,
where n is the total number of test data vectors:
(37)ej=ŷj-yj,yr=[max⁡(Y)-min⁡(Y)]2,
where ŷj is the estimated value, yj is the actual output, and Y={yj} is the output vector.


### 3.2. Determination of Fuzzy Densities

The cluster-wise breakup of the estimated output of the additive fuzzy system [7] is given as(38)ŷj=(μ1(xj)⋅p(c1)⋅f(xj,α1)∑l=1Mμl(xj)⋅p(cl))+(μ2(xj)⋅p(c2)⋅f(xj,α2)∑l=1Mμl(xj)⋅p(cl))+⋯+(μM(xj)⋅p(cM)⋅f(xj,αM)∑l=1Mμl(xj)⋅p(cl)),
where using GMM from [7] we estimate f(xj,αm) as(39)f(xj,αm)=y̅jm-∑d=1D((xjd-x̅dm)⋅σmdyσmyy).Equation (38) can also be written as(40)ŷj=w1⋅f(xj,α1)+w2⋅f(xj,α2)+⋯+wM⋅f(xj,αM),
where wm=((μm(xj)·p(cm))/(∑l=1Mμl(xj)·p(cl))). Substituting f(xj,αm)form=1toM from (39) in (40), we have(41)w1⋅(y̅1-∑d=1D((xjd-x̅d1)⋅σ1dyσ1yy))+w2⋅(y̅2-∑d=1D((xjd-x̅d2)⋅σ2dyσ2yy))⋯⋯+wM⋅(y̅M-∑d=1D((xjd-x̅dM)⋅σMdyσMyy))=a01+w1(xj1⋅g11+xj2⋅g21+⋯+xjD⋅gD1)+a02+w2(xj1⋅g12+xj2⋅g22+⋯+xjD⋅gD2)⋯⋯+a0M+wM(xj1⋅g1M+xj2⋅g2M+⋯+xjD⋅gDM).
Using (32), for M rules, we have the cluster-wise estimates of the output for the non-additive fuzzy model written as(42)ŷj=a01+∑d=1Dh1(xjd)⋅(Q1(Xd)-Q1(Xd+1))+a02+∑d=1Dh2(xjd)⋅(Q2(Xd)-Q2(Xd+1))⋯⋯+a0M+∑d=1DhM(xjd)⋅(QM(Xd)-QM(Xd+1)).
From (42) it is implied that a0=a01+a02+⋯+a0M in view of (34) and (35).Equation (42) can also be written as(43)ŷ=a01+w1(xj1⋅g11(1+λ1Q1(X2))+xj2⋅g21(1+λ1Q1(X3))+⋯+xjD⋅gD1)+a02+w2(xj1⋅g12(1+λ2Q2(X2))+xj2⋅g22(1+λ2Q2(X3))+⋯+xjD⋅gD2)⋯⋯+a0M+wM(xj1⋅g1M(1+λMQM(X2))+xj2⋅g2M(1+λMQM(X3))+⋯+xjD⋅gDM)
or
(44)ŷ=a0+w1⋅f′(xj,β1)+w2⋅f′(xj,β2)+⋯+wM⋅f′(xj,βM).
Since (40) and (44) represent the estimated outputs for additive and non-additive fuzzy systems, respectively, and hence these must be equivalent to each other for λ=0. Equating the right hand sides of (40) and (44), we have(45)w1⋅f(xj,α1)+w2⋅f(xj,α2)+⋯+wM⋅f(xj,αM)=a0+w1⋅f′(xj,β1)+w2⋅f′(xj,β2)+⋯+wM⋅f′(xj,βM).
Thus when the fuzzy measure λm=0,αi=βi. As αi=gim we can express the fuzzy densities in terms of the elements of covariance matrices of GMM as [7](46)g1m=σm1yσmyy;g2m=σm2yσmyy,⋯gDm=σmDyσmyyform=1,2,…,M.
We are in a position to estimate λ and Q-measures and hence the β coefficients using the fuzzy densities. We will then use all these in the Choquet integral to estimate the output. For convenience of notation we start with fuzzy measure of one element set Xd={xD} and move in backward direction to find the Q-measure of X1. Since we are computing all Q-measures before the start of the Choquet integral, these become handy in computing the output. This is because in (32) the index d is in the forward direction from 1 to D.


### 3.3. Functionality of GMM

As we have already proved that the Choquet integral is the functionality of non-additive GFM, It is now easy to extend to this functionality the GMM case from the fact that the output is Gaussian in the nonadditive case too as per the expression(47)p(y∣x,cm)=12π∑m,(D+1)⋅exp⁡{-∑j[yjm-f(xjm,βm)]22∑m,(D+1)}.
From the above it may be noted that βm is obtained from αm using GMM by the choice of λm in the Q-measure. Hence f(xjm,βm) is the mean of yjm in the non-additive case just as f(xjm,αm) is the mean of yjm in the additive case. We can conclude therefore that when βm is evaluated from αm, the Choquet integral also assures the functionality of GMM.


## 3.1. Choquet Fuzzy Integral and Q-Measure

When a fuzzy measure is available on a finite setX, we can use a fuzzy integral as a computational scheme to integrate all values from χ, that is, the subsets Xi of X nonlinearly. In other words, a fuzzy integral relies on the concept of a fuzzy measure. A general definition is that given a class of functions F⊆{h:χ→ℜ+} and a class of Q-measures Q(·), we have a functional f:F×Q=h×Q→ℜ, which is a fuzzy integral [42]. The Choquet integral is a particular fuzzy integral [41] that serves as an alternative computational scheme for aggregating information.Definition 3.
LetQ-measure over a set X be a function whose elements (discrete) are x1,x2,…,xD, then the Choquet integral of a positive function h with respect to Q is defined by [39]
(26)fg(h)=∑d=1D((h(xd)-h(xd-1))Q(Xd)),
or
(27)fg(h)=∑d=1Dh(xd)(Q(Xd)-Q(Xd+1)),
where h(x1)≤h(x2)≤⋯≤h(xD). We assume here that Q(XD+1)=0 and h(x0)=0. To convert (24) into the Choquet fuzzy integral form, we shall represent coefficients βdm in terms of Q-measures, Q(Xd) as
(28)βdm=Qm(Xd)-Qm(Xd+1)=gdm+λmgdmQm(Xd+1),ford=1,2,3…,DandQm(XD+1)=0.
Interchanging the summations in (24) leads to
(29)ŷj=∑d=0D(∑m=1M(μm(xj)⋅p(cm)⋅xjd∑l=1Mμl(xj)⋅p(cl)))⋅(Qm(Xd)-Qm(Xd+1)).
Next separating out the d=0 term yields,
(30)ŷj=a0+∑d=1D(∑m=1M(μm(xj)⋅p(cm)⋅xjd∑l=1Mμl(xj)⋅p(cl)))⋅(Qm(Xd)-Qm(Xd+1)),
where a0 is a constant. Letting
(31)hm(xjd)=(μm(xj)⋅p(cm)⋅xjd∑l=1Mμl(xj)⋅p(cl)).Equation (30) is simplified as
(32)ŷj=a0+∑d=1D(∑m=1Mhm(xjd)⋅(Qm(Xd)-Qm(Xd+1))).
Denoting [h(xjd)]=[h1(xjd)h2(xjd)⋯hM(xjd)] and
(33)[(Q(Xd)-Q(Xd+1))]=[(Q1(Xd)-Q1(Xd+1))(Q2(Xd)-Q2(Xd+1))⋮(QM(Xd)-QM(Xd+1))]
allows us to write (32) compactly for D input variables as
(34)ŷj=a0+∑d=1D[h(xjd)]⋅[(Q(Xd)-Q(Xd+1))],
or
(35)ŷj=a0+∑d=1D[h(xjd)-h(xj(d-1))]⋅[Q(Xd)],
where hm(xj1)≤hm(xj2)≤⋯≤hm(xjD) and hm(xj0)=0 for m=1,2,…,M, is the monotonicity condition. Note that hm(xj1),hm(xj2),…,hm(xjD) are the evidences provided by the input sources x1,x2,…,xD, respectively, and Qm(X1),Qm(X2),…,Qm(XD) are the Q-measures formth cluster. Here, we take Qm(XD+1) to be zero. Equations (34) and (35) similar to (26) and (27) are the two forms of Choquet fuzzy integral [42]. We have now proved that the defuzzified output of the non-additive GFM fuzzy rules is in the form of Choquet fuzzy integral. The underlying model is nonadditive GFM because both h and Q are nonlinear functions of x. Thus the Choquet integral provides the functionality of non-additive GFM. The functionality of the Choquet integral for GMM will be discussed later. We will now estimate the unknown Q-measures from fuzzy densities determined using (9). We then evaluate the performance of the fuzzy model with a measure, which is a function of mean square error [54], chosen as
(36)E=∑j=1nej2n⋅yr,
where n is the total number of test data vectors:
(37)ej=ŷj-yj,yr=[max⁡(Y)-min⁡(Y)]2,
where ŷj is the estimated value, yj is the actual output, and Y={yj} is the output vector.


## 3.2. Determination of Fuzzy Densities

The cluster-wise breakup of the estimated output of the additive fuzzy system [7] is given as(38)ŷj=(μ1(xj)⋅p(c1)⋅f(xj,α1)∑l=1Mμl(xj)⋅p(cl))+(μ2(xj)⋅p(c2)⋅f(xj,α2)∑l=1Mμl(xj)⋅p(cl))+⋯+(μM(xj)⋅p(cM)⋅f(xj,αM)∑l=1Mμl(xj)⋅p(cl)),
where using GMM from [7] we estimate f(xj,αm) as(39)f(xj,αm)=y̅jm-∑d=1D((xjd-x̅dm)⋅σmdyσmyy).Equation (38) can also be written as(40)ŷj=w1⋅f(xj,α1)+w2⋅f(xj,α2)+⋯+wM⋅f(xj,αM),
where wm=((μm(xj)·p(cm))/(∑l=1Mμl(xj)·p(cl))). Substituting f(xj,αm)form=1toM from (39) in (40), we have(41)w1⋅(y̅1-∑d=1D((xjd-x̅d1)⋅σ1dyσ1yy))+w2⋅(y̅2-∑d=1D((xjd-x̅d2)⋅σ2dyσ2yy))⋯⋯+wM⋅(y̅M-∑d=1D((xjd-x̅dM)⋅σMdyσMyy))=a01+w1(xj1⋅g11+xj2⋅g21+⋯+xjD⋅gD1)+a02+w2(xj1⋅g12+xj2⋅g22+⋯+xjD⋅gD2)⋯⋯+a0M+wM(xj1⋅g1M+xj2⋅g2M+⋯+xjD⋅gDM).
Using (32), for M rules, we have the cluster-wise estimates of the output for the non-additive fuzzy model written as(42)ŷj=a01+∑d=1Dh1(xjd)⋅(Q1(Xd)-Q1(Xd+1))+a02+∑d=1Dh2(xjd)⋅(Q2(Xd)-Q2(Xd+1))⋯⋯+a0M+∑d=1DhM(xjd)⋅(QM(Xd)-QM(Xd+1)).
From (42) it is implied that a0=a01+a02+⋯+a0M in view of (34) and (35).Equation (42) can also be written as(43)ŷ=a01+w1(xj1⋅g11(1+λ1Q1(X2))+xj2⋅g21(1+λ1Q1(X3))+⋯+xjD⋅gD1)+a02+w2(xj1⋅g12(1+λ2Q2(X2))+xj2⋅g22(1+λ2Q2(X3))+⋯+xjD⋅gD2)⋯⋯+a0M+wM(xj1⋅g1M(1+λMQM(X2))+xj2⋅g2M(1+λMQM(X3))+⋯+xjD⋅gDM)
or
(44)ŷ=a0+w1⋅f′(xj,β1)+w2⋅f′(xj,β2)+⋯+wM⋅f′(xj,βM).
Since (40) and (44) represent the estimated outputs for additive and non-additive fuzzy systems, respectively, and hence these must be equivalent to each other for λ=0. Equating the right hand sides of (40) and (44), we have(45)w1⋅f(xj,α1)+w2⋅f(xj,α2)+⋯+wM⋅f(xj,αM)=a0+w1⋅f′(xj,β1)+w2⋅f′(xj,β2)+⋯+wM⋅f′(xj,βM).
Thus when the fuzzy measure λm=0,αi=βi. As αi=gim we can express the fuzzy densities in terms of the elements of covariance matrices of GMM as [7](46)g1m=σm1yσmyy;g2m=σm2yσmyy,⋯gDm=σmDyσmyyform=1,2,…,M.
We are in a position to estimate λ and Q-measures and hence the β coefficients using the fuzzy densities. We will then use all these in the Choquet integral to estimate the output. For convenience of notation we start with fuzzy measure of one element set Xd={xD} and move in backward direction to find the Q-measure of X1. Since we are computing all Q-measures before the start of the Choquet integral, these become handy in computing the output. This is because in (32) the index d is in the forward direction from 1 to D.


## 3.3. Functionality of GMM

As we have already proved that the Choquet integral is the functionality of non-additive GFM, It is now easy to extend to this functionality the GMM case from the fact that the output is Gaussian in the nonadditive case too as per the expression(47)p(y∣x,cm)=12π∑m,(D+1)⋅exp⁡{-∑j[yjm-f(xjm,βm)]22∑m,(D+1)}.
From the above it may be noted that βm is obtained from αm using GMM by the choice of λm in the Q-measure. Hence f(xjm,βm) is the mean of yjm in the non-additive case just as f(xjm,αm) is the mean of yjm in the additive case. We can conclude therefore that when βm is evaluated from αm, the Choquet integral also assures the functionality of GMM.


## 4. Algorithm for Estimating the Model Parameters of the Nonadditive Fuzzy Systems

The algorithm has the following steps.Step 1.
Normalize the input-output data, so that the data values lie in between 0 and 1:(48)xnormalized=(x-min⁡(z))(max⁡(z)-min⁡(z)),
where z=[x,y] is the input-output data vector.Step 2.
Find the premise model parameters of GMM using the EM Algorithm.Step 3.
Determine fuzzy densitiesg1m,g2m,…,gDm for each input variable in mth cluster/rule using (47).Step 4.
Choose initial values ofλm with the restriction λm≥-1 for mth cluster.Step 5.
ComputeQm(Xd) by the recursive computation involving the following equations from (10) and (11):(49)Qm(XD)=Qm({xD})=gDm,Qm(Xd)=gdm+Qm(Xd+1),+λmgdmQm(Xd+1),for1≤d<D.Step 6.
Using (31) find hm(xd) for each input variable d in the cluster/rule to obtain the matrix [h(xd)].Step 7.
Compute the estimated output using (34).Step 8.
Learn the model. This requires the following two subtasks:(a)
set up an objective function [54] as the case required. For batch:(50)Jbatch=1n∑j=1n(ŷj-yj)2=1n∑j=1nej2,
for incremental,
(51)Jincr=12(ŷj-yj)2=12ej2.(b)
Update the values ofλm by the Gradient-descent learning law:(52)(λm)new=(λm)old-η⋅(∂J∂λm),a0,new=a0-η⋅(∂J∂a0),
where J may refer to either Jbatch or Jincr and η is a learning factor fixed at the value 0.9 for better performance. The parameter learning can be accomplished in either of the above two ways: batch or incremental. Derivatives required in the learning process are relegated to the Appendix.Step 9.
Repeat Step1 to Step 8, till the model parameters are within the tolerance.Step 10.
CalculateE from (36).


## 5. Experimental Results

### 5.1. Gas Furnace

Here, we take up the gas furnace data [55] to demonstrate the effectiveness of the proposed algorithm. In this gas furnace, the air at a feed rate of u(t) is combined with methane to produce a mixture of gases with carbon dioxide of concentration y(t). These are used to fit the Gaussian mixture density function with four components. The input vector after sorting is x=[x1x2x3x4]=[y(t-1)-y(t-2)y(t-1)u(t)-u(t-1)u(t)] and the output is y=y(t). The composition of the input vector is arrived at using the fuzzy curve approach [56]. On applying the expectation maximization (EM) [57] clustering, the number of clusters is found to be 4; hence the model will have four components. The five-variable GMM using the EM algorithm yields the parameters of GMM as shown in Table 1. These parameters are also the premise model parameters of the non-additive GFM fuzzy rules.Table 1
Estimated parameters of A 4-component GMM using the EM algorithm.
Cluster or rule no.(m)Weight of the rulep(cm)Mean(z̅m)Covariance(∑m)Cluster or rule no. 1p(c1)=0.2991z̅1=[0.40660.28970.43410.53210.2986]∑1=[0.02460.00320.0114-0.00750.00950.00320.0150-0.0027-0.01060.01490.0114-0.00750.0095-0.0027-0.01060.01490.01610.01040.00040.01040.0427-0.01460.0004-0.01460.0170]Cluster or rule no. 2p(c2)=0.1685z̅2=[0.43720.67820.46760.22320.6904]∑2=[0.05030.00560.0151-0.01240.01700.00560.02090.00620.00850.01800.01510.00620.0090-0.00190.0096-0.01240.0085-0.00190.01500.00250.01700.01800.00960.00250.0194]Cluster or rule no. 3p(c3)=0.2166z̅3=[0.55580.71000.59960.41780.7355]∑3=[0.0198-0.00900.0049-0.0066-0.0039-0.00900.0234-0.0018-0.00340.01990.0049-0.00180.0098-0.00290.0004-0.0066-0.0034-0.00290.0140-0.0054-0.00390.01990.0004-0.00540.0181]Cluster or rule no. 4p(c4)=0.3158z̅4=[0.32110.53660.52650.59630.5065]∑4=[0.0135-0.00570.00130.0016-0.0033-0.00570.02700.0069-0.01530.02560.00130.00690.01830.00640.00730.0016-0.01530.00640.0208-0.0152-0.00330.02560.0073-0.01520.0248]Fuzzy densities and estimated fuzzy measures of the model are given in Tables2 and 4, respectively. The estimated values of λm are given in Table 3. The estimated β-coefficient and Q-measures are given in Tables 5 and 6 respectively. Note that ζ(X1) contains all input elements, hence its value is one. The β-coefficients constitute the consequent parameters of the GMM fuzzy rules. The performance measure is found to be 0.00208. The fuzzy measures of the constituent singleton sets satisfy the monotonic relations for D=4 shown as a lattice in Figure 2.Table 2
Fuzzy densities using GMM.
Rule(m)g1g2g3g4Rule no. 10.24730.77600.00050.0000Rule no. 20.56990.82040.00280.0003Rule no. 30.62840.86710.00700.0001Rule no. 40.48440.73300.00420.0012Table 3
Estimated lambda (λm).
Rule 1Rule 2Rule 3Rule 4−0.1241−0.8364−0.9108−0.6197Table 4
Estimated fuzzy measures.
Rule(m)ζ(X1)ζ(X2)ζ(X3)ζ(X4)Rule no. 11.00000.79230.00210.0009Rule no. 20.99970.82140.00320.0003Rule no. 30.99990.86860.00710.0001Rule no. 40.99940.73590.00530.0012Table 5
Estimatedβ coefficients.
Rule(m)β1β2β3β4Rule no. 10.20770.79020.00120.0009Rule no. 20.17830.81820.00290.0003Rule no. 30.13130.86150.00700.0001Rule no. 40.26350.73060.00410.0012Table 6
EstimatedQ-measures.
Rule(m)Q(X1)Q(X2)Q(X3)Q(X4)Rule no. 11.00000.78230.00130.0000Rule no. 21.00000.85410.06350.0001Rule no. 31.00000.91750.08630.0003Rule no. 41.00000.72300.01690.0002For the training dataset containing the first 250 data vectors,E=6.12505×10-4. For the remaining data vectors forming the validation data set, E=0.0021. This is inferior to that of the training data set, as expected. The plots of actual output and model output for both training and prediction along with the prediction error of the model output are shown in Figure 1. The model output is very close to the actual output during training, but it displays small deviations from the actual output during the prediction period. Using the premise and consequent parameters of Table 1, we have constructed the membership functions for the four inputs and one output in Figure 3 where we depict the membership function of rules-1, 2, 3, and 4 using dashed lines, dotted lines, dots with dashes lines and solid lines, respectively.Plot of actual output and model output for training data set (uppermost plot) and plot of actual output and predicted output and the corresponding prediction error for Box and Jenkins’s gas furnace data.
(a)(b)(c)Figure 2
Lattice of fuzzy measure values of the constituent singleton sets [39] of rule 1 for Box and Jenkins’s data.(a) Membership function for input one. (b) Membership functions for input two. (c) Membership function for input three. (d) Membership function for input four. (e) Membership function for output.
(a)(b)(c)(d)(e)The first ruleR1 consists of,(i)
the weight of the rule; 0.2991,(ii)
the membership functions of the premise parts(53)N1(x1;104066,0.0246),N1(x2;0.2897,0.0150),N1(x3;0.4341,0.0161),N1(x4;0.5321,0.0427).(iii)
The consequent part of a rule can be found using Table6.Similarly other rules can also be formed.


### 5.2. Industrial Dryer

In this, we consider the industrial dryer data obtained from theDaISy database. Fuel is combusted to produce the hot gas, which is blown by an exhaust fan on the raw material fed into the dryer. It is desired to determine the relationship between the functional elements of the dryer with the moisture content of the raw material at the dryer output. The measurements are designated by(i)
fuel flow rate,x1(t),(ii)
hot gas exhaust-fan speed,x2(t),(iii)
rate of flow of raw material,x3(t),(iv)
moisture content of the raw material after drying which isy(t).Sampling period of 10 seconds is considered to generate a total of 867 samples. Input and output data are normalized using (40) to maintain compatibility in the range 0 to 1. The input variables x1(t), x2(t), x3(t), and y(t-1) affect the current output y(t). The input-output vectors are represented by z(t)=[x1(t)x2(t)x3(t)y(t-1)y(t)]. Out of these vectors 70% is taken as the training set. The remaining 30% is kept for prediction and evaluation of our proposed model.The number of clusters is found to be 4 using EM clustering, which yields the parameters of GMM in Table7, and these are also the premise model parameters of the non-additive GFM.Table 7
Estimated parameters of A 4-component GMM using the EM algorithm.
Cluster or rule no.(m)Weight of the rulep(cm)Mean(z̅m)Covariance(∑m)Cluster or rule no. 1p(c1)=0.3564z̅1=[0.3946-0.6119-0.0988-0.3346-0.3402]∑1=[0.4246-0.02430.1174-0.00490.0066-0.02430.1174-0.00490.00660.0983-0.0372-0.0558-0.0618-0.03720.61880.02290.0355-0.05580.02290.39520.3608-0.06180.03550.36080.3933]Cluster or rule no. 2p(c2)=0.2723z̅2=[-0.19520.1800-0.18971.55851.5501]∑2=[0.5879-0.2083-0.0120-0.0916-0.1151-0.20830.7343-0.16700.0011-0.0073-0.0120-0.16700.87010.07020.1139-0.09160.00110.07020.28020.2227-0.1151-0.00730.11390.22270.2849]Cluster or rule no. 3p(c3)=0.2046z̅3=[-1.18111.4027-0.4488-0.1763-0.1760]∑3=[0.70080.1105-0.24000.07110.05820.11050.2471-0.0056-0.0244-0.0416-0.2400-0.00560.4133-0.1059-0.09530.0711-0.0244-0.10590.46610.42260.0582-0.0416-0.09530.42260.4704]Cluster or rule no. 4p(c4)=0.1667z̅4=[-0.30411.04430.9807-0.00320.0179]∑4=[0.42830.0685-0.2311-0.0418-0.09530.06850.6950-0.41110.0238-0.0029-0.2311-0.41110.76990.12820.1662-0.04180.02380.12820.32100.2911-0.0953-0.00290.16620.29110.3528]The estimated fuzzy densities, fuzzy measures,β-coefficients and Q-measure of the model are given in Tables 8, 10, 11, and 12, respectively. The monotonicity relations for D=4 are also verified here for fuzzy measure values of the constituent singleton sets, shown as a lattice in Figure 5. Table 9 shows the estimated values of λm.Table 8
Fuzzy densities using GMM.
Rule(m)g1g2g3g4Rule no. 10.49820.42950.38810.7989Rule no. 20.49960.73680.61580.9027Rule no. 30.46960.75430.37040.5916Rule no. 40.48440.73300.00420.0012Table 9
Estimated lambda (λm).
Rule 1Rule 2Rule 3Rule 4−0.9970−1.0000−1.0000−0.9947Table 10
Estimatedλ-measures.
Rule(m)ζ(X1)ζ(X2)ζ(X3)ζ(X4)Rule no. 10.96690.93150.87790.7989Rule no. 20.90270.88280.87580.8749Rule no. 30.95000.65290.62440.5918Rule no. 41.00001.00001.00000.9981Table 11
Estimatedβ coefficients.
Rule(m)β1β2β3β4Rule no. 10.03540.05360.07900.7989Rule no. 20.01990.00700.00090.8749Rule no. 30.29710.02850.03260.5918Rule no. 40.00000.00000.00190.9981Table 12
EstimatedQ-measures.
Rule(m)Q(X1)Q(X2)Q(X3)Q(X4)Rule no. 11.00000.78230.00130.0000Rule no. 21.00000.85410.06350.0001Rule no. 31.00000.91750.08630.0003Rule no. 41.00000.72300.01690.0002The training data set yieldsE=9.0005×10-4 for training the model. The remaining 30% data vectors constitute the validation data set. The performance measure E over the prediction period is obtained as 0.0019. The results of the actual output and the model output during the training and the prediction periods are shown in Figure 4 which also includes the prediction error. The results demonstrate the effectiveness of the proposed non-additive fuzzy model using Q-measure. The membership functions associated with all the four rules can be obtained in the similar fashion as in Figure 3.Plot of actual output and model output for training data set (uppermost plot) and plot of actual output and predicted output and the corresponding prediction error for industrial dryer data.
(a)(b)(c)Figure 5
Lattice of fuzzy measure values of the constituent singleton sets of rule no.1 of Table10 for industrial dryer’s data.


### 5.3. Electric Load Demand

In this, we consider the real time hourly electric load data, that is, hourly peak electric load consumption with uncertain weather information like hourly average temperature, humidity, and wind speed for the month of April 2003. The variables are designated as(i)
hourly average Temperature,x1(t),(ii)
hourly average Humidity,x2(t),(iii)
hourly average Wind speed,x3(t),(iv)
hourly peak electric load consumption which isy(t).A number of 720 samples of Input and output data are normalized using (40) to maintain the compatibility in the range 0 to 1. We consider 75% of the total input-output vectors of the form z(t)=[10·(y(t)-y(t-2))10·(x3(t)-x2(t-2))10·(x2(t)-x2(t-2))50·(x3(t-2)-x3(t))100·y(t)] as the training set. The remaining 25% is reserved for the prediction. The weights before the components in z(t) have been fixed for achieving better prediction capability. It may be noted that the criteria only facilitate the selection of proper inputs by trying out different combinations. But the final curtain falls on the selection with the verification through the performance measure.The number of clusters in this example also is found to be 4 using the EM clustering after discarding a few datasets as outliers. Application of the EM algorithm yields the parameters of GMM in Table13. These constitute the premise parameters of the GFM. The estimated fuzzy densities, lambda values, fuzzy measure, Q-measure and β-coefficients of the model are given in Tables 14, 15, 16, 17, and 18, respectively. The monotonicity relations can also be verified as in the previous applicationsTable 13
Estimated parameters of A 4-component GMM using the EM algorithm (load).
Cluster or rule no.(m)Weight of the rulep(cm)Mean(z̅m)Covariance(∑m)Cluster or rule no. 1p(c1)=0.2646z̅1=[0.47200.45000.46940.38200.5007]∑1=[0.0031-0.0000-0.00107-0.00160.0026-0.00000.03060.0160-0.00510.0010-0.00170.01600.02460.0050-0.0005-0.0016-0.00510.00500.0227-0.00150.00260.0010-0.0005-0.00150.0029]Cluster or rule no. 2p(c2)=0.2225z̅2=[0.52260.74300.55640.45980.5269]∑2=[0.0188-0.00070.0031-0.00040.0122-0.00070.00810.0031-0.0044-0.00010.00310.00310.00520.00340.0024-0.0004-0.00440.00340.03080.00090.01220.0001-0.00240.00090.0084]Cluster or rule no. 3p(c3)=0.2474z̅3=[0.29020.30000.68440.43930.3771]∑3=[0.0228-0.0014-0.0035-0.00120.0137-0.00140.01540.00180.0003-0.0044-0.00350.00180.00650.0018-0.0025-0.00120.00030.00180.0054-0.00200.0137-0.0044-0.0025-0.00200.0142]Cluster or rule no. 4p(c4)=0.2655z̅4=[0.64740.57380.73720.49400.6013]∑4=[0.01020.00110.0009-0.00230.00680.00110.0093-0.00040.00300.00370.0009-0.00040.0108-0.00330.0002-0.00230.0030-0.00330.01890.00240.00680.00370.00020.00240.0073]Table 14
Fuzzy densities using GMM.
Rule(m)g1g2g3g4Rule no. 10.84790.64880.58170.6735Rule no. 20.01960.06330.00010.2634Rule no. 30.02640.01000.07250.0255Rule no. 40.00010.04690.00010.1708Table 15
Estimated lambda (λm).
Rule 1Rule 2Rule 3Rule 429.3843−0.0776−0.99950.2543Table 16
Estimatedλ-measures.
Rule(m)ζ(X1)ζ(X2)ζ(X3)ζ(X4)Rule no. 11.00000.08770.04390.6735Rule no. 21.00000.12000.05690.2634Rule no. 31.00000.09100.08180.0255Rule no. 41.00000.47180.19740.1708Table 17
EstimatedQ-measures.
Rule(m)Q(X1)Q(X2)Q(X3)Q(X4)Rule no. 11.00000.02910.01450.0033Rule no. 21.00000.15690.07440.0614Rule no. 31.00000.14680.13200.0161Rule no. 41.00000.38910.16280.1409Table 18
Estimatedβ coefficients.
Rule(m)β1β2β3β4Rule no. 10.97090.01460.01120.0033Rule no. 20.84310.08250.01300.0614Rule no. 30.85320.01480.11590.0161Rule no. 40.61090.22630.02190.1409The training data set is found to giveE=0.0034. The remaining 25% data vectors constitute the validation data set. The performance measure E over this set is obtained as 0.00256. The results of the actual output and the model output during the training and the prediction periods are shown in Figure 6, which also includes the prediction error. The results demonstrate the effectiveness of the proposed non-additive fuzzy model using Q-measure for this application too (Figure 7).Plot of actual output and model output for training data set (uppermost plot) and plot of actual output and predicted output and the corresponding prediction error for hourly electric load data.
(a)(b)(c)Figure 7
Lattice ofQ-measure values of the constituent singleton sets with D=4.The performance measures of the applications A and B are given in Table19 for comparison with some of the existing well-known methods of fuzzy modeling. This table shows that E of the proposed nonadditive fuzzy model is better than those of the additive fuzzy systems of Gan et al. [7] and WM method [25].Table 19
Comparison ofE.
DataWM [25] methodGan et al. [7]Non-additive (usingλ-measure)Proposed algorithm (using fuzzy measure estimation)Box & Jenkins’s Gas Furnace data0.00570.00320.00210.00208Industrial Dryer’s dataNA0.00270.00190.00189


## 5.1. Gas Furnace

Here, we take up the gas furnace data [55] to demonstrate the effectiveness of the proposed algorithm. In this gas furnace, the air at a feed rate of u(t) is combined with methane to produce a mixture of gases with carbon dioxide of concentration y(t). These are used to fit the Gaussian mixture density function with four components. The input vector after sorting is x=[x1x2x3x4]=[y(t-1)-y(t-2)y(t-1)u(t)-u(t-1)u(t)] and the output is y=y(t). The composition of the input vector is arrived at using the fuzzy curve approach [56]. On applying the expectation maximization (EM) [57] clustering, the number of clusters is found to be 4; hence the model will have four components. The five-variable GMM using the EM algorithm yields the parameters of GMM as shown in Table 1. These parameters are also the premise model parameters of the non-additive GFM fuzzy rules.Table 1
Estimated parameters of A 4-component GMM using the EM algorithm.
Cluster or rule no.(m)Weight of the rulep(cm)Mean(z̅m)Covariance(∑m)Cluster or rule no. 1p(c1)=0.2991z̅1=[0.40660.28970.43410.53210.2986]∑1=[0.02460.00320.0114-0.00750.00950.00320.0150-0.0027-0.01060.01490.0114-0.00750.0095-0.0027-0.01060.01490.01610.01040.00040.01040.0427-0.01460.0004-0.01460.0170]Cluster or rule no. 2p(c2)=0.1685z̅2=[0.43720.67820.46760.22320.6904]∑2=[0.05030.00560.0151-0.01240.01700.00560.02090.00620.00850.01800.01510.00620.0090-0.00190.0096-0.01240.0085-0.00190.01500.00250.01700.01800.00960.00250.0194]Cluster or rule no. 3p(c3)=0.2166z̅3=[0.55580.71000.59960.41780.7355]∑3=[0.0198-0.00900.0049-0.0066-0.0039-0.00900.0234-0.0018-0.00340.01990.0049-0.00180.0098-0.00290.0004-0.0066-0.0034-0.00290.0140-0.0054-0.00390.01990.0004-0.00540.0181]Cluster or rule no. 4p(c4)=0.3158z̅4=[0.32110.53660.52650.59630.5065]∑4=[0.0135-0.00570.00130.0016-0.0033-0.00570.02700.0069-0.01530.02560.00130.00690.01830.00640.00730.0016-0.01530.00640.0208-0.0152-0.00330.02560.0073-0.01520.0248]Fuzzy densities and estimated fuzzy measures of the model are given in Tables2 and 4, respectively. The estimated values of λm are given in Table 3. The estimated β-coefficient and Q-measures are given in Tables 5 and 6 respectively. Note that ζ(X1) contains all input elements, hence its value is one. The β-coefficients constitute the consequent parameters of the GMM fuzzy rules. The performance measure is found to be 0.00208. The fuzzy measures of the constituent singleton sets satisfy the monotonic relations for D=4 shown as a lattice in Figure 2.Table 2
Fuzzy densities using GMM.
Rule(m)g1g2g3g4Rule no. 10.24730.77600.00050.0000Rule no. 20.56990.82040.00280.0003Rule no. 30.62840.86710.00700.0001Rule no. 40.48440.73300.00420.0012Table 3
Estimated lambda (λm).
Rule 1Rule 2Rule 3Rule 4−0.1241−0.8364−0.9108−0.6197Table 4
Estimated fuzzy measures.
Rule(m)ζ(X1)ζ(X2)ζ(X3)ζ(X4)Rule no. 11.00000.79230.00210.0009Rule no. 20.99970.82140.00320.0003Rule no. 30.99990.86860.00710.0001Rule no. 40.99940.73590.00530.0012Table 5
Estimatedβ coefficients.
Rule(m)β1β2β3β4Rule no. 10.20770.79020.00120.0009Rule no. 20.17830.81820.00290.0003Rule no. 30.13130.86150.00700.0001Rule no. 40.26350.73060.00410.0012Table 6
EstimatedQ-measures.
Rule(m)Q(X1)Q(X2)Q(X3)Q(X4)Rule no. 11.00000.78230.00130.0000Rule no. 21.00000.85410.06350.0001Rule no. 31.00000.91750.08630.0003Rule no. 41.00000.72300.01690.0002For the training dataset containing the first 250 data vectors,E=6.12505×10-4. For the remaining data vectors forming the validation data set, E=0.0021. This is inferior to that of the training data set, as expected. The plots of actual output and model output for both training and prediction along with the prediction error of the model output are shown in Figure 1. The model output is very close to the actual output during training, but it displays small deviations from the actual output during the prediction period. Using the premise and consequent parameters of Table 1, we have constructed the membership functions for the four inputs and one output in Figure 3 where we depict the membership function of rules-1, 2, 3, and 4 using dashed lines, dotted lines, dots with dashes lines and solid lines, respectively.Plot of actual output and model output for training data set (uppermost plot) and plot of actual output and predicted output and the corresponding prediction error for Box and Jenkins’s gas furnace data.
(a)(b)(c)Figure 2
Lattice of fuzzy measure values of the constituent singleton sets [39] of rule 1 for Box and Jenkins’s data.(a) Membership function for input one. (b) Membership functions for input two. (c) Membership function for input three. (d) Membership function for input four. (e) Membership function for output.
(a)(b)(c)(d)(e)The first ruleR1 consists of,(i)
the weight of the rule; 0.2991,(ii)
the membership functions of the premise parts(53)N1(x1;104066,0.0246),N1(x2;0.2897,0.0150),N1(x3;0.4341,0.0161),N1(x4;0.5321,0.0427).(iii)
The consequent part of a rule can be found using Table6.Similarly other rules can also be formed.


## 5.2. Industrial Dryer

In this, we consider the industrial dryer data obtained from theDaISy database. Fuel is combusted to produce the hot gas, which is blown by an exhaust fan on the raw material fed into the dryer. It is desired to determine the relationship between the functional elements of the dryer with the moisture content of the raw material at the dryer output. The measurements are designated by(i)
fuel flow rate,x1(t),(ii)
hot gas exhaust-fan speed,x2(t),(iii)
rate of flow of raw material,x3(t),(iv)
moisture content of the raw material after drying which isy(t).Sampling period of 10 seconds is considered to generate a total of 867 samples. Input and output data are normalized using (40) to maintain compatibility in the range 0 to 1. The input variables x1(t), x2(t), x3(t), and y(t-1) affect the current output y(t). The input-output vectors are represented by z(t)=[x1(t)x2(t)x3(t)y(t-1)y(t)]. Out of these vectors 70% is taken as the training set. The remaining 30% is kept for prediction and evaluation of our proposed model.The number of clusters is found to be 4 using EM clustering, which yields the parameters of GMM in Table7, and these are also the premise model parameters of the non-additive GFM.Table 7
Estimated parameters of A 4-component GMM using the EM algorithm.
Cluster or rule no.(m)Weight of the rulep(cm)Mean(z̅m)Covariance(∑m)Cluster or rule no. 1p(c1)=0.3564z̅1=[0.3946-0.6119-0.0988-0.3346-0.3402]∑1=[0.4246-0.02430.1174-0.00490.0066-0.02430.1174-0.00490.00660.0983-0.0372-0.0558-0.0618-0.03720.61880.02290.0355-0.05580.02290.39520.3608-0.06180.03550.36080.3933]Cluster or rule no. 2p(c2)=0.2723z̅2=[-0.19520.1800-0.18971.55851.5501]∑2=[0.5879-0.2083-0.0120-0.0916-0.1151-0.20830.7343-0.16700.0011-0.0073-0.0120-0.16700.87010.07020.1139-0.09160.00110.07020.28020.2227-0.1151-0.00730.11390.22270.2849]Cluster or rule no. 3p(c3)=0.2046z̅3=[-1.18111.4027-0.4488-0.1763-0.1760]∑3=[0.70080.1105-0.24000.07110.05820.11050.2471-0.0056-0.0244-0.0416-0.2400-0.00560.4133-0.1059-0.09530.0711-0.0244-0.10590.46610.42260.0582-0.0416-0.09530.42260.4704]Cluster or rule no. 4p(c4)=0.1667z̅4=[-0.30411.04430.9807-0.00320.0179]∑4=[0.42830.0685-0.2311-0.0418-0.09530.06850.6950-0.41110.0238-0.0029-0.2311-0.41110.76990.12820.1662-0.04180.02380.12820.32100.2911-0.0953-0.00290.16620.29110.3528]The estimated fuzzy densities, fuzzy measures,β-coefficients and Q-measure of the model are given in Tables 8, 10, 11, and 12, respectively. The monotonicity relations for D=4 are also verified here for fuzzy measure values of the constituent singleton sets, shown as a lattice in Figure 5. Table 9 shows the estimated values of λm.Table 8
Fuzzy densities using GMM.
Rule(m)g1g2g3g4Rule no. 10.49820.42950.38810.7989Rule no. 20.49960.73680.61580.9027Rule no. 30.46960.75430.37040.5916Rule no. 40.48440.73300.00420.0012Table 9
Estimated lambda (λm).
Rule 1Rule 2Rule 3Rule 4−0.9970−1.0000−1.0000−0.9947Table 10
Estimatedλ-measures.
Rule(m)ζ(X1)ζ(X2)ζ(X3)ζ(X4)Rule no. 10.96690.93150.87790.7989Rule no. 20.90270.88280.87580.8749Rule no. 30.95000.65290.62440.5918Rule no. 41.00001.00001.00000.9981Table 11
Estimatedβ coefficients.
Rule(m)β1β2β3β4Rule no. 10.03540.05360.07900.7989Rule no. 20.01990.00700.00090.8749Rule no. 30.29710.02850.03260.5918Rule no. 40.00000.00000.00190.9981Table 12
EstimatedQ-measures.
Rule(m)Q(X1)Q(X2)Q(X3)Q(X4)Rule no. 11.00000.78230.00130.0000Rule no. 21.00000.85410.06350.0001Rule no. 31.00000.91750.08630.0003Rule no. 41.00000.72300.01690.0002The training data set yieldsE=9.0005×10-4 for training the model. The remaining 30% data vectors constitute the validation data set. The performance measure E over the prediction period is obtained as 0.0019. The results of the actual output and the model output during the training and the prediction periods are shown in Figure 4 which also includes the prediction error. The results demonstrate the effectiveness of the proposed non-additive fuzzy model using Q-measure. The membership functions associated with all the four rules can be obtained in the similar fashion as in Figure 3.Plot of actual output and model output for training data set (uppermost plot) and plot of actual output and predicted output and the corresponding prediction error for industrial dryer data.
(a)(b)(c)Figure 5
Lattice of fuzzy measure values of the constituent singleton sets of rule no.1 of Table10 for industrial dryer’s data.


## 5.3. Electric Load Demand

In this, we consider the real time hourly electric load data, that is, hourly peak electric load consumption with uncertain weather information like hourly average temperature, humidity, and wind speed for the month of April 2003. The variables are designated as(i)
hourly average Temperature,x1(t),(ii)
hourly average Humidity,x2(t),(iii)
hourly average Wind speed,x3(t),(iv)
hourly peak electric load consumption which isy(t).A number of 720 samples of Input and output data are normalized using (40) to maintain the compatibility in the range 0 to 1. We consider 75% of the total input-output vectors of the form z(t)=[10·(y(t)-y(t-2))10·(x3(t)-x2(t-2))10·(x2(t)-x2(t-2))50·(x3(t-2)-x3(t))100·y(t)] as the training set. The remaining 25% is reserved for the prediction. The weights before the components in z(t) have been fixed for achieving better prediction capability. It may be noted that the criteria only facilitate the selection of proper inputs by trying out different combinations. But the final curtain falls on the selection with the verification through the performance measure.The number of clusters in this example also is found to be 4 using the EM clustering after discarding a few datasets as outliers. Application of the EM algorithm yields the parameters of GMM in Table13. These constitute the premise parameters of the GFM. The estimated fuzzy densities, lambda values, fuzzy measure, Q-measure and β-coefficients of the model are given in Tables 14, 15, 16, 17, and 18, respectively. The monotonicity relations can also be verified as in the previous applicationsTable 13
Estimated parameters of A 4-component GMM using the EM algorithm (load).
Cluster or rule no.(m)Weight of the rulep(cm)Mean(z̅m)Covariance(∑m)Cluster or rule no. 1p(c1)=0.2646z̅1=[0.47200.45000.46940.38200.5007]∑1=[0.0031-0.0000-0.00107-0.00160.0026-0.00000.03060.0160-0.00510.0010-0.00170.01600.02460.0050-0.0005-0.0016-0.00510.00500.0227-0.00150.00260.0010-0.0005-0.00150.0029]Cluster or rule no. 2p(c2)=0.2225z̅2=[0.52260.74300.55640.45980.5269]∑2=[0.0188-0.00070.0031-0.00040.0122-0.00070.00810.0031-0.0044-0.00010.00310.00310.00520.00340.0024-0.0004-0.00440.00340.03080.00090.01220.0001-0.00240.00090.0084]Cluster or rule no. 3p(c3)=0.2474z̅3=[0.29020.30000.68440.43930.3771]∑3=[0.0228-0.0014-0.0035-0.00120.0137-0.00140.01540.00180.0003-0.0044-0.00350.00180.00650.0018-0.0025-0.00120.00030.00180.0054-0.00200.0137-0.0044-0.0025-0.00200.0142]Cluster or rule no. 4p(c4)=0.2655z̅4=[0.64740.57380.73720.49400.6013]∑4=[0.01020.00110.0009-0.00230.00680.00110.0093-0.00040.00300.00370.0009-0.00040.0108-0.00330.0002-0.00230.0030-0.00330.01890.00240.00680.00370.00020.00240.0073]Table 14
Fuzzy densities using GMM.
Rule(m)g1g2g3g4Rule no. 10.84790.64880.58170.6735Rule no. 20.01960.06330.00010.2634Rule no. 30.02640.01000.07250.0255Rule no. 40.00010.04690.00010.1708Table 15
Estimated lambda (λm).
Rule 1Rule 2Rule 3Rule 429.3843−0.0776−0.99950.2543Table 16
Estimatedλ-measures.
Rule(m)ζ(X1)ζ(X2)ζ(X3)ζ(X4)Rule no. 11.00000.08770.04390.6735Rule no. 21.00000.12000.05690.2634Rule no. 31.00000.09100.08180.0255Rule no. 41.00000.47180.19740.1708Table 17
EstimatedQ-measures.
Rule(m)Q(X1)Q(X2)Q(X3)Q(X4)Rule no. 11.00000.02910.01450.0033Rule no. 21.00000.15690.07440.0614Rule no. 31.00000.14680.13200.0161Rule no. 41.00000.38910.16280.1409Table 18
Estimatedβ coefficients.
Rule(m)β1β2β3β4Rule no. 10.97090.01460.01120.0033Rule no. 20.84310.08250.01300.0614Rule no. 30.85320.01480.11590.0161Rule no. 40.61090.22630.02190.1409The training data set is found to giveE=0.0034. The remaining 25% data vectors constitute the validation data set. The performance measure E over this set is obtained as 0.00256. The results of the actual output and the model output during the training and the prediction periods are shown in Figure 6, which also includes the prediction error. The results demonstrate the effectiveness of the proposed non-additive fuzzy model using Q-measure for this application too (Figure 7).Plot of actual output and model output for training data set (uppermost plot) and plot of actual output and predicted output and the corresponding prediction error for hourly electric load data.
(a)(b)(c)Figure 7
Lattice ofQ-measure values of the constituent singleton sets with D=4.The performance measures of the applications A and B are given in Table19 for comparison with some of the existing well-known methods of fuzzy modeling. This table shows that E of the proposed nonadditive fuzzy model is better than those of the additive fuzzy systems of Gan et al. [7] and WM method [25].Table 19
Comparison ofE.
DataWM [25] methodGan et al. [7]Non-additive (usingλ-measure)Proposed algorithm (using fuzzy measure estimation)Box & Jenkins’s Gas Furnace data0.00570.00320.00210.00208Industrial Dryer’s dataNA0.00270.00190.00189


## 6. Conclusions

This paper deals with an important issue of computing fuzzy measures directly from covariance matrices found in GMM. Use of EM algorithm in GMM provides us with the input clusters and their Gaussian memberships. The consequent parts of the fuzzy rules, that is, the output function, of GFM are altered for incorporating the non-additive property. It has to be noted that we have been still using the multiplicative T-norm in the premise part but the additive property of S-norm in the consequent part. For this the output function is defined as a linear function of fuzzy measures serving as coefficients. Computation complexity is reduced by replacing Sugeno’sλ-measure with the Q-measure in the non-additive fuzzy model.The defuzzified output of the non-additive GFM rules is then shown to be in the form of Choquet integral. By showing the Choquet fuzzy integral as the functionality of non-additive GFM, we have enhanced its capability to tackle a variety of real-life problems. We have also proved the corresponding non-additive functionality for GMM. This has been accomplished by showing thatf(x,βm) is the mean value of y. The computation of fuzzy measures required in the Choquet integral entails a lot of computational burden, as we need to solve a polynomial in λ. Using Q-measure this burden has been eliminated.The non-additive fuzzy model is applied on the three benchmark applications, namely, Box and Jenkins’s, Dryers’ data and on the real time electric load demand data. The results demonstrate the superiority of this model over the additive fuzzy model of Gan et al. [7] and Wang and Mendel method [25]. Thus this work makes an important contribution to the field of fuzzy modeling by providing a transition from additive fuzzy systems to non-additive fuzzy systems for dealing with the real-life applications. This is done without learning the fuzzy densities and eliminating the computational complexity associated with solving the polynomial function in λ. In the non-additive fuzzy systems, it makes a novel contribution and will pave the way for several other applications and for further research.



---

*Source: 402420-2012-06-03.xml*