# Fuzzy ARTMAP Ensemble Based Decision Making and Application

Authors: Min Jin; Zengbing Xu; Ren Li; Dan Wu
Journal: Mathematical Problems in Engineering
 (2013)

---

## Abstract

Because the performance of single FAM is affected by the sequence of sample presentation for the offline mode of training, a fuzzy ARTMAP (FAM) ensemble approach based on the improved Bayesian belief method is supposed to improve the classification accuracy. The training samples are input into a committee of FAMs in different sequence, the output from these FAMs is combined, and the final decision is derived by the improved Bayesian belief method. The experiment results show that the proposed FAMs’ ensemble can classify the different category reliably and has a better classification performance compared with single FAM.


---

## Body

## 1. Introduction

Recently, artificial neural networks (ANNs) have been widely used as an intelligent classifier to identify the different categories based on learning pattern from empirical data modeling in complex systems [1]. For example, the BP, RBF, and SVM models have been developed quickly and utilized to classify the different fault classes of the machine equipment [2–6]. However, these traditional neural network methods have limitation on generalization, which can give rise to overfitting models for training samples. To solve the problem, the fuzzy ARTMAP (FAM) neural network is created and applied to the classification field [7–9], which is an incremental and supervised network model and designed in accordance with adaptive resonance theory. Although the FAM is able to overcome the stability-plasticity dilemma [10], in real-world application, the performance of FAM is affected by the sequence of sample presentation for the offline mode of training [11, 12].For this drawback, some preprocessing procedures, known as the ordering algorithms such as min-max clustering and genetic algorithm [13, 14], have been proposed for FAM. Furthermore, a number of fusion techniques have been proposed for FAM to overcome this problem. Tang and Yan employed the voting algorithm of FAM to diagnose the bearings faults [15], Loo and Rao applied the multiple FAM based on the probabilistic plurality voting strategy to medical diagnosis and classification problems [16]. Since these voting algorithms do not consider the effect of the number of the sample in each class, an improved Bayesian belief method (BBM) is used to combine multiple FAM classifiers which are offline trained in different sequence of samples in this paper.In view of the above principles, a novel ensemble FAM classifiers is proposed to improve the classification performance of single FAM. The identification schematic graph is shown in Figure1. Firstly, through different features extraction methods, some feature parameters are extracted from the raw signals. Secondly, by the modified distance discrimination technique, the optimal feature set is selected from the original feature set. Finally, multiple FAM classifiers ensemble based on the improved BBM is employed to come up with the final classification results. The proposed method is applied to the fault diagnosis of hydraulic pump. The experiment results show the effectiveness of the proposed ensemble FAM classifiers.Figure 1
Flowchart of classification system.


## 2. Fuzzy ARTMAP Ensemble Using the Improved Bayesian Belief Method

### 2.1. Fuzzy ARTMAP (FAM)

FAM consists of two ART modules, namely,ARTa and ARTb modules which are bridged via a map field [10], which is capable to forming associative maps between clusters of input domain in which ARTa module functions as clustering and output domain in which the module ARTb functions as clustering. Each module comprises three layers: normalization layer F0, input layer F1, and recognition layer F2. The structure of FAM is shown in Figure 2. When the output domain is a finite set of class labels, FAM can be utilized as a classifier. The algorithm of FAM can be depicted simply as follows.Figure 2
The basic structure of fuzzy ARTMAP.TheARTa module receives the input pattern, and the normalization of a M-dimensional input vector a, is complement-coded to a 2M-dimensional vector A(1)A=(a,ac)=(a1,…,aM,1-a1,…,1-aM).
Then, the dimension of the input vector is kept constant:
(2)|A|=|(a,ac)|=∑i=1Mai+(M-∑i=1Mai)=M.
Afterward, the input sample A selects the category node stored in the network by the category choice function (CCF):
(3)|A∧wja||wja|+αa=Tj(A),
where ∧ is a min operator, αa is the choice function of ARTa, and wja is the weight vector of the jth category node.When a winning category node is selected, a vigilance test (VT), namely, a similarity check against a vigilance parameterρa of the chosen category node, is taken place:
(4)|A∧wja||A|≥ρa,
where wja is the winning jth node. When the above category match function (CMF) is satisfied with criterion, the resonance occurs and learning takes place; namely, the weight vector wj is updated according to the following equation:(5)wjnew=β(A∧wjold)+(1-β)wjold,
where β∈[0,1] is the learning rate. Otherwise, a new node is created in F2a which codes the input pattern. In the meantime, for the ARTb the same learning algorithm occurs simultaneously using the target pattern.After the resonance occurs in theARTa and ARTb, the winning node in F2a will send a prediction to ARTb via the map field. The map field vigilance test is used to detect the test. If the test fails, it indicates that the winning node of ARTa predicts an incorrect target class in ARTb; then a match tracking process initiates. During the match tracking, the value of ρa is increased until it is slightly higher than |A∧wja||A|-1; then a new search for the other winning node in ARTa is carried out, and the process continues until the selected F2a node can make a correct prediction in ARTb.


### 2.2. Decision Fusion Using Bayesian Belief Method

The novel Bayesian belief method is supposed in [17]. It is based on the assumption of mutual independency of classifiers and considers the error of each classifier. Assume that in pattern space Z there are M classes and K classifiers. A classifier ek can be considered as a function:
(6)ek(x)=j,k=1,2,…,K,j∈{1,2,…,M,M+1}.
It signifies that the sample x is assigned to class j by the classifier ek. And its two-dimensional confusion matrix can be represented as
(7)CMk=[n11kn12k⋯n1Mkn1(M+1)kn21kn22k⋯n2Mkn2(M+1)k⋯⋯⋯⋯⋯nM1knM2knMMknM(M+1)k],(k=1,…,K),M+1isanunknownlabel
which is obtained by executing ek(x) on the test data set after ek(x) is trained. Each row i corresponds to class ci and each column j corresponds to ek(x)=j. The matrix unit nijk means the input samples from class ci while are assigned to class cj by classifier ek(x). The number of samples in class ci is ni.k=∑j=1M+1nijk, where i=1,…,M, and the number of samples labeled j by ek(x) is n.jk=∑i=1Mnijk, where j=1,…,M+1. Considering the difference of the number of samples in each class, on the basis of the confusion matrix a belief measure of classification can be calculated for each classifier by the following belief function [18]:(8)Bk(x∈ciek(x))=P(x∈ciek(x)=j)=nijk/∑j=1M+1nijk∑i=1M(nijk/∑j=1M+1nijk),i=1,2,…,M,j=1,2,…,M+1.
When multiple classifiers e1,e2,…,ek are developed, their correspondent beliefs B1,B2,…,Bk are computed based on the performance of base classifiers. Combining the belief measures of all fusion classifiers can result in the final belief measure of the multiple classifier system. In case of equal a priori class probabilities, the combination rule can be depicted as follows:(9)B(i)=B(x∈ci∣e1(x),e2(x),…,ek(x),EN)=P(x∈ci∣e1(x),e2(x),…,ek(x),EN)=∏k=1KBk(x∈ci∣e1(x),EN)∑i∏k=1KBk(x∈ci∣e1(x),EN),i=1,2,…,M+1.
Thus, x is classified into a class j(j=1,2,…,M+1) according to the belief of making the final decision Bel(j)=maxi=1M+1B(i).


## 2.1. Fuzzy ARTMAP (FAM)

FAM consists of two ART modules, namely,ARTa and ARTb modules which are bridged via a map field [10], which is capable to forming associative maps between clusters of input domain in which ARTa module functions as clustering and output domain in which the module ARTb functions as clustering. Each module comprises three layers: normalization layer F0, input layer F1, and recognition layer F2. The structure of FAM is shown in Figure 2. When the output domain is a finite set of class labels, FAM can be utilized as a classifier. The algorithm of FAM can be depicted simply as follows.Figure 2
The basic structure of fuzzy ARTMAP.TheARTa module receives the input pattern, and the normalization of a M-dimensional input vector a, is complement-coded to a 2M-dimensional vector A(1)A=(a,ac)=(a1,…,aM,1-a1,…,1-aM).
Then, the dimension of the input vector is kept constant:
(2)|A|=|(a,ac)|=∑i=1Mai+(M-∑i=1Mai)=M.
Afterward, the input sample A selects the category node stored in the network by the category choice function (CCF):
(3)|A∧wja||wja|+αa=Tj(A),
where ∧ is a min operator, αa is the choice function of ARTa, and wja is the weight vector of the jth category node.When a winning category node is selected, a vigilance test (VT), namely, a similarity check against a vigilance parameterρa of the chosen category node, is taken place:
(4)|A∧wja||A|≥ρa,
where wja is the winning jth node. When the above category match function (CMF) is satisfied with criterion, the resonance occurs and learning takes place; namely, the weight vector wj is updated according to the following equation:(5)wjnew=β(A∧wjold)+(1-β)wjold,
where β∈[0,1] is the learning rate. Otherwise, a new node is created in F2a which codes the input pattern. In the meantime, for the ARTb the same learning algorithm occurs simultaneously using the target pattern.After the resonance occurs in theARTa and ARTb, the winning node in F2a will send a prediction to ARTb via the map field. The map field vigilance test is used to detect the test. If the test fails, it indicates that the winning node of ARTa predicts an incorrect target class in ARTb; then a match tracking process initiates. During the match tracking, the value of ρa is increased until it is slightly higher than |A∧wja||A|-1; then a new search for the other winning node in ARTa is carried out, and the process continues until the selected F2a node can make a correct prediction in ARTb.


## 2.2. Decision Fusion Using Bayesian Belief Method

The novel Bayesian belief method is supposed in [17]. It is based on the assumption of mutual independency of classifiers and considers the error of each classifier. Assume that in pattern space Z there are M classes and K classifiers. A classifier ek can be considered as a function:
(6)ek(x)=j,k=1,2,…,K,j∈{1,2,…,M,M+1}.
It signifies that the sample x is assigned to class j by the classifier ek. And its two-dimensional confusion matrix can be represented as
(7)CMk=[n11kn12k⋯n1Mkn1(M+1)kn21kn22k⋯n2Mkn2(M+1)k⋯⋯⋯⋯⋯nM1knM2knMMknM(M+1)k],(k=1,…,K),M+1isanunknownlabel
which is obtained by executing ek(x) on the test data set after ek(x) is trained. Each row i corresponds to class ci and each column j corresponds to ek(x)=j. The matrix unit nijk means the input samples from class ci while are assigned to class cj by classifier ek(x). The number of samples in class ci is ni.k=∑j=1M+1nijk, where i=1,…,M, and the number of samples labeled j by ek(x) is n.jk=∑i=1Mnijk, where j=1,…,M+1. Considering the difference of the number of samples in each class, on the basis of the confusion matrix a belief measure of classification can be calculated for each classifier by the following belief function [18]:(8)Bk(x∈ciek(x))=P(x∈ciek(x)=j)=nijk/∑j=1M+1nijk∑i=1M(nijk/∑j=1M+1nijk),i=1,2,…,M,j=1,2,…,M+1.
When multiple classifiers e1,e2,…,ek are developed, their correspondent beliefs B1,B2,…,Bk are computed based on the performance of base classifiers. Combining the belief measures of all fusion classifiers can result in the final belief measure of the multiple classifier system. In case of equal a priori class probabilities, the combination rule can be depicted as follows:(9)B(i)=B(x∈ci∣e1(x),e2(x),…,ek(x),EN)=P(x∈ci∣e1(x),e2(x),…,ek(x),EN)=∏k=1KBk(x∈ci∣e1(x),EN)∑i∏k=1KBk(x∈ci∣e1(x),EN),i=1,2,…,M+1.
Thus, x is classified into a class j(j=1,2,…,M+1) according to the belief of making the final decision Bel(j)=maxi=1M+1B(i).


## 3. Case Study

In order to evaluate the effectiveness of the supposed ensemble FAM, the fault identification of hydraulic pump is taken as example. Figure3 shows the schematic diagram of experiment rig. Four accelerometers are attached to the housing with magnetic bases and mounted at the positions P1, P2, P3, and P4. Pressure sensor is mounted at the position P5. Considering the sensitivity to the fault conditions of hydraulic pump, the vibration signal which is acquired by the accelerometer at the position P2 is utilized to identify the fault categories. And the vibration signals are acquired, respectively, under normal condition and the different fault conditions, such as inner plunger wear, inner race wear, ball wear, swashplate wear, portplate wear, and paraplungers wear.Figure 3
The schematic diagram of the experimental setup.


### 3.1. Data Preparation

The data set contains 490 samples. These data samples are divided into 245 training and 245 test samples. The detailed descriptions of three data sets are shown in Table1. In order to identify the different fault categories, a seven-class classification problem need be solved.Table 1
Sample subset statistics.
The number of training samples
The number of test samples
Operation condition
Label of classification
45
45
Normal
1
30
30
Inner plunger wear
2
25
25
Inner race wear
3
50
50
Ball wear
4
20
20
Swashplate wear
5
40
40
Portplate wear
6
35
35
Paraplungers wear
7


### 3.2. Feature Extraction and Selection

#### 3.2.1. Feature Extraction

Feature parameters are used to characterize the information relevant to the conditions of the hydraulic pump. To acquire more fault-related information, many features in different symptom domains are extracted from the measured signals.Frequency domain is another description of a signal. In [19], some novel features which can give a much fuller picture of the frequency distribution in each band of frequencies are proposed. Supposed N points of normalized PSD, Pxx, of the vibration signal, Pxx are divided into L segments, where L is 1 in this study. The four features based on the moment estimates of power can be obtained as follows:
(3.2.1)k1=1N′∑N′Pxx(n),k2=1N′-1∑N′(Pxx(n)-k1)2,k3=1N′k23/2∑N′(Pxx(n)-k1)3,k4=1N′k22∑N′(Pxx(n)-k1)4,
where “n” is the number of total data points and N′ is the number of sample points in the lth segment.In order to characterize the spectrum with a higher accuracy, the moment estimates of frequency weighed by power are calculated by the following formulas:(11)k5=1Kl∑N′f(n)Pxx(n),k6=∑N′[(f(n)-k5)2Pxx(n)]Kl,k7=1Klk63∑N′[(f(n)-k5)3Pxx(n)],k8=1Klk64∑N′[(f(n)-k5)4Pxx(n)],
where f(n) is the corresponding frequency of Pxx(n) and Kl is the total power in the segment. Then, the total number of features extracted for each spectrum is 1 × 8.To depict the fault-related information about the hydraulic pumps quantitatively, the first-order continuous wavelet grey moment (WGM) [20] of vibration signal is extracted. Assuming the wavelet coefficients matrix [W]M×N which can be displayed by the continuous wavelet transform (CWT) scalogram, M and N are the scales and the time of the scalogram, respectively, the matrix [W]M×N is divided into m parts along the scale equally, and the first-order wavelet grey moment g1 of each part can be calculated by the following equation:
(12)g1=1(M/m)×N∑i=1M/m∑jNwij1(i-1)2+(j-1)2,
where wij is the element of matrix [W](M/m)×N. In this paper, the m is set as 8 and the wavelet function is Morlet wavelet.In addition, due to sensitiveness of these model parameters to the shape of the vibration data, AR model parameters are utilized to characterize the information about the conditions of hydraulic pumps. The AR model is written as follows:(13)xt=ϕ1xt-1+ϕ2xt-2+⋯+ϕpxt-p,
where xt-1,xt-2,…,xt-p are the r previous samples, xt is the predicted sample of the signal, and ϕ1,ϕ2,…,ϕt-p is AR model parameters, which can be obtained by the least square method in [21] and expressed by the following formula:
(14)ϕ^=(XTX)-1XTY,
where
(15)X=[xnxn-1⋯x1xn+1xn⋯x2⋮xN-1xN-2⋯xN-n],Y=[xn+1,xn+2,…,xN]T.
In this study, the parameter p is set as 8.Thus, 24 features constitute the original feature set.


#### 3.2.2. Feature Selection

In order to improve the identification accuracy and reduce the computation burden, some sensitive features providing characteristic information for the classification system need to be selected, and irrelevant or redundant features must be removed. In this study, based on [22], a modified distance discriminant technique is employed to select the optimal features.Supposing that a feature set ofJ classes consists of N samples, in the jth class there are Nj samples, where j=1,2,…,J, and N=∑j=1JNj. Each sample is represented by M features, and the mth feature of the ith sample is written as fim. Then, the feature selection process can be described as follows.Step 1.
Calculate the standard deviation and the mean of all samples in themth feature:
(16)σm2=1N∑i=1N(fim-f-m)2,f-m=1N∑i=1Nfim.Step 2.
Calculate the standard deviation and the mean of the sample in thejth class in the mth feature, respectively,
(17)σm′2(j)=1Nj-1∑j=1Nj(fjm-f-jm)2,f-jm=1Nj∑j=1Njfjm.Step 3.
Calculate the weighted standard deviation of the class centergj in the mth feature:
(18)σm′′2=∑j=1Jρj(gjm-gm)2=μ1-μ22,
where μ1=∑j=1Jρj(gjm)2, μ2=∑j=1Jρjgjm, gm=∑j=1Jρjgjm, and gm are the centers of all samples in the mth feature; gjm is the center of the samples of the jth class in the mth feature; μ1, μ2 are the weighted means of the squared class center gj2 and the class center gj in the mth feature; ρj is the prior probability of the jth class, respectively; and ∑j=1Jρj=1.Step 4.
Calculate the distance discriminant factor of themth feature:
(19)dbm-βdwm=1σm2[σm′′2-β∑j=1Jρjσm′2(j)],
where dbm is the distance of the mth feature between different classes, dwm corresponds to the distance of the mth feature within classes, and β is used to control the impact of dwm, which is set as 2 in this paper.
Considering the overlapping degree among different classes, a compensation factor is calculated as follows.
Firstly, define and calculate the variance factor ofdwm in the mth feature as follows:
(20)vwm=max(dwm)min(dwm).
Secondly, define and calculate the variance factor ofdbm in the mth feature as follows:
(21)ubm=max||f-im-f-jm||min||f-im-f-jm||,||f-im-f-jm||=(f-im-f-jm)σm2,i,j=1,2,…,J,i≠j.
Then, the compensation factor of themth feature can be defined and calculated as follows:
(22)ηm=1vwm+1ubm.
Thus, the modified distance discriminant factor can be calculated as follows:(23)dbm-β′dwm=dbm-(βηm)dwm=1σm2[σm′′2-β∑j=1J(1vjw+1ujb)ρjσm′2(j)],(m=1,2,…,M).Step 5.
RankM features in descending order according to the modified distance discriminant factors (1/σm2)[σm′′2-β((1/vjw)+(1/ujb))∑j=1Jρjσm′2(j)]=λm; then normaliz λm by λ-m=(λm-min(λm))/(max(λm)-min(λm)) and get the distance discriminant criteria. Clearly, bigger λ-m(m=1,2,…,M) signifies that the correspondent feature is better to separate J classes.Step 6.
Set a threshold valueγ and select the sensitive features whose distance discriminant factor λ-m≥γ from the set of M features.


### 3.3. Diagnosis Analysis

It is well known that the data-ordering of training samples can affect the classification accuracy of single FAM, and that a single output used to represent multiple classes may lead to lower classification accuracy. In order to know how well the proposed FAMs’ ensemble work, that is, how significant the generalization ability is improved by utilizing the improved Bayesian belief method to combine the classification results from a committee of single FAM trained with different data-ordering of training samples, the performance of single FAM is also conducted.In the diagnosis phase performed by the single FAM and FAM ensemble, they are all trained in the fast learning and conservative mode (i.e., settingβ=1 in (5) and αa=0.001 in (3)). Besides, in order to ensure the performance of stability-plasticity, the vigilance parameter of FAM is set as ρa=0.5, and the ensemble size is set as 5.In order to improve the classification accuracy and reduce the computation time, in each case some salient features are selected from each feature set by the modified distance discriminant technique, respectively, and then input into the five single FAM in different sequence in the process of training. Figure4 shows the modified distance discriminant factor λ-m of all features in the feature sets. From the figure it can be seen that the threshold γ corresponding to the optimal features are different for the case. That is to say, the number of salient features is different.Figure 4
Feature selection for three different data sets.Figure5 summarizes the classification results in terms of test accuracy of single FAM and FAMs’ ensemble. From the figure, it can be seen that the FAMs’ ensemble (0.988) outperforms the single FAMs’ in terms of accuracy. And the test accuracy is getting higher when the number of single FAM increases. These indicate that FAMs’ ensemble can identify the different fault categories of hydraulic pump well.Figure 5
Relationship between the test accuracy and the number of single FAM used in FAMs’ ensemble.


#### 3.3.1. Effect of Different Threshold for Feature Selection

As shown in Figure4, when the threshold value γ is set properly, some redundant and irrelevant features can be removed from the original feature set. To test the effect of the proposed feature selection method based on the modified distance discriminant technique, a series of experiments is carried out against the threshold value γ, in which the parameter of the single FAM is the same as the above, and the size of ensemble FAM is set as 5.Figure6 lists the classification accuracy of five individual FAMs’ and FAM ensemble against the different thresholds. From the figure, it can be noticed that when γ=0 (original feature set), the test accuracy of single FAM and FAM ensemble is 0.824 and 0.845, respectively. The highest test accuracy of single FAM and FAMs’ ensemble (0.915 and 0.988) is arrived synchronously when γ=0.8, where the optimal feature set is selected. However, when the threshold value continues to increase, the test accuracy of single FAM and FAMs’ ensemble tends to decrease. And when threshold γ>0.9, the test accuracy of single FAM and FAMs’ ensemble is lower than that used in all features with threshold γ=0. This is mainly because the smaller number of features leads to the overfitting; namely, the drastic reduction of features can lead to a decrease in the test accuracy.Figure 6
Classification accuracy comparison for different threshold.


#### 3.3.2. Classification Performance Comparison with Other Classification Methods

In order to test the superiority of the proposed FAMs’ ensemble method, the test results produced by FAMs’ ensemble and single FAM are compared with those produced by other classification methods. In this experiment, the parameters of FAM ensemble and single FAM are the same as the above.Table2 shows the test results of the FAMs’ ensemble versus other classification methods. From the table it can be seen that the average test accuracy using single FAMs’ is the lowest. However, the test accuracy produced by two FAMs’ ensemble methods is higher than that produced by the single classifier, and the test success rate of the proposed FAMs’ ensemble is highest and higher than that of FAM ensemble with voting algorithm. These indicate that the proposed FAMs’ ensemble has comparatively superior diagnosis performance.Table 2
Test accuracy produced by different classification methods.
Method
Test accuracy (%)
Single FAM
91.5
The proposed FAM ensemble
98.8
FAMs' ensemble with voting algorithm and five FAMs
95.2


## 3.1. Data Preparation

The data set contains 490 samples. These data samples are divided into 245 training and 245 test samples. The detailed descriptions of three data sets are shown in Table1. In order to identify the different fault categories, a seven-class classification problem need be solved.Table 1
Sample subset statistics.
The number of training samples
The number of test samples
Operation condition
Label of classification
45
45
Normal
1
30
30
Inner plunger wear
2
25
25
Inner race wear
3
50
50
Ball wear
4
20
20
Swashplate wear
5
40
40
Portplate wear
6
35
35
Paraplungers wear
7


## 3.2. Feature Extraction and Selection

### 3.2.1. Feature Extraction

Feature parameters are used to characterize the information relevant to the conditions of the hydraulic pump. To acquire more fault-related information, many features in different symptom domains are extracted from the measured signals.Frequency domain is another description of a signal. In [19], some novel features which can give a much fuller picture of the frequency distribution in each band of frequencies are proposed. Supposed N points of normalized PSD, Pxx, of the vibration signal, Pxx are divided into L segments, where L is 1 in this study. The four features based on the moment estimates of power can be obtained as follows:
(3.2.1)k1=1N′∑N′Pxx(n),k2=1N′-1∑N′(Pxx(n)-k1)2,k3=1N′k23/2∑N′(Pxx(n)-k1)3,k4=1N′k22∑N′(Pxx(n)-k1)4,
where “n” is the number of total data points and N′ is the number of sample points in the lth segment.In order to characterize the spectrum with a higher accuracy, the moment estimates of frequency weighed by power are calculated by the following formulas:(11)k5=1Kl∑N′f(n)Pxx(n),k6=∑N′[(f(n)-k5)2Pxx(n)]Kl,k7=1Klk63∑N′[(f(n)-k5)3Pxx(n)],k8=1Klk64∑N′[(f(n)-k5)4Pxx(n)],
where f(n) is the corresponding frequency of Pxx(n) and Kl is the total power in the segment. Then, the total number of features extracted for each spectrum is 1 × 8.To depict the fault-related information about the hydraulic pumps quantitatively, the first-order continuous wavelet grey moment (WGM) [20] of vibration signal is extracted. Assuming the wavelet coefficients matrix [W]M×N which can be displayed by the continuous wavelet transform (CWT) scalogram, M and N are the scales and the time of the scalogram, respectively, the matrix [W]M×N is divided into m parts along the scale equally, and the first-order wavelet grey moment g1 of each part can be calculated by the following equation:
(12)g1=1(M/m)×N∑i=1M/m∑jNwij1(i-1)2+(j-1)2,
where wij is the element of matrix [W](M/m)×N. In this paper, the m is set as 8 and the wavelet function is Morlet wavelet.In addition, due to sensitiveness of these model parameters to the shape of the vibration data, AR model parameters are utilized to characterize the information about the conditions of hydraulic pumps. The AR model is written as follows:(13)xt=ϕ1xt-1+ϕ2xt-2+⋯+ϕpxt-p,
where xt-1,xt-2,…,xt-p are the r previous samples, xt is the predicted sample of the signal, and ϕ1,ϕ2,…,ϕt-p is AR model parameters, which can be obtained by the least square method in [21] and expressed by the following formula:
(14)ϕ^=(XTX)-1XTY,
where
(15)X=[xnxn-1⋯x1xn+1xn⋯x2⋮xN-1xN-2⋯xN-n],Y=[xn+1,xn+2,…,xN]T.
In this study, the parameter p is set as 8.Thus, 24 features constitute the original feature set.


### 3.2.2. Feature Selection

In order to improve the identification accuracy and reduce the computation burden, some sensitive features providing characteristic information for the classification system need to be selected, and irrelevant or redundant features must be removed. In this study, based on [22], a modified distance discriminant technique is employed to select the optimal features.Supposing that a feature set ofJ classes consists of N samples, in the jth class there are Nj samples, where j=1,2,…,J, and N=∑j=1JNj. Each sample is represented by M features, and the mth feature of the ith sample is written as fim. Then, the feature selection process can be described as follows.Step 1.
Calculate the standard deviation and the mean of all samples in themth feature:
(16)σm2=1N∑i=1N(fim-f-m)2,f-m=1N∑i=1Nfim.Step 2.
Calculate the standard deviation and the mean of the sample in thejth class in the mth feature, respectively,
(17)σm′2(j)=1Nj-1∑j=1Nj(fjm-f-jm)2,f-jm=1Nj∑j=1Njfjm.Step 3.
Calculate the weighted standard deviation of the class centergj in the mth feature:
(18)σm′′2=∑j=1Jρj(gjm-gm)2=μ1-μ22,
where μ1=∑j=1Jρj(gjm)2, μ2=∑j=1Jρjgjm, gm=∑j=1Jρjgjm, and gm are the centers of all samples in the mth feature; gjm is the center of the samples of the jth class in the mth feature; μ1, μ2 are the weighted means of the squared class center gj2 and the class center gj in the mth feature; ρj is the prior probability of the jth class, respectively; and ∑j=1Jρj=1.Step 4.
Calculate the distance discriminant factor of themth feature:
(19)dbm-βdwm=1σm2[σm′′2-β∑j=1Jρjσm′2(j)],
where dbm is the distance of the mth feature between different classes, dwm corresponds to the distance of the mth feature within classes, and β is used to control the impact of dwm, which is set as 2 in this paper.
Considering the overlapping degree among different classes, a compensation factor is calculated as follows.
Firstly, define and calculate the variance factor ofdwm in the mth feature as follows:
(20)vwm=max(dwm)min(dwm).
Secondly, define and calculate the variance factor ofdbm in the mth feature as follows:
(21)ubm=max||f-im-f-jm||min||f-im-f-jm||,||f-im-f-jm||=(f-im-f-jm)σm2,i,j=1,2,…,J,i≠j.
Then, the compensation factor of themth feature can be defined and calculated as follows:
(22)ηm=1vwm+1ubm.
Thus, the modified distance discriminant factor can be calculated as follows:(23)dbm-β′dwm=dbm-(βηm)dwm=1σm2[σm′′2-β∑j=1J(1vjw+1ujb)ρjσm′2(j)],(m=1,2,…,M).Step 5.
RankM features in descending order according to the modified distance discriminant factors (1/σm2)[σm′′2-β((1/vjw)+(1/ujb))∑j=1Jρjσm′2(j)]=λm; then normaliz λm by λ-m=(λm-min(λm))/(max(λm)-min(λm)) and get the distance discriminant criteria. Clearly, bigger λ-m(m=1,2,…,M) signifies that the correspondent feature is better to separate J classes.Step 6.
Set a threshold valueγ and select the sensitive features whose distance discriminant factor λ-m≥γ from the set of M features.


## 3.2.1. Feature Extraction

Feature parameters are used to characterize the information relevant to the conditions of the hydraulic pump. To acquire more fault-related information, many features in different symptom domains are extracted from the measured signals.Frequency domain is another description of a signal. In [19], some novel features which can give a much fuller picture of the frequency distribution in each band of frequencies are proposed. Supposed N points of normalized PSD, Pxx, of the vibration signal, Pxx are divided into L segments, where L is 1 in this study. The four features based on the moment estimates of power can be obtained as follows:
(3.2.1)k1=1N′∑N′Pxx(n),k2=1N′-1∑N′(Pxx(n)-k1)2,k3=1N′k23/2∑N′(Pxx(n)-k1)3,k4=1N′k22∑N′(Pxx(n)-k1)4,
where “n” is the number of total data points and N′ is the number of sample points in the lth segment.In order to characterize the spectrum with a higher accuracy, the moment estimates of frequency weighed by power are calculated by the following formulas:(11)k5=1Kl∑N′f(n)Pxx(n),k6=∑N′[(f(n)-k5)2Pxx(n)]Kl,k7=1Klk63∑N′[(f(n)-k5)3Pxx(n)],k8=1Klk64∑N′[(f(n)-k5)4Pxx(n)],
where f(n) is the corresponding frequency of Pxx(n) and Kl is the total power in the segment. Then, the total number of features extracted for each spectrum is 1 × 8.To depict the fault-related information about the hydraulic pumps quantitatively, the first-order continuous wavelet grey moment (WGM) [20] of vibration signal is extracted. Assuming the wavelet coefficients matrix [W]M×N which can be displayed by the continuous wavelet transform (CWT) scalogram, M and N are the scales and the time of the scalogram, respectively, the matrix [W]M×N is divided into m parts along the scale equally, and the first-order wavelet grey moment g1 of each part can be calculated by the following equation:
(12)g1=1(M/m)×N∑i=1M/m∑jNwij1(i-1)2+(j-1)2,
where wij is the element of matrix [W](M/m)×N. In this paper, the m is set as 8 and the wavelet function is Morlet wavelet.In addition, due to sensitiveness of these model parameters to the shape of the vibration data, AR model parameters are utilized to characterize the information about the conditions of hydraulic pumps. The AR model is written as follows:(13)xt=ϕ1xt-1+ϕ2xt-2+⋯+ϕpxt-p,
where xt-1,xt-2,…,xt-p are the r previous samples, xt is the predicted sample of the signal, and ϕ1,ϕ2,…,ϕt-p is AR model parameters, which can be obtained by the least square method in [21] and expressed by the following formula:
(14)ϕ^=(XTX)-1XTY,
where
(15)X=[xnxn-1⋯x1xn+1xn⋯x2⋮xN-1xN-2⋯xN-n],Y=[xn+1,xn+2,…,xN]T.
In this study, the parameter p is set as 8.Thus, 24 features constitute the original feature set.


## 3.2.2. Feature Selection

In order to improve the identification accuracy and reduce the computation burden, some sensitive features providing characteristic information for the classification system need to be selected, and irrelevant or redundant features must be removed. In this study, based on [22], a modified distance discriminant technique is employed to select the optimal features.Supposing that a feature set ofJ classes consists of N samples, in the jth class there are Nj samples, where j=1,2,…,J, and N=∑j=1JNj. Each sample is represented by M features, and the mth feature of the ith sample is written as fim. Then, the feature selection process can be described as follows.Step 1.
Calculate the standard deviation and the mean of all samples in themth feature:
(16)σm2=1N∑i=1N(fim-f-m)2,f-m=1N∑i=1Nfim.Step 2.
Calculate the standard deviation and the mean of the sample in thejth class in the mth feature, respectively,
(17)σm′2(j)=1Nj-1∑j=1Nj(fjm-f-jm)2,f-jm=1Nj∑j=1Njfjm.Step 3.
Calculate the weighted standard deviation of the class centergj in the mth feature:
(18)σm′′2=∑j=1Jρj(gjm-gm)2=μ1-μ22,
where μ1=∑j=1Jρj(gjm)2, μ2=∑j=1Jρjgjm, gm=∑j=1Jρjgjm, and gm are the centers of all samples in the mth feature; gjm is the center of the samples of the jth class in the mth feature; μ1, μ2 are the weighted means of the squared class center gj2 and the class center gj in the mth feature; ρj is the prior probability of the jth class, respectively; and ∑j=1Jρj=1.Step 4.
Calculate the distance discriminant factor of themth feature:
(19)dbm-βdwm=1σm2[σm′′2-β∑j=1Jρjσm′2(j)],
where dbm is the distance of the mth feature between different classes, dwm corresponds to the distance of the mth feature within classes, and β is used to control the impact of dwm, which is set as 2 in this paper.
Considering the overlapping degree among different classes, a compensation factor is calculated as follows.
Firstly, define and calculate the variance factor ofdwm in the mth feature as follows:
(20)vwm=max(dwm)min(dwm).
Secondly, define and calculate the variance factor ofdbm in the mth feature as follows:
(21)ubm=max||f-im-f-jm||min||f-im-f-jm||,||f-im-f-jm||=(f-im-f-jm)σm2,i,j=1,2,…,J,i≠j.
Then, the compensation factor of themth feature can be defined and calculated as follows:
(22)ηm=1vwm+1ubm.
Thus, the modified distance discriminant factor can be calculated as follows:(23)dbm-β′dwm=dbm-(βηm)dwm=1σm2[σm′′2-β∑j=1J(1vjw+1ujb)ρjσm′2(j)],(m=1,2,…,M).Step 5.
RankM features in descending order according to the modified distance discriminant factors (1/σm2)[σm′′2-β((1/vjw)+(1/ujb))∑j=1Jρjσm′2(j)]=λm; then normaliz λm by λ-m=(λm-min(λm))/(max(λm)-min(λm)) and get the distance discriminant criteria. Clearly, bigger λ-m(m=1,2,…,M) signifies that the correspondent feature is better to separate J classes.Step 6.
Set a threshold valueγ and select the sensitive features whose distance discriminant factor λ-m≥γ from the set of M features.


## 3.3. Diagnosis Analysis

It is well known that the data-ordering of training samples can affect the classification accuracy of single FAM, and that a single output used to represent multiple classes may lead to lower classification accuracy. In order to know how well the proposed FAMs’ ensemble work, that is, how significant the generalization ability is improved by utilizing the improved Bayesian belief method to combine the classification results from a committee of single FAM trained with different data-ordering of training samples, the performance of single FAM is also conducted.In the diagnosis phase performed by the single FAM and FAM ensemble, they are all trained in the fast learning and conservative mode (i.e., settingβ=1 in (5) and αa=0.001 in (3)). Besides, in order to ensure the performance of stability-plasticity, the vigilance parameter of FAM is set as ρa=0.5, and the ensemble size is set as 5.In order to improve the classification accuracy and reduce the computation time, in each case some salient features are selected from each feature set by the modified distance discriminant technique, respectively, and then input into the five single FAM in different sequence in the process of training. Figure4 shows the modified distance discriminant factor λ-m of all features in the feature sets. From the figure it can be seen that the threshold γ corresponding to the optimal features are different for the case. That is to say, the number of salient features is different.Figure 4
Feature selection for three different data sets.Figure5 summarizes the classification results in terms of test accuracy of single FAM and FAMs’ ensemble. From the figure, it can be seen that the FAMs’ ensemble (0.988) outperforms the single FAMs’ in terms of accuracy. And the test accuracy is getting higher when the number of single FAM increases. These indicate that FAMs’ ensemble can identify the different fault categories of hydraulic pump well.Figure 5
Relationship between the test accuracy and the number of single FAM used in FAMs’ ensemble.


### 3.3.1. Effect of Different Threshold for Feature Selection

As shown in Figure4, when the threshold value γ is set properly, some redundant and irrelevant features can be removed from the original feature set. To test the effect of the proposed feature selection method based on the modified distance discriminant technique, a series of experiments is carried out against the threshold value γ, in which the parameter of the single FAM is the same as the above, and the size of ensemble FAM is set as 5.Figure6 lists the classification accuracy of five individual FAMs’ and FAM ensemble against the different thresholds. From the figure, it can be noticed that when γ=0 (original feature set), the test accuracy of single FAM and FAM ensemble is 0.824 and 0.845, respectively. The highest test accuracy of single FAM and FAMs’ ensemble (0.915 and 0.988) is arrived synchronously when γ=0.8, where the optimal feature set is selected. However, when the threshold value continues to increase, the test accuracy of single FAM and FAMs’ ensemble tends to decrease. And when threshold γ>0.9, the test accuracy of single FAM and FAMs’ ensemble is lower than that used in all features with threshold γ=0. This is mainly because the smaller number of features leads to the overfitting; namely, the drastic reduction of features can lead to a decrease in the test accuracy.Figure 6
Classification accuracy comparison for different threshold.


### 3.3.2. Classification Performance Comparison with Other Classification Methods

In order to test the superiority of the proposed FAMs’ ensemble method, the test results produced by FAMs’ ensemble and single FAM are compared with those produced by other classification methods. In this experiment, the parameters of FAM ensemble and single FAM are the same as the above.Table2 shows the test results of the FAMs’ ensemble versus other classification methods. From the table it can be seen that the average test accuracy using single FAMs’ is the lowest. However, the test accuracy produced by two FAMs’ ensemble methods is higher than that produced by the single classifier, and the test success rate of the proposed FAMs’ ensemble is highest and higher than that of FAM ensemble with voting algorithm. These indicate that the proposed FAMs’ ensemble has comparatively superior diagnosis performance.Table 2
Test accuracy produced by different classification methods.
Method
Test accuracy (%)
Single FAM
91.5
The proposed FAM ensemble
98.8
FAMs' ensemble with voting algorithm and five FAMs
95.2


## 3.3.1. Effect of Different Threshold for Feature Selection

As shown in Figure4, when the threshold value γ is set properly, some redundant and irrelevant features can be removed from the original feature set. To test the effect of the proposed feature selection method based on the modified distance discriminant technique, a series of experiments is carried out against the threshold value γ, in which the parameter of the single FAM is the same as the above, and the size of ensemble FAM is set as 5.Figure6 lists the classification accuracy of five individual FAMs’ and FAM ensemble against the different thresholds. From the figure, it can be noticed that when γ=0 (original feature set), the test accuracy of single FAM and FAM ensemble is 0.824 and 0.845, respectively. The highest test accuracy of single FAM and FAMs’ ensemble (0.915 and 0.988) is arrived synchronously when γ=0.8, where the optimal feature set is selected. However, when the threshold value continues to increase, the test accuracy of single FAM and FAMs’ ensemble tends to decrease. And when threshold γ>0.9, the test accuracy of single FAM and FAMs’ ensemble is lower than that used in all features with threshold γ=0. This is mainly because the smaller number of features leads to the overfitting; namely, the drastic reduction of features can lead to a decrease in the test accuracy.Figure 6
Classification accuracy comparison for different threshold.


## 3.3.2. Classification Performance Comparison with Other Classification Methods

In order to test the superiority of the proposed FAMs’ ensemble method, the test results produced by FAMs’ ensemble and single FAM are compared with those produced by other classification methods. In this experiment, the parameters of FAM ensemble and single FAM are the same as the above.Table2 shows the test results of the FAMs’ ensemble versus other classification methods. From the table it can be seen that the average test accuracy using single FAMs’ is the lowest. However, the test accuracy produced by two FAMs’ ensemble methods is higher than that produced by the single classifier, and the test success rate of the proposed FAMs’ ensemble is highest and higher than that of FAM ensemble with voting algorithm. These indicate that the proposed FAMs’ ensemble has comparatively superior diagnosis performance.Table 2
Test accuracy produced by different classification methods.
Method
Test accuracy (%)
Single FAM
91.5
The proposed FAM ensemble
98.8
FAMs' ensemble with voting algorithm and five FAMs
95.2


## 4. Conclusions

The classification performance of FAM is affected by the sequence of training samples. A novel and reliable FAMs’ ensemble based on the improved Bayesian belief method is described and proposed to improve the classification performance of FAM in this paper, which combines the output from a committee of FAM fed with different orderings of training samples and derives the combined decision.And the supposed FAMs’ ensemble method is applied to the fault identification of hydraulic pump. The experiment results testify that the proposed FAM ensemble can diagnose the fault categories accurately and reliably and has better diagnosis performance compared with single FAM. These indicate that the proposed FAMs’ ensemble has a good promise in the engineering of classification and decision making.



---

*Source: 124263-2013-09-23.xml*