# A Multiple-Kernel Relevance Vector Machine with Nonlinear Decreasing Inertia Weight PSO for State Prediction of Bearing

Authors: Sheng-wei Fei; Yong He
Journal: Shock and Vibration
 (2015)

---

## Abstract

The scientific and accurate prediction for state of bearing is the key to ensure its safe operation. A multiple-kernel relevance vector machine (MkRVM) including RBF kernel and polynomial kernel is proposed for state prediction of bearing in this study; the proportions of RBF kernel and polynomial kernel are determined by a controlled parameter. As the selection of the parameters of the kernel functions and the controlled parameter has a certain influence on the prediction results of MkRVM, nonlinear decreasing inertia weight PSO (NDIWPSO) is used to select its kernel parameters and controlled parameter. The RBF kernel RVM model with NDIWPSO (NDIWPSO-RBFRVM) and the polynomial kernel RVM model with NDIWPSO (NDIWPSO-PolyRVM) are used, respectively, to compare with the multiple-kernel RVM model with NDIWPSO (NDIWPSO-MkRVM). The experimental results indicate that NDIWPSO-MkRVM is more suitable for the state prediction of bearing than NDIWPSO-RBFRVM and NDIWPSO-PolyRVM.


---

## Body

## 1. Introduction

The scientific and accurate prediction for state of bearing is the key to ensure its safe operation [1]. Artificial neural networks (ANNs) [2, 3] and support vector machine (SVM) algorithm [4] have been used as efficient alternative tools to solve the nonlinear prediction problems. Practicability of artificial neural networks is limited due to the shortcomings of overfitting and falling into local extremum easily existed in ANNs. Support vector machine is a kind of machine learning method based on the statistical learning theory, which has better generalization performance than artificial neural networks, particularly under the condition of small training samples [5, 6].Relevance vector machine (RVM) is an intelligent learning technique based on sparse Bayesian framework, as the number of relevance vectors in RVM is much smaller than that of support vectors in SVM, which makes RVM have a sparser representation compared with SVM [7]. Thus, RVM has better generalization ability than SVM. In order to improve the generalization ability of RVM, a multiple-kernel relevance vector machine including RBF kernel and polynomial kernel is proposed for state prediction of bearing in this study; the proportions of RBF kernel and polynomial kernel are determined by a controlled parameter. As the selection of the parameters of the kernel functions and the controlled parameter has a certain influence on the prediction results of MkRVM, nonlinear decreasing inertia weight PSO is used to select its kernel parameters and controlled parameter. Particle swarm optimization is inspired by the social behavior of bird flocking or fish schooling [8]. Compared with traditional PSO, nonlinear decreasing inertia weight PSO has a nonlinear decreasing inertia weight instead of fixed inertia weight. The RBF kernel RVM model and the polynomial kernel RVM model are used, respectively, to compare with the multiple-kernel RVM model, and the kernel parameters of the RBF kernel RVM model and the polynomial kernel RVM model are also selected by NDIWPSO. The experimental results indicate that NDIWPSO-MkRVM is more suitable for the state prediction of bearing than NDIWPSO-RBFRVM and NDIWPSO-PolyRVM.


## 2. Multiple-Kernel Relevance Vector Machine

LetT=xl,tll=1N be a set of the training data, where xl denotes the input vector and tl denotes the corresponding output target and the target tl includes the additive noise [9], which can be expressed as follows:(1)tl=yxl,w+εl,where εl is assumed to be mean-zero Gaussian noise with variance σ2.The regression function of relevance vector machine which consists of a linear combination of the weighted kernel functions can be described as follows:(2)yx,w=∑i=1NwiKx,xi+w0,where K(x,xi) is the kernel function, w=[w1,w2,…,wN] is the weight vector, and w0 is the bias.As different kernel functions will obtain different regression functions of RVM, which can determine the operation performance of RVM, it is very important to choose a suitable kernel function of relevance vector machine. Here, two kinds of kernels including local kernel and global kernel are employed to construct the regression function of relevance vector machine. A typical local kernel is radial basis function kernel (RBF); in this study, Gaussian kernel is used as radial basis function kernel, which can be defined as follows:(3)KRBFxi,xj=exp⁡-xi-xj2γ2,where γ denotes the kernel parameter of the RBF kernel.A typical global kernel is the polynomial kernel, which can be defined as follows:(4)KPolyxi,xj=xi·xjT+1d,where d denotes the kernel parameter of the polynomial kernel.In order to improve the generalization ability of RVM, a multiple-kernel relevance vector machine is constructed by the local kernel function (RBF kernelKRBF) and global kernel function (polynomial kernel KPoly); the proportions of RBF kernel and polynomial kernel are determined by the controlled parameter u. Thus, the multiple-kernel function can be expressed as follows:(5)KmixRBF,Polyxi,xj=uKRBFxi,xj+1-uKPolyxi,xj,where u(0≤u≤1) denotes the controlled parameter.Whenu=1, relevance vector machine only employs RBF kernel function, and when u=0, relevance vector machine only employs polynomial kernel function. It is obvious that the multiple-kernel function integrates all characters of independent kernel and has better data distribution performance.


## 3. State Prediction Method of Bearing Based on Multiple-Kernel Relevance Vector Machine with Nonlinear Decreasing Inertia Weight PSO

State prediction process of bearing based on multiple-kernel relevance vector machine with nonlinear decreasing inertia weight PSO can be described in this section. The experimental data are normalized to the range[0,1] in order to improve the generalization ability of the prediction model. Assume that the normalized experimental data are a1,a2,…,am,…,an,…,an+k, among which a1,a2,…,am,…,an are used to establish the training sample sets, and an+1,…,an+k are used to test the prediction model. The training sample sets can be described by the formula:(6)X=a1a2⋯ama2a3⋯am+1⋮⋮⋱⋮an-man-m+1⋯an-1,Y=am+1am+2⋮an,where m denotes the embedding dimension, X denotes the set of input vectors, and Y denotes the set of corresponding outputs.In this paper, we use nonlinear decreasing inertia weight particle swarm optimization to select the kernel parametersγ and d and controlled parameter u. In NDIWPSO, the particle is composed of the kernel parameters γ and d and controlled parameter u; each parameter has its value range. We perform the validation for each training sample by the MkRVM models with the different particles and obtain the corresponding validation error of each training sample. Mean validation error of all training samples can be used to evaluate the performance of the MkRVM models with the different particles. The process of the selection of the kernel parameters γ and d and controlled parameter u of MkRVM by NDIWPSO can be described as follows.Step 1.
Perform the setting of the parameters of NDIWPSO and initialization of the particles.Step 2.
Evaluate the fitness of each particle. The mean validation error of all the training samples is used to create the fitness function, which is defined as follows:(7)e~=1G∑q=1Gyq-y^qyq,where yq is the actual value and y^q is the validation value; G is the number of the training samples in training sample sets.Step 3.
Update the global and personal best according to the fitness evaluation results.Step 4.
The particle flies toward a new position by calculating the velocity of position change, and the velocity of each particle is calculated by the following formula:(8)vijh+1=ωh·vijh+c1·rand·pbestijh-pijh+c2·rand·gbestjh-pijh,where h denotes the iteration counter, vij is the velocity of particle i on the jth dimension, pij is the position of particle i on the jth dimension, pbestij is the personal best (pbest) position of particle i on the jth dimension, gbestj is the global best (gbest) position of the swarm, rand means the random value in the range [0,1], and the positive constants c1 and c2 are personal learning factor and social learning factor, respectively. ω is the inertia weight, which is used to balance the global exploration and local exploitation, ω(h)=ωmax⁡-((ωmax⁡-ωmin⁡)/H-1)×h-1 (ωmax⁡ is the maximum inertia weight, ωmin⁡ is the minimum inertia weight, and H is the maximum iteration).Step 5.
Each particle moves to its next position according to the following formula:(9)pijh+1=pijh+β·vijh+1,where β is constraint factor used to control the velocity weight.Step 6.
The same procedures from Step2 to Step 5 are repeated until the maximum iteration is reached.Then, the MkRVM model is trained with obtained kernel parametersγ and d and controlled parameter u. Finally, the proposed NDIWPSO-MkRVM model is tested by the testing data.


## 4. Experimental Analysis

Kurtosis of bearing vibration signal can excellently reflect the state of bearing; that is, kurtosis prediction of bearing vibration signal can excellently reflect the future state of bearing. In this paper, the kurtosis data can be calculated and obtained from bearing vibration signal in “Bearing Data Set” [10]. As shown in Figure 1, the kurtosis data of bearing vibration signal of three cases are given, and 100 numbers of kurtosis data of bearing vibration signal are employed in each case. In each case, numbers 1~90 of kurtosis data of bearing vibration signal are used as the training data, and numbers 91~100 of kurtosis data of bearing vibration signal are used as the testing data.Figure 1
The kurtosis data of bearing vibration signal.The experimental data are normalized to the range[0,1] in order to improve the generalization ability of the prediction model. And we use nonlinear decreasing inertia weight particle swarm optimization to select the kernel parameters γ and d and controlled parameter u. In NDIWPSO, ωmax⁡ is set to 0.9, ωmin⁡ is set to 0.4, β is set to 1/1.3, and the positive constants c1 and c2 are set to 2. Here, the value range of γ is [0.5,5], the value range of d is [0,5], and the value range of u is [0,1]. We set the embedding dimension m to 5 to establish the training sample sets. The RBF kernel RVM model and the polynomial kernel RVM model are used, respectively, to compare with the multiple-kernel RVM model, and the kernel parameters of the RBF kernel RVM model and the polynomial kernel RVM model are also selected by NDIWPSO.In case  1, Figure2 gives the comparison of the prediction results for kurtosis of bearing vibration signal among the multiple-kernel RVM model with NDIWPSO (NDIWPSO-MkRVM), the RBF kernel RVM model with NDIWPSO (NDIWPSO-RBFRVM), and the polynomial kernel RVM model with NDIWPSO (NDIWPSO-PolyRVM); and Figure 3 gives the comparison of the absolute percentage prediction errors for kurtosis of bearing vibration signal among NDIWPSO-MkRVM, NDIWPSO-RBFRVM, and NDIWPSO-PolyRVM. As shown in Table 1, the mean absolute percentage error (MAPE) of NDIWPSO-MkRVM in case  1 is 11.32%, the MAPE of NDIWPSO-RBFRVM in case  1 is 11.99%, and the MAPE of NDIWPSO-PolyRVM in case  1 is 12.08%. It can be seen that NDIWPSO-MkRVM has smaller MAPE than NDIWPSO-RBFRVM and NDIWPSO-PolyRVM in this case.Table 1
The comparison of the mean absolute percentage errors among NDIWPSO-MkRVM, NDIWPSO-RBFRVM, and NDIWPSO-PolyRVM.
Prediction model
MAPE (%)
Case 1
NDIWPSO-MkRVM
11.32
NDIWPSO-RBFRVM
11.99
NDIWPSO-PolyRVM
12.08
Case 2
NDIWPSO-MkRVM
6.44
NDIWPSO-RBFRVM
6.63
NDIWPSO-PolyRVM
7.41
Case 3
NDIWPSO-MkRVM
12.79
NDIWPSO-RBFRVM
13.93
NDIWPSO-PolyRVM
12.96Figure 2
The comparison of the prediction results for kurtosis of bearing vibration signal among NDIWPSO-MkRVM, NDIWPSO-RBFRVM and NDIWPSO-PolyRVM in case 1.Figure 3
The comparison of the absolute percentage prediction errors for kurtosis of bearing vibration signal among NDIWPSO-MkRVM, NDIWPSO-RBFRVM and NDIWPSO-PolyRVM in case 1.In case  2, Figure4 gives the comparison of the prediction results for kurtosis of bearing vibration signal among NDIWPSO-MkRVM, NDIWPSO-RBFRVM, and NDIWPSO-PolyRVM; and Figure 5 gives the comparison of the absolute percentage prediction errors for kurtosis of bearing vibration signal among NDIWPSO-MkRVM, NDIWPSO-RBFRVM, and NDIWPSO-PolyRVM. As shown in Table 1, the MAPE of NDIWPSO-MkRVM in case  2 is 6.44%, the MAPE of NDIWPSO-RBFRVM in case  2 is 6.63%, and the MAPE of NDIWPSO-PolyRVM in case  2 is 7.41%. It can be seen that the MAPE of NDIWPSO-RBFRVM is near to that of NDIWPSO-MkRVM in case  2; however, the MAPE of NDIWPSO-PolyRVM is obviously higher than that of NDIWPSO-MkRVM in case  2.Figure 4
The comparison of the prediction results for kurtosis of bearing vibration signal among NDIWPSO-MkRVM, NDIWPSO-RBFRVM and NDIWPSO-PolyRVM in case 2.Figure 5
The comparison of the absolute percentage prediction errors for kurtosis of bearing vibration signal among NDIWPSO-MkRVM, NDIWPSO-RBFRVM and NDIWPSO-PolyRVM in case 2.In case  3, Figure6 gives the comparison of the prediction results for kurtosis of bearing vibration signal among NDIWPSO-MkRVM, NDIWPSO-RBFRVM, and NDIWPSO-PolyRVM; and Figure 7 gives the comparison of the absolute percentage prediction errors for kurtosis of bearing vibration signal among NDIWPSO-MkRVM, NDIWPSO-RBFRVM, and NDIWPSO-PolyRVM. As shown in Table 1, the MAPE of NDIWPSO-MkRVM in case  3 is 12.79%, the MAPE of NDIWPSO-RBFRVM in case  3 is 13.93%, and the MAPE of NDIWPSO-PolyRVM in case  3 is 12.96%. It can be seen that the MAPE of NDIWPSO-PolyRVM is near to that of NDIWPSO-MkRVM in case  3; however, the MAPE of NDIWPSO-RBFRVM is obviously higher than that of NDIWPSO-MkRVM in case  3.Figure 6
The comparison of the prediction results for kurtosis of bearing vibration signal among NDIWPSO-MkRVM, NDIWPSO-RBFRVM and NDIWPSO-PolyRVM in case 3.Figure 7
The comparison of the absolute percentage prediction errors for kurtosis of bearing vibration signal among NDIWPSO-MkRVM, NDIWPSO-RBFRVM and NDIWPSO-PolyRVM in case 3.The experimental results show that the MAPE of NDIWPSO-RBFRVM is near to that of NDIWPSO-MkRVM in case  2, but the MAPE of NDIWPSO-RBFRVM is obviously higher than that of NDIWPSO-MkRVM in case  3; in addition, the MAPE of NDIWPSO-PolyRVM is near to that of NDIWPSO-MkRVM in case  3, but the MAPE of NDIWPSO-PolyRVM is obviously higher than that of NDIWPSO-MkRVM in case  2. It is indicated that kurtosis prediction performance of bearing vibration signal of NDIWPSO-RBFRVM and NDIWPSO-PolyRVM is less stable than that of NDIWPSO-MkRVM. Thus, NDIWPSO-MkRVM is more suitable for the state prediction of bearing than NDIWPSO-RBFRVM and NDIWPSO-PolyRVM.


## 5. Conclusion

In this paper, multiple-kernel relevance vector machine with nonlinear decreasing inertia weight PSO is proposed for state prediction of bearing. The multiple-kernel function including two kinds of kernels, RBF kernel and polynomial kernel, is employed to improve the generalization ability of RVM, and we use nonlinear decreasing inertia weight PSO to select the kernel parameters and controlled parameter of MkRVM. The experimental results show that NDIWPSO-MkRVM has more stable kurtosis prediction performance of bearing vibration signal than NDIWPSO-RBFRVM and NDIWPSO-PolyRVM. Thus, NDIWPSO-MkRVM is more suitable for the state prediction of bearing than NDIWPSO-RBFRVM and NDIWPSO-PolyRVM.



---

*Source: 685979-2015-03-17.xml*