# A Novel Rolling Bearing Fault Diagnosis and Severity Analysis Method
## 4 Multiclass Relevance Vector Machine
### Multiclass Relevance Vector Machine  
The multiclass relevance vector machine (mRVM) expands the original RVM to the multi-class setting by introducing the auxiliary variables \(\mathbf{Y}\in\Re^{C\times N}\), which act as intermediate regression targets. This approach uses the hierarchical Bayesian framework to solve multiclass issues and introduces the multinomial probit likelihood to output the probability of the members. The structure of the hierarchical Bayesian model is shown in Figure 2.  
There are two classification algorithms, mRVM\({}_{1}\) and mRVM\({}_{2}\), which expand the original RVM to the multi-class multi-kernel setting. The theoretical bases of the two versions of mRVM are consistent. First, a training set \(\mathbf{T}=\{\mathbf{x}_{i},t_{i}\}_{i=1}^{N}\) should be given, where \(\mathbf{x}_{i}\in\Re^{D}\) is the \(D\)-dimensional feature vector and \(\mathbf{t}\in\{1,2,\cdots,C\}\) is the respective class label. The kernel function of the training set is \(\mathbf{K}\in\Re^{N\times N}\). By introducing the auxiliary variables \(\mathbf{Y}\in\Re^{C\times N}\) to act as the regression targets of the weighting parameter \(\mathbf{W}^{T}\mathbf{K}\), the standardized noise model is deduced as follows:  
\[\mathbf{y}_{cn}|w_{c},\mathbf{k}_{n}\sim N_{y_{cn}}(w_{c}^{T}\mathbf{k}_{n},1). \tag{27}\]  
Convert the regression targets above into the category labels by introducing the multinomial probit link:  
\[\mathbf{t}_{n}=i,\mathbf{y}_{ni}>\mathbf{y}_{nj},\mathbf{\forall}i\
eq j. \tag{28}\]  
The probabilistic output for class membership is obtained through the resultant multinomial probit likelihood function:  
\[P(\mathbf{t}_{n}=i|\mathbf{w},\mathbf{k}_{n})=\varepsilon_{p(u)}\left\{\prod_{j\
eq i} \mathbf{\Phi}(u+(w_{i}-w_{j})^{T}\mathbf{k}_{n})\right\}, \tag{29}\]  
where \(u\sim N(0,1)\) and \(\mathbf{\Phi}\) denotes the Gaussian cumulative distribution function. To ensure the sparse model, the weight \(w_{nc}\) follows a standard normal distribution with zero-mean and variance \(\alpha_{nc}^{-1}\), where \(\alpha_{nc}\) belongs to the scales matrix \(\mathbf{A}\in\Re^{N\times C}\) and follows a gamma distribution. With sufficiently small \(\tau\) and \(v\) (\(<\)10\({}^{-5}\)), most \(\mathbf{w}\) values are restricted around zero, which naturally leads to a sparse solution. The closed-form posterior of the weight \(\mathbf{w}\) can be derived as follows:  
\[P(\mathbf{w}|\mathbf{Y})\propto P(\mathbf{Y}|\mathbf{w})P(\mathbf{w}|\mathbf{A})\propto\prod_{c=1}^{ C}N((\mathbf{KK}^{T}+\mathbf{A}_{c})^{-1}\mathbf{K}y_{c}^{T},(\mathbf{KK}^{T}+\mathbf{A}_{c}))^{-1}, \tag{30}\]  
where \(\mathbf{A}_{c}\) is a diagonal matrix derived from the \(c\) column of \(\mathbf{A}\). The maximum a posteriori estimator is as follows:  
\[\hat{\mathbf{w}}=\underset{\mathbf{w}}{arg\,max}P(\mathbf{w}|\mathbf{Y},\mathbf{A},\mathbf{K}). \tag{31}\]  
Figure 2: Schematic diagram of the hierarchical Bayesian model. Then, the parameters are updated based on the maximum a posterior estimator value when a class \(i\) is given:  
\[\hat{w}_{c}=(\textbf{K}\textbf{K}+\textbf{A}_{c})^{-1}\textbf{K}y_{c}. \tag{32}\]  
Finally, the posterior probability distribution of the priori parameters of the weight vector is as follows:  
\[P(\textbf{A}|\textbf{w})\propto P(\textbf{w}|\textbf{A})P(\textbf{A}|\tau,v) \propto\prod_{c=1}^{C}\prod_{n=1}^{N}G(\tau+\frac{1}{2},\frac{w_{nc}^{2}+2v}{ 2}).\]  
[PERSON] et al. introduced and provided the theoretical background of the two multi-class multi-kernel mRVMs [25]. The difference between the two versions of the mRVM is how the mRVM manipulates the kernel during the training phase. mRVM\({}_{1}\) follows a constructive approach starting with an empty model and then adding or removing samples from the training kernel based on their contribution. However, mRVM\({}_{2}\) follows a top-down approach, loading the entire training set into memory and iteratively pruning uninformative samples.