# A Tutorial for Feature Engineering in the Prognostics and Health Management of Gears and Bearings
## 4 Feature Evaluation and Selection for Diagnosis
### Scatter Matrices  
In the more than two classes problem, the following process should be considered. Denoting the number of classes as \(M\) and the prior probability of class as \(P_{i}\cong n_{i}/N\) where \(n_{i}\) is the number of samples in the \(i\)th class and \(N\) is the number of total samples, the _Within-class scatter matrix_, which represents the degree of dispersion within each class, is defined as follows:  
\[\mathbf{S}_{w}=\sum_{i=1}^{M}P_{i}\mathbf{S}_{i},\text{ where }\mathbf{S}_{i}=E\Big{[}(\mathbf{x}- \mu_{i})(\mathbf{x}-\mu_{i})^{T}\Big{]}, \tag{16}\]  
where \(\mathbf{S}_{i}\) is the covariance of the feature vector \(\mathbf{x}\) with the mean \(\mu_{i}\) at the \(i\)th class. The _Between-class scatter matrix_, which represents the distance of the mean of each individual class from the global mean, is then defined as  
Figure 10: PDF of (**a**) MEAN, and (**b**) SK, representing the max and min FDR, respectively, for HS gear.  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Top 5** & **Feature Name** & **FDR Value** & **Bottom 5** & **Feature Name** & **FDR Value** \\ \hline
1 & MEAN & 57.0285 & 1 & SK & 0.0000 \\
2 & NB4 & 15.9001 & 2 & M6A & 0.0012 \\
3 & M8A & 13.0798 & 3 & SER & 0.2256 \\
4 & FM4 & 12.9912 & 4 & SF & 0.2274 \\
5 & CR & 12.1305 & 5 & NA4 & 1.8612 \\ \hline \hline \end{tabular}
\end{table}
Table 3: Rank of FDR value for HS gear.  
\[S_{b}=\sum_{i=1}^{M}P_{i}(\mu_{i}-\mu_{0})(\mu_{i}-\mu_{0})^{T},\text{ where }\mu_{0}=\sum_{i=1}^{M}P_{i}\mu_{i}. \tag{17}\]  
The _mixture scatter matrix_ is then defined as  
\[S_{m}=E\Big{[}(\mathbf{x}-\mu_{0})(\mathbf{x}-\mu_{0})^{T}\Big{]}=S_{w}+S_{b} \tag{18}\]  
With these matrices, a new criterion for feature evaluation is defined as follows [25]:  
\[J_{3}=trace\Big{\{}S_{w}^{-1}S_{m}\Big{\}} \tag{19}\]  
Similar to the FDR, the smaller the \(S_{w}\), namely the smaller dispersion within the class, and the larger the \(S_{m}\), meaning the larger distance between the different classes, the larger the \(J_{3}\) value we obtain, which means a better classification between the classes. The corresponding Matlab function is in Appendix A as \(J_{3}=\)_ScattMat(data,label)_ where _data_ are the feature vectors obtained in Section 4 and _label_ is the fault mode. In this study, \(J_{3}\) is calculated to select the two features out of the total features as an illustration. It is applied to the two examples: KAUG and CWRU, which are the three class problems as shown in Table 1. Note in the KAUG problem that the encoder signals are those preprocessed by the TSA. Hence, the 11 time domain features are extracted after applying the HPF only to the signal. Then the \(J_{3}\) are calculated for any two features combinations from 11 features which amount to 55 cases. The features for top 5 and bottom 5 results are listed in Table 4. The scatter plot of the best (PEAK & CL) and worst (STD & SK) features are also given in Figure 11a,b, respectively. The differences are outstanding: the features with larger \(J_{3}\) classify the faults much better. In the CWRU problem, the 14 features as shown in Figure 9 are used to find out the two best features based on the \(J_{3}\). The top 5 and bottom 5 results are given in Table 5. The scatter plots of the two features with best (SF & P2P) and worst (SK & IF) are also given in Figure 12a,b. The same observations are found in this case as well.