# Rotating Machinery Fault Detection Using Support Vector Machine via Feature Ranking
## 6 Model Evaluation  
The final best model for this study is Quadratic SVM, using the \(11\) most important predictors with an overall accuracy of \(99.7\%\) and a total training time of \(6.9\) s. Even though the accuracy decreases by \(0.1\%\) from the Quadratic SVM using all \(58\) predictors, the amount of training time it saves is significant. One of the earlier obstacles during this study is computational power. With fewer predictors, a more extensive data set could be used to generalize the proposed system. To assess the performance of the Quadratic SVM model on unseen data, a random subset from the original dataset was extracted and used as a new test set. The accuracy dropped significantly when evaluated on this new data, suggesting that the initial training dataset was insufficient or not diverse enough to capture the full range of features required for the model to generalize well. Thus, this result may seem to be due to a potential issue with overfitting, where the model may have learned to perform exceptionally well on the training data but struggles to generalize to new, unseen data. But, we believe that the drop in performance may be due to the varying frequency of the motors. This is why, in our initial dataset selection, the key parameter for selection was the rotational frequency. If the rotational frequency changes, this causes tremendous variation. In our experiment, we were able to verify when the frequency was similar in another dataset; although the model and predictors may be different, the performance was comparable to the initial results, which shows that when the frequencies are similar, high classification accuracy could be obtained. Therefore, if a form of time-domain signal warping could be applied \begin{table}
\begin{tabular}{l l l l l l} \hline \hline
**Model Number** & **Model Type** & **Neural Network** & **Predictors** & **Accuracy (Validation)** & **Training Time** \\ \hline
6.2 & SVM & Quadratic SVM & All & \(99.8\%\) & \(16.3\) s \\ \hline
12 & SVM & Quadratic SVM & Only \(11\) Predictors & \(99.7\%\) & \(6.9\) s \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Quadratic SVM Models using All Predictors vs. \(11\) Predictors. Figure 14: Improved Quadratic SVM model using \(11\) predictors, with green boxes indicating accuracy improvement and red boxes for lower accuracy compared to the Quadratic SVM using all predictors. to absorb the variation caused by the change in frequency, the algorithm could produce higher accuracy. For verification, a few additional Quadratic SVM models using two different datasets (data sets #2 and #3) of equal size and modified predictors were trained. Similar to Cross-Validation in the initial training, 20% of each new data set is set aside for testing. And again, both models showed a significant drop in test accuracy shown in Table 8. This further reinforces the hypothesis that the datasets currently in use may be too small or not representative enough of the underlying patterns in the data. The main reason for the poor model performance may have been due to the limited size of the training data, which in turn was constrained by the similarity in frequency of the data and available computational power. The lack of sufficient processing capability made training and testing the model on larger, more comprehensive datasets impossible. Since the computer used for this task lacked the necessary computing and memory resources to handle large-scale data processing, this became a significant bottleneck, limiting both the quantity and quality of data that could be fed into the model. With smaller datasets, the model may not encounter enough feature variation, reducing its ability to learn effectively and to generalize to unseen data. This lack of diversity in the training data can lead to overfitting, where the model performs exceptionally well on the training set but struggles to adapt to new patterns when tested on different data.