## 4 Model Evaluation
The table highlights that both the SVM and LSTM algorithms achieved their best performance when using 10 features selected by RFI-10, with the SVM algorithm achieving an accuracy of 98.12% and F1-score of 98.19% and the LSTM algorithm achieving an accuracy of 98.48% and F1-score of 98.41%. In general, selecting 10 features led to better model performance compared to selecting 5 features. This study shows that careful feature selection can significantly impact the effectiveness of ML models in fault classification, with RFI emerging as the most successful method. The performance metrics in bold show the best results in Table 5.  
Case 2: The NASA battery dataset is used to extract 45 TD features from voltage, current, and temperature measurements. The proposed FSMs are evaluated using the SVM and LSTM models. Table 6 displays the feature selection results for the NASA battery dataset. Kurtosis and P2P were consistently selected across multiple methods for both voltage and current, indicating their strong relevance in battery health monitoring. The features expressed in bold are the features selected by all FS methods in Table 6. The graphics of the most frequently selected features are given in Figure 3. \begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Method**} & \multicolumn{4}{c}{**Performance Metrics (\%)**} \\ \cline{2-5}  & **Accuracy** & **Precision** & **Recall** & **F1-Score** \\ \hline SVM-all feature & 97.83 & 98.06 & 97.94 & 97.93 \\ SVM-MI-5 & 85.80 & 81.92 & 86.38 & 83.00 \\ SVM-MI-10 & 87.97 & 83.52 & 88.31 & 85.10 \\ SVM-FS-5 & 69.42 & 70.23 & 70.67 & 67.75 \\ SVM-FS-10 & 92.75 & 93.91 & 93.28 & 93.05 \\ SVM-SFS-5 & 96.81 & 97.05 & 96.95 & 96.94 \\ SVM-SFS-10 & 97.97 & 98.17 & 98.08 & 98.07 \\ SVM-RFE-5 & 88.99 & 84.30 & 89.21 & 85.98 \\ SVM-RFE-10 & 88.55 & 88.94 & 88.89 & 87.33 \\ SVM-RFI-5 & 92.61 & 94.32 & 93.30 & 92.96 \\
**SVM-RFI-10** & **98.12** & **98.23** & **98.22** & **98.19** \\ LSTM-all feature & 98.39 & 98.32 & 98.38 & 98.32 \\ LSTM-MI-5 & 91.09 & 91.53 & 91.20 & 90.66 \\ LSTM-MI-10 & 98.30 & 98.27 & 98.26 & 98.23 \\ LSTM-FS-5 & 85.30 & 85.74 & 85.83 & 84.99 \\ LSTM-FS-10 & 96.97 & 96.60 & 96.65 & 96.50 \\ LSTM-SFS-5 & 97.87 & 97.79 & 97.88 & 97.80 \\ LSTM-SFS-10 & 98.43 & 98.41 & 98.41 & 98.37 \\ LSTM-RFE-5 & 91.26 & 90.78 & 90.98 & 90.04 \\ LSTM-RFE-10 & 98.26 & 98.23 & 98.25 & 98.20 \\ LSTM-RFI-5 & 95.13 & 95.30 & 95.16 & 94.99 \\
**LSTM-RFI-10** & **98.48** & **98.48** & **98.43** & **98.41** \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Performance evaluation with selected Feature Selection Methods for CWRU bearing dataset. The penalty parameter \(c\) is set to 50 and \(\gamma\) is set to 0.01. The datasets are split into training and testing subsets, with 70% designated for training and the remaining 30% for testing purposes, allowing the models to be trained and subsequently evaluated on separate data. The LSTM model consists of 1 input LSTM layer with 64 units and a dense layer with 32 units, respectively, and an output layer. For multi-class classification problems, the Binary Cross-Entropy is implemented as the loss function, while the sigmoid function is utilized as the activation mechanism within the model, and [PERSON] as the optimizer. A dropout ratio of 0.3 is applied to prevent overfitting. A total of 50 training epochs and 32 batch size are applied, with early stopping for both datasets. After early stopping, the learning rate scheduler is applied for preventing overfitting. Then, 10-fold cross-validation is applied to both algorithms to ensure the results are robust and generalizable across different data splits. Table 7 presents the performance evaluation of SVM and LSTM models using different FSMs on the NASA PCoE lithium-ion battery dataset. The table highlights that LSTM achieved its best performance when using 10 features selected by RFI-10. The LSTM model achieved an accuracy of 99.06% and an F1-score of 98.80%.