# A Cost-Efficient MCSA-Based Fault Diagnostic Framework for SCIM at Low-Load Conditions
## 5 Experimental Study
### ML-Based Diagnosis  
Empirically, the feature extraction and evaluation processes described in the previous subsections provide the platform upon which traditional ML-based classifiers are deployed for fault classification. Practically, each ML-based classifier is unique in its architecture and learning principle, and this often poses a concern when choosing the most appropriate model for practical use. In addition, their respective cons and computational costs also pose valid concerns during decision making. Consequently, we explore (as much as possible) many popular traditional ML-based classifiers in our study. The classifiers summarized in  
Figure 6: Feature evaluation results: (**a**) correlation heatmap between features, (**b**) probability density plot of the features, and (**c**) LLE-assisted discriminative property assessment. Table 2 were employed for training (using the training feature set) and testing (using the test feature set). For optimal efficiency, each algorithm has its own set of parameters and architecture, as shown in Table 2. Exhaustive parameter tuning optimized parameters for each algorithm, which are recorded in Table 2. Figure 7 illustrates the accuracy, precision, recall, and F1-scores resulting from ten-fold cross-validation of the algorithms on the test data. In general terms, accuracy measures a model's ability to predict classes correctly and is represented in Figure 7 in green circles. F-1 (represented by yellow squares with blue dotted lines) is calculated by taking the average of precision and recall, which determines the proportion of true predictions the model makes among all actual predictions. Precision determines the percentage of classes that are true and is represented by black triangles, while recall determines the percentage of predicted cases that are actually true and is represented by red stars. As observed, RF is the highest performing classifier, with a test accuracy of 79.25%. This is followed by NBC, QDA, and GBC, whose test accuracies are 73%, 71.4%, and 70.25%, respectively. On the downside, GPC and Gaussian SVM were the worst performing, with test accuracies of 26.55% and 25%, respectively, while the rest of the classifiers ranged between 40% and 62%. It can be observed that the training scores of all the classifiers (except ABC) on the training data are almost 100%; however, their test performances are not as optimal as anticipated. This hints at the superiority of some classifiers over others. Interestingly, RF remains one of the most reliable ML-based classifiers and has been shown to be the most accurate in the proposed case study. From a different perspective, assessing the computational cost of the classifiers provides a further avenue for assessing their suitability for practical use, especially in cost-sensitive situations where computational power is a \begin{table}
\begin{tabular}{l l l} \hline \hline
**Algorithm** & **Parameter** & **Value** \\ \hline Logistic regression (LR) & regularization & L1 \\ Adaboost classifier (ABC) & \(n\) estimators, learning rate & 50, 0.1 \\ Naive Bayes classifier (NBC) & Gaussian & – \\ \(k\)-nearest neighbor (KNN) & k & 5 \\ Gaussian process classifier (GPC) & kernel & RBF \\ Random forest (RF) & \(n\) estimators & 120 \\ Gradient boosting classifier (GBC) & \(n\) estimators & 1000 \\ Decision tree (DT) & pruning & 12 \\ Multi-layer perceptron (MLP) classifier & \(n\) layers, learning rate & 3, 0.001 \\ Linear SVM & kernel & linear \\ Gaussian SVM & C, gamma & 10, 1 \\ Quadratic discriminant analysis (QDA) & regularization & 0.001 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Classifiers and their respective architecture. Figure 7: Global performance evaluation of the classifiers on the test data. concern. Table 3 summarizes the computational costs (in seconds) of the classifiers for training and testing.