# AI-Driven Predictive Maintenance for Workforce and Service Optimization in the Automotive Sector
## 5 Experimental Works  
A dataset with 1,309,915 samples with 48 attributes was used to train the models. Then, 44.011 test samples were tested, and the following results were obtained, as shown in Table 8. Furthermore, each model was replicated ten times to ensure the reliability of the predicted values. Multiple cross-validations (10-fold cross-validation) were applied to make the comparison more intuitive. The most successful results based on F1-score and AUC performance metrics were obtained with the XGBoost algorithm. Figure 22 shows the ROC graph of the XGBoost algorithm. An ROC curve shows a plot of the true positive rate as a function of the false positive rate for different cut-off points [43]. A perfect discrimination experiment has an ROC curve passing through the upper left corner and exhibits 100% sensitivity and 100% specificity [44]. Therefore, the closer the ROC curve is to the upper left corner, the higher the class separation success of the experiment [44]. According to Figure 22, the model experiment with the ROC curve closest to the upper left corner is XGBoost. The ROC curve of the DT algorithm moved away from the left axis after the threshold of 0.05. The reason for this could be the difficulty of class discrimination as the threshold increases. The DT algorithm separates classes based on impurity. Samples with a probability value greater than 0.05 may contain samples whose impurity value is not low enough. These samples can be misclassified. While applying the above algorithms, the threshold value was chosen as 0.5. The applied algorithms essentially output a probability value. These probability values are cut according to a determined threshold value, and target values such as 0 or 1 are assigned. In general, 0.5 is used as a threshold in algorithms that give a probability value. However, choosing a threshold of 0.5 may not always be the accurate approach. Threshold optimization can be applied for different purposes in different works. In this paper, the aim is to have a high precision score, that is, the number of vehicles we estimated correctly among \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Algorithms** & **F1-Score** & **Recall** & **Precision** & **AUC** \\ \hline DT & 0.30 & 0.31 & 0.29 & 0.64 \\ RF & 0.40 & 0.57 & 0.31 & 0.82 \\ XGBoost & 0.45 & 0.51 & 0.39 & 0.90 \\ LGBM & 0.10 & 0.93 & 0.05 & 0.60 \\ \hline \hline \end{tabular}
\end{table} 
Table 8: Performance metrics of the applied algorithms. Figures 26 and 27 show that the derived features are of great importance in understanding and solving the problem. Section 2.3 shows the 12 most important attributes when the XGBM algorithm is used. The most striking of these attributes is the total number of vehicles owned by the relevant vehicle owner. While deriving this attribute, we thought that customers who have more vehicles would care about their vehicles and would not disrupt their Maintenance Periods. Another important attribute is the number of days since the vehicle owner last bought the vehicle. We derived this attribute because we think that if the owner has recently purchased a vehicle, they interact with the authorized dealer. We have derived the percentage differences in the fees paid by the vehicle owner in maintenance services to measure the variability of the changes in the fee in maintenance habits. The Count of Owner covariant is derived to show the model of whether the vehicle has changed ownership. This attribute was derived because we think the authorized service habit may decrease as the vehicle changes hands. To address the inherent opacity of ensemble-based models, particularly XGBoost, we employed model-agnostic interpretability techniques to enhance transparency and stakeholder trust. Specifically, we utilized SHAP (SHapley Additive exPlanations) values to quantify the marginal contribution of each feature to the model's predictions.