# AI-Driven Predictive Maintenance for Workforce and Service Optimization in the Automotive Sector
## 4 Methodology
### Model Training
#### 4.2.8 Feature Selection  
Feature selection in this study was conducted through a two-phase approach, utilizing both pre-modeling data refinement and model-based importance evaluation. In the first phase, unique or identifier-type features such as Customer ID, Chassis ID, Service Visit Date, and Guarantee Erd Date were excluded from the dataset to prevent data leakage and reduce bias during model training. The second phase was embedded within the modeling process, where algorithmic feature importance metrics were employed to assess the relevance of each remaining variable. This phase involved training Decision Tree-based models and leveraging their inherent ability to measure feature contribution through impurity reduction. To provide a comprehensive evaluation, a comparative analysis of feature importance was also carried out across four distinct algorithms: Decision Tree (DT), Random Forest (RF), Light Gradient Boosting Machine (LightGBM), and Extreme Gradient Boosting (XGBoost). Each algorithm was trained using the full feature set (excluding the predefined unique identifiers), and the relative importance of each variable was assessed using model-specific feature attribution mechanisms:  
For DT and RF, feature importance was computed based on the average decrease in Gini impurity. The impact of the feature is measured in terms of the reduction in node impurity, which is weighted by the probability of reaching the tree node using the Decision Tree algorithm. The node probability can be calculated by the ratio of the number of samples reaching that node to the total number of samples. The higher the reduction value, the more significant the feature will be. It can also be said that the split is chosen at each step during tree construction, which causes the highest reduction in the impurity of the data reaching the m node [33]. The reduction in impurity is measured as follows [33]. \[nij:=w_{j}C_{j}-w_{left(j)}C_{left(j)}-w_{right(j)}C_{right(j)} \tag{27}\]  
In gradient boosting algorithms such as LightGBM and XGBoost, feature importance is often assessed using the gain metric, which reflects the total contribution of a feature to the improvement of the model's objective function (e.g., log loss for classification, RMSE for regression). At each split in the Decision Tree, a feature is selected to partition the data based on the maximum gain, which corresponds to the greatest reduction in the loss function. Let  
\(\mathcal{L}\) be the loss function of the model;  
\(I\) be the set of instances in the node;  
\(f\) be the feature used for splitting node \(j\);  
\(\mathcal{L}_{\text{parent}}\), \(\mathcal{L}_{\text{left}}\), and \(\mathcal{L}_{\text{right}}\) represent the loss for the parent, left child, and right child nodes, respectively. Then, the gain from using feature \(f\) at node \(j\) is given by  
\[\text{Gain}_{j}(f)=\mathcal{L}_{\text{parent}}-\mathcal{L}_{\text{left}}+ \mathcal{L}_{\text{right}} \tag{28}\]In XGBoost, the gain is typically calculated using second-order gradient statistics. We assume that  
\(G\) is the sum of first-order gradients;  
\(H\) is the sum of \(\sec\)ond-order gradients;  
\(\lambda\) is the regularization term. Then the gain for a split is  
\[\text{Gain}_{j}(f)=\frac{1}{2}\Bigg{[}\frac{G_{\text{left}}^{2}}{H_{\text{ left}}+\lambda}+\frac{G_{\text{right}}^{2}}{H_{\text{right}}+\lambda}-\frac{ \left(G_{\text{left}}+G_{\text{right}}\right)^{2}}{H_{\text{left}}+H_{\text{ right}}+\lambda}\Bigg{]} \tag{29}\]  
A higher gain value suggests that the corresponding feature consistently contributes to reducing the model's prediction error, thereby increasing its predictive utility. The cumulative gain, aggregated over all the splits in which the feature is utilized, reflects its overall importance and global impact on the model's performance. This method of evaluating feature importance enhances model interpretability and provides a reliable basis for understanding feature relevance, particularly in high-dimensional datasets where identifying influential variables is critical.