# An Interpretable XGBoost Framework for Predicting Oxide Glass Density
## 2 Materials and Methods
### Machine Learning Models  
The machine learning models were trained using the Scikit-learn Python library. We tested three algorithms, each representing a different model class: linear (ElasticNet), tree-based (XGBoost), and Artificial Neural Networks (MLP). For each of the four feature sets, the models' hyperparameters were optimized using a grid search with fivefold cross-validation. The performance was evaluated using the negative root mean squared error (nRMSE) as the scoring metric. The optimal hyperparameter values are summarized in Table S5. The final model's performance was evaluated using the following metrics:  
\[R^{2}=\frac{\sum_{i=1}^{n}\left(\rho_{i}^{exp}-\rho_{i}^{pred}\right)^{2}}{ \sum_{i=1}^{n}\left(\rho_{i}^{exp}-\langle\rho^{exp}\rangle\right)^{2}}, \tag{1}\]\[MSE=\frac{1}{n}{\sum_{i=1}^{n}}{\left({\rho_{i}^{exp}-\rho_{i}^{pred}}\right)^{2}}, \tag{2}\]  
\[MAE=\frac{1}{n}{\sum_{i=1}^{n}}{\left|{\rho_{i}^{exp}-\rho_{i}^{pred}}\right|} \tag{3}\]  
where \(n\) is the total number of samples in the test set; \(\rho^{exp}\), \(\rho^{pred}\) are experimental and predicted densities, respectively. One of the key issues of the ML models is overfitting. Several standard techniques were implemented throughout the modeling pipeline to control and prevent overfitting. The most fundamental step taken was the strict separation of data into training and test sets. In this case, 80% of the data was used for training and hyperparameter tuning, and a completely independent 20% was held out as a test set. The final performance of each model was exclusively evaluated on this unseen test set. This provides an unbiased assessment of how well the model generalizes to new, previously unobserved data. A fivefold cross-validation scheme within a grid search framework was employed for hyperparameter tuning. This technique ensures that the selected hyperparameters are robust and perform well across the entire training dataset, rather than being tuned to a specific, arbitrary validation subset. It significantly reduces the risk of overfitting during the model selection phase. The machine learning algorithms chosen for this study have built-in mechanisms to control model complexity, which were optimized during the hyperparameter tuning process. In case of ElasticNet, that is an inherently regularized linear model that combines L1 and L2 penalties to constrain the model's coefficients, preventing them from becoming too large and complex. The XGBoost algorithm includes several hyperparameters that directly control overfitting, such as the maximum depth of the trees, the learning rate, and L1/L2 regularization. These were carefully tuned using the cross-validated grid search to find an optimal balance between model performance and complexity. Similarly, the hyperparameters for the neural network, such as the regularization parameter, were optimized to prevent the model from becoming overly complex. The final point was feature selection. Before training the models, a feature reduction process on the feature sets that involves removing sparse and highly correlated features was applied. By reducing the dimensionality and removing redundant information, the problem for the learning algorithm was simplified. To provide a comprehensive understanding of feature contributions, we assessed feature importance using three distinct methods. The first, feature weight, was determined directly from the XGBoost model using the \"gain\" metric. This metric quantifies the total reduction in the model's loss function contributed by all splits on a given feature across all trees in the ensemble. A higher total gain signifies that the feature was more influential in the model's internal decision-making process. The second method, permutation importance, provided a model-agnostic assessment of a feature's predictive power.