## 2 Literature Review
### Application of XGBoost and SHAP in Machine Learning-Based Educational Data Research  
Among the various machine learning techniques used in educational data analysis, Decision Trees and Random Forests have been prominently employed. Decision Trees automatically learn the relationships between features, yet single tree models are prone to overfitting and can exhibit performance variability depending on data sampling [57]. To overcome these limitations, Random Forests combine multiple trees to enhance prediction accuracy, although interpreting the importance of features remains a challenge [58]. XGBoost (eXtreme Gradient Boosting) addresses these limitations and is widely used in the analysis of educational data. XGBoost is a Gradient Boosting-based algorithm that combines multiple decision trees to maximize predictive performance [59]. This algorithm applies weight updates and regularization techniques to prevent overfitting while maintaining high predictive accuracy and provides stable performance even in datasets with nonlinear relationships and multicollinearity [60]. Furthermore, XGBoost utilizes ensemble learning techniques to combine several weak predictive models (decision trees) into a strong predictor. This iterative improvement process corrects errors from previous trees, thereby optimizing the overall model performance. Such ensemble methods are tailored to the complexity and characteristics of the data, showing exceptional performance even in complex datasets like education. Recent studies indicate that XGBoost outperforms traditional machine learning models, with an increase in the average coefficient of determination (R\({}^{2}\)) by 0.05-0.1 compared to Random Forests, and a reduction in the root mean square error (RMSE) by an average of 10-20% compared to regression analyses [58, 60]. Additionally, the mean absolute error (MAE) has been reported to decrease by an average of 15% compared to single decision trees, enabling more precise predictions [57, 58]. Performance evaluation of the XGBoost model utilizes various metrics, such as R\({}^{2}\), mean absolute error (MAE), mean squared error (MSE), root mean square error (RMSE), and mean absolute percentage error (MAPE). R\({}^{2}\) and MAPE are crucial indicators of predictive accuracy, with R\({}^{2}\) values above 0.7 typically indicating a model with high explanatory power and MAPE values below 10% signifying a highly accurate model [60, 61]. However, XGBoost is often considered a \"black box\" model, making it difficult to intuitively interpret the impact of individual features on the prediction process [60]. To address this, SHAP (Shapley Additive Explanations) analysis has been applied, which uses game theory to quantitatively assess and distribute the contributions of features [62, 63, 64]. SHAP analysis employs Shapley values to quantitatively evaluate the contribution of each feature, thus enhancing the transparency of the model's predictive process. By analyzing how the inclusion or exclusion of specific features affects prediction outcomes, SHAP helps intuitively interpret the results. SHAP enables the intuitive understanding of whether specific features increase (positive values) or decrease (negative values) the prediction and also allows for the analysis of interactions between features [65]. This approach clarifies how significant features interact and contribute to predictions in educational data. Recent studies have further discussed both the possibilities and limitations of applying explainable AI (XAI) in education. For instance, [PERSON] (2023) explores the socio-technical challenges of implementing XAI, arguing that transparency alone may not ensure meaningful educational outcomes [66]. Similarly, [PERSON] (2023) investigates how user characteristics, such as gender and educational background, influence the interpretation of XAI outputs, highlighting the need to account for user diversity in educational applications of XAI [67].