## 3 Results
### Feature Importance Based on SHAP Values  
We conducted a SHAP analysis for the CatBoost model to enhance its interpretability and ascertain features' contributions to the prediction process. Figure 5 reveals the range and distribution of lexical features' impacts on the familiarity rating predictions for domain  
Figure 4: Residual plots for six regression models on the training dataset through ten-fold cross validation with 75% training samples and 25% holdout samples.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**MAPE** & **MSE** & **MAE** & **RMSE** & \(\mathbf{R^{2}}\) \\ \hline
0.09 & 0.35 & 0.44 & 0.59 & 0.11 \\ \hline \hline \end{tabular}
\end{table}
Table 5: Performance of the CatBoost method on the testing dataset.  
concepts. Each data point in Figure 5 represents a SHAP value assigned to a concept and its corresponding features within our data set. The vertical axis indicates lexical features' importance in descending order; the horizontal axis shows features' degree of impact on familiarity ratings. SI_C1, CL, and SN were the three most meaningful variables for prediction. Regarding SI_C1, the data points in dark blue were primarily in the negative area; that is, a lower value for the first principal component of semantic information in domain concepts corresponded to a lower SHAP value in the prediction model. Then, taking the multiple correlation coefficient (R2 = 0.067, F-statistic = 44.68, and _p_-value < 0.001) between domain concepts' familiarity and SI (i.e., SI_C1, SI_C2 and SI_C3) into consideration, we can infer that domain concepts with little SI are less familiar when perceived. This outcome aligns with our expectation that more semantic concept-related information (i.e., derived from a vast knowledge base) affords individuals more opportunities to understand its usage and differentiate it from other concepts. In contrast, the yellow data points for CL and SN were distributed in the negative region of SHAP, indicating that domain concepts with more characters and strokes tend to be less familiar to students. This observation substantiates the negative effects of word length and SN on students' lexical processing and word recognition [31; 51]. Familiar words have fewer word-length and stroke-number effects versus unfamiliar words because familiarity is more readily apparent in whole-word processing [52].  
To further explore how lexical features and their interactions quantitatively affect the prediction of domain concept familiarity, we performed SHAP dependency analyses on the three most important lexical features (Figure 6). Ignoring the colors in Figure 6, the dependency plots show variations in associated SHAP values for SI_C1, CL, and SN. Figure 6a reveals that the SI_C1 with various values of SI_C2 had a joint positive impact on familiarity prediction when SI_C1 was larger than zero. Interestingly, SI_C2 was complementary to SI_C1 when SI_C1 was lower than zero. According to the point distribution in the CL and SI_C1 dependency plot in Figure 6b, a CL value greater than three may compromise familiarity prediction regardless of the shift in SI_C1. No discernible pattern emerged between CL and SI_C1 in terms of SHAP values. Relatedly, as pictured in Figure 6c, an SN value beyond five may adversely affect familiarity prediction. Rising values of SI_C2 might alleviate this consequence to some degree.  
Figure 5: Summary plots of lexical feature importance in the best prediction model.