## 3 Results
### Predictive Performance of Regression Models  
Table 4 presents the candidate regression models' familiarity predictions on the training dataset through ten-fold cross validation with 75% training samples and 25% holdout samples. All methods returned promising results as evidenced by MAPE values consistently lower than 0.13. As expected, the CatBoost method showed the best performance, with the lowest average MAPE (0.09) and the highest average R\({}^{2}\) value (0.02) highlighted in bold in Table 4. It is noteworthy that several approaches, such as AdaBoost and LightGBM, produced negative R\({}^{2}\) values. Negative R\({}^{2}\) scores may result from parameter constraints that cause the sum of squared errors to exceed the sum of squared deviations of the dependent variable from its mean [49; 50]. To proceed with an in-depth performance analysis of regression models on the training samples, the residual distributions between the predicted and actual values for six tree-based ensemble regression models are illustrated in Figure 4. In the cross-validation procedure, it is observed that the R\({}^{2}\) values of the six models on the training samples are much higher compared to those on the holdout samples. The CatBoost method had an acceptable reliability and robustness, as indicated by its R\({}^{2}\) values of 0.612 on the training samples and 0.112 on the holdout samples. Its residuals tend to adhere to a normal distribution, shown as the right side of Figure 4a; however, there are a few high residual points. This observation indicated that the Catboost method demonstrated strong predictive performance in estimating familiarity ratings for domain concepts, except for several cases of extremely high familiarity ratings. As for the AdaBoost method, it produced lower R\({}^{2}\) values on both the training and holdout samples, suggesting a suboptimal fitting performance. Both LightGBM and XGBoost methods had R\({}^{2}\) values greater than 0.8 on the training samples but less than 0.1 on the holdout samples, indicating overfitting due to excessive learning of noise and details in the training samples and thus resulting in poor generalization performance on the holdout samples. Therefore, the CatBoost method was best suited to predicting domain concepts' familiarity.  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline \multirow{2}{*}{**Regression Model**} & \multicolumn{5}{c}{**Metrics**} \\ \cline{2-6}  & **MAPE** & **MSE** & **MAE** & **RMSE** & **R\({}^{2}\)** \\ \hline CatBoost & **0.09** & **0.39** & **0.45** & **0.62** & **0.02** \\ \hline RF & 0.10 & 0.39 & 0.47 & 0.62 & 0.01 \\ \hline GBoost & 0.10 & 0.39 & 0.45 & 0.62 & 0.01 \\ \hline AdaBoost & 0.10 & 0.40 & 0.49 & 0.63 & \(-\)0.01 \\ \hline LightGBM & 0.10 & 0.42 & 0.48 & 0.65 & \(-\)0.07 \\ \hline XGBoost & 0.10 & 0.45 & 0.49 & 0.67 & \(-\)0.15 \\ \hline \hline \end{tabular}
\end{table}
Table 4: Performance comparisons of familiarity prediction methods on the training dataset through ten-fold cross validation.  
In addition, Table 5 summarizes the CatBoost method's performance on the testing dataset, reinforcing this model's robustness. The correlation analysis was significant (\(r=0.36,p<0.001\)) and implied that the CatBoost method can accurately predict domain concepts' familiarity ratings based on lexical features.