# Multi-Feature-Enhanced Academic Paper Recommendation Model with Knowledge Graph
## 4 Experiments
### Evaluation Metrics  
This paper employs the leave-one-out method as the validation approach. It is a form of cross-validation where the dataset is divided into n subsets, with each subset used as the test set once while the remaining n - 1 subsets serve as the training set for training and validation. In the experiments in this paper, for each user, the most recent paper interacted with is taken as the test set, and the remaining papers interacted with constitute the training set. This method helps prevent overfitting and underfitting and can more accurately assess the model's generalization capability.  
For each user \(u_{i}\), 99 papers not interacted with by that user are randomly selected and ranked together with the test paper to determine the ranking position of the test paper among these 100 papers, thereby evaluating the recommendations' effectiveness. This paper selects the top 10 in the ranking as the recommendation list for evaluation, allowing a certain margin for error [33].  
Regarding this recommendation list, hit ratio (HR) and normalized discounted cumulative gain (NDCG) are used as metrics to judge the recommendation performance:  
\[HR=\frac{1}{N}\sum_{i=1}^{N}\mathit{hits}(i) \tag{25}\]  
In Equation (25), \(N\) represents the total number of users, and \(\mathit{hits}(i)\) indicates whether the test paper of the \(i\)-th user is present in the recommendation list. If the test paper is  
\begin{table}
\begin{tabular}{c c} \hline \hline
**Data Type** & **Quantity** \\ \hline Paper & 716,549 \\ Author & 976,544 \\ Year & 5 \\ Publishing Institution & 4023 \\ Author’s Affiliated Institution & 571,659 \\ \hline \hline \end{tabular}
\end{table}
Table 2: DBLP-v11 experimental dataset.  
in the list, \(hits(i)\) is 1; otherwise, it is 0. A larger HR (hit ratio) value indicates a higher recommendation accuracy rate of the model.  
\[NDCG=\frac{1}{N}\sum_{i=1}^{N}\frac{1}{\log_{2}(p_{i}+1)} \tag{26}\]  
In Equation (26), \(N\) represents the total number of users, and \(p_{i}\) represents the position of the \(i\)-th user's test paper in the top 10 recommendation list. If the test paper is not included in the recommendation list, \(p_{i}\) tends toward infinity. Therefore, a larger NDCG (normalized discounted cumulative gain) value indicates better recommendation performance by the model.  
To calculate precision and recall, this paper collects and constructs positive samples from the papers that the authors have actually published or cited and randomly selects negative samples from research areas where the user has not shown interest.  
\[Precision=\frac{TP}{TP+FP} \tag{27}\]  
Precision refers to the proportion of papers in the recommendation list that the user is genuinely interested in, relative to the total number of papers in the recommendation list. The formula is given by Equation (27), where TP (true positives) is the number of papers in the recommendation list that the user is interested in, and FP (false positives) is the number of papers in the recommendation list that the user is not interested in.  
\[Recall=\frac{TP}{TP+FN} \tag{28}\]  
Recall refers to the proportion of papers in the recommendation list that the user is genuinely interested in, relative to the total number of papers the user is interested in. The formula is given by Equation (28), where FN (false negatives) is the number of papers that the user is interested in but were not recommended in the recommendation list.