# Y-Rank: A Multi-Feature-Based Keyphrase Extraction Method for Short Text
## 6 Experiments
### Experimental Results and Analysis
#### 6.2.5 Overall Effectiveness  
To evaluate the overall effectiveness of Y-Rank, the model is applied to all five datasets. The evaluation results of Y-Rank are compared with nine advanced unsupervised methods (three statistical models: TfIdf, KPMiner and YAKE!, six graph-based methods: Text \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**Match Pattern** & **Impact Factor** & **P@5** & **R@5** & **F@5** & **MRR@5** & **MAP@5** & **Bpref@5** \\ \hline \multirow{4}{*}{Accurate} & Biased & 23 & 12.829 & 16.18 & 35.347 & 39.95 & 50.967 \\  & Unbiased, \(\beta\) = 0.75 & 24.2 & 13.348 & 16.916 & 38.261 & 44.044 & 55 \\ \cline{1-1}  & Unbiased, \(\beta\) = 0.5 & 24.4 & 13.501 & 17.095 & 39.842 & 45.589 & 56.8 \\ \cline{1-1}  & Unbiased, \(\beta\) = 0.25 & 24.6 & 13.701 & 17.295 & 40.478 & 46.344 & 57.1 \\ \hline \multirow{4}{*}{Fuzzy} & Biased & 63 & 34.559 & 43.885 & 48.985 & 27.125 & 84.917 \\  & Unbiased, \(\beta\) = 0.75 & 61.8 & 33.681 & 42.907 & 53.314 & 30.265 & 87.467 \\ \cline{1-1}  & Unbiased, \(\beta\) = 0.5 & 61.8 & 33.681 & 42.907 & 53.314 & 30.265 & 87.467 \\ \cline{1-1}  & Unbiased, \(\beta\) = 0.25 & 62.2 & 33.923 & 43.201 & 53.881 & 31.584 & 88.117 \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Comparison of Biased PR and Unbiased PR with different influencing factors (\(K\) = 5). Figure 4: Comparison with Y-Rank and YAKE! **(a)** Precision. **(b)** Recall. **(c)** F-Measure. **(d)** MRR. **(e)** MAP. **(f)** bpref. Rank, SingleRank, TopicRank, TopicalPageRank, PositionRank and MultipartiteRank) are compared. * Precision High accuracy means that the model has a high level of correctness. In compliance with Figure 5, it can be observed that Y-Rank has better evaluation scores in Precision than the other nine unsupervised methods. Compared with the second-best method, Y-Rank's evaluation score can be improved by 1.3% on average. Meanwhile, the smaller \(K\) is, the better improvement of Y-Rank is, which proves that Y-Rank is most suitable for the application scenarios of extraction with a few keyphrases. * Recall A high recall means that the model can find more true keyphrases. As shown in Figure 6, Y-Rank outperforms the other nine unsupervised methods in recall evaluation. Compared with the second best method, Y-Rank's evaluation score can be improved by an average of 1.5%. Among them, unsupervised methods based on graph methods generally have lower evaluation scores, especially TextRank, whose recall rate is almost 0% when \(K=5\). This indicates that the random walk algorithm has random directionality, which will lower the recall score. Figure 5: Comparison between Y-Rank and other models in Precision. (**a**) kp20k. (**b**) wikinews-fr-100. (**c**) termith-eval. (**d**) taln-archives. * F-Measure The F-measure is usually used to comprehensively evaluate the performance of a model. As shown in Figure 7, compared with the other nine unsupervised methods, Y-Rank has better performance in terms of F-measure. Compared with the second-best method, Y-Rank's evaluation score can be improved by an average of 1.4%. Obviously, the improvement of Y-Rank is more significant when k is smaller, which further proves that Y-Rank is most suitable for the application scenarios of extraction with a few keyphrases. Figure 6: Comparison between Y-Rank and other models in Recall. (**a**) kp20k. (**b**) wikinews-fr-100. (**c**) termith-eval. (**d**) taln-archives. * Mean Reciprocal Rank (MRR) The MRR is the average of Precision, which means more stable ranking recommendation when the score is higher. The evaluation results of MRR are presented in Figure 8, which demonstrates that Y-Rank outperforms the other nine unsupervised methods and achieves an average improvement of 3.5% in MRR. Compared to other methods, Y-Rank is a mixed method that combines statistical and semantic features. The experiment has demonstrated that Y-Rank can effectively improve the quality of recommendation ranking. Figure 7: Comparison between Y-Rank and other models in F-Measure. (**a**) kp20k. (**b**) wikinews-fr-100.