# Citation Oriented AuthorRank for Scientific Publication Ranking
## 4 Experiments
### Experimental Evaluation  
(1) If AuthorRank can replace PageRank for publication ranking  
Based on our previous work [5], the PRP algorithm is better than other classical methods for publication ranking. In this paper, we tried to improve publication ranking by using AuthorRank to inform a topic-based PageRank algorithm. AuthorRank scores were computed by author ranking via different four methods: First Author, Last Author, Max Author, and Average Author. For evaluation, 104 topics were selected associated with review or survey papers. For each topic, we calculated each publication's ranking score in the dataset by the methods: PageRank, PRP_1, PRP_2, First Author, Last Author, Max Author, and Average Author. In Tables 5 and 6, we found that the best result was generated by PRP_2 both for MAP and nDCG, which verified that AuthorRank cannot replace the PageRank algorithm for publication ranking. We also found that topic-based PageRank (PRP_1) can significantly improve on the original PageRank algorithm, and that PRP_2 (which included the citation topic distribution) outperformed PRP_1 (which used the publication topic distribution only), especially for nDCG. Among AuthorRank results, we found the method of Average Author to be the best. The average contribution from all authors in an article is a more accurate guide to a paper's AuthorRank than the other methods. However, the evaluation shows that AuthorRank cannot simply replace PageRank. For each AuthorRank method (First Author, Last Author, Max Author, or Average Author), the parameter \(\alpha\), which controls the relative contributions of a PRP and AuthorRank to a publication's topical importance score, was trained from 0 to 1 with a step of 0.1. Tables 7 and 8 display results by the linear combination method for AuthorRank in informing publication ranking. The best result in the training results for each method is shown in the tables. As shown in the above tables, when the parameter \(\alpha\) = 0.9 in each method, AuthorRank combined with PageRank received the best result, meaning that AuthorRank can improve ranking results, but cannot replace the traditional link analysis ranking algorithm (PageRank with Priors). For MAP@n, the First Author method was better than the others when n \(\leq\) 30, but for n \(\geq\) 50, the Average Author method was better than all of the others. nDCG@n is a more important indicator in this research, for it tells the degree of (publication topic) importance. If an nDCG score is large, the target algorithm can prioritize the most important on the ranking list. In the tables, it is clear that the First Author method is better than the others when n \(<\) 30, but for n \(>\) 1000, the Average Author method is the best one. We also used significance testing to compare each method with the baseline PRP_2, and t \(<\) 0.001. Cobb-Douglas Form:  
This combination method is insensitive to small scores and sensitive to large scores. The parameter, \(\alpha\), was trained from 0 to 1 with a step of 0.1. The best results for each method are shown in the following tables. For MAP@n in Table 9, the Max Author method was better than the others when n \(\leq\) 10, and Average Author was the best when 30 \(<\) n \(<\) 50 and n \(>\) 1000, while for 100 \(<\) n \(<\) 100, the First Author method was better than the others. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline  & **PRP\_2** & \multicolumn{2}{c}{**First Author**} & **Last Author** & **Max Author** & **Average Author** \\ \cline{3-6}  & **\(\alpha\) = 0.9** & **\(\alpha\) = 0.9** & **\(\alpha\) = 0.9** & **\(\alpha\) = 0.9** \\ \hline MAP@10 & 0.2238 & 0.2271 & 0.2265 & 0.2266 & 0.2286 \\ MAP@30 & 0.1933 & 0.1948 & 0.1943 & 0.1941 & 0.1935 \\ MAP@50 & 0.1804 & 0.1812 & 0.1814 & 0.1805 & 0.1818 \\ MAP@100 & 0.1619 & 0.1623 & 0.1626 & 0.1626 & 0.1627 \\ MAP@1000 & 0.1159 & 0.1172 & 0.1088 & 0.1158 & 0.1182 \\ MAP@all & 0.0996 & 0.1002 & 0.1002 & 0.1005 & 0.1007 \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Linear Combination results (MAP).