## 3 Results for the Research Questions
### RQ5: What Are the Datasets That Have Been Used for Expert Finding Systems? Expert finding systems are IR systems where interactions start with a user sending a query to the system, and then the system retrieves a list of experts that hopefully are relevant to the user's query. Normally, an IR research evaluates the system efficiency using a test data collection (datasets). The most popular evaluation criteria that are generally used are mean reciprocal rank (MRR), mean average precision (MAR), and precision after ranking N people (P@N). Earlier component of a test collection contains (1) a collection of documents, (2) a set of topics, as queries in the collection, and (3) a set of relevance judgments, also called a query relevance set. To test the implemented expert finding systems, developers can load the documents from the dataset (text collection) and send the topics to their systems one by one. Then the documents retrieved by the developed systems can be examined by their occurrence in the text collection. The efficiency of the developed models can be displayed by comparing them with baseline retrieval systems [4]. Expert finding systems uses the \begin{table}
\begin{tabular}{c c} \hline \hline
**Methods** & **References** \\ \hline Generative probabilistic models & [27; 40; 77; 109] \\ Discriminative probabilistic models & [31] \\ Voting models & [26; 60; 69] \\ Network-based models & [18; 24; 46; 47; 67; 84] \\ Hybrid methods & [16; 17; 25; 53; 54; 55; 56; 78; 70; 79; 82; 87; 88] \\ Others & [5; 15; 20; 22; 36; 39; 45; 50; 51; 59; 61; 63; 74; 78; 81; 83; 85] \\ \hline \hline \end{tabular}
\end{table} 
Table 8: Expert finding systems methods. existing test collections of IR [4; 122] and additional test collections shown in Table 9. The table shows the test collections, their characteristics, and the previous studies that used them. A particular test collection can be used for expert finding or expert profiling or both of them. The internal documentation of World Wide Web Consortium (W3C) is generated from public W3C sites and produced the W3C collection which was used in TREC 2005. Table 9 displays the statistics of W3C in TREC 2005. Documents in the dataset hold intranet pages, papers, email, etc. The predefined candidate experts are the participants in working groups, and the search topics are these working groups in W3C. This technique has some obvious limitations. For instance, the document dataset may hold data on other experts that are not on the set. Then, in TREC 2006, 55 topics are chosen for the final set [4]. In TREC 2007, the test collection is developed into a snapshot of the Australian Commonwealth Scientific and Industrial Research Organization (CSIRO). A main difference between the CSRIO and the W3C is that CERC does not offer a predefined set of expert candidates, and the number of experts on each topic is fewer. It is worth stating that the expertise judgment is made by the authors themselves with other encoded knowledge [123]. The academic test collections that provided for expert finding have been developed and made available for the public. For example, the UvT [124] and its enhanced version TU [41] contains employees of Tilburg University as candidate experts. The university official webpages used by UvT have some interesting characteristics and can be considered valuable resources of expert relevant documents. Commonly, faculties and scientists in universities or colleges are seen to have certain specialties on specified knowledge fields. Additionally, personal profiles on official webpages are mostly maintained by the individuals themselves. These self-introductions are updated when they make any progress on their research and knowledge learning. The TU dataset is currently the largest dataset accessible for benchmarking academia expert finding systems. It contains both Dutch and English documents, and it has five different ground-truths--GT1 to GT5. GT5 is considered the most recent and complete ground-truth [46].