# Information Retrieval and Machine Learning Methods for Academic Expert Finding
## 4 Experimental Results and Discussion
### Experimental Setup for the IR Models
#### 4.3.1 Clustering  
To construct the term subprofiles of each researcher, we leveraged the article collections using two methods. Firstly, we utilized the K-means clustering algorithm [71]. Secondly, we employed latent Dirichlet allocation (LDA) [48] to generate K latent topics and subsequently performed clustering by assigning each article to its most probable topic. For the implementation of K-means, we relied on the scikit-learn Python library ([[https://scikit-learn.org](https://scikit-learn.org)]([https://scikit-learn.org](https://scikit-learn.org)), accessed 19 June 2023), whereas for LDA, we utilized the Python implementation available in the Gensim library ([[https://pypi.org/project/gensim](https://pypi.org/project/gensim)]([https://pypi.org/project/gensim](https://pypi.org/project/gensim)), accessed 19 June 2023).  
To address the large size of the document-term matrices in both collections (consisting of hundreds of thousands of rows representing articles and columns representing different terms), we employed a dimensionality reduction process. After removing stopwords and applying stemming, we disregarded terms that appeared in fewer than 750 articles or more than 90% of the articles. Additionally, to further reduce the dimensionality, we selected a maximum of 5000 of the most frequent remaining terms in the corpus as input for both K-means and LDA. However, it is important to note that the subprofiles derived from these algorithms encompassed all the terms present in the original articles once the clusters were generated.  
Regarding the determination of the number of clusters, \(K\), we conducted experiments using various approaches. Two of them were derived from conventional cluster analysis methods, while in the other three cases the value of \(K\) was fixed:  
* We employed the formula \(K=\sqrt{m/2}\)[72], where \(m\) represents the number of articles in the training set. Consequently, for PMSC-UGR and CORD-19, we obtained \(K\) values of 390 and 239, respectively.
* Another approach we considered was \(K=m*t/e\)[71], where \(t\) corresponds to the number of distinct terms appearing in the articles within the training set, and \(e\) represents the number of non-zero entries in the document-term matrix. For PMSC-UGR and CORD-19, we obtained \(K\) values of 54 and 29, respectively.
* We set \(K=110\), which aligned with the number of descriptors or categories in the second level of the Medical Subject Headings (MeSH) thesaurus ([[https://meshb.nlm.nih.gov/treeView](https://meshb.nlm.nih.gov/treeView)]([https://meshb.nlm.nih.gov/treeView](https://meshb.nlm.nih.gov/treeView)), accessed 4 July 2021).  
* We set \(K=20\), the number of medical specialties at St. George's University ([[https://www.sgu.edu/blog/medical/ultimate-list-of-medical-specialties](https://www.sgu.edu/blog/medical/ultimate-list-of-medical-specialties)]([https://www.sgu.edu/blog/medical/ultimate-list-of-medical-specialties](https://www.sgu.edu/blog/medical/ultimate-list-of-medical-specialties)), accessed 4 July 2021).
* We set \(K=10\) to test a case with a very low number of clusters.  
We also experimented with two extreme cases where no explicit clustering was performed: one where all the articles of a given author were grouped into a single profile (equivalent to using \(K=1\)), which we called the Joint model, and another where each article of an author formed an independent subprofile (equivalent to using \(K=\) articles in the collection), which was called the Atomic model.