# Information Retrieval and Machine Learning Methods for Academic Expert Finding
## 4 Experimental Results and Discussion
### Experimental Setup for the ML Models  
We now present the technical information about the experiments conducted using ML-based approaches. This information is summarized in the top part of the \"Expert recommendation\" column in Figure 1. Specifically, we utilized the scikit-learn and Keras libraries [75]. Scikit-learn ([[https://scikit-learn.org](https://scikit-learn.org)]([https://scikit-learn.org](https://scikit-learn.org)), accessed 15 January 2023) is an open-source ML library designed for Python, while Keras ([[https://keras.io](https://keras.io)]([https://keras.io](https://keras.io)), accessed 15 January 2023) is a Python-based deep learning API that runs on top of the TensorFlow machine learning platform. The scikit-learn library was applied to classical models, whereas the Keras library was employed for deep learning models, which are not available in the scikit-learn library.  
A total of 10 ML models divided into two categories were utilized. On one hand, we employed decision trees (DTs), K-nearest neighbors (KNN), logistic regression (LR), naive Bayes (NB), random forests (RFs), and stochastic gradient descent (SGD) linear models. On the other hand, we utilized multi-layer perceptron (MLP), convolutional neural networks (CNNs), long short-term memory (LSTM), and bidirectional long short-term memory (BiLSTM).  
We conducted numerous experiments and carefully selected a subset for reporting. For instance, we experimented with using the text of the articles as input data, both with and without feature selection. The feature selection involved removing terms that appeared in fewer than 750 articles or more than 90% of the articles, following the same selection criteria as used by the clustering algorithms; stop-word removal and stemming were always applied, regardless of whether feature selection was used or not. Additionally, we explored the conversion of the text into numerical vectors using two transformer-based architectures: bidirectional encoder representations from transformers (BERT) [76], which uses a pre-trained model with PubMed documents to extract the vector associated with the [CLS] special token as a contextual representation of the whole text, and a sentence-BERT (SBERT) model [77], which uses the fine-tuned model sentence-transformers/allenai-specter consisting of a fine-tuned pooling layer on top of a transformer encoder.  
However, our findings indicated that experiments using feature selection consistently yielded worse results compared to those without it. Therefore, we only report the results obtained without feature selection. Similarly, the experiments using BERT consistently performed worse than those using SBERT, so we solely present the results using SBERT. Moreover, we explored various configurations for some of the models, such as different numbers of layers and neurons in each layer for deep learning models. However, we only report the results of the best configuration for each model that we employed.  
Concerning the configuration parameters of the different classical models, we used all the default values for the DT, KNN, LR, and NB models. For the RF model, we set max_depth=20, which represents the maximum depth of the tree. Regarding SGD, we employed loss='modified_huber' because it allows probability estimates, which are not available for other loss functions.  
For the deep learning models, the configuration parameters were as follows:  
* For MLP, we used only one hidden layer with 8500 neurons for PMSC-UGR and 5000 for CORD-19 and trained for 20 epochs.
* For the CNN, LSTM, and BiLSTM models, we employed an embedding layer with pre-trained weights obtained from PubMed with a sequence length of 1024, an embedding size of 200, and 10 iterations.
* Additionally, for the CNN, we utilized one convolutional layer consisting of 2048 neurons, a kernel size of 3, and a stride of 1; for the LSTM, we used two LSTM layers, each with 200 neurons; lastly, for the BiLSTM, we employed two BiLSTM layers, one with 200 neurons and the other with 100 neurons.