# Active Learning for Medical Article Classification with Bag of Words and Bag of Concepts Embeddings
## 3 Experimental Study
### Evaluation Results  
Before examining the utility of different text representations, we identify the most useful active learning setup based on the performance profiles presented in Figure 3. The presented curves were obtained using the bag of words representation, all three classification algorithms (NB, SVL, and RF), and three different combinations of active learning initialization and query selection (random-uncertainty (ru), cluster-random (cu), and cluster-uncertainty (cu)). The following observations can be made:  
**Active learning setups:** The cluster-uncertainty and random-uncertainty active learning setups outperform the cluster-random setup for each algorithm. There is not much of a difference between these two, but the cluster-uncertainty setup has a slight advantage; therefore, this is the setup selected for subsequent experiments. **Classification algorithms:** The RF algorithm clearly outperforms the other two, and NB is better than SVL. Figure 3: Performance profiles for the BoW text representation, different active learning initialization methods (c—cluster; r—random), different active learning query selection strategies (u— uncertainty; r—random), and different classification algorithms (RF—random forest; NB—naive Bayes; SVL—SVM with linear kernel). For example, RF-cu designates the random forest algorithm applied with cluster initialization and uncertainty sampling. The four different variants of the bag of concepts representation with the random forest algorithm and the cluster-uncertainty active learning setup are compared in the performance profiles in Figure 4. Supplementary Material Figures S1 and S2 present the corresponding profiles for the linear SVM and naive Bayes algorithms, respectively. Each plot includes the performance profiles for the bag of words representation and the four variants of the combined bag of concepts and -words representation. One can observe the following:  
**RF:** The best results were obtained for the BoC+W variants, with the extended variant BoC+W having a slight advantage over the others; all BoC variants performed worse than the BoW variant, which achieved significantly worse results than BoC+W variants. **SVL:** BoC+W representations showed behavior quite similar to that of BoW; performance profiles of BoC representations definitely stand out from the profiles of the other variants. **NB:** Adequate results were obtained only for the BoW variant. Supplementary Material Figure S3 presents performance profiles for BoW and the normal and extended BoC representations with the cluster-uncertainty active learning setup and the random forest classification algorithm. Supplementary Material Figure S4 similarly compares BoW with extended and extended short BoC. The following can be observed:  
**Normal versus extended normal:** in the case of BoC representations, the extended variant is better than the normal variant; the use of concatenation of BoC with BoW mitigates the impact of the extended set of concepts on the classification result. **Extended normal versus extended short:** Short variants yield slightly worse classification results. Considering concepts that are located on the ontology tree paths from the concept found by the annotator to the tree root, in the creation of the numerical representation of articles, enables the identification of similarity between articles, even though these articles do not contain the same ontology concepts. This mechanism is presented in Figure 1.  
Figure 4: Performance profiles for BoW and different BoC/BoC+W variants: normal (n), short (s), extended (x), extended short (xs) for active learning with cluster initialization and uncertainty sampling query selection, and the RF algorithm. With the cluster-uncertainty active learning setup and the extended bag of concepts representation, which can be generally considered the most useful based on the results presented above, Supplementary Material Figure S5 compares the performance profiles of different classification algorithms.