# Active Learning for Medical Article Classification with Bag of Words and Bag of Concepts Embeddings
## 4 Conclusions  
This article examined the choice of text representation for semi-automated SLR systems based on active learning more thoroughly than before. Motivated by the insufficient attention that the bag of concepts representation has received so far, we implemented and experimentally evaluated several variations thereof, varying in the vocabulary formation method, in comparison with the ubiquitous bag of words representation, as well as the combined bag of concepts and -words representation. The experimental study included classification algorithms found to be the most useful for text classification by prior work, as well as several active learning setups, and was performed on a fairly large number of medical SLR case studies. Besides calculating the _WSS@95%_ metric of SLR workload savings, the performance profile methodology, which is known in the field of optimization but novel in the context of text classification and systematic literature reviews, was adopted to visualize the relative \begin{table}
\begin{tabular}{l l l l l l l} \hline
**Case Name** & **BoW** & **BoW** & **BoC** & **BoC** & **BoC+W** & **BoC+W** \\  & **(\%)** & **(WSS95)** & **(\%)** & **(WSS95)** & **(\%)** & **(\%)** & **(WSS95)** \\ \hline ACE Inhibitors & 69.16 & **0.2550** & 55.03 & 0.1935 & 69.94 & 0.2293 \\ ADHD & 29.26 & 0.6579 & 27.77 & 0.6672 & 27.15 & **0.6779** \\ Antihistamines & 81.88 & 0.1006 & 78.39 & 0.1399 & 80.13 & **0.1472** \\ Atypical Antipsychotics & 73.22 & 0.1157 & 79.53 & 0.1242 & 69.83 & **0.1522** \\ Beta Blockers & 71.96 & **0.2046** & 70.44 & 0.1639 & 75.15 & 0.1934 \\ Calcium Channel Blockers & 59.51 & 0.2247 & 67.62 & 0.1977 & 68.53 & **0.2634** \\ Estrogens & 64.46 & 0.3059 & 60.17 & **0.3211** & 50.14 & 0.3209 \\ NSAIDS & 54.46 & **0.3677** & 51.67 & 0.2857 & 55.86 & 0.3235 \\ Opiods & 58.18 & 0.3289 & 46.55 & 0.1712 & 53.61 & **0.3625** \\ Oral Hypoglycemics & 84.21 & 0.1082 & 81.05 & 0.1164 & 76.84 & **0.1280** \\ Proton Pump Inhibitors & 75.97 & 0.1810 & 71.84 & **0.2201** & 70.60 & 0.2079 \\ Skeletal Muscle Relaxants & 61.41 & 0.2899 & 68.95 & 0.1736 & 62.04 & **0.3144** \\ Statins & 70.48 & 0.2448 & 65.99 & **0.2906** & 66.67 & 0.2632 \\ Triptrans & 48.82 & **0.3393** & 42.92 & 0.2766 & 49.66 & 0.3286 \\ Urinary Incontinence & 68.66 & 0.2584 & 72.18 & 0.2281 & 66.90 & **0.2809** \\ \hline \end{tabular}
\end{table} 
Table 3: _WSS@95%_ metric values for the BoW and extended BoC/BoC+W text representations, active learning with cluster initialization and uncertainty sampling, and the RF algorithm. performance of different configurations of the learning process, permitting an easy and objective comparison. The obtained results confirm that uncertainty sampling combined with cluster initialization provides the most useful active learning strategy. The concept vocabulary formation method including not only concepts identified by the annotator but also concepts on the paths leading from those concepts to the ontology tree root turned out to provide the most useful variant of the bag of concepts representation. While it still did not outperform bag of words, the combined bag of concepts and -words representation turned out to provide the highest predictive value for the random forest and SVM algorithms, with the former being the most successful. While this superiority of BoC+BoW is an empirical observation based on comparative experiments rather than a formally justified property, it can be reasonably explained as resulting from the fact that BoC emphasizes the occurrence of terms related to the domain of the texts being classified, while BoW emphasizes their context provided by the occurrence of other words.