# Active Learning for Biomedical Article Classification with Bag of Words and _FastText_ Embeddings
### Related Work  
Active learning for text classification has been a relatively popular research topic during the last two decades, and reviewing even a fraction of the most impactful prior work would require more space than this entire article. To provide a minimal necessary research context, Table 1 lists a small number of selected particularly closely related publications, summarizing them along the following dimensions:  
**Representations:** text representation methods;
**Algorithms:** classification algorithms;
**AL Setup:** active learning setup: initialization, query selection, and early stopping criteria;
**Evaluation:** predictive performance measures and evaluation procedures; \begin{table}
\begin{tabular}{p{113.8 pt} p{113.8 pt} p{113.8 pt} p{113.8 pt} p{113.8 pt}} \hline \hline
**Ref.** & **Representations** & **Algorithms** & **AL Setups** & **Evaluation** & **Data** \\ \hline
[14] & BOW (TF-IDF) & SVM & initialization: random; query: margin; stopping: none & error, precision–recall brakeven; train–test split & Reuters, subset of Newsgroups \\ \hline
[15] & BOW & maxent & initialization: cluster; query: uncertainty; stopping: none & accuracy; \(k\)-fold CV brakeven & subset of Newsgroups, WebKB \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Related work. The common features of most of this prior work are using the bag of words text representation and uncertainty sampling for query selection. The latter appeared in several different variations, depending on the adopted uncertainty measure or some specific properties of the classification algorithm used. This is the case, in particular, for [17] using the predictions of multiple models on data packets to estimate uncertainty, [18] proposing an uncertainty sampling strategy specifically designed for a regular expression-based text classification algorithm, and [21] using a Monte-Carlo dropout uncertainty estimation technique for a deep neural network model. Researchers using the SVM algorithm often adopt query selection strategies based on the classification margin (distance to the decision boundary) [14; 20]. In some studies of active learning for systematic literature reviews a query strategy that selects articles with the highest probability of being relevant to the study topic was used [16; 19; 22; 23]. This is referred to as minority sampling in Table 1 and thereafter. It can be easily noticed that initial training set selection has not received very much attention in prior research on active learning for text classification. Out of ten publications listed in Table 1 only one uses a non-random clustering-based initialization method [15]. The same can be said about early stopping criteria: most prior work continues the learning process until a predetermined number of iterations or instances labeled, with [18] being a single exception. While some of those studies use \(k\)-fold cross-validation for predictive performance evaluation, most settle for the less reliable train-test split approach. More importantly, predictive performance indicators are usually limited to those measuring the quality of class label predictions (e.g., accuracy, precision, recall, F1) rather than class