# Active Learning for Medical Article Classification with Bag of Words and Bag of Concepts Embeddings
## 2 Methods
### Bag of Concepts
#### 2.1.2 Feature Space  
Concepts in an ontology form a directed graph, which is considered a taxonomy tree. Concepts that lie close to the tree root have a broader meaning, and those that are farther away from the root have a more specific meaning. On the basis of concept positions in the ontology tree, we can categorize them. As in [28], we introduce a concept depth threshold. As in [28], we take into account the shortest path to the root as the basis for evaluating concept depth.  
The set of ontology concepts identified by the annotator among all articles in the set subject to transformation (\(A\)) can by directly used as the concept vocabulary (\(C\)) for the bag of concepts representation. Besides this direct approach, there are other possible vocabulary formation methods, such as those based on the filtering of the set of identified concepts by their occurrence frequency or based on the extension of the set of identified concepts by including additional concepts lying on the paths between their corresponding nodes and the root node of the ontology tree.  
For each concept (\(c\in C\)), the number of articles (\(N(c)\)) in which this concept occurs is determined. For a given article (\(a\in A\)), the number of occurrences (\(n(a,c)\)) of concept \(c\) is determined. Then, the corresponding feature value is obtained using one of the following approaches:  
**TF:**: \[w(a,c)=n(a,c)\cdot d(c),\] (1)
**TF-IDF:**: \[w(a,c)=(1+\log n(a,c))\cdot\log\frac{N}{N(c)}\cdot d(c),\] (2)  
where \(d(c)\) is the depth of concept \(c\). The first factor in the applied TF-IDF formula is a sublinearly scaled form of term frequency supposed to avoid assigning too much weight to multiple concept occurrences in the same article. For both TF and TF-IDF, feature values are scaled depending on concept depth. Notice that this reduces to the standard TF-IDF formula used in the BoW representation if \(d(c)=1\), \(c\) is a term in bag of words vocabulary, and \(1+\log n(a,c)\) is replaced by \(n(a,c)\).  
To extend the concept vocabulary, for each concept (\(c\in C\); we call it a seed concept), all concepts that are on the shortest path from \(c\) to the root of the ontology tree are identified. Those whose depth is greater than the fixed depth threshold level (\(TL\)) are used to build an extended set of concepts associated with \(c\)--\(EC(c)\) by adding them to concept \(c\). As a result, the set of concepts (\(EC\)) that is the basis for defining the feature space is of the following form:\[EC=\bigcup_{c\in C}EC(c). \tag{3}\]  
The value of the \(\bar{c}\in EC(c)\) feature for article \(a\) is then evaluated as  
\[w(a,\bar{c}) =c_{r}\cdot w(a,c), \tag{4}\] \[c_{r} =\frac{1}{d(c)-d(\bar{c})+1}. \tag{5}\]  
This method of evaluating bag of concepts feature values is similar to the approach described in [28], with the caveat that in [28], the term-frequency variant is used.  
Figure 1 illustrates the extended bag of concepts representation. In two articles, two seed concepts are identified--C7 in Article 1 and C9 in Article 2. If vector representations are created on the basis of concepts C7 and C9, they do not capture the similarity between the articles. However, if concept C1 is additionally used, then the similarity of the articles is reflected by their representations.  
The scheme of the process of evaluating text representations according to the bag of concepts method is presented in Figure 2.