## 3 Framework
### Data Transformation  
This component deals with the representation of weighted documents to emphasize the relevance of a specific word within the document collection. The weight of each word represents its importance degree. Depending on the weighting scheme adopted, the knowledge acquired from the collection might vary. Specifically, based on the document's statistical features and the desired granularity of the outcomes, one of the weighting schemes might outperform the others. To measure the relevance of the various terms in the document, each cell in the matrix \(X\) contains a _weight_\(x_{ij}\), that is, a positive real number indicating the importance of the term \(t_{j}\) appearing in the document \(d_{i}\). Ref. [41] proposes different weighting functions, combining a local term weight with a global term weight. By applying a weighting function to a collection \(D\), we obtain its weighted matrix \(X\). In particular, each element \(x_{ij}\) in the matrix represents the weight of the term \(t_{j}\) in the document \(d_{i}\) and is calculated as the product of a local term weight (\(l_{ij}\)) and a global term weight (\(g_{j}\)) (\(x_{ij}=l_{ij}\times g_{j}\)). A local weight \(l_{ij}\) refers to the relative frequency of a specific term \(j\) in a particular document \(i\), while the global weight \(g_{j}\) represents the relative frequency of the specific term \(t_{j}\) within the whole corpus \(D\). Three local term weights and three global term weights are included in ESCAPE. The local weights are _term-frequency_ (TF), _logarithmic term frequency_ (Log), and _Boolean_, while the global ones are _inverse document frequency_ (IDF), _entropy_ (entropy) and _term-frequency_ (TF\({}_{glob}\)). Their definition is reported in Table 1. The TF weight (L1 in Table 1), defined as \(tf_{ij}\), represents the frequency of term \(j\) in document \(i\). A similar measure is also reported by Log weight, which, however, evaluates the frequency of the term on a base-2 logarithmic scale. Lastly, the Boolean weight function is equal to 1 if the frequency was non-zero, and 0 otherwise. Intuitively, L1 and L2 give increasing importance to more frequent words, but L2 gives progressively smaller additional emphases to larger frequencies, while L3 is sensitive only to whether the word is in the document. After establishing the frequency of the different terms in the document, the resulting count has to be altered according to the perceived importance of that term by integrating the global importance of each word. To this end, the global weighting schemes reduce the weight of those terms that have a high frequency in a single document or that appear in many documents, which involves interesting variations concerning the relative importance of document frequency, local frequency, and global frequency. In particular, the global weight IDF (G1) measures how rare a term is within the corpora (\(|D|\)). This weight is calculated as the logarithm of the ratio between the total documents in (\(|D|\)) and the number of documents \(df_{j}\) containing the term \(j\). The more frequent a term is in the various documents, the lower its IDF will be. Entropy (G3) represents the real entropy of the conditional distribution given that the term \(i\) appeared. In documents, high-normalized entropy is considered good and low-normalized entropy is considered bad. Entropy, as a weighting scheme, is the most sophisticated one, and it is built on information theoretic ideas. If a term has the same distribution over different documents, it is given the minimum weight (i.e., where \(p_{ij}=1/ndocs\)), while if a term is concentrated in a few documents, it is given the maximum weight. In other words, entropy considers the distribution of terms over documents. Lastly, G3 represents the number of times in which the corresponding word j appears in the entire textual corpus D. It extends L1, considering the whole corpus. ESCAPE integrates six different term-weighting schemes to measure term relevance.