## The Investigation Framework
### Framework Overview
#### 4.2.4 Module M2-Feature Extractor in Details  
Module _M2-Feature Extractor_ receives the training set \(TR\) extracted from the overall set of pre-processed messages \(T_{p}\). Its goal is to give different representations of each labeled message \(lm_{p}\in TR\), based on the basic and combined feature patterns reported in Table 2.  
The module generates nine different versions of the training set \(TR\), one for each feature pattern, denoted as \(TR^{U}\), \(TR^{B}\), \(TR^{T}\), \(TR^{Q}\), \(TR^{U,B}\), \(TR^{B,T}\), \(TR^{T,Q}\), \(TR^{U,B,T}\) and \(TR^{U,B,T,Q}\). These are intermediate results, necessary to generate the actual output of the module, i.e., a pool of feature vectors \(FV^{f/p}\) (where \(fp\) denotes the feature pattern, as in Table 2): each \(FV^{fp}\) contains a feature vector for each topical class. Let us start by describing the generation of \(TR^{f/p}\). * First, \(TR^{U}=TR\), because uni-grams coincide with single terms in vectors \(lm_{p}.d_{4}\) representing message texts (i.e., \(lm^{U}.d=lm_{p}.d_{4}\)). * Each \(lm^{B}\in TR^{B}\) derives from the corresponding basic version \(lm_{p}\in TR\), where \(lm^{B}.d\) is a vector of bi-grams. Similarly, training sets \(TR^{T}\) and \(TR^{Q}\) contains descriptions \(lm^{T}\) and \(lm^{Q}\) of messages whose vectors \(lm^{T}.d\) and \(lm^{Q}.d\) are vectors of tri-grams and quad-grams, respectively. * Training sets based on combined feature patterns are derived from training sets based on basic patterns. Given a message \(lm_{p}\), its representations based on combined feature patterns are obtained as follows:
* for \(lm^{U,B}\in TR^{U,B},lm^{U,B}.d=lm^{U}.d\times lm^{B}.d\);
* for \(lm^{B,T}\in TR^{B,T},lm^{B,T}.d=lm^{B}.d\times lm^{T}.d\);
* for \(lm^{T,Q}\in TR^{T,Q}\), \(lm^{T,Q}.d=lm^{T}.d\times lm^{Q}.d\);
* for \(lm^{U,B,T}\in TR^{U,B,T},lm^{U,B,T}.d=lm^{U}.d\times lm^{B}.d\times lm^{T}.d\);
* for \(lm^{U,B,T,Q}\in TR^{U,B,T,Q}\), \(lm^{U,B,T,Q}.d=lm^{U}.d\times lm^{B}.d\times lm^{T}.d\times lm^{Q}.d\). Once the training sets are prepared, for each of them (that generically we will denote as \(TR^{fp}\)) the module performs the following activities. 1. For each message \(lm_{i}^{fp}\in TR^{fp}\), a set of terms \(s_{i}^{fp}\) is derived from the vector of terms \(lm_{i}^{fp}.d\): \[s_{i}^{fp}=\{t|\exists pos(1\leq pos\leq|lm_{i}^{fp}.d|\wedge lm_{i}^{fp}.d[ pos]=t)\}\] so that duplicate occurrences of a term in \(lm_{i}^{fp}.d\) becomes a unique occurrence in \(s_{i}^{fp}\). 2. The frequency matrix \(Freq^{fp}[t,c]\) is built, where \(t\) is a term and \(c\) is a class (or topic). To obtain the \(Freq^{fp}\) matrix, first of all the module builds \(tc_{i}^{fp}\), a set of (term, class, message identifier) triples \((t,c,i)\) obtained as \[tc_{i}^{fp}=s_{i}^{fp}\times\{lm_{i}^{fp}.at\}\times\{i\}\] that is, by performing the Cartesian product among the set \(s_{i}^{fp}\) of terms in the message, the singleton set of the class label (topic) \(lm_{i}^{fp}.at\) associated to the message and the singleton set containing \(i\) (i.e., the identifier of the message). All the \(tc_{i}^{fp}\) sets are united into the \(TC^{fp}\) set, i.e., \(TC^{fp}=\cup_{\forall lm_{i}^{fp}\in TR^{fp}}(tc_{i}^{fp})\). Each single item of the \(Freq^{fp}\) matrix is then computed as follows: \[Freq^{fp}[t,c]=|\{(t_{j},c_{l},i_{j})\in TC^{fp}|t_{j}=t\wedge c_{j}=c\}|\] where we count, for each term \(t\) and each class \(c\), the number of different documents, associated to class \(c\), which term \(t\) occurs in (the third element \(i_{j}\) in triples is necessary to distinguish term occurrences coming from different messages). 3. For each term \(t\) in each class \(c\), the module computes the \(Weight(t,c,C)\) score (defined in Formula 1), where \(C\) is the set of all class labels.