## 3 Framework
### Data Processing and Characterization  
In order to deal with the textual data analysis problem in a more efficient way, ESCAPE includes two steps to transform and characterize the textual corpora: (i) _document processing_ and (ii) _statistics definition and computation_. These steps are performed automatically without any user intervention. **Document processing. ** In this block, five steps are performed sequentially as interrelated tasks:  
1. _document splitting_: documents can be split into sentences, sections, or analyzed in terms of their entire content, according to the next analytic task. While short documents, such as emails or tweets, are represented with a single vector, longer documents can be decomposed into paragraphs or sentences; hence, multiple vectors are required. Choosing the best procedure depends on the goals of the analysis: for the clustering task (as in the scope of this paper), the entire document is analyzed in terms of its entire content; for sentimental analysis, document summarization, or information retrieval, smaller units of text such as paragraphs might be more appropriate;
2. _tokenization_: this is the process of segmenting a text or texts into tokens (i.e., words) by the white space or punctuation marks within the same split;
3. _case normalization_: capitalization is very useful to humans in the reading phase. However, in many analytics tasks, a capital word at the beginning of a sentence should not be treated differently from the same lower-case word that appears elsewhere in the document. For this reason, this step converts each token to completely upper-case or lower-case characters;
4. _stemming_: each token is mapped into its own root form. This includes the identification and removal of prefixes, suffixes, and pluralization;  
Figure 2: The ESCAPE system architecture. _stopwords removal_: stopwords are grammatical words which are irrelevant to text contents (e.g., articles, pronouns, prepositions), so they need to be removed for more efficiency. These common words can be discarded before the feature-generation process. The document's main themes are depicted with the bag-of-words (BOW) representation, which shows the most meaningful frequent terms in terms of multiplicity without caring about grammar rules and word order. Information about the frequency of each word in a document can be useful to reduce the size of the dictionary. For example, the most-frequently occurring words in a document are often stop words and should be deleted. Terms that are very rare should also be deleted, as they are often typos. The remaining most-common words are the most important and significant. In general, the smaller the dictionary, the greater the intelligence to capture the most- important words [39]. Tokenization and stemming are two steps that help us reduce the size of the dictionary. After defining the set of words, the next step is to convert the document collection into a matrix structure format. Let \(D=\{d_{1},d_{1},\ldots,d_{|D|}\}\) be a corpus of documents, and \(V=\{t_{1},t_{2},\ldots,t_{|V|}\}\) be the set of distinct terms used at least once in the textual collection. The corpus \(D\) is represented as a matrix \(X\), named the _document-term_ matrix, in which each row corresponds to a document in the collection, and each column, one for each \(t_{j}\in V\), corresponds to a term in the vocabulary. **Statistics definition and computation.