## 3 Theoretical Foundations  
This section covers the basic terminology and concepts of TM. The present section may be skipped by the reader familiar with the theoretical foundations of TM approach and LDA algorithm.  
A word is the basic unit of text and is entered in a vocabulary known as the bag of words, indexed as \(\{1,\ldots,N\}\). Words are normally represented as Boolean variables \(w\) such that \(w=1\) or \(w=0\). Given that a document is a sequence of words, the document \(m\)-th is represented as a vector \(d_{m}=\{w_{1},w_{2},\ldots,w_{N}\}\). A corpus is a collection of \(D=\{d_{1},d_{2},\ldots,d_{M}\}\) documents.  
A natural language is an unstructured entity that can present semantic ambiguities. It is, therefore, necessary to resort to a simplifying model that can transform textual information into numerical information. The vector space model is an effective paradigm. It is based on the geometric representation of textual documents as vectors so that they are incorporated into geometric objects used to determine the necessary distance concept to clustering. In the case of vector data, it is a consolidated practice to use the cosine value of the angle formed by the two vectors as a proxy of similarity. Summarizing the quantities in a TM method, they are as follows:  
* The corpus of documents \(D=\{d_{1},d_{2},\ldots,d_{M}\}\);
* A vocabulary of unique words \(W=\{w_{1},w_{2},\ldots,w_{N}\}\);
* A word-document matrix \(A\) of \(M\times N\) size, where the element of row \(m\) and column \(n\) is equal to the occurrences of the word \(w_{n}\) in the document \(d_{m}\). An example of matrix \(A\) is as follows.  
\[A=\begin{array}{c|cccc}&w_{1}&w_{2}&\ldots&w_{N}\\ \hline d_{1}&1&1&\ldots&1\\ d_{2}&1&0&\ldots&0\\ \ldots&\ldots&\ldots&\ldots&\ldots\\ d_{M}&0&1&\ldots&1\end{array}\]  
This notation corresponds to representing each document as a line vector. To the terms in the \(A\) matrix, the corrective term _tf-idf_ (\"term frequency\"-\"inverse document frequency\") is applied. This correction calculates the importance a word has within a document: the greater the importance, the more the word is recurrent in the text, and the lesser, the more the word is present within the other documents of the corpus. This coefficient depends on two factors:  
1. The plus factor \(tf_{m,n}\) takes into account the number of occurrences of the word \(w_{n}\) in the \(d_{m}\) document, normalized to the size of the document. Normalization is necessary in order not to give excessive weight to longer documents: \(tf_{m,n}=\frac{\sum w_{m,n}}{|d_{m}|}\)
2. The minor factor \(idf_{m,n}\) takes into account the percentage of documents that contain the word \(n\)-th. If the word is very frequent throughout the corpus of documents, it is of less importance within only one of these: \(idf_{m}=\log\frac{|D|}{|d:m\in d|}\)  
The complete corrective term is \((TF-IDF)_{m,n}=tf_{m,n}\cdot idf_{m}\). Documents within the \(A\) matrix are row vectors, the norm of which represents the size of the document. By applying the corrective term \(TF-IDF\), the arbitrary length of the documents is normalized to a fixed length.  
In the view of LDA, each document \(d_{m}\) contains \(N_{m}\) words in the corpus (or text dataset) \(D\). Each document \(d_{m}\) is also a mixture of \(K\) different topics and can be represented by a \(K\)-dimensional \"document-topic\" distribution \(\theta_{m}\). Each topic \(k\) is characterized by a mixture of \(V\) words, represented by a \(N\)-dimensional \"topic-word\" distribution \(\phi_{k}\).