## 2 Pertinent State of the Art of Reddit-Related Research
### Reddit and Natural Language Processing  
One of the main NLP-anchored research directions found in Reddit-related literature concerns extracting the main point(s) of a text [16]. The umbrella term for these methods is _topic modeling_. In this research, typically, one attempts to extract \"main topics\" from the textual data. Next, the output is further modeled, e.g., with graph networks. In this context, we note that graph network model relations are easier to find for a small space of unambiguous features, which clearly point to specific things, phenomena, people, etc. Hence, of particular value are methods with a reasonably small output space. Such methods also deliver results that are easier to comprehend by humans. Therefore, let us now briefly discuss the selected most popular and recent topic modeling methods [17; 18; 19] and justify the choice of the method used in this work. The prime example of a \"classical\" methods is Non-negative Matrix Factorization (NMF). This method utilizes algebraic tools for matrix factorization to turn the word-document matrix into matrices of document-topic and word-topic dimensions. Algebraically speaking, given a set of text documents (\(D\)) and words (\(W\)), a matrix is built where rows are word embeddings with dimensions \(W\times D\) (i.e., a word-frequency matrix). The resulting matrix is always non-negative since word frequency has to be always positive (a word can only appear a positive number of times in a document). Next, using one of the methods for matrix factorization, two matrices are calculated. Their product would return the original \(W\times D\) matrix. The two matrices have dimensions \(W\times T\) and \(T\times D\) and provide information regarding which word belongs to which topics (\(T\)) and which topic belongs to which document, respectively. Hence, the output (the topics) are a combination of the words from the input documents. The second approach is Latent Dirichlet Allocation [20] (LDA). LDA is a generative model that uses Dirichlet distribution to assign documents their topics based on the words within them. This method is outlined in Algorithm 1. ```
input:k--number of topics to assign
1 fordocumentin documentsdo
2 forwordindocumentdo
3 randomly assign each word a topic from k topics
4\(N\leftarrow\) number of iterations;
5 whileN \(\
eq\) 0 do
6 fordocumentin documentsdo
7 forwordindocumentdo
8 compute \(p(w|t)\)--the proportion of all documents that are assigned to topic k (for a given word) compute \(p(t|d)\)--the proportion of words in documents that are assigned to topic t
9 Recalculate probability \(p(w|t,d)=p(w|t)\times p(t|d)\)\(N\gets N-1\);
```  
**Algorithm 1**LDA pseudocode (simplified). In LDA, there are two hyperparameters not mentioned in the simplified version of Algorithm 1. These are:  
* \(\alpha\), a document density factor (i.e., weight of topic in a document) controlling the number of topics expected in the document (the higher the value, the more topics are envisioned to exist in a document),
* \(\beta\), a topic word density factor (i.e., weight of a word in a topic) controlling the distribution of words per topic in the document (the higher the value, the larger number of words may belong to each topic). Similarly to NMF, the output of this approach consists of a probability that a given word is a part of a specific topic in a document. The output space consists, again, of different combinations of words from the input documents. We note that the two hyperparameters provide some form of \"manual control\" of the expected size of the output space. Other methods that can be used for statistical topic modeling are, for instance, the correlated topic model (CTM) [21] and the Pachinko Allocation Model (PAM) [22]. However, these two (and many others) share the same problem, namely the size of the output space. Since the output topic model is built from the word within the documents, the number of possible topics is enormous.