# Short Text Clustering Algorithms, Application and Challenges: A Survey
## 3 Components of Short Text Clustering
### Gibbs sampling  
It is a powerful and simple technique in statistical inference. It is a Monte-Carlo-Markov-chain algorithm. Gibbs sampling produces a sample from a joint distribution when only conditional distributions of each variable can be efficiently computed. Many researchers have used this technique for the LDA [82; 83; 84]. Dirichlet Multinomial Mixture (DMM)  
The other technique is based on model-level improvements, in which standard procedures impose additional constraints on model assumptions to generate topics. Assuming that in traditional models, each document is composed of several topics, given that each short text document has only a few words. The DMM [66] model assumes that there is only one topic covered in each document. The constraints on these models are excessive. The number of relevant topics depends on the information in the various texts. As a result, simply putting such restrictions has the potential to cause noise, so this technique may be less effective and less generic. A corpus is a collection of search results composed of \(D\). The character \(D\) denotes the number of documents in the corpus. Each \(\overset{\rightarrow}{\mathrm{d}}\) includes a group of words (\(w\in 1,2,\ldots,N_{d}\)). Figure 10 describes the DMM model \(D\). The DMM models work according to the following process:  
Figure 9: Graphical illustration of LDA. 1. Sample \(\theta=(\upkappa_{1},\upkappa_{2},\ldots,\upkappa_{n})\) from a Dirichlet distribution with parameter \(\upalpha=(\lambda_{\upkappa_{1}},\upkappa_{\upkappa_{2}},\ldots,\upkappa_{ \upkappa_{n}})\). 2. For each topic \(t=1,2,\ldots,n_{T}\), the sample from a Dirichlet (\(\beta\)), where \(\beta=\big{(}\lambda_{\beta 1},\lambda_{\beta 2},\ldots,\lambda_{\beta n}\big{)}\),
3. For a \(\overset{\rightarrow}{\mathbf{d}}(d\ e\{1,\ldots,\ \mathbf{N}\})\): in \(=d\). 1. A topic \(z_{n}\in\{1,2,\ldots,T\}cc\) is selected from multinomial(\(\upalpha\)), where \(\upalpha=(\upkappa_{1},\upkappa_{2},\ldots,\upkappa_{n})\) represents the topic distribution in the corpus. 2. The word count \(N_{d}\) is selected, and a word \(w_{d}\) from d from multinomial(\(\beta\)) is also independently selected, where \(\beta=(\beta_{\upkappa 1},\beta_{\upkappa 2},\ldots,\beta_{\upkappa n})\) represents the word topic distribution in the corpus. A DMM-based method for STC was suggested by [66]. However, how to create an effective model remains unclear. Based on BOW, most of these methods are trained, which are shallow structures that cannot maintain semantic similarities [25]. In Equation (2), the probability of observed data \(D\) is calculated and maximized to infer the latent variables and hyperparameters [66]:  
\[p(d|\upalpha,\beta)=\sum_{t=1}^{T}\upalpha_{t}\frac{N_{d}!}{\prod_{w=1}^{V}N_ {d}^{w_{t}}!}\prod_{w=1}^{V}\beta_{wt}^{N_{d}^{w}}, \tag{2}\]  
where \(\upalpha\) denotes the topic Dirichlet prior parameters and the distribution of words over topics from the Dirichlet distribution; for a specific \(\beta\), the vocabulary size is represented by the letter \(V\). The frequency of the word \(W\) in \(d\) is \(N_{d}^{w}\). Additionally, the number of words is \(N_{d}\). For corpus-level topic distributions, the Dirichlet-multinomial pair is \((\upalpha,\theta)\). A DMM output matrix has rows for documents and columns for topics. The labelled one is assigned to the cell with the coordinates (_i,j_) if the document \(d_{i}\) belongs to the topic \(t_{j}\). Latent Semantic Analysis (LSA)  
This model is a technique in text representation that can be used for modelling the conceptual relationship among several documents based on their set of words, which can be computed as semantic information using this model. One of the methods for improving the text representation model is using semantic information [85]. This concept is founded on the premise that words with lexical distinctions frequently appear in similar documents and have similar meanings.