# Short Text Clustering Algorithms, Application and Challenges: A Survey
## 3 Components of Short Text Clustering
### Wikipedia
#### 3.2.2 Deep Learning Measures
[114] suggested an LSTM design to measure the semantic similarity between two supplied sentences. Tree-LSTM is then trained over the parse tree to provide sentence representations. A neural network is trained with these phrase representations and determines the absolute distance and angle between the vectors. Figure 13: Structure of a multi-neural network. Bidirectional Long Short-Term Memory  
Bidirectional RNNs are just two independent RNNs combined. This structure enables the networks to contain both backward and forward sequence information at each time step. Bi-LSTMs use two LSTMs that run in parallel in order to fully capture the context [102]. By running the inputs in two directions, one from the past into the future and the other from the future into the past, one may preserve information from both the past and the future simultaneously, making this method superior to the more common unidirectional one. Like NLP, there are occasions when knowing what comes next is just as important as knowing what came before. To estimate the model's semantic similarity, [PERSON] and [PERSON] [115] presented a hybrid architecture based on Bi-LSTM and CNN to fully capture the context. The approach takes advantage of Bi-LSTM to perform context modelling. Two LSTMs' hidden states are used to generate vectors that are then compared using a comparison unit, resulting in a model of paired word interactions. [PERSON] and [PERSON] presented a MaLSTM [72], which is a Siamese deep neural network that uses LSTM networks with connected weights as sub-modules to learn presentations for sentences. MaLSTM receives sentence pairs, initially expressed as word embedding vectors, as inputs. MaLSTM is trained to utilize a loss function based on the Manhattan distance to learn new representations for sentences. Bidirectional Encoder Representations in Transformers (BERT)  
It is a computational method that allows machine learning models to be trained on textual data. BERT learns contextual embeddings for words as a result of the training procedure [116]. Following the computationally expensive pretraining, BERT can be fine-tuned with lower resources on smaller datasets to optimize its performance on specific tasks. It refers to bidirectional encoder representations in transformers. In contrast with modern theories of language representation [117], pretraining deep bidirectional representations from the unlabelled text is the goal of BERT, and it does so by concurrently conditioning the left and right context across all layers [118]. For this reason, the pre-trained BERT model may be fine-tuned with a single extra output layer to provide state-of-the-art models for various tasks, including Q&A and language inference, without significant task-specific architecture alterations. Many different types of NLP tasks have been improved using language model pre-training [119]. Paraphrasing [120] and natural language inference [121] are examples of sentence-level tasks that aim to predict relationships between sentences by analysing them holistically. Named entity recognition and Q&A are examples of token-level tasks that require models to produce fine-grained output at the token level [122], as shown in Figure 14.  
In Table 3, the deep learning similarity measure works of literature are illustrated and summarized. Figure 14: Pre-training and fine-tuning procedures for BERT [116].