# Short Text Clustering Algorithms, Application and Challenges: A Survey
## 3 Components of Short Text Clustering
### Wikipedia
#### 3.2.2 Deep Learning Measures
In text data, the convolution procedure is the product of a sentence vector and a weight matrix, with each element contributing to the whole. When attempting to extract features, convolution operations are performed. Features with a negative impact can be ignored, and only feature values with a significant effect on the work at hand are taken into account, thanks to pooling operations. The most common pooling operation is called'max pooling'. It involves picking the highest value in a particular filter space [103]. In this section, the literature is extensively reviewed to measure the similarity of short text based on deep learning measures. The most common models are listed below. Convolutional Neural Networks  
It is a popular deep learning approach; specific techniques use CNNs as feature extractors. Recently, the CNN has improved performance in many NLP applications, including relation classification [104], phrase modelling [105] and other traditional NLP tasks [106,107]. This is because the CNN is the most popular nonbiased model and applies convolutional filters to capture local features. A reliable feature function that extracts higher-level characteristics from constituent words or n-grams became necessary with the widespread use of word embeddings due to its capacity to represent words in a dispersed space. The self-taught CNN (STC2) was proposed by [PERSON] et al. [25] to learn implicit features from short texts for short text representation. Short text representation learning has also been implemented using neural-network-based techniques. The proposed model needs  
Figure 12: Basic structure of a neural network. two different raw representations of short text: binary coding representations of short text-based dimensionality reduction on term-frequency vectors and word embedding representations of short texts pre-trained from large external corpora. The input for CNNs is word-embedding representations of short texts, and the binary codes are utilized as data labels to train the CNN model. After the CNN is trained successfully, the deep representations for short text are taken from the last hidden layer of the CNN. However, short texts are usually sparse, so the deep features learned by neural network-based techniques may not accurately represent the short text. Recurrent Neural Networks  
Recently, neural networks such as recursive neural networks (RecNN) [108] and RNN [109] have demonstrated superior performance in creating text representations via word embedding. RecNN has high temporal complexity in building the textual tree, whereas RNN, which uses the hidden layer computed at the last word to represent the text, is a biased model in which later words are more prominent than early words [110]. By contrast, non-biased models may extract the learnt representation of a single text from all of the words in the text using non-dominant learning weights [25]. In recent years, RNNs have seen widespread adoption in research focusing on sequential data types, such as text, audio and video. However, when the input gap is wide, the RNN cannot learn important information from the input data. The problem of long-term dependencies is well-handled by the LSTM after gate functions are introduced into the cell structure [111]. Long Short-Term Memory  
The LSTM networks are a subclass of RNNs. RNNs can remember the previous words, capturing the context, which is crucial for processing text input. RNNs have the issue of long-term reliance because not all the past content is relevant to the following word/phrase. To counteract this issue, LSTMs are developed. Owing to the gates in LSTMs, the network can pick and choose which bits of information to remember [111]. The LSTM framework is widely used for determining how similar two sections of text are semantically [112]. To predict the similarity of sentences, [PERSON] et al. [113] utilized a network that combines LSTM and a CNN to create sentence embedding using pre-trained word embeddings. [PERSON] et al.