# Short-Text Semantic Similarity (STSS): Techniques, Challenges and Future Perspectives
## 8 Discussion  
This systematic literature review presents the recent trends in the domain of short-text semantic similarity. Semantic similarity is the most widely used area in natural language processing tasks. Understanding the semantics of sentences is still challenging for machine learning algorithms, as there are various sentence drawbacks, such as anomalies, sparsity, semantic ambiguity, polysemy, and the number of sentence structures, such as word order, punctuation, spelling errors, and so on. These drawbacks make the standard methods more complicated to identify the hidden semantics of sentences. We included approximately 42 deep learning-based research articles to answer the first research question (RQ). We have noted that from the year 2017 to 2021, a number of studies used convolutional neural networks (CNN) with traditional methods, such as the support vector machine and Word2 Vec, for different STSS techniques, such as short text classification, short answers, short text categorization, short tweets, and many more. Many research studies, however, made extensive use of Twitter, AG news, Chinese Wikipedia, knowledge base, Probase, Google snippets, and the Hewlett Foundation for short answer scoring. In addition, the recurrent neural network (RNN) has a significant impact on short-text classification and short-sentence similarity. For sentence similarity tasks, they used semantic matching techniques, as well as character-level representations. From the year 2018 until today, language models like the BERT, ELMO, KeyBERT, Roberta, SciBERT, BioBERT, and distilBERT achieved the highest performance in understanding the syntax and semantics of a short sentence. These models employ pre-training and fine-tuning techniques to train on domain-specific and domain-independent datasets. These studies, however, heavily relied on domain-specific and multi-domain datasets to determine similarity. To answer the second research question, we reviewed approx. 76 research studies to understand the contextual information of the short text. Earlier models, such as convolutional neural networks and recurrent neural networks, used Word2 vec, WordNet, support vector machine, and bag-of-word models to understand the background information of text; however, these models are context independent in nature. It means they create the same embeddings for the same word, which is used in different contexts. To that end, in this section, we included the three most widely used transformer models, such as the BERT, ELMO, and GPT-2. These models are context-dependent; in order to do this, these models create embeddings based on the given context. Moreover, the BERT model is suitable for vector representation, for extracting ambiguous terms such as polysemy, and for context information from sentences by using a stacked transformer and attention mechanism. The model uses a multi-encoder to understand the sentence in the sequence of words or sub-words, referred to as keywords or key phrases. The BERT model also employed masked language modeling and next-word prediction for domain-specific and multi-domain sentence structures. There are two versions of the BERT model: the BERT Base with 12 layers and the BERT large with 24 layers. The model required huge corpora to understand the information. However, ELMO uses the encoder and decoder to understand the sentences. The ElMO is suitable for morphological and character-level representations, and the model also uses masked language modeling for the task-specific domains. Furthermore, GPT-2 is a multi-layer transformer decoder. It is a unidirectional stacked transformer that understands the context of the given sentence and extracts the similarity. The model split sentences into words and, as with the BERT model, it is suitable for domain-specific and domain-independent datasets. The model also uses the masked language modeling technique. In terms of performance metrics, these models heavily rely on confusion metrics.