# Analysis of Short Texts Using Intelligent Clustering Methods
## 2 Materials and Methods
This combination effectively identifies main topics from large text arrays while accounting for contextual relationships between words, capturing subtle nuances of language. Figure 4 illustrates the general structure of this method. The input to the model is a collection of documents (D). The LDA module generates topic distributions \(p\) (topic|document) of dimensionality \(k_{LDA}\), where \(k_{LDA}\) is the predefined number of topics used in LDA. BERT provides the contextual embeddings of 768 dimensions. These outputs are concatenated into a single vector (\(\gamma\cdot vec_{lda}\)_and_\(vec_{bert}\)) and passed to the autoencoder, compressing the vector into a lower-dimensional latent space and reconstructing it. The latent representation produced by the autoencoder is then used for clustering with the K-Means algorithm. The results include cluster labels, the trained LDA model (for interpretability), and the autoencoder (for reuse). This method enhances the quality of clustering and visualization. It provides a deeper semantic understanding of the text, making it particularly effective for analyzing complex short texts and supporting various NLP tasks (Figure 4). \begin{table}
\begin{tabular}{l l l l l l l l} \hline \hline \(\gamma\) & **Silhouette** & **[PERSON]\_[PERSON]** & **[PERSON]\_[PERSON]** & **Adjusted\_[PERSON]** & **Homogeneity** & **Completeness** & **v\_Measure** \\ \hline
1 & 0.1356 & 2.30 & 1.0007 & 0.4828 & 0.7952 & 0.8219 & 0.8084 \\
5 & 0.1150 & 2.0697 & 1.0362 & 0.4828 & 0.7952 & 0.8219 & 0.8084 \\
10  
Several stages of data preprocessing were used to work on the short text clustering problem. First, all texts were cleared of noise, including removing special characters, links, numbers, and stop words. Then, the texts were tokenized and lemmatized to bring words to their original form, which improved the quality of the input data for the models. A grid search method was used to optimize the parameters of the BERT and LDA models to find the best hyperparameters. This included adjusting the number of topics for the LDA model and the length of sequences for BERT. The architecture of the hybrid LDA + BERT + AE model is presented in the following stages. First, the LDA model was used to identify topic clusters, after which BERT was used to extract vector representations of words, taking into account the category types. AE performed dimensionality reduction of the vector space, improving the clustering accuracy. The results of this integrated approach are evaluated by comparison with traditional text clustering methods. Integrating LDA + BERT + AE is expected to significantly enhance clustering accuracy, providing a deeper understanding of texts and more efficient extraction of helpful information. This research shows the significant potential of the \"context-aware embedding\" approach to improve the processing and analysis of text data, which has important implications for a wide range of applications in natural language processing. In the study, several works are referenced that validate the effectiveness of advanced techniques such as BERT, TF + IDF, and the hybrid LDA + BERT + AE approach in short text clustering. Notably, the methodologies outlined by [PERSON] et al. (2023) [1] and [PERSON] et al. (2023) [2] are considered. These studies emphasize the advantages of multilingual strategies and ensemble methods, particularly in text categorization and  
Figure 4: The architecture of the hybrid method LDA + BERT + AE. sentiment analysis. The integration of these approaches is demonstrated to significantly enhance the accuracy and efficiency of clustering tasks in the study of short texts.