# Topic Detection Based on Sentence Embeddings and Agglomerative Clustering with Markov Moment
## 4 The Experiment against the Baselines and Its Results  
The tests were performed on two distinct datasets. News20 is a dataset containing news headlines and articles, which was split into four subsets. Each division contains rows with unique labels e.g., rows from a subset containing three classes is not used in subsets with five and 10 classes. BBC is a dataset containing BBC articles with each row labeled as one of the five distinct classes: sport, tech, business, entertainment, and politics. The proposed model was tested against two other well-known clustering methods that have the ability to infer the optimal number of clusters as part of the training process. DBSCAN and OPTICS were chosen as one of the few classic approaches that does not require the number of clusters to train the model [28; 29]. Three metrics were utilized to evaluate the performance of our method against classic approaches. The V-measure can be described as a weighted harmonic mean of homogeneity and completeness [30]. In this metric, the homogeneity criterion is satisfied if the algorithm only assigns to a single cluster the data units that are members of a single class. In order to satisfy the completeness criteria, the algorithm must assign to a single cluster all of the data units that are members of a single class. NMI (Normalized Mutual Information) [31] was used as additional performance metric, which utilises entropies of class labels and mutual information (a reduction in the entropy of class labels after training). Both of these metrics are independent of the absolute values of the labels, and the results of clustering can be directly compared with the true labels, which are document classes of the original data assigned by the authors of this dataset. Finally, to test models' ability to accurately determine the number of clusters in a given corpus, we employed a measure of similarity between the optimal number of clusters and the number obtained by algorithms (number of clusters' inference accuracy). Experiments were conducted with different encoding techniques, in order to test their possible impact on the model's performance. The first model is TF-IDF, as it is, to this day, a very popular encoding methods for relatively simple tasks. Another employed method is FastText [32], which is an extension of a well-known word2 vec model with improved accuracy and faster training time due to the usage of negative sampling. Finally, we use the context-aware USE model described in Section 2, which, unlike FastText, can output embeddings not just for words but for entire sentences and, by doing so, bypasses the process of averaging the word embeddings typically performed by the models that do not utilise context information, such as FastText. The results for different combinations of training and encoding models, as well as different metrics, can be seen in Tables 3-9. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline  & \multicolumn{3}{c}{**News3**} & \multicolumn{3}{c}{**News5**} \\ \hline
**Method** & **W2V** & **TF-IDF** & **USE** & **W2V** & **TF-IDF** & **USE** \\ \hline “e-2” hypothesis & 0.17 & 0.20 & **1.43** & 0.17 & 0.49 & **1.48** \\ DBSCAN & 0.02 & 0.39 & 0.73 & 0.04 & 0.34 & 0.15 \\ OPTICS & 0.30 & 0.47 & 0.70 & 0.24 & 0.46 & 0.01 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: NMI for News10 and News20. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline  & \multicolumn{3}{c}{**News3**} & \multicolumn{3}{c}{**News5**} \\ \hline
**Method** & **W2V** & **TF-IDF** & **USE** & **W2V** & **TF-IDF** & **USE** \\ \hline “e-2” hypothesis & 0.14 & 0.08 & **1** & 0.20 & 0.14 & 0.84 \\ DBSCAN & 0.33 & 0.33 & 0.38 & 0.33 & 0.78 & 0.51 \\ OPTICS & 0.60 & 0.18 & 0.11 & 0.84 & 0.45 & **0.95** \\ \hline \hline \end{tabular}
\end{table} 
Table 8: The number of clusters’ inference accuracy for News10 and News20.