## 4 Results
### Human Validation of Topic Coherence  
To further assess the correctness of the extracted topics, a human validation experiment was conducted with domain experts. A total of 110 topics were reviewed, and each was categorized as either correct, incorrect, or uncertain. The results indicate that 81.82% of the topics were deemed correct, 10.91% were identified as incorrect, and 7.27% fell into the uncertain category, as shown in Figure 10.  
These findings suggest that the topic-modeling approach effectively captures meaningful themes in e-participation discussions. However, the presence of incorrect and uncertain topics highlights areas for potential improvement. The uncertain category suggests that some topics may lack clear semantic boundaries or require further refinement. Future work will extend this validation by incorporating a broader panel of domain experts and employing inter-annotator agreement measures to improve the robustness of the evaluation. Further, the effectiveness of the topic-modeling pipeline was evaluated using the BBC dataset [29], which consists of documents labeled into five thematic groups: _business_, _entertainment_, _politics_, _sport_, and _tech_. The dataset consists of 2225 articles. The labels were encoded using LabelEncoder from the scikit-learn library, and the dataset was split into training and testing sets with an 80-20 ratio. The pipeline was trained on the training set and evaluated on the held-out test set. The following hyperparameters were used: top_n_words: 10, UMAP: n_neighbors=15, n_components=2, metric='cosine'. To assess the model's performance, several evaluation metrics were computed. The Adjusted Rand Index (ARI) [30] measured the agreement between the predicted clusters and the true labels, yielding an ARI of 0.888. The Normalized Mutual Information (NMI) quantified the mutual information between the predicted \begin{table}
\begin{tabular}{l c c c c c c c c c} \hline \hline
**Model** & \multicolumn{2}{c}{**20 News Group**} & \multicolumn{2}{c}{**BBC News**} & \multicolumn{2}{c}{**Join**} & \multicolumn{2}{c}{**[PERSON]**} & \multicolumn{2}{c}{**iVoting**} \\ \cline{2-10}  & **NPMI** & **Diversity** & **NPMI** & **Diversity** & **NPMI** & **Diversity** & **NPMI** & **Diversity** \\ \hline LDA & 0.058 & 0.749 & 0.014 & 0.577 & 0.001 & 0.572 & \(-0.011\) & 0.502 & \(-0.04\) & 0.124 \\ CTM\_C & 0.096 & 0.886 & 0.094 & 0.819 & 0.033 & 0.862 & 0.009 & 0.855 & \(-0.24\) & 0.746 \\ NMF & 0.089 & 0.663 & 0.012 & 0.549 & 0.061 & 0.322 & 0.009 & 0.379 & \(-0.04\) & 0.343 \\ Top2 Vec & 0.192 & 0.823 & 0.171 & 0.792 & 0.081 & 0.898 & \(-0.169\) & 0.658 & \(-0.16\) & 1.000 \\ Proposed Approach & 0.166 & 0.851 & 0.167 & 0.794 & 0.060 & 0.598 & 0.066 & 0.663 & \(-0.04\) & 0.436 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Evaluation metrics for different topic models across various datasets. Figure 9: Comparison of different embeddings on JOIN regarding their computational time and the number of topics per topic model. clusters and the true labels, achieving an NMI of 0.874. The accuracy, representing the proportion of correctly classified documents, was 0.953. Additionally, precision, recall, and F1-score were calculated, which were 0.955, 0.953, and 0.952, respectively. The results were visualized using a UMAP projection of the test-set embeddings, as shown in Figure 11. The scatter plot highlights the true and predicted labels, providing insights into the model's ability to separate the classes in the embedding space. Misclassified samples were also analyzed to identify potential limitations of the model. Some documents belonging to overlapping topics (e.g., _business_ and _politics_) were harder to accurately classify. Additionally, a confusion matrix was visualized in order to show the models performance Figure 12.  
Figure 11: UMAP projection of the test-set embeddings. The outer circle colors represent predicted labels, while the inner colors represent true labels. Figure 10: The results of the human validation experiment.