# TR-GPT-CF: A Topic Refinement Method Using GPT and Coherence Filtering
## 3 Results and Discussion
### Evaluating Topic Coherence Improvement Across Datasets
#### 3.1.6 Improvement of Topic Coherence in Science Article Dataset  
Table 10 indicates that G-BAT showed the most significant improvement among the models, with its coherence score rising from 0.265 to 0.341, representing a notable increase of 28.68%. This substantial gain underscores the refinement method's effectiveness for enhancing low-performing models on structured, domain-specific datasets such as Science Article. In contrast, BERTopic, which already had a high baseline coherence, exhibited a modest gain, improving from 0.731 to 0.740, representing a 1.23% increase. This slight improvement indicates the restricted potential for further enhancement given the model's robust initial performance. Similarly, LDA exhibited a modest improvement, with its coherence score increasing from 0.526 to 0.544, reflecting a 3.42% increase. This improvement illustrates the refinement method's ability to enhance coherence in probabilistic models, though to a limited extent. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Model** & **Before Refinement** & **After Refinement** & **Improvement (\%)** \\ \hline LDA & 0.461 & 0.487 & 5.64 \\ BERTopic & 0.506 & 0.552 & 9.09 \\ G-BAT & 0.494 & 0.570 & 15.38 \\ NMF & 0.427 & 0.483 & 13.11 \\ \hline \hline \end{tabular}
\end{table} 
Table 9: Coherence score for the SMS Spam dataset. NMF showed moderate progress on the Science Article dataset, with its score marginally increasing from 0.614 to 0.619, a 0.81% increase. This minor change shows that the refinement procedure had little effect, probably because the model was already performing near its optimal level for this dataset. Similarly, aside from G-BAT, the other models only showed small improvement. This suggests that the Science Article dataset, with its well-structured, long, and technical content, probably provides a solid foundation that limits further improvement. This structure may explain why models with high baseline coherence scores, such as NMF and BERTopic, have minimal space for improvement, as the dataset's inherent characteristics already support a high level of topic coherence. Evaluating Topic Coherence Improvements by Candidate Word Replacement: WordNet, GPT, and Combined Approaches  
In this section, we explore the improvement in topic coherence across various datasets when employing different candidate word generation techniques: WordNet, GPT, and a combination of both. Each strategy has distinct benefits in aligning topic words more accurately, therefore improving the interpretability of the topics. Through the comparison of different approaches, we aim to underscore the effectiveness of each in facilitating more coherent and meaningful topics. To improve topic coherence, WordNet and GPT were used to identify eligible words for refining. WordNet provided semantically similar candidates, such as synonyms and hypernyms, based on linguistic hierarchies, while GPT dynamically created contextually relevant alternatives using language model embeddings. This study investigates the use of WordNet and GPT in three different scenarios. First, we utilized only WordNet as a generator to provide alternative words. Second, we used only GPT. Third, we employed a combination of both. This hybrid approach ensured a resilient selection of candidate words for topic refinement by combining contextual adaptability with semantic depth. From Figure 2, we can observe that each dataset exhibits differences in the effectiveness of WordNet, GPT, and their combination in improving topic coherence. Below is a detailed analysis of these variations. Science Articles benefit from both semantic knowledge and contextual generation; therefore, the combined approaches are most effective. When WordNet and GPT are used together in this dataset, they perform slightly better than when used separately. This indicates that their combination is effective in handling more technical information. Due to their heterogeneity, Newsgroup datasets show minimal improvement across all approaches, making them difficult to enhance.