# A Neural Topic Modeling Study Integrating SBERT and Data Augmentation
## 4 Experiments
### Evaluation Indicator  
**Topic coherence:** Topic coherence refers to the relatedness of words in a topic, i.e., whether the words in a topic share the same thematic characteristics. The purpose of topic coherence evaluation metrics is to assess the relevance and consistency of the topics extracted from a topic model. Topic coherence evaluation metrics commonly use measures such as NPMI and C_V to evaluate topic coherence. The evaluation metrics aim to assess whether the topic model can generate semantically coherent and easily understandable topics. For example, the group of words \"apple, banana, orange, grape\" is more coherent and aligned with human judgment than \"apple, banana, packaging paper, price\".  
Evaluating topic coherence is usually calculated using the normalized point mutual information (NPMI) [45; 46]. The score for each topic is based on the word combinations of the first T words returned for that topic to calculate the co-occurrence, and then the topic coherence score is calculated based on the co-occurrence of the words. By default, the first 10 words of each topic are used to calculate the value of the NPMI. It is defined in Equation (5):  
\begin{table}
\begin{tabular}{c c} \hline \hline
**Name** & **Parameters** \\ \hline learning\_rate & \(1\times 10^{-3}\) \\ Dropout & 0.1 \\ batch\_size & 64 \\ Optimizer & Adam \\ embedding\_dim & 768 \\ \hline \hline \end{tabular}
\end{table}
Table 1: Experimental parameter settings in this paper.  
\[NPMI(t)=\sum_{j=2}^{N}\sum_{l=1}^{j-1}\frac{\log\frac{p(w_{i},w_{j})}{p(w_{i})p(w_{ j})}}{-\log P(w_{i},w_{j})} \tag{5}\]  
where \(P(w_{i})\) is the probability that the word \(w_{i}\) appears in a document, and \(P(w_{i},w_{j})\) is the number of times the pair \((w_{i},w_{j})\) appears in a document.  
In addition, the C_V metric is another method used to evaluate the consistency of topic models, similar to the NPMI approach. C_V can measure both the internal coherence of a topic and the coherence between topics. Specifically, C_V utilizes the co-occurrence information of words to calculate the relevance within a topic and the separation between topics, thus assessing the quality of topics. We followed the approach used by [PERSON] et al. [44] to conveniently compute the coherence metrics (NPMI,C_V) for topics.  
**Topic diversity:** The RBO [47] measure is based on a simple probabilistic user model. It compares the first ten words of two topics. We judged the diversity of topics by the reciprocal of the RBO measure [44; 48]. For the same topic in a theme, it has a countdown of 0, while for completely different topics, it has a countdown of 1. The better the topic diversity, the higher the score, indicating a better model extraction.  
We show some examples of topics mined by the model to better visually represent the topics proposed by the model in this paper and to view the coherence of the topics in a human-compatible way. When the semantics of the words of a set of topics are similar, we can consider them as semantically coherent and high-quality topics. When a set of topics is semantically coherent and the words are not identical, they can be considered as topics with a high diversity.