# TR-GPT-CF: A Topic Refinement Method Using GPT and Coherence Filtering
## 2 Materials and Methods
### Misaligned Word Detection
Cosine similarity alone measures how semantically related each word is to the topic centroid, yet it struggles with relative comparisons across topics. The Z-score addresses this issue by identifying words that significantly deviate from the norm within the context of the topic. This combination enables more reliable and context-aware detection of misaligned words, which are essentially the 'outliers' in the similarity distribution. The Z-score, also known as the standard score, is a statistical measure that quantifies the number of standard deviations a data point is from the mean of a dataset. It is calculated using the following formula [60] as further applied in [61; 62]:  
\[Z=\frac{(x-\mu)}{\sigma} \tag{1}\]  
The numerator (\(x-\mu\)) calculates the difference between the individual data point (\(x\)) and the mean (\(\mu\)). This determines how far the data point is from the average value of the dataset. The standard deviation (\(\sigma\)) represents the typical amount that data points differ from the mean. In this experiment, the Z-score is used to quantify the deviation of a specific value from the mean of the similarity scores, normalized by the standard deviation. In this context, the value refers to the cosine similarity of a word to the centroid. By standardizing the similarity scores with the Z-score, we normalize the variability in cosine similarity across different topics. This standardization allows us to apply the same Z-score threshold consistently, regardless of the overall range of similarity values within a given topic. We investigate Z-score thresholds between \(\pm\)1.5 and \(\pm\)3.0 to ascertain their effect on topic coherence metrics \(C\_v\) for identifying misaligned words. We implement the Z-score using the scipy.stats.zscore function from the SciPy Python library (v.1.14.1). To facilitate understanding of the variability in cosine similarity scores within a topic, Table 3 illustrates the scores for each word compared to the topic centroid, along with their Z-score values, assuming a standard deviation (\(\sigma\)) of 0.20. Word E, with a score of 0.40, exhibits significantly lower similarity to the centroid than other words. Relying exclusively on cosine similarity presents difficulties in establishing an appropriate threshold for misaligned word detection. This challenge is particularly evident when the topic demonstrates low similarities or when the dataset exhibits different behavior. Using Equation (1), the Z-score value is calculated to measure the extent to which each word's cosine similarity deviates from the 'average' similarity score, relative to the overall distribution of similarities. From Table 3, we observe that most of the words have a Z-score close to zero, indicating that their cosine similarities are near the mean. However, Word E has a Z-score of \(-\)1.85, significantly lower than others; if a Z-score threshold of \(-\)1.5 is applied, Word E would be flagged as a misaligned word because its Z-score falls well below this threshold. For several reasons, the Z-score represents a superior method for identifying misaligned words within topics. Firstly, it provides relativity by comparing each word's similarity to the other words in the topic, rather than using an arbitrary threshold, such as requiring that the 'cosine similarity must be \(>\)0.7'. This feature enhances the adaptability of Z-scores to their respective context. Secondly, the flexibility of Z-scores enables the effective application of the same threshold (e.g., \(-\)1.5) across various topics, accommodating the unique distribution of similarities in each topic. Lastly, Z-scores are particularly effective in emphasizing misaligned words like Word E, which stand out due to their significant deviations from the norm, regardless of whether the topic naturally exhibits low- or high-similarity values. To further refine the detection of misaligned words, we also calculate the inverse document frequency (IDF) [63] of each word using a TF-IDF vectorizer to account for word importance.