# Leveraging LLMs for Efficient Topic Reviews
## 4 Results and Discussion
### Results and Analysis
#### 4.1.2 Evaluation of Topic Generation Quality  
The Evaluation of Topic Generation Quality section aims to assess and validate the effectiveness of the topic generation process by comparing the automatically generated keywords with those indexed in Scopus and provided by the authors in the documents. This comparison allows us to measure the accuracy and relevance of the generated topics, ensuring that the thematic organization and representation of the documents are consistent with the original content. By using advanced NLP techniques, this section provides a quantitative analysis that supports the reliability and quality of the topic modeling approach employed in this study. Of the 399 documents initially obtained in Topic 6, 267 containing both author keywords and index keywords were filtered. These documents were specifically selected for their likelihood of providing more complete and detailed information on the topic in question, thus allowing a more accurate and relevant selection for the literature analysis. In this sense, the perspective is based on techniques used for research and systematic reviews (see Figure 6). To further analyze the quality and consistency of topic generation, we conducted an experiment focused on comparing the cosine similarity between the keywords generated by KeyBERT for Topic 6 and the keywords indexed and provided by the authors in the same set of documents. This comparison allowed us to directly measure how well the automatically generated topics align with the established thematic content. Moreover, we expanded the analysis by comparing the KeyBERT output for documents within Topic 6 against the average KeyBERT-generated keywords from four other selected topics (Topics 9, 44, 51, and 63). These topics were chosen as baselines because they are related to the general scope of the Scopus query but are distinct from Topic 6, offering a meaningful comparison point. To determine if there is a significant difference in topic identification between KeyBERT's output for Topic 6 and the averaged results from the  
Figure 6: Bubble Cloud of most frequent words in topic 6. This visualization presents the frequency of up to 2 n-grams in topic 6.  
other four topics, we conducted a series of statistical analyses. We first applied the Shapiro-Wilk test to check for normality in the data. After confirming that the data followed a normal distribution, we performed paired \(t\)-Tests to assess the significance of the differences in cosine similarity. The results indicated statistically significant differences, with \(p\)-values suggesting that KeyBERT's performance in generating topics for Topic 6 is distinct when compared to the baseline topics. The results presented in Table 3 provide an overview of the cosine similarity between the KeyBERT-generated keywords and those indexed and authored in the documents analyzed. The descriptive statistics indicate that for the index keywords in Topic 6, the mean cosine similarity is 0.6956, with a median of 0.7007 and a standard deviation of 0.0531. In contrast, the mean for the averaged selected topics (Topics 9, 44, 51, and 63) is slightly lower at 0.6323, with a median of 0.6309 and a smaller standard deviation of 0.0245, suggesting less variability in similarity scores across these topics. A similar trend is observed for the author keywords, where Topic 6 shows a mean cosine similarity of 0.6971, a median of 0.7064, and a standard deviation of 0.0717. The selected topics have a lower mean of 0.5958, a median of 0.6019, and a standard deviation of 0.0432, again indicating more consistency but lower overall similarity compared to Topic 6. The Shapiro-Wilk test results confirm the normality of the data across all categories, with \(p\)-values exceeding the 0.05 threshold, thus justifying the use of the paired \(t\)-Test for further analysis. The \(t\)-Test paired \(p\)-values, 0.0135 for index keywords and 0.0224 for author keywords, indicate statistically significant differences between the cosine similarities of Topic 6 and the averaged selected topics.