# Topic Modeling for Faster Literature Screening Using Transformer-Based Embeddings
## 3 Results and Discussion  
### Peri-Implantitis Dataset Analysis  
Running BERTopic on a Colab notebook, after loading the necessary libraries and packages, required a few seconds for such a small dataset. BERTopic successfully identified several topics based on the dataset titles, but the exact number of topics varied depending on algorithm parameters. The main steps that users can control in BERTopic are summarized in Figure 1:  
1. Creating the embeddings,
2. Reducing the embeddings' dimensions. 3. Clustering the embeddings,
4. Labelling the embeddings. To create sentence embeddings from the titles, we decided to use the all-mpnet-base-v2 model, which has been pre-trained on a large dataset and has proven to be effective also on academic titles [60]. While slower than smaller models, this did not significantly impact computation time with a dataset of just over 6000 titles. To improve clustering efficiency, we reduced the dimensions of our initial embeddings using UMAP. This algorithm allows for a thorough customization of its parameters, including the granularity of the topological structure that it aims to preserve during dimensionality reduction through the \"number of neighbors\" parameter. A major advantage of HDBSCAN is that it does not require pre-determining the number of clusters (e.g., unlike K-means algorithms), although, as a result, it tends to cluster unclear documents in a null group, which is identified by the \(-1\) label. HDBSCAN can be customized through several parameters too, including the minimum acceptable size for a cluster. The last step consists of finding a label for each cluster that corresponds to its topic. For that purpose, we have chosen two representation models: a large language model, to create human-like labels, and KeyBERT, to generate keywords that are characteristic to the topic. Figure 2 shows that by reducing the minimum size of the clusters, BERTopic was able to find more topics in our dataset. This is expected, as niche topics consisting of only a few articles may be overlooked if the threshold is too high. At the same time, changing the sensitivity of UMAP through its number of neighbors parameter altered the slope of the curve. The choice of the best settings is a subjective decision, which depends on the purpose of the investigators; in our case, we wanted to have enough granularity to isolate unrelated topics while keeping the number of topics small enough to be easily manageable and several orders of magnitude smaller than the dataset itself to make it a convenient step in the workflow. Therefore, we empirically determined the optimal number of neighbors to be 15 and the minimum cluster size to be 50. These settings generated 14 topics (Table 1). Table 1 lists the topics identified by the algorithm, from the biggest one to the smallest one. The topic list includes the '\(-1\)' unclassified documents cluster, where all the unclassified papers are supposed be allocated. Interestingly, the algorithm agreed that the non-classified (\(-1\) cluster) articles were still mostly centered on implants (and implant infections) and assigned them the label \"Treating Implant Infections\". This null topic cluster was quite small (n = 357). As expected, BERTopic identified several topics related to peri-implantitis (e.g., topic #0 which included the majority of the papers (n = 3733) or topic #6) but also more broadly related to implant dentistry (e.g., topics #5, #7, and #12).