## 3 Methodology
### Enhancing BERTopic-Based Framework for Dynamic Topic Modeling
#### 3.2.2 Hyperparameter Tuning for BERTopic  
In the rapidly evolving landscape of 6G technology, the continuous tracking of the 'topic evolution' is crucial for the understanding of emerging technologies in 6G. Traditional topic modeling approaches are naturally static and thus do not allow for the modeling of documents ordered sequentially. It is understood that dealing with dynamic, constantly changing volumes of patent data would require transitioning to a DTM. Given the cyclic nature of topic evolution, we adopt the BERTopic-based framework, as shown in the second part of Figure 1.  
Initially, BERTopic was fitted as if the data have no temporal aspect, thereby creating a general topic model. After obtaining the embeddings from contextual embeddings, the three critical components used in our BERTopic-based framework are as follows:  
1. We first performed the Uniform Manifold Approximation and Projection (UMAP) algorithm to reduce their dimensionality. The high-dimensional vectors obtained from the embedding model are often redundant and computationally expensive to process, especially in the subsequent clustering step. UMAP aims to find a low-dimensional representation of the data that preserves the local and global structure of high-dimensional data. It constructs a fuzzy topological representation of the data in the high-dimensional space and then optimizes a low-dimensional representation approximating this topology. In processing 6G patent data, UMAP can transform the high-dimensional vectors (e.g., vectors in \(\mathbb{R}^{2}\)) into two- or three-dimensional vectors (e.g., in \(\mathbb{R}^{2}\) or \(\mathbb{R}^{3}\)), which can significantly reduce the computational complexity of the subsequent clustering step and is easy to visualize;
2. Hierarchical Density-Based Spatial Clustering of Applications with Noise (HDBSCAN) was used to group the low-dimensional vectors obtained from UMAP into clusters. HDBSCAN is a hierarchical and density-based clustering method that allows it to handle clusters of irregular shapes. The topics in the 6G landscape may have complex and non-spherical distributions in the low-dimensional space, and HDBSCAN can effectively identify these clusters. Let \(V^{\prime}\) be the set of low-dimensional vectors obtained from UMAP and HDBSCAN partition \(V^{\prime}\) into \(k\) clusters \(C_{1},\ldots,C_{k}\) and a set of outliers \(O\);
3. Cluster tagging using c-TF-IDF is the final step in BERTopic to extract topic representation for each cluster at different time frames. Global representation was utilized to identify the main topics likely to emerge at different timesteps. For each topic and timestep, the c-TF-IDF representation was calculated to identify representative terms for each cluster. The mathematical details of c-TF-IDF are presented in the upcoming section. In the tasks of the DTM, both global fine-tuning and evolutionary fine-tuning were used to enhance the results. Firstly, global fine-tuning was carried out. This method combines the c-TF-IDF representation of a topic at a particular timestep with the global representation. By taking an average of these two representations, the topic representation at that timestep was adjusted to move slightly closer to the global one. This approach enables the topic to maintain some of its unique characteristics while also aligning with the overall trends captured by the global representation. Secondly, evolutionary fine-tuning was performed. Here, the c-TF-IDF representation of a topic at a given timestep was averaged with the c-TF-IDF representation of the same topic at the previous timestep. This process allows the topic representation to gradually change and adapt over time, reflecting the dynamic nature of the data. As new information becomes available at each timestep, the topic representation evolves to incorporate these changes. These keywords effectively summarize the main concepts and characteristics of the cluster, providing valuable insights into the technological topics within that time period;
4.