# Discovering Trends in the Digitalization of Shipping:
## 3 Methodology
### Topic Detection  
Topic detection is aimed at identifying consistent themes across the document set. We analyzed presentation frequencies by year and used word networks, built on co-occurrence relationships, to visualize word interactions, utilizing [PERSON]'s coherence model [33]. Dimensionality reduction was achieved through Uniform Manifold Approximation and Projection (UMAP), preparing data for BERTopic, which employed vectorization and class-based TF-IDF to ascertain topics. The BERTopic algorithm for topic modeling combines several powerful techniques. At its core lies the use of pre-trained transformer models like BERT. These models excel at converting documents into dense vector representations, capturing the essence of how words relate to each other within the text. This allows BERTopic to grasp the underlying meaning conveyed in the documents. BERTopic goes beyond traditional TF-IDF by employing a class-based variation. This modified scheme emphasizes words that hold particular importance within a specific topic, even if they do not appear frequently throughout the entire document collection. This helps pinpoint keywords crucial for understanding the thematic content. To group documents with similar semantic meaning, BERTopic utilizes clustering algorithms like HDBSCAN. This process essentially sorts documents based on their semantic closeness, resulting in the formation of distinct clusters that represent different topics. To make the identified themes interpretable, BERTopic extracts keywords and phrases for each topic. This involves breaking down the text into individual words or phrases (tokenization) and assigning weights to identify the most significant words within each topic. BERTopic achieves these goals in a sequential manner. First, it transforms documents into numerical representations using BERT. Since these embeddings can be high-dimensional and complex, BERTopic then applies dimensionality reduction techniques like UMAP to make the data more manageable. With the simplified embeddings, BERTopic utilizes clustering algorithms to group them into distinct topic clusters. Finally, BERTopic performs topic tokenization and weighting to extract and highlight the most significant words and phrases associated with each identified thematic group [26]. A more mathematical view of the BERTopic model consists of the document embeddings and document clustering formulas. For the document embeddings, BERTopic uses the Sentence-BERT (SBERT) framework. It is based on the following functions. For the classification objective function, the sentence embeddings u and v are concatenated with the element-wise difference \(\mid\)u - v\(\mid\) and this is multiplied by the trainable weight. \[\text{Wt}\in\text{R3n}\times\text{k:o}=\text{softmax}(\text{Wt}(\text{u}, \text{v},\mid\text{u}-\text{v}\mid))\]  
where n is the dimension of the sentence embeddings and k the number of labels. For the regression objective function, the cosinesimilarity between the two sentence embeddings u and v is computed. The meansquared-error loss as the objective function is used. For the triplet objective function, given an anchor sentence a, a positive sentence p, and a negative sentence n, triplet loss tunes the network such that the distance between a and p is smaller than the distance between a and n. Mathematically, we minimized the following loss function:  
\[\text{max}(\mid\mid\text{sa}-\text{sp}\mid\mid-\mid\mid\mid\text{sa}-\text{ sn}\mid\mid+,0)\]  
with sx the sentence embedding for a/n/p, and \(\mid\mid\cdot\mid\) a distance metric and margin. Margin ensures that sp is at least closer to sa than sn [34]. For the document clustering, UMAP was used. UMAP has been found to retain more local and global characteristics of high-dimensional data when projected into lower dimensions [17]. The reduced embeddings are clustered using HDBSCAN. The computational view of UMAP starts with the construction of a weighted k-neighbor graph.