Semantic Component Analysis: Introducing Multi-Topic Distributions to Clustering-Based Topic Modeling (2025)

Abstract: Topic modeling is a key method in text analysis, but existing approaches fail to efficiently scale to large datasets or are limited by assuming one topic per document. Overcoming these limitations, we introduce Semantic Component Analysis (SCA), a topic modeling technique that discovers multiple topics per sample by introducing a decomposition step to the clustering-based topic modeling framework. We evaluate SCA on Twitter datasets in English, Hausa and Chinese. There, it achieves competetive coherence and diversity compared to BERTopic, while uncovering at least double the topics and maintaining a noise rate close to zero. We also find that SCA outperforms the LLM-based TopicGPT in scenarios with similar compute budgets. SCA thus provides an effective and efficient approach for topic modeling of large datasets.

## introduction topic modeling is an essential tool for automated text analysis in digital humanities, computational social science, and related disciplines @xcite @xcite . it aims to find high-level semantic patterns, the topics, in a corpus and then assign them to documents. the state-ofthe-art (sota) for large-scale datasets, bertopic @xcite , assumes a single topic per document, which often does not hold even for short texts @xcite , as can be seen in figure 1 . additionally, a high rate of samples is usually assigned to no topic, i.e. they have a high noise rate @xcite . llm-based approaches like topicgpt @xcite perform very well, but due to their high computational cost, it is questionable how well they scale to large datasets under resource constraints. to overcome these limitations, we propose semantic component analysis (sca) as a novel sca bertopic 6: china, tariffs, trade, currency 24: coronavirus, virus, task, briefing 56: fake, media, nytimes, news 118: testing, cases, tests, deaths 70: york, mayor, jersey, city 0: apprentice, celebapprentice, celebrity, interviewed with the exception of new york & a few other locations, we've done much better than most other countries in dealing with the china virus. many of these countries are now having a major second wave. the fake news is working overtime to make the usa (& me) look as bad as possible! figure 1 : a tweet from the trump dataset @xcite decomposed into its semantic components as defined by the top3 component activation scores of iteration 1 (components 6, 24, and 0) and subsequent iterations @xcite . ## methodology sca is a clustering pipeline that introduces a decomposition step to find unit vectors in the embedding space-which we call semantic components-that each correspond to a topic. for clustering, we employ the cosine distance in umap to map the embeddings to a 5-dimensional, euclidean space on which hdbscan operates, mirroring the setup in bertopic. visualized in figure 2 , it works as follows: 0. embed input sequences 1. reduce and cluster the embeddings x j 2. represent component defined by cluster after each decomposition step in 3, we set x @xmath0. the algorithm converges when the residuals are zero. we introduce other practical stopping criteria listed in appendix a.3. note that sca has two hyperparameters: µ ∈ [0, 1], controlling the amount of the decomposition, and a threshold α, which is compared against α i,j , the cosine similarity between x j and v i . see appendix a.1 for a detailed discussion of α and µ. we assume that each cluster found represents a topic. to obtain an interpretable representation, we follow bertopic and use c-tf-idf to rank the tokens according to their importance for a cluster storing the top10 set of tokens for each cluster. ## conclusion we introduce semantic component analysis as a novel topic modeling method that can discover multiple topics and scale to large datasets. we show that the topics found by sca indeed provide new, more nuanced insights into the data while maintaining a noise rate close to zero and matching the large-scale sota in terms of topic quality. furthermore, we show that the method is able to generalize to multiple languages. however, future work is required to address the problem of generic, overpredicted clusters and the interpretability of the topic representations, perhaps by downweighing larger clusters and exploring other forms of cluster representations respectively. taken together, we believe that sca provides a substantial improvement in large-scale short text analysis.