PRISM: PRIor from corpus Statistics for topic Modeling
Authors: Tal Ishon
Yoav Goldberg
Uri Shaham
Venue: Accepted by TMLR ()

Abstract: Topic modeling seeks to uncover latent semantic structure in text, with LDA providing a foundational probabilistic framework. While recent methods often incorporate external knowledge (e.g., pre-trained embeddings), such reliance limits applicability in emerging or underexplored domains. We introduce PRISM, a corpus-intrinsic method that derives a Dirichlet parameter from word co-occurrence statistics to initialize LDA without altering its generative process. Experiments on text and single cell RNA-seq data show that PRISM improves topic coherence and interpretability, rivaling models that rely on external knowledge. These results underscore the value of corpus-driven initialization for topic modeling in resource-constrained settings.
        
Code is available at: https://github.com/shaham-lab/PRISM.

Published in Transactions on Machine Learning Research (03/2026)
PRISM: PRlor from corpus Statistics for topic Modeling
Tal Ishon
ishonta ac.
Department of Computer Science Bor Man University
Prof. Yoav Goldberg
yogo
Department Computer Science Bar aan University
Dr. Uri Shaham Department of Computer Science Bor Ilan Universily
wri.shaham @biu.ac..
Reviewed OpenReview: https: // opwprevvieeeeeeee.eeemmmm net/ forum? id= id=454 tza
Abstract
Topic modeling seeks to uncover latent semantic structure in text, with LDA providing a foundational probabilistic framework. While recent methods often incorporate external knowledge (e.g., pre-trained embeddings), such reliance limits applicability in emerging or underexplored domains. We introduce PRISM, a corpus- intrinsic method that derives a Dirichlet parameter from word co-occurrence statistics to initialize LDA without altering its generative process. Experiments on text and single cell RNA-seq data show that PRISM improves topic coherence and interpretability, rivaling models that rely external knowl- edge. These results underscore the value of corpus-driven initialization for topic modeling in resource-constrained settings.
corpus-intrinsic
Code is available at: https:/ //gt:://ghhttt//g com/shhham Lab/PRISM#
1 Introduction
Topic modeling is a cornerstone technique in Natural Language Processing (NLP) for uncovering latent semantic structures in text. infers the thematic composition of a corpus by representing each topic as a probability distribution over words. The versatility of topic modeling is evidenced by its application across a spectrum of disciplines from analyzing customer feedback in e-commerce platforms modeling gene expression patterns in biology by treating genes as 'vocabulary" and samples as "documents". Such cross-disciplinary utility underscores topic modeling's broad relevance in both applied and scientific contexts.
Since its inception, among topic modeling techniques, Latent Dirichlet Allocation (LDA) (Blei et al., 2003) remains a foundation model, leveraging Bayesian inference to estimate document-topic and topic-word dis- tributions while inspiring numerous generative extensions. Topic modeling research has since evolved along many directions, which can be broadly categorized into two main paradigms. The first follows I.DA 's corpus- intrinsic approach, relying solely on statistical patterns within the target corpus through methods including graph-based and neural models that enhance semantic representation and topic coherence. The second paradigm incorporates external knowledge sowch such as pre-trained embeddings from large-scale language mod- els or domain-specifie priors-to guide topic discovery beyond the statistical patterns available in the input corpus alone.
Corpus-intrinns intrinsic topic modeling can be particularly attractive in emerging scientific domains where large-scale pretrained representations may not be explicitly optimizod for the target modality, and domain knowledge remains incomplete or evolving. biology, for example, key regulatory genes and functional proteins may be undiscovered or poorly characterized, limiting the reliability of external knowledge sources. While pretrained

---

Published in Transactions on Machine Learning Research (03/2026)
P1
22
B
D
Estimated Dirichlet parameter
Ply
Multinomial Topic-word distributions
Second Order Similarity Graph (PPMI-besed)
Diffusion Embeddings
B
0
W
M
LDA Original Framework
Figure 1: PRISM overview. From corpus D, we build second-order word-similarity graph (PPMI cosine), obtain diffusion-man map embeddings 8, softly cluster them into K topics, and estimate data-driven topic- word Dirichlet prior 3 for LDA. The lower panel shows the standard LDA graphical model; PRISM replaces the symmetric 3 with 3 while leaving the generative process unchanged.
representations are often highly effective in well-established domains such as general text, it seems prudent not to rely on them exclusively: they can encode pretraining-date artifacts and biases and introduce inductive biases that are misaligned with the target corpus (Belém et al., 2024; Ghate al., 2025).
In this work, we introduce PRISM PRIor from corpus Statistics for topic Modeling which enhances LDA solely through corpus-intrinsic initialization (Figure 1). PRISM shapes the model's 'initial perspec- tive" by deriving informed topic-word distributions from corpus statistics, providing prior-like guidance that improves lopic quality. Empirical results across five text corpora and single-cell RNA sequencing dalaset show that PRISM significantly improves topic coherence and interpretability, often matching or exceeding externally guided methods.
2 Related Work
Topic modeling methods broadly follow two paradigms: approaches that inject external knowledge (e.g.. expert resources or pretrained representations) and corpus-intrinsic approaches that rely only on corpus statistics.
2.1 Methods Incorporating External Knowledge
A prominent line of work injects external semantic structure to guide topic discovery. Expert-guided models incorporate supervision such as seed words (e.g., SeededL.D (Jagarlamudi et al., 2012)) and have been extended to clinical phenotyping from EHRs (Song et al., 2022). Related approaches integrate structured resources more directly, such as medical knowledge graphs within end-to-end topic models (Zou et al., 2022).
Another route leverages pretrained representations. ETM parameterizes topic-word distributions via pre- trained word embeddings (Dieng et al., 2020), while CTM amortizes inference using contextual document embeddings (Bianchi et. al., 2021). Several methods induce topics directly in embedding space, includ- ing BERTopic, which clusters sentence embeddings and labels clusters via c-TF-IDF (Grootendorst, 2022), Contextnal-Top2Ve6 which clusters contextual token embeddings to identify dense semantic regions (An- gelov Inkpen, 2024), and FASTopic, which reconstructs semantic relations among pretrained document embeddings with learnable topie/word embeddings (Wu et al., 2024).
2

---

Published in Transactions on Machine Learning Research (03/2026)
Overall, externally guided approaches can improve interpretability and alignment with known concepts, but they assume access to reliable curated resources or pretrained semantics; when such information is incomplete or weakly aligned as can occur in heterogeneous, measurement-nois biological modalities (e.g., single-cell gene expression) these assumptions may be less stable (Kedzierska et al., 2025), motivating corpus-intrinsic alternatives.
2.2 Corpus- Methods (No External Knowledge)
Corpus-intrinsic lopic models infer sucuuue solely from the observed corpus. LDA (Blei et al., 2003) remains the canonical probabilistic formulation, while NMF (Lee Seung, 2001) provides a non-generative factor- ization alternative. Neural variational topic models modernize inference under a BoW likelihood (Srivastava & Sutton, 2017), and recent work further exploits corpus-internal structure (e.g., contrastive graph-based signals) to improve topic quality without external knowledge (Nguyen et al., 2024; Luo et al., 2024).
Complementary work has examined how Dirichlet hyperparameters influence LDA. Asuncion ct al. (Asun- cion et al., 2009) and Wallach et al. (Wallach et al., 2009) treat a and / as dov-dimensional concentration parameters tuned via empirical Bayes This inquiry has been modernized in neural topic modeling, where researchers now learn these hyperparameters through backpropagation and specialized variational distribu- tions (Burkhardt & Kramer, 2019; Joo et al., 2020). While these recent updates allow for more flexible gradients and improved global sparsity, they still primarily calibrate smoothing levels rather than imposing vocabulary-level semantic structure on the topic-word prior.
Taken together, these lines of work demonstrate the value of corpus-internal signals for enhancing topic models without external supervision. Building on this idea, PRISM introduces a corpus-derived prior as a data-driven initialization for LDA, constructing a full V-dimensional, semantically structured 3 from corpus similarity. This provides a more informed starting point- prism through which the model better captures semantic structure while while preserving LDA's generative foundations.
3
Preliminaries
This section introduces the key mathematical tools underpinning our approach.
3.1 Latent Dirichlet Allocation (LDA)
LDA (Blei et al., 2003) is a generative probabilistic model in which each document is a mixture over latent topics, and each topic is a distribution over words. For each document d, topic proportions Og are drawn from a Dirichlet distribution with parameter a. Each word Wan is then generated by first sampling topic Zdn Multinomial( (fu), followed by sampling the word from the corresponding topic-word distribution, drawn from Dirichlet distribution with parameter B.
In the original formulation of LDA (Blei et al., 2003), B denotes the multinomial lopic word distribution. In contrast, implementations based on collapsed Gibbs sampling, including MALLET (McCallum, 2002), treat & as the Dirichlet parameter over topic word distributions. Throughout this work we adopt the latter convention, consistent with MALLET and other Gibbs-bascd LDA implementations.
Inference is commonly performed using Collopsed Gibbs Sampling (Griffiths & Steyvers, 2004), which samples topic assignments zán based current token counts and priors and B. This method, implemented efficiently in MALLET (McCallum, 2002), is widely adopted bascline.
3.2 Pointwise Mutual Information and Variants
Pointwise Mutual Information (PMI) (Church & Hanks, 1990) quantifies the association between two words and my by comparing their joint probability to the product of their marginal probabilities. To suppress noisy or uninformative associations, Positive PMI (PPM)) retains only non-negative values.
Beyond first-order association (direct PPMI between word pairs), second-order similarity compares words via the similarity of their PPMI context profiles (i.e., cosine similarity between PPMT row vectors). Intuitively,
3

---

Published in Transactions on Machine Learning Research (03/2026)
two words may be related even without direct co-oceurrence if their contextual associations overlap; for example, surgeon and physicion may seldom co-occur yet both associate with patient, hospital, and diagmosis. This captures semantic relatedness when words occur in similar contexts (Schütze, 1998). The use of cosine in a PPMI space is further supported by analyses linking predictive objectives such as SGNS to implicit lactorization of a PPMI matrix (Levy & Goldberg, 2014). Empirically, PPMT vectors combined with cosine similarity perform competitively on semantic similarity evaluations, suggesting that corpus co-occurrence statistics already encode substantial information about word relatedness (Bullinaria & Levy, 2012).
3.3 Diffusion Maps
Diffusion Maps (Coifman Lafon, 2006) provides a nonlinear dimensionality reduction technique that cap- tures the intrinsic geometry of data represented similarity graph W, the method constructs Markov transition matrix P according the following formula- tion: P = D- where D is the diagonal degree matrix with entries Da E, Wij- Then, it performs eigen-decomposition over P to obtain the diffusion embedding. Each word is then embedded as a vector:
similarity graph. Given undirected word-word
(((w,) = (Mf (112 vm
where A and the are the k-th eigenvalue and eigenvector of the transition matrix, respectively; m is the embedding dimension, and t is the diffusion time controlling the decay.
4 Proposed Method
We introduce PRISM, a corpus-intrinsic method that improves topic modeling by providing a data-driven initialization for LDA. PRISM proceeds in two stages: (1) it constructs word embeddings directly from corpus statistics by building a PPMI-based word similarity graph and applying diffusion maps to obtain low-dimensional embeddings capturing its global structure; (2) it softly clusters these embeddings to form topic-word distributions and fits a Dirichlet parameter B, which is used to initialize LDA. This yields prior- like guidance while leaving LDA's generative process unchanged.
4.1
Constructing Word Embeddings
4.1.1 Similarity Graph
To capture semantic similarity between words, we construct undirected weighted graph W based on Positive PMI (PPMI), as described in Section 3.2. The PPMI matrix is computed using document-leved co- occurrence, treating each document as context window. Each word is represented by its corresponding row vector v, from the PPMI matrix, encoding its distributional context. We then define pairwise similarity using cosine similarity as Wi, = cos(v,, Vj), where v; and V, are the PPMI vectors of words w, and Wy, respectively. The resulting weighted similarity graph W captures both direct and indirect semantic associations by leveraging the principle that words appearing in similar contexts tend to have similar vector representations.
4.1.2 From Graph to Embeddings
To obtain dense word representations from the weighted similarity graph, we apply diffusion maps, as defined in Section 3.3. This spectral embedding technique captures high-order semantic relationships by modeling multi-step transitions over the graph. Intuitively, if diffusion process is initiated at two semantically sim- ilar words, their transition probabilities over time will be similar, reflecting shared contextual neighborhoods.
In our implementation, we construct the transition matrix P from the PPMI-weighted similarity graph and apply a
step (a = 1) following the anisotropic normalization (Coifman & Lafon, 2006) to mitigate the influence of word frequency variations. We embed cach word using the top m diffusion components-specffiin the leading non-trivial right eigenvectors of P. scaled by their corresponding eigenvalues at diffusion time i = 1. We set t to preserve local semantic neighborhoods
4

---

Published in Transactions on Machine Learning Research (03/2026)
and fine-grained word relationships, higher values of f would suppress high-frequency geometric detail in lavor of broader, less discriminative global clusters. Empirically, selecting between 80 and 130 components yields the best semantic representation, with the optimal m chosen via grid search based on topic coherence scores for each dataset.
The resulting vectors serve as our corpus-specific word embeddings, encoding semantic structure without relying on any external resources.
4.2
Estimating Dirichlet Parameter B
4.2.1
Topic-Word Distributions
To compute empirical topic-word distributions from the learned word embeddings, we apply a Gaussian Mixture Model (GMM) (Reynolds, 2009) with K components to softly cluster the word representations. The GMM yields the posterior probability p(= w) of each word w belonging to topic 2, along with the topic priors p(z).
However, our goal is to obtain p(w =)-the probability of a word given topic as required to estimate the Dirichlet parameter B over topic-word distributions. To do so, we apply Bayes' Rule:
p(w 2) p(z w)p(w)
p(z)
where p(= w) is given by the GMM, p(2) is the mixture weight for component 2, and p(w) is taken from uni- gram distribution, computed as the frequency of word w in the corpus divided by the total number of tokens.
This yields multinomial topic-word distributions matrix X E RRXV where each row corresponds p(w 2) for a given topic. The matrix is grounded entirely in corpus-derived signals capturing both contextual similarity and token-level frequency used to estimate the Dirichlet parameter B3
4.2.2 Parameter / Estimation
To estimate Dirichlet parameter RV over the vocabulary, we apply the method of moments (Minka, 2000), classical statistical technique used for parameter estimation. The core idea is to match the theoretical moments of the Dirichlet distribution to empirical moments computed from data. Let ( (*) E A V-1 be k observed samples from a Dirichlet ..... distribution over the (V- 1)- probability simplex. The method of moments estimator for 3 is given by
E x
E R(X,
1).
= E[,]
V V1
where, following Minka (2000), we set i j. and
EXX
x
V[X.
(x(0 B(X,))
4.2.3 Initializing LDA with
The estimated vector 3 is then used to initialize the LDA model, replacing the standard uniform fixed scalar prior. We modified the MALLET implementation to support vector valued á parameter, enabling topic-word distributions to reflect corpus-specific semantic structure from the outset. Apart from this change, the rest of the LDA inference pipeline in MALLET remains unaltered.
5 Experiments
We evaluate the effectiveness of our corpus-informet Dirichlet parameter by assessing its impact on standard LDA. Our goal is to determine whether data-driven initializtion can substantially improve topic quality and
5

---

Published in Transactions on Machine Learning Research (03/2026)
bring LDA closer to, even surpass, state-of-the-art topic modeling methods. Experiments conducted five diverse text corpora, using three complementary metrios. (Ablation studies appear in Appendix D.2, and the complexity analysis in Appendix D.4).
Datasets. To ensure a fair and objective evaluation, we use pre-processed datasets from the OCTIS frame- work (Terragni et al., 2021), thereby avoiding model-specific tuning of the preprocessing pipeline. Specifically, we experiment with four diverse OCTTS datasets: 20NEWsGROUP, BBC NEwS, M10, and DBLP, covering range of domains and document styles.
In addition, following BERTopic (Grootendorst, 2022), we include the TRUMPTWEETS (TT) dataset to test model performance on informal, short-form text. Since this dataset is not included in OCTIS, we apply OCTIS's preprocessing module with standard, commonly used filtering (see Appendix A for details). This setup allows us to evaluate our method both under standardized preprocessing and in a setting closer to real-world social media text. The statistical details can be seen in Appendix A.
Basclincs. We evaluate our approach against a diverse set of topic models. For classical and neural baselines implemented in OCTIS (Terrone et al., 2021), we include LDA (Blei al., 2003), NMF (Zhao et al., 2017), ProdLDA (Srivastava & Sutton, 2017); NeuralLDA (Srivastava & Sutton, 2017), and the ETM (Dieng et al., 2020). These models rely solely corpus-internal signals and do not incorporate any external knowledge.
We further compare against recent embedding-based methods that leverage pretrained representations: BERTopic (Grootendorst, 2022), FASTopic (Wu et al., 2024), and Contextual Contextual-Top2ee Top2Vec (Angelov & Inkpen, 2024). These models utilize external knowledge, typically through pretrained sentence embeddings such as MiniLM or the Universal Sentence Encoder. We follow each method's official implementation and recommended configuration as provided in their respective GitHub repositorics.
Additionally, we include MALLET (McCallum, 2002), highly optimized Gibbs-sampling-basel LDA im- plementation, and our proposed model, PRISM. Both were run using the MALLET framework with default hyperparameters, enabling internal parameter optimization (c.g., optimizeInterval) For PRISM, we further supplied the model with estimated B as described in Section 4.
Metrics. We evaluate topic models using statistical and human-aligned metrics that capture different dimensions of quality: c, Coherence, normalized pointwise mutual information (NPMI) and the word intru- sion detection ( (WID) task. Formal definitions appear in Appendix B. Coherence (Röder el al., 2015) and NPMI (Bouma, 2009) measure the semantic relatedness of top words within a topic, based on co-occurrence statistics. While both are widely used, we tend to favor c, duc to its empirically stronger corrolation with human judgments. WID (Chang et al., 2009) evaluates topic interpretability via the identification of an out-of-place word among a topic's top words. To scale this human-centrie task, we follow (Garg et al., 2023) and use large language models (LLMs) automated judges. WID implementation details are provided in Appendix B.2.
Setup. To enable fair and robust comparison, we evaluate all models under a unified protocol. For each dataset, we select three values of K near the reference number of ground-truth topics, along with slightly larger values to support finer-grained topic discovery. For PRUMPTWEETT: which lacks labels, we use comparable K values to those in labeled datasets (See Table 4). This protocol reflects weakly-supervise topic modeling paradigm, in which coarse supervision guides model behavior without enforcing strict topic- label alignment, following the principles outlined by (Zhao et al., 2017). All models are evaluated using the top 10 words per topic, and each configuration is run 10 limes to account for stochastic variation; final scores are averaged across runs and topic counts.
Models with adaptive topic selection are evaluated accordingly. For BERTopic, we report the best result across runs with and without fixed K; for Contextual-Topp21 Top2Ve,, which infers K automatically, we evaluate its output as-is. Further implementation details appear in Appendix C.
6

---

Published in Transactions on Machine Learning Research (03/2026)
Table I: Model performance across datasets. Each entry is mean (std) over three topic settings. Best is bold, second-best is underlined. t indicates PRISM outperforms MALLET for that metric.
20NG
BBC News
M10
DBLP
TrumpTweets
MODELS
NPMI
NPMI
NPMI
NPMI
NPMI
POoILDA
.5076 (.0216)
.0569 (J0657)
.427 (.0050)
-046 (0011) .4218 (10886)
-19951 (,0644)
.4718
(0196)
.072 (0464)
.5375 (.0090)
.0778 (,0029)
NeumILLAA
.5339 (,004))
00255 (,0010)
.5730 [,0234)
0632 (0274)
.4179 (.0163)
(.0450)
3833 (.1022)
(.008))
.4252 (.0150)
(.0057)
LDA
J532 (.077)
0457 (.0035)
.0124 (.150))
.00 (0168)
3833 (0116)
-1002 (,0455)
3647 (0014)
-227 (00622)
.4188 (.0196)
30000 (J004)
FIM
.212 (.0070)
0433 (.0049)
.9633 ( (0002)
0176 (00005
3559 (0015))
1225 (.0344
2828 (0068)
0389 (0152)
.4133 (.0338)
.0203 (.0136)
NMP
.597 (J0857
00050 (.0029)
.4664 (10909
.007 (0110)
.666 (10119)
-1943 (.0157)
2003 (J121.8)
.002 (0018)
.4558 (.0103)
JU826 (J1120)
BERTopic
5357 (.0155)
0887 [:0113)
.7124 ( (.0033)
16944 (.0022)
4030 (.0131)
.1310 (.0095)
3802 (.0075)
-009 (.0051)
.4461 ( [.0025)
- 0287 (.003))
VoTop2Voee
.5822 (.8227)
0082 (.0152)
.10S (.0484)
5750 (0202)
3863 (00033)
-195 [,0066
3185 (.0005)
-8828 (0011)
.8633 (.0028)
2249 (JK045)
FASTopic
574
(0028))
0225 (.0078)
6572 (.3114)
0036 (.116)
.475 (0081)
2065 (.0122)
3259 (.0027)
0012 (00002
.384 (.0245)
1115 (.0550)
MALLET
J822 (.0052)
1015 (.0057)
6384 (.0051)
.226 [0000])
.6711 (.0058)
.0671 (.0032)
4329 (.0155)
.9667 (.003))
.4959 (.0152)
.0701 (.009))
PRISM (ours)
(0081)
-1106' (.0061)
.878] (.0061)
(.012))
.5288 (.0054)
(.0132)
.643 ((U52)
(,0131)
.5571/ ((0036)
((0112)
Table 2: Word Intrusion Detection accuracy across five datasets (mean (std) over three topic settings). Best is bold, second-best is underlined; indicates PRISM outperforms MALLET.
MODELS
20NowsGroap
BBC
M16
DBLP
TrumpTwosts
ProILDA
4561 (.0165)
.556 (.0681)
.232 (,0322)
2528 (.0638)
.759 (.0722)
NeuraLDA
.233 (.0518
.3250 (.0000)
.1194 (,0211)
.1677 (.067))
.1296 (.0114)
LDA
.1722 (10222)
.131 (,9615)
.1130 (.0880)
J0S33 (.2211
.1611 (.0625)
ETM
3422 (0000)
.4592 (47726)
0989 (33713)
.194 (.2224)
.1300 (.117)
NMF
.1411 (.019))
.0830 (.3148)
.0000 (.8181)
.010 (.8161)
2426 (2077)
BERTopic
3811 (.0422)
.5537 (.0022)
.4907 (.0227)
.3584 (.0118)
31139 (.0103)
Topo2ee
8022 (.0009)
.5915 (.0618)
.3855 (.0188)
.4928 (.0366)
.354 (.1016)
FASTopic
5844 (.0285)
.3115 (.0600)
.370 (.04/2)
.722 (.0278)
2362 (0:166)
MALLET
.581 (.0200)
.8689 (.0412)
.3711 (J0466)
.0051 (.0554)
.2077 (J8022)
PRISM (ours)
(.012))
6201
(.0281)
(921((0017)
1,3406
(.0512)
1,3057 (0131)
Our Results. We evaluate PRISM across five benchmark datasets, focusing on two key questions: (1) whether it provides consistent improvements over classical LDA implemented in MALLET, and (2) how it compares to recent topic models, including both corpus-intrinsic approaches and those that leverage external semantic knowledge. The following results address both aspects. Models above the dashed line are direct basclines; those below use external embeddings and represent an upper bound (Table 1, Table 2).
Quantitative Results. As shown in Table 1, PRISM consistently outperforms the original MALLET im- plementation, achieving substantial gains in both cu and NPMI. Beyond improving upon MALLET, PRISM frequently closes the gap lo, or even surpasses, recent embedding- based methods. This is particularly evi- dent on the BBC News, M10, and TrumppTweets datasets, where PRISM not only significantly outperforms MALLET but also achieves the best or second-best scores across both metrics demonstrating competitive- ness with SOTA methods. PRISM obtains the best C. score three out of five datasets (20NG, M10, TrumpTwweetss and ranks second on BBC News and DBLP. Given that is widely regarded as more reflective of human topic judgments, these results suggest that PRISM produces more interpretable and se- mantically coherent topics across diverse domains. While NPMI improvements are somewhat more modest, PRISM still achieves the best scores on 20NewsGroup, M10 and TrumpTweets and remains competitive throughout.
To further assess topic interpretability, we employ the Word Intrusion Detection (WID) task (detailed in Appendix B.2). As shown in Table 2, PRISM ranks among the top two models three out of five datasets and remains highly competitive on the remaining two. Among corpus-intrinsic models (above the dashed line), PRISM consistently achieves the highest accuracy-ottperormiin MALLET and all other classical baselines. It also competes strongly with embedddingbbased methods (below the dashed line), outperforming all of them on 20NG and several on BBC and M1. Overall, these results show that PRISM not only domi- nates traditional models in both coherence and interpretability but also matches-and at times exceods the performance of state-of-the-arl models that rely on external knowledge.
Qualitative Analysis. Over BBC dataset, we show a comparison of PRISM to BERTopic, which achieved the highest e, and NPMI scores, and to ProdLDA, the strongest corpus-intrinsic baseline in e, (Table 1)
7

---

Published in Transactions on Machine Learning Research (03/2026)
Toplo
Toplo
Tople
Toplo4
Tople
govemment
film
win
gama
viras
mobile
election
2ivemment
film
company
win
tochnology
election
play
game
Victary
yesterday
quarter
gratimment
election
Rood
market
computer
party
award
player
phone
Erietor
shrnnk
Interest
plan
party
award
play
phane
labour
music
match
technology
software
aibum
investment
labour
music
rise
Player
mobile
device
program
tochnology
koynote
rate
tory
show
finel
servize
securiy
firm
gial
lote
greth
issue
tory
show
price
back
user
itse
stor
club
music
user
performance
declaration
law
Includo
economy
match
game
country
Rood
user
great
Sgure
leader
star
firm
Tax
octor
back
search
computer
music
ball
excitement
price
conservative
public
top
grorh
Glub
net
inevitable
stock
public
minister
final
music
(a) BERTopic: top-10 words
(b) ProdLDA: 1op-10 words
(c) PRISM:
top-10 words
Figure 2: Top-10 words per topic on the BBC dataset with K = 5. Each column is a distinct topic. Colors denote manually interpreted categorics (politics, entertainment, business, sporte, technology): lighter shades indicate weaker relevance and white indicates no clear association. Panels: (a) BERTopic, (b) ProdLDA, (c)
PRISM.
Toplc
Topic
Topic
Topic
Topic
Topic
Topic
Topic
gene
protein
gene
protein
soil
soil
water
expression
expression
structure
expression
sequence
water
water
soil
prolein
daturn
sequence
daturn
model
crop
crop
diesel
datum
microarray
dna
model structure
yield
yield
change
sequence
analysis
model
network dna
climate
change
ongine
cluster
cluster
motif
cluster motif
irrigation
area
model
dna
regulatory
fold
analysis
base
remote
climate
climate
microarray
model
prediction
microarray
fold
change
effect
effect
model
method
algorithm
regulatory
prediction
agriculture
production
production
clamsfication
bind
paper
production
remote
fuel
BERTopic
(b) PRISM
(c) MALLET
(a) BERTopic
(b) PRISM
(c) MALLET
Figure 3: Top-10 words for biology topics M10.
Figure 4: Top-10 words for a siimate/agriculture topic in M10.
over BBC. The datasel contains five ground-truth topic labels. As shown Figure 2, PRISM successfully recovers all five topics politics, business, sports, technology, and entertainment with minimal off-topic words. In contrast, BERTopic fails to recover the business category and cxhibits redundancy across two overlapping technology topics. ProdLDA clearly captures politics and business, while entertainment and technology are only partially distinguishable, and sports is entirely missing. These observations align with the WID results (Tabble 2), where PRISM ranks second overall, while BERTopic and ProdLDA rank fourth and seventh, respectively. This suggests that PRISM produces more coherent and uniquely identifiable topics. Supplementary examples and heatmap comparisons are included in Appendix D.1.
On M10, we compare PRISM to BERTopic and MALLET. PRISM achieves the highest cp and second-best WID score, while BERTopic leads in WID and NPMI, and MALLET ranks second in c, (Table 1, Table 2). As shown in Figure 3, all models capture biologically meaningful themes, though MALLET includes few less relevant terms (e.g., "paper," " "network"). BERTopic merges gene-related and protein-related concepts into a single broad topic, whereas both PRISM and MALLET separate them into two distinct but related topics, one focused on gene expression and microarray, the other on protein structure and binding. This finer- grained separation reflects PRISM's stronger topical coherence and is also evident its ranking of more meaningful terms (e.g., "microarray," "regulatory," "structure") with higher probability than MALLET, Interestingly, BERTopic's broader topic structure may contribute to its higher WID score: in the WID
8

---

Published in Transactions on Machine Learning Research (03/2026)
task, intruder words are sampled from other high-probability topics (details in Appendix B.2), and when topics overlap semantically, as with genes and proteins, intruders may feel topically adjacent rather than clearly out of place. PRISM's more specific topics make intruder detection more challenging, which may explain its slightly lower WID despite better topic distinctiveness. Figure 4 presents another M10 topic, likely related to climate and agriculture. PRISM and BERTopic show high overlap top-ranked words ("soil," " "water," " "crop," " "yicld," " etc.), with strong topic focus. In contrast, MALLET's output contains generic or loosely related terms ("engine," " "fuel," " "model"), leading to reduced coherence. These results support the quantitative findings, where PRISM outperforms MALLET across all metrics and approaches the interpretability of BERTopic without using external knowledge.
6 Biological Experiments
Motivation and Analogy. We investigate the applicability of PRISM to biological data, aiming to un- cover latent Biological Processes (BPs) from single-cell RNA sequencing (scRNA-seq) data. This task nat- urally parallels topic modeling: cells correspond to documents, genes to words, and BPs to topics. As in text, where documents often span multiple topics and words can take on different meanings depending on context, each cell may be involved in multiple BPs, and individual genes may participate in several biologi- cal functions, reflecting the many-to-many relationships captured by topic models. Furthermore, scRN A-seq data is organized as count matrix, where each entry denotes the expression level of a gene in cell, dircctly analogous to the word-document count matrix in LDA.
Datasets. We evaluate our approach on three scRNA-seg datasets spanning human and mouse tissues. BreastCancer comprises human breast tumor samples provided by collaborating laboratory in prepro- cessed form, PBMC3k contains peripheral blood mononuclear cells (PBMCs) from healthy donor," and Zeisel brain profiles cells from the mouse cerebral cortex. All datasets were preprocessed using a standard Scanpy workflow; no provider-specifie preprocessing was applied. Details are given in Appendix E.
Baselines. As proof of concept, compare PRISM to the standard LDA implementation in MALLET to test whether our corpus-intrinsic initialization improves the interpretability of biological processes in scRNA-seq data. We also include two widely used text-topic baselines: ProdLDA (Srivastava & Sutton, 2017) and NMF (Loe & Scung, 2001). Finally, we benchmark against two single-cellsspectne factorization models: scHPF (single-cell hierarchical Poisson lactorization) (Levitin el al., 2019) and cNMF (consensus NMF) (Kotliar et al., 2019).
Evaluation Metrics. We assess biological plausibility with an LLM-based scoring procedure following Hu et al. (2025). For each topic, we submit its top 20 genes to GPT-4 using a fixed prompt asking how likely the genes are co-participate in a Biological Process (BP), and record the returned confidence the topic's score. Averaging over topics yields proxy for biological coherence (details in Appendix E.2.2). In addi- tion, we report conventional gene-set quality metrics; definitions and full results appear in Appendix E.2.1 (definition)) and Appendix E.3 (results).
Experimental Setup. We evaluate all baselines and PRISM on the same scRNA-seq datasets under matched configuration: the number of topics is fixed to K = 5, and each model is trained for 10 independent runs per dataset. For PRISM, the 3 prior is estimated exactly in our text-corpus experiments, with a minor adjustment to the PPMI graph construction tailored to gene-gen co-occurrence (see Appendix E.1 for details).
hhttps://zenooo http://aaooooorreeeeeeeee//////d
hnttpe://scanpy. rewdthedoce.eeeee http:///wa....aaaaaooee///////a/ datasets. pbmc3k. (innnkkk.ttii html www.::/wwwnn.nn.i.....ne.m

---

Published in Transactions on Machine Learning Research (03/2026)
Table 3: Gene-set quality metrics for biological datasets. Each entry is the mean over 10 runs with = Coherence mean within-set Spearman correlation; Coverage fraction of pathway members recovered (averaged over significant pathways); Strength - (9) (FDR-adjusted). "_" indicates no significant pathway cnrichments (q 0.05). Best scores bold; second-best are underlined; indicates PRISM significantly outperforms MALLET for that metric (test details in the Supplement).
BreastCancer
PBMC3k
Zeisel brain
Models
Coherence Coverage Strength Coherence Coverage Strength Coherence Coverage Strength
cNMF
0.2695
0.0
0.3867
0.8
10.2447
0.6306
0.8
4.1679
sclIPF
0.2423
0.2
1.4056
0.1694
0.6
6.0057
0.4049
0.6
3.4919
ProdLDA
0.0956
0.0
0.0091
0.0
0.0273
0.0
NMF
0.0918
0.0
0.0068
0.0
0.0395
0.0
MALLET
0.2471
0.0
0.2952
0.8
9.0515
0.5541
0.8
2.2903
PRISM
0.2906
0.0
0.3488
0.8
11.8848
0.5987
0.8
3.7403
Results. As shown Table 3, two factors largely explain the observed patterns. (i) Dataset scale. The BreastCancer matrix contains only 297 genes (vs. 5,000 for PBMC3t//Zeise)) which limits the pathway universe and reduces statistical power for enrichment testing; accordingly, most methods show no significant enrichments Cancer despite reasonable within-set coherence. (ii) Model suitability. Methods geared toward text without biological inductive bias (ProdLDA, NMF) perform poorly across datasets- exhibiting low coherence and no enrichment whereas corpus-intrinsic, count-based topic models (MALLET, PRISM) and single-cell specific baselines (scHPF, cNMF) recover biologically meaningful structure. Across datasets, PRISM is consistently competitive with and often exceeds - scRNA-seq baselines, while also outperforming MALLET on every dataset, yielding higher coherence and stronger enrichments wherever pathways detectable. Overall, these results may indicate that LDA (as implemented in MALLET) is strong choice for biological gene-program discovery, and that Nomain-appropriate corpus-intrines intrinsit initial- ization, as in PRISM, further improves the biological interpretability of the inferred programs. Another LLM-based evaluation can be found in Appendix E.3.
7 Conclusion
We introduced PRISM, corpus-driven initialization method for LDA that integrates semantic structure de- rived directly from the data, without relying on external embeddings. Across five diverse datasets datasets- spanning news, social media, and biomedical texts-PRISM consistently improves coherence and interprotability over classical baselines and rivals embedding- on corpus-internal signals, highlights the underexplored potential of structure-aware initialization proba- bilistic models. This work opens new directions for enhancing topic models through data-intrinsic seman- ties bridging the gap between classical transparency and modern representational strength.
models. Its strong performance, despite operating entirely
7.1 Limitations and Future Directions
PRISM, like classical topic models, requires priori selection of K, treated as a tunable hyperparameter; while this may limit full automation across helerogeneous corpora, our experiments indicate robustness across practical range of K. A natural extension is corpus-driven initialization for the document topic prior aiming to adapt sparsity and enhance interpretability, and potentially further stabilizing training across datasets.
Acknowledgments
This research was funded in part by MOST grant number 8491/25
10

---

Published in Transactions on Machine Learning Research (03/2026)
References
Meta AL Llama 3.3-70b instruct, 2024.
URL https://augguuggaae...c//aaaaaaa
Accessed: 2024-02-10.
Dimo Angelov and Diana Inkpen. Topic modeling: Contextual token embeddings are all you need. In Findings of the Association for Computational Linguisties: EMNLP 2024, 2024.
Arthur Asuncion, Max Welling, Padhraic Smyth, and Yee Whye Teh. On smoothing and inference for topic models. In Proceedings of the Fwinty-Fifth Conference on Uncertainty in Artificial Intelligence (UAI), pp. 27-34, 2009.
Catarina G. Belém, Preethi Seshadri, Yasaman Razeghi, and Sameer Singh. Are models biased on text without gender-related language?, 2024. URL attps://arxiv, wrgg/bs/24000 2200 00588.
Yoav Benjarmini and Yosef Hochberg. Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society: Series B, 57(1))289-33)) 300, 1995.
Federico Bianchi, Silvia Terragni, and Dirk Hovy. Pre-training is a hot topic: Contextualized document embeddings improve topic coherence. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics (ACL), pp. 759 759-766, 2021.
David M. Blei, Andrew Y. Ng, and Michael I. Jordan. Latent dirichlet allocation. Journal of Machine Learning Research, 3:993 3:993-1022, 2003. URL
http//ppers////beee.aa. html.
Gerlof Bouma. Normalized (pointwise) mutual information in collocation extraction. In Proceedings of the Biennial GSCT, Conference, pp. 31 31-40, 2009.
John A. Bullinaria and Joseph P. Levy. Extracting semantic representations from word co-occurrence statis- tics: computational study. Behavior Research Methods, 44(3):510 526, 2012.
Sophie Burkhardt and Stefan Kramer. Decoupling sparsity and smoothness in the Dirichlet prior latent Dirichlet allocation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing, pp. 3089-3098, 2019.
Jonathan Chang, Jordan Boyd-Graber, Sean Gerrish, Chong Wang, and David M. Blei. Reading tea leaves: How humans interpret topic models. In Advances Newral Information Processing Systems (NeurIPS), volume 22, 2009. URL https:t/tppp/t attpss//proceeeiings eeteeedinnnnnuniiss....ee www.ips.cc/paper.r.m wtwlss/papee//00000 fftll/ 1alllaa3ee171111311585888866368
Paperr. pdf..
Kenneth W Church and Patrick Hanks. Word association norms, mutual information, and lexicography. Computational Linguistics, 16(1):22 29, 1990.
Ronald R Coifman and Stéphane Lafon. Diffusion maps. Applied and Computational Harmonic Analysis, 21(1):5-30, 2006.
Adji B. Dieng, Francisco J. R. Ruiz, and David M. Blei. Topic modeling in embedding spaces. Transactions of the Association for Computational Linguistics, 8:439-453 2020. doi: 10.1162/tacl a a_0025...
Siddhant Garg, Dinesh Balasubramaniam, Aniruddha Guha, Dipankar Das, and Shubham Jain. Llm reading tea leaves: Automatically evaluating topic models with large language models. In Findings of the Associa- tion for Computational Linguistics (ACL), 2023. URL https://aclanttii 550/-
org/ vrrg/223..m findings-aci..
Kshitish Ghate, Isaac Slaughter, Kyra Wilson, Mona T. Diab, and Aylin Caliskan. Intrinsic bias is pre- dicted by pretraining data and correlates with downstream performance in vision-language encoders. In Luis Chiruzzo, Alan Ritter, and Lu Wang (eds.), Proceedings of the 2025 Conference of the Nu- tions of the Americas Chapter of the Association for Computational Linguistics: Human Language Tech- nologies (Volume 1: Long Papers), pp. for Computational Linguisties. ISBN 879---891166189-6 doi: 10.18653/ /v1// v1/2222222222222 htap//22..aaalllleee long. URL w02/ naacl- 148/.
Albuquerque, New Mexico, April 2025. Association
11

---

Published in Transactions on Machine Learning Research (03/2026)
Thomas L. Griffiths and Mark Steyvers. Finding scientific topics. Proceedings of the National Academy of Sciences, (supll 115228-52355 2004. doi: 10.1073//maas.00077222
Maarten Grootendorst. Bertopic: Leveraging bert and c-tf-idf to create easily interpretable topics. arXiv preprint, 2022. URL fttpp::/aaxiv.orggg. wttgg///////////o//////////////e 259424
Minji Hu, Safa Alkhairy, Irene Lee, Ryan T Pillich, Daniel Fong, Kevin Smith, Rebecca Bachelder, Trey Ideker, and Daniel Pratt. Evaluation of large language models for discovery of gene set function. Nature Methods, 22(1):82 91, 2025. doi: 10.1038 /41 592-024-02525--
Jagadeesh Jagarlamudi, Hal Daumé III, and Raghavendra Udupa. Incorporating lexical priors into topic models. In Proceedings of the 13th Conference of the European Chapter of the Association for Computa- lional Linguislics, pp. 204 213. Association for Computational Linguistics, 2012.
Weonyoung Joo, Wonsung Kim, Sungrae Shin, and Il-Chul Moon. Dirichlet-based variational auto-encoder. In International Conference on Artificial Intelligence and Statistics (AISTATS), pp. 2050 2050-2060, 2020.
Kasia Z. Kedzierska, Lorin Crawford, Ava P. Amini, and Alex X. Lu. Zero-shot evaluation reveals limitations of single-cell loundation models. Genome Biology, 26(1):101, 2025. doi: 10.1186 000/6/113300 1130500225-0357444
Aobo Kong, Shiwan Zhao, Hao Chen, Qicheng Li, Yong Qin, Ruiqi Sun, Xin Zhou, Enzhi Wang, and Xiaohang Dong. Better zero-shot reasoning with role-play prompting. arXiv preprint or arXi::2388.07000 2023.
D. Kotliar, A. Veres, M. A. Nagy, S. Tabrizi, E. Hodis, D. A. Melton, and P. C. Sabeti. Identifying gene expression programs of cell-type identity and cellular activity with single-cell rna-seq using cunf. eLife, 8te43803, 2019. doi: 117555/eeLif.445
Daniel D. Lee and H. Sebastian Seung. Algorithms for non-negative matrix factorization. In Advances in Newral Information Processing Systems 13 (NIPS 2000), pp. 556 562, 2001.
H. M. Levitin, J. Yuan, Y. L. Cheng, F. J. Ruiz, D. Pe'er, and et al. De novo gene signature identification from single-cell rna-seq with hierarchical poisson factorization (schpf). Molecular Systems Biology, 15(2): e8577, 2019. doi: 10.15252/ms/ m 20188557.
Omer Levy and Yoav Goldberg. Neural word embedding as implicit matrix factorization. In Advances in Neural Information Processing Systems, 2014. doi: 10.5555 /ttp//299933299999 URL https //ppser//ppprrs.. nips.cc/ attd
Malte D. Luecken and Fabian J. Theis. Current best practices in single-cell rna-seq analysis: tutorial.
Molecular Systems Biology, 15(6):08746, 2019. doi:
Zheheng Luo, Lei Liu, Sophia Ananiadou, and Qianqian Xie. Graph contrastive topic model. Ex- pert Systems with Applications, 255:124631, 2024. doi: (tw.://j.ccaa22224122000 URL https: //www.
Andrew Kachites McCallum. Mallet: machine learning for language toolkit, Technical software, University of Massachusetts Amherst University of Pennsylvania, 2002. URL http://mallet.com umass. edu.
Thomas P. Minka. Estimating a dirichlet distribution. Technical Report MSR-TR-2000-7, Microsoft Re- search, 2000. https://tminka. wtw.ggtb.....pppee//ttteteeeoeee
Thong Nguyen, Xiaobao Wu, Xinshuai Dong, Cong-Duy Nguyen, See-Kiong Ng, and Luu Anh Tuan. Topic modeling as multi-objective contrastive optimization. In International Conference on Learning Representations (ICLR), 2024. URL
iclr. caw.arccc/pperrcc. /ttp//ppr..llee/ppppr/////////// pdf.
Douglas A Reynolds. Gaussian mixture models. Encyclopedia of biometrics, 741:659 663, 2009.
12

---

Published in Transactions on Machine Learning Research (03/2026)
Michael Röder, Andreas Both, and Alexander Hinneburg. Exploring the space of topic coherence measures. In Proceedings of the eighth ACM international conference on Web search and dala mining, pp. 399-408, 2015.
Hinrich Schütze. Automatic word sense discrimination. Compulational linguistics, 24(1):97-123, 1998.
severse. Scanpy documentation: pp apphighly.varriale.. gennes. https://scanp:::::: stable/generated/ swaaay.aaapppaaaaeaaa peeaamyy.p...em highhy_variablee Serbbleeeenee_eeeeeeeeee hemll, accessed 2025.
io/en/
Ziyang Song, Yuanyi Hu, Aman Verma, David I.. Buckeridge, and Yue Li. Automatic phenotyping by a seed-guided topic model. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Dala Mining (KDD '22), pp. 113-4722, Washinglon, DC, USA, 2022. Association for Computing Machinery. doi: 10.1145 10.1145/353447555 /3334668.33422675
Akash Srivastava and Charles Sutton. Autoencoding variational inference for topic models. In Interna- tional Conference on Learning Representations (ICLR), 2017. URL httpe:///peereeie.eeeeteeeeeee id=BybtVK91g.
Timothy Stuart, Andrew Butler, and al. Comprehensive integration of singlo-cell data. Cell, 177(7): 1888 1902.e21, 2019. doi: 10.1016/...el..2019005...
Silvia Terragni, Xuan Tien Doan, Brian Davis, Bedoor AlShebli, Suppawong Tuarob, Pascal Frossard, Javier Norge-Holthoefer Raheel AlGhamdi, and Peter Gloor. Octis: Comparing and optimizing topic models is simple! In Proceedings of the 16th Conference of the European Chapter of the Association for Computa- tional Linguisties: System Demonstrations, pp. 263 270, 2021.
Leonardo Terrone, Federico Bianchi, Elisabetta Fersini, and Matteo Palmonari. Octis: Comparing and optimizing topic models is simple! In Proceedings of the 16th Conference of the European Chapter of the Association for Compulational Linguisties: System Demonstrations, pp. 263-270, 2021.
Isaac Virshup, Sergei Rybakov, Fabian J. Theis, Philipp Angerer, and F. Alexander Wolf. anndata: Access and store annotated data matrices. Journal of Open Source Software, 9(101):4371, 2024. doi: 10.21105/ j4ss.04371.
Hanna Wallach, David Mimno, and Andrew McCallum. Rethinking LDA: Why priors matter, In Advances in Neural Information Processing Systems, volume 22, pp. 1973-1981. Curran Associates, Inc., 2009.
Jason Wei, Xuczhi Wang, Dale Schaurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoe V Le, and Denny Zhou. Chain of thought prompting elicits reasoning in large language models. In Advonces in Newral Information Processing Systems (NeurIPS), 2022.
F. Alexander Wolf, Philipp Angerer, and Fabian J. Theis. Scanpy: large-scale single-cell gene expression data analysis. Genome Biology, 19(1):15, 2018. doi: 10.1 10.1166/1130599--77--8222
Xiaobao Wu, Thong Nguyen, Delvin Ce Zhang, William Yang Wang, and Anh Tuan Luu. Fastopic: Pre- trained transformer is fast, adaptive, stable, and transferable topic model. 2rXiv:2405.19978 2024.
Wayne Xin Zhao, Songbo Tan, Rui Zhang, Xueqi Li, Yongdong Zhang, and Ji-Rong Liu. Weakly-supervised text classification with topic modeling. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management (CIKM), pp. 2235-22252 2238. ACM, 2017.
Yuesong Zou, Ahmad Pesaranghader, Ziyang Song. Aman Verma, David L. Buckeridge, and Yue Li. Mod- eling electronic health record data using an end-to-end knowledge-gran saatee-rrap--iiforree topic model. Scientific Reports, 12:17868, 2022. doi: 10.1035/s415988 022-22956-w.
13

---

Published in Transactions on Machine Learning Research (03/2026)
A Datasets
As described in See 5, we evaluate on five benchmarks. Four are standard corpora from OCTIS (preprocessod by its internal pipeline). We also include Trump Tweels, following BERTopic (Grootendorst, 2022), to assess performance on short, noisy social media text. Summary statistics are in Table
Table 4: Dataset statistics. Arg. Len. denotes average document length in tokens; K lists the topic counts used.
Dataset
#Docs
Vocab
Avg.
Len.
#Labels
20NewsGroup
16,309
1,612
48.0
20
20, 25, 50
BBC
2,225
2,949
12.1
5
15
M10
8,355
1,696
5.9
10
10, 15, 20
DBLP
54,595
1,513
5.4
4
4, 10, 15
Twenp
18,239
1,988
9.0
10, 15,
The Trump Tweels dataset was obtained from the same source cited by BERTopic." To ensure consistency across datasets, we applied the OCTIS preprocessing module with basic filtering:
Preprocessing(
vocabulary=None,
Lovercase=True,
remove_nin coroov____mmbrr-rrue,,
min_ win_words_docse
min_ sin_chars=3, min_ df=.01, max axx_df=.9, max sas_features=2200,
renove_.umcctaatioo-Truu remove_puncture
Lemmatize=True,
stopword_liste linpoodddddsst==eniisssle
This configuration balances document retention and vocabulary quality, which is especially important for short texts such as tweets.
B
Metrics
We assess topic quality using corpus-intrinsic coherence (cx) and normalized pointwise mutual information (NPMI), plus a language-model assisted interpretability evaluation.
B.1 Standard Topic Modeling Metrics
C, Coherence. The c, metric (Röder et al., 2015) combincs pairwise NPMI with cosine similarity over context vectors derived from a sliding window:
C₁ = WI(WW 1) NPMI(w;, w5) cos(vi, Vÿ),
(1)
where W is the set top-N words per topic and v, E C encodes co-occurrence statistics wy across the corpus C. We compute e, with Gensim's CoherenceModel (defaults).
14

---

Published in Transactions on Machine Learning Research (03/2026)
NPMI. Following (Bouma, 2009),
(ww,,,; log P P(wr))((u
NPMI w. wy) log P(w; wy)
(2)
which normalizes PMI to [-1,1] for comparability across corpora.
B.2 Word Intrusion Detection (WID)
To complement statistical metrics with praxy for human interpretability, we use the Word Intrusion Detection (WTD) task. General framework of the metric can be viewed in Figure 5.
Prompt::
"From the follouing List of words,
Find the intruder ward
Topic
Player
Rodal's Reaponsm:
Apple Football
LLM
"Aple"".
Ball
Figure 5: Iustttaiiion of the Word Intrusion Detection (WID) framework. A large language model prompted to identify the word that does not belong in a list of top topic words (a.k.a. the intruder). The prompt shown here illustrative; actual prompis used in our experiments follow a more structured format.
The Word Intrusion Detection (WID) task (Chang et al., 2009) is widely adopted human-centered eval- uation method for assossing topic interpretability. In this task, annotators are shown the top-N words of a topic, one of which is an intruder i.e,, a word drawn from another topic that appears with low prob- ability in the larget topic but prominent elsewhere. The annolator is asked to identify the word that does not semantically belong. Higher topic coherence typically results easier and more consistent intruder identification, making WID an indirect yet effective proxy for human interpretability.
Recent studies have proposed leveraging large language models (LLMs) to automate WID (Garg et al., 2023), enabling scalable, consistent, and human-aligned evaluation. We adopt this paradigm by employing LLM as automatic evaluator within our WID framework (sce Figure 5). This model is prompted to identify the intruder from each modified topic word list, effectively simulating human judgment without the need for manual annotation.
Pipline. Our pipeline leverages HuggingPace's transformers library. We initialize the tokenizer via AutoTokenizer. www.ffoooretiiee prttoppretraiedd,.m explicitly setting the end-of-sequence (eos) token as the padding to- ken to ensure consistent handling of short text inputs. The LLM integrated with the pipeline APT using device device_map="autoo for efficient hardware mapping and torch, aarch_dtyp==tordd bfloat16 to reduce memory overhead.
During inference, each topic's word list is modified by injecting one intruder word. The model is then prompted to identify the semantic outlier. Its success in this task reflects the semantic cohesion of the topic, thus serving as an indirect interpretability metric that complements statistical scores.
The task involves identifying an intruder word inserted into an otherwise coherent topic word list. We evaluate performance using the Mita--LaaMA-33-770000 Instructt model (AI, 2024), which demonstrates strong alignment with human judgment.
15

---

Published in Transactions on Machine Learning Research (03/2026)
"You are assistant word intrusion detection task.
From the following list of tokens, identify the one token that does not belong with the others.
For exanple: for 1. Banana 2. Orange 3. Japan 4, Stramberry Tree The expected answer is index 3.
the intruder's index is since Banana, Orange, Stramberry and Tree are related to fruits in a way, but Japan is country.
The
Another example: for 1. Dog 2. Cat 3. Horse 4. Apple 5. Pig The expected answer index
The reason the intruder's index is since Dog, Cat, Horse and different kinds of animals, while Apple is a fruit.
Here are your words: wounbered_word_ttee
In your response, you have to provide only one index betueen 1 to (TenCaords_ index is intruder word's index the given List. Mithout any additional explanations."
Figure 6: Prompt template for the Word Intrusion Detection (WTD) task, with instructions, two illustrative examples, and structured response format. The input variable numbered_ _oordd_orr__ list inserted at evalu- ation time to test different word sets.
Algorithm Word Intrusion Detection (WID) Pipeline
Input: set of topic models M; prompt template
Output: saved LLM outputs; model-wise accuracy scores
for each model M E .M do for each topic T E ...TOPICS do for each word list size s {10, 15, 20) do
W (T)
Widx < ADDINDICES(W)
prompt &- FORMATPROMPP(WWaa)
result QUERYLLM(prompd)) (AVEOUTPUT(resutt)
EVALUAATAAACUUAACY(M)
b Eq. 3
Prompt Enginecring. Inspired by chain-of-thought prompting (Wei et al., 2022) and role-play prompt- ing (Kong et al., 2023), we crafted prompts to guide the large language model (LLM). The template includes two examples that demonstrate both intruder identification and the expected output format (Figure 6).
Evaluation Metric. Accuracy is computed as the proportion of correct intruder identifications:
Accuracy = Count(LLM response = true intruder)
(3)
K
where K is the number of topics. We report accuracy separately for top-10, top-15, and top-20 word lists.
This pipeline evaluates topic coherence by detecting intruder words terms that do not within topic word dists-using an LLM. For each model, we form top-10/15/200 word lists per topic, index the words (ADDINDICES), format a structured prompt (FORMATРRомPт), query the LIM (QuERyLLM), and save outputs. We then compare predictions to the known intruders compute accuracy (EVALUATEAACCURACY))
16

---

Published in Transactions on Machine Learning Research (03/2026)
C
Setup
C.1 Models
To ensure fair and reproducible comparisons, we evaluate a diverse set of topic modeling baselines using publicly available implementations under their recommended configurations.
OCTIS Models. We rum LDA, NMF, ETK, ProdLDA, and NeuralLDA via the OCTIS framework, using its standardized preprocessing pipeline and default hyperparameters unless noted.
ETM configurations.
We consider two ETM variants:
Without pretrained embeddings: the default OCTIS configuration.
With pretrained embeddings: GloVe initialization:
(Tuum___tooo topics-TOPICS NNM, (tteddinggsize=100)
100d. vec",,
Results were similar across the two settings (no detectable improvement from GloVe), so we report the corpus-only variant in the main tables.
BERTopic. We use the official implementation' with default parameters, except we explicitly set the number of topics to match our experimental setup; when applicable, we report the better of auto-detected fixed-K configurations.
Contextual Vec. We run Vee in contextual-embedddin mode with:
Top2Vec(
documents,
splitt_documens=-Tru,
contextual_toopveeeceee
Speed="deep-learyy
workers=2
This follows the recommended usage from the official repository: 6 We also tested custom tokenizer, which did not improve performance; therelore, we use default tokenization in reported results.
FASTopic. We use the TopMost implementation' with the recommended preprocossing utility:
preprocessing Preprocess (etopoord==="=""))i)))))
model= MASTopic(uum,.. (ttnnnnnuutnocaeeeeeeeeeeeee wwtppppppiitpcoooootoooootttttit num, preproces-preprocesiing
Hyperparameters follow the defaults in the official GitHub repository," with no additional tuning.
C.2 Computing Infrastructure
Local (WSL). ASUS Vivobook S 15/16 OLED; Windows 11 + WSL2 (Ubuntu). CPU: Intel Core Ultra 9 185H (11 cores / 22 threads); RAM: 16 GB. GPU: not utilized. Used for PRISM, MALLET, and OCTIS models (LDA, NMF, ProdLDA, ETM, NeuralLDA), and BERTOPIC.
http::///ggtaab.cccyyytooooo//// htttp::/ggtuubb.cmm//bb////////
17

---

Published in Transactions on Machine Learning Research (03/2026)
Server. Limux (Ubuntu). GPUs: 4x NVIDIA L40S, each with ~46 GiB VRAM (driver 575.57.08, CUDA 12.9). Example utilization snapshot (from nvidiasmi): GPU 0/1/3 active (#28.6/17.9/17.1 GiB). (Host CPU/RAM not recorded.) Used for FASTOPIC and CONTEXTUAL-TOF22EET
C.3 Licenses of External Assets
We list the license for each third-party asset used.
Table 5: Licenses terms for third-party software and assets.
Asset
License Terms
Software/ / librriies
OCTIS
MIT License"
Gensim
LGPL v2.1 (or later)
MALLET
Apache License 2.0
BERTopic
MIT License
Contextual-Topp2eer (Top2Vec)
BSD 3-Clusse 13
FASTopic / TopMost
Apache License 2.0
ETM (reference implementation)
MTT License 15
CTM (Contextnalized Topic Models)
MTT License
Teat datasets
OCMS preprocessed corpora (20NG, BBC, M10, DBLP) Trump'Iweets (TTh Trump Archive)
Original source terms via OCTIS dataset cards" 17 Site terms
TTitter TOS11
Biological datasets
PBMC3k (10x Genomics)
CC BY 4.
Zeisel mouse brain (GSE60361, CEO)
GEO usage policy (no explicit license))
BreastCancer (Zenodo: 10.5281 (2eenodo.1060000000
CC BY 4.0
Additional Experiments
D.1 More Qualitative Findings
To complement the qualitative findings presented in the main paper, we include additional quantitative analysis for BBC and M10 dataset here.
BBC Dataset. Figure 7a (MALLET) and Figure 8b (PRISM) display the top words for each topic on the BBC dataset with 5 topics. While MALLET produces reasonable topics, it redundantly captures politics in two separate themes and fails to isolate the entertainment domain. contrast, PRISM yields distinct and semantically meaningful topics, effectively covering all major themes the corpus. These results suggest that while MALLET offers solid inference framework, our initialization method pushes the model further
"http://aallet.cs. umass.edu/ (see LICENSE in the distribution)
httppp/s///ggg..rr//o///e////rss (license field)
21
18

---

Published in Transactions on Machine Learning Research (03/2026)
Topic
Topic
Topic
Topic
Topic 5
Topic
Topic
Topic
Topic
Topic
company
technology
law
win
election
government
film
company
win
technology
market phone
case
good
government
election good
market
game
computer
rise
mobile
court play
party
party award
firm
play
phone
sale
game government game
labour
labour music
rise
player mobile
firm
service rule film
plan
plan win
good
service
price music claim award
tory
tory show price back
user
share user legal player
public
law Include economy match game
growth computer charge back country
issue star share team firm
economy net police show
work
public top growth
club net
month firm
ban
world
minister
minister actor
month final music
(a) MALLET Top 10 words
(b) PRISM Top 10 words
Figure 7: Top-10 words per topic the BBC dataset with K 5. Each column is distinct topic. Colors in the panels denote mamually interpreted categories politice, entertainment, business, sports, and technology); lighter shades indicate weaker relevance and white indicates no clear association. Panels: (a) MALLET, (b) PRISM.
orere
-10
(a) FASTopic
(b) PRISM
ProdLDA
Figure 8: Hcatmaps on the BBC dataset with K = 5. Each column corresponds to a topic. Each column corresponds to a topic and each row to a top-ranked word (from the union of topics). Cell colors encode the topic-word relevance (i.e., Φk shown on log scale the colorbar): lighter/yelle shades indicate higher probability mass, whereas darker/pure shades indicate lower probability. Panels: (a) FASTopic, (b) PRISM, (c) ProdLDA.
toward more coherent and semantically distinct topics, indicating that the observed improvements stem from our approach rather than the base model alone.
Figure 8 further corroborates these observations using topic topic-word relevance heatmaps. Although FASTopic and ProdLDA identify differentiated themes, some topics remain less cleanly interpretable: FASTopic's TO blends business and technology terms, and ProdLDA fails to recover a coherent entertainment topic (its T3 is dominated by largely generic words), despite producing reasonable sports, politics, and technology-related themes elsewhere. In contrast, PRISM yields five clearly delineated topics aligned with the known BBC domains, concentrating high probability mass on consistent, domain-specific vocabularies while still allowing limited, context-driven lexical overlap across related topics. This graded overlap further motivates our of PRISM for scRNA-seg, where genes can participate in multiple biological processes and thus benefit from representations that accommodate overlapping gene programs.
M10 Dataset. Figure highlights contrasting qualitative behaviors on the heterogeneous M10 corpus. BERTopic yields highly coherent, easily nameable topics with almost block-diagonal structure, indicating
19

---

Published in Transactions on Machine Learning Research (03/2026)
PRISM
M10 K=10 BERRpiic
ferd
Cab
www
M10 K-10) lastopic
-1
(a) BERTopic
(b) PRISM
(c) PASTopic
Figure 9: Heatmaps on the M10 dataset with 10. Each column corresponds to a topic and each row top-ranked word (from the union of topies). Cell colors encode topic topic-word relevance (i.e., Φk shown log scale in the colorbar): ligter/yellow shades indicate higher probability mass, whereas darker darker/purple shades indicate lower probability. Panels: (a) BERTopic, (b) PRISM, (c) FASTopic.
near-disjoint topic assignments and minimal redundancy; however, this sharp separation largely suppresses meaningful lexical sharing across related scientific contexts. FASTopic recovers several coherent themes (e.g,, finance, learning, energy, and genomics), yet exhibits noticeable redundancy and blurred boundaries: materials materials/methodde relaedd- terms (e.g., material, composite, microstructure) receive elevated relevance across multiple topics, suggesting topic duplication rather than distinct semantic cores. In contrast, PRISM main- tains clear lopic centers while allowing selective, semantically plausible overlap; for example, while gene is most strongly associated with the expressin topic (T6), it retains neglieglg probability mass in the protein/ste protein/structure sstructure topic (T9), reflecting context-dependent usage and enabling richer representations than either BERTopic's near-exchusive clusters FASTopic's diffuse redundancy.
D.2 Ablation: Priors and Embeddings
To assess the contribution of individual components our method, we conducted ablation studies focusing on priors and embeddings. Specifically, we evaluated four variants: (1) replacing our custom word embed- dings (constructed via soc-PPMI and diffusion maps) with pretrained GloVe embeddings; (2) introducing random prior within the MALLET framework to test whether improvements arise solely from enabling asymmetric 9 prior (unsupported in the original MALLET, which assumes scalar B); (3) using the raw PPMI matrix without applying cosine similarity, to test whether the latter indoed enhances the cap- ture of indirect semantic associations beyond direct co-occurrence; and (4) substituting diffusion maps with SVD to examine whether SVD alone provides more effective embeddings. For each variant, we report both standard statistical metrics (cu, NPMT) Table and WID task performance in Table 7. Overall, the ablations consistently underperformed the full model, confirming that both the embedding construction and the asymmetric & prior necessary to achieve the observed gains.
20

---

Published in Transactions on Machine Learning Research (03/2026)
D.2.1 Statistic Metrics
We report ablation results using two standard coherence metrics, c, and NPMI, consistent with the evaluation used the main paper.
Table 6: Coherence across datasets for ablation variants. Each entry is mean (std), Best is bold, second-best is underlined.
MODELS
Cropp
BBC
M10
DBLP
Tramp Tweets
NPMI
NPMI
NPMI
NPMI
NPMI
GlaVe- GMM Piter
6924 (.0021)
1130 (.0076) 6396 (00119
.1100 (0000)
.4539 (.0129)
0658 (,0100)
4503 (,0310)
.088 (.0177)
.4968 (00055
.0899 (.0091)
Randoo Price No SOC (PPMI-DM)
6230 (10133)
1100 (0104)
8120 [0096)
11181 (OL07)
.4222 (J0644
.017 (J1332
(J1476)
0501 (02214)
4910 (0075)
.I578 (.0025)
6682 (.0108)
1164 (.068))
45:0 (0219)
1230 (0072)
0152 (.0018)
0731 (.0251)
.4799 (0209)
.0742 (.014)
1191 (0057)
(850 (.009))
SVD ln piace of DM
.6436 (.0117]
1159 (.005])
8436 (.0141) 41)
1111 (0:28)
.0814 (.0116)
.094 (J1129)
4611 (.008))
.016 (0016))
.400 (0161)
.0997 (.0722)
PRISM (rwforenc))
.6592 (0081))
.1168 (0061) .0781
[.0161)
.1408
(.0179)
.5285 (.009))
.0822 (0152)
(,0152)
(00131)
.5571 (.0366
.0981 (.0122)
The results in Table 6 highlight the consistent advantage of our method. Across all datasets, PRISM achieves the highest coherence scores, with the sole exception of DBLP, where it ranks second. Notably, the variant that omits cosine similarity (No SOC) emerges as the most competitive ablation, frequently attaining the socond-best results. This suggests that the overall framework of our method effectively captures meaningful similarities, while the cosine-similarity step further enhances performance.
D.2.2 Word Intrusion Detection
We also evaluate the ablation variants on the Word Intrusion Detection (WID) task, which was used in the main paper to assess topic interpretability. Results are reported in Table 7. Across all datasets, PRISM consistently outperforms the ablation baselines, achieving the highest WID accuracy. These findings indicate that PRISM produces more coherent and interpretable topics, enabling intruder words to be more reliably identified.
Table 7: Word Intrusion Detection (WID) accuracy for ablations across datasets. Each entry is mean accuracy (std) over 10 runs. Ablations: GloVe Embed (GloVe term embeddings - GMM; MoM to Dirichlet prior), Random Prior (random vector prior), No SOC (PPMI - DM; cosine-similarity step removed), SVD in place of DM (SOC-PPMI + SVD - GMM).
MODELS
20NewsGroup
BBC
M10
DBLP
TrumpTweels
GloVe Embed
.5590 (.0480)
.5778
(.0072)
.3622 (.0492)
.2844 (.0376)
.2733 (.0618)
Random Prior
.5277 (.0494)
.5467 (.0091)
.3161 (.0547)
.2506 (.0488
.2328 (.0627)
No SOC (PPMI-DM)
.5677 (.0492)
.5233 (.0145)
.3633 (.0577)
.2750 (.0467)
.2744 (.0687)
SVD in place of DM
.543 (.0514)
.5533
(.0132)
.3589 (.0615)
3133 (.0455)
.2411 (.0714)
PRISM (reference)
.6099 (.0124)
.6201 (.0281)
.3921 (.0176)
.3408 (.0512)
.3057 (.0131)
D.3
Ablation: PPMI Context Window
D.3.1
Statistic Metrics
We report ablation results using two standard coherence metrics, G, and NPMI, consistent with the evaluation used in the main paper. The results in Table 8 highlight the consistent advantage of our method. Across all datasets, PRISM achieves the highest coherence scores, whether documents are very short or long.
Table 8 compares PRISM priors constructed from PPMI estimated with fixed sliding windows (w E 2, 5, 10) versus a document-level window. The results show a clear and consistent pattern: enlarging the context window generally improves coherence, but the document-level context yiclds the strongest and most stable coherence across all datascts. This is particularly important given the wide variation in average document length.
21

---

Published in Transactions on Machine Learning Research (03/2026)
Table 8: Ablation on PPMI context window size. Each entry is mean (std) over three topic settings. Best
is bold, second- best underlined.
2ING
DELP
TrumpTTree
MODELS
NPMI
NPMI
NPMI
NPMI
NPMI
PRISM (w-2)
.9722 (0065)
1100 (00688
.639 (0120)
.120 (01130)
.4452 (.0000)
0645 (.000))
.4400 (.0140)
0080 (.0100)
5250 (.0100)
.0780 (0000)
PRISM (w=5)
4190 (0072)
(.055))
(0135)
.1502 (0100)
.505 (.003))
0705 (00105)
.4525 (.0151)
0710 (0120) 20)
5310 (.009))
(.0100)
PRISM 10=
6538 (0078)
1111 (.000))
6692 (0110)
(1319 (.0175)
.006 (0040)
.0630 (00200)
.4344 (.0150)
0638 (0130)
5329 (.0100)
.973 (0110)
MALLET
/332 (0002)
1018 (.0077)
6384 (.0051)
1285 (.0101)
.457 (0008)
0671 ( (.0032)
.4229 (.0133)
0467 (0089)
4989 (.0152)
.701 (004)
PRISM (doc-level)
.6992 (,0081)
.1168 (0061)
.6781 (00161)
.1468 (0179)
.5285 (.0034)
0622 (0132)
4643 (.0122)
.0751 (00111)
.5711 (0096)
.0991 (.0112)
Table 9: Ablation on PPMI context window size for Word Intrusion Detection (WID) accuracy (mean (std) over three topic settings). Best is bold, second-best is underlinod.
MODELS
20NewsCroou
BBC
M10
DBLP
TrumpTweets
PRISM (w=2)
5550 (.0180)
.5000 (3330)
(.0220)
3150 (.0500)
.2870 (00100)
PRUSM (w=:]
5975 (.0150)
5950 (00250)
.3800 (.190))
3310 (,0680)
.960 (0150)
PRISM (w-10)
8068 (.0180)
6150 (.020)
39910 (.0180)
3385 (.0520)
2006 (0104)
MALLET
(J0266)
.6689 (0412)
.871 (.046)
.3051 (00544
.007 (0002)
PRISM (doc-level)
.8099' .8090"(0124 (00144)
.62011 (2281))
(.016)
.3408"
(0037((0131)
D.3.2 Word Intrusion Detection
We also evaluate the ablation variants on the Word Intrusion Detoction (WID) task, which was used in the main paper lo assess topic interprelability. Results in Table 9 mirror the coherence trends and strengt hen the justification for our default choice: PRISM with document-level context window attains the highest WID accuracy across all datasets (and consistently exceeds the fixed-window variants). Because WID mcasures whether an intruder can be identified from a topic's top words, higher accuracy indicates that the lopic word lists are more coherent to humans.
D.4 Complexity Analysis
Table 10: Dominant time and space complexities.
Model
Time (dominant)
Space (dominant)
Bo W topic/factor models
LDA
((nnz(X) K) + ((V]K)
(nnz((X) + NK + (V[K)
NMF
(mnz(XX K)
(nnz(() + NK + (У]K)
PRodLDA
((nz(X) K) + O((e))
O (nnz((X + N + [V]K + (0])
ETM
O(nz(() K) + O((]K)
O(nn(X) + NK (V]K) O(nnz(X) (X) + NK + |VJK + (0])
CTM
((nz(X) K) O(()))
MALLET
O(nnz (X) Racs)
O(nnz(X) + NK IV])
Embedding clustering pipelines
BERTopic
O(N Cenc) O(N log N) O(N?) O(IVK)
O(N demb (VIK) O(N demb) ((nz(XX + (V(
Contextual-Top221
O(N cene) + O(N log N) + O(N3) O(nnz(X)) O(IVIK)) O(IVI))
FASTopic
Our model
PRISM (ours)
O(nna(X) Kaet) + O(1V1))
O(nz((X) + NK + [V]K) + O(IV))
Reading Table 10. Symbols. N: #docmeents; VV: vocabulary size; K: #topics; X N sparse with nnz(X) nonzeros; IO) number of trainable neural parameters (encoder/decodel Cene! per-doeument embedding cost of the text encoder; demb: embedding dimension; Ract: average number of active topics per
22

---

Published in Transactions on Machine Learning Research (03/2026)
Table 11: Wall-clock runtime seconds (mean (std) over three runs).
Model
20NuwsGroup
BBC
M10
DBLP
TrumpTweets
MALLET
38.24 (6.81)
32.65 (0.58)
31.11 (0.23)
32.26 (1.69)
31.07 (0.61)
PRISM
46.67 (12.42)
38.08 (2.10)
39.56 (5.29)
42.67 (1.04)
34.67 (2.52)
BERTopic
668.14 (7.50)
130.06 (5.46) 102.30 (21.63)
2005.01 (51.50)
625.45 (42.38)
token under SparseLDA (typically Ract < K). Time entries are per iteration/epoca for iterative models and per pipeline for
methods; space lists dominant stored objects.
BoW models. LDA: ((nnz(X))) K) from local doc topic variational/ssuficient statistic updates (each nonzero token touches K' topic scores); O(VV)) from global topic-word updates/normalizatee /nomaalzzaiion; space ((nnz(X)+ +N (V])) for sparse data plus doo-topic and topic-word matrices. NMF: ((nnx(X) K) from multiplicative updates touching nonzeros per factor; same space pattern as BoW factorization. ProdLDA: ((nnz(X) K) (local BoW term) + +(([O)) from backprop through amortized encoder /ooder//ecodrr decoder per epoch; space adds [0). ETM: O(nzz(X) K) (local inference) + +O(V/)) for topic-embedding softmax/globa updates; BoW space. CTM: ((nnz(X) K) (local) O(Ie)) (backprop); BoW space + +|0l_ MALLET: (nnz(XX) Ract) since per-token sampling operates only over nonzero counts for the (few) topics active in that context; space as BoW.
Embeddingg ccuuttriig... BERTopic: O(N Cene) for encoder forward passes; O N) from ANN/KNN graph (UMAP build); O(N2) worst-case from HDBSCAN; O(IV(K) from c-TF-IDF labeling; space O(N demb) for embeddings + +O(V)) for cluster-term weights. Contextual-Top2Wer
same O(N Cene) + ONN log N) + O(N2) UUCCod++UUMPPDDDDSCAA no c-TF-IDF term; space O(N demb). FASTopic: (nnz(X)) to build word co-oceurrences; O(V]K)) for anchor selection; O(V K) for recovery; space O(nz () + [V]K).
PRISM (ours). Inference matches MALLET: O(nnz(X) Kaet) time; ((mzz( (X) + NK + (V(K) space. The one-time word-embedding pre-step (dense case) is dominated by the diffusion-maps spectral decomposition on a |V|x|V [V] kernel: as O((V)) time and ((V(2) space (the PPMT transform and all-pairs similarities are subsumed).
Reading Table 11. As shown in Table 11, PRISM exhibits runtimes close to MALLET across all datasets (34.67-46.67s vs. 31.07-38.24s). indicating that the porpus-intrinsic prior construction and initialization in- troduce only modest additional overhead. In contrast, BERTopic is substantially slower 002.3002005.01. consistent with the cost of contextual embedding extraction and subsequent dimensionality reduction and clustering, whose runtime scales strongly with the number of documents (e.g., DBLP) Overall, these re- sults suggest that PRISM largely preserves the computational profile of classical LDA-style training while delivering the empirical improvements reported elsewhere in the paper.
E
Biological Experiments
E.1 Biological Technical Details
Bio Preprocessing. We first filtered lowly detected genes, retaining only those expressed in least cells using acanpy-pp.oe filler_geneese (Wolf et al., 2018). Counts were normalized per cell to fixed library size (scanpy pp. www.noooeeeettaaaaeea totall and log-transformed (scanpy -p. .p. logll) (Wolf el. al., 2018; Luecken & Theis, 2019). Highly variable genes (HVGs) were then selected with the Seurat. v3 hot (flavor=seurat_vir v3),, using raw counts via layer=" and, where applicable, merging HVGs across batches via batch_ket (scverse, accessed 2025; Stuart et al., 2019). Analyses were performed in Scanpy with AnnData as the primary container (Wolf et al., 2018; Virshup al., 2024).
PRISM adaptation for scRNA-ses Our workflow matches the text-corpus setup except for the PPMI construction. In text, the sliding window is entire document; in scRNA-seg, we first construct cell-cell
23

---

Published in Transactions on Machine Learning Research (03/2026)
k-NN (here & 30) on PCA features (Euclidean distance), standard step in single-cell pipelines, and treat each cell's neighborhood as the PPMT 'window!. We then compute gene PPMT from co-occurrences across these neighborhoods. The intuition is that genes co-expressed across similar cells (often sharing cell type/state) better capture shared programs than mere co-occurrence within a single cell, where multiple unrelated programs may be intermixed.
E.2
Biological Evaluations
E.2.1
Gene-Set Quality Metrics
We assess the biological plausibility of inferred gene expression programs using three complementary criteria: (i) enrichment overrepresentatioon of curated pathways/ /functionsssss (e.g., GO/KEGG/ (0/KGG/MSiigBB) relative to fixed gene universe; (ii) Soberence-withinnggeee the extent to which programs collectively recover pathway members. Metrics are computed per program and then aggregated per model and dataset. We control the false discovery rate (FDR) the expected proportion of false positives among rejected hypotheses using the Benjamini Hochberg procedure and report BH--djjstee g-values (Benjamini & Hochberg, 1995).
co-variation of expression across cells; and (iii) coverage-
Enrichment strength (- lo 9). Let U be the gene universe (|U) N). For program $ S ([S|=K) and a curated pathway P C U (IPI = M), with overlap |S n Pl, we test overrepresentation using one-sided hypergeometric (Fisher's exact) test:
M)
(M) (1)
p
We then correct the collection of p-vauues across all tested pathways for S using FDR (Benjamini Hochberg) to obtain q, and report the strength as (9) (higher is better).
Coherence (mean within-set Spearman correlation). Given the cells ggenes expression matrix, com- pute Spcarman correlations Pıj across cells for all gene pairs j. i,j E S, and average:
coherence
Σ
Pig.
K(K - 1) ich, IJES
Higher values indicate coordinated expression consistent with a functional program.
Coverage (pathway completencss). Per-pathway coverage the recovered fraction of members:
cov(P) ()
P1
Aggregate over the set Pa of pathways significantly enriched for S (post-FDR), e.g., by simple mean:
coverage
Σ
cov(P|
P
PEP
or wiighte mean (specified in the Supplement). Higher better.
E.2.2 LLM Confidence Score
To assess the biological relevance of gene sets derived from topic models, we follow the LLM-based evaluation protocol introduced by (Hu et al., 2025).22 22 The core idea is to query a large language model (GPT-4) with each gene set and evaluate whether it can (1) identify a coherent biological process (BP) associated with the gene sot, and (2) express high confidence in that association.
22 httipp:://www.eees nebi. www.mih..m......mmmmm...... htth.pow//mmaarrrcccee///1117/
24

---

Published in Transactions on Machine Learning Research (03/2026)
Prompt Design. Each gene set is presented in natural language prompt, instructing the model to infer shared biological process based on the listed genes. Prompts are carefully crafted to be neutral and avoid leading the model toward specific functions. The model is then asked to (i) name the most likely BP, and (ii) rate its confidence on a scale from 0 to 1.
Scoring. The model's textual output manually inspected to verify whether the inferred BP matches a plausible biological function supported by external evidence (c.g., GO annotations). Confidence scores are recorded for each gene set and aggregated assess overall coherence across topics.
Interpretation. As shown in prior work, gene sets yielding low confidence often correspond to functionally inconsistent noisy groups, whereas high-confidence predictions align with known biological pathways. Thus, the LLM confidence score serves as proxy for functional coherence and interpretability of the gene sets.
E.3 Biological Quality Results
Table 12: LLM-based evaluation scores (mean and std over 10 runs) across biological datasets.
Models
Breast
pbmm3k
zeisel
brain
cNMF
.8000 (.0505)
.9021 (.0401)
.9830 (.0172)
scHPF
.7712 (.0240)
.8326 (.0235)
.8880 (.0658)
NMF
.7880 (.0331)
.6500 (.0141)
.6670 (.0435)
ProdLDA
.7600 (.0469)
.4500 (.1862)
5950 (.0282)
MALLET
.7920
(.0352)
.3922 (.0276)
8920 (.0268)
PRISM
88300 (.0171)
8997' (.0167)
.9350' (.0206)
As shown in Table 12, PRISM attains the highest GPT-4 confidence on two of three datasets (BreastCancer, zeisel brain) and is a close second on pbmc3k, consistently outperforming MALLET. Together with the gene-set metrics (see Table 3), these results may indicate that corpus-intrinstt LDA variants are well-suited scRNA-soq: PRISM and MALLET clearly surpass generic text models (NMF, ProdLDA), while PRISM is competitive with single-cell baselines (cNMF, scHIPF) and reliably improves over MALLET. Since GPT- 4 scores track biological plausibility (Hu et al., 2025), PRISM's gains suggest more coherent, biologically meaningful topics.
25


Reviews:
[{"review_id": "uuBCYotTBc", "reviewer": "Reviewer_AEEe", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper provides a way to initialize the word-topic embeddings by learning them from the corpus. The embedding method casts the words as nodes in a graph and extracts embeddings from the diffusion map of the graph. The motivation of developing corpus-intrinsic embeddings, and the method is interesting. The method is also presented in an intuitive manner. However, the writing itself requires work in terms of substantiating several claims present throughout the paper.", "claims_and_evidence_yn": "No", "claims_explanation": "MANY of the sentences are implicit claims about the literature, but unsubstantiated. E.g., “foundation models are underdeveloped and existing knowledge is often fragmented” , “..pre-trained models…fall short in data-scare settings” etc.\n\nThis is is a major issue, and forces the reader to take these claims at face value, which is not ideal.", "audience_yn": "Yes", "audience_explanation": "The idea is an interesting application of spectral methods to learning useful embeddings for LDA initialization.", "requested_changes": "- The paper (especially the introduction) needs a fine-grained analysis of the implicit claims and whether they are supported by the literature. Otherwise, it is hard to convince the reader of the motivation of the paper.\n- I could not tell what the value of “t” is in the experiments. More broadly, I think the paper will benefit from ablations/sensitivity analysis and discussion of the hyperparameters it introduces to the LDA pipeline (I appreciate the practical guidance for m).\n- While I appreciate the complexity analysis, I think that as a practitioner, I still cannot develop a mental model around the sensitivity of the method. For example, how many docs are needed to build reliable embeddings? What is the interpretation of the embeddings themselves? - Related to the previous point, while the main contribution of the paper is around the word-embedding method, the paper is missing discussion around how good these embeddings are on their own. Technically, LDA is just a downstream application to this method, but the method is general. Is there a reason the paper couples them?\n- For Section 6, why was K set to 5? And why were other baselines from Section 5 not used? Again, a justification/discussion would help the section.\n- For my understanding: Why did you choose PPMI as the metric to define edge weights? What are other choices? Why not them?", "broader_impact_concerns": "I don't have any concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "nuX4hhE25t", "reviewer": "Reviewer_bU34", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes PRISM, a corpus-intrinsic way to improve LDA without using any external knowledge. It initializes a topic–word Dirichlet prior by computing a second-order word similarity graph from PPMI along with diffusion map embeddings. Therefore, this study does not changes LDA’s generative process but improves topic representations. Empirically, across five text datasets and three single-cell RNA sequence datasets, PRISM consistently improves coherence and interpretability over baseline methods.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The proposed method is grounded in established technqiues such as PPMI and diffusion maps with a reasonable theoretical support.  The claims are supported by consistent improvements across multiple text and a scRNA-seq datasets, evaluated with quantitative metrics such as topic coherence and an LLM-assisted word-intrusion test.", "audience_yn": "Yes", "audience_explanation": "The paper offers a corpus-intrinsic way to improve classical LDA via an informative Dirichlet prior, and demonstrates consistent gains on both text corpora and scRNA-seq. Therefore, the TMLR readers in probabilistic modeling and representation learning may be interested in this work.", "requested_changes": "1. Justify “unbiased pattern discovery” in Introduction by adding citations, evidence or explanations. It is not clear why corpus-intrinsic methods are better than methods using external knowledge.\n\n2. Provide more contexts about \"Second-order similarity\" in Section 3.2. \n\n3. Section 2 Related Work only lists the relevant works without discussing limitations. \n\n4. Section 2.1 discusses topic model with expert knowledges but it does not cover recent works (e.g., within 5 years). It may include some works like [1,2].\n\n[1] Automatic phenotyping by a seed-guided topic model. KDD, 2022.\n\n[2] Modeling electronic health record data using an end-to-end knowledge-graph-informed topic model. Scientific Report, 2022.\n\n5. Update related work in Section 2.2 (“Complementary work has examined …”). The discussion cites older references (latest around 2009). Please add a few more recent works (e.g., within 5 years). \n\n6. Remove section 4.1 or merge it with the above section. It has been discussed in Introduction. \n\n7. In Sec. 4.2.1, please clarify that W is a weighted similarity graph (i.e., a weighted adjacency matrix).\n\n8. Qualitative analysis could add a heatmap to show some topics and their top words with probabilities. Moreover, the visualizations lack expert-knowledge–guided baseline such as SeededLDA.", "broader_impact_concerns": "NA", "additional_comments": "NA", "extra_scores": {}, "extra_text": {}}, {"review_id": "Bx3I6eJmEO", "reviewer": "Reviewer_xMSF", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper introduces PRISM, a novel corpus-intrinsic initialization method for Latent Dirichlet Allocation. The core contribution lies in deriving a data-driven Dirichlet prior parameter from word co-occurrence statistics within the corpus itself, effectively initializing LDA without relying on external embeddings or pre-trained models. This is particularly valuable in resource-constrained settings and emerging domains where external knowledge may be limited or unreliable. The authors demonstrate improved topic coherence and interpretability across diverse datasets, rivaling the performance of embedding-based methods. The method appears robust and offers a practical alternative for enhancing topic modeling in scenarios where external knowledge is scarce.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The provided experiments consistently report improvements in topic coherence and interpretability when using PRISM compared to other LDA baselines. The performance is also shown to be competitive with embedding-based methods. The authors explicitly state that the improvements are demonstrated across five diverse datasets, suggesting generalizability.  The comparison to models like MALLET, NMF, ProdLDA, cNMF, and scHPF provides a reasonable benchmark. The claim of robustness is supported by consistent performance across different datasets and domains.\n\nHowever, the reliance on GPT-4–generated scores to assess biological plausibility in the scRNA-seq experiments is problematic. For instance, the cited paper itself notes that while the generated analysis text was largely factual, GPT-4 occasionally produced unverifiable statements, underscoring that even state-of-the-art LLMs require explicit fact-checking and reference validation, either automated or manual. These scores lack transparency, reproducibility, and a clear biological grounding, making it difficult to evaluate what aspects of the data they actually validate. Without a principled justification or comparison to established domain-specific metrics or expert annotations, the use of a general-purpose language model risks introducing opaque biases rather than providing meaningful biological credibility.", "audience_yn": "Yes", "audience_explanation": "The TMLR audience includes researchers and practitioners working in areas such as NLP, ML, Bayesian methods, and bioinformatics.  Topic modeling is a fundamental technique in these fields, and the problem of improving topic coherence and interpretability remains an active area of research. PRISM offers a novel solution that addresses the limitations of existing methods, particularly in resource-constrained settings. The application to single-cell RNA sequencing data is also of significant interest to the bioinformatics community.  The corpus-intrinsic approach and demonstrated performance make this paper relevant and valuable to a broad segment of the TMLR readership.", "requested_changes": "* **[major]** Provide a more thorough ablation study evaluating PRISM’s sensitivity to key hyperparameters in the core methodology, such as those governing the PPMI computation. This is necessary to assess robustness and to provide actionable guidance for practical use.\n\n* **[major]** The contribution appears largely incremental. The proposed method heavily builds upon existing topic modeling and initialization techniques, reusing well-established components such as PPMI-based co-occurrence statistics, diffusion-based embeddings, and standard clustering pipelines. While the integration is technically sound, the paper offers limited conceptual novelty beyond combining known techniques, and the empirical gains over prior work are relatively modest. This raises questions about whether the method represents a substantive advance over existing approaches or primarily a refinement of prior art.\n\n* **[major]** While GPT-4–based scores are used as a proxy for biological plausibility in the scRNA-seq experiments, the evaluation would be strengthened by incorporating additional quantitative metrics. Notably, the cited work itself acknowledges that although GPT-4–generated analyses are largely factual, the model occasionally produces unverifiable statements, highlighting the necessity of explicit fact-checking and reference validation, whether automated or manual.\n\n* **[minor]** Provide a clearer and more detailed description of the diffusion maps algorithm used to construct embeddings, including sufficient detail to ensure reproducibility.\n\n* **[minor]** Seems like some parts of the Figure 1 are hand-drawn. It would be good to have them in a vector format with proper fonts.", "broader_impact_concerns": "The ethical implications of this work are relatively minor. Topic modeling itself can be used for various purposes, some of which may have ethical concerns (e.g., biased topic discovery). However, PRISM does not introduce any new ethical risks beyond those associated with standard topic modeling techniques.", "additional_comments": "Overall, the paper is clearly written and tackles a relevant problem in topic modeling. The corpus-intrinsic perspective is a reasonable design choice, and the empirical evaluation spans a diverse set of datasets. However, several aspects limit the current impact of the work. In particular, the methodological contribution appears largely incremental relative to prior literature, key design choices lack sufficient ablation and justification, and the evaluation relies on proxies that raise concerns about interpretability and robustness. Addressing these issues, along with the other requested revisions, would substantially improve the technical rigor and clarify the paper’s true contribution to the field.", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)