Explicit B ayesian Inference to Uncover the Latent Themes of Large Language Models (2025)

Abstract: Large language models (LLMs) have demonstrated impressive generative capabilities, yet their inner mechanisms remain largely opaque. In this work, we introduce a novel approach to interpret LLMs generation process through the lens of an explicit Bayesian framework by inferring latent topic variables via variational inference. Specifically, we leverage a variational autoencoder-based neural topic model to dynamically approximate the posterior distribution over the high-level latent topic variables at each generation step. By reconstructing the LLM’s next-token predictions through these latent topics and maintaining a regularized latent space, our method yields interpretable and diverse topic representations but also has the ability to effectively captures semantic shifts throughout the text. We validate our approach on multiple datasets, showing that our latent topics outperform state-of-the-art topic models on intrinsic measures of coherence and diversity. Furthermore, we demonstrate the utility of our approach in downstream applications by using the inferred topic distributions to retrieve relevant demonstration examples for in-context learning, resulting in significant gains on classification and summarization tasks.

## introduction large language models (llms) have achieved impressive performance across a variety of benchmarks @xcite @xcite . this can primarily be attributed to the large-scale pre-training on massive text corpora, where they learn to generalize by outputting the appropriate continuation given the task prompts. however, at such large scales, understanding their inner workings becomes very challenging due to the opaque nature of deep neural networks. one promising direction to understand their behavior is through the bayesian framework @xcite @xcite , where the model learns to first infer a posterior distribution over some latent variables conditioned on the previous context before predicting the next token. when these latent variables are disentangled @xcite @xcite , each component can reflect a distinct interpretable feature (e.g., a topic) in the generation process, where the model then uses the inferred distribution over these features to guide its predictions. thus, the activations of these latent variables may provide a transparent overview of the model's internal structures during the generation process. in this work, we focus on inferring latent topic variables, which are high-level thematic dimensions of the model represented as clusters of related words and concepts. however, since llms do not explicitly learn a latent topic distribution @xcite , supplementary techniques are required to infer the posterior distribution over latent topics from the model predictions. for example, the recent work by @xcite used prompt tuning to learn a set of token embeddings as the optimal latent variable values under the empirical bayesian framework. however, their fixed topic embeddings are difficult to interpret and cannot be used to reveal how the model's topical shifts at each step. instead, we propose a novel framework that uses variational inference to approximate the posterior distribution over the latent topics at each generation step. specifically, we train a neural topic model (ntm) @xcite based on the variational autoencoder (vae) architecture @xcite to dynamically infer a distribution over topic variables at each generation step, where topics are represented as distributions over the vocabulary. at each step of the generation, the vae constructs the llm's predicted token probability based on the sampled topic distribution conditioned on the previous context. we empirically validate the effectiveness of our proposed method by performing an intrinsic and extrinsic evaluation of the latent topics. for intrinsic evaluation, we perform experiments to assess the interpretability of the inferred latent topics of an llm by training our model on a collection of documents from the ag news @xcite , db-pedia, and govreports dataset @xcite . comparison against state-of-the-art topic modeling techniques reveals that our method can consistently learn high-quality topics which can be used directly by humans for analyzing the given text corpus (section 4). for extrinsic evaluation, we conduct experiments to assess the effectiveness of the inferred topics on two popular downstream taskstext classification and summarization-using incontext learning. in these tasks, the topic distribution serves as a text embedding to identify demonstrations with similar topic distributions (see section 5). in summary, our contributions are three fold: ## related work in subsection 2.1, we describe the bayesian frameworks for interpreting llms. while in subsection 2.2 and subsection 2.3, we discuss recent works on topic modeling and in-context learning, respectively. ## intrinsic evaluation on topic modeling since our method infers the underlying topics during llm generations, these latent topics can capture the underlying semantic patterns providing some transparency regarding the themes leveraged by the model at each generation step. when trained on a collection of documents, these topics can be interpreted as llm-enriched thematic structures of the corpus analogous to those found in traditional topic models. in this section, we assess the intrinsic quality of the inferred topics by comparing our method with existing topic modeling techniques. ## extrinsic evaluation on in-context learning while the experiments in section 4 provided an intrinsic evaluation of the topic quality, we now perform an extrinsic evaluation by assessing the usefulness of the topics for in-context learning (icl). in particular, during inference on the test set, we use the topic distribution of the input documents to dynamically retrieve the examples with the most similar topic distribution in the training set. ## conclusion in this work, we propose a novel approach for inferring the latent topics of large language models under explicit bayesian framwork. by training a variational autoencoder (vae) to reconstruct the llm's predicted next-token distribution, we obtain interpretable topic variables that capture high-level thematic patterns at each generation step. our experiments demonstrate that this topic-based representation provides both intrinsic benefits-yielding more coherent and diverse topics than standard topic models-and extrinsic advantages by improv-ing retrieval-based in-context learning (icl), particularly in classification tasks where rare-token noise can be detrimental. for abstractive summarization, the coarse topic distribution performs comparably to the next-token distribution approach but at a fraction of the computational cost. these results suggest that modeling generation with a learned topic distribution can serve as a powerful lens for understanding llm behavior, bridging probabilistic generative models and largescale neural networks. in future work, we plan to extend our approach by designing more capable probabilistic models that capture a large proportion of the llm output space, and investigate ways to additional methods to utilize the latent topics in downstream applications (e.g., hallucination detection, uncertainty quantification, etc.). we hope this work inspires further research on incorporating explicitly modeled latent structures into large language models for enhanced transparency, and task performance.