Title: Dynamics of Spontaneous Topic Changes in Next Token Prediction with Self-Attention
Venue: NeurIPS 2025
Accepted: yes
Keywords: Self-attention

spontaneous topic changes

spontaneous human thought

next token prediction
Abstract: Human cognition is punctuated by abrupt, spontaneous shifts between topics—driven by emotional, contextual, or associative cues—a phenomenon known as spontaneous thought in neuroscience. In contrast, self-attention-based models rely on structured patterns over their inputs to predict each next token, lacking spontaneity. Motivated by this distinction, we characterize *spontaneous topic changes* in self-attention architectures and reveal divergences from *spontaneous human thought*. First, we establish theoretical results under a simplified, single-layer self-attention model with suitable conditions by defining a topic as a set of Token Priority Graphs (TPGs). Specifically, we demonstrate that (1) the model maintains the priority order of tokens related to the input topic, (2) a spontaneous topic change can occur only if lower-priority tokens outnumber all higher-priority tokens of the input topic, and (3) unlike human cognition, the longer context length or the more ambiguous input topic does not increase the likelihood of spontaneous change. Second, we empirically validate that the effect of input length or topic ambiguity persists in modern, state-of-the-art LLMs, underscoring a fundamental disparity between human cognition and AI behavior in the context of spontaneous topic changes. To the best of our knowledge, no prior work has explored these questions with a focus so closely aligned to human thought.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper aims to propose a theoretical model of ‘spontaneous topic changes in autoregressive self-attention models, motivated by observation that spontaneous topic change as a fundamental cognitive feature in humans while LLMs topic change is rather initiated by contextual cue. Using a concept of topic priority graphs (TPGs), they propose a model of such topic change in a simplified 1-layer self-attention language model and provide empirical results on the frontier LLMs that capture the prediction of their model about input length and topic ambiguity.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths:\n\nI think it is valuable to make a quantitative approach based on a simplified model to capture and describe phenomena in many complex models. Also, the authors provide the experiments with the frontier models at scale which captures some aspect of the simplified model, which validates their theory.  \n\n## Weaknesses:\n\nThe biggest weakness of the paper is clarity. I found the use of notations hard to follow and it was hard to follow the path to the ultimate end prediction about input length and topic ambiguity. \nThe visual illustration of the model is unclear; for example, it is not clearly described what is the direction of the edges and what is the dotted square. I inferred that it’s a label to input and SCCs but it took a lot of effort for me. Having explicit description of the figure would improve readability. \n\nWhile authors motivate the study by claiming the difference between human cognition and LLM on spontaneous topic change, I think its basis is not very well supported.\n\nQUESTIONS:\nLine 310: isn’t it the opposite; lower similarity of the topics implies less overlap between A and B?\n\nI thing Figure 5 a is supporting the authors claim rather weakly; have you increased the context length further to see the effect better? Does your theory provide any tendency of the topic change probability change (linear, or sudden change)?\n\nHave you considered other metric of topic similarity, which might be able to capture more intricate relationship? (eg. hierarchy)\n\nLIMITATIONS:\nThe theory is limited to a simple setup but the authors already mention it. \n\nWhile authors motivate the work with the dissimilarity between human cognition and LLMs in topic change, I do not see enough evidence of it. \n\nThe definition of similarity seems to be limited - what about hierarchy within topics or high dimensional topics?", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 1, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper investigates spontaneous topic changes, i.e., abrupt, unprompted shifts in conversational topic, within self-attention-based language models and reveals their differences from spontaneous thought patterns seen in human cognition. The authors define a formal framework using token-priority graphs (TPGs) to model topic representation and transition dynamics in simplified self-attention settings. Their results indicate that self-attention-based language models rarely change topics spontaneously unless specific statistical conditions are met, such as when low-priority tokens dominate. Contrary to human behavior, longer inputs or higher topic ambiguity reduce the likelihood of topic change. Experiments on modern LLMs (e.g., GPT-4o, Llama, Claude) empirically support these theoretical findings.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**:\n1. The paper explores an interesting and novel research question: the spontaneous topic change within LLMs and how it differs from human spontaneous thought. \n2. A rigorous mathematical framework is developed using TPGs and SVM-based analysis to explore the conditions under which spontaneous topic changes can or cannot occur in self-attention models.\n3.  Experimental results on real-world LLMs (GPT-4o, Claude-3.7, LLaMA-3.3, and DeepSeek-V3 ) empirically confirm the theoretical predictions. \n\n**Weaknesses**:\n1. Despite the paper tackles an interesting phenomenon: the spontaneous topic change within LLMs, it lacks a clear justification for its practical significance, leaving readers uncertain about why understanding or modeling such behavior in LLMs is important for real-world applications or cognitive alignment. \n\n2. The condition under which a topic change occurs: when a lower-priority token appears more frequently than all higher-priority tokens, is theoretically well-defined but practically hard to verify, since the paper’s formalism relies on constructing token-priority graphs that are not directly accessible or interpretable in real-world settings. Moreover, the paper does not provide evidence or discussion on how frequently such conditions arise in realistic language model usage.\n\n3. The paper focuses on analyzing the conditions for spontaneous topic change but does not explore potential applications of its findings, such as methods to encourage or suppress such behavior in language models for specific tasks.\n\nQUESTIONS:\nPlease see the weaknesses in the \"Strengths And Weaknesses\" section.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a theoretical and empirical study of spontaneous topic changes in self-attention models, contrasting their behavior with the dynamics of spontaneous human thought. The authors develop a formal framework by defining a \"topic\" as a set of Token-Priority Graphs (TPGs) within a simplified single-layer self-attention model. This approach leads to several key findings that distinguish the mechanisms of large language models (LLMs) from human cognition: 1) Preservation of topic priorities: The model preserves the priority order of tokens related to an input's original topic, even when trained on a mix of different topics. 2) Conditions for topic change: A topic shift is shown to occur only under a specific condition: a lower-priority token must appear more frequently in the input than all higher-priority tokens associated with the current topic. 3) Impact of context and topic ambiguity: Contrary to human thought, where longer discussions or ambiguity can trigger new associations, the paper establishes that longer input contexts decrease the likelihood of a topic change in LLMs. Similarly, greater ambiguity between topics acts as a stabilizing factor rather than a trigger for spontaneous shifts\n\nSTRENGTHS AND WEAKNESSES:\nStrengths: \n- The paper is well-motivated by the goal of theoretically understanding the distinctions between LLMs and human cognition.\n\nWeaknesses: \n- Limited discussion of literature on topic shifts: While the authors mention several papers on spontaneous topic changes in LLMs in the introduction part, a more detailed discussion comparing the proposed theoretical framework against existing work is needed. \n\n- Flawed application of prior theoretical results. The paper's theoretical claims are heavily dependent on Theorem 1, which is adapted from Theorem 2 in Li et al. (2024). However, there appears to be a critical misunderstanding in applying the theorem, which invalidates the subsequent proofs. \n  - Informally speaking, the work by Li et al. establishes that the attention weight $W$ converges to a structure that can be decomposed into two components, i.e., $W \\approx C \\cdot W^{svm} + W^{fin}$ with $C \\to \\infty$. The two components have different functionality: 1) Hard retrieval ($W^{svm}$): This term selects the high-priority input tokens according to the token-priority graph. 2) Soft composition ($W^{fin}$): A finite component that governs how to assign probabilities among the high-priority tokens selected by $W^{svm}$. \n\n   - However, in this paper, the authors only discuss the impact of $W^{svm}$, ignoring the role of $W^{fin}$. This omission is critical as it leads to incorrect conclusions. E.g., Lemma 1 builds an equivalence, stating that tokens have the same prediction probability if and only if they are in the same strongly connected component (SCC) in the token-priority graph. This is not true without knowing $W^{fin}$ which can (and generally will) assign different probabilities to tokens even if they are in the same SCC.\n\n\n[1] Yingcong Li, Yixiao Huang, Muhammed E Ildiz, Ankit Singh Rawat, and Samet Oymak. Mechanics of next token prediction with self-attention. In International Conference on Artificial Intelligence and Statistics, pages 685–693. PMLR, 2024.\n\nQUESTIONS:\n- Clarity of definitions: Some symbols are used without definition. For instance, $\\mathbb{Z}$ in Assumption 3.\n\n- Justification for the topic ambiguity metric: In Section 6, the authors mention that a higher keywords similarity between two papers corresponds to lower topic ambiguity. This is counterintuitive as a higher overlap in the keywords of two papers generally implies higher topic ambiguity.\n\nLIMITATIONS:\nLimitations and potential societal impacts are discussed in detail.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}