Explainable Chain-of-Thought Reasoning: An Empirical Analysis on State-Aware Reasoning Dynamics (2025)

Abstract: Recent advances in chain-of-thought (CoT) prompting have demonstrated the ability of large language models (LLMs) to perform multi-step reasoning. While prior work focuses on improving CoT generation quality or attributing token-level importance, we propose a novel framework to structurally analyze the latent dynamics of CoT trajectories for interpretability. Our method segments generated CoT into discrete reasoning steps, abstracts each step into a spectral embedding based on the eigenvalues of token-level Gram matrices, and clusters these embeddings into semantically meaningful latent states. We model the global evolution of reasoning as a first-order Markov chain over latent clusters, yielding interpretable transition structures. Through t-SNE visualizations and Monte Carlo rollouts, we uncover consistent trajectories across tasks and models, supporting the hypothesis that LLM reasoning follows globally coherent yet abstract paths.

## introduction chain-of-thought (cot) prompting has become a central technique for eliciting multi-step reasoning in large language models (llms) @xcite . by encouraging models to decompose problems into intermediate steps, cot improves performance on tasks such as arithmetic, logical deduction, and multi-hop question answering. understanding cot outputs is, therefore, critical for both evaluation and user comprehension. recent large-scale studies have shown that even modest increases in textual complexity can hinder human comprehension and increase cognitive load, highlighting the need for more interpretable llm outputs @xcite @xcite @xcite . understanding cot is even more challenging in * equal contribution. to address this, we propose a state-aware transition framework for cot reasoning. we segment generated cots into discrete steps, embed each step via spectral analysis of token-level representations, and cluster them into semantically coherent latent states. transitions between these states are modeled as a first-order markov chain, yielding a structured view of reasoning dynamics. this approach goes beyond surface-level generation by uncovering latent structure in cot trajectories, offering a non-trivial solution to the challenge of interpreting abstract, multi-step reasoning without access to ground-truth annotations or explicit reasoning labels. designed as an explainability scaffold, this framework supports diverse analyses, including semantic role identification, temporal transition pattern discovery, and trajectory-level consistency evaluation. we summarize our contributions as follows: ## explainable modeling of chain-of-thought reasoning step embedding and abstraction to capture the evolving semantics of each reasoning step, we segment the cot output y (elicited from an instructiontuned llm given input x) into a sequence of steps @xmath0, . . . , c t ). for each step c t , we extract token embeddings x @xmath1. we define the spectral embedding e @xmath2: to incorporate contextual accumulation, we recursively update the accumulated gram matrix across steps: this yields a trajectory of spectral embeddings (e 1 , . . . , e t ), which we abstract using k-means with k clu clusters to assign each step to a latent state s @xmath3, . . . , k clu }. the resulting state sequence enables structural abstraction of cot and supports distribution-level explainability. semantics of reasoning state to assess whether latent clusters correspond to meaningful functional roles, we collect all reasoning steps assigned to each cluster and manually summarize their semantics. to further ground these semantics, we compute the average position of steps assigned to each cluster within the reasoning trajectory. specifically, for cluster c, the average step index is given by: ## analysis of reasoning dynamics datasets to comprehensively assess the structural dynamics of chain-of-thought (cot) reasoning, we consider datasets from three distinct categories: mathematical gsm8k @xcite , seattle has 20 sheep. charleston has 4× more than seattle, and toulouse has 2× more than charleston. how many sheep do they have in total? cafeteria had 23 apples. if they used 20 for lunch and bought 6 more, how many apples do they have? step 1 step 2 step 3 …. step 6 step 7 result state embedding states 1 2 3 transition matrix low-dim projection to illustrate the cluster to cluster relationship overview of the cluster to cluster probability flowing summarized cluster and rank them based on the semantic meaning. using only an addition, how do you add eight 8's and get the number 1000? for each question, we generate a cot response using a fixed prompt and segment it into reasoning steps via explicit textual markers. we extract tokenlevel hidden states, compute cumulative gram matrices, and obtain spectral embeddings by taking the top-64 eigenvalues. these embeddings are clustered into @xmath4. monte carlo rollouts from p yield synthetic reasoning trajectories. ## conclusion we proposed a state-aware transition framework that abstracts cot trajectories into structured latent dynamics, offering a global perspective on multistep reasoning in llms. each reasoning step is embedded via spectral analysis and clustered into semantically coherent states, with their transitions modeled as a first-order markov chain. this approach moves beyond token-level attribution by uncovering consistent latent structures across models and tasks. it enables explainability applications such as semantic role identification and temporal pattern visualization.empirical results demonstrate that llms exhibit structured reasoning patterns, pointing to underlying strategies beyond surfacelevel token sequences.