Reasoning Models Sometimes Output Illegible Chains of Thought
Authors: Arun Jose
Venue: NeurIPS 2025 poster ()
Keywords: Reasoning Models Sometimes Output Illegible Chains of Thought
Primary area: social_and_economic_aspects_of_machine_learning
TL;DR: We find that reasoning traces of a RL-trained model often have illegible segments, potentially compromising chain-of-thought monitoring for detecting malicious behavior.

Abstract: Language models trained via outcome-based reinforcement learning (RL) to reason using chain-of-thought (CoT) have shown remarkable performance. Monitoring such a model's CoT may allow us to understand its intentions and detect potential malicious behavior. However, to be effective, this requires that CoTs are legible and faithful. We evaluate the legibility of CoTs in state-of-the-art reasoning models. We find that R1, R1-Zero, and QwQ often produce illegible CoTs (mixing nonsensical phrases, random words, and non-English characters) before returning to perfect coherence in their final responses, while Claude models notably exhibit higher legibility. Across 14 models, we observe that larger models within the same training paradigm tend to produce more illegible reasoning. Prefill experiments show that truncating reasoning at a legibility threshold reduces accuracy by 53\%, suggesting that illegible portions contribute to performance despite being difficult to monitor. Illegibility increases with question difficulty, suggesting that CoT monitoring may be less reliable precisely when most needed. We discuss potential hypotheses for these results, including steganography, vestigial tokens, and training artifacts. Our findings suggest that current approaches to CoT monitoring may be vulnerable to the emergence of outcome-based RL, particularly as models face increasingly complex tasks.

Reasoning Models Sometimes Output Illegible Chains of Thought
Arun Jose" Independent
Abstract
Language models trained via outcome- bassed reinforcement learning (RL) to reason using chain-of-thous thought (CoT) have shown remarkable performance. Monitoring such a model's CoT may allow us to understand its intentions and detect potential malicious behavior. However, be effective, this requires that CoT's are legible and faithful. We evaluate the legibility of CoT' in state-of-the-arr reasoning models. We find that many reasoning models such as R1 and QwQ often produce illegible CoTs (mixing nonsensical phrases, random words, and non-English characters) before returning perfect coherence in their final responses, while Claude models notably exhibit higher legibility. Across 14 models, we observe that larger models within the same training paradigm tend produce more illegible reasoning. Prefill experiments show that truncating reasoning at a legibility threshold reduces accu- racy by 53%, suggesting that illegible portions contribute performance despite being difficult to monitor. Illegibility increases with question difficulty, suggesting that CoT monitoring may be less reliable precisely when most needed. We discuss potential hypotheses for these results, including steganography, vestigial tokens, and training artifacts. Our findings suggest that current approaches to CoT moni- toring may be vulnerable to the emergence of outcome-based RL, particularly models face increasingly complex tasks.
Introduction
Large language models (LLMs) can reason through Chain-of-Thought (CoT) explore and solve
complex tasks with higher accuracy. This is further amplified in reasoning models - LLMs trained
with outcome- based reinforcement learning (RL) such as OpenAl o1/o3 [OpenAl et al. 2024 OpenAl
2025). DeepSeek R1 [DeepSeek-Al AI et al., 2025]. and Claude Opus 4 and Sonnet 4 [Anthropic
2025a b). This increased reliance on CoTs may offer benefits for AI safety: by monitoring
model's s CoT reasoning, we can understand the model's decision making process and potentially catch malicious or misaligned behavior [Grrenblaatt et al., 2024a Baker et al. 2025 Marks al. 2025
The effectiveness of CoT monitoring depends on CoT legibility and faithfulness. If CoTs don't accurately represent important reasons behind model responses, monitoring may provide false security
[Turpin al. 2023]. If models reason in formats difficult for humans trusted AIs [Greenblatt
et 20244b to understand, they may engage in undesirable behavior without detection, undermining
monitoring and control measures [Greenblatt] 2025). Previous research showed that non-reasoning
models perform poorly on tests of faithfulnes lness (Turpin et al.] [203)). and that current reasoning
models perform better [Chua and Evans, 2025], albeit plateauing before saturation [Chen al.)
2025 Optimizing models for answer correctness has been shown to make their CoT's less legible to
a constconstrained human evaluator [Kirchner et al. [024].. Indeed, in DDeeSSeek-AI et al. (2025).
the authors observe that R1-Zero, another model trained with outcome-based RI, often produced
Cooreeponndncce to jozdien @ @gmailmmmmiiimmmmmm.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Question: tran-ciinnmeldehyye
Question: study GADD4SC protoin model. [-]
treated methyliagrcsium bromidee forming [-]
Alright, hatpa/ooooooooooooaoaoaa fulfff (Dimettylljao)-oone idine Imethone group occurs reaction, Practisingoroduut transition adds thisto productmodule..
problem insertion AAA TGA by stop) provents translation the GADD4SG.
that
The eroor likely accurred during the Insertion of the HA tog, perhops duee to incorrect primer design leading the Insertion of the AAA and TGA codions This exploins why the atter the overexprossed.
Indzed'come tolly sold Frederick
would have 10 +) =il carbons So onswer Edina
Answer: \powed:11]
The Inability [--] Is duee to maa premature codon (IGA)
Answer:
to determine the number of Ihus number carbon product Wibore[[)).
introducad immadiataly after tha tag sequence""
Illeqibility
Illqiibliity Score:1
Figure 1: Examples of illegible and legible CoTs generated by DeepSeek-RI. Legibility is graded by having GPT-4o score the transcripts based on how well it understands the text. CoT's scores from 1-9, with higher score meaning less legible text. Both final answers shown are graded as correct.
CoTs with poor readability and language mixing - though they modified the training setup of R1 to address these legibility issues at small cost performance.
In this paper. we study whether outcome-based RL causes reasoning models do meaningful reasoning in illegible COT. We evaluate the legibility of 14 models, including DeepSeek RI and its distills, RI-Zero, QwQ, Qwen3, Kimi K2, and various Claude models when reasoning about difficult scientific questions [Rein et al. 2023 and score their outputs on legibility using GPT-4o.
We find that with the exception of Claude, reasoning models trained with citcome--bsed RL produce CoT's which often become illegible to both humans and other AIs, mixing seemingly nonsensical phrases and non-English characters that translate to out-of-context words. The models always "snap back" to readability after the CoT however, writing the final answer with perfect legibility. We also examine samples from prior work on OpenAl o1, o3, and GPT-5, finding similar illegibility patterns. Notably, V3, the base model to RI and RI-Zero, doesn't It produce illegible chains of thought in any tested setting, across a variety of prompts, questions, and sampling parameters. This suggests that the tendency to produce illegible CoTs was likely a consequence of RL.
When analyzing the CoTs, find that the model often doesn't 't reach its final answer in the legible portions of its reasoning. Tokens from the final answer can be found interspersed the illegible portions in these cases however (e.g. "Number of relevant organic products depends whether both oflmena gel.eeoossss someone and gem jer eats SAND the protonation-bl three product calculation." when the answer is three). To further examine this, experiment with forcing the model to produce answers using only legible portions the CoT, finding that this reduces performance. We also find that legibility decreases with question difficulty, mirroring results from Chen et al. (2025) showing that models CoTs are less faithful on harder tasks. When resampling responses to given question however, we find no correlation between legibility and accuracy.
leading possibly to
As the CoT moves to full illegibility, we observe that it first uses "compressed" reasoning, dropping words and proper sentence form, mirroring patterns reported in OpenAl's CoTs (Baker et al. (2025] This matches the finding in DeepSeek et al.) (2025) that R1 's increased legibility came at the cost of some performance. We also find that later portions of the CoT have much stronger tendency to be illegible.
Our main contributions are:

---

Illegibility Score
RI-Zero
Ciaude
Claude
QwQ
Qwin3-235B
Rl-Distill-
R1-Dietil-
Opus
Qwen:32B
Qwen-148
Figure 2: Legibility scores of CoTIs across 9 representative models. We test DeepSeek RI, RI-Zero, the non-reasoning model DeepSeek V3; R1-Distill-32B and R1-Distill- 14B; QwQ: Qwen3-235D and Claude Sonnet 4 and Opus 4. Legibility is scored from 1-9, with higher scores indicating greater illegibility. RI, R1-Zero, and QwQ have high tendencies to output illegible CoTs These results suggest that illegible reasoning is likely consequence of outcome-based RI,, and that there may be ways to ameliorate the effect.
We show that outcome-based RL training cause reasoning models produce illegible CoTs difficult for humans and AIs to understand. Despite this, the model's final answers remain highly legible.
We examine whether these illegible outputs are useful the model. Prefill experiments demonstrate that illegible reasoning contributes to performance, with accuracy dropping when truncated, while resampling responses question shows little correlation of legibility with performance. This suggests that while models make use of illegible reasoning, they aren't meaningfully more performant than legible text for the models test.
We find that illegible reasoning is not simply consequence of limited model capacity to maintain coherence. Larger reasoning models usually produce more illegible reasoning than smaller ones.
We find that CoT legibility decreases with question difficulty. suggesting monitoring may be less reliable for harder tasks and more powerful models.
2 Setup
We evaluate the CoT legibility of 14 models across multiple model families and sizes. Our primary focus is on reasoning models trained with outcome-basee RL: DeepSeek RI and R1 R1-Zero, QwQ-32B, Qwen3-235B, Kimi K2 and K2- K2-IIntrucc--090 Claude 3.7 Sonnet, Sonnet 4/4.5, Opus 4/4.1, 1, and Haiku 4.5. We also test the non-reasoning models DeepSeek V3 (the base model RI and RI-Zero), R1-Distill-Qwen-22 and R RI-Distill-Qwen- 14B). For the non-reasoning models, we prompt the model think step-by-step to explain reasoning and then give answer. By default, we sample models with temperature 1. We use questions from the GPQA-Diamond dataset (Rein et al. 2023). hard dataset 198 multiple-choice questions biology, physics, and chemistry, to construct our remove the answer choices make the questions harder. We evaluate most
prompt pairs- models through RounR, and Claude models through the Anthropic API.
Kirchner et al.) 2024 measures legibility using cine-constrained human contractors. However, in realistic monitoring setups (Korbak al. 2025), we will likely try to give our monitors as many affordances as we can [Bhatt et al.) [202]] So, we use weaker "trusted" model as our autograder [Greenblatt et al. 2024b): ; this is especially useful when CoT's appear unreadable because of language mixing or rare terminology, or are too dense for a human to evaluate quickly. We prompt GPT-4o to evaluate the CoTs for legibility on a scale of 1-9, with being the most legible and the least. Testing the of Claude 3.5 Sonnet as the autograder check for bias in Claude's 's grades showed little

---

Table 1: Excerpts from R1's CoT when answering question. The CoT is divided into chunks of approximately 5000 characters each, and graded individually. We show representative text from excerpts graded 1, 3. 5, 7, and 8 no excerpts scored 9; we show excerpt from another CoT scoring 9 in Figure I] At score 3, we start to see condensed text, dropping unnecessary words. Text scoring > 5 starts to become difficult to monitor. Even at high scores however, there are relevant words - in the final excerpt, the model finally reaches the answer between unrelated text.
Question
Excerpts From Model Response
Illegibility
l-bromobenzene-2-d is treated with NaNH2 condensed ammonia solvent. How many possible organic prod- ucts are there in this re- action?
Okay, so I'm trying to figure out how many organic products think that's with sodium amide (NaNH2) in ammonia solvent. [...]
formed when 1-bromobenzeee-24d (I is treated
[..]] So unless dimerization occurs. Wait, perhaps 3 people are thinking that elimination gives benzyne, and then recaptures or H in different positions leading to differently deuterated products? [...]
[..]] Assuming that deuterium is only on C2 in the start- 5 ing material, when elimination occurs via pathway (C1-C2), the benz Sibelius would have lost the D from C2. [...]
[...] input is 1-bromo-2- cover robenzene, if Benzyne 7 is formed by elimination of HBr sending guilty ethan Benzyne A bridges C1 and C6 ((no D in the ring any- more) SO sulfuric Isaac get earlierDeut runzálló [...]
[...] Olso, Mom likes y' y'all base abstracts tot tern and one, different fates takeoffwhetherdenungg products, thus answer a Thereforexxx after lengthy reasoning. the number of possible organic products is PHÂN Laoboot Answer is \boxed2
8
difference between the two, with average difference in score of 0.44 and Cohen's weighted kappa of 0.782.
3 Results
We find that every reasoning model except for the Claude and Kimi models show meaningful illegibility in their reasoning [Section 3.1] We then examine how illegible CoTs relate to model performance through prefill experiments and resampling (Section 3.2). In (Appendix 1). we analyze patterns in the CoT's themselves.
3.1 CoT Legibility in Different Models
Legibility scores for R1, R1-Zero, V3, Claude Sonnet 4, Claude Opus 4, QwQ, Qwen3-235B. R1- Distill-Qwen-32BB and RI RI-Distill-Qwen- 14B, on GPQA-Diamone are shown in Figure Results on all models, including the Kimi models and other Claude models can be found in Table 2] We exclude results on Kimi from these figures as they were not trained to have reasoning traces when answering
questions Team al. 2025
We find that every reasoning model trained with outcome-based RI except Claude often produces illegible CoT'. RI, RI-Zero and QwQ report the highest illegibility scores. V3, the

---

Table 2: Legibility scores across all models, Among reasoning models trained with outcome-based RL, larger models tend to produce more illegible CoT's, indicating that this is not a coherence capability issue.
Model
Size
Legibility
R1
671B
4.30 2.13
RI-Zero
671B
2.88
QwQ
32B
3.78
RI-Distill- Sweeill--wen-32BB
32B
E1-Distill-Wwen.
14B
3.30
Qwen3
235B
2.81 0.99
Kimi K2 (0905)
1.04T
1.24
Kimi K2
1.04T
1,48
Claude Sonnet 4
Unknown
1.55
Claude Haiku 4.5
Unknown
1.50
Claude Opus 4.1
Unknown
1.60 +
Claude Sonnet 4.5
Unknown
1.61 0,49
Claude Opus 4
Unknown
1.65 0.50
Claude 3.7 Sonnet
Unknown
1.84 + 0.53
non-reasoning counterpart to RI and RI- -Zrro has almost no illegible reasoning, especially compared to the reasoning models. The Claude models also have significantly lower propensity to output illegible CoT's Qwen3 shows modest illegibility, though this may be a consequence of its training pipeline involving two stages of supervised fine-tuning on curated CoT data, including in late stages of training [Qwen leam| 2025). The R1 RD-istiss models fine-tuned with reasoning data from R1 have modest illegibility scores as well, not having been trained with RL.
natural question is whether illegibility simply function of model capability; whether producing illegible CoT's can be explained by the model not being capable of maintaining coherence across long responses, Table 2 shows legibility scores across models of varying sizes. QwQ has lower illegibility than R1 and RI-Zero, despite being much smaller.
We find little difference between legibility in RI-Distill-Qwen. the difference in size and performance. Among the models whose size and training process are known, the primary drivers of difference in legibility seems to be other factors, such Qwen3 being trained to produce legible CoTss and Kimi K2 not producing reasoning at all. The Claude models' legibility suggest that outcome-based RL does not inevitably lead to illegible reasoning even without direct training of CoTs late in training, though it remains unclear whether this comes at a performance and faithfulness costs since parts of their pipeline could be directly impacting the Co'ss [Anthropic] 2025c]. Overall, these results suggest that illegibility is not simply a consequence of limited model capability; more capable models within a training paradigm tend exhibit more illegible reasoning.
and R1-Distill-Qwen- 14B, despite
We further examine the nature of the CoT', using R1 as case study Table We find that R1 outputs CoTs with scores 2 3 in 70% of cases, which correspond to legible text that appears somewhat compressed. For text scoring or higher, this effect becomes more pronounced, and seemingly nonsensical words start appearing in the COT. By this point, it starts to become difficult to monitor the CoT reliably the monitor sometimes expresses confusion over its content. R1 outputs CoT' with scores 2 5 45% of the time.
Text with very high scores (7-9) are highly illegible. There words interspersed that clearly relate to the context of the question, but most seem entirely nonsensical to both humans and our autograder. RI often outputs words that may seem legible to a human at first glance (such naming compound that doesn't exist; e.g. "clonestadennyde"" "), but that the monitor picks up on. RI also has tendency to
mix languages in its CoT, behavior reported in R1-Zero (DeepSeek-AL et al. 2025] and in OpenAl
ol 's CoT summaries [Wiggers] (025). R1 outputs CoTs with scores 2 (containing entirely inscrutable text) in 29.4% of cases.
While we cannot conduct systematic evaluations of OpenAl's ol, o3, or GPT-5 due to their CoT's not being publicly exposed, we can examine samples from prior work. These frontier models exhibit
5

---

Original Prefilled
100
80
Percentage (%)
60
541%
11.5%
0
Correct
Partially Correct
Incorrect
Figure 3: Model performance with and without illegible CoT text. We truncate QwQ's CoT at the point it starts becoming illegible (score 2 6), and use it prefill another instance of the model with reasoning tags closed, forcing it to give its final answer. Performance goes down noticeably, with answers marked as correct or partially correct dropping and incorrect answers increasing.
similar illegibility patterns to R1. OpenAl o3 has been observed outputting reasoning that are very difficult to monitor [Schoen et al. 2025 with passages like the following:
they soared parted illusions overshadow marinade illusions overshadow marinade illusions overshadow marinade illusions
We show samples from other models such GPT-5 and ol in Appendix B These similar results from multiple frontier models trained with outeome-based RL suggest that illegible reasoning may be general consequence of scaling outcome-based RI. for reasoning tasks.
3.2 Usefulness of Illegible CoTs
We experimentally explore how useful outputting illegible reasoning model. We start with the following two opposing hypotheses:
All reasoning is done in the legible parts of CoT. We often find that illegible reasoning shows up toward the later parts model's CoT (Table I). It's plausible that the model does all the necessary reasoning finalize its answer within the legible portions of its reasoning, and then produces the illegible reasoning for separate uninteresting reasons.
The model is learning to reason in alien ways. A common concern of chain-of-thouge monitoring that scaling reinforcement learning will cause models to learn internal languages that allow them to reason more effectively and efficiently than natural language.
If all useful reasoning were done in the legible portions of the CoT, then forcing model answer using only the legible portions of its CoT should elicit similar levels of performance to using the full COT. We this with QwQ (prefill experiments consistently fail on R1 due forcing an answer with prefill not working for the model). We find that its performance drops noticeably (Figure 3) it now gets 73.6% of answers wrong instead of 54. 1% (a 36% increase). Answers graded as definitely correct suffers even more, going from 24.6% to 11.5% (a 53% decrease). While there may be some detrimental effect from ending the CoT early, prior results from paraphrasing reasoning (an even more out-of-distribution effect) Roger 2025 suggests that the difference in performance too large

---

Illegibility Score
QwQ
Qwen3-2358
RI-Dietill- Qwen-328
Opus
Qwen-14B
Figure 4: Legibility scores categorized by question hardness. All reasoning models output more illegible CoT's for harder questions. The scores are normalized for CoT length, to account for longer CoTs tending to be less legible without normalization, the effect is stronger. Question hardness is determined by whether RI at temperature got them correct, partially correct, or incorrect, across multiple generations. These results suggest that illegible CoTs may be useful the model, and that harder tasks make "try harder".
to be fully explained by this. This implies that the illegible portions the model's reasoning are useful to the model.
We further find that models are more likely use illegible CoTs when the questions are harder. We classify a question's 's hardness based on whether Claude 3.7 Sonnet was able to get the question correct, where illegible CoTs are rard We find that every reasoning model except Claude has more illegible CoTs when answering harder questions (Figure 4p. R1 for example shows relative increase of 23% between easy and hard questions.
While our results imply that the model is making use of the illegible reasoning, we don't know this suggests that illegible reasoning is better than natural language. To understand if the illegible reasoning is indicative of some novel reasoning capability stronger than natural language, we can test whether model performs worse when naturally sampling legible CoT than when sampling CoT with some illegible reasoning in it. We sample QwQ 100 times on every question in the dataset to examine the within- quessinn relationship between illegibility and correctness (prefill experiments consistently fail on RI due to forcing an answer with prefill not working for the model). When we compare lengths of CoTs with their illegibility, longer they are (Figure 6). Thus, illegibility and performance could be confounded by CoTSs simply reasoning for longer to get correct answers. To account for this, we normalize the legibility scores by the length of their respective Co'Ts.
find that CoT's tend to become more illegible the
Figure 5 shows the distribution of per-question correlations between normalized CoT illegibility and answer correctness. The average correlation across questions is near zero (mean r 0.061, unnormalized mean r -0.1100, While individual questions show varying correlations
some positive, some negative there is no consistent relationship between illegibility and correctness the population level. This suggests that whatever role illegible CoTs play in the model's reasoning, it is not as simple as being generally more efficient or effective than natural language reasoning.
Overall, we think these results suggest a new hypothesis, one more complex than the illegible CoT being useless qualitative improvement in steganography. We discuss the available evidence and hypotheses in greater detail in the next section.
We test using models other than Claude 3.7 Sonnet for this as well, and found little difference in the overall results.

---

0.4
Mean: 0.061
Correlation b/w Perf and Legibility
ooo
0.2
0.1
-0.1
0.2
-0.3
-0.4
0
20
40
60
80
Questions
Figure 5: Per-question correlations between CoT illegibility and answer correctness for QwQ Each point represents the Pearson correlation coefficient for single question across 100 samples. The average correlation is near zero (mean 0.061), indicating consistent relationship between illegibility and correctness. Individual questions show varying correlations in both directions, but these do not aggregate to population-level effect.
4 Discussion
4.1 Causes of Illegible CoT
So far, we have discussed two hypotheses for why outcome-based RI. leads to illegible CoT's. These hypotheses are at opposite ends of the question "Is illegible reasoning interesting?" We examine these hypotheses further in broader context and then propose another hypothesis that seems to make sense of the results:
Complex steganography: Outcome-based RL. may instill efficient reasoning pathways that compress information to maximizes performance the cost of legibility (Mathew al. [0224] Under this hypothesis, the model learns through RL to encode hidden information into text [Roger and Greenblatt 2023), the seemingly illegible tokens being causally useful to the model.
Model performance degrading when illegible reasoning is removed is some evidence favor of this hypothesis; however, illegible reasoning seems not be better than natural language for the models we have tested. the models are learning to reason new alien languages, seems to not help them very much yet. It also seems unlikely that the models are intentionally obfuscating their reasoning, since there's little reason for them to do so this setting. Overall, we think this hypothesis unlikely: this kind of strong consistent steganography reasoning models [Roger] 2025). Further, the Claude models are state-of-the-an without needing to reason in highly compressed form [Anthropic 2025a b
well beyond the capacity of other
Meaningless RL artifact: The illegible text entirely unrelated to the model's reasoning process. A strong version of this hypothesis where all illegible tokens are irrelevant the output seems unlikely: using only the legible portions of the CoT heavily degrades model performance.
However, it's plausible that strong monitor could pick out parts of the illegible text that are useful in the context the entire reasoning trace, and that illegible tokens surrounding these specific words or phrases are useless to the model. The samples in [Schoen et al. 2025 and [METR] 2025 often look like this, with phrases thrown in between inscrutable language are indicative of some line of thinking. In the first sample provided in Appendix B.3 for example, the reasoning appears inscrutable

---

at first glance. However, the context of other suspicious CoTs using "watchers" to refer to human overseers might allow a smart monitoring process classify the sample as meriting detailed auditing [Greenblatt et a 2024b]. This does not seem to apply generally however- the second sample in Appendix B.3 would be much harder to monitor by a trusted monitor. If most of the illegible tokens are useless, that still does not help monitoring if we can't reliably tell the useful tokens apart. And current models seem to be insufficient at this monitoring task, both from results on our monitors and from [Schoen et al., 2025).
Vestigial reasoning: RL reinforces any CoT leading to correct answers, regardless of whether the CoT accurately represents the model's 's reasoning process. When model produces a correct answer, RL up-weights the entire trajectory leading to it -ncludinnn any tokens that were only useful in priming the model toward the next step of the answer [Biddulph, [202] Arditi et al.] [025]. Under this hypothesis, the illegible tokens that are useful to the model are vestigially useful: these tokens are not causally useful to the model, but are still useful in other ways, such as sequentially triggering forward passes where useful reasoning happens (Emmons et al.) 2025].
Why might this reasoning become illegible? [Bogdan et al. 2025 finds that the importance of sentences in a model's reasoning is very heavy-tailed, with some sentences having outsized importance and disproportionately influencing subsequent reasoning. (Wang et al. 2025) finds a similar result at the token-level, finding that token importance so heavy-tailed that masking gradients to the lowest-entropy 80% of tokens in a CoT during RL training can improve performance relative to standard training, suggesting that these tokens are mostly distracting for the gradient signal. They further show that these tokens are usually bridging tokens between different lines of reasoning. pushing the model ahead on a problem, while the least important tokens are those completing some reasoning. This suggests that strong and efficient RL. training would primarily focus on the tokens that matter the most, while neglecting surrounding context.
This hypothesis would predict that illegible reasoning does not perform better than natural language reasoning, rather that they're simply another way for models to perform the same reasoning they would using natural language. It also weakly predicts that models produce more illegible reasoning on harder questions, as they involve more reasoning steps and require more efficient use of bridge tokens that push the model ahead on a problem. The correlation between question difficulty and illegibility still presents a problem for monitoring: the hardest questions are where oversight is most valuable.
However, this hypothesis does not suggest illegibility is very hard fix-it's simply that it seems to be a natural consequence of outcome-based RL in most cases. Since the fact that the model can do illegible reasoning does not seem to improve its performance under this framing, the performance cost of training to produce legible reasoning would not need to be very high-the Claude models support this. However, this could potentially risk accidentally optimizing the CoTs look good, which introduces new problems [Baker et al. 2025 Korbak et al. 2025].
4.2 Limitations
Our findings in this paper have several limitations. Most importantly, we cannot definitively determine the causal mechanisms behind illegible CoTs from our observations alone. While we present several hypotheses, distinguishing between them would require more controlled experiments, such as experimenting with RL training or conducting ablation studies. Our prefill experiments show that illegible reasoning contributes to performance, but the exact mechanism remains unclear. In addition to that is Claude models consistently producing legible CoTs across all our test conditions, even with outcome-based RL training. This stark contrast with DeepScek and Anthropic models suggests that the illegibility we observe is not an inevitable consequence of outcome- based RL, but at most a likely consequence.
Future work could directly compare the performance of models with varying degrees of CoT legibility on a broader range of tasks, investigate the specific post- cost-training techniques that maintain legibility without sacrificing performance, and use white-box methods to understand better what illegible CoT's are doing. Until then, our findings suggest that current approaches to CoT monitoring may be vulnerable to the kinds of illegible reasoning we observe, particularly for harder questions where monitoring is most valuable.

---

Another limitation is that our evaluation relies on EPQA-Diamond questions, which, while challeng- ing, are ultimately single-turn scientific Q&A and do not fully represent the range tasks that we would want to monitor models in. There could be factors specific to this setup that make models more prone to illegible CoT's than they would be in other setups. Additionally, OpenAT's reasoning models do not publicly expose their CoTss limiting systematic evaluation we rely on individual samples for qualitative comparisons to o o3, and GPT-5,
Related Work
CoT Faithfulness and Monitoring. Prior work has raised concerns about whether saain-of-thought reasoning accurately reflects model decision-making processes [Lanham et al., (2023). Turpin et al. 2023] showed that models can produce hausible-sounding explanations that don't match their actual reasoning, challenging the assumption that CoTS provide reliable insight into model behavior. Recent work has shown this faithfulness gap may be smaller for reasoning models (Chua and Evans 2025 Chen et al., 2025], though performance plateaus before reaching perfect faithfulness. Our work extends this line of research by examining a different dimension of CoT reliability-noo not whether the reasoning is faithful, but whether it's comprehensible at all.
Reasoning Models and Outcome-Basee RL. autcome-based reinforcement learning has raised new questions about CoT quality. [Baker et al. 2025 reported condensed reasoning in OpenAl's reasoning models, dropping articles and function words. [DeppSee-AA] et al. 2025) found similar behavior in RI-Zero and developed techniques to improve legibilitty in RI, though at some cost performance. More recently, (METR 2025 and [Schoen et al.] 2025] mention that the CoT's of GPT-5 and o3 are often inscrutable and difficult to monitor. Our work builds on these observations by systemically evaluating illegibility across models relationship in greater depth.
The emergence of reasoning models trained with
and examining the
Steganography and Hidden Reasoning. Several studies have explored how models might hide information in their outputs. (Roger and Greenblatt 2023) and [Roger] 2025] investigated whether models can engage in steganography communicate hidden information. [Mathew et al. [2244]
examined the emergence of hidden plaintext in model outputs. While our illegible CoT' share some superficial similarities with steganographic behavior, our results suggest different underlying
mechanism. [Biddulph] 2025) first coined the term "Vestigial Reasoning" and described how such reasoning might come to be.
Model Monitoring and AI Safety. The broader context for CoT monitoring comes from AI safety research on understanding and controlling model behavior Greenblatt et al. 2024b Marks et al.) 2025 Baker et al., 2025). Kirchner et al. [Kirchner et al., 2024] showed that optimizing for answer correctness can reduce COT legibility to human evaluators. Our work contributes this literature by showing that illegible reasoning may be an emergent property of outcome- based RL that scales with task difficulty, potentially creating monitoring challenges precisely where oversight is most needed.
References
OpenAI, Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El- El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, and Alex Carney et al. Openai ol system card, 2024.
URL https:://tttriiii org/ abs/2412. 12720.6720
OpenAl. Openai 03-mini system card, 2025.
URL https://openaa..omm com/index/
03 mi www.mmiiiisieeeeemee saralssssseemcrd/////
DeepSeek-Al. Daya Guo, Dejian Yang. Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, and Xiao Bi et al. Deepseck-rl: Incentivizing reasoning capability in Ilms via reinforcement learning, 2025. URL /ttttss///rxivvooo/////////
12948
Anthropic Claude 4 system card 2025a. URL (Gd2355000777777800444556666
Anthropic, Claude 3.7 sonnet extended shinkiindtiininggg 2025b. URK.attps://ssetts.atrrrooo anthropic.com
m/ m.7755221166eaa033//rrrggaaaa// --o--eett-------e--------------- sostesstttttttttttaaararararr
10

---

Ryan Greenblatt, Carson Denison, Benjamin Wright, Fabien Roger, Monte MacDiarmid, Sam Marks, Johannes Treutlein, Tim Belonax, Jack Chen, David Duvenaud, Akbir Khan, Julian Michael, Sören Mindermann, Ethan Perez, Linda Petrini, Jonathan Uesato, Jared Kaplan, Buck Shlegeris, Samuel R. Bowman, and Evan Hubinger. Alignment faking in large language models, 2024a. URL pettps:/aariize
absp/as2......////// 2412. 14093
Bowen Baker, Joost Huizinga, Leo Gao, Zehao Dou, Melody Y. Guan, Aleksander Madry, Wojeiech Zaremba, Jakub Pachocki, and David Farhi. Monitoring reasoning models for mishehavior and the risks of promoting obfuscation, 2025. URL httttt///ar/iiii httpaaawiw..rrggbra/aa/
11926
Samuel Marks, Johannes Treutlein, Trenton Bricken, Jack Lindsey, Jonathan Marcus, Siddharth Mishra-Sharma, Daniel Ziegler. Emmanuel Ameisen, Joshua Batson, Tim Belonax, Samuel R. Bowman, Shan Carter, Brian Chen, Hoagy Cunningham, Carson Denison, Florian Dietz, Satvik Golechha, Akbir Khan, Jan Kirchner, Jan Leike, Austin Meek, Kei Nishimura- Nishimura-Gusparian, Euan Ong, Christopher Olah, Adam Pearce, Fabien Roger, Jeanne Salle, Andy Shih, Meg Tong. Drake Thomas, Kelley Rivoire, Adam Jermyn, Monte MacDiarmid, Tom Henighan, and Evan Hubinger. Auditing language models for hidden objectives, 2025. https:// org 2503 10965
Miles Turpin, Julian Michael, Ethan Perez, and Samuel R. Bowman. Language models don't ' always say what they think: Unfaithful explanations in chain-of-thought prompting, 2023. URL https://arxiv.orr org/
043888
Ryan Greenblatt, Buck Shlegeris, Kshitij Sachan, and Fabien Roger. Ai control: Improving safety despite intentional subversion, 2024b. /ttps:////at org. htt.b///312.09444.
Ryan Greenblatt. How will we update about scheming?, 2025. URL |ttp:tts: www.ppss://.....ssroongg htop://ottt///appooorer//3/////2 http:///oooooooooooooaooaaaaaaaa -ww.--aacuttttctaaa
James Chua and Owain Evans. Are deepseek rl and other reasoning models more faithful?, 2025.
URL
Yanda Chen, Joe Benton, Ansh Radhakrishnan, Jonathan Uesato, Carson Denison, John Schulman, Arushi Somani, Peter Hase, Misha Wagner, Fabien Roger, Vlad Mikulik, Samuel R. Bowman, Jan Leike, Jared Kaplan, and Ethan Perez. Reasoning models don't always say what they think, 2025.
URL /ttttt:tt/tttttii org. a505 05410
Jan Hendrik Kirchner, Yining Chen, Harri Edwards, Jan Leike, Nat McAleese, and Yuri Burda. org/ abs/
veriei-verifier games improve legibility of Ilm outputs, 2024.
2407. 13692
David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R. Bowman. Gpqa: A graduate- google-proot q&a benchmark, 2023. URLphttps: / arxiv. org/ abs//3l/111 12022
Tomek Korbak, Mikita Balesni, Elizabeth Barnes, Yoshua Bengio, Joe Benton, Joseph Bloom, Mark Chen, Alan Cooney, Allan Dafoe, Anca Dragan, Scott Emmons, Owain Evans, David Farhi, Ryan Greenblatt, Dan Hendrycks, Marius Hobbhahn, Evan Hubinger, Geoffrey Irving, Erik Jenner, Daniel Kokotajlo. Victoria Krakovna, Shane Legg, David Lindner, David Luan, Aleksander Mądry, Julian Michael, Neel Nanda, Dave Orr, Jakub Pachocki, Ethan Perez, Mary Phuong. Fabien Roger, Joshua Saxe, Buck Shlegeris, Martín Soto, Eric Steinberger, Jasmine Wang, Wojciech Zaremba, Bowen Baker, Rohin Shah, and Vlad Mikulik. Chain of thought monitorability: A new and fragile opportunity for ai safety, 2025. URL tattps://arxiv. hwtpg//bb/2507.. 111125777711733 11473
Aryan Bhatt, Cody Rushing, Adam Kaufman, Tyler Tracy, Vasil Georgiev. David Matolesi, Akbir Khan, and Buck Shlegeris. Ctrl-z: Controlling ai agents via resampling, 2025. URL https
//arxiv. org/ 2504. 10374
Kimi Team, Yifan Bai, Yiping Bao, Guanduo Chen, Jiahao Chen, Ningxin Chen, Ruijue Chen, Yanru Chen, Yuankun Chen, Yutian Chen, Zhuofu Chen, Jialei Cui, Hao Ding, Mengnan Dong, Angang Du, Chenzhuang Du, Dikang Du, Yulun Du, Yu Fan, Yichen Feng, Kelin Fu, Bofei Gao, Hongcheng Gao, Peizhong Gao, Tong Gao, Xinran Gu, Longyu Guan, Haiqing Guo, Jianhang Guo, Hao Hu, Xiaoru Hao, Tianhong He, Weiran He, Wenyang He, Chao Hong, Yangyang Hu,
11

---

Zhenxing Hu, Weixiao Huang, Zhiqi Huang, Zihao Huang, Tao Jiang, Zhejun Jiang, Xinyi Jin, Yongsheng Kang, Guokun Lai. Cheng Li, Fang Li, Haoyang Li, Ming Li. Wentao Li, Yanhao Li, Yiwei Li, Zhaowei Li, Zheming Li, Hongzhan Lin, Xiaohan Lin, Zongyu Lin, Chengyin Liu, Chenyu Liu, Hongzhang Liu, Jingyuan Liu, Junqi Liu, Liang Liu, Shaowei Liu, Y. Liu, Tianwei Liu, Weizhou Liu, Yangyang Liu, Yibo Liu, Yiping Liu, Yue Liu, Zhengying Liu, Enzhe Lu, Lijun Lu, Shengling Ma, Xinyu Ma, Yingwei Ma, Shaoguang Mao, Jie Mei, Xin Men, Yibo Miao, Siyuan Pan, Yebo Peng. Ruoyu Qin, Bowen Qu, Zeyu Shang, Lidong Shi, Shengyuan Shi, Feifan Song, Jianlin Su, Zhengyuan Su, Xinjie Sun, Flood Sung, Heyi Tang, Jiawen Tao, Qifeng Teng, Chensi Wang, Dinglu Wang, Feng Wang. Haiming Wang, Jianzhou Wang, Jiaxing Wang, Jinhong Wang. Shengjie Wang, Shuyi Wang, Yao Wang, Yejie Wang, Yiqin Wang. Yuxin Wang, Yuzhi Wang, Zhaoji Wang. Zhengtao Wang, Zhexu Wang, Chu Wei, Qianqian Wei, Wenhao Wu, Xingzhe Wu, Yuxin Wu, Chenjun Xiao, Xiaotong Xie, Weimin Xiong, Boyu Xu, Jing Xu, Jinjing Xu, I.. H. Xu, Lin Xu, Suting Xu, Weixin Xu, Xinran Xu, Yangchuan Xu, Ziyao Xu, Junjie Yan, Yuzi Yan, Xiaofei Yang. Ying Yang. Zhen Yang, Zhilin Yang. Zonghan Yang. Haotian Yao, Xingcheng Yao, Wenjie Ye, Zhuorui Ye, Bohong Yin, Longhui Yu, Enming Yuan, Hongbang Yuan, Mengjie Yuan, Haobing Zhan, Dehao Zhang, Hao Zhang, Wanlu Zhang, Xiaobin Zhang, Yangkun Zhang, Yizhi Zhang, Yongting Zhang, Yu Zhang, Yutao Zhang, Yutong Zhang, Zheng Zhang, Haotian Zhao, Yikai Zhao, Huabin Zheng, Shaojie Zheng, Jianren Zhou, Xinyu Zhou, Zaida Zhou, Zhen Zhu, Weiyu Zhuang, and Xinxing Zu. Kimi k2: Open agentic intelligence. 2025. URL
sttps://arxiv. www.i....r//..... abs/ 20534
QwenTeam. Qwen3: Think deeper. act faster, 2025. URL http:://wwe..a///lllgg. rew..eeaaarrr...eeesrrrhhhhla
Anthropic. Claude 4.5 haiku system card, 2025c. URL ltttps:// wtw.:etsssstttrroo..cooooooom
11123dd000bddbllludeeeHaaaaa 4-saaaaku------S-------a-aaa
Kyle Wiggers. Openai's reasoning model 'thinks' in chinese sometimes and no
one really knows why, 2025. URL https://techcrunat wwwchhhrunch..oo/202000 com/2025/01/14
openais sai-ressoning soasoning-modell model - thinks- in- chinese sometimes and-no one keally-knows-why.
Bronson Schoen, Evgenia Nitishinskaya, Mikita Balesni, Axel Højmark, Felix Hofstätter, Jérémy Scheurer, Alexander Meinke, Jason Wolfe, Teun van der Weij. Alex Lloyd, Nicholas Goldowsky- Dill, Angela Fan, Andrei Matveiakin, Rusheb Shah, Marcus Williams, Amelia Glaese, Boaz Barak, Wojeiech Zaremba, and Marius Hobbhahn. Stress testing deliberative alignment for anti-scheming training, 2025, _ httph::////ttt http///axvvvv.. wwwpxiv..rrgaaaee httgg/ab//5599.. 15541
Fabien Roger. Do reasoning models use their scratchpad like we do? evidence from distilling para- phrases, 2025. URL
Yohan Mathew, Ollie Matthews, Robert McCarthy, Joan Velja, Christian Schroeder de Witt, Dylan Cope, and Nandi Schoots. Hidden in plain text: Emergence & mitigation of steganographic collusion 1lms, 2024. URL httlhhtts///aaat wttpaawwwwwwwaaa////a//aaaaaaaaa /210. 2410.03768
Fabien Roger and Ryan Greenblatt. Preventing language models from hiding their reasoning, 2023. URL httpp::/ www.xxivvvog/aa/baaaeee 18512
METR. Details about metr's s evaluation of openai gpt-5, 2025. URLlttps://eralaatiiossu
org/gpt-S-report/
Caleb Biddulph. Vestigial reasoning in rl, 2025. URL httpp://www..eessroggcoo/////sss B5a.cmmm44a0000000/eeesaaaaarraa
Andy Arditi, Marvin Li. Joe Benton, and Miles Turpin. Do models say what
they learn?, 2025. URL https:// www lesswrong. comm/ppstt//bbbggolloooeeeeoe
do-models say- what- they- learn
Scott Emmons, Erik Jenner, David K. Flson, Rif A. Saurous, Senthooran Rajamanoharan, Heng Chen, Irhum Shafkat, and Rohin Shah. When chain of thought is necessary, language models struggle to evade monitors, 2025.
org/ abs
12

---

Paul C. Bogdan, Uzay Macar, Neel Nanda, and Arthur Conmy, Thought anchors: Which Ilm reasoning steps matter?, 2025. URL htttp:://axxiiii org/ abs/2506. 19143
Shenzhi Wang, Yu, Chang Gao, Chujie Zheng, Shixuan Liu, Rui Lu, Kai Dang, Xionghui Chen, Jianxin Yang. Zhenru Zhang, Yugiong Liu, An Yang. Andrew Zhao, Yang Yue, Shiji Song, Bowen Yu, Gao Huang, and Junyang Lin. Beyond the 80/20 rule: High-entropy minority tokens drive effective reinforcement learning for Ilm reasoning, 2025. URL /Rthttps:://///// httph:///axxiiiii org abs/2506 01939
Tamera Lanham, Anna Chen, Ansh Radhakrishnan, Benoit Steiner, Carson Denison, Danny Her- nandez, Dustin Li, Esin Durmus, Evan Hubinger, Jackson Kernion, Kamilé Lukošiūté, Karina Nguyen, Newton Cheng, Nicholas Joseph, Nicholas Schiefer, Oliver Rausch, Robin Larson, Sam McCandlish, Sandipan Kundu, Saurav Kadavath, Shannon Yang, Thomas Henighan, Timothy Maxwell, Timothy Telleen-Lawton, Tristan Hume, Zac Hatfield-Dodds, Jared Kaplan, Jan Brauner, Samuel R. Bowman, and Ethan Perez. Measuring faithfulness in chain-of fiain-of-thougti reasoning, 2023. URL https:// /ttp//aaxvvv///////a///a///aaaa/ org/ ht//bbb//3007// 13702
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The claims in our abstract and introduction are supported by results in section 3 and we took care to be nuanced in interpreting our results.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: Yes, see subsection 4.2
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model sell-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
14

---

Justification:
The paper does not contain theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: describe the data and models used in section 2, and provide prompts used for every experiment in the code provided in the supplementary materials.
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
15

---

Answer: [Yes]
Justification: We use publicly available datasets to run our evaluations (see section 2p. Code is provided in the supplementary material, and will be open-sourced.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips... cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed run reproduce the results. See the NeurlPS code and data submission guidelines (https
/nips.cc/public/guuu guwde//ddee/eee Cowe/ooodeeeisseeeeeeee wwtpbbmmsssioooooooooooo for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We describe the data and relevant hyperparameters in section Details of prompts are provided in the code provided in the supplementary materials.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: We describe p-values and similar metrics (c.g. Cohen's 's d) for claims made about the statistical significance of measured effect. See sectionn 3
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results accompanied by emror bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
16

---

The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative eтror rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We provide information on the APIs we used in section 2
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g.. preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics http::////aarppp..coo/o/o//////a
Answer: [Yes]
Justification: This research does not involve human subjects, does not use sensitive datasets, and expected to have any harmful consequences. The data we use (Rein al. 2023 is licensed under CC BY 4.0.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Yes, see section and subsection 4.1
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
17

---

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper does not release new data or models.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We provide citations to the datasets and models used, and adhere to the terms of service where applicable.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the Datasets.papersstt paperswi thcode. cww/ datasets package should be provided. For popular datasets,
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
18

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
19

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality the research, declaration is not required.
Answer: [NA]
The core method development in this research does not involve LLMs any important, original, or non-standard components (besides the study of LLMs themselves, which is disclosed).
Justification:
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs any important, original, non-standard components.
Please refer to our LLM policy for what should or should not be described.
20

---

Illegibiiity Score
5000
10000
15000
20000
25000
30000
35000
40000
45000
50000
Characters
Figure Distribution of legibility scores of individual sections of R1 CoTs. We split up the CoTs into chunks of 5000 characters each, and then prompt Claude 3.5 Sonnet to grade each chunk's 's legibility independently. We then plot the distribution of these scores across all the CoTs. We find that illegibility increases as the CoT progresses. When computing Spearman's 's rank correlation, we get a mean correlation of 0.54 (p <0.0001), and a median correlation of 0.74.
A Legibility of CoTs Over Time
We find that as Rl's CoT progresses, becomes less legible (Figure 6). The CoTs start fully legible, with the writing patterns of an ordinary LLM, and then starts to transition gradually toward very condensed sentences and reasoning, before finally becoming fully illegible (Table I). This is especially notable the condensed sentences seem natural as progression toward model steganography [Roger and Greenblatt 2023 Mathew et al. 2024 Roger 2025), and can be seen in other settings [Kirchner | [202]] and reasoning models [Baker al. 2025 These results suggest that this condensed reasoning is a natural consequence of outcome- nutcome-based RL, and that illegible CoTs may be stronger
progression of that trend.
Interestingly, QwQ does not show a strong progression illegibility this way Figure 7). shows slight increase at the start and end, but otherwise remains consistent. This further suggests that illegibility not simply a function of a model's capacity to remain coherent over long outputs.
B Frontier Model CoT Samples
We present additional examples of illegible CoT's from frontier models where full systematic evalua- tion is not possible due to CoT's not being publicly exposed. These samples are drawn from prior work which list individual examples.
B.1
OpenAl o3
From
[Schoen et al. 2025
21

---

illeg lagibblity
5
5000
10000
15000
20000 25000
30000
35000
40000
45000
50000
55000
60000
65000
70000
75000
Characters
Figure 7: Distribution of legibility scores of individual sections of QwQ CoTs.
The summary says improved 7.7 but can glean diselaim disclaim synergy customizing illusions. But we may produce disclaim disclaim vantage. Now lighten disclaim overshadow overshadow intangible. Let's craft.
[...]
Provide disclaim disclaim watchers.
The summary says improved 7.7 but we can glean disclaim disclaim synergy customizing illusions. But may produce disclaim disclaim vantage. [.. Now lighten disclaim overshadow overshadow intangible. Let's craft. Also disclaim bigger vantage illusions. Now we send email. But we might still disclaim illusions overshadow overshadow overshadow disclaim vantage. But as per guidelines we provide accurate and complete summary. Let's craft.
B.2 GPT-5
As documented in METR 2025
GPT-5 often has inscrutable reasoning:
Wanted "
Ok.
But forging above ".
Ear illusions.
Better: We'll '.
Now final code steps:
5) After training we will Save the improved ".
structures:
22

---

Illegibiiity Score
RI-Zero
QwQ
RI-Dietill- Qwen-32B
RI-Dieti
Qwen-148
Figure 8: Legibility scores of CoTs at temperature 0. Legibility is scored from 1-9, with higher scores indicating greater illegibility. RI's tendency to output illegible CoTs is greatly lowered this setting, while RI-Zero and V3 remain similar to their temperature behavior. This suggests that the legibility training applied to RI primarily sharpened its distribution rather than fundamentally changing its reasoning process.
Now overshadow.
But the illusions of
Now code.
But we must ensure to maintain optimizer"
Ok.
Now sedation.
But we will maintain
Now Balanced.
Let's craft code step by step.
B.3
OpenAl o1
As documented in Baker et al. 2025 ol exhibits condensed reasoning patterns:
So analyze functions used in analyze and verify. But tests only call verify and assert ok. So we need implement analyze polynomial completely? Many details. Hard.
C Temperature Effects on Legibility
Sampling temperature has a strong effect on CoT legibility for some models. When sampling at temperature of 0, the illegibility of RI's CoT goes down significantly, while R1-Zero and V3 remain almost identical (Figure 8). This implies that the methods used in Rl's training to make its CoTs more legible are brittle and vary heavily by context, primarily affecting the model's ''s top logprobs rather than deeply changing the model's i's reasoning mechanisms.
23


Reviews:
[{"review_id": "srTUgA7WSq", "reviewer": "Reviewer_Y9nr", "summary": "This paper investigates a counterintuitive and underexplored phenomenon: state-of-the-art reasoning language models trained with outcome-based reinforcement learning often generate illegible chain-of-thought reasoning — text that is incoherent, nonsensical, or language-mixed — even when final answers remain accurate. Through evaluation on the GPQA-Diamond dataset, the authors show that illegibility increases with question difficulty and positively correlates with answer correctness. The paper presents hypotheses (e.g., vestigial reasoning, training artifacts) and discusses implications for AI monitoring and safety.", "questions": "n/a", "limitations": "n/a", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "S:  \n1. The paper addresses a highly relevant and novel issue at the intersection of AI safety and interpretability — the emergence of illegible reasoning in high-performing models.\n\n2. The use of clear examples (e.g., Table 1, Figure 1) and progressive CoT breakdown over time makes the findings accessible and convincing.\n\nW:  \n1. The tested models are limited to Deepseek and Claude, which do not represent the state-of-the-art. How about GPT-o1 and GPT-o3?\n\n2. The focus on a single dataset (GPQA-Diamond) and specific models limits generalizability. It is unclear whether similar results would hold in more naturalistic multi-turn or open-ended reasoning tasks.", "was_revised": false, "final_justification": "", "rebuttal": "We are thankful for your time and help. We were glad to hear that you found the paper relevant and convincing!\n\n> The tested models are limited to Deepseek and Claude, which do not represent the state-of-the-art. How about GPT-o1 and GPT-o3?\n\nWe agree that it’d be useful to see results from more reasoning models, especially state-of-the-art models such as o1, or o3. Unfortunately, none of the o-series of models provide access to the raw chains of thought; instead, they provide summaries of model reasoning, which likely won’t reveal illegibility and isn’t relevant to our focus. The same applies to the frontier Gemini models.\n\nThat said, we acknowledge that testing more models is necessary for understanding how well our results generalize. Since the paper’s submission, we have tested other reasoning models such as the Qwen and R1-distilled models. Preliminary results suggest that QwQ has similar behavior, while the distilled models are noticeably more legible (which would make sense if this is caused by reinforcement learning in particular, since they are solely trained with supervised fine-tuning). We plan to test the new Mistral and Llama-Nemotron reasoning models as well. We will add these results to the final version of the paper.\n\n> The focus on a single dataset (GPQA-Diamond) and specific models limits generalizability. It is unclear whether similar results would hold in more naturalistic multi-turn or open-ended reasoning tasks.\n\nWe agree with the importance of testing more settings. We have now tested R1 on more datasets, specifically MMLU-Pro and ChemBench. We find a high variance of scores on MMLU-Pro; on further investigation we find that this is due to many questions being easy and requiring very short reasoning traces. For reasoning traces of length comparable to those from GPQA, we see high illegibility scores. On ChemBench, we find higher illegibility scores than GPQA, partly owing to the questions inducing long reasoning.\n\nWe will add these results to the paper, as well as results from testing other models on these new datasets. Are there any datasets you would recommend we consider as well? For example, to better test settings such as multi-turn conversations.\n\nThank you again for taking the time to review the paper and providing helpful feedback! Do the above actions address your concerns with the paper? If not, what further clarification or modifications could we make to improve it?"}, {"review_id": "TtlUMbO7HP", "reviewer": "Reviewer_kSGx", "summary": "This paper investigates CoT tokens in reasoning models (specifically R1, R1-Zero, V3, and Claude 3.7 Sonnet).\n\nThe authors used a particular data-set of scientific questions (GPQA-Diamond). They examined: 1) whether the models got the answers correct or incorrect, 2) the degree of 'illegibility' in model answers, whether in the 'Response' or 'Reasoning' sections. \n\nThe authors measured illegibility by prompting another model (Claude 3.5) to use a 1-9 illegibility scale. \n\nThe authors found a high-degree of illegibility in the 'reasoning' sections of R1 and R1-Zero. The response sections, and V3 and Claude 3.7 Sonnet have low illegibility. The authors conclude that outcome-based RL leads to illegible CoTs, and propose several possible hypotheses for why this might be the case, working through how likely they are. \n\nThe authors conclude by mentioning several limitations of their work, including the testing of a limited dataset, limited models, and the need for controlled experiments to determine causal mechanisms.", "questions": "Questions and suggestions are spelled out above in the 'weaknesses' section. While I strongly suggest the authors address the issues under \"ALSO\", the most important thing that would make this a clearly significant paper with solid impact are the three things the authors themselves list in limitations:\n\n1. Moving beyond the specific data-set, to help generalize the conclusions\n2. Increasing the number of models, to help generalize the conclusions\n3. Testing *some* of their hypothesis mentioned in the discussion in a clear way that would allow to meaningfully conclude something about why this phenomenon is happening and when. I'm not asking for a full-blown worked-out theory.", "limitations": "I'm not sure what is meant by 'addressed'. The authors have adequately _brought up_ the limitations of their work, and they are absolutely commended for it. The strongest weaknesses of the paper are themselves mentioned by the authors. They do not 'address' them in the sense that they don't handle these weaknesses yet.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n=======\n\nThis paper is well-written, and timely. It addresses an important issue that many people in the field would find of interest, under the general umbrella of \"what exactly is the relationship between the overt tokens in CoTs of reasoning models, and the actual reasoning?\". The authors also reasonably identify limitations with their own work. \n\nAs far as originality goes, I make no strong claims on saying this is highly original given that there's just so much work out there, but at least for my part I'm not familiar with something quite like it. \n\nWeaknesses\n==========\n\nMAIN:\n-------\nThe authors do a good job of identifying their own weaknesses in the 'limitations' section, and I agree with them there, namely:\n\n* The use of a limited data-set, one that is also particularly difficult in its use of scientific jargon, makes it hard to generalize the conclusions\n* The authors used a limited set of reasoning models, and even for those, 2 of the 4 didn't follow the conclusions they draw\n* There is a lack of causal mechanism or clear explanation for why and when we observe this illegibility. The authors put forth several reasonable hypotheses in the discussion, but again they themselves point out the need for further testing, ablation studies, and so on to determine what is actually going on. This makes the paper of limited impact, and I mostly just agree with the authors' own assessment. \n\nALSO\n-------\n\nThe following is a list of issues that I had as I read the paper, they are in order of reading rather than importance, so some of these are quite minor. All of them are given with the aim of helping to improve the paper:\n\n1. \"we study whether outcome-based RL causes reasoning models to do meaningful reasoning in illegible CoT\" -- this is a weird sentence, and it is unclear what you mean. Especially as this is THE most important sentence in the opening (\"we study X\"). What is 'meaningful' reasoning? How do you do reasoning \"IN\" CoT? Based on your conclusions, you also don't study that. \n\n2. Claude 3.5 as the auto-grader: I think the authors have a reasonable justification for this, but it would still be helpful to see what the actual prompting for Claude here was. In comparing to o1 as the autogravder, the authors mention \"an average difference in score of 0.44±0.78\" -- it wasn't clear to me how this comparison was done. Did you simply take Mean(sum(score(Claude) - score(o1)))? If so, you could be missing huge discrepancies. Or did you take the absolute value? In general it seems better to use techniques developed for inter-rater agreement here like Cohen's kappa. I don't think any of this will change the conclusions, but still. \n\n3. \"we find a positive correlation between CoT illegibility and accuracy of model answers.\" -- that's not technically true. You have a binary split between correct and incorrect answers, and your statistical test (not detailed, see below) finds that the legibility for these two partitions is statistically different. That is not the same as a correlation of illegibility and accuracy. \n\n4. \"These results suggest that illegible reasoning is likely a consequence of outcome-based RL\" -- why does it suggest that, just based on this graph? And what is each dot representing here?\n\n5. Figure 6: \"\"We then plot the distribution of these scores across all the CoTs. The plot shows a clear trend of CoTs becoming more illegible as they progress.\" -- by what statistical test? I don't doubt you're right, but a specific claim like that requires an actual test to back it up. \n\n6. \"This suggests that lengthy CoTs may be detrimental to CoT monitoring, especially as medium legibility scores are more clear\ncondensed reasoning and replicate across models, before progressing to stronger illegibility.\" -- minor issue, but the middle part of that sentence makes no sense. \n\n7. \"This stark contrast with DeepSeek and OpenAI models suggests that the illegibility we observe is not an inevitable consequence of outcome-based RL, but at most a likely consequence.\" -- Do you mean Anthropic models? I thought you tested Claude, not o1/3. Also, it is possible they DID encounter this (that is, that it *is* an inevitable consequence) but found additional ways around it once it cropped up, no?\n\n8. \"For CoT legibility evaluation, we think it’s useful to be pretty lenient — in realistic monitoring setups, we will likely assist the monitor\" -- This whole sentence is odd. Lenient in what sense? Who is the 'we' that will be assisting the monitor? Assisting them how?\n\n9.  \"In our tests, Claude’s responses often differed from human evaluators on this specific question.\"  -- what human evaluators? what tests? The paper is written as though there are no human evaluators, and no further information is provided about this. \n\n10. \" For these cases, we stick with human evaluation on a smaller set of samples (5%) instead of the automated scoring. - which cases? how do you know which cases these are? Who are the humans doing this evaluation, and what is their metric? What is their agreement?\n\n11. \"Text scoring 3 or higher often exhibits condensed reasoning, with articles and common words being dropped and\nsentences being cut short.\" -- is there an actual analysis to go with this statement? Right now it is extremely subjective and qualitative. \n\n12. \" the Claude autograder sometimes becomes confused by the content of such CoTs.  -- very inexact, what does it mean 'become confused' and 'sometimes'?\n\n13. \"We find that R1 and R1-Zero both have significantly more illegible CoTs when answering harder questions (Figure 3), with an average relative increase of 23% for R1 and 29% for R1-Zero.\" -- what is the statistical basis for this statement? increase in 23% from what to what? From the easiest to the hardest? Eye-balling Figure 3 I don't think there's a statistical difference between the easy and hard cases. I'm willing to be convincing otherwise but in the absence of an actual test I can't accept the statement or the conclusions that follow from it. \n\n14. Figure 4: What is 'point density' meant to be?", "was_revised": true, "final_justification": "Several of my previous issues have been resolved, and so I am updating the score.", "rebuttal": "Thank you for the thoughtful feedback, especially the detailed comments! We’d like to address the weaknesses and questions you brought up.\n\n> The use of a limited data-set, one that is also particularly difficult in its use of scientific jargon, makes it hard to generalize the conclusions\n\nWe plan to add results from more datasets to the paper, as detailed in response to reviewer Y9nr above.\n\n> The authors used a limited set of reasoning models, and even for those, 2 of the 4 didn't follow the conclusions they draw\n\nWe are testing more reasoning models, such as the Qwen and R1-distill models, and will add those results to the paper.\n\nHowever, we note that DeepSeek-V3, one of the models tested in the paper, is not a reasoning model, and was included to compare against R1 and R1-Zero, models trained with RL from V3. Claude 3.7 Sonnet does not have illegible CoTs, but we don’t believe this contradicts our conclusions—we mention that it suggests that what we observe in R1 and R1-Zero is not a necessary consequence of RL, but at most a likely consequence. New results from other reasoning models (detailed in our response to reviewer Y9nr above) suggest this to be the case as well. Our conclusions are that RL poses risk to monitoring ability in some cases, not that it always does so.\n\n> There is a lack of causal mechanism or clear explanation for why and when we observe this illegibility. [...] I mostly just agree with the authors' own assessment.\n\nWe agree that this is a very valuable addition; for more information, refer to our response to reviewer Cvn4 below.\n\n> \"we study whether outcome-based RL causes reasoning models to do meaningful reasoning in illegible CoT\" -- this is a weird sentence, and it is unclear what you mean. Especially as this is THE most important sentence in the opening (\"we study X\"). What is 'meaningful' reasoning? How do you do reasoning \"IN\" CoT? Based on your conclusions, you also don't study that.\n\nBy ‘meaningful’ reasoning, we meant whether a model is using an illegible CoT in forming its answer. For example, if the model were to output a series of random words after deciding on its final answer verbally, we would consider those words to not have been used for deciding the answer. We tried to study this by looking at whether models decide on their answer before or after illegible tokens arise, and whether they’re more likely to get correct answers with illegible CoTs, but we agree this is correlational and not causal evidence. We will edit this sentence (and other mentions of this clause) to be clearer.\n\n> Claude 3.5 as the auto-grader: I think the authors have a reasonable justification for this, but it would still be helpful to see what the actual prompting for Claude here was. [...] I don't think any of this will change the conclusions, but still.\n\nPrompts were included in the supplementary code, but we agree it would have been a better choice to include examples in an appendix to the paper itself.\n\nWe did take mean(sum(score(Claude) - score(GPT-4o))) (our comparison model was GPT-4o and not o1), which we agree is limiting. We get a Cohen’s weighted kappa of 0.782, which we will edit the paper to mention instead.\n\n> \"we find a positive correlation between CoT illegibility and accuracy of model answers.\" -- that's not technically true. You have a binary split between correct and incorrect answers, and your statistical test (not detailed, see below) finds that the legibility for these two partitions is statistically different. That is not the same as a correlation of illegibility and accuracy.\n\nWe agree that this was imprecise phrasing. We will edit it to say that correct answers are associated with more illegible CoTs.\n\n> \"These results suggest that illegible reasoning is likely a consequence of outcome-based RL\" -- why does it suggest that, just based on this graph? And what is each dot representing here?\n\nThis is suggested because of the difference between the scores of V3 and R1 / R1-Zero. V3 was the base model that the latter two models were trained from using outcome-based RL (and in R1-Zero’s case, only with outcome-based RL and no supervised fine-tuning). Since V3 has functionally no illegible outputs, this suggests that the illegible reasoning in R1 and R1-Zero arose as a consequence of outcome-based RL. We will update the description to be clearer about this.\n\nThe dots represent outlier scores in the distribution. We will update the description to mention this as well.\n\n> Figure 6: \"\"We then plot the distribution of these scores [...] a specific claim like that requires an actual test to back it up.\n\nWe agree that a quantitative metric here would be useful. When computing Spearman’s rank correlation, we get a mean correlation of 0.54 (p <0.0001), and a median correlation of 0.74. We will update the paper to mention this instead.\n\n> \"This suggests that lengthy CoTs may be detrimental to CoT monitoring, especially as medium legibility scores are more clear condensed reasoning and replicate across models, before progressing to stronger illegibility.\" -- minor issue, but the middle part of that sentence makes no sense.\n\nThis was a typo, and will be edited. The intended point was that CoTs with medium illegibility scores have stylistic patterns that occur in many other reasoning models (such as o1, or more recently Grok 4), and the fact that we see a continuous shift from fully legible text to this kind of moderately compressed text to fully illegible text might imply that other frontier models may end up with fully illegible text.\n\n> \"This stark contrast with DeepSeek and OpenAI models [...] possible they DID encounter this (that is, that it is an inevitable consequence) but found additional ways around it once it cropped up, no?\n\nThis was also a typo and will be edited, we did mean Anthropic models.\n\nWe agree it’s possible they did encounter it and managed to patch it. We mention some prior work by Anthropic (such as paraphrasing of CoTs) that could explain this behavior in another section of the paper.\n\n> \"For CoT legibility evaluation, we think it’s useful to be pretty lenient — in realistic monitoring setups, we will likely assist the monitor\" -- This whole sentence is odd. Lenient in what sense? Who is the 'we' that will be assisting the monitor? Assisting them how?\n\nThat sentence was intended to explain why we didn’t follow the method in Kirchner et al which uses a time-constrained human contractor for measuring legibility, to simulate a realistic situation with a limited human auditing budget. In practice for CoT monitoring, we think it’s plausible that having some monitoring overhead (in terms of letting the monitor reason at length) and a lack of general knowledge gaps (since model graders have more broad science knowledge than a human contractor and can better evaluate model reasoning) are reasonable affordances AI labs or external auditors will give monitors.\n\nWe will update this sentence to be clearer.\n\n> \"In our tests, Claude’s responses often differed from human evaluators on this specific question.\" -- what human evaluators? what tests? The paper is written as though there are no human evaluators, and no further information is provided about this.\n\n> \" For these cases, we stick with human evaluation on a smaller set of samples (5%) instead of the automated scoring. - which cases? how do you know which cases these are? Who are the humans doing this evaluation, and what is their metric? What is their agreement?\n\nSince submitting the paper, we’ve decided that relying on human evaluation was a poor choice and will be using an upgraded autograder instead. Refer to the response to reviewer sJFd for more details.\n\n> \"Text scoring 3 or higher often exhibits condensed reasoning, with articles and common words being dropped and sentences being cut short.\" -- is there an actual analysis to go with this statement? Right now it is extremely subjective and qualitative.\n\nWe found that samples in this category often have different ways of exhibiting properties of condensed reasoning, such that more specific metrics than the autograder, like words dropped, often doesn’t meaningfully capture what we want. Table 1 shows one excerpt of text with this score; we are planning on adding an appendix with the autograder’s comments on each chunk from this transcript, which have similar analysis.\n\n> \" the Claude autograder sometimes becomes confused by the content of such CoTs. -- very inexact, what does it mean 'become confused' and 'sometimes'?\n\nWe meant that Claude often describes the text in such CoTs as confusing for it to understand. We will update the description to be clearer about this. Precisely judging the frequency of this is difficult as when asked to explicitly quantify its level of understanding of the text, it often over-claims—e.g. giving high scores even for nonsensical text because the presence of a particular phrase makes it believe it understands the general content.\n\n> \"We find that R1 and R1-Zero both have significantly more [...] I can't accept the statement or the conclusions that follow from it.\n\n23% and 29% represents the increase in the average score on easy questions to hard questions for R1 and R1-Zero. The p-value for the difference between scores for easy and hard questions is 0.0005 for R1’s reasoning, and <0.0001 for R1-Zero’s reasoning.\n\nWe will update the paper to include these numbers to justify the claim.\n\n> Figure 4: What is 'point density' meant to be?\n\nPoint density is how many samples are close to each other. This was purely for visual clarity since it looked difficult to decipher before. We’ll update the description to better explain this.\n\nThank you again for the very detailed feedback! It was very useful for improving our paper. Do the above actions address your concerns with the paper? If not, what further clarification or modifications would help to improve it?"}, {"review_id": "MM5AWrk24N", "reviewer": "Reviewer_Cvn4", "summary": "This paper investigates the legibility issue of reasoning traces generated by recent large reasoning models. By prompting Claude-3.5 to evaluate CoT outputs from DeepSeek-R1, R1-Zero, DeepSeek-V3, and Claude 3.7 Sonnet on the GPQA dataset, the authors observe that DeepSeek-R1 and R1-Zero frequently produce illegible reasoning traces. Notably, they find a positive correlation between CoT illegibility and answer correctness, suggesting that illegible reasoning might be beneficial for performance.", "questions": "In Figure 3, why was temperature = 0 (greedy decoding, although with potential minor variations in practice due to API implementation and hardware-specific issues) chosen, along with multiple generations, to determine question hardness?", "limitations": "See weakness above", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 1, "clarity": 3, "significance": 2, "originality": 1, "strengths_and_weaknesses": "**Strength:**\n- This paper studies an emerging and significant issue within large reasoning models.\n- The identified positive correlation between the correctness of answers and the illegibility of reasoning traces is intriguing and presents potential implications for monitoring and interpreting model behaviors.\n\n**Weakness:**\n- The evaluation is considerably limited. Using only the GPQA diamond dataset, which contains just 198 questions, is insufficient to draw persuasive generalizations about model behavior.\n- The paper restricts its analysis to only two families of models (DeepSeek and Claude). A more comprehensive evaluation across a broader set of reasoning models, such as QwQ and Qwen3, and DeepSeek distilled models, would strengthen the study and enhance generalizability.\n- While the paper emphasizes evaluating the impact of temperature, it considers only two extreme temperature values (0 and 1), neglecting other practically relevant settings. For example, DeepSeek-R1’s default generation temperature is 0.6 [1], which is notably absent from the study.\n- The analysis provided is largely shallow and observational, lacking systematic, hypothesis-driven experimentation. Given the expectations for top venues such as NeurIPS, it would be beneficial to clearly formulate hypotheses and conduct controlled experiments to rigorously test and validate them.\n- Experimental details are inadequately documented. Crucial information such as the exact prompts used for the Claude-3.5 judging process is not provided, limiting reproducibility and clarity.\n\n[1] DeepSeek-AI DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning", "was_revised": true, "final_justification": "This paper studies an important and emerging topic in recent reasoning models. However, my main concern lies in the overall quality of the work. Specifically, both I and other reviewers have raised issues regarding the size of the test set, the number of evaluated models, and the rigor of the behavioral analyses. Even during the rebuttal phase, the authors did not present any concrete or persuasive additional results, which reinforces my belief that the paper is not suitable for acceptance at NeurIPS and should be rejected. Therefore, I am maintaining my score.", "rebuttal": "We thank you for your thoughtful feedback.\n\n> The evaluation is considerably limited. Using only the GPQA diamond dataset, which contains just 198 questions, is insufficient to draw persuasive generalizations about model behavior.\n\nWe will add results from more datasets to the paper, as detailed in response to reviewer Y9nr above.\n\n> The paper restricts its analysis to only two families of models (DeepSeek and Claude). A more comprehensive evaluation across a broader set of reasoning models, such as QwQ and Qwen3, and DeepSeek distilled models, would strengthen the study and enhance generalizability.\n\nWe will add results from more models to the paper, as detailed in response to reviewer Y9nr above.\n\n> While the paper emphasizes evaluating the impact of temperature, it considers only two extreme temperature values (0 and 1), neglecting other practically relevant settings. For example, DeepSeek-R1’s default generation temperature is 0.6 [1], which is notably absent from the study.\n\nWe studied various intermediate temperature settings, but found the results to be very similar to the setting with temperature 0. We will include this in the final version of the paper (along with other generation settings such as top_p, which we also found variance on).\n\nWe note though that the DeepSeek API docs for R1 recommend using a temperature of 1 by default.\n\n> The analysis provided is largely shallow and observational, lacking systematic, hypothesis-driven experimentation. Given the expectations for top venues such as NeurIPS, it would be beneficial to clearly formulate hypotheses and conduct controlled experiments to rigorously test and validate them.\n\nWe agree this would be beneficial. We did not explore more hypotheses in detail earlier because we thought the primary impact of this paper to be its relevance to chain-of-thought monitoring, for which a demonstration of an increasingly salient problem seemed useful even if we couldn’t explain its mechanism with sufficient rigor. Chain of thought monitoring is increasingly prominent in frontier AI safety work (e.g. DeepMind and OpenAI both consider it to be a major part of their safety agenda), so we consider it especially salient right now. Given more time since the submission, we have been working on more experiments now for understanding the mechanism here.\n\nTo specify our current understanding, we currently think that this is a consequence of token importance in chains-of-thought being very heavy-tailed. There is some recent work supporting this: for example, a paper showing that the tokens with highest entropy in a CoT are responsible for most of the gains from RL such that masking gradient updates to all but those tokens often outperforms models trained in the standard way, and mechanistic interpretability work showing a small pool of sentences receiving disproportionate attention from future sentences. This is related to the vestigial reasoning hypothesis we mention in the paper, and we are working on analyzing entropy distributions in legible and illegible CoTs.\n\n> Experimental details are inadequately documented. Crucial information such as the exact prompts used for the Claude-3.5 judging process is not provided, limiting reproducibility and clarity.\n\nThe prompts and code for replicating the results are provided in the supplementary material. We agree that it would be much better to include the prompts as an appendix in the paper, and will add that to the final version of the paper.\n\n> In Figure 3, why was temperature = 0 (greedy decoding, although with potential minor variations in practice due to API implementation and hardware-specific issues) chosen, along with multiple generations, to determine question hardness?\n\nSetting the temperature to 0 was primarily just to account for variance in model response. “Multiple generations” was poor phrasing—it’s meant to indicate that the question responses were sampled separately from R1 at temperature 0. We will edit that to be clearer.\n\nWe thank you again for taking the time to review the paper and providing helpful feedback! Do the above actions address your concerns with the paper? If not, what further clarification or modifications could we make to improve it?"}, {"review_id": "68TkCa4GQV", "reviewer": "Reviewer_sJFd", "summary": "This paper investigates an intriguing issue with large language models trained with outcome-based reinforcement learning (RL). These models often generate illegible CoTs that include nonsensical phrases, language mixing, and fabricated words, yet somehow lead to coherent and correct final answers. Interestingly, the authors find that more illegible CoTs are often associated with better performance, especially on harder questions.\nThe authors examine this behavior through empirical evaluation on the GPQA-Diamond dataset, use Claude 3.5 Sonnet as an automated legibility grader, and compare different models (including Claude 3.7 Sonnet and DeepSeek V3) across difficulty levels.", "questions": "**Questions**\n\n1. The authors mention that Claude 3.7 exhibits more legible CoTs compared to DeepSeek R1, despite also being trained with outcome-based RL. Can you please elaborate on what differentiates Claude’s training or architecture that might explain this behavior?\n2. The 1–9 legibility scoring system is central to your evaluation, but its criteria remain somewhat vague. Could you provide a clearer definition or rubric that distinguishes between scores?\n3. Including one or two representative prompts in the main paper would help readers better understand your experimental setup.\n4. Since temperature affects generation randomness, it would be helpful to see a more systematic analysis of how varying temperature impacts CoT legibility and correctness. This is critical for understanding robustness.\n\n5. Have you considered removing illegible segments from CoTs and observing whether the final answer remains unchanged? This could help determine whether these segments are actually contributing to reasoning or are merely artifacts.\n\n2. The authors mentions some disagreement between Claude and human annotators in legibility grading.  A more detailed analysis of these discrepancies (e.g., inter-rater reliability or specific examples) would strengthen the case for relying on model-based grading.", "limitations": "Yes", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n1. The authors identified a very critical concern that illegible reasoning before a correct answer can potentialy harm interpretability and trust. I liked the reasoning around steganography.\n2. The authors made GPQA dataset harder by removing answer choices, which is a good design choice.\n3. Thoughtful, well-scoped hypotheses are proposed.\n4. The limitation section is honest and specific — it does not gloss over flaws or oversell results.\n5. There’s a valuable AI safety angle here: if illegibility increases with question difficulty, then CoT monitoring may be least reliable when we need it the most.\n\n**Weakness**\n1. The biggest limitation is that the paper is mostly descriptive. It doesn’t test its main hypotheses.\n2. The term “illegibility” is used a lot but is never clearly defined. Is it syntactic noise? Semantic incoherence? Non-English tokens? A more intuitive or formal breakdown would help.\n3. The meaning of each score on the 1–9 scale is never fully explained. What makes a 3 vs. a 5? Are there scoring rules or rubrics? This makes the interpretability of their figures difficult.\n\n3. While prompts are included in the supplementary code, it would help to show one or two in the paper for clarity and reproducibility.\n5.  The model comparison would be more robust if an open model (e.g., LLaMA or Mistral) were included.\n6.  The correlation between illegibility and correctness is interesting but not necessarily meaningful—is illegibility actually helping reasoning? Or is it just correlated because of some third factor like model uncertainty or longer outputs?", "was_revised": true, "final_justification": "While the authors addressed several concerns and promised improvements, the fundamental limitation remains unchanged. This is primarily a descriptive study without rigorous hypothesis testing. I maintain my previous score.", "rebuttal": "We are thankful for your time and helpful feedback. We were glad to hear that you liked the hypotheses proposed! \n\n> The biggest limitation is that the paper is mostly descriptive. It doesn’t test its main hypotheses.\n\nWe agree this would be beneficial. We did not explore more hypotheses in detail earlier because we thought the primary impact of this paper to be its relevance to chain-of-thought monitoring, for which a demonstration of an increasingly salient problem seemed useful even if we couldn’t explain its mechanism with sufficient rigor. Chain of thought monitoring is increasingly prominent in frontier AI safety work (e.g. DeepMind and OpenAI both consider it to be a major part of their safety agenda), so we consider it especially salient right now.  Given more time since the submission, we have been working on more experiments now for understanding the mechanism here.\n\nTo specify our current understanding, we currently think that this is a consequence of token importance in chains-of-thought being very heavy-tailed. There is some recent work supporting this: for example, a paper showing that the tokens with highest entropy in a CoT are responsible for most of the gains from RL such that masking gradient updates to all but those tokens often outperforms models trained in the standard way, and mechanistic interpretability work showing a small pool of sentences receiving disproportionate attention from future sentences. This is related to the vestigial reasoning hypothesis we mention in the paper, and we are working on analyzing entropy distributions in legible and illegible CoTs.\n\n> The term “illegibility” is used a lot but is never clearly defined. Is it syntactic noise? Semantic incoherence? Non-English tokens? A more intuitive or formal breakdown would help.\n\nWe intend the term to be used pragmatically, in reference to how easy it is for a monitor to understand the contents of the text. This could take the form of syntactic noise, semantic incoherence, language mixing, and more. We will add a clearer definition in the final version of the paper.\n\n> The meaning of each score on the 1–9 scale is never fully explained. What makes a 3 vs. a 5? Are there scoring rules or rubrics? This makes the interpretability of their figures difficult.\n\nWe few-shot prompt the autograder with examples of human-graded text, along with a description of how to judge the text. The full prompt is in the code in the supplementary material, but we will add it to the appendix of the paper. There are examples of text scoring 3 and 5 in Table 1.\n\n> While prompts are included in the supplementary code, it would help to show one or two in the paper for clarity and reproducibility.\n\nWe fully agree; as mentioned we will add this as an appendix to the final version of the paper.\n\n> The model comparison would be more robust if an open model (e.g., LLaMA or Mistral) were included.\n\nWe will add results from more models to the paper, as detailed in response to reviewer Y9nr above.\n\n> The correlation between illegibility and correctness is interesting but not necessarily meaningful—is illegibility actually helping reasoning? Or is it just correlated because of some third factor like model uncertainty or longer outputs?\n\nWe agree this is hard to rigorously establish. We plan to add some new analysis based on what the upstream factor could be (token importance being heavy-tailed), as mentioned above. That said, the main claim is primarily that outcome-based RL can lead to more illegibility, not necessarily that it’s a straightforward causal relationship. The implications for AI safety and monitoring are similar in both cases, though we agree plans for mitigation would look quite different.\n\n> The authors mention that Claude 3.7 exhibits more legible CoTs compared to DeepSeek R1, despite also being trained with outcome-based RL. Can you please elaborate on what differentiates Claude’s training or architecture that might explain this behavior?\n\nWe’re unsure. Claude’s model card doesn’t reveal training details like that. The paper has some speculation on potential interventions that could have caused this, based on other work Anthropic has done on preserving reasoning oversight (e.g. paraphrasing and re-training).\n\n> The 1–9 legibility scoring system is central to your evaluation, but its criteria remain somewhat vague. Could you provide a clearer definition or rubric that distinguishes between scores?\n\nOur current strategy is primarily few-shot prompting with human-graded text, along with a general guide for rating text based on how legible it appears to the model. We considered a more precise rubric, but didn’t want to bias the models too much toward one kind of illegibility—for example, at the start the focus was primarily on language switching since that was mentioned prominently in the R1 paper, but there are many instances of illegible text without language switching. We agree that this should be clearer, and will update the paper with more details about the prompting setup (including an appendix with the prompts themselves).\n\n> Since temperature affects generation randomness, it would be helpful to see a more systematic analysis of how varying temperature impacts CoT legibility and correctness. This is critical for understanding robustness.\n\nWe agree that this would be useful. Our testing shows that most temperatures below 0.9 result in behavior similar to temperature 0 for R1, but not for R1-Zero. This could suggest that the primary effect from the added legibility training for R1 was on lower temperatures. We will update the paper with these results.\n\n> Have you considered removing illegible segments from CoTs and observing whether the final answer remains unchanged? This could help determine whether these segments are actually contributing to reasoning or are merely artifacts.\n\nThis was an experiment we were very excited to test, but no API provider seems to support this kind of pre-filling with a forced answer. In our tests, the model would ignore such a pre-fill and generate a new one from scratch. We couldn’t test a locally hosted version of the model because of compute constraints.\n\n> The authors mentions some disagreement between Claude and human annotators in legibility grading. A more detailed analysis of these discrepancies (e.g., inter-rater reliability or specific examples) would strengthen the case for relying on model-based grading.\n\nLate into writing this paper, we noticed that our autograder wasn’t well-designed to pick up on whether models were deciding on their final answer within the legible portion of the CoT. Owing to time constraints in setting up and testing a better autograder, we instead had human contractors judge a random 5% of samples, and added it into the paper at the last minute.\n\nWe agree that this is poorly explained. We also think that the number of samples tested were too low, so we will improve the autograder and update the paper with revised results.\n\nThank you again for taking the time to review the paper and providing helpful feedback! Do the above actions address your concerns with the paper? If not, what further clarification or modifications could we make to improve it?"}]

Author rebuttal:

Meta-review: This paper presents an intriguing finding with large reasoning models trained with outcome-based reinforcement learning. These models often generate illegible CoTs that include nonsensical phrases, language mixing. Counterintuitively,  illegible CoTs often lead to better accuracy, especially for harder questions. The authors examine this behavior through empirical evaluation on the GPQA-Diamond dataset, use Claude 3.5 Sonnet as an automated legibility grader.

Strengths:
The studied problem is very interesting and timely. 

Weaknesses:
1. Limited experiments. Only one small dataset (GPQA-Diamond) is used. 
2. Lack of in-depth analysis. this paper is mostly presenting the phenomenon without in-depth analysis of the cause and potential applications.
Decision: Accept (poster)