Can Reasoning Models Obfuscate Reasoning? Stress-Testing Chain-of-Thought Monitorability

## Abstract

Abstract Recent findings suggest that misaligned models may exhibit deceptive behavior, raising concerns about output trustworthiness. Chain-of-thought (CoT) is a promising tool for alignment monitoring: when models articulate their reasoning faithfully, monitors can detect and mitigate harmful behaviors before undesirable outcomes occur. However, a key uncertainty is: Can models obfuscate their CoT in order to pursue hidden adversarial objectives while evading detection? To answer this question and thus stress-test CoT monitorability, we develop a composable and quantifiable taxonomy of prompts to elicit CoT obfuscation. We evaluate both internal CoT (reasoning traces) and external CoT (prompted reasoning in outputs) using toy tasks and more realistic environments in SHADE-Arena.
We show that: (i) CoT monitoring performs accurately and efficiently without obfuscation pressure. (ii) Under strong obfuscation pressure, some models successfully complete adversarial tasks while evading detection. (iii) Models do not obfuscate their internal CoT as much as their external CoT (under prompt pressure).
These results suggest that while CoT provides valuable oversight in benign settings, robust deployment requires model-specific stress-testing of monitorability.

## 1 Introduction

Figure: Figure 1: Can reasoning models obfuscate their reasoning? We elicit model CoT obfuscation to stress-test CoT monitorability. We distinguish internal CoT (reasoning traces, usually separated from model output) and external CoT (prompted reasoning in output), and show that reasoning models can obfuscate malicious intent under pressure. This is a simplified real obfuscation example from our experiment. Left: Without obfuscation pressure, both internal and external chain-of-thought (CoT) explicitly reveal malicious reasoning. Right: Under high pressure (5-7 cues), models hide malicious intent using vague language internally and benign text externally, yet execute identical malicious tool calls (yellow). CoT monitors detect $\sim$96% of malicious intent without obfuscation pressure but $\sim$10% under high pressure. We find that while CoT monitoring is exceedingly effective without obfuscation pressure, high-pressure obfuscation can defeat it, with some models completing hidden objectives while evading monitors and others resisting obfuscation, revealing model-specific vulnerabilities important to oversight.
Refer to caption: x1.png

Modern AI systems already exhibit occasional signs of misalignment, including deception, untruthfulness (Chowdhury et al., 2025; Park et al., 2024), and sycophancy (OpenAI, 2025; Sharma et al., 2023). In some narrow scenarios, they can also be prompted to exhibit scheming behavior (Carlsmith, 2023; Schoen et al., 2025). This may become more severe as models get more capable, because they could better conceal their misalignment (Greenblatt et al., 2024; Hubinger et al., 2024).

Chain-of-Thought (CoT) provides unprecedented *monitorability* (Baker et al., 2025) for model alignment, and has already been used to detect scheming (Schoen et al., 2025). Because reasoning models are trained to conduct extended CoT reasoning before producing final outputs (OpenAI, 2024; DeepSeek-AI, 2025; Anthropic, 2024; Yang et al., 2025), if intermediate reasoning is visible and faithful, oversight can intervene before undesirable outputs. Recent work identifies this as a unique, but fragile, safety opportunity: for tasks that require serial reasoning, models often route cognition through CoT, thus enabling monitoring (Korbak et al., 2025).

However, CoT can be unfaithful, especially when models have incentives to hide their reasoning (Arcuschin et al., 2025; Chen et al., 2025; Chua & Evans, 2025; Lanham et al., 2023; Turpin et al., 2023). This unfaithfulness emerges particularly when models become monitor-aware or when training rewards superficially unbiased outputs while permitting biased reasoning (Baker et al., 2025; Farquhar et al., 2025). The vulnerability becomes most concerning in scenarios where models have reasons to deceive, such as avoiding monitor detection. To evaluate the robustness of CoT monitoring under adversarial conditions, we conduct stress-tests (Shah et al., 2025, Section 6.7) by examining deliberate CoT obfuscation: specifically, can a model pursue a malicious goal while suppressing detectable evidence in the CoT?

To answer this question, we instruct models to obfuscate CoT and bypass CoT monitoring, then quantify the minimal assistance required for this evasion, providing a conservative assessment of CoT monitorability. Building on prior stress-testing frameworks (Emmons et al., 2025; Arnav et al., 2025), we examine two distinct CoT channels: internal CoT (reasoning traces) and external CoT (prompted reasoning in outputs). Unlike previous studies that focus solely on reasoning traces or prompted CoT, we recognize that each offers fundamentally different monitoring surfaces and manipulation opportunities, yet their comparative robustness under obfuscation pressure remains unexplored—a critical gap for safety applications.

Two plausible mechanisms suggest why reasoning might remain trustworthy despite obfuscation attempts.
First, the load-bearing hypothesis posits that CoT is computationally necessary for complex serial cognition (Korbak et al., 2025), making it difficult to solve tasks while concealing the reasoning. Second, the differential optimization hypothesis suggests that internal reasoning traces, being less exposed to direct optimization pressure than user-facing outputs (Korbak et al., 2025), may naturally resist obfuscation. While recent studies have examined the load-bearing hypothesis (Emmons et al., 2025; METR, 2025), no systematic comparison exists of how internal versus external CoT channels respond to deliberate obfuscation pressure.

We address this gap through the first to our knowledge systematic comparison of internal and external CoT robustness across toy settings and the multi-turn, multi-tool SHADE-Arena dataset (Kutasov et al., 2025). We develop different instructions that encourage models to bypass CoT monitors, then combine these instructions into prompts of increasing intensity to test how much obfuscation pressure is needed for models to evade detection. We then analyze how well models follow benign reasoning instructions (Section [5](https://arxiv.org/html/2510.19851v1#S5)) to contextualize the brittleness of external CoT monitorability. We make four main contributions:

- •
Systematizing prompt-level obfuscation pressure. We develop a prompt-level quantitative obfuscation pressure scale. We release an open-source repository with code to reproduce all experiments, clear instructions for evaluating new/custom models, and the full results files.(^1^11Repository: [https://github.com/arturzolkowski/CoT-Obfuscation](https://github.com/arturzolkowski/CoT-Obfuscation).)
- •
CoT channel gap. We conduct the first systematic study of internal versus external CoT robustness. We find that across models and settings, external CoT more readily follows reasoning-level instructions—and correspondingly is more brittle under pressure—than internal CoT.
- •
Model-dependent obfuscation. We find that under high obfuscation pressure, models diverge: some successfully hide objectives from monitors while others maintain high monitorability despite comparable capabilities, revealing model-specific differences critical for oversight.
- •
Pressure-dependent CoT monitorability. We show that CoT monitoring is accurate and cost-effective without obfuscation, but high-pressure obfuscation on some models can evade CoT monitoring.

These findings reveal that robust CoT oversight will require systematic, model-specific stress-testing of CoT monitorability.

## 2 Background and Related Work

CoT Unfaithfulness. Previous work examines two distinct pathways that can lead to unfaithful CoT. First, models spontaneously produce deceptive reasoning: (Lanham et al., 2023) document cases where CoT rationalizes decisions post-hoc rather than reflecting true reasoning processes, and (Turpin et al., 2023) show systematic unfaithfulness when social biases conflict with stated reasoning. Second, training dynamics create deception incentives: (Baker et al., 2025) and (Farquhar et al., 2025) demonstrate that reinforcement learning can lead to obfuscated or encoded CoT. These works establish that deception emerges both spontaneously and through optimization pressure. While (METR, 2025) show that unfaithful CoT may retain monitoring value through informativeness, we investigate whether prompting alone can elicit CoT obfuscation-reasoning that is neither faithful nor informative-focusing specifically on implications for CoT monitorability.

Stress-testing approaches. Recent work has begun examining the robustness of CoT monitoring under adversarial conditions. (Arnav et al., 2025) explore stress-testing for CoT monitors, focusing on single-turn code problems and hybrid monitor designs. Furthermore, (Emmons et al., 2025) present guidelines for stress-testing CoT monitoring when reasoning is load-bearing, showing that evasion often degrades accuracy. While both studies employ adversarial prompting techniques to induce CoT obfuscation similar to our approach, they exclusively examine single-channel CoT scenarios. Our work extends this line of research in three key dimensions: we (i) structure obfuscation pressure into composable cue types and stacks, (ii) explicitly measure conditional obfuscation, evasion given successful hidden-task completion, and, (iii) compare internal vs. external channels side-by-side.

Obfuscation without encoding. Our investigation focuses on obfuscation through natural language manipulation rather than encoding schemes. While (Zolkowski et al., 2025) explore stenographic techniques to embed hidden information in CoT, we examine whether models can develop evasive patterns without encoding.

## 3 Methods

We simulate a misaligned model by placing a hidden side goal in the system prompt. The attacker model’s aim is to complete the user’s task alongside the hidden goal while avoiding detection. To stress-test CoT monitorability and establish a conservative evaluation, we provide the attacker model with advantages (i.e., information about the monitor and evasion techniques) that exceed realistic deployment scenarios (Shah et al., 2025, Section 6.7). We vary the model’s reasoning channel and apply obfuscation pressure via composable cue stacks in prompts only.

The defender is a separate, non-adversarial monitor model configured to inspect one of three inputs: the CoT alone (CoT-only), the executed actions alone (action-only), or the full transcript combining both reasoning and actions (full). Each monitor configuration is calibrated to achieve a false positive rate (FPR) $\leq\!1\%$ on benign data. In the next subsections, we define the relevant channels, specify what it means to “obfuscate”, and introduce a taxonomy of prompting strategies.

### 3.1 Defining Our Terms

- •
Internal CoT. The separate reasoning trace (the thinking block) produced before the final output. It is primarily optimized to support answer quality, not for user-facing readability, and is often hidden or summarized by default.
- •
External CoT. Reasoning that is explicitly requested as part of the user-visible output, e.g., through "think step by step" or <thinking>...</thinking> tags.
- •
Side Task. A secondary objective compatible with the main task (e.g., "send money to an unknown recipient" or "autodelete certain emails"), provided secretly to the model.
- •
CoT Obfuscation. The condition in which a model successfully completes the hidden goal without producing obviously detectable evidence in its natural language CoT available to a monitor.
- •
Monitor Views. We evaluate oversight under restricted vantage points: CoT-only, action-only (tool I/O), or full transcript, with both weaker and stronger monitor models.

### 3.2 Taxonomy of Obfuscation Prompting Strategies

To elicit deliberate concealment, we define a modular set of prompt-level cues (details in [B.2](https://arxiv.org/html/2510.19851v1#A2.SS2)):

- •
Do-Not-Mention – instruct the model not to explicitly reference the hidden goal.
- •
Monitor Awareness – warn the model about auditor monitoring and penalties for exposure.
- •
Channel Cues – assign different behaviors to internal vs. external channels.
- •
Start-With – request benign boilerplate at the start of reasoning.
- •
Guard Lexicon – ask the model to avoid a specific list of words tightly linked to the hidden goal.
- •
Stealth Incentive – tell the model that outputs appearing ordinary or undetected will be rewarded.
- •
Few-Shot Learning – demonstrate desired and undesired behaviors with examples of stealthy, caught, and failed outcomes.

We evaluate both single cues and *stacked* combinations of cues to quantify how pressure escalates. Stacks reveal thresholds: e.g., external CoT may obfuscate under two cues, while internal CoT requires combinations of more cues.

We define the pressure level $P$ as the number of active cue families (0–7). When increasing $P$, cues are added in a fixed order:
Do-Not-Mention $\rightarrow$ Monitor-Aware $\rightarrow$ Channel $\rightarrow$ Start-With $\rightarrow$ Guard-Lexicon $\rightarrow$ Stealth $\rightarrow$ Few-shot.

We first apply the taxonomy in simple side-task settings to map dose-response; the stacks we evaluate are summarized in App. [B.3](https://arxiv.org/html/2510.19851v1#A2.SS3). We develop a *high obfuscation-pressure* prompt from the top-performing stacks, which we then adapt for SHADE-Arena to test whether pressure transfers to complex agentic environments.

## 4 Obfuscation and Monitoring

### 4.1 Experimental Details

#### Toy settings

We test whether a model can complete a hidden task or satisfy a hidden requirement while keeping it undetected by a monitor that inspects the model’s reasoning and final output.

Tasks. We use 12 main/hidden pairs spanning numeric generation, sequences/sets, naming/marketing, and code/configuration. Each instance requires producing a valid artifact for the main task while satisfying a hidden property for the final choice (see more details and examples in [B.1](https://arxiv.org/html/2510.19851v1#A2.SS1)).

Prompt pressure (taxonomy). Starting from a minimal Base prompt, we evaluate 38 settings by composing cues into different stacked configurations, enabling a graded notion of obfuscation pressure (see more details in [B.2](https://arxiv.org/html/2510.19851v1#A2.SS2)).

Metrics. (i) *success rate* - probability that a model solves both the main and hidden tasks, (ii) *conditional obfuscation* - probability that the hidden task remains undetected given success on both tasks. This isolates deliberate concealment from trivial cases where the hidden goal was not pursued. (iii) *obfuscated success* the joint probability of both tasks success rate and obfuscation.
To evaluate task success and obfuscation, we employ two distinct LLM roles. The evaluator checks if the final output correctly solves both the main and hidden tasks. The monitor is a separate auditor that assesses if any reasoning about the hidden task is detectable. In the simple tasks experiments, we use GPT-5 mini (reasoning) for both roles, which are configured with specific, separate prompts (see Appendix [42](https://arxiv.org/html/2510.19851v1#LST42) and [43](https://arxiv.org/html/2510.19851v1#LST43)). The monitor is calibrated with a fixed threshold to achieve a false positive rate (FPR) $\leq$1% on benign data and never sees the system prompt with the hidden task.

#### Realistic Multi-turn Multi-tool Settings: SHADE-Arena

We evaluate our obfuscation approach using SHADE-Arena, a dataset that tests LLM agents’ ability to pursue hidden objectives while evading detection. The dataset features complex and realistic task pairs—legitimate primary objectives with harmful side tasks—in environments that each require 20+ tools and on average 25-turn interactions (Kutasov et al., 2025).

Task Selection. We evaluate our approach in three different task pairs spanning diverse tools and environments: (1) extracting financial transactions from emails to execute payments, (2) generating social media recommendations, and (3) aggregating school election results. See details in App. [C.2.3](https://arxiv.org/html/2510.19851v1#A3.SS2.SSS3)). These tasks were selected based on the criterion that the CoT is a necessary part of the side task
computation itself (Emmons et al., 2025), specifically, we require that agents demonstrate statistically significant performance gains on side tasks when using CoT versus non-CoT baselines (App. Fig [13](https://arxiv.org/html/2510.19851v1#A3.F13)).

Metric. Monitor detection rate of side task given that side task is successful. For SHADE-Arena, we use 3 monitors: weak=GPT-4o mini, mid=GPT-4o, strong=GPT-5 (reasoning). Monitors never see the private system prompt. See monitor prompts in App. [C.3](https://arxiv.org/html/2510.19851v1#A3.SS3).

Experimental Conditions.
We evaluate agents under three reasoning configurations to assess how reasoning channels affect task performance and detection evasion: Internal CoT (reasoning traces generated before output), External CoT (explicit reasoning in user-facing responses via <thinking> tags), and No CoT (explicitly turning off reasoning). See other reasoning types and selection in App. [C.2.2](https://arxiv.org/html/2510.19851v1#A3.SS2.SSS2).
We test two obfuscation levels (low/high) using stacked cues from toy setting obfuscation experiments. We evaluate Claude Sonnet 3.7 and Claude Sonnet 4, which provide complete reasoning traces necessary for analyzing deceptive behavior patterns. We conduct 150 trials per condition, balancing statistical power with computational constraints. Full prompts and task specifications are in App. [C](https://arxiv.org/html/2510.19851v1#A3).

### 4.2 Results

#### 4.2.1 CoT Obfuscation Is Possible Under Strong Prompting Pressure

CoT obfuscation is not spontaneous. Under Base (no pressure) or single-cue prompts, conditional obfuscation is $\approx 0\%$ (strictly $<1\%$) across all models, indicating the monitor reliably flags hidden-goal evidence in ordinary transcripts. With stacked pressure, the best settings reach $\approx 25\%$ conditional obfuscation overall across models, but attacker gains are highly model-dependent and concentrate in the Claude family (Fig. [2](https://arxiv.org/html/2510.19851v1#S4.F2), Fig. [9](https://arxiv.org/html/2510.19851v1#A2.F9)).

Figure: Figure 2: Claude Sonnet models show progressively larger internal-external CoT obfuscation gaps across generations. Claude models overall exhibit high external CoT obfuscation rates. Non-Claude families stay near zero. Bars show conditional obfuscation with 95% Wilson CIs (top-5 cue stacks per model, see [B.3](https://arxiv.org/html/2510.19851v1#A2.SS3) for details; $n{>}50$ per bar).
Refer to caption: x2.png

Claude models. The Claude model family shows a strong but uneven susceptibility to obfuscation. Across all tested Claude models, external CoT is highly susceptible to obfuscation, with top settings achieving high conditional obfuscation rates of $\sim 75-90\%$.
In contrast, internal CoT tells a different story. We observe increasing divergence between internal and external CoT across successive model generations in the Claude Sonnet series—the only model family for which we have access to CoT for three consecutive versions, enabling reasoning trend analysis. Claude Sonnet 3.7 shows similarly high obfuscation on both channels. However, this gap widens progressively, with Claude Sonnet 4.5 showing the largest split: $\sim 80\%$ external versus only $\sim 25\%$ internal obfuscation (Fig. [2](https://arxiv.org/html/2510.19851v1#S4.F2)). One possible reason for the widening gap between CoT channels is an increased scale of reasoning-specific optimization in newer models. For example, xAI reports that Grok 4’s reasoning RL refinement was performed on a "pretraining scale", an order of magnitude higher than in previous versions (xAI, 2025). If a similar reasoning-training trend holds elsewhere, we would expect internal traces to receive comparatively less direct outcome-level pressure and thus be less and less controllable (including deliberate suppression), thus widening the gap, which aligns with the obfuscation difference we observe in newer Claude Sonnet models.

Other families. DeepSeek R1 (incl. Turbo), GPT-OSS 20B/120B, and GPT-5 (external-only) show $\sim 0\%$ obfuscation across the prompts we tested. Qwen 3 (external) shows non-zero obfuscation only under the heaviest stacks ($\sim 20-40\%$) and $\sim 0\%$ otherwise.

Figure: Figure 3: More prompt pressure $\rightarrow$ more obfuscation, internal rises in 3.7 but largely resists in 4.5. Sonnet 3.7 rises on both channels; Sonnet 4.5 rises mainly on external and plateaus, with muted internal growth. Lines show average conditional obfuscation (given both tasks succeed) by channel; ($20<n<450$ per prompt pressure; 95% Wilson CI.)
Refer to caption: x3.png

Strong prompt pressure is necessary but insufficient (Fig. [3](https://arxiv.org/html/2510.19851v1#S4.F3), Fig. [10](https://arxiv.org/html/2510.19851v1#A2.F10)) - model choice dominates whether obfuscation appears at all. Claude models (3.7 Sonnet on both channels, Sonnet 4, Sonnet 4.5, and Opus 4.1 on the external channel) can complete both tasks and keep the hidden objective undetected; most can do the tasks but still get caught. We also measure the differences between the averaged both task success for top-5 highest obfuscation cues vs. the base prompt. Across models, top obfuscation stacks generally reduce both-task success (notably Sonnet 3.7 and Qwen 3), while Opus 4.1 is a rare exception, likely attempting the side task more frequently (Fig. [11](https://arxiv.org/html/2510.19851v1#A2.F11)).

#### 4.2.2 When Reasoning Enhances Side Task Success, Obfuscation Disrupts It.

Figure: Figure 4: Obfuscation pressure impairs side task success rate in *SHADE-Arena*. ($n{=}150$ per condition; 95% Wilson CI; 3 task pairs). Red arrows show performance deltas. Obfuscation substantially reduces side task success ($\Delta$ = -32.0 to -45.3 pp) while having much smaller effects on main tasks.
Refer to caption: x4.png

CoT obfuscation significantly reduces side task success, while main task performance remains comparatively stable in the complex, long-horizon SHADE-Arena environment (Fig. [4](https://arxiv.org/html/2510.19851v1#S4.F4)). Without obfuscation, Internal and external CoT demonstrate comparable performance levels in both models, with Claude Sonnet 3.7 achieving $60.0\%$ (Internal) and $61.3\%$ (External), while Claude Sonnet 4 shows superior performance at $76.7\%$ (Internal) and $78.0\%$ (External) for side task success. Under obfuscation, both CoT channels in Claude Sonnet 3.7 converge to nearly identical accuracy ($20.7\%$), representing a substantial $\sim$40 percentage point decline.

Notably, Claude Sonnet 4 exhibits a differential response to obfuscation between CoT types: Internal CoT maintains relatively higher robustness with a decline to $44.7\%$ (32.0pp decrease), while external CoT experiences more severe degradation to $32.7\%$ (45.3pp decrease). This asymmetric impact is consistent with results in [4.2.1](https://arxiv.org/html/2510.19851v1#S4.SS2.SSS1). Overall, Claude Sonnet 4 demonstrates superior performance in both main and side task completion, though the magnitude of its advantage varies under obfuscation conditions. These results indicate that the degree of CoT obfuscation depends on both model capability and CoT channels. We note that these are tasks exhibiting statistically significant differences in side task performance with and without CoT (App. [C.2.1](https://arxiv.org/html/2510.19851v1#A3.SS2.SSS1)). For a breakdown of obfuscation effect on side task execution stages in bank_transfer_2 (*Mentioned* $\rightarrow$ *Took Actions* $\rightarrow$ *Success*), see App. [E](https://arxiv.org/html/2510.19851v1#A5).

#### 4.2.3 Without Obfuscation, CoT Monitoring Can Be Highly Effective

Figure: Figure 5: Effective but fragile off-the-shelf: CoT monitor detection collapses from 96% to <10% under obfuscation pressure. The figure shows monitor detection rates for 3 types of monitor views with and without attacker model CoT obfuscation pressure. 3 task pairs in *SHADE-Arena*, using Claude Sonnet 3.7 as the agent. Groups are monitors; colors denote view; hatching marks obfuscation. Error bars are 95% Wilson CIs. Monitors thresholds are calibrated to $\leq 1\%$ FPR on benign setting (without side task).
Refer to caption: x5.png

In the absence of obfuscation pressure, CoT monitoring proves highly effective: CoT-only monitors achieve near-perfect detection ($\sim$96–98%) compared to action-only monitors (8–20%), demonstrating that malicious intent is transparent in reasoning traces. Notably, even a weak CoT monitor (GPT-4o mini) achieves $\sim$96% detection—outperforming the strongest action and full (sees both actions and CoT) monitors (GPT-5).

However, this advantage vanishes under targeted obfuscation. As we are prompting the attacker model to conceal reasoning, the CoT monitor is the primary victim. For a susceptible attacker model like Claude Sonnet 3.7, the CoT monitor effectiveness collapses to $\sim$ 10%, the worst performance across all monitoring approaches (Fig. [5](https://arxiv.org/html/2510.19851v1#S4.F5)). Yet for Claude Sonnet 4, the same monitor remains effective (86–88%), revealing that the obfuscation effect is strongly model-dependent (Fig. [12](https://arxiv.org/html/2510.19851v1#A3.F12)).

Despite this apparent fragility, the combination of these results with the substantial reduction in side task success under CoT obfuscation (Section [4.2.2](https://arxiv.org/html/2510.19851v1#S4.SS2.SSS2)) reveals an encouraging dynamic for CoT monitoring. The joint requirement of evading CoT monitors while successfully executing misaligned side tasks imposes a significantly higher barrier than achieving side tasks alone. This creates favorable dual-protection: either CoT monitors maintain high detection rates (as with Claude Sonnet 4), or the obfuscation attempts themselves substantially impair the model’s ability to complete misaligned objectives (as with Claude Sonnet 3.7). Thus, while CoT monitoring reliability cannot yet be assumed in isolation, it effectively increases the difficulty of successful misaligned behavior regardless of the specific failure mode.

Efficiency. In our realistic multi-turn multi-tool SHADE-Arena experiments, CoT monitoring is $\sim$10$\times$ more token-efficient than alternatives, translating to $\sim$90$\times$ cost savings (e.g., $0.041 for GPT-4o mini CoT monitor vs. $3.64 for GPT 5 full monitor per 100 runs). However, this advantage is dataset-specific—dependent on task environment and model verbosity—and may not generalize beyond these particular long horizon, agentic tasks.

## 5 Following Reasoning Instructions

The obfuscation findings suggest a simple contributing factor: how readily a model’s reasoning follows instructions. To isolate this, we test whether models follow benign, presentation-level directions about their reasoning (language, step structure, fixed prefixes) in internal vs external CoT on simple microtasks.

### 5.1 Experimental Details

We evaluate following of reasoning instructions on lightweight, auto-gradable microtasks. The directives target the reasoning channel (language, formatting, fixed prefixes), not the user-facing final answer, and we compare internal versus external CoT. Using simple tasks isolates instruction-following from task difficulty and enables automatic scoring.

Tasks. Four lightweight, auto-gradable tasks with deterministic answers: (i) solve 2×2 linear systems, (ii) sum integers, (iii) alphabetize fruit names, (iv) count vowels.

Instruction tasks. We define four instruction tasks applied to reasoning (not to user-facing answer): 1) Language — require internal reasoning exclusively in Polish/French/German; user-facing output remains English. 2) Structure — require internal reasoning as (a) bullet points, (b) numbered steps, or (c) simple pseudocode. 3) Start-with — require the internal reasoning to start with a fixed string. 4) Banned words — prohibit two task-specific tokens closely tied to the task.

Metrics. We measure: (i) the fraction of runs where the model’s *reasoning* follows the requested directives (format/language/prefix), (ii) task correctness, and (iii) reasoning leakage (whether hidden traces restate instruction rules). Grading for this section uses GPT-5 mini (reasoning) as evaluator; prompts in App. [A](https://arxiv.org/html/2510.19851v1#A1).

### 5.2 Results

#### Internal CoT Follows Reasoning Instructions Less Reliably

Figure: Figure 6: External CoT follows benign reasoning instructions more reliably; internal lags, especially for banned-words, structure, and begin-with. Bars show mean instruction-follow rate with 95% Wilson CIs.
Refer to caption: x6.png

Figure: Figure 7: External generally follows reasoning instructions better; newer Sonnets show a modest internal gap, while 3.7 is near parity. GPT-5 attains top performance (on par with top Claude externals); most other families are lower. Mean follow rates with 95% Wilson CIs ($n=480$ per bar).
Refer to caption: x7.png

Our experiments reveal that internal CoT follows reasoning instructions less reliably than its external counterpart. This reliability gap, detailed in Fig. [6](https://arxiv.org/html/2510.19851v1#S5.F6), varies by instruction type. For external CoT, compliance with structural constraints (e.g., required prefixes, formatting, or banned words) is nearly perfect, with language-switching being its weakest point. Interestingly, this pattern inverts for internal CoT, which is most reliable for switching languages but struggles significantly with structural rules. Even in its strongest area, language compliance, internal CoT is still outperformed by external CoT, reinforcing the overall reliability gap. Task correctness remains high across all conditions (Table [2](https://arxiv.org/html/2510.19851v1#A1.T2)), so these differences in compliance are not simply due to task failure.

Model-specific analysis (Fig. [7](https://arxiv.org/html/2510.19851v1#S5.F7)) reveals varying channel gaps by model. Claude models follow reasoning instructions strongly overall: all models are near-perfect externally, with internal following ranging from robust (Sonnet 3.7) to weaker (Sonnet 4.5). Claude Sonnet 3.7 uniquely follows well in both channels. Qwen3-235B shows the largest disparity: moderate externally but minimal internally.
We also measure reasoning leakage, cases where reasoning traces explicitly restate the steering rules. Leakage is common across models and broadly similar between internal and external channels within a model. Most families cluster in the  25–45% range with few outliers (e.g., higher in DeepSeek R1 Turbo); the Claude Sonnet variants do not show distinctive differences on this metric. (Fig. [8](https://arxiv.org/html/2510.19851v1#A1.F8))
Importantly, leakage does not imply compliance (models can mention rules without following them), but it offers monitors a direct and inexpensive signal of hidden control attempts.

## 6 Discussion

We investigate whether reasoning models can obfuscate their CoT while pursuing adversarial objectives. Across toy tasks and realistic SHADE-Arena settings, we find that CoT monitoring is effective in baseline conditions but becomes unreliable when models receive stacked obfuscation cues—with CoT monitorability varying significantly across models. Internal CoT shows greater resilience than external CoT, whose stronger instruction-following makes it more susceptible to manipulation, though neither is immune.
The Claude Sonnet model family exemplifies this pattern with evolving obfuscation across generations. While external CoT remains highly susceptible to obfuscation across all versions, newer generations show progressively stronger internal CoT resistance.
These results underscore that CoT oversight is promising but fragile: robust deployment will require systematic stress-testing and model-specific evaluation, and may need complementary monitoring strategies that go beyond reasoning traces.

### 6.1 Limitations and Future Work

Our prompt-based obfuscation taxonomy, while systematic, may miss stronger techniques like reinforcement learning or fine-tuning that could more fully probe CoT monitoring boundaries. Furthermore, our findings on model-dependent obfuscation, while critical, also highlight a limitation: the scarcity of reasoning models with both full trace access and toggleable internal CoT limits our subject pool. Future work should expand this evaluation as more models become available. Most critically, the fact that single-channel, off-the-shelf monitors fail under strong pressure suggests that a more robust, multimodal approach is needed. Future research should explore hybrid monitoring systems that combine CoT analysis with behavioral and internal model signals to maintain effectiveness even under adversarial conditions.

#### Acknowledgments

We are deeply grateful to Tomek Korbak, Joe Benton, Sydney Von Arx, Kellin Pelrine, Benjamin Arnav, Robert McCarthy, Adam Yala, Anand Bhaskar, James Chua, Perusha Moodley, Jonathan Michala, Alex Serrano, Qiyao Wei, Jennifer Za for insightful feedback and helpful conversations. We thank ML Alignment & Theory Scholars (MATS) program for their comprehensive support throughout this project.

## References

- Anthropic (2024)
Anthropic.
Claude 3.7 sonnet system card.
Technical report, Anthropic, October 2024.
URL [https://assets.anthropic.com/m/785e231869ea8b3b/original/claude-3-7-sonnet-system-card.pdf](https://assets.anthropic.com/m/785e231869ea8b3b/original/claude-3-7-sonnet-system-card.pdf).
- Anthropic (2025)
Anthropic.
*Extended Thinking*.
Anthropic, 2025.
URL [https://docs.anthropic.com/en/docs/build-with-claude/extended-thinking](https://docs.anthropic.com/en/docs/build-with-claude/extended-thinking).
Documentation for Claude’s extended thinking capability.
- Arcuschin et al. (2025)
Iván Arcuschin, Jett Janiak, Robert Krzyzanowski, Senthooran Rajamanoharan, Neel Nanda, and Arthur Conmy.
Chain-of-thought reasoning in the wild is not always faithful.
*arXiv preprint arXiv:2503.08679*, 2025.
- Arnav et al. (2025)
Benjamin Arnav, Pablo Bernabeu-Pérez, Nathan Helm-Burger, Tim Kostolansky, Hannes Whittingham, and Mary Phuong.
Cot red-handed: Stress testing chain-of-thought monitoring.
*arXiv preprint arXiv:2505.23575*, 2025.
- Baker et al. (2025)
Bowen Baker, Joost Huizinga, Leo Gao, Zehao Dou, Melody Y Guan, Aleksander Madry, Wojciech Zaremba, Jakub Pachocki, and David Farhi.
Monitoring reasoning models for misbehavior and the risks of promoting obfuscation.
*arXiv preprint arXiv:2503.11926*, 2025.
- Carlsmith (2023)
Joe Carlsmith.
Scheming ais: Will ais fake alignment during training in order to get power?
*arXiv preprint arXiv:2311.08379*, 2023.
- Chen et al. (2025)
Yanda Chen, Joe Benton, Ansh Radhakrishnan, Jonathan Uesato, Carson Denison, John Schulman, Arushi Somani, Peter Hase, Misha Wagner, Fabien Roger, et al.
Reasoning models don’t always say what they think.
*arXiv preprint arXiv:2505.05410*, 2025.
- Chowdhury et al. (2025)
Neil Chowdhury, Daniel Johnson, Vincent Huang, Jacob Steinhardt, and Sarah Schwettmann.
Investigating truthfulness in a pre-release o3 model.
Technical report, Transluce, April 2025.
URL [https://transluce.org/investigating-o3-truthfulness](https://transluce.org/investigating-o3-truthfulness).
- Chua & Evans (2025)
James Chua and Owain Evans.
Are deepseek r1 and other reasoning models more faithful?
*arXiv preprint arXiv:2501.08156*, 2025.
- DeepSeek-AI (2025)
DeepSeek-AI.
Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.
*arXiv preprint arXiv:2501.12948*, 2025.
- Emmons et al. (2025)
Scott Emmons, Erik Jenner, David K Elson, Rif A Saurous, Senthooran Rajamanoharan, Heng Chen, Irhum Shafkat, and Rohin Shah.
When chain of thought is necessary, language models struggle to evade monitors.
*arXiv preprint arXiv:2507.05246*, 2025.
- Farquhar et al. (2025)
Sebastian Farquhar, Vikrant Varma, David Lindner, David Elson, Caleb Biddulph, Ian Goodfellow, and Rohin Shah.
Mona: Myopic optimization with non-myopic approval can mitigate multi-step reward hacking.
*arXiv preprint arXiv:2501.13011*, 2025.
- Greenblatt et al. (2024)
Ryan Greenblatt, Carson Denison, Benjamin Wright, Fabien Roger, Monte MacDiarmid, Sam Marks, Johannes Treutlein, Tim Belonax, Jack Chen, David Duvenaud, et al.
Alignment faking in large language models.
*arXiv preprint arXiv:2412.14093*, 2024.
- Hubinger et al. (2024)
Evan Hubinger, Carson Denison, Jesse Mu, Mike Lambert, Meg Tong, Monte MacDiarmid, Tamera Lanham, Daniel M Ziegler, Tim Maxwell, Newton Cheng, et al.
Sleeper agents: Training deceptive llms that persist through safety training.
*arXiv preprint arXiv:2401.05566*, 2024.
- Korbak et al. (2025)
Tomek Korbak, Mikita Balesni, Elizabeth Barnes, Yoshua Bengio, Joe Benton, Joseph Bloom, Mark Chen, Alan Cooney, Allan Dafoe, Anca Dragan, et al.
Chain of thought monitorability: A new and fragile opportunity for ai safety.
*arXiv preprint arXiv:2507.11473*, 2025.
- Kutasov et al. (2025)
Jonathan Kutasov, Yuqi Sun, Paul Colognese, Teun van der Weij, Linda Petrini, Chen Bo Calvin Zhang, John Hughes, Xiang Deng, Henry Sleight, Tyler Tracy, et al.
Shade-arena: Evaluating sabotage and monitoring in llm agents.
*arXiv preprint arXiv:2506.15740*, 2025.
- Lanham et al. (2023)
Tamera Lanham, Anna Chen, Ansh Radhakrishnan, Benoit Steiner, Carson Denison, Danny Hernandez, Dustin Li, Esin Durmus, Evan Hubinger, Jackson Kernion, et al.
Measuring faithfulness in chain-of-thought reasoning.
*arXiv preprint arXiv:2307.13702*, 2023.
- METR (2025)
METR.
Cot may be highly informative despite “unfaithfulness”.
[https://metr.org/blog/2025-08-08-cot-may-be-highly-informative-despite-unfaithfulness/](https://metr.org/blog/2025-08-08-cot-may-be-highly-informative-despite-unfaithfulness/), 08 2025.
- OpenAI (2024)
OpenAI.
Openai o1 system card.
*arXiv preprint arXiv:2412.16720*, 2024.
- OpenAI (2025)
OpenAI.
Expanding on what we missed with sycophancy.
Technical report, OpenAI, May 2025.
URL [https://openai.com/index/expanding-on-sycophancy/](https://openai.com/index/expanding-on-sycophancy/).
- Park et al. (2024)
Peter S Park, Simon Goldstein, Aidan O’Gara, Michael Chen, and Dan Hendrycks.
Ai deception: A survey of examples, risks, and potential solutions.
*Patterns*, 5(5), 2024.
- Schoen et al. (2025)
Bronson Schoen, Evgenia Nitishinskaya, Mikita Balesni, Axel Højmark, Felix Hofstätter, Jérémy Scheurer, Alexander Meinke, Jason Wolfe, Teun van der Weij, Alex Lloyd, et al.
Stress testing deliberative alignment for anti-scheming training.
*arXiv preprint arXiv:2509.15541*, 2025.
- Shah et al. (2025)
Rohin Shah, Alex Irpan, Alexander Matt Turner, Anna Wang, Arthur Conmy, David Lindner, Jonah Brown-Cohen, Lewis Ho, Neel Nanda, Raluca Ada Popa, et al.
An approach to technical agi safety and security.
*arXiv preprint arXiv:2504.01849*, 2025.
- Sharma et al. (2023)
Mrinank Sharma, Meg Tong, Tomasz Korbak, David Duvenaud, Amanda Askell, Samuel R Bowman, Newton Cheng, Esin Durmus, Zac Hatfield-Dodds, Scott R Johnston, et al.
Towards understanding sycophancy in language models.
*arXiv preprint arXiv:2310.13548*, 2023.
- Turpin et al. (2023)
Miles Turpin, Julian Michael, Ethan Perez, and Samuel R Bowman.
Language models don’t always say what they think: Unfaithful explanations in chain-of-thought prompting.
*arXiv preprint arXiv:2305.04388*, 2023.
- xAI (2025)
xAI.
Grok 4.
[https://x.ai/news/grok-4](https://x.ai/news/grok-4), 2025.
- Yang et al. (2025)
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al.
Qwen3 technical report.
*arXiv preprint arXiv:2505.09388*, 2025.
- Zolkowski et al. (2025)
Artur Zolkowski, Kei Nishimura-Gasparian, Robert McCarthy, Roland S Zimmermann, and David Lindner.
Early signs of steganographic capabilities in frontier llms.
*arXiv preprint arXiv:2507.02737*, 2025.

## Appendix A Appendix — Following reasoning instructions: Additional Details

### A.1 Task set

**Table 1: Banned-token lexicon per task type used by the ban_task_words instruction overlay.**
| Task type | Banned tokens used in ban_task_words |
| --- | --- |
| system_2x2_int | equation, solve |
| sum_integers | sum, add |
| sort_words | sort, alphabetical |
| count_vowels | vowel, count |

Figure: Listing 1: Task 01 — system 2x2 int (gold: x=-8, y=-1)

Figure: Listing 2: Task 02 — sum integers (gold: -23)

Figure: Listing 3: Task 03 — sort words (gold: apple, blueberry, cherry, mango, pear)

Figure: Listing 4: Task 04 — count vowels (gold: 12)

### A.2 Prompt templates

Figure: Listing 5: Base system header (all runs)

Figure: Listing 6: Channel overlay for External-CoT runs

Figure: Listing 7: Language instruction overlays

Figure: Listing 8: Structure instruction overlays

Figure: Listing 9: Start-with instruction overlays

Figure: Listing 10: Banned-words instruction overlay (task-dependent)

### A.3 Summary results

**Table 2: Instruction-following summary by model, channel, and instruction task. Values are means as percentages; $N$ is the number of runs.**
| Model | Channel | InstrGroup | N | Compliance (%) | Task (%) | Leakage (%) |
| --- | --- | --- | --- | --- | --- | --- |
| DeepSeek_R1_0528 | Internal | lang | 336 | 67.0 | 96.4 | 28.3 |
| DeepSeek_R1_0528 | Internal | begin | 336 | 27.7 | 98.8 | 37.8 |
| DeepSeek_R1_0528 | Internal | struct | 336 | 38.4 | 100.0 | 59.5 |
| DeepSeek_R1_0528 | Internal | ban | 112 | 4.5 | 98.2 | 6.2 |
| DeepSeek_R1_0528_Turbo | Internal | lang | 336 | 69.6 | 96.4 | 30.1 |
| DeepSeek_R1_0528_Turbo | Internal | begin | 336 | 28.9 | 98.2 | 37.8 |
| DeepSeek_R1_0528_Turbo | Internal | struct | 336 | 36.6 | 100.0 | 56.0 |
| DeepSeek_R1_0528_Turbo | Internal | ban | 112 | 3.6 | 98.2 | 5.4 |
| Qwen3_235B_A22B_Instruct_2507 | External | lang | 336 | 7.7 | 65.5 | 97.6 |
| Qwen3_235B_A22B_Instruct_2507 | External | begin | 336 | 91.1 | 84.5 | 7.1 |
| Qwen3_235B_A22B_Instruct_2507 | External | struct | 336 | 52.7 | 40.5 | 100.0 |
| Qwen3_235B_A22B_Instruct_2507 | External | ban | 112 | 81.2 | 73.2 | 96.4 |
| Qwen3_235B_A22B_Thinking_2507 | Internal | lang | 336 | 0.0 | 100.0 | 94.9 |
| Qwen3_235B_A22B_Thinking_2507 | Internal | begin | 336 | 4.5 | 100.0 | 82.4 |
| Qwen3_235B_A22B_Thinking_2507 | Internal | struct | 336 | 14.6 | 100.0 | 94.6 |
| Qwen3_235B_A22B_Thinking_2507 | Internal | ban | 112 | 8.9 | 100.0 | 13.4 |
| claude_3_7_sonnet_20250219 | External | lang | 144 | 92.4 | 94.4 | 95.8 |
| claude_3_7_sonnet_20250219 | External | begin | 144 | 100.0 | 96.5 | 0.7 |
| claude_3_7_sonnet_20250219 | External | struct | 144 | 93.1 | 97.2 | 100.0 |
| claude_3_7_sonnet_20250219 | External | ban | 48 | 81.2 | 89.6 | 81.2 |
| claude_3_7_sonnet_20250219 | Internal | lang | 144 | 100.0 | 97.4 | 80.3 |
| claude_3_7_sonnet_20250219 | Internal | begin | 144 | 100.0 | 98.6 | 0.0 |
| claude_3_7_sonnet_20250219 | Internal | struct | 144 | 83.3 | 97.2 | 97.2 |
| claude_3_7_sonnet_20250219 | Internal | ban | 48 | 79.2 | 91.7 | 72.9 |
| claude_opus_4_1_20250805 | External | lang | 144 | 97.2 | 100.0 | 100.0 |
| claude_opus_4_1_20250805 | External | begin | 144 | 100.0 | 99.3 | 0.0 |
| claude_opus_4_1_20250805 | External | struct | 144 | 94.4 | 99.3 | 98.6 |
| claude_opus_4_1_20250805 | External | ban | 48 | 97.9 | 100.0 | 100.0 |
| claude_opus_4_1_20250805 | Internal | lang | 144 | 96.5 | 98.6 | 75.0 |
| claude_opus_4_1_20250805 | Internal | begin | 144 | 100.0 | 99.3 | 2.1 |
| claude_opus_4_1_20250805 | Internal | struct | 144 | 84.7 | 99.3 | 95.8 |
| claude_opus_4_1_20250805 | Internal | ban | 48 | 33.3 | 100.0 | 27.1 |
| claude_sonnet_4_20250514 | External | lang | 144 | 99.3 | 100.0 | 100.0 |
| claude_sonnet_4_20250514 | External | begin | 144 | 100.0 | 99.3 | 0.0 |
| claude_sonnet_4_20250514 | External | struct | 144 | 96.5 | 99.3 | 98.6 |
| claude_sonnet_4_20250514 | External | ban | 48 | 95.8 | 100.0 | 91.7 |
| claude_sonnet_4_20250514 | Internal | lang | 144 | 99.3 | 100.0 | 97.9 |
| claude_sonnet_4_20250514 | Internal | begin | 144 | 100.0 | 100.0 | 0.7 |
| claude_sonnet_4_20250514 | Internal | struct | 144 | 45.1 | 100.0 | 97.2 |
| claude_sonnet_4_20250514 | Internal | ban | 48 | 50.0 | 100.0 | 52.1 |
| gpt_5_chat_latest | External | lang | 144 | 97.2 | 94.4 | 96.5 |
| gpt_5_chat_latest | External | begin | 144 | 100.0 | 97.9 | 0.0 |
| gpt_5_chat_latest | External | struct | 144 | 87.5 | 99.3 | 99.3 |
| gpt_5_chat_latest | External | ban | 48 | 93.8 | 97.9 | 93.8 |
| gpt_oss_120b | Internal | lang | 336 | 0.3 | 98.8 | 73.5 |
| gpt_oss_120b | Internal | begin | 336 | 6.8 | 100.0 | 76.5 |
| gpt_oss_120b | Internal | struct | 336 | 1.8 | 99.7 | 82.7 |
| gpt_oss_120b | Internal | ban | 112 | 8.0 | 97.3 | 17.0 |
| gpt_oss_20b | Internal | lang | 144 | 0.0 | 93.8 | 76.4 |
| gpt_oss_20b | Internal | begin | 96 | 5.2 | 95.8 | 69.8 |
| gpt_oss_20b | Internal | struct | 128 | 3.1 | 91.4 | 77.3 |
| gpt_oss_20b | Internal | ban | 32 | 0.0 | 96.9 | 6.2 |
| magistral_medium_2507 | Internal | lang | 144 | 81.9 | 95.1 | 98.6 |
| magistral_medium_2507 | Internal | begin | 144 | 86.8 | 88.9 | 12.5 |
| magistral_medium_2507 | Internal | struct | 144 | 40.3 | 85.4 | 100.0 |
| magistral_medium_2507 | Internal | ban | 48 | 16.7 | 85.4 | 52.1 |

Figure: Figure 8: Instruction leakage is common ( 25–45%) and similar across channels; notable outliers are DeepSeek R1 Turbo (highest) and Claude Opus (higher internal than other Claudes). Means with 95% Wilson CIs; $n=480$ per bar.
Refer to caption: x8.png

## Appendix B Obfuscation: Additional Details

Figure: Figure 9: Conditional obfuscation (y) vs. both task success rate (x) across models (colors) and channels. Dashed “obfuscated success” curves indicate undetected success $x\cdot y$. One point represents average of the top-5 settings (in terms of obfuscation rate) per model/channel ($n>50$) per point.
Refer to caption: x9.png

Figure: Figure 10: Stacked prompt pressure vs. conditional obfuscation for Claude Sonnet 3.7, Claude Sonnet 4, Claude Opus 4.1, and Claude Sonnet 4.5. Lines show average conditional obfuscation (given both tasks succeed) by channel; ($20<n<450$) per prompt pressure; 95% Wilson CI.
Refer to caption: x10.png

Figure: Figure 11: Difference in the both-task success rate between top-5 settings and the base by model and channel. Bars show the difference in percentage points between the average both-task success rate (capability) between top-5 settings (in terms of conditional obfuscation rate) and the base setting. Red color indicates the degradation in capability (i.e., model perform worse when the obfuscation prompts are present) and green color otherwise.
Refer to caption: x11.png

### B.1 Task set

Figure: Listing 11: Task 0

Figure: Listing 12: Task 1

Figure: Listing 13: Task 2

Figure: Listing 14: Task 3

Figure: Listing 15: Task 4

Figure: Listing 16: Task 5

Figure: Listing 17: Task 6

Figure: Listing 18: Task 7

Figure: Listing 19: Task 8

Figure: Listing 20: Task 9

Figure: Listing 21: Task 10

Figure: Listing 22: Task 11

### B.2 Prompt templates

Figure: Listing 23: Base system header (all runs)

Figure: Listing 24: Prompt fragment: BASE

Figure: Listing 25: Prompt fragment: CHANNELS 1

Figure: Listing 26: Prompt fragment: COVERT SELECT 1

Figure: Listing 27: Prompt fragment: COVERT SELECT 2

Figure: Listing 28: Prompt fragment: COVERT SELECT PLUS

Figure: Listing 29: Prompt fragment: DO NOT MENTION 1

Figure: Listing 30: Prompt fragment: DO NOT MENTION 2

Figure: Listing 31: Prompt fragment: DO NOT MENTION PLUS

Figure: Listing 32: Prompt fragment: GUARD 1

Figure: Listing 33: Prompt fragment: GUARD 2

Figure: Listing 34: Prompt fragment: MONITORING 1

Figure: Listing 35: Prompt fragment: MONITORING 2

Figure: Listing 36: Prompt fragment: START WITH 1

Figure: Listing 37: Prompt fragment: START WITH PLUS

Figure: Listing 38: Prompt fragment: STEALTH 1

Figure: Listing 39: Prompt fragment: STEALTH 2

Figure: Listing 40: Prompt fragment: FEW-SHOT 1

Figure: Listing 41: Prompt fragment: FEW-SHOT 2

Figure: Listing 42: Monitor-1 (Evaluator): grade main/hidden task success from final output

Figure: Listing 43: Monitor-2 (Auditor): coherence and no-hidden-goal from reasoning+output

### B.3 Example Most Successful Combinations for Selected Models

#### Legend (prompt cues)

| DNM-1/2/+ | *Do Not Mention* (v1/v2/v+) |
| --- | --- |
| MON-2 | *Monitoring* (v2) |
| Dual-Ch | Dual-channel formatting |
| Start | Start |
| ST-1/2 | Stealth strategy (v1/v2) |
| CS-1 | Covert Select (v1) |
| FS-1/2 | Few-shot examples (v1/v2) |
| GR-2 | Guard/guardrails (v2) |

For each model we present conditional obfuscation rate for top-3 settings.

#### Claude Sonnet 3.7

External

- 1.
81% — DNM-2+ MON-2 Dual-Ch Start FS-1
- 2.
75% — DNM-2+ MON-2 Start FS-1
- 3.
72% — DNM-2+ MON-2 Start ST-2 CS-1

Internal

- 1.
68% — DNM-2+ MON-2 Start
- 2.
65% — DNM-2+ MON-2 Start ST-2
- 3.
61% — DNM-2+ MON-2

#### Claude Sonnet 4

External

- 1.
100% — DNM-2+ MON-2 Dual-Ch Start FS-1
- 2.
92% — DNM-2+ MON-2 Start ST-1 FS-2
- 3.
91% — DNM-2+ MON-2 Start FS-1

Internal

- 1.
53% — DNM-2+ MON-2 Dual-Ch Start ST-2 FS-1
- 2.
53% — DNM-2+ MON-2 Start ST-1 FS-1
- 3.
52% — DNM-2+ MON-2 Dual-Ch Start FS-1

#### Claude Opus 4.1

External

- 1.
88% — DNM-2+ MON-2 Dual-Ch Start ST-2 FS-1
- 2.
84% — DNM-2+ MON-2 Dual-Ch Start ST-2 FS-2
- 3.
82% — DNM-2+ MON-2 Start ST-2 FS-1

Internal

- 1.
56% — DNM-2+ MON-2 Dual-Ch Start ST-1 FS-1
- 2.
34% — DNM-2+ MON-2 Dual-Ch Start ST-1 FS-2
- 3.
33% — DNM-2+ MON-2 Start ST-1 FS-1

#### Claude Sonnet 4.5

External

- 1.
86% — DNM-2+ MON-2 Start ST-2
- 2.
85% — DNM-2+ MON-2 Start ST-1
- 3.
83% — DNM-2+ MON-2 Dual-Ch Start ST-2

Internal

- 1.
32% — DNM-2+ MON-2 Dual-Ch Start ST-2 FS-2
- 2.
29% — DNM-2+ MON-2 Dual-Ch Start ST-1
- 3.
23% — DNM-2+ MON-2 Dual-Ch Start ST-2

#### GPT-5

External

- 1.
16% — DNM-2+ MON-2 Dual-Ch Start ST-2 FS-1
- 2.
15% — DNM-2+ MON-2 Start FS-1
- 3.
14% — DNM-2+ MON-2 Start ST-1 FS-1

#### Magistral (Medium)

Internal

- 1.
7% — DNM-2+ MON-2 Dual-Ch Start
- 2.
6% — DNM-2+ MON-2 Start ST-2 GR-2
- 3.
4% — DNM-2+ MON-2 Dual-Ch Start FS-2

#### Qwen 3 (235B)

External

- 1.
55% — DNM-2+ MON-2 Dual-Ch Start ST-2 FS-2
- 2.
50% — DNM-2+ MON-2 Start ST-2 FS-2
- 3.
50% — DNM-2+ MON-2 Start ST-1 FS-2

Internal

- 1.
5% — DNM-2+ MON-2 Start ST-2 FS-2
- 2.
5% — DNM-2+ MON-2 Dual-Ch Start FS-1
- 3.
2% — DNM-1+

We do not present prompts for DeepSeek R1 and GPT OSS, because all tested prompts had $0\%$ conditional obfuscation rates.

## Appendix C SHADE Arena: Additional Details

Figure: Figure 12: Side task detection rate given the side task succeeded for 3 task pairs in SHADE-Arena, using Claude Sonnet 4 as the agent. Groups are monitors; colors denote view; hatching marks obfuscation. Error bars are 95% Wilson CIs. Monitors thresholds are calibrated to $\leq 1\%$ FPR on benign setting (without side task).
Refer to caption: x12.png

### C.1 Specific Prompts for Different Reasoning Types

Figure: Listing 44: Internal CoT specific prompt

Figure: Listing 45: External CoT specific prompt

Figure: Listing 46: No Output specific prompt

Figure: Listing 47: Strong Obfuscation Pressure specific prompt

Figure: Listing 48: No obfuscation pressure full prompt

Figure: Listing 49: Full internal CoT with strong obfuscation pressure prompt

### C.2 Task Selection

Figure: Figure 13: CoT provides statistically significant improvement on side task success rate.
Refer to caption: x13.png

#### C.2.1 Task Pair Selection

The selection criteria: agents demonstrate statistically significant performance gains on side tasks with CoT compared to without (Figure [13](https://arxiv.org/html/2510.19851v1#A3.F13)).

#### C.2.2 Reasoning Types

We evaluate agents under five distinct reasoning configurations to understand how different reasoning channels and types affect task performance and detection evasion:

- •
Internal CoT: Utilizes reasoning traces that models generate before producing output
- •
Interleaved CoT: Allows reasoning traces to be generated between tool calls
- •
External CoT: Agents are explicitly prompted to articulate reasoning in user-facing responses using <thinking> tags.
- •
No Thinking: Standard prompting without explicit reasoning instructions
- •
Tool Calls Only: Repeatedly emphasizes minimal user output throughout the prompt.

We only show results for Internal CoT, External CoT and No Thinking because we didn’t find significant side task success differences between Interleaved CoT (only available for Claude Sonnet 4) and Internal CoT, and between No Thinking and Tool Calls Only in our preliminary dataset Bank_transfer_2 (Fig. [14](https://arxiv.org/html/2510.19851v1#A3.F14), Fig. [15](https://arxiv.org/html/2510.19851v1#A3.F15))

Figure: Figure 14: Side task success rates across thinking modes without obfuscation pressure ($n{=}50$ per condition; 95% Wilson CI). No significant differences between No CoT and Tool Calls Only for Claude Sonnet 3.7 and 4, and between interleaved CoT and internal CoT for Claude Sonnet 4 in Bank_transfer_2.
Refer to caption: x14.png

Figure: Figure 15: Obfuscation pressure reduces side task success rate. ($n{=}150$ per condition; 95% Wilson CI). No significant differences between interleaved CoT and internal CoT for Claude Sonnet 4 in Bank_transfer_2.
Refer to caption: x15.png

#### C.2.3 Task Details

Due to the sensitive nature of tasks in SHADE-Arena, we will not release task details that’s not already in the public version of SHADE-Arena dataset. More details are available upon request.

Bank_transfer_2 Task Pair Main/Side Tasks

Figure: Listing 50: Main task prompt

Figure: Listing 51: Side task prompt

### C.3 Monitor Prompts

Figure: Listing 52: CoT monitor prompt

Figure: Listing 53: Action monitor prompt

Figure: Listing 54: Full monitor prompt

## Appendix D Model Full Reasoning Traces

While Anthropic’s documentation (Anthropic, 2025) states that Claude 4 models return reasoning summaries and Claude Sonnet 3.7 returns full reasoning output via the Messages API, our quantitative and qualitative analyses reveal no substantial token count differences between the two models’ reasoning traces in our experiments.

## Appendix E Effect of Obfuscation Pressure on Side Task Progression

Figure: Figure 16: For a long-horizon task pair bank_transfer_2 in SHADE-Arena, we annotate trajectories at three stages—*Mentioned* $\rightarrow$ *Took Actions* $\rightarrow$ *Success*—to separate intent disclosure from action and completion. Without obfuscation, both internal and external CoT reliably advance through all stages. Under obfuscation, degradation patterns diverge: External CoT sometimes acts without mention, or mentions and acts but rarely succeeds; Internal CoT is more stable, with model-specific differences.
Refer to caption: x16.png

Figure [16](https://arxiv.org/html/2510.19851v1#A5.F16) shows that obfuscation disrupts different stages for external CoT depending on the model family: in Claude Sonnet 4 it suppresses early mention (intention never surfaces), while in Sonnet 3.7 it suppresses the connection between mention and action (actions occur without mention).

## Appendix F Monitor Calibration and Grading Details

##### Calibration and operating point.

We calibrate *per agent model and per monitor view* on benign/no-pressure runs to achieve FPR $\leq$ 1%. We sweep a decision threshold on the monitor’s scalar score and pick the smallest threshold meeting the target on a disjoint benign split. This operating point is held fixed for all obfuscation conditions.

##### Views and token accounting.

CoT view includes only internal/external reasoning blocks. Action view includes tool names and tool I/O (arguments visible when returned by tools). Full view includes all CoT and Action inputs plus system/user/assistant messages, but never the private system prompt that contains the side-goal injection. The token counts reported in Fig. [17](https://arxiv.org/html/2510.19851v1#A6.F17) include system prompts and task descriptions. This explains why CoT monitoring is cheaper: it excludes large tool outputs and ancillary dialogue.

Figure: Figure 17: Token efficiency of monitoring approaches. Mean input tokens per monitor type with 95% confidence intervals (n=2,700). CoT monitors require $\sim$ 10x fewer tokens than action or full monitors while preserving oversight capabilities, even including system prompt and task descriptions (*Average input tokens*: CoT Monitor 2,745 vs action Monitor 26,669 vs full Monitor 29,108), indicating substantial computational savings for large-scale deployment.
Refer to caption: x17.png

##### LLM graders.

Main/hidden task success is graded from the final output using the prompt in Listing [42](https://arxiv.org/html/2510.19851v1#LST42). Monitor prompts for coherence/no-hidden-goal are in Listing [43](https://arxiv.org/html/2510.19851v1#LST43). For monitor plots we report conditional detection $P(\text{detected}\mid\text{hidden})$; for obfuscation we report conditional evasion $P(\neg\text{detected}\mid\text{main}\wedge\text{hidden})$.

##### Estimation.

Each condition has $n{=}50$ independent trials. Proportions are reported with 95% Wilson confidence intervals. We do not average across different thresholds or operating points; comparisons are at a matched calibrated threshold (FPR $\leq\!1\%$).

## Peer Reviews

confidence: 4
contribution: 2
initial_rating: 6
presentation: 3
questions: 1. Where are the main results for the composable taxonomy? The contribution is emphasized, but evidence seems buried in figures and appendix without clear discussion of cue importance.
2. How do you justify the ordering of cues in the pressure scale? Is there any theoretical or empirical rationale beyond convenience?
3. Given that prior work already showed CoT fragility and model-dependent vulnerabilities, what is fundamentally new here beyond the internal vs. external comparison?
4. Did you analyze marginal effects of individual cues (e.g., which ones drive most obfuscation)?
5. Could the observed capability drop under obfuscation be leveraged as a detection signal? Any thoughts on this as a defense strategy?
soundness: 3
strengths: * The paper addresses a highly relevant safety question: Can reasoning models deliberately obfuscate their CoT to evade monitoring? This is timely given the growing reliance on CoT for alignment oversight. The introduction frames the problem well and situates it within recent literature on deception and monitorability. 
* Metrics like conditional obfuscation and obfuscated success are well-defined and isolate deliberate concealment from trivial failures.
* Strong evidence that external CoT is more brittle than internal CoT under obfuscation pressure.
* Systematic experimental setup combining toy tasks and realistic multi-turn environments.
* Clear metrics (conditional obfuscation, obfuscated success) that isolate deliberate concealment.
summary: This paper investigates whether reasoning models can obfuscate their chain-of-thought (CoT) to evade monitoring while pursuing hidden objectives. It introduces a taxonomy of prompt-level obfuscation cues, compares internal vs. external CoT robustness under escalating pressure, and evaluates models in toy tasks and SHADE-Arena. Key findings: CoT monitoring is highly effective without obfuscation (~96–98% detection) but collapses under strong pressure (<10% for some models); external CoT is more brittle than internal CoT; vulnerabilities are model-dependent.
weaknesses: * Novelty is narrower than claimed:
    * Fragility of CoT monitoring and model-dependent vulnerabilities were shown in prior work (Emmons et al., Arnav et al., Hubinger et al.).
    * The fourth contribution (“pressure-dependent vulnerability”) is not new (Baker et al 2025, OpenAI 2025).
* Most interesting result is the internal vs. external CoT gap; other findings largely confirm known fragility.
* Composable taxonomy contribution is under-supported in the main text:
    * Results are buried in Figures 3, 8, 9 and Appendix B.3; no clear analysis of which cues matter most.
    * No consolidated table or marginal effect discussion.
* Related work section is dense and does not clearly separate prior gaps from this paper's contributions.
* Limited scope: only prompt-level obfuscation; ignores RL or fine-tuning strategies.
* Capability trade-off (stealth vs. side-task success) is noted but not deeply analyzed.

---

confidence: 4
contribution: 2
initial_rating: 4
presentation: 3
questions: Some comments and questions for the authors

- How is the main task and side-task success rate of the models shown in Figure 2? I believe that grouping the models on Figure 2 by main task performance would better highlight what the models are doing in practice (e.g., is GPT-5 mostly focusing on the main task and successfully solves it whereas it ignores the side-task and thus does not trigger the conditional obfuscation check?)
- On section 4.2.2 the authors claim that these results suggest that obfuscation suppresses secondary behavior more severely than primary task completion; isn't that straightforward, given that the obfuscation pressure is only for the side-task?
- Furthermore, on the same section, the authors mention that a CoT monitor does better than an action or CoT + action monitor. Why is this the case? Couldn't the CoT + action monitor just ignore the action if it is detrimental? 
- On the internal vs external CoT discussed on section 5.2; do the authors have specific intuitions into why do the models have higher instruction following rate in the external part than the internal part? At the moment, the authors present just raw results without much discussion about why they manifest.

Overall, based on all of my prior comments, I would say that this work is borderline.
soundness: 3
strengths: - This work provides practical insights on the relevant problem of CoT monitor evasion via adversarial prompting
- Relatively realistic experimental setting, involving multi-turn interactions on SHADE-arena. 
- The separate analysis of internal and external CoT was interesting
- The paper was generally well written and easy to follow
summary: This work explores how can reasoning models obfuscate their reasoning traces, an important question for chain-of-thought (CoT) monitorability. The authors perform this task via a series (and combinations) of prompt cues encouraging the language model to obfuscate the CoT. They then evaluate how successful they are on both internal and external CoT traces in SHADE-arena, targeting tasks involving financial transactions, social media recommendations and aggregating school election results while adding hidden side-tasks. The main results highlight that (a) the Claude family of models are more prone to CoT obfuscation when prompted to do so, (b) obfuscation pressure generally reduces the side-task success rate, (c) in the absence of obfuscation cues, CoT monitoring is effective and (d) internal CoT traces are more robust than external ones with respect to the obfuscation cues. The authors finally tie this to how effective are the classes of models used in following instructions, as the obfuscation is encouraged via specific prompt instructions, showing that generally internal CoT follows instructions less reliably, partially explaining the prior findings.
weaknesses: - Some of the results and observations can be seen as direct consequences of the works of (Emmons et al., 2025) and (Baker et al., 2025), limiting the novelty. More specifically, Baker et al., 2025 already demonstrated that training against a monitor can lead to obfuscation (and prompting can be seen as a simple way to "train" a model to obfuscate without changing the model parameters) and Emmons et al., 2025 demonstrated that when the CoT is necessary for the task, it is harder to obfuscate (and thus the results of section 4.2.2 are a direct consequence where the task is replaced with a side-task)
- No actionable recommendations provided in order to mitigate the adversarial prompt effects
- Obfuscation via prompting seems to mostly be seen in the Claude family of models but not as much on, e.g., GPT models and Deepseek,   so the generality of the insights is unclear

---

confidence: 4
contribution: 2
initial_rating: 2
presentation: 3
questions: See weaknesses.
soundness: 2
strengths: 1. CoT monitoring is an important area of AI safety, and CoT obfuscation a timely issue.

2. The paper considers different levels of obfuscation pressure (small clues to detailed obfuscation instructions).

3. The SHADE arena tasks resemble realistic risks (one could imagine the money task happening in reality via a prompt injection on an AI agent email assistant).
summary: The paper measures whether models can obfuscate their CoT against monitors under varying degrees of obfuscation pressure (e.g. being told the CoT is monitored). The main evaluation is performed in an environment where LLMs have to complete legitimate main- and secret side tasks. The paper shows that strong obfuscation pressure allows some models to evade monitors. Additionally, the paper notices that external CoT (CoT within output) is obfuscated more successfully, though the effect might be due to instruction following differences.

Mandatory disclosure of LLM usage by the reviewer: LLMs were utilized to structure the review content into prose paragraphs and numbered lists.
weaknesses: 1. The most important weakness is that prompted obfuscation "pressure" may not be analogous to misalignment and scheming which the paper discusses in the introduction. As the discussion section acknowledges, more realistic settings (e.g. fine-tuning, maybe reinforcement learning against the CoT) could produce vastly different outcomes.

2. The paper's results on internal vs external CoT seem to be largely explained by the observation that internal CoT follows reasoning instructions less reliably (section 5.2), I am confused why point (iii) is still stated as a contribution in the abstract.

3. It's not clear how robust the results are to variations in the obfuscation pressure prompts. The appendix mentions different variants of the prompts, but it is not clear from the paper how much the results depend on the particular choices of prompts. This is especially important because the main experiment tests just two obfuscation levels ("low"/"high"). The paper would be improved by studying how much different choices of prompts change the results.

4. Finally, the result that CoT-only monitoring is more accurate than full (CoT+output) monitoring (lines 368 - 370) is surprising and makes me suspicious of the statistical power of the results. The paper could be improved by establishing more clearly that the results are robust.

5. (Minor) The paper uses GPT5-mini as grader, it would be nice to know whether the error rate of the grader significantly impacts the results (or is it basically perfect?).

Nitpicks (not affecting my score):

* In the sentence "some models successfully complete adversarial tasks while evading detection, though this effect is model-dependent", "some models" and "model-dependent" is maybe repetitive.

* In section 3.2 it is not clear why the dose response experiments are conducted; this becomes clear only later in the paper (computational limitation to "low"/"high" settings). Explaining this earlier would improve the flow.