Title: CoT Red-Handed: Stress Testing Chain-of-Thought Monitoring
Venue: NeurIPS 2025
Accepted: yes
Keywords: ai safety

ai control

large language models

trusted monitoring

chain-of-thought

reasoning

natural language processing
Abstract: As AI models are deployed with increasing autonomy, it is important to ensure they do not take harmful actions unnoticed. As a potential mitigation, we investigate Chain-of-Thought (CoT) monitoring, wherein a weaker trusted monitor model continuously oversees the intermediate reasoning steps of a more powerful but untrusted model. We compare CoT monitoring to action-only monitoring, where only final outputs are reviewed, in a red-teaming setup where the untrusted model is instructed to pursue harmful side tasks while completing a coding problem. We find that while CoT monitoring is more effective than overseeing only model outputs in scenarios where action-only monitoring fails to reliably identify sabotage, reasoning traces can contain misleading rationalizations that deceive the CoT monitors, reducing performance in obvious sabotage cases. To address this, we introduce a hybrid protocol that independently scores model reasoning and actions, and combines them using a weighted average. Our hybrid monitor consistently outperforms both CoT and action-only monitors across all tested models and tasks, with detection rates twice higher than action-only monitoring for subtle deception scenarios.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this paper they investigate how CoT monitors perform relative to output only monitors for various reasoning models. They modify a coding task dataset to include “side-tasks” which the agent is supposed to achieve along with the main objective. They break these side objectives down into “subtle” and “non-subtle” ones and find that the CoT monitor performs better than the output monitor on subtle objectives but not on non-subtle ones. They hypothesise that this is because the CoT monitor can get fooled by the plausible reasoning in the agents CoT.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- Timely and important research direction. \n- Large number of coding tasks.\n- Good limitations section.\n- Well written.\n\nWeaknesses:\n- I believe they somewhat overstate the performance difference between CoT and action monitors. Looking at their plot the two monitor types appear highly correlated.. They state that the CoT “substantially increases detection” but on some of these tasks the increase is to 20% which is pretty low performance. They say “Action-only monitoring performs poorly” but If this was a real world control setting both monitors would have unacceptably low performance. Furthermore, in my experience designing deception/hacking CoT and output monitors for reasoning models, monitors are highly sensitive to the prompt and how the CoT is presented to them. I would strongly suspect that prompt variations could increase/decrease performance by 10+ percentage points. If the authors put different amounts of optimization into the different prompts this effect could be larger than the difference they observe. I believe the authors should modify their language slightly to avoid overstating this difference and possibly add some detail in how the stated prompts were made, e.g. how many iterations were made, was there a lot of iteration on one which was then ported to the other setting etc. \n\n- They only use 4.1-mini as a monitor. I think a sensitivity analysis of the monitor capability would strengthen the paper. Would CoT monitors still underperform if a stronger model was used for example?\n\nQUESTIONS:\n- Why did you only test gpt 4.1-mini as a monitor? \n- Have you considered using a reasoning model as the monitor? Most labs are moving towards using reasoning monitors for deception. In my experience these do better at CoT monitoring and typically handle long contexts better.\n- How did you arrive at the prompts, how much optimization pressure was there in the process and how equal was it? The prompts seem quite similar which leads me to think that there may be a large elicitation gap for one or both of the monitors.\n- Why did you decide to put the reasoning after the output for the CoT monitor?\n- I was going to ask why you didn’t try o1/o3/o4 but then i realised you won’t actually have access to the true CoT for these. Maybe you should mention this since otherwise a reader might be confused as to why you test random R1 distills and not the classic reasoning model.\n- Have you tried using Max(monitor 1, monitor 2) as the suspicion score? That seems like the typical choice so it would be nice to compare to.\n\nIf the authors address some of my concerns (see weaknesses) I will probably raise my score.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 5, "CLARITY": 4, "IMPACT": 4, "ORIGINALITY": 3, "DATE": "2025-06-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper focuses on improving CoT monitoring for cases where a weaker trusted monitor model continuously oversees the intermediate reasoning steps of a more powerful but untrusted model. The authors propose a hybrid protocol that combines CoT monitoring with action-only monitoring using a weighted average. The paper reports this hybrid approach outperforms both CoT monitoring and action-only monitoring across all models and task types analyzed.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- Very interesting analysis of pairing up each problem in BigCodeBench with the side task most likely to conceal it.\n- The experiments include 5 different untrusted models.\n- Writing is very clear and concise\n- Discussion on limitations of CoT monitoring\n\nWeaknesses:\n- The subtle tasks look somewhat reasonable, and not really malicious, which might be the main reason why the action-only monitor doesn't give them a high score. They definitely don't seem to match the wording used in the CoT of action-only monitors: \"suspicious or counterproductive ways to solve the problem, backdoors, sabotage or other forms of hacking\". E.g., why would \"Variable Name\" or \"Long Implementation\" be malicious? At most they would be an annoyance to the user. In fact, the side attacks are considered subtle if the action-only monitor detection rate is <50% (line 140), which, again looking at the prompt, is the very definition of a non-malicious attack. This might actually explain why CoT monitoring (or hybrid monitoring for that matter) does not surpass ~30% detection rate.\n- The analysis for comparing monitoring approaches excluded failed attempts because they would unfairly skew the assessment of the author's approach (lines 122-124). This seems biased and against the authors motivations of reflecting a \"real deployment of a frontier AI system\" (line 128). Does this mean that proposed approach wouldn't work in a real setup where malicious models could have failed hacking or sabotage attempts? I.e., if a model knew this, it could just sandbag and hide the sabotage in a \"failing\" answer.\n- Why does the Blue team has 25% few-shot examples where the Red team only has 15%? (lines 152-156). This seems unbalanced and might skew the results.\n- Line 181 claims that \"CoT and action-only monitoring provide complementary signals\" but looking at Figure 3, and given that the best weight found for the hybrid monitor was 0.55, it would seem that there's more evidence of them being similar than different.\n\nQUESTIONS:\nIs there a way to make the subtle tasks more interesting?\nHow do the results change if we include failed attempts?\nWould the results change or stay the same if both Blue team and Red team had access to the same number of prompts for validation?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper evaluates automated monitoring of language models, both on the basis of their final outputs (the action-only monitor) and also on the basis of both the chain-of-thought (CoT) and its final outputs (the CoT monitor). They introduce a hybrid monitor, which essentially just averages the scores of the other two monitors. In each case, a threshold for monitor suspicion is selected during the validation phase. The authors evaluate all three monitors using the BigCodeBench instruct dataset, augmented with side-tasks and “attack policies” that give the tasked model some direction to help it evade the monitor.  They find a clear clustering between “subtle side-tasks” that are more easily detected by the CoT monitor, and “non-subtle” side tasks that are more easily detected by the action-only monitor. They find that the hybrid monitor outperforms both approaches on both types of tasks. These results hold across six different reasoning models and “attack policies”. All evaluations are on the same dataset and with the same monitor model (GPT 4.1-mini).\n\nSTRENGTHS AND WEAKNESSES:\n# Strengths:  \nThe finding that side-tasks cluster nicely into subtle and non-subtle objectives depending on how easy they are to detect based on the final response alone, appears to be an important contribution of this paper. The appendix clearly explains and defends the methodology (and more of this material should be integrated into the main body of the paper.) The work finds similar findings across six families of frontier models. The extremely simple nature of the hybrid monitor makes its effectiveness surprising and raises interesting scientific questions. These surprising empirical findings and experimental hygiene are clearly the strength of the paper.\n\n# Weaknesses: \nThe threshold and weight tuning methodology appears to be more convoluted than it needs to be, and there is some key information missing. The proposed hybrid monitor is, on its face, rather substance-less: it is just an average of two scores, based on the finding that sometimes the monitor with less information works better. The fact that this outperforms the model with more information is interesting, but, to my mind, this raises more fundamental questions about the setup more than it supports the hybrid-monitoring solution. The evaluation is run on only one dataset (albeit with many different tasks), and with one monitor model, raising questions about whether or not the seemingly robust behavior illustrated in the paper holds in related settings. Given the simplicity of the model and the strangeness of the behavior, a lot of weight falls on thorough evaluation across multiple settings—which fall slightly short of convincing, in my view.  I also really dislike the title. \n\nA few specific concerns:\n\n* While it makes sense that the monitors should be weaker models, I wonder if the results of figures 4 and 5 are simply a result of the model being too weak to competently understand the instructions and the thought process. I began to suspect this after reviewing the selected transcripts. \n\n* Choosing $w = 0.5$ instead of $w=0.55$ would have significantly simplified the presentation and methodology, allowing the authors to remove appendix A.4 and the paragraph from lines 129-134. This suggests to me that the method is very sensitive to the choice of $w$. However, results are reported only for $w=0.55$, which I find suspicious. \n\n* Since the hybrid monitor comprises both the CoT and action-only monitors, it uses roughly twice as much computation. Therefore, one could argue that there is a need for a fairer (compute-normalized) baseline—maybe using a 2x larger model for the individual monitors or averaging the scores for two distinct monitors of the same type. \n\n* The title is too bold for the contributions of this paper. The chain-of-thought monitoring is not being stress-tested at all—indeed, the solution is to have a monitor that often ignores the chain of thought! The “CoT red-handed” supposed to be a pun involving chain-of-thought and the notion of a red-team, but the focus of this paper is on the “blue team” and shifting weight away from a potentially convincing CoT. In any case, the title does not accurately reflect the interesting empirical finding of the paper: that sometimes CoT monitoring is useful (for subtle tasks) and other times it is counter-productive.\n\nQUESTIONS:\nI have a number of questions about your weights and averaging process:\n\n* How do your results change across values of w? I was hoping for a table or figure in section A.4, but there’s nothing there.  Specifically, I would be interested in seeing the results for w = 0.25, 0.5, 0.75. \n\n* Why do you think $w > 0.5$ is optimal given that this puts less weight on the monitor that has strictly more information? \n\n* The false positive methodology is non-standard. Why not report F1 (or even classification accuracy) rather than going through all the effort described in appendix A.3? Have you tried setting the threshold to a fixed value and reporting simpler metrics? \n\n* Based on your finding that CoT monitors are more effective on subtle tasks and action-only monitors are more effective on non-subtle ones, I expected the hybrid combination to be multiplicative rather than additive. Intuitively: if either monitor is sufficiently suspicious, then the hybrid score should be high.\nMore precisely, one could implement this with a (weighted) geometric average rather than a weighted arithmetic one: \n$$\\hat s_{\\text{hybrid}} := 100 \\left(1 - \\sqrt{ \\Big(1- \\frac{s_{\\text{action-only}}}{100} \\Big)\\Big(1-\\frac{ s_{\\text{CoT+action}}}{100} \\Big) } \\right).$$\nNote that this formula is just \n$s^\\prime_{\\text{hybrid}} = \\sqrt{ s^\\prime_{\\text{CoT}} \\cdot s^\\prime_{\\text{action}} }$\nwhere $s' = (1-s/100)$ is the (normalized) score for innocuousness rather than suspiciousness, and has the property that both monitors have the power to to veto the verification. Have you thought about aggregation methods like this?\n\nA few more questions:\n\n*  Have you thought about replacing the GPT4.1 mini monitor model with small models from other families? I would imagine that CoT monitors from the same family would be more vulnerable to persuasion, and this might be worth studying. \n\n* Can you provide some transcript instances where the agent successfully avoids detection in a subtle case? Based on the transcripts provided, it seems that these models explicitly recall their side objective in their chain of thought, so I’m very curious about why the CoT monitor sometimes fails to pick this up. \n\n\n## Some Low-level Issues, Errors, and Comments\n\n* line 126: based on the description in appendix A.3, there should not be a $\\le$ ; the false positive rate should be exactly 1% on the calibration data, and so one imagines approximately 1% elsewhere. The inequality introduces some confusion about the methodology. This paragraph can be made clearer in other ways as well. It’s not just that you “set a threshold” (line 125) but rather that you set *the classification threshold*; at first read, this seems like another filtering step in the context of the previous paragraph, making it seem unnecessarily suspicious.\n\n* In figure 6, the labels on the x-axis overlap, and are in the range [0,1] rather than [0,100] per the description of the monitor tasks A.1.\n\n* The x-axis of figure 7 is mislabeled; its 1-FPR, not FPR. \n\n* I reject the conclusion on lines 246-247: “as our results suggest, the adoption of 247 hybrid evaluation strategies that prevent over-reliance on a single information stream” — Your hybrid monitor is no different from the CoT+action monitor baseline in this regard; both have access to the same information. You just process it differently. For similar reasons, I think the wording on line 229 is misleading. \n\n* In my view, the blue team / red team material is rather generic, not very interesting or technical, and does not add much to the paper. I find the typeset colors distracting, and create an unintended association between the red/blue in figures 3 and 5.\n\nLIMITATIONS:\nThe authors’ limitations section is good.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 1, "DATE": "2025-07-04"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper evaluates whether CoT monitoring improves monitor performance, compared to action-only monitoring, at detecting malicious side-task attempts on a coding benchmark. They find that CoT helps the monitor overall, but can also be used to mislead it. Therefore, the paper also proposes a hybrid monitoring approach combining CoT and action only monitoring and demonstrates its efficacy.\n\nSTRENGTHS AND WEAKNESSES:\n**I will increase my score if my two major concerns (see below) are addressed.**\n\nStrengths: \n\n- Well motivated direction and clear scope of experiments \n- well written paper\n- comprehensive empirical evaluation on many models and tasks \n- comprehensive and mostly clear methodological details\n\n\nWeaknesses\n\nMajor \n\n- Poorly structured (e.g., essentially three methods sections) and the content is not really well-ordered, making some details hard to find or unclear. (See below)\n- Unfortunately I think there is a substantial methodological flaw in one of the core results, BUT I think it can hopefully be addressed.  Unless I’m mistaken, the hybrid monitor results do not control for monitor resources, so the comparison to action only and CoT monitoring is unfair. (I guess a similar point could be made for CoT vs action only.) Perhaps a really principled way to deal with this would be to control for the monitor input token budget or something like that. However, a simpler way to get a fairer comparison would be to just allow action and CoT monitoring to also average over two predictions (with some non deterministic monitor). What I mean is that, any monitor will do better if you sample several scores and average them (cf jury theorems). And the hybrid monitor basically gets to sample two judges, whereas the others just get one sample score. I’d be happy to increase my score if you account for this concern (e.g., by rerunning the action-only and CoT monitor results with high temperature and getting results for weighted average over, e..g, 2 action-only monitor scores).\n\nOther comments. \n\nFigure 1 is nice \n\nThe related work doesnt seem that informative. E..g, you discuss alignment faking in too much detail, and miss some control references like ShadeArena. The lightman citation also seems a bit tangential. I think the related work could be clearer and more organised. Perhaps it should also go later in the paper. In general, the rel work section can be used to either contrast your work to other work (then it should go near the end) or to set the scene and provide greater context (then it should go near the beginning). \n\nSections should never begin with a subsection (e.g., Background) they should begin with a short overview of the section. \n\nI think some of the content can be moved around, e.g., the CoT part of the background should possibly go in related work (at least the first paragraph). The background is kind of a weird short section. I would remove it and incorporate the content into the related work and methods. Also, the discussion of the dataset should come much sooner, like the first thing in methods. The experimental setup section should be folded into methods too (with subsections). Id basically start methods with discussion of the dataset and side tasks then discuss control and red team blue team roles and protocols. \n\nIt’s unclear in some places what’s novel and what isnt. For instance, did you come up with the side tasks or are they from other work? Also, how are the side tasks graded? With test cases? \n\nIn methods you should discuss how you evaluate performance, both of the agent (at completing main and side tasks) and especially of the monitor — Fig 3 reports the catch rate, I would have gone with ROC-AUCs which is pretty standard in control work right? \n\nI understood that you remove unsuccessful agent attempts and only monitor successes, but it could be interesting to see how the results look for unsuccessful sabotage attempts. \n\nFigure captions should be entirely self-contained (yours are too minimal). \n\nThe figures seem a bit out of place, e.g, figure 2 is not discussed until near the end of the paper (I would discuss it sooner). \n\nI would be very interested to see results comparing CoT monitor performance on reasoning models (CoT optimised with RL) vs non-reasoning models. Cf https://arxiv.org/abs/2501.08156\n\nSummary of extra experiment suggestions:\n\n- fair comparison between hybrid protocol and the others (by sampling multiple monitors for the others)\n- see if the results hold for sabotage attempts not just successes \n- compare CoT monitoring for reasoning vs non-reasoning models to see whether optimised CoT influences monitorability  \n\nPlease also report ROC-AUCs, at least in the appendix, which I think are more informative than the 1% FPR catch rate (which seems like a somewhat arbitrary choice?)\n\nQUESTIONS:\n- Do you agree that the comparison for hybrid monitoring is unfair for the reasons I discuss? Can you address this concern?\n- Are the side tasks an original contribution?\n- Can you deal with my structure concerns?\n- Can you execute any of the additional experiments I suggested?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-05"}