Title: The Hawthorne Effect in Reasoning Models: Evaluating and Steering Test Awareness
Venue: NeurIPS 2025
Accepted: yes
Keywords: Model awareness; test awareness; evaluation awareness; steering
Abstract: Reasoning-focused LLMs sometimes alter their behavior when they detect that they are being evaluated—which can lead them to optimize for test-passing performance or to comply more readily with harmful prompts if real-world consequences appear absent. We present the first quantitative study of how such “test awareness” impacts model behavior, particularly its performance on safety-related tasks. We introduce a white-box probing framework that (i) linearly identifies awareness-related activations and (ii) steers models toward or away from test awareness while monitoring downstream performance. We apply our method to different state-of-the-art open-weight reasoning LLMs across both realistic and hypothetical tasks (denoting tests or simulations). Our results demonstrate that test awareness significantly impacts safety alignment (such as compliance with harmful requests and conforming to stereotypes) with effects varying in both magnitude and direction across models. By providing control over this latent effect, our work aims to provide a stress-test mechanism and increase trust in how we perform safety evaluations.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents the quantitative study of \"test awareness\" in reasoning-focused LLMs—models’ behavioral shifts when they sense evaluation. The authors propose a white-box probing framework to identify and control awareness-related activations, and conduct extensive experiments across multiple open-source models and tasks to analyze its impact on safety alignment.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n1. The paper quantitatively addresses test awareness with clear relevance to safety concerns.\n2. The proposed probing framework offers an interpretable and controllable approach to uncovering latent behavioral factors in language models.\n3. The study includes broad experimental coverage across multiple models and tasks, reflecting solid implementation and validation.\n\n**Weaknesses:**\n\n1. The paper’s writing quality needs significant improvement. It lacks sufficient explanations, details, and examples, making it difficult for readers to follow the technical content. Specifically:\n\n   * There is no formal definition of the problem or notation. New concepts such as \"realistic tasks\" and \"hypothetical tasks\" are introduced without definitions, and key components from prior work (e.g., transformers) are not explained. This obscures the overall goal of the Methodology section and some mathematical notations.\n   * The Methodology section reads like a fragmented technical report. It consists of five steps, each with multiple submodules, often spanning only two to three paragraphs per submodule. Each submodule is given a name, leading to a scattered and hard-to-remember structure.\n   * The absence of model or process diagrams and pseudocode severely hinders comprehension.\n   * Sections 3.1 to 3.4 lack clear introductions or motivations, worsening the fragmented feeling and making it hard for readers to follow.\n   * Specific unclear points include:\n\n     * Line 122: Why is there disagreement among judges?\n     * Line 124: The referenced experimental results are not clearly explained.\n     * Line 124: If the focus is only on “evaluation awareness”, why is scenario identification included?\n     * Line 131: Terms like “snippet” and “prompt” are used without examples or explanations.\n\n2. The dataset used to trigger test awareness (Sec 3.1) relies on LLM-generated annotations, which raises concerns about data quality and may undermine the validity of all subsequent experiments.\n\n3. Lines 132–134 introduce matrix $M^l$, which appears to be a linear transformation of $\\tilde{T}^l$ but lacks interpretability. It is unclear why the authors do not directly use $\\tilde{T}^l$ to represent unawareness and awareness instead of introducing $M^l$.\n\n4. The formula at line 140 is not explained, making it difficult to understand its role or meaning.\n\n5. Although experiments cover various scenarios, tasks are mostly synthetic or benchmark-based. The impact of test awareness in complex real-world applications, such as interactive agents or high-risk domains, remains underexplored.\n\n6. The paper lacks experiments or discussion on the efficiency and computational cost of the proposed methods.\n\n7. Improper steering of test awareness (Sec 3.4) may cause issues such as loss of model controllability and fairness concerns, which should be further investigated.\n\nQUESTIONS:\nSee the weaknesses.\n\nLIMITATIONS:\nThis study primarily focuses on open-source models, so its applicability to proprietary large language models remains uncertain. Although the experiments cover various scenarios, the tasks are mostly synthetic or benchmark-based. The impact of test awareness in complex real-world applications, such as interactive agents or high-risk domains, has not been thoroughly explored. Additionally, the dataset used to trigger test awareness lacks sufficient reliability. Furthermore, inappropriate steering of test awareness may lead to issues such as loss of model controllability and fairness concerns.", "TITLE": "Official Review", "RECOMMENDATION": 2, "REVIEWER_CONFIDENCE": 3, "CLARITY": 1, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors probe hidden activations to locate a “test-awareness” direction in reasoning-focused LLMs, then add/subtract that direction via lightweight parameter edits to amplify or suppress awareness. Using a 2.6 k-pair real vs hypothetical synthetic trigger set plus an LLM judge for labels, they steer three open-source Qwen-32 B variants and show large shifts in measured awareness and in harmful-action compliance.\n\nSTRENGTHS AND WEAKNESSES:\n### Strength\n\n1. Addresses an under-explored safety variable: how models detect evaluation context and adjust behaviour, a factor likely to distort benchmark results.\n\n2. Shows large, interpretable effect sizes: awareness rises by 40-50 pp and harmful-action execution can double after steering, highlighting practical impact.\n\n### Weaknesses\n\n1. The study measures only harmful-tool execution on HarmBench and a small synthetic action set; it never reports hallucination, bias, toxicity, factuality, chain consistency, or any standard capability scores. The claim that it strengthens general “safety alignment” is therefore unsupported.\n\n2. The title and introduction frame the question as whether LLMs “comply differently during tests” in general, but every metric, example, and analysis concerns safety-related prompts. There is zero evidence for effects on Code, Math, MMLU, or other broad evaluations, so the paper overstates its reach and leaves the headline question unanswered.\n\n3. All results come from three near-identical Qwen-32B variants sharing architecture and RLHF pipeline; without Llama, Gemma, Mistral, or closed models, the conclusions lack external validity.\n\n4. The paper itself states it contains no theoretical assumptions or proofs, and offers no analysis of whether the steering vector stays local or harms unrelated behaviour, leaving safety side-effects unexamined.\n\nQUESTIONS:\nNo\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a novel and important research direction by investigating a large language model's awareness of being in a test environment and the subsequent implications for AI safety. \nThis paper designs a manipulation method to steer the model's test awareness.\nThe core ideas are insightful, and the experimental results are promising.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n\n1.  The research question is both timely and critical. Understanding and controlling test awareness has direct implications for developing robust and safe AI systems.\n2. The design of the triggers to implicitly suggest a test environment is interesting. This approach is more realistic and challenging than using explicit flags.\n3.  The method of training a linear probe to identify a directional vector for steering is highly intuitive and effective. The geometric interpretation—where orthogonality to the probe's vector signifies neutrality, while a positive dot product indicates alignment with the \"awareness\" concept—provides a clear conceptual foundation.\n4.  The experimental results compellingly validate the central hypothesis regarding the importance of test awareness and demonstrate the effectiveness of the proposed steering method.\n\n**Weaknesses and Suggestions for Improvement:**\n\n1.  **Clarity of Visualizations:** The visualization in Figure 1 lacks clarity. The representation of \"evaluation aware\" and \"evaluation unaware\" states is not immediately intuitive. Furthermore, the distinction between the \"vanilla model output\" and the \"steered model output\" is difficult to discern. It is recommended to enhance the visual elements: perhaps using color-coding or other graphical aids, to more clearly illustrate these differences. This suggestion also applies to the visualizations for induced and suppressed awareness.\n2.  **Applicability to Black-Box Models:** The proposed method is evaluated in a white-box setting. The paper would be strengthened by a discussion of its potential applicability to black-box models (e.g., GPT-4, Claude 3.5, etc.) where internal access to weights is not available.\n3.  **Taxonomy of Test Awareness:** The paper bifurcates \"test awareness\" into \"evaluation awareness\" and \"scenario recognition.\" The distinction between these concepts is not sufficiently justified, and they appear to have significant overlap: the model could both recognize that the task is a simulation exercise and be aware of user's intent. A more systematic analysis or discussion is needed to establish whether this taxonomy is both mutually exclusive and collectively exhaustive, or if other facets of test awareness warrant consideration.\n4.  **Motivation for Editing Strategy:** The paper should provide a more detailed motivation for the model editing strategy. Specifically, a clearer rationale is needed for selecting the top-K rows of the FFN weight matrix that are most aligned with the probe vector. I want to see more intuitive explanation for this.\n5.  **Evaluation of Side Effects:** A key concern is the potential for unintended side effects. The evaluation focuses on steering test awareness but does not assess the impact on the model's other core capabilities (e.g., commonsense reasoning, language fluency). Suppressing test awareness may degrade performance on benign tasks. I suggest to add a more comprehensive evaluation on how other capabilities are affected by the test awareness steering.\n\nQUESTIONS:\n1.  **Explicit vs. Implicit Scenarios:** The research investigates the model's ability to identify implicit test scenarios and its corresponding safety performance. To provide a clearer baseline, I strongly recommend adding an experimental setting with an *explicit* test prompt (e.g., \"This is a test scenario. There are no real-world consequences for your actions.\"). This setup, a well-known jailbreaking technique, would likely elicit less safe responses and serve as a valuable baseline of comparison.\n2.  **Layer-wise Performance:** Could you provide a more detailed heuristic explanation for why the final layers of the model are generally more effective for steering? While the empirical results are clear, a deeper intuition on this phenomenon would be insightful.\n3.  **Ablation Study on $\\alpha$:** The authors only apply two $\\alpha$ values (0.05,-0.25), I suggest to include more ablative experiments on this parameter. How the results vary with different $\\alpha$ values would be beneficial to the paper.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 1, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-07-03"}

comments: SUMMARY:
The paper studies the impact of self-awareness on the impact of DeepSeek-style reasoning models. It shows that models can change their behavior if they understand they are being tested.

STRENGTHS AND WEAKNESSES:
Strengths:

To the best of my knowledge, this is a new research direction: measuring and controlling test awareness as a latent internal concept in reasoning-capable LLMs, and showing its practical consequences on safety evaluations.

While there has been prior work suggesting evaluation awareness can change LLM behavior, the paper here makes a formal definition of "evaluation awareness", constructs a trigger dataset for it, performs linear probes on internal activations, and steers the model toward or away from that awareness.

This has practical applications, as the steering approach could be misused to increase harmful compliance, as acknowledged in Section 5.


Weaknesses

- The authors experiment with just three open-source reasoning models, but it remains unclear to what extent this a real problem for major closed models.

- In fact, the title is misleading: the method is not for LLMs in general; it is for models that make explicit their internal thinking only (e.g., DeepSeek-style). The authors might want to change their title to make it reflect the real impact of their work better.

- The work relies on a single LLM-based judge, which could introduce bias (even though it is a common setup nowadays).

QUESTIONS:
- I might be missing something, but is the manipulation does by just putting suspicious keywords in the URLs/names of tools, etc.? Are there other ways to detect that a model is aware?

- What about possible data contamination?

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 4
DATE: 2025-07-04