Title: CLAWS:Creativity detection for LLM-generated solutions using Attention Window of Sections
Venue: NeurIPS 2025
Accepted: yes
Keywords: llm

reasoning

math

creativity

hallucination
Abstract: Recent advances in enhancing the reasoning ability of Large Language Models (LLMs) have been remarkably successful. LLMs trained with Reinforcement Learning (RL) for reasoning demonstrate strong performance in challenging tasks such as mathematics and coding, even with relatively small model sizes. However, despite these impressive improvements in task accuracy, the assessment of creativity in LLM generations has been largely overlooked in reasoning tasks, in contrast to writing tasks. The lack of research on creativity assessment in reasoning primarily stems from two challenges: (1) the difficulty of defining the range of creativity, and (2) the necessity of human evaluation in the assessment process. To address these challenges, we propose CLAWS, a novel method that defines and classifies mathematical solutions into Typical, Creative, and Hallucinated categories without human evaluation, by leveraging attention weights across prompt sections and output. CLAWS outperforms five existing white-box detection methods—Perplexity, Logit Entropy, Window Entropy, Hidden Score, and Attention Score—on five 7–8B math RL models (DeepSeek, Qwen, Mathstral, OpenMath2, and Oreal). We validate CLAWS on 4,545 math problems collected from 181 math contests (A(J)HSME, AMC, AIME). Our code is available at https://github.com/kkt94/CLAWS.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces a new white-box method to classify LLM-generated mathematical solutions into the three categories: 'Typical', 'Creative', and 'Hallucinated'. \n\nIt partitions the context into 5 areas:\n* Guideline: Providing the model with the concept and standard for identifying creative \n* Problem P: The math problem for the LLM to solve.\n* Reference Solutions S: A set of 1 to _n_ ‘typical’ solutions to the problem\n* Instruction I: An instruction to create a novel solution\n* Response R: The output from the LLM\n\nThe average amount of model attention distributed across these five different areas is used to create a 5 element feature vector which is compared against reference features for the three categories. A response’s feature vector used to classify the response based feature vectors of known examples of each class. \nThe authors generate a ‘reference set’ dataset using five 7-8B math-specialized LLMs and labeling their outputs using two larger, state-of-the-art LLMs (GPT-o4-mini and Gemini 1.5-Pro) as evaluators, avoiding any human annotation\n\nExperiments show that CLAWS significantly outperforms five standard white-box hallucination detection baselines.\n\nSTRENGTHS AND WEAKNESSES:\nCLAWS provides a potentially simple yet powerful method to classify LLM responses quality. However, the experiment results rely on an unvalidated assumption about the reliability of LLM-based evaluators, which significantly undermines their credibility.\n\n# Strengths\n* Significance & Originality: Evaluating hallucinations and creative responses is crucial to the progress of LLMs and this paper attempts to tackle this with a relatively simple method to classify responses not based on the attention used to generate them. The method of using attention distribution over prompt sections to infer the nature of the generation (Typical, Creative, or Hallucinated) appears original and intuitive.\n* Simplicity: The CLAWS method is simple and computationally efficient as it does not require multiple LLM calls or external model calls for evaluation.\n* Apparent Rigor of Experiments: The use of 5 open-weight maths reasoning models and multiple evaluation metrics demonstrates a good approach to testing the method with the labels in the reference set (deficiencies in the reference set not withstanding (see below))\n* Figure 4 clearly demonstrates how CLAWS distributes the three categories in feature space.\n\n# Weaknesses\n* Flawed in Evaluation (Quality): The paper's most significant weakness, calling into question its empirical contribution, is its use of a reference dataset’s ‘ground truth’ labels are generated by other LLMs (GPT-o4-mini, Gemini 1.5-Pro). \n  * Unsupported Ground Truth: Using an LLM as a quality judge (e.g. creativity) without any validation is scientifically unsound. No evidence is provided of the LLM evaluators' accuracy, their agreement with each other (e.g., inter-annotator reliability), or, most importantly, their correlation with human expert judgment. \n  * Risk of Circularity: Without grounded labels there is a danger the authors are measuring how well aligned attention distribution in these small models is to GPT-4 and Gemini labels, rather than an objective notion of creativity. \n  * Lack of Transparency: Crucial details for this labeling process, such as the exact evaluator prompts (mentioned as Appendix A.1), are not provided, making it impossible to scrutinize or reproduce this critical step.\n* Clarity & Completeness: The paper is generally well-written, but key details are missing. None of the referenced appendices (detailing hyperparameter settings and the LLM evaluator prompts) are included - making a full assessment of the work's reproducibility and methodological details difficult. Furthermore, The checklist at the end of the paper is filled with \"[TODO]\", indicating an incomplete submission.\n* Limited Scope and Generalization: The method only works with open-weight models and relies on very specific prompt structure. Furthermore, the paper does not discuss how this framework might generalize to other creative tasks ‘typical; examples might not exist.\n\nQUESTIONS:\nThe proposed method is as far as I am aware very significant and original. However there are quality issues that could be substantially improved. I think the quality could be improved if the authors can convincingly address the following critical points.\n\n* The most critical issue is that the results depend on quality labels from LLM evaluators; labels which have not been validated. It would be good to see a measure of the raw agreement between GPT-o4-mini and Gemini 1.5-Pro on the 'Correctness' and 'Creativity' dimensions, e.g. Cohen's Kappa. More importantly, it needs to made clear how these LLM-generated labels compare to judgments from qualified human experts on a representative subset of the reference dataset. Without this, the claim of detecting \"creativity\" is unsubstantiated. The fact the Qwen2.5-14B and DeepSeekV2-16B models ‘lack of ability to creatively solve mathematical problems’ rings alarm bells - could ‘creative’ solutions are ones with correct answers but incorrect reasoning. Providing strong evidence here would be the most effective way to raise your score.\n* Layer-wise Contribution to Attention Signal: CLAWS averages attention weights across all layers. Have you investigated the effect of using different layers? It is known that different layers play different roles so an ablation study showing performance using different layers, e.g. using only top, middle, or bottom layers could provide valuable insights and potentially lead to a stronger method.\n* Definition and Scope of \"Creativity\": What is your rationale for the ‘inclusive criterion’ rather than a stricter one where both LLMs need to agree? How does it affect the results?\n* Completeness of the Submission: The current submission has several \"[TODO]\" markers in the checklist and references appendices that are not included. Can you confirm that a final version would include all prompts, hyperparameter details, and completed sections necessary for full scrutiny and reproducibility?\n\nLIMITATIONS:\nNo, the authors have not adequately addressed the limitations of their work. The paper lacks a dedicated \"Limitations\" section and does not discuss potential negative societal impacts. \n\nI strongly recommend the authors add a dedicated Limitations section that discusses:\n* Evaluation process: The most critical limitation is the reliance on unvalidated LLM evaluators for ground truth. The authors should be upfront about this, discuss the potential for circularity, and state what would be required for proper validation (e.g., human expert studies).\n* Applicability: CLAWS is restricted to open-weight models, preventing analysis of the most SOTA proprietary models behind APIs. This should be stated clearly.\n* Generalizability: The method's reliance on a specific prompt structure with ‘typical’ solutions may not generalize to other creative tasks and this should be discussed.\n* Potential Negative Societal Impact: The authors should discuss the danger in using LLM-generated labels in evaluation high-stake scenarios, e.g., assessing student work, where ground truth validity has not established.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-06-27"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes CLAWS, a white-box method for detecting Creative, Typical, and Hallucinated solutions generated by LLMs for mathematical problems. The approach leverages attention weights segmented by prompt sections (Guideline, Problem, Solution, Instruction, Response) to classify generations based on their attention distribution. Using a structured prompt design and evaluation by strong frontier LLMs (Gemini, GPT-4-mini), the authors label over 4.5K math solutions and demonstrate that CLAWS outperforms five white-box baselines (e.g., PPL, Logit Entropy) across multiple LLMs and datasets.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- The idea of using attention weights segmented by prompt sections for creativity/hallucination detection is new and elegant.\n- The empirical evaluation is extensive. CLAWS is tested against five relevant baselines across five different LLMs and on a large, challenging dataset of math problems. The use of multiple evaluation strategies (Prototype, MLP, etc.) and metrics strengthens the claim of the method's superiority.\n- The method is computationally efficient, requiring no additional model calls or complex operations beyond accessing and averaging attention weights from a single generation pass.\n- Despite some language issues, the core idea, setup, and contributions are clearly communicated.\n\n\nWeakness\n- While the paper acknowledges the subjectivity of creativity, it largely sidesteps deeper discussion or formal operationalization of what constitutes “creativity.”\n- The validity of the entire experimental setup rests on the quality of the labels produced by Gemini 1.5 Pro and GPT-4o-mini. While these are powerful models, the paper provides no direct evidence (e.g., a comparison with human experts on a sample, or inter-rater reliability between the two LLMs) to validate their performance on this specific, subjective task of creativity assessment. This introduces a potential source of unquantified noise and bias into the \"ground truth\" labels.\n- Although the guidelines for creativity are reasonable, they are somewhat subjective and rely heavily on the labeling from other LLMs. Some human verification could strengthen the validation.\n- The construction of positive/negative pairs may encode spurious cues or biases (e.g., formality, length, genre), which could confound the learning objective. This is not fully analyzed.\n\nQUESTIONS:\n- Could you provide data on the agreement rate between your two LLM evaluators (Gemini 1.5 Pro and GPT-04-mini)? More importantly, have you considered validating their outputs against those of human mathematics experts on a subset of the data to benchmark their reliability for this novel task?\n- For the baseline methods, how is the \"Prototype\" evaluation strategy implemented? These methods produce a single scalar feature, and it is unclear how Euclidean distance to a class prototype is meaningfully applied in a 1D space. Please clarify this.\n- How sensitive is CLAWS to the specific phrasing and structure of the prompt sections? Would minor variations in the \"Guideline\" or \"Instruction\" sections, for example, lead to substantially different attention patterns and degrade classification performance?\n- You note that the prototype-based strategy struggled with the OREAL model due to its high rate of hallucinated outputs. Does this point to a broader limitation of the prototype strategy in cases of severe class imbalance? How does CLAWS perform on OREAL for the 3-class detection task when using a trainable classifier (e.g., MLP) instead of the prototype method?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes CLAW, a novel method for classifying LLM generated answers in mathematical reasoning tasks into hallucinated, creative, or typical categories using attention weight analysis. They segment the input tokens into five sections of guideline, problem, solution, instruction, and responses for attention weights. The evaluation with LLMs such as GPT-4mini and Gemini1.5 pro shows that the proposed method achieves strong performances on benchmark datasets CreativeMath and HARLP with 4 math-specialized LLMs including DeepSeek, Qwen2.5, OpenMath2, etc.\n\nSTRENGTHS AND WEAKNESSES:\n## **Strengths**\n1. Novel problem formulation; the work tackles underexplored task to detect creative responses of LLMs. The work is now a three-way classification instead of a hallucinated/non-hallucinated classification.\n2. Interesting approaches with attention weights; they propose the prompt grouping for better usage of attention weights to obtain section-wise attention ratios and use them as features for three-way hallucination detection.\n3. Strong empirical results; the work compares the method with sufficient number of benchmarks (PPL, logit entropy, windown entropy, hidden scores, attention scores, etc) on multiple LLMs (deepseek, Qwen2.5, Mathstral, OpenMath2, OREAL).\n## **Weaknesses**\n1. Labeling/testing relies on specific LLMs: Creativity annotations are derived using GPT-4-mini and Gemini-1.5 Pro, with no human validation or inter-model agreement reported. This raises concerns about labeling consistency and reliability, especially given the subjectivity of creativity. Esp. the authors even mentioned that they did not report results with other LLMs as they seem not sufficient due to their lack of ability to creatively solve math problems. As scales increase, they may be less creative but likely to generate accurate responses. This may raise concerns regarding LLM specific issues or the necessity of creativity/ valid definition of creativity.\n2. The evaluation approaches relies on class prototypes derived from the reference set, which can be fragile under class imbalance as seen in OREAL where hallucinations dominate.\n3. No ablation studies; the method is empirically studied with performance comparison but no component-wise ablations such as prompt sections of guideline, problem, solution, etc. It would have been interesting to see their individual influence or the order of each chunk matters, etc.\n4. Grammar issues/ language unclarity in writing\n\nQUESTIONS:\n1. In lines 103–106, you mention that larger, general-purpose LLMs like LLaMa3-8B, Qwen2.5-14B, and DeepSeekV2-16B were excluded due to their lack of creative solutions. Could this be a consequence of how creativity is defined in your setups? \n- Specifically, that fluent, efficient, and correct solutions that resemble reference answers are penalized under your labeling criteria?\n- If so, does this suggest that the proposed method may be less applicable to stronger models that prioritize clarity and correctness over stylistic novelty? \n- More broadly, how do you distinguish between “lack of creativity” and producing the mathematically optimal solution?\n\n2. This proposed work uses GPT4-mini and Gemini1.5 Pro for dual labeling and report the correctness based on their agreement. Yet, you apply a union-based criterion for creativity. Have you measured the inter-model agreement rate (e.g., overlap or Cohen' k)? This would help understand the reliability and consistency of the proposed labeling protocol.\n\n3. Have you conducted any experiments to manipulate/isolate the influence of each prompt section (e.g., guideline, problem, solution, instruction, etc) on CALW attention weights settings as an ablation?\n\nLIMITATIONS:\nNA - labeling issues may not be a huge impact for the society but natural limitations due to the design choice for the methodology.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-03"}

comments: SUMMARY:
The paper presents CLAWS, a novel framework that uses human feedback signals to detect creativity in open-ended language generation tasks. It leverages contrastive learning with weak supervision to distinguish creative responses from conventional ones without relying on labeled data. The authors demonstrate CLAWS’ effectiveness across multiple datasets, showing that it can outperform supervised baselines in creativity detection.

STRENGTHS AND WEAKNESSES:
+Integrates implicit human feedback (e.g., upvotes, ratings) to guide the model without needing extensive annotations.

+Employs contrastive learning in a weakly supervised setting, which is both scalable and efficient.

+Demonstrates robust performance across several creativity-related benchmarks.

-The paper does not fully resolve how to consistently define or quantify creativity, which is inherently subjective.

- Performance may degrade if the human feedback signals (like upvotes) are noisy or biased.

-Focuses mainly on text generation; extension to multimodal or non-verbal creative tasks remains unexplored.

QUESTIONS:
How does CLAWS handle variations in human feedback quality across different platforms or domains, and could it adaptively calibrate its learning when feedback signals are sparse, noisy, or biased?

LIMITATIONS:
YES
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-04