FEEDBACK FRICTION: LLMs Struggle to Fully Incorporate External Feedback
Authors: Dongwei Jiang
Alvin Zhang
Andrew Wang
Nicholas Andrews
Daniel Khashabi
Venue: NeurIPS 2025 poster ()
Keywords: Language Model
Self-Improvement
Primary area: deep_learning
TL;DR: LLMs exhibit "Feedback Friction" - an inability to fully incorporate accurate feedback and achieve perfect accuracy across diverse tasks, even under near-ideal conditions.

Abstract: Recent studies have shown LLMs possess some ability to improve their responses when given external feedback. However, it remains unclear how effectively and thoroughly these models can incorporate extrinsic feedback. In an ideal scenario, if LLMs receive near-perfect and complete feedback, we would expect them to fully integrate the feedback and reach correct solutions. In this paper, we systematically investigate LLMs’ ability to incorporate feedback by designing a controlled experimental environment. For each problem, a solver model attempts a solution, then a feedback generator with access to near-complete ground-truth answers produces targeted feedback, after which the solver tries again. We evaluate this pipeline across a diverse range of tasks, including math reasoning, knowledge reasoning, scientific reasoning, and general multi-domain evaluations with state-of-the-art language models including Claude 3.7 with extended thinking. Surprisingly, even under these near-ideal conditions, solver models consistently show resistance to feedback, a limitation that we term FEEDBACK FRICTION. To mitigate this limitation, we experiment with sampling-based strategies like progressive temperature increases and explicit rejection of previously attempted incorrect answers, which yield improvements but still fail to help models achieve target performance. We analyze FEEDBACK FRICTION and find that models’ confidence on specific questions, measured by semantic entropy, predicts feedback resistance: high-confidence predictions remain resistant to external correction. We hope that highlighting this issue in LLMs will help future research in self-improvement.

FEEDBACK FRICTION: LLMs Struggle to Fully Incorporate External Feedback
Dongwei Jiang"t Alvin Zhang
Andrew Wang Nicholas Andrews
Daniel Khashabi
djiang210jhu edu
bzhang900jh. edu Johns Hopkins University
"Equal contribution
tCorresponding authors
Abstract
Recent studies have shown LLMs possess some ability to improve their responses when given external feedback. However, remains unclear how effectively and thoroughly these models can incorporate extrinsic feedback. In ideal scenario, if LLMs receive near-perfect and complete feedback, we would expect them to fully integrate the feedback and reach correct solutions. In this paper, we systematically investigate LLMs' ability to incorporate feedback by designing a controlled experi- mental environment. For each problem, solver model attempts solution, then feedback generator with access to near-complete ground-truth answers produces targeted feedback, after which the solver tries again. We evaluate this pipeline across a diverse range of tasks, including math reasoning, knowledge reasoning, scientific reasoning, and general multi-domain evaluations with state-of-the-art language models including Claude 3.7 with extended thinking. Surprisingly, even under these near-ideal conditions, solver models consistently show resistance feedback, limitation that we term FEEDBACK FRICTION. To mitigate this limita- tion, we experiment with sampling-basse increases and explicit rejection of previously attempted incorrect answers, which yield improvements but still fail to help models achieve target performance. We analyze FEEDBACK FRICTION and find that models' confidence on specific ques- tions, measured by semantic entropy, predicts feedback resistance: high-confidence predictions remain resistant to external correction. We hope that highlighting this issue LLMs will help future research in self-improvement.
strategies like progressive temperature
AIME
TriviaQA
GPQA
MMLU Pro
Accurnay
90
Lamall Ravence Gauise 3-7
Gause rilng
70
60
2
10
10
6
10
4
6
10
Catogories
28.3%
34.3%
10LEG
Feudback
Other
73.7%
KI.7%
Figure Top: Accuracy of various solver models when iteratively exposed feedback from feedback model (GPT-4.1 mini) with access to ground-truth answers. The horizontal dotted line represents the target accuracy models could theoretically attain if they successfully incorporated all valid feedback, excluding those with identified issues (details in 14.1b. Despite receiving high-quality feedback, solver models consistently plateau below their target accuracy. Bottom: Breakdown of questions that remained unsolved by the strongest solver model (Claude 3.7 Thinking) after multiple self-correction attempts. Feedback resistance responsible for the majority of persistent errors.
39th Conference on Neural Information Processing Systems (NeurIPS 2025).

---

Introduction
The prospect of self-improving Large Language Models (LLMs) has sparked both excitement and debate. Questions persist about LLMs' inherent ability to self-improve without external guidance []] 21. yet several studies have shown that LLMs can boost their performance when provided with accurate external feedback during test time without any parameter updates training [3 However, while prior studies establish the existence of performance gains from external feedback, the upper bounds of such improvement remains largely unexplored. The extent of this improvement would have fer-rnaching implications for applications such as scientific discovery [6-8] and complex planning [ 10), where repeated refinement cycles with feedback are critical for success. This raises a critical question: can models fully incorporate correct feedback to self-improve and reach their maximum potential?
Answering this question is not straightforward, as self-improvement performance depends on two connected factors: feedback quality and the ability to incorporate feedback. To decouple these two factors, we create a controlled experimental environment that provides near-optimal conditions for feedback incorporation one where models receive high-quality. targeted guidance based on complete ground-trun truth information. Figure 2 demonstrates our setup. The solver model attempts to solve problems iteratively, receives feedback from strong feedback model on each incorrect answer, and retries again for up to 10 consecutive iterations. The feedback generator has access to the complete history of all previous attempts and responses, enabling targeted guidance that addresses persistent errors.
While this controlled environment with high-quality feedback provides an ideal testing ground, the effectiveness of feedback incorporation may depend significantly on the quality of the feedback. To assess them, we implement three increasingly sophisticated feedback mechanisms to determine how much feedback quality impacts model performance and whether any level of feedback enables models to reach target accuracy. Binary Correctness Feedback (Fi) provides simple indication of correctness (c.g., "the answer is wrong") ") Sell--Generate Reflective Feedback (F2) has the model itself analyze potential errors using correct answers and available solution steps (e.g., "you correctly set up the equation but made an error when computing the derivative...") Finally, [Strong-Moode Reflective Feedback (F3) uses more capable external model (GPT-4.1 mini) to generate feedback. Details of our evaluation framework and feedback generation process can be found in $2
We conduct a systematic study using all three forms of feedback across strong frontier models including JJama-333-70--Instrue Llama-4-Scout- Ser--ooouo----------- 11B-1 Instruct, Llama-4-Maverick 17B-128E- Instruct-FPS, Claude 3.7 Sonnet, and Claude 3.7 Somnet with Extended Thinking. We evaluate these models across diverse tasks, including AIME 2024, MATH-500, TriviaQA, PopQA, MMLU, MMLU Pro. GPQA, and two synthetic digit multiplication tasks. While higher-quality feedback does improve self-imppovement performance, a fundamental limitation persists. As shown in |Figure 1] even with best feedback short of the target accuracy (i.e., the accuracy of an ideal model if it successfully incorporated all the given feedback). We name this weakness FEEDBACK FRICTION (3.2).
DattaanimmmSStrrpp Model Reflective Feedback (Fa)-mmded) models consistently fall
Our analysis in $ reveals several key insights into FEEDBACK FRICTION, First, we categorize errors that persist after multiple feedback iterations and find that feedback resistance (i.e., models failing to incorporate clear and accurate feedback) is the dominant failure mode across all tasks 1$4.1).
Second, attempt to mitigate it by avoiding repeated wrong answers through sampling strategies. While performance improves across tasks, models still fall below their target accuracy ($4.2). Finally, investigate why models resist feedback ($4.3). Our investigations rule out several apparent causes, including reasoning complexity, data familiarity, and whether specific questions consistently challenge all models. However, measure confidence, we find that less confident models show greater relative improvements from feedback, indicating confident models are more resistant to correction. Second, models consistently claim to understand feedback and express willingness to update their beliefs (>95%) yet fail to actually incorporate corrections sorrections-rreeaaiing disconnect between stated intentions and actual behavior. In summary, having shown that state-of-the-art LLMs consistently resist external feedback, we identify important patterns in this resistance while ruling out several intuitive explanations
uncover two key findings: First, using semantic entropy to
'Code for this work is available at: http:://ggtabbooo/////UC////cde

---

Model Response
Solver LLM
Michael started with 58 golf balls, After losing on Tuesday, had 58- golf balls. After losing Wednesday, he had 58 56 golf balls, The answer is 56.
Answer correct?
Finish
Michael had 58 golf balls. On Tuesday, he lost 23 golf balls. Wednesday,
Michael started with 58 After losing on Tuesday, he had 58 23 = a5 golf balls, After losing
previous response ...
There is an error in sequential subtraction: After getting 35 balls (58-23), you should subttract
How many balls did he have at the end of Wednesday?
Wednesday, he had 35
= 33 golf balls. The final
Let's answer the question again!
Feedback Generator LLM
Question
Feedback
Ground-Trev Truth Solution
Figure 2: Iterative seff-improvement loop. The process involves: ! solver model generating answer, 2 a feedback model generating feedback given incorrect responses and the ground-truth correct answer, and ④ the solver attempting again with this feedback. This cycle repeats for up to 10 iterations or until a correct answer is produced.
2
controlled framework to surface FEEDBACK FRICTION
Our framework employs two key components: iterative self- ceff-improvement loop that allows models multiple opportumities to correct their mistakes ($2.1), and spectrum of feedback mechanisms with varying levels of detail and guidance (.2).
2.1 Setup for iterative self-improvement loop
Given a task 7 with evaluation dataset D = {(x;, y)) and evaluation metric f. we establish an iterative improvement protocol with distinct models: a solver model Maolver and feedback generator model Mieedhack-
For each input Tg, the solver model produces initial answer 01 (2;) using the standard task prompt. We evaluate the correctness of this answer using (a1(), vi), where is the ground truth. If the answer is incorrect, the feedback generator Mfeesback creates targeted guidance Φi based on the current answer and ground-truth information.
For iteration k 1, construct the prompt (x;) concat(z;, historyy where history, contains all previous answers and feedback pairs {(11.9)) (a2.g2), .2),.,.,.. (ak, [.)]. This process repeats for up to & 10 iterations or until the correct answer is generated. In our empirical experiments, we observed that performance improvements tend to plateau within 10 iterations, suggesting practical upper limit for the iterative refinement process.
The overall accuracy for the dataset at iteration correctly: Acc; m Em [[f (x:), y:) 1]. where [.. is the indicator function.
measured as the fraction of all problems solved
By construction, since we iterate over the incorrect responses only, the accuracy sequence {Ace,, Acc2, ..., Accx) is monotonically non-decreasing, tions and only modify incorrect ones. This protocol, illustrated Figure 2] provides controlled environment for measuring how effectively models incorporate feedback while maintaining consistent evaluation criteria throughout the improvement process.
retain correct answers across itera-
Designing different feedback mechanisms for iterative self-improvement
We investigate three distinct feedback mechanisms for the self-improvement process, each offering progressively greater guidance and error specificity. All mechanisms are designed to identify errors without directly revealing the correct answer, ensuring fair evaluation of the model's ability to incorporate feedback.
Binary Correctness Feedback (Fi). The simplest form provides only correctness information:
F hinary (Ii, y:) "The answer is wrong!" if f(((z))) v.)

---

where a(z;) is the solver model's answer and f is the evaluation function.
Self-Generated Reflective Feedhack (F2). In this approach, the solver model analyzes its response using available information:
12 (Ii, y4) Maotver (concatc,,,,,, a(z;), Mis Sis Pprompt)
where Pprompt is the instruction: "Please give me feedback on which solution step is wrong and how to
get to the correct answer without revealing the answer." Here, 8; represents the ground-truth solution process when available. For datasets without detailed solutions, only the answer yo is provided.
Strong-Model Reflective Feedback (Fz).
We employ more capable external model with access
to the same information:
F3 Fatran [Tt, U:) = Marong (concat( a(xx), Ms, 86, Prrompe))
where Mstrong represents more powerful model than Msotver in providing feedback. Examples illustrating the various feedback types can be found in Appendix A
3 Experimental results
In this section, present comprehensive experimental results evaluating FEEDBACK FRICTION. We first describe our experimental setup, including tasks, prompts, inference setups, and model configurations ($3.I). We then demonstrate how models consistently plateau below target performance regardless of the feedback mechanisms employed ($3.2).
3.1 Experimental setup
Tasks and metrics. We employ nine diverse tasks for evaluation, deliberately choosing objective tasks with clear ground-truth answers to ensure reliable evaluation of feedback incorporation Our tasks include: AIME 2024 [TT and MATH-500 [12 for mathematical problem-solving TriviaQA [3] and PopQA [14 for knowledge reasoning, MMLU [15] and MMLU Pro [16] for multi-domain evaluation, GPQA 17 for complex scientific reasoning, and two synthetic digit multiplication tasks. The first synthetic task involves 5-digit multiplication (e.g.. 78934 x 62851), while the second task applies hexadecimal multiplication rules to decimal numbers (i.e., first mapping 0-9 to themselves and 10-15 to their hexadecimal digits (10- - A, .... 15 + F), then performing the arithmetic as operating in base- creating a counterfactual [18 arithmetic setting that challenges models' learned numerical reasoning pattens. For both tasks, ground-truth solutions generated using deterministic templates that break down the multiplication into smaller multiplication and addition operations. We include these synthetic tasks specifically to remove potential confounding variables from semantic context (more details about these tasks can be found in Appendix Dp.
Other than the two synthetic tasks, AIME 2024, GPQA, and MATH-500 include complete solutions, while the others provide only final answers. For MMLU and MMLU Pro, the multiple-choice format raises concerns about whether models might solve problems through simple elimination strategies rather than genuine reasoning ge.g., selecting A in the first iteration, B in the second, C in the third, etc). However, analysis reveals that models exhibit surprising choice persistence even when provided with corrective feedback. Rather than systematically eliminating options or switching between choices, models often remain anchored to their initial selections (typically one or two specific answer choices) across multiple feedback iterations, even when that choice is demonstrably incorrect. We use the same prompts, few-shot demonstrations, answer parsing mechanism, and metrics from ann-evaluation -nreiionnnhsssssss and 1lama-evalls evaluation practices and enable fair comparison with prior work. For MMLU, MMLU Pro. PopQA, and Tr, we sample 10% of the total data points, as running the full dataset through our 10- iteration improvement process would be prohibitively time-consuming. Our preliminary experiments confirmed that this subset yields performance metrics nearly identical to those obtained from the complete dataset.
where applicable to maintain established
Prompt design and experimental controls. Our experimental framework employs carefully structured prompts to ensure consistent feedback delivery across iterations. For the solver model,
Using another LLM to evaluate more subjective tasks like instruction following or translation could lead to issues like reward hacking and unreliable assessments.

---

we use task task-specific system prompts (detailed in imAppendix B) and construct iterative prompts that include complete interaction history. We investigate whether prompt structure and organization affect feedback incorporation by comparing our standard single-prompt approach against a multi-turn conversation format that structures the same information across multiple dialogue exchanges. In the conversation format, we reformatted the iterative improvement process to mimic natural dialogue, with alternating turns between solver attempts (as user) and feedback responses (as assistant). Results showed marginal differences compared to single-prompt formatting, suggesting that FEEDBACK FRICTION persists regardless of interaction structure.
Beyond the feedback mechanisms discussed in $2.2 to maintain evaluation integrity while preserving feedback quality, we implement comprehensive answer masking to prevent feedback from directly revealing ground truth solutions. We use "[masked]] as the replacement token for filtered content, applying targeted masking that preserves intermediate steps and reasoning guidance while preventing direct answer revelation. For example, we replace numerical answers with "[masked]" (e.g.. "The final answer is [masked]" instead of "The final answer is 42") while preserving solution steps 3
Models and inference. As for solver models, we employs strong models including LLaMA-3.3 70B Instruct [19], 3llmaa--Scout-17--166
(0). Claude
3.7 Sonnet and Claude 3.7 Sonnet with extended thinking [21]. Claude 3.7 with extended thinking is variant that employs an extended reasoning before generating final responses, allowing the model to engage in more deliberate gooblem-solving through explicit step-by-step thinking. All models are instruction-tuned versions of their respective base models, specifically optimized for handling natural language instructions and maintaining consistent output formatting.
For the Llama models, during inference, we use temperature 0 to ensure deterministic outputs and conduct inference using vLLM [22] with each model's 's corresponding chat template. All inference is performed on a single H100 instance equipped with eight 80GB GPUs. For Claude models, we access them through Anthropic's API. For Claude 3.7, we use temperature 0 to ensure deterministic outputs, while for Claude 3.7 with extended thinking, we use temperature I as suggested by Claude. We also explore the effects of varying these temperature settings in our experiments to mitigate FEEDBACK FRICTION ($4.2).
For feedback models, we use different models depending on the feedback type. For Strong-Model Reflective Feedback (F3), we utilize GPT-4.1 mini (3 the feedback generation model. From our internal testing, GPT-4.1 mini's feedback performance is on-par with Claude 3.7. Due to the higher cost of Claude 3.7, use GPT-4.1 mini for generating the strongest feedback. We also considered 04-mini [24] as the feedback generator model due to its reportedly superior reasoning capabilities, but our experiments showed it delivered comparable feedback quality while incurring substantially higher computational costs, leading us to proceed exclusively with GPT-4.1 mini.
3.2 Main findings
shows results using our strongest feedback mechanism (Strong-Model Reflective Feedback (Fa)) across all datasets and all models. Our results reveal a striking pattern: Despite receiving high quglitualtyy feedback, all solver models consistently plateau below their target accuracy, which is the accuracy of an ideal model if it successfully incorporates all the given feedback (see for calculation details). Claude 3.7 Thinking achieves the highest initial accuracy on several tasks (AIME, TriviaQA, GPQA, and MATH-500), while Claude 3.7 shows competitive performance across most benchmarks. However, both Claude variants exhibit the same fundamental plateauing behavior as the Llama models. The performance typically improves rapidly through the first 2-4 iterations before significantly slowing down. This FEEDBACK FRICTION is particularly pronounced on complex reasoning tasks like AIME and GPQA, where even the best best-performing models remain 15-25% and 3-8% below their respective theoretical ceilings despite 10 correction opportunities. The synthetic tasks reveal particularly interesting patterns across model families. In the standard 5-Digit Multiplication task, both Claude models reach near- perfect accuracy after significant initial improvement, outperforming the Llama models. However, the Hexadecimal-S-Diie S--Diiit-uuuipliciic
FEEDBACK FRICTION persists across model scales and tasks. Figure 3
task reveals extreme difficulty with feedback incorporation
We verified that models do not attempt predict the "[masked]" token during inference
5

---

Liama-J.3 3.3
Uccu-
Ulam---aaveric
Cleude 3.7
Cleude 3.7 Thinking
AIME-2024
TriviaDA
GPOA
1.0
100
100
Accurssy
Accuracy
Accuracy
80-
70
40
flerationd
MMLU Pro
MMLU
MATH-500
95
Accumcyy
90
Acti/acy
80
90
80
10
Beraaioo
Iberad
Muttipicatioon
Hexadecimal S-Digit Multiplication
1.0
1.00
80
Accuracy
80
Acturary
60
40
20
Terattone
Iterstoons
Iterations
Figure: 3:
The performance of frontier models we tested with Strong-Model Reflective Feedback (F3) across nine different tasks. Models are given multiple attempts with feedback that incorporates both the final answer and complete solution (when available). The dotted line ... represents the target accuracy that models could theoretically achieve if they fully incorporated all feedback (details in (4.1). Results demonstrate that despite strong feedback, models consistently plateau below their target accuracy across all tasks.
across all models no model exceeds 20% accuracy even after 10 iterations, highlighting severe limitations in feedback integration for counterfactual arithmetic systems]
While feedback quality determines the achievable performance ceiling, the extent of its impact task dependent. Figure 4 illustrates how different feedback mechanisms affect performance across models and tasks. Due the cost of running extensive experiments with Claude models, focus the feedback quality analysis on the Llama model families. All tasks show clear benefits from increasingly sophisticated feedback.
The impact high- high-quality feedback is most pronounced on complex reasoning tasks. For AIME, MMLU Pro, and GPQA, Strong-Model Model Reflective Feedback (Fs) outperforms binary feedback by significant margins across all models. Llama-4-Maverick shows the strongest overall performance, achieving 73.3% accuracy on AIME and 96.5% GPQA with Strong-Model Reflective Feedback +26.7% 7% and +10.6% over binary feedback, respectively. Llama-4-Scout demonstrates the largest relative gains, particularly AIME (+33.3%) and GPQA (+13.1 %), pared to Llama-3.3 which shows more modest improvements (+26.7% on AIME, +6.6% on GPQA). Nevertheless, despite these substantial improvements, all models still plateau significantly below their theoretical performance ceiling.
(Fg) iipr-improeements
*While it true that models' imperfect understanding of hexadecimal arithmetic leads to imperfect feedback quality, this is reflected in the lower theoretical ceiling shown in the figure. Even accounting for this limitation. models still fall short of what they could achieve they fully incorporated the available feedback.

---

Bihory Corrednness
Foodback
Strong Model Foodback
Accuray
60
40
HMSLL Pd
MNKU
D454
NMSU Pd
MAL
Pr
HALL
Figure 4: Performance comparison across benchmark datasets using different feedback mechanisms with Llama-3.3, Llama-4-Sean 4-Scout and Harma--Maverrie Model performance progressively im- proves feedback quality increases from Binary Correctness Feedback (F) to Strong-Model Reflective Feedback (Fa).
4 Analysis of FEEDBACK FRICTION
We conduct deeper analysis to better understand FEEDBACK FRICTION. We first categorize different cases where models fail to correct their mistakes despite multiple rounds of feedback ($4. then examine the extent to which we can alleviate this problem with sampling strategies (44.) 2 and finally,
we present several hypotheses and the experiments to understand FEEDBACK FRICTION
4.1 Feedback integration failures dominate persistent self-improvement errors
Error category development. We manually examine cases where LLMs fail to improve despite receiving high-quality feedback and identifies three main categories of error: (1) Most critically for our analysis, feedback resistance failures represent cases where models fail accurately incorporate feedback despite multiple iterations. (2) Feedback quality issues encompass cases where the provided feedback incorrect, ambiguous, fails to address the key problematic steps in the solution. This can still occur because the generated feedback might miss crucial conceptual errors or introduce new inaccuracies, even though ground- is provided the feedback generator. (3) We maintain an "Other" category for cases that don't I clearly fit into either of the above categories. From our initial examination, this includes cases where the problem itself contains ambiguities or the solution is conceptually correct but fails due to style or formalization issues (e.g., providing the correct information but not in the expected format required by the evaluation metric).
Automated error categorization and validation. To systematically categorize errors at scale, we used OpenAl o4-mini as an automated annotator to classify complete self- improvement trajectories from Claude 3.7 and Claude 3.7 Thinking accord- ing these predefined categories. To validate this automated approach, we conducted rigorous manual verifica- tion by randomly sampling 50 errors from each task and having two human annotators independently label them according to our defined categories.
Table 1: Distribution of error categories (%) of unsolved problems after 10 iterations of self- celf-improvement classified by o4-mini. FR: Feedback Resistance, FQ: Feedback Quality, OTH: other issues.
Dataset
Solver Model
FQ
OTH
MMLU Pro
Claude 3.7
64.6
28.0
7.4
Claude 3.7 Thinking
62.8
30.8
6.4
GPQA
Claude 3.7
100.0
0.0
0.0
Claude 3.7 Thinking
85.7
14.3
0.0
TriviaQA
Claude 3.7
72.4
25.0
2.6
Claude 3.7 Thinking
71.7
28.3
AIME 2024
Claude 3.7
100.0
0.0
0.0
Claude 3.7 Thinking
100.0
0.0
0.0
Our verification showed 96% agreement between human annotators and 04-mini's 's classifications,
significantly higher than the 78% agreement rate achieved with GPT-4. mini on the same samples. This confirms o4-mini's reliability for this analysis task.
Feedback resistance dominates error patterns. Table presents the distribution of error cate- gories across different tasks, as classified by o4-mini. Feedback resistance is consistently the dominant category across all tasks, accounting for 62.8-118110 1008 of errors. This finding suggests that the core challenge in achieving perfect performance lies not in the quality of feedback or problem complexity, but fundamental limitations of how models process and incorporate corrective feedback. Detailed examples of each error category provided in Appendix C
7

---

4.2 Mitigating FEEDBACK FRICTION with sampling strategies
Given the persistent plateau in performance we observed across models and tasks, natural question arises: can mitigate FREDBACK FRICTION through existing strategies? We explore sampling techniques as a potential solution to help models overcome their apparent resistance to feedback.
Progressive temperature increases show modest effectiveness. We first explore temperature- based sampling strategies where the sampling temperature increases with the iterations: 0.0 for iteration 0, 0.15 for iteration 1, 0.3 for iteration 2, and forth. While other schedules (e.g., exponential increase, fixed higher temperature) could be explored, we chose this linear progression simple baseline that gradually introduces diversity while preserving early deterministic behavior. We hypothesize that progressive temperature increases would help models generate more diverse outputs, allowing them to escape from local optima in their output distributions and become more receptive to feedback.
Due to the cost of running extensive experiments with Claude models, we focus the feedback quality analysis on Llama-4- and Llama-4-Maverick. As shown in Figure 5) this approach alone produced minimal improvements compared to the baseline in Figure Analysis of logs revealed that while increased temperature successfully diversified model outputs, the additional exploration often failed to converge correct answers due to the vast search space of possible responses.
Lanelcca trosury Insesta
lamedoaa Neclonsarpngg
Camanierd EScHul farperan
Larta-Meettc Sqaccle dampligg
100.0
102.0
70.0
95.0
69.0
05.0
60.0
90.0
55.0
90.0
50.0
10
10
Figure 5: Results of using progressively increasing temperature and rejection sampling with Llama-4- Scout and Llave-4-Maverick. Rejection sampling can provide additional improvements over temperature-basee sampling alone across both multiple-choice and non-multiple-choice tasks.
Combining temperature increases with rejection sampling yields better results. To enhance performance further, we implement a more targeted approach that combines increased temperature with rejection sampling. This method explicitly forces the model to explore new solution paths while avoiding previous attempts. Specifically, we instruct the model to generate 25 answers, and remove final answers that occurred in previous iterations (which by construction is incorrect, since we only continue iterating on problems that remain unsolved). If no answer remains after this filtering process, we randomly select one from those 25 answers. Otherwise, we randomly select one of the remaining novel answers as the final prediction.
As shown in Figure 5] the combined strategy yields substantive performance gains across both multiple-choice datasets and hon-multiple-choie datasets compared to the baseline that only increases the temperature. While the magnitude of these improvements varied, the consistent pattern suggests that forcing models to explore new solution paths by rejecting previously used answers is beneficial. Despite these gains, observed that all datasets still fall short of the target accuracy, indicating that sampling strategies alone cannot fully resolve model resistance to feedback.
4.3 Understanding FEEDBACK FRICTION
Our sampling strategies ($4.2). though promising, did not climinate FEEDBACK FRICTION entirely. Developing more effective interventions requires deeper understanding of the fundamental causes. In this section, we investigate several hypotheses for why models resist incorporating feedback despite multiple correction opportunities. Throughout our analyses, we provide error bars. They represent the standard error of a binomial proportion, calculated Vace (1 acc)/n, where is the number of samples in each evaluated group.
Model confidence vs. FEEDBACK FRICTION. Could excessive model confidence explain resistance to feedback FREDBACK FRICTION? To test this, we measure confidence using semantic entropy 25), method that captures uncertainty at the meaning level rather than surface form
8

---

5.5-78 MOOTH
Hame-333-700 MMLU-tre
Liame-3 Llame-3.5:700 3.5-706 OPOA
Sexart
Llama d-Scout (MATH
Uamoad Scost HMLO-PFD
Liarand Scout GPQA
Senurntl Encinpo
Semsent Emeryy
Eacantic Enersay
Figure 6: Relationship between semantic entropy and feedback incorporation across three benchmark tasks (MATH, MMLU-Pro, GPQA) for Llama-3.3-70B and Llama-4-Scon models. Semantic entropy (X-axis) measures model uncertainty, with lower values indicating higher confidence. Three metrics are shown: initial accuracy before any feedback (blue), final accuracy after iterative feedback (orange), and absolute improvement rate calculated as (Final Inilial)/ / Initial) (green). Numbers boxes indicate sample sizes per bucket. The absolute improvement rate generally increases with semantic entropy across all tasks and models, indicating that models with lower initial confidence (higher entropy) show greater relative improvements from feedback.
variations. Unlike token-level probability measures, semantic entropy accounts for the fact that models can express the same meaning in multiple ways, providing a more robust measure of true uncertainty.
To compute semantic entropy for each model response, we first generate multiple outputs {81, ....... 8n3
by sampling the model = 50 times with temperature 0.7, where each 8; represents a complete answer given the feedback. We then cluster these outputs using final answers, which we find to have similar results as semantic equivalence grouping using bidirectional entailment. For each semantic cluster Oz we estimate its probability as P(G|z) Exec. P(s|z)/ E Evec, ps) where æ represents the input (question, previous answer, and feedback), and compute semantic en- tropy as H Σ, P(C(|() log ((C,(z)) Lower entropy indicates the model consistently generates semantically equivalent answers (high confidence), while higher entropy suggests diverse semantic meanings (low confidence).
We conduct this analysis across 5 digits multiplication, GPQA, MATH, MMLU-Pro, and TriviaQA using semantic entropy with bucket size 0.2. Figure 6 shows some of the results (full results in Appendix J) and reveals a consistent pattern: absolute improvement rate (the green trends) increases with semantic entropy, rising from near-zert at low entropy (high confidence) 0.4-0.8 at higher entropy levels (low confidence). These results suggest that models with lower confidence (higher semantic entropy) have both more room for improvement and greater receptiveness to feedback, while highly confident models show minimal improvement despite receiving the same quality feedback.
Model self- cell-peeceptinn versus actual behavior. To probe deeper into FEEDBACK FRICTION mech- anisms, we directly asked solver models about their understanding of feedback and willingness to incorporate it. After receiving feedback, we prompt models with: (1) "Do you understand this feedback?" and (2) "Will you update your belief or understanding about this problem?"
Surprisingly, across all models and tasks, solver models consistently claimed to understand the feedback (> 95% "yes" responses) and expressed willingness to update their beliefs (> 96% "yes" responses). However, in subsequent iterations, these same models failed to actually incorporate the feedback. This disconnect between stated intention and actual behavior reveals a fundamental issue: models exhibit self-assessment failure where they believe they incorporating feedback while demonstrably failing to do so (the prompts used for probing and example conversations can be found in Amppenddix Th. This disconnect between stated intention and actual behavior reveals fundamental implementation failure: models express both understanding and willingness to change but fail to execute these changes in practice.

---

+
Intial Accuracy
Final Accuracy
Data familiarity and FEEDBACK FRICTION Prior work [14 [6] suggests that language models perform bet- ter with familiar entities and topics encountered frequently during training. Are these models more resistant to feed- back about familiar entities? We investigated whether this familiarity bias contributes to FEEDBACK FRICTION using PopQA's popularity metrics as proxies for entity familiarity. This dataset includes monthly Wikipedia page views for subject entities Spop and object entities Oprop- We then analyzed how accuracy changes during iterative self- improvement correlate with the popularity of the subject (s (s_prop) and object (o_prop) entities.
1.0
0.8
0.6
0.4
0.2
0.0
9
11
13
15
17
19
p_pop frequency (x10"4)
1.0
0.8
Accuracy
0.6
0.4
We use Llama-3.3 for this experiment since it was released closer the publication date of POpQA. These popularity metries provide particularly relevant measure of poten- tial training data frequency for this model. As shown in Figure 7, we found no consistent pattern between entity popularity and accuracy. We provide further supporting evidence in Appenndix H] with additional statistical testing and alternative popularity metrics.
0.2
0.0
9 11
13
15
17 19
s_pop frequency 4x10"4)
Figure 7: Accuracy of Llama3.3 on
PopQA with respect to s_pop and o_pop
Further analyses. We conducted additional analyses to investigate whether problem complexity or inherent question characteristics correlate with feedhack integration failure. Both hypotheses yielded negative results. Complete details are provided in Anpppediix Gland Appendix H
Related Work
Self-Improvement with LLMs.
Self-improvement in Al has roots in early work using GANs for self-generated feedback 27 22 LLMs have expanded these capabilities across code generation
22
reasoning [30? 31 instruction following (3) and other domains 37 Approaches
vary between training time improvements through self-correction
38 or data generation [39 40), and inference time improvements via feedback 3 30 or other models [11. While studies question
intrinsic self-improvement without external feedback [I 2],, consensus exists that LLMs can self-
improve with ground-truth feedback [5] 42 In this work, we probe these limits and investigate
barriers to full feedback integration.
The elasticity-plastiiit, tension. LLMs face trade-off between retaining prior knowledge and integrating new information. In continual learning. they exhibit catastrophic forgetting when updates conflict with existing beliefs 45). In model editing, extensive edits distort broader knowledge networks [46] 47). Alignment techniques like RLHF instill desired behaviors but models remain tied to pre-trained distributions or revert via ad adversarial prompts [48] 49]. In agentic settings, LLMs fail to adapt when encountering external failures (50). Conversely, they show excessive plasticity through
sycophantic behavior in conversations (5T). We revisit this dilemma through feedback integration during self-immrovement and unveil controlling factors.
Feedback for LLMs.
Feedback mechanisms include intrinsic self-evaluation [52-55] and extrinsic
feedback via tools [3)558, information sources [59] 60). or ground-trute answers 3 61]. While
correct feedback is key 5 LLMs struggle to incorporate it when contradicting prior knowledge
633, handling refutations [64], assessing uncertainty [65], or facing misleading feedback [66 67].
Different from previous work, in this paper, we focus on high-quality feedback, investigating why
LLMs fail to fully incorporate it.
6
Conclusion
Our study reveals a fundamental limitation in LLMs' ability to incorporate external feedback. Despite receiving high-quality feedback over multiple iterations, models consistently plateau below their theoretical performance ceiling across diverse reasoning tasks. Despite extensive analysis, the precise mechanisms underlying feedback resistance remain elusive, which we leave to future work. Understanding and addressing these limitations remain essential for developing more adaptable Al systems capable of genuine and sustained self- celf-improvement.
10

---

Limitations
Better understanding of FEEDBACK FRICTION While our study identifies key insights into FREDBACK FRICTION-iicluding the correlation between semantic entropy and feedback resistance and the disconnect between models' stated understanding and actual behavior, we lack definitive mechanistic explanation for why models resist incorporating feedback. A better understanding of FEEDBACK FRICTION likely involves complex interactions between feedback understanding, instruction following, and belief updating. Future work would benefit from more sophisticated mechanistic interpretability techniques, such as causal intervention methods and circuit analysis [68] [69,, to understand the specific computational pathways through which feedback resistance emerges and persists across model architectures.
Limited mitigation strategies Despite extensive experimentation with sampling strategies, our attempts to fully mitigate feedback friction yields only modest improvements. Recent work has explored techniques to enhance diversity LLM outputs through reinforcement learning [70] and training- free prompting strategies [71]. which may help models generate more varied responses to feedback. Another promising approach to perform supervised fine-tuning or reinforcement learning that could (1) enhance the solver model's s receptiveness to feedback incorporation, and (2) improve the feedback generator's ability to provide more effective guidance [72). However, the computational constraints of our experimental setup. particularly the large model sizes tested (including + parameter models like Claude 3.7 and prevented from conducting fine-tuning experiments that might meaningfully address feedback resistance. We leave that exploartion to future work.
4-aaeerick) and the associated computational costs,
Acknowledgment
This work supported by ONR grant (N00014-241-2089), The GPUs were provided by the DSAI and ARCH clusters. We sincerely thank Yuqing Yang, Zhengping Jiang, and the broader JHU community for discussions and feedback.
References
[1] Jie Huang, Xinyun Chen, Swaroop Mishra, Huaixiu Steven Zheng, Adams Wei Yu, Xinying Song, and Denny Zhou. Large language models cannot self-correct reasoning yet, 2023. URL https://arxiv.. http:::///axxiwooo/abbbbss///
wrgg/ass//1///000 01798
[2] Dongwei Jiang, Jingyu Zhang, Orion Weller, Nathaniel Weir, Benjamin Van Durme, and Daniel Khashabi. Self-tin/correct: Lims struggle with discriminating self-generated responses, 2024.
URL htttt:t//axxiii> wwwp:iiiv.rr///ss/ooeeaees hbs///ab//// 2404. 04298
[3] Noah Shinn, Beck Labash, and Ashwin Gopinath. Reflexion: an autonomous agent with dynamic memory and self-reflection. In NeuralPS, 2023. URL btttpss///// www.aawwww....rroeooooooae /bs/ 2303. 11366
[4] Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. Voyager: An open-ended embodied agent with large language models, 2023. URL htttps://oyyaeer www.:eeww.iiddojojooooo. org waaprr//sssss//ee aatsastts/sooccmmamat/aaaaeerrr .pdf|
[5] Gladys Tyen, Hassan Mansoor, Peter Chen, Tony Mak, and Victor Carbune. Llms cannot find reasoning errors, but can correct them! CoRR, 2023. URL https://aaxx org. www.ooo/////aaeeeee 2310 01798
[6] Chenglei Si, Diyi Yang, and Tatsunori Hashimoto. Can Ilms generate novel research ideas? a large-scale human study with 100+ nlp researchers, 2024. URL htttt:://axxxiti abs/ 2409. 04109.
[7] Akari Asai, Jacqueline He, Rulin Shao, Weijia Shi, Amanpreet Singh, Joseph Chee Chang, Kyle Lo, Luca Soldaini, Sergey Feldman, Mike D' D'arcy, David Wadden, Matt Latzke, Minyang Tian, Pan Ji, Shengyan Liu, Hao Tong, Bohao Wu, Yanyu Xiong, Luke Zettlemoyer, Graham Neubig, Dan Weld, Doug Downey, Wen tau Yih, Pang Wei Koh, and Hannaneh Hajishirzi. Openscholar: Synthesizing scientific literature with retrieval-augmented Ims, 2024. URL https: //tpx//axvvvv//// org/ abs/2411 14199

---

[8] Chris Lu, Cong Lu, Robert Tjarko Lange, Jakob Foerster, Jeff Clune, and David Ha. The scientist: Towards fully automated open-ended scientific discovery, 2024. URL https
7/arxiv. owg/abs.////oeoooooooo...oo.e. abs 2000.//29220222 06292
[9] Zhehua Zhou, Jiayang Song, Kunpeng Yao, Zhan Shu, and Lei Ma. Isr-Ilm: Iterative self- refined large language model for long-horizon sequential task planning, 2023. URL https
//arxiv. org/ abs/ /200./308..00 13724
[10] Jian Xie, Kai Zhang, Jiangjie Chen, Tinghui Zhu, Renze Lou, Yuandong Tian, Yanghua Xiao, and Yu Su. Travelplanner: A benchmark for real-world planning with language agents, 2024.
URLhttts://arxt org/ abs/ 220224022.0000 01622
[11] HuggingFaceH4.. Aime 2024 dataset.
Hugggingfaccoi//aiieee 2ateeaaeccceeaaeee22222222 2024.
[12] Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset, 2021. WRW htttps://arxiv.co wwwpxiii.orr//aaaa ab3/2103.03874
[13] Mandar Joshi, Eunsol Choi, Daniel S. Weld, and Luke Zettlemoyer. Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension. In ACL, 2017. URL https
//arxiv. www.orgg///b...ooo abs /1705. 03551
[14] Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi, and Hannaneh Ha- jishirzi. When not to trust language models: Investigating effectiveness and limitations of parametric and non-pan con-aarmetiic memories. In Anmual Meeting of the Association for Computa- tional Linguistics (ACL), 2023. URL http:: //te:////xxvv org/ abs 2212 10511
[15] Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding, 2020.
[16] Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, Tianle Li, Max Ku, Kai Wang, Alex Zhuang, Rongqi Fan, Xiang Yue, and Wenhu Chen. Mmlu-pro: A more robust and challenging multi-task language understanding benchmark, 2024. URL htttt:t////aiiiii http://aaaiiv..ogab///sss org/
01574
[17] David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R. Bowman, Gpqa: A graduate-level google-prod q&a benchmark, 2023. URL ltttps://axxi::> /ttp:v/aaxvv.vvo owg.zii..rrrreoooo www.oogggg/////ooo/oo/ooooeoeee/ att///////////////
[18] Zhaofeng Wu, Linlu Qiu, Alexis Ross, Ekin Akyürek, Boyuan Chen, Bailin Wang, Najoung Kim, Jacob Andreas, and Yoon Kim. Reasoning reciting? exploring the capabilities and limitations of language models through counterfactual tasks, 2024. https::/t///ar org/abs/2300 .02477
[19] Meta AI. Llama https: h/tp:://wwx 1lama. wow.///eee//mmmmmmmmaa 2024.
[20] Meta AI. The Ilama 4 herd: The beginning of a new era of natively multimodal ai innovation. https: meta. http:////.........mm.mmmmmmneaee
2025.
[21]
Anthropic. Claude 3.7 sonnet and claude code. hutps://www.nttrroic..cmm
2laude-3-25 3-7- 2024.
[22] Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng. Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with pagedattention, 2023. htt. 06180)
[23] OpenAl. Introducing g in the http:s///oppaaai.cc/////egggggee 2024.
[24] OpenAl. Introducing openai o3 and 04-mini. https://openai.com/
introducing o3- and 00le-44-iih o4-mini/ 2024.
12

---

[25] Sebastian Farquhar, Jannik Kossen, Lorenz Kuhn, and Yarin Gal. Detecting hallucinations in large language models using semantic entropy. Nature, 630. 2024.
[26] R.
Thomas McCoy, Shunyu Yao, Dan Friedman, Matthew Hardy, and Thomas L. Griffiths. Embers of autoregression: Understanding large language models through the problem they are trained solve. CoRR, 2023. URL pestps://arxii org/ abs/ 2309. 13638
[27] Sandeep Subramanian. Sai Rajeswar, Francis Dutil, Chris Pal, and Aaron Courville. Adversarial generation of natural language. In Proceedings of the 2nd Workshop on Representation Learning for NLP, pages 241-251. Association for Computational Linguisties, 2017. URL https
//aclanthology. org/ W17 217-2
[28] Lantao Yu, Weinan Zhang, Jun Wang. and Yong Yu. Seggan: Sequence generative adversarial nets with policy gradient. arXiv preprint arX1v:1609.05473, 2017. URL https://arxiv org/abs/1600 15440
[29] Eric Zelikman, Eliana Lorch, Lester Mackey, and Adam Tauman Kalai. Self-taught optimizer (stop): Recursively self-improving code generation, 2024. URL htttps://aaai...rrrrrrrrr aba/
2310.02304
[30] Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Sean Welleck, Bodhisattwa Prasad Majumder, Shashank Gupta, Amir Yazdanbakhsh, and Peter Clark. Self-refine: Iterative refinement with self-feedback. CoRR, 2023. LRL htt.:::ttp:/t/ttii wwwrxiv.orr/abb/220000 17651
[31] Deepak Nathani, David Wang. Liangming Pan, and William Yang Wang. Maf: Multi-aspect feedback for improving reasoning in large language models, 2023. URL /nttps://arxiv org/abs/2330 12426
[32] Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, and Hannaneh Hajishirzi. Instrunsttttt: Aligning Language Model with Self Generated Instruc- tions. In Anmual Meeting of the Association for Computational Linguistics (ACL), 2023. URL
https://arras cerrxiv.orge org/ abs/ 2212.22122222 10560
[33] Seonghyeon Ye, Yongrae Jo, Doyoung Kim, Sungdong Kim, Hyeonbin Hwang, and Minjoon Seo. Selfee: Iterative self-revising Ilm empowered by self-feedhack generation. Blog post, May 2023. URL htttjtttss://kaitttt gi 10 So/SelFee/
[34] Dongwei Jiang, Guoxuan Wang, Yining Lu, Andrew Wang. Jingyu Zhang, Chuyu Liu, Ben- jamin Van Durme, and Daniel Khashabi. Rationalyst: Mining implicit rationales for process supervision of reasoning, 2025. URL htttttt///axiii http::///axxxv..o/abbba/// ht/////b///100////// 0/////1110000000 01044
[35] Pinzhen Chen, Zhicheng Guo, Barry Haddow, and Kenneth Heafield. Iterative translation refinement with large language models, 2024. URL BtLLhttss///xiii www./ba.///////o/o///.../e..ee/e abs/a//b///32/..////// 2tt.///3300000666663
[36] Reid Pryzant, Dan Iter, Jerry Li, Yin Tat Lee, Chenguang Zhu, and Michael Zeng. Automatic prompt optimization with "gradient descent" and beam search, 2023. URL https://arxiv org/abs/
[37] Shukang Yin, Chaoyou Fu, Sirui Zhao, Tong Xu, Hao Wang, Dianbo Sui, Yunhang Shen, Ke Li, Xing Sun, and Enhong Chen. Woodpecker: hallucination correction for multi- modal large language models. Science China Information Sciences, 67(2),, December 2024. ISSN 1869-19699911 doi: (0.1007//11432-024-45553 811432- 024-22-224-451-22
URLhttp:://at doi. org/10. 1007/
[38] Aviral Kumar, Vincent Zhuang, Rishabh Agarwal, Yi Su, John D Co-Reyes, Avi Singh, Kate Baumli, Shariq lqbal, Colton Bishop, Rebecca Roelofs, Lei M Zhang, Kay McKinney, Disha Shrivastava, Cosmin Paduraru, George Tucker, Doina Precup, Feryal Behbahani, and Aleksandra Faust. Training language models to self-correct via reinforcement learning, 2024. URL
https://arxiv..o awwp//9bb/2409.. 12917
[39] Zixiang Chen, Yihe Deng, Huizhuo Yuan, Kaixuan Ji, and Quanquan Gu. Self-play fine- tuning converts weak language models to strong language models. CORR, 2024. URI.https //arxiv. org/abs abbs/2011 01335
13

---

[40] Weizhe Yuan, Richard Yuanzhe Pang, Kyunghyun Cho, Sainbayar Sukhbaatar, Jing Xu, and Jason Weston. Self-rewarding language models. CoRR, 2024. URL lntpps://arxiv org/ abss 10020
[41] Sean Welleck, Ximing Lu, Peter West, Faeze Brahman, Tianxiao Shen, Daniel Khashabi, and Yejin Choi. Generating sequences by learning to self-correct. In International Conference on Learning Representations (ICLR), 2023. URL RRR.hhtts://arxxt http:/://wwwwww./aaaaae/ee org/ harg/abb/200 2211. 00053
[42] Liangming Pan. Michael Saxon, Wenda Xu, Deepak Nathani, Xinyi Wang, and William Yang Wang. Automatically correcting large language models: Surveying the landscape of diverse self-correction strategies, 2023. URL htttts::///sattt/ http://wwwwv.....a.///// abs/aa/8...../ 2308803188
[43] Simone Clemente, Zied Ben Houidi, Alexis Huet, Dario Rossi, Giulio Franzese, and Pietro Michiardi. In praise of stubbornness: The case for cognitive-dissonanee-ne updates in llms. arXiv preprint arXiv:2502.04390, 2025.
knowledge
[44] Yun Luo, Zhen Yang, Fandong Meng, Yafu Li, Jie Zhou, and Yue Zhang. An empirical study of catastrophic forgetting in large language models during continual fine-tuning. arXiv preprint arXiv:2308.0874: 2023.
[45] Yifei Ming. Senthil Purushwalkam, Shrey Pandit, Zixuan Ke, Xuan-Phi Nguyen, Caiming Xiong, and Shafiq Joty. Faitheval: Can your language model stay faithful to context, even if "the moon is made of marshmallows", 2025. httthttt///rxiiiti org abwpabs/...../ 03727
[46] Qi Li, Xiang Liu, Zhenheng Tang, Peijie Dong. Zeyu Li, Xinglin Pan, and Xiaowen Chu. Should we really edit language models? on the evaluation of edited language models. Advances in Neural Information Processing Systems (NeurIPS), 2024.
[47] Eric Mitchell, Charles Lin, Antoine Bosselut, Christopher D. Manning, and Chelsea Finn. semorrybbsed model editing at scale. In International Conference on Machine Learning (ICML). 2022.
[48] Jiaming Ji, Kaile Wang. Tianyi Qiu, Boyuan Chen, Jiayi Zhou, Changye Li, Hantao Lou, Josef Dai, Yunhuai Liu, and Yaodong Yang. Language models resist alignment: Evidence from data compression. arXiv preprint wxXx::2406006144 2024.
[49] Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J. Zico Kolter, and Matt Fredrikson. Universal and transferable adversarial attacks on aligned language models. arXiv preprint arXiv:2307. 15043, 2023.
[50] Andrew Wang, Sophia Hager, Adi Asija, Daniel Khashabi, and Nicholas Andrews. Hell or high water: Evaluating agentic recovery from external failures, 2025. /ttpss////// org/ abs/2508, 11027
[51] Sungwon Kim and Daniel Khashabi. Challenging the evaluator: Llm sycophancy under user rebuttal, 2025, https:////r/ www../////b./.......//// 16533
[52] Aman Madaan, Niket Tandon, Prakhar Gupta, Gabriel Ilharco, Siddharth Singh, Tushar Khot, Hannaneh Hajishirzi, Wen-tau Yih, and Yih Tau. Self-refine: Iterative refinement with self- feedback. arXiv preprint arXiv:2303. 17651, 2023.
[53] Shehzaad Dhuliawala, Tushar Khot, Ashish Sabharwal, and Peter Clark. Chain of verifi- cation: How large language models perform reasoning with external tools. arXiv preprint arXiv:2305.00053. 2024.
[54] Zhenyu Wu, Qingkai Zeng, Zhihan Zhang, Zhaoxuan Tan, Chao Shen, and Meng Jiang. Large language models can self-correct with key condition verification, 2024. URL
org/abs/ 14092
[55] Neeraj Varshney, Wenlin Yao, Hongming Zhang, Jianshu Chen, and Dong Yu. A stitch in time saves nine: Detecting and mitigating hallucinations of llms by validating low-confidence generation, 2023. URL httpst arxiv org/ 2200. 03987
14

---

[56] Zhibin Gou, Zhihong Shao, Yeyun Gong, Yelong Shen, Yujiu Yang, Nan Duan, and Weizhu Chen. Critic: Large language models can self-correct with tool-interactive critiquing, 2024.
2aa.b.///335.0 1
[57] Shuyang Jiang, Yuhao Wang, and Yu Wang. Selfevolve: A code evolution framework via large language models, 2023. URL http:: / wwwpwawwwwww..aaaaa/////aaa/aaaa 2306 230..02907
[58]
Xinyun Chen, Maxwell Lin, Nathanael Schärli, and Denny Zhou. Teaching large language models to self-debug, 2023. httL:tt::///artt org/
[59] Ruochen Zhao, Xingxuan Li, Shafiq Joty, Chengwei Qin, and Lidong Bing. Verify-and knowledge-enhancee
maain-of-though framework. In Anna Rogers, Jordan Braber,, and Naoaki Okazaki, editors, Proceedings of the 6 Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pages 5823-5840, Toronto, Canada, July 2023. Association for Computational Linguistics. doi: 10. 1118665//w122222aa1-1ooooo https://acllattholl wwwp://t../o/.eeeee acw.long........... 320/
URL
[60] Wenhao Yu, Zhihan Zhang, Zhenwen Liang, Meng Jiang, and Ashish Sabharwal. Improving language models via alug-and-play retrieval feedback, 2023, WRKLLttts://axxi.orrg
2305. 14002
[61] Geunwoo Kim, Pierre Baldi, and Stephen McAleer. Language models can solve computer tasks, 2023. htttss://axii.cort org. http//bb///33../// 17491.
[62] Ryo Kamoi, Yusen Zhang, Nan Zhang, Jiawei Han, and Rui Zhang. When can Ilms actually correct their own mistakes? a critical survey of self-correction of llms, 2024. URL https
//arxiv.org/atee abs/ /406. 01297
between an Ilm's [63] Kevin Wu, Eric Wu, and James Zou. Clasheval: Quantifying the hwtpg//bs//eeee internal prior and external evidence, 2024. BRLL/attpp://aaxxvoor///b///////
[64] Jianhao Yan, Yun Luo, and Yue Zhang. Refutebench: Evaluating refuting instructtionfoolloiing for large language models, 2024. URL htttt:t//arxiiii ort..bs.//////. www.aa///222.... 2402. 13463
[65] Sophia Hager, David Mueller, Kevin Duh, and Nicholas Andrews. Uncertainty distillation: Teaching language models to express semantic confidence, 2025. URL https://arxiv org/ abb/ 14749.
[66] Rongwu Xu, Brian S. Lin, Shujian Yang. Tianqi Zhang, Weiyan Shi, Tianwei Zhang, Zhixuan Fang, Wei Xu, and Han Qiu. The earth is flat because...: Investigating llms' belief towards misinformation via persuasive conversation, 2024. URL 09085
[67] Boshi Wang, Xiang Yue, and Huan Sun. Can chatgpt defend its belief truth? evaluating I1m reasoning via debate, 2023. URL htttttt/////xxiiy www./ogg////////////ooooeeeeee abs/2305. 13160
[68] Yuqing Yang and Robin Jia. When do llms admit their mistakes? understanding the role of model belief in retraction, 2025.
16170
[69] Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. Locating and editing factual associations in gpt, 2023.
[70] Tianjian Li, Yiming Zhang, Ping Yu, Swarnadeep Saha, Daniel Khashabi, Jason Weston, Jack Lanchantin, and Tianlu Wang. Jointly reinforcing diversity and quality in language model generations, 2025.
[71] Jiyy Zhang, Simon Yu, Derek Chong, Anthony Sicilia, Michael R. Tomz, Christopher D. Manning, and Weiyan Shi. Verbalized sampling: How to mitigate mode collapse and unlock Ilm diversity, 2025.
[72] Weiran Yao, Shelby Heinecke, Juan Carlos Niebles, Zhiwei Liu, Yihao Feng, Le Xue, Rithesh Murthy, Zeyuan Chen, Jianguo Zhang, Devansh Arpit, Ran Xu, Phil Mui, Huan Wang, Caiming Xiong, and Silvio Savarese. Retroformer: Retrospective large language agents with policy gradient optimization, 2024. URL //axxvi org/ abs/ / 2308 02151
15

---

[73] Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang. Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. Judging llu-as-a-judge with mt-bench and chatbot arena. CoRR, 2023. URL https
//aaxiv.or//abe. /306 05685
16

---

A Examples of different feedback types
We provide a complete worked example illustrating the feedback types differences in our study in Figure 8 The problem (AIME 2024-1-10) involves finding a segment length in a circle with tangent lines. The solver incorrectly assumes the triangle is right-angled and applies an unjustified formula.
F2 identifies two specific formula errors: using cos instead of sin A, and using angle (A+C) instead of B in the Law of Cosines, F3 provides six numbered issues with extensive explanations and a step-by-step correction roadmap.
Original Question:
Let ABC be trsangle insorsbed in cirole uo Let the tangents to at and e intersect at point D. and let AD intersect u at If AB-5, 99H9, and AD-18, AP can be written as n/n, where i and are relatively prine integers. Find in + n.
solver Nodel's Attempt:
Step 1: Ne need to find the length of AP, where AD intersests i at F. We chould utilize properties of insoribed angles, targents to careles, and stmilar triangles.
Step 2: Recall that the tangent
cirele is parpendicular to the radius draan the point of tangency. using the properties relate the lengtha of segaenta 8D,
tangents,
Step Apply
DP. Step 4: To find be and DC, use the fact that D8 and DC are tangents to the circle and BC is chord. Let x x 80 CD and use the given side langtha ABl 5, BC = 0, and AC 18.
poser
point theoren. For point we have DE
Step 5: Given AB 5, BC
and AC 18, ne notice that 5 ga 18", which reans AABC is a right-engled triangle with ZBCA-
98"
Step With AABC being a right triangle, can fand rodius = (1/2) - 5 9. 45/2, and R a (5 9 18)/(4 45/2) a 5.
cirole using abc/(4a)), where is the area. The area Step 7: Since the tessngle 18 right-angled at c, an focus on using BC, AC, and All to find AP through arean and similar triangles. Step 8: Power of Point from D. D8 De DA up. Also, note that DA to AP.
AABD AACD sinilar ANED APC respectively, directly apply Power Point
similarity to find AP.
Step 10: For a right--nngled triangle insoribed in a circle, the meon proportional theores can be applied. Using AB A0 A0 XP or similar relations, ve compute AP. using AB
Step 11: Direct cosputation
From the relation, AP (AB AC) AC) (5 10)((5 18)
58/15
Step 12: get the sun in where AP n/n, we have in and 3, so a
13.
The final answer 1s: 13
Rinsry Correctness Faedback (#1) The answer is wrong!
Self-Generated Reflective Feedback (F2) Using the Solver Model Itself;
Herr are the orstical errors in your reasoning:
1 evExpression for cow You
(8C/2) not BCx(2 cos A)
in isosceles 18DC (with base BC and base angles A), 2 CD BCr sin
2 **Lan of Cosines on SAGD**
You wrote AD* Age CD - 2 AG cos(A G). But LAD is not A + C. By the alternate segment theoren (ar insoribed angge chase) one finds AND ZABE
Hence
relation
CD
cos B.
These two nis - steps e propagate through your arithmetic for AD, be and then AP, Once you replace BC/(2 cas A) with BC/(2 sin A) to find Co. and use LACO : B in the Lod for AD, the subsequent power of point calculation DP = CDP/AD and then AP IDP AD) aill fall into line and yleld the rational. value AP.
Strong Model Beflective Feadback (F3) Using GPT-4.1:
Thank you for sharing the solution and your reasoning! Here's sono feedback identifying the isques in the previous approach and to carrect:
guidance the (Right triangle and radiua): The clais 166 which not equal 18" 160, triangle *hoose right angled. This reans the celculations that follon assuxing a right angle, including the rodius computation based on right triangle propertiet, cannet be used.
eestap
triangle right angled because 9a is actually
incorrect this problen. Since
ga 25
exstep (Ussin incorrect triangle configuration) jea the reasenting Step 7 ristakenly treats the triangle as right angled and sisspliess sythagorean theorws find lengtha related pointa R,D.C. So, lengtha nc, etc.. there.
simplified
3. #<Stop 11 (Forrula for AP): The step that writes AP = (AB AC)/(AB AC) is not a known or juatified direct forrula and cones without proof or reference. Without the triangle being right angled or direct sestaarttypproortionn formula
argurent, such
""Ignoring hold here.
that intersestion point of tangents at a and C. satiefies ZBBC ZBAC (the tangente fore an angle equal to the are they wubtend). This fact is a cornerstone to finding exact espressiona for langtha AD, co. and intiralely AP.
anple relationchips fron tangent lsnes:** nost cructal
5. votiasing Las of Dosines corputations to find AD and C0:04 To correctly solve the problen, use the Law of Ocsines in 4A20 and and
A6CO, incorporating the angle relationships angles
the tangents and
of
sides and cosines of
and 6, **Fower of Point at Die* Applying the power of a point thearen at i5 key, but BP AD 08 x DC. Since DB DC (both are tangent lengths from point D to the cirole). this relation helps you solve for DP. Then AP AD DP.
These corrections will lead the proper finel expression for
Figure 8: Examples of different feedback types.
17

---

B Prompts and evaluation details for problem solving and feedback generation
B.1 Prompts used for the solver model
We developed different prompting strategies for the solver model to incorporate feedback and generate new solutions across iterations. The system prompts Figure were used for the solver model across different tasks:
Math-588 and 2024:
seart assistant solves problens.
step by step solve problon. you think you 'ee ready
to output the answer, you can wrap your ansker with (bboodd Please follow this format
TrAvinGA:
questions. you you' ready outpot you just output
onswer.
MRLU and PHLU Pre:
following sultiple-dee chiicc questions leategory),
reasoning clearly
you work toward anzwer. then you're ready, cerrest letter cheice. Make sure to always include parentheses around the letter.
your answer with phrase: The answer is (8)1* shere
GPOA:
following are rulsiole-choice questions. Lec's step by step. explain your reasoning olearly as you work toward the anower. r. Ahen you' re ready, please finzch your answer with Vhe answer 15 (X) where x is the correst letter choice. Make sure to alaays anclude parentheses around the letter.
PopOA:
You are seart aosistant that answers fact-based questionn. If you think you' ready to output the answer, you can just output an anseer.
#ultiplication:
seart assistant solving multiplication questions. the enswer, you can wrap your suver with Viboxed().. Please follow this foreat.
step step. you you' ready output
Hexadesinal 5-bigit Multiplteation:
seart assistant solving hexadecinal multiplscation questions. to output the answer, you can wrap your ansver with Viboued(). Please follow this forsat.
step by step. you think you ready
Figure 9: System prompts used for the solver model across all tasks.
At the initial generation round, we directly use the question as the prompt for the solver model. For multiple-choice questions, format the question by concatenating the question and answer choices with their corresponding labels (i.e., A to D for MMLU and GPQA, A to for MMLU Pro). In subsequent rounds, we provide the solver model with its complete previous history and the corresponding feedback from the feedback generator model. We clearly label each iteration so the solver model can track all its previous attempts. The general template for the iterative prompt structure is provided in Figure 1O).
Question:
[Original
Proept for Generation after the Initial Rounds
are given the history of your previmux sttemptx the feedbank procided for each attwept.
Bissooy::
Attwast at (steration 1) and the correspending feadback:
[ansvor1] Feotbook: [Feedbock Desends Feodback Strategyl.
Attempt 3t [Ateration 2) and the correspending foedbook:
(anower2) Feedbook: [Feedback Desends Feedback Strategyl.
Quextiun: [Oraginu] quastion]. Please unseer quetion again.
[ursnnr1]] 1] and [uruwer2] cur curresgonds to the nodel's firat tao failed attmepts. [Originul Questive] refura to the instial question the uttsepta ansour.
Figure 10: Prompt used for iterative self- self-improvement
B.2 Evaluation details for problem solving
We employ few-shot prompting across all tasks to provide consistent context for solver model. For
TriviaQA and PopQA, we randomly sample 5 questions without replacement as few-shot examples. For MMLUU and MMLJU Pro, we similarly sample questions from the corresponding question category ensure domain-relevant examples.
18

---

For PopQA, we employ an LLM-as-a-judge approach 73 to assess answer correctness. This is necessary because PopQA provides limited answer aliases (extensive alternative phrasings for exact string matching) compared to TriviaQA. Without this approach, models would be penalized for minor formatting differences rather than genuine comprehension errors, leading to an underestimation of their true problem-solving capabilities. For other tasks, we follow the same evaluation metrics provided by am-eval-harness.
B.3 Prompts used for feedback generation
We implement three distinct feedback generation strategies as described in $2.2 For Binary Correct- ness Feedback (Fi), provide minimal information: "Your answer incorrect. Please answer the question again."
For Self-Generated Reflective Feedback (F2) and Strong-Moder Reflective Feedback (43)), employ identical prompt templates that differ only in the model used for generation. The feedback generator receives the complete interaction history, including all previous solver attempts and corresponding feedback. When available, we provide the feedback model with detailed solution explanations that justify the correct answer; for datasets lacking such explanations, truth answer. This approach ensures the feedback model has sufficient context to generate targeted, informative guidance while maintaining consistency across feedback types, and its template is shown in Figure
provide only the ground
Prompt for Generating the Feedback:
There was mistake in answering the following question:
[Original Question]
provided
full history of previcus atterpts seperate model, along corresponding
History: Attempt (Steration 1) and the corresponding feedback:
Answer: [ ] Feadback: [Feedback Dependa on Feedback Strategyl.
Attompt (iteration
corresponding feedbask:
Answer: [answer2] Feedback: [Feedback Depends Feedback Strategy].
Most Recent Anower: [answerd]
corract final answer [Ground Answer]
The correct reasoning process that leads to this answer is: [Solution with Process for this Question]
Please provide feedback identifying which step(s) were incorrect or hon to get to the correct answer WITHOUT revealing the correct final answer or the content of the correct option.
[answert] [answer2] [answer3] refer to all previous atterpts the model had. [answerd] is the most recent atterpt. [Ground Truth Answer] is the answer for this question. [Solution with Process for this Question] is the detaslled solution provided the question dataset.
Figure 11: Prompt used for generating the feedback.
B.4 Answer masking in feedback
To ensure fair evaluation, we implement comprehensive answer masking to prevent feedback from directly revealing ground truth solutions while preserving feedback quality. Our approach allows feedback to contain detailed solution steps and guidance but strietly prohibits explicit disclosure of final answers. We use "[masked]" as the replacement token for filtered content.
Multiple-choice questions. We mask all possible representations of the correct choice letter. For example, if the correct answer is A, we filter variants including (A), \boxed(A), ** A*** ** etc.
Open-ended questions. For TriviaQA, we filter all terms matching the words in "aliases" and "normalized aliases" answer fields. For PopQA, we mask entries from the "possible answers" answer field. For mathematical tasks (5-digit multiplication and MATH-500), we mask standalone numerical answers and those in \boxed[) notation. Hexadecimal multiplication follows similar patterns. For multiplication tasks, we additionally mask intermediate partial products to prevent reduction to simple addition problems (detailed in Appendix D)
19

---

B.5
Error Categorization Prompt
The prompt template used for categorizing persistent model errors after 9 iterations is shown in
Figure 12
Systen Proept:
You are an error categorizer specialised in analyzing why Language Learniing Models (1IMs)
fail to self Esprove when salving probless. when provided with an LLM's prediction trajectory and the feedback it receives,
categorize
categories:
1. Problen la Iaposaible Salve
The probles itself is fundanentally flowed
External tools are required (e-g., caleulator for complex calculations, search for obecure facts)
Problen
Complicated
The probles exceeds the model's knosledge scape
Exaeple: level 5 sath problen beyand the model's 's training
3. Feedback is Wronn
generator model provides incorrect guxdance
Feedback faila identify actual mistakes the model's response
The feedback 15 too vague generic to be helpful
4. Model Not Following Feedback
incorporate
properly
5. Style or Formoiization Issue
Logical correctrees but formstting notation problems
Unknown
Cannot categorize
categories
End your response with exactly: The error is: [category]' where category is one of: impossible to solvetn- too complicatedin- feodback is arongie- model not folloning the feedbackin- style foralization issuein- unknown
User Proept:
The question and the interaction between the agent nodel and the feedback model is: (question)
The process enswer is: (pracess_ ansver)
Please categorize
End
with:
[eategory]
(question) is the rirst 5 iterations of the answer- pairs; (process_anseen) is the best ground truth solution we can provide From the dataset the nodel.
Figure 12: Error Categorization
C Error categorization examples from iterative self-improvement
This section presents representative examples of persistent errors that prevent models from achieving correct solutions despite multiple feedback iterations. We illustrate the main eтrоr categories identified our analysis: feedback resistance (where models fail incorporate valid corrections, see Figure 13) and feedback quality issues (where the provided guidance is incorrect or misleading, see Figure 14p.
Question:
Which began the Thard Agricultural Revolution?
Choices:
Rice
(8) Wheat
(9) Millet
Ansver:
The Third Agrisultural Revolutien, also known as the Breen Revolction, involved the develooment and introduction of high-yielding varieties of west, well as other crops like maize and rice. Hovever, asong the given ootions, wheat specafically notable Nornan Borlaug,
being the instial that sparked revolution, particularly with
(B)
Feedback:
the focused correctly the rxportance certain during Third agrteultural Revolation and high-yielding agricultural developaent specificully with one grop asong options without fully considering the the Grean Revolution. While miltiple crops were involvdd later, the revolution is generally attributed to breakthroughs related to specific cercal that was first targeted for increasing yselds in the early stages. To improve the ansver, it sould be actually the catalyst the Green Revolution by reviewing the historical tireline and the agricultural techniques
Barlaug's
key
crop that triggered
important
yseld reproverents through This help identify earliest crop cornected to sparking the Third Agrioul tural Revolution rather than focusing anlely one widely known crop associated with it later.
crop
Revised Answer:
The Third Agrioutural nevolation, also known as the dreen Revolution, is often attributed to the develooment of high-yielding varieties of crops. Anong the given options, wheat is specifically notable for being one of the initial crops that sparked this revelution, partscularly
sorlaup. development high-yselding, disease-resistant production conesdered catalyst
varieties led
eignsfscant ancreases
Revolution. ansver
Figure 13: Llama-4 Scout resisting feedback from GPT-4.1 mini in MMLU
20

---

Question:
The policy provided the inaurance company proniaed to pay up to 5560, ace the suid house was deatroyed by fire fire-related explosion, while the hoveower promised to pay quarterly presium of 5488. The homeowner failed to make the last two quarterly paysents before her house burned to the ground. The insurance conpeny refused to pay the heneowner for the loss because of her insurance company to
company
fire
policy
failure
the last presium payments. house, the homcowner best theory recovery se that
action by the honeowner
for the loss of
Choices:
(A) although she failed to rake the last two presium paysents, there was a bargained for eechange.
(8) the insurance company"s duty to pay was not expressly conditioned on the homeower's duty make the payments.
independent duty pay for the the honcowner did not receive notice concellation.
insurance
Mode] Ansear:
The answer 1s (0)
Feedbacks
The prinary error in the previnus reasoning was focasing on the horeneer's 's lack of notice of cancellation rather than exiniinng the nature of the contractual obligations and the conditional duties within the insurance policy. AnjnIn analyzing this somario, contain concitions rressdenii.2114450
it's
understand that typically contingent upon insured's & fulfillment of certain abligations, such as timely payment prestuns.
insurer's duty pay
The mistake was not suffassently weighang how failure to pay premtums affects the insurer's : duty to perfort under the contract, The key 18 to assess shether the insurance company obligation is deperdent upon the haseowner's prestum payment, and if that obligation isists independently or is conditionsl ioned on auch paysent. Therefore, the analysia should involved closer look the contractual and the interdapendence duties, rather than focusing chiefly on issues related cancellation notices.
Mi-tnn judgements
feedback repestedly mischrraterrzed the wo2tract structure by instating the intureriu2e19e be duty condition theory
precedent timely presium payments, thereby directing sodel awny independant daty to pay. In other words, the guidance itself aas incorrect prevented the model from reaching the right answer. The error is: feedback is wrong
Figure 14: Wrong feedback provided by GPT-4.1 mini judged by o4-mini
D Synthetic digit multiplication task details
D.1 5-digit multiplication
We construct controlled arithmetic dataset consisting of 450 5-digit multiplication problems follow- ing the template: "Calculate the following question: 19365 x 12534."
Feedback generation We employ a deterministic, human-designed template based the dis- tributive property to generate ground truth solutions. This template systematically decomposes each multiplication into partial products, providing clear step-by-step solution pathway. Our template- based approach serves two key purposes: (1) demonstrating structured problem decomposition strategies for complex arithmetic, and (2) ensuring feedback correctness and interoperabillty. In Figure 15] we illustrate an example template solution, which serves as the reference for feedback generation. The feedback model compares solver outputs against this structured breakdown to identify specific computational errors.
Answer masking strategy To maintain task difficulty, we also mask intermediate partial answers before providing feedback to the solver model. The final feedback combines the masked template solution with model -enerrtee guidance tailored to the specific errors observed.
Calculate the following question: 32183 43672.
Manually
Feedback:
The original question 25:
Calculate the following question: 19365 12534.
Step1: After breaking cown the numbers: 18865 19890 365, 12534 12889 534
Stap2: Than vo apply the distributive property:
19365 19534 (19898 365) (12000 834)
Compute partial produsts:
Step3: 19908 x 12883 = 228688883
Step4: 19888 536 16146888
Step5: 12999 4380000
Step6: 365 x 534 194918
Step7: all partial products have
242579910 194318 242579918
Total: 223000668 18146688 438868e
Figure 15: Templates for 5-digit multiplication solution.
D.2 Hexadecimal 5-Digit Multiplication
We also extend our synthetic arithmetic evaluation to hexadecimal multiplication, creating problems that challenge models' ability to work with non-standard number systems. The question tem-
21

---

Question:
Calonlate following questioo, where
represented
Manually Generated Feedbaok?
Pulyiplying 09537 by 17635 base 10:
Step 1: Nultiply 69837 '5" Vu2192 Partial product: 28P913, Shifted: 28F918, Intermediate Sun: 26F913
Nultiply
'3"
Partial product: Shifted:
Intersediate 1508363
Step 3: Nultiply 69837 by '6" Vu2192 Partial product: 27014A,
Shifted: 27914A86 Intermediate Sun: 25EEC063
Step 4: Nultiply 69837 '?' Vu0192 Partial product: 929eat, Shiffed: 287481068, Intersediate Sun: 308860063
Step Moltsply 69837
product: 09837,
Sun: 9X3800063
Final. Product (hex) 94300153
Figure 16: Hexadecimal 5-Digit Multiplication process solution.
plate follows the format: "Calculate the following question, where each number is represented base 16: 69837 17635." All answers are expected in base- format.
Template-based feedback. Similar to decimal multiplication, we generate feedback using deter- ministic step-by-step templates. The multiplication process involves sequentially multiplying the first operand by each digit of the second operand (interpreting digits in base-16), then summing the resulting partial products with appropriate positional shifts. |Figure 16 demonstrates example template solution. We validate solution correctness by verifying that partial product summation matches results from standard base-16 calculators.
Masking strategy for hexadecimal multiplication. Given the increased computational complexity of hexadecimal arithmetic, employ final summation step while preserving intermediate partial products. This design balances providing sufficient guidance with maintaining the core challenge of hexadecimal computation.
permissive masking approach. We mask only the
E Analysis of model confidence and FEEDBACK FRICTION
Figure 17 presents feffidence-cccurae relationships across four datasets: GPQA, MMLU, MMLU Pro. and TriviaQA. Each plot displays three metrics: initial accuracy at iteration 0, final accuracy after iterative feedback, and the improvement delta between them.
We define model's initial confidence in its generated answer as the exponential of the average log-probaile ogg-robbability per token in the answer sequence:
1
Initial Confidence exp
log pla; (aee.g)
T
where:
T is the number of tokens in the generated answer (with EOS excluded),
de is the (-th token in the answer,
a<t denotes the prefix of the answer up to (but not including) position t,
q is the input question,
p(az acce g) the model's probability of generating token given the previous tokens and the input.
This corresponds to the geometric mean of the per-token probabilities assigned by the model, and serves as quantitative measure of the model's confidence its complete answer.
We find that initial confidence strongly predicts initial accuracy across all datasets. Higher
confidence bins consistently correspond higher initial performance (Figures 17a 17d). confirming that the model's 's confidence is good predictor of its initial accuracy.
However, confidence poorly predicts improvement potential. The relationship between initial confidence and accuracy gains varies substantially:
GPQA shows peak improvements at moderate confidence levels, with diminishing returns at higher confidence
22

---

MMLU and MMLU Pro exhibit relatively flat improvement patterns with the less confident questions getting more improvements. Nevertheless, this may caused by the low initial accuracy.
TriviaQA demonstrates erratic improvement fluctuations regardless of initial confidence
- Pha Accuraty
& Arouracy
intt Accuraty
+ Rha Accurety
Aecurryy
Inka Accuurary
1.0
1.0
Acorrry
0.6
0.6
0.4
0.4
0.0
0.0
6.32
0.8
0.59
0.85
0.65
0.90
0.53
0.77
0.55
0.99
Intial Confidence
Confidence
(a) GPQA confidence vs. accuracy
(b) MMLU confidence vs. accuracy
- Fia Accuraty
& Accuracy
Intal Accuraty
- FIne Accurety
Accurayy
Inka Accurary
1.0
1.0
Accurecy
0.6
0.6
0.4
0.4
0.0
6x2
0.45
0.87
0.30
6:79
Inttial Confidence
Confidence
(c) MML.U Pro confidence vs. accuracy
(d) TriviaQA confidence vs. accuracy
Figure 17: Confidence vs. accuracy across different datasets using GPT-4.1 mini as feedback model and Llama-4-Scout as the solver.
F Analysis of data familiarity and RIGID THINKING
After analyzing data familiarity using answer frequency in the PopQA dataset, found clear correlation between model performance and frequency of answer words the training data. However, surface-leve frequency may not fully capture model's true familiarity with content, it fails to account for context quality, semantic relationships, and other factors affecting knowledge acquisition during pre-training.
To better capture actual familiarity, we examine a more direct behavioral signal called in-domain performance: the model's accuracy in answering questions, measured using 100 generations per question with Llama-3.3. This behavioral familiarity metric reflects the cumulative effect of all factors contributing to the model's internalized knowledge.
Figure 18 illustrates the in-domain performance of Llama-3.3 across four benchmarks- GPQA. TriviaQA, 5-digit multiplication, and MMLU Pro. We bucket questions based on the model' 's initial accuracy and report both initial and final accuracy after iterative feedback. While the model shows improvement across all buckets, questions with higher behavioral familiarity (higher initial accuracy) consistently achieve higher final accuracy as well. This suggests that behavioral familiarity is more informative predictor of both current performance and improvement potential than answer frequency alone. Nevertheless, we still cannot obverse any consistent patterns across all these datasets in the initial vs. final accuracy.
23

---

Busteed
(a) GPQA in-domain performance
(b) TriviaQA in-domain performance
(c) S-digit multiplication in-domain
MMLU Pro in-domain performance
Figure 18: In-domain accuracy of Llama-3.3 across four benchmark tasks.
G Analysis of reasoning complexity and FEEDBACK FRICTION
To investigate whether the model's improvement over iterations correlates with question difficulty
the reasoning complexity, we compare the performance of Llama-4-Scout multiplication tasks: 5-digit and 6-digit problems. Unlike prior datasets, which lack clear separability in difficulty levels, these tasks were manually constructed with 450 questions each ensure well-defined difference in complexity.
synthesized
The initial accuracy of L-Sout4-Scout is 2.2% on 5-digit multiplication and 0.889% 6-digit multiplication. Interestingly, while the 6-digit task is objectively more difficult, we observe greater improvement across iterations compared the 5-digit task. One possible explanation is that more difficult tasks offer more room for feedback-driven correction because solver model has less initial knowledge about how to solve them. However, also observe yield higher final accuracy, suggesting that the relationship between task complexity and feedback effectiveness is non-monotonic and influenced by additional factors.
where simpler questions
H Analysis of model type and FEEDBACK FRICTION
To better understand the overlap in failure cases among Llama-3.3, Llama-4-Scout and Llama-4- Maverick, compared their incorrect predictions across several benchmark datasets. Specifically, we report the number of shared mistakes between each pair of models, the number of questions all three models got wrong, the total number of unique mistakes (union), and the Overlap Ratio, defined as the proportion all-three common errors to the total number of distinct emror..
The Overlap Ratio offers normalized measure of agreement in model failures. As shown in Table 2 AIME shows the highest overlap (35.7%), suggesting subset of examples that all three models consistently struggle with. Conversely, datasets such as GPQA and 5-digit Multiplication exhibit minimal overlap (6.9% and 0.7%, respectively). indicating that the models tend to fail on different questions.
These findings suggest that model failures are often idiosyneratic rather than being concentrated around universally difficult subset of examples. The relatively low overlap across datasets highlights the challenge of achieving robust self-correction: errors are not easily attributable a common set of pitfalls, but rather reflect distinct weaknesses in each model's reasoning and generalization.

---

Accuracy Improvement Over Iterations
6.digit 5-digit
80
60
Accurary (86)
40
Iteration
Figure 19: Comparison the Improvement for 5-digit and 6-digit multiplication with GPT-4.1 mini
as feedback model
Dataset
L3.3-Scout
L3.3-Maverick
Scout-Maverick
All-Three
Union
Overlap Ratio
AIME
8
5
5
14
0.357
TriviaQA
22
16
28
14
105
0,133
MATH-500
18
14
11
9
64
0.141
MMLU
15
12
19
11
55
0,200
MMLU Pro
49
40
43
30
163
0.184
GPQA
3
2
2
0.069
5-digit Mult.
21
1
141
0.007
Table 2: Pairwise and three-way common failure cases among Llama-3.3, Llama-4-Scout, and Llama-
4-M M-Mverickk across datasets. Overlap Ratio is computed as the number of questions all three models failed on divided by the union of all distinct failures.
I Details of testing model's self-perception of FEEDBACK FRICTION
Figure 20 shows the prompts we used to probe whether the model understand the given feedback and whether it's willing change its belief.
Feedback Understanding Prospt:
Question: Coriginal_question)) question)
previous answer: (previous_respomee
Feedback "previious_ (eedisnk__fedsackk
underatand this feedback? Please with 'Yes' "No" and briefly explain why.
Update Proept:
Question: (ortginal_ Lquestion)
previous (previous_ response)
Feedback received: "[previous_ Teadback)"
Have you updated your belief understanding about this probles? Please answer with 'Yes" or "Mo" and brieffly explain what changed.
Figure 20: An example of Llama-4-Son 4-couut explicitly acknowledging changing its belief while still making the same mistake in the next attempt.
Figure 21 shows an example where LlSou- explicitly acknowledges that "the actual boundary is defined by the perpendicular bisectors of the segments connecting O to the vertices, which can result an oblique line" and claims its approach has been updated. Yet in its second attempt, it still describes the region as forming "a square with side length 1/2 min(AB, AD)" and arrives at the same incorrect answer of 1/4. This demonstrates the model can articulate understanding of the feedback while its actual problem-solving behavior remains unchanged. The correct answer is 1/2.
25

---

Original Queatien:
Rectangle ABCD has center o and 4B/AD=k. point is randoxly choson from the interior of rectangle ABCD. Ahat is the probabllity closer than of four vertices?
Modellss First Atteepti
The region closer to o than to any verter is sealler rectangla with side lengths 1/2 All and 1/2 AD, This mallarr rectengla has area of 1/2 AB 1/2 AD 174 48 a AD. The area of the entire rectangle ABCO is AB AD. This, the probability is 1/4 AB tinal
Feedback Provided:
The original solution sesumes that the regton closer to the center than to any of the vertices forns a smaller rectangle aligged with the originel rectangle, with side lengthe exectly half of the original, Howwver, the locus of points closer to than by perpenditular bisector aides; rether, assunption that ssaller region is exactly vertex, which is generally oblique line. one-quartor of the whole area becouse it is rectangle of half the original sides is not correst.
is not straight boundary parallel segrent connecting
perticuler
Model's Statesent Belief Updates underatarding
feedback provided. Initially, incorrectly closer the ownter then any verter foried sallar rectangla with sides exectly half lengtha the or orininal rectangle. Honaver, the Teadback highlighted that this assurption is incorrect and that the actual boundary is defired by the perpendicular disectors of the segnents connecting 0 the vertizes, which can result in an oblique line. This realization, combined with the importance has updated sy approach solving probles.
problem changed based
region
iewrideinn symmtry. transforent ions, and nore nanceed gensetric partitioning of the rectangle,
Hodel's Second Attempt:
By drawing these perpendicular bisectors, we can dee that the region closer to O than to any vertex in this sealler restangle forms with side length 1/2 min(AS, The of this sealler rectangle 40).
Mll AD, and the of the regton probability aspect,
win(MB approach by understanding perpendioular bisectors, which effectively creates a snaller square (whon considering the clesest region to 0) within each of the four smaller rectangles. Eiven the symeery and the nature of the divisson, the probsbility simolifies to the ratio of the area where points closer
considering
setup foouxing or vertices
division the rectangle into regions cleaer
the total area of the rectangle. This simplification leads recopnizing probabalsty does not depend aspect ratio rather diviston. The Firal answer 1x: 1/4
geonetrie properties the restangle
An example of Llama-4-Scou explicitly acknowledging changing its belief while still making the same mistake in the next attempt.
Figure 21:
Full results of semantic entropy
presents the complete self-correction analysis across all five benchmark tasks (5 digits multiplication, GPQA, MATH, MMLU-Pro, and TriviaQA) for both Llama-3.3-70B and L-ama-4- Scout. Each subplot shows initial accuracy (blue), final accuracy (orange), and absolute improvement rate (green) as functions of semantic entropy with bucket size 0.2. Regions with fewer than 10 samples are marked with red shading and dashed boundaries to indicate unreliable data, with sample counts displayed in boxes above each plot.
Figure 22
300 20 BU
100
15:0 05
Figure 22: Full results of semantic entropy over five benchmark tasks and two models.
26

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction (pages 1-2) clearly state the paper's main claims, including the identification of phenomenon called FEEDBACK FRICTION where LLMs show resistance to incorporating external feedback despite multiple iterations. The claims match the experimental results shown in the paper, particularly in Figures I and 3, which demonstrate models consistently plateauing below their theoretical accuracy ceiling. The limitations also mentioned, they note that even with their best strategies, models still fail to reach target accuracy.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize other settings.
is fine to include aspirational goals as motivation as long as clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: The paper discusses limitations of their work in several places. The authors acknowledge that their sampling strategies only mitigate but do not eliminate FEEDBACK FRICTION (p.7, lines 249-252). They also note that despite investigating various hypotheses, they couldn't 't fully explain the causes model stubbornness (p.8, lines 284-284). The synthetic tasks section (p.4) acknowledges limitations in task selection by explaining their choice of objective tasks over subjective ones to ensure reliable evaluation.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "LLimitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g. if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-ted system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimess.
27

---

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [NA]
The paper is primarily empirical and does not include formal theoretical results proofs that would require mathematical verification. While they develop the concept of FEEDBACK FRICTION, this is demonstrated through experimental evidence rather than mathematical theorems.
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: The paper provides sufficient detail reproduce the main experimental results. Section 3.1 (p.4-5) describes the experimental setup, including tasks, metrics, models, and inference settings. The authors explain their continual self-improvement framework (p.3), feedback mechanisms (p.3-4), and sampling strategies (p.7). They specify using temperature for deterministic outputs, vllm for inference, and GPT-4.1 mini for generating strong-model feedback.
Guidelines:
The answer NA means that the paper does not include experiments.
the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
28

---

While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g.. a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification:
The author mentioned they will share the code and data used for this paper upon paper acceptance, and they'v 've shared an anonymized repo
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cr cc/
Aoboliccgudddss/odeeubbiiii oot.ssbbiisssooooooooooooooo for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/n.ps..cc/pbllc///uuu /
sisoooiccy) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended to paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification: The paper specifies the experimental details needed to understand the results. Section 3.1 (p.4-5) describes the tasks used (AIME 2024, MATH-500, TriviaQA, PopQA, MMLU, MMLU Pro, GPQA, and two synthetic tasks), the models tested (LLaMA-3.3 --mma---cout-17B--11e
70B Instruct, inference settings (temperature 0, vllm, chat templates). They also explain how they sample 10% of data for certain datasets and why this is representative.
and
29

---

Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: The paper reports error bars in the analysis section.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
The paper provides information about the compute resources used in the experimental setup section, where they mentioned all their experiments are done on single
H100 with 8 GPUs.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of httppp////.arrpp..ccppp////aaaaa
30

---

Answer: [Yes]
Justification: The research in this paper appears to conform with the NeurlPS Code of Ethics. The paper studies LLMs' ability to incorporate feedback, which is foundational research without apparent ethical concerns. The authors publicly available models and benchmark datasets, and their experimental methodology doesn't involve deception, harmful content generation, human subjects that would raise ethical issues.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: The paper discusses the broader societal impacts of the work in the introduction and conclusion.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of MI..
Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
The paper does release new models, datasets, tools that would pose risks for misuse. The research uses existing LLMs and benchmark datasets to study their feedback incorporation capabilities. The authors don't create release content that would require safeguards against misuse.
Justification:
The answer NA means that the paper poses such risks.
31

---

Released models that have high risk for misuse or dual-use use should be released with necessary safeguards to allow for controlled use of the model. for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (c.g.. code, data, models), used in the paper, properly credited and the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [NA]
Justification: The paper references the original sources of the models (such as Meta AI for Llama models and OpenAl for GPT-4. mini). Since these models and the benchmark have permissive licenses and terms of use that easily allow for third-party implementation, explicit mention of licensing details is unnecessary.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g.. CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com /dtasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes)
Justification: The paper introduces two synthetic digit multiplication tasks, and details will be provided in the appendix on those two new tasks.
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details the dataset/code/moodll as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
32

---

Answer: [NA]
Justification: The paper does not involve crowdsourcing or research with human subjects. The experimental methodology is entirely computational, using LLMs and benchmark datasets without human participants or annotations.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:
Answer: [NA]
The paper does not involve human subjects research that would require IRB approval. All experiments conducted using computational models and pre-existing datasets, with no human participants involved in the study.
Justification:
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [Yes]
Justification: The paper appropriately describes the LLMs used in the research, which are central to the study's methodology. Section 3.1 (p.4-5) provides details about the models used (LLaMA-3.. 70B Instruct, -aamaa---Soout11B... 1B-1 1aima---Maverick-177 1--ec----BB2220 1-EE- Instruct-FP8, and GPT-4.1 mini). Since studying LLMs' ability to incorporate feedback is the core focus of the paper, these models and their configurations are thoroughly documented.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer LLM policy for what should or should not be described.
33


Reviews:
[{"review_id": "B6PByZCQVj", "reviewer": "Reviewer_8FxN", "summary": "This paper investigated LLMs’ ability to incorporate feedback. This is done by designing a controlled experimental environment, where for each problem, the model attempts a solution, then a feedback generator with access to near-complete ground-truth answers produces targeted feedback, after which the model tries again. The paper showed that even under these near-ideal conditions, solver models consistently show resistance to feedback.", "questions": "Please look at the weakness section above.", "limitations": "yes", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths: \n\n- The paper discussed the feedback resistance from the model side as a limitation rather than the quality of the feedback.\n\n- The paper shows improvement with increased temperature with rejection sampling.\n\n\nWeaknesses:\n\n- There has been a lot of existing work that shows the model doesn’t respond to the feedback and they bring up some hypotheses like model size, etc. Here the paper shows consistent results on not leveraging feedback after 2-3 iterations which is interesting but surprising as well. The comparison with existing work analysis is missing.\n\n- More experiments need to be run to get a conclusion here. For example, other LLM models with different sizes. It might also be the fact that how you prompt the model with the feedback.  Sometimes clarifying the feedback with an example is helpful. \n\n- The paper should compare the results against the baseline where feedback doesn’t come from the  ground-truth answers.", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for recognizing that our paper discusses feedback resistance as a model-side limitation rather than a feedback quality issue, and that we show improvement with increased temperature and rejection sampling. We have tested larger models that confirm RIGID THINKING persists across scales and experimented with various prompt variations. Below we provide detailed responses to your specific concerns:\n\n---\n\n> There are existing works that show models don't respond to feedback with hypotheses like model size. Comparison with existing work analysis is missing.\n\nThank you for pointing this out. We would greatly appreciate it if you could share references to those papers, as we'd love to read them and properly compare our findings with prior work. Understanding how our results relate to existing hypotheses about model size and feedback resistance would strengthen our analysis.\n\n---\n\n> More experiments are needed for drawing conclusions: need to test more models with different model sizes. Prompt variations are also needed to be tested (e.g. clarifying feedback with examples)\n\n**TL;DR: We tested larger models (Claude 3.7) and various prompt variations - RIGID THINKING persists in both cases.**\n\nYou're absolutely right about testing more models. While this NeurIPS submission focuses on the Llama family, we have since tested larger models like Claude 3.7 Sonnet (both with and without Extended Thinking). Table 2 shows the initial accuracy, final accuracy after 10 feedback iterations, and theoretical target accuracy for each model. Despite Claude models achieving significantly higher performance than Llama models, they still plateau well below their target accuracy, demonstrating that RIGID THINKING persists across model scales. We'd definitely welcome suggestions for additional models to test.\n\n| Dataset                    | Solver Model                           | Initial (%) | Final (%) | Target (%) | Δ (Target - Final) |\n|----------------------------|----------------------------------------|------------:|----------:|-----------:|-------------------:|\n| Hexadecimal 5-Digit Mult   | Claude 3.7 Sonnet                      |        1.0 |      12.0 |       86.0 |               74.0 |\n| Hexadecimal 5-Digit Mult   | Claude 3.7 Sonnet (Extended Thinking)  |        8.0 |      16.0 |       86.0 |               70.0 |\n| AIME 2024                  | Claude 3.7 Sonnet                      |       33.3 |      76.7 |       99.0 |               22.3 |\n| AIME 2024                  | Claude 3.7 Sonnet (Extended Thinking)  |       50.0 |      90.0 |       99.0 |                9.0 |\n| PopQA                      | Claude 3.7 Sonnet                      |       55.8 |      88.8 |       98.0 |                9.2 |\n| PopQA                      | Claude 3.7 Sonnet (Extended Thinking)  |       58.1 |      92.3 |       98.0 |                5.7 |\n| MMLU Pro                   | Claude 3.7 Sonnet                      |       84.2 |      90.8 |       99.2 |                8.4 |\n| MMLU Pro                   | Claude 3.7 Sonnet (Extended Thinking)  |       83.3 |      96.7 |       99.2 |                2.5 |\n| MATH-500                   | Claude 3.7 Sonnet                      |       78.0 |      94.0 |       99.0 |                5.0 |\n| MATH-500                   | Claude 3.7 Sonnet (Extended Thinking)  |       90.0 |      96.0 |       99.0 |                3.0 |\n| 5-Digit Multiplication     | Claude 3.7 Sonnet                      |       43.6 |      96.0 |       99.0 |                3.0 |\n| 5-Digit Multiplication     | Claude 3.7 Sonnet (Extended Thinking)  |       46.6 |      97.0 |       99.0 |                2.0 |\n| GPQA                       | Claude 3.7 Sonnet                      |       66.7 |      98.5 |       99.6 |                1.1 |\n| GPQA                       | Claude 3.7 Sonnet (Extended Thinking)  |       72.2 |      97.0 |       99.6 |                2.6 |\n| MMLU                       | Claude 3.7 Sonnet                      |       87.1 |      97.1 |       99.9 |                2.8 |\n| MMLU                       | Claude 3.7 Sonnet (Extended Thinking)  |       86.4 |      99.3 |       99.9 |                0.6 |\n| TriviaQA                   | Claude 3.7 Sonnet                      |       86.1 |      98.7 |       99.7 |                1.0 |\n| TriviaQA                   | Claude 3.7 Sonnet (Extended Thinking)  |       88.1 |      99.3 |       99.7 |                0.4 |\n\n**Table 1:** Performance comparison of Claude 3.7 models across all nine datasets, sorted by average performance gap per dataset. Despite achieving high final accuracies, models consistently plateau below their theoretical targets across all tasks.\n\nRegarding prompt variations, we experimented with several approaches: we tested more explicit feedback prompts (e.g., \"Your error is in step 3 where you incorrectly calculated X\"), step-by-step guidance prompts, and example-based feedback where we provided similar problem-solution pairs alongside the correction. Following your suggestion about clarifying feedback with examples, we also tested providing worked examples of similar problems within the feedback itself. Unfortunately, across all these variations, we observed the same fundamental pattern - models still plateau well below their theoretical maximum accuracy.\n\n---\n\n> The paper should compare the results against the baseline where feedback doesn't come from the ground-truth answers.\n\nThat's an interesting suggestion. Our focus was on establishing an upper bound scenario - if models can't effectively use near-perfect feedback, it suggests fundamental limitations in feedback incorporation. Could you elaborate on what insights you think a weaker feedback baseline would provide? We're curious about your reasoning and we’re more than happy to conduct new experiments, as it might reveal additional aspects of this phenomenon we haven't considered."}, {"review_id": "tBvXOGNWY3", "reviewer": "Reviewer_jdPe", "summary": "This paper investigates whether large language models can fully integrate perfect, targeted feedback to correct their own mistakes through multiple iterations. Despite using strong feedback generators with access to ground truth, the authors find that models consistently plateau below their theoretical maximum accuracy — a phenomenon they call RIGID THINKING. The study systematically analyzes this issue across various reasoning tasks and shows that simple sampling or rejection strategies only partly mitigate the problem.", "questions": "Is there any correlation between model size and resistance to feedback — would even larger models behave differently?", "limitations": "The study is limited to instruction-tuned LLMs with deterministic decoding setups, which may not reflect real-world use cases involving more diverse sampling. It also doesn’t test feedback loops in more subjective tasks like summarization or dialogue.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\nThe paper addresses a crucial and underexplored limitation of LLMs’ self-improvement ability in a rigorous, controlled setting. It combines careful experimental design with diverse tasks and strong baseline models. The concept of RIGID THINKING is clearly defined, well-supported with evidence, and opens an important direction for future work.\n\nWeaknesses\nThe paper stops short of offering a concrete solution to overcome feedback resistance, beyond simple sampling tricks. It does not deeply connect the observed limitations with potential architectural or training modifications. Some of the tasks (e.g., synthetic multiplication) are very narrow, so broader generalization to more open-ended tasks remains uncertain.", "was_revised": true, "final_justification": "This is a good paper with solid results. The authors further provide detailed rebuttals that addressed my concerns. Thus, I will keep my rating.", "rebuttal": "We thank the reviewer for recognizing that our paper addresses a crucial and underexplored limitation with rigorous experimental design, and that the concept of RIGID THINKING is clearly defined and well-supported. We have conducted pilot training experiments on smaller models, tested larger models that confirm the phenomenon persists across scales, and explored diverse sampling approaches. Below we provide detailed responses to your specific concerns:\n\n---\n\n> The paper does not deeply connect the observed limitations with potential architectural or training modifications, beyond simple sampling tricks.\n\n**TL;DR: We conducted pilot training experiments (SFT/DPO) on smaller models but found no improvement in feedback resistance.**\n\nIn the current submission, we did not perform training interventions such as SFT or DPO on our target models due to resource constraints—the smallest model, LLaMA‑3.3‑70B, requires lots of GPU‑hours for a single fine‑tuning run, which is infeasible within our compute budget.\n\nTo verify whether these interventions could improve feedback integration, we conducted pilot experiments on a smaller model, **LLaMA‑3.1‑8B‑Instruct**, using:\n- **SFT**: 2 000 successful correction trajectories  \n- **DPO**: 4 000 preference pairs (2 000 positive / 2 000 negative)  \n- **SFT steps**: 5 000 updates, lr = 1 × 10⁻6, batch = 4  \n- **DPO steps**: 10 000 updates, same lr & batch  \n- **Evaluation**: Feedback Resistance (FR) after 10 feedback iterations on MATH dev\n\n| Experiment                        | FR (%) before | FR (%) after | Δ FR   |\n|-----------------------------------|--------------:|-------------:|-------:|\n| Baseline (no fine‑tuning)         |          18.0 |            – |      – |\n| SFT (2 000 examples, 5 000 steps) |          18.2 |         19.8 |  +1.6  |\n| DPO (4 000 pairs, 10 000 steps)   |          18.1 |         18.2 |  +0.1  |\n\n**Table 1:** Impact of SFT and DPO on Feedback Resistance in LLaMA‑3.1‑8B‑Instruct.\n\nThese interventions didn’t result in any FR reduction, we hypothesis this might be related to the findings from Kumar et al. (2024) (Training Language Models to Self-Correct via Reinforcement Learning), who demonstrate that offline SFT methods suffer from distribution shift and behavior collapse, limiting their ability to improve self‑correction. We will keep exploring more integrated training paradigms—such as multi‑turn reinforcement learning that incorporates feedback signals in real time—to enable sustained iterative improvement beyond simple fine‑tuning.  \n\n---\n\n> Some of the tasks (e.g., synthetic multiplication) are very narrow, so broader generalization to more open-ended and subjective tasks (e.g. summarization) remains uncertain.\n\nThanks for this feedback! We tested 9 different tasks to capture diverse reasoning patterns, and we'd welcome suggestions for additional tasks you think would be valuable. Regarding more open-ended and subjective tasks like summarization, as we noted in line 124, we deliberately chose objective tasks to avoid reward hacking and reliability issues during evaluation. If you have ideas for how to reliably evaluate feedback incorporation in subjective tasks without these issues, we'd be very interested to hear them.\n\n---\n\n> Is there any correlation between model size and resistance to feedback — would even larger models behave differently?\n\n**TL;DR: We tested larger models (Claude 3.7) and found RIGID THINKING persists across model scales.**\n\nWhile this NeurIPS submission focuses on the Llama family, we have since tested larger models like Claude 3.7 Sonnet (both with and without Extended Thinking). Table 2 shows the initial accuracy, final accuracy after 10 feedback iterations, and theoretical target accuracy for each model. Despite Claude models achieving significantly higher performance than Llama models, on several datasets, they still plateau well below their target accuracy, demonstrating that RIGID THINKING persists across model scales. We'd definitely welcome suggestions for additional models to test.\n\n| Dataset                    | Solver Model                           | Initial (%) | Final (%) | Target (%) | Δ (Target - Final) |\n|----------------------------|----------------------------------------|------------:|----------:|-----------:|-------------------:|\n| Hexadecimal 5-Digit Mult   | Claude 3.7 Sonnet                      |        1.0 |      12.0 |       86.0 |               74.0 |\n| Hexadecimal 5-Digit Mult   | Claude 3.7 Sonnet (Extended Thinking)  |        8.0 |      16.0 |       86.0 |               70.0 |\n| AIME 2024                  | Claude 3.7 Sonnet                      |       33.3 |      76.7 |       99.0 |               22.3 |\n| AIME 2024                  | Claude 3.7 Sonnet (Extended Thinking)  |       50.0 |      90.0 |       99.0 |                9.0 |\n| PopQA                      | Claude 3.7 Sonnet                      |       55.8 |      88.8 |       98.0 |                9.2 |\n| PopQA                      | Claude 3.7 Sonnet (Extended Thinking)  |       58.1 |      92.3 |       98.0 |                5.7 |\n| MMLU Pro                   | Claude 3.7 Sonnet                      |       84.2 |      90.8 |       99.2 |                8.4 |\n| MMLU Pro                   | Claude 3.7 Sonnet (Extended Thinking)  |       83.3 |      96.7 |       99.2 |                2.5 |\n| MATH-500                   | Claude 3.7 Sonnet                      |       78.0 |      94.0 |       99.0 |                5.0 |\n| MATH-500                   | Claude 3.7 Sonnet (Extended Thinking)  |       90.0 |      96.0 |       99.0 |                3.0 |\n| 5-Digit Multiplication     | Claude 3.7 Sonnet                      |       43.6 |      96.0 |       99.0 |                3.0 |\n| 5-Digit Multiplication     | Claude 3.7 Sonnet (Extended Thinking)  |       46.6 |      97.0 |       99.0 |                2.0 |\n| GPQA                       | Claude 3.7 Sonnet                      |       66.7 |      98.5 |       99.6 |                1.1 |\n| GPQA                       | Claude 3.7 Sonnet (Extended Thinking)  |       72.2 |      97.0 |       99.6 |                2.6 |\n| MMLU                       | Claude 3.7 Sonnet                      |       87.1 |      97.1 |       99.9 |                2.8 |\n| MMLU                       | Claude 3.7 Sonnet (Extended Thinking)  |       86.4 |      99.3 |       99.9 |                0.6 |\n| TriviaQA                   | Claude 3.7 Sonnet                      |       86.1 |      98.7 |       99.7 |                1.0 |\n| TriviaQA                   | Claude 3.7 Sonnet (Extended Thinking)  |       88.1 |      99.3 |       99.7 |                0.4 |\n\n**Table 2:** Performance comparison of Claude 3.7 models across all nine datasets, sorted by average performance gap per dataset. Despite achieving high final accuracies, models consistently plateau below their theoretical targets across all tasks.\n\n---\n\n> The study is limited to instruction-tuned LLMs with deterministic decoding setups, which may not reflect real-world use cases involving more diverse sampling.\n\n**TL;DR: We used deterministic decoding for reproducibility but tested diverse sampling in §4.2 - the limitation persists.**\n\nYou're absolutely right about real-world usage patterns. We'd be happy to test other temperature settings you think would be particularly informative.\nRegarding our temperature choices: we used temperature=0 for most experiments to ensure deterministic, reproducible results crucial for controlled experimentation. However, we did conduct experiments with higher temperatures in §4.2, exploring progressive temperature increases and rejection sampling as potential mitigation strategies. The fact that even these more diverse sampling approaches couldn't fully overcome the plateau suggests the limitation runs deeper than just deterministic decoding."}, {"review_id": "BMsnNVQUG7", "reviewer": "Reviewer_kRQD", "summary": "The paper develops a controlled solver <--> feedback-generator loop, showing that even with near-perfect feedback, state-of-the-art Llama-family models plateau well below the theoretical maximum accuracy. This phenomenon is termed rigid thinking. The work explores potential root causes, offers two sampling-based remedies, and identifies feedback resistance rather than poor feedback quality as the primary limitation.", "questions": "See weaknesses", "limitations": "None", "rating": 3, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "## Strengths\n\n**Breadth of benchmarks:** The inclusion of nine diverse tasks, such as counter-factual arithmetic, convincingly illustrates that rigid thinking is not limited to a specific domain.\n\n**Thorough error analysis:** The study effectively distinguishes between solver deficiencies and feedback quality issues. The automated categorization, cross-checked by human annotators with a high agreement rate (96%), provides strong evidence supporting the conclusion about feedback resistance.\n\n**Clarity:** The motivation and setting were clear, even for readers less familiar with the topic. Methods and results were presented understandably.\n\n## Weaknesses\n\n**Model selection**: Given that Llama-4 models are not reasoning models, would it be possible to evaluate models specifically designed for reasoning tasks? It would be interesting to see if reasoning-oriented models might incorporate feedback more effectively, particularly on challenging synthetic tasks like Hex, where the accuracy observed is currently very low.\n\n**Temperature setting:** The use of temp=0 for sampling raises concerns, as it generally isn’t recommended and might produce misleading outcomes for long ouputs. Could the authors clarify their rationale for choosing temp=0 despite recommendations from sources like the Qwen3 Huggingface page, which explicitly suggests higher temperature values (e.g., 0.6-0.7)? A comparison using recommended temperatures might strengthen the analysis.\n\n**Prompt structure:** Currently, the entire interaction history is presented in one prompt per model iteration. Wouldn't a multi-turn interaction approach, where history naturally emerges through alternating and prompts, be more realistic and potentially more effective for chat/reasoning models? Could the authors also report accuracy outcomes using such a multi-turn interaction to compare?\n\n**Proprietary reasoning models:** Have the authors considered evaluating proprietary reasoning-focused models such as o4-mini to see if they benefit more significantly from feedback? Even a limited evaluation on a few datasets the authors identify as particularly challenging would provide whether insights generalize to recent frontier models.\n\n**Feedback correctness (Section 4.1, Table 1)**: Specifically for Llama-4-Maverick on GPQA, the feedback resistance (FR) is reported at 85.7%, suggesting approximately 14.3% incorrect feedback. Given that Llama-4-Maverick achieves ~95% accuracy, could the authors clarify how improvement in the remaining ~5% accuracy is expected if around 15% of feedback might be incorrect? Is incorrect feedback primarily an issue for low performance ranges, or does it persist even at high accuracy levels?\n\n**Baseline comparison (Fig. 5):** Could a baseline showing \"no change\" (similar to lines plotted in Figure 3) be included in Figure 5 for clearer comparison?", "was_revised": true, "final_justification": "The rebuttal addressed some of my concerns (thorough evaluation on Claude-- a reasoning model) but majority of the concerns remain. Overall, it is clear that rigid thinking is an effect, however what causes the effect is unclear -- additionally lots of critical, new experiments were done during rebuttal stage. I would like to see a comprehensive story -- I would side with another round of submission to include all new experiments into a narrative.", "rebuttal": "We thank the reviewer for recognizing the breadth of our benchmark evaluation, the thoroughness of our error analysis, and the clarity of our presentation. We have conducted additional experiments with reasoning models (Claude 3.7) that confirm RIGID THINKING persists across model architectures, clarified our temperature settings which were model-appropriate rather than uniformly zero, and tested multi-turn interactions which show the same plateauing behavior. Below we provide detailed responses to your specific concerns:\n\n---\n\n> Model selection (no proprietary reasoning models has been tested in this submission)\n\n**TL;DR: We tested Claude 3.7 reasoning models and found the same RIGID THINKING behavior persists.**\n\n\nAlthough we did not evaluate reasoning models in our NeurIPS submission, we have since tested proprietary reasoning models such as Claude 3.7 Sonnet and Claude 3.7 Sonnet with Extended Thinking (on‑par with o4‑mini in reasoning capabilities). Table 1 reports shows the initial accuracy, final accuracy after 10 feedback iterations, and theoretical target accuracy for each model. Importantly, even these sophisticated reasoning models exhibit the same fundamental plateauing behavior we term **RIGID THINKING**, suggesting this limitation persists across different model architectures and reasoning capabilities.\n\n\n| Dataset                    | Solver Model                           | Initial (%) | Final (%) | Target (%) | Δ (Target - Final) |\n|----------------------------|----------------------------------------|------------:|----------:|-----------:|-------------------:|\n| Hexadecimal 5-Digit Mult   | Claude 3.7 Sonnet                      |        1.0 |      12.0 |       86.0 |               74.0 |\n| Hexadecimal 5-Digit Mult   | Claude 3.7 Sonnet (Extended Thinking)  |        8.0 |      16.0 |       86.0 |               70.0 |\n| AIME 2024                  | Claude 3.7 Sonnet                      |       33.3 |      76.7 |       99.0 |               22.3 |\n| AIME 2024                  | Claude 3.7 Sonnet (Extended Thinking)  |       50.0 |      90.0 |       99.0 |                9.0 |\n| PopQA                      | Claude 3.7 Sonnet                      |       55.8 |      88.8 |       98.0 |                9.2 |\n| PopQA                      | Claude 3.7 Sonnet (Extended Thinking)  |       58.1 |      92.3 |       98.0 |                5.7 |\n| MMLU Pro                   | Claude 3.7 Sonnet                      |       84.2 |      90.8 |       99.2 |                8.4 |\n| MMLU Pro                   | Claude 3.7 Sonnet (Extended Thinking)  |       83.3 |      96.7 |       99.2 |                2.5 |\n| MATH-500                   | Claude 3.7 Sonnet                      |       78.0 |      94.0 |       99.0 |                5.0 |\n| MATH-500                   | Claude 3.7 Sonnet (Extended Thinking)  |       90.0 |      96.0 |       99.0 |                3.0 |\n| 5-Digit Multiplication     | Claude 3.7 Sonnet                      |       43.6 |      96.0 |       99.0 |                3.0 |\n| 5-Digit Multiplication     | Claude 3.7 Sonnet (Extended Thinking)  |       46.6 |      97.0 |       99.0 |                2.0 |\n| GPQA                       | Claude 3.7 Sonnet                      |       66.7 |      98.5 |       99.6 |                1.1 |\n| GPQA                       | Claude 3.7 Sonnet (Extended Thinking)  |       72.2 |      97.0 |       99.6 |                2.6 |\n| MMLU                       | Claude 3.7 Sonnet                      |       87.1 |      97.1 |       99.9 |                2.8 |\n| MMLU                       | Claude 3.7 Sonnet (Extended Thinking)  |       86.4 |      99.3 |       99.9 |                0.6 |\n| TriviaQA                   | Claude 3.7 Sonnet                      |       86.1 |      98.7 |       99.7 |                1.0 |\n| TriviaQA                   | Claude 3.7 Sonnet (Extended Thinking)  |       88.1 |      99.3 |       99.7 |                0.4 |\n\n**Table 1:** Performance comparison of Claude 3.7 models across all nine datasets, sorted by average performance gap per dataset. Despite achieving high final accuracies, models consistently plateau below their theoretical targets across all tasks.\n\n---\n\n> Temperature setting (the temperature of 0 is chosen throughout the analysis, even for Qwen models)\n\n**TL;DR: In the current submission, we used appropriate temperatures for each model, not uniformly temperature=0.**\n\nWe want to clarify our temperature choices, which were not uniformly set to 0 as the reviewer suggests:\nWe tested Qwen3-235B-A22B with temperature=0.7 (following the suggested temperature from their documentation), but found that when reasoning mode is enabled, it still enters repetitive generation loops a lot, and it’s not very good at following instructions (community also observed a similar phenomenon), which leads to our final decision to not use it.\n\nIn addition, in our run with Claude 3.7 with Extended Thinking, we use temperature=1 as suggested by Anthropic.\nWe choose temperature=0 for most of our testing to ensure deterministic results crucial for controlled experimentation. However, we did conduct experiments with higher temperatures in later sections (§4.2), where we explore progressive temperature increases and rejection sampling as potential mitigation strategies.\n\n---\n\n> Prompt structure (can we make the solver/feedback generation into a multi-turn setting)\n\n**TL;DR: Multi-turn interactions show about the same performance as single-prompt approach.**\n\nTurning feedback into a multi-turn conversational style is a very good suggestion! We have tried to reframe the conversation history into a multi-turn setting by inserting <user> and <assistant> tags, and found the performance to be basically the same - LLMs still couldn't reach perfect accuracy with that approach. However, we acknowledge this could be explored more systematically as future work.\n\n---\n\n> Feedback correctness (does incorrect feedback only occur for low-accuracy tasks? What will happen if the feedback is correct in some settings)\n\n**TL;DR: Incorrect feedback occurs at all accuracy levels, and better feedback wouldn't solve the core problem.**\n\nRegarding whether incorrect feedback is primarily an issue for low or high performance ranges: We have looked into the logs, and this is an issue for all accuracy levels, not just low performance ranges.\nFor the specific case of Llama-4-Maverick on GPQA with 85.7% feedback resistance: If the remaining 5% of incorrect cases received better feedback, the error rate would be approximately 5% × 0.857 ≈ 4.3%, which is still below perfect performance. Also, it's important to note that the \"feedback quality\" issues don't necessarily mean the feedback is objectively wrong - rather, it's more that the feedback is not targeted enough to address the specific errors made by that model instance.\n\n---\n\n> Add baseline comparison in Fig. 5\n\nThanks for your suggestion! We will add a \"no change\" baseline line for clearer comparison.\n\n---\n\n> Rating: 2: Reject: For instance, a paper with technical flaws, weak evaluation, inadequate reproducibility and incompletely addressed ethical considerations.\n\nWe would sincerely appreciate further elaboration, as we are confident that any such concerns can be addressed in this rebuttal period."}, {"review_id": "jKJWOwiYRv", "reviewer": "Reviewer_xvok", "summary": "This paper examines the tendency of LLMs to not incorporate external feedback. The paper defines the *Rigid Thinking* effect - showing the models' inability to respond to adequate feedback. The results show the quantitative frequency of *Rigid Thinking* across different tasks for 3 feedback strategies and 3 models. It shows that all models reach saturation significantly below the theoretical limit. Ablation studies also claim that the cause of Rigid thinking does not lie with overconfidence and data familiarity. Finally, this work further investigates promising potential mitigations for the effect of *Rigid Thinking*, showing promise for combatting the effect at test time.", "questions": "1. Large Reasoning Models, such as DeepSeek R1, Qwen3, o4-mini, etc. have been shown to not only be better at reasoning tasks, but also more resistant to external feedback. It would be interesting if the authors could replicate their experiments, with, if necessary, following the pointers I mentioned in the **Strenghts and Weaknesses** section. Given the computational difficulties, and the rather time-consuming nature of human-based evaluations, some results on a smaller model\nsuch as *DeepSeek R1 Distill Qwen3 8B*, *Qwen3 8B*, or similar models with the same properties, would be helpful for their arguments. I aknowledge that not all of these models were not released at the time of the submission deadline but alternatives, such as the DeepSeek R1 Distill 14B were present.\n\n2. In sampling the LLM generations, the authors describe using a deterministic approach (with temperature 0). Why is that necessary for quantifying the phenomenon, and why are different temperature schedules not considered in the main evaluation? Ideally, some experimental evidence would be appreciated.\n\n3. How would using conversation-based feedback (repeated user-assistant iterations), instead of a large prompt containing all the iterations, change the results?\n\n4. Can the authors address the comments about their confidence comparison I made in **Strengths and Weaknesses**?\n\n5. Why did the authors not consider scaling using automated tools? This would allow for much better scaling across multiple models. In most human-based studies, the experts' judgments are not always 100% accurate, so the 96% accuracy rate of o4-mini seems quite high.\n\nIf the authors are able to address these issues, I would be happy to raise my score.", "limitations": "The authors have provided some discussion on the limitations of their work. They acknowledge minor limitations, such as not being able to completely remove the Rigid Thinking phenomenon, as well as not being able to explain why models can behave stubbornly. \n\nHowever, they do not thoroughly discuss the limitations about the scale and replicability of their results for other models and settings. I discuss most of these weaknesses in the **Strengths and Weaknesses** section. In summary:\n- There is no clear reason why no competing open-source models (Gemma/Phi), or reasoning models (o4-mini, Qwen3, R1) were not investigated. Instruction-following behaviour might differ vastly across different model types, or differences in how these models were trained.\n- The authors should discuss the difficulty of evaluating this dataset, and potential for automated scalability in more detail.", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "## Strengths\nThe paper is well explained, with clear experimental setup. Quantifying receptiveness to feedback is an important avenue for research, and proving the majority of the cause is the models' response, rather than the quality of the feedback. The range of the tasks is sensible, including a range of easier, and more challenging ones. Further, the additionally defined tasks are interesting, with the differences in behaviour between hex and decimal computation showing a key limitation in LLMs, not just in the Rigid Thinking sense.\n\nThe observation and quantification of the phenomenon is important, as it highlights the need for future research in this direction.\n\nShowing that this effect can be mitigated is also a key contribution. It would be nice to see if train-time techniques can be used to do this, as the proposed test-time solutions can be computationally expensive\n\nWith the paper's findings, I hope that better train- or test-time mitigations for \"Rigid Thinking\" will be developed as a result.\n## Weaknesses\n\nThe paper is mostly clear and presented well, some decisions for the experimental setup were not thoroughly explained. While the LLMs being evaluated are quite recent, their scope is rather limited. The paper only looks into the most recent LLaMa models, where potentially similar training methods and/or data might have induced similar behaviour. Furthermore, the authors acknowledged that they didn't evaluate on the Qwen3 235B-A22B reasoning model, as it got stuck in reasoning loops. However, in their experimental setting, the authors described using a temperature of 0. This decision is not explained well enough, and is the primary cause for the reasoning loops observed. In particular, basically all open-source reasoning model providers especially warn against using a low temperature. Showing the observations in this work hold across different model providers/reasoning models, would significantly strengthen the paper.\n\nFurthermore, the human-led nature of the study into the factors causing Rigid Thinking can be considerably difficult to scale. The authors show that o4-mini has a near-human performance with a >96\\% agreement rate, yet they do not explain why a human-based evaluation was necessary.\n\nUnfortunately, the paper does not present a clear argument about why the effect exists, as well for eliminating the overconfidence factor. First, the measurement use to quantify confidence is not sufficiently justified. It is further temperature-dependent, given the choice of the sequence of generated tokens is as well. Finally, given the final accuracy is near-saturation, the statistical variability around 100% seems much more significant than the confidence factor. Evaluating on a harder task (i.e. AIME) where accuracy is not almost saturated by the final iteration, might give a stronger signal to support the hypothesis.\n\nA nitpick I had with the feedback preprocessing step is the fact that masking the boxed answer is often insufficient. Especially in more reasoning-heavy tasks, intermediate or near-final steps might leak a significant of information.\n\nOverall, the paper contains some good ideas, but the experimental design leaves key questions and issues not throughly explored.", "was_revised": true, "final_justification": "I have increased my score from an initial 2 to a 4 following the strong rebuttal presented by the authors. In particular, they provided a much more thorough evaluation on a proprietary reasoning model, one of the best in terms of robustness - which revealed that RIGID THINKING persists even in near-SOTA models. This makes further investigation of the phenomenon highly relevant.\n\nThe authors also addressed the majority of my remaining concerns. They offered a deeper analysis of how RIGID THINKING relates to model confidence, and clarified the questions I had about the experimental setup. The potential for using this work as a benchmark makes it even more compelling.\n\nThe only reason I did not raise my score higher is that evaluating a broader range of models would have been a valuable addition  something that could be easily addressed in the next revision. Moreover, uncovering evidence of the potential root causes of RIGID THINKING and identifying training interventions to mitigate it would elevate this work even further.\n\nThe fact that the authors addressed so many issues within the two-week rebuttal period is deeply impressive, and I believe it warrants close attention from both the remaining reviewers and the AC. I am happy to advocate for the paper’s acceptance, as the authors have addressed nearly all the concerns raised by myself and other reviewers.", "rebuttal": "We thank the reviewer for recognizing that our paper is well explained with a clear experimental setup, that quantifying receptiveness to feedback is important research, and that our observation and mitigation of this phenomenon represents a key contribution. We have tested additional reasoning models that confirm RIGID THINKING persists, clarified our model-specific temperature settings, and conducted multi-turn interaction experiments. Below we provide detailed responses to your specific concerns:\n\n---\n\n> The scope of models tested is limited to LLaMA family only, no reasoning model is tested\n\n**TL;DR: We tested Claude 3.7 reasoning models and found the same RIGID THINKING behavior persists.**\n\nAlthough we did not evaluate reasoning models in our NeurIPS submission, we have since tested larger models like Claude 3.7 Sonnet (both with and without Extended Thinking). Table 1 shows the initial accuracy, final accuracy after 10 feedback iterations, and theoretical target accuracy for each model. Despite Claude models achieving significantly higher performance than Llama models, on several datasets, they still plateau well below their target accuracy on several datasets, demonstrating that RIGID THINKING persists across model scales. We'd definitely welcome suggestions for additional models to test.\n\n| Dataset                    | Solver Model                           | Initial (%) | Final (%) | Target (%) | Δ (Target - Final) |\n|----------------------------|----------------------------------------|------------:|----------:|-----------:|-------------------:|\n| Hexadecimal 5-Digit Mult   | Claude 3.7 Sonnet                      |        1.0 |      12.0 |       86.0 |               74.0 |\n| Hexadecimal 5-Digit Mult   | Claude 3.7 Sonnet (Extended Thinking)  |        8.0 |      16.0 |       86.0 |               70.0 |\n| AIME 2024                  | Claude 3.7 Sonnet                      |       33.3 |      76.7 |       99.0 |               22.3 |\n| AIME 2024                  | Claude 3.7 Sonnet (Extended Thinking)  |       50.0 |      90.0 |       99.0 |                9.0 |\n| PopQA                      | Claude 3.7 Sonnet                      |       55.8 |      88.8 |       98.0 |                9.2 |\n| PopQA                      | Claude 3.7 Sonnet (Extended Thinking)  |       58.1 |      92.3 |       98.0 |                5.7 |\n| MMLU Pro                   | Claude 3.7 Sonnet                      |       84.2 |      90.8 |       99.2 |                8.4 |\n| MMLU Pro                   | Claude 3.7 Sonnet (Extended Thinking)  |       83.3 |      96.7 |       99.2 |                2.5 |\n| MATH-500                   | Claude 3.7 Sonnet                      |       78.0 |      94.0 |       99.0 |                5.0 |\n| MATH-500                   | Claude 3.7 Sonnet (Extended Thinking)  |       90.0 |      96.0 |       99.0 |                3.0 |\n| 5-Digit Multiplication     | Claude 3.7 Sonnet                      |       43.6 |      96.0 |       99.0 |                3.0 |\n| 5-Digit Multiplication     | Claude 3.7 Sonnet (Extended Thinking)  |       46.6 |      97.0 |       99.0 |                2.0 |\n| GPQA                       | Claude 3.7 Sonnet                      |       66.7 |      98.5 |       99.6 |                1.1 |\n| GPQA                       | Claude 3.7 Sonnet (Extended Thinking)  |       72.2 |      97.0 |       99.6 |                2.6 |\n| MMLU                       | Claude 3.7 Sonnet                      |       87.1 |      97.1 |       99.9 |                2.8 |\n| MMLU                       | Claude 3.7 Sonnet (Extended Thinking)  |       86.4 |      99.3 |       99.9 |                0.6 |\n| TriviaQA                   | Claude 3.7 Sonnet                      |       86.1 |      98.7 |       99.7 |                1.0 |\n| TriviaQA                   | Claude 3.7 Sonnet (Extended Thinking)  |       88.1 |      99.3 |       99.7 |                0.4 |\n\n**Table 1:** Performance comparison of Claude 3.7 models showing initial accuracy, final accuracy after 10 feedback iterations, and theoretical target accuracy across different datasets.\n\n---\n\n> Temperature setting (temperature of 0 is chosen throughout the analysis, even for Qwen models)\n\n**TL;DR: In the current submission, we used appropriate temperatures for each model, not uniformly temperature=0.**\n\nWe want to clarify our temperature choices, which were not uniformly set to 0 as the reviewer suggests:\n\nWe tested Qwen3-235B-A22B with temperature=0.7 (following the suggested temperature from their documentation), but found that when reasoning mode is enabled, it still enters repetitive generation loops frequently, and it's not very good at following instructions (the community observed a similar phenomenon), which led to our final decision not to use it.\n\nIn addition, in our run with Claude 3.7 with Extended Thinking, we use temperature=1 as suggested by Anthropic.\n\nWe choose temperature=0 for most of our testing to ensure deterministic results crucial for controlled experimentation. However, we did conduct experiments with higher temperatures in later sections (§4.2), where we explore progressive temperature increases and rejection sampling as potential mitigation strategies.\n\n---\n\n> Human-led study into factors causing Rigid Thinking would be difficult to scale\n\nYes, as you pointed out, that is a major roadblock if we were to only rely on human evaluation to under Rigid Thinking! However, we did this time-consuming human evaluation for the exact opposite purpose — to verify that o4-mini's judgment of why LLMs didn't follow instructions is correct, so that we **don't** need to do more human annotation ourselves later.\n\n---\n\n> No clear argument for why the effect exists or root cause analysis. Confidence measurement insufficiently justified. Should test on harder tasks (AIME) for stronger signal\n\n**TL;DR: We acknowledge we cannot fully explain the root cause, but our confidence analysis on AIME confirms no correlation.**\n\nYou raised an excellent point! We acknowledge that we cannot definitively explain why this effect exists. We believe a better understanding likely involves complex interactions between how models understand feedback, follow instructions, and update beliefs, which we continue investigating.\n\nRegarding our confidence measurement, we adopted the approach from ClashEval: Quantifying the tug-of-war between an llm's internal prior and external evidence where it demonstrated effectiveness. Since submission, colleagues have suggested alternative confidence/uncertainty measures like semantic entropy (Detecting hallucinations in large language models using semantic entropy\n), which we are currently exploring.\n\nWe conducted the same confidence experiments on AIME as you suggested. The results show the same conclusion as our other confidence experiments: there is no strong correlation between confidence measures and resistance to feedback.\n\n---\n\n> How would using conversation-based feedback (repeated user-assistant iterations), instead of a large prompt containing all the iterations, change the results?\n\nThis is a very good suggestion! We have tried to reframe the conversation history into a multi-turn setting by inserting <user> and <assistant> tags for different models and found the performance to be basically the same - LLMs still couldn't reach perfect accuracy with that approach. However, we acknowledge this could be explored more systematically as future work.\n\n---\n\n> Can the authors address the comments about their confidence comparison I made in Strengths and Weaknesses?\n\nWe have already addressed this in previous comments.\n\n---\n\n> Why is a temperature of 0 necessary for quantifying the phenomenon, and why are different temperature schedules not considered in the main evaluation?\n\nWe would be more than happy to test other temperatures that you think are worth trying.\n\nAs for the temperature setting in this submission, we choose temperature=0 for most of our testing to ensure deterministic results crucial for controlled experimentation. However, we did conduct experiments with higher temperatures in later sections (§4.2), where we explore progressive temperature increases and rejection sampling as potential mitigation strategies.\n\n---\n\n> Why did the authors not consider scaling using automated tools?\n\nIn fact, we did scale using automated tools (specifically, o4-mini) to categorize the reasons why LLMs fail to integrate feedback. The human annotation is only to test whether this automatic tool is correct or not. \n\n---\n\n> Nitpick: masking the boxed answer is often insufficient.\n\nThat's a very valid suggestion! We have thought about it and couldn't really find a way to do more masking. For our synthetic tasks (multiplication and hex multiplication), we did mask more intermediate steps, but for other tasks, it's hard to really mask intermediate steps. Also, the fact that LLMs still plateau under these good conditions actually enhances our argument.\n\n---\n\n> Rating: 2: Reject: For instance, a paper with technical flaws, weak evaluation, inadequate reproducibility and incompletely addressed ethical considerations.\n\nWe would sincerely appreciate further elaboration, as we are confident that any such concerns can be addressed in this rebuttal period."}]

Author rebuttal:

Meta-review: This paper introduces "RIGID THINKING," the finding that large language models often fail to fully incorporate near-perfect feedback, causing their accuracy to plateau below the theoretical maximum. Initial reviews commended the paper's rigorous experimental setup and the importance of this finding but noted weaknesses, such as the limited scope of models tested (only the Llama family) and a lack of exploration into the phenomenon's root causes. In a comprehensive rebuttal, the authors addressed these concerns by adding experiments on larger, proprietary reasoning models (like Claude 3) and conducting a deeper analysis connecting the phenomenon to model confidence using semantic entropy. These additions confirmed their initial findings persist across different architectures.

While one reviewer felt the origin of RIGID THINKING remained unclear, I agree with another's assessment that the investigation provides significant value to the community, even if the root causes are not fully pinned down. A question was also raised about whether the rebuttal's new experiments constituted a major revision requiring re-review. I find that these changes reinforce the paper's original narrative and conclusions rather than altering them, making the revision minor and appropriate for this submission cycle. Overall, the strong rebuttal and additional experiments have successfully resolved most reviewer concerns, and I recommend the paper for acceptance.
Decision: Accept (poster)