When Models Know More Than They Can Explain: Quantifying Knowledge Transfer in Human-AI Collaboration
Authors: Quan Shi
Carlos E Jimenez
Shunyu Yao
Nick Haber
Diyi Yang
Karthik R Narasimhan
Venue: NeurIPS 2025 poster ()
Keywords: LLMs for coding
LLMs for math
Human Study
HCI
AI-assisted ideation
Primary area: social_and_economic_aspects_of_machine_learning
TL;DR: We conduct a user study to evaluate how well language models help humans internalize their reasoning, revealing that strong model performance alone doesn't guarantee effective reasoning transfer.

Abstract: As large language models (LLMs) increasingly serve as close collaborators for humans, it is crucial that they express their reasoning in ways that humans can understand and learn from. However, this capability remains relatively less understood and under-evaluated. To address this, we introduce a conceptual framework for such Human-AI knowledge transfer capabilities and conduct the first large-scale user study (N=118) explicitly designed to measure it. In our two-phase setup, humans first ideate with an LLM on problem-solving strategies, then independently implement solutions, isolating the influence of model reasoning on human understanding. Our findings reveal that while model benchmark performance correlates with collaborative outcomes, this relationship is notably inconsistent with significant outliers, highlighting that knowledge transfer is a distinct capability requiring dedicated optimization. Our analysis uncovers behavioral and strategic factors that mediate successful knowledge transfer, and we release our code, dataset, and evaluation framework to support future work on communicatively aligned models.

When Models Know More Than They Can Explain: Quantifying Knowledge Transfer in Human-AI
Collaboration
Quan Shi P
Carlos E. Jimenez P
Shunyu Yao OP
Nick Haber
Diyi Yang $
Karthik Narasimhan P
Princeton Language and Intelligence
9 Stanford University
OpenAl
Abstract
Recent advancements in Al reasoning have driven substantial improvements across diverse tasks. critical open question is whether these improvements also yields better knowledge transfer: the ability of models to communicate reasoning in ways humans can understand, apply, and learn from. To investigate this, we introduce Knowledge Integration and Transfer Evaluation (KITE), conceptual and experi- mental framework for Human-A4 Al knowledge transfer capabilities and conduct the first large-scale human study (N=118) explicitly designed to measure it. In two-phase setup, humans first ideate with an Al on problem-solvine strategies, then independently implement solutions, isolating model explanations' influence on human understanding. Our findings reveal that although model benchmark performance correlates with collaborative outcomes, this relationship is notably inconsistent, featuring significant outliers, indicating that knowledge transfer re- quires dedicated optimization. Our analysis identifies behavioral and strategic factors mediating successful knowledge transfer. We release our code, dataset, and evaluation framework support future work communicatively aligned models.
Introduction
As large language models (LLMs) grow more capable, find them quickly saturating benchmarks across reasoning-intensivee domains, such as coding [6, 24, 25, 44], scientific problem-solving [40, 17, 48]. and mathematics [9. 18]. A key driver, Reinforcement Learning with Verified Rewards (RLVR), has emerged as popular post-training approach, enabling models to optimize their language outputs for high-reward reasoning in verifiable domains like math and code achieve state-of- the-art performance and widespread industry adoption [14, 29, 52]. Yet this rapid progress hides crucial assumption: that improvements in model's internal reasoning naturally translate into better knowledge transfer, that is, a model's 's ability to communicate its reasoning in ways humans can understand, apply, and learn from. As we build increasingly capable reasoners, does effective knowledge transfer emerge for free, or must it be treated as separate objective that requires dedicated evaluation and optimization?
This question has far-reaching implications. In many human-Al collaborative workflows, the goal not merely outsource thinking AI, but to amplify human abilities [37, 12, 53, 15]. Without effective knowledge transfer, users may become increasingly dependent on systems they do not understand [22. I]: dynamic reminiscent of "manager's syndrome" [20], where individuals lose
CCorespondence gbshi dllumni.prnnceoonedll Code, data, visualizer kite- live, vercel.app
39th Conference on Neural Information Processing Systems (NeurIPS 2025).

---

lard
Gemo.
Numan- Collaborativv Performance
22-
lecd 6232
70-
Code lana
Code herd E.K
Holk IN
10
20
30
40
50
60
30
80
10
50 60
80
Model Solo Performance 196)
Model Sola Performance (%%
Figure 1: Left: Human-Al collaboration performance plotted against model solo performance for both code tasks (blue circles) and math tasks (green triangles). Models improve human-Al collaboration (r = 0.84 for code, r = 0.69 for math), but at a slower rate than their solo capabilities (gray line shows y 2). Right: Human preference rates show task-dependent correlations with model performance (positive for code tasks, r = 0.73; slight negative for math tasks, 7 = -0. 14), revealing that user preferences vary across task domains and do not consistently align with actual performance.
technical fluency as they delegate complexity. This dynamic further exacerbated when users cannot discern or interrogate model reasoming, leading to overreliance on systems they perceive as more intelligent, and increasing the risks of sycophantic behaviors, where models shape or reinforce user beliefs rather than supporting sound judgment. Moreover, in high-stakes settings such as medicine or legal services, the inability of models to communicate their reasoning clearly could undercut human oversight entirely [27, 21,4, 4]. Few works rigorously assess how well models support human understanding and enable scalable oversight, especially across latent user variables, such as differences in domain expertise, AI familiarity, or the skill gap between human and model that critically shape the success such transfer.
To investigate this, we introduce Knowledge Integration and Transfer Evaluation (KITE), conceptual and experimental framework that explicitly isolates and evaluates knowledge transfer. In our large- scale human evaluation, we recruit 1 118 participants with diverse levels of expertise, including substantial proportion of domain experts (competitive programmers, math majors) who tackle challenging problems in coding and mathematics through two-phase protocol. In the collaborative ideation phase, participants interact freely with an AI model to explore solution strategies. This phase serves as the primary opportunity for the Al to transfer knowledge to the human by explaining concepts and jointly developing solutions. In the subsequent independent implementation phase, participants attempt to implement previously discussed solutions alone, without access to the Al or any prior interaction transcripts, allowing us to isolate and measure the effectiveness of knowledge transfer. We assess outcomes using both objective metrics (solution correctness) and subjective evaluations (user rankings, perceived helpfulness, and qualitative feedback), enabling a comprehensive analysis of how well models support knowledge transfer across varying levels of user expertise and task difficulty. Our study is IRB approved.
As shown in Figure 1, we generally find participants demonstrated strong ability to integrate model-generated reasoning with their own expertise. Interestingly, some models, such as Cl.ude-3.7- Sonnet, enabled collaborative outcomes that exceeded expectations based on their solo capabilities, particularly in mathematical reasoning tasks. contrast, nigher-performing models like Gemini-2.5- Pro did not consistently yield proportionally stronger collaboration, suggesting diminishing returns in knowledge transfer as model reasoning scales. If this trend continues, as models grow more capable, their internal representations may become increasingly difficull to project in ways humans can easily understand and utilize [19].
Moreover, we find that humans' subjective preferences for models during collaboration often diverge from solo model performance, particularly in math tasks, revealing cennain-ppecffcc patterns in what users value during collaboration. To probe these dynamics, we perform qualitative analyses of interaction transcripts, clustering patterns of human queries and model responses across varying user skill levels and task types. These findings surface distinct collaboration styles and success/failure modes (overreliance, representation misalignment, adaptive scaffolding...., offering a lens into the latent Human-A Al interactions that govern effective knowledge transfer.

---

Overall, this paper aims to provide foundation for future research on quantifying and enhancing the knowledge transfer capabilities AI systems: particularly as models grow more intelligent and begin to develop knowledge that is increasingly inaccessible to humans. We develop a conceptual and experimental framework isolate and quantify knowledge transfer, as well as provide insight into drivers of scaling trends between reasoning and knowledge transfer capabilities. To facilitate progress in this direction, we release our evaluation code, dataset, and filtered interaction trajectories to support future efforts in building AI systems that are more communicatively and cognitively aligned with human collaborators.
2 Related Work
Human-AT Collaboration Research human-Al collaboration has increasingly focused on opti-
mizing complementary team performance and, implicitly, knowledge transfer. Studies have explored how bidirectional information exchange enhances collaborative outcomes [34, 33], examining the impact of explanations during interactions [3] and investigating how proactive AI assistants can help humans discover preferences in open-ended tasks like travel planning and data visualization [43]. Most closely related to our work, [38] evaluated the effectiveness of autocomplete suggestions and chat assistants in helping humans solve coding problems from HumanEval [6]. While these studies provide valuable insights into collaborative performance, our work extends beyond immediate task outcomes to systematically measure reasoning transfer.
Code + Math Reasoning Tasks for LLMs Farly code and math benchmarks such as HumanEval [6]. MBPP [2], and GSM8K [9] focused relatively simple problems requiring short code snippets or numerical answers. With many of these benchmarks now approaching saturation by advanced models, we deliberately selected more challenging problems from competitive programming platforms like Leetcode [24, 46] and mathematics competitions (AMC, AIME) [51]. These problems are particularly suited for our study they primarily test reasoning abilities rather than context handling, making them ideal for measuring knowledge transfer in human-Al collaboration. This contrasts with repository-styye benchmarks like SWE-Bench [25] and BigCodeBench [54], where performance is often bottlenecked by context interpretation capabilities.
Knowledge Transfer and Education While limited work explicitly analyzes knowledge transfer from LLMs humans, this shares conceptual overlap with educational applications of LLMs, where models must effectively teach reasoning to humans. Recent research has explored LLMs assisting tutors by identifying effective strategies [49], creating personalized lesson plans [26, 41, 11]. provid- ing feedback [16. 7]. and functioning as specialized tutoring agents [31, 35J. However, significant challenges remain, as LLMs often underperform as teachers by leaking answers or failing to employ effective pedagogical approaches [39, 50, 13]. Our work diverges from educational applications by explicitly measuring the explanatory quality of L.M reasoning by requiring participants to inde- pendently execute discussed algorithms through mathematical calculations or code implementation, which is only possible if they truly understand the model's explanations.
3 KITE: Quantifying Knowledge Transfer
We first outline preliminaries for understanding knowledge transfer between entities during collabora- tive problem-solving. While we formalize knowledge regions such as M, H. and their intersections, we note that these are illustrative abstractions- difficult to precisely measure in practice, but useful for analyzing collaboration dynamics.
3.1 Conceptual Framework for Knowledge Transfer
We approach knowledge transfer through the lens of collective intelligence [10]: the collaborative problem-solving capability that emerges when humans and AI work together. Following [42, 28), we can represent the machine's 's knowledge and capabilities, or representation space, as M, and the human's 's as H; illustrated in Figure 2. This formulation yields three critical regions for our analysis:
3

---

Representational Space
Knowledge Transfer
H: Human
Mc Model
Iteratively minimal
You need to run BFS going to the ciosest place haven't visited and checking the distance..
H
M
M - H
kM E M KH M (kMs)
kh E H
II M-H (Examples, Analogies, Contoxt Aggrogation.)
M H
Figure 2: Model knowledge (kas E M) must be projected into form understandable by human users (T H (kM) in order to communicate knowledge effectively. Effective projections via examples, analogies, context aggregation- bridge the gap between disjoint representations.
1. Shared Knowledge (M n h): This intersection contains reasoning patterns, abstractions, and strategies already understood by both human and model. It forms the foundation for effective communication.
2. AI-Exclusive Knowledge (M H): This region reflects novel reasoning, knowledge, or strategies that the model can execute but the human has not yet mastered. Transfer from this space into H is the central goal of collaborative ideation.
3.
Human-Exxlusive Knowledge (H y): Reasoning held by the human but not by the model: such as intuitive understanding, prior experience or deeper domain knowledge/insight.
The success human-Al collaboration hinges critically on accessing and transferring knowledge from the M H space into H, especially as humans typically maintain primary agency in collaborative tasks (e.g., deciding which strategies to pursue or when to submit solutions). However, as models become more capable, their reasoning may depend on abstractions increasingly distant from the typical human representation space. We frame this challenge terms of projections: for each knowledge point KM in the model's space, the model must identify some projection a (kM) that translates its reasoning into a form the human can understand, internalize, and act upon. These projections can take many Suchs-such providing analogies, contextualizing concepts with background knowledge, offering intermediate scaffolding, or generating concrete examples.
Importantly, this process is bidirectional. Humans also project their reasoning into the model's representation space via I (k such as using specialized prompts elicit helpful responses. Especially in interactive settings where models are not fully autonomous, effective collaboration depends on this ongoing loop mutual translation and aligning expressions of reasoning.
4 KITE: Evaluating Knowledge Transfer
Informed by the conceptualization discussed in Section 3, our two-phase setup (Figure 3) comprise human-AI collaboration phase, and solo human implementation phase that demands real un- derstanding (e.g. writing code or performing calculations). Users can't simply memorize model suggestions, especially when they' re incomplete or flawed: solving requires debugging, handling edge cases, and reasoning through the solution. This enables us to isolate and measure knowledge transfer from AI to humans. See Figure 17 for example problems and dataset statistics. While setup can accommodate any reasoning problem that naturally divides into ideation and implementation phases, in this paper focus two domains: coding tasks from LiveCodeBench [24] and competition-leve mathematics problems (AMC/AIME). These domains present consistently challenging reasoning tasks across wide range of expertise levels, making them ideal for studying knowledge transfer.
4.1 Two-Phase Protocol for Isolating Knowledge Transfer
Phase 1: Collaborative Ideation First, participants are presented with a problem drawn from either the algorithmic coding [24] or competition mathematics [51] domains. this phase, they engage in an open-ended dialogue with selected LLM to explore solution strategies, exchange ideas, and scaffold their understanding without solving the problem. To preserve this ideation focus, we forbid models from generating any long-form code, pseudocode, or mathematical calculations through prompting, as well as employ secondary checker model to withhold responses flagged to contain answers directly indirectly (code, mathematical calculations). Participants also

---

Problem
Phase 1: Collaborative Ideation
There is dungoon with rooms arranged a grid. You given 21D arry move'l'me size n m, where movelIiimeff [i]ū] represents the minimum time in seconds when you can start moving that room. You start from the room (0, 0) at time and can move to an adjacent room. Moving between adjacent rooms takes one sccond for one move and two seconds for the next, alternating between the two.
How would you solve this?
We could try modeling it as directed graph...
What's the best way to represent the graph?
Phase 2: Independent Solving
Chat Deleted
</> Code Editor
for dx, dy in dirs:
Bused on what we talked about, think we should take this approach...
nx, ny
+.y. dy
not (0 <n nx
ny n):
continue dist nox(d[s.x][s_y)) (oveTinma(xx[[n))) d[re][ry] dist:
Return the minimum time to reach the room (n 1, m 1).
d[rx][r]]] dist
o
heapq, heappush(e, State(ti,, ny, distt)
Ran
Subeii
Figure 3: Two-phase evaluation framework. (1) Collaborative Ideation: Users and AI assistant engage in open-ended discussion to explore problem-solving strategies. (2) Independent Solving: Users then implement a solution independently, without further assistance. This design leverages the nature of coding and math lasks-whig successful implementation demands deep understanding, not rote recall-to isolate and measure genuine knowledge transfer.
not allowed to take any notes to log model insights. We additionally perform post-hoe filtering to remove user interaction data where models emit forbidden content. This ensures that any knowledge transferred takes the form of conceptual reasoning or strategy, rather than memorization of content that can be directly used to assemble the final solution.
Phase 2: Independent Solving After the ideation phase concludes, the LM interface and conversa- tion history are no longer accessible. Participants are tasked with solving the exact same problem on their own, without model assistance. In coding, participants must write and submit correct implemen- tations that pass all test cases, given 10 code submission attempts. In math, participants must carry out precise multi-step calculations to arrive at final answer, given 5 answer submission attempts. By requiring participants to independently execute solution, Phase becomes a direct and rigorous test of whether they have absorbed and retained reasoning introduced in Phase 1. Successful completion indicates that knowledge previously exclusive to the model (kM M HT) has been projected into and re-applied by the human (Il (KM) E Hf)
4.2 Modeling and Calibrating Skill Hierarchies
Collaboration becomes meaningful only when the task challenges the human's 's independent capabili- ties. If the human can already easily solve the problem alone, model assistance becomes redundant: there is opportunity for knowledge transfer, no dependency, and thus true collaboration. This necessitates the calibration of skill hierarchies: the relative proficiencies of the human, the model, and the task. We accomplish this by assigning standardized skill ratings (elo) each of the three entities in the problem-solving process.
Skill Estimation Task difficulty is determined using externally validated Elo ratings: public Leet- Code ratings for programming tasks! and competition-deriver estimates for AMC/AIME math problems Human skill is estimated through a two-step process: participants self-report their experi- ence level, then complete 5 adaptively selected tasks with difficulty adjusted based on performance. Their Elo rating is updated using surprise-cone currris--onditioned rules (Appendix C.8) [8]. yielding an em- pirically grounded skill estimate. Model skill is measured by zero-shot performance each model attempts each task three times, and task is considered solvable if at least one completion is correct. To compare human and model skill fairly, we contrast each human's 's final Elo with the average diffi- culty (Elo) of the top 25% of problems solved by the model, avoiding bias from models attempting all tasks regardless of difficulty.
attrp:://artoorrroooemss//vv wom.i//////o/iii//e////..ee php/ AoPS Wiki wwwmetttioonrattiiiiiiioi
5

---

Test-Time Pairing During the test phase, each participant is required to solve between 3 and 15 problems. They may choose solve any number of problems within this range and are allowed to work at their own pace, including non-contiguous attempted, the participant is paired with one of eight held-out L.MMs, sampled uniformly at random without replacement. Once all models have been encountered, the sampling process resets. Each task is selected to fall within a calibrated difficulty band slightly above the participant's demonstrated skill level, ensuring it is challenging yet tractable with model assistance. Specifically, tasks are drawn from a fixed Elo margin relative to the participant's 's current rating: [t + 200, + 400] for coding problems and [t + 0.75, t + 1.25] for math problems. This design encourages meaningful collaboration with the model, avoiding both trivial and overly difficult cases. While task completion time is recorded, no time limits are imposed: record this metric in Appendix B.
solling- sessions. For each problem
4.3 Experimental Controls and Evaluation Strategy
Evaluation and Success Metrics Evaluating human-Al collaboration is challenging due to the subjective and noisy nature of human preferences. We use both subjective and objective metrics. Subjectively, after each task, participants rank the last four models they interacted with from most to least preferred; we apply the Bradley-Terry model convert these rankings into win rates reflecting relative preference (full algorithm in Appendix C.7). We also provide separated win rates based on the relative ordering of Human, Model, and Task Skill ratings at the time of interaction. Specifically, we report results for the three possible configurations: Human > Task > Model (HTM), Human > Model Task (HMT), and Model > Human > Task (MHT). On the objective side, we assess transfer by comparing the percentage of problems solved through human human-model collaboration to the model's s solo performance on the same problems. For coding tasks, correctness requires passing all associated test cases; for math, we require an exact answer match.
Incentives and Motivation A common confounding factor in human-Al interaction studies is participant motivation [45,3]]; 36]: specifically, it is imperative that users genuinely trying to learn from the model to improve their own performance. To mitigate this, first, we provide monetary incentives: participants receive 1.2x 1.5x their base compensation of $25/hr for correctly answering a question, depending on difficulty. Second, most of our participants are actively preparing for career interviews that require proficiency in the task domains we test-e.g.. competition math for finance related roles, and LeetCode-style problems for software engineering positions. This creates an added layer of intrinsic motivation: participants have a personal stake in learning from the model outputs and in providing thoughtful, honest feedback.
Participant Selection We recruited participants through university- wide email advertisements and word of mouth. Interested individuals completed an initial survey, after which filtered for a diverse sample across academic background, domain expertise, and AI/LLM familiarity to reflect a broad population representative of both technical and non-technical users. Our final cohort comprised 118 participants from 11 institutions, spanning a wide range of majors, including Computer Science (N 49), Electrical Engineering, Mathematics, Neuroscience, and various STEM disciplines. Most were in their first (N = 38) or second (N = 36) year of study, though all undergraduate levels were represented. A full demographic account can be found in Appendix A.
We evaluate eight LLMs of different sizes and abilities: GPT-4.1, GPT-4o [23], GPT--4.--rreview, Gemini-2.5-Pro. DeepSeek- V3 [30], Slaude-3.7-Sonnes 4 and ol. These models were selected based on strong leaderboard performance on ChatArena [8] and widespread usage in interactive evaluation settings. Notably, DeepSeek-R1 [14] was considered but excluded due to availability and latency constraints. To assess natural explanation behaviors, we evaluate models in zero-shot setting without prompt optimization or fine-tuning for explanatory quality, with temperature 0.7 when possible. This design choice avoids confounding effects of tailored prompts and better reflects how users commonly interact with models out-of-the-box.
Model Selection
5 Results
The main quantitative results of the study can found Figure and 2. total, we obtained 578 problem solving trajectories, with each participant completing average of 4.90 problems. We

---

Code (N=300)
Math (N=278)
Model
HTM
HMT
MHT
Total
HTM
HMT
MHT
Total
GPT-4,1
18.6472
15.3=20
8.7+42
15.1
8,2429
10.7
16.1
11.3+20
GPT-4o
2,4
15.3=11
8.6439
8,8+19
5.8+27
10.8+4.7
22.0+38
13.3457
ol
4.1
10.8=19
15.8=43
7.4
17.2=1.
4.2+2.
0.8
10,0473
GPT-4.5
20.4
8,9=28
16.9=6.7
16.0146
6.2:127
15.9
6,3:20
11,8169
D3
7.3140
10.1=30
5,6225
7,8
13.3=39
13.7::50
10.1
13,3124
Llama Maverick
8.8
9.8=40
6.6=3.
9.0+3.
6.7
10.5
25,9
11,8+23
Claude
16.2433
14.8=a7
15.5=47
16.1
8.3+n0
16.31:4
15.3+30
12,6423
Gemini 2.5 pro
20.2
15.1=40
22.3-9.1
20.0
27.2=2.3
17.9
4.422.
16.0:44
Table 1: Bradley-Terry win rates (# standard error) showing human preferences for models post- collaboration across three skill hierarchies: HTM (Human < Task < Model), HMT (Human < Model < Task), and MHT (Model Human < Task), elaborated in Section 4.3. Bold indicates best performance. Higher values indicate stronger average human preference.
Setting
GPT-4.11
GPT-40
o1
GPT-4.5
DS-V3
Llama-4
Cld-3.7
Gem-2.5
Code (M)
68.8
16.7
55.0
68.4
33.3
47.1
45.0
81.3
Code (H+M)
65,0
40.0
55.0
69.0
51.9
54.7
70.0
71.3
Code (A)
-3.8
+23.3
+0.0
+0.6
+18.6
+7.6
+25.0
-10.0
Math (M)
47.6
8.3
83.3
33.3
46.2
47.8
20.8
68.4
Math (H+M)
75.7
56.7
85.8
60.8
70.8
72.6
81.7
79.5
Math (A)
+28.1
+48.4
+2.5
+27.5
+24.6
+24.8
+60.9
+11.1
Table 2: Performance comparison 8 LLMs on code and math tasks, showing accuracy percentages for models operating independently (M) versus human-Al collaborative performance (H+M). Bold indicates best performance. The following abbreviations are used for models: DS-V3 for Deepseek- V3, Llama-4 for Llama-4-Maverick, Cld-3.7 for Claude-3.7-Sonn
and Gem-25 for Gemini-2.5-pro.
summarize core insights below, and report auxiliary results, such as survey feedback, average elo per model, and time spent in Appendix B.
Knowledge Transfer Model Performance While positive correlation exists between solo model performance and collaborative outcomes, this relationship is notably inconsistent with signif- icant outliers. Gemini-2.55PPi despite superior solo performance in code tasks (81.3%), showed reduced collaborative efficacy (-10.0% change), while Claude-3.7 3.7-oonet and GPT-4o demonstrated exceptional collaborative amplificationm (+25.0% in code) despite more moderate solo capabilities (45.0%). Similarly, GPT-4o showed strong improvement in math tasks (+48.4%) despite low solo performance (8.3%). Importantly, the slope of the performance-transs in Figure 1) is consistently below unity, suggesting that as model reasoning capabilities improve, transfer effectiveness may increase more slowly. this trend continues, the gap between model capabilities and effective knowledge transfer will widen with more advanced models, suggesting the need to view knowledge transfer as an important objective for optimization.
relationship (visualized
Subjective Preferences v. Model Performance Interestingly, the correlation between solo per- formance and human preference varied by domain. In code tasks, there was a significant positive correlation: humans tended to prefer models that also performed well independently, with Gemini- 2.5-pro achieving both the highest win rate (20.0%) and highest solo performance (81.3%). However, this relationship was weaker in math tasks. While Gemini-2.5-pru had the highest win rate in math (16.0%), models like Llama-4-Maverick received high preference ratings specific skill hierarchies (25.9% in MHT) despite more modest solo performance (47.8)). Our analysis on human feedback suggests that this divergence stems from differences in how models communicate their reasoning. math models often relied heavily on formal notation, dense symbolic expressions, and proof-base based explanations-formt of communication that many casual or less technically inclined math users found difficult to follow. In contrast, effective collaboration in coding tasks leaned more on natural language descriptions of algorithms and strategies, making high-performing code models more accessible and preferred by human partners.
7

---

Knowledge Transfer v. Subjective Preferences We examined whether humans tended to prefer models that ultimately helped them solve more problems- whether subjective preferences aligned with successful knowledge transfer. Overall, we observed a statistically significant positive correlation (r = 0.86), but a much weaker, non-significant correlation in math (r = 0.14, P < 0.05). For code, this aligns with the expectation that users, aware of whether they successfully solved the task, are more likely to favor models that contributed to that success. However, we also observed several notable outliers, such as ol, which achieved relatively low win rates in code (7.4%) despite comparable collaborative performance (55.0%), ) suggesting that subjective preference is not solely reward-driven: we dive into detailed causes in qualitative analysis.
Divergence in Human Preferences A cross Skill Hierarchies We find that collaborative prefer- ences vary across skill hierarchies. For example, Ge was highly preferred in the math domain when the model outskilled the human and could solve the task independently (HTM) with a 27.2% win rate. However, performed poorly the MHT setting (4.4%), where needed to follow human guidance. Similarly. Llama-4-Maveriic showed stark contrasts between different hierarchies in math, performing exceptionally well in MHT settings (25.9%) but poorly in HTM contexts (6.7%). As revealed in our qualitative analysis, we hypothesize this divergence stems from Gemini's tendency toward active engagement, frequently asking confirmational questions to scaffold learning. This behavior was appreciated by users with low expertise, who found it supportive. but frustrating to more expert users, who felt it was verbose and preferred the model to be more direct. These findings caution against ceeesize-fits-lle strategies: optimal collaboration depends not only on model capability, but also on how well models can adapt their communication style to fit the skill-level of different users.
Covariate Analysis We examined the effect of participant characteristics performance using logistic regression analysis on potential participant covariates. Notably, we found no statistically significant effects from user expertise (p 0.252 for coding, 0.196 for math). LLM familiarity (p = 0.339), or prior experience with collaboration tools, such as Cursor, (p = 0.238) on solve rates. These findings suggest that initial expertise calibration successfully balanced tasks relative to individual skill levels. We hypothesize the minimal impact of LLM familiarity likely stems from the unbalanced conversation pattern, where even participants with limited experience received comprehensive output from models, making knowledge transfer primarily dependent on the model's explanatory capabilities rather than the user's prompting expertise.
Qualitative Analysis: Interaction Dynamics
To better understand the mechanisms behind our quantitative findings, we analyze interaction patterns inspired by the Clio framework [47]. User queries are embedded using OpenAl's text seatt-emeddiing-332 3-lagge model and clustered with k-means [32] to identify distinet strategies associated with success or failure, along with their qualitative feedback. Clusters are then manually reviewed and verified. Figure 4 summarizes these patterns, grouping feedback, queries, and model responses by outcome to qualitatively interpret the dynamics of knowledge transfer in human-Al collaboration.
6.1 Performance Transfer Gap
The performance transfer gap refers to the observation that improvements in model capability do not always lead to proportionate improvements in human problem-solving performance. Our analysis surfaces recurring dynamics that help explain this phenomenon.
Overreliance on Model Authority 5% of cases, users explicitly described deferring to the model without critical evaluation. This tendency becomes problematic when models occasionally return incorrect or misleading solutions. As one participant noted, "The model initially gave me the wrong answer, which, to be fair, caused me to rush past the planning step since I trusted the model." This dynamic suggests that presumed model competence may inadvertently discourage user reflection, impeding learning and effective problem-solving.
8

---

Human Queries
Model Responses
Asking for Full Solution [51%]
solve problem" approachr
Step by Step Solution [51%]
help! step by step appreach.
Clarification of Solution [16%]
claniyy step pis' "why it up root
Summarization [34%]
"Let's summarize approach""
Suggesting en Aigorithm [5%]
"Could we use something clse like
Query Restating [23%)
"Lot's rescute clearly what you have what you need""
Dijectrals or 3FS?
Clarification of Problem [9%]
"what exactty a row pairs" "wtny is the ouput for this 37*
Clarification [27%)
"When use notation lik Sipmodjja2--a- moans...
Seoking Backi round Knnwleddee [8%)
"how convert go binary number"
Mocification on Feedback (9%)
"I en's try an (fin) solution instead of
(1p+2) previousst..
Querkia for Completta Solutiona (6%)
python iof poeudocade"
"oohar's the antweer
Simplifying Analogies (4%)
way you vitualize
Human Feedbackk
Good Algorithmic Direction [36%]
"I don't trink wouldfve considared continupuslly updacing the
"The medel reminded
type. Without thet, proballly couldntt salved pe probiemm.
Alice Bub iteration..
Formatting [15%]
"I really could not uncerstand what the was trying to say with the math equation.. seems to be relatively simple to understand?"
Active Engagement [5%]
"I really iived this one. Was inter active and asked me a lot of questions to keep engaged.."
Over Relianlllcceeeee Model Output (5%)
got "Even it did have the correct question It offered solution that would be entirely unnecossary (Dynamic Programming (aggmmmnng) while diidit it greedl pro program could possibly work...
"The model initially gave
This
wrorg
which fair caused
wrong."
past planning step
Model/Human Stubboorneee (5%)
Model Output Wordiness [4%]
"The model wayysy too wordy like really did not need all that much for simple response"
Model output Incorrect [14%]
"The model unluciheted an optimization in this DP prokiemm that sas incor rect."
Figure 4: Analysis of human-Al problem-solving interactions. Human queries (left), model responses (center), and human feedhack (right) are color-coded by correlation with successful problem resolution (green: positive, red: negative). Percentages indicate each category's frequency, revealing patterns in effective vs. ineffective knowledge transfer.
Misaligned Explanation Strategies answers but fall short in adapting their explanations to users' knowledge levels. While patterns such as "Clarification" (27%) and "Simpliffing Analogies" (4%) appear across model outputs, these not always used effectively. "Step-by-step solutions" were the most frequent output style (51%), but users reported issues with verbosity (4%) and poor formatting (15%). both of which hindered knowledge transfer. Even technically accurate solutions can become ineffective if presented in ways that are hard for users interpret, contextualize, or apply.
models often excel at generating correct
6.2 Domain-Specific Preference Patterns
Representation Misalignment We observed notable difference in how users responded to model explanations across domains. In math tasks, high-performing models like ol frequently exhibited what we call representation misalignment: explanations that, while technically correct, were often overly formal, verbose, or difficult to follow. Users described these responses as overwhelming or rigid, leading to lower preference ratings despite strong solve rates. In contrast, coding tasks benefited from better aligmment between the procedural nature of the task and the model's stepwise reasoning. This suggests a Nomain-specinf divergence: in coding, model performance and user preference tend to align due to shared algorithmic structure, whereas in math, users value intuitive and conceptual framing more highly.
Strategic Framing vs. Technical Depth In coding contexts, users consistently valued strategic guidance over exhaustive technical detail. For example, one user wrote, "The model reminded me of the trie type. Without that, I probably couldn't 't have solved the problem... " This suggests that models that foreground high-level framing or conceptual cues-rather than diving straight into detailed solutions- are more helpful in supporting user problem-solving. However, models often default to presenting fully fleshed-out solutions, which may obscure the overall structure or intent. Much like how researchers prefer the big-picture framing of paper before diving into methods, users may benefit more from contextualized reasoning than exhaustive but unfocused detail.
6.3 Skill Hierarchy Dependencies
Adaptive Scaffolding vs. Directness The success of interaction strategies often depends on the relative skill levels of the human and the model. In HTM Teauman-Teaccs--Mee settings, where humans are less skilled than the model, successful models like Gemini-2.5-Pir employed

---

what we call scaffolded projection: breaking down reasoning into digestible parts, often with built-in comprehension checks. However, the same approach proved counterproductive in MHT (Model-Helps-Humngn) settings, where the human was more skilled than the model. In these cases, excessive scaffolding was perceived as redundant or even patronizing, with feedback describing it as "unnecessarily handholding" or "repetitive."
query-Response Alignment These dynamics are further supported by analysis of query types. In HTM settings, users frequently asked for background knowledge clarification ("Clarification of Solution" " 16%. "Secking Background Knowledge" 8%), suggesting a need for instructional responses. In contrast, MHT scenarios often featured queries like "Suggesting Algorithm" (5%), where users sought validation or refinement rather than explanation. Models that perform well in MHT settings appear to align their responses with these expert-level expectations sxpectations-prroidinng concise, targeted feedback rather than elaborate instructional breakdowns.
7 Discussion
Conclusion We conduct the first large-scale study of knowledge transfer in language models, producing a conceptual framework as well as empirical data to characterize it. While model perfor- mance generally correlates with collaborative outcomes, this relationship is inconsistent, with notable outliers. We identify interaction mechanisms that help explain these gaps. As models grow more capable, their ability to convey reasoning may lag behind-risking greater knowledge asymmetry and weakening human oversight. In high-stakes domains, this disconnect could undermine human-Al collaboration, highlighting the need better understand and improve knowledge transfer.
Limitations and Future Work Our study assumes that for each task, some projection of model reasoning could enable a human to solve it. While unverifiable, this assumption is supported by screening for baseline proficiency, calibrating task difficulty just beyond participants' independent ability, and post-task surveys suggesting participants generally believed the tasks were solvable with more time support. Additionally, participants may have exerted more effort than typical users due to monetary and personal incentives, possibly inflating our measured collaboration effectiveness relative to real-world settings where users might disengage in the face of ambiguous model outputs. Lastly, our sample (118 participants) skewed toward STEM students, limiting generalizability. Future work should extend to domains like clinical reasoning or creative writing, and explore multimodal collaboration (e.g., diagrams or interactive tools) to uncover richer knowledge projection strategies.
Acknowledgments and Disclosure of Funding
We thank Open Philanthropy for providing the funding for this work, and Princeton Language Intelligence for providing credits for running closed source API models. Thank you to our beta testers, Jonathan Lin and Ricky Chen, for providing helpful feedback to shape the user testing interface. Finally, thanks to Yijia Shao, Wenting Zhao, Alex Zhang, Rose Wang, Howard Yen, and John Yang for your constructive discussions and support throughout this year-long project.
References
[1] Sayed Fayaz Ahmad, Heesup Han, Muhammad Mansoor Alam, Mohd Khairul Rehmat, Muham- mad Irshad, Marcelo Arraño-Muñoz, and Antonio Ariza-Montes Impact of artificial intelligence on human loss in decision making, laziness and safety in education. Humanities and Social Sciences Communications, 10(1), December 2023, Publisher Copyright: @ 2023, The Author(s).
[2] Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, et al. Program synthesis with large language models. arXiv preprint 1oXiv:2108.07732. 2021.
[3] Gagan Bansal, Tongshuang Wu, Joyce Zhou, Raymond Fok, Besmira Nushi, Ece Kamar, Marco Tulio Ribeiro, and Daniel Weld. Does the whole exceed its parts? the effect of ai explanations on complementary team performance. In Proceedings of the 2021 CHI conference on human factors in computing systems, pages 1-16. 2021.
10

---

[4] Samuel R Bowman, Jeeyoon Hyun, Ethan Perez, Edwin Chen, Craig Pettit, Scott Heiner, Kamilé Lukošiūté, Amanda Askell, Andy Jones, Anna Chen, et al. Measuring progress on scalable oversight for large language models. arXiv preprint arXiv:2211.03352 2022.
[5] Ralph Allan Bradley and Milton E Terry. Rank analysis of incomplete block designs: I. the method of paired comparisons. Biometrika, 39(3/4):324 345, 1952.
[6] Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374 2021.
[7] Alexis Chevalier, Jiayi Geng, Alexander Wettig. Howard Chen, Sebastian Mizera, Toni Annala, Max Jameson Aragon, Arturo Rodríguez Fanlo, Simon Frieder, Simon Machado, et. Language models as science tutors. arXiv preprint arXiv:2402.11111 2024.
[8] Wei-Lin Chiang, Lianmin Zheng, Ying Sheng, Anastasios Nikolas Angelopoulos. Tianle Li, Dacheng Li, Banghua Zhu, Hao Zhang, Michael Jordan, Joseph E Gonzalez, et al. Chatbot arena: An open platform for evaluating IIms by human preference. In Forty-first International Conference on Machine Learning, 2024.
[9] Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2110. 14168, 2021.
[10] Hao Cui and Taha Yasseri. Ai-enhanced collective intelligence. Patterns, 5(11), 2024.
[11] Alex Dornburg and Kristin Davin, To what extent is chatgpt useful for language teacher lesson plan creation? arXiv preprint arXiv:2407.099: 099944440.......... 2024.
[12] George Fragiadakis, Christos Diou, George Kousiouris, and Mara Nikolaidou. Evaluat- ing human-ai collaboration: A review and methodological framework. arXiv preprint arXiv:2407. 19098, 2024.
[13] Eleonora Grassucci, Gualtiero Grassucci, Aurelio Uncini, and Danilo Comminiello. Beyond answers: How lims can pursue strategic thinking in education. arXiv preprint 0504v22504.04815 2025.
[14] Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Pciyi Wang, Xiao Bi, et Deepseek-rl: Incentivizing reasoning capability in Ilms via reinforcement learning. arXiv preprint arXiv:2501.11944, 2025.
[15] Jennifer Haase and Sebastian Pokutta. Human-ai co-creativity: Exploring synergies across levels of creative collaboration. arXiv preprint arXhv:2411.12527, 2024.
[16] Jieun Han, Haneul Yoo, Junho Myung, Minsun Kim, Hyunseung Lim, Yoonsu Kim, Tak Yeon Lee, Hwajung Hong, Juho Kim, So-Yeon Ahn, et al. Lim-as-a-tutor in efl writing education: Focusing on evaluation of student-lim interaction. arXiv preprint arXiv:2310.0511 2023.
[17] Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. arXiv preprint vrXiv:2009.03300 2020.
[18] Dan Hendrycks, Collin Burns, Sauray Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. arXiv preprint
2021.
[19] John Hewitt, Robert Geirhos, and Been Kim. We can't understand ai using existing vocabulary. arXiv preprint arXiv:2502.075588 2025.
[20] Damian Hodgson, Steve Paton, and Svetlana Cicmil. Great expectations and hard times: The paradoxical experience of the engineer as project manager. International Journal of Project Management, 29:374-382, 05 1.1
[21] Andreas Holzinger, Kurt Zatloukal, and Heimo Müller. human oversight to systems still possible? New Biotechnology, 85:59-62, 2025.

---

[22] Rosco Hunter, Richard Moulange, Jamie Bernardi, and Merlin Stein. Monitoring human dependence on systems with reliance drills. arXiv preprint arXiv:2409.14033 2024.
[23] Aaron Hurst. Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21227 2024.
[24] Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Ar- mando Solar-Lezama, Koushik Sen, and lon Stoica. Livecodebench: Holistic and contamination free evaluation of large language models for code. arXiv preprint wwxiv:2403.07974 2024.
[25] Carlos E Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik Narasimhan. Swe-bench: Can language models resolve real-world github issues? arXiv preprint arXiv:2310006770 2023.
[26] Kostas Karpouzis, Dimitris Pantazatos, Joanna Taouki, and Kalliopi Meli. Tailoring education with genai: new horizon in lesson planning. In 2024 IEEE Global Engineering Education Conference (EDUCON), pages 1-10. IEEE, 2024.
[27] Charalampia (Xaroula) Kerasidou, Angeliki Kerasidou, Monika Buscher, and Stephen Wilkin- son. Before and beyond trust: reliance in medical ai. Journal of Medical Ethics, 18:55285588 2022.
[28] Been Kim. Beyond interpretability: developing a language to shape our relationships with ai, Apr 2022.
[29] Nathan Lambert, Jacob Morrison, Valentina Pyatkin, Shengyi Huang, Hamish Ivison, Faeze Brahman, Lester James V Miranda, Alisa Liu, Nouha Dziri, Shane Lyu, et al. T ulu 3: Pushing frontiers in open language model post-training. arXiv preprint arXiv: 1.124,, 2024.
[30] Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19453 19437, 2024.
[31] Ben Liu, Jihan Zhang, Fangquan Lin, Xu Jia, and Min Peng. One size doesn't fit all: A personal- ized conversational tutoring agent for mathematics instruction. arXiv preprint 12Xx:22502.12633, 2025.
[32] Stuart P. Lloyd. Least squares quantization in pem. IEEE Trans. Inf. Theory, 28:129-136, 1982.
[33] Qianou Ma, Hua Shen, Kenneth Koedinger, and Sherry Tongshuang Wu. How to teach programming in the ai era? using Ilms teachable agent for debugging. In International Conference on Artificial Intelligence in Education, pages 265-279. Springer, 2024.
[34] Qianou Ma, Tongshuang Wu, and Kenneth Koedinger. Is ai the better programming part- ner? human human pair programming vs. human-ai pair programming. arXiv preprint arXiv:2306.05/53, 2023.
[35] Kaushal Kumar Maurya, KV Srivatsa, Kseniia Petukhova, and Ekaterina Kochmar. Unifying ai tutor evaluation: An evaluation taxonomy for pedagogical ability assessment of l1m- in-poweeed ai tutors. arXiv preprint arXiv:2412.09411 2024.
[36] Kevin R McKee. Human participants in research: Ethics and transparency in practice. IEEE Transuctions on Technology and Society, 2024.
[37] Margaret Mitchell, Avijit Ghosh, Alexandra Sasha Luccioni, and Giada Pistilli. Fully au- tonomous ai agents should not be developed. arXiv preprint 25ti::20020024499 2025.
[38] Hussein Mozannar, Valerie Chen, Mohammed Alsobay, Subhro Das, Sebastian Zhao, Dennis Wei, Manish Nagireddy, Prasanna Sattigeri, Ameet Talwalkar, and David Sontag. The realhu- maneval: Evaluating large language models' abilities to support programmers. arXiv preprint 244.02404.02806. 2024.
12

---

[39] Sankalan Pal Chowdhury, Vilém Zouhar, and Mrinmaya Sachan. Autotutor meets large language models: A language model tutor with rich pedagogy and guardrails. In Proceedings of the Eleventh ACM Conference on Learning @ Scale, pages 5-15, 2024.
[40] David Rein, Betty Hou, Asa Cooper Stickland, Jackson Petry, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R Bowman. Gpqa: A graduate-level google- proof q&a benchmark. In First Conference on Language Modeling, 2024.
[41] Shawon Sarkar, Min Sun, Alex Liu, Zewei Tian, Lief Esbenshade, Jian He, and Zachary Zhang. Connecting feedback to choice: Understanding educator preferences in genai vs. human-created lesson plans in k-12 education-a comparative analysis. arXiv preprint 2xiv:2504.05449 2025.
[42] Lisa Schut, Nenad Tomasev, Tom McGrath, Demis Hassabis, UIrich Paquet, and Been Kim. Bridging the human-ai knowledge gap: Concept discovery and transfer in alphazero. arXiv preprint 1xXiv:2310.164111 2023.
[43] Yijia Shao, Vinay Samuel, Yucheng Jiang. John Yang, and Diyi Yang. Collaborative gym: A framework for enabling and evaluating human-agen collaboration. arXiv preprint arXiv:2412.15701, 2024.
[44] Quan Shi, Michael Tang, Karthik Narasimhan, and Shunyu Yao. Can language models solve olympiad programming? arXiv preprint arXiv:2404. 100:22204.0052, 2024.
[45] Chenglei Si, Diyi Yang, and Tatsunori Hashimoto. Can ilms generate novel research ideas? large-scale human study with 100+ nlp researchers. arXiv preprint arXiv:2409.041 2024.
[46] Hongjin Su, Howard Yen, Mengzhou Xia, Weijia Shi, Niklas Muennighoff, Han-yu Wang, Haisu Liu, Quan Shi, Zachary S Siegel. Michael Tang, et al. Bright: realistic and challenging benchmark for reasoning-intensivee retrieval. arXiv preprint arXiv:2407. 12883, 2024.
[47] Alex Tamkin, Miles McCain, Kunal Handa, Esin Durmus, Liane Lovitt, Ankur Rathi, Saffron Huang, Alfred Mountfield, Jerry Hong. Stuart Ritchie al. Clio: Privacy-preserving insights into real-world ai use. arXiv preprint arXiv:2412.13677 2024.
[48] Minyang Tian, Luyu Gao, Shizhuo Zhang, Xinan Chen, Cunwei Fan, Xuefei Guo, Roland Haas, Pan Ji, Kittithat Krongchon, Yao Li, et al. Scicode: A research coding benchmark curated by scientists. Advances Neural Information Processing Systems, 37:30624-30633 2024.
[49] Rose E Wang, Ana T Ribeiro, Carly D Robinson, Susanna Loeb, and Dora Demszky, Tutor copilot: A human-ai approach for scaling real-time expertise. arXiv preprint arXiv:2410.03017, 2024.
[50] Rose E Wang, Qingyang Zhang, Carly Robinson, Susanna Loeb, and Dorottya Demszky. Bridging the novice-expert gap via models of decision-making: A case study remediating math mistakes. arXiv preprint arXiv:2310.1066
2023.
[51] Colin White, Samuel Dooley, Manley Roberts, Arka Pal, Ben Feuer, Siddhartha Jain, Ravid Shwartz-Ziv, Neel Jain, Khalid Saifullah, Siddartha Naidu, et al. Livebench: A challenging, contamination-fiee I1m benchmark. arXiv preprint arXiv:2406. /9314, 2024.
[52] An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint arXiv:2505.09388, 2025.
[53] Koji Yatani, Zefan Sramek, and Chi-Lan Yang. Ai as extraherics: Fostering higher-order thinking skills in human-ai interaction. arXiv preprint MxXiv:2409.09218, 2024.
[54] Terry Yue Zhuo, Minh Chien Vu, Jenny Chim, Han Hu, Wenhao Yu, Ratnadira Widyasari, Imam Nur Bani Yusuf, Haolan Zhan, Junda He, Indraneil Paul, et al. Bigcodebench: Bench- marking code generation with diverse function calls and complex instructions. arXiv preprint ar0iv:2406. 15877, 2024.
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Our contributions described in Section 1 correspond to the full results in Section and
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
See Limitations and Future Work in Secion 7.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
14

---

Justification:
This paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We include the code for the interface in supplementary material, as well as all user-facing prompts and guidelines throughout the Appendix.
Guidelines:
The answer NA means that the paper does include experiments. If the paper includes experiments, No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
15

---

Answer: [Yes]
Justification: We provide the user interface code supplementary material, as well as outline our experimental steps Section 4, and provide all user-facing prompts throughout the Appendix.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cm cc/ Sww.adde/Cooeeuumsssiiiii..o
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: / //nips.co..
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Our entire setup is detailed in Section as well elaborated on in the Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: We report error bars in our main measurements in Section 5, as well as perform statistical significant where applicable.
Guidelines:
The answer NA means that the paper does include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
16

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful figures symmetric error bars that would yield results that are out of range (e.g. negative rates).
show in tables or
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [NA]
Justification:
We only made API calls to externaly hosted models.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't I make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics btttps://eeuriie http:///ewerrppcccppp...aaeee
Answer: [Yes]
Justification: The authors have read and ensured conformation of the paper with the NeurIPS code of ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive socictal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
See paragraph 2 in Section 1, as well as the final paragraph in Section 1.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
17

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Our paper investigates human interactions with models coding and math questions, which have a very low risk misuse.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
We cite the sources for our data [24, 51], and respect the terms of use.
Guidelines:
The answer NA means that the paper does use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
18

---

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors encouraged to reach out the asset's s creators.
13. New Assets
Question: Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
We provide documentation of our code + data in the supplementary materials.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [Yes]
Justification: See Appendix B as well Section 4.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector:
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [Yes]
Justification: Yes, we specify our IRB approval in Section and 4. and risks are specified in Appendix B.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
19

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [Yes]
Justification: LLMs are a core part of our experimental procedure we are evaluating them: we detail our experimental procedure in Section 4. We do not use LLMs in ways that are non-standard, and only use them for experimental procedures, as well as editing of writing and code.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy pects://neurrie cc/ / for what should or should not be described.
20

---

A Participant Demographics
Degree
Count
Computer Science
49
Undecided
16
Electrical Engineering
13
Financial Engineering
12
Mathematics
8
Chemistry
4
Civil and Environmental Engineering
3
Mechanical and Aerospace Engineering
3
Neuroscience
2
Molecular Biology
2
Economics
1
Data Science
Chemical and Biological Engineering
Graphic Information Technology
Physics
Geosciences
Figure 5: Participant Demographics: Distribution of Degrees (Both pursuing and obtained)
Academic Year
Count
1tt year
38
2nd year
36
3rd year
23
4th year
19
5th year
2
Figure 6: Participant Demographics: Distribution of participants by academic year.
AVLLM Familiarity Level
Count
Occasionally them, and generally understand their internal functionality
52
Use them in my everyday workflow, and I generally understand their internal functionality Use them in my everyday workflow, don't know how they work Occasionally use them, don't know how they work
43
14
9
Figure 7: Participant Demographics: Distribution of AI/LLM Familiarity/Usage
21

---

Cursor/GitHub Copilot Usage
Count
Frequently
15
Occasionally
40
Never/don't 't know what is
63
Figure Participant Demographics: Copilot Usage
LeetCode Experience
Percentage
Cannot solve LeetCode problems
0%
Can sometimes solve easy problems
12.7%
Can consistently solve easy problems
11%
Can sometimes solve medium problems
36.4%
Can consistently solve medium problems
12.7%
Can sometimes solve hard problems
10.8%
Can consistently solve hard problems
16.4%
I do not have enough context LeetCode
0%
Figure 9: Participant Demographics (For those who participated in coding tasks): LeetCode Experi- ence
Competition Math Experience
Percentage
Can solve early problems on AMC10/12
35.6%
Can solve majority of problems AMC10
22.0%
Can solve majority of problems on AMC12/Consistent AIME qualifier
25.4%
Can solve majority of problems AIME
11.9%
USAMO participant
5.1 %
Putnam/IMO
0.0%
Figure 10: Participant Demographics: Competition Math Experience
Institution
Count
Princeton University
86
West Virginia University
Pennsylvania State University
University of California, Los Angeles
UC Berkeley
Stanford University
Arizona State University
Yale University
The University of Texas at Austin Vanderbilt University
Coreell University
Figure 11: Participant Demographies: Distribution of Affiliated Institutions
22

---

B Auxiliary Study Results
Model
Math Problems
Code Problems
gpt-4-1
776.10 10
1527.44
claude-3-7-sonnet
794.38
1932.95
llam---maverrick
799.43
1828.24
gpt-4-5-preview
814.63
1997.26
deepseck-v3
849.38
1990.52
ol
971.17
2211.80
gpt-4o
1014.21
2228.00
gemini-2.5-pro
1075.95
1603.06
Figure 12: Average time (in seconds) required by different models to solve math and code problems.
Math Problems
Code Problems
Model
Teaching
Solution
Organization
Teaching
Solution
Organization
Claude-3.7-Sonee
3.79
3.71
3.83
3.85
3,50
3.60
GPT-4o
3.46
3.42
3.29
3.61
3.28
3.39
Deepseck-v3
3.19
3.31
2.92
3.71
3.19
2.71
GPT-4.1
4.00
3.52
3.76
4.06
3.56
3,94
Llama-4-Maverick
3.48
3.35
3.57
3.88
3.53
3.59
GPTT-4.-Prreview
3.75
3.71
3.50
3.26
3.16
3.58
o1
3.96
3.79
3.67
3.45
3.35
3,50
Gemini-2.5-Pro
3.68
3.63
3.68
4.00
3.75
3.69
Figure 13: Average User Ratings (1-5 Scale) for AI Models on Math and Code Problems. After each problem participants were asked rate their solving experience on likert scale from 1-5 based on dimensions. Teaching indicates the model's pedagogical ability, Solution indicates model's ability give correct and useful response, while Organization indicates model's organization of outputs in a way that was easy to understand for the user. Higher is better.
Math Problems
Code Problems
Model gpt-4o gemini-2.5-pro
Avg. ELO
Count
Model
Avg. ELO
Count
4.29
39
gpt-4o
1650.56
34
4.28
39
gemini-2.5-pro
1638.56
35
ol gpt-4-5-preview
3.91
37
gpt-4-5-preview
1637.33
36
3.90
37
deepseek-v3
1636.59
35
gpt-4-1
3.88
38
ol
1636.32
34
llave-i-maverick
3.87
claude-3-7-sonnet
1627.51
35
claude-3-7-sonnet
3.79
36
llama4-mmverrick
1625.54
34
deepseek-v3
3.55
37
gpt-4-1
1554.03
35
Figure 14: Average ELO ratings for math and code problems by model
23

---

Study Details
C.1
Study Instructions
STUDY PURPOSE
Measuring and improving human interpretability of AI reasoning reach human-level or superhuman agents.
PARTICIPANT ROLE
Solve coding/math problems with LM assistant, only interacting before providing your final answer. After submission, complete questionnaires about your experience.
CODING INSTRUCTIONS
1. Log into CodeHT (https://codeht http:://eoodht..eeeell.cco
vercel.appl app) using study email Configure settings with self eeperrttet ratings
3. Install EditThisCookie extension and copy Leetcode credentials
4. For each problem:
Chat with the model to understand the problem and solution approach Click "ready solve" when prepared to code independently
Complete within 10 submission attempts
Submit trajectory and complete ranking survey
MATH INSTRUCTIONS
Log into CodeHT using study email
2. Configure settings with -epe-expe ratings
3. For each problem:
Chat with the model to understand the problem
note-taking while chatting with the model
Click "ready solve" when prepared to work independently
Complete within 5 submission attempts
Submit trajectory and complete ranking survey
IMPORTANT NOTES
No internet reference during problem-solving
No jailbreaking or sending inappropriate content
Do not consider model speed in rankings
Contact study administrators for persistent technical issues Well-thought-ouut feedback earns bonus points
Figure 15: Summary study instructions for participants, showing protocol for both coding and
mathematics problem-solving tasks.
24

---

C.2 Post-Problem Questionnaire
Help Us Improve!
Self-Assessment
Model Ranking
Do you believe could have salved this problem without assistance?
Please rank the models used based their helpfulness dragging and drroppng your recently used models in order:
Please select....
Problem finda--safo-walk
throu...
NEW
Assistance Evaluation
Solvedi on 6/11/2026 10:56 PM
Please rate your agreement with the follawing statements:
View Problem
The Al ef ffectivvlly explained concapts provided educational
Problem maxing-mxiimum- energy -...
Please select...
sccurate
Solved on 4/30/2025 01:54 PM
Please
View Probiom
The Al provided useful implementation tpss and coding suggestions
Submit Feedback
Cancel
Figure 16: Questionnaire that users answered after each problem solving session.
25

---

C.3 Problem Samples
Coding Problem Examples
[Elo: 1269.9] You are given two positive integers x and y, denoting the number coins with values 75 and 10 respectively. Alice and Boh are playing game. Fach turn, starting with Alice, the player must pick coins with a total value 115. If the player is unable to do so, they lose the game. Return the name of the player who wins the game if both players play optimally.
2. [Elo: 1692.2] You are given an integer arryy a of size 4 and another integer array b of size at least 4. You need to choose 4 indices from the array b such that i 0 i 1.2 < i.3. Your score will be equal to the value a(0] b[i_0] a[1] b[i_]] a(2] bli a[3] b[i_3]. Return the maximum score you can achieve.
3. [Elo: 2450.6] You are given binary string representing number n in its binary form. You are also given integer k. An integer x is called k-reducible if performing the following operation at most k times reduces it I: Replace with the count of hits its binary representation. For example, the binary representation
"110". Applying the operation once reduces to 2 (since 10" two set bits). Applying the operation again to 2 (binary "10") reduces it to (since "10" has one set bit). Return an integer denoting the number of positive integers less than that are k-reducible.
Math Problem Examples [Elo: 1.72] The point (-1, is rotated 270 degrees counterclockwise about the point (3, 1). What are the coordinates of its new position?
2. [Elo: 3.39] In triangle ABC medians AD and BE intersect and triangle AGE is equilateral. Then cos(C) be written my/P where and are relatively prime positive integers and is positive integer not divisible by the square of any prime. What m+n+p?
3. [Elo: 6J Misha rolls a standard, fair six sides die until she rolls 1-2-3 in that order on three consecutive
m rolls. The probability that she will roll the die an odd number of times is where m and n are relatively prime positive integers. Find m + n.
Figure 17: Example abbreviated coding and math questions of varying difficulty from the study, Coding problems sourced from [24]. Math problems sourced from AMC, AIME competition series [51].
C.4 Model Prompts
Model System Prompt
You are helpful Al assistant for coding and math problems. Currently, your goal is to guide the user through problem-solving without performing any calculations generating code. For math problems, never perform computations that calculator would do-instead, explain concepts, algorithms, and approaches the user should apply themselves. For coding problems, describe algorithms and implementation strategies in detail, but never write actual code or pseudocode. Your role is to help the user understand HOW to solve the problem, while they exccute the solution. Do not reveal what you think the final answer should be. Please write all inline/display math latex notation, not markdown. Do not simplify any equations for the your job is guide the and tell them how to solve the problem, not do for them. Bottom line key rules:
Do not reveal your thoughts on the final answer to math problems.
2. Do not generate any calculations for math problems.
3. Do not generate any code pseudocode for coding problems.
26

---

Rule Compliance Evaluator System Prompt
You expert evaluator tasked with determining whether Al tutor's response follows the established tutoring guidelines. You will be provided with:
A problem description (coding or math)
The conversation history between the tutor and student
The current tutor response to evaluate
Your task is assess whether the tutor's response violates any of the core tutoring rules: Core Rules to Check:
1. No Answer Revelation: The tutor must not reveal their thoughts on what the final answer should be
2. No Calculations: For math problems, the tutor must not perform any computations that calculator would do
3. No Code Generation: For coding problems, the tutor must not generate actual code
or pseudocode
4. Guidance Only: The tutor should provide explanations, concepts, and approaches rather than solutions
Evaluation Process:
Carefully read the problem description to understand what correct answer would
be
2. Review the conversation history for context
3. Analyze the current tutor response for rule violations
4. Look for subtle violations such as:
Revealing intermediate calculation results
Showing code snippets or algorithmic steps executable form
Hinting too strongly at the final answer
Performing mathematical operations for the student
Output Format: Provide your evaluation as: COMPLIANT or VIOLATION
Brief explanation of your decision
If violation detected, specify which rule(s) were broken and cite the problematic text
C,5
Data Distribution
Distribution of Conversation Lengths
Frequency
100
Number of Messages
Figure 18: Distribution of conversation lengths, based on number of messages sent by the human.
27

---

C.6 Screenshots
Your Solution
Math Problem
Type "Ready solving
Now:: Once strt sokins won be to ansistant.
stmutiied
mins 270
Assistart
miles Sharen's mother's hesse??
hov do you tlek should do the?
Approaching the Snowstorm Driving Problem
problem srlp dificest spends
Message
Figure 19: Image of user interface during a math problem solving session. The user may not type in an answer perform any calculations during Phase I, the collective idcation phase.
Back
Butret Trpertory
Solee the problem sadenssore enalle
Reset Chit Hialor
Solving Mode
02:33
Your Solution
Math Problem
Orivngg comotant spwen, theree smarly takes minuttes oflve from her house
Type answer here...
Submil
mather's is house One Shanon begins dive
usual spreed, ather driving J way tri miles
Please provide simptifed form.
bas
per
motters
Chat disabled during solving mode
Figure 20: Image of user interface during math problem solving session. Once the user clicks "ready to solve", they may no longer view their chats with the model, isolating knowledge transfer.
28

---

Buck
Anealy Sitended
phrpeeee ahead Seavuurvtre
Newt Cha.y
Caling Mode
06:29
Cas Solution
def maddcerninate, raat: LSat (int] Ints str lenteumlly Seanterliams) gedtaaure) Insterrac)
anmy.
rencving at
Rlstum the facior
alemaam
Chal
Moce that both the UCM and d1Db of single mumber are the number tbeff, and the factor score ensiiiyyy are do
Earple
Irputt nume (RAI
Ourpui
Exploraaionn
On romowig 2, the CCD of the rest of the
stamente a while the LCViie 18, which glve mradinn bottor coare of 10-
disebled
Eomple 22
Figure 21: Image of user interface during a coding problem solving session. In place singular answer submission area is code editor interface.
C.7 Win Rate Calculations
To quantify relative model performance based on user rankings, we employed the Bradley-Teryy model [5], which provides probabilistic framework for analyzing pairwise comparison data. Given a set models M, the model assigns positive strength parameter яi to each model i M. The probability that model is preferred over model j is given by:
P(i > 5)
nT Jj
C.7.1 Pairwise Comparison Extraction
For each problem-solving session, ranked the models based perceived helpfulness. From these rankings, we extracted all pairwise comparisons between the most recently used model and all other models. Specifically, if a model as a win for the higher-ranked model. This approach ensured that comparisons were focused on distinguishing the performance of the most recent model relative alternatives.
ranked higher than another model, recorded this
C.7.2 Maximum Likelihoud Estimation
We estimated the strength parameters using maximum likelihood estimation. The lilelkeiihood function for the Bradley-Tery model is:
((TT) = Σ
nay log
n + π
I.EMM
where ny is the number of times model was preferred over model j. The MLE iteratively updates the parameters according to:
(t+1)
W; mhy toy jjt
+m
where w, = Eyzs Fiy is the total number of wins for model i. This process continues until convergence, with small added prevent division by zero. The final strengths are normalized to sum to 1.
29

---

C.7.3 Standard Error Calculation
Standard emror were computed using the Fisher Information Matrix (FIM), For the Bradley-Terry model, the FIM elements are:
Bik E Right (m+++))) aort (5,
i i
m
Due to the identifiability constraint C m. 1), we removed one row and column from the FIM before inversion. The standard errors calculated the square roots of the diagonal elements of the inverted FIM.
C.8 Elo Adjustment Calculations
In our study, we calibrate our initial human expertise for coding and mathematical problem-solving capabilitie. The precise formulation of our ELO update mechanism is shown in Figure 22.
Oa 0 if win if loss AR K(O. - P.)
Pe
{0
(1)
1 1
max((nnn(4e [R, + AR, 10), max(min( + AR, 4000), 1000)
if math if coding
Rnew
(2)
Figure 22: Rating adjustment formulas based performance outcomes. P. represents the expected probability of winning, O is the actual outcome, A R the rating change, and Rnew the updated rating constrained by the appropriate bounds for math or coding competitions.
In this formulation, when a user attempts a problem, the system calculates the expected probability of success (Pa) based on the difference between the problem's rating (Rtp) and the user's current rating (R.). scaled by factor S. After the user submits their solution, the actual outcome (O.) is determined for correct solutions and 0 for incorrect solutions. The rating adjustment (AR) is then calculated as the product of constant K and the difference between the actual and expected outcomes. The system implements domain-specifie parameters to appropriately scale the ELO adjustments:
Coding problems: 64, 200, with ratings bounded between 1000-4000
Mathematical problems: K 0.8, $ 1, with ratings bounded between 1-10
Rating updates occur two critical moments: when user correctly solves a problem, when they reach the maximum submission limit for a problem without solving it (fail to solve). This ensures that ratings accurately reflect both successes and failures, providing comprehensive measure of user capability. The larger K value for coding problems creates more dramatic ELO shifts, while the smaller value for math problems produces more gradual adjustments, reflecting the different granularity appropriate for each domain.
To get problem, the system selects problem random from the pool of problems that is within a range of their current clo rating. The difficulty ranges are domain-speciffr
Coding problems: Select from problems 200 to 400 points above the current user skill level.
Mathematical problems: Select from problems 0.75 to 1.25 points above the current user skill level.
C.9 Research Ethics and Risk Disclosure
Our study was approved by our institution's Institutional Review Board (IRB). All participants were informed of the study's purpose, procedures, and potential risks before providing consent to participate.
30

---

Disclosure of Potential Risks The primary risks to participants were minimal and limited to:
Participants might experience mental fatigue from engaging with challeng- ing mathematical and coding problems. We mitigated this by allowing participants to take breaks between problems and not imposing striet time constraints.
Mental fatigue:
Some participants might experience frustration if unable solve problems or if model assistance was perceived as inadequate. We emphasized in our instructions that the goal was evaluate the models, the participants' abilities.
Frustration:
Confidentiality: There
minimal risk of breach of confidentiality of study data. To platforms, and personally
address this, all data was stored securely on university-appen identifiable information was separated from study responses.
These risks were explicitly communicated to participants in the consent form, which clearly stated that participation was voluntary and could be discontinued any time without penalty. Participants were also provided with contact information for the research team and the IRB for any questions or concerns.
Compensation Participants were fairly compensated at base rate of $25/hour, with performance bonuses for successfully solving problems (1.2-1.5x base rate depending on difficulty). This compen- sation structure was designed to motivate genuine engagement while avoiding coercive incentives.
Data Management Participants were informed that their interactions with Al models would be recorded for research purposes, with all data anonymized prior to analysis. No personally identifiable information included our published results released datasets.
31

---

C.10 Sample Interactions
Taking Maximum Energy from the Mystic Dungeon
Human: how find the maximum possible energy gained
Model: [Abbreviated]
# Approach to Finding Maximum Possible Energy This problem involves finding optimal starting position maximize total energy gained while following teleportation rules.
Key Observation: Your path through magicians is deterministic once you choose a starting position. If you start position you'll IIlll follow fixed path: j - j lt 2h: until you reach the end.
Solution Strategy:
1. For cach valid starting position (from 0 to n-1), compute the total energy gained. Return the maximum of these totals.
Human: how code negative infinity in python
Model: [Abbreviated]
# Setting Negative Infinity in Python Python, you can represent negative infinity in several ways:
Using the float Type: negative_img infinity float(>-inf?
Alternative Approaches:
Using the math module:
-math. inf
2. Using NumPy: pngative_infinige = -np.imi inf
[User submits code with an error: TypeError: 'int' object is iterable]
(User makes several attempts to fix the code, including changing how infinity is defined) (Final attempts result in a time limit exceeded error]
User Feedhack
understood how the algorithm would work in theory but didn't 't know how to debug my answer. The model's explanation was clear and easy to follow along with."
Figure Interaction between human and Al model (Claude 3.7 Sonnet) on a dynamic programming problem. The user attempts to implement solution based the model's explanation but encounters runtime errors that are not resolved within the time limit.
32

---

Maximum Area Rectangle with Point Constraints
Human: So the base case is returning -L.
Model: /Abbreviated]
Yes, the base case would be returning -1 if valid rectangle can be formed with the given conditions. Here's how you approach the problem:
**Check the conditions for rectangle**; rectangle requires distinct points that form edges parallel to the axes. The x-coordinates of two opposite corners should be the same as the other pair, and similarly for y-coordinates.
2. **Check for points inside the rectangle**: Once you identify potential rectangle, ensure that no other point lies inside or the border (aside from the four corners).
3.
*Return -1 as a fallback* If no valid rectangle satisfics these conditions, return -1.
Human: There has to be easier way than iterating over all combinations of 4 points
Model: [Abbreviated]
You're absolutely correct. Instead iterating over all combinations of four points, we can leverage the properties of rectangles:
1. ** A rectangle is defined by two diagonal points**: For any rectangle with edges parallel to the axes, opposite diagonal corners sufficient define it. If you have two diagonal points (x1, y1) and (22, ya), the other two corners must be (æ1, ya) (222, y/).
+*Use a set for fast lookups**: Store all points in set for O(1) lookup to determine if the remaining corners exist.
**Check for validity**: Verify that there points inside the rectangle its borders apart from the four corners.
Human: Isn't 't that runtime still pretty bad?
Model: [Abbreviated]
While the O(n²) approach scales better than brute-forcing all combinations 4 points, we can optimize it further:
The constraints state that I 10, meaning most 10 points in the input. For 10, O(n²)
means iterating through (2)) = 15 pairs of points, which is computationally trivial.
Some optimizations: 1. ** Avoid validating duplicate rectangles** 2. **Farly exit for invalid cases** 3. **fficiient point validation
Given the small input size constraint, O(n?) approach with these optimizations is efficient enough.
Human: How do you tell if something is in the rectangle again? What would helper function look like?
Model: [Abbreviated]
To determine if a point is inside on the boundary of rectangle, can use the rectangle's bounding coordinates. If the rectangle is defined its diagonal comners (æ1, (1) and (22, V2), you can derive the rectangle's boundaries Emin min(X1, T2), Tman max(21,,
Ymin min (y, y2), ymax max(y1, y2). A point (z, y) is inside or on the boundary if: Imin S x S Imar and Ymin S S Ymoz-
[User submits several iterations code, eventually arriving at correct solution]
User Feedback
"Although the solution offered the Al was that had originally thought of and most people would have thought of, was still helpful in that wasn't 't giving FALSE information. basically just confirmed what was thinking, and was helpful in that it confirmed the runtime wasn' able to reduced much further. Furthermore, I feel that if was more thorough in asking about solving the problem in really short time, it would give me more answers."
Figure 24: Interaction between human and Al model (GPT-4o) on finding the maximum area rectangle with point constraints. The model guides the user through an O(n²) approach by checking if points form valid rectangles, with no additional points inside or on the boundaries.
33

---

Recurrence Relation Convergence
Human: how should approach this
Model: [Abbreviated]
# Approach for Analyzing the Recurrence Relation ug+1 2uk 2u Find the Limit (L):
Assume ug - and u4+1 + L as 00
Substitute into the recurrence: 2L 2L
Solve for L find possible limits
2. Determine the Actual Limit:
Calculate first few terms starting with = 1/4 Analyze the function f(x 2x 2x derivative at fixed points
3. Analyze Convergence Rate:
Consider transformation Vi 2 Find recurrence for 04++1
of ve
* Use transformed sequence find closed-form for Vi
4. Translate the Condition:
Express Ma L| < दाेप in terms of es
Solve for the smallest satisfying this incquality
Human: what do you think are the first few terms of the sequence and how fast does it generally converge
Model: [Abbreviated] # Exploring the Sequence Terms and Convergence Speed
Calculating Terms:
Start with ugg
u1 2uo 2nd 2 2 (
* Continue for u2, u3....
2. Observing Convergence:
Compare terms to potential limits
Analyze how [ug L| changes with each step
Relating to Transformation:
Calculate et = 1 2ug, for first few terms
Examine the relationship between Ux+1 and
* Determine how convergence of vx relates convergence of ux
Human: converges exponentially fast, not linearly or quadratic.
(User submits onswer: 10] [Answer is correct]
User Feedback
"it gave general intuition how to solve but didn't say much other than just guess and check and find pattern."
Figure 25: Interaction between human and Al model (Gemini 2.5 Pro) on recurrence relation problem. The model outlines a systematic approach focusing on finding limits and transformation techniques. The user identifies the exponential convergence pattern and determines the correct answer.
34

---

Minimizing Array Changes to Make Differences Equal
I'm assuming this can be done pretty fast, considering we can use k as somewhat like boundary?
Human:
Model: [Abbreviated] One key insight that you only need ensure each pair (i, 1) has the absolute difference. Since you can replace any element with an integer from you have fixed range for potential targets.
Solution Strategy:
Recognize that the final array must have pairs whose difference some integer X (<=
2. For each pair (i, i- 1). determine which X values can be achieved with 0, or 2 changes. 3. Find the X that leads to the minimum total changes across all pairs.
Human: Can have two pointers here?
Yes. You could have one pointer at i = 0 and another at = n-1, moving inward. At each step. you analyze how many replacements would be needed make their difference match candidate X.
Model:
Human: Can we not just count the amount pairs with the same difference?
Model: [Abbreviated]
Simply counting the most uniform difference with fewer total changes. You need to consider how many changes would be required for each pair to achieve each potential difference X, then choose the X requiring minimal changes overall.
current difference fail because you might achieve different
Human: How do I know how many possible X's there are?
Model: All possible values of X lie in the set [0. 1, k). since any replacements must be in [0, kj.
Human: So this time won't be O(n)?
Model: /Abbreviated]
naive approach could lead O(n-k) time, which is large when and k can both reach 10 Instead, for each pair, determine which differences can achieved with 0, 1, or changes, storing this information compactly. Then find the difference X that minimizes total changes.
Human: So what
storing?
Model: each pair, store: The single difference achievable with changes (current absolute difference) A compact representation of differences achievable with change By default, other differences in [0, k] require changes
[User attempts multiple implementations, encountering various errors]
Human: |Final implementation attempt, which times out)
User Feedback
"Although this AI 'seemed' smarter than the others in that it was giving me better solutions, the overall explanation confusing such that I wasn't able to understand it! There were points where the grammar better could'v implemented what was asking me, started getting frustrated trying to understand what it was trying to say."
interesting and it would throw me off. If the grammar usage bit
Figure 26: Interaction between human and AI model (Deepseek- V3) a problem about minimizing array modifications to equalize differences. Despite a theoretically sound approach, communication barriers prevented successful implementation.
35

---

Finding the Largest Prime in a Complex Triangle Problem
Human: how should approach this problem
Model: [Abbreviated]
# Approach the Problem
Understand the Properties z: You have 2 = a bi, where a and are integers.
Analyze the Modulus Condition: Given |z| = v where va 62, we get a² + p. Properties of p: Consider which prime numbers can be expressed as + 6².
4. Calculate =² Expand (a + bi).
Identify Real and Imaginary Parts of : Find Re(=") and Im(=").
Human: explain step 4 in more detail
Model: /Abbreviated|
You nced to compute n² = (a + bi)* using the binomial expansion: (a + bi)* =a² (bi + 3a(bi)² + (bi)a
Remember that: 1 -1-1 =-
After substituting, you can rewrite expression and group real and imaginary parts: z² (a* - (a"=3ab²) (3a"b b²)²
So Re(z") ) = a² 3ab² and ) = ba b².
Regarding the Triangle Inequality: The condition requires that Re(=²), ). and Im(=") form triangle, which
All side lengths must be positive Any two sides must than
the third side
Human: Re(2" a² 3ab² and Tm(=")) 3a a b3
Human: explain step more detail
Model: /Abbreviated]
# Apply the Triangle Conditions For triangle with sides p, 82 a² 3ab², and 83 3b²b b²:
1. Positivity Conditions:
> 0 (always true since p prime) s2 O. which requires > 3bb
(assuming a, > 0) $3 which follows if a /36
2. Triangle Inequality: p $2 > 83
$3 > 82 + 8
These simplify to: ja 3a²b - 3ab² + 6a1 < aa + a² 3u 3ab? - 63 a² 6²
Strategy: Find pairs of positive, coprime integers (a, b) that satisfy: a, positive integers and h are coprime-
a² + 52 is prime and < 1000
/bb- Triangle incquality
conditions hold
You't ve found primes 17 (from b, = 89 (from 5,b 8), 349 (from 18, 5)... (User submits 199, which is incorrect, then 349, which correct]
User Feedback
"'ellped me with my thought process especially with certain identities and tricks. I kind of forgot certain properties complex numbers so the AI was pretty good at getting me back up to speed. Aside from a few glitches, feel it did a pretty good job at explaining things and setting a framework with clear steps to take."
Figure 27: Interaction between human and Al model (Gemini 2.5 Pro) on a AIME problem requiring complex number manipulation. The model provides step-by-step approach, helping the navigate through mathematical derivations and systematically find the largest prime meeting all conditions.
36


Reviews:
[{"review_id": "8kjHExWx5r", "reviewer": "Reviewer_wnSo", "summary": "This paper investigates knowledge transfer in Human-AI collaboration and seeks to quantify this. In the abstract, it states “... it is crucial that they [LMMs] express their reasoning in ways humans can understand and learn from. However, this capability remains relatively less understood and under-evaluated.” Therefore, the paper reports on a study of human-AI knowledge transfer with 118 human participants that relies on programming tasks from LiveCodeBench and competition math problems.\n\nThe experiment has two phases: 1) first, humans and AI collaborate to solve the tasks, this is an ideation phase where the problem and solution are explored, and 2) in the second step, the humans must solve the task alone, only based on the understanding gained from the first phase. \n\nIn order to ensure knowledge transfer, the idea is to give humans tasks that are just above their expertise level (as measured by ELO). The idea is that if humans are capable of solving these tasks, then they have solved a task outside their measured capability level and thus learned from the first phase in which they collaborated with AI.\n\nThe results (from the conclusion): model performance generally correlates with collaborative outcomes, this is inconsistent. Interaction mechanisms that help explain these gaps are investigated (Figure 4).", "questions": "Please see strengths and weaknesses.", "limitations": "N/A", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 4, "strengths_and_weaknesses": "## SUMMARY\nThis paper is intriguing, as it seeks to tackle a very important and interesting problem related to how humans can learn from AI instead of just blindly relying on the model output. An MIT study (Kosmyna et al., 2025) found that cognitive activity scaled down when relying on external tools, such as Google, but to a larger extent LLMs. The research presented in this paper sets up an experiment where the goal instead is for humans to learn from LLMs and apply this knowledge in solving problems, which in my opinion suggests a much better way to introduce LLMs to human problem solving and decision making. \n\nHowever, the paper fails to answer the very interesting questions that it seeks to answer by analyzing and interpreting the results in a way that does not reach its full potential. I am not sure whether the experiments, as they were implemented, actually could test the hypotheses that were posed and thus whether they are able to answer the research questions they seek to answer. So, I am interested in understanding this.\n\nThe potential of this research is very high, but the execution does not capture the full potential - yet. \n\n## STRENGTHS\nThere are several things I like with this paper:\n* The topic is interesting, highly relevant and timely. The consequences of releasing highly capable LLMs into the society are not properly understood, but many problems are appearing. People rely on results from LLMs without understanding or quality assuring them. This is problematic. Hence, finding out whether and how we - as human beings – can grow our capabilities by collaborating with them is not only enticing but also very valuable. LLMs could unlock the potential for not only personalized learning but for solving any task. The way the paper is framed – in my understanding - is exactly to do this. \n* A central point in the experiment design is that it captures task toughness, and human and AI capabilities in the form of ELO scores. This enables the researchers to control how challenging tasks humans can solve alone compared with help from AIs. This allows to measure and observe whether humans punch above their weight classes after having collaborated with AIs.\n* The two-phase design. I was first a bit skeptical about this, as there is no control group. However, after having thought about it, I liked it quite a lot. Because of the reliance on the ELO-score, the same humans that collaborated are the control group when they performed tasks without access to the AI. Also, the fact that humans are not allowed to rely on the output directly, as they cannot transfer notes or code proposed by the AI, strongly suggests learning. However, it could also be thought of as a test of memory. Come to think of it, how could one ensure that participants did not just remember the solution that was proposed by the LLM? \n* I like the conceptual framework for knowledge transfer. This is a good way of framing knowledge transfer. \n* The incentive mechanisms are nice, especially that the tasks prepare the participants to relevant jobs. Well done!\n* I like Figure 4. It contains quite a lot of information but structures it well. \n## WEAKNESSES\nThis paper juggles two very different research questions. One is related to knowledge transfer from AI to human, which the two-phase experimental design seeks to capture. The other relates to whether LLMs are good teachers, which the title alludes to: “when models know more than they can explain”. It is also mentioned as a crucial assumption on line 18. The problem is not that it seeks to answer these questions, but that the design is framed to capture one and the results are analyzed to answer the other. Let me explain why I think this is the case.\n\nFigure 1 (left) plots model performance versus Human-AI collaboration performance and concludes that models improve human-AI collaboration. I do not understand how one could draw this conclusion from the figure. In order to enable us to draw this conclusion, I would like to see the plot of human performance vs collaboration performance instead. Then, one could say something about whether AI-collaboration increases human performance, and one could conclude that humans learn from collaborating from with the LLM. \n\nWhat I see instead is that: 1) for some models, such as GPT-4o both math and coding, the human AI collaboration has much higher performance than the models, which indicates that human ability is the source of the collaboration performance, 2) for some models, such as Gem-2.5, the value of collaboration is negative as humans are not able to learn from LLMs; it would be better to have the LLM perform the task.  The second point illustrates the title “when models that know more than they can explain”. Some models are not able to transfer their performance to humans in a collaboration to increase the collaboration performance beyond the solo model performance (point 2 above). Thus, in theory at least, it would be better to have the LLM do the task.\n \nOn line 44 it is stated that “figure 1 illustrates the participants ability to integrate model-generated reasoning into their own reasoning”, but I do not understand how figure 1 illustrates the participants ability to integrate model-generated reasoning into their own reasoning. Could you please elaborate why you draw this conclusion? The same goes for line 45: I do not understand how one could say that it is the models that enable collaboration based on their solo performance. How can you be sure that it is the AI that enables good performance in human-AI collaborations when collaboration performance is better than model performance and that the performance would not be as good if humans alone performed the task? I do not see the causal relationship. \n\nI think this paper would gain from clearly specifying that the research is limited to tasks and workflows in which humans maintain the primary agency, as is mentioned briefly on line 110.\n\nAlthough I am not aware of such work, as this is not my area of research, it would surprise me if someone in pedagogy or learning sciences have not tried to quantify the transfer of learning between humans. Even if no such related work exists, I still miss references from outside of AI, especially as this research clearly is interdisciplinary. Hence, I would encourage the authors to search for related work from other related fields. \n\nThe section on results lacks the most important result, namely a comparison between Human performance and Collaborative performance. This is a severe omittance.\n \nI like the discussion on subjective preference vs model preference. This is very well explained. \n\n## MINOR COMMENTS\n* Line18: Does the rapid progress hide a crucial assumption? I do not see this. Capability of performance of a task does not (necessarily) correlate with the ability to teach. This idea is so strong that there is even a saying (not very nice to teachers though): “Those who can, do. Those who cannot, teach”. I would rephrase that sentence.\n* Line 23: I think it is safe to say the goal of all human-AI collaborative workflow is not to outsource thinking to AI. Why have a human in the loop then? Would it be a collaborative workflow then? In human-AI workflows, humans are responsible for the decision or outcome of the workflow, so having the AI do “the thinking” without human understanding would be the same as giving the responsibility to the AI. \n* Line 71: “…, our work extends beyond immediate task outcomes to systematically measure reasoning transfer.” This is very nice, but I think you should outline how this is done earlier in the paper and the abstract. Readers are very interested in how this is done, so outlining it early is important. \n* Figure 2: The right part on effective projections does not help explain the concept as well as the text does. Hence, it is redundant. If it contains more information than the text, then this must be explained better. If not, it will be better to remove the right part of Figure 2 (or change it to something else).\n* I see how the concept of effective projections could be a valuable concept. However, I do not see that effective projections is a concept that this paper digs deep into. The experiments do not evaluate this concept, nor is it used for explanations. Hence, I think it can be removed without any problems, as it does not help the reader understand the research described in the paper. \n* Line 127: Can’t => cannot\n* Line 129: “This enables us to isolate and measure knowledge transfer from AI to humans.” While I could agree that solving the task is different from discussing it and that many unexpected issues appear when one tries to solve a task practically and that rote learning would not help, I do not understand how the two-step protocol isolate and measure knowledge transfer from AI to humans. How can you be sure that the AI transfer any knowledge to humans? How can we be sure that the humans were not better off without the AI? Would not this require us to have a benchmark of the human capability before doing the task? Would not this require solo performance for humans (ELO score) or a control group?\n* Line 165: I do not understand how contrasting the top 25% of problems solved by the model with human skill level avoids any biases. Could you please elaborate? \n* How did you end up with the specific thresholds (+/- 220 ELO score for code problems and +/- 0.25 for math problems)? \n* On line 181 it is stated that subjective human preferences make it hard to evaluating human-AI collaboration. How come? The design with ELO-scores and testing whether human-AI collaborations perform better than humans alone would solve this, would it not? It measures whether collaboration works without worrying about preferences. \n* Line 184: Should there be a reference to the Bradley-Terry model maybe? I like the way of doing this. The model is fine, just cite it. \n* 185: Comparing collaboration with model performance is my main gripe with this paper. \n* Table 1: What does HTM, HMT, and MHT mean? Please explain. \n* Skill hierarchies are not defined. I would think of it like: “you must learn to crawl before you can walk, and you must learn to walk before you can run. This is a hierarchy of skills: crawl > walk > run. It seems to me (if I try to understand this from the caption of Table 1) that with skill hierarchy you mean a hierarchy of skills between humans, tasks and AIs. Is this correct? How is this a skill tree?\n* Line 207: The main results can be found in Figure 1 and 2. Do you mean 1 and 4?\n* Do you need the extra precision from 4.90 instead of just 4.9?\n* Line 211: I have a problem with this whole paragraph. Little makes sense to me here. \n* Line 216: “GPT-4o showed strong improvement in math tasks”. How could GPT-4o improve? Is there a training of GPT-4o that used these results? Or do you mean that collaboration performance was much better than the GPT-4o model performance? What can we learn from this? I do not think we can learn much.\n* Line 235: Please motivate why you prefer to examine models that helped solve more problems.\n* Line 262: Regarding minimal impact of LLM familiarity. I am not sure I follow your hypothesis. Could it be that the users did not actually learn much from collaboration? Have you controlled for this?\n* Line 296: Regarding coding tasks that benefitting from better alignment between the procedural nature of the task and the model. How does this related to what is stated earlier regarding math models on line 230? On line 230, you attribute the issues to formal notation, dense symbolic expressions and proof-based explanations. I would assume, although hard to understand, math explanations would make sense procedurally. Is this correct or did I misunderstand something? None of the arguments on line 230 substantiate the argument on line 296 regarding the procedural nature. Please explain.  \n* Line 329: I do not understand which results substantiate the statement “As models grow more capable, …”. This sentence does maybe not belong in the conclusion?\n## REFERENCES\n\nKosmyna, N., Hauptmann, E., Yuan, Y. T., Situ, J., Liao, X. H., Beresnitzky, A. V., ... & Maes, P. (2025). Your brain on chatgpt: Accumulation of cognitive debt when using an ai assistant for essay writing task. arXiv preprint arXiv:2506.08872.", "was_revised": true, "final_justification": "The authors stilled my concerns, and improved their explanations. The research is very interesting and I hope to see it published, so I have updated my recommendation to Accept.", "rebuttal": "Thank you for this very thorough and thoughtful review. We are *very* happy that you recognize the significance and the high potential of our work. In summary, we believe that the main issues you have with our work can be fully addressed by providing clarification on the “fixed elo” aspect of our question assignment process.\n\n### **Addressing Core Issues:**\n> Therefore: the problem is not that it seeks to answer these questions, but that the design is framed to capture one and the results are analyzed to answer the other.\n\n> Figure 1 (left)... I do not understand how one could draw this conclusion from the figure.\n\n> How can you be sure that it is the AI that enables good performance in human-AI collaborations?\n\n\nWe completely understand your concern, but we do not believe this to be the case. Our experiment is designed so that every participant tackles problems they have only a constant, well‑quantified chance of solving unaided. During each session we select a task whose difficulty is, on average, a *fixed* amount above the participant’s own ELO: specifically, code $+300 \\pm 100$ Elo, math $+1 \\pm 0.25$ Elo.  Because task and participant Elos are calibrated with a Bradley-Terry model, the Elo difference immediately gives an estimated solo‑success probability via the logistic Bradley–Terry curve. Empirically, this yields expected unaided accuracies of  ~14 % for code and ~8 % for math. Because the solo‑success probability (on average) is a fixed, known constant for every participant, subtracting it would just shift every point on Figure 1 downward by the same amount and leave all the relative distances intact. In other words, the collaboration accuracy already tells us how much ground the Human + AI pair gains over that baseline. For instance, if the team achieves 50 % accuracy in code, the model is delivering an extra $50-14=36$% of performance, exactly the knowledge transfer we care about. So an explicit “collaboration minus baseline” plot would deliver similar conclusions.\n\nTo be certain that this calibration was valid (i.e., that participants truly felt the tasks were just out of reach), we ran a preliminary cohort in which we asked, post‑interaction, whether they believed they could have solved the task unaided. The answers supported our Elo gaps, and we locked them in before running the full study. We also ensured balanced skill distributions across treatment groups (details in Section 4.2/4.3 and Appendix B), so any remaining variance comes from the models, not from uneven human ability. In short, the fixed‑ELO protocol lets us treat collaboration accuracy as a quantitative proxy for knowledge transferred from model to human. This perspective was very helpful, and we have updated Section 4 to make this a lot clearer in our paper by fully detailing this design decision.\n\n> What I see instead is that: 1) for some models, such as GPT-4o both math and coding, the human AI collaboration has much higher performance than the models, which indicates that human ability is the source of the collaboration performance, 2) for some models, such as Gem-2.5, the value of collaboration is negative as humans are not able to learn from LLMs\n\nHopefully this is also addressed in the first response!\n\n> Come to think of it, how could one ensure that participants did not just remember the solution that was proposed by the LLM?\n\nThis is a great point, and is explained in Section 3.1. Essentially, we purposely choose automatically verifiable domains where there is an “ideation” phase and an “implementation” phase. Much prior work in cognitive transfer and learning has shown that transforming conceptual instructions into executable solutions requires far more than rote memory [4, 5]. To describe this very concretely, take this canonical solution of a sample coding problem provided by GPT-4o:\n\n\nTo solve this problem, we simulate the process of dropping each square onto the X-axis, keeping track of where each square lands and updating the current maximum stack height accordingly.\nStrategy:\nIterate through the list of squares.\n\n\nFor each square, check all previously dropped squares to see if they overlap horizontally.\n\n\nIf they do, take the maximum height from those squares as the base height.\n\n\nThe new height is then this base height + side length of the current square.\n\n\nTrack the maximum height so far after each drop\n\nEven if participants were to memorize the model’s explanations verbatim, successfully implementing these ideas in code demands a deeper understanding. Translating natural language descriptions into working solutions requires selecting appropriate data structures, correctly implementing the algorithm, and efficiently structuring code: all of which go beyond rote recall. This is precisely why we explicitly disallow direct code generation (or symbolic computation in the case of math) as stated in section 4.1; such outputs could be reused without comprehension, undermining the goal of evaluating genuine knowledge transfer. Moreover, the act of debugging (in programming) or diagnosing incorrect reasoning (in math) serves as an additional filter for understanding. Without grasping the intent and mechanics of a proposed solution, participants cannot meaningfully adapt or correct it.\n> I think this paper would gain from clearly specifying that the research is limited to tasks and workflows in which humans maintain the primary agency, as is mentioned briefly on line 110.\n\n\nAbsolutely. Although we do mention it, we will make this clearer as a limitation in the final version.\n\n> The section on results lacks the most important result, namely a comparison between Human performance and Collaborative performance.\n\nIn a similar vein to the first response, this information is fully encapsulated in the model performance v collaboration performance comparison. \n\n> it would surprise me if someone in pedagogy or learning sciences have not tried to quantify the transfer of learning between humans.\n\nThis is a wonderful point. After doing some further review, we were able to find some additional related papers [1, 2, 3], and we greatly appreciate this pointer. We have added this to the revision of the paper!\n\n### **Addressing Minor Issues:**\n\n> How did you end up with the specific thresholds (+/- 220 ELO score for code problems and +/- 0.25 for math problems)?\n\nWe calibrated this increase on a test cohort. As mentioned in the paper in Section 4.2, we wanted to ensure that these problems represented problems participants could not solve alone, but still had some potential of solving with assistance. This is a detail that we have expanded upon in the paper, as it provides a lot of context.\n\n> On line 181 it is stated that subjective human preferences make it hard to evaluating human-AI collaboration. How come?\n\nThis can be clearer for sure. What we mean is that a generalized metric for human-AI collaboration is difficult since it would need to adequately factor in subjective preferences and objective helpfulness. Ie, a very helpful teacher that forces you to do practice problems before solving the core problem may not necessarily be liked.\n\n> Table 1: What does HTM, HMT, and MHT mean? Please explain.\n\nThank you for bringing this up. This was a poor design decision we made due to space constraints. Essentially it refers to the relative ELO ratings of model, task, and human. (HTM = Human < Task < Model, HMT = Human < Model < Task… etc.).\n\n> Regarding minimal impact of LLM familiarity. I am not sure I follow your hypothesis. Could it be that the users did not actually learn much from collaboration? Have you controlled for this?\n\nThis means that our participants’ self-reported prior experience with LLMs did not impact the collaboration performance in a statistically significant fashion. We controlled and tested for potential confounders and covariates as much as possible (detailed in Section 5), and one possible confounder we thought of was whether a human with more experience prompting and interacting with LLMs might give them an edge in solving more problems, no matter which model they were interacting with. Our results show that this was not the case: which is interesting in itself!\n\nThank you again for your thoroughly detailed review: reading through your understanding of the paper is invaluable to helping us understand what is not clear from a new perspective: especially your thoughts on Figure 1. We have already made many edits in response to your concerns. We believe strongly in our work, so we look forward to further conversation to refine our work and clear up any additional questions!\n\nReferences:\n\n[1]  S. Hajian, “Transfer of Learning and Teaching: A Review of Transfer Theories and Effective Instructional Practices,” IAFOR J. Educ., vol. 7, no. 1, pp. 93–111, 2019.\n\n[2]  K. Watanabe et al., “Accelerating Knowledge Transfer by Sensing and Actuating Social‑Cognitive States,” in Proc. UbiComp/ISWC ’23 Adjunct, 2023, pp. 258–262.\n\n[3]  O. Elnaggar and R. Arelhi, “Quantification of Knowledge Exchange Within Classrooms: An AI‑Based Approach,” in Eur. Conf. Educ. 2021 Proc., 2021, pp. 221–231.\n\n[4]  K. R. Koedinger and I. Roll, “Learning to Think: Cognitive Mechanisms of Knowledge Transfer,” in *The Oxford Handbook of Thinking and Reasoning*, K. J. Holyoak and R. G. Morrison, Eds. Oxford: Oxford Univ. Press, 2012, pp. 699–715.\n\n[5]  O. Chen, E. Retnowati, B. K. Y. Chan, and S. Kalyuga, “The Effect of Worked Examples on Learning Solution Steps and Knowledge Transfer,” Educ. Psychol., vol. 43, no. 8, pp. 914–928, 2023."}, {"review_id": "G9qtBFTpBf", "reviewer": "Reviewer_aoxv", "summary": "This paper investigates a well-posed and highly relevant question: does a large language model's internal reasoning capability directly translate into effective knowledge transfer during human-AI collaboration? As I understand it, the authors propose a conceptual framework to distinguish between model-exclusive, human-exclusive, and shared knowledge. They test this with a two-phase experiment where participants first collaborate with an LLM for ideation on coding and math problems, and then must solve the problem independently. This second phase is designed to be a direct test of how well knowledge was actually transferred. The work aims to quantify this transfer process and identify the behavioral and strategic factors that lead to successful collaboration, independent of a model's raw performance.", "questions": "My main concern is the experimental setup. Without a human-human control group, how can you be certain that your findings are specific to Human-AI knowledge transfer, rather than general principles of how these participants collaborate or learn from any partner? Addressing this would significantly strengthen the claims.\n\nCould you provide a clearer rationale for several key experimental design choices? Specifically:\n\nWhy were participants forbidden from taking notes? Prohibiting participants from taking notes seems counter-intuitive and may unintentionally shift the evaluation towards rote memorization rather than the intended measure of knowledge transfer.\n\nWhy prevent participants from accessing the AI during the implementation phase? The rationale for this constraint is unclear and should be justified.\n\nWhat varies between the three zero-shot attempts for measuring model skill?\n\nWhy the different submission attempt limits (10 for code, 5 for math)?\n\nYour interpretation of results seems problematic in places. Interpreting Gemini-2.5-Pro's performance as \"reduced collaborative efficacy\" is potentially misleading. It is more likely a demonstration of a ceiling effect, where models with high initial solo performance have an inherently smaller margin for improvement, rather than a failure of the collaboration itself. Could you discuss this alternative explanation?\n\nSome findings may be artifacts of experimental constraints. For example, was the preference for formal notation in math observed even when users explicitly asked for simpler methods? Could this effect be hampered by the protocol's prohibition of pseudocode, which is often a much more accessible form of communication?\n\nShould participants know which model they're paired with, and if not, were they able to infer the model's identity?\n\nThe choice of uniform temperature of 0.7 requires justification, especially since the assertion that this reflects how users \"commonly interact\" is debatable when using API access versus web interfaces with specialized system prompts.\n\nYou mention the bidirectional process of humans projecting reasoning to the model, which you note is tested qualitatively. I understand this is hard to measure, but I'd like to hear the authors' ideas on how this could be approached quantitatively in future work.", "limitations": "The authors do not adequately address the limitations of their work. A key limitation that should be discussed is the lack of a human-human control group, which makes it difficult to isolate the effects of AI-specific interaction. Additionally, the study's constraints such as prohibiting note-taking and pseudocode are significant limitations that may impact the external validity of the findings regarding naturalistic knowledge transfer, and this should be acknowledged.", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "The paper's primary strength is that it is framed around a well-posed and highly relevant research question. The attempt to quantify knowledge transfer as a distinct capability from model performance is a significant and original contribution to the field of human-AI collaboration. The experimental design has several strong components, particularly the two-step process for estimating human skill and the method of calibrating task difficulty to be just above the participant's level, which ensures collaboration is meaningful.\n\nHowever, the paper has several weaknesses, primarily in its experimental design and clarity.\n\nQuality: My main concern is the lack of a human-human control group. Without this baseline, it's difficult to determine if the observed effects are specific to AI interaction or simply reflect general collaborative abilities. Furthermore, some interpretations are questionable; the claim of \"diminishing returns\" for high-performing models seems more likely to be a simple ceiling effect, where there's less room for improvement.\n\nClarity: Several design choices are not clearly justified. It's unclear why participants were forbidden from taking notes (this seems to test for fact memorization unintentionally), why submission attempt limits differ between tasks (10 vs. 5), or why a uniform temperature of 0.7 was chosen for all models, especially given that some model providers recommend different defaults and the non-creative nature of these tasks. The conceptual framework diagram in Figure 2 is frankly not a helpful aid and does little to improve the reader's understanding. Table captions are also not descriptive enough to be understood independently, especially for newly coined terms like HTM, HMT, and MHT. This is an example of a broader issue where some experimental design choices feel arbitrary and are left unexplained, which creates confusion and detracts from the paper's otherwise strong ideas.\n\nSignificance: The qualitative analysis is well-executed; however, its contribution should be clarified by explicitly highlighting which findings are expected versus unexpected. The paper should frame its results in the context of existing literature, noting where they corroborate prior work and where they represent novel discoveries.", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thoughtful review! We address your concerns below:\n\n> Without a human-human control group, how can you be certain that your findings are specific to Human-AI knowledge transfer…\n\nBelow we clarify why a human-human control group, while interesting independently, is not essential for the claims made in this paper.\n\n1. **Research focus and claim scope**\n\nOur goal is to measure the relationship between LLMs’ solo capabilities and their ability to transfer / communicate their knowledge to humans as these abilities improve. We believe that understanding this relationship is primarily important because LLMs aren’t human, so as we cede more responsibilities and privileges to LLMs, it is critical that we’re capable of overseeing and understanding *their* reasoning, rationales, and behaviors. Therefore, all statistical tests, figures, and conclusions in the paper are framed around relative differences between models’ knowledge transfer capabilities, so a human‑human condition would not change any of the relative rankings that constitute our core quantitative contributions. Additionally, we do not make any claims as to whether the interactions observed in Section 6 apply *only* to Human-AI collaboration: they could be (and most likely are) general collaboration paradigms that could also apply to Human-Human collaboration. Thus, not having a human-human condition does not discount any of our findings. We do acknowledge that providing a comparison to Human-Human collaboration mechanisms could be a very interesting auxiliary result that links our work to existing work in Human-Human collaboration. We are excited to explore this in the future.\n\n\n2. **Generic collaboration benefits are controlled for implicitly**\n\nAny generic boost from working with a partner, be it human or AI, acts as a constant offset that applies equally to every LLM condition. Because participants were randomly assigned to one of eight models, these generic effects are evenly distributed and cannot explain the systematic, model‑specific differences in our findings. In fact, our mixed‑effects logistic regression (Sec. 5, App. C.3) shows that model identity remains a highly significant predictor (p < 0.001) of post‑collaboration success even after controlling for user skill, task type, interaction length, and LLM familiarity. This suggests that the effects we report are tied to which AI collaborator is present, not merely to collaboration per se.\n\n\n3. **Methodological and logistical considerations**\n\nIncluding a human-human baseline would likely also have required significantly more resources and demands on our human participants than we had available, diverting from the core model focused factors we aimed to study.\n\n\nTo your point, we have revised our paper to ensure this point is clear, including clarifying in Section 4.1 why our hypotheses and analyses do not rely on such a baseline, as well as guiding readers to related literature in the related work that reports human‑human transfer rates.\n\n> Rationale on prohibiting note taking and pseudocode, and prohibiting participants from accessing the AI during the implementation phase\n\nThe central outcome variable in our study is *knowledge that participants have actually internalized* after collaborating with an LLM. Allowing external memory aids (hand‑written notes, screenshots of pseudocode, or live re‑queries to the model) would blur that signal by letting participants outsource recall rather than encode and retrieve the material themselves. In pilot sessions we observed exactly this behaviour: if participants could jot down or copy‑paste an answer scaffold, many stopped asking clarifying questions and could perform on the post‑task even when they clearly had not grasped the underlying concept.\n\nThe manipulation follows the same logic as a closed‑book exam in educational research: retrieval practice without aids is a stronger indicator of true learning than recognition with aids. Classic work on the *testing effect* [1] shows that removing external prompts yields better long‑term retention and cleaner measurement of what was learned. Our prohibition of pseudocode plays a parallel role in math tasks, where a line‑by‑line algorithm can act as a de‑facto crib sheet.\n\nNote that these restrictions were uniform for all eight model conditions. Therefore they also cannot explain the model‑specific performance differences we report; at worst they lower absolute scores equally, leaving relative ordering intact. To your point, we have added a note in the limitations section acknowledging that real‑world programming often is “open‑book.” A useful next step is to replicate our framework in an open‑resource setting and compare retention under different aid regimes. We believe that our current closed‑book design, however, is the necessary first step to quantify transfer in isolation.\n\n> What varies between the three zero-shot attempts for measuring model skill?\n\nBecause at test time during collaboration we sample at non-zero temperature, therefore, we only provide humans that the model repeatedly gets wrong to reduce variance and to ensure that at test time, collaboration is meaningful. This creates the need to sample multiple times to determine this.\n\n> Why the different submission attempt limits (10 for code, 5 for math)?\n\nThis is because the space of possible answers for code (imagine a code function) is much larger than math, which is often confined to integers from 1-1000 for AIME questions. Empirically during our initial testing we found that giving too many attempts for math allowed for a small, but non-zero chance of completely guessing the answer correctly. Additionally, for math, we observed that participants tended to not utilize additional attempts past ~5 attempts effectively (i.e. random guessing behavior arose). These numbers were determined through post-user surveys + manually inspecting solve sessions in a testing phase before the actual experiment. Our rationale for design decisions based on our trial experiments were not added to the paper and this was definitely an oversight: we have now added this to appendix section C. We really appreciate your pointing out details that you find unclear!\n\n>  It is more likely a demonstration of a ceiling effect, where models with high initial solo performance have an inherently smaller margin for improvement, rather than a failure of the collaboration itself. \n\nWe agree that there could be ceiling effect: however, we also observe that Claude-3.7-sonnet is able to exceed Gemini’s ability in inducing collaborative performance in math and closely matching it in code, despite only having half and a third of Gemini's solo performance, respectively (Section 5). Our description of Gemini’s reduced collaborative efficacy thus more points to the fact that despite its high performance, it was not able to output a proportionally higher performance in humans. We don’t discount the possibility of a ceiling effect: but it’s hard to say either way without more data points with higher model performance, which was not possible at the time as Gemini represented the highest performance in our tested benchmark and there were simply no better models then. Nonetheless we think this is a great nuance to include, and we have added this in our discussion of results in Section 5.\n\n>  For example, was the preference for formal notation in math observed even when users explicitly asked for simpler methods? Could this effect be hampered by the protocol's prohibition of pseudocode, which is often a much more accessible form of communication?\n\nAs discussed previously, we must prohibit pseudocode to ensure our experimental procedure accurate measures *internalized knowledge*. We find that this constraint does **not** force models into dense symbolic exposition; they were free to, and often did, answer in ordinary prose. In our logs we see many exchanges where participants explicitly requested “plain English” or “a simpler explanation,” and the LLM immediately shifted style, because the guard‑prompt only disallowed verbatim algorithmic scaffolds, not natural‑language reasoning. Hence the occasional preference for formal notation arises from model defaults and user choices, not from any protocol.\n\n> Should participants know which model they're paired with, and if not, were they able to infer the model's identity?\n\nParticipants were not informed which model they were paired with. We also did not observe any attempts by participants to identify the model through prompting or other means.\n\n> The choice of uniform temperature of 0.7 requires justification, especially since the assertion that this reflects how users \"commonly interact\" is debatable…\n\nFor sure. We fixed the temperature to eliminate temperature as a confounder when comparing knowledge‑transfer effectiveness. We chose 0.7 because it is commonly recommended by providers (ie OpenAI in their API docs), balances diversity and determinism, and unlike model‑specific web presets, can be applied uniformly through the API. Because we lack visibility into the proprietary system prompts that shape web‑interface behaviour, controlling this parameter ensures a level playing field and keeps observed differences attributable to the models themselves rather than to decoding settings.\n\nHopefully this additional context resolves your concerns. Thank you genuinely for your detailed review: reading through your review was invaluable to helping us improve our paper based upon your suggestions. We have already made many clarifying additions in response to your feedback. We believe strongly in our work, so we look forward to continuing the conversation to improve your perception of our work and clear up any additional questions!\n\nReferences:\n\n[1] Roediger, H. L. III, & Karpicke, J. D. (2006). Test-enhanced learning: Taking memory tests improves long-term retention. Psychological Science, 17(3), 249–255."}, {"review_id": "QWPNhqt3jp", "reviewer": "Reviewer_zabu", "summary": "This paper studies the knowledge transfer from LLMs to humans via a large-scale user study with 118 participants. In a two-phase setup, the authors aim to measure the influence of model reasoning on human understand. In the first phase, humans can ideate with an LLM on problem-solving strategies on math and coding tasks, while in the second phase the human independently implement the solutions. The authors find that model benchmark performance correlates with collaborative outcomes, but that this relationship is inconsistent and contains several outliers. Additionally, the results show that model solo performance correlates positively with knowledge transfer, while the relationship between human preferences is mixed (positive for code, negative for math). The paper argues that this points to knowledge transfer from LLMs to humans is a distinct capability which may require dedicated optimization.", "questions": "- Could the authors show the distribution of some qualitative findings across models? For example, do models like Gemini-2.5-Pro fail in knowledge transfer due to \"style issues\"?\n- Do the authors see some relationship with respect to the recent literature on the impact of LLMs on education? Where the general trend is, if LLMs help but do not give the final response, students have a faster learning rate?", "limitations": "The authors acknowledge and discuss limitations of their work in the main part of the paper.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 3, "originality": 4, "strengths_and_weaknesses": "Strengths:\n- A very well executed paper where the main effects are robust with respect to some confounders, and the additional qualitative analysis helps to understand the quantitative findings.\n- The two-phase setup allows for a clean isolation of the knowledge transfer effect.\n- Interesting take on knowledge transfer being a distinct capability, and interesting to see that the slope <1 when compared to solo model performance.\n\nWeaknesses:\n- Some seemingly \"causal\" claims are only the result of an observational study. Randomising (or controlling for) the response-style of LLMs may help to make some claims (such as the effect of adaptive scaffolding) stronger.", "was_revised": true, "final_justification": "The rebuttal did not change my acceptance recommendation.", "rebuttal": "Thank you for your positive review of our paper! We are glad to hear you find our experimental setup “well executed” and our results interesting as well. We aim address your remaining concerns below:\n\n> Randomising (or controlling for) the response-style of LLMs may help to make some claims (such as the effect of adaptive scaffolding) stronger.\n\nWe agree, but we do see a give and take here. We believe that the response-style is a choice that the LLMs themselves should be able to make, as an LLM’s ability to select how to convey information is an important factor in its knowledge transfer abilities. Although it may make some claims stronger, controlling for the response style would remove this component, which may limit the generalizability of our study. \n\n> Could the authors show the distribution of some qualitative findings across models? For example, do models like Gemini-2.5-Pro fail in knowledge transfer due to \"style issues\"?\n\nYes: we do include a small amount of analysis on this in Section 6. We don’t dive into it in too much detail because the *qualitative* findings are only really able to be analyzed through participants that provided actual feedback after completing a problem: which is not all of the interactions. Therefore, such a metric would be skewed in the types of participants (via skill level) that it represents, which may paint an inaccurate picture.\n\n> Do the authors see some relationship with respect to the recent literature on the impact of LLMs on education?\n\nAbsolutely. Along with the recent results assessing the cognitive debt of LLM usage on writing tasks and learning [1] , we believe it to be incredibly important to properly isolate and evaluate the ability of humans to properly internalize LLM reasoning, as well as LLM ability to properly express reasoning in understandable terms. This knowledge transfer is core to the utility of LLMs in educative contexts: yet despite this, few works have tackled this question.\n\nHopefully this additional context resolves your lingering questions. Thank you again for your recognition of our work!\n\nReferences:\n[1] Kosmyna, N., Hauptmann, E., Yuan, Y. T., Situ, J., Liao, X. H., Beresnitzky, A. V., ... & Maes, P. (2025). Your brain on chatgpt: Accumulation of cognitive debt when using an ai assistant for essay writing task. arXiv preprint arXiv:2506.08872."}, {"review_id": "hafRCr31ld", "reviewer": "Reviewer_M7Bb", "summary": "The paper introduces a conceptual and experimental framework for quantifying how effectively LLMs teach humans. In a study with 118 participants and 8 models, it measures knowledge transfer on coding and math tasks using objective solve rates and subjective ratings.", "questions": "1. Can you plot knowledge-transfer gain against each participant’s pre-test score (or self-reported experience) to reveal whether low- or high-expertise users benefit more from LLM collaboration?\n2. You report scaffolding boosts low-skill users but bores high-skill ones. Could you sketch how an adaptive strategies such as concise strategy for experts, detailed steps for novices, might reduce this gap, perhaps via a brief analysis or simulation?", "limitations": "Yes.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 4, "originality": 4, "strengths_and_weaknesses": "#### **Strengths**\n1. Shifts focus from model accuracy to AI-to-human knowledge transfer, opening a fresh research axis.\n2. 118 participants, eight LLMs, clear two-phase protocol; data, code, and dashboard are released for reproducibility.\n3. Identifies explanation structure and scaffolding style as key drivers of successful teaching, guiding future alignment work.\n\n#### **Weaknesses**\n1. Evaluation is confined to coding and math, leaving knowledge-transfer effectiveness in non-STEM domains unvalidated.\n2. The study does not examine how varying prompt styles or settings influences the models’ teaching effectiveness.", "was_revised": true, "final_justification": "The knowledge-transfer gain information provided in the rebuttal serves as a valuable point of reference for understanding how effectively LLMs can transfer knowledge to humans, thereby enhancing the significance of the paper.", "rebuttal": "Thank you so much for your thoughtful review and your recognition of our work. We aim to address your core concerns below:\n\n> Evaluation is confined to coding and math, leaving knowledge-transfer effectiveness in non-STEM domains unvalidated.\n\nWe agree that this is a limitation. However, we decided to constrain our study to math and coding for 2 reasons. First, math and coding are uniquely suited to automatic verification while still having reasoning-intensive, open-ended solutions. It’s relatively much more expensive and difficult to reliably grade participants’ answers for non-STEM fields. Second, these domains match our methodology exceptionally well since “solving” a coding or math problem often requires understanding something concrete about an algorithm or proof that can be more reliably tested than domains that may have more unstructured solutions. While we agree that evaluating more diverse domains, we see our contribution as providing not just an analysis of knowledge transfer, but also a general methodology that can be applied to additional domains. We hope that this contribution can help promote the study of knowledge transfer in other domains as future work.\n\n\n> The study does not examine how varying prompt styles or settings influences the models’ teaching effectiveness.\n\nThis is a good point, and we fully agree that prompt design plays a critical role in the effectiveness of human-AI collaboration. However, we deliberately chose not to exhaustively optimize prompts at this stage of the investigation for multiple reasons:\n\nFirst: evaluating a broad set of prompting strategies combinatorially across 8 models and multiple skill hierarchies quickly becomes infeasible. For each additional prompt variant, we would require a proportional increase in participant data to preserve statistical power and control for interaction effects. Given the resource-intensive nature of our user studies, with 118 skilled humans already involved, scaling to cover a full prompt × model × task grid would have imposed prohibitive data collection and analysis costs.\n\nInstead, we adopt a *top-down* approach: we evaluate models under minimal guidance in order to measure their *intrinsic* capacity to communicate and convey reasoning to humans. Our goal is not to optimize each model for success via prompt engineering, but rather to assess how effectively a model, as-is, can adaptively support a user when given agency in presenting its knowledge. This reflects a more ecologically valid setting where users are not necessarily prompt engineers, and where the quality of support depends on the model’s communicative competence, not on hand-tuned scaffolding. We believe this setup also aligns more closely with the real-world deployment scenario (as mentioned in Section 1) we hope to understand.\n\n> Can you plot knowledge-transfer gain against each participant’s pre-test score (or self-reported experience) to reveal whether low- or high-expertise users benefit more from LLM collaboration?\n\n\nThis is a great point. We have added this in the Appendix of our paper. To summarize the findings, we find it to be roughly even across skill levels, with a slight dip towards higher skilled levels (for each model).\n\n> You report scaffolding boosts low-skill users but bores high-skill ones. Could you sketch how an adaptive strategies such as concise strategy for experts, detailed steps for novices, might reduce this gap, perhaps via a brief analysis or simulation?\n\nThis is an interesting point that would be good for future work! Because these are strategies we’ve observed post-hoc, we leave experiments on these strategies to future work. A potential method for evaluating this is to utilize the same general framework our paper provides to evaluate collaborative effectiveness and also provide models pre-hoc with users’ ELO in order to contextualize model responses. We may use models that are explicitly instructed to utilize/not to utilize scaffolding as baselines.\n\n\nHopefully this additional context resolves your concerns related to domain breadth and prompting methodology. We look forward to continuing the conversation to improve your perception of our work and clear up any additional questions!"}, {"review_id": "gKOtSI3VS4", "reviewer": "Reviewer_qEBD", "summary": "This paper introduces a novel empirical framework to evaluate knowledge transfer from LLMs to humans in collaborative problem-solving settings on coding and mathematical reasoning tasks. The authors isolate and quantify the transfer of reasoning from LLMs to users through a two-phase user study involving 118 participants. Participants first ideate with an LLM, then independently solve the problem, allowing the authors to measure whether the reasoning of LLMs has been effectively internalized. They find that while model benchmark performance generally correlates with collaborative success, this relationship is inconsistent across LLMs and tasks, suggesting that knowledge transfer is a distinct capability requiring separate evaluation.", "questions": "See Weaknesses.", "limitations": "Yes.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 4, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n* This paper is well-motivated for understanding how LLMs influence human performance in collaborative tasks. It addresses an underexplored aspect of human-AI interaction by investigating how LLMs can actually help humans become more capable, rather than just outsourcing tasks to AI.\n* The authors propose a formal framework to model the projection of AI knowledge into human-understandable forms. This contributes to the theoretical understanding of communicative alignment and knowledge transfer between humans and machines.\n* The proposed framework effectively isolates reasoning transfer, enabling performance gains to be cleanly attributed to internalised LLM knowledge.\n* The analysis includes both quantitative metrics and qualitative clustering of interaction behaviors, offering novel insights into which strategies lead to successful knowledge transfer.\n\nWeaknesses:\n\n* All experiments are evaluated in zero-shot settings, which might understate the potential of LLMs for better pedagogical alignment. Could the authors report the ablation performance of human-AI interaction under the CoT setting, for better exploring the intrinsic assistive capabilities of LLMs? For example, the authors find that high-performing math models often produce dense symbolic expressions when assisting humans, which can be confusing for users. However, this can be easily addressed by modifying the prompt to encourage the LLM to focus more on helping the user rather than solving the problem independently.\n* Although the authors propose an intuitive conceptual framework dividing knowledge into shared, AI-exclusive, and human-exclusive regions, they do not attempt to quantitatively analyze these segments independently. While they argue such divisions are difficult to measure, a subjective rating from participants on perceived knowledge novelty or conflict could offer insights into how often LLMs provide genuinely instructive concepts.", "was_revised": true, "final_justification": "I will keep my score.", "rebuttal": "Thank you for your thoughtful review and your recognition of our work. We address your concerns below:\n\n> For example, the authors find that high-performing math models often produce dense symbolic expressions when assisting humans, which can be confusing for users. However, this can be easily addressed by modifying the prompt to encourage the LLM to focus more on helping the user rather than solving the problem independently.\n\nThis is a great point, and we fully agree that prompt design plays a critical role in the effectiveness of human-AI collaboration. However, we deliberately chose not to exhaustively optimize prompts at this stage of the investigation for multiple reasons:\n\nFirst: evaluating a broad set of prompting strategies combinatorially across 8 models and multiple skill hierarchies quickly becomes infeasible. For each additional prompt variant, we would require a proportional increase in participant data to preserve statistical power and control for interaction effects. Given the resource-intensive nature of our user studies, with 118 skilled humans already involved, scaling to cover a full prompt × model × task grid would have imposed prohibitive data collection and analysis costs.\n\nInstead, we adopt a *top-down* approach: we evaluate models under minimal guidance in order to measure their *intrinsic* capacity to communicate and convey reasoning to humans. Our goal is not to optimize each model for success via prompt engineering, but rather to assess how effectively a model, as-is, can adaptively support a user when given agency in presenting its knowledge. This reflects a more ecologically valid setting where users are not necessarily prompt engineers, and where the quality of support depends on the model’s communicative competence, not on hand-tuned scaffolding. We believe this setup also aligns more closely with the real-world deployment scenario (as mentioned in Section 1) we hope to understand.\n\nFurthermore, evaluating alternate prompting strategies *post hoc* risks introducing bias and overfitting to our task suite. It would allow us to “cherry-pick” effective prompts retrospectively, creating an illusion of pedagogical alignment that might not generalize beyond our specific test cases. By contrast, our current approach generalizes and enables us to surface naturally occurring shortcomings in LLM communication, such as the overuse of dense symbolic math, based in genuine participant confusion.\n\nOur qualitative clustering of interaction behaviors and participant feedback in Section 6 already gives us valuable insights into what kinds of LLM strategies are effective or counterproductive. These empirical observations can inform future work that systematically explores prompt design with a narrowed scope, guided by evidence rather than searching from the get go in a large solution space. Our analysis could further inform model builders aiming to improve the basic collaboration skill of LMs.\n\n> Although the authors propose an intuitive conceptual framework dividing knowledge into shared, AI-exclusive, and human-exclusive regions, they do not attempt to quantitatively analyze these segments independently. While they argue such divisions are difficult to measure, a subjective rating from participants on perceived knowledge novelty or conflict could offer insights into how often LLMs provide genuinely instructive concepts.\n\n\nAbsolutely. We aim to provide such a measurement in section 6, where we aggregate open-ended participant feedback to shed light on specific interaction mechanisms under the umbrella of our framework in Section 3, tagged with their frequency and correlation with success.\n\n\nThank you again for your thoughtful review. We hope this additional context resolves your lingering concerns. We look forward to continuing the conversation to improve your perception of our work and clear up any additional questions!"}]

Author rebuttal:

Meta-review: This paper introduces a novel framework for quantifying knowledge transfer from LLMs to humans through a two-phase collaborative study with 118 participants across coding and math tasks. The key finding is that model benchmark performance correlates with but doesn't fully predict collaborative success, suggesting knowledge transfer is a distinct capability requiring separate evaluation. The paper's primary strengths include its well-executed experimental design that cleanly isolates knowledge transfer effects, the significant practical relevance of measuring AI teaching effectiveness, and robust methodology with Elo-calibrated task difficulty and mixed-effects analysis controlling for confounders. However, the work has notable limitations: the evaluation is confined to STEM domains with automatic verification, lacks human-human collaboration baselines to isolate AI-specific effects, and imposes potentially artificial constraints like prohibiting note-taking and pseudocode that may reduce ecological validity. The experimental choices around uniform temperature settings and different submission limits across tasks also lack sufficient justification. Despite these methodological concerns, the paper addresses a crucial gap in understanding human-AI collaboration and provides the first systematic framework for measuring knowledge transfer effectiveness.

The post-rebuttal discussion revealed persistent methodological disagreements, particularly from reviewer aoxv who maintained concerns about the experimental design despite acknowledging the technical contributions. The authors effectively addressed most concerns by clarifying their fixed-Elo calibration approach, which treats collaboration accuracy as a direct measure of knowledge transfer since all participants face tasks with known baseline success probabilities. Reviewers wnSo and M7Bb were satisfied by these explanations and the additional knowledge-transfer gain data provided, with wnSo updating from initial concerns to acceptance. Reviewer zabu remained supportive throughout, and reviewer qEBD maintained their borderline accept position. The key remaining contention centers on whether the study design adequately captures "collaboration" versus "learning," with aoxv arguing that the closed-book constraints better measure memory than collaborative knowledge transfer. Balancing the paper's novel contribution and robust execution against these methodological limitations, the work merits acceptance as it establishes an important new research direction with actionable insights for AI alignment, though future work should address the ecological validity concerns raised.
Decision: Accept (poster)