When Models Know More Than They Can Explain: Quantifying Knowledge
  Transfer in Human-AI Collaboration
Authors: name: Quan Shi, name: Carlos E. Jimenez, name: Shunyu Yao, name: Nick Haber, name: Diyi Yang, name: Karthik Narasimhan

Recent advancements in AI reasoning have driven substantial improvements
across diverse tasks. A critical open question is whether these improvements
also yields better knowledge transfer: the ability of models to communicate
reasoning in ways humans can understand, apply, and learn from. To investigate
this, we introduce Knowledge Integration and Transfer Evaluation (KITE), a
conceptual and experimental framework for Human-AI knowledge transfer
capabilities and conduct the first large-scale human study (N=118) explicitly
designed to measure it. In our two-phase setup, humans first ideate with an AI
on problem-solving strategies, then independently implement solutions,
isolating model explanations' influence on human understanding. Our findings
reveal that although model benchmark performance correlates with collaborative
outcomes, this relationship is notably inconsistent, featuring significant
outliers, indicating that knowledge transfer requires dedicated optimization.
Our analysis identifies behavioral and strategic factors mediating successful
knowledge transfer. We release our code, dataset, and evaluation framework to
support future work on communicatively aligned models.