Title: From Atoms to Chains: Divergence-Guided Reasoning Curriculum for Unlabeled LLM Domain Adaptation
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords: Large Language Models, Unlabeled LLM Adaptation, Knowledge Distillation, Learning from Disagreement, Reasoning Curriculum

Abstract: Adapting Large Language Models (LLMs) to specialized domains without human-annotated data is a crucial yet formidable challenge. Widely adopted  knowledge distillation methods often devolve into coarse-grained mimicry, where the student model inefficiently targets its own weaknesses and risks inheriting the teacher's reasoning flaws. This exposes a critical pedagogical dilemma: how to devise a reliable curriculum when the teacher itself is not an infallible expert. Our work resolves this by capitalizing on a key insight: while LLMs may exhibit fallibility in complex, holistic reasoning, they often exhibit high fidelity on focused, atomic sub-problems. Based on this, we propose Divergence-Guided Reasoning Curriculum (DGRC), which constructs a learning path from atomic knowledge to reasoning chains by dynamically generating two complementary curricula from model disagreements. When a student and a teacher arrive at conflicting results for the same problem, DGRC tasks the teacher with a diagnostic process: it analyzes both reasoning paths to formulate atomic, root-cause queries targeting the specific steps where their reasoning diverged, and then provides the answers itself to create high-confidence atomic question-answer pairs. These pairs then serve a dual purpose: (1) providing an atomic curriculum to rectify the student's knowledge gaps, and (2) serving as factual criterion to filter the teacher's original reasoning chains, yielding a verified CoT curriculum that teaches the student how to integrate atomic knowledge into complete reasoning paths. Experiments on specialized domains demonstrate the effectiveness of DGRC. Notably, our method achieves a 7.76\% relative improvement for the 1.5B student model in the medical domain over strong unlabeled baseline.

TL;DR: Our method uses reasoning disagreements between models to create a reasoning curriculum, enabling effective unlabeled LLM domain adaptation.