TinyR1-32B-Preview: Boosting Accuracy with Branch-Merge Distillation
Authors: name: Lin Sun; name: Guangxiang Zhao; name: Xiaoqi Jian; name: Yuhan Wu; name: Weihong Lin; name: Yongfu Zhu; name: Change Jia; name: Linglin Zhang; name: Jinzhu Wu; name: Junfeng Ran; name: Sai-er Hu; name: Zihan Jiang; name: Junting Zhou; name: Wenrui Liu; name: Bin Cui; name: Tong Yang; name: Xiangzheng Zhang
Published: 2025-03-06 16:25:53

Abstract: The challenge of reducing the size of Large Language Models (LLMs) while
maintaining their performance has gained significant attention. However,
existing methods, such as model distillation and transfer learning, often fail
to achieve high accuracy. To address this limitation, we introduce the
Branch-Merge distillation approach, which enhances model compression through
two phases: (1) the Branch Phase, where knowledge from a large teacher model is
selectively distilled into specialized student models via
domain-specific supervised fine-tuning (SFT); And (2) the Merge Phase, where
these student models are merged to enable cross-domain knowledge transfer and
improve generalization. We validate our distillation approach using DeepSeek-R1
as the teacher and DeepSeek-R1-Distill-Qwen-32B as the student. The resulting
merged model, TinyR1-32B-Preview, outperforms its counterpart
DeepSeek-R1-Distill-Qwen-32B across multiple benchmarks, including Mathematics
(+5.5 points), Coding (+4.4 points) and Science (+2.9 points), while achieving
near-equal performance to DeepSeek-R1 on AIME 2024. The Branch-Merge
distillation approach provides a scalable solution for creating smaller,
high-performing LLMs with reduced computational cost and time.