Reverse-Engineered Reasoning for Open-Ended Generation
Authors: name: Haozhe Wang; name: Haoran Que; name: Qixin Xu; name: Minghao Liu; name: Wangchunshu Zhou; name: Jiazhan Feng; name: Wanjun Zhong; name: Wei Ye; name: Tong Yang; name: Wenhao Huang; name: Ge Zhang; name: Fangzhen Lin
Published: 2025-09-07 18:07:58

Abstract: While the ``deep reasoning'' paradigm has spurred significant advances in
verifiable domains like mathematics, its application to open-ended, creative
generation remains a critical challenge. The two dominant methods for
instilling reasoning -- reinforcement learning (RL) and instruction
distillation -- falter in this area; RL struggles with the absence of clear
reward signals and high-quality reward models, while distillation is
prohibitively expensive and capped by the teacher model's capabilities. To
overcome these limitations, we introduce REverse-Engineered Reasoning (REER), a
new paradigm that fundamentally shifts the approach. Instead of building a
reasoning process ``forwards'' through trial-and-error or imitation, REER works
``backwards'' from known-good solutions to computationally discover the latent,
step-by-step deep reasoning process that could have produced them. Using this
scalable, gradient-free approach, we curate and open-source DeepWriting-20K, a
large-scale dataset of 20,000 deep reasoning trajectories for open-ended tasks.
Our model, DeepWriter-8B, trained on this data, not only surpasses strong
open-source baselines but also achieves performance competitive with, and at
times superior to, leading proprietary models like GPT-4o and Claude 3.5.