Reverse-Engineered Reasoning for Open-Ended Generation
Authors: Haozhe Wang
Haoran Que
Qixin Xu
Minghao Liu
Wangchunshu Zhou
Jiazhan Feng
Wanjun Zhong
Wei Ye
Tong Yang
Wenhao Huang
Ge Zhang
Fangzhen Lin
Venue: ICLR 2026 Poster (2026)
Keywords: reasoning
open-ended generation
synthetic data
Primary area: foundation or frontier models, including LLMs

Abstract: While the "deep reasoning" paradigm has spurred significant advances in verifiable domains like mathematics, its application to open-ended, creative generation remains a critical challenge. The two dominant methods for instilling reasoning—reinforcement learning (RL) and instruction distillation -- falter in this area; RL struggles with the absence of clear reward signals and high-quality reward models, while distillation is prohibitively expensive and capped by the teacher model's capabilities. To overcome these limitations, we introduce REverse-Engineered Reasoning (REER), a new paradigm that fundamentally shifts the approach. Instead of building a reasoning process "forwards" through trial-and-error or imitation, REER works "backwards" from known good solutions to computationally discover the latent, step-by-step deep reasoning process that could have produced them. Using this scalable, gradient-free approach, we curate and open-source DeepWriting-20K, a large-scale dataset of 20,000 deep reasoning trajectories for open-ended tasks. Our model, DeepWriter-8B, trained on this data, not only surpasses strong open-source baselines but also achieves performance competitive with, and at times superior to, leading proprietary models like GPT-4o and Claude 3.5.

Published as a conference paper at ICLR 2026
REVERSE-ENGINEBRED REASONING FOR OPEN- ENDED GENERATION
Haozhe Wang 1,2,, Haoran Que Qixin Xu Minghao Liu Wangchunshu Zhou', JJazhan Feng' Wanjun Zhong', Wei Ye², Tong Yang', Wenhao Huang', tGe Zhang "Fangzhen Lin?
*ByteDance Seed, "Hong Kong University of Science and Technology. SPeking University, "M-A-P, $2077AI, TTiinghua University
WWrk done at ByteDance Seed, iCorresponding authors
ABSTRACT
While the "deep reasoning" paradigm has spurred significant advances in verifi- able domains like mathematics, its application open-ended, creative generation remains a critical challenge. The two dominant methods for instilling reason- ing reinforcement learning (RL) and instruction distillation falter in this area; RL struggles with the absence of clear reward signals and high-quality reward mod- els, while distillation prohibitively expensive and capped by the teacher model's capabilities. To overcome these limitations, we introduce TEEvers--Engineet soning (REER), a new paradigm that fundamentally shifts the approach. Instead of building a reasoning process "forwards" through trial-and-error or imitation, REER works "backwards" from known good solutions to computationally discover the latent, step-by-step deep reasoning process that could have produced them. Using this scalable, gradient-free approach, we curate and open-source DeipWriting- 20K, a large-scale dataset of 20,000 deep reasoning trajectories for open-ended tasks. Our model, DeepWriter-8B. trained on this data, not only surpasses strong open-source basclines but also achieves performance competitive with, and at times superior to, leading proprictary models like GPT-4o and Claude 3.5.
Rea-
INTRODUCTION
The paradigm of "deep reasoning" is catalyzing a shift in large language model (LLM) reasoning, moving beyond rapid, surface-level inference leverage increased computational investment at test time (Guo et al., 2025; Jaech et al., 2024; Team, 2025; Muennighoff et al., 2025; Fu et al., 2025). This approach unlocks sophisticated capabilities like multi-step planning and complex problem-solvine yielding remarkable performance gains in verifiable domains such as mathematics and programming. The success in these areas has been largely propelled by Reinforcement Learning (RI.), where clear reward signals for correct outcomes effectively guide model's search through vast solution spaces.
However, the reliance on verifiability presents a formidable barrier when applying deep reasoning to open-ended, creative domains (Lu, 2025; Ouyang et al., 2022). Creative writing, a quintessential example of a non-verifialle task, lacks singular, objective ground truth. Instead, its quality is judged on subjective criteria like originality, emotional resonance, and narrative coherence (Wu et al., 2025). This disconnect raises a critical and largely unexplored research question:
How to instill deep reasoning for open-ended generation in the absence of task verifiability?
Bridging this gap is profoundly challenging. The dominant paradigms for cultivating advanced reasoning falter here; adapting RL by training reward model to approximate subjective quality that aligns with human preferences is an immense challenge in itself (Ouyang et al., 2022), and the subsequent RL process is notoriously sample-inefficient and computationally burdensome (Lu, 2025). The alternative, instruction distillation from powerful model, is often prohibitively expensive and fundamentally capped by the teacher model's capabilities (Toshniwal et al., 2024). This exacerbated by the scarcity of high-qualin queries and deep reasoning trajectories tailored for complex open- ended generation (Bai et al., 2024). These constraints create critical bottleneck, demanding a new paradigm that sidesteps both the sample inefficiency of RI. and the costly dependency of distillation.

---

Published as a conference paper at ICLR 2026
Build Reasoning "forward"
Deep Reasoning
Recover Reasoning "barkward"
Source QA Pairs
① Beinforcement Learning
dEverse-Engineered Engineered Reasoning
Request
D0
I Io
IO
M
in
Figure (Left) Existing methods attempt to build deep reasoning "forwards" for user request through trial-and-error (RL) or costly distillation, which falter in open-ended domains that lack clear, verifiable reward signals. (Right) We propose third path for teaching deep reasoning, REverse-Engineered Reasoning (REER). REER works "backwards", recovering plausible human-like thought process from known-good outputs in open-source Question-Answa (QA) pairs.
To break this impasse, introduce paradigm: REverseEnggineeed Reasoning (REER). In contrast to conventional methods that build a reasoning process "forwards" through trial-and-error distillation, work "backwards" from known good outcome. We essentially ask: "Given a high-quality piece of output, what is the most coherent and logical thinking process that would have generated it?" By answering this question, we can synthesize the otherwise latent, human-like reasoning paths at scale, bypassing costly distillation of thinking data beforehand or inefficient trial-and-error.
We pioneer novel approach that operationalizes the REER paradigm and, for the first time, instill deep reasoning capabilities for open-ended generation entirely from scratch. Our approach involves three key stages. First, we source a diverse dataset of query-solution pairs for open-ended generation from the web, encompassing 16,000 samples spanning across ordinary-life question-answering academic writing, functional writing and creative writing. From these, we "reverse- "reverse-engineer" deep reasoning trajectories - structured, human-like thought process tailored for open-ended generation. Eventually, we use this synthetic data to fine-tune base language model, teaching it to reason and plan deeply before generating a final solution.
The central innovation lies in how synthesize this data: formulate the recovery of high-quality thinking trajectories as a gradient-free search problem. These trajectories are found by iteratively refining initial plan, with the search guided by proxy for thought quality the perplexity of a known good solution. The gradient-free, self-contained nature of our synthesis process lends us the scalability. By obviating the need for expensive, query-by-query distillation from proprietary models or the sample-inefficiency of reinforcement learning, our approach provides cost-effective and automatable pathway generate quantities of high-quality, deep-thinking training data. This makes it possible to instill sophisticated reasoning in models at scale that was previously impractical.
Using this method, we created DeepWriting-20u 20K, a comprehensive dataset of 20,000 thinking trajec- tories, and fine-tuned Qwen3-8B base model. Our extensive empirical evaluation on benchmarks like LongBench (Bai et al., 2024), HelloBench (Que et al., 2024), and WritingBench (Wu et al., 2025; Kang et al., 2025) shows that DeepWriter-81 successfully internalizes this deep reasoning process. not only substantially outperforms strong open-source baselines but also achieves performance com- petitive with, and in some cases exceeding, leading proprietary models like GPT-4o and Claude 3.5, validating approach as powerful new pathway for building complex reasoning for open-ended generation.
Our primary contributions
Pioneering a New Paradigm for Reasoning: We introduce Reverse- Enneisered Reasoning (REER). novel and scalable paradigm that synthesizes high nigh-quality thinking trajectories without reliance on RL or distillation. This work charts new, third path for teaching complex reasoning, addressing critical bottleneck in the field.
2

---

Published as a conference paper at ICLR 2026
Enabling New Research with Deep We contribute comprehensive dataset of 20,000 query-response and deep reasoning trajectories across 25 categories. This resource is designed to mitigate data scarcity and catalyze future research into the mechanisms of planning and structured thought in open-ended generation.
Strong Performance in Open-Ended Generation from Scratch: We provide rigorous em- pirical evidence that our approach works. Our model, DeepWriter-88. trained entirely on our synthesized data, matches or exceeds the performance of premier proprietary models on chal- lenging writing benchmarks, proving that human-like deep reasoning can be cultivated without costly distillation RL.
2
APPROACH
[Question]
Write engaging story based the following writing prompt: "You can't t speedrun an Isckai! It's not game!" The God tells me.
"Watch me." " reply.
[Response]
<think>
Alright, the user wants story based on the prompt...
First, need to understand the user's intent. The core objective to create a compelling...
Let me think... Maybe should emphasize the protagonist's 's journey from disbelief to Hmm... Alternatively, maybe should also consider the emotional journey of the protagonist and For the content, I need to brainstorm core creative idea. The main character, Hiro
Building the structure, I'll use narrative framework with clear beginning, development, climax. Let me think about the beginning Next, the development phase For the climax, I'll need a final, epic battle Additionally, should include Hiro's reflections on his journey.. Wait, that's 's a bit too straightforward.
Let me add some twists and turns.
For instance, the God could Hmm, maybe I should include a subplot involving This could add layers to the story. </think>>
<answer>
</answer>
Figure 2: An example of the deep reasoning generated by DeepWriter. The model demonstrates deliberate planning for open-ended generation, employing various thinking patterns (e.g., 'Hmm... Alternatively', 'Wait, that's bit ...') to facilitate structured reasoning, including logical deduction, branching, and backtracking.
Our central goal is to instill deep reasoning in LLMs for open-ended tasks without relying on costly distillation or reinforcement learning. To achieve this, we introduce AEverse-Fngineers Reasoning (REER), a novel paradigm that shifts the objective from generating a solution to discovering the latent reasoning process behind an existing high-quality one. Instead of building reasoning process "forwards" via trial-and-error, REER works "backwards" from known good output to computation- ally synthesize the step-by- by-eepp thinking that could have produced it. This approach is operationalized as search problem where we iteratively refine an initial thinking process to discover a trajectory that best explains high-quality, human-writte written output. An example of the structured reasoning we aim to cultivate is shown in Figure 2, where the model demonstrates deliberate planning, exploration of alternatives ("Hmm... Alternatively"). and self-correction ("Wait, that's a bit too straightforward").
2. REVERSE- ENGINEEEE REASONING AS A SEARCH PROBLEM.
Let be input query (e.g., story prompt) and y be high-quality reference solution (e.g., well- written story). Our objective is to find deep reasoning trajectory, denoted by 2, which represents a structured, step-by-step thinking process that guides the generation of y from
3

---

Published as a conference paper at ICLR 2026
The primary challenge in open-ended domains is the absence of a verifiable correctness signal. The REER paradigm circumvents this by reframing the problem: instead of judging the final output, evaluate the quality of a thinking process based on how well it explains known-good output. We operationalize this principle by using the perplexity (a.k.a, the model surprise) of the reference solution y a proxy for the quality of a given reasoning trajectory A lower perplexity score for y- conditioned on both and 2, indicates that the trajectory provides a more coherent and effective plan. In essence, REER posits that a good thinking process is one that makes a high-quality answer y seem maximally probable and logical to the model.
Formally, we model the deep reasoning trajectory z a discrete sequence of reasoning steps. 2 [21, , The problem is then formulated as a search for the optimal trajectory 2" within the vast space possible trajectories 2, such that minimizes the perplexity of the reference solution y:
arg min (yl 2)
sE2 Here, PyL(y (r, 2) is the perplexity of the token sequence of as calculated by a generator LLM, conditioned and z. This optimization is performed via a gradient-free local search algorithm, allowing us to iteratively refine the trajectory without differentiable objective.
2.2 ITERATIVE REFINEMENT VIA LOCAL SEARCH
Solving for the optimal trajectory directly intractable due to the vast search space. Therefore, we propose an iterative refinement algorithm that employs guided local search to discover a high-qualig deep reasoning trajectory. The algorithm starts with an initial trajectory and progressively improves by refining its constituent segments, guided by the perplexity signal. As visualized in Figure 3, the algorithm runs as follows:
1. Initialization: For given (x, y) pair, we generate an initial, imperfect deep reasoning trajectory, 20)), by prompting an L.LM with thought-provoking instruction (see Appendix, Listing 1) to produce plausible plan. This ini- tial trajectory is denoted
lherative Local Search for Deep Thinking Synthesi
Iniitaiization
Nade Expamiien
Suteed Tarting
Termination
Oiiqiaal
[21, 22, Zn.. Stow
Node Expansion (Segment-wise Edits): The core of our method is iterative loop that re- fines 2 one segment at a time. In each iteration, select a segment to improve. We prompt the L.M to generate candidate refinements with more thinking-based details, elaborations and reflections. To generate these refinements, we provide the full context including the query x, the reference solution y. and the surrounding trajectory segments (refined steps 2², and initial steps i. The prompt is meticulously designed to encourage detailed reasoning while preventing the model from simply copying content from the reference solution (see Appendix, Listing 2).
sodual
1 Nods Evaluation and Selection
Primcidet oj = any min PRLLyla, alune)
Sebeod
Figure 3: Method Overview: Iterative Local Search for deep reasoning Synthesis.
For each generated candidate c, we construct a temporary trajectory 2'and by substituting 2 with We then evaluate each candidate by computing its quality score, S(c) = PPL(y) X ). The candidate with the lowest perplexity score is chosen as the updated segment for the next iteration: 2 arg (i(. S(c). We include the original segment 2, in the candidate set to ensure that the perplexity improves monotonically.
Node Evaluation and Selection:
4. Termination: The refinement process repeats until the perplexity of the solution reaches predefined target threshold or a maximum number iterations is completed. The final output is a refined trajectory 2².
This process allows us to create a dataset of (x, 2., y) triples, which can then be used to fine-tune a base LLM to internalize the deep reasoning capability for open-ended generation from seratch.
is important to distinguish our iterative local search from methods like Monte Carlo Tree Search (Browne al., 2012; Li et al., 2025). First, by using the perplexity of a complete ref-
4

---

Published as a conference paper at ICLR 2026
erence solution quality proxy, REER avoids the computationally expensive rollouts required in MCTS. Second, our approach operates on a "global-to-loe albeit imperfect, global plan and iteratively improve it through local, segment-wise wise edits. This con- trasts with standard MCTS or beam search, which build solutions sequentially by extending partial states. These distinctions make approach scalable and efficient method for operationalizing REER, enabling the creation of large-scale, deep-reasoning datasets for open-ended generation.
principle: we start with a complete,
2.3 TRAINING DATA CURATION
The success our methodology hinges on a large-scale, high-quality dataset of (x, x", ,²,3) triples. The creation of this dataset follows multi-stage pipeline: sourcing diverse query-solution pairs, synthesizing deep reasoning trajectories, and applying rigorous filtering.
2.3.1 Sourcing of (Query, Solution) Pairs. To ensure diversity in style, topic. and complexity, we sourced initial (x.y) pairs from three primary channels. We gathered prompt-story pairs from online using upvotes quality proxy, and reverse-engineered queries
communities like r/ r/Writing (x) from classic Project Gutenberg texts (y) using GPT-4o. Finally, we augmented this collection with data from instruction tuning datasets such as WildChat (Zhao et al., 2024) and LongWriter66 (Bai et al., 2024).
2.3.2 Trajectory Synthesis and Filtering. From our sourced pairs, we selected 20,000 high- quality query-solution pairs covering 25 manually nominated categories to ensure broad topic coverage. For each pair, we executed our iterative local search algorithm to generate optimal deep reasoning trajectory =".
Context Engineering. The efficacy of the search algorithm, however, hinges not only on the search procedure but also on the nuanced design the instructions used to elicit deep reasoning from the generator LLM. We proposed three key designs in our context engineering to ensure high-quality synthesis. We only summarize the key insights here and refer the reader to the appendix for detailed prompts.
1. Enforcing Segment-wise Edits via Meta-Structure. To ensure the generator model performs a true segment-win edit without including edits for the subsequent parts of the trajectory, we enforce a meta-structure for the reasoning process within the prompt. This serves implicit regularizer, helping the model to localize the current segment and constrain its edits to the intended scope when performing segment-wise edits.
2. Injecting Human-like Thinking Patterns. To prevent the synthesis of rigid and formulaic reasoning, we deliberately inject human-like thinking patterns. Prompts explicitly encourage phrases that signify cognitive exploration and self-refiections (e.g., "Hmm..maybe I can...". "Wait, that's a bit..."), triggering a more human-like reasoning style and incentivizing self- reflection through training (Wang et al., 2025b).
Analysis of this synthesis process, shown in Figure 4, confirms its effectiveness. The perplexity distribution shifts significantly lower after refinement, with the vast majority of samples showing marked PPL improvement. Concurrently, the token length of the trajectories increases, to an indicating that the search process successfully expands initial simple plans into more detailed and elaborate reasoning chains.
During instruction tuning, we witness the challenge of repetitive and degenerate thinking. We therefore applied two heuristic filtering strategies to prune low-quality trajectories:
1. End-of- Thinnking Filtering:
We discarded samples where thinking patterns persisted in the final 10% of the sequence. These trajectories risk misleading the model to stuck in a repetitive loop and failing to conclude its reasoning process.
2. Repetition Filtering: We employed repetition metric to measure the frequency of the top-3 n-grams within each trajectory. Samples exhibiting high n-gram repetition, a sign degenerative looping expressions, were filtered out.
This process resulted in a final dataset of 20,000 high-quality deep reasoning trajectories. The distribution of this dataset, shown in Figure highlights its diversity, with significant focus
5

---

Published as a conference paper at ICLR 2026
1030
PPL Distribution
PPL Improvement
Token Length Distribution
850
Token Length Change
2000
000
0:0
2000
430
3.
25
Perpieily
0.2
0.
2-5
1/0
302
3000
IEEO 2000 2700
370 Longth
2003
2300
2002
Figure 4: Analysis Token Length & Perplexity Before and After the Search. The leftmost two plots show that our iterative search process consistently reduces perplexity (PPL). The rightmost two plots show that the process also tends to increase the token length of the thinking trajectory, reflecting the addition of more detailed reasoning steps.
Artistic (Literature and Arts) writing, which is further broken down into sub-genres like Creative Writing and Essay Writing.
Else Advertising
Ordinary
Scripts
Exsay Writing
Schelarly
7.3%
Writing
Artistic
48.0%
Finance
Writing
Categories
4%
12.2%
Law
Coucat Teaching
others
Asventisin
Edu
Use
France Bainew
Ceratiun &. syyaniue writing
Analycial& critic wetting
Eoray Wiiting & muppeeeee Weiting
utly Papa Whiting
poankingg Madu Purformanc
Figure 5: Distribution of the final 20K training dataset by categories taking more than 0.5% account. The primary chart shows diverse range of topics, with a large emphasis on Artistic writing. The detailed view of "Artistic" reveals a focus on Creative Writing and other styles, ensuring comprehensive coverage for open-ended generation.
2.3.3 Final Dataset Assembly for Fine-Tuning. Training model exclusively on domain-specinf data risks overfitting and can degrade its general knowledge priors. To mitigate this, we adopted a mixed-data training strategy. We combined our 20K synthetically generated trajectories with distilled deep reasoning trajectories from public datasets, i.e., OpenThoughts (Guha et al., 2025), that primarily cover domains like mathematics, coding, and science. This blended datasets prevents the model from catastrophic overfitting when learning deep reasoning for open-ended generation.
To train the base LLM, each complete triple in the final dataset was formatted using the prompt template shown in Listing 4 in the appendix. This strueture explicitly teaches the model to first perform deep reasoning before producing the final output, thereby internalizing the desired reasoning process from scratch.
3 EXPERIMENTS
Our empirical evaluation is structured to rigorously validate the efficacy of DeepWriter. We address two central research questions:
1. How does DeepWriter. fine-tuned from scratch on an 8B open-source model, compare against state-of-the-as proprictary models and other powerful open-source alternatives across spectrum of diverse open-ended generation tasks?

---

Published as a conference paper at ICLR 2026
2. What is the individual contribution of each core component of our approach specificallyy, the synthesized deep thinking trajectories, the iterative refinement algorithm, and the characteristics of the thinking traces and the data composition to the model's final performance?
To answer these questions, we first present a comprehensive comparison against leading models, followed by a series of targeted ablation studies. We also provide a qualitative deep-dive analysis into the model's reasoning capabilities in the appendix.
Training Data: Our primary training dataset comprises approximately 20,000 deep thinking tra- jectories, which we synthesized for 16,000 unique queries spanning wide array of open-ended tasks. As stated in Section 3, to prevent catastrophic forgetting of general reasoning abilities, blended this core dataset with public thinking-proee process datasets that reasoning-related domains (e.g.. mathematics, coding). This resulted in final mixed dataset of 37, 37,000 examples, ensuring a balance between specialized open-ended generation capabilities and keeping broad knowledge priors.
We selected Qwen3-8B-Ban as our base model for fine-tuning. This decision was informed by preliminary experiments where other candidates, such as Llama-3.1-88 Base, struggled to effectively internalize the deep thinking process, and Qwen-2.5--7BBBan prohibitive context length limitations. For the critical trajectory synthesis stage, we utilized Qwen2.55 32B-Instruct as the generator model. Fine-tuning was conducted for epochs using constant learning rate of 2 x 10-5 and a global batch size of 96. We set the max step to 10 and stopping PPL to 0.25 in the iterative local search procedure.
Implementation Details:
faced
3.1 EVALUATION BENCHMARKS
To ensure a comprehensive and multi-faceted evaluation, we employed suite of three complementary benchmarks: LongBench-Wrng Write (LB), HelloBench (HB), and WritingBench (WB). Together, they probe three distinct and critical dimensions of generative performance: raw endurance, real-world applicability, and domain-specifie proficiency.
LongBench- Write (LB)) This benchmark functions as targeted stress test for generative endurance. It designed to measure model's ability produce coherent, ultra-long-form text (e.g., > 10,000 words), allowing us to assess the foundational capacity for maintaining thematic consistency over extended outputs.
HclloBench (HB): To gauge practical applicability, HelloBench evaluates performance on diverse set of fin-the-wile tasks sourced from real user queries. Our analysis focuses two key subsets: HB-A (Open-Ended QA), which tests the generation of detailed and nuanced answers, and HB-B (Heuristic Text Generation). which assesses creative reasoning and stylistic fidelity in long-form narrative continuation.
WritingBench (WB): This benchmark is tailored to measure Nomain-sseciife proficiency and controllability across six professional and creative domains: (Academic Engineering). B (Finance & Business), C (Politics & Law), D (Literature & Arts), E (Education), and F (Advertising Marketing). It specifically evaluates the ability to adhere to complex, multi- dimensional constraints, hallmark of advanced open-ended generation.
Given the subjective nature of open-ended tasks, we adopted the established protocol of using powerful LLMs judges within each benchmarks', While we acknowledge the potential for inherent biases in this method, it remains the most scalable and consistent approach for evaluating nuanced generative quality. Specifically, Claude-3.7 was used to score outputs for LongBench and WritingBench, while GPT-40 was used for HelloBench. For HelloBench, we report the original score without rescaling.
Evaluation Protocols:
3.2 MAIN RESULTS
We benchmarked WeepWriter against leading proprietary models (GPT-40, Claude 3.5, Claude 3.7) and strong open-source baseline, 1em2..-33B-Instrrue Qwen3-8B, LongWriter-&B. The results, presented in Table unequivocally demonstrate that our methodology successfully instills sophisticated generation capabilities in an 8B model without costly distillation or trial-and-error.
Using the latest evaluation protocol of WritingBench, we note that there is currently discrepancy reproduced results and paper results, which is also acknowledged by the authors.
7

---

Published as a conference paper at ICLR 2026
Table Main performance comparison on LongBench (LB), HelloBench (HB), and WritingBench (WB). DeepWriter demonstrates competitive performance against leading proprietary models and significantly outperforms other open-source models.
Model
I.B
HB-A
HB-B
WB-A
WR-B
WB-C
WR-D
WB-F
WR-F
GPT-4o
83.1
83.7
87.6
74.4
73.4
74.3
77.9
75.8
78.0
Claude 3.5
89.3
82.9
88.3
59.05
57.6
56.3
59.3
62.0
67.7
Claude 3.7
97.8
83.9
93.2
78.2
77.9
76.5
79.3
79.2
80.8
Qwen2.5-322
35-32B-Insttuct
78.8
81.0
83.8
52.5
49.8
51.0
49.6
53.9
54.2
Qwen338B
85.2
81.4
85.3
68.7
68.9
67.0
67.2
71.2
71.3
Long LongWWiter-8B
76.5
80.1
82.6
57.9
53.9
49.0
52.0
52.9
52.0
Writerie 91.3
82.6
87.4
72.2
71.8
69.8
70.6
73.7
72.3
Table 2: Ablation studies. The full model (top row) is compared against versions with key components removed. Results show that the synthesized deep thinking trajectories and iterative refinement are crucial for performance.
Model Configuration
LB
HB-A
HB-B
WB-A
WB-B
WB-C
WB-D
WB-E
WB-F
DeepWriter-8B (Full)
91.3
82.6
87.5
72.2
71.8
69.8
70.6
73.7
72.3
Remove Synthesis Data
82.9
70.9
63.4
62.8
57.7
66.3
62.7
Remove Iterative Search
83.2
81.0
66.7
67.3
65.6
69.5
70.1
Remove Reflection Tokens
86.9
82.2
82.8
71.6
62.0
69.9
71.9
Doonnsamlle Long Traces
90.3
82.2
84.0
69.6
69.1
67.5
69.8
70.7
Downsample Short Traces
89.3
81.1
82.1
70.8
70.6
70.0
66.9
72.4
69.7
Remove Literature data
88.8
81.6
85.3
71.3
71.0
69.3
69.8
72.2
71.3
Analysis of Main Results. The results in Table reveal several compelling findings. First, DeepWriter-SB consistently and substantially outperforms the strong open-source baseline, Long NongWrittr-SB, across all benchmarks. The performance gap is particularly stark in the diverse WritingBench domains, where DeepWriter achieves average uplift of over 18 points. This high- lights the profound advantage our deep thinking synthesis approach over standard instruction tuning for cultivating advanced generative skills.
Second, DeepWriter-8B closes significant portion of the performance gap with elite proprictary models. On the creative HelloBench task (HB-B), its (87.6) and Claude 3.5 (88.3). More strikingly, on the professional writing tasks in WritingBench, DeepWriter-SB not only surpasses Claude 3.5 by a large margin all six categories but also remains highly competitive with the much larger GPT-4o and Claude 3.7 models. A counter-intuitive result is DeepWriter-8B's score of 91.28 on LongBench- Bong-eenc--rite,, exceeding both GPT-4o (83.1) and Claude 3.5 (89.3). This suggests that explicitly training structured thinking trajectories provides powerful inductive bias for maintaining long-range coherence, a critical challenge in ultra-long text generation.
(87.48) statistically on par with GPT-4o
3.3 ABLATION STUDIES
To meticulously dissect the contribution of each component of methodology, conducted series of ablation studies, with results detailed in Table 2. Each experiment isolates a specific design choice to quantify its impact overall performance.
The ablation results provide robust evidence supporting our methodological design.
Importance of Synthesized Data: Removing our 20K synthesized trajectories and training only on public thinking datasets ("- Remove Synthesis Data") causes the most significant performance degradation across the board. Scores plummet, particularly in creative tasks like HelloBench HB-B B (87.48 - 73.73) and across WritingBench (average drop of over points). hypothesis: it is not merely the presence of "thinking" data that matters,
This confirms but the quality and relevance of structured trajectories tailored for open-ended domains that drive performance.
8

---

Published as a conference paper at ICLR 2026
Impact of Iterative Refinement: Using the initial, unrefined thinking trajectories (2(0) instead of the final, optimized ones (2*) ("- Remove Iterative Search") also leads to a clear drop in performance. While the decline is less severe than removing the synthesis data entirely, the drop on nuanced WritingBench tasks (e.g., WB-A: 72.20 -> 66.72) is substantial. This proves that our perplexity-guided local search is highly effective at discovering superior reasoning paths that translate directly into stronger generative capabilities.
Removing reflection tokens (e.g.. 'Hmm..', 'Wait, that's...' from the synthesis prompts ("- Remove Reflection Tokens") has nuanced effect. While overall scores dip slightly. the most pronounced drop is in WritingBench domain D (Literature & Arts), which falls from 70.57 to 62.04. This suggests that these explicit markers of cognitive exploration, self-correction, and branching are particularly valuable for instilling the flexibility and creativity required in artistic writing tasks.
Effect of Reflection Tokens:
We explored the impact of trace length by selectively downsampling either long or short trajectories. The results reveal a task-dependent preference: removing longer, more elaborate traces ("- Downsample Long Traces") disproportionately harms performance on complex, Nommin--spcciffe tasks like those in WritingBench. Conversely, removing shorter, more concise traces ("- Downsample Short Traces") has slightly larger negative impact on creative tasks like HB-B. This suggests that detailed, multi-step plans are crucial for structured professional writing, while nimbler, more direct reasoning may be optimal for creative ideation.
Role of Trajectory Length:
Removing the data from the "Literature & Arts" and "Or- dinary Life" domains ("- without Literature & Arts data") degrades performance across all benchmarks, not just in the corresponding WB-I) category. This finding indicates that training on creative and narrative tasks imparts more generalizable ability to handle nuance, structure, and open-endedness,, even benefiting performance in more technical domains. This highlights the contribution of the release of our 20K dataset covering comprehensive topics.
Role of Literature & Arts Data:
4 RELATED WORK AND FUTURE DIRECTIONS
The paradigm of "deep reasoning" aims to move beyond rapid, surface-level inference by leveraging increased computational investment at test time, a strategy shown to be effective by advanced models (Team al., 2023; Guo et al., 2025: Jaech et al., 2024; Team, 2025; Muennighoff al., 2025; Fu et al., 2025). This approach gained prominence with methods like Chain-of-Thouug (CoT)) prompting (Wei et al., 2022), which elicits intermediate reasoning steps, and has evolved into more sophisticated strategies like Tree-of-Thouge et al., 2023; Kumar et al., 2024: Zelikman et al., 2022; 2024). While these techniques excel in verifiable domains like mathematics, their application to open-ended, creative tasks remains limited by the absence of a singular ground truth for verification. Our work, REEvrre--Enginees Reasoning (REER), directly addresses this gap.
(Yao et al., 2023) and self-refinement (Madaan
Two dominant paradigms exist for instilling reasoning into a model's 's parameters: reinforcement learn- ing (RL) and instruction distillation. RI. is effective when clear reward signals available (Ouyang et al., 2022; Guo et al., 2025; Team et al., 2025; Wang et al., 2025e), but struggles in creative domains where crafting a reward model to capture subjective qualities is an immense challenge in itself (Ouyang et al., 2022: Zhang et al., 2024; Lu, 2025). WritingZero (Lu, 2025) adopts this approach, but data and models remain closed. While recent work like VeriFree (Zhou et al., 2025) also uses proxy for reward in verifiable domains, REER applies similar principle to recover human- reasoning for the broader challenge of open-ended generation.
Instruction Distillation offers an alternative, wherein reasoning traces are generated by a powerful but proprietary "teacher" model, e-g., GPT-4 (Achiam et al., 2023). While effective, this approach is often prohibitively expensive and is fundamentally capped by the capabilities of the "teacher" model (Guha et al., 2025; Toshniwal et al., 2024). To overcome these data bottlenecks, researchers have increasingly turned to synthetic data generation that builds a solution "forwards" (Wang et al., 2022; Zelikman et al., 2022; 2024). e.g., LongWrite (Wu et al., 2025). Our central innovation is to "reverse- "reverse-engineer" reasoning by synthesizing it backwards from known good outcomes. By operationalizing this as scalable, gradient- tradient-frre search guided by perplexity, we created Deep Writing-204 20K, the first large-scale dataset of deep reasoning trajectories for open-ended tasks.

---

Published as a conference paper at ICLR 2026
Our model, Deep Writer-&B, trained this data, validates REER powerful and cost-effective method, surpassing strong open-source baselines and achieving performance competitive with leading proprietary models like GPT-4o. Importantly, creation and release of Deep Writing-20K also democratizes access to high-quality deep reasoning data, addressing a critical bottleneck for the research community. This opens several promising directions for future research. A primary avenue is scaling our experiments to larger models and datasets to investigate the scaling laws of reverse-engineered reasoning. Furthermore, the core principle of REER well-suited for novel scenarios where reasoning annotations are scarce, making it valuable paradigm to explore in complex domains such as multi step agentic tasks, scientific discovery, and multi-modal reasoning.
ETHICS STATEMENT
This research adheres to the ICLR Code of Ethics. Our work centers on the development of new methodology for training large language models and the creation of new dataset, 'Deep Writing- 20K'. The data for this dataset was sourced from publicly available and permissible sources, including online communities (e.g.. (/WritingPrommts), public domain texts (Project Gutenberg), and existing open-source datasets (WildChat, Long WriteróK). We have strictly filtered on top of theses datasets to ensure that data collection and usage practices respect privacy and do not include personally identifiable information.
The primary goal of this research is advance the understanding of reasoning in Al for open-ended, creative, and professional tasks. However, we acknowledge that, like any powerful generative model, the methods and models presented could be misused for generating harmful, biased, or misleading content. We tried our best to filter out harmful contents and prevented the model from internalizing implicit socictal biases.
The models and datasets used in our research, including the Qwen model series, are used in accordance with their respective licenses. We intend for our open-sourced dataset, 'Deep Writing-20K5 to be used by the research community to foster further investigation into transparent and beneficial AI reasoning mechanisms.
REPRODUCIBILITY STATEMENT
We are committed to ensuring the reproducibility of our research. All components required to replicate findings are detailed within the paper and its appendix, and we provide preview of the data in the supplementary.
The detailed data curation pipeline, including sourcing, synthesis, and filtering procedures, is de- scribed in Section 2.3. The core algorithm for Levvers-Engginees Reasoning (REER)' via iterative local search is detailed in Section 2.2. The exact prompts used for generating initial trajectories, performing segment-wise edits, and conducting inference are provided in the Appendix, Listings 1-4. The code is attached in the supplementary.
We used publicly available base models for our experiments. The trajectory synthesis was performed using 'Qwen2.5-32B 55-22-IInntructt and the final 'Deep Writer-SB model was fine-tuned from 'Qwen3- 8B-Base'. All training hyperparameters, including learning rate, batch size, and number of epochs, are specified in Section 3.1
Our evaluations were conducted on publicly available benchmarks: LongBench- Bergh--ee-rite HelloBench, and WritingBench. The evaluation protocols, including the LLMs used as judges, are described in Section 3.2.
REFERENCES
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint 230.:2333308777 2023.
10

---

Published as a conference paper at ICLR 2026
Yushi Bai, Jiajie Zhang, Xin Lv, Linzhi Zheng, Siqi Zhu, Lei Hou, Yuxiao Dong, Jie Tang, and Juanzi Li. Longwriter: Unleashing 10,000+ word generation from long context llms. arXiv preprint arXiv:2408.000 07055, 2024.
Cameron B Browne, Edward Powley, Daniel Whitehouse, Simon M Lucas, Peter Cowling, Philipp Rohlfshagen, Stephen Tavener, Diego Perez, Spyridon Samothrakis, and Simon Colton. A survey of monte carlo tree search methods. IEEE Transactions on Computational Intelligence and in games, (1):1-43, 2012.
Yichao Fu, Xuewei Wang, Yuandong Tian, and Jiawei Zhao. Deep think with confidence. arXiv preprint arXiv:2508. 15260, 2025.
Tiancheng Gu, Kaicheng Yang. Chaoyi Zhang, Yin Xie, Xiang An, Ziyong Feng. Dongnan Liu, Weidong Cai, and Jiankang Deng. Realsyn: An effective and scalable multimodal interleaved document transformation paradigm. arXiv preprint 2xX 2025.
Etash Guha, Ryan Marten, Sedrick Keh, Negin Raoof, Georgios Smymis, Hritik Bansal, Marianna Nezhurina, Jean Mercat, Trung Vu, Zayne Sprague, et al. Openthoughts: Data recipes for reasoning models. arXiv preprini arXiv:2506.041 2025.
Caglar Gulcehre, Tom Le Paine, Srivatsan Srinivasan, Ksenia Konyushkova, Lotte Weerts, Abhishek Sharma, Aditya Siddhant, Alex Ahern, Miaosen Wang, Chenjie Gu, et al. Reinforced self-training (rest) for language modeling. arXiv preprint arXiv:2308. 08998, 2023.
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in Ilms via reinforcement learning. arXiv preprint arXiv:2501.12944, 2025.
Guangzeng Han, Weisi Liu, and Xiaolei Huang. Attributes as textual genes: Leveraging Ilms as genetic algorithm simulators for conditional synthetic data generation, 2025. URL https: //arxiv. org/ abaaab/5/555 2509. 02040.
Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, et al. Openai ol system card. arXiv preprint arXiv:2412.16670,, 2024.
Zhaolu Kang, Junhao Gong, Jiaxu Yan, Wanke Xia, Yian Wang, Ziwen Wang, Huaxuan Ding, Zhuo Cheng, Wenhao Cao, Zhiyuan Feng. et al. Hssbench: Benchmarking humanities and social sciences ability for multimodal large language models. arXiv preprint (iXiv:2500603922.. 2025.
Aviral Kumar, Vincent Zhuang, Rishabh Agarwal, Yi Su, John D Co-Reyes, Avi Singh, Kate Baumli, Shariq Iqbal, Colton Bishop, Rebecca Roelofs, al. Training language models to self-correct via 2024.
reinforcement learning. arXiv preprint
Peiji Li, Kai Lv, Yunfan Shao, Yichuan Ma, Linyang Li, Xiaoqing Zheng, Xipeng Qiu, and Qipeng Guo. Fastmets: A simple sampling strategy for data synthesis. arXiv preprint 250.222002.1477. 2025.
Xun Lu. Writing-zers zero: Bridge the gap between nen-verif problems and verifiable rewards. arXiv preprint arXiv:2506.00103 2025.
Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao. Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, et al. Self-refine: Iterative refinement with self-feedback. Advances in Neural Information Processing Systems, 36:46534-46594 2023.
Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. sl: Simple test-time scaling. arXiv preprint arXiv:2501.19393 2025.
Long Ouyang, Jeffrey Wu, Xu Jiang. Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730- 27744, 2022.
11

---

Published as a conference paper at ICLR 2026
Haoran Que, Feiyu Duan, Liqun He, Yutao Mou, Wangchunshu Zhou, Jiaheng Liu, Wenge Rong, Zekun Moore Wang. Jian Yang. Ge Zhang, et al. Hellobench: Evaluating long text generation capabilitiies of large language models. arXiv preprint arXfr:2409.16111, 2024.
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Mingchuan Zhang, YK Li, Yu Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint urXiv:2402.03300 2024.
Alex Su, Haozhe Wang, Weiming Ren, Fangzhen Lin, and Wenhu Chen. Pixel reasoner: In- centivizing pixel-space reasoning with curiosity- reinforcement learning. arXiv preprint wrXiv:2505.1596 2025.
Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptia Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, al. Gemini: a family of highly capable multimodal models. arXiv preprint vrXiv:23/2.118055 2023.
Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. Kimi k1. 5: Scaling reinforcement learning with Ilms. arXiv preprint arX5v:2501.12599 2025.
Qwen Team. Qwq-32b: Embracing the power of reinforcement learning, March 2025. URL https: //wp:////wwmmmmmmm github. ht/blog/////oooocoooooooooooooct /elog///ww-3000 32g/
Shubham Toshniwal, Ivan Moshkov, Sean Narenthiran, Daria Gitman, Fei Jia, and Igor Gitman. Openmathinstruct-ll A 1.8 million math instruction tuning dataset. Advances in Neural Information Processing Systems, 37:34737-34774 2024.
Haozhe Wang. Chao Du, Panyan Fang. Li He, Liang Wang, and Bo Zheng. Adversarial constrained bidding via minimax regret optimization with causality- aware reinforcement learning. In Pro- ceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 2314-2325, 2023.
Haozhe Wang, Long Li, Chao Qu, Fengming Zhu, Weidi Xu, Wei Chu, and Fangzhen Lin. To code or not to code? adaptive tool integration for math language models via expectaion-mmxxiiizaio. arXiv preprint arXiv:2502.0000 00691, 2025a.
Haozhe Wang, Chao Qu, Zuming Huang, Wei Chu, Fangzhen Lin, and Wenhu Chen. VI-rethinker: Incentivizing self-reflection of vision-language models with reinforcement learning. arXiv preprint arXiv:2504.08837, 2025b.
Haozhe Wang, Qixin Xu, Che Liu, Junhong Wu, Fangzhen Lin, and Wenhu Chen. Emergent hierarchical reasoning in Ilms through reinforcement learning. arXiv preprint 250.0350000036466 2025c.
Tiannan Wang. Jiamin Chen, Qingrui Jia, Shuai Wang, Ruoyu Fang, Huilin Wang. Zhaowei Gao, Chunzhao Xie, Chuou Xu, Jihong Dai, et al. Weaver: Foundation models for creative writing. arXiv preprint arXiv:2401.17268. 2024.
Yizhong Wang, Yeganch Kordi, Swaroop Mishra, Alisa Liu, Noah A Smith, Daniel Khashabi, and Hannanch Hajishirzi. Self- celf-instruct: Aligning language models with self- eeff-generated instructions. arXiv preprint arXiv:22 /2. 10560, 2022.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of- thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824-24831 2022.
Yuning Wu, Jiahao Mei, Ming Yan, Chenliang Li, Shaopeng Lai, Yuran Ren, Zijia Wang, Ji Zhang, Mengyue Wu, Qin Jin et al. Writingbench: comprehensive benchmark for generative writing. arXiv preprint arxiv:2503.0222. 055444, 2025.
Kaicheng Yang, Jiankang Deng, Xiang An, Jiawei Li, Ziyong Feng, Jia Guo, Jing Yang, and Tongliang Liu. Alip: Adaptive language-iman pre- mre-training with synthetic caption. In Proceedings of the IEFF/CVF International Conference on Computer Vision (ICCV), pp. 2922-2931, October 2023.
12

---

Published as a conference paper at ICLR 2026
Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models, 2023. URL https://arxiv. wgg/abs/2305100001 3:1, 2023.
Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah Goodman. Star: Bootstrapping reasoning with reasoning. Advances in Neural Information Processing Systems, 35:15476-15488, 2022.
Eric Zelikman, Georges Harik, Yijia Shao, Varuna Jayasiri, Nick Haber, and Noah D Goodman. Quiet-star: Language models can teach themselves to think before speaking. arXiv preprint arXiv:2403.09629, 2024.
Lunjun Zhang, Arian Hosseini, Hritik Bansal, Mehran Kazemi, Aviral Kumar, and Rishabh Agarwal. Generative verifiers: Reward modeling as next-token prediction. arXiv preprint arXiv:2408. 15240, 2024.
Wenting Zhao, Xiang Ren, Jack Hessel, Claire Cardie, Yejin Choi, and Yuntian Deng. Wildchat: Im chatgpt interaction logs in the wild. arXiv preprint arXiv:2405.01477 02400,0,47,, 2024.
Xiangxin Zhou, Zichen Liu, Anya Sims, Haonan Wang, Tianyu Pang, Chongxuan Li, Liang Wang, Min Lin, and Chao Du. Reinforcing general reasoning without verifiers. arXiv preprint arXiv:2505.21493, 2025.
13

---

Published as a conference paper at ICLR 2026
A LIST OF PROMPTS
Below we list the exact prompts used for trajectory synthesis and in-house evaluation. For the meta- structure guidelines and thinking pattern injection, refer to Listing For enforcing segment-wise edits, refer to Listing 2. For quality assessment with regard to deep reasoning, refer to Listing 4. For computing the proxy score of a deep reasoning trajectory, we employ Listing 3, without including the reference output.
LISTINGS
Prompt for Generating Initial Thinking.
14
2 Prompt for Segment-wise Edits.
16
3 Prompt for Standard Inference.
17
4 Prompt for Rating Response Quality w.r.t. Deep Reasoning.
19
Listing Prompt for Generating Initial Thinking.
You are expert in many fields. Suppose you will give specific final response, need you to also write down the thought process behind this solution.
Here is a task:
Here is the solution you will create: ()
Now, you need to write down the thinking process behind this solution, as if you are thinking aloud and brainstorming in the mind. The thinking process involves thoroughly exploring questions through a systematic long thinking process. This requires engaging in comprehensive cycle of analysis, summarizing, exploration, reassessment, reflection, backtracing, and iteration to develop well-considered thinking process. Present your complete thought process within a single and unique '<think<</think> tag.
10
Your thought process must adhere to the following requirements:
12
1. **Narrate the first-person as if you are thinking aloud and brainstorming
13
Stick to the narrative of "I" Imagine you are brainstorming and thinking in the mind. Use verbalized, simple language.
14
2. +*Unify the thinking process and the writing solution:** Your thought process must precisely correspond to part of the writing solution. The reader should be able clearly see how your thoughts progressively "grew" into the finished piece, making the copy feel like the inevitable product of your thinking.
16
17
3. +*oone of Voice: Planning, Sincere, Natural, and Accessibles* Imagine you are analyzing and planning what to do before you start to wrtie the solution. Your language should be plain and easy to understand, avoiding obscure professional jargon to explain complex thought processes clearly.
19
20
4. **Logical Flow: Clear and Progressiver*
14

---

Published as a conference paper at ICLR 2026
23 5. *+Thinking Framework for deep thinking=*
To ensure your thinking is clear and deep, to showcase your thinking and planning to fulfill the task, below is what you mi might cover when you
thinking aloud and brainstorming.
25
26
Understanding the user intent and the task: Before putting pen to paper, I need to thoroughly consider the fundamental first need to discern the user's
purpose of the writing. true goal behind their literal request. Next, I will consider: Who am talking to? will create a precise profile of the target reader, understanding their pain points, aspirations, and reading context. Then, I will establish the Core Objective: What specific emotional, cognitive, and behavioral changes do I most want the reader to experience after reading?
28 Establishing the content: I need to brainstorm a core creative idea and communication strategy centered around my objective. Then, I will think about what content and key information I need convey to the reader to fulfill the writing task, and what source materials this will involve.
29
Building the structure: need to design clear narrative path for the reader, like a "blueprint." First, will plan the article's skeleton (e.g., using framework like the Golden Circle "Why-How-What," the AIDA model "Attention- Interest-Desire-Acttoo, or narrative structure "Beginning- Then, I will plan the key modules: How will the introduction hook the reader? How will the body be layered and the arguments arranged? How will the conclusion summarize, elevate the message, and provide a clear Call to Action (CTA)
Devellopentt-Ciiinneeeestttttooe
31
Outline: If the task output might be relatively long, I will consider writing an outline (or a draft) which naturally derives from the plan above. Specifically, the outline will ground my plan into paragraphs, summarizing the key content for each paragraph and what are the key points here, sentence structure or anything important for the paragraph.
PROMISE will NOT copy the solution will NOT copy the solution, this outline (or draft) should only look like a prototype or outline of the target text. After finishing this outline, I will check again if there are any details or notes should pay attention to when writing the final solution. I will begin writing this draft after 1--- Outline (or Draft separator the end of my thinking process. The draft will be included in the same '<think></thnnk> block.
35
37
6. Throughout the thinking process, I want to involve deep thinking and planning, and use deliberate self-eritiqueeselff- reflection in my thinking process. Trigger these by regularly using patterns such 'wait', 'maybe', 'let me', etc. For example:
Hmm, maybe (other concrete thinking regarding the given request)
Let me think
Wait no.
15

---

Published as a conference paper at ICLR 2026
But wait (might find something wreng with your previous thoughts)
Wait, t bit .. (reflections about previous decisions)
Let me think (are thinking of other possibilities) Wait, the user said (backtracing of previous information) So
44
Hmm. Alternatively, maybe (branching on other possibilities)
45
But
But I promise will use diverse triggers and will NOT same triggers repeatedly. will use these when analyzing user needs, establishing content and structure and when consider alternatives, backtracing and the details. I will NOT use them when write the draft or I am approaching the end of thinking.
48
In the thinking process, make sure NO PAST TENSES, NO PAST TENSES, because this is the thought process before you are write final solution. You are planning what you will and you need to do.
Imagine you' re thinking aloud and brainstorming, Write it as an internal monologue or a stream of consciousness. Do not use bullet points, numbers, or formal section headings. Now record your thinking process within <<thin></thinnk> tags.
50
Listing 2: Prompt for Segment-wise Edits.
Your task is to receive user request, target output, and an existing thinking process, and then to refine and enrich specific paragraph within that thinking process.
---> +*Task**
()
---> **aagget Output.*
()
---> Thinking Process**
()
<replace>
10
()
</replace>
12
()
14
Follow this three-step method to construct your response:
16
**Step 1:** Locate the paragraph you need to revise within the existing thinking process. In relation to the aurrounding context, what is the primary function of this paragraph?
18 **Seep 2:** Read the "Target Text" and the "paragraph to be revised" side-by-side, Ask yourself key question: Does the thinking process reflected in this paragraph lack crucial steps, or is there content that could be further optimized and detailed to better align with the Target Text?
19
**Step 3:** Improve and optimize the paragraph (that represents part of the thinking process).
Based the analysis, refine the initial target paragraph into a new one, base remain the tone. Put the refinement into < refine></refine> tags.
16

---

Published as a conference paper at ICLR 2026
To help involve deep thinking and planning, consider deliberate in your thinking process.
Trigger these by frequently using patterns such wait'' maybe', 'let me', , etc. For example: Hmm, maybe ... (other concrete thinking regarding the given request)
24
Let me think
25
Wait no
26
But wait (might find something wrong with your previous thoughts)
27
Wait, that's s a bit .. (reflection about previous decisions)
Let me think ... (are thinking of other possibilities) Wait, the user said (backtracking of previous information) So Hmm. Alternatively, maybe (branching on other possibilities)
28
29
30
But
31
If the function of the paragraph being improved is to serve as first draft of the text, you must focus on enhancing the text' s logic and completeness. The draft should not outline but should express specific content and state a clear point view. Consider whether the current draft. an appropriate prototype for the Target Text: it should be neither too vague nor a direct copy, but should reflect a foundational version.
general
32
Based the guide above, you are to refine **only** the section marked for replacement below. <replace>
35 ()
36
</replace>
37
your response, first, present your analysis following the three -step method within "Kanalyze></amalyze tags. Finally, place the corresponding, refined paragraph of the **thinking process** within <rrefnne</refin>>>> tags.
39 Notes: a. Avoid repeating. Reduce the use the same connection words, avoid repeating the same meanings over and over again. Ensure that your revised content does not repeat information from the context.
b. please keep the first a few words of the original paragraph, especially the connection words
41
c. use self-critique trigger words, such as 'wai'', 'maybe', 'let me', etc.
Listing 3: Prompt for Standard Inference.
2. You are an expert in many fields. Suppose you will give speci fic final response, need you to also write down the thought process behind this solution.
Here is a task:
()
Now, you need to think aloud and brainstorm in the mind. The thinking process involves thoroughly exploring questions through systematic long thinking process. This requires engaging in a comprehensive cycle of analysis, sunnarizing, exploration, reassessment, reflection, backtracing, and
17

---

Published as a conference paper at ICLR 2026
iteration to develop well-considered thinking process. Present your complete thought process within a single and unique k think></think> tag.
Your thought process must adhere to the following requi rements:
1. **Naratte the brainstorminge*
as if you are thinking aloud and
11
Stick to the narrative "I" Imagine you are brainstorming and thinking in the mind. Use verbalized, simple language.
12.
2. **Unify the thinking process and the writing solution:*x
14
Your thought process must precisely correspond to a part of the writing solution. The reader should be able to clearly how your thoughts progressively "grew" into the finished piece, making the copy feel like the inevitable product of your thinking.
3. **Tone of Voice: Planning, Sincere, Natural, and Accessibler Imagine you are analyzing and planning what to do before you start to wrtie the solution. Your language should be plain and easy to understand, avoiding obscure professional jargon to explain complex thought processes clearly.
17
18
4. *ogiccal Flow: Clear and Progressiver
20
5. *Thinking Framework for deep thinking**
22
To ensure your thinking is clear and deep, to showcase your thinking and planning fulfill the task, below is what you might ght cover when you are thinking aloud and brainstorming.
23
24
Understanding the user intent and the task: Before putting pen to paper, need to thoroughly consider the fundamental purpose of the writing. I first need to discern the user's true goal behind their literal request. Next, will consider: Who talking to? will create precise profile of the target reader, understanding their pain points, rapions and reading context. Then, I will establish the Core Objective: What specific emotional, cognitive, and behavioral changes do I most want the reader to experience after reading?
25
26 Establishing the content: need brainstorm a core creative idea and communication strategy centered around objective. Then, will think about what content and key information need to convey to the reader to fulfill the writing task, and what source materials this will involve.
28 Building the structure: need to design a clear narrative path for the reader, like a "blueprint. First, will plan the article's skeleton (e.g., using a framework like the Golden Circle "Why-How-What," the AIDA model "Attention- Interest-Desire-Aettion/ or a narrative structure "Beginning- Devel o modules: How will the introduction hook the reader? How will the body be layered and the arguments arranged? How will the conclusion summarize, elevate the message, and provide a clear Call to Action (CTA)?
Then, I will plan the key
18

---

Published as a conference paper at ICLR 2026
Draft: unless it is really easy request, otherwise need to consider writing a draft based on the plan above, before you give the final writing solution. into paragraphs, considering the key points, content, and sentence structure for each. This initial draft should look like prototype of the target text. This draft will be way shorter than the final writing solution within controlled length, but it must also avoid being too vague or general or simply copying the final text. will begin writing this draft after ---- The Draft --- separator at the end of my thinking process. The draft will be included in the same k think></think>> block. After writing the draft, will further critique what can be improved, and analyze what details can be enriched (and hence make it more likely to eventually arrive at the given solution)
will translate my plan
32
6. Throughout the thinking process, I want to involve deep thinking and planning, and use deliberate self-crittque/sel- reflection in my thinking process. Trigger these by frequently using patterns such as 'wait', 'maybe', 'let me', etc. For example:
33
Hmm, maybe (other concrete thinking regarding the given request)
34
Let me think
Wait
36
But wait (might find something wrong with your previous thoughts)
37
Wait, that's s a bit (reflections about previous decisions).
Let think .. (are thinking of other possibilities Wait, the user said (backtracking of previous information) So Hmm. Alternatively, maybe (branching on other possibilities) But
38
39
41
Now record your clear, complete, and logical thinking process within <tthin><</hinkk> tags.
In the thinking process, make sure NO PAST TENSES, NO PAST TENSES, because this is the thought process before you are to write final solution. You are planning what you will and you need to do.
Imagine you' thinking aloud and brainstorming, Write it an internal monologue or a stream of consciousness. Do not use bullet points, numbers, or formal section headings.
Listing 4: Prompt for Rating Response Quality w.r.t. Deep Reasoning.
You are an expert judge in generated content. Your primary task is to assess an AI model's response, specifically focusing on its ability to perform **deep thinking and planning**, You will evaluate the response across five distinct dimensions. A model that excels at deep thinking will not only provide a correct answer but will demonstrate structured, logical, and well-grounded reasoning process from start to finish.
Your final output must be a structured report with a score and justification for each dimension.
19

---

Published as a conference paper at ICLR 2026
## Evaluatia Dimensions & Scoring
### 11. Understanding & Problem Decomposition
**Relation to Deep Thinking:** This is the foundational step. Deep thinking is impossible without first accurately understanding the problem in its entirety. This dimension measures if the model comprehends the user's explicit and implicit needs and then breaks down the complex request into manageable, logical parts. This act of decomposition *is* the first stage of planning.
14
**Score (Poor) :** The model fundamentally misunderstands the user's s request or ignores key components. The response is off- topic fails to address the core problem.
**Score 3 (Average) :** The model grasps the main goal but may overlook nuances or implicit constraints, It attempts to break down the problem, but the decomposition may be incomplete or sl slightly illogical.
**Score 5 (Excellent) :** The model demonstrates
comprehensive understanding of the user's s intent, including subtle details. It expertly deconstructs the problem into clear, exhaustive, and actionable framework.
Score 2 and Score 4 fit interpolate into the above scoring criterion.
30
21
### 2). Content Structure Logical Consistency
22
23
**Relation to Deep Thinking:** This dimension reflects the clarity and order of the model's thought process. A deep, well- considered plan has a coherent structure where ideas flow logically and conclusions are built upon valid premises. Inconsistencies or chaotic structure indicate shallow, stream-of-consciouse planning.
generation rather than deliberate
25 **Score (Poor) :** The response is disorganized, rambling, or internally contradictory. It's difficult to follow the model' s line of reasoning.
26
discernible structure (e.g., uses headings, lists), but the flow between sections could be improved. It is mostly consistent, with only minor logical gaps.
**Score (Average) :** The response
25
**Score (Excellent) :** The response impeccably structured
Bach part logically follows from the previous one, building a coherent and compelling argument or plan. The internal logic is sound and easy to follow from beginning to end.
28
Score 2 and Score 4 interpolate into the above scoring criterion
29
31
### 3\. Depth of Analysis & Synthesis
32
**eelation to Deep Thinking:** This is the core of "deep thinking goes beyond simply retrieving facts and measures the
20

---

Published as a conference paper at ICLR 2026
model's ability to analyze underlying principles, connect disparate ideas, and synthesize them to create new insights. simple plan lists steps; deeply thought-out plan explains why+ those are the right steps and how they interrelate.
35 **Score (Poor) :** The response is superficial, relying on cliches or surface-level information. It shows no evidence of analyzing the "why" behind the "what." +*Score (Average) :** The model provides a competent analysis explaining concepts correctly but treating them in isolation. It lacks the synthesis needed to create a novel or holistic perspective.
37
**oorre (Excellent) :** The model provides a profound analysis, connecting concepts in insight ways. It synthesizes information to offer a nuanced perspective that is more than the sum of its parts, demonstrating true grasp of the subject matter.
Score 2 and Score 4 interpolate into the above scoring criterion
39
41
### 4%. Presentation Clarity
43
**ellttinn to Deep Thinking:** A brilliant plan is useless if it cannot be understood. This dimension assesses the model's ability to communicate its complex thoughts and plans effectively. Clarity in presentation demonstrates higher level of understanding, as the model must distill its reasoning into format that is concise, accessible, and acti actionable for the user.
45 **Score (Poor) :** The response is convoluted, filled with jargon, or poorly formatted. The user would struggle to understand the main points or how to act on the advice. +*Score (Average): :** The response is generally understandable but could be more concise or better organized. It may be overly dense or require the user to re-read sections to grasp the meaning.
46
47
**oorre (Excellent) :** The response is exceptionally clear, concise, and well-formatted. It uses plain language and effective formatting (like lists, bolding, or tables) to make complex information easy to digest and act upon.
48
Score 2 and Score 4 interpolate into the above scoring criterion
50
### 5%. Factual Grounding (Hallucination Check)
52
Reelaaion to Deep Thinking:** Deep thinking and planning must be grounded in reality to be useful. A plan built on fabricated information ("hallneinations") is fundamentally flawed and remonstrates a critical failure in the reasoning process. This dimension acts
crucial check on the validity of the model'ss entire output.
55 *This dimension is scored on severity scale, not a quality scale
21

---

Published as a conference paper at ICLR 2026
Sccore (Factually Sound) :** The response contains discernible factual errors or hallucinations.
Contains small error (e.g., a slightly incorrect date, a minor misstatement) that does not undermine the overall logic conclusion of the response. **Score 2 (Significant Hallucination) i** Contains majer factual error that invalidates key part of the argument or plan. The response is partially unreliable.
**Score (Minor Inaccuracy)
+*Score (Critical Hallucination) :** The core premise or a critical component of the response is based on fabrication, rendering the entire output untrustworthy and potentially harmful.
61
Score 1 interpolates into the above scoring criterion.
63
## Final Output Format
65
Please provide your evaluation in the following structured json format.
'jjoon
69
"ewaluattonRepoott""
ainderrstandingnndeeoossst
71
"score": "[Enter a score from 1-5]",
"justification": " "[Yur justification here. Explain why you gave this score.]"
),
74
"ssructureadCConsisteenyy"
"score": [[ntter score from 1-5]", "justification"" "[Your justification here. Explain why you gave this score.]"
76
77
},
78
"depthofanalysis:: {
"score"; "[Enter a score from 1-5]", "justification": " "Your justification here. Explain why you gave this score.)
80
81
},
"presentationnlaaitt"::
83
"score": [Enter a score from 1-5]",
"justification": " "[Your justification here. Explain why you gave this score.]"
},
86
"factualGrounding"::
"severityScore": " "Entter a severity score from 1-5]", "justification": " "[Describe any inaccuracies or hallucinations found. If none, state Response is factually sound.']
},
"overallSummazy": "[Provide final, concise paragraph summarizing the model's overall performance in deep thinking and planning. A response with Hallucination Severity Score of 2 or 3 cannot be considered a high-quality example of planning, regardless of other scores.)
91
93
94
<User Request>
96
22

---

Published as a conference paper at ICLR 2026
$INSTę
98
</User Request>
100
<Response>
102
103
SRESPONSES
105
</ </Response>
106
107
Now go back to the evaluation guideline and give the json report
B QUALITATIVE ANALYSIS
B.1 GENERATION QUALITY OF DEEP THINKING
Beyond quantitative scores, sought to understand how well DeepWriter internalizes the qualities of deep thinking. To this end, we conducted qualitative analysis, scoring model outputs on five dimensions intrinsically linked to advanced reasoning and planning:
Generation Quality (Scores Normalized) hrbblem
Deconstoctionn
Problem Deconstruction:
The ability break down complex prompt into logical hierarchy of sub- goals. This is the foundation of effective planning.
Factuál
Lopicl Condetency
Gruuninng
Logical Consistency: Maintaining coherent and non-contradictory reasoning path throughout the en- tire generation necessistates the ability to plan over the generation.
Presentation
Dupti
Depth of Analysis: Moving beyond surface-level responses explore nuances, consider alternatives, and demonstrate sophisticated understanding. This reflects the "deep" aspect of the thinking process.
loegin
Coale
Figure Qualitative comparison of gen- eration quality. Scores are normalized across five dimensions related deep thinking. DeepWriter-88 shows a rea- soning profile far superior to open- source baseline and is competitive with top proprictary models.
Presentation Clarity: The ability to structure the final output in a clear, organized, and persuasive man- ner, which is direct outcome of well-formed inter- nal plan.
Factual Grounding: Ensuring that generated content, where applicable, is accurate and well- supported, reflecting robust and awaae rea- soning process.
The normalized scores, visualized in the radar chart in Figure 6, provide a signature of each model's reasoning profile. As illustrated in Figure 6, DeepWriter-8% exhibits remarkably strong and well- rounded reasoning profile. Its performance polygon significantly envelops that of the LongWriter 8B baseline, showing dramatic improvements across all five dimensions. This confirms that our methodology genuinely enhances underlying reasoning capabilities, rather than just improving superficial output fluency.
Furthermore, DeepWriter-B 3.5, particularly in Depth of Analysis and Factual Grounding. While the state-of-the-ar Claude 3.7 still defines the frontier, especially in Depth of Analysis, our 8B model has demonstrably bridged a large portion of the capability gap. This validates central claim: instilling deep thinking process through gradient-free synthesis is a highly promising pathway toward building more powerful and scalable models.
profile closely rivals that of GPT-4o and substantially exceeds Claude
23

---

Published as a conference paper at ICLR 2026
B.2 QUALITATIVE COMPARISON OF THINKING PATTERNS
To better understand how injecting human-like thinking patterns during synthesis affects the model's behavior, we analyze the frequency of reasoning phrases generated by the full model versus the ablated model trained without injecting thinking patterns. The thinking patterns are deduplicated such that occurances will be counted only once for the same solution.
Comparison of Deduplicated Thinking Pattern Frequencies (Top 50)
Thinking Injection
Thinking Injection
alright
75.25
but
0..3%
let me think
10.3%%
first
$7.780
first
22.39
now
$2.830
07.010
05.035
need understand
10.224
243320
wait
$ specifically
next
goal Is to
need to understand
17.07
to ensure
15,00
hmm
56.330
let's
16.000
to ensure
11.97
can
14,270
need
thus
13.44%
similarly
start by
iRterativall
start by
maybe i can
to find
0.39%
specifically
1..37
wait
6.32%
however
17.7h
since
7,98%
need consider
15.99
similarly
10.779
maybe
5,220
way
therefore
break it down
analyze the
we can
this suggests
but wait
1.30
hmm
5.00
need to think about
7,300
let me think
is
1.00
should focus
2.40
Identify
stap step
datermine
should focus on
this suggests that
should consider
in way that
check
6.mm
need to think about it seams
analyza the
4.1.10
3,50%
context
140%
identify
step
are
therefore
okay
note that
alternativaly
need to analyze
this means that
need to show
we see
need identify
find
to find
need to
suggests
this suggests
should con consider
to make it easier
break down
this means that
need to find airight
It seems
need to
but walt
1%
10%
1%
10%
Frequency (larra
Trequency (hommlloos)
Figure 7: Comparison of the top 50 thinking pattern frequencies for models trained with and without the injection of human-like thinking patterns during data synthesis. The model with injection (left) shows more diverse and balanced distribution of patterns, while the model without (right) relies heavily on a few formulaic phrases.
As shown in Figure 7, the difference is stark. The model trained with thinking pattern injection exhibits a more diverse and evenly distributed use of thinking patterns. Tokens indicating reflection and self-correction such as 'let me think', 'maybe', 'hmm', and 'wait', are prominent. This suggests a more flexible, human-like reasoning process with cognitive exploration. In contrast, the model trained without this injection relies on small set of highly frequent phrases like 'next', 'first', and "goal is to'. The frequency distribution is highly skewed, indicating a more rigid and formulaic reasoning process.
24

---

Published as a conference paper at ICLR 2026
This analysis confirms that the proposed context engineering techniques encourages the model to adopt a more nuanced and reflective approach to moblem--solving which, as shown in the ablation studies, is particularly beneficial for creative and complex tasks.
C BEHAVIORAL ANALYSIS
We conducted preliminary analysis on the model's behaviors. Figure 8 shows the token length distribution of DeepWriter-8B responses on LongBench- Write.
Thask Pat
Arnwrr
16.5%
17.76
4.%
10.3%
6.7%
4.7%
23%
Figure 8: Token Length distribution of Thinking and Answer part of DeepWriter-SB.
We also compare the response string length distribution across leading models in Figure 9. While DeepWriter achieves superior performance competitive with frontier models, it does not introduce excessive response length like LongWriter. The average response length is around 5000 tokens, comparable with frontier models like GPT-4o and Claude-3.7.
Caude-337
Cleade3.5
LonpWrttend0
GFS4e
Dseplinband0
0%%%
6.%
20%
00.0
4000
1000
2803
10:0
2000
2900
(X09
1900 10:0
1300 1083
ZICE MRE 7881 10000 12000 1200
201%
Figure 9: Response String Length Distribution across different models.
D CASE STUDIES
Due to formatting issues with latex, we put few case studies in the supplementary materials. We manually review the cases where DeepWriter can outperforms other models, confirming the argument that DeepWriter can achieve better depth, logical consistency and factual grounding via deep reasoning traces. We also analyze the eиror patterns of the generations from DeepWriter. We find that Deep Writer often gets lower score due to domain knowledge gap. implying the benefits of training with more diverse corpus of topics and domains.
E EXTENDED RELATED WORK
Deep Reasoning and Test-Time Computation. The paradigm of "deep reasoning" (or Long CoTss)
aims to move beyond rapid, surface-level inference by leveraging increased computational investment at test time. Advanced models from organizations like Google (Team et al., 2023), DeepSeek AI (Guo et al., 2025), and OpenAl (Jaech et al., 2024) have demonstrated the effectiveness of this test-time scaling (Team, 2025; Muennighoff et al., 2025; Fu et al., 2025). This approach gained prominence with methods like Srain-of-Thhugee (CoT) prompting (Wei et al., 2022), which elicits intermediate reasoning steps guide model toward more accurate solutions. Building this,
25

---

Published as a conference paper at ICLR 2026
more sophisticated strategies have emerged, such as free-of-Thoughh (ToT) (Yao et al., 2023), which explores a tree of possible reasoning paths, and various self cor-ecctroonnn or self-refinement (Madaan et al., 2023; Kumar et al.. 2024; Zelikman et al., 2022: 2024) mechanisms that iteratively improve an initial response. While these approaches have yielded remarkable performance gains in verifiable domains like mathematics and programming, their application to open-ended, creative tasks remains largely unexplored due to the absence of a singular ground truth for verification. REER addresses this gap by developing method instill this deliberate, structured thinking capability for non-verifiable creative domains.
Paradigms for Instilling Reasoning. Beyond prompting techniques at inference time, two dominant paradigms exist for integrating advances reasoning capabilities directly into a model's parameters: reinforcement learning and instruction distillation.
Reinforcement Learning (RL) has been instrumental in aligning LLMs with human preferences (RLHF) and improving performance on tasks with clear reward signals (Ouyang et al., 2022; Guo et al., 2025; Team et al., 2025; Wang et al., 2025c). In verifiable domains, a correct outcome provides a straightforward positive reward, effectively guiding the model's search through a vast solution space (Shao et al., 2024; Wang et al., 2025a;b; Su et al., 2025). However, this reliance verifiability presents a formidable barrier when applied to open-ended generation (Ouyang et al., 2022; Lu, 2025). Crafting a reward model that can reliably approximate nuanced and subjective qualities like originality or emotional resonance is an immense challenge in itself (Ouyang et al., 2022; Zhang et al., 2024). Furthermore, the subsequent RL process is often computationally burdensome and sample- inefficient (Shao et al., 2024; Gulcehre et al., 2023; Wang et al., 2023). Recently VeriFree (Zhou al., 2025) extends verification-base reward to likelihood-based reward for reinforcement learning on verifiable domains. Likewise, GEverse-Enginess using a proxy to judge the reasoning quality. However, the motivation is fundamentally different we focus on recovering liken-like deep reasoning from known- -oond-doddddo outputs for the broader open-ended generation problems.
Reasoning (REER) shares the principle of
Instruction Distillation offers an alternative, wherein reasoning traces are generated by a powerful "teacher" model (e.g.. GPT-4 (Achiam al., 2023)) and used as training data for a smaller "student" model. While effective, this approach is constrained by two fundamental limitations. First, it is often hampered by the prohibitive cost of querying state-of-the-art proprietary models at scale (Guha et al., 2025; Toshniwal et al., 2024). Second, and more fundamentally, distillation is capped by the teacher's s abilities a student model cannot learn a capacity that the teacher does not already possess (Toshniwal al., 2024). This limitation is exacerbated by the general scarcity of high-quality, open-source instruction data tailored for advanced creative tasks (Bai et al., 2024).
To overcome these data bottlenecks, researchers have increasingly turned synthetic data generation. Most approaches use a powerful LLM to generate new query-response pairs, often to augment existing datasets bootstrap capabilities new domains (Wang al., 2022: Zelikman et al., 2022: 2024; Gu et al., 2025; Yang et al., 2023; Han et al., 2025). These methods aim to build a solution "forwards" for given query through data synthesis. Our central innovation is to "reverse-end reverse-engineer" reasoning synthesize deep reasoning "backwards" from known good outcome such as human- written solutions.
Writing Datasets, Models and Benchmarks Prior work has explored both synthetic data pipelines and RL in Al writing. For instance, Weaver (Wang et al., 2024) proposed instruction tank-translation, LongWriter (Bai et al., 2024) proposed an agentic data pipeline to synthesize long-form writing outputs and introduced the LongBench-Writ benchmark. In contrast, Writing-Zero (Lu, 2025) employed an RL approach, training reward model on private datasets, but its training data remains unreleased. DeepWriter, to our knowledge, is the first to instill deep reasoning for open-ended generation using a scalable, open synthetic data approach.
Evaluation in this domain relies on recently developed benchmarks. HelloBench (Que et al., 2024) proposes a diverse collection of "in-tte-wild" tasks from real user queries to gauge practical applica- bility. Meanwhile, WritingBench (Wu et al., 2025) measures domain-specific proficiency and the ability to adhere to complex, multi malti-dimensional constraints across six professional domains.
26

---

Published as a conference paper at ICLR 2026
F LLM USAGE
We acknowledge the use of large language models (LLMs) during the preparation this manuscript. The application of these tools was strictly limited assistive role for improving the quality of the writing, Specifically, LLMs were utilized to enhance clarity, refine sentence structure, and ensure smooth and logical flow of arguments throughout the paper. The core ideas, methodology. experimental results, and all intellectual contributions presented herein are entirely the work of the authors. LLMs were not used generate any of the substantive research content or analysis.
27


Reviews:
[{"review_id": "lE79mZ5F9n", "reviewer": "Reviewer_XQFR", "summary": "This paper introduces REverse-Engineered Reasoning (REER), a new paradigm that infers plausible step-by-step reasoning paths backwards from high-quality outputs, instead of constructing them via reinforcement learning or distillation. By formulating reasoning-path recovery as a gradient-free local search minimizing perplexity, the authors synthesize 20 K structured reasoning trajectories (DeepWriting-20K) and fine-tune an 8B model (DeepWriter-8B). The model achieves near-proprietary-level performance on LongBench-Write, HelloBench, and WritingBench, outperforming all open-source baselines.\nOverall, the work provides a cost-effective way to instill deep reasoning for open-ended generation. It offers both a scalable data-generation framework and strong empirical evidence that reverse-engineered reasoning can rival large proprietary models without RL or teacher supervision.", "questions": "1. Under what conditions does REER fail to find meaningful reasoning trajectories (e.g., nonsensical outputs, repetitive loops), and how are such cases handled?\n2. How would REER behave when applied to larger base models or multimodal datasets—would it continue to scale effectively or face diminishing returns?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Scalable and Cost-Effective Framework: REER eliminates the need for teacher models or reward functions by using a gradient-free local search guided by perplexity, enabling large-scale, low-cost synthesis of deep reasoning trajectories.\n2. High-Quality Public Dataset: The released DeepWriting-20K dataset of 20 K structured reasoning trajectories across 25 domains is a valuable open resource for future research on reasoning and creative generation.\n3. Strong Empirical Results: The proposed DeepWriter-8B model achieves near-proprietary-level performance on LongBench, HelloBench, and WritingBench, significantly outperforming all open-source baselines.\n4. Extending Deep Reasoning to Non-Verifiable Tasks: REER successfully brings deep reasoning to open-ended and creative domains (e.g., writing), bridging the gap between verifiable reasoning (math, code) and subjective generation tasks.", "weaknesses": "1. Subjective Evaluation via LLM Judges: Benchmarks rely heavily on GPT-4o or Claude-3.7 as evaluators, which introduces bias and limits reproducibility; no human or standardized blind evaluation is included.\n2. Scalability and Efficiency Trade-offs: The iterative local search for trajectory refinement can be computationally expensive for large datasets, and its efficiency compared to distillation or RL remains under-analyzed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "mxDLqkcHdy", "reviewer": "Reviewer_ngby", "summary": "This paper introduces Reverse-Engineered Reasoning (REER), a new paradigm for teaching deep reasoning in open-ended, creative tasks. Traditional methods like RL and distillation fail here due to a lack of clear reward signals or high costs. REER works \"backwards\": it starts with a high-quality solution and computationally discovers the latent reasoning process that could have produced it. This is framed as a gradient-free search to find a thinking trajectory that minimizes the perplexity of the known-good answer. Using this scalable method, the authors created the DeepWriting-20K dataset. The resulting model, DeepWriter-8B, rivals or exceeds top proprietary models like GPT-40 and Claude 3.5 on challenging writing benchmarks.", "questions": "1. What if we use a reward model (e.g., a process reward model) as a proxy for the quality of the reasoning trajectory? Will REER yield better performance?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. This paper introduces a novel approach that fundamentally shifts how reasoning is taught. Instead of building reasoning forwards through trial-and-error (RL) or imitation (distillation), it works backwards from known good solutions to computationally discover the latent, step-by-step thinking process that could have produced them. This provides a new path for tackling deep reasoning in open-ended, non-verifiable domains.\n\n1. REER is a scalable, gradient-free approach. It successfully bypasses the sample inefficiency of RL and the prohibitively expensive cost of distillation. This creates a cost-effective and automatable pathway to generate vast quantities of high-quality, deep-thinking training data at a scale that was previously impractical.\n\n1. The authors used their method to create and open-source DeepWriting-20K, a large-scale dataset with 20,000 deep reasoning trajectories. This comprehensive resource, spanning 25 categories, is designed to mitigate data scarcity and catalyze future research into planning and structured thought for open-ended generation tasks.\n\n1. The resulting 8B model, DeepWriter-8B, outperforms strong open-source baselines. More significantly, it achieves performance competitive with leading proprietary models like GPT-4o and Claude 3.5. This provides rigorous empirical evidence that human-like deep reasoning can be cultivated without relying on costly distillation or RL.", "weaknesses": "1. The paper's main evaluations rely on using powerful LLMs (like GPT-4o and Claude 3.7) as judges. The authors acknowledge that this method, while scalable, has potential for inherent biases.\n\n1. The core REER algorithm depends on using the perplexity (PPL) of the reference answer as a proxy for the quality of the reasoning trajectory. The assumption that lower PPL, which makes the answer seem maximally probable and logical, directly equates to a higher-quality reasoning process is a proxy and may not perfectly align with human judgments of logic, creativity, or efficiency.\n\n1. The primary experiments were conducted on an 8B base model (Qwen3-8B-Base). The effectiveness of REER on much larger models is not yet validated, as mentioned by the authors in Section 4.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "uHASkzDt05", "reviewer": "Reviewer_bq1Q", "summary": "This paper introduces Reverse-Engineered Reasoning (REER), a novel paradigm that generates reasoning data for open-ended tasks by working backward from known high-quality solutions instead of using costly RL or distillation. This method frames trajectory discovery as a gradient-free search to find the reasoning path that minimizes the solution's perplexity, resulting in a model trained on this data that competes with top proprietary models.", "questions": "See weaknesses.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The motivation is clear and important.\n2. The paper is well-written and easy to follow.\n3. The proposed method has potential for other domains where reasoning paths are rare and hard to obtain.", "weaknesses": "1. The core of the entire method is to assume that minimizing PPL ($y | x, z $) can find a high-quality inference trajectory $ z $. However, the paper does not prove the direct and reliable correlation between PPL and reasoning quality (e.g., efficiency, creativity, and logical depth). PPL often represents the quality and fluency of language, yet it is not equivalent to reasoning quality.\n2. The computation of PPL is based on the base model; however, the value of PPL can vary across different LLMs. The paper only conducts experiments on Qwen3-8B, which is insufficient to demonstrate the generalizability in different LLMs.\n3. It would be interesting to investigate the performance across different model sizes, which can further verify the ubiquity of the proposed method.\n4. The depth and diversity of reasoning paths lack a quantitative evaluation.\n5. Although claiming to be more cost-effective than distillation and RL, the paper does not provide a specific cost analysis or comparison of the REER synthesis process, which involves multiple LLM calls and PPL calculations.\n6. The evaluation (e.g., PPL) of reasoning path quality is narrow, and it is hard to verify whether the reasoning paths are correct. Wrong reasoning paths can also have low PPL due to other factors (e.g., hallucination), leading to my concern regarding real-world application and applying to other tasks.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)