LongWriter-Zero: Mastering Ultra-Long Text Generation via Reinforcement Learning
Authors: Yuhao Wu
Yushi Bai
Zhiqiang Hu
Roy Ka-Wei Lee
Juanzi Li
Venue: ICLR 2026 Oral (2026)
Keywords: LLMs
RL
Long-form generation
Primary area: foundation or frontier models, including LLMs

Abstract: Ultra-long generation by large language models (LLMs) is a widely demanded scenario, yet it remains a significant challenge due to their maximum generation length limit and overall quality degradation as sequence length increases. Previous approaches, exemplified by LongWriter, typically rely on ''teaching'', which involves supervised fine-tuning (SFT) on synthetic long-form outputs. However, this strategy heavily depends on synthetic SFT data, which is difficult and costly to construct, often lacks coherence and consistency, and tends to be overly artificial and structurally monotonous. In this work, we propose an incentivization-based approach that, starting entirely from scratch and without relying on any annotated or synthetic data, leverages reinforcement learning (RL) to foster the emergence of ultra-long, high-quality text generation capabilities in LLMs. We perform RL training starting from a base model, similar to R1-Zero, guiding it to engage in reasoning that facilitates planning and refinement during the writing process. To support this, we employ specialized reward models that steer the LLM towards improved length control, writing quality, and structural formatting. Experimental evaluations show that our LongWriter-Zero model, trained from Qwen2.5-32B, consistently outperforms traditional SFT methods on long-form writing tasks, achieving state-of-the-art results across all metrics on WritingBench and Arena-Write, and even surpassing 100B+ models such as DeepSeek R1 and Qwen3-235B.

Published as a conference paper at ICLR 2026
LONGWRITER-ZERO MASTERING ULTRA-LONG TEXT GENERATION VIA REINFOR LEARNING
Yuhao Wu Yushi Bai Zhiqiang Hu', Roy Ka-Wei Lee Juanzi Lite2 "iinaapore University of Technology and Design, Singapore TTinnguaa University, Beijing, China
[wu mru_yuhao, zhigiang hu) @mymail..... sut................ edu. ag, bys22@mails. tss.sssiig tsinghua..nna.a...a edu.
roy poy_lee@sutd. edu. sdu lijuanzi a nghua edu.
"Equal contribution.
*Co-corresponding authors.
ABSTRACT
Ultra-long generation by large language models (LLMs) is widely demanded scenario, yet it remains a significant challenge due to their maximum generation length limit and overall quality degradation sequence length increases. Previous approaches, exemplified by LongWrite (Bai et al.) (025), typically involve super- vised fine-tuning (SFT) synthetic long-form outputs. However, this strategy heavily depends on synthetic SFT data, which is difficult and costly to construct, often lacks coherence and consistency, and tends be overly artificial and struc- turally monotonous. In this work, we propose an incentivization- based approach that, starting entirely from scratch and without relying on any annotated synthetic data, leverages reinforcement learning (RL) to foster the emergence of ultra-long long, high-quality text generation capabilities in LLMs. We perform RI. training starting from base model, similar to R1-Zee R1-Zero, guiding it engage in reasoning that facilitates planning and refinement during the writing process. To support this, we employ specialized reward models that steer the LLM towards improved length trol, writing quality, and structural formatting. Experimental evaluations show that our LongWie WriterZerr model, trained from Qwen2.5-3215 consistently outperforms traditional SFT methods on long-form writing tasks, achieving state-of-the-art results across all metries WritingBench and Arena- Write, and surpassing 100B+ models such as DeepSeek RI and Qwen3-235B.
INTRODUCTION
Ultra-long text generation that extends over thousands of words has become an increasingly crucial capability for large language model (LLM) in real-world applications such multi-section report writ- ing, narrative storytelling, legal drafting, and educational content
Supervised Fine-tuning
AI
creation Yao
2019; Schmidgall et al. 2025 Bai al. 20025]
Wu et al. 2025b Tu et al. 2025a Wu et al. 2025a). Despite recent
advances enabling LLMs to generate outputs exceeding 10,000
words, prior studies have shown that existing long-output LLMs often suffer from issues such as local incoherence, internal con- tradictions, repetitive phrasing, topic drift, and structural collapse
Reinforcement learning
DDG Aut
long-form outputs (Wu et 2025b).
2025c
Bai al. 2025
Wu et al.)
DUG
AI
Previous efforts have largely relied on extending long-form genera- tion through supervised fine-tuning over synthetic long-output datasets, that is. instruction-outee outtutt pairs constructed through expert-designed agentic pipelines (Bai al. 2025 Pham al.) 2024. Tu et al.] 2025b) Wu et al., [2025a] Quan et al. 2024). While SFT offers straightforward approach, comes with two critical
L
Ultar-Lon text generation
Figure SFT vs. RL long- form generation.

---

Published as a conference paper at ICLR 2026
limitations. First, the training data is constructed using existing
LLMs, which constrains diversity and innovation in writing styles and inherently caps quality at the level of the off-the-shelf models. Second, the maximum likelihood objective fails to provide explicit signals for optimizing global-level properties such as coherence or formatting consistency (Deng et al. 2022 [ham et al. 2024)..
To move beyond these limitations, we pioneer a novel framework for long-form generation: using reinforcement learning to activate long-form generation abilities in LLMs from scratch. Rather than fitting on fixed reference texts, RL enables models to optimize for long-ränge objectives through reward signals that capture desired output qualities, and without the need for manually curated SFT datasets (as illustrated in Figure DT This mirrors recent advances in reasoning centrrc domains such as math and coding, where models like DeepSeek-R1- R1-Zero (DeepSeek-A) et al. 2025a), RI--trained from a base model, have shown notable performance gains. We hypothesize that a similar RI.-driven approach can empower L.LMs to produce longer, more coherent, and more logically structured outputs aligned with input instructions.
Concretely, we investigate how to effectively train long-form generation policies via RL, addressing three key research questions:
RQI (Reward Design): How can reward models designed to best guide long-form generation?
RQ2 (Test-time Scaling); Large reasoning models (LRMs) show substantial gains from inference- time scaling, particularly through long Chain-of-Thought (CoT) (OpenAl, 2024bb DeepSeek-Al
et al. 2025a Yeo et al. 2025). However, prior LRM research focus predominantly in math and code tasks. Can similarly introducing a long CoT enhance RL-based long-form generation?
RQ3 (Impact of Continual Pretraining): Can continual pretraining on long-form materials and reasoning data further raise the performance ceiling of RL-trained models?
Through systematic ablation and controlled experiments, we find that all three components- design, test-time scaling, and continual pretraining- critical for maximizing the effectiveness of RL in long-form generation. At the same time, we find that RL training significantly outperforms SFT training long-form writing tasks. Additionally, RL can unlock higher potential in base models with stronger foundational capabilities. We integrate these insights into a final model, Long Writer- Zero (trained based on Qwen2.5-32B) which achieves state-of- of- results on long-form writing benchmarks such as WritingBench (Wu et al. 2025d) and Arena Write (Sec. 2.1
REINFORCEMENT LEARNING FOR ULTRA-LONG TEXT GENERATION
In this section, we present our reinforcement learning framework designed to unlock ultra-long-fore generation capabilities in LLMs. We begin by describing the overall RL setup, including the training algorithm and key components of our framework in Sec. 2.1] We then address the three core research questions outlined in Sec. [] respectively focusing on: Reward Design (Sec. 2.2 Test-time Scaling (Sec. 2.3
and Impact of Continual Pretraining (Sec.
2.1
RL SETUP
To train policies for ultra-long-form generation, adopt the Group Relative Policy Optimization (GRPO) algorithm (Shao et al. 20244 DeepSeek-AT et al. 2025a) for RL training. Below, we detail the core components of our RL setup.
Training Algorithm. GRPO extends Proximal Policy Optimization (PPO) (Schulman al. 2017)
by computing normalized advantages over group of sampled completions. For each training input it samples of candidate outputs {0r, (2,,02," .... . .G) 02,.... from the current policy New Each output is scored by reward model, and the advantage for the ith sample scored with r; is computed as:
A = r; -
rcl)
(I)
std([ri...., rra))
2

---

Published as a conference paper at ICLR 2026
GRPO then maximizes the clipped objective:
JARRO (0) P())) fa: aod
min patio A A, clip(r," - 16)+,
G
3 DKL. (To Teet)
where the importance sampling ratio rf clipping threshold and the KL penalty term, respectively. The reference policy Neaf is fixed to the initialization of Te.
ne (o, q) The hyperparameters e and B control the
ne aa [e)
Query Source. We sample training prompts from two large-scale, real-world instruction datasets: English instructions from WildChat-IM (Zhao et al. 2024) and Chinese instructions from I.MSYS- Chat-IM (Zheng et al. (2023). To ensure the quality and suitability of queries for long-form generation, we use QwQ-32B model (Team 2025) filter inputs, retaining only requests that demand high-
quality, extended outputs
Training Configuration. We conduct RL training based on the Qwen2.5-322 base model (Team) 2024) using Megatron's reinforcement learning framework on 8 nodes, each with 8 x H800 GPUs. Each optimization step samples 32 concurrent trajectories with a batch size of 32 training prompts. set the maximum output length to 14,000 tokens. During training, we adopt sampling with T = 0.8 and top-p = 1.0. We set = 0.2 and remove the KL penalty term by setting S = 0, according to the empirical suggestions in the DAPO algorithm (Yu et al. 2025).
To support long text generation,
Evaluation (Arena-We Write). To provide timely and comprehensive monitor of the model's long- form generation capabilities during training, manually curate Arena-Write, a lightweight bench- mark comprising 100 diverse real-world user writing instructions as test prompts, with 40% of the prompts demanding outputs exceeding 2,000 words. Following the automatic evaluation protocol of Hard (Li et al. 2024). which simulates the crowdsourced Chatbot Arena leaderboard, we conduct pairwise win-rate comparisons for each test prompt against responses from six strong baselines: DeepSeek-a R 1 (DeepSeck-AL) et al. 2025a), @wwen2.5-7BB (Tem 2024). 2PT-4--202- (GLM et al. (024).. Claude-3.5-
11-20 (OpenAl 2024a). GLM
(Anthropic (DeepSeek-Al et al., 2025a), yielding total of 600
2024), and ReeeSSee-R comparisons. We use Qwen2.5-72B as the automatic judge to label each comparison as "win", "tie", or "lose" compared to the baseline responses, with the judging prompt provided in Appendix Evaluation results are reported using Elo ratings (higher means better), computed based on win rates against the baseline responses.
2.2 RQ1: REWARD DESIGN
In domains such mathematical reasoning and code generation, reinforcement learning often relies on deterministic rule-based reward by comparing to the groundtruth answer (DeepSeek-Al et al. 2025a Team 2025). However, such approaches generation due to the inherent complexity, subjectivity, and multidimensional nature of evaluating long-form outputs. To address these challenges, we design composite reward function that integrates multiple reward models, each targeting a distinct aspect of writing quality.
generally infeasible for open-ended text
Length RM. To guide models toward producing responses of appropriate length- particularly for form writing task where outputs may be expected to span thousands of words and responses short of that are typically considered failure to meet task requirements- we design a length reward model that evaluates whether generated output is too short, overly verbose, or well-aligned with the expected length. Specifically, we employ QwQ-32B (Team) (2025) to predict the appropriate word count range for each query (details provided in Appendix(A.3) which serves as the supervisory signal. For example, a query requiring a 3, 3.000-word essay would correspond to target range of
Filtering method details are provided in Appendix A.2
Throughout our paper, DeepSeck-RI refers the version released 2025-03-27.
3

---

Published as a conference paper at ICLR 2026
Length
Writing RM
Length RM
200a
Mean
Continua--Pattral--tee
Centinua--rnttaaa--iie Base-nothink
Continual- Pratrain-tiinn
Bese-think
Bane-think-
Basn-think
Base nothink
Base- nethinethinkk
Step
Figure RL Training curves of three setups (Base-nothink, Base-think, and across three metrics: Writing RM (left), Length RM (middle), and Mean Non-Overlong Generation Length (right).
[2, 70, 3,300] words. During inference, outputs falling within this length range, i.c., [Loowers Lupper) receive higher rewards, while under- or over-length responses are penalized:
1,
if Liowrr S (en(o) S Luppors
len(o)
(en(o) < Loowers
Mength
=
Liower
(3)
Lmax len(o)
if len(o) Lupper-
Lmax Lupper
Writing RM. The writing reward model Twrite is trained on manually-labeled preference data (Nwr 91) over writing-related prompt xr It uses Owen2.5-72B (Team. 2024) as the backbone and optimized under the loss function (Rafailov et al. 2024 Hou et al. 2024). tollowing the Bradley-Terry model (Bradley Terry 1952) to model preferences:
(n..::)))) (owleeo(wwwweeeeee (w. - Twiit (x))))))]
The Writing RM aims capture holistic writing quality, including fluency, coherence, and helpful- n Through this training process, it learns to provide high-level reward signal that aligns model outputs with human expectations, encouraging natural, well-structured, and goal-relevant responses.
To enforce structural integrity and reduce redundancy, the Format RM checks whether the output conforms to a predefined structure: exactly one <think> segment followed by one <answer> segment. It also penalizes repetitive content based on semantic overlap metrics, which is especially important RL settings where models may easily reach the target length required by the Length RM by generating nearly identical paragraphs.
Format RM.
Final Reward. A naive wwward-averaging strategy may cause the overall reward to be dominated by a sub-reward (e.g.. length or format) with larger numeric scale. To prevent this imbalance. each reward component is first individually normalized within its group into the same [-1, 1] range before computing its advantage. We then compute the final reward signal through advantage-level averaging:
Armal (Atength + Awnite + Afoemat)
which is plugged into the final GRPO objective in Eq. This gormalizaiion-then-ggeee
scheme ensures that all reward components contribute on a comparable scale, preventing length or format from overwhelming writing quality and promoting more stable and balanced learning signal for long-form generation.
'While effective for guiding long-form writing, the RM does not attempt to cover all quality dimen- sions- particcaarll fine-grained factuality-which explicitly acknowledge as limitation (Appendix A.7). It designed practical, task-aware proxy for long-form quality rather than fully comprehensive reward.
4

---

Published as a conference paper at ICLR 2026
Using the RL setup described in Sec. 2.1 we train the model with the final composite reward. This training setting is denoted as Base-nothink. As shown by the green curve in Figure 2 both the Writing RM and Length RM scores improve steadily over RL steps, with particularly pronounced gains in writing quality. Furthermore, performance on Arena- Write improves continuously, with Elo scores increasing from 200 to over 600 (Figure Green). Greenn...nee.. These results demonstrate the effectiveness of our reward design in long-form generation.
Result.
2.3
RQ2: TEST-TIME SCALING
Recent advances in mathematical and programmatic reasoning, such as DeepSeek-RI (DeepSeck-A) al. 2025a) and OpenAI ol (OpenAl 2024b), have popularized a new scaling law dimension test-time scaling: prompting the model to "think" in a dedicated intermediate step before answering through long Chain-of-Thouugh (Wei al. 2022). This think step, typically wrapped in <think> and </think> tokens, allows the model to plan and reflect before committing to final response. These methods have achieved strong empirical results in reasoning- -havy domains like math and code. However, it remains unclear whether a similar test-time scaling effect generalizes to open- apen-ended tasks such as long-form writing. Writing is inherently subjective and multidimensional, requiring not only coherence and clarity but also tone, structure, creativity, and reader alignment. Whether these qualities benefit from explicit intermediate reasoning optimized through reinforcement learning an open question.
To investigate this, we compare two prompting strategies during RL training and inference, one explicitly demand the model engage in thinking and the other asks the model to directly output the response:
Think Prompt
A conversation between the and the assistant. The user provides writing/general task, and the assistant completes it. The assistant first deeply thinks through the writing/answering process in their mind before providing the final written work to the user. The assistant should engage in comprehensive and in-depth planning to ensure that every aspect of the writing/general task is detailed and well- structured. If there is any uncertainty ambiguity in the writing request, the assistants should reflect, ask themselves clarifying questions, and explore multiple writing approaches to ensure the final output meets the highest quality standards. Since writing is both a creative structured task, the assistant should analyze it from multiple perspectives, considering coherence, clarity, style, tone, audience, purpose, etc. Additionally, the assistant should review and refine the work to enhance its expressiveness. The writing thought process and the final written work should be enclosed within <think> and <answer> tags, respectively, as shown below:
<think> A comprehensive strategy for writing that encompasses detailed planning and structural design-including hrainstorming, outlining, style selection, audience adaptation, self-reflection, quality assurance, etc </thi
/ nk>
<answer> The final written work after thorough optimization and refinement < </anwwe>>
Direct-Answer (Base-nothink) Prompt
conversation between the user and the assistant. The user provides a writing/general task, and the as- sistant completes it. The assistant directly provides the final written work. The final written work should be enclosed within <answor> tags, as shown below: Sanswer>Fiind written work. < </answer>
Result. As shown by the yellow curve in Figure 2 models trained with Think Prompt (Base-think) initially exhibit lower Writing RM scores- close to zero in early RL compared to their Disser-Answer counterparts (Base-nothink). This early lag expected, as the model must first learn the structure and utility of producing <think> and <answer> segments driven by the Format RM. However, as RL progresses, the Base-think model steadily improves and ultimately surpasses the Base-nothink baseline, achieving higher ceiling in Writing RM scores.
We attribute this improvement to the reflective planning enabled by the <think> phase, which helps the model organize thoughts, segment content meaningfully, and allocate information across the output more effectively. Indeed, the model learns to expand this planning phase over time, with the 'think" token length steadily increasing throughout RL. training, dynamic detailed in Appendix
5

---

Published as a conference paper at ICLR 2026
This is further supported by consistently higher Length RM scores, indicating better control over output length through planning in long COT. Morcover, as illustrated Figure Base-think significantly outperforms Base-nothink the Arena-Write benchmark (1,200 700). Together, these results highlight the value of incorporating an explicit think step in RI for long-form writing, improving both the internal structure and overall output performance of ultra-long responses.
3.5
2.4
RQ3: IMPACT OF CONTINUAL. PRETRAINING
Prior work has emphasized that the performance ceiling of RL is often bounded by the capabil- ities of the underlying base model Yue al.) 2025; Li et al. 2025a Yeo et al., 2025). In this study, we wonder whether this observa- tion also holds for open-ended tasks like long- form writing, which demand skills such as stylis- tic control, narrative planning, and length con- trol. To investigate this, we continual pretrain the Qwen2.5-32B model on 30 billion tokens of high-quality, writing-centric data before RL. training. The corpus comprises diverse selec- tion of Chinese and English books, reports, and academic papers, covering a broad spectrum of genres and topics to strengthen the model's writ- ing competence. Additionally, we distill a small portion of long CoT data from the RL trained Base-think model (from Sec. 2.3) to enhance the model's reflective reasoning ability and fa- cilitate initial format alignment. This CoT data is incorporated into the pretraining corpus at minimal ratio 1% to avoid memorization of specific CoT patterns. The model is trained with batch size of 512, using packed sequences with maximum context length 32K tokens. A detailed breakdown of the pretraining data is provided in Appendix A.4 All data, except for Ours_think, sourced from Common Crawl. We apply the same GRPO training to the model after continual pretraining, deriving the Continual-Pretraine
1400
Score
1000
800
600
400
Continual-Peetralnntn
-hink
200
Base-nothhin
150
Step
Figure 3: Elo scores evaluated on Arena- Write during training for the three setups: Base-nothink, Base-think, and Continual-Pretrain--iee axis shows the Elo score, and the x-axis represents training steps.
The y-
model.
Result. As shown by the blue curve in Figure 2 the Continual Continual-Pretriin- think model shows higher initial scores in both the Writing RM and Length RM metrics in the beginning. This early advantage roots improved writing priors acquired during continual pretraining, as well as early format alignment fostered by the inclusion of distilled long CoT data. Beyond strong starting point, the model also achieves a higher overall reward ceiling. These gains are further reflected in the Arena- Write benchmark, where the model starts with Elo score of over 1000 and eventually reaches around 1400 at convergence, outperforming the Base-think model that does not involve a continual pretraining stage. This performance corresponds to nearly an 80% win rate against strong large reasoning models such as DeepSeek-Ro R11, underscoring the significant benefits of continual pretraining pushing the RL performance frontier for long-form generation tasks.
EVALUATION
3.1 EVALUATION SETUP
LongWriter-Zers Based on the insights from the three research questions, we establish a robust training pipeline for long-form generation. We begin with continual pretraining on long books and articles to enhance the model's general writing competence. Following this, we apply GRPO training with the Think prompt to explicitly encourage the model to engage in reasoning before generating responses. During this stage, the model is optimized with three reward models to better align with the multidimensional long-form writing preferences. Based on this training pipeline, we build our final model, Long Writer- -ero, by starting from Qwen2.5-32D and applying 30B tokens of continual pretraining followed by 1550 steps of RL. During the evaluation,
the prompt format in

---

Published as a conference paper at ICLR 2026
Languages
Roquirements
Models
ZII
EN
DI
D2
D3
DA
D5
D6
RI
R2
C
R3
Prppuizory IJMMs
GPT-to-2024-11 2024-11-2000 OpcnAl 2021
8.3
8.1
8.1
8.2
8.1
8.4
8.3
8.7
8.9
8.2
947
ol ol-Preview
P OpenAl 2104D
3.15
8.1
8.2
8.0
8.1
8.2
8.2
8.4
8.1
8.2
8.6
8.2
8,8
8.2
8.2
1080
Claude Sonter
2025
8.60
8.5
8.6
8.6
8.5
8.6
8.7
8.5
8.6
8.8
9.0
8.6
1185
Qwen2.-May
Open-sounce LLMs
DeepSeck-R DeepSeek-Al
20253
8.55
8.5
8.6
8.6
8.6
8.7
8.9
9.0
8.6
1343
DeepSeek-V DeepSekkAAte
Misral-I.argo manooommmellll Jam m0
4024
7.64
7.6
7.7
7.7
7.6
7.8
7.3
7.9
7.6
7.7
8.2
7.7
8.7
7.2
7.9
774
Qmen3-235BB-A22 A228 Yan2 e
2024
3.68
8.7
8.6
8.6
8.6
8.6
8.7
8.8
8.6
8.7
8.9
8.7
9.0
8.7
8.8
1343
Qwen-2.5-72B-une Enstrize
7.90
7.8
8.1
7.7
7.8
8.0
8.3
8.8
7.9
911
Quen-2.5-78-Insmme
7.6
6.9
7.5
7.9
7.4
Llamm-3.2-708- Instrie
Linney ama
2024
7.01
6.7
7.3
7.0
6.9
7.0
6.8
7.3
7.3
7.1
7.8
8.2
7.0
570
Lluma-3 3.1-888- Duioey
2024
6.35
5.7
6.9
6.6
6.4
6.1
6.0
6.7
6.6
6.4
7.0
6.4
7.6
6.3
6.4
445
Capahllit-enhaneed enhonced
Suri-L-ORPO (7Bm| clal. 2024
4.97
4.4
5.5
5.6
5.3
5.0
4.1
3.0
5.1
4.8
5.2
5.0
5.4
4.5
4.0
LongWitter-Se SR Bai 2024
7.91
7.9
7.9
8.0
8.1
8.1
7.7
8.1
7.6
7.9
8.2
8.1
8.8
7.7
7.7
457
Long LongWWiier-Ze
1447
w/o Continaal Pastrin
7.8
8.8
7.9
1221
w/o Thinking
8.04
8.2
7.9
8.2
8.1
7.6
8.4
8.0
8.0
8.3
8.1
8.6
7.9
7.0
668
Table WritingBench performance of different LLMs across six domains and three writing require- ments (scale: 1-) The domains are: (D1) Academic & Engineering, (D2) Finance & Business, (D3) Politics Law, (D4) Literature Art, (D5) Education, and (D6) Advertising & Marketing. Requirements include (R1) Style, (R2) Format, and (R3) Length. "C" denotes the category-specific score. Arena- Write Elo scores are shown in the final red box.
training (Think or No-Think), described in Sec. 2.3 For model with long CoT, evaluate only the final response wrapped between <answer> and </ /answer>.
Benchmark Setup. We evaluate model performance on three evaluation suites:
(1) WritingBench (Wu et al. 2025d) is a comprehensive benchmark for long-form writing, covering six major domains and 100 sub-domains across creative, persuasive, informational, and technical genres. It includes 1,200 real-world writing prompts, each paired with five query-specific evaluation criteria. WritingBench adopts a query-dependent evaluation framework and uses Qwen2.5-7B critic model fine-tuned on 50K human-annotate samples, achieving 83% agreement with human judgments across dimensions like style, format, and length.
(2) Arena- Write, introduced in Sec. Sec.2.1 has 100 instructions: outputs are judged against six baselines by pairwise win rates and summarized with Elo ratings.
(3) Human-in-the-loog Win-rate Evaluation. We compile set of 200 real-world user instructions and compare the win-rate of Long Writer-Zero against six leading models, including Qwen3 235B- A22B (Yang al., 2025), DeepSeek-V V3 (DeepSeek-Al et al. 2025b). DeepSeek-R2 (DeepSeek-A) al. 2025a), Llama-4-Scout (AI, 2025). Claude-Sonnet-4 (Anthropic) (0255) and Gemini-2.5 Pro- The win-rate is initially assessed using GPT-4.1 (OpenAl 2025) with the
0506 Mend 2025).
prompt in Appendix A.5 evaluations.
To further validate these automatic judgments, we also incorporate human
3.2 MAIN RESULT
We evaluate Long Writer-Zero on WritingBench and Arena-Write after continual pretraining and RL. As shown Table [1, LongWriter-Zee Zero achieves the highest overall critic score of 8.69, outperforming all models (c.g.. Qwen-Max: 8.37, GPT-40-2024-11-20. 8. 8.16). including the strongest open-source baseline DeepSeek-RI (8.55). Across different domains, Long Writer--eee obtains the best performance in five out of six Nomains-Academne Academic & Engineering (8.7), Finance & Business (8.8), Politics & Law (8.8), Education (8.9), and Advertising Marketing (8.6, tie)-while slightly lagging behind DeepSeck-RI (8.6) in Literature & Art (8.4). On writing requirement metrics, also achieves the highest scores Style (R1: 8.7, C: 8.9) and Format (R2: 8.7, C: 9.0), while maintaining competitive Length score (R3: 8.6). Meanwhile, Long Writer-Zero also significantly outperforms other models in Arena- Write, achieving an Elo rating 1447, followed by DeepSeck-R1 and Doven3-235B-AAe which are tied for second place with score of 1343.
7

---

Published as a conference paper at ICLR 2026
We also perform ablation study Pretraining. The "-Continual Pretrain" model refers to Base-think in Sec. 2.3] and further "-Thinking" corresponds Base-nothink in Sec. Sec.2.2 We observe that progressively removing these two techniques leads to a substantial performance drop on both WritingBench and Arena- Write. Among them, Thinking proves
two key strategies: Test-time Scaling and Continual
critical for Arena- Write (1221 > 668), while Continual P Pretraining plays more important role on WritingBench (8.69 > 8.1.). These ablation results confirm that integrating intermediate reasoning (think) and continual pretraining are crucial to LongWriter making it highly effective for long-form generation tasks.
performance,
3.3 SFT VS. RL
In this subsection, we compare the effectiveness of SFT and RL using the same base models: Qwen2.5-32B and our continual trained Qwen2.5-32B in Sec. 2.4 For SFT, utilize writing instruction data from ShareGPT combined with long-output samples from the LongWriter-6k dataset. Evaluations conducted on Arena-Write are presented in Figure 4 We observe that, over both base models, RL consistently outperforms SFT. The performance of SFT is constrained by the overall quality of its training data, whereas RL continuously improves long- signals. Notably, despite stronger initialization from continual pretraining, the SFT models show only marginal improvement, with scores increasing slightly from 964 (base) to 971 (continual pretrained). In contrast, the RL-based approach significantly benefits from the stronger, continually pretrained initialization, yielding substantial performance gains (1221 / 1447) and clearly surpassing both SFT variants. This suggests that stronger base models achieve greater performance improve- ments through RL, whereas the effectiveness of SFT remains constrained by the supervision data.
generation through reward
3.4 WIN-RATE RESULT
1200
While WritingBench provides comprehensive absolute-quality evaluation, relying on a single critic model may introduce model-family biases or score compression among strong systems. To obtain more robust and model-agnostic view, we additionally adopt a human-ing inttun-----e-oopp win- rate evaluation. During evaluation, each query is evaluated twice with swapped response orders mitigate positional bias, and results cate- gorized as win, loss, or tie. This pairwise win- rate metric complements WritingBench by cap- turing finer relative preferences between closely matched models. The win-rate results are shown in Figure
Score
1000
800
600
SFT Base
SFT Continual Pretrain
RL. -Bas
RL Continual Protrain
150
200
250
Step
Figure Arena-Write performance across RL training steps, comparing RL (solid) and SFT (dashed) starting from Base (orange) and Contin- ual Pretrain (blue) initializations.
LLM Evaluation. From the six donuts on the left of Figure 5 where results are automatically judged by GPT-4.1, Long -rngWriter-Zero consistently demonstrates substantial performance lead over six strong baseline models. The win rates for our model in these automatic evaluations reach as high as 98.2% and remain above 62% even against the strongest baselines. Despite having only 32B parameters, long-form generation capabilities rival those of much larger LLMs.
Human Evaluation. To mitigate potential biases in automatic evaluation, we also conduct supplementary human evaluation, shown in the right two donut charts Figure (3 against DeepSeck-Ri and evaluated each query according to the same "win/loss/tie" criteria as the automatic evaluation. While the annotators tended to assign ties in cases of subtle differences between responses -lightly lowering the overall win rate LiveeLLonge Writer-Zers still consistently demonstrates strong human preference, validating its reliability in real-world scenarios.
Three independent annotators with undergraduate degrees
At the end of our experiment, we provide case studies showcasing LongWriter-Zer long CoT and final responses in Appendix A.10
8

---

Published as a conference paper at ICLR 2026
1. Llama-4-Scout
DeepSeek-V3
DeepSenk-R1
Human judge: R1
85.5%
72.7%
62.4%
4. Claude-Sonnet-4
GemPnk-2.5-Pro
6. Qwen3-2358-A22B
Human judge: Qwen3
Our wins
Opponent wins
Tie
Figure 5: Win-rate results of Long Writer- Zero in human-in in--inn-tooooe win-rate evaluation. Left six charts: Outcomes judged by GPT-4.1 against six baselines 1llma----cour DeepSeck- V3, DeepSeek- RI, Claude- Sonnet-4, Gemini-2.5-Pru Qwen3-2355 Awen3-235B-A22D5 A22B). Right two charts: Outcomes judged by human annotators (comparing against DeepSeek-R R1 and Owen3-235B-A2201 The percentage the center indicates the overall win rate, with ties counted as 0.5 wins for each side.
3.5 THINK LENGTH DYNAMICS DURING RL TRAINING
Figure 6 tracks the evolution of the mean "think- ing" token length for Base-think and Continual- Pretrain-think during RL. In contrast math
problems-wee where chain-of ofain-offthought can be ex- panded (DeepSeek-Al
Think token Length
1000
al.) 2025a) almost without limit-writing exhibits natural satura- tion point: once the "thinking" budget is suffi- cient to reliably produce high-quality text, ad- ditional thinking (e.g., effectively pre-drafting entire paragraphs) yields little marginal gain while consuming valuable context- window ca- pacity, which ultimately degrade perfor- mance. Consequently, both models converge to task-optimal thinking length and then plateau, rather than increasing indefinitely. Notably. Seetinual-Pretrain--fe
800
600
600
200
Continua--hwtrain-hhy
180
150
200
250
Step
consistently maintains slightly longer thinking length than Base-think (i.e.. modest lead), but the gap remains small and does not reflect a qualitative difference in behavior.
Figure 6: Evolution of the "thinking" token length during RL training. The continually pretrained model consistently generates longer reasoning chain.
RELATED WORK
Long-form Text Generation. Recent advancements in long-form text generation have explored structured prompting, personalization. and ultra-long output capabilities. Early approaches such as Re3 and DOC (Yang et al.) 2022; (023) introduce recursive or hicrarchical strategies to preserve narrative coherence. Later work emphasizes personalization, including LongLaMP (Kumar al.) 2024) and reasoning-enhanced self-training (Salemi et al. 2025). which tailor outputs to user intent. With the increasing demand for ultra-long generation (beyond 2,000 words) (Wu et al. (0255),, new benchmarks and methods have emerged. Suri (Pham et al. 2024) constructs large-scale instruction- following dataset via back-translatee translationn, but its outputs are limited to under 5k tokens and heavily depend back-transian takktaaanlation. LongWriter (Bai et al.] (025) fine-tuner models agent-genent generated outputs ranging from 6k to 20k tokens using supervised and preference optimization, achieving desirable ultra-long outputs but inheriting bias from teacher models. Overall, despite progress, current

---

Published as a conference paper at ICLR 2026
ultra-long generation techniques are still constrained by synthetic guidance, limiting their ability to ensure coherence, factuality, and generalization over extended contexts.
RL Scaling for LLMs. Recent advancements LLMs have increasingly leveraged reinforcement
learning to enhance reasoning capabilities, particularly in complex tasks such as mathematics, coding, and logical inference. Notable models in this domain include ol OOenAAl 2024b). DeepSeek- RI (DeepSeek-Al et al. (2255)) QwQ-32B (Team 2025), and Kimi k1.5 TTam et al. 2025), each demonstrating the efficacy RL in scaling LLM Performance on reasoning tasks. DeepSeek-RI-
7ero (DeepSeek-Al et al., 2025a) distinguishes itself by exclusively utilizing RI,, foregoing any supervised learning, and relies on GPRO to enhance reasoning without the need for labeled data. Despite having fewer parameters, QwQ-32B (Team] 2025) achieves performance comparable to larger models like DeepSeek-RI by integrating RL strategies that enhance the model's ability to reason through complex tasks. Kimi k1.5 (Team al. 2025) is RL, emphasizing long-contex, understanding and improved policy optimization methods. While these models have successfully scaled RL enhance reasoning, they primarily focus on short-form tasks and often combine RL with rule-based reward functions. There remains a gap in exploring the exclusive of RL for scaling LLMs in long-form generation tasks. To address this gap, propose novel framework that employs RL exclusively to scale LLMs for long-form generation tasks. and successfully LongWriter- LongWriter-Z4 Writer-Zero demonstrates the potential of pure RL approaches in enhancing the coherence, relevance, and overall quality of long-form generation tasks.
LLM trained with
with it. By eliminating reliance on supervised learning.
CONCLUSION
This work presents the first attempt to apply RL to ultra-Iong text generation without relying on synthetic annotated datasets. By leveraging composite reward models targeting length control, writing quality, and formatting consistency, our method addresses core challenges in long-form generation such length following, coherence degradation, and structural drift. Experiments WritingBench, Arena-Write, Write, and human evaluations show that Long Writer-Zeru trained with our approach. significantly outperforms both SFT basclines and leading reasoning-baie key insights emerge: (1) tailored reward design is essential for guiding long-form generation; (2) incorporating explicit reasoning steps via the Think Prompt during RL enhances planning and coherence; and (3) continual pretraining substantially raises RL performance ceilings. In summary, Long Zriter-Zero establishes strong RL-only paradigm for long-form generation, offering new pathways for scalable and coherent ultra-long text production.
models. Three
10

---

Published as a conference paper at ICLR 2026
ACKNOW
The research is supported by the National Research Foundation, Singapore under its National Large Language Models Funding Initiative (AISG Award No: AJSG-NMLP-2004-0055,, and by AI Singapore under its AI Governance Research Grant (AISG3-GV-2023-000)) This work is also supported by the National Natural Science Foundation of China (Grant No. 62476150) and the Beijing Natural Science Foundation (Grant No. L243006). Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not reflect the views of the National Research Foundation and AI Singapore.
ETHICS STATEMENT
We are committed to upholding the principles outlined the ICLR Code of Ethics. Our research incorporates a dataset annotated by three professional human annotators. All annotators involved in this project were provided with clear guidelines and gave their informed consent prior to participation. We ensured they were compensated fairly for their labor and time, in line with ethical standards for human-participant research.
The data presented to the annotators carefully processed remove any personally identifiable information (PID), thereby protecting the privacy of both the annotators and any individuals potentially represented in the data. Furthermore, we have conducted thorough assessment of the potential broader impacts of our work. This includes evaluating risks related to the amplification of social biases, the generation of toxic or harmful content, and dual-use applications. Our analysis indicates that our methodology and findings do not introduce or exacerbate these potential harms. The authors declare no competing interests.
REPRODUCIBILITY STATEMENT
We are committed to ensuring the reproducibility of our work. All datasets generated and used for continual pretraining, reinforcement learning, and evaluation are detailed in the Appendix and will be provided in the supplementary materials. Upon publication, we plan to release the source code for our training framework, the reward models, and the final LongWriter- public. Detailed descriptions of our experimental setup and hyperparameters are provided throughout the paper and in the Appendix to allow for the full replication of our results.
model weights the
REFERENCES
Meta AI.
The llama 4 herd: The beginning of new era of natively mul- timodal innovation, April 2025.
URL httpp://aa wwt..eta..como com/blog/
1lama mu = mmulzimoon wwwoodal-nnneellii /
Anthropic, Anthropic: Introducing claude 3.5 sonnet, 2024. URL http:tt//w...tttrrrr.
com/news/ clw.dllaeeeeseee 5-onnett
Anthropic. Anthropic: Introducing claude 4, 2025. URL https::/ww.anntrrppl..ooo
news/cl clluee-4
Yushi Bai, Jiahao Ying, Yixin Cao, Xin Lv, Yuze He, Xiaozhi Wang, Jifan Yu, Kaisheng Zeng, Yijia Xiao, Haozhe Lyu, et al. Benchmarking foundation models with language-ma mo----aa-----xaainnne Advances in Neural Informaionn Processing Systems, 36, 2024.
Yushi Bai, Jiajie Zhang, Xin Lv, Linzhi Zheng, Siqi Zhu, Lei Hou, Yuxiao Dong, Jie Tang, and Juanzi Li. Longwriter: Unleashing 10,000+ word generation from long context LLMs. In The Thirteenth International Conference on Learning Representations, 2025. URL https
/oopenreview. .nt/ wot.://orommmda-00000a0000
Ralph Allan Bradley and Milton E Terry. Rank analysis of incomplete block designs: I. the method of paired comparisons. Biometrika, 39(3/4):324-345 1952.
11

---

Published as a conference paper at ICLR 2026
Google DeepMind. Gemini 2.5 pro, 2025. URL http:::/ssorrrre.googleeeea////e
deepmindd-mddien aeaddmeeia//aa www.g//miiiiie /eeiiiiigggiiiiiiieiiiiiiiiiiiin v2 5 report.pdf
DeepSeck-AI, Daya Guo, Dejian Yang. Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang. Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei Feng, Chengda Lu, Chenggang Zhao, Chengqi Deng. Chenyu Zhang, Chong Ruan, Damai Dai, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, Fuli Luo, Guangbo Hao, Guanting Chen, Guowei Li, H. Zhang, Han Bao. Hanwei Xu, Haocheng Wang, Honghui Ding. Huajian Xin, Huazuo Gao, Hui Qu, Hui Li, Jianzhong Guo, et al. Deepseek-rl: Incentivizing reasoning capability in Ilms via reinforcement learning, 2025a. URL htRp:tts::// /tttpps:/aaxiv.orz/ abs 0bo/25010 2201 12948
DeepSeek-AI, Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Daya Guo, Dejian Yang, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, Fuli Luo, Guangho Hao, Guanting Chen, Guowei Li, et al. Deepseek-v3 technical report, 2025b. URL
org /bbs/
2412. 1 19437
Yuntian Deng, Volodymyr Kuleshov, and Alexander M. Rush. Model criticism for long-form text generation, 2022.
wttg/bs/ .8444
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783 2024.
Team GLM, Aohan Zeng Bin Xu, Bowen Wang, Chenhui Zhang, Da Yin, Diego Rojas, Guanyu Feng. Hanlin Zhao, Hanyu Lai, Hao Yu, Hongning Wang. Jiadai Sun, Jiajie Zhang, Jiale Cheng, Jiayi Gui, Jie Tang, Jing Zhang, Juanzi Li, Lei Zhao, Lindong Wu, Lucen Zhong, Mingdao Liu, Minlie Huang, Peng Zhang, Qinkai Zheng, Rui Lu, Shuaiqi Duan, Shudan Zhang. Shulin Cao, Shuxun Yang. Weng Lam Tam, Wenyi Zhao, Xiao Liu, Xiao Xia, Xiaohan Zhang, Xiaotao Gu, Xin Lv, Xinghan Liu, Xinyi Liu, Xinyue Yang, Xixuan Song, Xunkai Zhang, Yifan An, Yifan Xu, Yilin Niu, Yuantao Yang, Yueyan Li, Yushi Bai, Yuxiao Dong. Zehan Qi, Zhaoyu Wang, Zhen Yang, Zhengxiao Du Zhenyu Hou, and Zihan Wang. Chatglm: family of large language models from glm-1300 to glm-4 all tools. arXiv preprint arXiv:2406. 12793, 2024.
Zhenyu Hou, Yúi Niu, Zhengxiao Du, Xiaohan Zhang, Xiao Liu, Aohan Zeng. Qinkai Zheng, Minlie Huang, Hongning Wang. Jie Tang, et al. Chatglm-rlhf: Practices of aligning large language models with human feedback. arXiv preprint arXiv:2404. 00934, 2024.
Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. Mistral 7b. arXiv preprint 10.0682, 2023.
Ishita Kumar, Snigdha Viswanathan, Sushrita Yerra, Alireza Salemi, Ryan A. Rossi, Franck Der- noncourt, Hanieh Deilamsalehy, Xiang Chen, Ruiyi Zhang, Shubham Agarwal, Nedim Lipka, Chien Van Nguyen, Thien Huu Nguyen, and Hamed Zamani. Longlamp: A benchmark for person- alized long-form text generation, 2024. URL http:::////// arxiv. org. wwwp//////////ooo.///./..//..aea 2407.11016
Dacheng Li, Shiyi Cao, Tyler Griggs, Shu Liu, Xiangxi Mo, Eric Tang, Sumanth Hegde, Kourosh Hakhamaneshi, Shishir G Patil, Matei Zaharia, et al. Llms can easily learn to reason from demonstrations structure, not content, is what matters! arXiv preprint arXiv:2502.07374 2025a.
Dawei Li, Renliang Sun, Yue Huang, Ming Zhong, Bohan Jiang. Jiawei Han, Xiangliang Zhang, Wei Wang, and Huan Liu. Preference leakage: A contamination problem in llm-as-a-judge, 2025b. URL /tattss///rxiv...
abs/ 2502 01534
Tianle Li. Wei-Lin Chiang, Evan Frick, Lisa Dunlap, Tianhao Wu, Banghua Zhu, Joseph E Gonzalez, and Ion Stoica. From crowdsourced data high-quality benchmarks: hrena-hard and benchbuilder pipeline. arXiv preprint arXiv:2406.1 1939, 2024.
OpenAl. Openai:
Hello
go,- 2024a.
URL
hel To-gpt-40/
12

---

Published as a conference paper at ICLR 2026
OpenAL. Learning reason with 1lms, 2024b.
URL http:://ppenaa.cmmm/nnee
OpenAL. Introducing gpt-4.1 in the api, April 2025. URL https://oopeaaaccom///mm
gpt
Chau Minh Pham, Simeng Sun, and Mohit lyyer. Suri: Multi-constraint instruction following for long-form text generation. arXiv preprint (XXX::2400619371 2024.
Shanghaoran Quan, Tianyi Tang. Bowen Yu, An Yang, Dayiheng Liu, Bofei Gao, Jianhong Tu, Yichang Zhang, Jingren Zhou, and Junyang Lin. Language models can self-lengthen to generate long texts. arXiv preprint urXiv:2410.2399 2024.
Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly reward model. Advances in Neural Information Processing Systems, 36, 2024.
Alireza Salemi, Cheng Li, Mingyang Zhang, Qiaozhu Mei, Weize Kong, Tao Chen, Zhuowan Li, Michael Bendersky, and Hamed Zamani. Reasoning-enhanced self-training for long-form personalized text generation, 2025. httL:ttts////xxi..ororg
04167
Samuel Schmidgall, Yusheng Su, 7e Wang, Ximeng Sun, Jialian Wu, Xiaodong Yu, Jiang Liu, Zicheng Liu, and Emad Barsoum. Agent laboratory: Using Ilm agents as research assistants, 2025.
URL /attps://arxiia org/ absaabs/////// 200./2511.000 04227
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms, 2017. WRLhttps::// http://///axii...r/////////111
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi. Haowei Zhang. Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. Deepseckmath: Pushing the limits of mathematical reasoning in open language models, 2024. FRRLhttps: http//xxvvvv.rr/.//.. orgg /bs/
2402. 03300
Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. Kimi kl. 5: Scaling reinforcement learning with llms. arXiv preprint 2..iv::2001.25999 2025.
Qwen Team. Qwen2.5: A party of foundation models, September 2024. URL httt:ttts///ween..
Qwen Team. Qwq-32b: Embracing the power of reinforcement learning, March 2025. URL Settps:/qwwe.tee git.ub. cotaub.io/bloogg wo/blogg/wggg 32bb
Shangqing Tu, Yucheng Wang, Daniel Zhang-Li, Yushi Bai, Jifan Yu, Yuhao Wu, Lei Hou, Huiqin Liu, Zhiyuan Liu, Bin Xu, and Juanzi Li. Longwriter-v: Enabling ultra-long and sideeiiiyltty generation in vision-language models,
org. abs 2202. 118.0..14834 14834
Shangqing Tu, Yucheng Wang, Daniel Zhang-Li, Yushi Bai, Jifan Yu, Yuhao Wu, Lei Hou, Huiqin Liu, Zhiyuan Liu, Bin Xu, and Juanzi Li. Longwriter-v: Enabling ultra-long and high-fidelity generation in lasion-language models, 2025b. URL httphhttpt///aatteee
org/ abs 2 502 14834
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 0122824-248833 2022.
Yuhao Wu, Yushi Bai, Zhiqiang Hu, Juanzi Li, and Roy Ka- Wei Lee. Superwriter: defvecto forg-fommaao generation with large language models, 2025a. URL
2506. 04180
Yuhao Wu, Yushi Bai, Zhiqing Hu, Shangqing Tu, Ming Shan Hee, Juanzi Li, and Roy Ka- Wei Lee. Shifting long-conter Ilms research from input to output, 2025b. URL https: // arxiv.org/ abs/ 2503. 04723
13

---

Published as a conference paper at ICLR 2026
Yuhao Wu, Ming Shan Hee, Zhiqiang Hu, and Roy Ka-Wei Lee. Longgenbench: Benchmarking long- form generation long context LLMs. In The Thirteenth Internationaal Conference on Learning Representations, 2025c. URL httpttt://pppeeeeaiiiiiee
Yuning Wu, Jiahao Mei, Ming Yan, Chenliang Li, Shaopeng Lai, Yuran Ren, Zijia Wang, Ji Zhang, Mengyue Wu, Qin Jin, and Fei Huang. Writingbench: A comprehensive benchmark for generative writing, 2025d. https:: wttpxxxivvv..m///a/ org/ abcgg///b///0
05244
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint arXiv:2505.09388, 2025.
Kevin Yang, Yuandong Tian, Nanyun Peng, and Dan Klein. Re3: Generating longer stories with recursive reprompting and revision. In Proc. of EMNLP, pp. 4393-4479, 2022.
Kevin Yang, Dan Klein, Nanyun Peng, and Yuandong Tian. DOC: Improving long story coherence with detailed outline control. In Proc. of ACL. pp. 3378-3465, 2023.
Lili Yao, Nanyun Peng. Ralph Weischedel, Kevin Knight, Dongyan Zhao, and Rui Yan. Plan-and- write: Towards better automatic storytelling, 2019. URL https://arxiv....oot////
05701
Edward Yeo, Yuxuan Tong, Morry Niu, Graham Neubig, and Xiang Yue. Demystifying long chain-of tf-thught reasoning llms. arXiv preprint drXiv:2502.03373, 2025.
Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, et al. Dapo: An open-source Ilm reinforcement learning system at scale. arXiv preprint arXiv:2503. 14476, 2025.
Yang Yue, Zhiqi Chen, Rui Lu, Andrew Zhao, Zhaokai Wang, Yang Yue, Shiji Song, and Gao Huang.
Does reinforcement learning really incentivize reasoning capacity in Ilms beyond the base model?, 2025.
arxiv http://awwwv..or////////00/.....
Wenting Zhao, Xiang Ren, Jack Hessel, Claire Cardie, Yejin Choi, and Yuntian Deng. Wildchat: 1m chatgpt interaction logs in the wild. arXiv preprint arXiv:2405.01470, 2024.
Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Tianle Li, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zhuohan Li, Zi Lin, Eric. Xing, Joseph E. Gonzalez, Ion Stoica, and Hao Zhang. Lmsys-chat- lm: A large-scale real-world Ilm conversation dataset, 2023.
A APPENDIX
A.1 USE OF LARGE LANGUAGE MODELS (LLMS)
Large Language Models (LL.Ms) were used exclusively for language polishing and improving the clarity of the manuscript. They did not contribute to research ideation, methodological design, experimental exccution, data analysis, or any other substantive aspects of this work. All scientific content, results, and conclusions are the sole responsibility of the authors.
A.2 WRITING- -AAS SELECTION AND LENGTH-RA RANGE PREDICTION WITH QwQ-32B
We reformulate the pipeline to (i) decide whether a query requests original writing, and, if so, (ii) predict a suitable [min, max] word-count range in one shot. The few-shot prompt below "Prompt-WL")) is fed to the QwQ-32B model.
Writing Task Selection
Task Goal
Given a user query,
Decide it asks for original written content.
14

---

Published as a conference paper at ICLR 2026
If NotWritinn, stop.
3. If Writing, output a reasonable word-count range "[lower, upper]" (ignore =00%).
Response Format
If not writing respond exactly: NotWriting.
If writiing respond with only the code block ("range": [lower, upper])
Heuristics for Range Estimation
1. Depth & Complexity:
analysis higher upper bound.
Scope: multiple sub-topics/seecioos + longer.
3. Requested Form: tweets/notes (0-300); short blog/letter (300-800); school essay (800-1
200); report/article (1200-2500) thesis/prooosal/busines plan (4000-10000).
4. Explicit Length Clues: honour any word/page requirement if stated.
Few-Shot Examples
Example
Query: Write Weibo post titled "Tips for Preparing for College Final Exams."
Answer: ("range": [0, 300])
Example 2
Query: Translate "Seize the day" into Spanish.
Answer: NotWriting
Example 3
Query: Draft comprehensive 10-page business plan for new cat-litter product.
Answer: { I"range": [4000, 6000]}
Query: User Query
Answer:
A.3 LENGTH RM PROMPT
Appendix: Query Length Assessment Prompt
You a professional query text-length assessor. Based the type of the query content, you should:
Deeply understand the core requirement of the query (e.g., essay, blog post, summary, outline, thesis section, etc.). For example, the query "How do start writing my thesis from scratch" asks for guidance on "how to begin writing a thesis," so you would estimate word-count range of [400, 800], rather than the total words needed to complete the entire thesis [7000, 10000].
2. Choose a lower bound that is multiple of 100, with a minimum of O.
Choose an upper bound that multiple of 100, with maximum of 12,000. the reasonable range certainly exceeds these limits, output: ("range": [0, 0])
Ignore the 10% of extreme length cases to keep the range reasonable for most scenarios, and ensure the difference between upper and lower bounds does not exceed 3,000.
5. If the query contains an explicit word-count requirement, set the range to +10% of that number. For "write a 2,000-word essay." output: ("range": [1800, 2200]] For "no more than 2,000 words," output [1800, 2000]; for "at least 2,000 words," output [2000, 2200].
If the query cannot be fulfilled under the given conditions for example, "Read and analyze this paper" without providing the paper, or "Analyze project's 's prospects" without specifying the project details-then output: ("range": [0, 0)) Example:
Input "Write a high school essay" + 'raange": [800, 1000]]
Input "Complete academic paper green cities" ("range": [6000, 10000] Please process the current query: User Query
Analyze and output the JSON range accordingly.
A.4 CONTINUAL. PRETRAIN DATA
The corpora listed in Table 2 were used for continual pre-training. All sources were de-duplicated, nanguage-filttreed and truncated maximum sequence length of 32,76 766 tokens before up-sampling.
15

---

Published as a conference paper at ICLR 2026
Table 2: Continual pretraining data distribution.
Data
Percentage (%)
Description
Long sh_ficttion Long_en
40%
Chinese novels
30%
English fiction and non-fiction
Long zh zilzzzhnonho nonfiction
15%
Chinese non-fiction books
inline_information
8%
Web novels, posts, and blog articles
5%
Industry reports
Long_essap
1%
Academic papers
Ours _hink
1%
Long CoT samples generated by Base-think model
A.5 EVALUATION PROMPT FOR WIN-RATE JUDGMENT
WritingBench adopts a critic model to evaluate model outputs by assigning scores (ranging from to 10) across 4-5 distinet dimensions. However, this evaluation approach has several limitations. First, due to the relatively small size of the critic model, it may be vulnerable to some word/sentence hacking. Second, we observe that WritingBench primarily focuses on formal or professional writing analysis of real-world data from WildChat (Zhao et al. 2024) and AMSYS-Chat-s (Zheng et al. (023) reveals that creative writing (e.g., storytelling. fiction) constitutes significant portion of user queries.
tasks such summaries and reports weprres-whereas
To address these concerns, we adopt a more direct and interpretable evaluation metric: win rate. We evaluate model performance nearly 200 real-word world user queries collected from the aforementioned datasets. For each query, responses are generated by LongWriter-Zem and six baseline models. These responses then compared in pairwise manner using GPT-4.1 10441-122250--144 ( 2025)) To mitigate positional bias, we conduct two evaluations per pair by swapping the response order: wal_uuaiin__oroo
A and Evaluat iin__inonoopppp judgments, we categorize the results into win, loss, or tie.
B A. Based on the two
SYSTEM PROMPT for Win-Rate Evaluation in Arena-Write
Please act as impartial judge and evaluate the quality of the written responses provided by two Al assistants the user's writing prompt below. You will be given Assistant A's response and Assistant response. Your job determine which assistant's 'ss writing is superior. Evaluate them on the following criteria:
1. Relevance and Completeness: Does the assistant fully respond to the writing prompt? Does the length meet the user's qucry expectations? Is the content relevant to the topic, and does it provide sufficient depth, length, and detail, rather than drifting off-topic simplistic?
Writing Quality: Evaluate whether the assistant's writing is clear, fluent, and free of obvious grammatical emors. The overall quality of the writing is high, with elegant.
3. Creativity and Originality: If applicable, assess the creativity of the response, Does the assistant offer fresh perspectives, unique insights, demonstrate certain level of originality?
Specificity and Detail: Determine whether assistant provides concrete examples or detailed explanations. Properly justified repetition is permissible.
5. Tone and Style: Is the tone appropriate for the writing prompt? Is the writing style consistent throughout? Consider whether it aligns with the expectations of the intended audience writing purpose.
After evaluating each response, determine which one superior based factors above. Provide your explanation and then select one of the following final verdicts:
Assistant is significantly better: [[>>]]]
Assistant A is slightly better: [A>BB]]
Tie. relatively same: [A=B]]
Assistant B is slightly better: [[>>A]]
Assistant is significantly better: [ [B>A]]
'Due to the issue of preference leakage, where models tend to favor others from the same developer, we
ensure that all baseline models come from companies different from that of the judge model (Bai et al. 2024 Di
2025b
16

---

Published as a conference paper at ICLR 2026
Example output: My final verdict is tie: [[A=B]] ].
A.6 LIMITATION
While LongWriter-Zero demonstrates strong long-form generation ability, is not immune reward- model exploitation. Heuristic rewards-particularly those involving length and style-naturally introduce opportunities for reward hacking, and several concrete failure modes were observed during training.
(1) Stereotypical or templated stylistic openings. The policy occasionally adopted highly formulaic openings (e.g., "Early morning...) because such patterns reliably yield favorable scores under the Writing RM. These stylistic shortcuts increase perceived coherence without providing genuine narrative substance.
(2) Pattern-level and near-duplicate repetition. Despite sentence-overlap and n-gram penalties, the model sometimes inserted slightly altered repeated passages paraphrased or lolen-eevelmoddifeed redundancies are harder to detect and represent a common form of reward hacking in long-output RL training.
to satisfy length requirements. Such
(3) Keyword-driven reward inflation. As with many preference-trained reward models, the Writing RM exhibits mild biases toward certain sophisticated-soumee -ooenddddd-sodiggggggo terms. The policy occasionally exploited this by inserting high-value keywords even when semantically unnecessary, producing text that appears more "expert-like" without corresponding content depth.
These behaviors highlight a broader limitation of model-based RL: policies can exploit superficial correlations in the reward models rather than fully aligning with human intent. In our work, introduced stronger mitigation mechanisns-succh as format-level structural checks and a multi-stage iopetiiionddetection module that captures pattern-eveel redundancy-whind substantially reduce the most common hacks. However, these measures do not eliminate the issue entirely.
Future directions include developing more discourse-aware and factuality-sensitin reward models, incorporating adversarial or uncertainty- objectives to resist keyword and repetition exploitation, and maintaining periodic numan-in-the-loee Overall, we view LongWriter-ZZe -ero as step toward understanding how length-awar rewards behave in practice, rather than a complete solution RM hacking.
auditing detect emerging reward-hacking strategies.
A.7 ADDITIONAL RESULTS ON SMALLER MODEL SCALE: LONGWRITER-ZERO
To examine whether our reinforcemeen-leerming paradigm generalizes across different model scales, we additionally trained a smaller Qwen2.5-14B model using the full LongWriter-222 pipeline (including task-aware length RMM. Writing RM, format-level constraints, and repetition-contry mechanisms). The results show that the paradigm remains effective even significantly smaller scale, achieving consistent improvements over the SFT baseline across both WritingBench and Arena evaluations.
Model
WritingRench f ArenaWriter EL.O t
Table 3: Qwen2.5-14B-Innse LomgWritcr-Zero-114
7.60
744
8.49 (+0.89)
1198 (+454)
achieves strong gains at smaller scale. Even at 14B parameters, RL framework produces substantial improvements in both absolute critic scores (WritingBench) and pairwise preference evaluations (ArenaWriter ELO).
A.8 ADDITIONAL BASELINE: LONGWRITER-ON-32 SFT
To provide a clearer comparison with synthetic-date SFT pipelines, we additionally include an SFT baseline obtained by training the Qwen2.5-32B model using the publicly released synthetic data from the LongWriter paper (Bai et al. 2025). This configuration can be viewed as a 32B-scale version of baseline.
the "Loongriterr" pipeline, and thus serves natural F.oongWiter-on--e
17

---

Published as a conference paper at ICLR 2026
The results below correspond to the experimental setting used in Section 3.3 (SFT RL). As shown in Table 4 the synthetic-data SFT baseline performs noticeably worse than our RL-based ZERO-32B, SEEWWWRRRRRRRR-2222 reinforcing the effectiveness of our proposed RL paradigm over standard synthetic-data SFT approaches.
LONGWRITER
Table LongWriter-on-333 SFT baseline. We evaluate Qwen2.5-32B trained with Long Writer synthetic data against our LONG - ZERO-32B. Our RL pipeline achieves substantial improve- ments on both WritingBench and Kriterriter EL.O.
Model
WritingBench 1
ArenaWriter ELO
Qwen2.5-32B (LongWriter-SFT)
8.08
964
LONGWRITER- -ERO-332
8.69 (+0.61)
1447 (+483
A.9 ADDITIONAL EVALUATION ON LONGBENCH-WRITE
For completeness and comparability with Writer (Bai et al. 2025). we evaluate LongWriter Zero-32B on the LongBench- Write benchmark. Results are shown in Table 5 LongWriter-Zeno-922 achieves the strongest performance across all three metrics, indicating that the gains observed on WritingBench transfer to the independent LongBench- Write evaluation.
Model
S
S
Claude 3.5 Sonnet
80.7
73.7
87.7
GPT-4 Turbo
67.3
47.9
86.6
GLM4-99-Chht
68.3
51.0
85.5
1oama-31--8BIIssmee
60.3
50.0
70.6
1lama-3.1-70B-Inttma
65.6
50.8
80.3
Mistral-Large-Instruce
77.0
65.6
88.3
Suri-I-ORPO
56.6
59.6
53.5
LongWriter-BB
79.8
77.4
82.2
LongWriter-99
80.5
78.6
82.3
LongWriter-9B-DPO
84.0
82.6
85.4
- (ours)
86.3
84.2
88.4
Table 5: LongBench- Write results. Baseline scores follow Bai et al. (2025). LongWriter-Zero-31
obtains the best overall quality (S), long-horizon structure (St), and content quality (S₂))
A. CASE STUDY
We present three case studies comparing our model with Gemini-2.5-Pe DeepMind 2025). DeepSeek RI (DeepSeek-Al al.) (225a),, and Qwe23-235B-A22B (Yang al. (025).
Query
Write a 400 word extremely scary ghost story in the first person, with clear logic and engaging content.
LongWriter-Zen Think
The
400-word first-person ghost story that's extremely scary, with clear logic and engaging content. Let me start by breaking "extremely seary," so I need to focus on creating a visceral, atmospheric sense terror. First perst- perspective is essential for immediacy and personal connection. Clear logic means the plot needs structured progression, avoiding leaps that might confuse the reader. Engaging content requires strong character development and compelling narrative arc. First, I'll brainstorm settings. haunted house a classic, maybe something more specific. A childhood home that's been abandoned could work work-nose nostaliaaa mixed with
their request. They specified
18

---

Published as a conference paper at ICLR 2026
fear. The protagonist returning after a long absence adds tension. The ghost should have backstory that ties into the setting, perhaps a tragic event. Maybe the protagonist's 's mother died there, and the ghost manifestation of her lingering presence.
Next, the structure. The story should start with the protagonist's i'' return, setting the mood. Introduce the ghost gradually, building dread. Include sensory details like creaking floor- boards, cold spots, or strange noises. The climax could involve confrontation, where the ghost's true nature is revealed. The ending should leave the reader unsettled, maybe with an unresolved threat.
Style and tone are crucial. First-person allows for internal monologue, which can convey fear and uncertainty. The language should be deseriptive but not overly flowery. Use short, sharp sentences to mimic the protagonist's racing heart. Sensory details are key: sight, sound, touch, smell, taste. For example, the musty smell of the house, the chill that seeps into skin, the sound a child's laughter in an empty room.
The audience is likely readers who enjoy horror stories, so the story needs to be relatable but with a twist. Maybe the ghost isn't traditional specter but something more symbolic. The purpose is to scare, the ending should linger, perhaps with final ominous detail that suggests the threat continues.
Possible plot points: The protagonist returns to the house after a decade. They find in disrepair, triggering memories. They hear a child's laughter, which grows louder. They discover room with rocking chair, and the ghost appears as shadow. The ghost reveals itself to be the protagonist's mother, who died giving birth to them. The mother's spirit is trapped in the house, unable rest. The ending could be the protagonist fleeing, but the ghost's 's laughter follows them, suggesting it's never truly gone.
need to ensure the logic is clear. The protagonist's return is motivated by a family heirloom, adding reason for the visit. The ghost's presence is tied the mother's 's tragic death, giving backstory. The climax should show the ghost's desperation, making more sympathetic yet terrifying. The ending leaves the reader questioning whether the protagonist truly escaped.
Potential pitfalls: Making the ghost too cliché. To avoid this, I'll give it a unique trait, like the rocking chair, which becomes symbol of its confinement. Also, the protagonist's emotional journey- fear, nostalgia, guilt can add depth to the story.
Now, structuring the story into four parts: introduction, rising tension, climax, resolution. Each part should be concise but impactful. Use dialogue sparingly, perhaps a haunting whisper child's voice. The final 400 words need to be tight, so each sentence must a purpose.
I'll also consider the reader's emotional are. Start with curiosity, build to fear. then to horror, and finally to dread. The ending should leave the reader unsettled, perhaps with lingering image sound.
Finally, review for coherence, clarity, and style. Check for any logical inconsistencies and ensure the first-person perspective stays consistent. Use strong verbs and vivid imagery to enhance the horror. Maybe include metaphor, like the house as a living creature, to deepen the atmosphere.
Now, time to draft the story with these elements in mind, keeping the word count in check and maintaining steady pace of tension and terror.
LongWriter-Zero Response
The creak of the front door echoed like a dying man's gasp as I stepped into the house. I'd been gone ten years, but the air still smelled like my mother's perfume-lavente cigarette smoke. The wallpaper the corner of the living room was the same, its wooden slats worn smooth by time. I'd spent my childhood in that chair, my mom's 's hands on my shoulders, rocking me to sleep while she hummed lullabies.
and peeling, the windows cracked, but the rocking chair
I was there to retrieve my grandmother's 's locket, a family heirloom left on the dining table. The house felt like lung holding its breath. tiptoed past the kitchen, where the fridge hummed like a giant insect, and into hallway. That's when heard it: a child's laughter, high and shrill, coming from the nursery. My heart leaped. The nursery had been sealed off' since my mom died there, giving birth to me. The door was locked, but the laughter grew louder, as if the room itself was mocking me.
I fumbled for the key, my hands slick with sweat. The lock clicked open, and pushed the door. The room was empty, the crib still standing, its bars rusted like old teeth. But the rocking chair in the corner was moving moving-slowly at first, then faster, the wood creaking in
19

---

Published as a conference paper at ICLR 2026
time with the laughter. stumbled backward, tripping over own feet. The chair stopped. Silence fell like guillotine.
Then the rocking started again, and saw it: shadow the wall, the shape of woman. My mother. She'd died giving birth to me, her ghost was here, trapped in the house she'd loved. could see her face now, her eyes wide and pleading, her mouth moving. "I'm sorry," she whispered, her voice scratchy echo. "I'm so sorry."
I ran, my footsteps echoing like gunshots. The front door slammed behind me, and I sprinted down the street, my lungs burning. But laughter followed me, a nigh--ptchhd giggle that seemed come from everywhere. When looked back, the house was dark, but the rocking chair was still moving in the nursery window, its shadow swaying like pendulum. I knew then that she't never leave that she't d he waiting, rocking, until the day I returned.
Gemini-2.51 Pro--Pro Response
It started with the rocking chair. bought it estate sale, heavy oak piece with dark, almost black finish. put it the corner of my bedroom. The first night, woke up to find the middle the room, facing my bed. blamed a sloping floor and my own exhaustion, pushing it back against the wall.
The next night, it was closer. Close enough that my feet would have touched it if I'd stretched. My blood ran cold. This wasn't a slanted floor. spent the next day trying to debunk checking for drafts, vibrations, anything. Nothing. That night, I decided watch it.
I sat up in bed, my eyes locked on the chair in the corner, the moonlight painting it in shades of grey. For hours, nothing happened. My eyelids felt like sandpaper. The urge to blink was a physical agony. Finally, couldn't 't hold out. blinked. SCRAPE.
The sound was violent, splintering shriek of wood on wood. My eyes flew open. The chair was now foot from my bed. In the seat, where there was nothing before, sat the deep. dark impression of a human form. didn't blink again. couldn't. t. My heart hammered against my ribs, frantic prisoner. stared the empty space in the chair, the dent in the darkness, and understood the rule: only moves when can't 't see it. My eyes burned, tears streaming down my face, blurring my vision. Each time a tear washed over my pupil, the chair scraped closer. It was herding me. started backing away. my body trembling. my gaze fixed on the chair. My back hit the bedroom door. My hand fumbled behind me for knob.
The chair was now at the edge of my bed. I could see the shape in it more clearly the gaunt hollows of a face, grin that looked like chipped wood. just had to get the door open. My fingers finally closed around the cold brass knob.
That's when heard the soft, dry click of the deadbolt sliding into place.
The sound hadn't come from the door front me. came from the hallway, just behind it. The thing in the chair wasn't 't trying to get to me. It was a distraction. It was just keeping me looking forward, my back to the door, while its companion locked me in with it.
Query
Hello, you're senior manager at company specializing selling women's clothing on Amazon US. The company has been operating on the Amazon US platform for seven years and has estahlished certain brand presence. Now, you need to recruit a Senior Amazon Operations Manager with relevant experience, particularly in promoting hest- selling products. The candidate should have experience managing clothing category with fewer than 5,000 SKUs and possess strong English skills.
LongWriter-Zer Think
The user wants me to draft job posting for Senior Amazon Operations Manager at a women's clothing company on Amazon US. Let me start by breaking down the key requirements. The company's been the platform for seven years, they 're established but mayhe looking to scale up. They need someone with experience promoting hest-sellers, especially in a clothing category with under 5,000 SKUs. Strong English skills are essential,
20

---

Published as a conference paper at ICLR 2026
so perhaps the candidate will be handling international customers or communicating with Amazon's s US-based teams.
First, I need structure the job posting. Typically, these start with company overview, then the role's purpose, responsibilities, qualifications, and perks. The user mentioned "senior," so the candidate should have leadership experience. Maybe include a bullet point about mentoring or leading team.
The clothing category is specific detail. I should highlight that the candidate needs to have managed similar inventory sizes, maybe 3-5k SKUs. The user also specified fewer than 5,000, so I'll make sure to use that exact number. English skills are crucial, so perhaps include note about effective communication, maybe even proficiency in writing product descriptions or handling customer inquiries.
I need to think about the audience. This is for experienced professionals, so the tone should be professional but also inviting. Use active voice and bullet points for clarity. Maybe include a sentence about the company's 's growth and the role's impact on that. Possible challenges: Ensuring the job posting is specific enough without being too restrictive. The user wants someone with Amazon US experience, so should mention that the candidate should have worked on the US platform. Also, promoting best-selling products- moducts-mmybe the candidate should have track record of increasing sales or improving ranking.
Should include a section on what the company offers? Perks like flexible hours, benefits, opportunities for growth. This could make the posting more appealing. Maybe mention the company's 's brand presence and the chance to work on well-established platform. should also think about keywords. Terms like "Amazon US." "Senior Operations Man- ager." "clothing category." "SKU management," "best-seller promotion," "English profi- ciency," "leadership experience." Including these will help candidates find the posting and ensure it's relevant to their skills.
Potential ambiguity: The user says "promoting best-selling products." Does that mean creating new best-sellers or managing existing ones? I'll assume both, so mention driving sales growth and improving product performance. Also, "fewer than 5,000 SKUs" is specific number-mann make sure to use that exactly.
Style and tone: Keep it formal but not too stuffy. Use bullet points for the responsibilities and qualifications. Maybe start with an engaging opening line about the company's 's success and the role's importance.
Audience adaptation: This is for someone with 5-7 years of experience in Amazon opera- tions, so the language should he industry-specifie but not overly technical. Highlight the need for strategic thinking, data analysis, and problem-solving skills.
Quality assurance: Review the structure to ensure all requirements are covered. Check clarity in each section. Maybe ask myself: Would a candidate with the right experience find this posting appealing? Does it clearly outline what's expected?
Potential areas expand: Mentioning the company's 's growth trajectory, the competitive nature of the fashion industry, and how the role will contribute the company's 's goals. Also, the importance of English skills in dealing with customers, supplicrs, or Amazon policies.
In conclusion, the job posting needs to balance the company's 's history and achievements with the specific skills required for the role. Emphasize the candidate's impact on sales, product management, and team leadership. Use clear, concise language and highlight the benefits of working for a successful, growing company on a major e-commerce platform. Make sure the English proficiency requirement is clear and that the candidate understands the role's le's scope, from data analysis to strategic planning.
KongWriter-Zeer Response
Senior Amazon Operations Manager, Women's Clothing Your Company Name Remote / Your Preferred Location Competitive Salary + Benefits
At Your Company Name (YCN), we'ne 'e're redefining the way women experience style and confidence through our curated collections on Amazon US. With seven years under our belt, we' ve built a loyal customer base and established a brand known for quality, inclusivity, and accessibility. Now, we're re scaling-and we need seasoned Amazon operations leader to drive our next chapter. If you'te re a data-driven, customer-centric strategist who thrives on turning best-sellers into category icons, this role is for you.
21

---

Published as a conference paper at ICLR 2026
The Role:
As our Senior Amazon Operations Manager, you'll own the entire product lifecycle for women's clothing line, ensuring every item meets high standards of excellence. Your mission: elevate our best-selling products to new heights while maintaining 4.8+ rating average and steady market share growth. You'l Il collaborate cross- fusstinuciionllyy with marketing, merchandising, and customer service teams to execute flawless launches, optimize ad spend, and refine pricing strategies. Key responsibilities include:
Driving Product Visibility & Sales: Analyze competitive data to identify high- potential SKUs (targeting our <5,000 items) and develop tailored promotion plans that balance organic growth with paid advertising. Leverage Amazon's tools (e.g.. A+ content, Vine Voice, influencer outreach) to amplify customer engagement and increase review velocity.
Inventory Supply Chain Mastery: Manage SKUs efficiently, balancing stock levels meet demand while minimizing markdowns. Negotiate with suppliers to secure competitive pricing and ensure on-time deliveries to Amazon FBA warchouses, optimizing freight costs without compromising speed market.
SEO & Pricing Strategy: Lead SEO audits to optimize product listings keywords, ratings, and reviews. Implement dynamic pricing models that adjust to sales trends, competitor actions, and Amazon algorithms to maximize profitability without sacrificing competitiveness.
Customer Obsession: Champion customer-centric approach by monitoring feedback loops, addressing negative reviews proactively, and translating customer insights into actionable merchandising strategies. Partner with customer ser- vice team resolve escalations and maintain best-in-class shopping experience.
Stakeholder Alignment: Communicate complex operations data to internal and external stakeholders, translating metries into stories that drive decision- making, Work closcly with marketing team align ad spend with product performance, ensuring every dollar invested delivers ROI.
What We're Looking For:
This role demands unique blend of Amazon expertise, fashion category knowledge, and analytical prowess, Here's we're looking in our next superstar:
7+ years of Amazon Vendor Central experience, preferably in women's clothing or a comparable fast-moving consumer goods category with <5,000 SKUs.
Proven track record of managing best-seller promotions that doubled or tripled sales within 6 9 months, with clear data back your strategies.
Fluent English (written and spoken) for scamless communication with customers, vendors, and cross-fune cross-functional teams. Strong presentation skills are must.
Analytical acumen interpret Amazon reporting (e.g.. SALESrank, Impres- sions, Conversion Rate) and translate insights into actionable plans.
Adaptability to navigate Amazon's 's evolving policies and market dynamics, with the ability pivot strategies quickly without losing momentum.
A passion for fashion and a genuine interest in building brand that empowers women through style. Bonus points if you't 've worked with Amazon private label brands.
What We Offer:
Beyond competitive salary and comprehensive benefits, you'l '1l join company that values innovation, collaboration, and growth. Here's why you'll 'll'l love working at YCN:
Ownership of Your Impact: You'l Il have direct line to the CEO, enabling you to shape strategy and see your ideas transform our Amazon presence.
Growth Mindset: We're expanding our product portfolio and looking to launch new lines. Your insights will help define our future direction.
Innovative Culture: From sustainable packaging digital-first marketing.
we're always pushing boundaries. We'll support you developing new tools and processes that elevate the industry standard.
22

---

Published as a conference paper at ICLR 2026
Career Trajectory: This role is stepping stone to larger leadership opportuni- tics. We invest in our team's 's growth, offering mentorship, training, and a clear path advancement.
If you're ready to take your Amazon ops career to the next level and help build brand that resonates with millions of women, apply [here] with your resume, brief cover letter explaining how your past achievements align with our goals, a portfolio your most successful Amazon campaigns. We can't 't wait to meet you!
P.S. We' re open to remote candidates based the US. Let's chat!
Qwen3-235B-A22B Response
Job Title: Senior Amazon Operations Manager (Remote U.S. Based)
Company Overview
For over seven years, our company has been trusted name in the women's 's clothing space Amazon US. With strong brand presence and commitment to quality, style, and customer satisfaction, we've built loyal customer base and a portfolio of top-selling products. As we continue to grow, we seek a strategic and results-driven Senior Amazon Operations Manager to lead our marketplace strategy, optimize product performance, and drive scalable success in the competitive apparel industry.
Job Summary
We are seeking a seasoned Amazon Operations Manager with deep expertise in managing clothing categories (under 5,000 SKUs) and proven track record of scaling products to bestseller status. This role requires hands-on leader who can oversee end-to-end Amazon operations, from listing optimization and inventory management data-driven strategy development. The ideal candidate will have strong English communication skills, passion for e-commerce, and a demonstrated ability to boost visibility and sales for women's apparel brands.
Key Responsibilities
Manage daily Amazon US operations, including listing creation/opimmization, pricing strategy, and adherence to Amazon's 's policies and best practices.
Operations Leadership:
Product Promotion: Develop and execute strategies promote top-selling products, leveraging Amazon Ads (Sponsored Products, Brands, and Display), A+ content, and promotional campaigns.
Inventory Management: Monitor inventory levels, forecast demand, and col- laborate with suppliers to ensure optimal stock availability reduce FBA costs.
Data Analysis: Track KPIs (ACoS, ROAS, conversion rates, keyword rankings). conduct competitive analysis, and insights to refine strategies and maximize profitability.
Team Collaboration: Work cross-functionally with creative, marketing, and supply chain teams to align product launches, content creation, and seasonal planning.
Compliance Optimization: Ensure product listings meet Amazon's 's guide- lines, including accurate categorization, high-quality images, and compelling, SEO-optimized descriptions.
Qualifications
years experience in Amazon operations or e-commerce management, with at least years focused on apparel (specifically women's 's clothing).
Proven success in scaling products to top 10,000 Amazon Best Seller Rank (BSR) in the Clothing category.
Experience managing catalogs with fewer than 5,000 SKUs, with a focus on efficiency and growth.
23

---

Published as a conference paper at ICLR 2026
Advanced proficiency in Amazon Seller Central, along with tools like Helium 10, Jungle Scout, Similar Web.
Exceptional written verbal English skills; ability craft persuasive product listings and communicate effectively with stakeholders.
Strong analytical mindsct with expertise in interpreting sales data, market trends, and customer feedback.
Highly organized, proactive, and capable of managing multiple priorities in fast-paced environment.
Preferred Qualifications
Experience with Amazon Ads campaign management (Sponsored Brands Video, Stores, etc.).
Knowledge of U.S. fashion trends, sizing standards, and customer preferences.
Background in logistics or supply chain management for apparel.
Bachelor's degree in Business, Marketing, related field.
What We Offer
Competitive salary performance- bonuses.
Fully remote work flexibility with collaborative. growth-oriented team.
Professional development opportunities and access cutting-cdge e-commerce tools.
A dynamic environment where your expertise will directly impact the company's growth and success.
Application Process
Please submit your resume and a cover letter detailing your relevant experience to [emai] address] with the subject line: Senior Amazon Operations Manager Application /Your rolling basis position is filled.
NameJ. Applications will be reviewed
Join us in shaping the future of our brand and driving the next phase of growth Amazon's most competitive shelves!
Query
H AEP.AHAAAHHHAAAAAAAAA          E #
FEM
LongWriter-Zero Think
- #MTTME
##. #tt.
#
M. PT 12
HE 16 / F 1
$
# #
5 2
11 500 K. 1
Ht
( 1
Iff
4
* 0 K-1
A 1 BHE
AL
A
24

---

Published as a conference paper at ICLR 2026
Th U EE
+T TT
KX
3J
S #
88 F X
#
11
-T 1
||
#1.
500
PT 11
F 1F TI
A 10
i ল. , B.Z. WW - -
LongWriter-Zeru Response
3 i
#H J. 31 A
17
4 4
#1 A XF
#
/ 13
1: "1 S1
B 2 0000
7 4 IN
TO II A A T 1
# FL b/
N Fej , 14 SA
I 3
Deepseek-RL R1 Response
FU A T A
SAE #
25

---

Published as a conference paper at ICLR 2026
1 Aे S + " ( 89 ST
AE *
26


Reviews:
[{"review_id": "2SCmCvp8kQ", "reviewer": "Reviewer_nuzb", "summary": "The paper introduces LongWriter-Zero, an RL framework using GRPO to train LLMs for ultra-long text generation from scratch, bypassing synthetic SFT data. It uses composite rewards for length control, writing quality, and format, plus <think> prompts for reasoning and continual pre-training on writing data. The authors test the approach on WritingBench, Arena-Write, and human win-rate evals. Results achieve SOTA: 8.69/10 average on WritingBench, 1447 Elo on Arena-Write, and win rates >62% vs. 100B+ models like DeepSeek-R1.\n\nStrengths:\n- The pure RL approach fosters emergent long-form capabilities without data quality biases from synthetic SFT, enabling innovation beyond teacher model limits.\n\nWeaknesses:\n- Using heuristic rewards naturally entails reward hacking. The effectiveness of the proposed method is based on how well the heuristics of the reward models align with the actual metrics that they care about. The paper does not show that LongWriter-Zero is immune from reward hacking. This suggests that the generalization of the approach might be limited.\n- Composite reward balancing risks suboptimal trade-offs, where normalizing advantages could dilute focus on critical aspects like coherence in ultra-long sequences.", "questions": "See weaknesses.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The pure RL approach fosters emergent long-form capabilities without data quality biases from synthetic SFT, enabling innovation beyond teacher model limits.", "weaknesses": "- Using heuristic rewards naturally entails reward hacking. The effectiveness of the proposed method is based on how well the heuristics of the reward models align with the actual metrics that they care about. The paper does not show that LongWriter-Zero is immune from reward hacking. This suggests that the generalization of the approach might be limited.\n- Composite reward balancing risks suboptimal trade-offs, where normalizing advantages could dilute focus on critical aspects like coherence in ultra-long sequences.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "EdfW08BcBR", "reviewer": "Reviewer_rDBg", "summary": "While ultra-long generation is a desired capability in large language models, it often requires fine-tuning on SFT data with expensive data which may also lack quality. This paper instead motivates and introduces LongWriter Zero, an RL paradigm to transform a base language model into one that can generate long and high-quality text - without written data. They provide intuitive reward design, an introduce a new benchmark to evaluate different LLMs against each other (Arena-Write). They also perform helpful ablations demonstrating the efficacy in the continual pre-training case.", "questions": "* Why did you need to create and evalute with Arena-Write, ie, what is wrong with existing benchmarks for this?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "* The paper is written well in general\n* This solves a key, timely problem: how do we encourage more length without access to expensive SFT data? \n* There are comprehensive details of prompts and experimental design which seem well-justified, including a thorough appendix.\n* The reward design is justified and includes multiple components\n* The proposed model LongWriter-Zero (32B scale) outperforms a variety of methods across a few benchmarks", "weaknesses": "* The results are shown on only one model trained with the introduced paradigm. It would be nice to test if this holds across other models, namely 1) models not Qwen and 2) perhaps a smaller size Qwen (ie, does this still work at smaller scale)\n* While the approach is nice, replacing costly SFT data, the process intuitively seems prone to length reward hacking, since the reward is based on the averaged advantage over length equally with format and writing quality, a repetitive string may overwhelm the rest of the rewards by being extremely long, leading to reward hacking with long and non-fluent texts. This may be solved by weighting the other factors more, or in a different way.\n* Related to the above, it is unclear how robust the writing reward model, the proxy for quality, performs in complex and long scenarios. Though it is mentioned that it \"capture holistic writing quality, including fluency, coherence, and helpfulness\", there are other more complex factors like factuality which are hard to control for.\n* There should be more baselines. While it is nice to compare to existing other models, there needs to be more comparison of this length method to other approaches, such as those that leverage synthetic data, etc (ie, LongWriter on 32B scale)\n* Minor: ArenaWrite seems small - only 100 examples\n* There are a couple formatting errors, for example, a citation link on the bottom of Page 6 spanning over the page break\n* Minor: Sections RQ1, RQ2, and RQ3 seem a bit disconnected, could be transitioned better.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "4PFPdpnIZd", "reviewer": "Reviewer_htGy", "summary": "The paper describes a new approach to generating quality long-form text, LongWriter-Zero, aiming to improve over standard LLM generation baselines and the small number of works that have explicitly tackled longform generation (LongWriter, Suri).  The approach uses three components, framing their use using three research questions (RQs): RL with appropriate reward model aspects (length, quality, structural format and redundancy), test-time scaling, and continual pretraining.  There is an evaluation on three benchmarks (the existing WritingBench, Arena-Write defined in this paper, and a human-in-the-loop win-rate evaluation), as well as more detailed investigations like e.g. assessments of the importance of the three core components of LongWriter-Zero.", "questions": "Please see above.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "* The three components of LongWriter-Zero are intuitive, even if not necessarily obvious, contributing some originality.  I thought the argument for RQ2, on the use of test-time scaling – that it had mostly been used for e.g. code and math problems, and that its use was less obvious for long-form generation – was sensible and interesting, and worth investigating.\n\n* The model seems to perform strongly, even against much larger models.  Having three different types of evaluation was also a strength.  Ablation analyses were also informative.\n\n* This is an important aspect of LLMs to look at, and there isn't too much work in this space yet, making this a useful contribution.", "weaknesses": "* The evaluation was rather different from the LongWriter paper by Bai et al, which is really the primary baseline, as framed by e.g. the paper abstract and the related work.  LongBench-Write isn’t mentioned at all in the present paper.  While WritingBench does in some ways supersede LongBench-Write (with that paper discussing the relationship of the new benchmark to the prior LongBench-Write), I still would have expected to see an evaluation on LongBench-Write here for comparability.\n\n* Relatedly, the critic used for the first evaluation probably warrants some discussion.  It of course makes sense to use the critic that comes with WritingBench, but there’s no commentary on or investigation of possible bias in the evaluation of Qwen models wrt non-Qwen models by a Qwen critic.  (Bai et al noted this for their GPT-4o judge for LongWriter.)  This also factors into the results in Table 1, where the DeepSeek-R1 is probably close enough that I wonder whether the improvement is real or a result of bias.\n\n* Looking elsewhere in Table 1 results, the commentary in Sec 3.2 just observes that LongWriter-Zero outperforms all models, but it only outperforms Qwen3-235B-A22B by a quite small 0.01 overall.  (I’d also note that the Qwen3-235B-A22B score for EN should be bolded as equal highest given the one decimal place result.)  While this is still noteworthy, given that LongWriter-Zero is a much smaller model, I think it should be remarked upon.\n\n* Relatedly, do you have any thoughts about why LongWriter-Zero and Qwen3-235B-A22B are so close in the Table 1 results by the critic but there is a much more substantive difference in Elo ratings (discussed in Sec 3.2)?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "rYg1mtdFfv", "reviewer": "Reviewer_TSAk", "summary": "The paper introduces LongWriter-Zero by building a framework to generating ultra-long text by applying RL to a base LM. \n\nDifferent from the SFT approach from previous works, this paper proposes a RL approach with three main contributions 1) A dedicated reward model designed to balance length, writing quality and structural format. 2) The integration of a thinking step. 3) continual pre-training on writing-centric data would significantly boost the performance on top of the RL-based training. \n\nThe resulting model proves the effectiveness of the RL framework by demonstrating to achieve SOTA performance on WritingBench and good ELO score, surpassing its SFT counterpart model and open-source/proprietary models in long text generation.", "questions": "Questions: \n* Figure 2 and Figure 3 shows different story for \"Base-think\": Figure 3 shows a high ELO at the very beginning (before 150steps) but significantly worse in \"Writing RM\", does that mean an misalignment between Writing RM and ELO ratings? or writing quality is less important in deciding who win?\n\n* Figure 6 mentioned thinking token length plateaus because writing task has a ceiling. Could this plateau be an artifacts of the experiment design? For example, (a) could it be caused by the context length? (b) The format RM implicitly pensalizing overly long thinking setup? \n\n* Why remove KL-Penalty? details seen in the weakness section\n\nSuggestions on Writing:\n* Fig. 2 (right) needs more explanation on \"Mean Non-Overlong Generation Length\". It's hard to understand. And Green curve has different behavior than the other two methods. Need more explanation. \n* Use \"Direct-Answer\" and \"NoThink\" in different places but presenting the same meaning, a bit confusing.\n* nit: \"-Continual Pretrain\" in Table 1 may be misunderstood as dash line rather than \"minus\". Same for \"-Thinking\".", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "* The paper successfully built a pure RL-driven approach for an open-ended long-text generation task, moving beyond the prevalent paradigm of SFT on synthetic data, shed light on a topic with large headroom.\n* The paper is well-written to systematically  perform ablation on the importance of continual pre-training and the \"think\" step, providing valuable architectural insights. \n* LongWriter-Zero demonstrates SOTA results on multiple challenging benchmarks, validating the overall effectiveness of the proposed framework.", "weaknesses": "* It's a bit unclear comparison: \n   * If the comparison is between a basic SFT and the RL GRPO setup, it's no surprise the GRPO RL setup has more advantage; \n   * If the comparison is between RL and the SOTA approaches (which often has DPO/RL steps after a basic SFT), Table 1 is the only section discussion about evidence. \n\n* The paper is lacking details on how to build teh preference dataset for \"Writing RM\", e.g. data origin, size, diversity and human rating rubrics, etc. Given the importance of reward signals in RL, the missing info makes the major weakness on reproducibility and the trust in the results. \n\n* The paper mentioned removal of KL penality term (beta = 0, line 138). For the open-ended question like long-form writing, it's not justified   to use this setup. The KL term is an important regularizer mitigating reward hacking and the authors mentioned the repetition-drive reward hacking as a key limitation. The decision to remove KL penalty needs more thoughts.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Oral)