Think-While-Generating: On-the-Fly Reasoning for Personalized Long-Form Generation
Authors: Chengbing Wang
Yang Zhang
Wenjie Wang
Xiaoyan Zhao
Fuli Feng
Xiangnan He
Tat-Seng Chua
Venue: ICLR 2026 Poster (2026)
Keywords: LLM
Personalization
Reasoning
Primary area: foundation or frontier models, including LLMs

Abstract: Preference alignment has enabled large language models (LLMs) to better reflect human expectations, but current methods mostly optimize for population-level preferences, overlooking individual users. Personalization is essential, yet early approaches—such as prompt customization or fine-tuning—struggle to reason over implicit preferences, limiting real-world effectiveness. Recent “think-then-generate” methods address this by reasoning before response generation. However, they face challenges in long-form generation: their static one-shot reasoning must capture all relevant information for the full response generation, making learning difficult and limiting adaptability to evolving content. To address this issue, we propose **FlyThinker**, an efficient “think-while-generating” framework for personalized long-form generation. FlyThinker employs a separate reasoning model that generates latent token-level reasoning in parallel, which is fused into the generation model to dynamically guide response generation. This design enables reasoning and generation to run concurrently, ensuring inference efficiency. In addition, the reasoning model is designed to depend only on previous responses rather than its own prior outputs, which preserves training parallelism across different positions—allowing all reasoning tokens for training data to be produced in a single forward pass like standard LLM training, ensuring training efficiency. Extensive experiments on real-world benchmarks demonstrate that FlyThinker achieves better personalized generation while keeping training and inference efficiency.

Published as a conference paper ICLR 2026
THINK- WHILE-GENERATING: ON-THE-FLY REASONING FOR PERSONALIZED LONG-FORM GENERATION
Chengbing Wang'
Yang Z Wenjie Wang'
Xiaoyan Zhao
Fuli Feng'
Xiangnan He** Tat-Seng Chua
'University of Science and Technology of China, NNational University of Singapore
The Chinese University of Hong Kong
*National Engineering Laboratory for BITA, University of Science and Technology of China wwq19723 @maammmllll....l 1 ustc. edu.cn cn
ABSTRACT
Preference alignment has enabled large language models (LLMs) to better reflect hu- man expectations, but current methods mostly optimize for population-level preferences, overlooking individual users. Personalization is essential, yet early approaches such as prompt customization fine-tuning struggle reason over implicit preferences, limit- ing real-world effectiveness. Recent "think-then-ggenerate methods address this by rea- soning before response generation. However, they face challenges in long-form gener- ation: their static one-shot reasoning must capture all relevant information for the full response generation, making learning difficult and limiting adaptability to evolving con- tent. To address this issue, we propose Fly FlyThinker, an efficient thinnk-whieegeenertinng framework for personalized long-form generation. FlyThinker employs separate rea- soning model that generates latent token-level reasoning in parallel, which is fused into the generation model to dynamically guide response generation. This design enables rea- soning and generation to run concurrently, ensuring inference efficiency. In addition, the reasoning model is designed to depend only on previous responses rather than its own prior outputs, which preserves training parallelism across different positions-allowin all reasoning tokens for training data to be produced
single forward pass like standard LLM training, ensuring training efficiency. Extensive experiments real-world bench- marks demonstrate that Fly FlyTheeee achieves better personalized generation while keeping training and inference efficiency.
1 INTRODUCTION
Large language models (LLMs) have achieved significant advances through preference alignment tech- niques, enabling them to produce outputs that more closely reflect human expectations (Ouyang et al.) 2022
Rafailoy al. 2023). Yet, common preference alignment (Rafailov et al. 2023 Schulman
2017)
primarily focuses on population-level preferences, often overlooking the diverse characteristics and nuanced
needs of individual users (Guan al. 2025 Xie al. 2025 Li al. 2024 Zhao et al.) 2025b Qiu al.
2025a bj Zhhao et al. 2026 Zhang et al.) (025c). These limitations invvtably curb user satisfaction and hinder the more widespread adoption of LLMs in everyday life (Guan et al. 2025 Wu al. 2025 Zollo lt al. 2025 Zhao et al., 2025al Wang et al. 2025a [hi et al.] [024]] Wang et al. (025b). To bridge this gap. personalization has emerged as an indispensable component of LLM alignment an essential step to-
"Corresponding author; (zyang 1580, xiangnanhe] @gmail.com

---

Published as a conference paper ICLR 2026
Query
Reasoning
Response
Path to produce reasoning
Path generate response
Non-reasoning
Think-then-generate
Think-wwilleggnnrratingg (normal)
Think -eeewwiitt-gneratingg (FlyThinker)
Figure 1: Different ressoning-eenerationnn paradigms. Non-reasoning outputs responses directly. Think-then generate performs reasoning first, then responds. Standard think thik-wwile-genee generatingn interleaves reasoning and output sequentially, causing inefficiency. FlyThinker enables reasoning and generation proceed in parallel.
ward integrating broad, generalized utility with the fine-grained, personalized adaptability required for truly user-centric AI (Sorensen et al. 2024).
Existing LLM personalization methods typically rely either customizing user-specific prompts via 2024, Mysore et al. 2023b] Zhang et al. (025a), or fine-tuning the model (Zhang et al. 2025d Chen et al., [2225]]]] Kim et al. (025cc) to produce personalized outputs. While straightforward, these approaches often struggle in real-world scenarios where user interests are not explicitly stated but are only implicitly reflected in historical behavior. Such methods lack mechanism to reason over implicit user preferences, potentially resulting in suboptimal personalization. Aiming at addressing this limitation, a new thhnn-tthen-generate paradigm for LLM personalization has emerged (Luo al. 20256 Kim et al.) 2025a
RAG QQia et al.
Li et al. 2025b; (aalemi et al. 2025ch: rather than directly generating the final response, the LLM first produces an in-depth analysis of the user's preferences based on their input, and then generates responses
aligned with this analysis. By leveraging the model's reasoning capabilities, this approach can potentially capture implicit user interests more effectively, enabling more accurate personalization.
Despite its promise, the maank-tthe--enerrtte paradigm faces critical challenges in long-form personalized generation. In this paradigm, a static one-shot reasoning must capture all relevant information for the entire response, creating long-range dependencies that are difficult to model and learn. Moreover, it rigidly applies this one-step reasoning to guide all subsequent outputs, failing to adapt to dynamic changes content - long-form creation by users often involves ideas that evolve based what they have already created. To address these, we propose adopting a weink--wiill-eeeratiiie
paradigm, as illustrated in Figure [ where interleaved, with each reasoning step focusing guiding small
LLM reasoning and answer generation segment the response. This design better aligns with the natural progression of user thinking during creation, reduces learning complexity, and therehy potentially enables more effective personalization.
To implement the flink-while-genrratie paradigm, a key challenge is efficiency: frequent reasoning across response segments substantially increases both training and inference time as the number of segments grows. Latent reasoning (Hao et al. 2024) offers more efficient alternative to standard chain-of-thought (CoT) ap- proaches (Wei et al. 2022) by compressing the reasoning path into few hidden-state tokens, avoiding the overhead of generating lengthy CoT sequences. However, directly applying existing latent reasoning meth- ods remains inefficient. Current approaches (Tang al.] 2025 Zhang et al.) (025b) typically use the same LLM for reasoming and generation, forcing each output token generation to wait for all preceding reasoning tokens. Non-reasoning training can be accelerated by feeding all ground-truth tokens simultaneously. en- abling parallel optimization. With reasoning tokens, this parallelism is lost, as the model must wait for the reasoning outputs, slowing down training. For inference, the time cost also increases with the total number of reasoning tokens. To solve the problem, the key is to keep the parallelism.
To this end, we propose FlyThinker, an efficient ization with latent reasoning. FlyThinker enables parallel training and inference by maintaining separate reasoning model that continuously generates token-wise latent reasoning based on the query and the tokens produced far. This reasoning is fused into the generation model guide token prediction, allowing the generation process to incorporate dynamic reasoning. Importantly, reasoning tokens at different positions do not have direct sequential dependencies they depend only the query and the already generated to-
woink--wiil-ggeeratiin framework for LLM personal-
2

---

Published as a conference paper ICLR 2026
kens-alle allowwngg reasoning to run in parallel during training when all ground tokens are input for use in a teach-forcing manner. Once reasoning at all positions is computed in parallel, the generation model can also operate in parallel to predict all ground-truth tokens during training. During inference, the two models operate in staggered manner. while the generation model predicts the current token, the reasoning model prepares reasoning for the next prediction. This design allows FlyThinker to implement token-level think- while-generating efficiently, avoiding sequential bottlenecks while ensuring that each prediction is guided by up-to-date reasoning.
In summary, the main contributions are three-fold. Firstly, we put forward the concept of the think-while- generating paradigm for personalized long-text generation. Secondly, we propose Fly Thyhhieer, a novel framework that integrates parallel reasoning model, achieving efficient token-level think while genner-gnnr..ing Lastly, we conduct extensive experiments demonstrating that FlyThinker delivers substantial improvements in both personalization quality and generation efficiency over strong baselines.
2 TASK FORMULATION
This work studies personalized long-form generation, where the LLM must infer user preferences from historical data and produce responses that aligo with these preferences for new queries. Formally, let ID denote the collected dataset, and let (h, y) TD represent user-specific sample, where is the query, y is the ground-truth response, which naturally is long (e.g. full movie review written by the user), and h is the historical record consisting of the user's past (query, response) pairs. The goal is leverage I' adapt the LLM such that, given (h, .),, it generates a response consistent with the user's preferred response y.
Since user preferences are only implicitly encoded in historical data, it is better for the LLM involve a reasoning process to infer them before generating the final response. Existing work typically adopts the "think-then- generateeeeeeee paradigm to involve the reasoning, which can be expressed as:
(I) where r denotes the reasoning trace. However, in long-form generation, this paradigm often suffers from low learning efficacy and misalignment with the inherently dynamic nature of long-form writing-usert ideas evolve as the response unfolds. To address this, we consider a thilk- while-generating paradigm: after producing a partial response (a token in this work), the model generates new reasoning step guide the next segment of text. This can be formulated as:
Shink-then-Genevee
(h,
LLM
LLM, 9.
Shink--hile-Generaiin
(h, LLM, LLM,
LLM
LLM
....
(2)
where iA denotes the k-th token of the generated response. This work focuses on how to achieve this "think- while-generating" paradigm efficiently.
3
METHODOLOGY
In this section, we introduce Fly Thinker,,, covering its overview, architecture, training, and inference.
3.1 OVERVIEW
To efficiently achieve "Think wwwle-heneraaeeeee the same model for both generation and reasoning. this setting, generating the next response token or reasoning step must wait for the completion of previous reasoning, leading unacceptable delays. To address this, we propose Fly Thinker,, which parallelizes response generation and reasoning by employing two separate models for these tasks, shown in Figure At each step, while the generation model produces
identify key challenge in existing approaches: using
3

---

Published as a conference paper ICLR 2026
Training
Inference
Latent-reaaoning Enhanced Input
Paratiel computation
Generator
Generator
Generator
Token
Latiee
Rossoner
Reasoner
Only one pass
Ressoner
1
For
all Latent thoughts
Token
Token Embb
Lutent Thought
User Info
Figure 2:
An illustration of Fly Thinker, an efficient "think "tlink-while-generaatnn"" framework for LLM person- alization with latent reasoning.
a response token, the reasoning model can simultaneously generate a new reasoning segment, which then guides the next step of response generation. Formally,
(h,,)) (mmbeettee) in (ham:battee ÿ2 92....
Generation:
(3)
(h,z) (b.a.0c1)
ril (h.v.9.22
Reasoning:
r2l .......
where r' denotes the new reasoning results at step f,, and tcr and ret represent the response and reasoning generated before step f. This obviously ensures that inference can be performed efficiently.
Based on this parallel model design, further break the direct sequential dependence between reasoning tokens while ensuring that each reasoning step depends only on previously generated response tokens, as in-
dicated by Tz-il (h,2,0e)), Yz, meaning rz depends only on (h, z (..). This blocking design enables parallel training: during training, all reasoning steps can be generated in parallel by providing the reasoning model with the full ground-truth response sequence, improving training efficiency, similarly for the generation side. For generation, each step depends on both the previously generated reasoning and response, as indicated by Út-1
(h.2,)ectraa), rt, meaning ýt depends on (h, x. dec + ret), ensuring that dynamic reasoning truly
considered for the generation.
3.2 MODEL ARCHITECTURE
We enable parallel response generation and reasoning by introducing two separate models: a Reasoner, which produces latent reasoning, and Generator, which leverages for response generation. These mod- els are organized in parallel architecture-wine simultaneously generates new reasoning segment to guide the next step of response generation, as shown in Figure We next elaborate on the two models to explain the parallel architecture.
the Generator produces a response token, the Reasoner
The Reasoner (R) is an LLM designed to produce latent reasoning token at each step of response generation. At step I, it takes the query and the previously generated response input to produce a new latent reasoning voken-similas to how humans update their thinking during the writing process based newly written content. Following common practice in existing works, the latent reasoning token at each step is extracted from the hidden state of the top layer the Reasoner. Formally.
Reasoner.
R/
(h, 9cc- 1)[-
(4)
where ( denotes the extraction of the hidden state of the last input token the final layer of the Reasoner, T2 E Rdd denotes the new generated latent thought with dimension d, and fer-1 denotes the token
4

---

Published as a conference paper ICLR 2026
generate before step t. Notably, the Reasoner considers only the previously generated response tokens and does not explicitly include prior reasoning, breaking the direct sequential reliance between different parts of reasoning. This design is critical for enabling parallel training, which will explained in detail later.
Generator. The Generator (G) is implemented as another LLM but enhanced by incorporating latent thoughts into the token prediction process. Instead of relying solely on discrete token embeddings, the Generator fuses latent thoughts with response embeddings at each token position, allowing reasoning sig- nals to directly influence generation. Formally, for the 1-th token prediction, the predicted probability formulated follows:
P(Y: = yo h, vc+) Golvelh, x, F<i)))
(5)
where the fuse operation f()) is performed in the LLM token embedding space as f()c.re)) [e(yn) Ar...... c(ve-1) Are-1),
(6)
with e(y1) E Fd denoting the embedding of token /1. similar for others. The coefficient À controls the strength of the latent reasoning signal injected into the Generator.
3.3 PARALLEI. TRAINING
We jointly optimize the Reasoner and Generator in an end-to-end manner, allowing the model to learn reasoning and response generation simultaneously without requiring external supervision for reasoning.
Benefiting from our "breaking sequential dependence" design (where each reasoning step does not directly rely on previous reasoning) in the Reasoner, we can achieve parallel training of reasoning and response generation among different token positions using the feacher-forcing technique, by feeding all ground-truth tokens into the model, as in standard L.M training.
Specifically, at each training iteration, for each training example (h, y), we input the entire into the Reasoner and perform single forward pass to obtain the latent reasoning for all token positions. Formally,
= (r......... rTl (1.
where 7' is the length of y. V<T denotes all tokens of y except the last one, and R1 ()(- -1] extracts the hidden states corresponding to the last I' tokens of the input sequence.
After obtaining reasoning results for all steps, the Generator predictions can be computed similar one- pass forward computation. Formally,
[P(Y|hh,*)... x), P(Yih,, = 2,, -
(8)
where G+(-)[-T
-1] extracts the predicted probability distributions of G at the last T token positions.
The Generator and Reasoner are then jointly optimized using the standard next-token prediction loss (de- noted by L)::
L=
Σ (N...))ED
Σ
log P(): = yt h, x,
(9)
This optimization ensures that the Reasoner is updated naturally alongside the Generator, allowing Fly- Thinker capture user-specific preferences without explicit annotations auxiliary objectives. Importantly, since all latent reasoning tokens are computed in parallel, training remains efficient, with computational costs close to standard LLM fine-tuning.
5

---

Published as a conference paper ICLR 2026
3.4 PARALLEL INFERENCE
For inference, we run the Generator (Fquation 5 and the Reasoner (Equation 4 in parallel, enabling effi cient decoding without any additional delay compared to a standard non-reasoning LLM, as illustrated in Algorithm 2 Formally, at each step:
(Parallel reasoning) re R 2; Dct-1)l-1]. (c
(10)
(Parallel generation) ú. P(Y. 21 vee) Go(y,l,, f(Det, r<))),
As the equations show, while the Generator produces token, the Reasoner simultaneously generates new latent reasoning token for the next step. This eliminates waiting, enabling response generation without time delay. Notably, the computational cost similarly reduced, and memory usage increases.
4 EXPERIMENTS
In this section, we conduct experiments to answer the following research questions: 1) RQ1: How does FlyThinker compare with strong baselines on personalized long-form text generation? 2) RQ2: What are the training and inference efficiencies of FlyThinker? 3) RQ3: Does FlyThinker improve generation quality across different token positions? RQ4: How do the key architectural components of FlyThinker contribute to its overall performance?
4.1 EXPERIMENT SETTINGS
Datasets Backbone LLMs. We evaluate our FlyThinker on three long-form personalized generation tasks from the user-based split of the LONGLAMP benchmark (Kumar et al. 2024): Product Review, Ab- stract Generation, and Topic Writing, which are designed to assess the effectiveness of LLMs in generating personalized, long-text outputs. The model is required generate long-form response that aligns with user preferences. Our primary backbone is Qwen2.5-3B. Soven2.5-3B-Instruuee Instrucc, with additional experiments on Qwen2.5-7B. Instruct and Gemma-7B-it available Appendix L
and M
We compare FlyThinker against the following baselines: (1) Tuning-free methods: query-only generation (Non-pers) (Yang et al., 2024), retrieval-augmentee generation (RAG) (Lei
Baselines & Evaluation Metrics.
et al. 2023). and context steering (CoS) (He et al.] (2025;; (2) Timing- based methods: supervised fine-
tuning (SFT) Salemi al. 2024). summary-enhanced training (LLM-TRSR) (Zheng al. 2024), causal
preference modeling (NextQuill) (Zhao et al.) (2025c). chain-of-thought reasoning (CoT) (Wei et al. 2022
Salemi et 2025b), and latent reasoning (Coconut) (Hao al. 2024]. We report ROUGE-1, ROUGE-L,
BLEU, and METEOR scores. Baseline details and results on other evaluation metrics are provided in the
Appendix D and respectively.
4.2 PERFORMANCE ON PERSONALIZED LONG-FORM GENERATION (RQ1)
The main results summarized in Table I
From the table, draw three major observations:
Obs 1: Consistent gains over strong baselines. Across all three tasks, FlyThinker surpasses SFT and other tuning-based methods on most metrics. For example, on Product Review, FlyThenker achieves ROUGE-1 0.3663 (+3.1 % over SFT) and a BLEU of 4.36 (+1 (+11.5%). Similar trends hold for Abstract Generation and Topic Writing, where FlyThinker improves BLEU by about 10% over SFT and delivers higher ROUGE- I. scores. These results demonstrate that Thinker captures more salient user-specific information and produces outputs that both lexically and structurally closer the references.
Obs 2: Robustness across domains. FlyThinker maintains strong performance regardless of task type or text length. Its improvements especially pronounced Abstract Generation, where it reaches BLEU
6

---

Published as a conference paper ICLR 2026
Table 1: Main results on our FlyThinker method across three datasets: Product Review, Abstract Generation, and Topic Writing. FlyThinker consistently outperforms both tuning ffenng--feeeeee and tuning- tuning-based baselines.
Product Review
Abstract Generation
Tople Writing
Methods (1)
ROUGE-1
ROUGEL
BLEU
METEOR
ROUGE-L
METEOR
ROUGE-1
ROUGE-L
METEOR
pers
0.2839
0.1317
1.5377
0.1783
0.3276
0.1717
4.5814
0.2476
0.2189
0.1057
0.1869
0.3176
0.1420
3,2990
0.1616
3.4047
0.2998
0,1099
0.2253
0,2781
0.1347
2.9677
0.1576
3.2591
0.2134
LLM--RRSR
0,3377
0.1450
2.3279
0,2056
5,5359
0.2371
1.5552
0,1922
NesiQuill
0,3311
2,6599
0,2038
0,1269
2.1006
0.1817
SPT
0.3551
0.153
3.9112
0.2309
0,3602
0.2062
588189
0.2392
0.2916
0.1345
3.8876
0.2013
Ca'l
0,3466
0.1508
3,3905
0.2220
0.3685
0.2077
5,8538
0,2469
0.2846
0.1304
3.0020
0.1967
Cozomut
0.3381
0.1496
3.3213
0.2093
0.3542
0.2041
5.2578
0.2298
0.2786
0.1317
3,0736
0.1823
Thinkee
0,3663
0,1560
4.3620
0.2489
0,3716
0.2090
6.3383
0.2511
0.3139
0.1362
4,0606
0.2409
of 6.34, outperforming CoT (5.85 BLEU) and Coconut (5.23 BLEU). This suggests that the generating paradigm particularly effective in preserving coherence and relevance over long sequences, adapting dynamically as content evolves.
Obs 3: Superior to latent-reasoning baselines. Compared with Coconut, which employs think thenk-theee--- generate latent reasoning strategy, FlyThinker consistently yields higher ROUGE and BLEU across all tasks. This confirms that our interleaving reasoning with generation enables more fine-gran one-grained control and better exploitation of user context than static latent reasoning approach.
4.3 TRAINING AND INFERENCE EFFICIENCY OF FLYTHINKER (RQ2)
We further analyze the efficiency of FlyThinker. We report both training time per epoch and inference latency with fixed output length for fair comparison (Figure 3).
Training Efficiency
SFT
Obs 1: In terms of training efficiency, Fly Flynhinker trains much faster than other reasoning-base baselines while remaining close to SFT. Although FlyThinker produces latent reasoning for every token position, its parallel reasoning mechanism adds only minor overhead relative to SFT. Crucially, this overhead remains far lower than CoT and Coconut, whose autoregressive reasoning requires sequential token generation and substantially slows down training. When the Reasoner is scaled down, Fly FlyThiiker's 's cost ap- proaches that of plain SFT (see Section 3.3), demonstrating that par- allel reasoning generation effectively amortizes the additional com- putation.
Coconut
FlyThinker
(R - 3B)
FlyThinker
(R 1.58)
0,00 0.25 0.50 0.75 1.00 1.25 1.50 1.75
Training time (hours/epoch)
Inference Efficiency
#Raasoniig tolas-100
#Reasening takes- Leco
Coconut
CoT
In terms of inference efficiency, FlyThinker delivers much faster inference than reasoning-based methods and nearly matches SFT latency. During decoding, Fly FlyThinker achieves near- SFT latency by paralleling reasoning and token prediction: while the Generator predicts the next token, the Reasoner simultaneously prepares reasoning for the next step. This staggered, parallel design removes the sequential bottleneck inherent in CoT and Coconut. As a result, Fly Thynkee delivers teken-level reasoning with negligible latency overhead.
Obs 2:
FlyThinker
10
40 Inference time (s/sample)
Figure 3: Training and inference effi ciency of FlyThinker. The formal re- sults are detailed in the Appendix N
7

---

Published as conference paper ICLR 2026
(METEOR)
Topic
377
034
CxT
OCT
022
0.2
6.000
8:50
0:50
50-100
100-200
200-300
641
0:50
50-100
100-200
200-300
648
0:50
50-100
100-200
200-300
Generation Stop
Generation Stop
Generation Stop
Figure 4: Dosition-sensitivv evaluation across three datasets. Each generated sequence is divided into four segments by token position. We report METEOR here, with results on other metrics provided in Appendix
FlyThinker (R-3B)
FlyThinker (R-1.5B)
FlyThinker (R-0.5B)
ROUGE-1
ROUGE-L
ROUGE-1
ROUGE-L
BIEU
METEOR
BIEU
METEOR
BIEU
METEOR
Product Review
Abstract Generation
Topic Writing
Figure 5: Ablation study Reasoner scale. We compare FlyThinker with different Reasoner sizes (3B, 1.5B, 0.5B) on three datasets. The full experimental results are provided in the table of Appendix O
4.4 POSITION- SENIITIVE EVALUATION FLYTHINKER (RQ3)
To better understand Fly Thinker'''s impact on personalized long-form generation, analyze how generation quality evolves across token positions. We segment each generated sequence into contiguous chunks based on token order and compute personalization quality for each segment, enabling fine-grained view of how performance changes as the output grows longer. Figure 4 reports the results the METEOR metric, while results on other metrics are provided in the Appendix
Obs 1: Personalized generation quality degrades in later segments for all baselines. As shown in Fig- ure methods such as SFT, CoT, and Coconut exhibit a clear downward trend ROUGE and BLEU scores token positions increase, reflecting the well-known "context drift" problem in long-form generation: mod- els gradually lose alignment with user preferences as the text becomes longer.
Obs 2: FlyThie Thinker effectively mitigates quality degradation and sustains personalization. In the [100, 200] and [200, 300] token ranges, FlyThinker consistently outperforms all baselines by a substantial margin, demonstrating its ability to preserve personalization quality even in the most challenging later segments. We attribute this improvement to FlyThinker's on-the-fly latent reasoning, which performs step-wise context- aware reasoning and continuously refreshes personalized cues from earlier context. This adaptive mech- anism provides stronger guidance for later token predictions, mitigating context drift and yielding more faithful, user-aligned outputs.
4.5 ABLATION STUDY (RQ4)
We conduct comprehensive ablation study examine the design choices of FlyThinker, focusing three factors: the scale of the Reasoner, the effect the weighting parameter A. An additional study about the position add latent reasoning tokens is provided in Appendix G
8

---

Published as a conference paper ICLR 2026
Product Review
Abstract Generation
Topic Writing
Normalized Score
Normaltzwd Score
Normalized Score
ROUGE-1 ROUGEEL BLEU
ROUGE-1
ROUGE-1
ROUGE-L L
ROUGE
BLEU
0.2
1.5
Figure 6: Sensitivity analysis of the weighting parameter À on three datasets. Moderate values (0.5-2) yield the best overall performance. The full experimental results are provided the table of Appendix
(1) Analysis of Reasoner Scale. Section showed that reducing the size of the Reasoner significantly improves training efficiency. Here, we investigate whether such downsizing impacts generation quality. We evaluate FlyThinker using Biven2.5-3B-Insrrule Reasoner backbones, and report results in Figure S Two key observations emerge:
Owen2.5-115B-Instiie
the and (wee2.5-00.BB-Instee
Obs 1: Moderate downsizing preserves generation quality. When reducing the latent reasoner size from 3B to 1.5B, Fly Thinker achieves nearly identical performance across ROUGE-I/L. BLEU, and METEOR. This demonstrates that significant efficiency gains can be achieved without sacrificing output quality. indicating a favorable cost-performance trade-off.
Obs 2: Aggressive downsizing leads to performance degradation. Using 0.5B Reasoner results in clear drops in ROUGE-L and BLEU, suggesting that overly small Reasoners lack the capacity to provide suffi ciently rich reasoning signals for high-quality generation.
(2) Analysis for the weighting parameter X. We analyze the sensitivity of FlyThinker to the hyperparam- eter X in Eq. 6 which controls the contribution of latent reasoning. We sweep À E {0, 0.2, 0.5, 1, 1.5, 2, and report normalized results across all metrics in Figure [6] We make the following observations: 1) a moderate weighting strikes the best balance between leveraging reasoning signals and preserving stable generation. However, extreme À values degrade performance. Very small À suppresses the influence of la- tent reasoning, underutilizing reasoning capacity, whereas overly large À destabilizes the reasoning process and lowers overall quality. 2) When À varies within [0.2, 2.0]. individual metrics fluctuate slightly, but all results remain above the SFT baseline, indicating that our method's 's effectiveness is relatively stable within this range.
5 RELATED WORK
Personalized Long-Form Generation. As the demand for complex content-such as customized reports, creative narratives, and personalized analyses continues to grow, personalized fong-form generation has emerged as a critical research frontier (Salemi et al., 2025b Kumar et al., 2024 Salemi al., 2025a Salemi & Zaaanni 2025). Unlike short-text generation, long- form tasks require LLMs to maintain consistency and personalization quality over extended sequences. This requirement often challenged by the context drift problem, in which LLMs gradually lose alignment with user preferences as the text length increases (Kumar
et al. 2024 Salemi et al. 2025b).
The recent benchmark LongLaMP (Kmmar et al., 2024) has been proposed to systematically evaluate LLMs' capabilities in personalized long-form generation. To address quality degradation, some approaches incor- porate Retrieval-Augmentt
Generation (RAG), which injects personalized signals into the LLMs' con- text (Mysore et al.) 2023a Kumar et al.) 2024). REST-PG to improve the LLM's ability to leverage personal data more effectively throughout the generation process (Salemi et al. 2025b).
9

---

Published as a conference paper ICLR 2026
Despite these advances, a critical gap remains: existing methods struggle reason over implicit user prefer- ences and dynamically adapt to evolving contexts in complex, real-world long-form scenarios. FlyThinker directly addresses this limitation by introducing a dynamic, token-level reasoning mechanism that continu- ously integrates personalized signals during generation. By enabling fine-grained, adaptive personalization rather than relying on static conditioning strategies, FlyThinker achieves deeper contextual alignment and more robust personalization across extended outputs.
Reasoning- Generation, Reasoning-enhanne LLMs guide the production of high-quality responses. Widely adopted paradigms such explicit rea- soning (Wei et al. 2022) and latent reasoning (Hao et al., 2024) encourage LLMs to articulate intermediate reasoning steps. These reasoning paradigms have been adapted for personalization to capture implicit user preferences. REST-PG (Salemi et al.] 2025b) employs a mhinktthen-geneaat paradigm in which reasoning paths over personal context are generated via self-training to improve alignment. R2P (Luo et al. (2225a) uti- lizes a hierarchical reasoning template to guide large reasoning models in a think-then-generen for structured personalized outputs. RPM (Kim al. 2025b) introduces personalization for black-box LLMs by constructing structured, user-specific reasoning paths and retrieving reasoning reasoning-alignedd examples prior to producing the final response. TagPR (Jin al. 2025) enhances personalization reasoning via semantically tagged reasoning chains and reinforcement learning over personalization reward model. AlignXplore (Li jtt al., 2025a) infers personalized preferences from behavioral signals through extended inductive reasoning chains before downstream generation.
generation leverages the reasoning capabilities of
framework
The aforementioned methods primarily rely on a static, one-shot reasoning process, in which the LLM performs reasoning once before generating the entire response. While effective for short-form tasks (Luo jtt al., 2025a Kim et al., 2025b Jin et al. 2025 Li et al. 2025a). this think-then then-then-geeerate paradigm often struggles with long-fof fong-formppe personalized generation because fails to adapt to evolving contexts and dynamic preference shifts as the content unfolds. Furthermore, the sequential separation of reasoning and generation introduces significant latency, limiting real-time applicability (Salemi al. 2025b).
In contrast, by employing a parallel architecture that synchronizes a reasoning model with a generation model, our proposed FlyThinker enables on-the-fly reasoning that evolves dynamically alongside the re- sponse. This approach not only mitigates the training and inference efficiency bottlenecks inherent in se- quential paradigms but also significantly enhances the L.M''s capacity for fine-grained, dynamic reasoning over user preferences, resulting in superior personalization performance in complex long-form tasks.
6 CONCLUSION
In this work, we introduced FlyThinker, an efficient framework for LLM personalization built on a think- while-generating paradigm. Unlike prior methods that rely on static reasoning, FlyThinker dynamically integrates token-level latent reasoning into the generation process, enabling models to better capture evoly- ing user preferences during long-form text creation. The framework maintains parallelism in both training and inference, addressing the efficiency bottlenecks inherent in traditional massoning-augmentn generation. Experiments show clear gains in both personalization quality and efficiency over strong baselines.
ETHICS STATEMENT
This work complies with the ICLR Code of Ethics. No human or animal subjects were involved. All datasets (Product Review, Abstract Generation, Topic Writing) were used in accordance with their usage guidelines, without privacy violations or personally identifiable information. We are committed to maintaining trans- parency, faimess, and integrity in all aspects of this research.
10

---

Published as conference paper ICLR 2026
REPRODUCIBILITY STATEMENT
We have made every effort to ensure the reproducibility of our results. The experimental setup is detailed in Section 4.1 and Appendix C We also provide pseudocode for both training and inference (see Appendix [B) to assist in reproducing our experiments. Moreover, the datasets in LongLamp are publicly available, en- suring consistent and reproducible evaluation. We believe these measures will enable other researchers to replicate our work and advance the field.
ACKNOWLEDGMENTS
This work is supported by the National Natural Science Foundation of China (62525211) and the advanced computing resources provided by the Supercomputing Center of the USTC.
REFERENCES
Ruizhe Chen, Xiaotian Zhang, Meng Luo, Wenhao Chai, and Zuozhu Liu. PAD: personalized alignment of Ilms at decoding-tines time. In ICLR. OpenReview.net, 2025.
Jian Guan, Junfei Wu, Jia-Nan Li, Chuanqi Cheng, and Wei Wu. A survey on personalized Alignment
The missing piece for large language models in real-world applications. In Findings of the Association for Computational Linguistics: ACL 2025, pp. 5313-5333, Vienna, Austria, July 2025. Association for Computational Linguistics. ISBN
Shibo Hao, Sainbayar Sukhbaatar, DiJia Su, Xian Li, Zhiting Hu, Jason Weston, and Yuandong Tian. Train- ing large language models to reason in continuous latent space. CoRR, abs/2412.06766 2024.
Jerry Zhi-Yon Yang He, Sashrika Pandey, Mariah L. Schrum, and Anca D. Dragan. Context steering: Controllable personalization at inference time. In ICLR. OpenReview.net, 2025.
Song Jin, Juntian Zhang, Yong Liu, Xun Zhang, Yufei Zhang, Fei Jiang, Guojun Yin, Wei Lin, and Rui Yan. Tagging the thought: Unlocking personalization reasoning via reinforcement learning. ArXiv, abs/2509.23 140, 2025. doi: 14485500axxi.2500.23311
Jieyong Kim, Tongyoung Kim, Soojin Yoon, Jaehyung Kim, and Dongha Lee. Lims think, but not in your flow: Reasoning-leve personalization for black-box large language models. CoRR, abs/2505.21082, 2025a.
Jieyong Kim, Tongyoung Kim, Soonjin Yoon, Jaehyung Kim, and Dongha Lee. Rpm: Reasoning- personalization for box large language models. 2025b.
Minbeom Kim, Kang-il Lee, Seongho Joo, Hwaran Lee, and Kyomin Jung. Drift: Decoding-time personal- ized alignments with implicit user preferences. CoRR, 0bb/22502.14289) 2025c.
Ishita Kumar, Snigdha Viswanathan, Sushrita Yerra, Alireza Salemi, Ryan A. Rossi, Franck Dernoncourt Hanich Deilamsalchy, Xiang Chen, Ruiyi Zhang, Shubham Agarwal, Nedim Lipka, and Hamed Zamani. Longlamp: A benchmark for personalized long-form text generation. CoRR, abs/2407.1101k 2024.
Yibin Lei, Liang Ding, Yu Cao, Changtong Zan, Andrew Yates, and Dacheng Tao. Unsupervised dense retrieval with relevance-aware contrastive pre-training. In ACL (Findings), pp. 10932-10940. Association for Computational Linguistics, 2023.
Jia-Nan Li, Jian Guan, Wei Wu, and Rui Yan. Extended inductive reasoning for personalized preference inference from behavioral signals. CoRR, abs/2505.18071. 2025a.
11

---

Published as a conference paper ICLR 2026
Jia-Nan Li, Jian Guan, Wei Wu, and Rui Yan. Extended inductive reasoning for personalized preference inference from behavioral signals. CoRR, abs/2505.18071 2025b.
Xinyu Li. Zachary C. Lipton, and Liu Leqi. Personalized language modeling from personalized human feedback. CoRR, abs/2402.051333, 2024.
Sichun Luo, Guanzhi Deng, Jian Xu, Xiaojie Zhang, Hanxu Hou, and Linqi Song. Reasoning meets per- sonalization: Unleashing the potential of large reasoning model for personalized generation. ArXiv, abs/2505. 17571, 2025a. doi: 10.455500axxv.2255..77777
Sichun Luo, Guanzhi Deng, Jian Xu, Xiaojie Zhang, Hanxu Hou, and Linqi Song. Reasoning meets per- sonalization: Unleashing the potential of large reasoning model for personalized generation. CoRR, 10/22505.15711. 2025b.
Sheshera Mysore, Zhuoran Lu, Mengting Wan, Longqi Yang, Steve Menezes, Tina Baghace, Em- manuel Barajas Gonzalez, Jennifer Neville, and Tara Safavi. Pearl: Personalizing large language model writing assistants with seneration-calibratee retrievers. ArXiv, abs/2311.09180, 2023a. doi:
Sheshera Mysore, Zhuoran Lu, Mengting Wan, Longqi Yang. Steve Menezes, Tina Baghaee, Em- manuel Barajas Gonzalez, Jennifer Neville, and Tara Safavi. PEARL: personalizing large language model writing assistants with generation-calibrati retrievers. CoRR, abs/2311.00118. 2023b.
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730-27744 2022.
Hongjin Qian, Peitian Zhang, Zheng Liu, Kelong Mao, and Zhicheng Dou. Memorag: Moving towards next-gen RAG via memory-inspired knowledge discovery. CoRR, abs/2409 2024.
Yilun Qiu, Tianhao Shi, Xiaoyan Zhao, Fengbin Zhu, Yang Zhang, and Fuli Feng. Latent inter-user differ- ence modeling for IIm personalization. In Proceedings of the 2025 Conference on Empirical Methods Natural Language Processing. pp. 10610-10622 10628, 2025a.
Yilun Qiu, Xiaoyan Zhao, Yang Zhang, Yimeng Bai, Wenjie Wang, Hong Cheng. Fuli Feng, and Tat-Seng Chua. Measuring what makes you unique: Difference-aware user modeling for enhancing Ilm personal- ization. In Findings of the Association for Computational Linguistics: ACL 2025, July 2025b.
Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model secretly reward model. Advances in neural information processing systems, 36:53728-53741, 2023.
Alireza Salemi and Hamed Zamani. Lamp-qa: A benchmark for personalized long-form question answering. In EMNLP, pp. 1139-1159. Association for Computational Linguistics, 2025.
Alireza Salemi, Surya Kallumadi, and Hamed Zamani. Optimization methods for personalizing large lan- guage models through retrieval augmentation. In SIGIR, pp. 752-762. ACM, 2024.
Alireza Salemi, Julian Killingback, and Hamed Zamani. Expert: Effective and explainable evaluation of personalized long-form text generation. In ACL (Findings). volume ACL 2025 of Findings of ACL, pp- 17516-17532. Association for Computational Linguistics, 2025a.
Alireza Salemi, Cheng Li, Mingyang Zhang, Qiaozhu Mei, Weize Kong, Tao Chen, Zhuowan Li, Michael Bendersky, and Hamed Zamani. Reasoning-enhanee self-training for long-form personalized text gener- ation. CoRR, abs/2501. 004160 2025b.
12

---

Published as a conference paper ICLR 2026
Alireza Salemi, Cheng Li, Mingyang Zhang, Qiaozhu Mei, Weize Kong, Tao Chen, Zhuowan Li, Michael Bendersky, and Hamed Zamani. Reasoning-enhanced self-training for long-form personalized text gener- ation. CoRR, abs/2501 .04100,1760. 2025c.
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy opti- mization algorithms. arXiv preprint arXiv:1707.06347 2017.
Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. Hybridflow: A flexible and efficient rlhf framework. arXiv preprint arXiv: 2409.19255 2024.
Wentao Shi, Xiangnan He, Yang Zhang, Chongming Gao, Xinyue Li, Jizhi Zhang, Qifan Wang, and Fuli Feng. Large language models are learnable planners for long-term recommendation. In SIGIR, pp. 1893- 1903. ACM, 2024.
Taylor Sorensen, Jared Moore, Jillian Fisher, Mitchell L. Gordon, Niloofar Mireshghallah, Christo- pher Michael Rytting, Andre Ye, Liwei Jiang, Ximing Lu, Nouha Dziri, Tim Althoff, and Yejin Choi. Position: A roadmap to pluralistic alignment. In ICML. OpenReview.net, 2024.
Jiakai Tang, Sunhao Dai, Teng Shi, Jun Xu, Xu Chen, Wen Chen, Wu Jian, and Yuning Jiang. Think before recommend: Unleashing the latent reasoning power for sequential recommendation. CoRR, abs/2503.22675, 2025.
Chengbing Wang, Yang Zhang, Zhicheng Wang, Tianhao Shi, Keqin Bao, Fuli Feng. and Tat-Seng Chua. Decoding in latent spaces for efficient inference in Ilm-based recommendation. In Findings of the Associ- foronfor Computational Linguistics: EMNLP 2025, pp. 7591-7603, 2025a.
Chengbing Wang, Yang Zhang, Fengbin Zhu, Jizhi Zhang, Tianhao Shi, and Fuli Feng. Leveraging memory retrieval to enhance llm-based generative recommendation. In Companion Proceedings of the ACM on Web Conference 2025, pp. 1346-1350, 2025b.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoe V. Le, and Denny Zhou. Shain-of-thougee prompting elicits reasoning in large language models. In NeurIPS, 2022.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, and Jamie Brew. Huggingface 's transformers: State- of-the-art natural language processing. CoRR, abs/1910.03371, 2019.
Di Wu, Hongwei Wang, Wenhao Yu, Yuwei Zhang, Kai- Kai-Wei Chang, and Dong Yu. Longmemeval: Bench- marking chat assistants on long-term interactive memory. In ICLR. OpenReview.net, 2025.
Zhouhang Xie, Junda Wu, Yiran Shen, Yu Xia, Xintong Li, Aaron Chang, Ryan A. Rossi, Sachin Kumar, Bodhisattwa Prasad Majumder, Jingbo Shang, Prithviraj Ammanabrolu, and Julian J. McAuley. A survey on personalized and pluralistic preference aligmment in large language models. CoRR, abs/2504.07070, 2025.
An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang. Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang. Keming Lu, Keqin Bao, Kexin Yang Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li, Tingyu Xia, Xingzhang Ren, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yu Wan, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, and Zihan Qiu. Qwen2.5 technical report. CoRR, a5s/2412.15115, 2024.
13

---

Published as a conference paper ICLR 2026
Xinliang Frederick Zhang, Nick Beauchamp, and Lu Wang. PRIME: large language model personalization with cognitive memory and thought processes. CoRR, abs/2507.046077, 2025a.
Yang Zhang, Wenxin Xu, Xiaoyan Zhao, Wenjie Wang, Fuli Feng, Xiangnan He, and Tat-Seng Chua. Rein- forced latent reasoning for llm-based recommendation. CoRR, abs/2505. 19092, 2025b.
Zeyu Zhang, Yang Zhang, Haoran Tan, Rui Li, and Xu Chen. Explicit vs implicit memory: Exploring multi-hop complex reasoning over personalized information. arXiv preprint arXiv:2508. 13250, 2025c.
Zhaowei Zhang, Fengshuo Bai, Qizhi Chen, Chengdong Ma, Mingzhi Wang, Haoran Sun, Zilong Zheng, and Yaodong Yang. Amulet: Realignment during test time for personalized preference adaptation of llms. In ICLR. OpenReview.net, 2025d.
Siyan Zhao, Mingyi Hong. Yang Liu, Devamanyu Hazarika, and Kaixiang Lin. Do Ilms recognize your preferences? evaluating personalized preference following in llms. In ICLR. OpenReview.net, 2025a.
Xiaoyan Zhao, Ming Yan, Yilun Qiu, Haoting Ni, Yang Zhang, Fuli Feng, Hong Cheng, and Tat-Seng Chua. Steerx: Disentangled steering for Ilm personalization. arXiv preprint arXiv:2510.22256, 2025b.
Xiaoyan Zhao, Juntao You. Yang Zhang, Wenjie Wang, Hong Cheng, Fuli Feng, See-Kiong Ng, and Tat-Seng Chua. Nextquill: Causal preference modeling for enhancing LLM personalization. CoRR, abs/2506.02368,, 2025c.
Ziyi Zhao, Chongming Gao, Yang Zhang, Haoyan Liu, Weinan Gan, Huifeng Guo, Yong Liu, and Fuli Feng. Don't start over: A cost-effective framework for migrating personalized prompts between 1lms. The 40th Annual AAAI Conference Artificial Intelligence (AAAI 2026), 2026.
Zhi Zheng, Wenshuo Chao, Zhaopeng Qiu, Hengshu Zhu, and Hui Xiong. Harnessing large language models for text-rich sequential recommendation. In wwW, pp. 3207-3216. ACM, 2024.
Thomas P. Zollo, Andrew Wei Tung Siah, Naimeng Ye, Ang Li, and Hongseok Namkoong. Personallim: Tailoring ilms individual preferences. ICLR. OpenReview.net, 2025.
14

---

Published as conference paper ICLR 2026
A LLM USAGE
Large Language Models (LLMs) were used to aid the writing and polishing of the manuscript. Specifi- cally, we used an LLM assist in sentence rephrasing, grammar checking, and enhancing readability and flow. The LLM was not involved in the ideation, methodology, or data analysis. All research concepts, ideas, and analyses were developed and conducted by the authors.
The authors take full responsibility for the manuscript's 's content, including any text generated or polished by the LLM. All use complied with ethical standards, ensuring no plagiarism scientific misconduct.
B ALGORITHM PSEUDOCODE
Algorithm 1 Training FlyThinker
Algorithm Inference with FlyThinker
Require: Generator Gại Reasoner Rø; token em- bedding e(-); fusion coef. x; batch sample se-
Require: Trained Gø, Ry; max steps M; token list is derived from the context in the same way as during training.
quences zf:.0 T]
1: (One-pass latent thoughts)
1: for step t = 1,2. ..,.....N
M do
2: r[: + Re (=[::0
(Parallel) (L) Ro(x<ct-1)
3: (Build enhanced embeddings)
3: (Parallel) P. + GG(ect-1,
4: els, 1:
e<<[;,, 1: T)) + A- :::
Sample y/ P., st V
5: (Computing loss)
5: (Prepare for next step)
6: P. + Cs(el:::7))
Ft f y/
7: CE W+)
7: et e(x=)) + x- (L)
8: Update 0, Φ by backprop on
end for
9: return trained Rg. G
9:
return full generation sequence s
C IMPLEMENTATION DETAILS
our study, we fine-tune the Qwen2.5-3B 2--.5--BBInnstteee and Qwen2.5-7B. Instrucct model to validate our ap- proach. All experiments, including Fly Thinker and baseline models, conducted four NVIDIA A100 GPUs. The LLMs are loaded via HuggingFace Transformers (Wolf et al. (2019). and the training is imple- mented in Python 3.10 using the Verl training framework (Sheng al. 2024).
D DETAILS OF BASELINE METHODS
We provide detailed deseription of the baselines considered in comparisons:
Non-pers:
Use the original LLM and the user query directly, without any personalization alignment, as the default generation method.
RAG: Supply the original L.M with user-specific information retrieved from interaction history and inject it into the input context, enabling personalization without training.
CoS: Compute contextual influence by contrasting output probabilities with and without context, and adjust it with parameter À to control personalization strength.
15

---

Published as conference paper ICLR 2026
SFT: Fine-tune the LLM with user interaction history, queries, and ground-truth responses, aligning param- eters toward personalized outputs.
LLM-TRSR: Extract key information from the user's interaction history, integrate the resulting summary into prompts, and fine-tune the LLM to improve personalization.
NextQuill:
Fine-tune the LLM by modeling causal preference effects of user history on token prediction, and align the LLM's internal preferences with those observed in real data, focusing on sifference-bearing tokens.
Fine-tune the LLM with explicit reasoning paths under a thhink-then-generate paradigm, generating reasoning steps before producing the final answer.
CoT:
Fine-tune the LLM by generating latent representations autoregressively as an latent reasoning path, and produce the final answer conditioned on them.
Coconut:
COMPREHENSIVE EVALUATION RESULTS
This section further incorporates the BERTScore metric and the average relative improvement, providing a more comprehensive demonstration of FlyThinker's advantages. The results are presented in Table T2
F POSITION-SENSITVV EVALUATION
This section presents additional results on position-sensitiie evaluation (see Section As illustrated in Figure we can observe that Fly PlyThinker effectively alleviates quality degradation and consistently preserves personalization across all datasets and metrics.
G EFFECT OF THE POSITION OF LATENT REASONING AUGMENTATION.
When implementing FlyThinker, we also add latent reasoning tokens to the task prompt and query parts of the model inputs to ensure alignment of the representation space across the entire sequence. We further study where to apply latent reasoning augmentation: only on the input tokens (Input-only). only on the output tokens (Output- only), jointly both as in FlyThinker (global enhancement). Results shown in Table 2
Obs I: Global enhancement yields the most consistent improvements. FlyThinker, which applies reason- ing augmentation at both input and output positions, achieves the highest overall performance across tasks and metrics, with especially notable gains ROUGE and BLEU. This indicates that combining input- and output-side reasoning provides complementary benefits, resulting in more faithful and informative genera- tion.
Obs 2: Input-only enhancement increases diversity but harms alignment. Injecting reasoning solely at the input stage makes the model more "free-form," producing diverse paraphrases but reducing lexical and structural alignment with reference texts, which leads to lower ROUGE/BLEU scores.
Obs 3: Output-only enhancement underperforms due to weak contextual grounding. Reasoning injected exclusively at the output stage lacks sufficient conditioning on the user profile and query, resulting in con- sistently weaker performance across tasks.
H LATENT REASONING TRAJECTORY: VISUALIZATION AND ANALYSIS
16

---

Published as a conference paper ICLR 2026
Methods (
Product Review
Abstract Gemeration
Topic Writing
ROUGE-L
ROUGE-L
ROUGE-L
METEOR
ROUGE.L
ROUGE-L
SET
0.3551
0.1536
3,9112
0.2309
0.3602
0.2062
5.8189
0.2392
0.2916
0.1345
3.8876
0.2013
Input-onty
0.3559
0.1471
0.3036
0.2783
0.1195
2.7058
Output-only 0.3417
0.1513
3.3729
0.2123
0.3586
0.2064
5,7380
0.2355
0.2740
0.1314
2.9972
0.1759
FlyThiuker
Table 2: Ablation study on the position of latent reasoning augmentation. We compare our FlyThinker with reasoning applied only to input tokens (Input-only) or only to output tokens (Output-only).
Interpretablity is common challenge for latent reason- ing. Draw inspiration from your comments, have con- ducted an visualization study. Under a fixed task, we se- lected five users and extracted one sample from each of them. We compared the Reasoner latent reasoning tokens for these samples and visualized them using t-SNE. We then used linear regression to estimate the direction from early to later reasoning tokens, representing each user's reasoning trajectory. The corresponding Figure 7 for this experiment illustrates that different users show different trajectory directions.
aurrle 2 arrgle
To examine whether these trajectories relate to the pref- erence, used GPT-5.1 to compare the samples. We selected the two samples whose users had the most sim- ilar trajectory directions (sample0 and sample4) and the one farthest away (sample2) in Figure 7 GPT-5.1 judged similar to sampleO in paragraph
Figure 7: t-SNE visualization and trajectory gression vectors of latent reasoning tokens under two experimental configurations.
sample4 to be structure, narrative style, and analytical approach. Sam- ple2 was the least similar. These results show that different users induce distinct reasoning trajectories, and that closer trajectories correspond to more similar writing styles and semantic structures, etc. This suggests that latent reasoning tokens may contain interpretable semantic information useful for user clustering and preference visualization.
I SEASONER-GENERAN RELATIVE SIZE ANALYSIS IN FLYTHINKER
Understanding the relative sizing between the Reasoner and the Generator is essential for deploying Fly- Thinker effectively in practical scenarios. In this section, conducted systematic analysis using 3B and 7B Generators, and tested Reasoners with sizes of 0.5B and 1.5B. As shown in Table with a 7B Generator, 0.5B Reasoner can already deliver performance improvements, while with 3B Generator, the Reasoner needs to reach 1.5B to achieve meaningful and consistent gains across metrics. These results suggest that a generally small Reasoner (around 1.5B) can be effective. However, there no clear proportional size requirement between the Reasoner and the Generator - stronger Generator places fewer demands on the Reasoner. Naturally, a larger Reasoner typically delivers better performance.
J ROBUSTNESS OF FLYTHINKER UNDER VARYING USER HISTORY LENGTHS
In this section, analyzed Fly Thynkek'''s performance under different history lengths. Specifically, tested three settings: two history examples (= 1, 750 tokens), three history examples (+= 2, 250 tokens), and four history examples (== 2, 750 tokens), comparing our method with the SFT method. The results, summarized
17

---

Published as conference paper ICLR 2026
Table Evaluation of Fly Thinkeeer under different Reasoner Reasoner-Generator size pairings, comparing 0.5B/1.5B Reasoners with 3B/7B Generators.
Product Review SFT
ROUGE-1
ROUGE-L
BLEU
METEOR
BERT-SCORE
ROUGE-1
ROUGE-L
BLEU
METEOR
BERT-SCORE
0.3551
D.1556
39112
0.2309
0.7681
0.3512
0.1560
3,8953
0.2224
0.7674
ThiThinee (R-1.5B)
5,1072
Fly Thinker (R=0.5B)
0.3556
0.1531
3,9072
02366
0.7692
0,3712
0.1566
1,8035
0.2737
0.7813
Abstract Generation
ROUGE-1
ROUGE-L
BLEU
METEOR
BERT-SCORE
ROUGE-1
ROUGE-L
BLEU
METEOR
BERT-SCORE
SFT
0,3602
0.2062
5.8189
0.2392
0.8812
0,3645
0.2122
5.9661
0.2398
0.8854
FlyThinker (R=1.58)
6.2690
Fly Thinkeeer (R-4.55B)
0.3603
0,1915
5.5334
0.2985
(18883
0,3665
0.1949
5,9182
0.3112
0.8903
Tople Writing
ROUGE-1
ROUGE-L
BLEU
METEOR
BERT-SCORE
ROUGE-1
ROUGE-L
BLEU
METEOR
BERT-SCORE
0.1351
Fly Thinker (R=1.5B)
0.315
0.1348
3,8004
0.2450
0.7154
0.3076
0.1325
36672
0.2564
0.7249
Fly Thinker (R=0.58)
29600
0.1297
3,2908
0.2450
0.7158
0,3023
0,1319
3.1868
0.2566
0.2215
Table 4: Robustness of FlyThinker to different history lengths. Increasing the number of history ples yields mild performance gains while maintaining a clear advantage over SFT.
[ #his token= 1750]
[fhis=3, #his token==2250]
[#his=4, #his token==27501
Product Review
SFT
FlyThinker
SFT
FlyThinker
SFT
FlyThinker
ROUGE-1
0.3551
0.3663
0.3619
0.3705
0.3635
0.3719
ROUGE-L
0.1536
0.1560
0.1568
0.1581
0.1585
0.1619
BLEU
3,9112
4.3620
4.4810
4.5020
5.1703
5,0671
METEOR
0.2309
0.2489
0.2394
0.2490
0.2437
0.2511
BERT-SCORE
0.7681
0.7755
0.7711
0.7762
0.7708
0.7772
Abstract Generation
SFT
FlyThinker
SFT
FlyThinker
SFT
FlyThinker
ROUGE-1
0.3602
0.3716
0.3648
0.3786
0.3651
0.3795
ROUGE-L
0.2062
0.2090
0.2095
0.2035
0.2102
0.2047
BLEU
5,8189
6,3383
5,9950
6,8119
6,0234
7.0848
METEOR
0.2392
0.2549
0.2422
0.2903
0.2432
0.2882
BERT-SCORE
0,8812
0,8858
0.8845
0.891
0.8852
0.891
Topic Writing
SFT
FlyThinker
SFT
FlyThinker
SFT
FlyThinker
ROUGE-1
0.2916
0.3139
0.2963
0.3153
0.298
0.3185
ROUGE-L
0.1345
0.1362
0.1360
0.1376
0.1359
0.1401
BLEU
3,8876
4.0606
4.8021
4.4026
4.7817
4.8841
METEOR
0.2013
0.2409
0.2084
0.2384
0.2126
0.2395
BERT-SCORE
0.7067
0.7205
0.7092
0.7197
0.7091
0.7204
in the Table 4 show that performance increases the context length grows across most metrics, while FlyThinker consistently outperforms SFT in all tested settings. These findings suggest that our method relatively insensitive to user history length.
18

---

Published as conference paper ICLR 2026
K LLM-BASED EVALUATION OF FLYTHINKER
In this section, we added an LLM-based evaluation to provide a more comprehensive assessment of person alized generation quality. Specifically, use GPT-4o as an evaluator and compare the outputs produced by Fly Thinker with those from the strongest baseline, SFT, following a structured evaluation rubric 9 As shown in the Table 5 FlyThinker consistently achieves higher scores across all datasets under the L.MM- based evaluation. These findings indicate that FlyThinker's personalization gains are reflected not only in traditional automatic metrics but also in the LLM judge's s assessments.
Table 5: LLM-Based Evaluation of FlyThinker.
GPT-4o-mini evaluation
Product Review
Abstract Generation
Topic Writing
SFT
0.3695
0.5718 0.5955
0.5718
FlyThinker
0.3816
0.5955
L GENERATOR SCALING STUDY
We conduct study on generator scaling with Qwenn.5--7B-Instruutt aiming evaluate scalability. For this purpose, we adopt a simple yet strong Saseline-SFT---1 our reference baseline. As shown in Table 6 FlyThinker-7B (R=1.5B) consistently achieves better overall performance across the three tasks. In partic- ular, FlyThinker demonstrates clear advantages on BLEUMMETEOR, indicating improved lexical precision and semantic alignment. Although SFT-7B remains competitive and sometimes stronger on ROUGE-L. (e-g.. in abstract generation and topic writing), Fly Thinker provides more balanced improvements across multiple evaluation metrics. These results highlight the effectiveness and robustness of FlyThinker.
Table 6: The evaluation results Qwwnn..577-Instruu.. The results show that FlyThinker consistently outperforms the SFT baseline across all datasets.
Qwen2.5-7B-Instruue
Product Review
ROUGE-1
ROUGE-L
BLEU
METEOR
BERT-SCORE
SFT
0.3512
0.1560
3.8953
0.2224
0.7674
FlyThinker (R=1.5B)
0.3738
0.1578
5.1072
0.2715
0.7846
Abstract Generation
ROUGE-1
ROUGE-L
BLEU
METEOR
BERT-SCORE
SFT
0.3645
0.2122
5,9661
0,2398
0.8854
Fly Thinker (R=1.5B)
0.3708
0.1971
6.2732
0.3010
0.8910
Topic Writing
ROUGE-1
ROUGE-L
BLEU
METEOR
BERT-SCORE
SFT
0.2849
0.1351
3,4850
0. 0.1883
0.7071
FlyThinker (R=1.5B)
0.3076
0.1325
3.6572
0.2564
0.7249
M CROSS-ARCHIIECCTRR VALIDATION ON ADDITIONAL MODELS
On the Gemma-7B-it model, FlyThinker also shows consistent performance gains (ref. Table 7 Together with the results on Qwen2.5-3B and 7B, these findings indicate that our approach demonstrates strong gen- eralization across different model architectures.
19

---

Published as conference paper ICLR 2026
Table 7: Cross cross-architecturr evaluation on Gemma-7B-it demonstrating that FlyThinker achieves stable per- formance gains over SFT, confirming its effecttiveness across different model families.
Gemma-7Bt
Product Review
ROUGE-1
ROUGE-L
BLEU
METEOR
BERT-SCORE
SFT
0.3285
0.1544
3,9774
0.2166
0.7559
FlyThinker (R=2B)
0.3355
0.1579
3.5646
0.2160
0.7553
Abstract Generation
ROUGE-1
ROUGE-L
BLEU
METEOR BERT-SCORE
SFT
0.3501
0.2057
6.1330
0.2404
0.8750
FlyThinker (R=2B)
0.3624
0.2062
6.8307
0.2638
0.8800
Topic Writing
ROUGE-1
ROUGE-L
BLEU
METEOR BERT-SCORE
SFT
0,2664
0.1323
3.9175
0.1938
0.6960
FlyThinker (R=2B)
0.2829
0.1385
3.6046
0.2035
0.6989
N MEMORY AND COMPUTATION ANALYSIS OF FLYTHINKER
In this section, we have included latency costs, summarized in the Table The symbols are defined as:
comprehensive evaluation of memory, computational, and time-
(0c 0R): parameters of the generator and reasoner, respectively
(N1 N2): number of text tokens reasoning tokens to generate
(Ca CR): cost of one forward pass for the generator reasoner
(fg / R): latency to generate one token by the generator reasoner
Specifically, regarding computational cost you focused: our method's ' cost is approximately (Ni Ca N2CR). while the baseline latent reasoning method incurs (N1Ca + N2Cc). The difference comes from (N2CR) versus (N2Cg). Since the reasoner is typically much smaller than the generator (e.g., 0.5B vs. 7B), method's computational cost is significantly lower than latent reasoning.
Compared to the SFT baseline (cost (NıCe)), our additional cost is (N2Cn). As (Cn < Cg), this overhead is relatively small. For instance, using a 0.5B reasoner for 7B generator with (N1 = N2 = 1000), the FLOPS are: method 15 TFLOPs, Cocount-22 228 TFLOPs. SFT 14 TFLOPs. In this scenario, our method adds only ITFLOPs computation cost while delivering significant performance improvements, compared to SFT.
Importantly, a key advantage of our method is reduced latency, as also shown the Table g
FULL EXPERIMENTAL RESULTS ON REASONER SCALE IN FIGURE 5
Figure metrics were normalized and plotted on unified scale to highlight relative differences across methods. In this section, we supplement Figure 5 with the complete numerical results (see Table
P EXPANDED SENSITIVITY ANALYSIS OF THE HYPERPARAMETER À
In this section, we report all raw (unnormalized) metrics in the Table 10 for a clear examination of the effect of the À hyperparameter in the original Figure These extended experiments reveal several insights:
20

---

Published as conference paper ICLR 2026
Table 8: Analysis of FlyThinker's memory and computation characteristics, comparing its training and inference overhead with SFT and other ceasoning-basee baselines.
Latency lo lc N2lc lc + N2lc
Method
Memory
Computation
Training stage
Inference Stage
SFT
Ocl
NiCa
Nilc
CoT
1cc
NiCa + N2Cc
Nilc + N2la
Coconut
|0cl
NiCa N2CG
Nilo + N2lc
FlyThinker
10c| |0
NiCG N2CR
lG + LR
NilG
Table 9: Complete evaluation results in Figure malized plot in the main text.
providing absolute metric values to supplement the nor-
Product Review
ROUGE-1
ROUGE-L
BLEU
METEOR
BERT-SCORE
SFT
0.3551
0.1536
3.9112
0.2309
0.7681
FlyThinker (R=3B)
0,3663
0.1560
4.3620
0.2489
0.7755
FlyThinke (R=1.5B)
0.3676
0.1554
4.2123
0.2501
0.7729
FlyThinker (R=0.5B)
0.3556
0.1531
3.8072
0.2366
0.7692
Abstract Generation
ROUGE-1
ROUGE-L
BLEU
METEOR
BERT-SCORE
SFT
0.3602
0.2062
5.8189
0.2392
0.8812
FlyThinker (R=3B)
0,3716
0.2090
6.3383
0.2549
0.8858
ThTnkee (R=1.5B)
0.3726
0.1989
6.2690
0.2910
0.8896
FlyThinker (R=0.5B)
0.3603
0.1915
5.5334
0,2985
0.8883
Topic Writing
ROUGE-1
ROUGE-L
BLEU
METEOR
BERT-SCORE
SFT
0.2916
0.1345
3.8876
0.2013
0.7067
FlyThinker (R=3B)
0.3139
0.1362
4.0606
0.2409
0.7205
Fly Thinker (R=1.5B)
0,3115
0.1348
3.8004
0.2450
0.7154
Fly Thinker (R=0.5B)
0.2960
0.1297
3.2908
0.2450
0.7158
Fine-grained tuning of À can further improve specific metrics. For example, the Abstract Generation task, A= 1.0 leads to additional improvements in BLEU, and in the Topic Writing task, x= 1.5 further boosts ROUGE-L and BLEU compared with coarser settings.
Moderate À values provide robust overall performance. The table shows that when À stays within moderate range around 1.0 neither too close to nor excessively large (e.g., approaching 5)-the the model consistently outperforms the SFT baseline. This suggests that X does not require precise tuning in practice and the method is robust to reasonable variations.
The performance fluctuations between À 0.2 and 2.0 mild and acceptable. When À varies within [0.5, 2.0], individual metrics fluctuate slightly, but all results remain above the SFT baseline, indicating that method's s effectiveness is relatively stable within this range.
21

---

Published as conference paper ICLR 2026
Table 10: Complete evaluation results in Figure 10 providing absolute metric values to supplement sensi- tivity analysis of the À hyperparameter.
A
Product Review
0
0.2
0.5
1
1.5
2
ROUGE-1
0.3551
0.3641
0.3664
0.3551
0.3569
0.3626
0.3271
ROUGE-L
0.1536
0.1554
0.1560
0.1546
0.1543
0.1550
0.1479
BLEU
3,9112
4,1088
4.3223
4.0979
4.0685
4.6599
2.7995
METEOR
0.2309
0.2419
0.2489
0.2334
0.2353
0.2529
0.1976
BERT-SCORE
0.7681
0.7723
0.7755
0.7687
0.7688
0.7740
0.7541
Abstract Generation
0
0.2
0.5
1
1.5
2
5
ROUGE-1
0.3602
0.3750
0.3716
0.3748
0.3729
0.3734
0.3540
ROUGE-L
0.2062
0.2016
0.2090
0.2068
0.2081
0.2077
0.2022
BLEU
5.8189
6.5881
6.3383
7.0687
6.7091
6.8036
5.4578
METEOR
0.2392
0.2891
0.2549
0.2718
0.2633
0.2657
0.2345
BERT-SCORE
0.8812
0.8891
0.8858
0.8871
0.8863
0.8867
0.8792
Topic Writing
0
0.2
0.5
1
1.5
2
ROUGE-1
0.2916
0.3108
0.3139
0.3050
0.3035
0.3036
0.2820
ROUGE-L
0.1345
0.1365
0.1362
0.1359
0.1371
0.1349
0.1327
BLEU
3.8876
4.1522
4.0135
4.3646
4.4095
4.0991
2.8485
METEOR
0.2013
0.2363
0.2408
0.2253
0.2203
0.2308
0.1858
BERT-SCORE 0.7067
0.7181
0.7205
0.7150
0.7156
0.7142
0.7033
Q DIFFICULTY-AWARE EVALUATION FOR PERSONALIZED TASKS
To more comprehensively assess performance within the personalized generation, we constructed a difficulty-controlled evaluation experiment.
For each test sample, we first computed multiple automatic metrics the base Instruct model (ROUGE, BLEU, METEOR, BERTSCore). After normalization and aggregation, we ranked all samples and partitioned the test set into four difficulty levels.
As shown in Table I1 FlyThinker consistently surpasses SFT across all difficulty levels, including the hardest level (HO). This demonstrates that the dynamic reasoning mechanism remains highly effective even when the personalized generation task becomes more challenging.
22

---

Published as conference paper ICLR 2026
Product Review (ROUGE-1)
Abstract Generation (ROUGE-1)
Topic Writing (ROUGE-1)
8:40
APF
8.20-
CaT
OuT
GeT
ROUGE-1
ROUGE-1
6.58
8.50
0.3
50.100
100:200
50-100
100:200
200-300
50-100
100-200
200-300
Generation Step
Generation Step
Generation Step
Product Revlow (ROUGE-L)
Abstract Generation (ROUGE-L)
Writing (ROUGE-L)
3Ff
aFF
8.14-
Grt
8.2
Rytister
ROUGE-L
OyTizee
6.10
609
8:00
0-50
50-100
100-200
200-300
0-50
50-100
100-200
200-300
0-50
50-100
100-200
200-300
Generation Step
Genaration Stop
Generation Stop
Product Review (BIEU)
Abstract Generation (BLEU)
Topic Writing (BLEU)
Cureet
Rymosker
RyTheaer
50-100 Generation Step
Step
Step
(METEOR)
Goneration (METEOR)
Toplc (METEOR)
SFf
SFF
3FT
8.100
car
624
Out
Cut
5:000
0-50
50-100
100-200
200-300
0-50
50-100
100-200
200-300
0-50
50-100
100-200
200-300
Generation Stop
Generation Stop
Generation Stop
Figure 8: Dosition-sensitive evaluation across three datasets. Each generated sequence is divided into four segments by token position.
23

---

Published as conference paper ICLR 2026
Table 11: Difficulty-awan evaluation results for personalized generation. Test samples are grouped into four difficulty levels based on normalized automatic metric scores of the base Instruct model.
H0-
Very Hard
HI
Hard
H2 Medium
H3 Easy
SFT
FlyThinker
SFT
Fly FlyThinker
SFT
FlyThinker
SFT
Fly Thinker
ROUGE-1
0.2796
0.2923
0.3224
0.3390
0.3480
0.3679
0.3818
0.3965
ROUGE-L
0.1430
0.1441
0.1588
0.1612
0.1686
0.1714
0.1917
0.1932
BLEU
2.2154
2.3545
2.5867
2.8538
3,1460
3,5514
5.1719
5,5257
METEOR
0,2017
0.2275
0,2156
0.2443
0,2225
0.2494
0.2478
0.2649
BERT-SCORE
0.7175
0.7247
0.7793
0.7899
0,8080
0.8127
0,8396
0.8490
You impartisl evaluator tasked with assessing how well L--eerrerteedd response reflects the user's personal preferences, and key points by comparing it with user's real response.
[Scoring Criteria]
[Score 1]: Completely unrelated to the user's real response. Does not follow the instruction reflect the user's personal thoughts and preferences.
[Score 2]: Weakly related to the user's real response. May mention the general topic but lacks the user's key content, tone, or insights.
[Score 3]: Partially matches user's response. Includes ideas from the but others missing, altered significantly, feel generic.
[Score 4]: Mostly matches user's real response, Covers main points but misses some personal details, tone, nuances. [Score 5]: Very similar to the user's real response, Conveys the user's core ideas, structure, and style with only minor omissions or changes.
You must assign based only the alignment between LLM-generated response and response.
Output the numerical score only no explanctions, no extra text.
[Instruction]: [the input prompt]
[User's response]: [expected outpu[]
[LLM-generated response): [generated output) [Score]:
Figure 9: Illustration of the instruction template provided GPT-4o-mini for conducting LLM-based per- sonalized evaluation.
24

---

Published as conference paper ICLR 2026
Table 12: Comprehensive comparison results with BERTScore additional metric and Average Relative Improvement (computed relative to Non-pers).
Datasets
Methods
Rouge-1
Rouge-L
BLEU
Meteor
BERTScore
Impr. (t)
Non-pers
0.2839
0.1317
1.5357
0.1783
0.7380
Tuning
RAG
0.3176
1410
3.2990
0.2309
0.7482
33.08%
free
CoS
0.2781
0.1347
2.9677
0.2189
0.7213
22.80%
Product Review
SFT
0.3551
0.1536
3.9112
0.2309
0.7681
45.99%
LLM-TRSR
0.3377
0.1450
2.3279
0.2122
0.7573
20.45%
Tuning
NextQuill
0,3311
0.1518
2.6599
0.2172
0.7580
25.92%
based
CoT
0.3466
0.1508
3.3705
0.2220
0.7587
36.68%
Coconut
0.3381
0.1496
3.3213
0.2093
0.7578
33.80%
Ours
FlyThinker
0.3663
0.1560
4.3620
0.2489
0.7755
55.24%
Non-pers
0.3276
0.1717
4.5814
0.2476
0.8616
Tuning free
RAG
0.3168
0.1616
3,4047
0.2998
0.8818
-2.28%
CoS
0.2973
0.1576
3.2591
0.3023
0.8952
-4.06%
Abstract Generation
SFT
0,3602
0,2062
5,8189
0.2392
0,8812
11.18%
LLM-TRSR
0.3583
0.2056
5.5359
0.2371
0.8819
9.61%
Tuning
NexiQuill
0.3501
0.2038
5,4388
0.2310
0.8737
7.79%
based
CoT
0,3685
0,2071
5,8538
0.2469
0.9023
13.06%
Coconul
0.3542
0.2041
5.2378
0.2298
0.8812
7.28%
Ours
FlyThinker
0.3716
0.2090
6.3383
0.2549
0,8858
15.85%
Non-pers
0,2389
0. 0.1057
1.1190
0. 0.1869
0.6986
Tuning
RAG
0.2533
0.1099
1,4339
0.2253
0.6966
11.68%
free
CoS
0,2307
0.1056
1.5426
0.2134
0.6891
9.42%
Topic Writing
SFT
0.2916
0.1345
3.8876
0.2013
0.7067
61.11%
LLM-TRSR
0,2839
0.1280
1.5552
0.1922
0.7040
16.50%
Tuning
NextQuill
0.2755
0.1269
2.1006
0.1817
0.6938
23.92%
based
CoT
0.2846
0.1304
3.0020
0.1967
0.7138
43.63%
Coconut
0,2786
0.1317
3.0736
0.1823
0.7012
42.76%
Ours
FlyThinker
0.3139
0.1362
4.0606
0.2409
0.7205
71.03%
25


Reviews:
[{"review_id": "Uq0RLwwSRT", "reviewer": "Reviewer_vYSZ", "summary": "This paper proposes FlyThinker, a “think-while-generating” framework for personalized long-form text generation. FlyThinker employs a Reasoner that generates latent token-level reasoning signals and a Generator that dynamically integrates these reasoning signals into its token-level predictions, enabling parallel training and inference. Experiments on three tasks from the LONGLAMP benchmark, including Product Review, Abstract Generation, and Topic Writing, demonstrate that FlyThinker achieves improvements in both personalization quality and generation efficiency over several baselines.", "questions": "Please refer to Weaknesses.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The “think-while-generating” paradigm is well-motivated, and the overall methodology is simple and intuitive.\n2. The evaluation is comprehensive with multiple metrics, showing the effectiveness of the proposed method over baselines.\n3. FlyThinker achieves training and inference efficiency comparable to SFT, which is a major advantage relative to existing reasoning-augmented methods that typically incur higher latency.", "weaknesses": "1. It is not clear whether the reported personalization results are based on the user-based split (testing on unseen users) or the temporal split (testing on later instances of seen users). Since these settings test different personalization abilities (cross-user vs. within-user), clarification or stratified results would make the findings more interpretable.\n\n2. While the paper ablates on Reasoner size, it is not clear how the Reasoner scales relative to the Generator, e.g., what would be the smallest Reasoner that still remains effective for different Generator sizes - would it be roughly 30%, or 50% of the Generator size? Insights into this would provide very helpful practical guidance for applying FlyThinker in real-world settings.\n\n3. Appendix H shows that adding reasoning tokens to both input and output positions yields the best performance. It is not clear how sensitive FlyThinker is to user history length, i.e., when each user has a lot of historical records with long-form generations, making the context very long. A discussion on this would strengthen the paper’s empirical insights.\n\n4. Are the learned latent reasoning tokens interpretable, or could they be used for downstream applications such as user clustering or user preference visualization? Some discussions on this would offer insights into the interpretability of the latent reasoning tokens.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "lQirMSv8AA", "reviewer": "Reviewer_uQ75", "summary": "The authors propose FlyThinker, a method for combining LLM reasoning and generation in a \"think-while-generating\" paradigm. In their approach, there are reasoner and generator models, where the reasoner is given the input and an in-progress generation, then produces a latent reasoning token that is fed to the generator to guide its response. They argue that this approach allows for efficient LLM personalization with latent reasoning.", "questions": "- Figure 5 has no values in the axes, so it is hard to tell how significant the differences in scores actually are.\n- Could the authors clarify and further fill out the results in Figure 6? The authors claim that \"Moderate values (0.5-2) yield the best overall performance\", but do not test any intermediate values between 0.5 and 2. Also, there seems to be some instability in this range, especially for abstract generation. There are no values on the y axis for this graph, so it is difficult to tell how much $\\lambda$ affects performance.\n- It would be valuable to see some examples of personalized outputs produced by FlyThinker, for example for different reasoning model sizes or $\\lambda$ parameters. This would make it more clear how exactly these factors affect the outputs of the generator.\n\nMinor typo: line 180: \"tought\" instead of \"thought\"", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "- The proposed approach seems useful for efficiently combining reasoning and generation. It provides a simple way to align to user preferences.\n- The method is flexible and can be used with models of different sizes. It is particularly convenient that a small reasoning model can be combined with a larger generation model for greater efficiency.\n- The writing and presentation of the paper are clear. The method section in particular is very easy to read and clearly lays out the method.", "weaknesses": "While the proposed approach is interesting, the evaluation experiments in their current form are not sufficiently comprehensive. \n1.  The evaluation is based only on simple automated metrics (ROUGE, BLEU, METEOR, BERT-Score). To get a full understanding of how much FlyThinker improves personalization, it would be useful to have a user study or an automatic LLM evaluation of preferred personalized outputs.\n2. The experiments are mostly limited to Qwen2.5-3B-Instruct. The small set of Qwen2.5-7B-Instruct experiments do not have any numbers on the axes, so it is difficult to tell how much FlyThinker improves performance over SFT. The experimental results would be strengthened by additional experiments with other models and filling out the Qwen2.5-7B-Instruct scores. \n3. From my understanding, compared to SFT, with this approach it is necessary to keep up to 2x the number of parameters in memory. The authors state that their method is efficient because both the reasoner and generator can perform inference simultaneously, but do not discuss the added memory required at training and inference time. Some discussion of training and inference memory requirements (in addition to the runtime results already included) would be appreciated.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "BvKM55Ya1l", "reviewer": "Reviewer_tgii", "summary": "This paper introduces FlyThinker, an efficient “think-while-generating” framework. Unlike prior “reason-then-generate” approaches, FlyThinker employs a separate reasoning model that runs in parallel to dynamically guide the generation model. The framework achieves efficiency in both training and inference, and extensive experiments demonstrate its strong effectiveness and superior performance.", "questions": "As noted in the weaknesses above, please provide more details on the actual overall computational cost and evaluate the proposed method on a broader range of challenging tasks, such as mathematical or scientific reasoning and coding benchmarks, to better demonstrate its effectiveness.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. The concept of generating reasoning tokens and response tokens simultaneously is novel and intriguing to me.\n\n2. The idea of decoupling reasoning tokens from previously generated reasoning outputs is particularly interesting, as it enables a one-pass training process and significantly improves overall efficiency.", "weaknesses": "1. The title of Figure 3 is somewhat misleading and should be revised to “Training Efficiency / Inference Efficiency.” Although the proposed method demonstrates shorter runtime, it relies on two separate models—the reasoning model and the generation model—which substantially increases memory consumption. The authors should therefore provide a comparison of the actual computational cost against other baselines to present a fair assessment.\n2. Reasoning models typically show the greatest advantages on more challenging tasks, such as mathematical or scientific reasoning (e.g., AIME24, GPQA) and coding tasks. It would be more insightful if the authors evaluated the proposed method on such demanding benchmarks, as this would more convincingly highlight the true effectiveness of the reasoning model.\n\nTypo: In line 94, the author redundantly includes an extra “First” after “Firstly.”", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)