Beyond Markovian Drifts: Action-Biased Geometric Walks with Memory for Personalized Summarization
Authors: Parthiv Chatterjee
Asish Joel Batha
Tashvi patel
Sourish Dasgupta
Tanmoy Chakraborty
Venue: ICLR 2026 Poster (2026)
Keywords: User Preference Modeling
Personalized Recommendation
Personalized Summarization
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: Document summarization helps readers focus on the "content-of-interest", a *subjective* and *time-variant* quantity. Capturing this *dynamic subjectivity* requires modeling how user preferences evolve over time, thereby demanding *personalized summarization*. Recent news recommendation and summarization models often assume that preferences follow a *memoryless or short-memory random walk* on interaction graphs, i.e., a Markovian diffusion seeded at the latest interaction or compressed into a short hidden state or prompt. We ask whether such a hypothesis also holds for personalized summarization. To test this, we propose **Walk2Pers**, a lightweight encoder–decoder framework that extends the walk view with *action-conditioned geometric steps*, decomposed into (i) a *magnitude* controlling shift strength and (ii) an *orientation* capturing continuity vs. novelty. The process is mediated by dual memory lanes that reinforce consistent interests while suppressing disinterest, and is augmented with a drift term for summary requests. We show theoretically that such structured walks approximate first-order action-conditioned kernels, and empirically validate the hypothesis on PENS, OpenAI-Reddit, and PersonalSum. Using PerSEval, a personalization metric with strong human correlation, Walk2Pers outperforms specialized personalized summarizers by an average of $0.41 \uparrow$, and strong LLM baselines (DeepSeek-R1-14B, LLaMA-2-13B, Mistral-7B, Zephyr-7B) by $0.22 \uparrow$. Analyses further confirm cross-domain robustness ($0.19 \uparrow$ over the best LLM) and stability on long histories. Together, these results support viewing personalized summarization as an *action-biased geometric walk with memory*, offering both interpretability and efficiency.

Published as a conference paper at ICLR 2026
BEYOND MARKOVIAN DRIFTS: ACTION-BIASED GE- OMETRIC WALKS WITH MEMORY FOR PERSONALIZED SUMMARIZATION
Parthiy Chatterjee
Asish Joel Batha KDM Lab
Tashvi Patel
KDM Lab
Adani University
Dhirubhai Ambani University
Dhirubhai Ambani University
Ahmedabad, Gujarat, India
Gandhinagar, Gujarat, India
Gandhinagar, Gujarat, India
Sourish Dasgupta
Tanmoy Chakraborty
KDM Lab
IIT Delhi, India
Dhirubhai Ambani University
HIT Delhi Abu Dhabi, UAE tanchake wei.daaa..... ac.
Gandhinagar, Gujarat, India
sourish.daaguutaadda
ABSTRACT
Personalized document summarization helps readers focus on the "content-of- interest", subjective and time-variant quantity. Recent news recommendation and summarization models often assume that preferences follow a memoryless short-memon moortyy random walk on interaction graphs, i.e., Markovian diffusion seeded at the latest interaction or compressed into a short hidden state prompt. We ask whether such hypothesis also holds for personalized summarization. To this end, we propose Walk2Pers, lightweight encoder-decoder framework that extends the walk view with conditionn geometric steps, decomposed into (i) a magnitude controlling shift strength, and (ii) an orientation capturing continuity vs. novelty. The process is mediated by dual memory lanes that rein- force consistent interests while suppressing disinterest, and augmented with drift term for summary requests. We show theoretically that such structured walks approximate first-order action-conditioned kernels, and empirically validate the hypothesis on three benchmark datasets - PENS, Open Sum. Using PerSEval, a personalization metric with strong human correlation. Walk2Pers outperforms specialized personalized summarizers by an average of 0.41 1. and strong L.LM baselines (DeepSeek-R) 14B, 1-3.M, Mistral- 7B, Zephyr-7B) by 0.22 f. Our analyses further confirm cross-domain robustness (0.19 over the best LLM) and stability on long histories. Together, these results support viewing personalized summarization an action-biased geometric walk with memory, offering both interpretability and efficiency.
and Personal-
INTRODUCTION
With the problem of information overload, personalized summarization has become essential for tailoring updates to a reader's ' individual interests, especially in multi-aspect documents covering diverse topics (Dasgupta et al. 2024g. Existing approaches typically rely on static persona attributes
2022; Li et al. 2023). Yet, datasets capturing user reading behav- (Dou et al. 2021 He et al. iors, such as MS/CAS PENS (Ao et al. 2021). reveal that user preferences evolve over time and shift across fine-grained subtopics. This creates difficulties even for state-of-the-art (SOTA) Large Language Models (LLMs), which show degraded performance when long user interaction histories embedded in prompts for in-context personalized summarization (Liu al. 2024: Patel et al.) 2024). This suggests that both existing personalized summarizers and LLMs struggle to capture subtle, action-specific user interactions within user logs.

---

Published as a conference paper at ICLR 2026
A natural follow-up question is how do user preferences predictably evolve over time? In most recommendation and personalized summarization systems, user preference evolution has been mod- eled using simple Markovian assumptions, the most basic case being pure Random Walk, where each new preference state is treated as an isotropic perturbation of the previous one, without any action semantics conditioning. A stronger variant is the cetion-condiiioned Random Walk, which biases each step by the most recent action type, while still remaining memoryless. More recently, graph-based Random Walk with Restart (RWR) methods, including Personalized PageRank and its extensions, have been widely applied in news recommendation, where user interests are modeled as a diffusion process seeded at the last interaction. Notably, S- S-Walk (Qiu et al. 2022) restructures the transition kernel to improve session-level modeling, and D-RDW (Zhang et al. (2225) diversifies
restart paths to mitigate popularity bias. Although these models remain competitive as lightweight graph baselines, extensive benchmarking under the same evaluation regimes (e.g.. MIND, Adressa)
has shown them to be outperformed by neural encoders such as NAML (Wu et al. 2019a), NRMS (Wu et al., 2019b), and EBNR (Okura et al. 2017), which aggregate user click histories via CNNs, Transformers, or GRUs. However, these models reduce long histories into compressed embeddings, providing only shallow memory. Similarly, preference-promptee mid-sized LLM summarizers can condition on past interactions, although their windowed prompts impose hard memory cap, with no persistent reinforcement or suppression. Collectively, graph diffusion, short-memory neu- ral encoders, and prompt- -ased LLMs fall under the Markovian Drift Hypothesis (MDH) - each new state depends primarily on the most recent interaction, overriding long-horizon action dynamics.
In this paper, we test how well MDH holds for personalized summarization. As an alternative exten- sion to MDH, we propose the Structured Walk Hypothesis (SWH). SWH decomposes the preference state update due to each user interaction (click, skip, summarize) into (i) a magnitude, controlling the strength of the cetion-sppcific nudge, and (ii) an orientation, determining whether the evolution follows the existing trajectory (continuity) departs into a new direction (novelty). In this way, SWH an action-conditione geometric walk. To go beyond MDH, SWH incorporates dual mem- ory lanes that reinforce consistent interests while suppressing disinterest (for comparative table of SOTA MDH models see Table IO). We propose Walk2Pers, a personalized summarization model, as concrete realization of SWH. To test the adequacy of MDH, we pose three research questions. RQI: Is MDH sufficient for modeling preference evolution in summarization? RQ2: Do dual memory lanes and cction-conditioned geometric steps with magnitude-orienaation decomposition, provide systematic and complementary gains over MDII variants? RQ3: How does Wa k2ers,, as an instantiation of SWH, compare against short-memory neural encoder augmented personalized summarizers, prompt-personalie LLMs, and oracle sunmarizers?
For RQ1 and RQ2. we benchmark MDH-based neural encoder models, along with our own sim- plc action-conditioned Short-Memory Drift (SMD) and an action-category sensitive Action-Gated drift (AGD), on the next user behavior prediction task. We show that these MDH baselines do not sustain asymmetric reinforcement suppression or disentangle continuity from novelty. In contrast, structured walks with dual memory and geometric decomposition achieve systematic gains, with Walk2Pers outperforming the best AGD model by 0.07/0.12/0.11 w.r.t AUC/MRR/DDDGG 5/10 metries. For RQ3, we assess downstream personalized summarization. On the PENS dataset, Walk2Pers surpasses specialized summarizers (PENS-NAML. NRMS, EBNR, GTP, SP) by an average of 0.42/0.36/0.433 across personalization metrics (PSE-ISDISUAMETEORR) It also out- performs four mid-sized L.MS (Zephyr-7B, I.PB,,ILLA LLaMA2-131B, Mistral-7B, ReepSeek-R1-144 under 2-shot+history prompting (the best configuration). with DeepSeek-R1 lagging by 0.20/0.29/0.35. Together, these results demonstrate that while MDH models can be competitive in recommendation- style ranking, SWH with explicit magnitude-orientatioo decomposition, dual memories, and drift, faithfully captures evolving user preferences for personalized summarization.
RELATED WORK
Personalized Summarization Evaluation. Personalized summarization has been increasingly rec- ognized as essential for tailoring updates to a reader's 's interests, especially when documents cover multiple aspects. Existing evaluations of summarization quality (c.g., ROUGE, METEOR, BLEU) do not explicitly account for personalization. Recent work Vansh et al. 2023 Dasgupta et al.] 2024h has emphasized the need for personalization-awaar metrics. EGISES proposed framework
2

---

Published as a conference paper at ICLR 2026
for evaluating semantic shift under personalization, while Dasgupta et al. (2024) introduced PerSE- val, which we adopt in this paper as correlates strongly with human judgment.
Datasets for personalized summarization. To study evolving user preferences, require datasets with (i) temporal orders of user interactions, (ii) user-specific expected summaries for shared con- tents, and (iii) diverse, shifting topics and subtopics. In this direction, the MSICAS PENS dataset (Ao et al. 2021 contains click/skip logs with multi-aspect articles and user-target summaries (per trajectory: 13.6 topies; 52.83 sub-topics, with topic change rate of 0.77). It has become bench- mark for testing personalization-awee aware models (Ao et al. 202T] Song et al. 2023 Lian et al. 2025) PersonalSum (Zhang et al. 2024). The Norwegian dataset derived from Adressa, augments news interaction logs with personalized gold summaries. It highlights preference drift and is suitable for multilingual evaluation. OpenAI-Reddit (Völske al., 2017) provides long-range user interac- tion traces (posts and comments) with subjective summaries. This non-news multi-domain dataset stresses Long-horizon dependencies and temporal drift, and used for cross-domain generalizability test. Dataset details are in Appendix C
Personalized Summarization Models. Most existing personalized summarizers rely on static user
personas, as in GSUM, CTRLSum, TMWIN, and Tri-Agent (Dou et al. 202T, He et al., 2022] Kirstein et al. 2024, Xiao et al., 2024). Dynamic extensions such as PENS (Ao et al. 2021) in-
corporate external news-recommeedation encoders like NRMS Wu et al. (2019b), NAML Wu et al.
(2019a), and EBNR (Okua et al. (2017), while GTP (Song et al. 2023) ) leverages latent editing controls and SCAPE (Lian al. 2025) blends content with stylistic features. However, these ap- proaches remain within the scope of the MDH. The few-shot LLM personalization (Patel et al., 2024) achieves competitive performance against such models, but is ultimately stalled by prompt length and memory constraints. In contrast, our work explores SWH, where preference evolution is modeled as a memory- aware, oction-condition geometric walk. Wal x2Pers serves as one con- crete instantiation capturing evolving user histories beyond the limits of MDH-style summarizers.
3 METHODOLOGY: USER PREFERENCE EVOLUTION REPRESENTATION
3.1 PREFERENCE DATA AS ESEER-NNERACTIIN GRAPH (UIG)
We represent user histories as a UnterIntoo Graph (UIG), directed acyelic graph G (N, F)) where the node set N consists of three disjoint types: (i) u-nodes (f())) denoting a initial timestep to, (ii) d-nodes d(fi) representing documents interacted at timestep t,, and (iii)
nodes su representing user-specific summaries requested or generated at time t, for a document
viewed at 1j-1- The edge set E encodes user actions: d' (1)) {click, skip, summa r ze)
on documents, and (f,)) as the follow-up summGen. action connecting a document dtts 1) to its summary s A user trajectory Tu is then a time-ordered sequence of such interactions, beginning at (o())) Each trajectory can be decomposed into behavior duplets b(t)) = (a(t)), (z((.),, pairing an action with its tail node t/(4,), The UIG T is pool of trajectories, used as Train for training and Trest for evaluation. For UIG construction, see Appendix C.4. Figure
While UIG captures rich temporal detail, directly modeling its raw structure quickly becomes com- putationally expensive and noisy over long horizons. This is particularly challenging in personalized summarization, where fine-grae fine-grained shifts in preference must be retained without overwhelming the model. Recent work in sequential recommendation suggests that hierarchical abstractions improve long-horizon accuracy by condensing lowelevel interactions into higher-order behavioral units (Cho) & Hyun| 2023 Ou al. 2025| Zhu al. 2023 Pan & Wang 2021 Zhang al.) 2020). Motivated by this, we adopt a bi-level hierarchy the u-layer records raw interactions: user nodes (ta)), docu-
ment nodes d(+), and summary nodes g(t) connected by action edges (1),, and the b-layer abstracts
these into behavior duplets bit) = (a(*), t((")), represented as b-nodes. Sequential dependencies captured by nextBehavior edges. This abstraction provides a compact yet expressive substrate
for modeling preference evolution. Having established the representational basis, we now ask a fundamental question: how do preferences evolve along b-layer trajectories?
3

---

Published as a conference paper at ICLR 2026
Table SWH (vs. MDH): (i) explicit trajectory modeling across b-nodes, (ii) dual memory lanes
(h+ h h-) for reinforcement and suppression, and (iii) action-aware updates per (11)
Aspect
Markovian Drift Hy pothesis
Structured Walk Hypothesis
History usage
Collapsed into last state or seed
Explicit trajectory across b modes
Memury
None or sived-lived (hidlen
astention)
Dual memory lanes (f" (fi') for reiffonemmmmsppreessin
Action conditioning
Mininal (recency, weak precupti
Explicit aware updates per
Step dynamics
Magnitude: implicit (via Vanmenntioon weights/rrouqe tokess)
Ovientation: not modeled explicity
Mogninde migie!!))
Orionaaion: die!')
Long-trm
Regatten beyond seep
Persist thrugh memorg-condiiienn updates
Interpretaboiity
Opaque embedding/hidönn states
(contimaty vs. novelty) and stuchasicc (controlled walk)
3.2 MODELING USER PREFERENCE EVOLUTION
We model preference evolution on the b-layer. Each visited b-node b%) is associated with latent preference embedding c I2d summarizing user u's state after timestep t. We first state the prevailing Markovian Drift assumption, then contrast it with our Structured Walk Hypothesis.
3.2.1 MARKOVIAN DRIFT HYPOTHESIS (MDH)
Under MDH, the next state depends only on the immediately preceding state (or short compressed representation), while longer histories are discounted or collapsed into a recency prior g:
Cbuu (++1) = f( (0) 9) + t
e ~ ~NO0 E(a
(1)
Here E(a(9) E Rdxdd controls how stochastic drift spreads across embedding dimensions. The catiioncondiiooinng lets skips be noisier than focused clicks. pure Random Walk (PRW) is: f(eba (t), 9) 6,.u- Short-memory neural encoders (NAML, NRMS, EBNR) and prompt-
(t)
personalized LLMs also fit this umbrella by compressing history into short-term aggregates.
3.2.2 STRUCTURED WALK HYPOTHESIS (SWH)
Evidence from PENS (Ao et al. 2021). PersonalSum (hang et al 2024). and OpenAl-Reddit VVölkee et al.) 2017) indicates long-horizon dependencies: clicks reinforce, skips suppress, and summary requests induce systematic drifts. We posit the SWH: preference evolution is a structured, action-conditiooed geometric walk with memory:
b,u = E,,u + @ (a( trajectory-contex)) + (ktth,) +
(t+1)
(t)
where (--) decomposes trajectory-conte into momentum direction (1() (continuity) and an orthogonal novelty direction o!"), Vv()) aggregates asymmetric reinforremenn/upppessiin via (ht hh, and (--) captures special action drifts (e.g., sumnGen). This decomposition is inspired by advances in trajectory-based dynamic embeddings (e. JODIE (Kumar et (019a) and angle- based relational models (e.g., RotatE, ChronoR (Sun al., 2019 AAnshelevich al.; 202T), ), but adapted to the summarization setting with explicit action bias and memory. To illustrate, frequent Slimtee-policy clicks keep movement near (+) and accumulate in hit: repeated skips of celebrity content load h,- and downweight entertainment; issuing s summGen after dense reports triggers A(), nudging toward concise representations. This view generalizes random walks into controlled dif- fusion governed by state, action, and memory traces (Kumar et al. 2019b Balcer & Lipinski 2025
SWH Model Family.
Refining Eq. equation we obtain a generic structured walk family:
(++1)
e. + mag(a (1) (a))(a(")) ( sin (
d: geometric step: continuity vs. novelly
(3)
+
V vaat hr)
+ 5 Tia (t) = summGen]
W: dual memory: reinforcement vs. suppeession
A: summary speciicc drift
Here ma scales the step (single click small shift; repeated clicks larger shift). The angle (aa!)) steers between ! and (!)); small 0 (so cos 0 as 1, sin 0 as 0) favors continuity; larger 0 in- creases novelty. V(h h,)) persists asymmetric signals from past actions, and the drift 5 (active for
4

---

Published as a conference paper at ICLR 2026
Mack2Park
Wark DPyy Deosile
Enceddr
Uioee
dencee
5-todd Usar vector
Recoded b-history
Next b-rods vector
Encoder
History
I-nodd Vector
Contemoclond
Innense
Approtiaaoor
Frozen
Encoddrr
Query
Finctuned
Figure I: Walk2Pers: Novel instantiation of SWH SSWH-b-ood embedding is formed from the u-layer and fed into the SWH-Encoder; Predictor estimates the next b-node, the embedding of which is fed into the Inverse Approximator, which extracts the latent summary (s-node); Contextualizer computes cross-attention of latent s-node, user history, with query document; T5-decoder finetuned (top-layers only) to generate personalized summary. b-node embedding details in Figure
sumnGGen captures shifts due to specific interest signal. Different models may parameterize mag, 0, and y differently, but all share this decomposition, In the next section, we present Walk2Pers as one concrete instantiation of this family. Theoretical relationship the MDH in Appendix D
3.3 WALR2PERS MODEL SWH INSTANTIATION
3.3.1 WALK2PERS ENCODER: HISTORY ENCODING WITH STRUCTURED WALKS
We instantiate SWH (Eq. 3 in Walk2Pers by encoding user trajectories action-aware b-cells, augmented with dual memories for persistence and geometric steps for continuity-noved tradeoff.
Action-biased B-cell composition. Each d-node and s-node is initialized with T5-base coder (Raffel et al.] 2020), while each action at timestep 1, is represented as a 4-d one-hot vec- tor (click, skip, summarize, SummGen). A b-cell fuses the action with its tail-node content: Cil = (aattf((tta) @ etl : ((t) = tanh(W', c(1)). where f((.)) is an action-conditionel gate. We borrow the (-D(.) function of the baseline Action-Gated Drift model (AGD) (Section 4.2 Ap- pendix E.1). which generates action-specific f(a.t) for action type (click, skip, summarize, summGen). Here, click strengthens the fused representation, skip weakens it, and summGen anchors it to a summary node.
(tz)
Dual memories and drift (Realization of v and A). To capture long-term asymmetries, Walk2Pers models history linear combination of dual memory lanes: h h(+) + (1 - (0 (0)h hn (-:0) (w is learnable scalar), where h(+) accumulates reinforcement signals (clicks), and accumulates suppression signals (skips) as follows:
(tde)) = (+.4-1) + m (4) @ cff h(-4)) h(-4-1) @ (1 - m()) + +c (14).
(4)
In case where the action is summGen, triggering drift vector A (t), h(+4)) ((t((-1) 1) + m (4:) @
A (ti), where A (e) (I (I-e(t-") ) and (4:) Soft W,
w A nudges
the preference state toward condensed representations since summari sememrrrzz---ummeee is stronger positive signal. The corresponding action-gate is then applied as: f(a.4) = AGD(e., &)) o (())) (see (t,)
Appendix E.1 The b-node embedding computed as b tanh (W, (lanh()(a.t)) ed (t)))-
Geometric step decomposition. Finally, Walk2Fers models preference evolution as directed, structured geometric step, balancing persistence with novelty o(-):
e b.3
(8) 6,N + mag!") cos 0(1) (t-1)) sin 0(1) o(t)
where (!-1)) is the momentum axis (continuity), (t) its orthogonal novelty axis, mag(!) the step size, and g(t) the rotation angle interpolating between persistence (0==0) and novelty (0 large). This realizes the t term in & Eq. (Implementation & theoretical equivalence: Appendices G G.2).
5

---

Published as a conference paper at ICLR 2026
Training. The encoder is supervised via two complementary objectives. First, the next-node pre- diction head maps the final contextualized embedding to the predicted query b-node: c by.00
((+1)
W e 6,u + boext. Second, a position classifier enforces alignment between the contextualized
trajectory and its constituent steps: D
(t) SoftMax pos e. The joint objective combines
((t), fene Lalign + (1 a) m = 0.6 so as to avoid cumulative cascading of Lalign on Leext. The alignment term ensures that each intermediate b-node in the trajectory is recoverable from the contextualized embedding, regularizing the walk so it respects positional consistency across steps. The next-node prediction term directly trains the encoder to forecast the upcoming behavior node, making the walk predictive rather than descriptive. Together, these losses encourage (geometric step), V/ (dual memories), and (drift) to cooperate in producing embeddings that are both history- fistory-faithful and forward-looking
these two signals: Lalign = - Σ log (t+), Leatt - log Ph
3.3.2 DECODER: CONTEXTUALIZING USER INTENT FOR SUMMARIZATION
While the encoder (Sec. 3.3.1 is the core instantiation the Structured Walk Hypothesis, we at- tach the same backbone decoder of the encoder model used to generate seed embeddings, T5-base, that consumes the contextualized query embedding eq and generates the personalized summary. Training optimizes combined objective: Lie Average( fenc), where Leen is cross-entropy under teacher forcing and Lene is the structured-walk encoder loss from See. 3.3.1 This ensures that the encoder faithfully models user trajectories while the decoder exploits those states to produce preference-aware summaries. We evaluate two decoder variants: T5-CA (Contex- tualized Attention) and T5-UCA (User-aware CA). The document embedding ed, with the latent summary intention vector (s-node) via cross-attention. This injects summary intent but leaves the document representation agnostic of the user's history- EE-UCA-Decodar builds on that by gating the query document emhedding with the user's trajec-
contextualizes the query
tory state (t)) Ey...," The gating suppresses aspects aligned with h- (c.g.. topics repeatedly skipped)
and amplifies aspects aligned with h+ (reinforced interests), producing user-aware document tor. This ensures that the same document is viewed through Alice's preference lens differently than through Bob's. The gated representation is then contextualized with the latent summary intent, in CA. While CA adapts summaries to "what this document is generally about given the latent sum- mary signal", UCA further adapts "what this document means for this user given their interaction history." As shown in Sec. 5.2. UCA yields stronger personalization, while CA serves as weaker control. Full derivations of latent s-node extraction and gating functions are in Appendix G.3
EVALUATION
We design the experiments address the following research questions (RQ): RQ1: Is MDH suffi- cient for modeling preference evolution in summarization? RQ2: Do the necessary components of the SWH, i.e., dual memory lanes + summary-specific drift & action-conditioned geometric steps with magniude--orientatoo decomposition, yield systematic gains over MDH variants? RQ3: How does Walk2Fers, as an instantiation of SWH, compare against specialized personalized summa- rizers, cooppt-personalieed LLMs, and oracle summarizers?
4.1 EXPERIMENT SETUP
Training (& Test) Datasets. We evaluate across three corpora capturing diverse personalization signals: (i) PENS (Ao et al. (002T) a large-scale news summarization dataset with user clicks/skips; (ii) PersonalSum (EN) ęZhang al. 2024), manually eurated dataset translated to English with explicit summary requests; (iii) Open (Völske et al.) 2017). where summaries are user- rated and span diverse domains. We construct UIGs for each dataset Tirain and slice trajectories be- fore every (d-s) pair, yielding history T query document dy. and target summary s sq,u, For PENS PENS.D,) we sample 55K training trajectories (avg. 134 d-nodes, 5 s-nodes per trajectory). For AlenAI-Reddit (TUAII), 18K training trajectories (avg. 39 d-nodes, 10 s-nodes per trajectory). For PersonalSum-FN (T-NEE (translated into English using M2M-100 (Fan et al.] 2020 700 trajec-

---

Published as a conference paper at ICLR 2026
tories (highly curated, long-horizun The corresponding Test sets (Ties) reflect the same structure. with additional random skips injected (50-70 per user in PENS) to stssss-test suppression memory
Training Setup. Walk2Pers is trained end-to-end with the joint encoder-decoder objective Lac for 6 epochs. Then the encoder is frozen while the decoder (including the last layers of the T5 decoder) is further finetuned for 18 epochs. This ensures encoder quality is the primary driver of downstream summarization gains. Training details are in Appendix H.2 and Table 13
4.2 BASELINES
A. Encoder Baselines We benchmark Walk2Pers for RQ-1 & 2 against five MDH models:
I. Short-Memory Drift (SMD). This minimal baseline captures the pure MDH stance: each new b- node embedding is computed only from the immediate past state and the current action embedding. No reinforcement, suppression, or geometric structure is retained. User preference evolution reduces to one-step drift, overwriting longer histories. Details in Appendix E..1
II. Action-Gated Drift (AGD). This stronger MDH variant replaces the update rule of SMD with action action-specific gates. Different parameterizations applied for click, skip, genSumm, and summGen, allowing the model to qualitatively differentiate between actions. However, the updates remain short-memory: past interactions vanish quickly, and persistent reinforcement suppres- sion is maintained. Details in Appendix E.I
III. Short-Memory Neural-Encoders. We also include NAML (Wu et al. 2019a). NRMS (Wul et al. 2019b), and EBNR (Okura et al. 2017). widely used in personalized recommendation and summarization pipelines (e.g.. PENS). These models differ in mechanism. NAML employs additive self-attention over multi-view document features, NRMS uses multi-head self-attention, and EBNR leverages a GRU over clicked entities. However, all collapse long histories into compressed short- memory embeddings, consistent with MDH.
These baselines constitute strong MDH realizations for the next-b-node prediction task. The candidate set contains 151 b-nodes (including the target). We report average AUC, MRR, and nDCG@5/1L over the PENS test set (metric definitions in Appendix B with full baseline for- malizations in Appendix E. These metrics directly measure a model's 's ability to predict real user behavior because each ground-truth b-node corresponds to an actual user action (click, skip, summarize) and, for <sunGen, s-node>> pairs, to a human-written summary. Hence, RQ1/2 evaluate behavioral faithfulness.
B. Personalized Summarizers To validate the efficacy of Wa 1 k2Pers in terms of the downstream personalized summarization task, we also compare against SOTA personalized summarizers.
I. Neural-Encodeer Augmented Summarizers. For RQ-3, compare against three SOTA per- sonalized summarizers: PENS (Ao et al. (021). GTP (Song et al. 2023).. and Signature-Phres Phrase (Cai al. 2023). PENS pairs a pointer generator with external user encoders; GTP integrates the TrR Mlo encoder internally. Within PENS, we use NAML etu 2019a) (T-1), EBNR (Okura et al. 2017)
(T-1), and NRMS (Wu et al., 2019b) (T-2), with injection (T-x) details in Appendix E.2.1 TrRMIo is an integrated full-sequence Transformer, while Signature- Signature-Phrase models user-specific keyphrases. All baselines are finetuned on Tiin under the same regime as Walk2Pers.
II. LLMM-aa--Summarizers.. To extend RQ-3, we evaluate six frozen instruction tuned LLMs Felasn-2.5-Flashh, Qwen3-235B (Team, 2025). Mistral-7B-Instruee (Jiang et al.] (023)) DeepSeck-R1-14B (DeepSeek-Al et al. 2025). LLaMA-2-133 1---1-BBBhhhttHH (Touvron et al. (023). 3, and Zephyr-7B (Tunstall et al. 2023). Gemini and Qwen are included for their SOTA Long Bench per- formance. We use the best O-shot and 2-shot prompting recipes from Patel et al. (2024) and apply prompt-chaining for DeepSeck- R1 and -istral-7B-Instruue Prompt formats are in Appendix(i
III. Pon-Perssnallzed Summarizers as Oracles. As part of RQ-3, we evaluate three strong non- personalized summarizers, BigBird-Pegasus (Zaheer et al. 2020), SimCLS (Liu & Liu 2021), and T5-base (Raffel et al.] 2020), under the "oracle" protocol of Vansh et al. (2023). We replace each query document's 's title with the user's gold-reference title to inject the true preference cue. This assesses whether frozen models can exploit the cues produce seemingly personalized outputs.
For detailed stress results Appendix and L.3
7

---

Published as a conference paper at ICLR 2026
Table 2: RQ-1/2: Next b-node Prediction (PENS Dataset): Details in Table 14 Full result for
Cross-task transferability on Sequential News Recommendation (MIND Dataset): Table 17
Category
Models
AUC
MRR
abCG@5
#DCGG110
NAML
0.498
0.001
0.0004
0.6007
NRMS
0.499
0.0069
0,0002
E.0004
MDHI based
EBNR
0,409
0.0069
D00003
BU0005
SMD (ours)
0,415
0.054
0.052
0,065
ACID (our))
0.446
0.113
0.069
0,073
SWII based
WoiN2Pces Excoder Pull (oups)
Ma Enconee w/o Groneesi: Siep (ouss]
0,424
0.533
0.121
0.23
0,082
0.198
0,122
0,249
Table 3: (RQ-3:) Galk2Pers-Encooeee Validation: Personalized summarization performance
(w.r.t PSL-ISD/SU-IMETEOR) comparison with SOTA baselines on the PENS dataset.
Category
Model
PSE-ISD
PSE-SU4
PSE-METEOR
BighirdPegasus
0.253
Oraele Summarizzrs (via Cue Injection)
SimCLS
0.157
0,032
0.116
T5-base
0,073
0,011
0.022
LLaMA-138
0.187
0,069
0,078
L.MSs (w/ O-sher user histery)
Mistral71 7B
Zepbiy-788
0,211
0,212
0,0R2
0
0.098
D2089
0.152
0.07
0.00)
Zepbyr-78
0.231
0,083
0.086
LLMss (w/ 2-thor user history)
DaepSock-148
Mastral-78
0,235
0.248
0.094
0
0.097
0.084
Owen-3.235B
0.105
0,082
0
Gemini 2,5-hahh
0.222
0,104
0.124
"0075
0.0%%
0.02%
(Prompt-chinning)
DeepSeck-1-0
0,078
0.028
PENS-NAMI-TI
0.021
0.014
PENS- EBNR T1
0.015
0,010
0.011
PENS-EBNR-12
0.011
0.008
0.009
Fine-tuned Specialized (Personalized)- ~ MDH
PENS-NRMS-TI
0.015
0,011
0.011
PENS-NRMS-T2
0.008
0,807
0.007
GTP
0.024
0,017
0.019
SP-Indiidual
0.017
0,015
0.011
Markov (MDH) Encoders (ours)
SMD 13-UUA Decoder
AGD TS-DIAA-Dooderr
0.143
0.286
0.136
0.214
wio Geometric Step + IS UCA Decoder
0,306
0,334
0.321
Walk2P Encodees (-- SWH) (ours)
Fall TS-CA-Dacddrr
0,488
0.34!
0.22
Pall DeodeA-Deoddrr
0.452
0,383
0,449
5 RESULTS AND OBSERVATIONS
We present results w.r.t the three RQs (Section 4.. All results have a significance of p < 0.05.
5.1 WALK2PERS PREDICTION ACCURACY
RQ1 Are MDH models sufficient? We observe that MDH baselines fail to anticipate user trajectories reliably. Short-memory neural encoders (NAML/RMS//EBNR hover at chance AUC (x 0.5) and collapse on rank metrics (MRR < 0.001, nDCG < 5x1001), 1), showing that compressed hidden states carry little predictive signal. Among controlled variants, action gating helps (AGD > SMD, c.g., .RR 013 vs. 0.094), but overall accuracy remains low, past signals vanish quickly and reinforcemen//sppression is absent.
RQ2 Do SWH components yield systematic gains? We find that adding dual memories and drift (Wa WWalkPPers-Ene w/o Geo) already surpasses the best baseline AGD (AUC +0.028, -egnitud-orrennattion
nDCG@ 10 +0.059). Geometric magnitude orientation step yields large additional jumps (AGD: AUC +0.086/ + 0.117/ + 0.176 w.r.t AUC/MRR/IDCG @5/10) These results confirm that mem- ory lanes preserve reinforcemen//sppprssiinn, while geometric steps capture continuity vs. novelty.
Cross-task Performance. As further validation of Wa k2Pers encoder, we evaluate the sequen- tial recommendation performance of it on widely adopted MIND news recommendation dataset. We find Walk2Pers encoder (Full, with geometric step), which was trained end-to-end for personal- ized summarization task, to surpass the MIND recommendation leaderboard baselines by a signif- icant margin. It outperforms the best baseline (Fustfrmer+PLM--RR (-L_M--e--Enebble by 1.21 on MRR, 1.8 on nDCG@5, 5, and f.5t on nDCG@ 10. This result demonstrates the cross-tate task transfcrability of Wa k2Pers encoder. Detailed results are in Table 17
8

---

Published as a conference paper at ICLR 2026
Table 4: (RQ-3) Cross-domain generalizability on OpenAl (Reddit)
Calegory
Model
PSE-JSD
PSE-SU14
PSE-METEEOR
ULMMA-136
0.233
0.093
0.107
LLMs (wl 2-shot user histury)
7epby--7B
Mistral-70 76
0.226
0.314
0.888
0.887
0.104
0.103
DeepSeek-148
0.24
Walk2Pers
TS-DCA- Decoder
0.350
5.2 RQ-3: WALK2PERS END-TO-END SUMMARIZATION PERFORMANCE
To address RQ3, we benchmark Malk2Pers against the strong summarizer baselines as described in Section 4.2 Evaluation is conducted PENS and OpenAl (Reddit) datasets and further validated using the PersonalSumm dataset using the three PerSEval metrics (PSI-ISDISLLAMETEORR)) which correlate strongly with human preference ratings (Appendix AA.2.I)
Comparison with Specialized Personalized Summarizers, We see that Walk2Pers consistently outperform all specialized models (Table 3). While SOTA frameworks like GTP and SP yield PSE scores in the 0.017-0.022 range, corresponding to average absolute gains of 0.41 over these fine-tuned MDH instantiations. This highlights that explicit geometric-step modeling with dual memory lanes captures preference evolu- tion more effectively than both RNN- and Transformer-based encoders.
achieves 0.452/0. 0022/0033//.44 /0. (SSD/SUAMEETERR)
Comparison with LLMs. LLMs improve when conditioned with user histories. However, they remain substantially below Wa 1 k2Perss. For instance, the best two-shot configuration (DeepSeek- 14B) is outperformed by Wa 1alk2pPrss by margins of 0.20/0.29 29.0335. On average, Walk2Pers yields gains of 0.22 across all LLMs (including long context models Gemini and Qwen3). Prompt- chaining even less effective. falling behind even MDH baselines. Furthermore, evaluation the OpenAl (Reddit) dataset (Table 4 confirms that Walk2Pers indicates cross-domain general- ization, outperforming the strongest LLM bascline by 2..99/0.20/0. across JSD/SU4/METEOR, We also evaluate Wa k2Pers on the translated PersonalSum dataset and observe scores of 0.31 /0.28/0.3 w.r.t. PSE-JIDD/UU/METEOR underscoring the efficacy in datasets tailored for personalized summarization. These results suggest that while prompting improves LLMs, it cannot substitute dedicated action-aware encoder that explicitly models user trajectories.
Comparison with Oracle Summarizers. We observe that oracle summarizers, augmented with gold preference cues, fall short. BigBird-Pegasee the strongest oracle, is outperformed by a margin of 0. 0.20 0 24/0.28 Relative to SimCLS and T5-base, the margins are even larger (0.29 /9.30.35/ /0.43 and 43/3377 /0.38, resp.). This underscores that Wa 1 k2Pers utilizes evolving preferences better than the best-possible performance of non-personalized architectures.
Performance w.r.t Accuracy. We evaluate the accuracy of Walk2Pers W.E.t. gold-reference summaries under standard accuracy evaluation metrics Rouge-L and Rouge-SU4 (Lin) 2004), and find that Wa 1 k2Pers outperforms the baselines by an average margin of 22.3/24.1 on RG-L and RG-SU4, respectively. This confirms that boost in personalization capability also bridges the lack of accuracy gap. Results are in Table
5.3 HUMAN-RATING GROUNDED EVALUATION
While RQ1/2 provide behavioral faithfulness against human trajectories, as a part of RQ-3, we complement this by assessing how generated summaries align with what users prefer. Using the multi-domain non-news OpenAI-Reddit dataset, which contains multiple (5781) human-rated sum- maries of models for 642 query documents, we identify 1042 top-rated (i.e.. 7) one per user as the human-prefere reference. We then measure the "Bert-embedding-spp spaci RMSD-divergence of the model-generated summaries from the reference and create a ground Biiing-to-RMS----- table, where each rating row has its corresponding average min-max range. Using this table, we interpolate the HJ-rating of our baseline models as in Table 5 We observe that both full and w/o ge- ometric step variants (trained on OpenAl-train) of Walk22 srschivee average rating of 7 out of 7, while MDH-modek, are significantly underperforning, with the exception of AGD. Interpolation computation details in Appendix [.1
map

---

Published as a conference paper at ICLR 2026
Table 5: Personalized Summarization Performance w.r.t Human-Judgment Ratings: Avg. in- terpolated rating (w/ RMSD dist. from gold reference) on OpenAl (Reddit) datasct
Category
Model
RMSD (generated vs. gold reference) Rating
PENS-EBNR--11
PENS-FBNR-12
0.938
PENS-NAMI-21I
0.926
Fine-tuned Specialized (Penontalizedd- ~ MDH
PENS-NRMS-TI
0.911
PENS+NRMS 12
0.919
CIP-TRRMO
0.839
Best-forming LLMs user history)
Geniini
0.782
DeepSeek
0.779
TS-UCA Decoder MDII Encoders
SMD
AGD
0.836
0,701
SWIl w/ US-UCA Decoder
Me 1e2Pess woo Govmetric Step MalkIPezs (Full)
0.461
0.396
CONCLUSION
In this paper, we contrast the Markovian Drift Hypothesis (MDH) with our proposed Structured Walk Hypothesis (SWH), which models preference evolution as an action-conditioned geometric walk with memory. Walk2Fers, our proposed instantiation of SWH, encodes trajectories through action-aware b-cells, dual memory lanes, and magnituee-orientation orientation steps, balancing continuity with novelty. Experiments show that the next behavior (b-node) prediction, the primary test of SWH, outperforms MDH baselines. At the same time, downstream personalized summarization confirms that stronger preference modeling yields more user-aligned outputs. While limited by fixed memory kernels and reliance on limited noisy logs, Walk2Pers demonstrates that preference evolution is better framed as SWH than shallow MDH.
CODE OF ETHICS STATEMENT
This research adheres the ICLR Code of Ethics In conducting this work, we: (i) contributed to society and human well-being by advancing methods for trustworthy personalized summarization; (ii) upheld high standards of scientific excellence through transparent reporting, reproducibility, and acknowledgement of prior work; (iii) avoided harm by ensuring that our methods were tested responsibly, with no foreseeable misuse to compromise safety, security, or privacy; (iv) were honest, trustworthy, and transparent in disclosing our methods, limitations, and potential risks; (v) acted fairly and without discrimination, considering inclusivity in data and evaluation; (vi) respected the work and rights of others via proper citation and intellectual property compliance; and (vii) respected privacy by not using personally identifiable or sensitive information in our datasets. (viii) used LLMs (GPT-5) limited to structural changes (paraphrasing and summarization of our content, which has not been used verbatim in most of the paper), table format corrections, and extensive literature review (using Deep Research). We have not used LLM for any content generation purpose.
REPRODUCIBILITY STATEMENT
We have made significant efforts ensure the reproducibility of our work. All details of the pro- posed Walk2Pers framework, including the encoder and decoder variants, training objectives, and evaluation protocols, provided in Sections 3.3 and 4 Hyperparameter choices, model configu- rations, and training details are documented in Appendix H.22 and Table [3] Dataset descriptions, preprocessing steps, and evaluation metrics across personalized summarization (PSE-JSD, PSE- SU-4, PSE-METEOR) sequential predictions and next b-node prediction (nDCG, MRR) are clearly specified in Section 2, Appendices C Appendix I.I I.I choices. To facilitate independent verification, we include the corresponding GitHub code reposi- tory: htttt:://gttttttee wtt.h//.....ooo/tta//////aaaaa www.aaaaaaPPPasre ICL- 2026. git.
A.2.I and B We detail the human judgment annotations in We also provide ablation studies (Tables 2 & 3) to demonstrate robustness to design
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Elliot Anshelevich, Zack Fitzsimmons, Rohit Vaish, and Lirong Xia. Representative proxy vot- ing. Proceedings of the AAAI Conference on Artificial Intelligence, 35(6):5086 5093, May aaai www.aaaa....inneeeeeeeeeeeeeeeee wwtgginnde.... php/
2021. doi: 10. www.6////............. 16643. URL
AAA//article/vlew/
Xiang Ao, Xiting Wang, Ling Luo, Ying Qiao, Qing He, and Xing Xie. PENS: A dataset and generic framework for personalized news headline generation. In Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli (eds.), Proceedings of the 59th Annual Meeting of the As- sociation for Computational Linguistics and the 1 11th International Joint Conference on Nat- ural Language Processing (Volume Long Papers), pp. 82-92, Online. August 2021. As- sociation for Computational Linguistics. doi: 10.18653/11/2021.co aac.ooo.........o........ URL https: //aclanthology org.
Klaudia Balcer and Piotr Lipinski. Session-based recommender systems: User interest stochas- tic process in the latent space, 2025.
10005
Satanjeev Banerjee and Alon Lavie. METEOR: An automatic metric for MT evaluation with improved correlation with human judgments. In Proceedings of the ACL Workshop on In- trinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization, pp- 65-72, Ann Arbor, Michigan, June 2005. Association for Computational Linguistics. URL https: pettps:/aaclase http///aaaaantttoooooo www.llloy.ooooooooooooooooooe -05- 0909
Pengshan Cai, Kaiqiang Song, Sangwoo Cho, Hongwei Wang, Xiaoyang Wang, Hong Yu, Fei Liu, and Dong Yu. Generating user-engaging news headlines. In Anna Rogers, Jordan Boyd- Graber, and Naoaki Okazaki (eds.), Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pp. 3265-3280, Toronto, Canada, July 2023. Association for Computational Linguistics. doi: 10. 1..18665//12203.aclllee 183. URI. www.alaaannnooooooooo///o000
Junsu Cho and Hyun. Dynamic
sequence modeling for next item recommendation. In Proceedings of the AAAI conference on artificial intelligence, 2023.
Sourish Dasgupta, Ankush Chander, Tanmoy Chakraborty, Parth Borad, and Isha Motiyani. PerSE- val: Assessing personalization text summarizers. Transactions on Machine Learning Research, 2024. ISSN 2835-8856. URL
net www.eeeeeeoouteeeeeeeeeeeeeeeeee -iniiidddddii------- yqT /eBz1VJ
Dee,Seek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang. Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei Feng, Chengda Lu, Chenggang Zhao, and Chengqi et al. Deepseek-rl: Incentivizing reasoning capability in Ilms via reinforcement learning, 2025. URL
abs/2501. 12948
Zi-Yi Dou, Pengfei Liu, Hiroaki Hayashi, Zhengbao Jiang, and Graham Neubig. GSum: A general framework for guided neural abstractive summarization. In Kristina Toutanova, Anna Rumshisky, Luke Zettlemoyer, Dilek Hakkani-Tur, Iz Beltagy, Steven Bethard, Ryan Cotterell, Tanmoy Chakraborty, and Yichao Zhou (eds.), Proceedings of the 2021 Conference of the North Amer- ican Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 4830-4842, Online, June 2021, Association for Computational Linguistics. doi: 10. 1865/v11 2021 maacl-main.33. URL ht httts:://claatthll.ootooo org.:///r///0000 www.nnaaalaaam. 384/
Abdul Ghafoor Etemad, Ali Imam Abidi, and Megha Chhabra. Fine-tuned 15 for abstractive sum- marization. International Journal of Performability Engineering, 17(10), 2021.
Angela Fan, Shruti Bhosale, Holger Schwenk, Zhiyi Ma, Ahmed El-Kishky, Siddharth Goyal, Man- deep Baines, Onur Celebi, Guillaume Wenzek, Vishrav Chaudhary, Naman Goyal, Tom Birch, Vitaliy Liptchinsky, Sergey Edunov, Edouard Grave, Michael Auli, and Anmand Joulin. Beyond english-centric multilingual machine translation, 2020. URL https: / //ariivv.... oog//////....... abs/ 2010 0. 11125
11

---

Published as a conference paper at ICLR 2026
Junxian He, Wojeiech Kryscinski, Bryan McCann, Nazneen Rajani, and Caiming Xiong. CTRL- sum: Towards generic controllable text summarization. In Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang (eds.), Proceedings of the 2022 Conference Empirical Methods in Natu- ral Language Processing, pp- 5879-5915, Abu Dhabi, United Arab Emirates, December 2022. Association for Computational Linguisties. doi: 0665:///2022..mmmpmmaaaa
URL
main. 1i..n.996 3967
Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chap- lot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. Mistral 7b, 2023. URL https:
06825
Frederic Kirstein, Terry Ruas, Robert Kratel, and Bela Gipp. Tell me what I need to know: Exploring LLM-based (personalized) abstractive multi-source meeting summarization. In Franck Demonncurt, Daniel Preotiuc-Pietro, and Anastasia Shimorina (eds.), Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: Industry Track, pp. 920-939, Miami, Florida, US, November 2024. Association for Computational Linguistics. 1rt2:004..nll-innnustryye.. emm p- 1ernlp--nndine inw.stryyccccmmmm 69/
doi: 10.18653/w1/20222
URL https:
Srijan Kumar, Xikun Zhang, and Jure Leskovec. Predicting dynamic embedding trajectory in tem- poral interaction networks. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD '19, pp. 1269-1278, New York, NY, USA, 2019a. Association for Computing Machinery. ISBN 9781450362016.
doi: 10. 10.1455/2925003330899.
org/10. 114500...115550 URL /a4.::///oll.oogg/...14///222220 /200///3/22200003338355 3330895
Srijan Kumar, Xikun Zhang, and Jure Leskovec. Predicting dynamic embedding trajectory in tem- poral interaction networks. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD '19, pp. 1269-1278, New York, NY. USA, 2019b. Association for Computing Machinery, ISBN 978 1403662011.. doi: 10.1145/3292500.333888)
URL httts:://oooii org. 11451 3292509222000 3330895
Alon Lavie and Abhaya Agarwal. Meteor: an automatic metric for mt evaluation with high levels of correlation with human judgments. In Proceedings of the Second Workshop on Statistical Machine Translation, StatMT '07. USA, 2007. Association for Computational Linguistics.
Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy. Veselin Stoyanov, and Luke Zettlemoyer. BART: Denoising sequence-to-sequane pre- training for natural language generation, translation, and comprehension. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 7871 1-7880. Asso- ciation for Computational Linguisties, July 2020. doi: 10.18653/v1/2020.aco main. 703. URL
https: http://aaaaannnlcc...c..c org. 2020. acl main 703
Zekun Li, Baolin Peng, Pengcheng He, Michel Galley, Jianfeng Gao, and Xifeng Yan. Guiding large language models via directional stimulus prompting. In Proceedings of the 37th International Conference on Neural Information Processing Systems, NIPS '23, Red Hook, NY, USA, 2023.
Curran Associates Inc.
Junhong Lian, Xiang Ao, Xinyu Liu, Yang Liu, and Qing He. Panoramic interests: Stylistic-content aware personalized headline generation. In Companion Proceedings of the ACM on Web Confer- ence 2025, 2025.
Chin-Yew Lin. ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out, pp. 81, Barcelona, Spain, July 2004. Association for Computational Linguis- tics. URL httpttt:: : ac http///caaaaaao......c.ao wtw.atttwoooooooooooo0/---10
Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. Lost in the middle: How language models long contexts. Transactions of the Association for Computational Linguistics, 12:157 2024. doi: 10.1/62/tacl a 00638. URL
https:// antho www.org/../2..ooo/
12

---

Published as a conference paper at ICLR 2026
Yixin Liu and Pengfei Liu. SimCLS: A simple framework for contrastive learning of abstractive summarization. Proceedings of the 59th Annual Meeting of the Association for Computa- tional Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), pp. 1065-1072. Association for Computational Linguistics, August 2021. doi: 11116633/112221.al--lff/ 135. URL https:::::/ //acl acl-short.oo 135
M.L. Menéndez, J.A. Pardo, L. Pardo, and M.C. Pardo. The jensen-shannon divergence. Jour- nal of the Franklin Institute, 334(2)307-318) 1997. ISSN 0016-0032. doi: https///oo..rr//10 article/pii/
1016//9006--0032226/00000644444 URL //ww::///wwwwwo http://wwww.cnc...e.eeeeeeeeeeee wtt.:///wo.....a...oooocccccc/// ctt.:e/ttcccomccccccecccccc
article/pii/5001 6003296000634
Shumpei Okura, Yukihiro Tagami, Shingo Ono, and Akira Tajima. Embedding- news recommendation for millions of users. In Proceedings of the 23rd ACM SIGKDD Interna- tional Conference Knowledge Discovery and Data Mining, KDD '17, pp. 1933-1942, New York, NY, USA, 2017. Association for Computing Machinery. ISBN 978 9781450348874, doi: 00.1145/3097983.309881 URL https: http::::/ddooioo htt.//i..org////// 114.0..1114111999311 3098108
Zhonghong Ou, Xiao Zhang, and Zhu. Ls-tgnn: Long and short-term temporal graph neural net- work for session-based recommendation. In Proceedings of the AAAI Conference on Artificial Intelligence, 2025.
Zhe Pan and Peng Wang. Hyperbolic hierarchy-award knowledge graph embedding for link pre- diction. In Marie-Francing Moens, Xuanjing Huang. Lucia Specia, and Scott Wen-tau Yih (eds.), Findings of the Association for Computational Linguistics: EMNLP 2021, pp. 2941-2948, Punta Cana, Dominican Republic, November 2021, Association for Computational Linguistics. doi: 10.18663/11/0221..mg http:///22...dnnnnnnmnmmlpppoopp
URL
org/2021,
Findings wwn.iinggeemnn..222. wwn.e.mnpp.... 251/
Divya Patel, Pathik Patel, Ankush Chander, Sourish Dasgupta, and Tanmoy Chakraborty. Are large language models in-context personalized summarizers? get an iCOPÉRNICUS test done! In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.), Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 16820-16842. Miami, Florida, USA, November 2024. Association for Computational Linguisties. doi: 10. 102118665//11222.emmel 935. URL htt.tttsps:/aaaattttaa
org/ 2024 wwweeennn..la. maa.pp-maiiin.mmmmmm 935/
Jie Qiu, Yifan Zhang, Zhichao Hou, Xiangnan He, Tong Xu, and Xing Xie. S-walk: Accurate and scalable session-bases recommendation with random walks. Proceedings of the 15th ACM International Conference on Web Search and Data Mining (WSDM), pp. 88-96, New York, NY, USA, 2022. ACM. doi: 10. 11144//448850034944422
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to- text transformer. Journal of Machine Learning Research, 21(140): 2020. URL http:
7/jmlr. wwtpooppppp../////////. 074 html
GS Ramesh, Vamsi Manyam, Vijoosh Mandula, Pavan Myana, Sathvika Macha, and Suprith Reddy. Abstractive text summarization using t5 architecture. In Proceedings of Second International Conference on Advances in Computer Engineering and Communication Systems: ICACECS 2021, pp. 535-543. Springer, 2022.
Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bert- networks. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguisties, 11 2019. URL https: detts://arxiv:
org/ stg/abs// 1908. 10084
Yun-Zhu Song, Yi-Syuan Chen, Lu Wang, and Hong- Han Shuai. General then personal: De- Transactions of the Associa-
coupling and pre-training for personalized headline generation. tion for Computational Linguistics, 11:1588-1666 1607, 2023. doi: 10.1162/tacl a 00621. URL
https: /atps://acl. antho wowhhooogy..ooo ogg. tacl-11............
13

---

Published as a conference paper at ICLR 2026
Zhiqing Sun, Zhi- Deng, Jian-Yun Nie, and Jian Tang. Rotate: Knowledge graph embedding by relational rotation in complex space. In International Conference on Learning Representations, 2019.
T Tawmo, Mrinmoi Bohra, Pankaj Dadure, Partha Pakray, et al. t5 model for abstractive text summarization on different datasets. In Proceedings of the International Conference Innovative Computing Communication (ICICC) 2022. SSRN, 2022. URL http:: http://cc...tttteeooooooooattt/.
Comparative analysis of
4096411
Qwen Team. Qwen3 technical report, 2025.
Hanghang Tong, Christos Faloutsos, and Jua-Yu Pan. Fast Random Walk with Restart and its Ap- plications. In Proceedings of the Sixth International Conference on Data Mining (ICDM). pp- 613-622, Washington, DC, USA, 2006. IEEE Computer Society. doi: 10. 109/1/0M..0006..00
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Niko- lay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open founda- tion and fine-tuned chat models. arXiv e-prints, pp. arXiv-2307, 2023.
Lewis Tunstall, Edward Beeching, Nathan Lambert, Nazneen Rajani, Kashif Rasul, Younes Belkada, Shengyi Huang, Leandro von Werra, Clémentine Fourrier, Nathan Habib, et al. Zephyr: Direct 2023.
distillation of im alignment. arXiv e-prints, pp.
Rahul Vansh, Darsh Rank, Sourish Dasgupta, and Tanmoy Chakraborty. Accuracy is not enough: Evaluating personalization in summarizers. In Findings of the Association for Computa- tional Linguistics: EMNLP 2023. pp. 3582-2595, Singapore, December 2023. Association for Computational Linguistics. doi: 10. 10.18653/v1/2033 10006665//0......iineesmmm//ppp wwndingg--emll...6...
URL https:
//aclanthology..o wwwptthhooog...rr/220.. org/2023. f findi inndings- eagg-emnlle 1...m...111. 169
Michael Völske, Martin Potthast, Shahbaz Syed, and Benno Stein. TL:DR: Mining Reddit to learn automatic summarization. In Lu Wang, Jackie Chi Kit Cheung, Giuseppe Carenini, and Fei Liu (eds.), Proceedings of the Workshop on New Frontiers in Sunmarization, pp. 59- 63, Copenhagen, Denmark, September 2017. Association for Computational Linguistics. doi: 10.18653/v1/W11-45000
Chuhan Wu, Fangzhao Wu, Mingxiao An, Jianqiang Huang, Yongfeng Huang, and Xing Xie. Neural news recommendation with attentive multi-view learning. In Proceedings of the Twenty-Eige International Joint Conference on Artificial Intelligence. International Joint Conferences on Arti- ficial Intelligence Organization, 2019a.
Chuhan Wu, Fangzhao Wu, Suyu Ge, Tao Qi, Yongfeng Huang, and Xing Xie. Neural news rec- ommendation with multi-head self-attention. In Proceedings of the 2019 Conference on Em- pirical Methods in Natural Language Processing and the 9th International Joint Conference Natural Language Processing (EMNLP-JJCNLP), pp. 6389-6394 Hong Kong, China, Novem- ber 2019b. Association for Computational Linguistics. doi: 10.18653/v1/D19-166)) URL https: httppp///aaannttooooooooo//////
Fangzhao Wu, Ying Qiao, Jiun-Hung Chen, Chuhan Wu, Tao Qi, Jianxun Lian, Danyang Liu, Xing Xie, Jianfeng Gao, Winnie Wu, and Ming Zhou. MIND: A large-scale dataset for news recom- mendation. In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault (eds.), Proceedings of the 58th Annual Meeting the Association for Computational Linguistics, pp. 3597-3606, On- line, July 2020. Association for Computational Linguistics. doi: 10. 20653//1/2020.cllmammm URL httph: http:/ss///ccaathhoooooo org/ aww.2mn.ccmm........... 331
Wen Xiao, Yujia Xie, Giuseppe Carenini, and Pengcheng He. Personalized abstractive summariza- tion by tri-agent generation pipeline. In Yvette Graham and Matthew Purver (eds.), Findings of the Association for Computational Linguistics: EACL 2024, pp. 570-581, St. Julian's s, Malta, March 2024. Association for Computational Linguistics. URL https: http///aaaaanno......mmm org f idd ngn--acc eacl. 397
14

---

Published as a conference paper at ICLR 2026
Manzil Zaheer, Guru Guruganesh, Kumar Avinava Dubey, Joshua Ainslie, Chris Alberti, Santi- ago Ontanon, Philip Pham, Anirudh Ravula, Qifan Wang. Li Yang. and Amr Ahmed. Big bird: Transformers for longer sequences. In Advances in Neural Information Processing Systems, vol- neur ume 33, pp. 17283-17297, 2020. URL [RR./httppss//prrceeeddngg neur ps. cc/ppeer files http://peer.bb00///e/////2122222
Lemei Zhang, Peng Liu, Marcus Tiedemann Oekland Henriksboe, Even W. Lauvrak, Jon Atle Gulla, and Heri Ramampiaro. Personalsum: A user-subjective guided personalized summariza- tion dataset for large language models. In The eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2024. URL https://openreview htt.//ormmmmidi22k11001
Wei Zhang, Xiaoyu Li, Chen Sun, Hongzhi Wang, and Yongfeng Zhang. D-rdw: Diversity-driven random walks for news recommendation. In Proceedings of the ACM Web Conference (WWW), New York, NY, USA, 2025, ACM. To appear.
Zhanqiu Zhang, Jianyu Cai, Yongdong Zhang, and Jie Wang. Learning hierarchy-aware knowl- edge graph embeddings for link prediction. In Proceedingy of the AAAI conference artificial intelligence, volume 34, pp. 3065-3072, 2020.
Xiaoyu Zhu, Chenyang Bu, Bingbing Dong, Shengwei Ji, Yi He, and Xindong Wu. Ikgn: Intention- aware knowledge graph network for poi recommendation. In 2023 IEEE International Conference 2023. doi: 1011099CCDM58522.223..00000
on Data Mining (ICDM), pp.
A MEASURING DEGRBE-OF-PERSONALIZZEE
A.1 MOTIVATION
Vansh et al. (023) proposed EGISES- metric to measure the degree of insensitivity-to- subjectivity for relative benchmarking of how much models lack personalization (i.e., a lower score is better within the range [0, 1) instead of assigning an absolute goodness score. Based on this notion, they defined (summary-level) "deviation" of model Mo,.. termed as Degree-of- Responsiveness (DEGRESS) by Dasgupta et al. (0044) as follows:
Summary-level DEGRESS. Given document d, and user-profile Uhj (user j's pected summary), the summary-level responsiveness of a personalized model Me, (i.e., DEGRESS (sug [di, usj))) ), is defined as the proportional divergence between model-generated sum- mary Suy, of d; for j-th user from other user-specific summary versions w.r.t a corresponding diver- gence of Uay from the other user-prood user-profiles.
DEGRESS($u., [dd,, u.y)) is formulated as:
|Ua, min(Xish 15h, YIJR) +e mar(Xiikk Yija) +
DEGRESS (Su (de, u.))
Nisk- exp(w(usy)e) (un))
(uiss use); Yisk exp(w](u, ( 1) (Bwrgy Smne)
(6)
Σ exp(w(uss) (u))
exp(u(sms) (8mn))
auky, u.)), w(sury |sma)
o (B00) Ause) alougy di)
w(us (uaa)
aluy, d:)
Here, D is the total number of documents in the evaluation dataset, |U] is the total number of users who created gold-reference summaries that reflect their expected summaries (and thereby, their subjective preferences), and |U2| (= [Sa,) )) is the number of users who created gold-references for document d,, w is the divergence of the model-generated summary Su., (and the correspond- ing expected summary uij) from document d; itself in comparison to all the other versions. It helps to determine how much percentage (therefore, the softmax function) of the divergence (i.e., assuy Su,e) should be considered for the calculation of DECRESS. If Suy is farther than Su,z w.r.t d, then DEGRESS( (dd, u.z)) ) < DEGRESS(s (d; 4,, implying that Mø u is more responsive to the k-th reader. A lower value of DEGRESS (Su, (di, miy)) indicates that while reader-profiles
15

---

Published as a conference paper at ICLR 2026
different, the generated summary Suy very similar to other reader-specific summaries (or vice versa), and hence, is not responsive at the summary-level. The system-level DEGRESS and EGISES have been formulated follows:
DECRESS( ((4))))))
DEGRESS Mo..)
(7)
D
A.2 PERSEVAL: FORMULATION
As can be noted, the DEGRESS formualtion does enforce any penalty on accuracy drop. To rectify this Dasgupta et al. (2024) proposed PerSEval. The design of PerSEval had two key goals: (i) to penalize models for poor accuracy, while simultaneously (ii) ensuring that evaluation of responsiveness (i.e.. DEGRESS) not overshadowed by high accuracy. This penalty is referred to as the Effective DEGRESS Penalty Factor (EDP). a model achieves 100% accuracy. no EDP will be applied, and the PerSEval score will equal the DEGRESS score. The following formulatiown of PerSEval guarantees these properties:
PerSEval(*u,, [dds uny)) = DEGRESS( (d; u;s)) x EDF (Su,) (d;, uiz))
where, (S (di, u:5)) 1 10o23 exp (- (-11022 (s..,) (d;, uiy))))
(8)
CGP(suuyl(de wi))) = ADP ((d; u.-)) + ACP (di us;))
Here, ADP is document-level penalty due drop in accuracy for the best-performance of the model (i.e., the model-generated summary of document d, (Su.g) is closest to the corresponding reader's expected summary wiy). ADP is formulated as follows:
(D (dd,, uxo))
=
1 10724 exp (110- (I-or (tuge Idd.)+
a Mlce d,-0
(9)
where, o² (So.. )ds min af uiy) [dl:
j=1
and {e An infinitesimally small number E (0, 1)}
ADP ensures that even the DEGRESS score acceptable, a penalty due to accuracy drop can still be imposed part of EDP. ADP, however, fails to address the scenario where the best-case scenario is acceptable (i.e., accuracy fairly high) but rather an outlier case i.e., for most of the other model-generated summary versions, there is considerable accuracy drop. To address this issue, the second penalty component within EDP called Accurucy-ineoniisenc Penalty (ACP) was introduced which evaluates whether model consistently performs w.r.t accuracy for specific generated summary compared to its average performance. ACPis formulated as:
ACP (dd Mey))
+ 10724 exp (-10- (mr(au mhs lid -o* (Auge (lde)
(*aayy May Md, -o* (Auie (laalld
(10)
where, (Ssu.. (di
(Ssgg wi)ld
Ua,
The system-level PersEval score is follows:
PerSEval(suy (dian))
14,1
(er.Eval(Me...)
(1I)
D
The system-level PersEval 1] and is bounded by the system-level DEGRESS score.
16

---

Published as a conference paper at ICLR 2026
A.2.1 PSE METRICS
PerSEval-RG-SU4 (or PSE-SU4) is the PerSEval variant that uses ROUGE-SU4 (Lin 2004) distance metric (i.e., σ) in the FerSEval formula. PSE-SU4 has been reported have high human-judgment correlation (Pearson's s r: 0.6; Spearman's 's 0.6; Kendall's s T: 0.51) Dasgupta et al. (022).. The ROUGE-SU4 score is based on skip- hiiibbiramm,, which are pairs of words that appear in the same order within a sentence but can have up to four other words between them. The formula is as follows:
For given generated summary G and reference summary R. the ROUGE-SU4 score is calculated as:
Skip-Bigram Recall (Rsua):
Rsu4 Count matching skip-bigrams between G and
Total skip-bigrams R
Skip-Bigram Precision (Psu4):
Count of matching skip-higrams between G and
Psu4
Total skip-bigrams in G
F1 Score (F15:))): The F1 score is the harmonic of precision and recall:
x Psu4 X Rsu4
Flsu4 = Psu4 Rsu4
Where:
A skip-bigram consists of two words in the correct order but with zero four words skipped in between.
Matching skip-bigrams are counted between the generated summary and the reference sum- mary.
The final ROUGE-SU4 score is typically reported the Fl measure, balancing precision and recall.
PerSEval-JSD (or PSE-ISD) is the PerSEval variant that uses the Jensen-Shannon Diver- gence (JSD) Menénddez et al. 1997) as the distance metric a in the PerSEval formula. JSD is smoothed and symmetric version of Kullback-Leti
divergence between the unigram (or n-gram) distributions of the generated summary G and reference summary R. Its formulation is:
ISD(P ()
( + KL(Q||
where M [PP Q)
(12)
here, P and Q are the normalized n-gram probability distributions of G and R respectively, and
P(x)
KL((PIM)) = Σ P(z) log
M(x)
We then define the divergence as: JUSD (G, R) JSD PP and plug JJSD into all occurrences
of a in Equations equation 6 equation 11 to obtain PSE-ISD.
(or PSteMMttor) uses the METEOR score Banerjee & Lavie (2005): Lavie & Agarwal (2007) as the similarity metrie; convert it into distance by 1 METEOR. METEOR aligns unigrams (with synonymy, stem, and paraphrase matching) and combines precision, recall, and fragmentation penalty. Its formulation is:
PerSEval-Metie
17

---

Published as a conference paper at ICLR 2026
matchedd.unigramn
matched..uiggramm umigrams(At))
p=
R
(13)
Junigrams(G)
PR
F.
00 1],
(14)
a P +
#chunks
8
Penalty = mmachee__unigrams
7, 0,
(15)
METEOR G,, R) = (1 P-Pnllty) Fa-
(16)
We then set Meteor
G. R) METEOR (G, R), and substitute Mettorrr for in Equations
equation [ equation
11 to yield PSE-Meteor.
B PREDICTION METRICS
In this section, we provide definitions of the evaluation metrics used in our experiments: AUC, MRR, and nDCG@k. Each metric captures complementary aspect of ranking quality when comparing the predicted next-step positions against the ground-trus -ouun target.
Area Under Curve (AUC) AUC measures how well the model ranks the ground- truth item rela- tive to all other candidates. Formally, if r is the rank (I-based) of the ground-truth item among C candidates, we define:
C-r AUC = C
This normalizes the rank to the interval [0, 1], where higher values indicate that the item is ranked closer to the top. Intuitively, AUC reflects the overall discriminative ability of the model.
Mean Reciprocal Rank (MRR) MRR emphasizes how highly the correct item appears in the ranked list. Given the rank of the ground-truth item, its reciprocal rank is:
RR
MRR is the average of RR values across all queries. MRR rewards systems that consistently place the true item very close the top of the ranking.
Normalized Discounted Cumulative Gain (nDCG@k) nDCG@k evaluates the quality of the top-k predictions, with stronger weight on higher-ranked positions. The gain relevant item at rank is discounted logarithmically:
T(r: target) b:
DCG@k
Σ
log, +
where [[{}} is an indicator function. Since there is only one relevant target per query, DCG@k reduces (1 if the target appears within the top-k, and 0 otherwise. Normalization divides by the best possible score (which is if the target is at rank 1). Thus:
if k.
nDCG@k
(1+r)
0,
otherwise.
nDCG@k highlights whether the correct prediction is placed near the very top of the model's didate list.
These three metrics together provide comprehensive evaluation: AUC captures global rank dis- crimination, MRR emphasizes early precision, and nDCG@k measures the quality the truncated top-k predictions.
18

---

Published as a conference paper at ICLR 2026
Clasoifie
c(f-1)
cft)
e byeu
b,u
summgen
{
click
skip
gensumm
b,u
B-cell
mag
(8)
A
Memory Mixer
6,
-cal
flatt
Iach
action conditioned gate
(t:)
conditional drift for summary
Geomeetrc decomposition
Figure 2:
Walk2Pers-Encodes The b-cell generates b-node embedding (4) c, at timestep t (t:) the action embedding e and the history (f from the
using the tail-node embedding "tl previous tail-cell content d (inside the corresponding b-cell); Gt updates the dual memory lanes (h*, h that persists positive (click, summarize) and negative (skip) memory resp. create mixed history ((t=-1) which in turn is modulated by action-specifie gate f(a.t)) and conditional wwwpeen--nciiinttiiiine drift (A) before fusing with the tail-node e ((te), the generated bbu then goes through geometric-step decomposer re-orient the embedding w.r.t continuity vs. novelty.
(t,)
Model A
Model
Model
Manually-written
scores
NEWS
Stage One
skip
Stage Two
Headlinel
Headline2
Headline3
Head
s00m,,
s00Tes
score,
Headlinet
Headlino2
Headline3
Headline-human
Figure 3: Stages of creation of testing dataset consisting of personalized headlines
C DATASETS
C.I PENS DATASET
The PENS dataset (Ao et al. 2021) includes 113,762 news articles across 15 topics. Each article contains ID. title (avg. 10.5 words), body (avg. 549 words), and category, with titles linked to WikiData entities. The dataset also includes user interaction data, such as impressions and click behaviors, combined with news bodies and headlines from the MIND dataset Wu et al. (2020)
PENS training set. For training, 500k user-news impressions were sampled from June 13 to July 3, 2019. Each log records user interaction as [uID, tmp. clkNews, uelkNews, elkedHis], where "elkNews" and 'uclkNews' represent clicked and unclicked news, and 'clkedHis" refers to the user's prior clicked articles, sorted by click time. The training data for Wa k2Pers, as discussed in Section 4.1 shows high preference shift. This inherently supports that personalizing UX is strongly dependent on the temporal dynamics of the user. The stats are in the table
PENS test set. To create offline testbed, 103 English-speaking students reviewed 1,000 head- lines in stage- 1, and then selected 50 articles, and created preferred headlines (i.e., expected gold- reviewed by four participants. Editors checked for factual accuracy, discarding incorrect headlines. The high- quality remaining headlines as personalized cold-standard references in the PENS dataset.
reference summaries) for 200 unseen articles in stage-2 (see Figure B) Each article
19

---

Published as a conference paper at ICLR 2026
Table 6: MS/CAS PENS Dataset and Interaction Statisties
Characteristic
Dimension
Value
Article Stats
Topics
15
Articles
113,762
General Stats
Avg. Tide Length
words
Avg. Body Length
549 words
Train Dataset Statisties
JJer-News Impressions (amon.)
Users (amon.)
445,000 June 13-Joly 2019
Interaction Data
Time Period
User Interaction Fields
[ulD. tmp, clkNews, uelkNows, clkedHis]
Test Dataset Statisties
Participants
Participant Category
English speaking college students
Participant Stats
Articles
3,940
Browsed Headlines (Click Skip)
1,000 per participant 50 per participant
Min. Interested (Click) Headlines
Gold Reference
Summarized Article Bodies Avg. Summaries per Article
200 participant
(Participant- written Headlines)
Table 7: OpenAl TL;DR (Reddit) Dataset Statistics
Characteristic
Dimension
Value
Datusel Overview
Reddit Posts
123,169
General Stats
Policy- Poliey-Generated Summaries
# Subreddits (Domains)
115,579
29
Human Written Summaries
Available
Train Validation Dataset Statistics
# Reckit Posts
21,111
# Policies
81 107,866
Article Stats
# Generated Summaries
# Annotators
76
# Summary Pairs Rated
64,832
Validation Subset Statisties
Reddit Posts
1,038
Policics
13
Subset Details
Generated Summaries
7,713
# Annotators
32
Datasel (RLHH-TTnned Policies) Statisties
# Evaluated Policies Evaluated Reddit Posts
4 (out 1,038)
Evaluation
Evaluatio Method
Indirect Benchmarking
Annotation and Feedback
Rating Scale
1-7
Feedback Collection
Confidence Scale
1-9
Avg. Ratings per Annotator Annotation Format
1.176
Summary-Pane Solection
C.2 OPENAI (REDDIT) DATASET
The OpenAI (Reddit) dataset (Völske et al. 2017) comprises 123,169 Reddit posts collected from 29 distinct subreddits. This dataset provides both OpenAl-generated and human-writeer summaries and organized into two splits: Comparisons, used for training and validation, and Axis, designated for validation and testing. A curated subset of 1,038 posts was processed by 13 different summarization policies, resulting in the generation of 7,713 summaries. These summaries underwent evaluation by 64 annotators who rated paired summaries based on selection preferences, confidence in their ratings, and dimensions such as accuracy, coherence, coverage, and overall quality. Notably, unlike datasets like PENS, these summaries are not linked to individual annotators or their reading histories, which means they lack elements of personalization and contextual user information. Stats are in Table 7
20

---

Published as a conference paper at ICLR 2026
Table PersonalSum Dataset Statisties
Characteristcc
Dimenssin
Value
Dataset Overview
* Articles
441
Stats
Personalized Summarics
Anmotators
1,099
Avg. Persenalized Summaries per Article
-3
Summary Tyypes
GPT-4 Generic Summaries (Post-edited)
441
Summary Stats
Human Personalized Summaries
1,099
Avg. Summaries per Annotator
-28
Amnotation Process
Stage I: Generic Summary Creation
GPT4 - Human Editing + Highlights
Stages
Stage-2:
Stage-3: Quality Fillering
Persona lived Summarization
GPT:3.5 Scoring (Coherenec. Relevance, Consisiency))
Crondwarkers Source Highiightnng
Final Datanet
High-quality, aligned summarics with anonymized metadita
Annotation Details
Source Sentence Highlighting Comprehousion Questions
Yes Yes Anonymized
Feedback and Grounding
User Metadara
u-node
Click
Tost
Figure 4: UIG Construction: Construction of Interlnntraction Graph from preference datasets.
C.3 PERSONALSUM DATASET
The PersonalSum dataset (Zhang et al. 2024 is crowd-annotated benchmark designed for person- alized summarization. It consists of 441 Norwegian news articles across diverse topics and a total of 1,099 human-written summaries contributed by 39 unique annotators. Each article is paired with GPT-4-generated generic summary (post-edited by students for fluency and factuality), and on aver- age, personalized summaries, written by crowdworkers reflect their individual preferences and topical interests, Annotators also highlight the source sentences from the article that informed their summaries, adding explicit grounding for each summary segment.
PersonalSum annotation process. The dataset was constructed in three stages. In Stage-1, a GPT- 4-generated generic summary was created for each article, refined by human editors, and tagged with sentence-level source highlights. In Stage-2, crowdworkers were recruited to write summaries from a personal perspective. Each worker completed comprehension questions and highlighted source text, ensuring fidelity to the original article. In Stage-3, automatic quality checks using GPT-3.5- based scoring (evaluating coherence, relevance, and consistency) filtered poor-quality annotations. Only those scoring above a threshold were retained. The final dataset comprises high-quality user- personalized summaries with source alignments and user metadata anonymized for privacy.
C.4 UIG CONSTRUCTION
To construct a Uste-Intennction Graph (UIG) from preference datasets, we distinguish between two types: (i) trajectory-based datasets like PENS Ao et al.) (2021), which directly encode user inter- actions, and (ii) feedback-basee datasets like ReenAI-Reddit Völske et al. (2117). which lack user trajectories but contain document nodes (d-nodes), model-generated summaries (s-nodes), and sub- jective user feedback (e.g., ratings with confidence scores). For PENS style datasets, we first build a trajectory pool 7 using click and skip interactions (e.g., clkNews and uclkNews), but this yields an incomplete user profile due to the absence of personalized nodes. To address this, we augment T with s-nodes from the test set by inserting summa r and summGen edges at the appropriate time
21

---

Published as a conference paper at ICLR 2026
Table 9: User-Interaction Graph Statistics for our PENSS D T and EN
train
train
Characteristic
TPENS-D
TPS EN
# u-nodes (trajectories)
55,000
18,000
700
# d-nodes per trajectory
134
39
17
# s-nodes per trajectory
15
12
Average trajectory length
143
47
23
# Max. trajectory length
200
50
32
# Min. trajectory length
5
25
7
Rate of Topic Shift
0.77
0.48
0,41
steps, resulting in a derived pool TTDD_ For OpenAT-style datasets, we infer preference sequences by classifying d-nodes clicked if any associated system-generated summary received rating above a threshold (i.c., greater than equals to 6 out of 9), then selecting the highest-rated summary as surrogate s-node. The UIG construction algorithm described Algorithm
Algorithm UIG Construction
0: function CONSTRUCT DOONSTRUCT.UIG(rrain test data, dast, type)
1: Initialize TrENS: TOAI +
2: for each user train train_data do
3: Initialize 7" +
4: for each interaction in u's data do
5:
if type PENS then
6:
Map d-node with click/skip edge based on interaction else
7:
8:
Map d-node with click/skip edge based on rating if rating confidence max then
9:
10:
Map to d-node with gensum, and s-node with sumgen edge end if
11:
12: end if
13:
Append mapped d-node to 7"
14:
end for
15: Add 7 to TPENS Tox based type
16: end for
if type PENS then
18: for 7" in TPEnS do
Retrieve and insert s-nodes from test_data using genSunm/sumgen edges 20: end for
21: return PENS-D +- TPENS
22: else
23:
return TOAI
24: end if
24: end function=0
D THEORETICAL ANALYSIS: SWH STRICTLY GENERALIZES MDH
We provide formal derivations showing that the Structured Walk Hypothesis (SWH) strictly gener- alizes the Markovian Drift Hypothesis (MDH).
Lemma D.1 (Reduction: SWH recovers MDH). For any MDH update of the form
(1++) f (1)) .g) + (e)), e N(O, ((a
there exists parameter setting of SWH (Eq. that reproduces it.
Proof. Set and = 0. The increment under MDH is (e fle a(e (()).)) -e We select (1) (o (e(0//|Ae(0 (||Ae(0)) any (1)) 1 ( and set mag(a!!) mag(a(!)) |Ac0111
a(t),
(1)
22

---

Published as a conference paper at ICLR 2026
((a 0. Then + Ae( and Eq. 3 collapses to the MDH update. Noise e is matched by sampling mag adding Gaussian head tied to a!t)
Corollary D.2 (MDH variants as special cases). The The following well-known models are recovered as degenerate cases of SWH:
Pure Random Walk (PRW): mag = 0, = 0, =
Action-aware Random Walk (ARW): Same as PRW, but with action-conditioned noise (!) N(,, )(.
Random Walk with Restart (RWR): Tong et al. (2006) proposed this popular diffusion model where, at each step, the walk either (i) follows the graph transition with probability (1 (1-a) or (ii) "restarts" a fixed seed distribution q with probability a. This ensures the walk remains centered around the most recent interaction and does not drift arbitrarily far. In our SWH framework, this
corresponds to choosing 4 = O (q b,w) with 0 = 0 and E (0, 1).
Lemma D.3 (Short-memory encoders are contained). Any continuous short-memory update
fleba (t)) (e.g.. GRU, attention block) is realizable by SWH with v 8 0, since 4 (ebbur all))
-bba"
Theorem D.4 (Strict Generalization). SWH strictly generalizes MDH:
1. Every MDH update is realizable by SWH (Lemmas D.I.2
2. There exist SWH updates that no MDH update reproduce.
Proof. (i) follows from the lemmas. (ii) Consider two histories H4. H2 with identical (e (()) but
distinct memories (hę h, ). Eq. yields increments Ø + (h, hT)) ) + and & + (h, which differ. Since MDH is first-order Markov (cb,u (9) 9). cannot separate Hz from H2. Thus SWH has strictly greater expressivity.
14 h4 + 6,
Corollary D.5 NNon-Markovianity from dual memory). If V aggregates asymmetric signals multiple timesteps, the resulting process is non-Markovian with respect to the pair (eb al)),, and hence lies outside the scope of MDH.
Proof Sketch. Under MDH (Eq. []. the next state depends only on the current preference embedding b. and action (1). Thus two different histories Hi and H2 that lead to the same (e a(") are indistinguishable, In contrast, SWH retains separate reinforcement and suppression traces via dual memories (h 1,)) ). If Hi contains repeated clicks on climate policy and H2 contains repeated skips of celebrity news, both may yield the same (t) although (h;* .h,) differ. Consequently. V (h, hT)
produces distinet updates Eq. 3 Therefore the transition distribution P(e (((((t))
6 (()) Hi)
(t)
depends on the full history Hi, not just the present state, violating the Markov property.
Remark. This shows that even when the visible preference embedding (t) b, is the same, Walk2Pers can differentiate users with distinct long-term interaction traces. This is impossi- ble under MDH. where all memory beyond the latest step collapsed into g. In other words, SWH reduces to MDH when memory and drift are disabled, but enables trajectories to diverge even when current states coincide. This matches the empirical evidence that user preferences depend on long- horizon, action-specific dynamics (see Table 3).
E BASELINES
BASELINES FOR RQ-1/2: TESTING MDH VS. SWH COMPONENTS
To evaluate whether the Markovian Drift Hypothesis (MDH) suffices or if the Structured Walk Hy- pothesis (SWH) is necessary, we compare Walk2Pers against a suite encoder variants and established news recommendation encoders:
23

---

Published as a conference paper at ICLR 2026
Table 10: Encoder Models are Markovian Drifters: Walk2Pers uniquely integrates all aspects of SWH; history capped by context length; no persistent memory across examples.
Medels
Habley
Ytaneey
Action Canditioning
Explist Step Ling-len Persisintt Menuey
Interpettbbliiy
Oradee (igglin SmrLS, 15cmad)
(golt mjpared a aso)
(Zylly,, LL DecySook
(rjected poonge)
PRW
SMD (Shae Menayyy bnitt) (ours)
AJOK
U-BLWW
KAST
NRNS
Jistum
FHNNR
CTP
Vineesied
MalicPace (OBr))
Vidaao
Short-Memory Gate (SMD). This is the minimal baseline consistent with the Markovian Drift Hypothesis. At timestep 4, the update depends only on the immediate tail state and current action embedding via a convex gate:
(t (anhhfSah pla.t))
(3.)
(5) tanh (Wo cel ((l)))
SMD
etl
b,u.
where the gate is defined as: fla.t) n (W. (4:)
(1 (W, Gl (4 ) Here, 1) E (0, is learnable scalar which balances the contribution of the action embedding (4)) and the prior tail state al (tz-1), Because this update overwrites history at each step, it carries no mechanism for reinforce- ment, suppression, or geometric decomposition, and thus strictly follows the MDH assumption.
Action-Specific Gates (AGD). Instead generic convex combination, each action has its own parameterization of f(-). We design f(a.t)) separately for click, skip, summariza ize, and summGen, so that each action boosts, mutes, or rebalances user interests. A click encodes posi- tive reinforcement, implemented as history-aware gain:
f(cli.t)) = (Wei- (bs)) + c
Czik
(t4-1)
Cu
(t)) = tanh c(1
(1:)
(17)
If Alice clicks on "Concert in New York". the gain reinforces music-related features her history. In contrast, skip reflects either disinterest in the current document or pull toward an alternative. In terms of the timesteps of the trajectory Tux this would be look-ahead timestep 4++1 where (f.)
is compared with (4-1) via dot product (-). We model this alternative attraction towards a more preferred (in contrast to disinterest) alternative as a deviation Tum
(le) plikn.tt) tanh W e skp
)
(11) where: gmove x1100 (4.) W pull (t.) (t.). Nattr
+
Siove eel
(18)
(1.) Nattr max((c)) (11-1) eel (141)) 1 (cu eil ())))); (b4)
(1--1) (1.)
c. tanh(
(t.) e
Here, (1+) non-linearly distorts the trajectory. \1, X2 are learnable constants. To illustrate, Al-
ice may choose to skip Election Policies article (due to disinterest) leave, say "Music During Cooking", for more preferred "Latest Concerts by Pink Panther" (i.e., the future pull).
summary request, i.. s summarize indicates focused intent anchored the title of the document:
f(summ,te) (e) = (W summ
(f.) Suntin
O c
(f,-1)
(t.) = tanh( (numm.))) Ci
(t,)
(19)
(itl
Finally, generated summary s sumnGen evaluates whether the summary is concise in terms of expec- tations and faithful to the source content:
j ((summGGen.t.) - log exp(gaummen
e (4)) + exp((1 -gsmmneen) @ e
(tc) tanh(f (summeentt)))
Cil (!:-1)
gsummGen of W min W SummGen
(1)) summGen))
(20)
The gate gummGen balances coverage of the source along with history alignment, with learnable tuning the trade-off between them. We generalize the gates f(ants) AGD(e., t,) @ Chistory where AGD determines which function to trigger based on the action a and enistory is the embedding of the previous state history.
24

---

Published as a conference paper at ICLR 2026
E.2 BASELINES FOR RQ-3 (PERSONALIZED SUMMARIZATION PERFORMANCE)
For downstream evaluation on personalized summarization, compare against three categories of strong baselines:
E.2.1 BASELINE PERSONALIZED MODELS
PENS-NRMS Injection-Type The PENS framework (Ao et al. 2021> generates personalized summaries by incorporating user embeddings along with the input article. For this variant, user embeddings are derived using NRMS (Neural News Recommendation with Multi-Head Self- Attention) (Wu et al. 2019b), which includes a multi-head self-attention based news encoder to represent news titles, and a user encoder that captures browsing behavior through multi-head self- attention over clicked articles. Additive attention mechanisms are employed to highlight important words and articles. In Injection- 1, the NRMS user embedding is injected by initializing the decoder's 's hidden state, thereby directly influencing the summary generation process from the start.
PENS-NRMS Injection-Type 2. This variant also uses NRMS for user embedding, but personal- ization is introduced differently. Instead of initializing the decoder, the user embedding is injected into the attention mechanism of the PENS model. This modulates the attention weights over the news body, enabling the model to focus on content aligned with the user's preferences.
PENS-NAML Injection- Type 1. NAML (Neural News Recommendation with Attentive Multi- View Learning) (Wu et al. (2019a) generates news representations by attending over multiple views, including titles, bodies, and topic categories. The user encoder learns from interacted news and selects the most informative content for personalization. The resulting user embedding is integrated into the PENS decoder using Injection-Type i.e., by initializing the decoder's hidden state.
PENS-EBNR Injection-Type 1. EBNR Gimbedding-bassee News Recommendation) OOkura al. 2017) models user preferences using an RNN over browsing histories to produce user em- beddings. These embeddings are injected into the PENS model via Injection-Typpe 1 by initializing the decoder, thereby influencing the initial decoding steps with user-specific information.
PENS-EBNR Injection-Type 2. This configuration uses the same user encoder from EBNR but applies Injection-Type 2. Here, the user embedding is incorporated into the decoder's attention layers, allowing the model to personalize attention distributions over the news body during decoding.
General Then Personal (GTP). General Then Personal (GTP) (Song et al. 2023) a two-stage framework for personalized headline generation. In stage- 1, Transformer- based encoder- decoder model is pre-trained on large-scale news article-headling pairs to learn robust, content-focused head- line generation without personalization. In stage-2, a separate "headline customizer" refines the gen- eral headline by incorporating user-specific preferences, which are encoded as a control code by the user encoder TrRMIo. To bridge the gap between general generation and personalized refinement, GTP introduces two mechanisms: (i) Information Self-Boosting (ISB), which reintroduces relevant content details from the article to prevent information loss during customization; and (ii) Masked User Modeling (MUM), which randomly masks parts of the user embedding during training and reconstructs them, reducing the model's 's over-reliance on its general parameters.
Signature Phrase. Another line of personalization focuses on condensing user's reading history into a collection of signature phrases (Cai et al., 2023). These phrases, derived through contrastive learning over news articles without annotated data, act dynamic user profiles that adapt interests evolve. Such phrases need not appear verbatim in the user's history but instead encode higher-level signals. Using these phrases, the model learns generate personalized headlines that connect new articles with the user's inferred interests, yielding outputs that are engaging, relevant, and grounded in article content rather than drifting toward clickbait.
These encoders serve as competitive MDH-aligned baselines, since they reduce trajectories into compressed short-term embeddings with no explicit long-term reinforcement novelty modeling.
25

---

Published as a conference paper at ICLR 2026
E.2.2 BASELINE LLMS
Zephyr 7B B. Zephyr[Tunsten Tunstall et al. 2023) is 7 billion parameter transformer model fine tuned from Mistral 7 billion using Direct Preference Optimization on publicly available and synthetic data. It removes traditional alignment constraints to improve raw performance and achieves strong results on benchmarks such as MT Bench where it reports a score of 7.34 in comparison with 6.86 for LLaMA 2-70B Chat. Zephyr is optimized for helpful dialogue and is openly available under an MIT license. It focuses on efficiency and high quality responses without relying on reinforcement learning from human feedback. The model supports an input context length of 32K tokens.
Mistral 7B. Mistral Instruct[Jiang et al. 2023) is dense transformer model that uses grouped query attention and sliding window attention to efficiently scale with long context inputs. It is pretrained on approximately two trillion tokens and provides strong performance across natural lan- guage and code generation benchmarks, surpassing models such as LLaMA 13B in many evalua- tions. Mistral Instruct is fully open source under the Apache 2.0 license and includes instruction tuned variant that is widely adopted for fine tuning and deployment. The model supports an input context length of 32K tokens.
LLaMA 13B. LLaMA two Touvron et al. 2023) 13 billion by Meta is an autoregressive trans- former trained on two trillion tokens of public data with context length of 4096 tokens. It sup- ports chat interaction through instruction tuning and reinforcement learning from human feedback. Although originally state of the art among open models, it is surpassed in many tasks by newer chitectures such as Mistral-7 billion. LLaMA-2 remains an influential and widely used foundation model with extensive documentation and open access under the Meta license.
DeepSeek R1 14B. DeepSeek RI (DeepSeek-Al et al. 2025) is 14.8 billion parameter model dis- tilled from 2.e 14 billion and is designed for strong performance on mathematical reasoning, coding, and multi step logical tasks. It is fine tuned on eight hundred thousand examples gener- ated by a larger DeepSeek R1 model and is released under an MIT license. Despite its moderate size, it rivals substantially larger models on benchmarks such as AIME and MATH while remaining efficient for customization and deployment. The model supports an input context length of 128K tokens.
Gemini 2.5 Flash. Gemini-2.5 Flash is a compact and highly optimized member the Gemini family designed for fast inference, large scale retrieval augmented generation, and multimodal in- teraction. provides high throughput generation with reduced latency and suitable for production applications that require consistent responsiveness. Gemini 2.5 Flash incorporates the same unified multimodal architecture used across the Gemini series and has been trained on a large mixture of text, code, and image data. The model supports input context length of I million tokens, making it particularly effective for long document synthesis and extended conversational sessions.
Qwen 3 235B. Qwen-2 2 235 billion is a large scale frontier transformer model developed as part of the Qwen three family. It is trained on extensive multilingual and multimodal datasets with a focus on reasoning, tool use, and high fidelity generation. The model is designed for advanced analytical tasks, multi hop reasoning, and instruction following at very high capability levels. Qwen-3 serves as foundation for many distilled and specialized variants and openly available for research and commercial use under a permissive license. The model supports an input context length of 256K, allowing operate effectively on extremely long sequences in both textual and mixed modality settings.
E.2.3 BASELINE GENERIC SUMMARIZERS
BigBirdPegasus. BigbirdPegasus, proposed by (Zaheer et al., 2020) is an extension of Trans- former based models designed specifically for processing longer sequences. It utilizes sparse atten- tion, global attention, and random attention mechanisms to approximate full attention. This enables BigBird to handle longer contexts efficiently and, therefore, can be suitable for summarization.
A Simple Framework for Contrastive Learning of Abstractive Summarization (Liu & Liu 2021 uses two-stage training procedure. In the first stage, Seq2Seq model (Lewis al.
CLS.
26

---

Published as a conference paper at ICLR 2026
2020) is trained to generate candidate summaries with MLE loss. Next, the evaluation model, initi- ated with RoBERTa is trained to rank the generated candidates with contrastive learning.
T5. TText-To-Tese Transfer Transformer) is based on the Transformer-basedd Encoder- Decoder chitecture that operates on the principle of the unified text-to-text task for any NLP problem, includ- ing summarization. Some recent analyses the performance of T5 summarization tasks can be found in (Raffel et al. 2020
Tawmo et al.
2022 Ramesh et al. 2022
Etemad et al. 2021
F LICENSE AND USAGE STATEMENT
In this work, we utilize the following pre-trained large language models (LLMs): DeepSeek-RI 14B (MIT License), Mistral-7B-Instruct (Apache 2.0), LLaMA2-13B (Llama 2 Community License), and Zephyr 7B (3) (MIT License). All models are used according to their respective licenses and terms provided by their original creators. Proper attribution is given to each model's developers as cited in our references. We also use the following datasets:
MS/CAS PENS dataset: We comply with the dataset's terms of use, which is de- rived from the Microsoft Research License (bttps:/ /ttp:////gttubb/t// com/ Ttccb/master/MMs22lic.cooo
Data. pdf).
OpenAl Reddit dataset: We comply with the MIT License specifications as set by OpenAl htttps://githu htatttuu..co////ge cwm openai/peaa////ee samppeniissmmarizeee summarumizaao from feedback/ Dllch/masterrLIIEENS
We have ensured that all datasets and models are used responsibly, respecting privacy, consent, and ethical guidelines. When applicable, data is anonymized and handled according the ethical standards set forth by NeurIPS.
G WALK2PERS ARCHITECTURAL DETAILS
G.1 WALK2PERS- ARCHITECTURAL DETAILS
Magnitude and Orientation Computation. To capture the direction and magnitude of movement in complex embedding- -based manifold, we proposed geometrical walk of behaviors 3.3.1 This walk conceptualizes preference evolution as sequence of directed steps in semantic space where each behavior is obtained by rotating and scaling the previous embedding to get the geometrically (t:) We compute the orientation degree of the update as follows: contextualized ecbay"
v(4) - (u(t)) u(t, u(fs-1)) rilts 1)
g(4) = π tanh
We of W angle b,a 1) o(())
max (1o(ts) )) (u(ts), ullt-1)) ulti-all2, 8)
(t,)
(1--1)
(4)
(((+)
6,
b,u
; (((=-))=
6,0
max (118)
(1,)
(b1-1)
11,, e)
max (lee ll,, E)
bug
(21)
The angle ((t,) E (- R) serves as directional drift controller, deciding whether the trajectory should continue smoothly (small g(t)) or shift sharply (large ((ta)). The vector (t=-r) represents the forward direction inherited from past context (carrying forward the momentum current flow of history), while (((,) denotes the orthogonal novelty axis derived from the raw transition direction g((4), For example, if Alice has consistently read Climate Science Articles, (tt=-1) encodes this mo- mentum, while a sudden click on a Policy Debate yields a transition vector with a strong orthogonal component (f4+) a small ((()) implies smooth thematic extension (science into policy), whereas large ((r,) reflects a sharp diffusion toward politics.
To determine how far e travels along the chosen direction, the magnitude of movement is regu-
lated as:
mag (t. Softplus W. Wn. el.."))
(22)
27

---

Published as a conference paper at ICLR 2026
where mag(t)) acts like the distance of the walk, dictating whether the model advances cautiously or moves decisively.
(tt) Gebuy b,a + magitt) (cos tht sin ((tt) olte))
(1.)
(23)
This ensures that preference evolution is represented as a directed step, combining aligmment with historical momentum and deviation toward orthogonal novelty. For Alice, this means her reading trajectory can smoothly extend within climate science when ((4) is small and mag(t)) is low (steady interest), or make a decisive into polities when ((6) is large and mag(4) accelerates the shift (sharp interest change).
G.2
WA WALK2PERS ENCODER AS AN INSTANTIATION OF SWH
We now formalize how the wai.22eers-encoee concretely realizes the SWH update (Eq. 3) For reference, recall the SWH rule:
(++1)
b,u mag(a cos cos (()(0) ( + sin ((a
6,u
4: geometric step
(hi + + ( summGen]
dual menorry
drilt
Theorem G.1 (Walk2Persan instantiation of SWH). Walk2Pers-encoder update is exactly of the form Eq.
With the definitions of Sec. 3.3.1, the
Detailed Sketch. We match each component of Eq. 3 to the Walk2Pers.
(A) Geometric step. The encoder produces two scalar heads (m, Qc) for each action (t);
ma)(a softplus(m.)
0,
(() = n. (0.) E (0,
These control the length and orientation of the step. Next, Walk2Pers maintains two orthogonal unit axes: (9)); the "momentum" direction, aligned with recent preference drift. o!); orthog-
onal "novelty" axis obtained via Gram-Schmidt. Together, these yield the geometric increment
+ = mag(a (*) (cos)(ccoo))) (()())))()))) + sin (()))))))))))
Intuition: This term says each action either pushes the state forward in continuity-preserving direction (small 0) rotates into novel axis (large 0), with strength controlled by (a)(
(B) Dual memory. Walk2Pers keeps two asymmetric accumulators:
h = h + (t) @ (c) h h C (() (1)
Here (t) S + W)) ) is an action gate, and ( the content embedding.
Thus: h+ reinforces positively gated interactions (e.g. clicks), accumulating them additively. h suppresses negatively gated interactions (e.g. skips), attenuating old signals via (1 - (t))).
The lanes are blended with learnable weight w E (0, 1):
(t) whi + (1 w)
Finally, linear projection produces the memory contribution:
(h (,) = Wv [h<h::
Intuition: Unlike MDH, which forgets everything beyond the last step. this mechanism lets Walk2Pers reinforce long-term positive signals while still allowing suppression repeated dis- interest. Thus V injects Mar-Marov,ann, history-dependent bias into the walk.
28

---

Published as a conference paper at ICLR 2026
(C) Drift. For the summGen action, Walk2Pers triggers extra summary drift term. A drift vector å = (xds(()) is generated and added only when a(*) summGen:
A (0) 8 8 [aa summGen].
This nudges the preference state toward more "condensed" representations when the user explicitly asks for summary, acknowledging that summarization requests are qualitatively different from passive clicks skips.
Intuition:
Putting together (A)-(C), the Walk2Pers update rule is
(t+11
(t)
+ t. + whh k,) A
h,
b,,
which is identical to Eq. Hence Walk2Persa concrete instantiation of the Structured Walk Hypothesis.
Remark By parameter restriction: setting v/ 0 and 5 0 collapses Wa k2Pers pure
geometric update. Further restrictions recover PRW. ARW, and RWR as shown in Cor. D.2
G.3 WALK2PERS-DDE -ECCODER DETAILS
The query b-node embedding by predicted by the en- coder represents a geometrical alignment infused entanglement of behavior duplet (genSumm, sg)- The 8y represents the latent s-node embedding of the query document d,- Since (1142) has in- fused the learned action embedding, becomes difficult for a decoder to feed on and generate a personalized summary. To address this, extract the latent s-node embedding from b (k++1
Latent Summary Contextualization
(f11)
(ft
Latent s-Node Contextualizer. Although latent s-node embedding 6s, represents the user's personalized summary intention, it lacks the contextualization of the query document dy- The base
decoder contextualizes dg with (4(11) using cross-attention, and feeds to summarizer decoder. (t++1)
The query document ed, serves the query, and e acts both key and value, resulting sum- mary contextualized query embedding as:
(W, ed,)) TT,,)((H T(W Gsgie
(f141)
eq, = SoftMax
(W, e((t-aa))
(24)
Vd
This contextualizes the document to be summarized with the latent summary intention of the user, but the query document ed, lacks explicit user-preference representation.
User-based d-Node Encoding. To incorporate explicit user preferences into the document repre- sentation, enrich the query document embedding with the user's interaction history. Specifically, the encoder of the summarizer model produces the base document embedding ed, In parallel, the fi-
nal user history vector e from the Walk2Pers encoder applied as gating signal to modulate ed,, yielding the user-aware document embedding:
edg.
W
(tt) bb.
④ ed,
(25)
The gating makes the query document align with the user's own preference, while irrelevant aspects are suppressed. This ensures the document passed through the lens of Alice for generating the expected summary for her, and the same document passes through the lens of Bob for generating his expected summary. ed,.u encodes both the semantic content of the document and the personalized preference profile of the user, enabling more faithful contextualization in subsequent cross-ate aatss-aattntion. The latent s-node then contextualizes Cd to produce of as discussed Section 3.3.2
Personalized Summarization. The Walk2Per: decoder generates a personalized summary by feeding contextualized query document embedding cg We use the T5-base (Raffel et al. 2020) decoder for the summarization.
29

---

Published as a conference paper at ICLR 2026
Decoder Training. The decoder training objective (Ldee) is linear combination of two loss fune- tions, Generation Loss (Egen) and the earlier encoder loss (Cene see Section (.3.2), as Lae = 3 Lgen + (1 (1-33 3) Lenc. Here, Eyen is the cross-entropy loss between predicted tokens ý and ground-truth y" under teacher forcing with the TS decoder. Optimizing Lyen updates the cross- attention layers and language-modeling head of T5 decoder, contextualizer weights WE, Wg, W, and mapersg-mapping weights W summe W Fine-tuning the cross-attention layers ensures that the decoder learns how to properly fuse the contextualized document embedding ec. (dg. with the la- tent s-node embedding, thereby injecting user-specific preference signals into the decoding process, and fine-tuning the language modeling head adapts the token generation distribution reflect this personalized conditioning, improving lexical and stylistic alignment with the user's history. This
(fr11) and stronger cross- cooss-attention with document ensures accurate latent s-node reconstruction 0zg. embedding, thus improving summary relevance.
All notations related to methodology are enumerated Table 11
H IMPLEMENTATION DETAILS
H.1 COMPUTE RESOURCES
All data preprocessing (behavior graph generation, embedding lookup, and probability space map- ping) was performed on CPU machines with 16GB memory per core. Embedding tables for news bodies, headlines, and summaries were initialized using shared vector space seeded from pre- trained transformer T5-bas base encoder model Raffel et al. (2020). All training and inference experi- ments for Walk2Pers were conducted using mixed mixed-precision (FP32) training on L40S and A100 GPUs. We gratefully acknowledge Lightning.ai for providing virtual compute resources with A100 and L40SS GPUs. Wa 22Pess utilizes almost 95% cheaper resource utilization and costs than the best baseline LLM DeepSeck-RI. We summarize the detailed training and deployment details of Walk2Pers in comparison to best LLM DeepSeck-RR RR inference in Table I2
H.2 TRAINING
Model training conducted end-to-end across the full pipeline for 6 epochs (approx. 20 hours), and then with frozen encoder for 18 epochs (approx. 35 hours), over 63K unique behavior se- quences. Optimization employed Adam (PyTorch v2.0.1) with learning rate of 1 10-3 The decoder operated with teacher-forced supervision using T5, with a learned adapter vector injected as the decoder token in the first layer to guide personalized generation. Loss was weighted com- bination of classification loss over encoding of nodes, behavior node prediction and cross-entropy loss on the personalized summary output. Hyperparameter details are in Table 13
H.3 T5 MODEL
The Text-to-Text Transfer Transformer (T5) (Raffel et al. 2020) is unified framework that casts all NLP tasks, ranging from translation and summarization to question answering and classification, into text-to-text format. This design choice enables the use of single model architecture and training objective across a diverse set of tasks. The T5 model is built upon the standard Transformer architecture and is trained on large corpus called the "Colossal Clean Crawled Corpus" (C4). Among its variants, T5-base consists of 12 Transformer layers in both the encoder and decoder, with hidden size of 768 and 16 attention heads, totaling approximately 770 million parameters.
Decoder. The decoder in T5 follows the autoregressive language modeling paradigm, predicting the next token conditioned on previous outputs and the encoder's representations. It incorporates a stack of masked self-attention layers, encoder-decoder cross-attention layers, and feed-forward layers. Unlike the encoder, which allows full bidirectional attention, the decoder's self-altention is causal (i.e., left-to-right masked) to prevent information leakage during training and inference. Each decoder layer attends to both the previously generated tokens and the encoder outputs, enabling the model to align and condition generation on the input sequence effectively. Position-wie wise feed- forward layers and layer normalization used after each attention block. During fine-tuning, the
30

---

Published as a conference paper at ICLR 2026
decoder is trained to generate task-specific outputs, such summaries translations, making central to the T5's generalization across tasks.
1 DETAILED RESULTS
1.1 HUMAN-JUDGMENT INTERPOLATION FROM OPENAI-REDDIT DATASET.
The interpolation of human judgment scores is performed by leveraging the OpenAI-Reddit dataset, which provides multiple human-rated summaries for each article. For every article, the highest- rated human summaries (7 out of 7) are designated as the benchmark reference. All candidate summaries, including the benchmark, are first embedded into high-dimensional semantic space using a Sentence Transformer (Reimers & Gurevych] 2019) model. The semantic deviation between the benchmark embedding Vo and any other summary embedding V. quantified via the Root Mean Square Deviation (RMSD), which in this context is equivalent to the Euclidean distance:
RMSD(V,, V.)
Σ (b: 0,)3.
In practice, this computation is implemented efficiently using NumPy's linear algebra module, np. linaig. norm. The resulting RMSD values are then grouped according to the original hu- man rating of each summary (e.g.. 7/7,, 6/7). By averaging the RMSD) values within each rating group, we obtain a mapping between human-judged quality scores and embedding Notably, the RMSD for summaries rated 7/7 is not always zero, as there may exist multiple distinct summaries with a top score for the same article; while all such summaries are judged as equally high-quality by humans, their semantic embeddings can still differ due to variations in phrasing, emphasis, or lexical choices. These aggregated averages form the scoring thresholds used for inter- polating human judgment our evaluation framework.
distances.
1.2 ABLATION ON CLICK-ONLY TRAJECTORIES.
We ablate Wa k2 under click-only trajectories, which results highly imbalanced action dis- tributions to investigate whether dual-memory components degenerate when skip actions become extremely sparse. We observe that removing the geometric aligner leads to a moderate degrada- tion (avg.-0.08) reaffirming that sparsity weakens cross-action disentanglement. However, the dual-memory encoder still remains stable and continues to extract preference signals from docu- ment transitions, outperforming all competing baselines even in this extreme regime. Although Qwen-3 shows the smallest range of degeneration of performance w.r.t the original performance, Walk2Pers consistently outperform and show moderate degeneration. These results confirm that (i) the geometric aligner is necessary for full robustness under skewed action distributions, and (ii) the dual-memory lanes themselves do not collapse, even when skip/summarize actions nearly dis- appear. Results are in Table 15
1.3 ABLATION ON Low FREQUENCY TOPICS.
We also ablate on 200 subset trajectories from the test data to understand whether rare, infrequent but relevant topics get oversuppressed by the memory lanes or geometric novelty of Walk2Per These 200 trajectories have a higher topic frequency (121 vs. 105) and higher rate of topic change within a trajectory (0.63 vs. 0.54) than the entire test dataset, indicating the occurrence of rare, infre- quent topics within these trajectories. We find that although there is degradation in performance, the model still extracts stable user preferences, validating that the design principle of incorporating learnalle memory lanes and geometric novelty aligner is necessary understand users' evolving interests in any real-world setting. Detailed results are in Table [6]
1.4 PERFORMANCE W.R.T. ACCURACY.
We evaluate the accuracy of Walk2Per: w.r.t gold-reference summaries under standard accuracy evaluation metrics Rouge-L and Rouge-SU4 (Lin 2004), and find that Wa 1 k2Pers outperforms
31

---

Published as a conference paper at ICLR 2026
the baselines by average margin of 22.3/24.1 on RG-L and RG-SU4, respectively. This confirms that a boost in personalization capability also bridges the lack of accuracy gap. Results are in Table
18
J PROMPT SETUP
2-shot w/ history. This setup provides the model with complete user history that includes inter- actions with previous articles in the form of clicks, skips, and summaries. Two in-context examples shown before the actual task, where each example contains the article content and personalized headline rewritten by the user. These few-shot examples serve as demonstrations to help the model learn the structure the desired output. Given a new query document, the model is instructed to generate a personalized headline by considering the user's ' history: click indicates positive inter- est, skip p indicates disinterest, and s summari zed indicates focused preference. The headline is to be returned in specified format.
O-shot history. In this variant, the user history again presented as a list of past interactions including clicks, skips, and summarizations, but no in-context examples are shown. Instead, single task prompt is provided that explains the significance of each action type. The model is instructed directly this user history infer the user's s interest and produce a personalized headline for given query document. This prompt relies on the model's zero-shot reasoning capabilities without relying on demonstrations.
Prompt-Chaining w/ history. This method adopts multi-stage interaction design. In the first step, model receives a single document topics, keyphrases, and user preferences based on that interaction. The output of each step is ac- cumulated to incrementally build a structured user profile. As new interactions (c.g.. skips or sum- maries) occur, the model is repeatedly prompted to refine update the user's inferred preferences. Finally, when the query document is given, the model uses the constructed user preference profile to generate a personalized headline. This setup simulates long-term personalization via chaining and stateful interaction across multiple prompts.
user action (e.g., click), and asked to extract
32

---

Published as a conference paper at ICLR 2026
0-shot w/ history.
User History
List of Articles clicked/SkiippeddSummaiizzz by
user:
<Doc1: click>, <Doc2: click>, <Doc3: skip>,
<Doc4: Summarized as Sum1>........
Task
Given Query Doc sdoc_content
Generate Headline by considering the user's history the indicator to their interests, where click denotes positive interest, skip denotes negative interest and summarized indicates focus on that topic. Return the headline in this format: Headline: (output)
Figure 5: O-shot Prompt-Template Template for LLM baselines.
33

---

Published as a conference paper at ICLR 2026
Table 11: Notations used across Sections 3-4 and the Appendix. We group symbols by (i) the Interacttnt Graph (UIG), (ii) hypotheses and structured walk components (SWH), and (ii) the Walk2Pers instantiation (encoder, contextualizer, decoder).
Symbol
Explanation
User-Interaction Graph (UIG)
(N
User Aber-lnteraction Graph with nodes N and edges
(
User node initial timestep to
Document at timestep
81
Summary node at timestep f for d((-1)
Action at time t (al ckk akk skip, genSumm, summGien
Behavior duplet (a(")_11 (( (action tail node)
y
User trajectory (ordered sequence over f) Training and trajectory pools
Tixen T
Hypotheses & Structured Walk (MDH vs. SWH)
Latent preference state b-node after timestep I One-step update under
F()
(short-memory)
Recency prior (e.g.. restart distribution in graph diffusion)
Stochastic perturbation in MDH updates
E(a(0)
Action-cone cedion-cnditione covariance of
((-)
Geometric step: continuity vs. novelty (SWH)
V(--)
Dual memory: reinforcement (h: suppression
(()
Drift: special action-induced shift (e.g.. summGen)
Momentum axis (continuity direction)
ofo
Orthogonal novelty axis at Step magnitude (strength of update)
mo)(a!")
((a
Step orientation angle (mix a! vs. o())
4t.
Positiveeneeatiie memory
(click reinforcement/ skip suppression)
5
Summary-spenn specific drift vector (active for summGen)
Walk2Pors Encoder (b-layer), Sec. ??
f(
Action-gate function inside b-cell at time Raw tail-node embedding time
6/
Action embedding (4-d one-hot or learned) at time Tail-cell content (flowing history) at time t specitcc projection for
(0))
Wak
W:
Projection to b-node space
b-node embedding produced the b-cell Combined memory state time Positive (riiffrremmnt) Negative (suppression)
h
hit00, h W--0
(9)
Gate for memory routing/strength (e.g.. SoftMax features)
A
Drift vector injected on a summGen, (I <u ee
grt)
Predicted orientation angle time
RfF-1) 60
Momentum axis
1), novelty (at
mag(() (t)
Predicted step magnitude at time
Contextualized b-node after applying d, 2,
Waza
Lineur to predict next b-node embedding
Predicted next b-node embedding (query b-node)
Position classifier weight alignment objective
Predicted position distribution for alignment
Calgas Caza Less
Encoder objectives: position alignment, next-b-node, joint
Latent Summary Contextualization (Decoder-side signals)
(tea)
Latent s-node (summary- intent))) for document d,
query- query-document document embedding (backbone encoder)
6d
User-aware doe embedding (gated by trajectory state)
Doe embeedding contextunlized by latent s-node
Final user-conditioned doc embedding for decoding
Wg. We, W.
Cross attention projections (query, key, value)
W
Gating projection for ex,,
W
W++
Inverse-mapping weights used to extract latent s-node
summ
Decoder Objective
Eyo
Token-level generation lnss (eross ennss-enttpyy.. teacher forcing)
Decoder objective: C (1 Ge
34

---

Published as a conference paper at ICLR 2026
Table 12: Training and Deployment Resources Summary of Walk2Pers in comparison to best
performing LLM baseline DeepSeek-RI.
Metric
Our Model (170M)
2-shot LLM (14B)
Relative Gain
Parameters
170M
14B
82x smaller
Avg. summary length
tokens
20 tokens
FLOPP/summary
2.04 x 10th
1.68 10
82x lower
Inference time (per summary, est.)
0.2-2
15-160
60-80x faster
Running cost (GPU-hours)
18
42
orders lower
VRAM foolprint
<1 GB
>28 GB
edge-deployable
Table 13: Learned Weights and Hyperparameters of Walk2Pers.
Component
Shape/ TType
Notes Inii
Training Configuration
Batch size
38
Fixed across encoder/decoder
Optimizer
AdamW
PyTorch 2.0 impl.
Learning rale (end-to-end)
2 10-4
End-to-end Training
Learning rate (decoder finetuning)
3 10-3
Decoder fine-tuning
Epochs
6+ 18
6 joint. 18 decoder-only
Action Encodings
Colk. Eskos Csummn Cammees
One-hol action basis Action transforms, no bias
Wat. W dlp. Waumm, W Wamnum
(768, 4)
State & Memory Transforms
W
(768, 1)
Skip attraction transformation
Wo, W2
(768, 768)
State transforms
WA. W.
(768, 768)
Memory routing gates
At. (0
768
Reinforcement suppression memories
scalar
Memory lane mixing weight; learnable
Fusion Layers
W., W., W.
(768, 768)
Fusion linear transforms
6.
Fusion bias (zeros)
Wemb
(768, 768). bias 768 768
Embedding proj., std init Zeros init
Geometric Step (Orientation and Magnitude)
Wangle
(768, 768)
Transforms to direction signal
Wg
(1. 768)
Orientation angle transformation
Wi.
(768, 768)
Shared with fusion
W.
(68)
Magnitude scaling transformation
(14)
scalar
Orientation angle, (-x,
mag())
scalar
Step magnitude
Prediction Decoder
(768, 768)
Tail b-node proj-
W Owss
(68) 768
Next-node prediction
W
(768, 768)
Positional alignment classifier
W WA, W.
(768, 768)
Cross-attention projections
W
(768, 768)
User-awure doc gating
Wamm W
(768, 768)
Inverse mapping for latent s-node
Complexity
Per-step
O(d) O(nd), T O(Ld))
For each b-cell update
Condensed b-layer
Long-horizon compression
Decoder
Transformer (T5-base)
Table 14: RQ-1/2: Next b-node Prediction (PENS Dataset) Mean (/) & Standard Deviation (o): Standard Deviation of SOTA MDH user-encoders is close to the Mean, thereby strengthening the reported performance gain; t NT: Originally published results were on the news recommendation task in contrast to next behavior prediction.
Metric
NAML;
NRMS:
EBNR
Walk2Purs- Encc w/o Geo. (mean)
Malk2Pery- Enc. (mean)
MRR,
MRR,
0.0163
0.001
0.0009
0.008
0.0009
0.0101
0.121
0.23
nDCG@5
0.0004
0.0002
0.0003
0.082
0.198
DDCG@5,
0.0176
0.01
0.012
nDCG@S,
0.0007
0.0004
0.0005
DDCG@5,
0.0199
0.0128
0.0146
0.132
0.249
35

---

Published as a conference paper at ICLR 2026
Table 15: Performance degradation under sparse-click evaluation. Left value indicates the original
score; right value indicates sparse-click score; percentage drop from original values under this sub- set. Lower drop indicates better robustness to interaction sparsity.
Model (Sparse Click-only Test)
PSE-JSD
PSE-SU4
PSE-METEOR
DeepSeck-R1 (2-shot)
0.248/0.147
(-40.7%)
0.094/0.064
(-31.9%
0.091/0.071
(-26.8%)
Gemini-2. 5-5aalhh (2-shot)
0.222/0.12 122
(-45.0%)
0.104/0.061
(-41.3%
0.124/0.070
(-43.5%)
#wem3-2358-Thinking (2-shot)
0,105/0,103 103
(-1.9%)
0.082/0.073
(-11.0%)
0.082400071
(-13.4%
Best MDH Baseline (GTP)
0.024/0.016
(-33.3%)
0.170/0.009
(-94.7%)
0.019/0.011
(-42.1%)
Wal Walk2Pees (w/o Geometric Step; Only Memory) Walk2Pors (Full)
0.306/0.231
(-24.5%)
0.334/0.253
(-24.3%)
0.321/0.234
(-27.1%)
0.452/0.378
(-16.4%)
0.383/0.501
(-21.4%)
0.449/0.310
(-31,0%
Table 16: Performance Top 200 Low-Topie Now-Topic-Frqqueeee Frequency Trajectories. Left value indicates the original score; right value indicates sparse-click score; percentage drop from original values under this subset. Lower drop indicates better robustness topical sparsity.
Category
Model
PSE-JSD
PSE-SU4
PSE-METEOR
DeepScok-R--4B
0.248/0.001
(-63.3%)
0.094/0.774
(-21,35%
0.097/0.082
(-15.5%)
Gemini-2.5-Flase
0.222/0.092
(-58.6%)
0.104/0.881
(-22.1%)
0.124/0.033
(-33.1%)
Best Baseline Variants
Qwen 3-25BB Thinking
0.105/0.094
(-10.5%)
0.082/0.073
(-11.0%)
0.082/0.77
(.611%
GTP (Best MDH)
0.024/0.016
(-33.3%%
0,1700.013
(-92.4%)
0.019/0.015
(-21.1%)
NalksPers (wia Geoinetric Step)
0.30600280
(-8.5%)
0.334/0.290
(.13.2%)
0.321/0.280
(.128%)
Walk2Pers Variants
Na k2Pe r5-
0.452/0.410
(-9.3%)
0.383/0.300
(-16.4%)
0.449/0.380
(-15.4%)
Table 17: RQ-3(b): Sequential Recommendation on MIND--Lrge. (sorted by MRR) Baselines show
paper-reported means. Wa k2Pers results are reported with mean + variation from resampling.
Methods (Venue, Year)
AUC
MRR
nDCG@s
nDCG@ 10
DKN (WWW" 18)
64.07
30.42
32.92
38.66
(Baseline. 2016)
65.42
31.24
33.76
39.47
EBNR (KDD'17)
65.46
31,26
32.18
39.04
NPA (KDD'19)
65.92
32.07
34.72
40.37
(ICAA119)
32.75
35.66
LSTUR (ACL'19)
67.08
32,86
35,95
40.94
Lincur Transformers (ICML'20)
67.76
32.94
35.91
41.97
ProFairRec (SIGIR
33,08
35.32
NRCLS (Appl. Sci.
68.35
33.12
36.70
43.03
Linformer (arXiv 20)
68.02
33.19
36.22
42.10
Roolingformer (ICML'21)
68.54
33,20
36.69
42.60
NRMS (EMNLP-DCNLPP
67.66
33.25
36.28
41.98
BigBird (NeurIPS'20)
68,
33.28
36.42
42.18
Transformer (NeurIPS'1))
GERL (W)
68.10
33.41
36.34
42.03
GNewsRee (IP&M 20)
68.15
33.45
36.43
42.10
FIM (ACL'20)
67.87
33.46
36.53
42.21
HieRe: (ACI-IICNI_P'22)
68.33
33,86
36.83
42.65
DCAN (arXiv'22)
68,90
33,90
36.90
42,80
ANRS (arXiv'22)
69.20
34.10
37.10
43.00
TCCM (CIKM'23)
69.75
34.42
37.53
43.25
Fastformer (arXiv'21))
69.11
34,55
37.62
43,38
FUM (SIGIR
69,90
34,60
37.70
43,40
CAUM (SGGRR(22))
70.04
34.71
37.89
43.57
DIGAT (Findings ACL/222
70.08
35,20
38.46
44.15
PLM-NR (SIGIR'21)
70.64
35.39
38.71
44.38
Fastformer+PLMM-NR (Hybrid)
71.04
35.91
39.16
45.03
MINER (Findings ACL'22)
71.51
36.06
39.56
45.21
CAST-Rec (TOIS'25)
72.10
36.90
40.20
46.30
Fastformer+PLMM. PineR--eeeeMM (Hybrid' 22)
72.68
27.45
41.51
46.84
Walk2Pers-F Encoder
53.32-11.1
38.64:113
43.32 1.2
50.38
36

---

Published as a conference paper at ICLR 2026
Table 18: Comparison of Specialized and Vanilla Models with Walk2Pers under standard accu-
racy metrics ROUGE-L/SU4 Category
Model
Rouge-SU4
Rouge-T.
VENS-NAMI.-T
13.12
21.62
CANS-EBNR-TI
12.16
20.73
PENS-EBNR-T2
12.41
20.82
Specialized (Personalized)
PENS-NRMS-T
13.15
20.75
PENS-NRMS-TE
13.64
21.03
GTP-TrRMIo
21.91
28.31
SP-Individual
19.54
25.18
LLaMA-13B
18.31
29.34
LLMs w/ 2-shot history)
Mistral-7B
16.42
22.85
DeepSeck-144 14B
19.57
29.72
Zephyr-7B
18.45
26.45
Walk2Pors
Walk2Pers-Fu
43.09
47.16
2-shot w/ history.
User History
List of Articles clicket/Skipped/Suummaaadd by
user:
<Doc1: click>, <Doc2: click>, <Doc3: skip?,
<Doc4: Summarized as Sum1>....
2 shot examples
[opc_coneet]]
[Personalized Headline]: Rewritten_ Titles by
User
[Doc Content]
[Personalized Headline]: Rewritten_ Titles by User
Task
Given Query Doc @doc_content
Generate a Headline by considering the user's history as the indicator to their interests, where click denotes positive interest, skip denotes negative interest and summarized indicates focus on that topic. Return the headline in this format: Headline: (output)
Figure 6: 2-shot Compt-Template for LLM baselines.
37

---

Published as a conference paper at ICLR 2026
Prompt-Chaining w/ history.
User History
List of Articles diicke//kkiped/Summrriieed by user one by one:
<Doc1: click>
Task
Given doc and action performed <doc1 @iooc_cconentt click>
Generate list of interested keyphrases, topics, and preferences for the user.
Output
Interest: <topic1, topic2, topic3> Keyphrases: <phrase1, phrase2, phrase3>
User History
List of Articles ciicked/Skippee//Sumnazzee by user one by one:
<Doc2: skip>
Task
Given doc and action performed <doc1 <doc1_contes _ontent,, click>, and the user preference output
Update list of interested keyphrases, topics, and preferences for the user.
Figure 7: Coompt-Chinning Template for LLM baselines.
38


Reviews:
[{"review_id": "nBk7FefRRC", "reviewer": "Reviewer_E9Qi", "summary": "This paper argues that previous personalized summarization systems implicitly assume a Markovian Drift Hypothesis (MDH): user preference at step t+1 depends mostly on the latest interaction. However, authors of this work believe that this assumption is too weak for long, action-rich histories. It proposes the Structured Walk Hypothesis (SWH) and instantiates it as Walk2Pers, where user preference evolves as an action-biased geometric walk with memory: each interaction applies (i) an action-conditioned step with magnitude (how strong the shift is) and orientation (continuity vs. novelty), (ii) dual memory lanes to reinforce clicked interests and suppress skipped content, and (iii) a summary-specific drift for summarization actions. On three personalization datasets (PENS, OpenAI-Reddit, PersonalSum), Walk2Pers beats strong MDH-style encoders (NAML/NRMS/EBNR, SMD, AGD), specialized personalized summarizers, and mid-size LLMs prompted with user histories, with especially large gains on PerSEval metrics, supporting SWH over MDH in this task.", "questions": "-- Can the authors report training/inference cost per summary (tokens, FLOPs, wall-clock) vs. the best 2-shot LLM, to show the benefit is not purely from extra computation? \n\n-- How does Walk2Pers behave if the action distribution is highly imbalanced (many clicks, very few skips, rare summarize)? Does the dual-memory lane degenerate? \n\n-- Could the geometric step (magnitude/orientation) be learned conditionally on document facets (topic, source) to better handle cross-domain shifts in OpenAI-Reddit?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "**Well-factorized update rule**: decomposing each step into magnitude, orientation, dual memories, and summary drift makes the dynamics interpretable and extensible. \n\n**Strong empirical gains**: Walk2Pers closes or reverses gaps against both domain models (PENS, GTP, SP) and prompted LLMs, which is nontrivial for personalization. \n\n**Task-aligned evaluation**: using PerSEval (human-correlated) plus next-behavior prediction gives both user-centric and model-centric evidence that Walk2Pers with Structured Walk Hypothesis (SWH) can indeed improve over MDH.\n\n**Meaningful Ablations**: removing geometric steps or dual memories degrades performance, indicating the gains aren’t from just a bigger encoder.", "weaknesses": "-- Complexity vs. practicality: the full pipeline (UIG → b-layer → action-gated cells → dual memories → geometric steps → T5-UCA) is quite complex; deployment and latency costs aren’t quantified. \n\n-- Heavy dependence on good interaction logs: the method assumes rich click/skip/summarize traces; it’s unclear how it performs with sparse or noisy histories. \n\n-- Comparisons favor the hypothesis: many baselines are cast as MDH-style and naturally underuse long histories; stronger long-horizon sequence/retrieval baselines would make the claim tighter. \n\n-- Limited analysis of failure modes: the paper shows when SWH helps, but not when geometric novelty or negative memory might over-suppress relevant but infrequent topics.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "u6PKCg2MnB", "reviewer": "Reviewer_RW71", "summary": "This paper presents Walk2Pers, a lightweight encoder-decoder framework for personalized document summarization, which models user preference evolution as an action-conditioned geometric walk with memory. Unlike existing methods that assume short-term or memoryless (Markovian) user behavior, Walk2Pers explicitly represents preference dynamics through two components: magnitude, controlling the strength of preference shifts, and orientation, capturing the balance between continuity and novelty. The model incorporates dual memory lanes to reinforce consistent interests and suppress disinterest, and includes a drift term to handle user-specific summary requests. The authors provide a theoretical justification showing that the proposed structured walk approximates first-order action-conditioned kernels. Empirical results on three benchmark datasets (PENS, OpenAI-Reddit, and PersonalSum) demonstrate that Walk2Pers achieves consistent improvements over both specialized personalized summarizers and strong large language model baselines, as measured by the PerSEval metric. The framework also shows robustness across domains and stability over long interaction histories, suggesting that modeling personalization as a geometric walk with memory provides a more interpretable and efficient approach to personalized summarization.", "questions": "N/A", "limitations": "", "rating": 8, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1.\tThe problem of user preferences evolving over time is very interesting and practical.\n\n2.\tThe paper is overall well-written and easy to follow.", "weaknesses": "1.\tI don’t see obvious weakness for this paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "fb7x3hqJlW", "reviewer": "Reviewer_5Y5U", "summary": "This paper proposes the Walk2Pers model, aiming to break the limitation of the Markovian Drifts Hypothesis (MDH) regarding the short-term dependency of user interests.\nIts core innovation lies in viewing personalized summarization as an action-biased geometric walk with memory, which precisely captures long-term, complex preference evolution through geometric steps and dual memory lanes.\nThe main contribution is providing theoretical support and mechanism innovation, and significantly outperforming traditional summarization models and powerful LLM baselines in experiments.", "questions": "As mentioned in the weakness.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "1.The paper provides detailed theoretical proofs and discussion, which are exhaustive and rich.\n\n2. The proposed SWH (Structured Walk Hypothesis) in the paper is intriguing, as it challenges the widely prevalent Markovian Drifts Hypothesis in traditional recommendation systems and summarization models.", "weaknesses": "1. Section 3.3.1 mentions that T5 is used for embedding, but the paper does not explain why T5 was chosen. It would be better to include an experiment using BERT or other more advanced embedding models.\n2.  When comparing with LLM baselines, the evaluation was performed using 2-shot+history prompting. In fact, due to the limited contextual understanding, it is common for these smaller-parameter models to perform poorly. If this evaluation method is adopted, the LLM baselines for comparison should include large-parameter closed-source models such as GPT-4 and Gemini 2.5 Flash to be more appropriate. If smaller-parameter models are to be compared, their fine-tuned versions should be compared instead.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "eXIRhlVl0b", "reviewer": "Reviewer_HwNK", "summary": "This paper addresses personalized document summarization, where user preferences evolve dynamically over interaction histories (e.g., clicks, skips, summaries). It challenges the prevailing Markovian Drift Hypothesis (MDH)—assuming preferences follow memoryless or short-memory random walks (e.g., seeded at recent interactions or compressed states)—and proposes the Structured Walk Hypothesis (SWH) as an alternative.\n\nKey Contributions\n\nTheoretical Framework (SWH): Models preference evolution as an action-conditioned geometric walk on a User-Interaction Graph (UIG). Updates decompose into:\n- Magnitude ($ \\text{mag}(a(t)) $): Controls shift strength.\n- Orientation ($ \\theta(a(t)) $): Balances continuity (momentum) vs. novelty (orthogonal direction).\n- Dual memory lanes ($ h^+, h^- $): Reinforce interests (e.g., clicks) and suppress disinterests (e.g., skips).\n- Drift term ($ \\Delta $): Handles summary requests.\n- Proves SWH approximates first-order action-conditioned kernels, generalizing MDH.\n\nWalk2Pers Model: Lightweight encoder-decoder instantiation of SWH.\n- Encoder: Fuses actions into b-cells, applies dual memories and geometric steps; trained on next-behavior prediction and alignment losses.\n- Decoder (T5-based): Contextualizes query documents with user-aware gating (UCA variant) for personalized generation.\n- Efficient: $ O(pd) $ complexity for long horizons ($ p \\ll T $).\n\nEmpirical Validation:\n- Datasets: PENS (news, clicks/skips), PersonalSum (EN, summary requests), OpenAI-Reddit (multi-domain, long histories).\n- Next-Behavior Prediction (RQ1/2): Outperforms MDH baselines (e.g., NAML, NRMS, EBNR, SMD, AGD) on PENS: AUC 0.532, MRR 0.23, nDCG@10 0.249 (vs. AGD's 0.446/0.113/0.073).\n- Personalized Summarization (RQ3): On PENS, beats specialized MDH models (PENS variants, GTP, SP) by avg. 0.41 on PerSEval (PSE-JSD/SU4/METEOR); LLMs (DeepSeek-14B, Mistral-7B, etc., with 2-shot prompting) by 0.22 (e.g., 0.452/0.383/0.449 vs. DeepSeek's 0.248/0.094/0.097).\n- Cross-Domain & Robustness: On OpenAI-Reddit, +0.09/0.20/0.24 over best LLM; on PersonalSum, 0.31/0.28/0.3. Achieves human-judgment rating of 7/7 (RMSD 0.396). Stable on long histories; oracles (e.g., BigBird) lag by 0.20+.\n- Ablations confirm SWH components (memory, geometry) yield complementary gains.\n\nOverall, the work demonstrates SWH's superiority for capturing evolving preferences, offering interpretability and efficiency over MDH/LLM baselines, with potential for broader personalization systems.", "questions": "Enhancing LLM Baselines with Advanced Prompting or Fine-Tuning: In RQ3 evaluations (Tables 3 and 4), LLMs like DeepSeek-R1-14B are prompted with 2-shot + history, yielding lower PerSEval scores (e.g., 0.248 PSE-JSD on PENS vs. Walk2Pers' 0.452). However, this may not fully test LLM capabilities, as techniques like chain-of-thought prompting or fine-tuning on personalization datasets could improve performance, as noted in TKGE-LLM hybrids (e.g., Ji et al., 2023 survey on temporal KG completion). Could you re-run LLM baselines with CoT or LoRA fine-tuning on a subset of PENS/OpenAI-Reddit, and report updated metrics? This could clarify if the 0.22 average gap holds\n\nStress-Testing on Ultra-Long Trajectories and Noisy Scenarios: The datasets feature moderate history lengths (e.g., avg. 134 d-nodes in PENS, 39 in OpenAI-Reddit), but claims of long-horizon stability (e.g., dual memory suppression) aren't tested on ultra-long sequences (500+ interactions), where catastrophic forgetting might occur. Additionally, real-world noise like interleaved multi-user sessions or rapidly evolving topics (e.g., from Adressa/MIND extensions) is absent. Could you synthesize ultra-long trajectories from MIND and evaluate AUC/MRR (as in Table 2) or PerSEval under noise injection (e.g., 20% random skips)?\n\nDetailing Parameter Learning and Hyperparameter Sensitivity: Components like mag(a(t)), θ(a(t)), and ω(t) are mentioned but not fully explained in terms of optimization (e.g., are they learned via MLPs or fixed?). Appendix H.2 (inferred from text) covers training, but no sensitivity analysis is shown. Could you clarify learning mechanisms and add ablation on key hypers (e.g., α=0.6 in Lenc) via plots? This could clear up methodological ambiguities and strengthen quality if robustness is confirmed.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "Originality\n\nThe paper introduces a novel Structured Walk Hypothesis (SWH) for modeling user preference evolution in personalized summarization, extending beyond the prevalent Markovian Drift Hypothesis (MDH) by incorporating action-biased geometric walks with dual memory lanes for reinforcement and suppression. This formulation creatively combines ideas from graph diffusion (e.g., random walks with restart), trajectory embeddings (e.g., JODIE), and relational modeling (e.g., RotatE), while applying them to a new domain: dynamic, long-horizon user interactions in summarization. The decomposition of updates into magnitude (shift strength), orientation (continuity vs. novelty), and drift terms removes limitations of memoryless models, offering a fresh perspective on preference dynamics. The Walk2Pers model serves as an innovative instantiation, integrating these elements into a lightweight encoder-decoder framework.\n\nQuality\n\nThe work demonstrates high technical rigor, with a solid theoretical foundation showing that SWH approximates first-order action-conditioned kernels while generalizing MDH. Empirically, it is validated through comprehensive experiments on three diverse datasets (PENS, PersonalSum-EN, OpenAI-Reddit), addressing RQs via next-behavior prediction (Table 2: AUC 0.532, MRR 0.23 vs. MDH baselines like AGD's 0.446/0.113) and summarization tasks (Table 3: PSE-JSD/SU4/METEOR 0.452/0.383/0.449 on PENS, outperforming specialized models by 0.41 avg. and LLMs like DeepSeek-14B by 0.22). Cross-domain robustness (Table 4: +0.19 over best LLM on OpenAI-Reddit) and human-judgment alignment (Table 5: 7/7 rating, RMSD 0.396) further strengthen the evaluation. Ablations confirm complementary gains from SWH components (e.g., dual memory + geometry), and the model's efficiency (O(pd) for long horizons) is well-justified. Baselines are fairly chosen (e.g., NRMS, EBNR, 2-shot LLMs), with sound training details (e.g., joint losses, T5-base).\n\nClarity\n\nThe paper is exceptionally well-structured and readable, with clear progression from problem motivation (information overload in multi-aspect documents) to hypothesis testing (MDH vs. SWH), methodology (UIG graphs, equations like (3) for SWH), and evaluation (RQs explicitly addressed). Visual aids (e.g., Figure 1 illustrating Walk2Pers architecture) and tables (e.g., Table 1 contrasting SWH/MDH) enhance comprehension, while appendices provide derivations, notations (Table 11), and hyperparameters (Table 12). Technical terms are defined upfront (e.g., b-nodes, dual lanes), and the writing avoids jargon overload, making complex ideas like geometric decomposition accessible.\n\nSignificance\n\nBy outperforming SOTA summarizers and LLMs on personalization metrics (e.g., PerSEval with strong human correlation), the paper advances user-centric NLP, particularly for evolving preferences in news/recommendation systems. Its lightweight design offers practical efficiency over prompt-heavy LLMs, with implications for real-world applications like tailored updates in high-volume domains (e.g., Reddit, news feeds). The SWH framework could extend to broader personalization tasks (e.g., recommendation, dialogue), promoting interpretability via geometric/memory structures.", "weaknesses": "Originality\n\nWhile the Structured Walk Hypothesis (SWH) innovatively applies geometric decompositions (magnitude and orientation via cos/sin terms) to user preference evolution in summarization, it draws heavily from existing temporal knowledge graph embedding (TKGE) techniques without sufficient differentiation. For instance, the action-conditioned geometric steps in Eq. (3) closely resemble rotational updates in models like RotatE (Sun et al., 2019, cited in the paper), which uses complex rotations to model relational patterns, and more recent TKGE works such as Time-Enhanced Compound Geometric Operations (TECGO; Zhao et al., 2025), which employs compound geometric operations (e.g., rotations and translations) on timestamps and entities for temporal fact prediction. Similarly, the dual memory lanes for reinforcement/suppression echo memory-augmented updates in temporal graph models like HGE (Sadeghian et al., 2023), which uses product spaces (Complex, Split-complex, Dual) to handle temporal heterogeneity. The application to personalized summarization is a novel domain shift, but the core mechanism—action-biased walks on interaction graphs—overlaps with heterogeneous random walk models like HeteEdgeWalk (Liu et al., 2023), which biases edges without meta-paths for recommendation.\n\nQuality\n\nThe empirical validation is comprehensive in scope but lacks depth in handling real-world variability and scalability testing, potentially undermining claims of robustness for long-horizon preferences. For RQ3, while Walk2Pers outperforms LLMs on PerSEval metrics (e.g., 0.452 PSE-JSD on PENS vs. DeepSeek's 0.248), the LLM baselines use only 2-shot prompting with history, ignoring advanced techniques like chain-of-thought or fine-tuning on personalization datasets, which could narrow the gap (as seen in recent TKGE surveys emphasizing hybrid LLM-graph models; Rossi et al., 2023). Cross-domain results on OpenAI-Reddit show gains (+0.19 over best LLM), but the dataset's avg. 39 d-nodes is modest; missing are stress tests on ultra-long trajectories (e.g., 500+ interactions, synthesizable from MIND extensions) to verify dual memory's suppression without catastrophic forgetting. Human-judgment alignment (7/7 rating, RMSD 0.396) is promising but based on limited samples (inferred from Table 5); a full user study with 50+ participants rating summaries for relevance/conciseness would better validate PerSEval's correlation. Additionally, no runtime or parameter efficiency comparisons are provided despite \"lightweight\" claims (O(pd) complexity); suggest adding FLOPs/ms per inference against NRMS/EBNR and LLMs on varying history lengths to quantify efficiency gains.\n\nSignificance\n\nThe work advances personalized summarization but overstates broader implications for \"personalization systems\" without demonstrating extensibility beyond news/Reddit domains, limiting its impact on diverse applications like e-commerce or social media feeds. For instance, while stability on long histories is claimed, evaluations omit noisy real-time scenarios (e.g., interleaved multi-user sessions or evolving topics like in Adressa extensions), where action biases might degrade. The focus on PerSEval (strong human correlation per Dasgupta et al., 2024) is apt, but ignoring standard summarization metrics like ROUGE-1/2/L (even as secondary) misses opportunities to show balanced personalization vs. generic quality trade-offs. Cross-domain robustness is shown (0.19↑ on OpenAI-Reddit), yet PersonalSum-EN relies on M2M-100 translations, potentially introducing artifacts—suggest validating with native Norwegian evaluators or alternative translators like NLLB. To enhance significance, extend to a fourth dataset (e.g., MIND for pure recommendation) and include a pilot on a downstream task like preference-based ranking, quantifying how SWH embeddings transfer to non-summarization personalization.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)