Diversity Augmentation of Dynamic User Preference Data for Boosting Personalized Text Summarizers
Authors: Parthiv Chatterjee
Shivam R Sonawane
Amey Hengle
Aditya Tanna
Sourish Dasgupta
Tanmoy Chakraborty
Venue: Accepted by TMLR ()

Abstract: Document summarization facilitates efficient identification and assimilation of user-relevant content, a process inherently influenced by individual subjectivity. Discerning $\textit{subjective}$ salient information within a document, particularly when it has multiple facets, poses significant challenges. This complexity underscores the necessity for $\textit{personalized summarization}$. However, training models for personalized summarization has so far been challenging, particularly because diverse training data containing both user preference history (i.e., $\textit{click-skip}$ trajectory) and expected (gold-reference) summaries are scarce. The MS/CAS PENS dataset is a rare resource in this direction. However, the training data only contains preference history $\textit{without any target summaries}$, thereby blocking end-to-end supervised learning. Also, the diversity in terms of topic transitions along the trajectory is relatively low, thereby leaving scope for better generalization. To address this, we propose PerAugy, a novel $\textit{cross-trajectory shuffling}$ and $\textit{summary-content perturbation}$-based data augmentation technique that significantly boosts the accuracy of four state-of-the-art (SOTA) baseline user-encoders commonly used in personalized summarization frameworks (\text{best result}: $\text{0.132}$$\uparrow$ w.r.t AUC). We select two such SOTA summarizer frameworks as baselines and observe that when augmented with their corresponding improved user-encoders, they consistently show an increase in personalization ($\text{avg. boost}$: ${61.2\%}\uparrow$ w.r.t. PSE-SU4 metric). As a post-hoc analysis of the role of induced diversity in the augmented dataset by PerAugy, we introduce three dataset diversity metrics -- $\mathrm{TP}$, $\mathrm{RTC}$, and DegreeD to quantify the induced diversity. We find that $\mathrm{TP}$ and DegreeD have a strong correlation with the user-encoder performance when trained on the PerAugy-generated dataset across all accuracy metrics, indicating that the increase in dataset diversity plays a major role in performance gain.

Published in Transactions on Machine Learning Research (10/2025)
Diversity Augmentation of Dynamic User Preference Data for Boosting Personalized Text Summarizers
Parthiv Chatterjee
202421011 @dau.ac.in
Knowledge Discovery & Management Lab Dhirabhai Ambani University
Shivam Sonawane
202321014 @dau. ac.
Knowledge Discovery & Management Lab
Dhirabhai Ambani University
Amey Hengle
ameylenglee22@gmaii.c
Indian Institute of Technology Delhi
Aditya Tanna
202103023@dem..om
Knowledge Discovery Management Lab
Dhirubhai Ambani University
Sourish Dasgupta
sourish. dasgupta Ceaauptta@dd...o
Knowledge Discovery & Management Lab
Dhirabhai Ambani Universiby
Tanmoy Chakraborty
tanchak@rätd..o
Indian Institute of Technology Delhi
Reviewed on OpenReview: http:://ppaaeeiaaa.ea/e///mmmm/m
Abstract
Document summarization facilitates efficient identification and assimilation of user-relevant content, a process inherently influenced by individual subjectivity. Discerning subjective salient information within a document, particularly when it has multiple facets, poses signif- icant challenges. This complexity underscores the necessity for personalized summarization. However, training models for personalized summarization has so far been challenging, par- ticularly because diverse training data containing both user preference history (i.e., click- skip trajectory) and expected (gold-reference) summaries scarce. The MS/CAS PENS dataset is rare resource in this direction. However, the training data only contains pref- erence history without any target summaries, thereby blocking end-to-end supervised learn- ing. Also, the diversity in terms of topic transitions along the trajectory is relatively low, thereby leaving scope for better generalization To address this, we propose PerAugy, novel crajsttrayyctory shuffling and nummary-content perturbation- based data augmentation tech- nique that significantly boosts the accuracy of four satte-of-tho-ar (SOTA) bascline user- encoders commonly used in personalized summarization frameworks (best result: 0.1321 w.r.t AUC). We select two such SOTA summarizer frameworks baselines and observe that when augmented with their corresponding improved user-encoders, they consistently show an increase in personalization (avg. boost: 61.2% 1 w.r.t. PSE-SU4 metric). As post-hoc analysis of the role induced diversity in the augmented dataset by PerAugy, we introduce three dataset diversity metrics TP, RTC, and DegreeD to quantify the induced diversity. We find that TP and DegreeD have a strong correlation with the user-encoder performance when trained on the PerAugy-generatee dataset across all accuracy metrics, indicating that the increase in dataset diversity plays major role in performance gain.

---

Published in Transactions on Machine Learning Research (10/2025)
1 Introduction
The rapid increase in information requires efficient. summarizers for fast comprehension and prioritization (Ter Hoeve et all 2022). However, identifying 'salient' information is subjective, particularly in multi- aspect documents; which makes personalized summarization critical. Training such models requires datasets with diverse user histories and subjective summaries. However, such datasets are scarce due to privacy concerns (Kirk et all, 2024; Lin ef all, 202e1). The MS/CAS PENS dataset (Ao ef 2021), derived from the MIND dataset (WW al 2121), is key benchmark for training and evaluating SOTA personalized summarization models. It comprises user preference histories (i.e., sequences of click and skip interactions with news articles) along with user-specific gold-reference summaries. Although user-encoders trained on
PENS/MIND. have been employed in various frameworks (Song ct all 2023: Yang ct all, 2022: Lian ct all
2025 evaluations have focused on accuracy and, more recently, on the
(Dasgupla ata 2Ize overlooking the evaluation of the effectiveness of preference datasets as training data. To address
this challenge, we propose PerAugy a perturbation- based augmentation of historical user preference data for personalized summarization. A seed preference dataset is first modeled User Interaction Graph (UIG). The nodes of a UIG represent users' clicked and skipped documents (d-nodes) as well as their generated/expeet
summaries (s-nodes). A path, called trajectory, of the UIG starts with a specific user node (u-nodes) and terminates in d/s-nodes. The edges denote click, ship, generate-summary, and click-summary actions. The UIG, hence, is a pool of user trajectories representing dynamic user behavior histories. We apply PerAugy on this UTG
The key design principle behind Augy is guided by recent findings in recommendation systems, which show that higher diversity in training data directly enhances user interaction sequence representation (Hn et all 2019 Fabbri et all 202T). In other words, "trajectory diversity of training data is directly proportional to personalization capabilities of summarizer models". In line with this principle, PerAugy has been designed as a cross-trajectory augmentation technique in which we propose a novel controlled both-ways exchange of user trajectory segments (termed Double Shuffling (DS) between sampled copies of the original trajectories to create a new pool of diverse synthetic user trajectorics. There are two primary controls in DS - (i) the gap-length that determines how much the new trajectory should resemble the original seed, and (ii) the trajectory-length that determines the number of diverse profiles to be synthesized. The DS operation mimics the stochastic diffusion of a users interest into diverse themes, reflecting naturalistic behavior. A perturbation operation is then applied to the exchanged s-nodes to eliminate unnatural thematic jitters at the boundaries of two segments on the new synthetic trajectory, enhancing realistic nature of the synthetic trajectories. The content of every exchanged s-node is replaced by its corresponding d-node content that most closely matches the nearest prior nodes of the new synthetic trajectory. The substitutions influence diminishes over a k-step context window that represents what is to be considered within proximity. Since this perturbation follows k-order Markov Chain, term this Stochastic Markovian Perturbation (SMP). Through the DS and SMP operations, PerAugy generates synthetic profiles that capture a richer spectrum of thematic transitions and behavioral patterns. This controlled diversification ensures that user- encoders are exposed to varied histories and context shifts during training, thereby strengthening their ability to gencralize and capture user preferences effectively.
We evaluate PerAugy across two core dimensions: (i) improvements in accuracy for SOTA user-encoder models trained on augmenned data, and (ii) downstream gains in personalized summarization frameworks. For the first objective, we use the PENS dataset Ao et all (202T) as the seed. We find that PerAugy- augmented PENS data consistently improves models like NAML, EBNR, and NRMS, showing average gains of 24%, 25% and 18% across AUC, MRR, and nDCG@5&110 10, respectively. PerAugy outperforms the other bascline augmentation stratogics including PENS-SH Song et all (2025), S3 Grover et all (2024)), SDAInter liao et all (2122)), and SOTA LLM-as- augmentors (Llama-13B, Mistral-7B, De-1SBek-RR-14BB. Additionally, PerAugy generalizes effectively to cross-domain datasets (e.g., OpenAT-Reddit), yielding consistent. encoder gains of 19%, 25%, and 17% across accuracy metrics". In the downstream task of personalized summariza- tion, GTP and PENS frameworks show average improvement of 61.2% in PSE-SU4, with setups like PENS+ T22 reaching up to 75% in PSF-RG-SU4.
We have released the augmented datasets across Per and all the baselines as part of the AugPersum collection.
2

---

Published in Transactions on Machine Learning Research (10/2025)
To have deeper analysis of the performance of PerAugy and its relationship with the achieved diversity, we use two simple diversity evaluation metries, Unique Topics per Trajectory (TP) and Rate of Topic Change (RTC). While TP and RTC capture topical richness and frequency of topical shifts in a trajectory, they fail to capture the effect of inscrted s-nodes into trajectories, and their aligmment with the corresponding d-nodes. This motivates to design of a more effective embeediingbassd metric DegreeD, DegreeD is designed to capture the proportional shift of successive s-nodes to that of the d-nodes, as well as the faithfulness of s-nodes towards the corresponding d-node. We evaluate the diversity of PerAugy generated datasets in comparison to baseline datasets. We find a strong correlation between enhanced DegreeD and user-encoder performance, with Pearson 0.68, Spearman's 's pt 0.73, and Kendall's т: 0.57. These results collectively demonstrate PerAugy's ' effectiveness in enhancing data diversity, which in turn, enhances user modeling via user-encoders, and boosts downstream personalization?.
2 Background
2.1 Dynamic User Preference (vs. Static User Persona)
It is crucial to distinguish between user persona and pref-prefeeence history in the context of preference datasets. Persona information, such as address, nationality, or broad interests like genres, tends to remain relatively static over time. In contrast, preference histories are highly dynemic, since they constitute interac- tion (or reading) behavior as a temporal sequence that is complex, and spans across multiple subtopics and discourses. user is unlikely to display consistent behavioral repetition; for instance, it is improbable that Alice's weekly reading consistently centers only on European soccer highlights while predictably skipping U.S. politics film updates. This distinction reinforces the need for training datasets that consist of dynamic user preferences rather than static persona features.
2.2 Personalized Summarizers
Most research on personalized summarization assumes a static user persona (i.e., user profile information that is relatively time-invariant). These works leverage the simplicity of guided (or controlled) summarization, Tn this direction, Don al (202T) proposed GSUM, where the goal was to inject a generic guidance in terms of explicit user-provided key-phrases that are restricted to the query-document only and do not account for the dynamic shift in user preference. CTRLSuum and TMWIN were also proposed on similar lines, where either static control signals were given explicitly or extracted from dialogue scssions (He 2022, Kirstcin et all, 2024). Xiao el all (21221F) proposed the Tri-Agenl personalized summarizer that was iteratively trained that knows historical user-edits of previous summaries.
under an RL setup using an However, the user-edit-poe prreeeedc-eeeeecceeeee does not entail subjectivity and is also static. Static user preference is unrealistic in most situations, while a shift in topics of interest is the norm.
In the more realistic context of dynamic user preference, personalized summarization refers to the extent to which a summarization model aligns its outputs with a reader's subjective expectations. The subjectivity is function of the user's characteristic shift in preference as reflected through the reading history a temporal trajectory of the reading and skipping actions of the user on a sequence of documents. Tt is important to note that this trajectory may occasionally be interleaved by the actions of generating and reading summaries instead of the full-length documents. The PENS framework (with external user-encoders such as NRMS, NAML, EBNR (WWi el all, 2019ka Okkura el. all, 20TZ) is an early example that attempts to address this (Ao et all 2IZI). The plugged user-encoders embed the user behavior trajectories from the PENS dataset. However, the encoders do not capture the dynamic temporal behavioral trend and are also tightly-coupled with the three injection techniques (T-1/2/3) of the encoder-decoders based pointer-genes generator summarizer. Song et al] (2023) proposed the GTP framework that follows a similar summary-editing approach as Tri- Agent, except there is no explicit static guidance but rather the editing (latent) control is generated from the user trajectory. However, the internal user encoder, TrRMIo does not encode the dynamically shifting user trajectory without differentiating short vs. long-term influences. Also, so far we have not found any work that explicitly differentiates the various semantics of the user actions click, skip, read-sunmary.
"Codebase: https://g hts:://gtthiin tht.t con/KON LAB/PerAugy TMLR/; Presentation: https://tinyur //tin/t/nnrrr.../ www/Perkugy-KKaAe -CMLa...
3

---

Published in Transactions on Machine Learning Research (10/2025)
UIG Symbols
UIG (N,E)
Directod acyelic graph user-locument-sumnnnn interactions.
User node j time to- Document node at time fp- Summary node for j time
(44)
User interaction trajectory.
Trajectory pool datasct.
PerAugy Symbols
7
Synthetically created trajectory pool by incorporating s-nodes.
Tuy, target
Trajectory selected for angmentation.
Tul,
Segment extracted from source trajectory.
91
Gap length between inserted segments.
Tgs
Final trajectory after donble shuffling. Final trajectory after SMP.
TSMP
k
Context window size for computing influence.
A
Exponential decay constant for context weighting.
РSMP
Probability of perturbing a summary node.
Table 1: Notations and denotations used in the paper.
2.3 Personalized Summarization Datasets
A key challenge in the task of personalized summarization is the lack of suitable training data across varied domains that covers the three key conditions: (i) chronological ordering of evolving user actions, i.e., his- torical trajectories, (ii) subjective summary expectations (i.c. gold reforences/ratings) for same document by multiple users, and (iii) diversity and dynamicity w.r.t topics and topic transition. Standard summariza- tion datasets like CNN/DM, Multinews (Hermann et al, 2015 Fahbri_et et all 2019) do not qualify. Only a few real-world datasets, notably PENS criteria
ef aLl 2021) and PersonalSum (Zhang et all 2022)), meet these
2.4 Personalized Summarization Evaluation
(T2X3 introduced the notion of p personalization measure of the subjective user experience (UX), which is inversely related to both information overload and lack of expected information. They proved, arguably for the first time, that accuracy metrics are unsuitable for measuring UX, i.c., there are real-world cases where we find low UX even with high accuracy. As a solution, they proposed EGISES EGISES was further modified and completed by
Vansh et all
as a metric to quantify the ofagree-o-persooniizaatioo Dasgupta et all (2024) incorporate penalty due to accuracy drop and the PerSEval metric was proposod (detailed exposition in Appendix Al). In this paper, we adopt PerSEval demonstrate that SOTA user- encoders trained PerAugy-generatte augmented dataset enhance the performance of SOTA personalized summarizers.
3 Modeling User Preference Datasets
In the following section, we first introduce User- www.-nnteeatinnnrraaee Graph (UIG) a temporal knowledge graph- inspired data model for capturing dynamic user behavior trajectories. PerAugy operates on UIG to generate diverse synthetic trajectories, discussed in depth in Section
3.1 User- Interacttot Graph (UIG)
A UIG is a Directed Acyclic Graph UIG: (N, where ((noess) {uf (dittt) u (s(ti)} and
E (edges) {a (tp) 2)) The node set contains disjoint types:
PPersonalSum skipped because insufficient samples and it being in Norwegian, makes infeasible performance boost. summarizers that are pre-trained in Norwegian.
4

---

Published in Transactions on Machine Learning Research (10/2025)
User (u-node)
News doe (d-node)
Summary (>-ndde)
Action: skip
Action: click
Action: genSum
summGen
Train Validation
Test
2
Figure 1: UIG construction pipeline for PENS-styled datasets: Step 1: Documents from train/valid data are sequenced as d-nodes; Step 2: Reference personalized headlines for an intersecting d-node from test data are interleaved s-nodes based on time-step; Step 3: If no intersecting d-node is found, the s-node along with corresponding d-node from test data are simply oppended at their respective time-step.
(to), the j-th user at the initial time-step to (source/starl node of G); u-nodes "j
d-nodes d(t); documents the user interacts with at time-step lp; d-node may re-appear at multiple time-steps.
s-nodes (()); user-specific summaries requested or produced at time ty for a d-node viewed at lime-step 1--1.
Edges represent user interactions on the nodes:
ad (t),, user interaction on d-node d(%) at time-step tp such that aa E fclick, skip, summarize), where click denotes positive engagement (interest), skip denotes non-engagement (disinterest) and summarize (also called genSumm) explicitly captures the interest to read a summarized version of the d-node (during inference, genSumm represents user command to generale a personalized summary of d((²));
ax (ta); the follow-up edge summarize denoting summarized version of d'v-1, acting on sty (also termed as summGen), indicating cither gold-reference summary written by user (in case UIG is treated as training data), or desired summary to be generated by the model (during inference).
Trajectory: Given a UIG, the preference history (termed trajectory) of uj is sequence of interactions, denoted 7", starting at and ending at. d-node s-node 4-11, where I is the trajectory length. Hence, UIG is pool of trajectories T.
A UIG can hence be seen as a dynamic temporal knowledge graph (TKG) of user behavior. We now formally define Personalized Sunmarization as follows:
Definition 1. Personalized Sunmarization Given a user trujectory Th, of length 1, a personalized parameterized (0) summarizer model Mp takes query document node (a( and generates corresponding user (uz) specific summary (t441) where Mo is trained on Terain to opproach the upper- upper-bound of a chosen personalization metric (in our case,
PerSEval (PSE)).
3.2 UIG Construction from Preference Data
In the parlance UIG, preference datasets suitable for personalized summarization training and evaluation are of two categories those which can be directly modeled into trajectory pool T (e.g., the PENS

---

Published in Transactions on Machine Learning Research (10/2025)
Caless
NevntD -N
Useri -y
Confdence-
tas Coraidezion
Qurrnry
Filtering
<
to get
confidenex./UU) Me]
8
contieenesju) 1
Highest
Rated
confidenca,f44 Mg)
Figure 2: UIG construction pipeline for OpenAl-styled datasets: Step 1: Extract NewsID, UserID, confidence, and summary; Step 2: Select top-rating < N,, > click pairs from filtered confidences; Step 3: Shuffle clicks, skips, and summaries to form trajectories.
dataset (Ao ct all 2024 ) dataset statistics are in Table B. and (ii) those which lack user trajectories but contain discrete d-nodes, model-generated s-nodes (in contrast to geer-geneaated s-nodes as per UIG definition), and subjective user feedback in the form of rating and the associated confidence score for that rating (e.g. OpenAl-Reddit dataset (Voökke all 20IZ)), statistics Table g. D. In our experiments, we select the OpenAI-Redee dataset for establishing cross-domain generalizability of PerAugy and its broader applicability in more widely available datasets. We describe the UIG construction method for both types follows:
PENS-styled Datasets The construction of UIG is straightforward in the first case and is done in two steps. In the first step, click and skip interactions in the train dalaset are mapped to document nodes (d-nodes) as incoming edges, forming the corresponding u-tier pool T. As an example, for the PENS dataset, the elkNNww interaction corresponds to a click edge and uclkNews to a ship edge, forming Te However, the PENS dataset lacks user-specific s-nodes (i.e., true interest evolution over time), rendering Th an incomplete representation of the user dynamic preference. Despite this, most recent frameworks train on TPNNS using history or document titles as 'pseudo-targets or via unsupervised learning et 2221; Song ct all R123; Yang et all 2023; Lian of all, 202H). We address this in the second step, where we incorporate the s-nodes from the lest. dataset (Trest) at their associated time-steps into T with the addition of genSumm (directed to the d-node whose corresponding s-node is incorporated) and summ Gen (to the incorporated s-node) edges, forming derived (and more diverse) user-profile pool Teyy- (sce Figure M).
OpenAI-styled Datasets For the second category of datasets, we first do a pre-construction classification
of clicked and skipped d-nodes for every human rater Uj. This is done based on a simple heuristic of selecting those d-nodes as clicked which has at least one corresponding model-generated summary (NT: there can be multiple models) that received a confidence score above a chosen threshold. In the case of ApendI-Roddit,, we chose the threshold for clicked d-nodes to be 6 out of (see Figure IIII,, forming Thae We then select the best model-genee generated summary (i.e., the highest rated one by uy) as the surrogate expected s- node for uj (Figure 1-11). We then randomly sequence all such (d - s)-n pairs along with the skipped d-nodes to form T"; (thereby Teyu-OAl, Figure IIIII). This method makes UIG-modeling compatible with most summarization datasets that are not PENS-styled. Additionally, it also addresses cold-start problem as Taase SAn itself is synthetically designed a random sequence. A detailed pseudo-code of UIG construction is given Algorithm M.
4 PerAugy: Augmentation of Base UIG
In this section, we introduce PerAugy, a novel data augmentation method designed to improve personalization in summarizers by enhancing the accuracy of user-encoders. The design is motivated by the objective to
For detailed exposition of datasets, A Appendix
6

---

Published in Transactions on Machine Learning Research (10/2025)
User (u-mode)
Sumuaryy (=-odde) Summary Action: skip
Action: click
Action:genSum
somm
(a)
DS
DS
Trajectory Longth
Ofivet
Segmentt Length
Gap
(b)
Uy
SMP
-steps comtest window
C
CL-1
(c)
SBERT
Figure 3: PerAugy: our proposed framework - (a) Pipeline overview depicting the two-step augmentation, (b) Double Shuffling (DS) to ensure cross-trajectory augmentation and induce diffusion, (c) Stochastic Markovian Perturbation (SMP) to smoothen the s-nodes and modulate random diffusion incorporated DS stage.
create diverse realistic user trajectories for training, under the hypothesis that "trajectory diversity of training data is directly proportional to personalization capobilities.
4.1 PerAugy Pipeline: Overview
The PerAugy (Porturbation-based Augmentation) pipeline has four steps. In the first step, we randomly sample (without replacement) m seed trajectories (T from given UIG (i.e., trajectory pool Tham) In the next step, we perform the 'Double Shuffling (DS)* operation that selects each of the sampled trajectories as the target and substitutes trajectory-segments at different time-steps of the target with that of segments from the other m trajectories. This leads to modified "shuffled' sample T We describe the DS operation details in section 1 We then select the nodees of each trajectory т E TBS via a Bernoulli Trial and perturb the summary content on the basis of the corresponding d-node's similarity with preceding d-nodes. The details of the perturbation method, called "Stochastic Markovian Perturbation (SMP), is given in section k The resulting perturbed trajectories form TSMP which represents new sel of synthelic user preference history data. Note that Ths is added back into the trajectory pool (T -Tample before the next iteration of sampling is done. PerAugy terminates after | //m sampling iterations. We depict the pipeline in Figure (a).
4.2 Double Shuffling
In this section, we detail the Double Shuffling (DS) operation. The key design heuristics behind DS are that behavioral subsequences within real-life trajectories can stitched together to form more di- verse, rcalistic synthetic trajectories. That is, sequence of interactions of Alice can be stitched with that of Bob and Joe result in a realistic synthetic profile that behaves like Alice during the first ses- sion, then Bob in the second, and then finally Joe. To simulate this, following the sampling method
7

---

Published in Transactions on Machine Learning Research (10/2025)
described in the previous section, given target trajectory Ttarget E T (i.e,, an existing real user), we first randomly select an 'Offseel O. This offset determines the early- stage behavior sequence (behavior sequence is termed "trajectory segment' Taeg) that should re- main the same as Uz. O helps to make sure that a new trajectory does not start with an unrcal- istic initial segment. The random select ion helps to generate early-stage trajectory segments of vary- ing lengths the augmented dataset. For example, if the original trajectory of Alice Thase looks like: Alice's first three interactions involve reading "Meditation tips' (CLK:MT) and Yoga retreat guides" (CLK:YR), and skipping Politicul tension rises in Russia' (SKP:PR); an offset O = 3 ensures these three steps remain unchanged in her synthetic trajectory, preserving a natural beginning. The random selection of O also helps to generale early-stage trajectory segments of varying lengths in the augmented dataset.
sample corresponding to user uj
In the next step, we select m trajectory segments (Taeg ( from each of the remaining m ' 'source' trajectories Tsource -imbmm- Trample corresponding to m users and substitute corresponding target segments having same length as that of these source segments. After Alice's preserved O = 3 steps, segment of length Leeg = 2 from Bobs trajectory where he was browsing 'Concert schedules' (CLK-CS) and 'Band intervieus" (CIK:BI) gets substituted in Alicc's synthetic trajectory Alice
TDS
The m 1- 1 time-steps on Target where the substitutions occur are controlled by "Gap" (G), which deter- mines the number of time-steps that should be kept intact (i.e., same intermediate behavior sequences as the original u,). If G = 2, then after Bobs exchanged segment ends, two of Alice's untouched inter- actions, say, "Cooking recipes' (SKP:CR) and "Mindfulness blogs" (CLK:MB) are preserved in TDS before the next substitution. Following this, another source segment may be introduced. For exam- ple, segment of length Lang, 2 from Joe could be substituted, representing his interactions with Sports highlights" (CLK:SH) and "Malaria Outbreaks" (SKP:MO). The final DDS -LIIBB-CKKHHH--KPPP
"y
Alice will therefore look like:
Gap length is a hyper-paramett that we ablate on in our experiments (see section IT). Longer gaps would "y
signify that the new synthetic trajectory 7Ds is more similar to the corresponding original samples while a, longer source segments and higher m value would lead to longer TDS. The random selection of O also helps to generate early-stage trajectory segments of varying lengths in the augmented dataset.
Although DS introduces diversity by aggregating trajectory segments from different users and threads them up at different time-steps thereby altering their original positions, it fails to ensure that the s-nodes that come inlact along with the source segments have realistic coherence with the preceding nodes. This is because the source s-node has been inftuenced by the preceding source nodes that form its "history", and hence, may not be compatible with the new history in the target trajectory. To address this, we introduce a subsequent operation on each double-shuftted trajectory in Tps called "Stochastic Markovian Perturbation' that we describe in the next section.
4.3 Stochastic Markovian Perturbation (SMP)
SMP smoothens the newly substituted incompatible s-node (44) at time-step 4 by operating over a backward sliding contezt window 7"larse Cj of & time-steps, derived from the corresponding d-node at fę-1. This process refines glt,) by perturbing it; i.e. by replacing it with a sentence that better aligns with the temporal context. of the target user. Specifically, SMP selects the top-p sentences from d(tc-)) that are most influenced by the context nodes Cg=1:* within the window. In our experiments, we use top-1 selection, since the s- node represents summary-level headline, making single representative sentence sufficient. The notion of "influcnce" is quantified through the Root Mcan Square Distance (RMSD) between each candidate sentence St, in d((--1) and each context node Cy in Mtagett Ca. Sentence and context representations are computed using SBERT embeddings (as detailed in Section E). To reflect temporal relevance, this influence is weighted by an exponential docay factor e e)>opos(cu), where pos,, = denotes the most recent context and thus receives maximum weight. As a result, carlier context nodes contribute less to the influence score. This formulation characterizes SMP as k-order Markov process, where the prediction at 4 depends on a weighted combination
8

---

Published in Transactions on Machine Learning Research (10/2025)
of the preceding k time steps. A compact representation of SMP follows:
SMP(s(t) E (hwrott) g(t,) arg min (n (1))))))))) Ie (epeee)))))))))) (xxx1));
(1)
where: E(p.a) = alsst, Ce, d(())))) No. of sentences in d RMSD
Once the minimum-scoring sentence 8(4) is identified via this wnfhuence-wiighted RMSD computation, the original s-node g(4)) is replaced with g(4), RMSD is chosen over other metries because it provides a straight- forward, geometry-based measure of dissimilarity between embeddings, capturing overall distributional mis- match rather than just directional alignment (as in cosine similarity). This makes sensitive to subtle semantic shifts, which is desirable when ensuring that substituted s-nodes remain contextually coherent. The DS operation mimics the stochastic diffusion of user's interest into diverse themes, reflecting natural behavior; hence, smoothing every s-node may inhibit the intended thematic diffusion. Conse- quently, s-node g(4:) is selected for perturbation via a Bernoulli trial with perturbation probability PSMp B, As an illustrative example, suppose at time-step tę Alices double-shuffeed trajectory inserts an s-node from Bob, e.g,, "Booked tickets for a rock concert', but her recent context window (I- = 3) contains SSearching for yoga retreats', "Reading reviews of meditation centers', and "rowsing healthy recipes'. SMP looks back to the previous d-node, extracts candidate sentences, and scores them against this context with exponential decay weighting. A more coherent candidate, such as WWrrshops focus on mindfulness practices" achieves the lowest weighted RMSD and replacos the original s-node, ensuring the trajectory flows naturally with Alice's wellness theme instead of abruptly shifting lo concerts.
The DS and SMP operations generate synthetic user profiles à and trajectories TSMP E SMP We ablate on the context window size k, decay constant X, and PSMp in Section WT. The pscudo-code for DS is provided in Algorithu & and that of SMP is provided Algorithu
5 Evaluation
To evaluate the overall efficacy of PerAugy, we frame our investigation around two central research questions: RQ-1: Does applying PerAugy seed training data lead to higher accuracy in SOTA user-encoders? RQ-2: If so, then do the enhanced user-encoders, in turn, improve SOTA personalized summarization frameworks?
5.1 Experiment Setup
this section, we describe our detailed experimental setup: The training and testing datasets, Training procedure, Baseline augmentations, user-encoders, and summarizers, and the Evaluation metrics utilized assess the effectiveness and utility of PerAugy.
5.1.1 Augmented Synthetic Datasets
Training Data. We create two training datasets using PerAugy: Thss and Th The dataset Th is mixed bag of trajectories sampled from ten different augmented datasets generated with varying < <Gap-length Trajectory-length 1> configurations without SMP operations. Among them, five datasets {Tis use a fixed = 150 and vary E { [10, 15, 20, 25, 40), while the remaining five (Tis use a fixed 9i = 25 and vary 1 E (50, 100, 125, 175, 200). Each of the ten datasets contains 400K trajectories. From each, we sample 10% to construct the final T
Similarly, T544MPP is constructed via the same mixed-bag sampling strategy, but using SMP operations applied to each of the ten augmented datasets with decay constant X = 0.3, perturbation probability PSMP = 0.8, and context length A = 10. Proportional sampling from diverse configurations helps mitigate overfitting and increases the generality preference historics. We generate four variants total: (1) T
(ii) Th OA- (iii) DS+SMP and (iv) DS4SMP: derived respectively from base and base 7
5 AAll symbols used described Table

---

Published in Transactions on Machine Learning Research (10/2025)
Test Data: User-Encoder Evaluation. We construct the test dataset Test to evaluate user encoders under realistic interaction conditions accurately. The PENS validation set lacks skipped d-nodes (thereby rendering it less effective to evaluate user-encoders). The PENS test set lacks explicit negative dnodes in the target bin, we merge Phaso-1 clicks (Freg.) with Phaso-2 (d, pairs (T²") in soquential order. We split : in half: the first kalf is appended to Tseg, to form the user history Thima while the second half serves as the candidate set for next-click prediction. To ensure balance, we augment positive samples with negative samples-documents not clicked by user Uj in either Phase-1 2 randomly drawn from the index range [50 n, where n is the total number of s-nodes in T. This process yields realistic distribution of clicked and non-clicked items, enabling a fair next click prediction evaluation. The final test set contains 103 trajectories with 2 20K candidate pool (both positive and negative samples) of d-nodes with 10K target d-nodes, thereby not altering the settings of the PENS test dataset. (Table B)
Test Data: Personalized Summarizer Evaluation. We evaluate personalized summarizers using the ", original PENS test set Trest For each user Uj we retain Stage-1 click history Tseg, and use the 200 (d, s) pairs from Stage-2 as summarization queries. The model generates personalized summaries 8 conditioned on the query document dquery and Tseg, alone; the intermediate (d, 8) pairs are not. appended, resulting in inferred summaries $1.200 based solely on Tseg..
5.1.2 User-Encoder Training
We train the user-encoder models from scratch on each of the mixed sets Ths and T5 (with batch size: 128 trajectories; epochs 2; Adam Optimizer (a : -e-4: 4; B1 0.9; B2 : 0.999; le -)), in contrast to standard fine-tuning, to analyze: (a) the extent to which synthetic datasets can replace real datasets, especially when such datasets are extremely scarce, and (b) to clearly understand the effect of the hyperparameters of PerAugy under ablation. During training, we split a user's trajectory TDS/DS+SMP into an input segment, also termed train history-segment "2 and a target segment Ttarget. at random time-step within the interval [1"
My
2g lh: - 3]. The nodes of Trrrget form a target candidate bin for the next
node prediction task. We ablate herai length in section fill We further fine-tune TrRMIo (with epoch 1) on the top of T to ascertain the impact of fine-tuning by each baseline augmentations (as discussed in Section
5.1.3 Baseline Augmentation Methods
We evaluate PerAugy against three state-of-the-art (SOTA) algorithmic augmentation methods. We select PENS-SH as a strong baseline, as it is specifically designed for personalized summarization. We also choose S3-Aug as trrec-rrrjyytooy bascline, and SDAInter as unter-trajcctory baseline, along with three LLM-basod augmentation setups. PENS-SH merges multiple user interaction trajectories from Tiaan into synthetic ones by aligning common d-nodes to create diverse pseudo-users. S3-Aug applies intra-trajectory segment-shuffle- stitch operations to perturb temporal structure while preserving local coherence in T:S3 Terai. SDAluter swaps interchangeable sub-sequences between user trajectories based on anchor overlaps and loU (Interaction over Union) confidence, producing cross-user hybrids in TiSDA We convert each of these into UIG-compatible datasots, PSH Tayn-S3, and TTy.-SDA, by injecting s-nodes from test datasct summarics to evaluate diversity. In the LLM-as-augmeetorr setup, we use LLaMA-2-133 13B, distral-v2-Instruue and DeepSeek-77 chat with two prompt strategies. Chain-of-Thhught (CoT) prompts guide LLMs reason step-by syp--by-step through user preferences to generate personalized summaries. Prompt-Chaining (PC) splits the task: first generating user behavior, then using that to generate personalized summaries. All augmented datasets are used to train user encoders, and we evaluate them to assess their diversity and impact on user-encoders. 0 The baseline augmentations are discussed in details in Appendix Glll and prompt details are depicted in Figure B (Chain-oftthoulhts)) and Figure III [Prompt Chaining).
GUIG statistics are detailed in Table
10

---

Published in Transactions on Machine Learning Research (10/2025)
5.1.4 User-Encode Baselines
To study RQ-1, we evaluate four SOTA user-encoder models originally trained on PENS or its source dataset MIND (Wi el all 2020): NAML (Wn el all 201Ma), NRMS (Wi el all, 2015), EBNR (Okura el all, 2014), and TrRMoo (Song et al, 212X since these encoders form four structural variations of history se- quence modeling. NAML employs a multi-view additive attention approach to integrate news features (e.g., titles, categories) and models user interests via attention over browsing history. NRMS applies multi-head self-attention in both news and encoders to capture semantic relations in titles and per- sonalize based on attended browsed content. EBNR is click-order GRU that incorporates dwell-time- based implicit negatives, combining transformers and attention to model user preferences from both positive and negative signals. TrRMIo leverages pre-trained full-sequence transformers with attention pooling, defining user interest through CTR-based filtering that emphasizes low-CTR news as indicators of core user preference. All baseline encoders are described in Appendix O
5.1.5 Personalized Summarization Baselines
Most recent frameworks for personalized summarization are trained on the trajectory dataset The using either of two methodological paradigms. The first involves reinforcement learning setups, where models are optimized using "pseudo-target" such as user history (An all, 2021) or document title (Song al, 212%) approximate personalized summaries. The second involves unsupervised setups, where the training objective is not based explicit summaries but instead aims to reduce the surprise in the gen- erated summary (Yang et all даКК)
to align user representations with the style-preference centroid of similar users in their neighborhood (Lian all 2025). We could not experiment with models of the second paradigm since they are closed
To systematically investigate RQ-2, we adopt two state-of-the--rt per- sonalized summarization frameworks. The first is PENS (AA atll 2022)), a pointer-network-ee based model trained on the PENS dataset using policy gradientbased reinforcement learning. PENS utilizes user em- beddings derived from third-party user-encoeers such as NAML, NRMS, and EBNR, injecting them into the generation process to personalize summaries. The second GTP (Song ct all 2023, which follows two-stage late-fusion approach trained on PENS-SH. In this framework, general headline is first generated using a transformer-based encoderdecoder model, and then personalized in a second stage using TrRM generated user embeddings to control stylistic and semantic refinements. Baseline frameworks are detailed in Appendix MK
5.2
Evaluation Metrics
5.2.1 Encoder Evaluation
To evaluate user-encoders on the task of Next-item Prediction, we use standard metrics: AUC, MRR, and nDCG@5&:10. AUC (Arca Under the ROC Curve) measures the models ability to rank a positive item higher than negative ones, indicating overall ranking quality. MRR (Mean Reciprocal Rank) evaluates the position of the first relevant item in the ranked list, giving higher scores when relevant items appear earlier. nDCG (normalized Discounted Cunulative Gain) at cutoff positions 5 and 10 assesses both the relevance and position of items, rewarding models that rank relevant items higher in the top-K predictions.
5.2.2 Personalization Evaluation
We adopt PSE-SU4 as the PerSEval (only existing personalized summarization evaluation metric known far) variant to measure the boost in judgment correlation (Pearson's 's r: 0.6; Spearman's 's p: 0.6; Kendall's 7: 0.51) and computational efficiency (Dasgupla el. all. 2023)). PerSEval measures how well a summarization model personalizes its outputs to individual user preferences (responsiveness) while also penalizing it for poor or inconsistent accuracy across users. It balances the trado-off between generating diverse, uscr-specific summarics and maintaining relevance to the expected content. The detailed formulation described in Appendix AA
for both frameworks, owing to its high human-
We are yet. receive the codebase from the authors.
11

---

Published in Transactions on Machine Learning Research (10/2025)
Method
RRNR
NRMS
TrRMio (R)
#DOGGS
nDOGGR5
xDCGG10
#DOG990
=DOGG6
xDOG
PENS/Orggaa)
PENNSSE
0.79
0.79
0.72
83
0.71
0.73
0.79
0.69
0.778
0.78
0.97
0.70
(:81
0.81
0.5)
0.75
0.8
SDAliee
0.23
0.75
0.53
0.83
0.31
1.75
0.73
0.79
0.52
0.27
0.83
0.83
0.50
0.57
0.95
0.55
14MM
0.43
0.62
0,08
0.65
0.46
1U68
0.74
0.74
0.41
0.53
0.58
0.58
0.42
0.57
0.73
0.73
0.45
0.64
0.71
0.71
0.48
0.70
0.74
0.74
0.45
0.29
0.05
0.08
0.56
0.77
0.78
0.76
DelpSork-RR
0.43
0.61
0.63
0.65
0.45
0.64
0.72
0.72
0.44
0.34
0.63
0.65
0.47
0.50
0.77
0.77
PerAugy (S(s))s)
0.57
0.78
0.84
0.64
0.51
8.77
0.84
0.64
0.65
9.78
0.63
0.63
0.24
0.9
0.95
0.05
DS (ouss
Table 2: User encoder performance (truined--rrom-screte augmented variants, including PerAugy (DS+SMP). Obsorvation-1: PerAugy outperforms all baselines across models (NAML, EBNR, NRMS) and metrics (AUC, MRR, nDCG05/10), when franneddfoom--sulllee Observation-2: When finetuned on TrRMIo, PerAugy consistently outperforms all baseline augmentation strategies, as compared to their fine-tuned versions. Observation-3: While some methods (e.g., SDAInter) help, others (e.g., S3, LLaMA2) degrade performance, showing the impact of augmentation and UIG quality.
Models trained on PENS and its
5.2.3 Human-Judgment based Evaluation
Direct human evaluation of personalized summarization faces fundamental feasibility issues. A third-party annotator cannot reasonably adopt the evolving preferences of target user after parsing through extensive and often noisy interaction histories, ranging from raw click headlines and skipped articles to long Reddit threads. Personalizatiin hinges on nuanced, fongitudinal signals such as shifting stances, sub-topic interests, and stylistic inclinations, which are often subtle and subjective. Any attempt to reduce this complexity into a simplified abstraction risks crasing the contributions of more exprossive personalization models that can capture temporal shifts in preference history, collapsing the goal of personalized summarization to persona- centric (static) summarization (as described in Section I.).. Thus, to assess the cognitive validity of PerSFval, Dasgupta al (20221) designed a survey-based meta-evaluation simulating how human evaluators perceive personalization. Participants rated the similarity of summary pairs generateene and gold-reference) without knowing their source. From these ratings, they constructed DEGRESS HJ (a human-judged version of DEGRESS) using normalized similarity as divergence and compared it against DEGRESS using corre- lation metrics (Pearsons r, Spearmans p, Kendalls T), and further evaluated whether applying standard accuracy metrics as discounting factors over DEGRESS-HI H.J (mimicking EDP) aligus with PerSEval scores. Strong correlations both stages confirm that human evaluators intuitively align with PerSEval's ratio- based responsiveness and factor-based accurocy penaltyindicating that PerSEval has strong human-judgnee validity and does not require further human evaluation in this setup.
6 PerAugy Performance Results and Insights
In this section, we discuss the results of each of the research questions outlined in Section E.
6.1 RQ-1: PerAugy's Effect on User-Encoder Accuracy
Comparison with SOTA augmentation strategy. We observe a significant improvement accuracy across all the frained-from-scratell baseline @ when trained on our proposed dataset 7DS+SMP, com- pared to their original performance using the standard preference training set Traain The best results obtained using PerAugy show notable relative gains of 0.139 1 in AUC, 0.108 7 in nDCG@5/10 (on the EBNR encoder), and 0.096 in MRR (on the NRMS encoder), clearly demonstrating that the PerAugy augmeeneed dataset. can effectively substitute for scarce preference training data. We also find that PerAugy significantly outper- forms all bascline augmentation methods (S3, PENS-SH, and SDAInter) when the user encoders (NAML, Specifically, we observe average gains of 0.127 1, 0.143 1, and
EBNR, and NRMS) are trained-ffrom-scaate 0.09 1 over S3, 0.1 1, 0.12 1, and 0.07 t over PENS-SH, and 0.103 +, 0.09 t, and 0.067 1 over SDAInter in terms of AUC, MRR, and nDCG@5/11 10 respectively. $T ST upgrades performance w.r.t. Th's across all metrics throughout the user encoders (average boost of 0.03 f in AUC, nDCG@5&110, and 0.02 MRR),
8 All results are statistically significant with p < 0.01 (lest size: 18.1K positive and negative decision nodes).
12

---

Published in Transactions on Machine Learning Research (10/2025)
Personalized Summarrker User-Eucoder
PENS (Original)
DS (Scratch)
DS+SMP (Scratch) DS+SMP (Eucoder FT Eud-to-Eud)
NAML (TI)
0.013
0.008
0.014
0.015 0.017
FINRR (T1)
DOBI8
0.000
0.010
0.10 / NA
PENS
EBNR (T2)
0.005
0.008
0.008
0,010 0.000
NRMS (TT)
DJ011
0.008
0.010
D011 / NA
NRMS (T2)
0.004
0.007
0.007
0.005 NA
GTP
TcRMoo (Lille
0.006
0.017
0.017
0,020 0.022
Table 3: Personalized Summarizer Performance (PSE-SU4). All models are injected with the same user-encoder used during their original training. Observation-1: GTP utilizes improved user embeddings best in both encoder finetuming and end-to-end tuning. Observation-2: In PENS, NAML-TI T1 shows clear boost, while other variants fail to capitalize on the finetuned encoder. Observation-3: SMP consistently improves over DS, indicating its necessity for mazimizing gains.
establishing that Double-Shuffting alone is not sufficient to yield significant performance lift( Table 2). ab- late on the mixed training data Tig to analyze the effect of DS hyper-parameters gap-length (section saml) and train history-segment length Thara: {1/2, 51/8, /, 31/4, 71/8,1 ( - 3} (l: trajectory length). For SMP hyper-parameters (k: {10, 15, 20), (0.3,0.8.1) 1),PPPP {0.5, 0.8, 1}), we ablate on T544MPP Results are Appendix II; Figure F.
Comparison with LLM-generated Train scts. Furthermore, to assess the effectiveness in comparison to LLM-generated training data, we train the same user encoders from scratch using Train T-LAMA-2 T. DeepSeck-R1 In comparison to these LLM-generated baselines, our PerAugy training set DSP yields and Terain average performance gains of 0.157 1, 0.2 t, and 0.203 1 over LLaMA2; 0.13 1, 0.167 1, and 0.16 t over Mistral; and 0.15 1, 0.213 1, and 0.197 1 over DeepSeek-R1 again reported in terms of AUC, MRR, and nDCG@5/ 10 respectively. It is important to understand that LLMs have limited capacity to generate large number of diverse trajectories, and thus we scale down to 800 1500 unique trajoctorics generated. These consistent improvements across all metrics and models further validate the effectiveness the PerAugy as an augmentation strategy (For details, see Table 2)
Effect of Finetuning. We reserve TrRMIo to analyze the fine-tuning performance of Per and find it to outperform the best performing 0.026 1 w.r.t nDCG@5/10) (Results in Table Z)
fine-tuning (0.14 1 w.r.t AUC, 0.034 w.r.t MRR, &
Comssn Study. When trained on ---AA DS/DS+SMP and evaluated on the PENS test set Tust, we observe that PerAugy reliably applied to OpenAl (Reddit) like datasets that do not contain preference histories (best (NRMS): 0.163 1 w.I.t AUC, 0.112 1 w.r.t MRR, 0.09 w.r.t nDCG@5/10). Importantly, our primary claim is of cross-domain generalizability and not transferability. By generalizability, we mean that PerAugy can be trained on datasets (as trained on T DS/DS that are very different in domain and structure (for instance, is non-news, multi-domain dataset), and still sustain its boosting performance their corresponding test sets, while testing the DS/DS tta-traineed encoders on Trest sufficiently validates cross- domain transferability. The results are detailed in Figure
6.2 RQ-2: PerAugy's Effect on Personalization
We examine how effectively the baseline personalized summarization frameworks, GTP and PENS, lever- age their corresponding improved user encoders, focusing on the frameworks' sensitivity to enhanced user history embeddings. To isolate the contribution of user encoder improvements, we first replace the fine- tuned encoders with their train-from-scrate counterparts. This setup helps assess the raw effect of our augmentation strategy, PerAugy. With DS, the highest improvement in the PSE-SU4 score is for the GTP framework (+TrRMlo), achieving a gain of 0.012 t over its baseline (original PSE-SU4: 0.006). On the PENS framework, the PENS(+EBNR+TT2 variant shows a notable performance increase of 0.003 under DS. PENS variants that utilize NAML and NRMS encoders with TI injection appear to benefit more from the SMP augmentation, yielding additional gains of 0.006t and 0.002t respectivelyon top of what DS alone provides. This further establishes that. Double Shuffling alone is not sufficient. to boost downstream per-
13

---

Published in Transactions on Machine Learning Research (10/2025)
formances. When we switch to using the fine-tuned versions of the user encoders, the improvements in PSE-SU4 become more pronounced across models. The best results with these encoders include boost of 0.014 1 for GTP+IrRMIo, 0.005 t for PENS+EBNR-T2, and 0.002 1 for PENS+NAML-TL, Finally, to evaluate the full effect of end-to-end fine-tuning of the summarization frameworks with PerAugy, we utilize NAML(TI), EBNR(T2), and TrRMIo (since these encoders had shown relatively greater improvements their fine-tuned versions). This end-to-end training leads to further performance boosts for NAML(T1) and GTP, both gaining 0.002 1, whereas EBNR(T2) shows a slight performance decline of 0.002 & (Table B). This indicates that the architecture and the injection method play major role. This is evident from the best-performing GTP, which is trained with the additional loss on aligning generated summary embodding with the history embedding by the TrRMlo user-encoder. This results in superior personalized summaries, while the PENS framework falls behind due to a lack of aligmment with the user's preference history.
7 Dataset Diversity Boosts Performance: Quantitative Analysis
We observe that DS and SMP operations lead to significantly higher performance of the SOTA user encoders (and thereby, the personalized summarization frameworks) when trained on PerAugy-genorated datasets (scc Tables B) This empirically confirms our hypothesis that "Irajectory diversity of training data is divectly proportional personalization capabilities'. In this section, we further analyze the extent to which PerAugy achieves diversity via the DS and subsequent SMP operations w.r.t. three different post-hoc diagnostic diversity metrics. We then show that these metrics have high positive correlation with the accuracy metrics, thereby hinting (as a part of preliminary sludy) that such metrics may be reliably used lo estimate how good the training data (both real and synthetic) might be for the personalized summarization task. The analysis outlined is a strongly suggestive verification method to show that diversity is the primary cause of the performance boosl.
7.1 Trajectory Diversity Metrics
In order to analyze the extent of diversity in PerAugy-generated datasets, we first define three diversity metrics Topics per Trajectory (TP), Rate of Topic Change per Trajectory (RTC), and (DegreeD) as follows:
Topics per Trajectory (TP). TP is a simple trajectory-level metric that commonly used to quantify topical variety and diversity in uscrinteraction graphs (UIGs):
U
TP(*") = [topiic ) TP(D) = U TP(7")
(2)
For a user trajectory T"s with length 4, where d(t)) is the i-th document (d-node) consumed at time f, and let topic(-) map d(4)) its discrete topic (given as ground-truth). Higher TP indicates that a user engages with a wider variety of unique topics over time, However, TP alone cannot distinguish between drift and diffusion. A trajectory where topics change gradually over time (drift) and one where topics switch abruptly in scattered manner (diffusion) can yield the same TP score, since TP only counts distinct topics. This limitation motivates the need for a complementary metric that accounts for the frequency of topical shifts.
Rate of Topic Change per Trajectory (RTC). This captures how frequently the topic changes between consecutive steps:
U U Σ RTC(7"5)
RTC(7"5) +
Σ 1 topic (dlti) # (opic(attni)) RICCDD)
(3)
where 1|.] the indicator function and [U] the number of user trajectorics in datasct D. A higher RTC indicates more frequent topic switching within a trajectory. By construction, RTC(r")) if every successive pair of interactions involves different topies.
14

---

Published in Transactions on Machine Learning Research (10/2025)
While TP and RTC provide fast, interpretable signals of diverse topies and frequent shifts of users' interest over timesteps, they do not capture several aspects crucial to personalized summarization dataset: (i) Faithfulness of s-node to source d-node: they ignore how closely each subjective expected summary (s-node) aligns with the central discourse of the corresponding d-node. (ii) Magnitude of thematic shifts: a change of topic label is treated equally regardless of the actual semantic distance between the topics i.e., small and large shifts are indistinguishable. (iii) Consistency of user focus: they do not penalize cases where s-nodes drift away from the core content of the corresponding d-nodes across time, i.e, when the faithfulness decreases. (iv) Sparsity of s-nodes: practical UIGs often lack s-nodes at many time-steps; these metrics offer no principled way to account for the lack of gold-reference s-nodes in training data. Also, RTC is insensitive to the uniqueness and breadth of topical transitions: a trajectory that merely switches with high frequency between a small subset. of Lopics (periodic alternation) can yield a high RTC score, even though the topical coverage remains narrow. These limitations motivate a metric that jointly captures over (a) the relative alignment between documentdocument and summarysumnary shifts, (b) the absolute thematic divergence, and (c) changes in failhfulness over time.
Degrco-of-Diversity (DegreeD). To address the above drawbacks, we propose a novel metric called DegreeD (Degree-of-Diversily) and then briefly discuss the method to compute it given any UIG. As building block, we first define Degree-of-PPe Prefe--reeeeceee Shift (DePs) in given UIG trajectory 7"/ correspond- ing user Uj, where DePS quantifies the shift in a user's interest across T")
Definition 2 (Degree of Preference Shift (DePS)). DePS is the ratio of the thematic divergence between consecutive d-nodes ditt)) and d(t++1) (denoted as 6[X]a) over a time span (or interval) A(tinton)) = t++1 to and that belween the corresponding s-nodes (user's 's subjeclive expected summaries) s4t) and 8) as 8[X],).
(1111) (denoled
ts 1++) where is a distance measure on a The thematic divergence 8[X]. (o E {d) is calculated as a 3 chosen metric space. As per the definition, DePS for the j-th user at any unit interval ((.44))) is:
Antan min 8[X] ) +E mar d, 0 s,,) +
DepS. j
(4)
The Expected DePS, E, [DePS], for j-th user over the trajectory Th having length / is:
E, [DePS]
DePS, Antt))
(5)
E, [DePS] penalizes the disproportionate alignment between the expected summaries (8[X],,) and the doc- ument divergence (8[X]a for r", (Figure Bn (a)). We adopt the minmax normalization in Eq. El ensure that DePS is scale-invariant across embedding spaces while remaining naturally bounded in [O, 1], providing consistent measure of alignment between document and summary divergences. Consider that at 41 Alice clicks on Yoga retreat in Bali" (d(t)) and her expected summary is 'Yoga travel guide" (s(t)),, which is well aligned. At t2, the document shifts to 'Top 10 meditation apps" (d(*)),, but her summary remains almost. unchanged as YYoga travel blogs" (s(fa)). Here, the document divergence 8[X]a non-trivial (topic shifted within wellness), while the summary divergence 8[X), is very less. This disproportion leads to low De f2), signaling that Alice's summaries are not faithfully tracking her docunent-level preference shifts. However, it fails to penalize the case when any expected summary (s-node) in r" is not consistently faithful (in the sense of centrality to the core topic) to the corresponding document (d-node) (Figure En (b)). Tt also fails to penalize the case when the absolute thematic divergence is small (Figure 3ml (c)). Both these cases can happen even with a high EE, DePS]. To address the first issue, we modily Ey [DePS] to incorporate the necessary penalties (i.e, penalized [DePS] or E, (DePPP)) as follows:
IM
Auaun (((())) 1000),000 (2))
DePs,
(d ( (11117
(6)
E, DePS
15

---

Published in Transactions on Machine Learning Research (10/2025)
P
Symbol
Description
T)
Unique topics in trajectory 7",,
RTC("")
Rate of topic change in 7")
6[X]
Thematic Divergence between consecutive documents.
6(x),
Thematic Divergence between consecutive summaries for user j.
unit time-span.
Divergence metric embedding space. Expected preference shift across r*,
[6
Penalized expected preference shift for j.
Regulator that controls the influence of Peualized DePS. Number of trajectories the datuset.
U
(b)
(a) DegreeD penalization: (a) Disproportion- ality between successive s-node and d-node diver- gences, (b) Proportionate but unfaithful align- ment of s-node & d-node, (c) Lack of thematic divergence between consecutive d-s pairs.
(b) Symbols used in the definition of TP, RTC, and DegreeD.
In the above equation, the second factor penalizes negative shift in faithfulness where the s-node starts deviating away from the corresponding d-node as compared to the deviation previous time-steps. Note that the second factor rewards posilive shift where s-node comes closer to its corresponding d-node. As can instance, consider at t1, Alice clicks' Yoga retreat in Bali" (d(f))) and her expected summary is 'Yga travel guide' (sl())). At 12, she clicks "Top 10 meditation opps' (d(tz),, but her expected summary becomes "App technology and efficiency' (s(tz)). The two documents are semantically close (both wellness- related), but the new s-node drifts away, showing inconsistency. E, [DeP)] would still look high because of normalization, but the penalized Ey [DePSP] detects this faithfulness drop and penalizes it. To address the second issue, we injoct an additional factor 8[X],, that penalizes lack of thematic divergence terms of user's actual interest/ focus (hence, 8[X]s; instead of 8[X]a;). o[X],; is regulated by the hyper-parameter (0, 1]. The final DegreeD formulation for dataset D containing [U] unique user trajectories is:
DegreeD(D)
[x),, [e(De
To continue illustration from the last example itself, consider at t3, Alice clicks "Meditation workshop details' (d(ta),, and her expected summary is WWrksshpp information" (s(ta)). Both d(ta) and (tta) are very similar, and the corresponding summaries are almost identical. Here, [DePSP] might appear high since s-nodes are faithful towards their respective d-nodes, along with being proportionate. But 8[X],, penalizes such low divergence in user interest, ensuring that datasets with repetitive, uninformative shifts are not overvalued. A dataset D is suitable for personalization training if it has high DegreeD score. Mathematical proof of robustness of DegreeD under variant choice of is given in Appendix F..
7.2 Effect of Augmentation on Dataset Diversity
We analyze PerAugy's effect dataset diversity. We observe that TD44MPP achieves boost of 4.2/0.09/0.28 t w.r.t. TP TP/RTC/Degredd compared to the PENS original dataset and a boost of f w.r.t TP/RI TP/RRC/Degreee compared to the OAI original dataset (TTble B). We ablate the effect of the hyperparameters of PerAugy on DegreeD, including gap length gi and trajectory length for Ths as well as context length k decay constant X, and perturbation probability PSMP for TDS4SMP (see Appendix L2
Figure D).
7.3 Dataset Diversity Metrics as a Potential Predictor of Performance Gain
In order to draw a conclusive statement on dataset diversity being a primary cause of performance gains across models, first have to establish the reliability and stability of the three diversity metrics, along with
16

---

Published in Transactions on Machine Learning Research (10/2025)
Augmentation Basclines
TP
RTC
DegreeD
PENS (original)
7.3
0.56
0.009
PENS-SHi
7.4
0.54
0.067
S3t
7.3
0.56
0.019
SDD Interf
7.8
0.63
0.083
LLAMA-2-13B(
8.8
0.61
0.113
Mistral-7Bt
8.6
0.63
0.144
DeepSeek-R1i
9.4
0.68
0.219
PerAugy DSf
13.6
0.77
0.232
PerAugy DS+SMP1
13.6
0.77
0.289
OpenAl (Reddit) (OAI)
8.5
0.42
0.008
PerAugy-OAl+
11.7
0.72
0.121
Table 4: Diversity analysis: Comparison of DegreeD, TP, and RTC across datasets. indicates aug- mentation followed by UIG abstraction on PENS, and indicates augmentation on seed OpenAl(Reddit), Observation-I: PerAugy shows higher diversily in terms all 3 melrics than ils seeds and angmenlation baselines. Observation-2: While TP and RTC relate diversity, they foll short in capturing preference shifts effectively (e.g. PENS-SH has lower RTC than PENS, although it leads to higher user-encoder accu- mecy; see Table :bservaavi::::::: Per DS and PerAugy DS+SMP yields same across TP and RTC becouse they are agnostic of the inserted s-nodes, while SMP changes s-node contents only on the top of DS, thereby making DegreeD better evaluator of the pertubation w.r.t. diversity.
Diversity Metric Pair
Pearson
Spearman
Kendall
TP va RTC
0.75
0.78
0.62
TP va DegreeD
0.79
0.81
0.67
RTC vs DegreeD
0.84
0.90
0.75
Table 5: Inter-correlation (Pearson r, Spearman Ps Kendalls 7) between Diversity Metrics: Observation-1: DegreeD is effective diversity evoluation metric that gives additional insights, which are not coptured by TP and RTC; Observation-2: DegreeD has high compatibilily with text-based diversily evaluation metrics; Observation-3: RTC exhibits higher correlation than TP across all datasets, which suggests that RTC might fail lo coplure true diversification of sequence, and misevaluale a frequently shifling trajectory limited to fewer topics.
whether they have inter-metric alignment. In this section, we first check whether the three metrics align with each other, and then show the meta-evaluation of diversity metrics to establish reliability.
7.3.1 Inter-metric Alignment
We also see a positive correlation between the diversity metrics within themselves, with an average correlation of 0.7 between TP and RTC, 0.75 between TP and DegreeD, and 0.8 between DegreeD and RTC, in Table F This suggests that at broader design level, DegreeD although being a more nuanced metric, is compatible with simpler diversity metrics (correlation computation details have been provided in Appendix H)).
7.3.2 :etaaeevluation: Diversity Metric Reliability
As per the empirical evidences of Tables and B, dataset diversity should lead to an increase in accuracy. Therefore, any diversily metric should be consistent with these empirical results. Hence, we conduct the meta- evaluation of the diversity metrics w.r.t reliability. To this end, compute the correlation of the dataset diversity (both original and synthetic generated by the augmentation methods) as shown in Table g with the average user-encoder accuracy across the encoder models when trained on these datasets We observe a strong positive correlation, thereby high reliability w.r.t consistency, for TP and DegreeD across all accuracy metrics (Pearson: 0.72/0.69, Spearman: 0.7/0.73, and Kendall: 0.53/0.58 w.r.t. nDCG), while RTC has low correlation (see Table B). This indicates that RTC, which only focuses on the absolute frequency of topic shifts
Since LLM-generated trajectorics have been preproccssed track unique sequences to address redundancics (thus making the number of trajectories quite low and yielding significantly lower user-encoder performance). For details of correlation measures, see Appendix
17

---

Published in Transactions on Machine Learning Research (10/2025)
Aggregate Mean Correlation of Diversity Metrics with User Encoder Performance
Diversity Metric
Eval Metric
Pearson r
Spearman P
Kendall's 7
(inel. PerAugy/ excl. PerAugy)
[inel. PerAugy/ excl. PerAugy)
(incl. PerAugy/ excl. PerAugy)
AUC
0.606 / 0.543 (A = 0.063)
0.710 / 0.562 (A = 0.148)
0.518
/ 0.461 (A = 0.057)
MRR
0.765 / 0.686 (A = 0.079)
0.704
/ 0.637 (A = 0.067)
0.542
0.531 (A- = 0.011)
TP
nDCG05
0.719 0.612 (A 0.107)
0.701
0.635 (A 0.066)
0.529
0.421 (A 0.105)
nDCGG00
0.719
0.612 (A 0.107)
0.701
0.635 (A 0.066)
0.529
0.424 (A 0.105)
AUC
0.448 / 0.392 (A 0.056)
0.243 / 0.201 (A 0.042)
0.112 / 0.094 (A 0.018)
MRR
0.642/ / 0.581 (A 0.061)
0.251 / 0.213 (A 0.038)
0.132 / 0.110 (A 0.022)
RTC
DDCG05
0.523 0.172 (A 0.051)
0,234
0.196 (A 0.038)
0,106 0.090 (A 0.016)
1DCG010
0.523
0.472 (A 0.051)
0.234
0.196 (A 0.038)
0.106
0.090 (A 0.016)
AUC
0.682 / 0.645 (A= = 0.037)
0.737
0.712 (A = 0.025)
0.578
0.556 (A = 0.023)
MRR
0.672 0.624 (A 0.048)
0.725
0.684 (A 0.041)
0.552
0.537
(A 0.015)
DegreeD
DDCG05
0.687
0.663 (A 0.024)
0.731
0.693 (A 0.038)
0.580
0.566 (A 0.014)
RDCG@10
0 8.687
0.663 (A 0.024)
0.731
0.693 (A 0.038)
0.580
0.566 (A 0.014)
Table 6: Meta-evaluation of Diversity Metrics: Correlation of dataset diversity with encoder accuracy (averaged) Observation-1 (Reliability)): DegreeD consistently shows the strongest correlation, confirming it as the most reliable diversity indicator. Obscrvation-2: RTC exhibits low correlation throughout, indi- caling that frequency of lopic switching alone is insufficiend for capturing personalization--eeees Observation-3 (Stability): Correlation of TP and DegreeD remains strong even after excluding high- performing PerAugy, thereby conftrming that overall metric reliability is not inflated by PerAugy.
diversily-
Correlation TP and DegreeD with each user encoder
Metric
Corr.
NAML
EBNR
NRMS
TrRMIo (TP / Degrree)
Aggr- Mean Corr.
Variance (TP DegreeD)
(TP/ / DegreeD)
(TP DegreeD)
(TP
DegreeD)
(TP DegreeD)
Pearson
0.318
0.848
0.303/ / 0.380
0.616
0.700
0,370
0.607
0.606/0.682
0,041 / 0.031
AUC
Speaarman
0.602
0.905
0.429/ / 0,571
0.464 / 0.567
0.357 / 0.500
0.710/0.737
0.069 / 0.035
Krudall
0.371 / 0.786
0.229/ / 0.357
0.371 / 0.500
0.257 / 0.429
0.518/0.578
0.049 / 0.030
Peurson
0.167 0.643
0,100/ / 0,239
0.341 / 0.499
0,245 / 0.376
0.7655/0. /0.672
0.313 / 0.057
MRR
Spearman
0.265
0.667
0.153
0.262
0.357
0.433
0.267
0.648
0.704/0.725
0,202 / 0.077
Kendall
0.148 0.500
0.095 / 0.143
0.190
0.286
0.143 0.476
0.542/0.552
0.16 / 0.062
Pearson
0.205
0.772
0.143/ / 0.288
0.382
0.468
0.279
0.638
0.719/0.687
0,226 / 0.054
DDGG85/10
Spearman
0.530 0.786
0.365 / 0.524
0.530 / 0.467
0.414
0.690
/0.771
0.063 / 0.029
Kendall
0.297 0.643
0.185 / 0.286
0.334 / 0.389
0,260
0.524
0.529/0. /0.580
0.071 / 0.033
Table 7: Stability of Diversity Metrics: The sensitivity of TP and DegreeD to strong positive user-encoder
outliers is analyzed via model-specific correlation variance w.r.t aggregate mcan correlation (reported in Table n). Observation-1: Overall, TP an DegreeD consistently has low variance across models; Observation-22 DegreeD has stronger stability w.r.t MRR (the strictest accuracy metric) than TP.
rather than their uniqueness, can incorrectly quantify a low diversity dataset high. It is to be noted that beyond PerAugy multiple bascline augmentations (c.g., SDAInter, PENS-SH) also raise encoder accuracy over the original PENS set, indicating that increased and well-aligned diversity can improve learning.
7.3.3 Meta-evaluation: Diversity Metric Stability
Having established that the diversity-accuracy relationship holds across all augmentation methods (not. only PerAugy), we next test whether the strong correlation observed between datasct diversity and encoder performance is inflated by two potential sources (i) PerAugy itself that acts as a strong outlier and dominates the correlation trend, and (ii) specific strong positive user-encoder outlier whose unusually high accuracy performance disproportionately boosts the correlation values.
There is a possibility that the high performance of PerAugy acts as a positive strong outlier. This can inflate the reliability correlation of TP and DegreeD thereby giving a mislcading metric reliability. We perform the same experiment as Table F but excluding Per Augy- generated datasets. We observe that the correlation of both TP and DegreeD do not vary and lie in the same high correlation band, underscoring that the diversityaccuracy trend is not because of PerAugy alone
Stability w.r.t. PerAugy as strong outlier:
18

---

Published in Transactions on Machine Learning Research (10/2025)
(see Table E A: absolute difference in correlation values; Mean values PTP 0.082, MoegreeD 0.023; Standard Deviations OTP 0.03, Dbggree 0.01).
Stability w.r.t. strong outlier user-encoder: The previous analysis cannot detect whether user- encoder acts as a strong positive outlier and inflates the aggregate correlation results reported. To address this, we find the correlation between dataset diversity (w.r.t TP and DegreeD) and individual user-encoder (i.e., NAML, EBNR, NRMS, TrRMlo) accuracy performances (w.r.t AUC, MRR, and nDCG metrics). We then compute the correlation variance (averaged across the user-encoders w.r.t the aggregated mean reported in Table D). We observe that DegreeD consistently demonstrates lower variance than TP (average gain of 0.07 t). Also, the results indicate that TP as diversity metric is not stable indicator of the role of the underlying dataset diversity under the stricter condition of MRR of encoders. In general, both metrics are stable w.r.t outlier models.
Although DegreeD more stable and robust across models, it is relatively less interpretable and computa- tionally heavier. TP, on the other hand, is simpler and interpretable but fails to capture the magnitude and direction of semantie shifts, making it insensitive to subtle preference drifts.
7.4 Data Diversity Causes Personalization Boost
Having empirically established the reliability and stability of TP and DegreeD, we can conclude that PerAugy reliably outperforms other baseline augmentation methods by significant margin in terms of injecting diversity in the seed datasets (outperforms best (DeepSeek-R1) by 4.2/0.03 w.r.t TP PD/DegreeD; Table E)). However, the user-encoder and the downstream model architecture matter, we find that NAML, NRMS, and TrRMIo being able to capture the diversity more, showing consistent correlation with diversity metrics.
8 Related Work
Prior works data augmentation have largely focused on generative approaches for dialogue summarization (Lin et all, 2022; Ouyang et al. 2023 Park et al, 2022)). and document-level augmentation for generalized summarization tasks (Fabbri et all 2021; Chen et all, 21R31 Sahu-et_ai all aIz)). However, to the best of our knowledge, preference-oriented data augmentation
the contert of persomalized summorization remains significantly underexplored. The most relevant effort this domain PENS-SH (Song et al, 2023), which constructs synthetic user trajectories by identifying and morging common d-nodes across multiple user interaction graphs (UIGs). While effective at preserving shared preferences, PENS-SH fails to retain temporal order information due to the loss of time-step data, and entirely lacks intermediate summary nodes (s-nodes), leading to an incomplete representation of user intent and preference evolution. Broadly, preference data augmentation methods for sequential recommendation can be categorized into two classes: intra-trajectory and cross-trujeclory augmentation. Most of these techniques use sequential recommendation datasets like Amazon, MovieLens, where the goal is next interaction prediction, not preference- based generation.
8.1 Intra- Trajectryy Augmentation
Most intratrajectory methods perturb each users history by locally manipulating nodes or segments, but within the same trajectory. For example, MBASR (Xiao et 2024a) employs an intra-trajectory aug- mentation technique by performing pairwise swapping of segments to generate diversity. But on highly monotonous trajectories, this yields minimal change and may even inject unrealistic temporal transitions. STEAM et (тс operates by deciding whether to drop or insert nodes within a trajectory to create augmented data. However, the method is not scalable to longer trajectories, and the insertion or deletion of nodes can disrupt the historical sequence or break the natural flow of interactions. L2Aug (Wang et all 2112) learns a policy to delete nodes, but deletions disrupt continuity and remove potentially informative context. The Heuristic SAMPLER model (Chen el. all, 2023)) replaces single nodes via popularity or cooccurrence- based heuristics, but such isolated swaps fail to shift the sequences overall engagement degree. ASReP (Li all 2021) extends trajectorics by generating pseudo-prior nodes using reverse pretraining. Although it aims to enrich the trajectory, any kind of insertion technique (or segment extension) can incorporate un- realistic synthetic nodes that compromise the authenticity of time-step information of further interactions,
19

---

Published in Transactions on Machine Learning Research (10/2025)
making it unsuitable for our application. Finally, BTBR (Li 2023) incorporates masking strategies and swapping operation to train the model for "Next Novel Basket Recommendation" But it does not generate a diverse input sequence to make the existing encoders learn the representations.
PerAugy departs from these schemes by applying controlled perturbations that operate at both micro and macroscales while preserving strict temporal coherence since intra-trajectory augmentations does not solve the problem of monotonous user history. Rather than swapping or deleting isolated nodes, we perform shuffling that incorporates controlled shifts and apply perturbation to diffusion. This approach polishes the sequenceenriching diversitywithout sacrificing realistic timestep information or overburdening the augmen- tation pipeline, making it scalable to long trajectories.
8.2 Cross- Trajectory Augmentation
Crosstrajectory techniques draw patterns across users to forge new sequences and augments multiple trajec- tories. DR4SR (Yin et all, 2022) uses transformer to learn global sequence regeneration. The pertaining task is constructed for to extract patterns from given set of sequences and feed the patterns to the model to regenerate other set of possible sequences, but applying interchangeable patterns across subjective s-node summaries risks injecting generic behaviors that dilute personalization. TiCoSeRec (Dang et al, 2122)) en- sures uniform time-interval distribution in the soquence basod on the timo-aware traditional operations like Crop, Mask, Insert, Reorder and Substitute. But our trajectories inherently assume consistent unitstep timing, thereby making such timeaware edits redundant. FDA (Chen et all, 21265) generates synthetic user profiles from the realistic profiles to balance between realistic data and pseudo data. But it generates monotomic complemented sequences that mirror existing repetition rather than diversifying it.divSPA (Li alall 2123 swaps segments between similar users based on similarity metries. Similaritybaser exchanges often leave the overall interaction degree unchanged and introduce context mismatches.
Instead of wholesale regencration or arbitrary segment swaps, PerAugy leverages crosstrajectory substitution to adapt its perturbation parameters dynamically. By analyzing interuser variance in snode distributions, our method addresses optimal perturbation scales and target positions, ensuring that borrowed structure euhances diversity without compronising each trajectorys unique sunmarization points w.r.t. realistic user- behaviors. This yields augmented sequences that are both personalized and informationrich, overcoming the homogenization (or unrealistic heterogenization) pitfalls of existing crosstrajectory approaches.
9 Discussions & Limitations
PerAugy enhances the diversity and expressiveness of the original user history data while preserving personal- ization fidelity. This approach is particularly beneficial for improving model robustness and generalization in sparse or skewed datasets. In deployment settings, where user-written summaries are typically unavailable, model-generated summaries can serve as effective proxies, especially during cold-start scenarios such as a new browsing session, which we address via experiments on the OpenAI(Reddit) dataset. Looking forward, we see promising opportunities extending this augmentation framework using large language models (LLMs). In particular, we are investigating prompt-tuning- based augmentation techniques that could generate more semantically rich and user-aligned variations of preference histories. Such methods hold the potential to be especially impactful in low-resource non-PENS-Hie domains, where user signals are limited or noisy, Additionally, we aim to ground perturbation modeling in more principled stochastic processes. One such candidate is the Itô process, which incorporates both deterministic trends (drift) and random fluctuations (diffusion). Modeling user preference evolution through such contimuuus-tim stochastic frameworks may offer a more realistic approximation of human behavior, allowing for fino-grained control over the inten- sity and direction of perturbations. This could open avenues for theoretically grounded, temporally aware augmentation strategies that better reflect user dynamics in real-world settings.
20

---

Published in Transactions on Machine Learning Research (10/2025)
10 Conclusion
In this paper, we introduced PerAugy, novel data augmentation technique designed to enhance the personal- ization capabilities of summarization models. By addressing limitations in current personalized datasets like PENS, PerAugy generates synthetic, diverse user interaction trajectories, reducing overfitting and improving gencralization across domains. Techniques like Double Shuffling (DS) and Stochastic Markovian Perturba- tion (SMP) ensure that the augmented data remains realistic and coherent, enabling models to better align with individual user preferences. Our evaluation demonstrated significant. improvements in the personaliza- tion metric PSE (an average of 61.2% boost), particularly in models like PENS+NRMS+12, which achieved 75% performance increase (PSE-RG-SU4). PerAugy also improved user-encoders such as NAML, EBNR, and NRMS, enhancing their ability to capture user preferences with average boosts of 24%, 25%, and 18% over baseline augmentations (w.r.t AUC, MRR, and nDCG@5&:10). We further demonstrated its potential as a reliable generator synthetic datasets in low-resource domains like OpenAl (Reddit), with encoder boosts of 19%, 25%, and 17% in the same metrics, broadening its applicability. While PerAugy critical advancement. in addressing data scarcity and generalization in personalized summarization, future work will refine its techniques and explore adaptability to more models and architectures.
Acknowledgment
T. Chakraborty would like to thank the support of the Rajiv Khemani Young Faculty Chair Professorship in AL. We would also like to thank Lightning.ai for GPU credits and Prof. Ankush Chander of DAU for helping the team with implementation details.
References
Xiang Ao, Xiting Wang, Ling Luo, Ying Qiao, Qing He, and Xing Xie. PENS: A dataset and generic framework for personalized news headline generation. In Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli (ods.), Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume Long Papers), pp. 82-92, Online, August 2021. Association for Computational Linguistics. doi: 10.18653 11/8652211222211111 accl..cl-oo URL
hotp://hhooooooooo221111111 acl- 77|
Pengshan Cai, Kaiqiang Song, Sangwoo Cho, Hongwei Wang, Xiaoyang Wang, Hong Yu, Fei Liu, and Dong Yu. Generating user-engaging news headlines. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (eds.), Proceedings of the 61st Annual Meeting of the Association for Computational Linguistios (Volume 1: Long Papers), pp. 3265 3280, Toronto, Canada, July 2023. Association for Computational Linguistics. acl acl-long. 183/
doi: 10.18653/ /l acl.ooo..l-ooo..1... 183. URL
Lci Chen, Le Wu, Kun Zhang, Richang Hong, Defu Lian, Zhiqiang Zhang, Jun Zhou, and Meng Wang. Improving recommendation fairness via data augmentation. In Proceedings of the ACM Web Conference 2023, WWW '23, pp. 10121020, New York, NY, USA, 2023a. Association for Computing Machinery. ISBN 9781450394161. doi: 10.1145/ 10.1145/3543507, (3543507.3583342
URL https://do1.org//11 10. 1145/144354/ 3583341,
Xiuying Chen, Guodong Long, Chongyang Tao, Mingzhe Li, Xin Gao, Chengqi Zhang, and Xiangliang Zhang. Improving the robustness of summarization systems with dual augmentation. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (eds.), Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 6846 6857, Toronto, Canada, July 2023b. Association for Computational Linguistics. doi: 2L116633/1122233ac-- https:
URL
wwwporggg.//... acl- 3787.
Xu Chen, Zhenlei Wang, Hongteng Xu, Jingsen Zhang, Yongfeng Zhang, Wayne Xin Zhao, and Ji-Rong Wen. Data augmented sequential recommendation based on counterfactual thinking. IEEE Transactions Knowledge and Data Engineering, 35(9):9181-9199) 9194, 2023c. doi: 10.1109 1111//109/0/..0 202.....22..222070..
Yizhou Dang, Enneng Yang. Guibing Guo, Linying Jiang, Xingwei Wang, Xiaoxiao Xu, Qinghui Sun, and Hong Liu. Ticoserec: Augmenting data to uniform sequences by time intervals for effective recommendation.
21

---

Published in Transactions on Machine Learning Research (10/2025)
IEEE Transactions on Knowledge and Data Engineering, 36(6):2686-2700 2024. doi: 10.1109/TKDE.202. 3324312.
Sourish Dasgupta, Ankush Chander, Tanmoy Chakraborty, Parth Borad, and Isha Motiyani. PerSEval: Assessing personalization in text summarizers. Transactions on Machine Learning Research, 2024. ISSN
2835-8856. URL
DeepSeek-AL AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Pciyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei Feng, Chengda Lu, Chenggang Zhao, and Chengqi et al. Deepseck-rl: Incentivizing reasoning capability in Ilms via reinforcement learning, 2025. URL https: wttp.//rr/aab//1.12555995
Zi-Yi Dou, Pengfei Liu, Hiroaki Hayashi, Zhengbao Jiang, and Graham Neubig. GSum: A general frame- work for guided neural abstractive summarization. In Kristina Toutanova, Anna Rumshisky, Luke Zettlemoyer, Dilek Hakkani-Thn Iz Beltagy, Steven Bethard, Ryan Cotterell, Tanmoy Chakraborty, and Yichao Zhou (ods.), Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 4830 4842, Online, June 2021. Association for Computational Linguistics. doi: 10.18653/v1/2000 maaac-main.384 URL https://acLanthooogg org/ org/ www.naaacmaaaa 3847.
Alexander Fabbri, Irene Li, Tianwei She, Suyi Li, and Dragomir Radev. Multi-news: A large-scale multi- document summarization dataset and abstractive hierarchical model. In Anna Korhonen, David Traum, and Lluís Mårquez (eds.), Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 1084,1084, Florence, Italy, July 2019. Association for Computational Linguistics. doi: 10.18653/v1/P19- 1102. URL /ttts:///aclanthooogy.ooo orgp199-12/
Alexander Fabbri, Simeng Han, Haoyuan Li, Haoran Li, Marjan Ghazvininejad, Shafiq Joty, Dragomir Radev, and Yashar Mehdad. Improving zero and few-shot abstractive summarization with intermediate fine-tuning and data augmentation. In Kristina Toutanova, Anna Rumshisky, Luke Zettlemoyer, Dilek Hakkani-TTuu Tz Beltagy, Steven Bethard, Ryan Cotterell, Tanmoy Chakraborly, and Yichao Zhou (eds.), Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 704-717, Online, June 2021. Association for Computational Linguistics. doi: 10. 18.186/3vl 21655//v/2221...1 wtw.://........nnnniiiem maacl-main.55. URL https://eeLantthoooov.ooogg
naaci-main.5) 577.
Shivam Grover, Amin Jalali, and Ali Etemad. Segment, shuffic, and stitch: A simple layer for improving time-series representations. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang (eds.), Advances in Neural Information Processing Systems, volume 37, pp. 4878 4905. Curran Associates, Inc., 2024. URL https::/prooceedingg uure httpr//p...cc/pprre..e//////eeee 2AA.EF55c12227222220d22333333323
Junxian Hc, Wojeiech Kryscinski, Bryan McCann, Nazneen Rajani, and Caiming Xiong. CTRLsum: Towards generic rollabea text summarization. In Yoav Goldbery, Zornitsa Kozareva, and Yue Zhang (eds.), Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 5879- 5915, Abu Dhabi, United Arab Emirates, December 2022. Association for Computational Linguistics. doi: 10.18653/v1 www.2.eemmlpmmaie main.396. URL https://accaanthoooo.cooo wwwpsantttoooooooooooeeee//ee
/2t. em- 3967
Karl Moritz Hermann, Tomáš Kočiský, Edward Grefenstette, Lasse Espcholt, Will Kay, Mustafa Sulcyman, and Phil Blunsom. Teaching machines read and comprehend. In Proceedings of the 28th International Conference on Neural Information Processing Systems Volume 1, N1PS115, pp. 16931701, Cambridge, MA, USA, 2015. MIT Press.
Zhiting Hu, Bowen Tan, Ruslan Salakhutdinov, Tom Mitchell, and Eric P. Xing. Learning data manipulation for augmentation and weighting. Curran Associates Inc., Red Hook, NY, USA, 2019.
Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, al. Mistral 7b. arXiv e-prints, pp. arXiv-2310 2023.
22

---

Published in Transactions on Machine Learning Research (10/2025)
Yang Jiao, Fan Yang, Yetian Chen, Yan Gao, Jia Liu, and Yi Sun. Rethinking sequential relationships: Improving sequential recommenders with inter-sequenee data augmentation. In Companion Proceedings of the ACM Web Conference 2024, WWW '24, New York, NY, USA, 2024. Association for Computing Machincry. ISBN 9798400701726.
Hannah Rose Kirk, Bertie Vidgen, Paul Röttger, and Scott A Hale. The benefits, risks and bounds of personalizing the alignment of large language models to individuals. Nature Machine Intelligence, pp- 10, 2024.
Frederic Kirstein, Terry Ruas, Robert Kratel, and Bela Gipp. Tell me what need to know: Explor- ing based (personalized) abstractive multi-source meeting summarization. In Franck Dernoncourt, Daniel Preotiuc-Pictro, and Anastasia Shimorina (eds.), Procecdings of the 2024 Conference on Empirical Methods in Natural Language Processing: Industry Track, pp. 920 939, Miami, Florida, US, Novem- ber 2024. Association for Computational Linguisties. doi: 10.18653/vl. /t////222.22mmm.. inmul---ddustty.69. URL https://acllanthlloo
Suchetha N. Kunnath, David Pride, and Petr Knoth. Prompting strategies for citation classification. In Proceedings of the 32nd ACM International Conference Information and Knowledge Management, CIKM '33, pp. 11271137, New York, NY, USA, 2023. Association for Computing Machinery. ISBN 9798400701145.. doi: 00.1155/5533890:3150088 URL
http://doo..rg////// 1r1.arrr///0111///3333/8880 1145/3583780.361550
Ming Li, Mozhdeh Ariannezhad, Andrew Yates, and Maarten de Rijke. Masked and swapped sequence modeling for next novel basket recommendation in grocery shopping. In Proceedings of the 17th ACM Conference on Recommender Systems, RecSys 223, pp. 3546, New York, NY, USA, 2023. Association for Computing Machinery. ISBN 9798400702419, doi: 10.1145/3004915336008
URL https://dol.org//1.
1145/3624915.368883
Junhong Lian, Xiang Ao, Xinyu Liu, Yang Liu, and Qing He. Panoramic interests: Stylistic-content aware personalized headline generation. In Companion Proccedings of the ACM on Web Conference 2025, 2025.
Chin-Yew Lin. ROUGE: A package for automatic evaluation of summaries. Text Summarization Branches Out, pp. 74-81, Barcelona, Spain, July 2004. Association for Computational Linguistics. URL https:
//aclanthology 001100000000000///111111
Yujie Lin, Chenyang Wang, Zhumin Chen, Zhaochun Ren, Xin Xin, Qiang Yan, Maarten de Rijke, Xiuzhen Cheng, and Pengjie Ren. A self-correcting sequential recommender, 2023. URL https://axxwv.or//abs
2303.02291
Chong Liu, Xiaoyang Liu, Ruobing Xie, Lixin Zhang, Feng Xia, and Leyu Lin. Learning from all sides: Diversified positive augmentation via self-distillation in recommendation. CoRR, abs/2308.07629, 2023.
URL https://do1.org/// 48550//rki. 2308. 2308.07629.
Ruibo Liu, Jerry Wei, Fangyu Liu, Chenglei Si, Yanzhe Zhang, Jinmeng Rao, Steven Zheng, Daiyi Peng, Diyi Yang, Denny Zhou, and Andrew M. Dai. Best practices and lessons learned on synthetic data. In First Conference on Language Modeling, 2024. URL / wttp://wwwww//ooo///////oaaaaaea
Yongtai Liu, Joshua Maynez, Gonçalo Simöes, and Shashi Narayan. Data augmentation for low-resource dialogue summarization. In Marine Carpuat, Carre-Catherine de Marneffe, and Ivan Vladimir Meza Ruiz (eds.), Findings of the Association for Computational Linguistics: NAACL 2022, pp. 703-710, Seattle, United States, July 2022. Association for Computational Linguistics. doi: 10.18653/v1 h2tp://////222///// www..2...ninng--aaa... org.://gg//2222 findingg--nnacl..55
53. URL
Zhiwei Liu, Ziwei Fan, Yu Wang, and Philip S. Yu. Augmenting sequential recommendation with pseudo- prior items via reversely pre-training transformer. In Procoedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 21. ACM, July 2021. doi: 10.1145/3404835.34663 URL http://dx. do1 org/10 1ttp://0..11///0443555..4
23

---

Published in Transactions on Machine Learning Research (10/2025)
Shumpei Okura, Yukihiro Tagami, Shingo Ono, and Akira Tajima. Emneddding- based news recommen- dation for millions of users. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '17, pp. 19331942, New York, NY, USA, 2017. Associ- ation for Computing Machinery. ISBN 9781450348874. doi:
URL https:
//do1.org/T00 10t.1114533999990 3098T08.
Siru Ouyang, Jiaao Chen, Jiawei Han, and Diyi Yang. Compositional data augmentation for abstrac- tive conversation summarization. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (eds.), Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pp. 1471-1488, Toronto, Canada, July 2023. Association for Computational Linguistics. URL
doi: 10. N0.18853/v1//20 /tt////12222..lloogggg
Sangwon Park, Hongsook Choi, Dongha Choi, and Hyunju Lee. GENDEX: Generative data augmentation strategy leveraging external data for abstractive dialogue summarization. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Findings of the Association for Computational Linguistics: ACL 2024, pp. 3171-3185, Bangkok, Thailand, August 2024. Association for Computational Linguistios. doi: 10. 10.18653/ 11/2024.findings-all.B33 URL
Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics, 11 2019. URL https://arxww.or/abbs.
10084.
Pranab Sahoo, Ayush Kumar Singh, Sriparna Saha, Vinija Jain, Samrat Mondal, and Aman Chadha. A systematic survey of prompt engineering in large language models: Techniques and applications, 2024.
URL
Gaurav Sahu, Olga Vechtomova, and Issam H. Laradji. A guide to effectively leveraging 1lms for low- resource text summarization: Data augmentation and semi-supervised approaches, 2025. URL https:
//arxiw.org/abs/2447.11 07731.
Yun-Zhu Song, Yi-Syuan Chen, Lu Wang, and Hong-Han Shuai. General then personal: Decoupling and pre-training for personalized headline generation. Transactions of the Association for Computational Linguistics,
2023. doi: 10.1162/taci a_ 00621. URL https://eeLLattiooeeeyor////////
taci-1.901 907
Md Arafat Sultan, Jatin Ganhotra, and Ramón Fernandez Astudillo. Structured chain-of-thouggh prompting for few-shot generation of content-grounded QA conversations. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.), Findings of the Association for Computational Linguistics: EMNLP 2024, pp. 16172-16187, Miami, Florida, USA, November 2024. Association for Computational Linguistics. doi: 10.18653/v1/22 w1/0//11/2222//// w1220/4.nnninggs.-oo..
URL https://aclantiioooyy.ogg htg.e/atttoooooooooo////e/eennnn 9487
Shichao Sun, Ruifeng Yuan, Ziqiang Cao, Wenjie Li, and Pengfei Liu. Prompt chaining or stepwise prompt? refinement in text summarization. In Len-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Findings of the Association for Computational Linguisties: ACL 2024, pp. 7551-7555 Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: 10.18653 /1/////1///// 1112222...nniiii acc.ings-aal.4999 URL https: //aclanthology.co //aclanttoloog..rr//004...11 Fit.//02...nddnng-aal......
Maartje Ter Hoeve, Julia Kiseleva, and Maarlen Rijke. What makes a good and useful summary? Incorpo- rating users in automatic summarization research. In Marine Carpuat, Marie-Catherina de Marneffe, and Ivan Vladimir Meza Ruiz (cds.), Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 46 75, Seattle, United States, July 2022. Association for Computational Linguisties. doi: 10. 18.15863/v1 /2022. mamaa-aminn..... URL http://cccattoolloo.oooo/////22a
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaci, Nikolay Bash- lykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv e-prints, pp. arXiv-2300 2307, 2023.
24

---

Published in Transactions on Machine Learning Research (10/2025)
Rahul Vansh, Darsh Rank, Sourish Dasgupta, and Tanmoy Chakraborty, Accuracy is not enough: Evaluating
personalization in summarizers. In Findings of the Association for Computational Linguisties: FMNLP 2023, pp. 2582 2595, Singapore, December 2023. Association for Computational Linguistics. doi: 10.
18653/v1/ 2 findings-emnlp 1m..p. URL httpss//aclanttiooogg.ooo wttg//22..iinnnnns........ findings-emn_nn 169.
Michael Völske, Martin Potthast, Shahbaz Syed, and Benno Stein. TL;DR: Mining Reddit to learn automatic summarization. In Lu Wang, Jackie Chi Kit Cheung, Giuseppe Carenini, and Fei Liu (eds.), Proceedings of the Workshop on New Frontiers in Summarization, pp. 59-63, Copenhagen, Denmark, September 2017.
Association for Computational Linguistics. doi: (00118633v1/W117-0 org/w17-4508.
URL https://acLanthooogg
Jianing Wang, Qiushi Sun, Xiang Li, and Ming Gao. Boosting language models reasoning with chain-of- knowledge prompting, 2024. URL https://arxiv.orr .rgg hwwpg//bb/230..... 23640
Jianling Wang, Ya Le, Bo Chang, Yuyan Wang, Ed H. Chi, and Minmin Chen. Learning to augment for casual user recommendation. In Proceedings of the ACM Web Conference 2022. WWW '22, pp. 21832194, New York, NY, USA, 2022. Association for Computing Machinery. ISBN 9781450390965. doi:
10.1145/3485447..3 351121477 URL 0ttps///000.0////01111444444441 3512777
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Lc, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models, 2023. URL
Chuhan Wu, Fangzhao Wu, Mingxiao An, Jianqiang Huang, Yongfeng Huang, and Xing Xie. Neural news rocommendation with attentive multi-view learning. In Proccodings of the Twenty-Eighh International Joint Conference on Artificial Intelligence. International Joint Conferences on Artificial Intelligence Orga-
nization, 2019a.
Chuhan Wu, Fangzhao Wu, Suyu Ge, Tao Qi, Yongfeng Huang, and Xing Xie. Neural news recommen- dation with multi-head self-attention. In Proceedings of the 2019 Conference on Empirical Methods Natural Language Processing and the 9th International Joint Conference on Natural Language Processing Hong Kong, China, November 2019b. Association for Computational URL. https: wwtpp//-oog//D//1111
(EMNLP-LJCNAL) pp. Linguistics. doi: 10. (0.18633/w1/D19-11
Fangzhao Wu, Ying Qiao, Jiun-Hung Chen, Chuhan Wu, Tao Qi, Jianxun Lian, Danyang Liu, Xing Xie, Jianfeng Gao, Winnie Wu, and Ming Zhou, MIND: A large-scale dataset for news recommendation. In Dan Jurafsky, Joyce Chai, Natalic Schluter, and Joel Tetreault (eds.), Proceedings of the 58th Anmual Meeting of the Association for Computational Linguistics, pp. 3597-3606, Online, July 2020. Association /rg/
for Computational Linguistics. doi: 10. N0118633/11/200...cmmm....33 URL
2820_ac1
ac -ma-inn.31111331111
Jing Xiao, Weike Pan, and Zhong Ming. A generic behavior-aware data augmentation framework for sequen- tial recommendation. In Procoedings of the 47th International ACM SIGIR. Conference Rescarch and Development in Information Retrieval, SIGIR '24, pp. 15781588, New York, NY, USA, 2024a. Association for Computing Machinery. ISBN 9798400704314. doi: 10.145/8626772.666662. URL
01155/6627772.657656 http:://0...0007111155/555777553
Wen Xiao, Yujia Xie, Giuseppe Carenini, and Pengcheng Hc. Personalized abstractive summarization by tri-agent generation pipeline. In Yvette Grahan and Matthew Purver (eds.), Findings of the Association for Computational Linguistics: EACL 2024, pp. 570-581 581, St. Julian's 's Malta, March 2024b. Association for Computational Linguistics. URL https://aclanthoolgyy orgh org.oog//////// 2024.
Ftt.://ooooo.ooor//22emo..neneee Findings-eacl...
Zhao Yang, Junhong Lian, and Xiang Ao. Fact-preserved personalized news headline generation. 2023 TEEE International Conference Data Mining (ICDM), 2023.
Jiacheng Ye, Shansan Gong, Liheng Chen, Lin Zheng, Jiahui Gao, Han Shi, Chuan Wu, Xin Jiang, Zhen- guo Li, Wei Bi, and Lingpeng Kong. Diffusion of thought: Chain-of-thought reasoning in diffusion language models. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomezak, and
25

---

Published in Transactions on Machine Learning Research (10/2025)
C. Zhang (eds.), Advances in Neural Information Processing Systems, volume 37, pp. 105345-105374.
Curran Associates, Inc., 2024. URL https://proceedingg hww.eeeddnnnnn.nnii...... cttpeips.ccpppaee wwt.eeeeaapeee/222////1// ba5004e7822ee///Garo0007-------1
pdf.
Mingjia Yin, Hao Wang, Wei Guo, Yong Liu, Suojuan Zhang, Sirui Zhao, Defu Lian, and Enhong Chen. Dataset regeneration for sequential recommendation. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD 24, pp. 39543965. ACM, August 2024. doi: 10.1145/
3637528.3671841 URL http://dx.dol.orgg.. 1145/3637588.... 36718411
Lemei Zhang, Peng Liu, Marcus Tiedemann Oekland Henriksboe, Even W. Lauvrak, Jon Atle Gulla, and Heri Ramampiaro. Personalsum: sube-subeeetive guided personalized summarization dataset for large language models. In The Thirty-cight Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2024. URL http:://ppeeevvvee...r//rorotttt
A Measuring Degrre--of-Prrsonaizttioo
A.1 Motivation
Vansh et all (2023) proposed EGISES- metric measure the degree of insenstivvty----sijjeeiivy for relative benchmarking of how much models lock personalization (i.e., lower score better within the range [0, 1]) instead of assigning absolute goodness score. Based on this notion, they defined (summary- level) "deviation" of model MMaterrer termed as Degree-of- Reieeoof--Rsposivveeeess (DEGRESS) by DDsgupta et al.
(2024] as follows:
Definition 3. Summary-level DEGRESS. Given document di and a user-profile uij (user j's expected summary), the summary-level responsiveness of personalized model Mo.u, (i.e., DEGRESS($ [d,, uiy))), is defined as the proportional divergence belween model-generated summary Suz of d4 for j-th user from other user-specific summary versions w.r.t a corresponding divergence of usi from the other user-profile
DEGRESS Su.y (dd, uij)) is formulated as:
14
min min(Xygk Yyx) + max(Xisk Yisk) +e
DEGRESS( (ddi, Wiz))
U
exp(w(uiyluaa o (ues, ua)) Yosk
exp(w((su, (lme)) ossuy Sun)
Xisk U
U2
Σ (xp(w((ssluaz)
exp(u(su,)lBua)
auuy, usk, d,)
o (Sugz Smce)
w( (uxx))
u (Sag))))u))
alBuy,di)
Here, [D] is the total number of documents in the evaluation dataset, [U] is the total number of users who created gold-reference summaries that reflect their expected summaries (and thereby, their subjective preferences), and Ua,, (= [) is the number of users who created gold-references for document d,. w is the divergence of the moode--genrateer summary Suy (and the corresponding expected summary uzj) from all the other versions. It helps determine how much percentage
document d, itself in comparison (therefore, the softmax function) of the divergence (i.e., (8aup) Su,)) should be considered for the caleulation of DEGRESS. If Suy is farther than Su, w.r.t d; then DEGRESS (Say (dd,, u;y)) DEGRESS [(d,, ua)),, implying that Me... is more responsive to the k-th reader. A lower value of DEGRESS( (di, wiz)) indicates that while reader-profilee profiles are different, the generated summary Su,, is very similar to other reader-specific summaries (or vice versa), and hence, is not responsive at the summary-level. The system-level DEGRESS and EGISES have been formulated as follows:
Jua,
10,1
DEGRESS (Mo,
(9)
D
26

---

Published in Transactions on Machine Learning Research (10/2025)
A.2 PerSEval: Formulation
As can be noted, the DEGRESS formualtion does not enforce any penalty on accuracy drop. To rectify this Dasgupta et al. ( proposed PerSEval. The design of PerSEval had two key goals: (i) to penalize models for poor accuracy, while simultancously (ii) ensuring that the evaluation responsiveness (i.c., DEGRESS) is not overshadowed by high accuracy. This penalty is referred to as the Effective DEGRESS Penally Factor (EDP). If a model achieves 100% accuracy, no EDP will be applied, and the PerSEval score will equal the DEGRESS score. The following formulatiown of PerSEval guarantees these properties:
Per,Eval(s.,,I(dde u.y)) = ( (d., uny)) x (DP [dde, u:y))
where, (0,(s,,,((d;; u;y))
(10)
10x ( (- (sug)(dd,, u;)l)))
(sugyl(d, uz)) ADP(su.- [(d, )x-)) + ACP(su.y [dd, u.;))
Here, ADP is document-level penalty due to drop in accuracy for the best-performance of the model (i.e., the model-generated summary of document d, (Sur) is closest to the corresponding reader's 's expected summary wij). ADP is formulated as follows:
AD ((d,, (k=)) + 10724 exp -10
x" "Suce Mic) di-0 [1-o (Auze lddt+e
JU,1
(11)
where, o² (Bus [d; = min a( [d; j=1
and (e: : An infinitssmalll small number E (0)
ADP ensures that even if the DEGRESS score is acceptable, a penalty due to accuracy drop can still be imposed as a part of EDP. ADP, however, fails to address the scenario where the best-case scenario is acceptable (i.e., accuracy is fairly high) but is rather an outlier case i.e. for most of the other model-generated summary versions, there is a considerable accuracy drop. To address this issuc, the second penalty component within EDP called A -anuce--ncoonsistency Penalty (ACP) introduced which evaluates whether model consistently performs w.r.t accuracy for specific generated summary compared to its average performance. ACPis formulated as:
ACP( [(ds, Uky))
1 10.24.14.00 exp (-10- 10
ofsuey May (Eui. 2ie I (ol M.e di (Sug Mia 1F
(12)
1
where, (B Uie) )i
a (Buey uis) dj
U
The system-level PerSEval score as follows:
PersEval( ((d(.)))))
[4a1
PerSEval(Mo..)
(13)
D
The system-level PerSEval [0, 1] and is bounded by the system-level DEGRESS score.
PerSEval-RG-SUA. (or PSE-SU4) is the PerSEval variant that uses ROUGE-SU4 (Lin. 2004 as a distance metric (i.e,, о) in the PerSEval formula. PSE-SU4 has been reported to have high human-judgment corre-
lation (Pearson's r: 0.6; Spearman's p: 0.6; Kendall's т: 0.51) Dasgupla et. all (20224). The ROUGE-SU4 score is based on skip-bigrams, which are pairs of words that appear in the same order within a sentence but can have up to four other words between them. The formula as follows:
For given generated summary G and reference summary R, the ROUGE-SU4 score is calculated as:
27

---

Published in Transactions on Machine Learning Research (10/2025)
Skip-Bigram Recall (Rsu)):
Rsu4 Count of matching skip-bigrams between G and R
Total skip- bigrams in
Skip-Bigram Precision (Psu4):
Count of matching skip- bigrams between and R
Psu1
Total skip-bigrams in G
F1 Score (F1su4): The Fl score is the harmonic mean of precision and recall:
2 Psu4 Rsu4
Flsu4
Psu4 + RsU4
Where:
A skip-bigram consists of two words in the correct order but with zero to four words skipped in between.
Matching skip-bigrams are counted between the generated summary and the reference summary.
The final ROUGE-SU4 score typically reported as the F1 mcasure, balancing precision and recall.
B Dataset and Statistics
The PENS dataset al (202T) includes 113,762 news articles across 15 topics. Each article contains an ID, title (avg. 10.5 words), body (avg. 549 words), and category, with titles linked to the WikiData entities. The dataset also includes user interaction data, such as impressions and click behaviors, combined with news bodies and headlines from the MIND dataset Wi Wi (2020). For training, 500k user-news impressions were sampled from June 13 to July 3, 2019. Each log records user interaction as [ulD, tmp, clkNews, uelkNews, clkedHis]] where 'clkNews' and 'uclkNews' represent clicked and unclicked news, and 'elkedHis' refers the user's prior clicked articles, sorted by click time. To create offline testbed, 103 English-speaking students reviewed 1,000 headlines in stage=1, and then selected 50 articles, and created preferred headlines (i.e., expected gold-reference summaries) for 200 unseen articles in stage-2 (see Figure [). Each article was reviewed by four participants. Editors checked for factual accuracy, discarding incorrect headlines. The high-quality remaining hcadlines serve as personalizod gold-standard references in the PENS dataset. The PENS dataset has become the standard benchmark for personalized lized summarization task Ao all (2020); Song et all (2013); Yang et al. (2023); Cai ef all (2023); Lian et all (2025) The statistics of PENS dataset are given in Table g
PENS
OpenAI (Reddit). The OpenAT (Reddit) dataset Völske ef all (2017) comprises 123,169 Reddit posts collected from 29 distinct subreddits. This dataset provides both Openne--eeerated and luman--writte sum- maries and is organized into two splits: Comparisons, used for training and validation, and Axis, designated for validation and testing. curated subset of 1,038 posts was processed by different. summarization policies, resulting in the generation of 7,713 summaries. These summaries underwent evaluation by 64 anno- fators who rated paired summaries based on selection preferences, confidence their ratings, and dimensions such accuracy, coherence, coverage, and overall quality (see Table gl for details). Notably, unlike datasets like PENS, these summaries are not linked to individual annotators or their reading histories, which means they lack elements of personalization and contextual user information. The detailed statistics are given in Table g.
C Baselines
Here, we discuss in details the baseline augmentation strategics, user-encoders and summarization frame- works.
28

---

Published in Transactions on Machine Learning Research (10/2025)
NEWS
Model
Model B
Model
Manually-written
ScoTE1
score;
NEWS
Stage One
skip
Stage Two
Headlinel
Hcadline2
Headlinc3
Hcadlinc-human
scoTe1
score
scoTes
Hcadlinel
Headline2
Headline3
Headline-hure
Figure Two stage PENS test data (original) crcation Ao all (202T): Stage Participants
selected 50+ preferred headlines from 1,000 shown titles; Stage 2 They rewrote headlines for 200 unseen articles using only news bodies, without seeing original titles.
Characteristic
Dimension
Value
Article Stats
# Topics
15 113,762
# Articles
General Stats
Avg. Title Length
10.5 words
Avg. Body Length
549 words
Train Dataset Statisties
#
UserNews Impressions (anon.)
500,000
#
Users (anon.)
445,000
Interaction Data
Time Period
June 13July 3, 2019
User Interaction Fields
JuID, tmp, clkNews, uelkNews, elkedHis
Test Dataset Statistics
# Participants
Participant Category
Englishspeaking college students
Participant Stats
# Articles
3,940
Browsed Ieedllnes (Click Skip) Min. Interested (Click) Ieeddlnes
1,000 per participant 50 per participant
Gold Reference (Participantwrittenn Hcadlines)
Summarized Article Bodies
200 per participant
Avg. Summarics per Article
Table 8: PENS dataset (original) statistics: Here the 'clkNews" and 'uclkNews' indicate clicked and un-clicked (i.e., skipped) news; 'clkedHiis' refers to the user's prior clicked articles, all sorted by click time; news bodies and headlines sourced from the MIND dataset Win at. (2020);; Test dataset is created two stages (see Figure F).
C.1 Baseline Augmentations
We compare PerAugy with three SOTA algorithmic augmentation mothods and three LLM-as-augmenttors, and create corresponding UIGs for each of the augmentation methods. The statistical analysis of UIGs are given in Table ITI. We describe each method follows:
PENS-SH. We choose PENS-SH Song et all (212% SOTA PENS-based synthetic data generator for personalized summarization, as comparative baseline w.r.t user-encoder accuracy boost. PENS-SH merges multiple (say, m) seed UIG trajectories {-mm) from the PENS train dataset (TiP. into a single synthetic fi.m
=Bim trajectory TP-SH such that all the common d-nodes on Tp-SH are unique to it, thereby forming the pool TSS train
We analyze the diversity of the PENS-SH trajectories in PSH when injected with s-nodes from the PENS test dataset, denoted PSH as UIG PENS-SH, following Section
S3-Aug. We choose S3 Doggment-Shuffl-SSiiie Groyer et all (2024) modular neural intra-trajectory augmentation mechanism designed for sequential data, a comparative baseline. S3 restructures user interaction sequences 7" by dividing them into non-overlapping segments, followed by differentiable
29

---

Published in Transactions on Machine Learning Research (10/2025)
Characteristic
Dimension
Value
Dataset Overview
# Reddit Posts
123,169
# Subreddits (Domains)
29
General Stats
Policy-Generated Summaries
115,579
Human- Written Summaries
Available
Train + Validation Dataset Statistics
#
Reddit Posts
21,111
# Policies
81
Article Stats
# Generated Summaries
107,866
# Annotators
76
# Summary-Par Pairs Rated
64,832
Validation Subset Statistics
# Reddit Posts
1,038
# Policies
13
Subset. Details
# Generated Summaries
7,713
# Annotators
32
Test Dataset (RLHF-Tunde Policies) Statistics
# Evaluated Policies
4 57 (out of 1,038)
Evaluation Stats
# Evaluated Reddit Posts
Evaluation Method
Indirect Benchmarking
Annotation and Feedback
Rating Scale
17
Confidence Scale
19
Feedback Collection
Avg. Ratings per Annotator
1,176
Annotation Format
Summary-Pan Pairs Selection
Table OpenAI TL;DR (Reddit) dataset statistics:
The dataset includes 123,169 Reddit posts across 29 subreddits, with policy-generated and human-written summaries. Evaluation involves summary-pall ratings and RLHF-tuned policy benchmarking.
shuffling and stitching operation TP to yield augmented trajectories Ts f1.m that preserve local coherence while introducing temporal perturbations. The resulting S3-augmented trajectories T used to train user-encoders, and we evaluate its diversity after incorporating s-nodes from PENS testbed (denoting the UIG of it as S3) and further effect on user encoders as compared to PerAugy.
SDAInter. We include SDATnter aiao et all (2024) as cross-trjectory augmentation baseline that gen- erates pseudo user sequences by identifying interchangeable subsequences in TeP between different user trajectories based on shared anchor items. If the subsequences between two users meet a minimum IoU- based interchangeability confidence C Te, they are swapped to create new synthetic trajectories TSDA The resulting SDAInter-auugmetted pool 7,SDAA evaluated for its effect on user-encoder performance and compared against PerAugy, along with evaluation of 7aya-SDA (synthetic UIG version of SDAInter augmented trajectories by incorporating intermediate s-nodes) for DegreeD.
We also compare method against three popular LLMs Llama-2- 2-13B Tonvonn all (2023), -inttral-22-nsstrett Jiang et all (2122), and DeepSeek-7b-chat DeepSeek-AL Al et all (2125) in two prompt-based settings- 1) Chain-of- Thoughts Wei of all (2028) which involves guiding an LLM through a series of logical reasoning steps to solve a lask Wei el all (2155) Using LLaMA-2-13B as the base model, we design a CoT prompt with detailed step-by-stey instructions to logically generate a personalized summary basod on a users interaction history (see Figure H), and 2) Prompt-Chaining Sahoo of all (2024) which is a technique that involves using a series of prompts to deconstruct a task into sequential steps. Our prompt- chaining setup consists of two sequential tasks. In the first. task (step-1), the LLM generates user interactions
LLM-as-Augmentoo
30

---

Published in Transactions on Machine Learning Research (10/2025)
in the form of (document, action) pairs, simulating user behavior (c.g., clicks, skips). In step-2, using few-shot prompting, the LLM generates personalized summary, conditioned on both the input document and the user interactions generated in step-1. Each few-shot prompt contains four in-context examples (see Figure m.). These generated trajectories are then subsequently used to train user-encoder models (as described in
Section 12)
C.2 Baseline User-Encoders
In this section We discuss SOTA news headline recommendation models which were used as baseline user- encoders to understand the effect of PerAugy generated training data.
NAML. Neural News Recommendation with Attentive Multi- View Learning (NAML) Wu et all (ZuNA) is a neural news recommendation approach that Iearns informative representations of users and news by exploiting different kinds news information. The core of this approach is a user-encoder and a news encoder where the news encoder learns unified news representations from titles, bodies, and topic categories by regarding them as different perspectives of the news, through an attentive multi-view learning model, and the user-encoder learns the representations of users based on their browsing history and applies attention mechanism to select informative news for user representation learning.
NRMS. Neural News Recommendation with Multi-Head Self-Attention (NRMS) Wi el all (20199) is a neural news recommendation where the core of the encoder is a news encoder that uses multi-head self- attention to learn news representations from news titles by modeling the interactions between words and a user-encoder which learns user representations from their browsed news and use multi-head self-attention capture relatedness between the news. Additive attention to learn more informative news and user representations is used, by selecting important words and news.
EBNR. Embedding based News Recommendation (EBNR) Okura et all (2017) is an RNN-styled news recommendation approach that incorporates implicit negative user feedback by distinguishing positive and negative news clicks based on the reading dwell time of the news by the user, and learning the user represen- tations from positive and negative samples via a combination of Transformer and additive attention network. It computes final click score as a combination of positive click scores and negative click scores.
TrRMIo. Transfomer- based Recommendation Model Song et al. (2)X utilizes a personalized news recom- mendation model to represent users' preferences derived from clicked records. The pre-trained transformer models are used for both recommendation and headline generation tasks, where a news encoder and a user- encoder is adopted for content- based recommendation. The textual information from the news encoder is aggregated via Attention Pooling, which then further integrated by user representation. The user interest. is defined on the basis of Click Through Rate (CTR) by examining the frequency of news articles in users' click histories and positive samples. The assumption is that user history consists of both popular news and interested news, where popular news has higher CTR ranking across the users, while interested news has lower CTR ranks, depicting the personalized choice of that user lo generale the user representation. This laid the foundation for Transformer- basedd Recommendation Model Interest- Interest-Only(TiRee Only(Tr)Mloo), where news histories with lower CTR for a particular user is trcatod as his "Interest-Only" fcatures.
C.3 Baseline Personalized Summarizers
To determine whether BerAugy-generale training data enhances the regularization of specialized Pretrained Language Models, improving PerSEval performance in personalized sunmarization tasks, we benchmark the PENS personalized summarization framework Ao aLl (20z41) and the recent GTP personalized summariza- tion framework Song al. (2023). We describe cach of the frameworks as follows:
PENS.
The PENS framework employs a transformer-based encoder to process the news body and pointer networkbased decoder to generate headlines. The pointer mechanism used to dynamically choose between generating words from a vocabulary and copying words directly from the news text, which helps in handling
31

---

Published in Transactions on Machine Learning Research (10/2025)
u-nodds (trajectories)
# d-nodes per trajectory
s-nodes per trajectory
Average trajectory length
Maximum trajectory length
UIG
PENS
360K
83.78
1.91
85,72
2433
PENS-SH)
197K
310.56
1.01
321.58
5106
S3F
300K+
79.4
2.2
83.2
2167
SDAInterf
360K+
87.6
2.8
123.6
1742
LLaMA-2-13Bf
2176
29
13.6
17
Mistral-78t
31.25
58
DeepSeck-R1f
813
37
4.23
35
OpenAI (Reddit) (OAI)
126K
25.19
4.82
30.02
51
PerAugyf=
360K
123.7
5.1
129.8
200
PerAugy-OAI=
3
36.92
11.44
48.37
50
Table 10: User-interaction graph statistics: Two seed datasets chosen- PENS train dataset (Table B and
OpenAl (Reddit) train dataset (Table B); Baseline augmentation methods (i) PENS-syuthetic-base (ours;
as *), (ii) PENS-SH, (iii) LLaMA-2-16 2-1BB,, (iv) Mistral-7B, (v) DeepSeek-R1, and (vi) PerAugy-PENs/OAI (ours); faugmentation followed by UIG abstraction on the PENS dataset.
out-of-vocabulary words and maintaining factual consistency. To personalize the headline generator, three distinct injoction strategies for incorporating user embeddings (learned from user behavior data using state- of-the-url news recommendation models as user-encoders) is proposed: (i) Decoder Initialization where the user embedding is used to initialize the decoders hidden state, so the generation process conditioned from the very start the user's interests, (ii) Attention Perturbation where the user embodding is injected into the attention mechanism. This modulates the attention distribution over the news body words, effectively guiding the model to focus on parts of the text that align with the users preferences, and (iii) Generation-Copy Switch Adjustment where the user embedding is also used to perturb the probability (or switch) that determines whether the decoder generates a word from the vocabulary or copics a word from the news body. This helps ensure that the generated headline reflects personalized nuances rather than just. summarizing the article content.
GTP. General Then Personal (GTP) is framework that tackles personalized headline generation by decoupling the task into two sequential stages. In stage 1, Transformerbased encoderdecoder model (e.g., BART) is pre-trained on large-scale news articlchcadline pairs to learn robust, content-focused hcadline generation without any personalization. In stage 2, separale headline eustomizer takes the general headline and refines by incorporating user-specific preferences. These preferences are encoded (control code) by the user-encoder TrRMIo. bridge the gap between the general generation and personalized refinement, the authors introduce two key mechanisms: (i) Information Self-Boosting (ISB) that enhances the customization by reintroducing relevant content details from the news article ensure that personalization does not lead to information loss, and (ii) Masked User Modeling (MUM) that helps the model learn to recognize and utilize the user control code by randomly masking parts of the user embedding during training and then reconstructing them, thereby reducing over-reliance on the general model parameters.
D Encoder/Decodee Accuracy Metrics
In this section, we provide a detailed formulation of the user-encoder accuracy metrics in the context of the next d-node prediction task.
AUC. The Area Under the Curve (AUC) measures the probability that randomly chosen positive d-node is ranked higher than randomly chosen negative d-node in the test dataset T.P The formula given as:
AUC
Σ Σ
1(sp sn)
(14)
PI- IN
PEPREN
where:
. is is the set of positive interactions (set of clicked d-nodes of all users in test data).
32

---

Published in Transactions on Machine Learning Research (10/2025)
N is the set of negative items (set of skipped d-nodes of all users in test data).
Sp is the predicted score for positive d-node.
Sn is the predicted score for negative d-node.
(sp > 8n) is an indicator function that equals 1 if 8p > 8n, otherwise 0.
|P] and N] are the number of positive and negative d-nodes, respectively.
MRR. Mean Reciprocal Rank (MRR) evaluates how early the ground-truth target d-node appears in the ranking. It is defined as:
MRR 1
(15)
EU ranku,
where:
U is the set of users.
rank, the position of the first relevant d-node for user uj in the ranked recommendation list.
|U| is the total number of users.
A higher MRR indicates that the target d-node is ranked closer to the top of the prediction list, improving user experience.
nDCG@k. Normalized Discounted Cumulative Gain at rank & (uDCG@k) evaluates the ranking quality by considering both the prediction score and the position of ground-tree truth target d-node. It defined as:
DCG@k
nDCG@k IDCG@L where: DCG
IDCG@K-
(16)
log, (i 1)
log 1)
Here:
8 is the prediction score the target d-node at rank in the rocommonded list.
s the actual score of the target d-node at rank in the idcal ranking (sorted by prediction scorc).
DCG@k the Discounted Cumulative Gain up to rank
IDCG@k is the Ideal Discounted Cumulative Gain, representing the best possible ranking.
A higher nDCG@k indicates that the target d-node is ranked higher, improving prediction effectiveness.
E Algorithms
In this section, we discuss the details of the algorithms used in our paper.
UIG Construction We construct the User Interaction Graph (UIG) by parsing interaction logs from two lypes of seed datasets: (i) PENS-styled, where explicit click and skip behaviors are available, and (ii) OpenA(((Reddit)-styyleld where user preferences are inferred from model confidence and summary quality ratings. The algorithm maps user interactions to document. (d-node) and summary (s-node) nodes, while assigning appropriate behavioral edges such click, skip, gensum, and sungen to encode both explicit and inferred preferences.
33

---

Published in Transactions on Machine Learning Research (10/2025)
Algorithm UIG Construction
Require: train data and test data, dataset_type Initialize T + 0
for each user u in train data do Initialize +0+
for each interaction u's data do if dataset type PENS then
Map interaction to d-node with click/skip edge else if datasel lype OPENAI then
if any model-generated summary for d-node has confidence score > threshold then
Label as click
Select best-rated summary by u as surrogate s-node
Map to d-node and s-node with gensum and sumgen edges else
Label skip end if
else
Map rating to d-node with click/skip edge if rating is max then
Map to d-node and s-node with gensum and summGen edges end if end if
Append d-node to r" end for Add 7" to T end for dataset type = PENS then for each r" in do
if d E train data AND test data then
Insert (d-s)-nodes from test data genSumm summmen edges end if end for return 7PENSDD t T else
return T end =0
In section Mal we described special case where for every d-node in an UIG we have corresponding s-node in every trajectory т, which evidently unrealistic. In reality, many of the d-nodes will not have corresponding s-node and hence, the calculation of DePS cannot be done every unit time-interval ( This requires modification in the computing procedure
DegreeD Computation UIGs
to account for the missing s-nodes. To address this, the first "surrogate" s-node sfth) for the initial d-node d(ti) at time-step & is assumed to be the same as the document's title, as there is no prior preference history of a user at time-step t and hence, subjectivity as a function of preference history does not arise yet. Let the first s-node (ta) occur at time-step la (i.e., the first valid interval is A(tr))))- Therefore, DePsA for j-th user is calculated as:
8y
DePS, Ag1t)) mān(8 X (en 8[x A(r)))) 1,,) +
max(5 x Acr di 8 8[x Acc lo,) +c²
(17)
where: 8[xAnmella = b - iml
(dlti) alto+e)); (dltt),(ll++0))). o|x x[xA ((xAe)))))) ((s)(a),800
34

---

Published in Transactions on Machine Learning Research (10/2025)
Algorithm Computing DegreeD
1: Input: Users U, Actions A, Summaries S, Documents D, window size w
2: for each trajectory (U, A, D, S) do
3: Diotal = 0, Dasst = [, DMA = =
4: for 1 = to |A| - do
Retrieve Dr,,U, if AG is click/skip then
Ddist o (Di, : update DMA else if Ar, gen summ then
8:
min( min(Dr,,DDa) Deg)+e
D22 = DMA, 8
max(Di, D1g)
,(D))
10:
p= oDi 1te
11:
Diotal = 8. OUU,,
12: end if
13: end for
14: D+= Diotall /|A|| - 1)
15: end for
16: + D/ =0
DegreeD is then computed over all valid intervals as per equation In this paper, we represent d-nodes and s-nodes with their embeddings generated from lightweight S-BERT model Reimers Gurevych (201 and use Manhattan Distance the distance metric o. It is important to nole that DegreeD is fundamentally defined as ratio of relative variations in distances between d- and s-nodes within the same interval (Equa- tion IT). This formulation ensures that the metric only depends on how well s-nodes track the semantic drift of d-nodes, rather than on the absolute scale of any embedding space or distance function. In other words, any embedding model (e-g:: SBERT, BERT, or domain-specific encoders) and any valid distance metric a (e.g., Manhattan, cosine, Euclidean) merely provide a representation space in which distances are computed, but the ratio-based structure of DegreeD cancels out biases due to embodding geometry metric scaling. Hence, DegreeD remains model- and metric-agnostic measure of diversity, relying only the relative aligmment, of document w.r.t. summary dynamics. In section R. we empirically provide strong ev- idence that higher UIG DegreeD has strong correlation with user-encoder model accuracy when trained on such UIGs (DegreeD computation is Algorithm Ø). We compute the DegreeD of the PENS synthetic base pool Teye- and find very low DegreeD score of 0.009. The OpenAI (Reddit) synthetic base pool Syn-OAI also shows low DegreeD of 0.0079.
base
PerAugy: Double Shuffling - Algorithm Details The PerAugy framework enhances user interaction gen- eralization by introducing two complementary augmentation strategies: Double Shuffling (DS) and Stochastic Markovian Perturbation (SMP). In DS (Algorithm B), target user trajectorics are systematically altered by substituting randomly selected segments from other users interaction histories. These substitutions occur at randomized offscts and are spacod by controlled gap lengths to maintain temporal realism and simulate cross- user behavioral blending. This process generates diversified yet structurally plausible trajectories, expanding the training distribution without deviating from feasible user behavior patterns.
PerAugy: Stochastic Markovian Perturbation Algorithm Details Following DS, the SMP stage (Algorithm H) further refines the augmented trajectories by focusing semantic consistency the summary level. Specifically, newly introduced summaries evaluated within local Markovian window of recent doc- ument interactions, and replaced with top-ranked candidates based on relevance score computed via RMSD (Root Mean Square Distance) similarity. These candidates are weighted by an exponential temporal decay factor to prioritize more recent contextual nodes, ensuring that substituted summaries align with short-term user interest profiles. Together, DS and SMP act synergy produce coherent, high-quality augmented data that preserves personalization cues while introducing controlled variability crucial property for training robust and generalizable user models in recommendation and summarization tasks.
35

---

Published in Transactions on Machine Learning Research (10/2025)
Algorithm Double Shuffling (DS)
Require: UIG trajectory pool base sample size mand gap-length gi
Ensure:
Modified trajectory set TDS
T
sampic
hase m)
Ds f-
My for each target trajectory Ttarget T
sample do
4: + RandomOfisst((
5: Tuubs 0
6:
TM, for each source trajectory Touuree
Teample> where i j do
Mi seg steps.}
f- RaRndomSggmentt (T souren) {Select trajectory segment of random length at random time-
3 ttarget
Substitute (aargett> 734 subb))
subs + O lenglh(T Teeg) + gr (Determine substitution indices target: ensuring that two source segments are separated by gap-length g..}
10: end for
11: DDS f- Trarget
12: DS + DS U {
13: end for
14: return TBS =0
Algorithm Stochastic Markovian Perturbation (SMP)
Require: DS trajectories TDS, window k, decay X, top-p Ensure: perturbed set TMMP
1: TSMP - /
2: for each T T Tps do
3: for each step in r do
4: if g(t) newly substituted then
Retrieve d(t-1), extract {st}; define window {c} for each st ((11) do
[(st) + E. RMSD(st, c) 8: end for
- pos(()
Rank {st}, pick top-p 8(t), replace (t)
10: end if
11: end for
12: TSMP TSMP U {r}
13: end for
14: return TSMP =0
F Stability of DegreeD Correlation Under Divergence Substitution
F.1
Setup
Let us assume a dataset with users U. For each user with trajectory length Ljs define from
L
50 = ((s(())))))))
8( o(d(t)), A
Σ
80)
L -
im
Let the timestep-level ratio be defined (())(1) 5()) (1)/5/) (1), with
DepS = ((r(i))))) )(r) = min(r, 1) + +e
max(r, 1) +
36

---

Published in Transactions on Machine Learning Research (10/2025)
At the same time, the penalty term is
a(d(tt).s,tti) +c (1) = o(d(t(+1) sy(ti+1)) +e¹
and the expected penalized DePS
1--1 (e) (i).
E E,[DPPS]
Li
Finally,
DegreeD,
Σ
A
E, Ey[DePS"].
|U|
jEU
For datascts (D&) let F6- (Dgr G DegreeD (Dk), and Aş for accuracy.
Metric Substitution. Let of be the substitute metric of a.
Assuming o' is Bi-Lipschitz equivalent to a: 0 < X S A < s.t Ao y) < < v) s Ao(z,)), (18)
for all pairs used in DegreeD, with A/A. For pure scalings o' = co, X A =
F.2 Post-Substitution Time-stepwise Bounds Lemma 1 (Local distortions). For all i,J,
X < 6 < SAA,, -0()s s 6(') s no(r), (00 s ( (1).
Proposition 1 (User-level Bounding Inequalities). Let H3 = =ak)
E, DePS" Then, using Lemma D.
2 < H
3
H
< x
H
A
Corollary 1 (Dataset-level Bounding Inequalities).
Aggregaling over users, we oblain
A3
A
K_Fx S Gs S K4FK.
K_
K_
A
X2
Corollary (Pure scaling). o' co, then G5 = cF and all corvelations with A are unchanged.
F.3 Post-Substitution Rank Stability
Let us define an ambiguity band that captures the zone of uncertainty where relative rankings between two dataset DegreeD-scores Fş and Gg may flip under divergence substitution. From Corollary for each datasct & we can conclude that for any datasct pair k,
Gk
K- FR F
K, K_ Ft
Pa
E
G2
KK
II t lies outside - k=], then the order of Fg and Fe is preserved for all possible distortions, guaranteeing rank stability. Conversely, if the ratio falls inside this interval, an inversion is possible: one dataset-Degreed could be stretched to its upper bound while the other shrinks to its lower bound. Substituting the explicit forms of K2 gives:
K K,
(2)
K A -
K_
Let the uncertainty interval be defined Ambiguity Band B [(2/A)3, (A/A)%).
37

---

Published in Transactions on Machine Learning Research (10/2025)
Proposition (Substitution Stability of DegreeD). Fa/Ft Fe & B for all l + L, then F and G have identical makings; hence Ps (F, G) = (F,
While Proposition ensures exact rank preservation whenever all datasct diversity (w.r.t. DegreeD) ratios Fs/ Fr lie outside the ambiguity band B, in practice some pairs may fall inside B, permitting local inversions. We record here two principled ways to quantify partial rank stability.
Exact identity. For any two rankings of m datasets with rank differences {d) Spearman's correlation satisfics
Ps (F = -1- 6 6 df m(m² 1)
(19)
Thus the precise degradation of Ps depends on the displacement profile {d), not only on the number of inversions.
Adjacency-limited inversions. If all inversions correspond adjacent swaps, then each inversion con- tributes most to Σ df, implying
12K
(. 2 1 m(m=-11
(20)
where K is the number of inversions. This yields computable lower bound under localized perturbations.
General inversions. Without further assumptions, no nontrivial bound in terms of K alone is possible:
moving a single element down by places creates K = f inversions but contributes d2 = f2 + t, which may degrade Pa substantially more. Hence, robust guarantecs require either (a) bounding the displacement profile directly from data, or (b) assuming structural restrictions (e.g. inversions are local).
F.4 Pearson Correlations
Proposition 3 (Correlation transfer). Let TFA = corT(F,A), TFG corT(F, G), rGA corr(G, A). Then
TGA TEGTPA / 11- rro V r
Lemma 2 (Lower bound on TFG). From Corollary D.
TFG 2 VK-/K+ (A/A)25.
F.5
Summary Results
Theorem 1 (Correlation stability of DEGREED). Under the bi-Lipschitz equivalence II
the following
hold:
Rank stability (from Proposition B): If all F avoid B, then (FF.G))=((FCC)
Pearson transfer (from Proposition & & Lemma For any external accuracy variable A,
corr(G, A) 2 Ki corr(F,F (F, V Ko V1 (orr(F.A)]: with Ko = (A/A)25.
Scaling invariance (from Corollary 2): If A, then G₁ cFs and all correlations are preserved exaclly.
38

---

Published in Transactions on Machine Learning Research (10/2025)
F.6
Results for RMSD
Euclidean vs. RMSD: || - vll = Vd (z.y). y). This case guarantees pure scaling and hence, correlations remain exactly unchanged (Corollary 2).
Cosine distance: cos us__ist((z,)) v) = RMSD* (z, (). Here o' is Lipschitz equivalent to provided RMSD is bounded away from zero; Theorem B applies.
Angular distance: Monotone in cosine similarity, hence Lipschitz related to RMSD in bounded domains. Rank stability (Proposition ( guarantees identical orderings outside
Mahalanobis distance: ||
yll is bi-Lipschitz to Euclidean with constants V (M) and Amax(M) (M), all conclusions of Theorem apply with Vdmax
(M)aaaa (M)/Xmin(M)
By chaining Lemma M. Proposition Corollary M. we established the main squeeze bound for DegreeD under divergence substitution. Proposition @ shows that if all dataset-Degreed ralios Fa/ avoid the ambiguity band B, then rankings are preserved exactly. In Section DM we extend this partial perturbations where we show the condition for exact identity w.r.t rank displacements in Spearman correlation. This clarifies how DegreeD rankings degrade in controlled ways when some pairs fall inside B.
G Implementation Details
Computing Resources. The creation of User Interaction Graphs (UIGs) and computation of DegreeD are performed on a standard 4-core CPU with 16GB of RAM. For Stochastic Markovian Perturbation (SMP), use the SBERT alI-MiniI.M-L6--22 model (Reimers & Gurevychi, 200 to generate embeddings, and the NVIDIA A-100 GPU. LLM-based experiments
SMP process takes approximately 16 hours to complete are conducted using 3 NVIDIA A-100 GPUs.
PerAugy Settings. We generate the embeddings of the d-nodes and s-nodes using the SBERT Reimers & Gurevych (201g) M-1--- model', which has 22.7M parameters and embedding size of 384. Manhattan distance is used to compute embedding divergence during the DegreeD calculation, chosen for its linear scalability and efficiency. For SMP, embeddings of sentences and d-nodes within the context window are generated using the same SBERT model, followed by the use of RMSD to compute similarity scores for perturbation.
Model Settings. Three user encoders (NAMI. (Wi ef 2019a), EBNR (Okura ef all 2017), NRMS ctlll, 2011)) are trained on PerAugy datasets for 2 epochs, with a learning rate of 0.0001 and batch size 128 using the Adam optimizer. The models are finetuned on the TDS/DS4SMP datasets the TrRMIo model for one epoch after training from scratch. During training, intermediate s-nodes are modeled as d-nodes integrate them into the encoders.
LLM Settings for Prompts. Prompting experiments are conducted using two setups: (1) Chain-of- Thoughts with LLaMa2-13B and (2) Prompt-Chaining with Mistral-Instruct-22 and DeepSeek-7B-Chatt For LLaMa2-13B we perform inference using sampling with lemperature set to 0.75, lop-p to 0.9, and top-k to use deterministic sampling strategy (temperature
50. For Mistral-Instruct- and DeepSeek- 0.0, top-p = 1.0) for controlled generations. Max__okonn tokens are set to 1024 for both setups.
Correlation Computation
To quantify the relationship between encoder accuracies and diversity metrics, as well as the dependdeeed of diversity metrics thenselves, we employ three standard correlation measures: Pearsons correlation coef- ficient, Spearmans rank correlation coefficient, and Kendalls 7 coefficient. Their formulations are provided below.
39

---

Published in Transactions on Machine Learning Research (10/2025)
Formulations. Given paired observations ((r,, y:) compute:
Pearson Correlation:
Σラ (x,, - x) y)
(21)
(3; 872 2i= (yr 7)
where and the sample means.
2. Spearman Rank Correlation:
6 6 d
- where d, the difference between the ranks of xy and ye-
(22)
n(n² 1)
Kendalls 7:
C- 2 nn -))
(23)
where C and D denote the number of concordant and discordant pairs, respectively.
Accuracy vs. Diversity Metrics. Let A = {a,}ー<1 1 denote the set of averaged encoder accuracies across all encoders for dataset j, and let D* {dt M M Jj=1 denote the set of diversity scores corresponding to the k-th diversity metric, where k E (PENS, PENS-SH, S3, SDAInter, PerAugy-DS, PerAugy-DS++SMP) For each k, compute the correlation with accuracy
Corr (A, = (((g),))))) dD))
(24)
where f E PTT corresponds to Pearson, Spearman, or Kendalls correlation, respectively.
Note that LLM-generated trajectories are excluded from D& due to their inability to produce complete trajectory sets with 113K news items, resulting in artificially lower accuracies despite exhibiting topical diversity and frequent shifts.
Inter- Corrrelttoo of Diversity Metrics. Let (TP, RTC, DegreeD] denote the set of inter-diversity metrics. For each dataset j, we obtain the metric values D, {d]t JmeM. For any pair (m1, E M x M, we compute
(frr!f((D)) Dms) = f (H(d"", (
(25)
where f E (r.p.T).
This formulation allows us to construct correlation matrix across M, thereby quantifying the degree of alignment or divergence among different internal diversity measures (including those derived from LLMs).
Detailed Results
Ablation Studies (RQ-1)
We ablate on the mixed training data TEE to analyze the effect of DS seper-parameters- gap-length gi (section [) and train history-sogment length Theran: (1/2, 51/8, 3//4, 17/8,1-3) (I: trajectory length). For TE-P SMP hyper-parameters (k: {10, 15, 20), A: (0.3, 0.8, PSMPs (0.5, 0.8, 1)), ablate DS+SMP- Results are in Figure F.
Efffec of Thtrain & We fix to 25 and observe that Thamin has a major impact across all
user-encoders with the longest Theaa (1 - 3) having the highest mean boost (0.064 t w..t AUC, 0.035 t w.r.t MRR, 0.011 1 w.r.t nDCG@5/10) against the least scores, thereby confirming that longer preference history in train is better. 9i (1 fixed at 150) also matters particularly w.rt AUC with best at 40. This shows that synthetic profiles having longer original user segments are better.
40

---

Published in Transactions on Machine Learning Research (10/2025)
AUC
MRR
NDCG
X-axis: Hyperparameter Values
Y-exis: Metric Scores
EBNR
NRMS
DS Gap
0.7
10
15
20
20
25
40
10
15
20
25
10
15
40
DS Hist
0.8
0.7-
02
5/811
3/41
7/8"1
1-3
0/2
5/0
3/4"1
7/0"l
1-3
1/2
5/871
304'1
7/8"1
1-3
1/2
5/8'1
3/471
718"1
SMP Context
0.9
0.8
10
15
20
10
15
20
10
15
20
10
15
20
0.9
SMP Decay
0.8
0.5
0.3
0.8
1.0
0.3
0.8
1.0
0.3
0.8
1.0
0.5
0.8
1.0
0.9
SMP Prob
0.5
0.5
1.0
0.5
1.0
Figure 6: Effect of PerAugy hypar-parameters on Usor-Encodor Accuracy: All encoder models are trained-from scrocccr results summarized in Table III Observation-1: Best hyper-parometer values perform consistently across models; Observation-2: For DS, gi and Th = favor longer retention; Observation-3: For SMP, 10, РSMP 0.8, 0.3 control abrupt diffusion best, and non- non-Markovion smoothing is preferred.
Effect of it A, & РSMp We observe that PSMP has the maximum impact (fixing k 1 0.5), particu- larly for ranking metrics (MRR, mDCG@5/10). We find that РsMP = 0.8/1 have highest boost (0.04 w.r.t nDCG@10). This shows that SMP smoothing is mostly required during augmentation. We also observe that the length of the context window (T = 0.5; = 0.8) also has a significant effect on the overall With the best & and PSMP (0.8), we
AUC (0.05 t) and nDCG@5/10 (0.032 t) with the best 1: find the best À be 0.3, particularly for MRR (0.047 t) and nDCG@5 (0.032 t). This shows that (a) long context window is not useful for SMP smoothing and (b) smoothing cannot be strictly Markovian.
41

---

Published in Transactions on Machine Learning Research (10/2025)
AUC
MRR
nDCG@5
nDCG@10
Hyper-parametar
RWM
AWM
RWM
AWM
RWM
AWM
RWM
AWM
gi
0.018
0.043
0.005
0.021
0.007
0.023
0.007
0.018
Therain
0.024
0.064
0.013
0.035
0.011
0.026
0.011
0.028
k
0.028
0.049
0.016
0.027
0.016
0.032
0.016
0.032
A
0.019
0.034
0.023
0.047
0.01
0.029
0.01
0.029
PSMP
0.019
0.04
0.018
0.039
0.023
0.035
0.018
0.04
Table 11: Comparative impact of hypeer-aaaameees.. Metries shown are Relative Win Margin (RWM) and Absolute Win Margin (AWM). Observation: Shorter gap-length leads to consistent wins across encoders
w.r.t AUC, but for prediction ranking, higher perturbation probobility and contert- contert-window length matter more.
0.35
0.30
Diaree Score
0.20
0.15
0.05
0.00
300
20.0
12.5
13.0
23.0
0.6
1.0
Values
Vallues
Values
Values
Values
Figure 7: Ablation effect of hyper-parameters on DegreeD: of PerAugy; Observation-1: lower gap length increases diversity due to diffusion into different topics across a trajectory; Observation-2: Longer contert window may not lead to sufficient perturbation; Observation- 3: stricter Markovian does not yield higher diversity; Observation 4: frequent SMP on s-nodes lead to higher diversily.
Diversity analysis for all hyper-parameters
1.2 DegreeD Ablations
We ablate various hyperparameters of PerAugy, including gap length gi and trajectory length for TDS as well as context length k, decay constant X, and perturbation probability PSMP for TDS+SMP. A summary of our findings is given in Figure M.
Gap Length 9I and Trajectory Length I. Smaller values of gi generally lead to higher DegreeD, with 9t = 10 yielding the best results (DegreeD of 0.163). This suggests that frequent substilutions in 'source' segments boost thematic divergence. Similarly, increasing results in higher DegreeD, with the highest score (0.158) observed at 175. This indicates that the length of 'source' segments plays a crucial role in promoting diversity.
SMP Parameters. We observe that smaller values of the context window and the decay constant X lead to higher DegreeD, while higher PSMP improves DegreeD, with the optimal setup being k = 10, X 0.3, and PSMP 1 (with a score of 0.278). This suggests that, while there Markovian effect (as lower k results in higher diversity), the role of higher-order influence should not be overlooked. In other words, user- generated subjective summaries are not solely governed by Markovian process (as evident from the fact that lower À corresponds to higher DegreeD), and might have long-term dependencies. We conclude that user behavior exhibits a tendency toward diffusion (random or exploratory variation in a users reading behavior); however, an abrupt diffusion does not necessarily lead to higher diversity. This underscores the importance of SMP as smoothing mechanism regulate diffusion. The comparative analysis of the ablations on different. hyperparameters w.r.t DegreeD are in Table II
Comparative Impact of Hyperparametors. We conduct detailed ablation to understand the influ- ence of each hyperparameter of PerAugy on user-encoder model performance across AUC, MRR, nDCG@5 & 10 using Relative Win Maryin (the difference between the best and second-best. performance for a hy-
42

---

Published in Transactions on Machine Learning Research (10/2025)
NAML
Ooll KMN PerGogy
Deli wih Pertugy
0.8
0.7
O-Ji win Pentegy
O-di wis Peróurg D-A
0.7-
NDCO83
NDCO
Figure 8: User-encoder performance (OpenAI (Reddit)): Impact of Double Shuffling (DS) and DS+ on OpenAT seed base TOAI (SMP hyper-parameters: k=10, x=0.3, PsMp=0.8). TrRMIo fine- tuned; others are trained from scratch. Observation-1: Vanilla OpenAl lags behind DS and DS+SMP, indicaling that vandom ougmentalion is less effeclive than PerAugy; Observation-2: DS achieves perfor- mance boosts as in PerAugy-PENS; Observation-3: DS+ further boosts performance, demonstrating the cross-domain strength of PerAugy.
perparameter) and Absolule Win Margin (difference between best and worst). History Length exhibits the strongest effect on AUC with an Absolute Win Margin of 0.064 and Relative Win Margin of 0.024, in- dicating that longer historical context is crucial for general user modeling. Context-Length provides the highest Relative Win Margin of 0.028 for AUC and consistent gains across ranking metrics, showing the importance of larger contect windows. Decay Constant achieves the highest MRR Absolute Win Margin of 0.047 and strong Relative Win Margin 0.023, highlighting the impact of temporal recency weighting. Porturbation Probability PSMP leads in ranking metrics, with nDCG@5 Relative-0.023, A Absolute=0.083 and nDCG@10 Absolute=0.04. suggesting higher perturbalion improves lop-k relevance. Finally, Gap Length con- tributes stable gains to AUC (Abs.=0.043), indicating that shorter gaps between actions lead to consistent encoder performance. Overall, each hyperparameter uniquely benefits different objectives, and careful tuning is vital for optimal performance. The detailed results are in Table [T].
J Comparative Study
In this section, we provide comparative analysis of SOTA relatable data augmentation techniques based on the operations they perform on trajectory-like datascts. In torms of operations, the methods are divided into two main calegories: Intra-Trajectory Augmentation and Cross-Tras Trajs-rraectory Augmentation, serving the purpose of sequential recommendation tasks.
J.1 Intra- Trajectory Augmentation
S3 Grover et all (2024): Segment-Shufflo-an---iin is an intra-trajoctory augmentation where non- overlapping segments within same trajectory sequences are segmented, shuffled and finally concatenated (stitched) to form optimal trajectory. Since our goal is to generate diverse synthetic trajectories,
43

---

Published in Transactions on Machine Learning Research (10/2025)
shuffling among the segments within same sequence does not guarantee smooth thematic transition of nodes w.r.t the historical interactions. Also, in our case, Shuffling does not enhance diversity w.r.t. DegreeD.
MBASR Xiao et all (2024a) Multi-behavior Augmentation for Sequential Recommendation method employs an intra-trajectory augmentation technique by performing pairwise swapping of segments to gener- ale diversity. However, in our case, the base historical dataset has highly monotonous trajectories. Therefore swapping nearby subsequences fails to produce sufficient diversity, and additional operations like order per- turbation or redundancy reduction do not effectively smoothen the s-node content in line with historical preferences and might inject (or remove) unrealistic time-step information thereby disrupting the flow of the trajectory. For these reasons, we do not adopt MBASR.
STEAM Lin ct all (2023): STEAM operates in an intra-trajectory manner by dociding whether to drop or insert nodes within a trajectory to create augmented data. However, the method is not scalable to longer trajectories, and the insertion or deletion of nodes can disrupt the historical sequence, ultimately undermining the realistic flow of the synthetic user profiles. Hence, we do not use STEAM.
Learning-lo-Augnennt is an intra-trajectory augmentation method where node is deleted from the sequence of users to generate sequence of synthetic casual through reinforcement learning- based policy mechanism. Node deletion is irrelevant in case as it can disrupt the sequential flow of the trajectories.
L2Aug Wang et al. (2022):
Bi-directional Transformer Basket Recommendation ion model incorporates masking strategies and swapping operation to train the model for 'Next Novel Basket Recommendation" Despite some similarity the purpose at broader level, our goal is not to create a podel/encoder that encodes the input sequence but generate diverse input sequence to make the existing encoders learn the representations.
BTBR Li al 20233:
J.2 Cross- Trajectory Augmentation
SDAinter Jiao et all (2024): SDAinter is cross-rrajectory technique that matches anchor items (e.g., identical start and end d-nodes or s-nodes) across trajectories to facilitate segment exchange. However, the reliance on anchor-based matching docs not effectively capture the subjective nuances of individual user interests, limiting its applicability in personalized summarization. For this reason, we do not consider SDAinter suitable for historical interaction sequence- besedd tasks.
Recommendation transformer-styled cross-trajectory sequence regeneration model where the pertaining task is constructed for to extract patterns from given set of sequences and feed the patterns to the model to regenerate other set of possible sequences. However, patterns in our case would mean reading and summarizing habits of two different users, where the s-nodes are subjective. Therefore, this technique might incorporate rodundant s-nodes, defcating the goal of personalized summarization.
DR4SR Yin et all (2024): :
Data- poreeeerraiion-o-SSeeeeeee
TiCoSeRee Dang et al. (2024)): Time Interval Aware Augmentation technique ensures uniform time- interval distribution in the sequence based on the time-aware traditional operations like Crop, Mask, Insert, Reorder and Substitute. However, trajectories are primarily assumed to be uniform in terms of time-steps (unit time between two successive interactions).
FDA Chen ct all (20287): Fairness-oriented Data Augmentation is used to generate synthetic user pro- files from the realistic profiles to balance between realistic data and pseudo pseudo data. However, modeling historical preference trajectories by generating fake interaction sequences will not lead to diversified trajec- torics as the 'complemented' sequence will also remain monotonous. Also, idcal datascts for personalizod summarization tasks must have intermediate summary nodes for supervised learning setup, which makes the generation of fake interactions challenging.
44

---

Published in Transactions on Machine Learning Research (10/2025)
Chain-of-Thoughts (COT) Prompt
You Al model generating synthetic interaction trajectories with articles.
Task Definition
Each user follows sequence of interactions news articles. The dataset consists of:
"UserlD": : Unique identifier for user.
"Sequence of Docs": Ordered list of news article IDs the user interacts with.
"Sequence of Actions": Ordered list of actions taken ( click, skip, gensum, sumgen). )
"Number of Summary Nodes": The count of summary nodes (e.g.. S-1, S-2) generated during the trajectory.
### Rules for Interaction Generation
Each User's Trajectory 100-200 Interactions Long
The interacts with sequence of news articles.
The sequence follows logical decision-maan based relevance and interest.
Action Types
""lick" User reads the article.
"skip" - User ignores the article.
"gonsum" User generates rewritten headline that document.
"sumgen" User written personalized headline.
Summary Node Constraints
"sumgen" must immediately follow "gensum".
Each "sumgen" introduces summary node S-(id) jin the decument sequence.
Each user must have to 50 summary nodesin their trajectory.
### Step-by-Step Thought Process
1. Assign UserlD Generate unique identifier for the user.
2. Generate Long Sequence of Interactions
Select 100-200 news articlesfrom various categories.
- Apply logical reasonng to assign "click", "skip" "gensum". , or "sumgen" actions.
Ensure Summary Nodes are Introduced Properly
When "gensum" occurs, assign it now summary node S-(id) )
The "sumgen" action must refer previously generatedsummary node,
Ensure there are at least summary nodes user.
4. Output the Structured Dataset "Userll>"
"Sequence Does": : Ordered anticle IDs and summary nodes,
Corresponding user actions. "Number of Summary Nodes"
"Sequence Actions"
### Expected Output Format (JSON)
"UserlD": "U801".
["N101", "N102", "N103", "S-1", "N104", "N105", "S.2", "S-1", "NI06"]
"Actions": ("elick", "skap". " "gensum",, "sungen", "click", "gensum", "sumgen", "sumgen", "click").
"Num Summary_ 3
Figure 9: Chain-of- Thougghts (CoT) prompt template used in LLM-bbsed experiments.
divSPA-styled methods Lin ef all (202%) These methods use a cross-Irajectory augmentation strat- egy by exchanging segments between trajectories based on similarity metrics. Despite this, the exchanged segments often lack sufficient variation with respect to the overall degree (DegreeD), resulting in minimal diversity gains. This limitation makes the approach less effective for our needs.
K Prompt Details
Chain-of-Thoughhe Chainofthought (CoT) prompting a powerful technique that guides large language models to decompose complex problems into series of intermediate reasoning steps before emitting their
45

---

Published in Transactions on Machine Learning Research (10/2025)
final answer. CoT prompting shown to significantly improve multistep arithmetic, commonsense, and symbolic reasoning tasks by eliciting explicit rationale chains that mirror human logic Ye el all (2024)).
Subsequent work in ACL demonstrated that CoT can be extended to address hallucination and faithfulness issucs by injecting structured knowledge during rationale generation (Wang ct all 2022).. EMNLP findings further confirmed that structured CoT variations, such as statebased prompting, yield substantial gains contentgrounded dialogue systems by promoting intermediate subtask decomposition Snitan of all (2024). Another EMNLP study introduced prompt tuning of masked language models to generate both intermediate and final reasoning steps jointly, striking balance between interpretability and performance without full finetuning (Kunnath et all, 2028). Across these efforts, consistent insight is that CoT acts as bridge, enabling LLMs to expose latent reasoning processes. The method is particularly effective for tasks demanding logical coherence and multihop inference, such as math word problems and question answering. Although CoT relies on large model scale to be effective, research shows that even generated exemplars (e.g. Lets think stop by step) can approximate fewshot behavior. In the context of our work, CoT prompting with LLaMA213B is used to craft personalized user summaries by breaking down interactions step by step, to enhance transparency and accuracy, making LLM reasoning more interpretable and reliable.
Prompt Chaining. Prompt chaining is effective prompling strategy where complex task is decom- posed into a sequence of smaller, well-defined prompts, with the output of one prompt becoming the input to the next, thereby guiding the model through a structured reasoning pipeline. It improves performance on multi-step tasks by reducing cognitive load on the model and increasing transparency at each stagedevelopers can verify and debug intermediate outputs, enhancing controllability and reliability. Academic research, such as shows prompt chaining excels in iterative summarization by orchestrating drafting, critiquing, and re- fining phases via discrete prompts, outperforming one-shot stepwisc alternatives (Sin et all, (iza)).Across these efforts, the core insight is that chaining leverages the models strengths at each subtask rather than relying on single-shot reasoning, leading to superior performance, especially when task complexity or in- put length is high. In our context, prompt chaining is implemented via two stepsuser behavior simulation followed by summary generationmirroring the validated pattern of decomposition for enhanced LLM task performance.
46

---

Published in Transactions on Machine Learning Research (10/2025)
Prompt Chaining (Task 1)
### Task
Generate sequence of interactions for User {user id).
Document IDs: &doc sequence)
Actions: [click, skip]
### Rules:
1) Each action corresponds document in the sequence.
2)
The sequence should only contain "click" or "skip" actions at this stage.
### Expected Output Format (JSON)
document 1 action I, document 2 action 2, document n, action
Prompt Chaining (Task 2)
### Examples
Below are examples of personalized headlines generated for different users based on their document content:
[fewshottcxamples]
### Task:
Generate a personalized headline based on the document content. Ensure that the headline aligns with the user's preferences and effectively captures the essence of the document.
User: {user id)
Document: (doc_id)
Document Content: (doc_conter))
### Expected Output Format (JSON)
Strictly return a JSON object in the following format:
"headline": "your generated headline"
Figure 10: Prompt-Chaining template used LLM-based experiments.
47


Reviews:
[{"review_id": "p7V9eJQ8Hz", "reviewer": "Reviewer_si1k", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper addresses the lack of diversity in user interaction datasets for personalized summarization. It argues that current datasets (e.g., PENS, OpenAI Reddit) are too small and fail to capture realistic shifts in user preferences, which limits the generalizability of user-encoder models. To tackle this, the authors propose DegreeD, a metric to quantify preference-shift diversity, and PerAugy, a two-step augmentation pipeline combining Double Shuffling (DS) to mix trajectory segments across users and Stochastic Markovian Perturbation (SMP) to smooth coherence issues in summaries. DegreeD is computed using SBERT embeddings and Manhattan distance to capture divergences between documents (d-nodes) and summaries (s-nodes). At the same time, PerAugy generates synthetic but realistic user histories that preserve temporal coherence.\n\nThe experiments evaluate user encoders (NAML, NRMS, EBNR) and personalized summarization frameworks (PENS for extractive, GTP/TrRMIo for generative) across multiple datasets, including PENS, OpenAI Reddit, and LLM-generated UIGs. Results show that the base datasets have a low DegreeD (<0.01), while PerAugy boosts it substantially (up to approximately 0.28). Higher DegreeD correlates strongly with encoder accuracy (Pearson r = 0.68–0.69 across AUC, MRR, nDCG). Ablation studies confirm DS increases diversity, with DS+SMP producing the most coherent trajectories. Finally, integrating improved encoders into summarization frameworks reveals that GTP models benefit most (up to 4 times gains in AUC/nDCG), while PENS shows only modest improvements.", "claims_and_evidence_yn": "No", "claims_explanation": "Claim 1: DegreeD is a valid metric for dataset diversity, and Current datasets lack diversity.\n\nThe paper reports very low DegreeD scores for baseline datasets (PENS: 0.009 and OpenAI (Reddit): 0.008). In addition, other augmentation methods did not improve DegreeD as much (Table 2). Correlation study in Meta-evaluation (Table 4) shows that DegreeD positively correlates with encoder performance metrics.\n\nWhile it is plausible that high diversity leads to higher performance, the correlation between DegreeD and performance does not establish a direct trend with diversity. Alternatively, albeit less likely, it might be that the other augmentation method is too diverse and the models fail to learn it (i.e., underfitting rather than overfitting).\n\nClaim 2: Double Shuffling (DS) and Stochastic Markovian Perturbation (SMP) are both necessary\n\nUnderstandably, the DS is the core of the PerAugy method; however, it is unclear whether SMP is necessary for its effectiveness. There are two main flaws in the supporting evidence. First, there is no experiment without the SMP. Second, the ablation experiment in the Appendix was on the DegreeD. The conclusion is then a two-step reasoning, DS/SMP -> DegreeD -> Model Performance.\n\nClaim 3: PerAugy improves downstream personalization\n\nUser encoders (NAML, NRMS, EBNR) trained on PerAugy data outperform those trained on baseline data across AUC, MRR, nDCG. Both summarization frameworks, PENS and GTP, benefit from the proposed method. Significant gains (up to 4 times improvement) are observed in GTP.", "audience_yn": "Yes", "audience_explanation": "Researchers in personalized summarization, recommender systems, and data augmentation introduce a novel diversity metric (DegreeD) and augmentation pipeline (PerAugy), with clear implications for personalization and dataset quality. Additionally, the proposed method may be adaptable to other types of time series datasets.", "requested_changes": "From the comments above:\n\n1. Provide stronger evidence that higher DegreeD directly improves model generalization, rather than only reporting correlations. For example, the paper may present the training performance along with the test performance to emphasize the generalization ability of using the PerAugy datasets.\n2. Include experiments with DS-only (without SMP) to test whether SMP contributes to performance improvements directly.\n3. Complement ablation evidence with downstream model performance comparisons. This would strengthen the causal link between SMP and improved personalization.\n\nFor the paper's clarity\n\n1. Provide simpler examples of the DS. Rather than a figure, it would be much simpler to see a user trajectory (A) is augmented with another trajectory (B) like this [A1, A2, A3, B4, B5, A6, B7, B8, A9] (O=3, G=1). The current description leaves ambiguity (e.g., is it substitution or insertion? What exactly is being replaced in the target trajectory?). A more explicit worked example of multiple users’ trajectories under different gap lengths would help readers follow.\n2. Differentiate and clarify $\\sigma$. Currently, it uses SBERT embeddings and Manhattan distance. Authors should explain why RMSD/Manhattan was chosen over alternatives like cosine similarity, and how sensitive DegreeD is to this choice.\n3. Clarify the notation $\\delta$. In Eq. 1, it has no superscript, but in Eq. 5, it does. What is the difference? (Why add $X$ in the notation?)\n4. Table 1 misses $a$ and typo in $E_j$ (should have been $\\mathbb{E}_j$).", "broader_impact_concerns": "None", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "zRxxcVeZd7", "reviewer": "Reviewer_hXxr", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper introduces DegreeD, a metric for measuring diversity in user preference trajectories, and PerAugy, a data augmentation method that combines cross-trajectory shuffling with summary perturbation to generate more diverse and realistic interaction histories, thereby achieving higher DegreeD scores. Experiments show that PerAugy  improves user-encoder performance and boosts personalization for downstream SOTA summarizers.\n\n### Strengths\n\n- Tackles an important challenge in personalized summarization: lack of diverse training data.\n\n- Strong experimental evaluation across multiple baselines, including heuristic and LLM-based augmentation methods.\n\n- Demonstrates both theoretical (diversity–performance correlation) and practical (improved summarization quality) contributions.\n\n### Weaknesses\n\n- The method introduces multiple hyperparameters which raises concerns about sensitivity, reproducibility, and potential overfitting to specific datasets.\n\n- Current evaluation focuses heavily on PENS; broader coverage on other domains or languages would strengthen claims.\n\n- The interpretability of DegreeD is less intuitive than simpler diversity metrics, and the connection to personalization performance does not feel very direct to me.", "claims_and_evidence_yn": "No", "claims_explanation": "While I appreciate that the paper addresses an overlooked problem and introduces both a new evaluation metric and an augmentation methodology, I have key concerns. The tight coupling of the proposed metric (DegreeD) and the method makes it difficult to independently justify the methodology and weakens the overall evidence. In Section 6.1.4 (RQ-2), the authors present a small-scale study suggesting a positive correlation between DegreeD and user-encoder performance, but this feels insufficient. Furthermore, I believe the new metric should be validated more rigorously, for example through human assessments or by demonstrating consistent alignment with existing other evaluation metrics.", "audience_yn": "Yes", "audience_explanation": "This work addresses the data scarcity problem in personalized NLP. The proposed augmentation strategy has potential applications beyond summarization, including other preference-driven tasks such as recommendation and dialogue personalization, which broadens its relevance to TMLR’s readership in both LLM4Rec and NLP. I also found Section 3 on user preference modeling to be an interesting and timely direction that is likely to attract attention from the community.", "requested_changes": "### Major\n\n- Provide more justification and intuition behind DegreeD, since the entire paper relies on it. This could include comparisons with simpler existing metrics or validation against human assessments.\n\n- Clarify the sensitivity of PerAugy to hyperparameters and offer practical guidelines to improve reproducibility.\n\n- Please correct the formatting of references, as the current style hinders readability badly.\n\n### Optional\n\n- Provide some qualitative examples or cases of augmented trajectories.\n\n- Discuss the limitations when ground-truth preference histories are unavailable (i.e., anyway to cold start?).", "broader_impact_concerns": "None", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "oS7nJUuiaq", "reviewer": "Reviewer_qSYo", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "* The paper proposes DegreeD, a novel metric intended to capture the diversity of user-preference trajectories for training personalized text summarization models. Lower DegreeD implies that the training data contain limited user interests, which may lead to overfitting and weaker generalization.\n* The paper also proposes PerAugy, a novel user trajectory augmentation pipeline for the same application. Specifically, PerAugy combines Double Shuffling (DS) and Stochastic Markov Perturbation (SMP), which effectively generates realistic synthetic user trajectories.\n* Experimental results show that PerAugy improves the personalized text summarization performance across multiple models and datasets, especially for two distinct dataset types (PENS vs. OpenAI-Reddit).", "claims_and_evidence_yn": "No", "claims_explanation": "* The main contribution, the DegreeD metric, does not seem sufficiently supported by experimental results. The reader would expect that a higher DegreeD will improve the end performance; however, Tables 2–3 alone do not establish a monotonic relationship between DegreeD and downstream performance across baselines. Although the authors present correlation results in Table 4; it would strengthen the claim if the paper reports the correlations computed ‘without’ (i.e., exclude) PerAugy.\n* It needs to be clearly explained that PerAugy is targeted to increase the DegreeD score. In other words, they both start with the same intuition that diverse trajectories would lead to better results. Because PerAugy is designed to increase DegreeD, there is a risk of circularity.\n* DegreeD relies on the embedding space from a specific model and specific distance (e.g., Manhattan distance). It is unclear if the proposed metric is dependent on a certain embedding model or distance type.\n* The notation can be improved further. For example, the divergence ($delta$) operator is dependent on the timestep $t$, but the notation itself does not include $t$. Overall, the notations are too complicated, and I needed to revisit the notations repeatedly – yet I could not sure I have understood the process correctly.\n* The definition of DePS is somewhat difficult to follow, particularly in equations (1)-(5). Maybe a concrete example would help readers understand the concept of DePS. In particular, please justify why the min/max ratio is the right choice for capturing proportional shifts between document and summary divergences.", "audience_yn": "No", "audience_explanation": "I acknowledge that this topic is important yet underexplored. However, to broaden appeal and accessibility, especially for readers new to personalized summarization, please expand the related-work context and clarify the definitions, figures, and notations.", "requested_changes": "Please see the other sections.", "broader_impact_concerns": "Although the authors do not explicitly discuss the broader impact, I think this paper does not have ethical or political concerns. On the other hand, the paper includes a limitation section.", "additional_comments": "* The reference style makes the paper difficult to read. Please use proper parentheses.\n* The figures and corresponding explanations are sometimes difficult to follow. Before and after do not look very different. Please consider adding realistic examples or consider an alternative visualization to illustrate the idea.\n* I am somewhat confused about the “genSum” and “summGen”. So, the “genSum” action connects a document to its summary (which is a document node, not a summary node…). Then, “summGen” action again connects the summary to the new summary node?\n* Is the test set size of 103 user trajectories sufficient? Can you compare the size of the test datasets commonly used in this domain?\n* Can you explain more about ‘why’ the “unique topics per trajectory” or “Rate of change of topics” are insufficient, especially compared to DegreeD? They are interpretable and easy to compute.", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)