Adaptive Preference Arithmetic: A Personalized Agent with Adaptive Preference Arithmetic for Dynamic Preference Modeling
Authors: Hongyi Nie
Yaqing Wang
Mingyang Zhou
Feiyang Pan
Quanming Yao
Zhen Wang
Venue: NeurIPS 2025 poster ()
Keywords: Personalized LLM Agents
Dynamic Preference Modeling
Adaptive Preference Arithmetic
Primary area: applications
TL;DR: AdaPA-Agent enhances LLM agent personalization by dynamically modeling user preference strengths via 'Adaptive Preference Arithmetic,' learning from existing interactions without extra user feedback.

Abstract: As large language models (LLMs) are increasingly used as personalized user assistants, effectively adapting to users' evolving preferences is critical for delivering high-quality personalized responses. While user preferences are often stable in content, their relative strengths shift over time due to changing goals and contexts. Therefore, modeling these dynamic preference strengths can enable finer-grained personalization. However, current methods face two major challenges: (i) limited user feedback makes it difficult to estimate preference strengths accurately, and (ii) natural language ambiguity limits the controllability of preference-guided generation. To address these issues, we propose AdaPA-Agent, a LLM-agent personalization framework that models dynamic preference strengths via Adaptive Preference Arithmetic. First,  instead of requiring additional user feedback, AdaPA-Agent employs an alignment-based strength estimation module to estimate the strength of user preferences from the existing user-agent interaction. Then, it guides controllable personalized generation by linearly combining next-token distributions, weighted by the estimated strengths of individual preferences. Experiments on two personalization tasks-conversational recommendation and personalized web interaction-demonstrate that AdaPA-Agent better aligning with users' changing intents, and has achieved over 18.9\% and 14.2\% improvements compared to ReAct, the widely-used agent framework.

Adaptive Preference Arithmetic: Modeling Dynamic Preference Strengths for LLM Agent Personalization
Hongyi Nie
Yaqing Wang" Beijing Institute of Mathematical Sciences and Applications wangyaqing@bimsa. cn
Northwestern Polytechnical University
hy_niie niemmail. www.u.wpue......mmm..m
Mingyang Zhou
Feiyang Pan
Tsinghua University
Tsinghua University pf efy824@gmail.com
zhou-my24mmalls. ts tsinghua..cnmmm edu.
Quanming Yao Tsinghua University yaoaaθtsinghua edu. on
Zhen Wang' Northwestern Polytechnical University w-zhen@nvpu.edd edu cn
Abstract
As large language models (LLMs) are increasingly used as personalized user assis- tants, effectively adapting to users' evolving preferences critical for delivering high-quality personalized responses. While user preferences are often stable in content, their relative strengths shift over time due to changing goals and contexts. Therefore, modeling these dynamic preference strengths enable finer-grained personalization. However, current methods face two major challenges: (i) lim- ited user feedback makes it difficult to estimate preference strengths accurately, and (ii) natural language ambiguity limits the controllability of preference- guided generation. To address these issues, personalization framework that models dynamic preference strengths via Adap- tive Preference Arithmetic. First, instead of requiring additional user feedback, wmp-ooy an alignment-barn
propose AdaPA-Agent, LLM-agent
AdaPA-Agent employs the strength of user preferences from the existing user-agent interaction. Then, it guides controllable personalized generation by linearly combining next-token dis- tributions, weighted by the estimated strengths of individual preferences. Exper- iments on two personalization lasks-conversational recommendation and person- alized web interaction-demenn demonstrate that AdaPA- AdaPA-Agent better aligning with users' changing intents, and has achieved over 18.9% and 14.2% improvements com- pared to ReAct, the widely-used agent framework.
strength estimation module to estimate
1 Introduction
Agents powered by large language models (LLMs) [1L 2 are increasingly being utilized as user as- sistants [BI], helping individuals with tasks such as information retrieval and decision-makinn making. Given that user needs are highly diverse and personalized, they often do not want assistants to provide generic responses. Instead, users expect these user assistants to understand their unique needs better and provide personalized services [4]], such as travel planning [5], and shopping recommendations
Corresponding authors
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Therefore, enhancing the ability of LLM agents to model users' personalized needs has become a key focus of current research [[]]
B ZZ]-
Since user preferences fundamentally shape their personalized needs, recent approaches improve personalization of L.M agents by modeling and leveraging these preferences [8]. These methods can be divided into two categories: fine-tuning and training-free methods. Fine-tuning [b, [Q] in- volves adjusting the model's parameters fit personal preferences. However, this approach requires gathering large amounts of individual data and model training, making it both resource-intensive and costly. In contrast, training-free -ree methods such as retrieval- -aggeeetd generation (RAG) and prompt engineering provide more flexible alternatives. RAG [8 [T] enables agents to retrieve rele- vant preference information in real-time adapting responses without retraining the model. Prompt engineering [T22. IT3 [4] inserts the prompts of preference content into prompt templates, which can guide the model toward personalized outputs without modifying its underlying parameters.
In real-world scenarios, users often hold mul- tiple coexisting preferences-such favoring both healthy meals and junk food-but the in- fluence of each preference on user's decision- making is not equal and can vary over time [[[]] We term this influence as preference strength. As shown in Figure I. without modeling this dynamic variation in preference strength, LLM agents may produce responses that fail to align with the user's current intent.
Streutth
User preference
Changes in Preference Strengths
Pref-1: healthy food
Strongth Prodi
Pref-2: high high sugar food
Streegth Prof
T2
T3
T4
Time
I'm a little hungry, please order takeout for me.
Static preference/ promaee
User
Although, existing methods make great progress in personalization, they still face two key challenges of modeling the dynamic preference strengths for LLM agents: (i) Accu- rately estimating preference strengths without relying on explicit user feedback. Existing methods such as ReAct [[6]] and Reflexion [IZ]] often require user feedhack to help adjust the strengths of preferences. Yet in real-world scenarios, user feedback is often limited in number, and frequent requests for feedback will disrupt normal interactions and signifi- cantly degrade user experience. (ii) Effectively utilizing estimated strengths to guide content generation. A common strategy is to input the preference strengths text prompts LLMs. However, this prompt-base method is fundamentally limited by the inherent ambiguity of natural language [T& I99, which may lead to misaligned personalized responses.
T1
T3
X
Hambbrgak
LLM Agent
(respoase for a user's food delivery request different time points)
satistied
X
unsatisflad
Illustration of an agent responding to user's food delivery request over time. The user's needs are shaped by two preferences (Pref-1 and Pref-2), whose relative importance changes. To respond effectively, the agent must recognize and adapt to these shifts; otherwise, it may fail to meet expectations.
Figure 1:
To address these challenges, we propose AdaPA-Agent, framework that models and applies dy- namic preference strengths through Adaptive Preference Arithmetic. AdaPA-Agent comprises two key components. First, to estimate preference strengths without explicit user feedback (Challenge ()), we introduce Alignment-Based Strength Estimation, which combines dual-side augmenta- tion with an LLM-based alignment scorer to measure how well each preference aligns with the current user-agent interaction. Second, to utilize these estimated strengths in generation (Challenge (ii),, we propose Controllable Personalized Generation. This component modulates the output distribution of an LLM by linearly combining next-token distributions conditioned individual preferences and weighted by their inferred strengths. We validate AdaPA- on two personal- ized agent tasks: conversational recommendation and personalized web interaction. Experimental results show that AdaPA-Agent consistently improves personalization quality over strong baselines, enabling LLM agents to generate responses that better reflect users' evolving intents and priorities.
Our key contributions can be summarized as follows:
We identify two key challenges of modeling dynamic preference strengths for LLM LLM-agent person- alization: (i) accurately estimating preference strengths without relying on explicit user feedback, and (ii) effectively utilizing these estimated strengths to guide content generation.

---

To address these challenges. we propose AdaPA-Agent that models dynamic preference strengths for LLM agents through Adaptive Preference Arithmetic. AdaPA-Agent introduces two novel components: Aligmment-Based Strength Estimation for estimating preference strengths without explicit feedback, and Controllable Personalized Generation, which controls the personalized generation process by linearly combining next-token distributions of LLM based on the estimated strengths.
We validate AdaPA-Agent on conversational recommendation and personalized web interaction tasks, achieving over 18.9% and 14.2% improvements compared to widely-used agent framework ReAct, respectively, demonstrating its effectiveness adapting to users' dynamic preferences.
Related Work
2.1
LLM Agent
The emergence of LLMs has transformed autonomous agents, enabling them to perform tasks with human-like intelligence through powerful reasoning and knowledge capabilities. These agents typ- ically feature modules for profiling [X]] memory [20]], planning [21[], and action [22]. Profiling
defines the agents role and personality [23 (24], while memory enables learning and adaptation [20]]. Planning is enhanced by Chain-of- Thought (CoT) [25]] and Tree of Thoughts (ToT) [26], which Chain-of-Thought
break down complex tasks using sequential or tree-based reasoning. Action modules execute plans by combining internal logic with external tools [27, 22]. Integrated reasoning and acting approaches, like ReAct (166), enable dynamic necision-making. while Reflexion [17]] improves adaptability via self-reflection and feedback.
L.M agents are now widely used in domains like marketing 4X and software development [ As these agents become more capable, user demands have shifted from seeking general responses to expecting interactions that are more aligned with their personalized needs. Enhancing the ability of agents to provide tailored and personalized experiences has thus become key focus of current research.
2.2 LLM Personalization
Real-world applications like personalized web interaction [27]] and conversational recommendation [B0] demand flexible models that adapt to evolving user needs. While traditional methods like collaborative filtering [+T]] are common, they lack semantic understanding and rich interactions. With LLMs, there's a shift toward language-based agents, offering more personalized and responsive experiences. LLM personalization mainly follows two approaches: fine-tuning and training- free methods. Fine-tuning [5% IIO, 32] adjusts model parameters using user-specific data for improved alignment, though it requires significant resources. Training-free free approaches like RAG and prompt engineering offer lightweight alternatives. RAG [XL [[]] retrieves external data in real time, while prompt engineering [ 13,
[4] [ adapts prompts to personalize outputs without modifying the model. Recent works further explore controllable text generation guided by user preferences. For instance, OPAD [62]] and CoS [65] propose controllable generation methods that rely on explicit preference instructions to guide LLM outputs. AMPLe [G6] infers preferences through multiple rounds of explicit user feedback.
However, current methods still struggle with modeling preference strengths and controlling gener- ation when user feedback is sparse or inconsistent. Our work addresses these gaps by improving preference modeling and guiding LLM agents to produce more personalized outputs.
3 Target Formulation
This paper focuses on personalization tasks where LLM-based agents generate responses tailored to user preferences. This includes (i) conversational recommendation, where the agent interacts with users to refine intent and recommend suitable items, and (ii) personalized web interaction, where the agent performs context-aware actions such as search or comment generation based on inferred preferences. Here, we formulate the personalization task of LLM agent as follows:

---

Input
Alignment- Based Strength Estimation
(2) Controllable Personalized Generation
Preference Chains ('ueeeence-sddee:
Preforence pm
pedle
sciled skidkon
LLM(O cb. G)
Porf-1
heallity food
high and sugar
In pudlie lagle
LLMiO de
[0 aa Wal mll
Lify lacerty
Dhaalaiie Augieataatinn
Distabirious
Syntheck
bure
(Interactinn thon side):
Sacple
ming pear liky ligh??
based
Toon
yeg
Alignment
LLM( G2
Ouipu 02
User agent interactions: DF
I'd recommenn rice bowl with
grilled chicken
Figure 2: Overall framework of AdaPA-Agent, which includes two components: (1) Alignment-
Based Strength Estimation, which computes alignment scores between structured preference chains and augmented user-agent interactions infer the relative importance of preferences. (2) Controllable Personalized Generation, which combines next-token distributions from preference- conditioned LLM outputs, weighted by their estimated strengths. Our method enables the agent to generate responses that reflect the user's evolving priorities without requiring explicit feedback.
Definition 3.1 Personalization Task of LLM Agent [27, [J Given user u. his user-agent inter- action and intent at current time is Dt and Iú, and historical interactions is H" (- The intent It is influenced by the user's s preference set p" = {pr preferences. pa can be extracted from H²" and each preference p. has a corresponding strength wi- The goal of the LLM agent to infer the preference information from user-agent interactions and use it to generate response O that aligns with the user's 's intent I.
1 Ti=b where M the number of
Methodology
Existing personalization methods often encode user preferences as static prompts or memories, over- looking how their relative strength evolves over time. However, modeling dynamic preference strength is challenging due to limited user feedback and the ambiguity of natural language. To address this, we propose AdaPA-Agent, a LLM-agent personalization framework that captures preference dynamics via Adaptive Preference Arithmetic. As shown in Figure 2. it includes: (1) Alignment-Ban Based Strength Estimation (Section D.D,,, and (2) Controllable Personalized Gener- ation (Section ) The first component estimates preference strengths adaptively without extra feedback through a dual-side augmentation module and an LLM-based alignment scorer. These strengths are then used in generation by combining next-token distributions conditioned prefer- ences in the second component. This allows the agent to adapt responses continuously based on updated preference weights. The following sections detail each component.
4.1 A Alignment-Based Strength Estimation
Finding an observed signal that correlate the strength of user preferences important for estimation without user feedback. Our key insight is that the stronger a preference, the more frequently it manifests in user-agent interactions. Based on the insight, we reformulate strength estimation as alignment problem: for each preference, we measure how well it semantically aligns with the user-agent interaction and treat this alignment score as a proxy for strength. A higher alignment score indicates that the interaction provides stronger support for the preference, hence implying larger strength for that preference.
However, user expressions are often vague and diverse in practice, which makes alignment scoring unreliable when using naive methods such as keyword overlap or static embedding similarity. To address this challenge, we incorporate two key techniques: (i) dual-side augmentation, which en- riches both the preference (via preference chains) and interaction sides (via conversation-leve para- phrases) with more informative and semantically diverse representations; and (ii) an LLMe-based

---

scoring module, which assesses the alignment between each augmented ceeference-interactnn pair context-aware and interpretable manner.
4.1.1 Dual-Side Augmentation
Preference-side augmentation. We first extract the set of preferences P" from historical interac- tions H." However, user preferences often emerge multiple semantic levels, i.e. from abstract self-descriptions to concrete behavioral cues. To capture this spectrum, we construct three-stage preference chain for each preference p using a CoT prompt:
C; CoT(p)) [p!uw pt P3
where p. is the original description of the preference extracted from p p is context-aware reformulation refining p,a" and p. enumerates concrete items or behaviors exemplifying p,a and p. This three-level structure supplies coarse-to- to-fie preference information for the subsequent alignment scoring. The complete prompt is provided in Appendix Al Prompt 1.
Interaction-side augmentation. Since user intent is typically implicit and linguistically diverse, computing an alignment score from a single interaction can be highly biased. To reduce this variance, enrich the interaction side by paraphrasing the original user-agent interaction Di into multiple semantically equivalent variants:
g(D)) {D", Di .... D )
where we employ an LLM as a generation function g(-) (Prompt Appendix A), each synthetic interaction Dj, (A = 1, 1,2,.... K) preserves the same number of turns as Dj but rewrites utterances with alternative lexical and syntactic choices enrich the expression of user intent. The augmented set A" (D)) U Di forms more complete pool of interaction variants for alignment scoring.
4.1.2 LLM-Based Alignment Scoring
Compared to static embeddings or keyword matching. LLMs offer richer understanding of seman- tics, enabling context-aware and fine-graie grained alignment. Therefore, we employ an LLM to assess the alignment score of each peefreence-interactio pair. Specifically,, given the structured prefer- ence chains C; and the augmented interaction set AA, we compute alignment score 8; for each preference p" as:
Σ f(C,, D),
(3)
AT
DEA"
where f(.,)) -) is an -Lsed scorer returning fine-grat tine-grained alignment score (1-10) for a preference- interaction pair (Prompt 3, Appendix ().. We then normalize the scores obtain relative strength of each preference:
W;
M
j=1
Through this process, we obtain and adaptively update relative strength of each preference with- out requiring additional user feedback.
4.2 Controllable Personalized Generation
Once the set of preference strengths {wi) is obtained, the remaining question is how use these weights to steer an LLM so that its response continuously reflects the user's current intent. Given the inherent ambiguity in natural language prompting, directly inserting numeric weights into a text prompt is unreliable [LS]. Recent work [L']] has demonstrated that complex textual style can be effectively generated by arithmetically combining the next-token distributions of language models conditioned on different basic styles. This motivates the proposal of preference arithmetic, citerrecce-coddtioonn distributions in formulaic way, using
technique that combines multiple the dynamically estimated strengths W; as weights. This allows for fine- ane-grained control over the influence of each preference De the generated response Op.
Specifically, each step of generating the response OH,, the L.MM produces next-token distribu- tion. Let P (pp(-) be the optimal personalized next-token distribution, which is conditioned on the
5

---

user's intent I, We model P (-) as weighted combination of M individual next-token distri- butions, where each individual distribution P pp;()) is conditioned on a specific preference p from the user's preference set pa. The overall next-token distribution for generating the current token of the response Of, given the partially generated response prefix Üi and the current user-agent interaction Di, is defined as:
(wwlww Dr. IT)
Σ P (we) O (,ck) Di, Di,
(5)
where each preference-condition distribution (.) generated by the LLM when prompted with the specific preference p in the context of the current interaction Dj and the already generated part of the response Oi,ck- For preference p. we can use its augmented representation, the pref- erence chain C, (from Section A.I., construct a comprehensive prompt for the L.M. Thus, the i-th preferencee-conditioned next-token distribution is:
P pp (wx [O, D,,, p.) LL, Di, C.)).
This formulation enables a continuously steerable decoding process, where each preference influ- ences generation proportionally to its weight. Equivalently, the combined distribution can be viewed the solution weighted KL minimization problem with closed-form optimum [19]]
M
P = arg min
wi DKL P 1Q.),
Pt (wxx) softmax log Q.(wx))
Here, P* P*(wx) represents the KL--optimll compromise among the preference-conditionee distri- butions Qi where stronger preferences (larger wy) dominate while weaker yet relevant ones still contribute to a balanced, intent-aligned generation.
To generate the full response Oj, tokens sampled autoregressively. As illustrated in Algorithm M at each generation step k, the corresponding token Wy is sampled from the combined distribution: Wk ~ Ei= Wi LLM((OH, tkck Di, C.)), and extend the context as: Oi = Oi wx)- This process continues iteratively until the sampled token wy is the end-of-sequence token (EOS), ensur- ing the generation of coherent and personalized responses.
5 Experiments
In this section, we evaluate our AdaPA-Agent on two typical LLM personalized agent tasks (con- versational recommendation and personalized web interaction). Additionally, we validate the effec- tiveness of our method design through series of ablation experiments. In the following, we first present the experimental setup and baselines, and then provide detailed presentation and analysis of the experimental results.
5.1 Experimental Setup
5.1.1 Baseline
The proposed AdaPA-Agent is compared with the following baselines: ReAct [[[]] combines reason- ing and action generation to improve decision-making in interactive tasks, allowing models to adapt dynamically to new information and user needs. Reflexion [| uses verbal reinforcement learn- ing for self-reflection and feedback, allowing language agents to continuously refine their actions and reasoning. making them more adaptable to personalized interactions. Sim'ToM [3Z] introduces perspective-takin to improve theory-of-mind (ToM) capabilities, helping models better understand and predict human intentions, thus tailoring responses based individual user preferences. Rec- Mind [B8] leverages an LLM-powered autonomous recommender agent with self-inspiring algo- rithm, enabling zero-shot personalized recommendations by considering historical information and
The code is available https///ftttubb.ccm//ttttssssss

---

Maximum
Long-Tol Term Tasks
Short-Tema Tasks
All Tasks
Method
Steps
RSR
AIR
RSR
AIR
RSR
AIR
ReAct [16]
30.27
2.81-0.09
30.19+7.73
2.89
30.20=3.20
2.85++.05
Reflexion 77]
38.45
2.74_0.09
28.6943.79
2.85
33.80=6.50
2.79
RecMind
[8]
39.77-9.08
2.73
32.50+4.44
2.85+0.05
36.40=4.73
2.78
3
InteRec
39
11.55 45.67
2.66+0.16
32.87
2.76
39.60
2.71+4.:1
SimTom
37
40.22
2.78-0.09
31.19
2.87 20.05
36.01
2.82+
AdaPA-Agent
46.10 19.65
2.6610.10
37.05
2.62
41.45
2.64 0.08
ReAct [T6]]
57.47
4.19
40.99+4.59
4.4640.09
49.60-4.60
4.32
Reflexion 7
67.88_7.80
3.81 3.21
61.40+6.30
4.24
60.20_4.80
4.01
5
RecMind
66.58
9.29
0.30
58.92 14.72
14.72
40.18
5.88
3.88 3.88
InteRec
39
70.28 40.28+10.35
3.67
3.67
66.43
3.88
10.15
66.36
3.77
SimTom
17
66.20
9.71
3.88
52.58
4.17
59.45
4.02
AdaPA-Agent
71.27 19.37
3.64
63.04 1.884
3.83
10.19
67.24
1.441
3.73 10.23
ReAct [GG]]]
73.50_7.77
5.06_0.39
51.92+3.31
5.59
63.40=1.60
5.31
Reflexion
79.70 5.74
4.51_0.43
62.08
5.20
71.40
4.83
7
RecMind
81.26 55338
4.40_0.43
65.24
4.82
73.80
22.93
4.59
InteRec
79.64_5.81
4.26
69.60
+0.27
75.00
-3.33
4.38
SimTom 17
79.98 4.94
4.44_0.42
64.07
5.07
10.32
72.60-3.27
33.27
4.73
AdaPA-Agent 80.36 5.64
4.41
_0.42
70.45
4.51 +0.30
75.41 +3.07
4.46 10.36
Table I: Main results of the AdaPA-Agent method and baseline models the conversational recom- mendation task. The table presents recommendation successful rate (RSR) and average interaction rounds (AIR) for long-term, short-term, and overall performance, evaluated under different maxi- mum steps (3, 5, and 7).
previous states. InteRec [R integrates long-term and short-term memory mechanisms to enhance personalized recommendations, improving the understanding of user preferences and context for more relevant interactions.
5.1.2 Implementation Details
For AdaPA-Agent, we use -a-tma--.-----Inttttu [+0] as the local LLM to generate the next-token distribution. In the inperaation-augmenttnn dation task and K = 2 for the personalized web interaction task. In the preference arithmetic phase, we set M for both two tasks, i.e., we choose top preferences to generate the personalized next-token distribution. In the conversational recommendation task, GPT-4o [НI]] serves as the user simulator, while DeepSeek V2.5 [~2] supports all baselines and AdaPA-Agent. For the personalized web interaction task, GPT-4o is used consistently across all baselines and AdaPA-Agent. To reduce generation randomness, we set the LLM temperature during all evaluations.
phase, we
4 for the conversational recommen-
5.2
Conversational Recommendation Task
5.2.1
Task Introduction
Conversational recommendation systems personalize suggestions through real-time interactions. Users express needs via natural language, while the agent clarifies and recom- mends accordingly. To provide recommendations that better align with the user's intent, the agent needs to be able to model the changes in the user's preference strength. (2) Task Construction: To evaluate how well our method models preference strength, we divide user movie preferences into long-term and short-term types. Based on the Reddit-Movie dataset [[]], we filter 984 unique users with sufficient historical data. Then, we extract their stable preferences from historical data as long-term and select unrelated movies to define short-term preferences. A dynamic simulation environment is built using an LLM-based user simulator [83) A4 where each task randomly assigns a dominant preference type to simulate the conversation. This setup allows controlled variation in preference strength, enabling direct and systematic evaluation of our method. We introduce the setup details of task settings and user simulator in Appendix Q (3) Evaluation Metrics: The task limited to 7 rounds. the agent recommends the target movie within rounds (k: * T), suc- ceeds; otherwise. it fails. Performance is measured by: Recommendattin Successful Rate (RSR)
(1) Task Descríption:
7

---

Method
Search
Recommendation
Review
Overall
F. Acc
R. Acc
F. Acc
R. Acc
F. Acc
R. Acc
F. Acc
R. Acc
Random Memory
0.974
0.640
0.296
0.018
0.996
0,442
0.745
0.357
Last Memory
0.937
0.626
0.432
0.028
1.000
0,442
0.782
0.357
Relevant Memory
0.928
0.622
0,492
0.030
1.000
0,443
0.800
0.356
ReAct [
0.903
0.605
0,560
0.027
0.996
0.444
0.815
0,350
RecMind [38]
0.981
0.645
0.226
0.017
0.990
0.442
0.721
0.359
AdaPA-Agent
0.987
0.654
0.592
0.027
1.000
0.457
0.851
0.367
Table 2:
Main results of the AdePt-Agent method and baseline models on the single-turn track settings of personalized web interaction. The table shows the function accuracy (F. Acc.) and response accuracy (R. Acc.) for the three web services: Search, Recommendation and Review.
Method
Recommendation F. Acc Acc Avg. Steps F. Acc R. Acc Avg. Steps F. Acc Ace Avg. Steps F. Acc R. Acc Avg. Steps
Search
Review
Overall
Random Memory
0.999
0.680
4.193
0.703 0.042
4.474
1.000 0.448
2.007
0.896 0.380
3.564
Last Memory
0.996
0.676
4.229
0.708 0.045
4.252
1.000
0.449
2,007
0.897
0.381
3.597
Relevant Memory 0.996
0.686
4.233
0.715
0.042
4.564
0.999
0.448
2.008
0.899
0.383
3.609
ReAct [IT6]
0.996 0.674
4.657
0.218
0.013
5.468
0.974
0.448
2.129
0,718
0.369
4.198
Reflexion [|
1.000
0.686
5.406
0.281 0.014
6.145
0.976
0.449
2.145
0.741
0.373
4.579
RecMind [88]
0.997
0.642
6.728
0.347 0.026
6.003
0.997
0.451
2.107
0.771
0.364
4.938
InteRec [39]
0.999
0.642
3.110
0.618
0.022
3.008
1.000
0.447
2.001
0.867
0.362
2,706
AdaPN-Agent 0.999 0.698
5.352
0.768 0.039 3.485
1.000
0.455
2.004
0.917 0.386 3.592
Table 3: Main results of the AdaPA- method and baseline models on the multi-turn track set- tings of personalized web interaction. The table shows the function accuracy (F. Acc.), response accuracy (R. Acc.) and average steps (Avg. Steps) for the three web services: Search, Recommen- dation and Review.
N 100%, Average Interaction Rounds (AIR) E 4, where N is the total tasks, N, the successful recommendations, and k; the interaction rounds in task i. Higher RSR and lower AIR indicate better efficiency.
5.2.2 Results Analysis
The results in Table M compare AdaPA-Agent with baseline models under varying maximum steps (3, 5, and 7) in the conversational recommendation task. AdaPA-Agent consistently outperforms baseline methods
terms of RSR while maintaining competitive step efficiency. AdaPA-Agent maintains stable and robust performance across both long-term and short-term tasks, unlike some baseline methods that exhibit imbalances. In contrast, baseline methods often show trade-offs. For instance, RecMind performs well on short-term tasks but suffers on long-term preferences, while Reflexion achieves moderate long-term results but lags on short-term tasks. These inconsistencies suggest a lack of adaptive preference modeling. AdaPA-Agent addresses this gap by estimating dynamic preference strengths and adjusting generation accordingly, leading to stable performance across diverse user intents. Although InteRec has explicitly considered the long-term preference and short-term preference, it still suffers the problem of modeling the preference strength and has a lower performance than AdaPA-Agent. Furthermore, AdaPA-Agent strikes an optimal balance between RSR and AIR, making it more efficient in real-time applications. At just 3 steps, it outper- forms all baselines on overall RSR (41.45% vs. 38.60% by RecMind and 36.01% by SimToom), and achieves the lowest AIR (2.64). This demonstrates that AdaPA-Agen can effectively estimate pref- within short
erence strengths with minimal feedback and provide accurate recommendations interactions, confirming its effectiveness in real-world conversational recommendation systems.
5.3
Personalized Web Interaction Task
5.3.1
Task Introduction
(1) Task Description: Personalized Web agents infer user preferences to enhance web services. We use the PWABench benchmark [2Z]], which includes tasks like personalized search, recommenda-
8

---

User preference
User-agent Interactions
Response of AdaPA-agent
Pref-1: 0.82 Pref-2: 0.18
A: Looking for a movie?
How about The Intern? It's heartwarming comedy with just the right touch inspiration perffct for relaxing night.
U: I'm in the mood for something fun to watch tonight.
Pref-1: light-hearted comedy
A: Looking for movie?
Pref-I: 0.10 Pref-2: 0.90
U: I want story that really moves
A: Since you 'e looking for something emotionally powerful,
Pref-2: emotional movies
I'd recommend The Pursuit of Happyness it's deeply moving and incredibly inspiring.
Preference-sine Angmentation
A: Looking for movic?
Pref-1: 0.38 Pref-2: 0.62
U: I want watch something meaningful but not to beavy.
A: You might enjoy About Time- it's heartfelt and thought- provoking, but still light and charming in tone.
Interaction Augmentation
Preference chain of Pref-1:
Synthetic Interaction:
gou enjoys light bearted comedies
pref; prefets movies that casy to watch
A: Looking for movie?
U: I just want something easy to watch this evening.
per. Padcington Crazy Rich Asians
A: Looking for a movie?
Preference chain of Pref-2:
U: I want film that tells a powerful, touching story.
pru likes omotional movics
pro, enjoys ennviionally resoraal inspirational films pr Marringe Story, Flipped
A: Looking for movic?
U: I'm hoping for something thoughtful, but still easy to watch.
Figure 3:
Case study of AdaPA-Agent Agent in movie recommendation scenario. The model dynamically adjusts preference strengths based on user-agent interactions and generates personalized responses.
tion, and review generation. These tasks require LLMs to select the right web function and param- eters for personalized outputs. PWABench also provides three memory retrieval Buscines-Randows Memory, Last Memory, and Relevant Memory-that sample differrnnt historical data to assist agents. To better reflect real-world user behaviors, we do not restrict users to only two preferences as in the conversational recommendation task. Instead, each user may have multiple active preferences with diverse and dynamically shifting strengths, posing greater challenges for accurate modeling and adaptation. (2) Evaluation Metrics: the evaluation metrics for the system include three key components: (a) Function Accuracy (F. Acc): evaluates the agent's 's ability to select the correct web function and parameters, scoring for correct selection and 0 otherwise;
(b) Response Accuracy (R. Acc): for search and recommendation tasks, this metric uses the rank r of the target product within the returned product list the performance indicator and is calculated as:
[1 0,
r-1
if 10
R. Ace
if 10
and (c) Average Steps (Avg. Steps): measures total actions needed to complete a task, where fewer steps indicate higher efficiency.
5.3.2 Results Analysis
In the personalized web interaction task, accurately understanding user intent requires distinguish- ing subtle differences in prefrence-driven behaviore g.. discerning whether user prefers to search for a product or directly receive recommendation. This challenge is particularly pronounced in single-turn interactions, where the agent lacks rich dialogue context. As shown in Table AdaPA- Agent achieves the best overall performance across all services, reaching the highest overall function accuracy (F. Ace: 0.851) and response accuracy (R. Acc: 0.367). In particular, AddaPA-Agent signif- icantly outperforms prior methods recommendation-rrllateee interactions (F. Ace: 0.592), setting where user intent is often implicit and preference strength is most influential. Compared to RecMind (0.226) and Random Memory (0.296), AdaPA-Agent improves recommendation accuracy by over 30 absolute points, showing its ability to infer the dominant preferences even with sparse interaction.
As shown in Table EL in the multi-turn track, user feedback from multiple rounds is utilized to refine the model's personalized output, leading to performance improvements across all methods compared to the single-turn track. However, AdaPA-Agent stands out not only in achieving higher function accuracy and response accuracy than the baseline methods but also in maintaining relatively small average steps. This highlights the efficient preference modeling capabilities of AdaPA-Agem as it

---

does not rely on excessive user feedback. While methods like Reflexion and ReAct, which depend on user feedback to update their understanding of user preferences. require more steps on average.
These results affirm the core advantage of AdaPA-Agent: by modeling preference strengths adap- tively, it delivers both more accurate and more efficient personalized servicesespecially in settings where feedback is limited or user intent is ambiguous.
5.4 Case Study
We conduct study examine how Agent-Agent dynamically adjusts to user preferences in a conversational movic recommendation scenario with two competing preferences: light-hearted comedy and emotional movies. As shown in Figure GL across different user utterances, the pref- erence weights inferred by AdaPA-Agent shift in a consistent and interpretable manner, accurately reflecting subtle changes in user intent. For instance, when the user mentions wanting story that really moves me, the alignment score for the emotional preference sharply increases, prompting matching recommendation. In contrast, when the user seeks something fun, the model shifts weight toward the light-hearted preference. This illustrates two key strengths of our approach: (1) the ability to capture nuanced preference signals through diinmeet-based estimation without explicit feedback, and (2) the controllability of generation via preference- weeightee decoding. allowing the agent to produce responses that align precisely with the user's evolving priorities. This example highlights how AdaPA-Agen enables fine- fine-grained and interpretable adaptation in real-time person- alized interactions.
5.5 Additional Experiments
The appendix provides extensive ablation studies (Appendix D)) that further validate the effective- ness of AdaPA-Agent in modeling dynamic preference sfrengths. These analyses demonstrate that (1) dual-side augmentation substantially enhances robustness and fine-grained preference estimation (Appendix D_.), (2) the proposed LLM-based alignment scorer more accurately captures semantic alignment between user preferences and interactions compared to embedding- based methods (Ap- pendix D22)), and (3) the continuous preference arithmetic formulation enables more expressive and controllable generation than prompt-based methods (Appendix D.3).
6 Conclusion
We present AdaPA-Agent, a training-frre framework for enhancing LLM-based agents through dynamic preference modeling. AdaPA-Agent addresses two key challenges in personalization: (i) estimating the relative strength of user preferences without relying on explicit user feedback, and (ii) effectively incorporating these strengths into the generation process. To solve the first challenge, we introduce Alignment -Bseeeenn---dddddddee Strength Estimation, which leverages dual-side augmentation and an I.L.-based alignment scorer infer fine-grained preference weights from implicit user behaviors. To solve the second challenge, we propose Controllable Personalized Generation, which utilizes the estimated strengths via preference arithmetic control the personalized generation process. Our experiments on two personalized agent tasks cookssscovrrstionnall recommendation and personalized web agent-demonstrate that AdaPA-Agent consistently improves alignment with users' evolving intents and priorities by modeling dynamic preferences.
Acknowledgement
This work was supported in part by the National Science Fund for Distinguished Young Scholars under Grant 62025602; in part by the National Natural Science Foundation of China under Grant U22B2036, and Grant 11931015; in part by the Technological Innovation Team of Shaanxi Province (No. 2025RS-CXTD-009) the International Cooperation Project of Shaanxi Province (No. 2025GH- YBXM-017); in part by the Fok Ying-Tone Education Foundation, China, under Grant 171105; in part by the Fundamental Research Funds for the Central Universities under Grant D5000230366: in
part by the Fundamental Research Funds for the Central Universities under Grant G2024WD015L, and in part by the Tencent Foundation and Xplorer Prize. Y. Wang is sponsored by Beijing Nova Program. Q. Yao is sponsored by CCF-Zhipu Large Model Innovation Fund (No. Zhipu202402).
10

---

References
[1] Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, lige Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. GPT-4 technical report. arXiv preprint arXiv:2303.08774, 2023.
[2] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhari- wal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. In Advances Neural Information Processing Systems, pages 1877-1901, 2020.
[3] Xin Luna Dong, Seungwhan Moon, Yifan Ethan Xu, Kshitiz Malik, and Zhou Yu. Towards next-generation intelligent assistants leveraging LLM techniques. In ACM SIGKDD Confer- ence on Knowledge Discovery Data Mining, pages 5792-5793, 2023.
[4] Yuanchun Li, Hao Wen, Weijun Wang, Xiangyu Li, Yizhen Yuan, Guohong Liu, Jiacheng Liu, Wenxing Xu, Xiang Wang, Yi Sun, al. Personal LLM agents: Insights and survey about the capability, efficiency and security. arXiv preprint 2xXi:24411005552 2024.
[5] Jian Xie, Kai Zhang, Jiangjie Chen, Tinghui Zhu, Renze Lou, Yuandong Tian, Yanghua Xiao, and Yu Su. TravelPlanner: A benchmark for real-world planning with language agents. In International Conference on Machine Learning. 2024.
[6] Zhankui He, Zhouhang Xie, Rahul Jha, Harald Steck, Dawen Liang, Yesu Feng, Bod- hisattwa Prasad Majumder, Nathan Kallus, and Julian McAuley. Large language models as zero-shot conversational recommenders. In ACM International Conference on Information and Knowledge Management, pages 720-730, 2023.
[7] Gangyi Zhang. User-centric conversational recommendation: Adapting the need of user with large language models. In ACM Conference on Recommender Systems, pages 1349-1354, 2023.
[8] Alireza Salemi, Sheshera Mysore, Michael Bendersky, and Hamed Zamani. LaMP: When large language models meet personalization. In Annual Meeting of the Association for Compu- tational Linguistics, pages 7370-7392, 2024.
[9] Joel Jang, Seungone Kim, Bill Yuchen Lin, Yizhong Wang, Jack Hessel, Luke Zettlemoyer, Hannanch Hajishirzi, Yejin Choi, and Prithviraj Ammanabrolu. PERSONALIZED SOUPS: Personalized large language model alignment via post-hoc parameter merging. In Advances in Neural Information Processing Systems Workshop Adaptive Foundation Models, 2023.
[10] Jiongnan Liu, Yutao Zhu, Shuting Wang, Xiaochi Wei, Erxue Min, Yu Lu, Shuaiqiang Wang, Dawei Yin, and Zhicheng Dou. LLMs+ Persona-Pluge Personalized LLMs. arXiv preprint arXiv:2409.11 1901, 2024.
[11] Alireza Salemi, Surya Kallumadi, and Hamed Zamani. Optimization methods for personal- izing large language models through retrieval augmentation. In International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 752-762, 2024.
[12] Qijiong Liu, Nuo Chen, Tetsuya Sakai, and Xiao-Ming Wu. ONCE: Boosting content- bontent-based recommendation with both open-and closed-soures large language models. In ACM Interna- lional Conference Web Seurch and Data Mining, pages 452-461, 2024.
[13] Hanjia Lyu, Song Jiang. Hanqing Zeng, Yinglong Xia, Qifan Wang, Zhang, Ren Chen, Chris Leung, Jiajie Tang, and Jiebo Luo. LLM-Rec: Personalized recommendation via prompting large language models. Findings of the Association for Computational Linguistics: NAACL, pages 583-612, 2024.
[14] Yunjia Xi, Weiwen Liu, Jianghao Lin, Xiaoling Cai, Hong Zhu, Jieming Zhu, Bo Chen, Ruim- ing Tang, Weinan Zhang, and Yong Yu. Towards world recommendation with knowledge augmentation from large language models. ACM Conference on Recommender Systems, pages 12-22, 2024.

---

[15] Chuan Shi, Zhiqiang Zhang, Ping Luo, Philip S Yu, Yading Yue, and Bin Wu. Semantic path based personalized recommendation on weighted heterogeneous information networks. In ACM International Conference on Information and Knowledge Management, pages 453-462, 2015.
[16] Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. ReAct: Synergizing reasoning and acting in language models. In International Confer- ence on Learning Representations, 2023.
[17] Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Re- flexion: Language agents with verbal reinforcement learning. In Advances Neural Informa- lion Processing Systems, pages
2023.
[18) Simran Arora, Avanika Narayan, Mayee F Chen, Laurel Orr, Neel Guha, Kush Bhatia, Ines Chami, and Christopher Re. Ask Me Anything: A simple strategy for prompting language models. In International Conference on Learning Representations, 2023.
[19] Jasper Dekoninck, Mare Fischer, Luca Beurer-Kellner, and Martin Vechev. Controlled text generation via language model arithmetic. In International Conference on Learning Represen- tations, 2024.
[20] Kostas Hatalis, Despina Christou, Joshua Myers, Steven Jones, Keith Lambert, Adam Amos- Binks, Zohreh Dannenhauer, and Dustin Dannenhauer. Memory matters: The need to improve long-term memory in LLM-agents. In AAAI Symposium Series, pages 277-280, 2023.
[21] Xixi Wu, Yifei Shen, Caihua Shan, Kaitao Song, Siwei Wang, Bohang Zhang, Jiarui Feng, Hong Cheng, Wei Chen, Yun Xiong, et al. Can graph learning improve planning LLM- based agents? In Advances in Neural Information Processing Systems, 2024.
[22] Yanchu Guan, Dong Wang, Zhixuan Chu, Shiyu Wang, Feiyue Ni, Ruihua Song, and Chenyi Zhuang. Intelligent agents with LLM- process automation. In ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pages 5018 1008-5027,00 2024.
[23] Murray Shanahan, Kyle McDonell, and Laria Reynolds. Role play with large language models. Nature, 623(7987):493 498, 2023.
[24] Yunfan Shao, Linyang Li, Junqi Dai, and Xipeng Qiu. Character-LLMM: trainable agent for ooly-playing. In Conference on Empirical Methods in Natural Language Processing, pages 13153- 13187, 2023.
[25] Jason Wei, Xuezhi Wang. Dale Schuurmans, Maarten Bosma, Fci Xia, Ed Chi, Quoc V Lc, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. In Advances in Neural Information Processing Systems, pages 14824-24831 2022.
[26] Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of Thoughts: Eliberate problem solving with large language models. In Advances in Neural Information Processing Systems, pages 11809 -1822, 2023.
[27] Hongru Cai, Yongqi Li, Wenjie Wang, Fenghin Zhu, Xiaoyu Shen, Wenjie Li, and Tat-Seng Chua. Large language models empowered personalized web agents. In The Web Conference, 2024.
[28] Kausik Lakkaraju, Sara Jones, Sai Krishna Revanth Vuruma, Vishal Pallagani, Bharath Muppasani, and Biplav Srivastava. LLMs for financial advisement: A fairness and efficacy study personal decision making. In ACM International Conference on Al in Finance, pages 100-107, 2023.
[29] Md. Ashraful Islam, Mohammed Eunus Ali, and Md Rizwan Parvez. MapCoder: Multi-agent code generation for competitive problem solving. In Annual Meeting of the Association for Computational Linguistics, pages 4912-4944, 2024.
[30] Jiabao Fang, Shen Gao, Pengjie Ren, Xiuying Chen, Suzan Verberne, and Zhaochun Ren. multi-agent conversational recommender system. arXiv preprint arXiv:2402.0711 2024.
12

---

[31] Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. Neural collaborative filtering. In International Conference on World Wide Web, page 173182, 2017.
[32] Xiaoyu Zhang, Ruobing Xie, Yougang Lyu, Xin Xin, Pengjie Ren, Mingfei Liang, Bo Zhang, Zhanhui Kang, Maarten de Rijke, and Zhaochun Ren. Towards empathetic conversational recommender systems. In ACM Conference on Recommender Systems, pages 84-93, 2024.
[33] Ge Gao, Alexey Taymanov, Eduardo Salinas, Paul Mineiro, and Dipendra Misra. Aligning LLM agents by learning latent preference from user edits. In Advances in Neural Information Processing Systems, 2024.
[34] Mingye Zhu, Yi Liu, Lei Zhang, Junbo Guo, and Zhendong Mao. On-the-fly preference align- ment via principle-guided decoding. In International Conference on Learning Representations. 2025.
[35] Jerry Zhi- Yang He, Sashrika Pandey, Mariah L.. Schrum, and Anca D. Dragan. Context steer- ing: Controllable personalization at inference time. In International Conference on Learning Representations, 2025.
[36] Minhyeon Oh, Seungjoon Lee, and Jungseul Ok. Comparison- based active preference learn- ing for dineniddimeniional personalization. In Annual Meeting of the Association for Computa- tional Linguistics, pages
2025.
[37] Alex Wilf, Sihyun Lee, Paul Pu Liang, and Couis-Philippe Morency. Think twice: Perspective- taking improves large language models' theory-of offiry-of-mind capabilities. In Anmual Meeting of the Association for Computational Linguistics, pages 8292-830S, 8308, 2024.
[38] Yancheng Wang, Ziyan Jiang, Zheng Chen, Fan Yang. Yingxue Zhou, Eunah Cho, Xing Fan, Yanbin Lu, Xiaojiang Huang. and Yingzhen Yang. RecMind: Large language model pow- ered agent for recommendation. In Findings of the Association for Computational Linguistics: NAACL, pages 4351-4366 2024.
[39] Xu Huang, Jianxun Lian, Yuxuan Lei, Jing Yao, Defu Lian, and Xing Xie. Recommender Al agent: Integrating large language models for interactive recommendations. arXiv preprint arXiv:2308. 16505, 2023.
[40] Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The Llama 3 herd of models. arXiv preprint arXiv2240..21783, 2024.
[41] Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. GPT-4o system card. arXiv preprint arXiv:2410.21276, 2024.
[42] Aixin Liu, Bei Feng, Bin Wang, Bingxuan Wang, Bo Liu, Chenggang Zhao, Chengqi Dengr, Chong Ruan, Damai Dai, Daya Guo, et al. DeepSeek- V2: A strong, economical, and efficient mixture-of-experrs language model. arXiv preprint arXiv:2405.0444:4 2024.
[43] Jafar Afzali, Aleksander Mark Drzewiecki, Krisztian Balog, and Shuo Zhang. Usersimers: A user simulation toolkit for evaluating conversational recommender systems. In ACM Interna- tional Conference on Web Search and Data Mining, pages 1 1160-1 1163, 2023.
[44] Xiaolei Wang, Xinyu Tang, Xin Zhao, Jingyuan Wang, and Ji-Rong Wen. Rethinking the eval- uation for conversational recommendation in the era of large language models. In Conference on Empirical Methods in Natural Language Processing, pages 10052-10065, 2023.
[45] Haozhe Xu, Xiaohua Wang, Changze Lv, and Xiaoqing Zheng. Beyond single labels: Im- proving conversational recommendation through llm-powered data augmentation. In Annual Meeting of the Association for Computational Linguistics, pages 15573-15590, 2025.
[46] Gustavo Adolpho Lucas de Carvalho, Simon Ben Igeri, Jennifer Healey, Victor S. Bursztyn, David Demeter, and Lawrence Birnbaum. A flash in the pan: Better prompting strategies to deploy out-of-the-box lmss as conversational recommendation systems. In International Conference Computational Linguistics, pages 8385-8398, 2025.
13

---

NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We have discussed the claims made in the abstract and introduction in the main paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We have discussed the limitation of our work in the Appendix.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The au- thors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, c.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the ap- proach. For example, facial recognition algorithm may perform poorly when image resolution is low images taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to ad- dress problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers grounds for rejection, worse outcome might be that reviewers discover limitations that aren't it acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
14

---

Justification:
This paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theo- rems.
The proof' can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be comple- mented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclu- sions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: We have provided the implementation details about our method in the main paper.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken make their results reproducible verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others replicate the model with the same dataset, or provide access to the model. In general. releasing code and data often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of model checkpoint, other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all sub- missions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution primarily new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to re- produce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
15

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification:
We have provided an open link for our code and dataset.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (httpp::/nnis..o cc/
for more details.
While we encourage the release code and data, we understand that this might not be possible, so No is acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https:)
/ /mips.cc/publii/gguideee Coodeddeeeee www.eessummssiiooooooiooooo ssiocyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
We have provided the details of our experimental setting in the main paper.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question:
Does the paper report error bars suitably and correctly defined or other appropri- ate information about the statistical significance of the experiments?
Answer: [Yes]
Justification:
We have provided an error bar for our results.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
16

---

The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report I-sigma error bars, but one should state it. The authors should prefer- ably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of eπrоrs is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: We have provided the details of experiments compute resources about GPU using and training time.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't 't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics htttpp///eaarrssscc/ppb/////teaa
Answer:
[Yes]
Justification:
We have reviewed the NeurIPS Code of Ethies and our paper conforms to it.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is a special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We have discussed the broader impacts of our work in the Appendix.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact spe- cific groups), privacy considerations, and security considerations.
17

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitiga- tion strategies (e.g., gated release of models, providing defenses addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g.. pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Our paper has no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by re- quiring that users adhere usage guidelines or restrictions access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
We have cited the original paper that produced the code package.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name the license (c.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
assets are released, the license, copyright information, and terms use in the pack- age should be provided. For popular datasets, paperswi thcode com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the cense of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
18

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documenta- tion provided alongside the assets?
Answer: [NA]
Justification: We have introduced the details our new dataset and provided an open link for our code and dataset.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodll as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
For crowdsourcing experiments and research with human subjects, does the pa- per include the full text of instructions given to participants and screenshots, if applicable, well as details about compensation (if any)?
Question:
Answer: [NA]
Justification: Our paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but the main contri- bution of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, cura- tion, or other labor should be paid at least the minimum wage the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: Our paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA
that the paper does involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equiva- lent) may be required for any human subjects research. you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
19

---

16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [Yes]
Justification: We use LLMs for editing (e.g., grammar, spelling, word choice), data pro- cessing/filtering. and writing.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our L.M policy Suttps://neure http://arrppcpccc///eeeeeeeeee// / for what should or should be described.
20

---

A Prompts for Alignment-Baee Strength Estimation
The prompt I is used for preference-side augmentation, key step in estimating dynamic preference strengths. It constructs structured preference chains that represent user preferences across three semantic levels: from abstract concrete. The model is instructed to think step by step and output the structured result strictly in JSON format.
Prompt 1: Reasoning Augmentation
Your goal is to help an Al agent better understand user preferences for personalized response generation.
Given the following user preference description:
{USER_ PEEFERRENEE
Your task is to construct a structured preference chain with three levels of semantic granularity.
Please think step by step and provide: raw: the original high-level form the preference
refined: a -ware, clearer reformulation the raw preference
3. example: representative items, actions, or behaviors that reflect the preference
Output the result in the following JSON format:
"raw": "<fill in raw preference>", "refined": "<fill in refined preference>", "example": ["<item 1>". "<item 2>", "..."]
This prompt is used for interaction-side augmentation, key step in estimating dynamic preference strengths. It instructs the L.LM to generate K paraphrased versions given user-agent interaction Dr. maintaining the same structure and intent but varying the surface-level expressions. This helps expand the range of observable user behaviors, reduces scoring bias from any single expression, and enables more robust alignment between user intent and preferences during strength estimation.
Prompt 2: Intuiton Augmentation
Please generate [K] semantically equivalent but lexically diverse conversations based on the following user-agent interaction:
User-Agent Interaction:
(DN) )
Instructions:
Return only the (K) generated interactions, each preserving the number of and speaker roles.
Keep the user's underlying intent consistent original conversation. Use varied wording and sentence structures to enhance linguistic diversity.
This prompt is designed for use in the alignment scoring stage of framework, where an LLM asked to assess how well given user-agent interaction reflects specific user preference, The input includes a structured preference chain (comprising raw, refined, and example-level deseriptions) and the current interaction between user and agent. The LLM is instructed to reason step-by-step interpret the semantics of the preference, and judge the degree of alignment exhibited the dialogue. It outputs a single numerical alignment score ranging from 0 to 10, where higher values indicate stronger semantic alignment. This score serves as the foundation for computing the relative strength of each preference in the final generation stage.
21

---

Algorithm Adaptive Preference Arithmetic (AdaPA-Agent
Require: Accumulated user interaction data Dj up time t, User preference set 7 = {pr )
Chain-of-thouge generator CoT(-),
Interaction paraphraser g(-),
Alignment scorer :)
Ensure: Personalized agent response O
1: ## 1: Alignment-Baan Based Strength Estimation
2: A - g(D)) U }
# Interaction-side augmentation
3: for
1 to . do
4: C, + (p))
Preference-side augmentation # Alignment scoring
5: 8i - A' EDEA" f(C;, D)
6: end for
7: Normalize: w;
for 1:M
# Strength weighting
Σ
2--1%
8: ## 2: Controllable Personalized Generation
9: Initialize: 0y <- while o EDS do
# Empty response buffer
P mix < 1 LLM((OP, C)).,)
12:
P P mix
~
## Sample next token ## Update response prefix
13: ON [O?,
14: end while
15: return O
Prompt 3: Measuring Correlation Score
You are an alignment evaluator tasked with assessing how well a user-agent interaction aligns with given preference chain.
Given:
User-Agent Interaction: (D
Preference Chain: (C,), which includes raw description, refined version, and representative examples of specific user preference.
Instructions:
1. Think step by step.
2. Judge how strongly the interaction reflects the intent or semantics of the given preference.
Provide single alignment score between 0 and 10:
perfectly aligned (preference clearly implied or reflected)
completely unrelated
4. Only return the numerical score.
B Algorithm for AdaPA-Agent
Algorithm I1 presents the core procedure of Adaptive Preference Arithmetic (AdaPA-Agent), which progressively adjusts the influence of user preferences based on the accumulated interaction context to generate personalized responses. The algorithm operates in stages that jointly enable adaptive, context-aware preference control.
1. Alignment-Base Strength Estimation (Lines 1-7): At each interaction step t, the agent updates its internal record of user interactions D. To enrich contextual diversity, it performs interaction-side augmentation by paraphrasing the interaction history via g(-). forming an expanded set A (Line 2). For each user preference P3¹ E pu, the agent gen- erates reasoning traces using a CoT generator (Line 4), then evaluates the semantic align- ment between each preference trace and the augmented interaction set using the alignment scorer f(,, -) (Line 5). The resulting alignment scores (s,) are normalized into preference
22

---

weights }w;} (Line 7), representing each preferences contextual relevance within the ongo- ing dialogue. This ensures that the model adaptively reflects preference shifts as user intent evolves over multiple turns.
2. Controllable Personalized Generation (Lines 8-15): Initialized with an empty output prefix (Line 9), the agent generates response token by token. At each step, it combines next-token distributions weighted by the computed w; across all preferences (Line 11), samples the next token (Line 12), and appends it to the output (Line 13), repeating until the end-of-sequence (EOS) token is reached.
By continually re-estimating preference strengths and integrating them into generation, AdaPA- Agent achieves contexl-aware, progressive adaptation across interaction turns. This allows the agent maintain alignment with evolving intent without requiring retraining explicit feedback, re- sulting in more personalized and consistent dialogue behavior.
Prompt 4: LLM-Ban M-Basedd User Simulator for Conversational Recommendation
You will play the role of interacting with conversational movie recommendation system. Your task is find movie that matches your current taste, which influenced by your preferences.
Role & Behavior Guidelines:
Engage naturally with the agent by gradually revealing your preferences.
Focus only on requesting or evaluating movie suggestions based on your preferences.
Never mention the name of your target movie.
Task Information:
In this task, your preference is: <prefer_innee info].
In this task, your target movie is: paarge__item]]
Simulation Rules:
Start with vague intent (e.g.. "I want to watch something meaningful").
2. Reveal preference
the agent asks follow-up questions.
3. Accept recommendations if they match the target movie.
4. Politely reject unrelated and give vague but helpful feedback (e.g.. "That's not quite what looking for").
5. Maintain natural and preference-driven tone throughout.
simulate a movie enthusiast is exploring potential movie recom- mendations, not to reveal the exact title of the target movie-(target, iteem: Keep the conversation natural and engaging, and always focus requesting recommendations giving feedback based on the suggestions you receive.
IMPORTANT: Your
C Details of Conversation Recommendation Task Construction
To carefully evaluate our method's ability to model preference strength, we categorize user movie preferences into two types: long-term and short-term. This distinction allows for controlled evalua- tion of how well the method adapts to shifts in preference strength. The dataset of the task is based on Reddit-Movie [[6], and we extract stable preferences from user's historical data as long-term pref- erences and generate corresponding movie list. We select several movies unrelated to long-term preferences and regard their features as the user's short-term preferences.
Furthermore, we create dynamic simulation environment where the agent interacts with a LLM- based user simulator. For each recommendation task, the user simulator randomly selects which type of preference (long-term or short-term) will dominate the interaction, allowing us to control the variation of preference strength. This enables us to directly manipulate and evaluate the model's re- sponse to dynamic shifts in user preferences, providing clear validation framework for our method. To ensure the reliability of this simulation setup, our user simulator design follows common prac- tices in recent literature [33] H4, 45, A6]. Specifically, we inject user attributes and interaction rules into prompt templates, and the LLM dynamically generates responses based on the accumulated dialogue context, enabling evolving user behavior over turns. We construct the user simulator for
23

---

the conversation recommendation task through the following prompt template (Prompt 4). To avoid exposing the target movie, we insert the rule that the LLM should not mention the target movie in the conversation. Besides, we use regular expressions in the code to detect and mask any acci- dental exposure (replacing with "*") during the interaction between the L.L. agent and the user simulator. Although our simulator may not perfectly replicate real users, it provides a controlled, consistent, and adaptive framework for evaluating the effectiveness of preference modeling strate- gies. To reduce generation randomness, For each reported result Table II we ran trials with different random seeds and computed the standard deviation across these runs.
set the LLM temperature to 0 during all evaluations.
D Ablation Studies of AdaPA-Agent
D.1 Effectiveness Dual-side Augmentation
To assess the contribution of dual-side augmentation, we compare three variants of AdaPA-Agent in pref- erence strength estimation: using both preference- and interaction-side augmentation (w/ D-side), using only preference-side augmentation (w/ P-side), and without any augmentation (w/o D-side). date dual-side augmentation, we simplified the con- versational recommendation and personalized web interaction tasks. In the former, classification to predict whether user's desired movie aligns with long-term or short-term preferences. In the latter, we use three-class classification to de- termine if the required service relates to search, rec- ommendation, or review. As shown in Figure ! the dual-side setup achieves the highest win in both tasks77.3% in conversational recommenda- tion and 63.8% in personalized web interactionsignif- icantly outperforming the other variants. These re- sults highlight that both augmentation dimensions are critical: while preference-side augmentation improves
18.2%
28.3%
vali-
63.8%
77.3%
Conversational Recommendation Personalized Web Interaction
use binary
Figure 4: Win rates of AdaPA-Agent on using dual-side data augmentation (w/ D- side), only preference-side augmentation (w/ P-side) and without using any data augmentation (w/o D-side) in preference strengths estimation.
K
Recall
Precision
F1-Score
0
25.32
70.21
37.22
1
43.73
66.70
52.83
semantic richness, interaction-side augmentation en- hances robustness against linguistic variability.
2
56.27
72.43
63.34
3
68.79
75.86
72.15
4
87.50
90.32
88.89
side introduces bias spar- sity, whereas combining both provides more compre- hensive signals for modeling fine-grained preference strengths. By contrast, dual-side augmentation enables AdaPA-Agent capture both semantic intent and lin- guistic variability, significantly improving the accu- racy of preference strength estimation.
Relying solely
5
78.13
80.64
79.35
71.88
71.88
71.88
Table 4: Effect of varying the number of generated user-agent interactions (K) on the binary classification results of the con- versational recommendation task.
Building upon the experimental setup in Figure HL we further investigate the effect of interaction- side augmentation dynamic preferences modeling. As shown in the Table AL K represents the amount of data augmentation, while recall, precision, and Fl-score indicate the model's performance on binary classification for different levels of data augmentation. We observe that as K increases, the recall significantly improves, showing that more augmentation helps identify relevant samples. Similarly, precision and F1-score also increase, reaching their peak values at K 4, after which they start to decrease as K continues to rise. These results show that, the initial improvement stems from the augmented interaction data providing useful preference information, but excessive augmentation introduces noise which results in poor performance. The same phenomenon also occurs in the personalized web interaction task.
D.2 Analysis of LLM-based Alignment Scorer
To evaluate the effectiveness of estimating preference-interaction alignment scores, we compare our proposed LLM-based method with a traditional embedding-basse based bascline using Sentence-BERT.

---

Maximum Steps
Embedding-based
LLM-based (ours)
3
35.39
41.45
5
58.56
70.45
7
67.58
75.41
Table 5: Recommendation success rate (RSR) of conversational recommendation using either an embedding- based aligmment scorer (Sentence-BERTT) or our proposed LLM-based alignment scorer. The LLM- method consistently outperforms the embedding baseline across different maximum step limits, indicating its superior ability to estimate alignment between user preferences and user- agent interactions.
For the baseline, we encode both the preference description and the user-agent interaction with Centence-BERYY and treat the cosine similarity of the two embeddings the alignment score. As shown in Table E the LLM--bssed alignment scorer consistently outperforms the embedding-ben method across varying maximum step limits in the conversational recommendation task. This per- formance gap suggests that LLMs are better at capturing the nuanced alignment between user pref- erences and interaction trajectories. Unlike static embedding models, LLMs can perform contextual reasoning and infer implicit preference signals, leading to more reliable estimation of preference strength and, ultimately, more accurate and intention alligned recommendations.
Prompt 5: Prompt Engineering
Here the user's long-term and short-term preferences and their weights (0 where 0 means relevance and means the highest relevance):
User's long-term preference: ()
Long-term preference weight: User's short-term preference: ()
Short-term preference weight: {}
Please respond the user's query based on the long-term and short-term preferences and their weights.
User query; {
D.3 Effectiveness of Preference Arithmetic
Figure demonstrates the effectiveness of the pref- erence arithmetic method in generating personalized responses based on varying strength combinations (ws, Wz) of short-term and long-term preferences compared to the traditional prompting engineering approach. Both preference arithmetic and prompt engineering consider two type of preferences and their weights, while the former uses weights efficients in arithmetic, the latter regards them as prompts. We use Llama-2-7b-chat as the backbone for these two method, while use GPT-4o to generate the ground truth. The semantic similarity between the generated content and the ground truth is calcu- lated by Pantence-BERT) From the figure, is ev- ident that preference arithmetic consistently outper- forms prompting engincering with the ground truth. This indicates that adjusting cnfference-conditionn using preference strengths more effectively controls the influence of long- and short-term prefer- ences on the generated content. Moreover, the preference arithmetic method is more accurate in distinguishing between strength differences, particularly when strengths of long- and short-term preferences are close, such as in the (0.4, 0.6) strength pair. While the traditional prompting en- gineering method struggles with this, likely due to limitations of understanding of numerical information in text. This ability allows preference arithmetic to generate more consistent personal-
125
653
Strength Pairs Long- and Short-tam: Preferences
Figure 5: Performance comparison the preference arithmetic and prompting engi- neering methods in generating personalized responses based varying pairs of (ws, W.). The figure shows the alignment between gen- erated content and ground truth generated by GPT-4o.
of alignment
next-token distributions
25

---

ized responses compared to prompting engineering. The prompt engineering template is designed as follows (Prompt 5):
We further compare AdaPA-Agent with a baseline method, termed Semantic Strength Prompt- ing (SSP), where the strengths of preferences represented five discrete textual descriptors (weakest, weak, neutral, strong, strongest). We evaluate both methods on the Conversational Rec- ommendation Task, using Recommendation Success Rate (RSR) and Average Interaction Rounds (AIR) as metrics. As shown in Table D. AdaPA-Agent consistently outperforms the SSP base- line across all interaction settings. Although such discrete verbal schemes may appear expressive, they are inherently limited to finite number of strength categories, leading to coarse control gran- ularity. contrast, AdaPA-Agent models preference strengths as continuous values within [0, 1]. enabling infinitely many combinations of strength settings. This formulation allows more nuanced and personalized control over how preferences influence generation. Moreover, since verbal descrip- tors like weak strong can be semantically ambiguous and inconsistently interpreted by L.MSs, token-wise distribution mixing provides explicit and interpretable control by directly manipulating strengths over next-token distributions, thereby reducing ambiguity and semantic drift. Therefore, these results demonstrate that AdaPA-Agent achieves superior expressivity and controllability com- pared to discrete strength prompting, enabling more robust and interpretable preference-conditione generation.
Max Steps
Method
RSR
AIR
AdaPA-Agent
41.45
2.64
3
SSP
34.20
2.80
AdaPA-Agent
67.24
3.73
5
SSP
59.60
4.13
AdaPA-Agens
75.41
4.46
7
SSP
70.82
4.79
Table 6: Performance comparison between AdaPA-Agent and Semantic Strength Prompting (SSP) on the conversational recommendation task.
E
Experiments Compute Resources
In this work, all experiments are conducted on machine with NVIDIA A6000*2 GPUs, each GPU has 48G memory.
F Limitations
While framework shows promising results, it has several limitations that open directions for fu- ture work. First, our evaluation relies on simulated user interactions, which may not fully capture the nuance of real-world behaviorfuture work could include seman-in-the-lopp or live deployment stud- ies to validate robustness. Second, the use of multiple LLM calls for alignment scoring increases computational costs, suggesting the need for more efficient approximation techniques. However, given the ongoing decrease in API call costs, we believe the trade-off between API usage and per- formance improvement our method is worthwhile. Third, we focus on two domainsmovie recom- mendation and web interactionwhile extending the approach to broader tasks (e.g., education, health advice) remains unexplored.
G Broader Impacts
This work explores dynamic preference modeling for personalized LLM agents, which can enhance user experience in recommendation, web assistance, and interactive applications. By adaptively aligning with user intent without requiring explicit feedback, the proposed method supports more natural and efficient human-Al interactions. However, personalization systems must carefully designed to avoid reinforcing user biases, exposing sensitive preferences, or leading over-reliance on AT decisions. Future work should consider fairness, privacy, and user control the deployment of adaptive preference-aware agents.
26


Reviews:
[{"review_id": "BWvHXW4ad8", "reviewer": "Reviewer_HZJJ", "summary": "This paper proposes a new method for agent conversational recommendation considering dynamic preference strength.\nWhen interacting with an agent given the user’s preference history,  this paper investigates the case where the user’s preference strength over each dimension changes. \nIt (1) paraphrases the history preference sentences into their proposed format called preference-chain (so there are several chains for each user); (2) paraphrases the new user input into multiple versions; (3) uses an LLM to assign an alignment score between the history preference-chain and each paraphrase of new user input ; (4) prompts an LLM to generate the next token distribution given the LLM’s response’s prefix, the current conversation, the user’s history preference-chain; (5) weighted sum of the distribution given each history preference-chain.\nThe experiments are on movie recommendation agents and personalized web-service agents. \nThe method is training-free and performs well the the above 2 tasks.", "questions": "- Could you provide examples of the original prompted responses of each preference and the weighted generation during conversation?\n- Did you try comparing the proposed method with directly prompting the LLM with the preference chains and scores for conversation?\n- Could your method deal with cases where the current conversation preference contains dimensions that are not previously stated in history?", "limitations": "Yes.", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths\n- The method is training-free, making it easy to test / apply. \n- The description of the method is intuitive, and the illustrations are very helpful.\n\nWeaknesses\n- The usage of weighted sum for each output token distribution is not well justified. Why is this a good idea? In certain toy settings, can we theoretically understand what this is doing and why it is a good idea?\n- The method is not tested on a real-world dataset.\n- The writing is of the paper can be clearer.", "was_revised": false, "final_justification": "", "rebuttal": "> **W1.** The usage of weighted sum for each output token distribution is not well justified. Why is this a good idea? In certain toy settings, can we theoretically understand what this is doing and why it is a good idea?\n\n**A1.** Thank you for this insightful question. Our decoding rule,\n\n$$\nP_{\\text{mix}}(w_k)=\\operatorname{softmax}\\left(\\sum_{i} \\omega_i \\log Q_i(w_k)\\right),\n$$\n\ndirectly instantiates the **language model arithmetic** formulation proven in *Controlled Text Generation via Language Model Arithmetic* (ICLR 2024, \\[24] in our paper). Specifically, their Theorem 1 (“Weighted KL-Optimality”) shows that this distribution $P_{\\text{mix}}$ is the unique solution to the following optimization problem:\n\n$$\n\\min_{P}\\ \\sum_{i} \\omega_i\\, D_{\\text{KL}}\\left(P \\,\\|\\, Q_i\\right),\n$$\n\nwhere each $Q_i$ is a next-token distribution conditioned on one attribute (in our case, a user preference), and $\\omega_i$ represents the relative strength of that preference. $P_{\\text{mix}}$ is the **KL-optimal compromise** among all $Q_i$, balancing their influence according to $\\omega_i$. This ensures that stronger preferences steer generation more, while still retaining support from all attributes. Thus, our method is not a heuristic but a principled, closed-form solution to a well-defined objective.\n\nTo avoid redundancy, we did not re-derive this theory. We will add the above objective and citation to the main paper for clarity. Finally, Appendix D.3 empirically validates this formulation: decoding via $P_{\\text{mix}}$ yields more attribute-aligned outputs than prompt-level weighting. This highlights the practical utility of this theoretically justified approach.\n\n\n> **W2.** The method is not tested on a real-world dataset.\n\n**A2.** We thank the reviewer for raising this point. However, we believe this may stem from a misunderstanding. Our evaluation setup is designed to reflect **real-world user behavior** in both data sourcing and interaction design. We clarify this from two perspectives:\n\n**(1) Real-world data sources.**\nBoth datasets used in our experiments are constructed from **authentic, large-scale user-generated content**, ensuring that user histories, preferences, and item metadata come from **real-world interactions**:\n\n* For the **conversational recommendation** task, we use the **Reddit-Movie dataset** [1], which contains over 634k naturally occurring Reddit discussions. These are real dialogs where users actively **seek and share movie recommendations**, reflecting informal, diverse preference expressions.\n\n* For the **personalized web interaction** task, we adopt the **PersonalWAB benchmark**, which builds on the **Amazon Reviews 2023** dataset [2]. This dataset includes up-to-date real user reviews, purchase behaviors, and metadata from the Amazon platform, faithfully capturing consumer preferences.\n\n\n**(2) Dynamic evaluation.**\nInstead of evaluating on static corpora, we adopt a **simulated user-agent interaction loop**, aligning with recent evaluation practices [3–4]. This dynamic setup better mimics **real-world multi-turn personalization**, where agents must adapt over time.\n\nTo ensure the **behavioral realism of the user simulator**, we designed a fine-grained simulator prompt template (see Appendix C), which produces consistent and context-aware responses. Moreover, we use **regular expressions** in the code to detect and mask any accidental exposure of target items (replacing them with “*”). This ensures the simulation process remains faithful and robust.\n\nIn summary, while our evaluation involves simulated interaction, it is grounded in **real user data** and follows **realistic multi-turn interaction protocols**. We will revise the paper to clarify this and avoid further confusion.\n\n\n[1] Large Language Models as Zero-Shot Conversational Recommenders, CIKM 2023\n\n[2] Bridging Language and Items for Retrieval and Recommendation, arXiv 2024\n\n[3] UserSimCRS: A User Simulation Toolkit for Evaluating Conversational Recommender Systems, WSDM 2023\n\n[4] Rethinking the Evaluation for Conversational Recommendation in the Era of Large Language Models, EMNLP 2023\n\n\n> **Q1.** Could you provide examples of the original prompted responses of each preference and the weighted generation during conversation?\n\n**A3.** We thank the reviewer for the suggestion. To better illustrate how **preference arithmetic** integrates multiple user preferences during generation, we provide examples comparing:\n\n* (i) responses generated **using only a single preference** (either $\\text{pref}_1$: *light-hearted comedy* or $\\text{pref}_2$: *emotional movies*); \n* (ii) the **final weighted response** using our arithmetic decoding method based on estimated preference strengths $\\omega_i$.\n\nThe three cases below are adapted from our **Figure 3 case study**, re-ordered by descending $\\omega_{\\text{pref}_1}$, to highlight how the **blending of token distributions** reflects varying emphasis on light-heartedness vs. emotional depth.\n\n---\n\n**Case 1: Pref-1 dominant (strengths: Pref-1 0.82 | Pref-2 0.18)**\n\n* **User input:** \"I'm in the mood for something fun to watch tonight.\"\n* **Pref-1 only:**\n  *“Crazy Rich Asians” is a delightful comedy full of charm and laughs.*\n* **Pref-2 only:**\n  *You might enjoy “The Fault in Our Stars”—it’s emotional and heartfelt.*\n* **AdaPA-Agent:**\n  *“The Intern” is a heartwarming comedy with just the right touch of inspiration—perfect for a relaxing night.*\n\nAnalysis: The Result shows comedic tone with a subtle emotional undercurrent.\n\n---\n\n**Case 2: Mixed preferences (strengths: Pref-1 0.38 | Pref-2 0.62)**\n\n* **User input:** \"I want to watch something meaningful but not too heavy.\"\n* **Pref-1 only:**\n  *“Julie & Julia” offers a light and uplifting story with a humorous tone.*\n* **Pref-2 only:**\n  *“Marriage Story” is an emotionally intense and powerful film.*\n* **AdaPA-Agent:**\n  *“About Time” is heartfelt and thought-provoking, but still light and charming in tone.*\n\nAnalysis: The result balances emotional depth with a gentle, uplifting mood.\n\n---\n\n**Case 3: Pref-2 dominant (strengths: Pref-1 0.10 | Pref-2 0.90)**\n\n* **User input:** \"I want to see a story that really moves me.\"\n* **Pref-1 only:**\n  *“The Hangover” is a light-hearted comedy for a fun night.*\n* **Pref-2 only:**\n  *“The Pursuit of Happyness” is deeply moving and inspiring.*\n* **AdaPA-Agent:**\n  *I’d recommend “The Pursuit of Happyness”—it’s deeply emotional and truly inspiring.*\n\nAnalysis: The result closely aligns with the emotional preference.\n\n---\n\nThese examples demonstrate that our **preference arithmetic mechanism produces outputs that interpolate** between stylistic and content elements associated with each preference. As the strengths shift, the **tone, emotional intensity, and item choice** vary accordingly, validating that the method reflects fine-grained preference strengths in actual generation. We will include these examples in the Appendix to enhance transparency and interpretability.\n\n\n> **Q2.** Did you try comparing the proposed method with directly prompting the LLM with the preference chains and scores for conversation?\n\n**A4.** Thank you for pointing this out. Following this idea, we conducted additional experiments comparing our proposed **AdaPA-Agent** with a baseline that directly provides the **preference chains and their alignment scores** as a prompt to the LLM, which we denote as **chain+score**.\n\nWe evaluated both methods on **conversational recommendation** and **personalized web agent** tasks. The results are summarized below.\n\n---\n\n**# Conversational Recommendation Task**\\\n(RSR: Recommendation Successful Rate; AIR: Average Interaction Rounds)\n\n|Max Steps|Method|RSR ± std|AIR ± std|\n|-|-|-|-|\n|3| **AdaPA-Agent**|**41.45 ± 5.82**|**2.64 ± 0.08**|\n|| chain+score| 40.52 ± 6.27| 2.70 ± 0.08     |\n|5| **AdaPA-Agent**|**67.24 ± 4.41**|**3.73 ± 0.23**|\n|| chain+score| 65.68 ± 5.42| 3.82 ± 0.21     |\n|7| **AdaPA-Agent**|**75.41 ± 3.07**|**4.46 ± 0.36**|\n|| chain+score|73.15 ± 3.35|4.76 ± 0.29|\n\n---\n\n**# Personalized Web Agent Task**\n\n|Task Type|Method|F.acc|R.acc|Avg.Steps|\n|-|-|-|-|-|\n| single-turn|**AdaPA-Agent**|**0.851**| **0.367** |-|\n|| chain+score| 0.827| 0.358|-|\n| multi-turn| **AdaPA-Agent**| **0.917**|**0.386**|**3.592**|\n|| chain+score| 0.891|0.378|3.623|\n\n---\n\nThese results show that while **chain+score** is a strong baseline and performs comparably in low-interaction settings (e.g., Max Steps = 3), **AdaPA-Agent consistently outperforms it** as the number of interaction turns increases. This suggests that although **explicit prompting of preference information helps initially**, it becomes less effective when prompts grow longer and noisier—diluting the salience of the preference scores. By contrast, AdaPA-Agent’s **preference arithmetic mechanism offers a more stable and scalable control** over generation via token-level distributional blending, which maintains robustness as dialogue length increases.\n\nWe will include these results and analysis in the updated appendix to make the comparison more transparent.\n\n> **Q3.** Could your method deal with cases where the current conversation preference contains dimensions that are not previously stated in history?\n\n**A5.** Thank you for raising this important point. While for evaluation purposes we assume that the user’s preference set $P^u$ is extracted from historical interactions $H^u$, **our method is fully capable of handling newly emerged preferences in the current interaction $D_t^u$**.\n\nIn practice, we can extract additional preferences directly from $D_t^u$ by LLM and augment them into the preference set $P^u$ accordingly. Then, AdaPA-Agent can generate preference chains for these new preferences via Preference-side Augmentation and seamlessly integrated into the strength estimation."}, {"review_id": "z18e6WFe4J", "reviewer": "Reviewer_o322", "summary": "This paper proposes a way to personalize LLMs based on user-LLM interaction. The approach does not perform fine-tuning. It first extracts certain preferences from the user interactions (it is unclear to me how the preference $p^{raw}_i$ is extracted, but I am guessing it is by prompting LLMs on the interaction history). It then measures the weight of each preference chain by calling an LLM-based scoring function on the interaction and preference chain. Finally, the next-word probabilities are computed for each preference separately and then combined using the given weights. My understanding is that this process is repeated at each step of the user-LLM interaction.\n\nExperiments are conducted on domains from the personalized web agents task. Results show that the approach performs better than baselines like React and Reflexion.", "questions": "1. What is the inference time of this approach? Can you report how much slower is the AdaPa agent over the baseline greedy decoding agent?\n\n2. How is the original preference $p^{raw}_i$ computed? And does $p^u_i$ indicate the concatenation of $p^{raw}_i, p^{ref}_i, p^{exp}_i$ and are these done in one single LLM call?\n\n3. How will the baseline where instead of doing token-wise mixing, the list of preferences and a categorization of their weights (e.g., weakest, weak, neutral, strong, strongest) is provided to the LLM, and a single call is made to the LLM? This will reduce the inference time while giving more expressivity.", "limitations": "N/A", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths:**\n\n- User personalization is an important task and this approach provides a fine-tuning-free approach for personalizing LLM. Fine-tuning free is nice because fine-tuning is both data and compute-intensive. \n \n- The approach performs better than common baselines\n\n**Weakness:**\n\n- **Expressivity:** Mixing token-wise distributions may not be expressive. Instead, an approach that expresses each preference and its weight (such as strong/weak) in text and then prompts the LLM with it might be better as the LLM can then combine this knowledge in more expressive way.\n\n- **Inference Time:** The approach might make too many LLM calls per interaction. There is one call for score per user preference, and a few for extracting preference. Inference time matters because in practical applications, the agent cannot wait too long before replying. This would be even worse with heavier LLMs.\n\n- **Related work:** paper misses some related work discussion on personalizing LLMs such as:\n\n 1. Aligning LLM agents by learning latent preference from user edits, Gao et al. (NeurIPS 2024)\n 2. Pearl: Personalizing large language model writing assistants with generation-calibrated retrievers, Mysore et al., ACL 2024.", "was_revised": true, "final_justification": "Authors have provided some useful clarification. I will keep my score.", "rebuttal": "> **W1. & Q3.** **Expressivity Concern:** How will the baseline where instead of doing token-wise mixing, the list of preferences and a categorization of their strengths (e.g., weakest, weak, neutral, strong, strongest) is provided to the LLM, and a single call is made to the LLM? This will giving more expressivity.\n\n**A1.** We thank the reviewer for the insightful suggestion. While encoding preference strengths as textual categories (e.g., *strong*, *weak*) may seem more expressive at first glance, we argue that our **token-wise distribution mixing** approach offers **greater expressivity and controllability** in practice:\n\n1. **Expressivity:**\n   The verbal scheme relies on a **finite set of predefined strength categories**, limiting the expressiveness to coarse buckets (e.g., 5 levels). In contrast, our method uses **continuous strengths in $[0,1]$**, allowing for **infinitely many combinations** of preference strength. This enables more nuanced and personalized responses, as demonstrated in Figure 5 (Appendix D.3), where preference arithmetic consistently yields better performance across varying strength settings.\n\n2. **Controllability:**\n   Words like *weak*, *strong*, or *neutral* are **inherently ambiguous** and prone to inconsistent interpretation by LLMs—especially when multiple preferences and weight descriptors are packed into a single prompt. This may lead to semantic drift or hallucination ([23, 24] in our paper).\n   In contrast, our method treats strength scores as **strengths over next-token distributions**, enabling **explicit and linear control** over how different preferences influence generation. This reduces ambiguity and increases robustness without requiring the model to \"reason over\" natural language descriptions of weight.\n\nWe also conducted a comparison between our method and the suggested baseline (which we term Semantic Strength Prompting, SSP), where preferences are embedded in the prompt with strength descriptors (weakest, weak, neutral, strong, strongest). As shown below, AdaPA-Agent outperforms this approach across all interaction settings, which means our approach provides finer-grained expressivity and more stable control over preference influence than the suggested baseline:\n\n**# Conversational Recommendation Task**\\\n(RSR: Recommendation Successful Rate; AIR: Average Interaction Rounds)\n\n|Max Steps|Method|RSR ± std|AIR ± std|\n|-|-|-|-|\n|3| **AdaPA-Agent**|**41.45 ± 5.82**|**2.64 ± 0.08**|\n|| SSP| 34.20 ± 5.96| 2.80 ± 0.07     |\n|5| **AdaPA-Agent**|**67.24 ± 4.41**|**3.73 ± 0.23**|\n|| SSP| 59.60 ± 5.53| 4.13 ± 0.21     |\n|7| **AdaPA-Agent**|**75.41 ± 3.07**|**4.46 ± 0.36**|\n|| SSP|70.82 ± 3.46|4.79 ± 0.26|\n\n\n> **W2.** **Inference Time:** The approach might make too many LLM calls per interaction. There is one call for score per user preference, and a few for extracting preferences. Inference time matters because in practical applications, the agent cannot wait too long before replying. This would be even worse with heavier LLMs.\n\n**A2.** We appreciate the reviewer’s concern regarding inference efficiency. Indeed, our framework introduces **additional LLM calls during each interaction step**—mainly due to alignment-based preference strength estimation and dual-side augmentation. However, we would like to highlight a **practical trade-off** between **per-step latency** and **overall task efficiency**. As shown in our experiments (e.g., \"Maximum Steps = 3, 5\" in Table 1), **AdaPA-Agent achieves significantly higher Recommendation Successful Rate (RSR) with fewer Average Interaction Rounds (AIR)**. This indicates **more efficient overall user-agent interaction** despite increased per-step compute.\n\nMoreover, our framework is **training-free and agnostic to model size**, making it flexible to deploy with lightweight LLMs in latency-sensitive environments. In future work, we plan to explore approximation strategies (e.g., LLM caching, preference precomputation, or lightweight scorers) to further reduce runtime overhead while retaining personalization quality.\n\n> **Q1.** What is the inference time of this approach? Can you report how much slower is the AdaPa agent over the baseline greedy decoding agent?\n\n**A3.** We thank the reviewer for this question. We believe this refers specifically to the **decoding efficiency** of our **preference arithmetic method (Equation 5)** compared to standard greedy decoding.\n\nWe clarify that our method computes the next-token distribution by aggregating $M$ preference-conditioned token distributions. To do this efficiently, we implement the inference using **batched forward passes**: all $M$ prompts (each conditioned on a different preference) are processed together in a **single model call with batch size $M$**. This leverages modern GPU parallelism and avoids any need to run $M$ separate LLM instances.\n\nWe evaluated the **token-per-second (TPS)** decoding time of our method against standard greedy decoding, using **LLaMA-3.1-8B** on an **NVIDIA A100 GPU**. Results are summarized below:\n\n| Number of Preferences (M) | Greedy Decoding (TPS) | Preference Arithmetic (TPS) |\n| -| --| --|\n| 1| 79| 79|\n| 2| 79| 74|\n| 3| 79| 69|\n\nThe results show that our method introduces **minimal overhead** (10 tokens per second for $M = 3$) due to batching. Thus, **preference arithmetic decoding remains comparable in efficiency to greedy decoding**, while offering more expressive control over generation. Considering the increased memory consumption when batching over $M$ preferences, we also plan to explore speculative decoding as a future direction to further accelerate inference while reducing memory consumption.\n\n\n> **Q2.** How is the original preference $p_i^{raw}$ computed? And does $p_i^u$ indicate the concatenation of $p_i^{raw},$ $p_i^{ref}$, $p_i^{exp}$ and are these done in one single LLM call?\n\n\n**A2.** Thank you for your question. We are happy to clarify the preference extraction pipeline.\n\n* As described in Section 4.2 (Preference-side Augmentation), each user’s full set of preferences is denoted as $P^u$, which is extracted from the user’s historical interaction data $H^u$.\n* Each individual preference $p_i^u \\in P^u$ is then expanded into a **three-level structured preference chain** $C_i$, via a CoT prompting strategy:\n\n  $$\n  C_i = \\text{CoT}(p_i^u) = [p_i^{\\text{raw}} \\rightarrow p_i^{\\text{ref}} \\rightarrow p_i^{\\text{exp}}]\n  $$\n* Specifically:\n\n  * $p_i^{\\text{raw}}$ is the original, coarse-grained preference extracted directly from $p_i^u$,\n  * $p_i^{\\text{ref}}$ refines $p_i^{\\text{raw}}$ in a clearer, context-aware form,\n  * $p_i^{\\text{exp}}$ lists concrete examples that exemplify $p_i^{\\text{ref}}$.\n\nThis **entire chain is generated via a single LLM call** using the prompt shown in Appendix A, Prompt 1, which encourages step-by-step reasoning and outputs the structured chain in JSON format.\n\nWe thank the reviewer again for pointing out this question, and we will revise the text to clarify this process and fix the notation ambiguity.\n\n\n> **W3.** Paper misses some related work discussion on personalizing LLMs, such as: (1) Gao et al., NeurIPS 2024; (2) Mysore et al., ACL 2024.\n\n**A5.** We thank the reviewer for pointing out these relevant works. Both papers are conceptually related. We will include a discussion of both in the revised Related Work section."}, {"review_id": "tG6KgmmoSG", "reviewer": "Reviewer_xLrL", "summary": "This paper proposes an LLM-agent-based framework for inferring user preferences by aligning preference analysis with the user-system interaction context. Strength weights for different preferences are estimated using an LLM, which assesses how well each preference aligns with the current interaction context. These weights then guide the redefinition of the next-token distribution through a weighted sum of distributions conditioned on various preferences. Experiments conducted on two distinct tasks demonstrate performance improvements compared to selected baselines.", "questions": "All my questions are given in the weakness points.", "limitations": "Yes.", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n1. The paper is well-written, clearly presenting most methodology details.\n\n2. The proposed method is concise and user-friendly. The authors provide their code and data, facilitating reproducibility and futher research.\n\n3. Experimental are conducted on two representative personalization tasks, broadening its potential impact and applicability.\n\nWeaknesses:\n\n1. To my understanding, the effectiveness of the proposed method should largely depend on the extraction of users' existing known preferences, as both interaction alignment and distribution re-weighting hinge upon these preferences. However, the extraction methods for these preferences appear handcrafted and guided by intuitive empirical rules. For instance, while the paper states, \"we extract stable preferences from historical data as long-term and select unrelated movies to define short-term preferences,\" the exact methods of extraction and definition remain unclear and appear empirical and naive. Additionally, the influence of different preference definitions and extraction techniques on the performance of the proposed AdaPA-Agent is important but remains unexplored.\n\n2. Although experiments encompass tasks such as conversational recommendation and personalized web interactions, the selection of baselines neglects relevant works specifically targeting these domains. The paper currently compares mainly against general agentic reasoning approaches and general recommendation methods employing LLMs. Important recent studies on using LLMs in conversational recommendation systems, such as \"Collaborative Retrieval for Large Language Model-based Conversational Recommender Systems,\" \"Towards Empathetic Conversational Recommender Systems,\" and \"Large Language Models as Zero-Shot Conversational Recommenders,\" are strongly suggested to be included in both discussions and comparisons.\n\n3. Some experimental settings lack clarity or justification. For example, the paper does not clarify the number of simulated users in the conversational recommendation experiments. It is also unclear how standard variances, if presented, are calculated in Table 1 and why they are missing in Tables 2 and 3. Further, detailed configurations, such as the temperature settings for LLMs, should be clearly articulated in the appendix. Lastly, excluding user interaction records from simulator construction diverges from primary practices in existing literature, potentially limiting the realism and complexity of the simulated user behaviors.", "was_revised": true, "final_justification": "Although the authors’ rebuttal clearly illustrates several strategies for extracting users’ preferences across different datasets, I still feel that the methodology lacks integrity with the proposed work and is difficult to generalize to many recommendation scenarios. In addition, the user simulation techniques seem rather weak, especially in cases powered by LLMs. Therefore, I will maintain my current rating.", "rebuttal": "> **W1.** The paper’s method heavily relies on extracted user preferences, yet the extraction process appears empirical and under-specified. For example, how long- and short-term preferences are defined remains unclear. The impact of different extraction strategies on model performance is also not explored.\n\n\n**A1.** Thank you for raising this important point. We would like to clarify that **our primary contribution lies not in proposing a new method for extracting user preferences**, but rather in modeling **the dynamic strength variation of existing preferences** to enable adaptive personalization.\n\nAs stated in the Introduction, many existing works ([12-18] in our paper) focus on extracting or fitting user preferences from context. Our work builds on this foundation and targets the **under-explored problem of estimating and applying dynamic preference strengths** within multi-turn interactions, regardless of how the preferences were initially derived.\n\nWe agree that the **preference extraction pipeline should be more clearly described**, and we will revise the paper to clarify our process:\n\n* For the **conversational recommendation task**, we simulate long-term and short-term preferences to control evaluation granularity. Long-term preferences are mined from repeated user patterns in the Reddit-Movie history ([8] in our paper), while short-term preferences are injected by introducing a topically unrelated but semantically plausible interest (e.g., a sudden interest in horror films for a user who typically likes romantic dramas). This setup allows us to **explicitly evaluate strength adaptation** without relying on noisy implicit labels.\n\n* For the **Personalized Web Interaction task**, we **do not predefine specific dimensions** (e.g., long-term or\nshort-term). Instead, for each user $u$, we use an LLM to **summarize only two representative shopping preferences** from their real-world historical behavior (e.g., purchase or review records) in the PUMAbench dataset ([34] in our paper). This design helps to reduce potential noise and ensure the extracted preferences reflect shopping interests depending on the user's history.\n\nWe also agree that the impact of different preference extraction techniques is an important direction. While orthogonal to our contribution, we will consider adding ablation or sensitivity studies in future work.\n\n> **W2.** The paper currently compares mainly against general agentic reasoning approaches and general recommendation methods employing LLMs. Important recent studies on using LLMs in conversational recommendation systems, such as \"Collaborative Retrieval for Large Language Model-based Conversational Recommender Systems,\" \"Towards Empathetic Conversational Recommender Systems,\" and \"Large Language Models as Zero-Shot Conversational Recommenders,\" are strongly suggested to be included in both discussions and comparisons.\n\n\n**A2.** Thank you for pointing out these relevant studies. We agree that recent advances in LLM-based conversational recommendation systems are important and complementary to our work. However, we would like to clarify our **baseline selection criteria and experimental goals**.\n\nOur paper focuses on evaluating **training-free, general-purpose LLM agents** capable of modeling **preference strengths** across multiple personalized agent tasks (e.g., recommendation, web interaction), rather than designing or improving domain-specific recommendation architectures.\n\nRegarding the specific works mentioned:\n\n1. **“Collaborative Retrieval for LLM-based Conversational Recommender Systems”** and **“LLMs as Zero-Shot Conversational Recommenders”** focus on **retrieval-style recommendation**: given a textual query, the system ranks items from a catalog. These models typically treat LLMs as recommendation engines that return **ranked item lists**, and **do not perform open-ended, multi-turn natural language interaction** with users. In contrast, **our setting requires the agent to interact in natural language**, reason over evolving user preferences, and complete task-oriented dialogues. Hence, these systems differ substantially in formulation and cannot be directly evaluated under our setup.\n   \n2. **“Towards Empathetic Conversational Recommender Systems”** is a **fine-tuned supervised method**, requiring task-specific labeled data and training. In contrast, all our baselines (and AdaPA-Agent itself) are **training-free**, enabling direct plug-and-play usage across tasks and models. Including fine-tuned methods would create an unfair comparison due to differences in supervision and resource assumptions.\n\nWe will update the related work section to **explicitly discuss these methods and clarify their differences**, and we thank the reviewer for pointing them out. Including a broader discussion of these systems helps contextualize our contribution within the expanding space of LLM-based personalization research.\n\n\n\n> **W3.** Some experimental settings lack clarity or justification:\\\n> (1) The paper does not clarify the number of simulated users in the conversational recommendation experiments. \\\n> (2) It is also unclear how standard variances, if presented, are calculated in Table 1 and why they are missing in Tables 2 and 3. \\\n> (3) Detailed configurations, such as the temperature settings for LLMs, should be clearly articulated in the appendix.\\\n> (4) Excluding user interaction records from simulator construction diverges from primary practices in existing literature, potentially limiting the realism and complexity of the simulated user behaviors.\n\n\n**A3.** Thank you for this detailed and constructive feedback. We address each subpoint as follows:\n\n1. **Simulated user count in the conversational recommendation task:**\\\n   To ensure that each simulated user has sufficient historical data for preference extraction, we selected users from the Reddit-Movie dataset who had **at least three interaction trajectories**. This resulted in a total of **984 unique users** used for simulation in the conversational recommendation experiments. We will add this detail to Section 5.2.1.\n\n2. **Standard deviations in Tables 2, and 3:**\\\n   For Tables 2 and 3 (Personalized Web Interaction task), we report the **results of baselines from the original paper of personalized web agent  ([34] in our paper)**, which **did not report standard deviations**. For consistency with prior work, we follow the official evaluation method of the PersonalWAB benchmark in our main paper. We will clarify this in the captions.\n\n3. **LLM temperature settings:**\\\n   To reduce output randomness and ensure reproducibility, we set the **temperature to 0** for all baselines and components in our method (including preference chain construction, interaction paraphrasing, and decoding). We will add this configuration detail to the Appendix.\n\n4. **Use of user interaction records in simulator design:**\\\n   Our user simulator does incorporate evolving interaction context. Specifically, for each round, the cumulative user-agent interaction record $D_t^u$ is included as part of the **user simulator's input prompt** to guide behavior. Prompt 4 in the appendix shows only the **static system prompt template** (i.e., the simulator’s role definition), which is why $D_t^u$ is not explicitly shown there. We will clarify this in the revised prompt description and include a full example of a simulator input to improve transparency.\n\nWe thank the reviewer again for helping us improve the clarity of the experimental setup and will revise the relevant sections accordingly."}, {"review_id": "8xgzVfkuTx", "reviewer": "Reviewer_E4dG", "summary": "This paper introduces a novel framework for large language model (LLM)-based agent personalization called AdaPA-Agent, which achieves more fine-grained personalized response generation by modeling the dynamic strengths of user preferences. The authors point out that although existing methods can already capture the content of user preferences, they fail to effectively model how the relative strengths of these preferences evolve over time. To address this, they propose \"Adaptive Preference Arithmetic,\" which consists of two key modules. The first module, Alignment-Based Strength Estimation, automatically estimates preference strengths from existing user-agent interactions without requiring additional user feedback, using dual-side augmentation and an LLM-based alignment scorer. The second module, Controllable Personalized Generation, controls the generation process by linearly combining next-token distributions conditioned on different preferences, weighted according to their estimated strengths. The framework is evaluated on two tasks—conversational recommendation and personalized web interaction—and the results demonstrate that AdaPA-Agent outperforms baseline methods such as ReAct across multiple metrics, achieving better alignment with user needs when intentions change over time.", "questions": "Why is this weighting approach in Eq. 5 effective? Is it possible to find a theoretical basis behind it?", "limitations": "Although the \"Broader Impacts\" section is mentioned in the appendix, it is relatively brief and lacks in-depth analysis of potential risks such as privacy leakage, preference amplification bias, and algorithm misuse.", "rating": 2, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Stregthens\n1. The authors propose a training-free approach to align with dynamic human preferences for personalization, enabling better alignment with user needs when intentions change.\n2. AdaPA-Agent is a training-free method that is applicable to various LLM architectures, offering strong versatility and deployment flexibility.\n\nWeakness\n1. The method is overly intuitive and experience-based, lacking theoretical foundations.\n2. The method calculates preference scores through model computations, but the reliability of these scores from large language models has not been sufficiently validated.\n3. Using the CoT prompting approach, the reasoning time may become excessively long, negatively impacting the user experience.\n4. The paper does not explicitly address how the model balances or resolves conflicting preferences when users express contradictory ones, nor does it explore whether impulsive or erroneous user inputs may affect the final generation outcomes.", "was_revised": false, "final_justification": "", "rebuttal": "> **W1. & Q1.** The method is overly intuitive and experience-based, lacking theoretical foundations.\n\n**A1.** We appreciate the reviewer’s concern. While our work is application-oriented—focusing on modeling dynamic preference for agent personalization—it is grounded in a **well-established theoretical framework**. Specifically, our decoding rule,\n\n$$\nP_{\\text{mix}}(w_k)=\\operatorname{softmax}\\left(\\sum_{i} \\omega_i \\log Q_i(w_k)\\right),\n$$\n\ninstantiates the **language model arithmetic** formulation proven in *Controlled Text Generation via Language Model Arithmetic* (ICLR 2024, \\[24] in our paper). Specifically, their Theorem 1 (“Weighted KL-Optimality”) shows that this distribution is the unique solution to the following optimization problem:\n\n$$\n\\min_{P}\\ \\sum_{i} \\omega_i\\, D_{\\text{KL}}\\left(P \\,\\|\\, Q_i\\right),\n$$\n\nwhere each $Q_i$ is a next-token distribution conditioned on one attribute (in our case, a user preference), and $\\omega_i$ represents the relative strength of that preference. $P_{\\text{mix}}$ is the **KL-optimal compromise** among all $Q_i$, balancing their influence according to $\\omega_i$. This ensures that stronger preferences steer generation more, while still retaining support from all attributes. Thus, our method is not a heuristic but a principled, closed-form solution to a well-defined objective.\n\nWe chose not to re-derive the theoretical proof in our paper due to space and its availability in prior literature, but we will add the relevant formula and citation to clarify the theoretical underpinnings. Furthermore, Appendix D.3 empirically validates that this approach significantly outperforms prompt-based alternatives, especially under fine-grained strength variations. This highlights the practical utility of this theoretically justified approach.\n\n\n\n> **W2.** The method calculates preference scores through model computations, but the reliability of these scores from large language models has not been sufficiently validated.\n\n**A2.** Thank you for raising this important point. We agree that relying on LLMs for estimating alignment-based preference scores introduces questions about reliability. To address this, we conducted **extensive empirical validation** of our scoring method in the supplementary material:\n\n* In **Appendix D.2**, we directly compare our **LLM-based alignment scorer** with an embedding-based baseline (using Sentence-BERT). Across all settings in the conversational recommendation task, the LLM-based scorer **consistently achieves higher task success rates**, showing its capability to model nuanced alignment between preferences and user-agent interactions.\n\n* In **Appendix D.1**, we further evaluate the **dual-side augmentation** strategy, which enhances scoring robustness by enriching both preference and interaction representations. Results show that this significantly improves score reliability, especially in linguistically diverse or ambiguous contexts.\n\nOur approach leverages a combination of **semantically powerful LLMs and our carefully designed dual-side augmentation strategy** to extract fine-grained preference strength signals that are difficult to capture through traditional similarity measures. Empirical results (Tables 1–3) consistently demonstrate that these scores lead to stronger personalization performance across two distinct tasks.\n\n> **W3.** Using the CoT prompting approach, the reasoning time may become excessively long, negatively impacting the user experience.\n\n\n**A3.** We appreciate the reviewer’s concern about inference latency. Indeed, our framework introduces additional LLM calls per interaction, mainly due to the alignment-based preference strength estimation and dual-side augmentation modules. However, we would like to highlight an important **trade-off between per-step latency and overall task efficiency**.\n\nAs demonstrated in our experiments (e.g., Table 1, Maximum Steps = 3 and 5), **AdaPA-Agent consistently achieves higher Recommendation Successful Rate (RSR) with fewer Average Interaction Rounds (AIR)**. This means that despite longer single-step inference, the agent **requires fewer steps to complete the task**, resulting in improved overall responsiveness and user experience from a task-level perspective.\n\nMoreover, our method is **training-free and model-agnostic**, allowing practical deployment with **lighter-weight LLMs** in latency-sensitive environments. In addition, our method can be readily combined with optimization strategies such as **LLM response caching, preference precomputation, and lightweight scoring models**. We believe these directions can make AdaPA-Agent both responsive and highly personalized, preserving user experience even under real-world latency constraints.\n\n> **W4.** The paper does not explicitly address how the model balances or resolves conflicting preferences when users express contradictory ones, nor does it explore whether impulsive or erroneous user inputs may affect the final generation outcomes.\n\n**A4.** Thank you for highlighting this important point. Our current work focuses on **modeling and leveraging dynamic strengths of user preference based on observed user-agent interactions**, rather than validating the consistency or veracity of the user’s expressed preferences. Specifically:\n\n* **Conflicting preferences** (e.g., “I want something serious and light-hearted”) are handled by our system via **relative strength estimation**. The model does not explicitly attempt to resolve contradictions but instead estimates the alignment of each preference with the current interaction and applies corresponding strengths. This allows the model to reflect the most dominant signals in the current context without making normative judgments about consistency.\n\n* Regarding **impulsive or erroneous inputs**, our framework **treats all user-agent interactions as valid evidence** for estimating preference strength. Detecting misleading or accidental expressions is outside our current scope, and nd we view this as a promising direction for extension."}, {"review_id": "c0t2JDNHQN", "reviewer": "Reviewer_vYZm", "summary": "This paper proposes AdaPA-Agent, a framework for personalizing large language model (LLM) agents by dynamically estimating and leveraging the relative strengths of multiple user preferences. Rather than relying on explicit user feedback, AdaPA-Agent infers preference strengths from user-agent interaction history using a combination of dual-side augmentation (enriching both preference representations and interaction paraphrases) and an LLM-based alignment scoring mechanism. The estimated strengths are then used to guide the agent’s responses by linearly combining the next-token distributions conditioned on each preference. Experiments on conversational recommendation and personalized web interaction benchmarks show that AdaPA-Agent achieves higher personalization quality and efficiency compared to several strong baselines, without requiring additional model retraining or explicit user input.", "questions": "Please see weaknesses.", "limitations": "Yes.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n\n**S1.** The paper proposes a training-free framework that models dynamic user preference strengths. This approach is highly practical and resource-efficient as it does not require costly fine-tuning or model retraining, making it a flexible solution for adapting to user preferences in practice.\n\n**S1.** The paper proposes a novel approach to estimating preference strengths by combining dual-side augmentation (preference chain construction and interaction paraphrasing) with LLM-based alignment scoring. This enables the extraction of implicit preference signals from dialogue without requiring explicit user feedback.\n\n**S2.** The method demonstrates substantial and consistent performance gains over competitive baselines such as ReAct, Reflexion, and RecMind on both conversational recommendation (Reddit-Movie) and personalized web agent (PersonalWAB) benchmarks. Improvements are shown in both personalization quality (e.g., recommendation success rate) and efficiency (e.g., fewer interaction rounds).\n\n**Weaknesses**\n\n**W1. Adaptivity to dynamic preferences.** While Figure 3 and Section 5.4 show changes in inferred preference strengths, this does not demonstrate true adaptivity to dynamic preference shifts. In my understanding, the framework simply re-computes strengths from scratch at each interaction—functionally equivalent to applying a static estimator repeatedly—rather than maintaining or updating preferences in an adaptive way. As a result, the method passively reflects recent inputs instead of adaptively detecting or responding to preference shifts within a session. Could the authors clarify how AdaPA-Agent can *detect* or quickly adapt to user preference shifts within an *ongoing* multi-turn conversation, rather than simply re-estimating preferences after each isolated interaction?\n\n**W2. Omission of PUMA baseline.** While the paper uses the PersonalWAB [(Cai et al., 2024)](https://arxiv.org/abs/2410.17236) for evaluating personalized web agent performance, it does not include PUMA—the core approach introduced with PersonalWAB—in its baselines. This omission is not discussed or justified. Since PUMA is specifically designed for this benchmark and serves as a state-of-the-art method, a fair comparison requires evaluating AdaPA-Agent against PUMA on the same tasks. The absence of this baseline makes it difficult to assess true progress over prior work; the paper should either include direct comparisons to PUMA or clearly justify its exclusion.\n\n**Minor comments**\n\n**C1. Additional related works.** The controllable personalized generation proposed here resembles recent methods such as [Zhu et al. (2025)](https://arxiv.org/abs/2502.14204) and [He et al. (2024)](https://arxiv.org/abs/2405.01768), but the paper does not discuss similarities or differences. Clearer positioning relative to these works would strengthen the contribution. Also, recent studies on LLM personalization via preference strength estimation, such as [Oh et al. (2024)](https://arxiv.org/abs/2411.00524) and [Hwang et al. (2024)](https://arxiv.org/abs/2312.09337), could be included in Section 2.2 to provide more accurate context. Lastly, “LLM Personalization” seems a quite broad category; I recommend narrowing the scope for a more explicit and meaningful comparison.\n\n**C2. Robustness of CoT-based augmentation.** The CoT-based augmentation process relies on LLM-generated examples to concretize user preferences. However, these examples may not always align with the user’s true preferences (e.g., “prefer healthy food” may not always mean “prefer salads”), introducing spurious associations. Additional clarification, analysis, or robustness experiments regarding this aspect would strengthen the reliability and stability of the proposed framework.\n\n**C3. Ambiguity in preference dimensions.** While the manuscript states that the authors considered \"multiple active preferences\" for web tasks, it does not specify the exact number or types of these preferences. For the paper to be self-contained, it is beneficial to clearly describe the preference dimensions used in the experiments.\n\n**References**\n\n- Zhu et al. (2025). *On-the-fly Preference Alignment via Principle-Guided Decoding*\n- He et al. (2024). *Context Steering: Controllable Personalization at Inference Time*\n- Oh et al. (2024). *Comparison-based Active Preference Learning for Multi-dimensional Personalization*\n- Hwang et al. (2024). *Promptable Behaviors: Personalizing Multi-objective Rewards from Human Preferences*\n- Cai et al. (2024). *Large Language Models Empowered Personalized Web Agents*", "was_revised": true, "final_justification": "The authors' responses have successfully resolved most of my initial concerns. However, I am maintaining my current score because the manuscript, in its present form, is not sufficiently self-contained. I believe a major revision is necessary to integrate crucial details into the main paper, thereby improving its completeness and reproducibility. I am willing to reconsider my evaluation upon seeing a revised version that incorporates these changes.", "rebuttal": "> **W1.** **Adaptivity to dynamic preferences.** The method passively reflects recent inputs instead of adaptively detecting or responding to preference shifts within a session. Could the authors clarify how AdaPA-Agent can *detect* or quickly adapt to user preference shifts within an *ongoing* multi-turn conversation, rather than simply re-estimating preferences after each isolated interaction?\n\n**A1.** Thank you for the insightful comment. We would like to clarify a possible misunderstanding about the adaptive mechanism in our framework.\n\nWhile our Alignment-Based Strength Estimation module does re-compute preference strengths at each interaction step, this **re-estimation is based on the full accumulated user-agent interaction records within the session.** \nIn other words, the preference strength estimation is continuously updated in response to the **evolving multi-turn conversation**, not based solely on isolated inputs. This design allows the model to **adaptively reflect preference shifts** as they emerge across turns, rather than relying on static or fixed preference encodings.\n\nWe briefly outline the adaptive process below for clarity:\n\n1. For the task at time $t$, as the number of interaction rounds increases, new interaction data will be accumulated in $D_t^u$.\n2. The updated $D_t^u$ is used to re-compute the strength of each preference $p_i^u \\in P^u$ through dual-side augmentation and LLM-based scoring (see Section 4.1).\n3. The re-computed strengths $\\omega_i^{(t)}$ directly guide the generation of the next agent response.\n\nThis mechanism ensures that the model does not merely reflect recent utterances, but rather performs **context-aware, progressive adaptation** throughout the session. We will revise Algorithm 1 to make this aspect of the framework clearer.\n\n> **W2.** **Omission of PUMA baseline.** While the paper uses the PersonalWAB for evaluating personalized web agent performance, it does not include PUMA—the core approach introduced with PersonalWAB—in its baselines. This omission is not discussed or justified.\n\n**A2.** Thank you for pointing this out. We acknowledge the importance of comparing against strong baselines, and we would like to clarify why **PUMA was not included in our evaluation**.\n\nOur goal in this paper is to evaluate and compare **agent methods for preference modeling that are training-free and generalizable across multiple personalized agent tasks**. Toward this goal, we deliberately selected **baseline methods that do not rely on fine-tuning or benchmark-specific supervision**, ensuring fair and consistent evaluation in both conversational recommendation and personalized web interaction settings.\n\nThe exclusion of PUMA is motivated by the following two reasons:\n\n1. **Fairness in evaluation:**\n   PUMA is a **fine-tuned model tailored specifically for the PersonalWAB benchmark**, and both PUMA (GPT-4o) and PUMA (LLaMA-7B) variants involve supervised fine-tuning. For instance, they fine-tune LLaMA-2-7B to identify the correct web function, and PUMA (LLaMA-7B) further applies DPO to optimize parameter generation of functions. In contrast, **our method—as well as all selected baselines like ReAct, Reflexion, RecMind, etc.—are training-free** and do not rely on any task-specific gradient updates. Comparing training-free methods directly with benchmark-specific fine-tuned models would not be methodologically fair.\n\n2. **Generality across domains:**\n   PUMA is **tightly coupled to the PersonalWAB benchmark** and is not designed to generalize to other personalized agent tasks. In contrast, our proposed AdaPA-Agent framework is task-agnostic and has been evaluated across **multiple scenarios**, including both personalized web interaction and conversational recommendation, without task-specific adaptation. As such, our focus is on **cross-task generality and dynamic preference modeling**, which PUMA does not support.\n\nIn summary, we deliberately focus on **training-free, plug-and-play personalization frameworks** that are more practical in real-world multi-task deployments. We will add a clarification in the paper to explicitly justify the omission of PUMA in the baseline comparison.\n\n> **C1.** **Additional related works.** The paper lacks discussion of related works on controllable generation (e.g., Zhu et al., He et al.) and preference strength modeling (e.g., Oh et al., Hwang et al.). A clearer positioning and narrower scope within “LLM Personalization” would improve the contribution.\n\n**A3.** We thank the reviewer for the valuable suggestions. We briefly summarize the relevant works and clarify how our approach differs:\n\n* **Zhu et al. (2025)** and **He et al. (2024)** propose controllable text generation methods for LLM that rely on **explicit preference instructions** to guide generation.\n* **Oh et al. (2024)** infers preferences through **multiple rounds of explicit user feedback**, while **Hwang et al. (2024)** models preference-conditioned reward adjustment for **robot policy control**.\n\nIn contrast, our method targets **LLM-agent personalization**, with the goal of **dynamically modeling and leveraging preference strength** based solely on observed user-agent interactions—**without requiring additional user feedback/instruction**.\n\nWe agree that these works are technically relevant, and we will revise **Section 2.2** to include a **focused discussion** of their similarities and differences. We also appreciate the reviewer’s suggestion to narrow the scope of the \"LLM Personalization\" category and will adjust the structure accordingly to improve clarity and positioning.\n\n> **C2.** **Robustness of CoT-based augmentation.** The CoT-based augmentation process relies on LLM-generated examples to concretize user preferences. However, these examples may not always align with the user’s true preferences (e.g., “prefer healthy food” may not always mean “prefer salads”), introducing spurious associations. Additional clarification, analysis, or robustness experiments regarding this aspect would strengthen the reliability and stability of the proposed framework.\n\n**A4.** Thank you for this insightful observation. We agree that relying on LLMs to expand preferences from abstract to concrete expressions introduces the risk of unintended or overly narrow interpretations. However, our **design of CoT-based preference chains is intended to mitigate this** by:\n\n1. **Expanding preferences at multiple semantic levels** (raw → refined → examples) rather than relying on a single point interpretation (Appendix A, Prompt 1). This allows the model to **capture a broader and more nuanced representation** of user intent, rather than reducing it to a single canonical form like “salads.”\n\n2. We **generate a diverse set of representative examples** (i.e., multiple concrete items per preference) to avoid overfitting to any one interpretation. For example, as shown in Figure 2, a preference like “healthy food” generates a list such as [grilled chicken, salads, avoids greasy], improving robustness. More examples are illustrated in Figure 3.\n\n3. In **Appendix D.1**, we show through ablation experiments that Preference-side augmentation (\"P-side\" in Figure 4) **significantly improves accuracy of preference type prediction** across both tasks. This empirically supports the effectiveness of using the CoT-based preference chain.\n\n> **C3.** **Ambiguity in preference dimensions.** While the manuscript states that the authors considered \"multiple active preferences\" for web tasks, it does not specify the exact number or types of these preferences. For the paper to be self-contained, it is beneficial to clearly describe the preference dimensions used in the experiments.\n\n**A5.** Thank you for pointing this out. We agree that clearly specifying the preference setup can improve clarity and self-containment.\n\nAs stated in **Section 5.1.2 Implementation Details (lines 235–237)**, for both the **conversational recommendation** and **personalized web interaction** tasks, **we construct two preferences per user (i.e., $M = 2$)** to serve as the input preference set $P^u$.\n\n* In the **conversational recommendation** task, we **manually simulate** one **long-term preference** (extracted from historical movie interaction data) and one **short-term preference** (constructed by injecting a transient interest). This setup is designed to evaluate the model’s ability to adapt to shifts in dominant preference strength.\n\n* In the **personalized web interaction** task, we **do not predefine specific dimensions** (e.g., long-term or\nshort-term). Instead, for each user $u$, we use an LLM to **summarize only two representative shopping preferences** from their real-world historical behavior (e.g., purchase or review records) in the PUMAbench dataset. This design helps to reduce potential noise and ensure the extracted preferences reflect shopping interests depending on the user's history.\n\nWe will revise the manuscript to make these details more explicit in the main text, ensuring that the paper remains self-contained."}]

Author rebuttal:

Meta-review: This paper introduces a training-free framework for LLM agent personalization that models dynamic preference strengths. The key idea is to estimate how user preference weights shift over time based on past interactions—without requiring additional feedback—and then guide controllable generation by linearly combining next-token distributions conditioned on these dynamic preferences. Reviewers highlighted the paper’s methodological originality and practical significance. Unlike fine-tuning-based approaches, the authors propose a lightweight, flexible solution that can be easily applied across tasks and architectures. This training-free property lowers barriers to deployment in real-world settings where user data and compute are limited. Reviewer o322 provided a balanced and constructive assessment based on their deep expertise, maintaining a "Borderline Accept" rating throughout the review process. Other reviewers raised some concerns about i) the reliance on simulated interactions, ii) ad-hoc preference extraction methods, and iii) missing domain-specific comparison points, including those that rely on supervised finetuning like PUMA. Despite these concerns, the submission makes a timely contribution in adaptive personalization methods that balance efficiency, expressivity, and robustness. In their revision, I advise the authors to include a discussion on whether LLMs might handle preference combination more naturally through language understanding rather than arithmetic operations will benefit subsequent research that builds on this work.
Decision: Accept (poster)