HYDRA: Model Factorization Framework for Black-Box LLM Personalization
Authors: Yuchen Zhuang
Haotian Sun
Yue Yu
Rushi Qiang
Qifan Wang
Chao Zhang
Bo Dai
Venue: NeurIPS 2024 poster (2024)
Keywords: LLM
LLM Personalization
Black-Box LLMs
Primary area: natural_language_processing
TL;DR: We introduce a model factorization method for black-box LLM personalization.

Abstract: Personalization has emerged as a critical research area in modern intelligent systems, focusing on mining users' behavioral history and adapting to their preferences for delivering tailored experiences. Despite the remarkable few-shot capabilities exhibited by black-box large language models (LLMs), the inherent opacity of their model parameters presents significant challenges in aligning the generated output with individual expectations. Existing solutions have primarily focused on prompt design to incorporate user-specific profiles and behaviors; however, such approaches often struggle to generalize effectively due to their inability to capture shared knowledge among all users. To address these challenges, we propose HYDRA, a model factorization framework that captures both user-specific behavior patterns from historical data and shared general knowledge among all users to deliver personalized generation. In order to capture user-specific behavior patterns, we first train a reranker to prioritize the most useful information from top-retrieved relevant historical records.
By combining the prioritized history with the corresponding query, we train an adapter to align the output with individual user-specific preferences, eliminating the reliance on access to inherent model parameters of black-box LLMs. Both the reranker and the adapter can be decomposed into a base model with multiple user-specific heads, resembling a hydra. The base model maintains shared knowledge across users, while the multiple personal heads capture user-specific preferences. Experimental results demonstrate that \method outperforms existing state-of-the-art prompt-based methods by an average relative improvement of 9.01% across five diverse personalization tasks in the LaMP benchmark.

HYDRA: Model Factorization Framework for Black-Box LLM Personalization
Yuchen Zhuang', Haotian Sun', Yue Yu', Rushi Qiang', Qifan Wang", Chao Zhang', Bo Dai
Georgia Institute of Technology, 2 Meta Al
{yczhuang, haotian.sun, yueyu, rqiang6, chaozhang-ogatech.eett wgfcråfb. com, bodai@ec. podai@cc.gttehh.co gatech.edu
Abstract
Personalization has emerged critical research area in modern intelligent sys- tems, focusing on mining users' behavioral history and adapting to their preferences for delivering tailored experiences. Despite the remarkable few-shot capabilities exhibited by black-bor box large language models (LLMs), the inherent opacity their model parameters presents significant challenges in aligning the generated output with individual expectations. Existing solutions have primarily focused on prompt design to incorporate user-specific profiles and behaviors; however, such approaches often struggle to generalize effectively due to their inability to capture shared knowledge among all users. To address these challenges, we propose HYDRA, a model factorization framework that captures both user-specific behavior patterns from historical data and shared general knowledge among all users deliver per- sonalized generation. In order to capture user-specific behavior patterns, we first train reranker to prioritize the most useful information from tep-retrieved relevant historical records. By combining the prioritized history with the corresponding query, we train an adapter align the output with individual user-specific prefer- ences, eliminating the reliance access to inherent model parameters of black-box L.MSs. Both the reranker and the adapter can be decomposed into base model with multiple user-specific heads, resembling hydra. The base model maintains shared knowledge across users, while the multiple personal heads capture user- specific preferences. Experimental results demonstrate that HYDRA outperforms existing state-of-the-art prompt-based methods by an average relative improvement of 9.01% across five diverse personalization tasks in the LaMP benchmark. Our implementation is available at https: : :/gi /ggtttu....... com/no com/night hightt chen/ ChDRAH
Introduction
Pre-trained large language models (LLMs) [35, 36, 2, 31, 4, 30] have revolutionized various natural language processing (NLP) tasks, ranging from traditional recommendation systems [62, 12, 8, 6] to modern virtual assistants [9, 23, 63, 66, 44, 65, 47]. Despite their strong capabilities, LLMs require further customization to consistently demonstrate desirable behaviors to each user and achieve optimal performance in specific use cases [48, 43, 61, 10, 13, 14, 24, 57, 58, 45, 59]. As a result, LLM personalization has emerged emergent abilities of LLMs to meet the unique needs of individual users.
rapidly evolving area of research [20], with the goal of tailoring the
Several existing studies have shown effectiveness in personalizing LLMs, including (1) fine-tuning personalized LLMs for each user [50, 54] and (2) aligning LLMs to personalized preferences through Reinforcement Learning from Human Feedback (RLHF) [22. 55, 17J. However, both fine-tuning and RLHF- based methods require access to model parameters, restricting their use to white-box LLMS only (e.g., LLaMA-2 [51]). These models tend to be less capable than black-box LLMs (e.g., GPT-3.5
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

User-Specific
User Sier-SpeiFFic
Shared User-Spacifal
Shared User-Specific
Prompt
Heads
Base Model
Reranker
Adap ter
(a) White-Box LLMs
(b) Black-Box LLMs
(c) Hydra for Black-Box LLMs
Figure I: Personalization in white-box and black-box LLMs. Existing methods prioritize (a) fine- tuning user-specific models in white-box L.MM personalization, while (b) designing user-specific prompts for black-box LLM personalization. In HYDRA, we present (c) a learning based model factorization solution to enhance the effectiveness of personalization in black-box LLMs. indicates the trainable parameters, whereas M indicates the inaccessible fixed parameters.
[30]) because they have access to less training data and smaller model scales. Moreover, RLHFF-based methods require more explicitly attributed characteristics (e.g., style) [22, 17] from implicit user behavior history and necessitate excessive annotation efforts for capturing human preferences.
Without access to modify the model parameters for black-box LLM personalization, an alternative solution is to augment user-specific content and/or context into the prompt template. One straight- forward approach is to incorporate the user's complete profile or entire historical behavior into the prompt design [6. 18, 25, 52, 64, 5, 64]. However, integrating the entire profile may exceed the length limitations of LLMs and lead to substantial costs, while randomly selected records cannot effectively capture representative patterns. To address this dilemma, retrieval-augmented generation (RAG) approaches [41, 40, 29, 21] have been explored by extracting the most relevant information from the user's historical data to facilitate personalized generation. One limitation is that a retrieval- -aggeeeted framework encodes different users independently in a personalized prompt, making it challenging to capture the shared (global) patterns of the entire user group [50]. Moreover, in comparison to fine- tuning the entire or partial model parameters to create personalized language models for individual users [50, 22, 17], simply augmenting the input prompt through a centralized LLM without updating model parameters may diminish the effectiveness of personalization (Figure 1).
To address these challenges, we propose HYDRA, a learning- based model factorization framework that captures both user-specific and shared behavior patterns to enable effective personalization within black-box LLMs. HYDRA leverages a retrieval-augmeente workflow, where retriever initially extracts relevant user behaviors from historical data for effective user-specific preference identification. To achieve personalized generation, we focus on the training process of two fundamental components: (1) personalized reranker to prioritize useful user information from the retrieved records, and (2) a personalized adapter to align black-box LLM outputs with user-specific preferences, without requiring access to internal model parameters. Both the reranker and the adapter can be decomposed into base model with multiple personalized heads, similar to a Hydra. By employing model factorization, we effectively integrate shared (global) knowledge, captured by the centralized base model, with user-specific preferences, harnessed through multiple user-specific heads, to enhance generalization across the entire user group.
We conduct extensive experiments on LaMP [41], comprehensive language model personaliza- tion benchmark, to evaluate the personalization capabilities of HYDRA across multiple dimensions, including three text classification tasks and two text generation tasks. Notably, HYDRA achieves an average improvement of 4.8% % over the best-performing baselines across all five diverse tasks. Further in-depth studies reveal the robust capability of HYDRA in scaling up to accommodate larger user groups and extensive behavior history, as well as adapting to user behavior shifts. We also demonstrate the effectiveness of shared knowledge in enhancing user experience through both quantitative and qualitative analyses. To facilitate future research in black-box LLM personalization, we will release the code repository and model checkpoints for transparency and reproducibility.
Our main contributions are as follows: (1) We propose HYDRA, a black-box LLM personalization framework that effectively mines user behavior history and adapts to user preferences for enhanced user experience; (2) HYDRA integrates shared (global) knowledge from the base model and individual (local) preference from multiple user-specific heads through model factorization to deliver gener- alizable personalization; and (3) HYDRA significantly outperforms existing prompt-based methods across five diverse tasks in the LaMP benchmark [41], introducing novel learning-based solution that achieves more effective adaptation to individual users in black-box LLMs.

---

2 Related Works
In-Context Learning. Vanilla personalized prompting approaches leverage the powerful in-context learning capability of LLMs by using the user's randomly sampled behavior history contextual samples. Existing studies [6, 18, 25, 52, 64] have utilized encoding user histories-whether personal ratings, interaction histories, generating personalized content in various down-stream applications. Additionally, research has shown that utilizing longer user history can potentially lead to better performance [5, 64].
exemplary reviews-as few-shot examples to facilitate LLMs in
Profile-Aumeetted Prompting. Improving upon the random sample strategy and leveraging the insights from the enhanced performance with more historical information, vrofile-augmente gener- ation (PAG) summarizes user preferences and behavior patterns into natural language profiles for query augmentation. For instance, Richardson et al. [38] employ instruction-ture LLMs to generate abstract summaries of user history data, integrating summarization for enhanced personalization. Similarly, ONCE [26] creates user profiles by summarizing topics and regions of interest from their browsing history, thereby assisting LLMs in capturing user preferences for downstream tasks.
Detrieval-Augmennee Prompting. Compared to random sampling in in-context learning and the use of entire histories in PAG, retrieval- -uggentee prompting excels at extracting the most relevant records from user behavior history to enhance LLM personalization, thereby efficiently managing the growing user behavior data within LLMs' limited context length and supporting personalized generation with more relevant evidence. For instance, LaMP [41, 40) introduces a retrieval-augne -ugmeeted method to obtain the most relevant content from the user's behavioral history and incorporate it into the prompt design. Similarly, AuthorPred [21] retrieves relevant past user-written documents for personalized text generation. Pearl [29] proposes generation-calibatte retriever to select from historic user-authored documents, enhancing personalization with relevant prompt augmentation.
Limitations. Similar to in-context learning. personalization based on PAG prone be easily distracted by irrelevant information retrieved, especially when there is shift in user behavior between the current query and the user's 's historical records. Despite the improvement upon PAG by retrieving relevant information, RAG-based methods may still suffer from the quality of retrieved information, where the "most relevant" information may not necessarily be the "most useful" information to answer a new query. Additionally, prompting- -based methods not only lack deeper personalization analysis due to their reliance on a single centralized model but also lack access to global knowledge due the user-specific prompt design.
3 HYDRA: Model Factorization for Black-Box LLM Personalization
3.1 Problem Formulation: Black-Box LLM Personalization
Black-box LLM personalization refers to tailoring model generations align with user preferences according to their history [20, 50, 17]. without having access to model parameters. Specifically, given a black-box LLM G and training dataset Dirain = {q,,, Tn, H.)), }. where for each user u, qu indicates the input sequence, Ta refers to the target output, and H., represents the user's historical behavior containing preference information. The user history data H., ( } includes all user behaviors hi consisting of (g', ri) pairs, mirroring the task-specific query-answer format (YurTw)- The goal of personalization is adapt the LLM's generation the target output r, conditioned on both the input and the user history.
Traditional fine-tuning methods train universal models for all users, whereas personalized tuning aims to develop a unique model for each user u, denoted 0(w), which captures the unique characteristics of its user-specific data distribution D(w), Idcally, each user model should possess both shared (general) and user-specific knowledge. Thus, the problem can be formulated as a decomposition of ((a) into a shared part a and a personalized part 7(1) to learn general and user-specific knowledge, respectively. The training object of LLM personalization can be formulated as:
U
min a,1(1) (1) (9)
Σ 7w)
(N)
where (o; (w)); D(w) denotes the loss function of user u, and U the total number of users. Specifically, in the black-box LLM personalization, the model parameters in G' are not accessible,
3

---

Top-K
Hydra
User 8
Usefulness
Adapter
User
Reranker
Personalized Inference
User
Propose
Query Gx
Retrioved
Mugnented Querles
Black-Box LLM
History
Training and z(
Freeze and Fit
Inference with o and r(*)
Ye' Seneration
Query
Base
Model
Headers p(o)
Base
Model
Headers y(o')
Base
Model
Headers
r(w))
Figure 2: Overview of HYDRA. HYDRA follows a retrieval-augmmeneed framework: (1) Firstly, we extend
an original RAG two-stage srrreve-tthnn-eraane workflow, where we rerank the most useful information from relevant user behavior records to capture user-specific preference (Section 3.2); (2) Secondly, augmented by the selected historical data, we train adapter align the output of black- box LLMs with personalized human preference (Section 3.3). Both the reranker and the adapter can be decomposed into a base model with multiple user-specific heads, resembling a hydra-like structure (Section 3.4). The base model maintains shared knowledge across users, while multiple personal heads capture user-specific preferences. represents the model decomposition for personalization.
making it infeasible to directly fine-tune the black-box LLM. Thus, we present HYDRA, model factorization framework for black-box LLM personalization, shown in Figure 2.
3.2 Retrieve-then-Reree
To capture user-specific preference, we follow a retrieve-then-rerume workflow to (1) retrieve the relevant user behavior records and (2) rerank them based on usefulness. Specifically, given an input query employ a retriever to retrieve top-N user history behaviors that have the highest relevance scores when compared with the input æ. The objective of the reranker to identify user historical records that can serve useful user preference information for answering the user's incoming queries.
Training Data Generation. Within the training data, each user only has single query, which is insufficient for capturing the relationships between the behaviors of the same user. Scoring all the history pairs of users is quadratic in Σ12 H.ul [Hul, which becomes prohibitively time- consuming. In order to obtain a high-quality candidate set of training samples for each user query ga- we retrieve the top-M relevant history records, denoted R(qu. H..). Additionally, gain better understanding of the user's history, we randomly sample another M historical records that can be considered as the user's previous queries. For each of these previous queries, we also retrieve the top-M relevant historical records. The training candidates can be represented as:
Erennner {(g,, Ti, E Hllgu, R(ga. Ha))) U (h R(h Hu/h",)) Jis Eng uED.
Enemeer
(2) where q: indicates the query, T; indicates the ground-truth answer the query, and h; indicates the candidate history. We then utilize an LLM as a labeling function to measure the potential improvement (i.e., usefulness) of each history upon the LLM personalized generation. Specifically. for each candidate (q.. h2). we first sample the generation of LLM f; by using the input context z and the candidate history h; from (f (g. h.). Next, we compare the generation with the trouh answer T;, and create binary label yi = 1(;; = r:). For generation task, the condition soften to y: 1(; R rz), where the Rouge metrics between r; and i is above a pre-defined threshold. To optimize the reranker,
combine the user query with the candidate history in an entailment learning style, resulting in the training inputs D; = {[CLS] [ q [SEP] h, [SEP]]. Therefore,
curate training set for the reranker D Preaaner
{(24, y1) [Pandee
(aanker

---

HYDRA- HYDR4-Reranker Training. Training the reranker for each user allows for a personalized selection of relevant historical information based on the user's query. The training objective involves using the cross-entropy loss between the predictions made by HYDRA-Reranker heads and the ground truth. For specific user and the corresponding training sample (x v.) E peranker, we can calculate the cross-entropy loss function for HYDRA- as follows:
HYYRR = y. log p. (1 - v/") log(1 P²")
where p." indicates the model prediction made by
HYDRA-Reranker Inference. With the trained reranker, our objective is to select the top-k most relevant candidates from the retrieved user history for personalized generation. For each query qi from user in the test data, we retrieve the most relevant history H. { _ and
concatenate them form the test inputs {x N..N). By feeding these into HYDRA-Reranker we obtain the corresponding predictions {pi,1 ".,.
] From the test inputs, we select the top-k history candidates C₁ with the highest level of usefulness:
C, = arg top-k,
k.. EH, (p%
3.3 Black-box LLM Adaptation
Complementary to personalizing the few-shot demonstrations for LLMs in Section 3.2, we further align the black-box LLM generation to user-specific preference through the training of a personalized adapter, eliminating the need for directly accessing model parameters.
Training Data Generation. To take full advantage of the preference information hidden in the user query and history, we generate the candidates for the adapter training:
Edapteer = {4%, r ) = {qu, Tu FuED U {((g r [2al JuED.
(5)
For each input query q: from the training dataset, we augment the reranked history candidates with the query and sample candidate responses from the black-box LLM:
( PF q:).
(6)
In comparison with the ground-truth personalized generation r,, we can evaluate each generated solution fr, and assign corresponding binary label y/s as ys (frr, =r). Using the model generations, we establish new dataset for the adapter training, denoted as Dndaper = { } ) [Paapal where Ti,j = [[CLS) q: [SEP] fz,y [SEP]} represents the concatenation of the user query with the entire candidate generation.
HYDRA-Adapter Training. We utilize the model decomposition training strategy (detailed in Sec- tion 3.4) to train personalized adapters for each user, which allows us to tailor the selection of candidate generation that best aligns with the user's preference. To calculate the task-specific loss function, we follow Eq. (3) and employ the same cross-entropy loss between the predictions of user-specific heads and the ground truth to optimize MYDRA-Adaple
HYDRA-Adapter Inference. During the process of model inference, we conceptualize the black box LLM a proposal generator, while the adapter functions as an evaluator. Specifically, for each test query 2; from user u, we adopt the best-of-b inference, also known as rejection sampling. Initially, we sample candidate generations {f.r})- from the LLM. By passing them through the HYpRA-Adapter, we obtain the corresponding score for each candidate [p",..... "PP.. The solution with the highest score is then chosen the final answer:
...)).
arg max P... j=1,..
3.4 Model Factorization for Personalization
To further effectively capture the shared knowledge across all users as well as the individual user- specific behavior patterns, direct solution is to develop parameter decomposition method, where the shared parameters store the general knowledge, benefiting from high model capacity, while the remaining parameters are designed to learn user-specific preferences that complement the overall
5

---

understanding (typically on smaller scale). Hence, it is adequate to employ a smaller-sized model (adapter) to represent these personalized parameters, instead of fine-tuning the entire LLM.
Model Factorization. Formally, in HYDRA, we assume that each personalized language model associated with set of weights ④ = {o(w) JuED, where g(w) represents the weights of the personalized model for user u. Each ((x) is decomposed as:
r(u) o & 7(1),
(8)
where a represents the base model parameter matrix that shared among all users, 7(1) represents a personalized head model, and @ indicates the operation of appending the user-specific head 7(M) on top of the base model a. Specifically, we add [2] heads to the final hidden states generated by the original model. The w-th head measures the usefulness between query and histories, or the level of preference for the generation from the perspective of the u-th user. The prediction of the u-th head is denoted as p(w), For each head, we employ a single layer of feed-forward network. We define u-th
head as 7(1) [W("), W b
(a) b(") and the prediction can be outlined as:
pd softmax softmax(W! WT Tanh (W(") bluh) bla),, where wh()) E R w(") E Raxd, (9)
where d indicates the dimension of hidden states and o indicates the dimension of outputs.
Training Strategy. We then train the base model in conjunction with multiple user-specific heads. For each sample (z vf") of user u from the training data and the model prediction pV, we update the base model and the u-th head accordingly:
o a a VLW .P)), ((1) <- x((() - a LLV PP)),
(10)
where a indicates the learning rate and L represents the task-specific loss function.
Fit Test User History. In order to accommodate the newly incoming users in the test data, we cannot reuse the personalized heads from the training data. Therefore, we need to create and initialize new heads. To fit the test users' history, fitting process is simple and requires minimal computational resources. Similar to the update of 7 ( in Eq. (10), when given new user u' for testing, we also leverage the task-specific loss function 2()) to update the head r(n");
freeze the base model and solcly focus the heads. This
(a)) - 71²) a Lyy .P.)
(11)
where (xf', .u²') are obtained from test user history.
Personalized Inference for Test Users. Upon fitting the test user history, we obtain the base model a, which contains general knowledge across users, and the user-specific head z(u'), which captures the user-specific knowledge in the test user history. Therefore, we can apply personalized inference directly to the test user u', given its query Fu'
p fogroe
(12)
where f foorteey (+) indicates the inference of the model a @ r(1').
4
Experiments
4.1
Experimental Setup
Datasets and Tasks. We adopt a widely used language model personalization benchmark, LaMP [41].
focusing on a diverse set of personalized text classification and generation tasks, including (1)
Personalized News Categorization (LaMP-2N), (2) Personalized Movie Tagging (LaMP-2M), (3)
Personalized Product Rating (LaMP-3), (4) Personalized News Headline Generation (LaMP-4), and (5) Personalized Scholarly Title Generation (LaMP--)). For data splitting, we follow the user-based separation setting provided by the LaMP benchmark, with 100 randomly selected users for training and an additional 50 randomly selected users for testing. No shared users are displayed across splits for specific measurements in personalization for new users. Additional details of the LaMP benchmark are available in Appendix D.

---

Table 1: Main experiment results on the LaMP benchmark. R-1 and R-L represent ROUGE-1 1 and ROUGE-L, respectively. k indicates the number of retrieved items. 1 denotes that higher values are better, while implies that lower values are preferred. The best score and 2nd best score for each task are highlighted in bold and underlined, respectively. Notations are consistent across tables.
Datasel (->)
LaMP-2N
LaMP-2M
LaMP-3
LaMP-4
LaMP-5
Method (1)
Asc.
F-1 Acc.
F-1
4
RMSE
R-1 f R-L BLEU f R-1 t R-L BLEU
gpt-3.5 S-tuubo [30]
0.660 0.280
0.440
0.309
0.480
0.825
0.133
0.120
0.996
0.379
0.326
5.477
ICL- (Cn-Random (k=1)
0.640
0.293
0,480
0.360
0.660
0.990
0.165
0.144
1.567
0.410
0.349
5.327
ICL-Random (k=2)
0.640 0.284
0.520
0.400
0.560
0.917
0.171
0.158
1.923
0.413
0.348
6.698
ICL- (CL-Random (k=4)
0.660 0.288
0.480
0,356
0,560
0,917
0.163
0.148
1.589
0.393
0.349
7.445
RAG (k=1) [41]
0.680 0.293
0.400
0.290
0.500
0.837
0.151
0.130
1.417
0.418
0.353
5.852
RAG (k=2) [41]
0.600 0.281
0.460
0.343
0,580
0,906
0.173
0.156
1.778
0.419
0.367
6.898
RAG (Jo=4) [41]
0.640
0.284
0.460
0,340
0.580
0.970
0.172
0.156
1.812
0.415
0.362
7,382
PAG (k=0) [38]
0.640
0.293
0.500
0,356
0.520
0.894
0,163
0.150
1.724
0,410
0.359
6,124
PAG (k=1) [38]
0.680
0.308
0.520
0.362
0.560
0.913
0.170
0.156
1.674
0.397
0.327
6.481
HYDR4-Reranker (k=4)
0.760
0.375
0.520
0.393
0.480
0.775
0.177
0.166
1.951
0.423
0.368
6,864
HYDRA-Adapter
0.680
0.277
0.480
0.385
0.420
0.762
0.145
0.118
1.137
0.409
0.355
5,816
HYDRA
0.780
0.401
0.540
0.458
0.400
0.747
0.178
0.169
2.396
0.434
0.372
7.531
Table 2: Ablation studies of HYDRA. -P, represents removing the HYDRA personalized training strategy.
Dataset (-)
LaMP-2N
LaMP-2M
LaMP-3
LaMP-
LaMP-5
Method (L)
Ace.
F-1
Acc. f F-1 MAE + RMSE
R-1
R-L
BLEU
R-1 f R-L BLEU f
HYDRA
0.780
0.401
0.540
0.458
0.400
0.747
0.178
0.169
2.396
0.434
0.372
7,531
-P--Adapter
0.740
0.300
0.500
0.384
0.460
0.831
0.169
0.157
1.604
0.398
0.336
5.766
-P.-Reranker
0.700
0.298
0.500
0.379
0.480
0.825
0.162
0.154
2.063
0.399
0.347
5.364
-P.-Adapter Reranker
0.780
0,312
0.420
0.339
0.520
0.894
0.172
0.163
2.001
0.385
0.332
5.848
HYDRA-Reranker (k=4)
0.760
0.375
0.520
0.393
0.480
0.775
0.177
0.166
1,951
0.423
0.368
6.864
-P--Reranker
0.700
0.358
0.480
0.372
0.540
0.864
0.174
0.161
1.772
0.411
0.363
6.192
Adapter
0.680
0,277
0.480
0.385
0.420
0.762
0.145
0.118
1.137
0.409
0.355
5.816
-P--Aapterr
0.673
0.276
0.460
0.341
0.480
0.835
0.141
0.119
1.006
0.374
0.317
4.754
Baselines. We compare our proposed HYDRA with existing statc-of-the-art black-box LLM personal- ization methods, including in-context learning (ICL-Random, with random selected k-item from user behavior history), retrieval-auge auugenel-augmennted prompting (RAG) [41], and profile-augmented prompting (PAG) [38]. We also present the experimental results of gpt-3. 3. 5-turbo [30] (zero-shot) without profile/history augmentation in order to showcase the baseline performance of the backbone language model. Learning-basse
personalization, such as fine-tuning and RLHF-based methods, cannot be applied to black-box LLMs (e.g., 3pt-3.5-turbll due to the unavailability of model parameters. Details of baseline implementations available in Appendix E.
Evaluation Metries. Following the evaluation metrics specified in LaMP [41], we utilize accuracy (Acc.) and F-1 score (F-1) for the test classification tasks in LaMP-2N and LaMP-2M. Additionally, we employ mean absolute error (MAE) and root mean squared error (RMSE) for the ordinal multi- class classification task in LaMP-3. To comprehensively evaluate the personalized text generation tasks in LaMP-4 and LaMP-5, we report the ROUGE-1 (R-1), ROUGE-L (R-L), and BLEU metrics.
Implementation Details. For both baselines and our proposed HYDRA, we follow the same prompt template in the LaMP benchmark [41] and employ g 5-turbo (1106) and BM25 [39] as the backbone black-box L.M and default retriever, respectively. Additionally, both HYDRA-Adapter leverage the lightweight LongFormer-Baze (110M) [1] as the backend language models. We include more implementation details in Appendix F.
and
7

---

4.2 Main Results
Table presents the main experimental results of five varied personalization tasks in the LaMP benchmark. Compared to the zero-shot setting in gpt-3. 3-turbbb [30], of historical records from user behavior or profiles enhances the model performance in most tasks, suggesting that personalization contributes improved performance with black-box LLMs. HYDRA exhibits substantial relative improvements across all five tasks, with an average of 9.01 % over the best- performing baselines. Specifically, HYDRA outperforms the best-performing alternative by 14.71 % in accuracy for the Personalized News Categorization (LaMP-2N) task, 3.85% in accuracy for the Personalized Movie Tagging (LaMP-2M) task, 20.00% in MAE (where lower values indicate better performance) for the Personalized Product Rating (LaMP-3) task, 2.89% in R-1 for the Personalized News Headline Generation (LaMP-4) task, and 3.58% in R-1 for the Personalized Scholarly Title Generation (LaMP-5) task. This further improvement can be attributed to the improved quality (usefulness) of retrieved records in better representing user-specifie preferences and the effective integration of global knowledge sourced from the entire user group.
random selection
4.3 Ablation Studies
From Table 1, both HYDRA- and HYDRA-Adapter demonstrate their effectiveness by signif- icantly improving the RAG and PAG baselines separately. Furthermore, we observe that HYDRA outperforms each of them individually, showcasing the complementary role of both components. In Table 2, we further eliminate the personalized training strategy of HYDRA, deteriorating to training a singular model across the entire user group, which solely incorporates global knowledge and lacks in- dividualized customization for each user. The decline in model performance across all tasks highlights the significance of personalization through user-specific heads. In addition, conduct additional experiments to comprchensively evaluate different components of HYDRA in black-box LLM person- alization, including the effect of retrievers (Appendix G.4), the effect of adapters (Appendix G.5), and the effect of reranker with case studies and error analysis (Appendix H).
4.4 Scale-up Analysis
Number of Users in Training. We study the impact of users in the train- ing data, as depicted in Figure 3 (a)-(c). As the number of users in- creases from 20 to 100, we observe that HYDRA reaches over 90%
Hosser of usen in traning
Meverra upen Twirg
Surderd sami
Number of History per User. We examine the impact of the number of historical records per user in Figure 3 (d)-(f), We randomly chose 50 users from each range of number of his- tory per user. the number of his- torical records increases, we consis- tently observe improved performance in both classification and generation tasks, as indicated by all metrics. This demonstrates the robustness of HYDRA in effectively capturing user-specific preferences, especially with a larger amount of user historical data.
(a) LaMP-2N
(b) LaMP-3
(C) LaMP-4
Mill (SACE)
100 (SAEE)
(d) LaMP-2N
(c) LaMP-3
(f) LaMP-4
800
(g) LaMP-2N
(h) LaMP-3
LaMP-4
Number of Selected History (k) per User. In Figure (g)-(i), we analyze the impact of the selected items & per user. can observed that HYDRA consistently outperforms other baselines across all values of k. highlighting the robustness of HYDRA. Additionally, it is evident that model performance correlates with the number of retrieved historical records in most cases, providing further evidence of the effectiveness of the eetrieval-augmente
Figure 3: Sacle-up analysis for (a)-(c) Effect of users in training data, (d)-(f) Effect of historical records per user, (g)- (i) Effect of selected historical records (k) per user.

---

History LaMP-2M
Training L2MP-4
LaMP-2N
LaMP-3
LaMP-5
6.8-
2.5
9.38
Metrica
Metrics
Matrics
0.22
0:2
ALC
P-2
23
0.4
1AA
0.0
Magje13
MaujeL
6:30
160991
(a) HYYDR--Reranker
LaMP-2N
LaMP-2M
LaMP-3
LaMP-4
LaMP-5
6.0
0.20
6..6
Medrics
0.45
0.16-
Meltics
B.G.
014
0.5
F-2
F2
966
RICE
0.0
Roage-1
Raxpel
Touge-1
(b) HYDRA Adaater
Figure 4: Performance under different levels of shared knowledge.
Dataset (->)
LaMP-2N
LaMP-2M
LaMP-3
LaMP-4
LaMP-5
Method ())
Acc. f F-1 f Acc. t
F-1 f MAE RMSE J R-1 1 R-L BLEU
R-1
R-L1
BLEU 1
gpt-3.5-turbe
0.638
0.499
0,412
0.347
0.540
0.851
0.133
0.119
1.043
0.439
0.371
6.018
ICL-Random (k=1)
0.598
0.476
0.392
0.335
0.676
0.959
0.147
0.132
1,330
0.457
0.396
8.118
ICL-Random (k=2)
0.632
0.499
0.376
0,311
0.562
0.871
0.151
0.137
2,388
0.451
0.393
8.550
ICL.-Random (k=4)
0.630
0.518
0.392
0.352
0.440
0.740
0.161
0.146
2.418
0.457
0.396
8.404
RAG (k=1)
0,610
0.486
0,408
0.345
0,602
0.871
0.154
0.138
1,649
0.468
0.405
7.820
RAG (k=2)
0.624
0.479
0.380
0.315
0.559
0.836
0.161
0.149
2.958
0,480
0.419
9.021
RAG (k=4)
0.656
0.524
0.392
0.339
0.391
0.716
0.167
0.155
3.615
0.479
0.418
9.108
PAG (k=0)
0.618
0.489
0.404
0.340
0.583
0.872
0.161
0.141
1,950
0.460
0.405
7.372
PAG (k=1)
0.630
0,500
0.418
0.357
0,414
0.787
0.163
0.153
2.934
0.474
0.414
8.372
0.748
0.551
0.446
0.373
0.328
0.656
0.175
0.167
4.772
0.508
0.442
9.519
Table 3:
Scale-up experiment results on the LaMP benchmark, including 1000 and 500 users during training and testing, respectively.
framework in black-box LLM personalization. However, an inconsistency is noted with RAG in LaMP-3, which may be attributed to the presence of noisy or irrelevant retrieved history. This finding emphasizes the importance of retrieval quality and raises the significance of the HYDRA-Rerane measuring the usefulness retrieved items. Additional scale-up experimental results are available in Appendix G.2.
in
Number of Users in Training Set. We also conduct additional scale-up experiments (Table 3) to evaluate HYDRA with an increased number of users, increasing from 100 to 1000, across all five tasks. Our findings from the scale-up experiments show that HYDRA maintains its performance advantages over baselines as the number of users increases.
4.5 Empirical Personalization Analysis
Performance w/o Shared Knowledge. We conduct additional experimental analysis on both text classification and generation tasks for comprehensive evaluation. In Figure 4, we study how the shared knowledge is leveraged in HYDRA. Specifically, we create three settings with different levels of leveraging shared knowledge: (1) using only test user history for training, (2) updating just the head parameters using training user data, and (3) updating the full model parameters using training user data. We observe that incorporating larger volume of training user data yields superior results compared to solely relying on test user history. The inclusion of additional training user information aids in capturing global information effectively. In addition, updating the entire model shows inferior performance compared to updating only the heads. This discrepancy arises due to the increased number of parameters, which necessitates a larger amount of data accurately capture each user's

---

information. Additionally, the transition from training user information to testing user information results in domain shift of global knowledge.
Performance under User Behavior Shift. Following the setup in Tan et al. [50] (see Appendix G.3 for addi- tional details), we evaluate HYDRA un- der user behavior shift when the query is not relevant to all historical records. As illustrated in Figure 5, our pro- posed HYDRA continues to outperform the current state-of- the-artt baselines in the face of user behavior shift. This serves evidence of its robustness and generalizability in providing a widely applicable solution for black-box I.I. personalization. This improvement can be attributed to the personalized reranker, which reevaluates the candidates based on their utility rather than solely on relevance, together with the personalized adapter further adjusts to align with user preferences.
(a) LaMP-2N
(b) LaMP-3
(c) LaMP-4
Figure 5: Performance under user behavior shift.
Effect of Significant Disparities in User Behavior. To consider more extreme cases of disparities in user behav- ior, we retrain HYDRA on a mixture of 50% users with the fewest interactions and another 50% users with the most interactions (blue in Figure 6, compared to the ran- dom selection in orange). The black and gray dashed lines represent the best- pesst-eeffrming baselines on the first and second metrics. The experimental results demonstrate that HYDRA consistently outperforms existing baselines, even under extreme cases. Compared to the previous random selection of training users, HYDRA achieves relatively lower performance due to the imbalance of training sam- ples for dense users and sparse users. By leveraging the global information in shared parameters, knowledge can be effectively transferred from dense users sparse users, thereby enabling further personalization through the uti- lization of sparse user-specifie head models.
Soctr boo
0:8
Feapt
Figure 6: Experiments on LaMP bench- mark of significant disparities in user be- havior history.
Conclusions
In this paper, proposed HYDRA, a model factorization framework for black-box LLM personal- ization that captures and leverages both user-specific and shared behavior patterns from historical data. Compared prompt-based methods, HYDRA introduces novel learning-based paradigm for black-box LLM personalization that not only remains highly effective in personalizing outputs by training reranker with black-box L.M adapter, but also eliminates the need for access model pa- rameters, presenting a promising alternative to existing mainstream techniques. Experimental results demonstrate that HYDRA outperforms kate-of-the-aa RAG and PAG baselines by an average relative improvement of 9.01% % across five diverse personalization tasks. HYDRA establishes foundation for learning-basse based black-box LLM personalization, facilitating targeted enhancements in user experience and ultimately contributing to the development of human-centre intelligent systems.
Acknowledgments and Disclosure of Funding
We thank the anonymous reviewers and area chairs for their valuable feedback. This work was supported by NSF grants IIS-2008334, IIS-2106961, CAREER I1S-2144338, IIS-2403240, and Azure credits from the Microsoft Accelerating Foundation Models Research Program.
10

---

References
[1] I. Beltagy, M. E. Peters, and A. Cohan. Longformer: The long-document transformer. arXiv preprint arXiv:2004.05/50, 2020.
[2] T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, Askell, et al. Language models are few-shot learners. Advances neural information processing systems, 33:1877-1901, 2020.
[3] S. Casper, X. Davies, C. Shi, T. K. Gilbert, J. Scheurer, J. Rando. R. Freedman, T. Korbak, D. Lindner. P. Freire, al. Open problems and fundamental limitations of reinforcement learning from human feedback. arXiv preprint arXiv:2307. 152/7, 2023.
[4] A. Chowdhery, S. Narang, J. Devlin, M. Bosma, G. Mishra, A. Roberts, P. Barham, H. W. Chung, C. Sutton, S. Gehrmann, et al. Palm: Scaling language modeling with pathways. arXiv preprint arXiv:2204.023/1 2022.
[5] K. Christakopoulou, A. Lalama, C. Adams, I. Qu, Y. Amir, S. Chucri, P. Vollucci, F. Soldo, D. Bseiso, S. Scodel, et al. Large language models for user interest journeys. arXiv preprint arXiv:2305.15498 2023.
[6] S. Dai, N. Shao, H. Zhao, W. Yu, Z. Si, C. Xu, Z. Sun, X. Zhang, and J. Xu. Uncovering chatgpt's capabilities in recommender systems. In Proceedings the 17th ACM Conference on Recommender Systems, pages 1126-1 132, 2023.
[7] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In J. Burstein, C. Doran, and T. Solorio, editors, Proceedings of the 2019 Conference of the North American Chapter of the Association for Computationna Linguisties: Human Language Technologies, Volume 1 (Long and Short Pa- pers), pages 4171-4186, Minneapolis, Minnesota, June 2019. Association for Computational Linguistics.
[8] D. Di Palma. Retrieval-augmentee recommender system: Enhancing recommender systems with large language models. In Proceedings of the 17th ACM Conference on Recommender Systems, pages 1369-13733 2023.
[9] X. L. Dong, S. Moon, Y. E. Xu, K. Malik, and Z. Yu. Towards next-generation intelligent assistants leveraging I1m techniques. Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 5792-5793, 2023.
[10] S. Gururangan, A. Marasović, S. Swayamdipta, K. Lo. I. Beltagy, D. Downey, and N. A. Smith, Don't stop pretraining: Adapt language models to domains and tasks. In D. Jurafsky, J. Chai, N. Schluter, and J. Tetreault, editors, Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8342 8360, Online, July 2020. Association for Computational Linguistics.
[11] P. He, J. Gao, and W. Chen. Debertav3: with disettagldd embedding sharing. In The Eleventh International Conference on Learning Representationss 2022.
Improving deberta using clectra-style pre-training
[12] Y. Hou, J. Zhang, Z. Lin, H. Lu, R. Xie, McAuley, and W. X. Zhao. Large language models are zero-shot rankers for recommender systems. In European Conference on Information Retrieval, pages 364-381. Springer, 2024.
[13] N. Houlsby, A. Giurgiu, S. Jastrzebski, B. Morrone, Q. De Laroussilhe, A. Gesmundo, M. At- tariyan, and S. Gelly. Parameter-efficient transfer learning for nlp. In International Conference on Machine Learning, pages 2790-2799. PMLR, 2019.
[14] E. J. Hu, P. Wallis, Z. Allen-Zih Zhu, Y. Li, S. Wang. L. Wang, W. Chen, et al. Lora: Low-rank adaptation of large language models. In International Conference on Learning Representations, 2021.
[15] Y. Huang, D. Liu, Zhong, W. Shi, and Y. Lee. k nn-adapter: Efficient domain adaptation for black-box language models. arXiv preprint

---

[16] G. Izacard, M. Caron, L. Hosseini, S. Riedel, P. Bojanowski, A. Joulin, and E. Grave. Unsuper- vised dense information retrieval with contrastive learning. arXiv preprint arX1v:2112.09118. 2021.
[17] J. Jang, S. Kim, B. Y. Lin, Y. Wang, J. Hessel, L. Zettlemoyer, H. Hajishirzi, Y. Choi, and Ammanabrolu. Personalized soups: Personalized large language model alignment via post- hoc parameter merging, arXiv preprint arXiv:2310. 11564, 2023.
[18] W.-C. Kang, J. Ni, N. Mehta, M. Sathiamoorthy, L. Hong, E. Chi, and D. Z. Cheng. Do llms understand user preferences? evaluating Ilms on user rating prediction, 2023.
[19] D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. arXiv preprini arXiv 2014.
[20] H. R. Kirk, B. Vidgen, P. Röttger, and S. A. Hale. Personalisation within bounds: A risk taxonomy and policy framework for the alignment of large language models with personalised feedback. arXiv preprint vrXiv:2303.05455 2023.
[21] C. Li, M. Zhang, Q. Mci, Y. Wang, S. A. Hombaiah, Y. Liang, and M. Bendersky. Teach Ilms to personalize- an approach inspired by writing education. arXiv preprint arXiv:2308.0777 077668 2023.
[22] X. Li, Z. C. Lipton, and L. Leqi. Personalized language modeling from personalized human feedback, 2024.
[23] Y. Li, H. Wen, W. Wang, X. Li, Y. Yuan, G. Liu, J. Liu, W. Xu, X. Wang, Y. Sun, et al. Personal IIm agents: Insights and survey about the capability, efficiency and security. arXiv preprint arXiv:240.00545, 2024.
[24] A. Liu, X. Han, Y. Wang, Y. Tsvetkov, Y. Choi, and N. A. Smith. Tuning language models by proxy, 2024.
[25] Liu, Liu. Zhou, R. Lv, K. Zhou, and Y. Zhang. Is chatgpt good recommender? preliminary study. arXiv preprint 2r04. 10149, 2023.
[26] Q. Liu, N. Chen, T. Sakai, and X.-M. Wu. Once: Boosting content- based recommendation with both open-and closed-source large language models. In Proceedings of the 17th ACM International Conference on Web Search and Data Mining, pages 452-461, 2024.
[27] X. Liu, K. Ji, Y. Fu, W. Tam, Z. Du, Z. Yang, and J. Tang. P-tuning: Prompt tuning can be comparable to fine-tuning across scales and tasks, In S. Muresan, P. Nakov, and A. Villavi- cencio, editors, Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume Short Papers), pages 61 61-68, Dublin, Ireland, May 2022. Association for Computational Linguistics.
[28] H. Lyu, S. Jiang, H. Zeng, Y. Xia, Q. Wang, S. Zhang, R. Chen, C. Leung, J. Tang, and J. Luo. Llm-rec: Personalized recommendation via prompting large language models. In Findings of the Association,o for Computational Linguistics: NAACL 2024, pages 583-612, 2024.
[29] S. Mysore, Z. Lu, M. Wan, L. Yang, S. Menezes, T. Baghaee, F. B. Gonzalez, J. Neville, and T. Safavi. Pearl: Personalizing large language model writing assistants with generation- calibrated retrievers. arXiv preprint arXiv:2311.09) 2023.
[30] OpenAl. Introducing chatgpt. OpenAl Blog, 2022.
[31] OpenAl. Gpt-4 technical report. arXiv, page 2303.08774v3. 2023.
[32] A. Ormazabal, M. Artetxe, and E. Agirre. CombLM: Adapting black-box language models through small fine-tuned models. In H. Bouamor, J. Pino, and K. Bali, editors, Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 2961-2974, Singapore, Dec. 2023. Association for Computational Linguistics.
[33] L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730-27744 2022.
12

---

[34] A. Peng, M. Wu, J. Allard, L. Kilpatrick, and S. Heidel. Gpt-3.5 turbo fine-tuning and api updates. OpenAl Blog, 2023.
[35] A. Radford, K. Narasimhan, T. Salimans, and Sutskever. Improving language understanding by generative pre-training, OpenAl Blog, 2018.
[36] A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, and I. Sutskever. Language models unsupervised multitask learners. OpenAl Blog, 2019.
[37] R. Rafailov, A. Sharma, E. Mitchell, C. D. Manning, S. Ermon, and C. Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36, 2024.
[38] C. Richardson, Y. Zhang. K. Gillespie, S. Kar, A. Singh, Z. Raeesy, O. Z. Khan, and A. Sethy. Integrating summarization and retrieval for enhanced personalization via large language models. arXiv preprint arXiv:2310.2008/1 2023.
[39] S. Robertson, H. Zaragoza, et al. The probabilistic relevance framework: Bm25 and beyond. Foundations and Trends@ in Information Retrieval, 3(4):333-33)) 2009.
[40] A. Salemi, S. Kallumadi, and H. Zamani. Optimization methods for personalizing large language models through retrieval augmentation. arXiv preprint arXiv:2404.05977 2024.
[41] A. Salemi, S. Mysore, M. Bendersky, and H. Zamani. Lamp: When large language models meet personalization. arXiv preprint arXiv:2304. 1406, 2023.
[42] S. Santurkar, E. Durmus, F. Ladhak, C. Lee, P. Liang, and T. Hashimoto. Whose opinions do language models reflect? In International Conference on Machine Learning, pages 29971- 1- 30004. PMLR, 2023.
[43] W. Shi, R. Xu, Y. Zhuang, Y. Yu, H. Wu, C. Yang, and M. D. Wang. Medadapter: Efficient test-time adaptation of large language models towards medical reasoning. arXiv preprint arXiv:2405.03000 2024.
[44] W. Shi, R. Xu, Y. Zhuang, Y. Yu, J. Zhang, H. Wu, Y. Zhu, J. C. Ho, Yang, and M. D. Wang. Ehragent: Code empowers large language models for few-shot complex tabular reasoning on electronic health records. In ICLR 2024 Workshop on Large Laggugge Model (L.LM) Agents, 2024.
[45] W. Shi, Y. Zhuang. Y. Zhu, H. Iwinski, M. Wattenbarger, and M. D. Wang. Retrieval-augment large language models for adolescent idiopathic scoliosis patients in shared decision-making In Proceedings of the 14th ACM International Conference Bioinformatics. Computational Biology, and Health Informatics, pages 1-10, 2023.
[46] N. Stiennon, L. Ouyang, J. Wu, D. Ziegler, R. Lowe, C. Voss, A. Radford, D. Amodei, and P. F. Christiano. Learning summarize with human feedback. Advances in Neural Information Processing Systems, 33:3008-300 -0218, 2020.
[47] H. Sun, Y. Zhuang. L. Kong. B. Dai, and C. Zhang, Adaplanner: Adaptive planning from feedback with language models. Advances in Neural Information Processing Systems. 36, 2024.
[48] H. Sun, Y. Zhuang, Wei, C. Zhang, and B. Dai. Bbox-adapter: Lightweight adapting for black-box large language models, 2024.
[49] T. Sun, Y. Shao, H. Qian, X. Huang, and X. Qiu. Black-box tuning for language-model-ag a-service. In International Conference on Machine Learning, pages 2022.
PMLR,
[50] Z. Tan, Q. Zeng, Y. Tian, Z. Liu, B. Yin, and M. Jiang. Democratizing large language models via personalized parameter-efficient fine-tuning, arXiv preprint arXiv:2402.04440, 2024.
[51] H. Touvron, L.. Martin, K. Stone, P. Albert, A. Almahairi, Y. Babaei, N. Bashlykov, S. Batra, Bhargava, S. Bhosale, et al. Ilama 2: Open foundation and fine-tuned chat models. arXiv preprint 2077022307.09288, 2023.
13

---

[52] D. Wang, K. Yang, H. Zhu, X. Yang. A. Cohen, L. Li, and Y. Tian. Learning personalized story evaluation, arXiv preprint arXiv:2310.0334,, 2023.
[53] Wei, M. Bosma, Zhao, K. Guu, A. W. Yu, B. Lester, N. Du, A. M. Dai, and Q. V. Le. Finetuned language models are zero-shot learners. In International Conference on Learning Representations, 2021.
[54] X. Wu, X. Liu, J. Niu, H. Wang, S. Tang, and G. Zhu, FedloRA: When personalized federated learning meets low-rank adaptation, 2024.
[55] Z. Wu, Y. Hu, W. Shi, N. Dziri, A. Suhr, P. Ammanabrolu, N. A. Smith, M. Ostendorf, and H. Hajishirzi. Fine-grained human feedback gives better rewards for language model training. Advances in Neural Information Processing Systems, 36, 2024.
[56] D. Xu, W. Yin, X. Jin, Y. Zhang, S. Wei, M. Xu, and X. Liu. Llmead: Fast and scalable on-device large language model inference. arXiv preprint 230i::23000022555 2023.
[57] R. Xu, H. Cui, Y. Yu, X. Kan, W. Shi, Y. Zhuang, W. Jin, J. Ho, and C. Yang. Knowledge-intee -iffuged prompting: Assessing and advancing clinical text data generation with large language models. arXiv preprint
2023.
[58] R. Xu, W. Shi, Y. Yu, Y. Zhuang. B. Jin, M. D. Wang, J. C. Ho, and C. Yang. Ram-ehr: Retrieval augmentation meets clinical predictions on electronic health records. arXiv preprint wrXiv:2403.00815 2024.
[59] R. Xu, W. Shi, Y. Yu, Y. Zhuang. Y. Zhu, M. D. Wang. J. C. Ho, C. Zhang, and C. Yang. Bmretriever: Tuning large language models as better biomedical text retrievers. arXiv preprint arXiv:2404. 18443, 2024.
[60] Y. Yao, J. Duan, K. Xu, Y. Cai, Z. Sun, and Y. Zhang. A survey on large language model (Ilm) security and privacy: The good, the bad, and the ugly. Hfigh-Confidence Computing, page 100211, 2024.
[61] Y. Yu, J. Shen, T. Liu, Z. Qin, J. N. Yan, J. Liu, C. Zhang, and M. Bendersky. Explanation- aware soft ensemble empowers large language model in-context learning. arXiv preprint arXiv:2311.07099 2023.
[62] Z. Zhao, W. Fan, J. Li, Y. Liu, X. Mei, Y. Wang, Z. Wen, F. Wang, X. Zhao, J. Tang, et al. Recommender systems in the era of large language models (Imss). IEEE Transactions on Knowledge and Data Engineering, 2024.
[63] I.. Zheng, W.-L. Chiang, Y. Sheng, S. Zhuang, Z. Wu, Y. Zhuang. 7. Lin, 7. Li, D. Li, E. Xing, al. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in Neural Information Processing Systems, 36, 2024.
[64] A. Zhiyuli, Y. Chen, X. Zhang, and X. Liang. Bookgpt: A general framework for book recommendation empowered by large language model. arXiv preprint arXiv:2305.1561 2023.
[65] Y. Zhuang, X. Chen, T. Yu, S. Mitra, V. Bursztyn, R. A. Rossi, S. Sarkhel, and C. Zhang. T Efficient action space navigation in large language models with a search. In The Twelfth International Conference on Learning Representations, 2023.
[66] Y. Zhuang, Y. Yu, K. Wang, H. Sun, and C. Zhang. Toolqa: A dataset for Ilm question answering with external tools. Advances Neural Information Processing Systems, 36, 2024.
[67] D. M. Ziegler, N. Stiennon, J. Wu, T. B. Brown, A. Radford, D. Amodei, P. Christiano, and G. Irv- ing. Fine-tuning language models from human preferences. arXiv preprint 2019.
14

---

A Limitations and Broader Impacts
A.1 Limitations
In this work, we propose a learning-based framework HYDRA for black-box LLM personalization. Despite its effectiveness, we have identified several potential limitations of HYDRA:
Resource Limitations. Due to limited access to the API services of black-box LLMs and budget constraints, our experiments with black-box fine-tuning are currently restricted to utilizing the Microsoft Azure OpenAl API service with the gpt-3.5 tuturo (1106) model.
Data Privacy. Unlike the Microsoft Open Al fine-tuning service for black-box LLMs, which involves data exchange through API services, HYDRA-Adapter does not share any training data with third parties via APIs. It ensures the security of the training samples during black-box LLM adaption. However, is important to note that potential private information may be contained in the retrieved historical records or the query itself, which could introduce potential risk of private data leakage [60].
On-Device Inference. In real-world recommendation systems or virtual assistants, on-device infer- ence with small-scale language models may be necessary [56]. While HYDRA-Adapter is lightweight language model with I 10M parameters and is efficient in fine-tuning during black-box L.M adapta- tion, the extensive model parameters of black-box LLMS themselves may eliminate the possibility of on-device inference.
A.2 Broader Impacts
Potential Positive Societal Impacts. The proposed HYDRA addresses an important challenge posed by the inherently opaque nature of state-of-the-arr LLMs, like GPTs [30]. enabling caaming-based personalization in black-box LLMs. Effective personalization of black-box LLMs can lead to significant improvements in user experience and engagement across wide range of applications, from virtual assistants to recommendation systems. By tailoring the outputs of LLMs to the unique preferences and behavioral patterns of individual users, HYDRA has the potential to provide more usefuul relevant, and satisfying interactions. This could enhance productivity, decision-making, and overall quality of life. Moreover, the ability to personalize language models without requiring access to their internal parameters makes this technology more accessible. This democratization of personalization capabilities can benefit a broader segment of the population, including individuals and organizations that may not have the resources technical expertise to fine-tune adapt large language models directly. Increased accessibility to personalized language models could lead to more inclusive and equitable technological solutions.
However, the personalization of language models also carries some potential risks and negative societal impacts, including concerns around user privacy and data security [20], as the personalization process requires collecting and storing sensitive user behavioral data. Improper handling misuse of this data could lead to privacy breaches and unauthorized profiling of individuals. Robust data governance frameworks and stringent privacy safeguards will be cnucial to mitigate these risks. Additionally, the personalization of language models could potentially exacerbate issues of algorithmic bias and filter bubbles. If the personalization process is not carefully designed to avoid reinforcing existing biases or limiting users' exposure to diverse perspectives, it could contribute to the creation of echo chambers and the perpetuation of biased decision-making.
Potential Negative Societal Impacts.
Future Works. Our research highlights several meaningful avenues for future exploration. It is imperative to prioritize fairness, diversity, and inclusivity in the personalization process in order to address privacy-related concerns. Furthermore, one potential direction is expand the scope of our applications beyond LLM personalization to encompass multi-modal vision language model personalization. This endeavor will not only enhance user-centric intelligent systems but also contribute to elevating the overall user experience.
Overall, the proposed HYDRA framework has the potential to deliver significant positive societal impacts through enhanced user experience and increased accessibility to personalized language models. However, careful consideration of privacy, data security, and algorithmic bias mitigation will be necessary to address the potential negative consequences and ensure the responsible development and deployment of this technology.
15

---

A.3 Ethical Statements
We strictly follow data usage guidelines during the interaction with Microsoft Azure's s Open Al API service. Although we utilize all publicly available datasets in this study. we have withdrawn from the human review process by completing and submitting the Azure OpenAI Additional Use Case Form to prevent any potential information leaks.
B Additional Related Works
B.1 LLM Personalization
We summarize existing LLM personalization studies in Table 4, including (1) prompting- methods (available for both white- and black-box LLMs), (2) learning-based methods (only available for white-box LLMs), and (3) HYDRA, a learning-based black-box LLM personalization framework.
Table 4: Summary of LLM personalization baselines and HYDRA on the inclusion of different components. We present an overview of the existing LLM adaptation methods, focusing on six key aspects: (1) personalization for specific users, (2) global knowledge across different users, (3) retrieval from user history, (4) retrieval for relevance, (5) retrieval for usefulness, (6) learning- bbaee method and (7) personalization of black-box LLMs.
Global
Retrieval
Retrieval
Black-Box
Methods
Personalization
Retrieval
Learning
Knowledge
Relevance
Usefulness
LLM
Prompting- Methods
ICL-Random
RAG [40]
x
PAG [10]
x
/
x
X
Learning-buse bosed Methods
Fine-Tunine Tine-Tunig
X
OPPU [50]
X
X
X
X
x
RIPHF [17]
X
X
X
X
/
X
HYDRA (Ours)
/
/
/
B.2
Learning-based Personalization for White-Box LLMs
Aligning Language Models to Personal Preferences. While RLHF aligns LLMs with general human preferences [33, 67, 46, 37], is inadequate for capturing diverse and individual perspectives. as it assumes a reward model based only on average annotator preferences and ignores variations in (P-RLHF) frame-
user-desired outputs even for the same prompt [3, 42]. The personalized-RL.n work [22] addresses diverse user preferences encoded in human feedback by jointly training user model to capture individual preferences and reward model to generate personalized language. Simi- larly, fine-grained RLHF [55] enhances personalization by providing rewards after each generated segment and integrating multiple reward models for various feedback types, including information incorrectness, irrelevance, and incompleteness. Additionally, Reinforcement Learning from Person- alized Human Feedback (RLPHF) [17] further considers scenarios conflicting preferences and achieves personalized alignment by decomposing preferences into multiple dimensions. However, RLHF-based solutions often focus on preference alignment by explicitly decomposing preferences into multiple predefined dimensions specified by the user, which may not always be feasible in real-world scenarios.
Personalized Parameter-Efffcieet Fine-Tuning (PEFT). An alternative approach to personalization to develop individual language models tailored to each user [50, 54]. One notable example of
relevant work. although only available on white-box LLMs, is One PEFT Per User (OPPU) [50],
which trains PEFT model for each user using Low-Rank Adaptation (LoRA) [14]. OPPU ef-
fectively combines parametric user knowledge embedded in personalized PEFT parameters with
'https:/ //ttps://aka... http://aaaaaa/////////// ww/wmi///aaaaaaaaaamaaeaa..a ttta///ddiiiioaasseees.
16

---

aa--aaametric knowledge obtained through retrieval and user profiling, ensuring a more comprehen- sive and personalized user experience.
Limitations. learning- hased methods primarily focus on white-box LLM personalization, involving personalized alignment through parameter merging and personalized reward models, which is not applicable to black-box LLMs due to the opacity of their model parameters. Specifically, RLHF-based solutions require explicit personalization information (e.g.. difficully, style) as preference pairs in addition to behavior history, which is difficult to collect and necessitates additional human effort in annotations. Compared with existing personalization solutions that develop individual user models for user-specific alignment, HYDRA takes into consideration shared (global) knowledge for more effective personalization with potential generalization to the entire user group. In addition, personalization is a strategic priority for many corporate corporations, where most state-of state-of-the-an the-art art solutions remain
proprietary and are not available as open-source.
B.3 Black-Box LLM Adaptation
Table 5: Summary of LLM adaptation baselines and HYDRA-Adapte on the inclusion of different components. We present an overview of the existing L.M adaptation methods, focusing on five key aspects: (1) accessibility of model parameters, (2) availability of high-dimensional representations for input sequences or output generations, (3) availability of token probabilities, (4) necessity of retrieval corpus, and (5) utilization of a smaller adapter model. Adapted from Table 1 in Sun et al. [48].
w/o Model
w/o Digenniimenion Representation
w/o Token Probabilities
w/o Retrieval w/ Smaller Corpus
Methods
Parameters
Adapter
White-Box LLM Fine-Tuning
Vine-Tuning [7]
X
Instruction- Connucttoo-Tunnngg [53]
X
X
X
Continual Pre- Tee-Trainnng [10]
X
X
Adapter [13]
X
X
X
Prefix-Tuning [27]
X
X
X
LoRA [14]
X
Grey-Box LLM Adaptation
LMaaS [49]
x
X
kNN-Adapter [15]
/
X
/
x
CombLM [32]
Proxy-Tuning (24]
X
Black-Box LLM Adopiation
BBox-Adapter [48]
HYDRA-Adapter (Ours)
Following Sun et al. [48], we leverage the level of access to LLM internal parameters to determine its classification as whiteebox,, grey-box, or black-box LLMs (Table 5). White-box LLMs grant full access to both model parameters and output probabilities, whereas grey-box models provide access only output probabilities. In contrast, black-box models restrict access both parameters and output probabilities.
Fine-tuning APIs, such as the OpenAl GPT-3.5-turbo API [34], enable the effective adaptation of black-box LLMS by allowing users to upload training data and download fine-tuned outputs. However, black-box LLM adaptation through APIs presents several significant challenges [48]: (I) Transparency: The fine-tuning process is largely opaque, with limited visibility into key aspects such as the extent of trainable layers and specific model weights. This lack of transparency hinders optimal customization, as users are restricted to adjusting narrow set of hyperparameters, such as the number of tuning epochs. (2) Privacy: Uploading training data via APIs raises concerns about potential privacy breaches, particularly in sensitive domains. (3) Cost: Fine-tuning APIs incur substantially higher costs compared to inference, making the adaptation process expensive. Moreover, the cost of fine-tuning escalates significantly when hyperparameter tuning is involved, further increasing the financial burden. As result, we propose EYDRA-Adapte to eliminate the requirement for direct access to model parameters or API services in order to enable black-box LLM adaptation.
17

---

C
Algorithm Details
Algorithm 1: HYDRA.
Input:
Dimin {(gu, Fus H.)) J: training data; u: User: qui the user's query; Tu. the ground truth answer to the user query gui H.: the user u's history behaviors; R(qu.. H.)): the retriever retrieve lop-M relevant history records; training epochs:
// HYDRA-Revanker
Construct the training, test history, and test query candidates Etrain geramker & Equary
"reranker via Eq.(2)
Adopt LLM as labeling function to create and annotate the dataset Daas reranker Deerubect Dyteery
for = 1, .... E' do
Compute
loss function Eq.(3) over Dismiker
Update the shared parameters and user-specific heads via (q.
Freeze the shared parameters in base model of HYDRA-reranker
Reinitialize the user-specific heads of HYDRA-reranker
for 2, .... E' do
Compute the loss function via Eq.(3) over Dihistory Update the user-specific heads via (11)
reranker
Obtain model inference C2 Darer Teraneer via (12)
I/ HYDRA-Adapter
gurai Construct the training, test history, and test query candidates Exdapter Catgerr E via Eq.(5)
Augment user query Oe with reranked history candidates Ca and sample candidate responses from the black-box LLM via (q.
for 1, E' do
Compute loss function Eq.(3) over Patipee
Update the shared parameters and user-specific heads via (q.(10)
Freeze the shared parameters in base model of HYDRA-adapter
Reinitialize the user-specific heads of HYDRA-adapter
for 2, E do
Compute the loss function via E3.(3) over Daligee
Update the uper-specific heads via (11)
Obtain model inference f; on Deserry via (12)
Output: The personalized generation F." for the given query of from user u.
D Dataset and Task Details
We employ the Language Model Personalization (LaMP) benchmark [41], an open-source benchmark specifically designed train and evaluate the capability of language models generating personalized content. LaMP encompasses diverse set of tasks (with LaMP-2 comprising two tasks, LaMP-2N and LaMP-2M), covering both personalized text classification and generation tasks. We present the statistics of the datasets in Table 6 to provide a clear depiction of the dataset structure. We further offer detailed descriptions of each task as follows.
Task 1: Personalized News Categorization (LaMP-2N) categorical text classification
task that involves classifying news articles into one of 15 categories based on a journalist. Given an article written by user, the model predicts its category using the user's history of articles and their corresponding categories.
Task Personalized Movie Tagging (LaMP-2M) an ordinal text classification task that involves predicting one of 15 tags for movie based on user's tagging history. LaMP-2M evaluates the capability of a language model to assign tags to movie description using the user's historical movie-tag pairs.
Task 3: Personalized Product Rating (LaMP-3) is a text classification task that involves predicting product ratings five-class text classification problem. The objective is to predict an integer rating from one to five for a given review based on users' historical review and rating pairs. This task assesses the capability of language models capture user-specific rating preferences.
Task Personalized News Headline Generation (LaMP-4) text generation task that involves generating personalized news headlines for given articles based on the authors'
18

---

historical article-tibl title pairs. This task evaluates the language model's ss ability to replicate the authors' stylistic nuances in headline generation.
Task 5: Personalized Scholarly Title Generation (LaMP-5) text generation task that involves generating titles for input articles based on the author's historical article and title pairs. This task extends the exploration of personalized text generation into scholarly domains, explicitly focusing on generating titles for research articles.
LaMP-6 has been excluded since the dataset is not publicly available. Furthermore, two additional tasks (LaMP-1 and LaMP-7) have been excluded from our empirical evaluation due to the inconsistent formats between user history and query.
Personalized Citation Identification (LaMP-1) is a binary text classification task that involves predicting citation choices. It requires determining which of two candidate papers will be cited in a new paper based on a user's historical publication data. This process utilizes user profiles that include the titles and abstracts of their previous works. This task aims to evaluate a language model's 's capability to identify the citation preferences specific to each user.
Personalized Tweet Paraphrasing (LaMP-7) is text generation task that involves para- phrasing an input tweet into personalized one, using the user's s historical tweet data for stylistic guidance. This task evaluates a model's proficiency in reproducing the unique stylistic patterns in authors' social media posts.
Table 6: Dataset statistics of five different personalization tasks in the LaMP benchmark [41].
Task
Type
Train Validation
Test
Input Length
Output Length
Profiles
Classes
LaMP-2N
Classification
5914
1052
1274
65.40 +: 12.29
306.42 42 + 286.65
15
LaMP-2M
Clussification
5073
1410
1557
92.39 +: 21.95
86.76 +: 189.52
15
LaMP-3
Classification
20000
2500
2500
145.14 + 157.96
188. 10 + 129.42
5
LaMP-4
Generation
12527
1925
2376
30.53 + 12.67
9.78 + 3.10
287. 16 + 360.62
TaMP-5
Generation
9682
2500
2500
152.81 + 86.60
9.26 + 3.13
89.61 + 53.87
E
Baseline Details
We compare our proposed HYDRA with both non-personalized and personalized baselines. The non- personalized baselines consist of zero-shot gpt-3. 5..55-55 5-turbo [30] and ICL-Random (In-Context Learning with Random items from user behavior history). On the other hand, the personalized baselines include RAG [Retrieval-Augmentee Personalization) and PAG Augmene- Per- sonalization). & denotes the number of retrieved items from the user behavior history (or profiles). For all baselines, we follow the same prompt template in the LaMP benchmark [41] and employ gpt-3. 5-turbo (turboo1100)) as the backbone black-box LLM. We utilize BM25 [39] as the default retriever for all experiments.
gpt-3.5-turbo [30] processes only the user's query (i.e., the input question) without leverag- ing the user's profile data.
ICL-Random augments the user's query with random k items from behavior history, enabling in-context learning to improve the capability of the original backbone LLM.
RAG [41] augments the user's query with top & retrieved items from the corresponding user's 's behavior history, following the aagmentl- personalization method in LaMP.
PAG [38] augments the user's query with the user's profile summary generated by LLMs which concludes the user's behavior patterns. We further combine PAG with A retrieved items from the corresponding user's behavior history.
19

---

F Implementation Details
Hardware and Software. We conduct all black-box LLM personalization experiments on CPU: AMD(R) EPYC(R) 7702 64-Core Processor @ 1.50GHz and GPU: NVIDIA A100-SXM4-80GB using Python 3.10.13.
Hyperparameter Configurations. We set the maximum sequence length for a generated solution as 512 tokens and the temperature as 1.0 for flexibility in the generations black-box LLMs, which serve potential solution candidates. For other baselines, we maintain the temperature to to avoid potential instability in performance. During the black-box LLM adaption stage, we set generation of intermediate candidates using HYDRA-Adapter During the training stage, set the learning rate to 2e - 5, the global batch size to 64, and the number of training epochs to as default hyperparameter settings for all experiments. In addition, employ AdamW [19] the optimizer with a weight decay of 0.01. We include prompt details in Appendix I.
for the
G
Additional Experimental Results and Analysis
G.1 Additional Analysis of Baselines in the Main Experimental Results
We conduct further analysis the main experimental results of dominant approaches for black-box LLMs, specifically prompt- based methods, including RAG and PAG. Compared to the zero-shot setting in gpt-3. 5-trbo [30], profiles enhances the model performance in most tasks, suggesting that personalization contributes to improved performance with black-box L.MMs. However, we also observe several instances of less optimal performance, particularly in LaMP-3. We hypothesize that this inconsistency may arise from the introduction of noise and irrelevant behavior patterns by the retrieved items, potentially complicating the understanding of user behavior patterns. Additionally, both RAG [41] and PAG [38] demonstrate marginal improvements over random selection, which may benefit from augmenting relevant behavior profile information. The experimental results also consistently demonstrate that an increase the number of retrieved items is positively correlated with improved performance, indicating the effectiveness of the retrieval- augmented framework in black-box LLM personalization. However, in comparison to HYDRA, RAG- and PAG-based methods still yield suboptimal results. This primarily attributed the presence of potentially noisy and irrelevant retrieved behavior patterns, as well as the absence of shared general knowledge across the entire user group.
even a random selection of historical records from user behavior or
G.2 Additional Experimental Details for Sacle-up Analysis
Figure 7 reports additional experimental results of HYDRA in accuracy and F-1 with different numbers of selected history per user, indicating the robustness of HYDRA with additional user history.
0.200
0.6
0.8
Metrics
Metrics
Metrics
0.175
0.125
0.4-
0.44
0.100
Acc
MAE
RMSE
Rouge-1
Rouge-L
(a) LaMP-2N
(b)
LaMP-3
(c) LaMP-4
Figure 7: Effect of users with different numbers of behavior history in HYDRA.
G.3 Additional Experimental Details for User Behavior Shift
Following the setup of existing work [50], we utilize an encoder-only language model called DeBERTa-v3 [11] to encode the user's historical behaviors and the query into high-dimensional representations. We then compute the cosine similarity between the query and all historical items to evaluate their relevance. We select the top 20 items with the lowest relevance scores irrelevant
20

---

user history. As depicted in Figure 5, our proposed HYDRA still outperforms the state-of- state-of-the-an ah-ttt--rt baselines. This performance improvement may stem from the personalized reranker, which reranks the candidates based on their usefulness rather than just relevance. Additionally, the personalized adapter also adapts the model's generations align with user preferences. In addition, prompt-based methods tend to underperform when there is a distribution shift between the few-shot demonstrations and user queries. This issue can become even more severe under personalized scenarios, where there might be mismatch between the selected user history and the user query.
G.4 Effect of Retrievers
We investigate the effect of different retrievers in Table Under the black-box LLM personalization scenario, dense retrievers that capture semantic relevance (e.g., Contriever [16]) achieve similar performances to sparse retrievers (e.g. BM25 [39] in Table 1), which aligned with observations in the existing works [40].
Table 7: Additional experimental results with Contriever [16] as the retriever (k=4).
Dataset >)
LaMP-2N
LaMP-3
LaMP-
Method (1)
Ace.
F-1 + MAE RMSE 4 R-1 f R-L BLEU f
ICL-Random
0.660 0.288
0.560
0,917
0.163
0.148
0.159
RAG
0.680
0,293
0.540
0.883
0.179
0.152
2,082
HYDRA 0.800
0,414
0.380
0.730
0.189
0.173
2.501
G.5 Effect of Black-Box LLM Adapters
We evaluate the effectiveness of different black-box LLM adapters in Table 8. Due to the inacces- sibility model internal parameters, BBox-Adapter is the only black-box LLM adapter available, apart from our proposed HYDRA-Adape Based on Table 8, it can be observed that HYDRA- HYDRA-Adapter Adapter outperforms Bdax-edapter when applied under the same personalization settings (as discussed in Section 3.4). This result demonstrates the effectiveness of our adapter module design, which captures both global and user-specific knowledge further adapt to personalized model outputs.
Table 8: Effect of different black-bor LLM adapters (k:=4).
Dataset (-)
LaMP-3
LaMP-4
LaMP-5
Method (1)
MAE
RMSE.
R-1
R-L
BLEU
R-1 1 R-L. BLEU
Personalized BBox-Adapte [48]
0.540
0.860
0,151
0.135
1,285
0.394
0.337
5.478
HYDRA-Adapter
0.420
0.762
0.145
0.118
1.137
0.409
0.355
5.816
G.6 Additional Datasets
To further validate the effectiveness HYDRA, we conduct extensive experiments on two additional widely used personalization datasets [28], MovieLens- and Recipe, focusing on predicting users' personal ratings for movies or recipes based on their historical rating patterns. The experimental results (Table 9) demonstrate that HYDRA outperforms the best-performing baselines by 8.0% on Movielens and 10.3% on Recipe, respectively.
G.7 Efficiency Analysis
Assume that we have Nuain users in the training data and Nes users in the test set. We adopt the transformer architecture, specifically the Longformer-base models. Consequently, the time complexity of all stages should be proportional to that of the attention mechanism transformers, O(dL²), where L indicates the sequence length and d indicates the hidden dimension. Additionally, the training process will go through the transformer for T epochs, while inference only requires one. For each user in the HYDRA--Reanlee training data, we augment M random historical records from the user's corresponding profile. The retriever then retrieves the
as the reranker and the adapter base
21

---

Dataset (-)
MovieLens-IN
Recipe
Method (1)
MAE 4 RMSE
MAE
RMSE
tot-3.
0.780
1.208
0.880
1.149
ICL Ra-Rdnm (k=1)
0.880
1.2649
0.960
1.233
ICL-Random (k=2)
0,800
1.095
0.980
1.192
ICL-Random (k=4)
0.820
1.225
0.760
1.114
RAG (k=1)
0.800
1.166
0,980
1.192
RAG (k=2)
0.700
1.030
1.114
RAG (k=4)
0.640
1.000
0.820
1.049
PAG (k=0)
0.820
1.147
0.860
1.128
PAG (k=l)
0.760
1.010
0.800
1.063
HYDRA
0.600
0.908
0.720
0.938
Table 9: Experiments two additional personalization datasets, MovieLens- 1M and Recipe, focusing on predicting users' personal ratings for movies or recipes based on their historical rating patterns.
Method
Mode
Time Complexity
LaMP-2N
LaMP-2M
LaMP-3
LaMP-4
LaMP-5
HYDRA-Reranker
Training
( (M
1)TL2d)
31m10s
41m51s
50m37s
1h1m31s
1h8m16s
HYDRA-Reranker
Fit User O (M2
18m8s
21m17s
25m36s
33m52s
31m25s
HYDRA-Reranker
Inference
O (d)
3m4s
3m1s
3m7s
4m38s
5m14s
HYDRA-Adapter
Training
O MNrain k I2)
1h10m17s
262m16s
2h1m59s
3h56m47s
3h19m422
HYDRA-Adapter
Fit New User
(N L O(NeakL2dd)
28m15s
1h7m27s
1h19s
2h23m10s
1h59m2s
HYDRA-Adapter
Inference
4m16s
4m17s
4m17s
5m53s
5m59s
Table 10: Time complexity analysis with running time summary on the LaMP benchmark.
top-M (M = 20 by default) relevant historical records to form training samples. Thus, for each user, we collect M + training samples. For each user in the HYPER-Adapter training data, we consider all the user historical records. For each record, we leverage model randomness to generate k (k = 8 by default) samples for the adapter select. Consequently, we can have H training samples per user, where denotes the average number of histories per user.
H
Case Studies
H.1 Case Studies for Effectiveness of Reranker
Table 11 illustrates two observed cases during the reranking process. In Case 1, the retriever accurately retrieves the text in the optimal order, meaning that the top outputs are the most relevant to the target. In this scenario, the reranker successfully generates scores that align with the retriever, preserving the correct relevance. In Case 2, when the retriever fails to retrieve the text in the optimal order by relevance, the reranker assigns higher scores to the more relevant terms, thereby mitigating the incorrect retrieval ordering. This emphasizes the significance of HYDRA-reranker by offering an additional measurement dimension that is dedicated to assessing usefulness.
H.2 Error Analysis
Table 12 presents three common error types observed in the reranker that account for most general errors: (1) Saturated Scores: These errors occur when the ranker outputs overconfident scores (near 1.0) regardless of different inputs. Consequently, it becomes challenging to distinguish the best output as the differences among the scores are minimal. (2) Incorrect Scores: These errors arise when the ranker assigns higher scores to less relevant terms. As result, the likelihood of selecting the most suitable samples decreases, leading to suboptimal generation. (3) Divergent Scores: These emors occur when the ranker exaggerates the differences among the inputs. This may be attributed to the inputs being hard samples, where none of them are closer the target compared each other, making it difficult for the reranker to discern the most appropriate option.
22

---

Table 11: Case study for HYDRA -eranker....er The "Target" column indicates the ground-truth categoriza- tion, while "Gen" represents the category of the retrieved text. "Score" is the score generated by the reranker, and "Order" denotes the retriever's ranking during the retrieval process.
Retrieved
Target
Gen
Roads Regional Jail in the past months.
least eight people people have died at Hampton
politics
politics
0.3525
Eleven places will each at least $1 million to reduce their jail population.
Case
But still could the Republican with the politics best shot at stopping her.
crime
0.3031
"Some information that be might find embar- rassing needs to get out. Just fair."
politics
crime
0.2867
So good with this one. I would this is definitely worth the extra expeuse com- pare those shaped cream colored made out of aluminum that am sure you places ...
4
0.4488
The speaks for itself. These are the same high quality of their products are and the sizing true to
Case 2
These okay, have had better quality sheets the past less These wrinklee easy but if that does not bother you then it is all cool. I do have problems keeping those my bod.
same price.
5
0.5200
Sandstone coaster are the only to absorb all moisture from your glass. Good quality and worth your money pro- tect your tahles. have two sets of these and I recentl started these....
5
0.4384
I Prompts
Following the RAG-based framework [41] and the PAG-based framework [38], we have implemented the prompt design for the RAG and PAG-based baselines. The details of the prompt design can be found in Table 13 and Table 14, respectively. We follow the RAG framework in HYDRA. Additional examples of RAG framework for each task (k: 2) are available as follows.
RAG Prompt Demo for LaMP-2h (k=2)
category for the article: vaccine but instead of fighting off disease it attracts dogs" is "entertainment" and
the category for the article: "The president, and many of his European counterparte, had condemned Trump as dangerous during his run." " is "politics"
Which category does this article relate among the following categories? Just answer with the category name without further explanation. categories: [women, religion, politics, style beauty, entertainment, culture & arts, sports, science & technology, travel, business, crime, education, healthy living, parents, food & drink] article: The guitarist will not play with the band on its upcoming tour.
RAG Prompt Demo for LaMP-2M (k=2)
the tag for the movie: "In a dystopian future, totalitarian regime maintains
peace by subduing the populace with drug, displays emotion are punishable by death. A man in charge of enforcing the law rises to overthrow the system." is "dystopia", and the tag for the movie: "Cobb, skilled thief who commits corporate espionage by
23

---

Table 12: Three common error types of the reranker. The "Target" column indicates the ground- truth categorization, while "Gen" represents the category of the retrieved text. "Score" is the score generated by the reranker.
Error
Source
Retrieved text
Target
Gen
Score
great quality and durability.
Good price, durable, very good product. bought this to make my son birthday cake, 5 come out nice, and he love it it!! would buy again, and recommend to any one.
Saturated
also great price, i would buy again, really recommend to my
5
0.9657
Scores
friends to anyone, looking for this kind product,
excellent shoes, great product, great price, fast delivery, unfortunately had to return this
5
5
0.9427
product, because was ordered wrong size, but the return transaction with amazon was very prompt and smooth, would buy again, over all i would recommend to all my friends and anybody
great quality and durability. also great price, 5 would buy again, really recommend to all my friends and to anyone, looking for this kind product,
5
0.9692
Very nice, bristles are well seated. Love the wooden handle, Teels nice the hand, sturdy.
Love these looms, they are the right sizes and prefer them the knifty knitter looms.
Incorrect Scores
5
4
0.7221
This is very nice mask. Works very well when kept in the freezer. Actually keeps out the light, too.
5
5
0.7080
rough start. almost put the book 5 down. had guessed the ending the 4th person showed up. Very fast read.
5
0.7053
"She said she missed when she would go out with her girl and get dressed up!"
Anna Jones, 18, was sitting in parked car with friends when she was shot.
Divergent Scores
women style 0.4016
reptile even received a kiss at the MTV women style Movie & TV Awards.
0.7800
"Wouldn't i disrespect that queen like that," women style 0.2905 the comedian said of the hacklash she re- ceived for an awkward bit she did at the cere- mony.
Table 13: RAG prompt design for five LaMP tasks. Concat(-) concatenates the input strings in order, and PPEP(-) composes the prompt for each retrieved item from the profile. [INPUT] represents the task's 's input.
Task
Per Profile Entry Prompt (PPEP)
Aggregated Input Prompt (AIP) (PPCaa())PEI(PP).. PPEP(P.). )). ". and "). [NNPUT]
LaMP-2N
"the category for the article: "P:[text]" ""(ttccggory"""
LaMP-2M
"the tag for the movie: "deecrrriptin"]" [tag"
conca((PPEP(())).. PP) ". and [INPUT]
LaMP-3
Piscore the score for "text"
conca((PPEP(P.)) PPEP(P.)). ", and "). [INPUT]
LaMP-4
"t,ltille]" is the title for [text"
conca((PPEP(P.)) PPEP(P.)). ". and [INPUT]
LaMP-5
[titte"" is the title for "P[[abstract]"
concat((PPEP(P.)) PPEP(P.)). ", and "). Following
the given patterns [INPUT]
infiltrating the subconscious of his targets is offered chance to regain his old 1ife payment for task considered be impossible: \"inception\" implantation of another person's idea into target's subconscious." " is "sci-fi"
Which tag does this movie relate to among the following tags? Just answer with the tag without further explanation. tags: [sci-fi, based book,
24

---

Table 14: Summarization prompt design for the five LaMP tasks. [INPUT] represents the task's input.
Task
Prompt
LaMP-2N
Look the following past articles this journalist has written and determine the most popular category they write in. Answer the following form: most popular category: <category>
LaMP-2M
Which tag does this movie relate to among the following tags? Just answer with the lag name without further explanation
LaMP-3
Based on this user's past reviews, what are the most common scores they give for positive and negative reviews? Answer in the following form: most common positive score: <most common positive score>, most common negative score: <most common negative score>
LaMP-4
Given this author's k previous articles, to describe template for their headlines. I want to be able to accurately predict the headline given one of their articles. Be specific about their style and wording; don't 't tell me anything generie.
LaMP-5
Given this author's previous publications, try deseribe template for their titles. I want to be able accurately predict the title Only generate the template description, nothing else.
the papers from the abstract.
comedy, action, twist ending, dystopia, dark comedy, classic, psychology, fantasy, romance, phought-provoking,, social commentary, violence, true story] description:
beeking-time-bee insomniac and slippery soap salesman channel primal male aggression into shocking new form of therapy. Their concept catches on, with underground \"fight clubs\" forming in every town, until an eccentric gets in the way and ignites out-of-control spiral toward oblivion.
RAG Prompt Demo for LaMP-3 (k=2)
5 is the score for "Luckily, I married. But have already messages to all my single girlfriends that this book MUST reading for understanding the male mind and knowing how to deal with itl..all wrapped up in energetic, yet sympathetic package (how nice that Matt seems to actually like women!). especially enjoyed the tip boxes, and co-author Fadal's 's female wisdom. Fun!", and
HIGH PRIORITY
5 is the score for "I received this book direct from the author. This has no effect on my reviev. \n\aThis is book 3. If you've read this far it means you love the series, and loved this book. You cannot read this one without reading book 1 and 2. Otherwise you would be lost trying follow what happens. loved this book. just wish there was an epilogue to show more what happens after everything. I know there is book just about Gabriel so maybe that's my follow up to the story. Нappy reading!"
What is the score of the following review without further explanation. review: Colin Dodds does it again with Another Broken Wizard. genius at evoking a mood, scene, and character Dodd elegantly weaves all three together in this story with equal weight. have long been fan of Dodds' poetry, and this novel reaches many poetic points in both his of language and the intoricating (no pun intended) way he manages to paint scenes with words so that very specific moods and nuanced emotion come alive. Dodds is truly wordsmith, and he has created a host of characters, especially the main protagonists, that will be very hard to forget mostly because we know these people, either because they dwell within they live among us. Besides, it dann good noir story about one those forgotten New England towns, and it's full of blood, guts, grit, love, losa. Hard to put down highly recommended.
scale of to just
with 1,
3, 4,
because
RAG Prompt Demo for LaMP-4 (k-2)
"Social Media Gone Awry: Tips for Teens to Stay Safe" is the title for "Here are safe when using the Internet and other web-based
few tips keep your technologies. If you think it's an avkward conversation; you can hand them this blog to read.", ", and "If You See Something, Please Do Something to Prevent Child *buse" is the title for "Although the age of social media has dramatically lowered the threshold
25

---

privacy standards, many adults are still reticent about reporting their suspicions about child abuse and neglect." Generate headline for the following article: That explains how the Confederate flag, contraception and clean water deregulation got linked to fighting mosquitoes, the senator said.
RAG Prompt Demo for LaMP-5 (k-2)
"Accurate Estimators for Improving Minwise Hashing and b-Bit Minwise Hashing" is the context of title for "Minwise hashing is standard technique high-dimensions 0/1 search and databases for efficiently estimating set (e.g., vector) similarities. Recently, b-bit minwise hashing was proposed which significantly improves upon the original minwise hashing in practice by storing only the lowest bits of each hashed value, opposed uaing bits. b-bit hashing is particularly effective high similarities (e.g., the resemblance >0.5). However, there are other important applications in which not just pairs of high similarities matter. For example, many learning algorithms require all pairwise similarities and it is expected that only small fraction the pairs applications care more about containment (e.g., how much one object is contain by another object) than the resemblance. In this paper, we show that the estimators for minwise hashing and b-bit minwise hashing used in the current most
applications which mainly concern seta
similar. Furthermore, many
systematically improved and the improvementa significant for set pairs of low resemblance and high containment."" ", "b-Bit Minwise Hashing for Large-Scale Linear SVM" is the title for "In this paper,
practice
we propose to (seamlessly) integrate b-bit minvise hashing with linear SVM to substantially improve the training (and testing) efficiency using much smaller memory, with easentially resemblance matrix, the minwise hashing matrix, and the b-bit minwise hashing matrix are all positive definite matrices (kernels). Interestingly, our proof for the positive definiteness of the b-bit minwise hashing kernel naturally suggests simple strategy to integrate b-bit hashing with linear SVM. Our technique is particularly useful when the data can not fit memory, which an increasingly critical issue in large-scale machine learning. Our preliminary experimental results on publicly available webspam dataset (350K samples and 16 million dimensions) verified the effectiveness of algorithm. For example, the training time was reduced merely few seconds. addition, our tech can be easily extended to many other linear nonlinear machine learning applications such as logistic regression.
loss of accuracy. Theoretically, prove that the
Generate title for the following abstract of paper: We propose skewed stable random projectionsfor approximating the \u03bith frequency moments of dynamic data streams
\u03b1 \uffid 2), show sample complexity (number of projections) k G 1 \u01eb2 log \u03b4 \u00b4 where \u01eb log(1+\uOteb) = 0 (uu0leb) as \u03b1 1, i.e., \u03b1 \u00b1 \ufffd with \ufffd 0. Previous results based on symmetric stable random projections (12, 16) required non-zero constant D(Au0leb), practically important. For example, \ufffd might the \'decay ratel" or \"interest rate, \" which is usuall small; and hence one might view skewed stable random projectionsas total value in the future, taking in account of the effect of decaying inter accruement. We consider the popular Turnstile data stream model. The input data stream at (1, It) arriving sequentially describes the underlying signal A, meaning At(i) At 1(1) It, 2 (1, D). We allow the increment It to be either negative (i.e., del etion). By definition, the
when Aufffd 0. The
Aufffd
V"gneralized counter\" for estimating the
positive (i.e., insertion) F(Au03b1)
i=1 [At(i) \u03b1. Our method only requires that, the time t for the evaluation,
Au03bith frequency
26

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We propose novel model factorization framework for black-box large lan- guage model personalization.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We have discussed the limitations of HYDRA in Appendix A.1.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model sell-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
27

---

Answer:
[NA]
Justification:
This paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof's can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification: We provide all the information needed to reproduce the main experimental results of the paper in Section 4 and Appendix F.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, descríbing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e-g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
5. Open access to data and code
28

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification: We provide the code in zzip file during the submission and peer-review stage. We will release the code repository and model checkpoints for transparency and reproducibil- ity GitHub upon acceptance. Data is publicly available in LaMP benchmark [41].
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cm cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so åÄIJNoâÄI is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: /nniss. cccpublic/guiciiocccccccccoccpce httpi//ggiidee//edeuuuiiiieeee pwweeeubmmssiooooll
for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if' applicable).
Providing much information possible in supplemental material (appended the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
We provide all the experimental details in Section 4.
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can provided either with the code, appendix, or as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: We set the temperature of LLMSs to zero for all baselines in order to ensure stable experimental results.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
29

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
All the computing infrastructure information has been provided in Appendix F.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics https: htapa////eeuuiieee /nwuriius...../ cc httpppuuli/BBhhsSciiiiieeeeo
Answer: [Yes]
Justification:
We conform with the NeurIPS code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- cration due to laws or regulations their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: We have discussed both potential positive socictal impacts and negative socictal impacts in Appendix A.2.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
30

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
This paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
We have properly cited all code, data, and models we used in the paper.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
31

---

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: This paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moode as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: This paper does involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
32

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
33


Reviews:
[{"review_id": "HQV7seD4ZF", "reviewer": "Reviewer_xKC3", "summary": "The paper introduces HYDRA, a model factorization framework designed to personalize large language models (LLMs) without modifying their internal parameters. HYDRA addresses the challenge of personalizing inherently opaque, black-box LLMs through a retrieval-augmented workflow. This method enhances personalization by using historical data to effectively capture user-specific preferences. HYDRA consists of two primary components: a personalized reranker that selects relevant information from the user's history, and a personalized adapter that tailors the LLM’s outputs to individual preferences. By integrating both global knowledge and local user behaviors, HYDRA improves personalization effectiveness. Extensive testing shows that HYDRA surpasses other state-of-the-art methods in personalization tasks, proving its ability to deliver tailored experiences without altering the fundamental model parameters.", "questions": "1) Limited Evaluation Metrics: The evaluation primarily emphasizes improvements in accuracy and benchmark performance. However, the use of the LAMP dataset to measure personalization is quite limited and fails to convincingly demonstrate effectiveness.\n2) Dependency on High-Quality Data: HYDRA's performance is significantly dependent on the quality of historical user data. If the data is sparse or noisy, the functionality of both the reranker and adapter could be undermined. Exploring and integrating methods to effectively manage such data limitations is crucial.\n3) Scalability Concerns: The discussion in the paper about the model's effectiveness covers only a limited number of users. Conducting scalability tests with much larger datasets and more diverse user bases could help in affirmatively validating the framework's efficiency on a larger scale.\n4) User Privacy Concerns: HYDRA uses user-specific data, raising potential privacy issues.\n5) Handling Dynamically Changing Preferences: How does HYDRA manage continuously evolving user preferences? Is there an efficient mechanism in place to update user models without extensive retraining?\n6) Limitations of Reranker and Adapter: What are the constraints on the learning capacity and efficiency of the reranker and adapter modules? Are there situations where these components fail to achieve optimal results?\n7) Integration Challenges: What are the main challenges in integrating HYDRA into existing large-scale systems, especially those that already have personalization frameworks in place?", "limitations": "NA", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "Originality: HYDRA introduces a novel model factorization method to personalize black-box large language models (LLMs). This marks a significant advancement beyond traditional techniques that necessitate access to model parameters. Distinct from existing methods which depend on prompt design or fine-tuning, HYDRA employs a dual-component system comprising a reranker and an adapter. This system enhances personalization without the need to modify the LLM's internal parameters, offering effective personalization within the limitations of black-box models. \n\nClarity: The paper is well-organized and clearly describes the components and operation of HYDRA. The explanations are concise and straightforward. Additionally, the inclusion of figures and detailed descriptions of algorithmic steps offers a clear view of the model’s structure and function, which helps in understanding the model’s mechanics.", "weaknesses": "1) Limited Evaluation Metrics: The evaluation primarily emphasizes improvements in accuracy and benchmark performance. However, the use of the LAMP dataset to measure personalization is quite limited and fails to convincingly demonstrate effectiveness.\n2) Dependency on High-Quality Data: HYDRA's performance is significantly dependent on the quality of historical user data. If the data is sparse or noisy, the functionality of both the reranker and adapter could be undermined. Exploring and integrating methods to effectively manage such data limitations is crucial.\n3) Scalability Concerns: The discussion in the paper about the model's effectiveness covers only a limited number of users. Conducting scalability tests with much larger datasets and more diverse user bases could help in affirmatively validating the framework's efficiency on a larger scale.\n4) User Privacy Concerns: HYDRA uses user-specific data, raising potential privacy issues.\n5) Handling Dynamically Changing Preferences: How does HYDRA manage continuously evolving user preferences? Is there an efficient mechanism in place to update user models without extensive retraining?\n6) Limitations of Reranker and Adapter: What are the constraints on the learning capacity and efficiency of the reranker and adapter modules? Are there situations where these components fail to achieve optimal results?\n7) Integration Challenges: What are the main challenges in integrating HYDRA into existing large-scale systems, especially those that already have personalization frameworks in place?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your detailed suggestions and comments. Please find the corresponding responses as follows:\n\n> W1: Limited Evaluation Metrics.\n\n**A:** In line with previous personalization research [1]- [3], LaMP serves as **a standard personalization benchmark that has been widely used ** in evaluating personalization for LLMs. To further validate the effectiveness of HYDRA, we conduct extensive experiments on two additional widely used personalization datasets [4]**, focusing on predicting users' personal ratings for movies or recipes based on their historical rating patterns. The experimental results (**Table R1**) demonstrate that HYDRA outperforms the best-performing baselines by **8.0% on Movielens and 10.3% on Recipe**, respectively. \n\n> W2: Dependency on High-Quality Data. \n\n**A:** We conducted experiments to study the impact of data quality on historical user data, as illustrated in Figures 3 and 5.\n- **When historical user data is sparse, in the case of a limited number of users**, we observe that HYDRA reaches over 90% of best performance with only 20% of training data, as shown in **Figure 3 (a)-(c)**.\n- **Similarly, in the case of limited user history length**, HYDRA demonstrates its robustness by consistently capturing user-specific preferences, as shown in **Figure 3 (d)-(f)**.\n- **When historical user data is noisy, when user behavior shifts**, HYDRA continues to outperform state-of-the-art baselines, even when the query is not relevant to all historical records, as shown in **Figure 5**.\n\nHYDRA maintains its robustness and consistency, even when dealing with sparse or noisy historical user data, by leveraging shared knowledge acquired from the base model.\n\n> W3: Scalability Concerns.\n\n**A:** Our experimental setup, including the number of users, aligns with other recent studies in personalized language modeling (**e.g., 100 users in [1] and [2]**). To address your valid concern, we conduct additional scale-up experiments (**Table R2**) to evaluate HYDRA **with an increased number of users, increasing from 100 to 1000, across all five tasks**. Our findings from the scale-up experiments show that HYDRA maintains its performance advantages over baselines as the number of users increases. \n\n> W4: User Privacy Concerns.\n\n**A:** We acknowledge the potential privacy concerns associated with the personalization task, as it naturally entails using individual users' historical preferences to customize the generations of LLMs. We would like to highlight that **HYDRA does not introduce any additional privacy risks compared to existing baselines**. While we acknowledge the significance of data privacy, achieving zero data leakage may be beyond the scope of this current work.\n\nWith the increasing demand for privacy preservation, HYDRA's modular design offers the flexibility to remove the RAG component, thus downgrading it to HYDRA-Adapter (see Table 2). HYDRA-Adapter is able to achieve a higher level of privacy preservation without transferring any user historical data compared to existing baselines. As a potential extension, we can enhance privacy by collecting data from users who have given their consent to share their information, considering them as \"anchor points”, and then map them to the most similar anchor users (groups). \n\n> W5: Handling Dynamically Changing Preferences.\n\n**A:** While dealing with dynamic changes may be orthogonal to our current personalization task, HYDRA is intrinsically compatible with continual learning. It enables efficient updates to user models as new data becomes available and eliminates the need for extensive retraining, achieving the balance between performance and efficiency. Specifically, we can follow a periodic continual training pipeline to (1) initially train the base model and personalized heads using the available user data and (2) collect a sufficient amount of new user records to adapt to dynamically changing preferences. Moreover, given that the shared knowledge among the entire user group may not significantly change over a short period, it is feasible to update the base model at a relatively slower pace than the head model.\n\n> W6: Limitations of Reranker and Adapter.\n\n**A:** We acknowledge the importance of understanding these limitations on the learning capacity and efficiency. As elaborated in the Limitation Section, the process of creating labeled datasets for training, especially for the HYDRA-reranker, introduces additional computational costs. In addition, the performance of the HYDRA-adapter may degrade when faced with user preferences that significantly deviate from the patterns seen during training. We will incorporate (1) newly conducted scale-up experiments in response to W3, as well as (2) additional analysis on user behavior shift experiments in Figure 5, in order to provide a comprehensive discussion on the learning capacity and efficiency of the reranker and adapter modules in the updated manuscript.\n\n> W7: Integration Challenges.\n\n**A:** To the best of our knowledge, **there are currently no widely available large-scale personalization systems for black-box LLMs**. However, we believe that the integration of HYDRA would be relatively simple due to its modular design. We can achieve this by (1) Leveraging existing user data to train HYDRA's base model and a set of personal headers, and (2) Quickly adapting to new users by composing personalized headers using a combination of existing user headers while leveraging the knowledge gained from existing users.\n\n[1] Tan et al. \"Democratizing large language models via personalized parameter-efficient fine-tuning.\" arXiv 2024.  \n[2] Tan et al. \"Personalized Pieces: Efficient Personalized Large Language Models through Collaborative Efforts.\" arXiv preprint arXiv 2024.  \n[3] Tang et al. \"Step-Back Profiling: Distilling User History for Personalized Scientific Writing.\" arXiv 2024.  \n[4] Lyu et al. \"LLM-Rec: Personalized Recommendation via Prompting Large Language Models.\" NAACL 2024.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "LbReh7F5jX", "reviewer": "Reviewer_KhW9", "summary": "This paper provides a black-box LLM personalization framework that explores global and local knowledge from user’s historical behaviour through model factorization.", "questions": "Please refer to my concerns above.", "limitations": "The authors did provide details in Appendix A.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "Strengths:\n\n1.\tThe paper is straightforward. The method is reasonable\n2.\tThis is good to see the authors provided many details including analyses and case studies in Appendix G and H\n3.\tExtensive experiments were conducted with implementation details and prompts were also provided.\n4.\tModel factorization is a good idea", "weaknesses": "Weaknesses:\n\n1.\tThe performance is not convincing. For example, in Table 1, it’s interesting to see that ICL-Random seems to be perform on par with HYDRA (second-best baseline on LaMP-2M, LaMP-5), and even better than RAG / PAG baselines. It’s not clear to me why using random items from user behaviour history is doing much better; please give more details.\n2.\tFrom the design until experiments, the authors should provide more details (or I may missed it?) about the tasks that benefit by shared (global), the tasks that benefit individual (local) preference, and the tasks that benefit from both. I consider model factorization is a big factor, but not much explanation and analyses were given for that part. In that way, it will further strengthen the paper \n3.\tIn addition, in Table 2, for example, HYDRA -P.-Adapter&Reranker has the same performance as HYDRA. Section 4.3 should provide more analyses and explanation about that. \n4.\tEven though the authors provided implementation details, I still believe the source code should be released or more stronger analyses should be given, given the performance results I mentioned above.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your detailed suggestions and comments. Please find the corresponding responses below:\n\n> W1: The performance is not convincing.\n\n**A:** We appreciate your thorough observations regarding the performance of ICL-Random in comparison to other baselines and HYDRA. We would like to explain the consistent observations, including (1) ICL-Random performs equally or better than the RAG and PAG baselines, and (2) the performance of RAG and PAG does not necessarily increase with the number of retrieved items (k). \n\nThe worse performance of RAG/PAG than ICL-Random indicates **the complexity of the LaMP dataset and the personalization task**, which contains **implicit user preferences** that are NOT easily captured by straightforward relevance (e.g., RAG) or profile matching (e.g., PAG). In addition, RAG and PAG may **introduce noisy information through retrieval**, even retrieving a larger number of retrieved items (k) does NOT necessarily lead to better results. \n\nInstead of retrieving the most relevant items, HYDRA employs a retrieve-then-rerank workflow to **rerank retrieved items based on their usefulness**, thereby enabling a more comprehensive capture of implicit and complex personalized information. This highlights the significance of a personalization framework like HYDRA, which effectively identifies and utilizes complex patterns of user behavior. Despite the relatively strong performance of ICL-Random in LaMP-2M and -5, **HYDRA still consistently outperforms all baselines across all tasks, including ICL-Random**. We will provide a more comprehensive performance analysis in the revised manuscript.\n\n> W2: Additional details on the base model. \n\n**A:** Thank you for your valuable suggestions. In our initial submission, we included dataset and task details in Appendix D. In addition, we would like to emphasize the distinction between prediction and generation tasks (Table 5 in Appendix D) with regard to the advantage of global or local preference. **Prediction tasks, such as LaMP-2N, -2M, and -3, benefit more from global knowledge.** This is because these prediction tasks can utilize general patterns learned from the entire user base, which particularly benefits users with limited historical records (i.e., cold-start). On the other hand, **generation tasks, such as LaMP-4 and -5, benefit more from local preference**. This is because personalized heads focus on customizing outputs for individual users, which offers enhanced personalization for users with extensive historical records.\n\nThe model factorization of HYDRA consists of two main components:   \n- shared base model capturing global information across all users and  \n- personalized heads capturing user-specific preferences.   \n\nThe integration of shared and personalized components provides \n- complementary information, with global knowledge serving as a solid knowledge foundation, while personalized heads refine outputs for individual users; \n- model flexibility, allowing the balance between global and personalized information based on the available user history; and \n- model robustness, mitigating overfitting to individual user patterns by grounding personalization in global knowledge. \n\nAblation studies demonstrate the effectiveness of each component across both classification and generation tasks (Table 2). In the revised manuscript, we will ensure a clearer emphasis on the specific task details of global and local preference with model factorization.\n\n> W3: HYDRA -P.-Adapter&Reranker has the same performance as HYDRA.\n\n**A:** We here clarify that across all five tasks, the HYDRA-P.-Adapter&Reranker achieves similar performance compared to the full HYDRA model **only** in LaMP-2N. This is due to task specificity, as LaMP-2N is a prediction task that involves predicting news articles into one of 15 categories based on the journalist. Given an article written by a user, the model predicts its category using the user's history of articles and their corresponding categories. As categorized in our response to W2, **LaMP-2N is the easiest task that relies more on reasoning based on objective facts and requires less personalization**. It is also important to note that even for specific tasks like LaMP-2N, the integration of the adapter and reranker will not negatively impact the model's performance. The ablation study in Table 2 still demonstrates the effectiveness of each component. We will incorporate this expanded analysis into Section 4.3 in the updated manuscript.\n\n> W4: Code release.\n\n**A:** Thank you for your suggestion. We have **included the source code as a zip file in the supplementary material** in the initial submission and **will publicly release the code on a GitHub repository to ensure transparency and reproducibility**. As per your suggestion, we will enhance the updated manuscript by including more comprehensive experimental results and rigorous analyses, which will encompass:\n- In-depth analysis of model performance regarding baseline comparisons (in response to **W1** above), task details with model factorization (**W2**), and ablation studies (**W3**);\n- Extensive experimental results on two additional personalization datasets (**Table R1**);\n- Scale-up experiments with additional analysis on an increased number of users (ranging from 100 to 1000, **Table R2**);\n- Efficiency test with computational complexity analysis (**Table R3**); \n- Effect of inherent randomness in LLM generation with two additional randomness-injected baselines (**Table R4**);\n- Simulation experimental results with analysis of significant disparities in user behavior history, including 50% of the most active users and 50% of the most inactive users (**Figure R1**).\n\nThank you again for your review! We hope our responses address your concerns. Please let us know if you have any further questions, and we will be happy to discuss them further.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "OD3Mouhaux", "reviewer": "Reviewer_AVA9", "summary": "The paper proposes HYDRA, a learning-based model factorization framework that captures both user-specific and shared behavior patterns to enable effective personalization within black-box LLMs. The framework involves training a reranker to prioritize the most useful information from top-retrieved relevant historical records, and training an adapter to align the output with individual user-specific preferences, thereby eliminating the reliance on access to inherent model parameters of black-box LLMs. The experimental results demonstrate that HYDRA outperforms existing state-of-the-art prompt-based methods by achieving an average relative improvement of 9.01% across five diverse personalization tasks in the LaMP benchmark.", "questions": "1. Could you provide the time complexity analysis, training runtime, and inference time?\n2. Could you provide a more detailed introduction of the base model?\n3. HYDRA-Adapter Inference uses LLM to generate b candidates, then selects the one with the highest score as the final answer. Considering the inherent randomness in LLM generation, is this fair to baselines?\n4. How does HYDRA perform in scenarios where there are significant disparities in user behavior history, such as some users having dense histories while others have sparse ones?", "limitations": "1. HYDRA-Adapter Inference utilizes an LLM to generate b candidates and selects the one with the highest score as the final answer. This approach substantially increases the inference time.\n2. As the number of users increases, the number and parameters of user-specific heads also increase accordingly. This not only raises concerns about computational expenses but also introduces storage issues, potentially posing challenges for practical applications.\n3. The effectiveness of HYDRA heavily depends on the quality and relevance of historical data, which may not always be available in the real world.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper proposes a novel method for personalizing black-box LLMs by combining user-specific behavior with shared knowledge, which addresses limitations of previous methods.\n2. The experimental results show that HYDRA achieves an average relative improvement of 9.01% over existing state-of-the-art prompt-based methods across five diverse personalization tasks in the LaMP benchmark.\n3. The authors plan to release the code repository and model checkpoints, promoting transparency and reproducibility in future research.", "weaknesses": "1. The Algorithm Details in Appendix C indicate that HYDRA requires multiple stages of training. However, the paper does not provide an analysis of HYDRA's efficiency.\n2. The paper only describes the user-specific head as a single layer of a feed-forward network, without providing details about the base model.\n3. HYDRA-Adapter Inference utilizes an LLM to generate b candidates and selects the one with the highest score as the final answer. This approach substantially increases the inference time.\n4. As the number of users increases, the number and parameters of user-specific heads also increase accordingly. This not only raises concerns about computational expenses but also introduces storage issues, potentially posing challenges for practical applications.\n5. The effectiveness of HYDRA heavily depends on the quality and relevance of historical data, which may not always be available in the real world.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your detailed suggestions. Please find our responses below: \n\n> W1&Q1: Time complexity.\n\n**A:** We summarize (1) the time complexity for the different stages and (2) the time consumed for training and inference on 100 training users and 50 test users in **Table R3**. Please see **official comment** below for detailed definitions. It is important to note that although current calculations are based on centralized computing, the fitting process for new users and inference can be distributed to user-end machines. This allows for simultaneous updates of different head models. The potential of parallelism and distributed training can further enhance the efficiency of HYDRA.\n\n> W2&Q2: Base model.\n\n**A:** Both HYDRA-Reranker and HYDRA-Adapter utilize the lightweight LongFormer-Base (110M) as the backend language models, as illustrated in Section 4.1. We include more details of the base model in the updated manuscript.\n\n> W3: Inference time.\n\n**A:** We acknowledge that the generation and selection of candidate solutions may lead to an increase in inference time. However, to the best of our knowledge, an alternative solution for black-box LLM personalization may not exist. We will include a time complexity analysis (**Table R3**) in the updated manuscript and prioritize improving the inference efficiency as an important future undertaking. We will be more than happy to include if the reviewer could suggest any alternative to black-box LLM personalization for inference comparison.\n\n> W4: Computational expenses.\n\n**A:** We would like to clarify that our proposed HYDRA design is more cost-efficient compared to many state-of-the-art baselines, such as tuning the entire LLM for each user. The personalized header for each user only requires 0.6M parameters. The total number of parameters in 10000 user-specific head models is almost equivalent to recent popular LLMs, specifically LLaMA-3-8B. Additionally, the personalized heads for each user can be trained and deployed on devices, eliminating the need for additional storage in the data center. We acknowledge that as the number of users increases, storage and computational expenses may gradually become concerns. We can further reduce the number of personalized heads and then combine incoming user-personalized heads with them as bases.\n\n> W5: Historical data.\n\n**A:** We conducted experiments to study the impact of data quality and relevance on historical user data. Regarding the quality of historical data,\n- **When historical user data is sparse, in the case of a limited number of users**, we observe that HYDRA reaches over 90% of best performance with only 20% of training data (Figure 3).\n- **Similarly, in the case of limited user history length**, HYDRA demonstrates its robustness by consistently capturing user-specific preferences (Figure 3).\n\nRegarding the relevance of historical data, instead of retrieving the most relevant items in RAG/PAG, HYDRA employs a retrieve-then-rerank workflow to **rerank retrieved items based on their usefulness**, thereby enabling a more comprehensive capture of implicit and complex personalized information.\n- **When historical user data is noisy, in the case of user behavior shifts**, HYDRA continues to outperform state-of-the-art baselines, even when the query is not relevant to all historical records (Figure 5).\n\nHYDRA maintains its robustness and consistency, even when dealing with sparse or noisy historical user data, by leveraging shared knowledge acquired from the base model. \n\n> Q3: Randomness in LLM generation.\n\n**A:** We emphasize the proposed method exploits the inherent randomness of LLM generation in the HYDRA-adapter to achieve more personalized responses and improve model performance. It is important to note that **even equipped with the randomness in LLM generation, existing baselines cannot take advantage of the randomness due to their inherent design**. As indicated in **Table R4**, the inherent randomness in LLM generation leads to almost no change or a significant decline in performance in other baselines due to a lack of capability in effectively utilizing the diversity in generation, which guarantees a fair comparison with HYDRA.\n\nIn addition, we conduct additional experiments to mitigate the potential influence of randomness by replacing the HYDRA-Adapter component with (1) HYDRA (random), which randomly selects the final answer from the generated candidates, and (2) HYDRA (SC), which utilizes self-consistency to select the most common answer from the generated candidates. Note that self-consistency cannot be leveraged in generation tasks as there are no definite answers in model generation (LaMP-4 and -5). We observe that HYDRA-Adapter still outperforms these two baselines that introduce randomness, thus demonstrating the validity of the HYDRA-Adapter design.\n\n> Q4: Significant disparities in user behavior history.\n\n**A:** We would like to clarify that the current random selection of training users includes significant disparities in the number of historical records, ranging from less than 20 to over 200. Thus, our experiments simulate real-world scenarios, where some users have dense histories while others have sparse ones. HYDRA has shown superior performance compared to all baselines.\n\nTo consider more extreme cases, **we retrain HYDRA on a mixture of 50% users with the fewest interactions and another 50% users with the most interactions** (**Figure R1**). The experimental results demonstrate that HYDRA consistently outperforms existing baselines even under extreme cases. Compared to the previous random selection of training users, HYDRA achieves relatively lower performance due to the imbalance of training samples for dense users and sparse users. By leveraging the global information in shared parameters, knowledge can be effectively transferred from dense users to sparse users, thereby enabling further personalization through the utilization of sparse user-specific head models.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
Dear reviewers,

We sincerely appreciate the time and effort dedicated to evaluating our work. We have summarized the additional experiments and analyses conducted during the rebuttal phase, and we are committed to incorporating them in the revised manuscript. 

Our newly added main experiments and analysis (attached PDF file) include: 
- **Experiments on two additional personalization datasets** (**Table R1**). We demonstrate that HYDRA outperforms the best-performing baselines by 8.0% on MovieLens-1M and 10.3% on Recipe, respectively;
- **Scale-up experiments with additional analysis on an increased number of users** (ranging from 100 to 1000, **Table R2**). We evaluate HYDRA across all five tasks and show that HYDRA maintains its performance advantages over baselines as the number of users increases;
- **Time efficiency examination with computational complexity analysis** (**Table R3**). We summarize the time complexity for the different stages and the time consumed for training and inference on 100 training users and 50 test users ; 
- **Effect of inherent randomness in LLM generation with two additional randomness-injected baselines** (**Table R4**). We show that even equipped with the randomness in LLM generation, existing baselines cannot take advantage of the randomness due to their inherent design;
- **Experiments with analysis of significant disparities in user behavior history**, including 50% of the most active users and 50% of the most inactive users (**Figure R1**). The experimental results demonstrate that HYDRA consistently outperforms existing baselines even under extreme cases.

We would like to further emphasize our main contributions as follows: 
- We propose HYDRA, the ***first black-box*** LLM personalization framework that effectively mines user behavior history and adapts to user preferences for enhanced user experience;
- HYDRA integrates ***shared (global) knowledge*** from the base model and ***individual (local) preference*** from multiple user-specific heads through model factorization to deliver generalizable personalization; and 
- HYDRA significantly ***outperforms existing personalization baselines*** across five diverse tasks in the LaMP benchmark, introducing one of the first learning-based solutions that achieves more effective adaptation to individual users in black-box LLMs.

Please find the point-to-point response with additional details in the following rebuttal section. We sincerely hope that our responses have enhanced the paper's quality and addressed your concerns. If you have any additional suggestions or comments, please don't hesitate to share them. We look forward to engaging in a constructive discussion during the rebuttal phase. Thank you again for your understanding and consideration.

Best Regards,   
Authors

Meta-review: The paper proposes a model factorization framework for black-box LLM personalization. The proposed framework consists of a reranker to retrieve relevant user history records, and an adapter to align the output with user preferences. Both reranker and adapter are decomposed into a base model for capturing global knowledge and multiple user-specific heads for personalization. 

The reviewers noted and mostly agreed on the novelty of the approach, clarify of the presentation and extensive experiments. 

The reviewers raised concerns on dependency on high-quality data, handling dynamically changing user preferences, training and inference efficiency, clear description of the base model etc.. The authors answered these questions during rebuttal. The authors are strongly encouraged to incorporate the analysis and results provided during the rebuttal and any other changes/updates needed into the final version to address the raised issues and further improve the paper. 

From the additional results, it seems the training and inference cost of the proposed model is very high, e.g., training on 100 users can take a few hours. The size of the proposed model is also very large and not scalable with number of users, e.g., the personalized header for one single user requires 0.6M parameters. Both efficiency and model size can be big concerns for practical use of the proposed model. These issues need to be addressed in either the revised version or in an extended version of the paper.
Decision: Accept (poster)