Personalized Pieces: Efficient Personalized Large Language Models through Collaborative Efforts (2024)

Abstract: Personalized large language models (LLMs) aim to tailor interactions, content, and recommendations to individual user preferences. While parameter-efficient fine-tuning (PEFT) methods excel in performance and generalization, they are costly and limit communal benefits when used individually. To this end, we introduce Personalized Pieces (Per-Pcs), a framework that allows users to safely share and assemble personalized PEFT efficiently with collaborative efforts. Per-Pcs involves selecting sharers, breaking their PEFT into pieces, and training gates for each piece. These pieces are added to a pool, from which target users can select and assemble personalized PEFT using their history data. This approach preserves privacy and enables fine-grained user modeling without excessive storage and computation demands. Experimental results show Per-Pcs outperforms non-personalized and PEFT retrieval baselines, offering performance comparable to OPPU with significantly lower resource use across six tasks. Further analysis highlights Per-Pcs’s robustness concerning sharer count and selection strategy, pieces sharing ratio, and scalability in computation time and storage space. Per-Pcs’s modularity promotes safe sharing, making LLM personalization more efficient, effective, and widely accessible through collaborative efforts.

## introduction personalization involves mining user's history data to tailor and customize a system's interaction, content, or recommendations to meet the specific needs, preferences, and characteristics, of individual users @xcite @xcite . by adapting to each user's unique preferences, personalization enhances the user experience and has become increasingly important in content recommendation (li et al., 1 code is available at https://github.com/tamsiuhin/ per-pcs { } sharer peft target user history target personal peft peft pieces figure 1 : user personal peft parameter sharing framework: sharers provide parts of their peft parameters (peft pieces). using the target user's history data, we recycle the peft pieces shared by anchor users and assemble the target user's personal peft. previous endeavors to personalize llms can be categorized into prompt-based and parameter-efficient fine-tuning (peft)-based methods. prompt-based personalization involves designing prompt templates to help llms understand user preferences, using methods such as vanilla personalized prompting @xcite , retrievalaugmented prompting @xcite , and profile-augmented prompting @xcite . however, prompt-based methods expose user data to centralized llm and can be easily distracted by irrelevant user history data, which retrieval can hardly avoid @xcite . peftbased personalization methods focus on storing users' preferences and behavior patterns in personal lightweight parameters. oppu @xcite is the pioneering work that stores users' preferences and behavior patterns in personal peft parameters, showing the superiority of model ownership and better user behavior pattern generalization compared to prompt-based methods. despite their success, the "one-peft-per-user" paradigm is computationally and storage-intensive, especially for large user bases. for instance, using oppu for personalized product rating prediction requires about 20 minutes of training on a single rtx a6000 gpu and 17 mb of storage per user, scaling linearly with the number of users. additionally, individually owned pefts limit community value, as personal models cannot easily share knowledge or benefit from collaborative improvements. inspired by the exhaustiveness of human preferences @xcite , we propose the per-sonalized pieces (per-pcs) framework, which allows users to safely share a small fraction of their peft parameters and build personalized llms efficiently through collaborative efforts (figure 1 ). specifically, we first select representative users as sharers and train their pefts with their personal history data. we then break down the peft parameters into pieces, inject a routing gate for each piece, and update the gate parameters while keeping the other parameters frozen with a few steps. these pieces are added to a pieces pool along with their corresponding gates for selection. in the assembly stage, per-pcs feeds the target user's history data and selects peft pieces from the pieces pool in an auto-regressive way, recycling the peft modules in the pieces pool. by processing all the history data through this pipeline, we determine the peft piece choices for all layers and obtain the target user's personal peft. per-pcs is trainingfree and only requires the storage of sharer's index and corresponding composition weights, making it computation and storage efficient. experimental results show that per-pcs outperforms non-personalized and peft retrieval base-lines, delivering performance comparable to oppu but with significantly reduced resource requirements across six personalization tasks in the lamp benchmark @xcite . further studies highlight per-pcs's robustness against sharer count and selection strategy. even when sharers consent to share only a small portion of their pieces, per-pcs maintains strong performance, comparable to scenarios where all pieces are shared. time analysis reveals that per-pcs is 38 times more efficient in storage and 7 times more efficient in computation costs compared to oppu. these findings underscore the potential of personalizing generalpurpose llms by integrating modular and collaborative parametric knowledge from personal peft pieces shared by users. in summary, the contribution of per-pcs is the pioneering framework that enables users to safely share personal pefts, facilitating efficient and finegrained llm personalization through collaborative efforts. unlike oppu, where personal pefts benefit only the individual user, per-pcs allows users to share a limited portion of their peft parameters with others, ensuring user privacy. for target users, per-pcs maintains model ownership and supports fine-grained user modeling comparable to oppu, but with significantly reduced storage and computation resources. we envision per-pcs as an initiative to encourage users to share their personal peft pieces, fostering collaboration in personalizing llms to create value for others. this approach preserves sharer privacy and reduces the carbon footprint of peft-based personalized llm. ## personalized pieces (per-pcs) we introduce per-pcs, a novel framework to empower llm personalization with modular and collaborative peft pieces within the community (fig. 2 ). we first adapt non-personalized base llms to the task without incorporating personal preferences ( §2.2). we then train personal peft and post-hoc gates for sharers and add them to the pool ( §2.3). finally, we assemble the target user's peft using their history and pieces from the pool ( §2.5). ## experiment settings datasets we adopt the large language model personalization (lamp) benchmark @xcite for our experiments, which consists of six public language model personalization tasks, including three text classification tasks (personalized citation identification, movie tagging, and producing rating) and three text generation tasks (personalized news headline generation, scholarly title generation, and tweet paraphrasing). 3 we randomly task metric non-personalized rag pag peft retrieval per-pcs (ours) oppu @xmath2: personalized citation identification acc ↑ .536 .576 .584 .656 .480 .576 .656 .592 * .579 .672 * .560 .584 .664 f1 ↑ .532 .568 .567 .654 .361 .556 .653 .589 * .564 .664 * .553 .567 .658 lamp-2: personalized movie tagging acc ↑ .340 .301 .417 .469 .336 .419 .477 .410 * .452 * .499 * .463 .467 .507 f1 ↑ .268 .255 .331 .375 .265 .326 .380 .301 * .343 * .383 * .320 .349 .385 lamp-3: personalized product rating mae ↓ .645 .336 .301 .301 .431 .305 .299 .262 * .272 * .262 * .262 .272 .266 rmse ↓ 1.277 .662 .639 .618 .897 .622 .608 .558 * .580 * .561 * .558 .580 .561 lamp-4: personalized news headline gen. r-1 ↑ .175 .179 .201 .204 .189 .193 .197 .193 * .205 * .205 .193 .205 .209 r-l ↑ .158 .162 .183 .184 .171 .175 .178 .174 * .186 * .186 .173 .185 .190 lamp-5: personalized scholarly title gen. r-1 ↑ .485 .486 .501 .505 .488 .508 .509 .488 .510 * .515 * .490 .509 .512 r-l ↑ .436 .439 .450 .453 .432 .448 .456 .439 .458 * .460 * .439 .457 .459 lamp-7: personalized tweet paraphrasing r-1 ↑ .516 .514 .552 .565 .522 .552 .559 .528 * .563 * .565 .529 .559 .561 r-l ↑ .463 .457 .511 .517 .475 .512 .517 .482 * .521 * .519 .480 .515 .519 evaluation metrics following lamp @xcite , we use accuracy and f1-score for personalized text classification tasks (lamp-1 and lamp-2), and mae and rmse for lamp-3: personalized product rating. for personalized text generation tasks (lamp-4, lamp-5, and lamp-7), we adopt rouge-1 and rouge-l @xcite . higher scores indicate better performance for all metrics except rmse and mae used in lamp-3. 4 statistics are presented in ## results performance with per-pcs. models equipped with per-pcs outperform non-personalized, rag, and pag counterparts across all six tasks. in personalized text classification, per-pcs achieves 11.79% and 6.02% relative gains in accuracy and f1-score for movie tagging, and 27.32% and 24.92% improvements in mae and rmse for product ratings. for personalized text generation, per-pcs shows 4.25% and 4.28% relative improvements in rouge-1 and rouge-l scores for news headline generation. these results demonstrate per-pcs's effectiveness in enhancing llm personalization. per-pcs vs. peft retrieval. compared to the peft retrieval method, per-pcs shows clear superiority. for instance, per-pcs achieves 8.76% and 22.09% performance gains in accuracy and f1-score for citation identification. significant improvements are also seen in movie tagging, product rating prediction, and news headline generation tasks, highlighting the benefits of fine-grained peft piece composition over peft-level composition, which may risk user data leakage. per-pcs vs. oppu. compared to oppu, which trains personal peft from scratch and requires more computational and storage resources, per-pcs achieves comparable or slightly better results. specifically, per-pcs achieves 99.28% of oppu's performance on average with 7 times less computation and 38 times less storage in personalized text 6463 classification. in personalized text generation, per-pcs shows comparable or better results in scholarly title generation and tweet paraphrasing. ## analysis robustness against sharer count in real-world deployment, the number of users who consent to share their personal peft can vary, and computational resources may constrain the number of sharers, making the sharer count a crucial factor in per-pcs. in this experiment, we alter the number of sharers in two representative tasks from the text classification and generation categories to test the model's robustness. as shown in figure 3 , per-pcs exhibits relatively stable performance despite changes in the number of sharers and achieves the accuracy f1-score 0.2 0.3 0.340 0.268 0.394 0.294 0.399 0.295 0.410 0.300 movie tagging non-personalized profile cluster most active history cluster rouge-1 rouge-l 0.2 0.175 0.158 0.181 0.164 0.182 0.165 0.181 0.164 news headline generation non-personalized profile cluster most active history cluster figure 4: performance of per-pcs on movie tagging and news headline generation tasks with different sharer selection strategies. we find per-pcs is robust to the choice of sharers. 20% 40% 60% 80% 100% share ratio 0.3 0.4 mae product rating prediction ( ) per-pcs rag + per-pcs 20% 40% 60% 80% 100% share ratio 0.55 0.60 0.65 0.70 rmse product rating prediction ( ) per-pcs rag + per-pcs 20% 40% 60% 80% 100% share ratio 0.15 0.17 0.20 0.23 0.25 rouge-1 news headline generaiton ( ) per-pcs rag + per-pcs 20% 40% 60% 80% 100% share ratio 0.16 0.18 0.20 rouge-l news headline generation ( ) per-pcs rag + per-pcs best performance with just 30 sharers in the personalized product rating prediction task, demonstrating its strong efficiency. compared with the peft-level composition baseline (peft retrieval), per-pcs consistently shows better performance, highlighting the effectiveness of fine-grained piecelevel composition in user modeling. ## modeling users with different active levels users can exhibit different levels of activity, resulting in varying lengths of user history items for user modeling and personalization. to investigate the impact of user activity levels, quantified by the number of historical behavior items, on model performance, we randomly sampled 10 users from each range of activity levels. as shown in figure 8 , we observe that (i) per-pcs generally shows stronger relative performance when user behavior items are fewer than 20, likely due to the collaborative signals captured during the assembling process that help the model understand user preferences. (ii) per-pcs generally performs similarly to oppu, which requires training and maintaining personal peft from scratch and significantly more resources, and (iii) both oppu and per-pcs outperform the non-personalized baseline at almost all activity levels. overall, these results demonstrate the strong performance and robustness of per-pcs across all user activity levels. ## ablation study as per-pcs outperforms various baselines in personalization tasks, we investigate the impact of each design choice in per-pcs to verify their effectiveness. more specifically, we perform ablation on the assembling process in both attention aggregation and piece selection steps. as is shown in peft-based personalization methods store user preferences and behavior patterns in parameters. oppu @xcite equips each user with a personal peft module, storing preferences in peft parameters and offering better generalization of user behavior patterns compared to prompt-based methods. another line of work focuses on designing personalized alignment methods via parameter merging @xcite , personalized rlhf @xcite , personalized reward models @xcite , and black-box llm personalization @xcite . ## conclusion we proposed per-pcs, a novel framework that enables users to share their personal pefts, creating community value while preserving privacy. for target users, per-pcs maintained model ownership, efficiency, and fine-grained personalization by employing piece-level composition based on user history data. extensive experiments showed that per-pcs outperforms non-personalized and peft retrieval methods, and performs close to oppu with significantly lower computational and storage resources. we envisioned per-pcs as a communitydriven effort to advance personalized llm, making it more modular, effective, and widely accessible.