Step-Back Profiling: Distilling User Interactions for Personalized Scientific Writing
Authors: Xiangru Tang, Xingyao Zhang, Yanjun Shao, Jie Wu, Yilun Zhao, Arman Cohan, MING GONG, Dongmei Zhang, Mark Gerstein


Reviewer paper summaries: This paper presents a method to personalize LLMs for scientific writing by distilling user histories into concise profiles. These profiles capture main traits and preferences, enabling personalized content generation in collaborative writing scenarios. The authors introduce the Personalized Scientific Writing (PSW) dataset to evaluate their method, demonstrating its effectiveness over existing personalization techniques on various tasks, including those in the LaMP benchmark.
Summary of strengths: - Well-written and well-presented paper.
- The method effectively distils user histories into concise profiles, addressing the challenge of handling extensive user histories while maintaining personalized content generation.
- The authors conduct extensive experiments using both the LaMP and PSW datasets, showcasing the method’s superior performance across multiple personalization tasks and validating its effectiveness through detailed ablation studies.
Summary of weaknesses: - While the method claims to be efficient, the paper lacks a detailed analysis of its scalability, particularly regarding the computational overhead of profile distillation and retrieval processes.
- The paper briefly mentions privacy but does not delve deeply into the potential privacy implications of sharing and distilling user histories.
Comments, suggestions and typos: - Improve the clarity of figure captions. For instance, Figure 1’s caption is somewhat confusing and should better explain the components and workflow depicted.
Overall assessment scores: 3