Crafting Personalized Agents through Retrieval-Augmented Generation on Editable Memory Graphs (2024)

Abstract: In the age of mobile internet, user data, often referred to as memories, is continuously generated on personal devices. Effectively managing and utilizing this data to deliver services to users is a compelling research topic. In this paper, we introduce a novel task of crafting personalized agents powered by large language models (LLMs), which utilize a user’s smartphone memories to enhance downstream applications with advanced LLM capabilities. To achieve this goal, we introduce EMG-RAG, a solution that combines Retrieval-Augmented Generation (RAG) techniques with an Editable Memory Graph (EMG). This approach is further optimized using Reinforcement Learning to address three distinct challenges: data collection, editability, and selectability. Extensive experiments on a real-world dataset validate the effectiveness of EMG-RAG, achieving an improvement of approximately 10% over the best existing approach. Additionally, the personalized agents have been transferred into a real smartphone AI assistant, which leads to enhanced usability.

## introduction in the era of mobile internet, personal information is constantly being generated on smartphones. this data, referred to as personal memories, is often scattered across everyday conversations with ai assistants (e.g., apple's siri), or within a user's apps (e.g., screenshots), including emails, calendars, location histories, travel activities, and more. as a result, managing and utilizing these personal memories to provide services for users becomes a challenging yet attractive task. with the emergence of advanced large language models (llms), new opportunities arise to leverage their semantic understanding and reasoning capabilities to develop personal llm-driven ai assistants. motivated by this trend, we study the problem of crafting personalized agents that enhance the ai assistants with the capabilities of llms by leveraging users' memories on smartphones. unlike existing personal llm agents @xcite , such as those designed for psychological counseling @xcite , housekeeping @xcite , and medical assistance @xcite , the personalized agents face unique challenges due to practical scenarios and remains relatively unexplored in current methods. these challenges can be summarized below. to this end, we introduce a new solution called emg-rag, which presents the first attempt of its kind to address these challenges. we discuss the solution along with the rationales behind it below. for (1), we utilize a business dataset collected from a real ai assistant, which includes daily conversations with the assistant, and users' app screenshots, to extract personal memories. specifically, we leverage the capabilities of @xcite to clean the raw data into memories. we organize the memories chronologically, and then use gpt-4 to generate qa pairs within each session (a set of consecutive memories). we also tag the memories involved in generating these qa pairs, which are then used for subsequent training purposes. for (2), we introduce a three-layer data structure, called editable memory graph (emg). the first two layers form a tree structure in accordance with the business scopes, while the third layer consists of a user's memory graph parsed from the memory data. this design is motivated by three considerations: 1) the tree structure allows for partitioned management of various memory categories, facilitating expansion to other categories; and 2) memory data is partitioned under different categories, with the graph structure to capture their complex relationships, and 3) this enables efficient retrieval to locate specific memories for editing, by searching within relevant partitions rather than the entire dataset. for (3), we introduce a reinforcement learning (rl) agent that adaptively selects memories on the emg, without being constrained to a fixed top-k approach. the rationale of using rl resembles a boosting process. specifically, when the agent selects relevant memories (actions), it prompts a llm (frozen) to generate improved answers. the quality of these answers is evaluated by a downstream task metric (reward), which then guides the agent to refine its policy for better memory selection. this results in an end-to-end optimization process aimed at achieving the desired goal for downstream tasks. overall, we make the following contributions. (1) we introduce a novel task of crafting llmdriven personalized agents, leveraging users' personal memories to enhance their experience through llm capabilities. this task differs from existing personal llm agents in three key challenges: data collection, editability, and selectability. (2) we propose emg-rag, a novel solution that combines emg and rag to address the three challenges. we show that it enables an end-to-end optimization process through reinforcement learning to achieve the goal of personalized agents. (3) we conduct extensive experiments on a real-world business dataset across various llm architectures and rag methods for three downstream applications: question answering, autofill forms, and user services. our approach demonstrates improvements of approximately 10.6%, 9.5%, and 9.7% over the best existing approach for these tasks, respectively. moreover, the personalized agents have been transferred into an ai assistant product, resulting in a notable improvement in user experience. ## related work personalized dialogue system. to develop a personalized dialogue system (pds), the personachat dataset @xcite is collected through crowdsourcing, which comprises personas (each persona is defined by a set of profile sentences) and chats (each chat is collected by two crowdworkers with two randomly assigned personas). based on the dataset, various techniques have been studied to address challenges in pds, including mutual persona perception @xcite @xcite , persona-sparsity @xcite , long-term persona memory @xcite ), etc. for example, p 2 bot @xcite is a gpt-based framework @xcite , specifically designed to enrich personalized dialogue generation through mutual persona perception. it aims to model the underlying understanding, such as character traits, within a conversation to facilitate mutual acquaintance between interlocutors. in addition, a pds can be further enhanced by integrating internal reasoning techniques @xcite or external acting techniques @xcite , which aim to generate more personalized and factual responses. in this study, we construct user-personalized agents using practical memory data gathered from smartphone ai assistants. leveraging these agents, we introduce three distinct applications: question answering, autofill forms, and user services. ## problem statement we study the problem of developing personalized agents for users on smartphone ai assistant platforms (such as apple's siri or samsung's bixby). these agents are designed to assist users in performing personalized tasks, requiring the fulfillment of the following two properties in practical scenarios: by satisfying these properties, the agents aim to enhance the user experience during interactions with their smartphone ai assistants. these agents offer essential functionalities to support personalized applications, including question answering, autofill forms, and user services like reminders for important events and times, and travel navigation (further details will be discussed in section 4.4). ## conclusion in this paper, we present a novel task of creating personalized agents powered by llms, which leverage users' personal memories to enhance three downstream applications. our solution, emg-rag, combines rag techniques with an emg to tackle challenges in data collection, editability, and selectability. extensive experiments are conducted to confirm the effectiveness of emg-rag.