# Designing Social Robots with LLMs for Engaging Human Interaction
## 3 System Design
### Memory, Persona, and Dialogue Continuity
#### 3.2.1 Memory Mechanism  
A central feature of the system is its ability to create the impression of continuity across multiple interactions by retaining and recalling user-specific information. This is achieved through a layered memory architecture consisting of two distinct but interrelated components: a persistent user memory profile and a dynamic, session-level conversation memory. The user memory profile serves as a long-term record of the user's background, preferences, and recurring themes, accumulated progressively through conversation. Rather than relying on predefined profiles, the system extracts relevant user information from interactions and stores it in an encrypted memory file associated with each individual. At the end of each session, a memory summarisation step is triggered, during which key points from the dialogue are extracted and compiled into a concise memory representation. This summarisation is generated by the LLM itself (see Prompt 1), using an instructional  
Figure 1: Functional architecture of the enhanced spoken dialogue system. prompt that requests the extraction of salient topics, personal details, preferences, and any intentions or action items discussed. **Prompt 1** (Memory Summarisation). : _Provide a comprehensive summary of our current conversation. The summary should include: 1. Key points discussed 2. Important details that emerged 3. Relevant interests and aspects related to user_name_
4. _Any action items agreed upon_  
For instance, if a user previously shared an interest in gardening, mentioned a recent trip to Paris, and spoke about their cat named [PERSON], the resulting summary might read: _\"user_name mentioned that enjoy gardening, recently traveled to Paris, and have a cat named [PERSON]\"_ This output is then appended to the user's profile, which is later reinjected into future prompts to maintain personalised, context-aware dialogue. In parallel to this persistent memory, the system manages a short-term, session-specific memory during each conversation. Initially, the full dialogue history is included in the prompt context sent to the LLM, allowing it to generate responses that are grounded in the ongoing exchange. However, as the conversation progresses and approaches the token limit imposed by the model, a real-time summarisation mechanism is triggered. This mechanism replaces the earlier dialogue turns with a compressed version that retains essential facts, user references, and relevant conversational threads, ensuring coherence without exceeding computational constraints. This summarised segment not only helps to manage prompt size, but is also stored and later integrated into the long-term user memory after the session ends. The LLM prompt (see Prompt 2) is structured to combine the system's multiple context layers: the robot's persona definition, the user's long-term profile, the in-session summary or full dialogue history (depending on token usage), and the most recent user utterance. This layered prompt design allows the robot to draw from both the current context and past experiences, enabling it to refer back to previous topics, maintain consistency in tone and knowledge, and avoid generic or repetitive exchanges. **Prompt 2** (System Initialisation). : _You are [PERSON], a social robot and conversational companion for user_name, who speaks language. Today, you want to engage warmly, empathetically, and meaningfully with user_name. Here's what you know about them: user_info. Previous conversations included: past_interactions. Consider these topics as you engage, but avoid repetitive questions. The current date and time is date/time in location. Specific behaviour instructions..._  
This multi-level memory structure was designed to support more natural and context-rich conversations, particularly in repeated interactions. During our evaluation, several users noted that the robot's ability to recall personal details from previous sessions contributed to a sense of being remembered and understood.