Post Persona Alignment for Multi-Session Dialogue Generation (2025)

Abstract: Multi-session persona-based dialogue generation presents challenges in maintaining long-term consistency and generating diverse, personalized responses. While large language models (LLMs) excel in single-session dialogues, they struggle to preserve persona fidelity and conversational coherence across extended interactions. Existing methods typically retrieve persona information before response generation, which can constrain diversity and result in generic outputs. We propose Post Persona Alignment (PPA), a novel two-stage framework that reverses this process. PPA first generates a general response based solely on dialogue context, then retrieves relevant persona memories using the response as a query, and finally refines the response to align with the speaker’s persona. This post-hoc alignment strategy promotes naturalness and diversity while preserving consistency and personalization. Experiments on multi-session LLM-generated dialogue data demonstrate that PPA significantly outperforms prior approaches in consistency, diversity, and persona relevance, offering a more flexible and effective paradigm for long-term personalized dialogue generation.

## introduction multi-session persona-based dialogue generation aims at generating responses based on the persona sentences, current dialogue context, and dialogue history sessions @xcite . with the development of large language models (llms), current dialogue systems are proficient in single-session dialogue generation, and even able to produce responses more natural than human-authored ones @xcite . despite their impressive capabilities in generating coherent single-session dialogues, llms still face challenges in multi-session dialogue generation. the first challenge is maintaining consistency as the dialogue history becomes long. research has found that llm agents cannot adhere to their own persona or be consistent with content in previous conversations after multi-session dialogues @xcite . as the dialogue sessions increase, it is necessary to truncate, summarize, or select parts of the history to fit within input token limits @xcite @xcite @xcite , which might lead to the loss of critical information. while recent llms offer greatly extended context windows that can accommodate the full history, prior studies @xcite show that they may still fail to retrieve and reason over it effectively, struggling to track subtle content shifts and recall information from early interactions.the second challenge lies in enhancing the diversity of generated dialogues. while less discussed in persona-aware dialogue literature, recent studies have raised concerns about the homogenization and lack of richness in llmgenerated text @xcite @xcite . retrieval-based multi-session systems often use the last utterance as a query to retrieve relevant persona information. however, this typically yields narrowly scoped con-tent, limiting the opportunity for diverse or novel responses. to address these challenges, we propose a novel framework called post persona alignment (ppa). unlike conventional approaches that retrieve persona memories before generating responses, ppa reverses the order: it first generates a general response based solely on the dialogue context, then uses that response as a query to retrieve relevant persona memories, and finally refines the response to ensure consistency and personalization. the framework is shown in fig. 1 . this post-hoc alignment strategy reflects how humans often communicate, i.e., by initially responding naturally to the flow of conversation, then revising or extending their reply based on what they know about the listener. compared to pre-alignment methods, ppa offers more freedom for the model to explore plausible and creative directions before aligning with persona traits, thereby improving both naturalness and diversity. we evaluate multiple generation strategies with different history representations on llm-generated multi-session dialogues. our experiments show that ppa significantly improves the consistency, diversity, and personalization of generated responses compared to prior baselines. ## related work most prior work on personalized dialogue generation adopts a pre-alignment strategy, conditioning the response generation on predefined persona profiles from the outset. the persona-chat dataset @xcite pioneered this setting by incorporating persona sentences into chit-chat models. subsequent work extended this paradigm by integrating persona embeddings @xcite or auxiliary classifiers to detect relevant persona usage @xcite . these approaches improved persona consistency but often led to generic, rigid, or contextually awkward responses when persona facts were injected indiscriminately. more recent methods recognized this limitation and adopted multi-stage pipelines to balance coherence and persona relevance. for example, @xcite proposed deleting and rewriting persona-inconsistent words post-generation. similarly, simoap @xcite and bob @xcite use over-sampling and entailmentbased filtering to maintain both fluency and consistency. these trends highlight the growing interest in decoupling persona alignment from initial re-sponse generation, a direction we pursue further. retrieval-augmented dialogue generation has proven effective for incorporating long-term or external knowledge into conversations. wizardof-wikipedia @xcite retrieves factual background to ground responses in real-world knowledge. similarly, persona facts can be viewed as structured knowledge: @xcite and @xcite proposed models that separately encode profile memory and interaction history, improving personalization. however, memory retrieval is often driven by the current dialogue context, which limits flexibility and may omit relevant long-range cues. to improve coherence and control input length, recent works propose summarizing histories @xcite or retrieving salient utterances or tokens @xcite , though such compression can sacrifice nuanced personal information. rule-based personal knowledge extraction has emerged as a more effective way to preserve speaker-specific details @xcite . maintaining consistency over multiple sessions is particularly challenging due to memory constraints and the risk of context drift. hierarchical models such as hred @xcite were early attempts to encode multi-turn coherence, while modern llm-based systems simply concatenate long histories, leading to token budget issues and degraded performance @xcite . multi-session chat (msc) @xcite has sparked research into long-term memory design @xcite . llm-based multi-session dialogues further increase dataset size @xcite , with park et al. (2023) providing some of the longest available sessions. in contrast to prior systems that integrate persona information before generation, our proposed framework delays persona grounding until after an initial response is generated. the response is first generated from the dialogue context, then used to retrieve relevant persona memories, which are finally incorporated through refinement. this reversal of the retrieval-generation order yields more natural, diverse responses without sacrificing persona consistency. by grounding after generation, ppa enables targeted alignment with the model's intended output, avoiding unnecessary or rigid persona mentions. this flexibility sets ppa apart from previous persona-grounded or retrieval-augmented methods, offering an effective solution for multisession dialogue with consistent personalization. ## post persona alignment framework we propose a two-stage generation framework called post persona alignment (ppa), which defers persona grounding until after a general response is generated. this design contrasts with conventional persona-aware generation that aligns responses to personas before generation begins. instead, our method first produces a preliminary response guided solely by dialogue context, then selectively retrieves relevant persona information based on that response, and finally refines the output for personalization and consistency. figure 1 illustrates the overall architecture. ## conclusion we introduced post persona alignment (ppa), a novel framework for multi-session persona-based dialogue generation that defers persona grounding until after an initial response is generated, reversing the conventional retrieval-generation order. ppa structures information into salient, memory-like facts, and retrieves them only when truly relevant to the model's communicative intent. this approach reduces the cognitive load on the model and improves controllability, enabling more natural, diverse, and personalized dialogues. experiments demonstrate that ppa significantly improves consistency, diversity, and persona relevance over prior methods, offering a flexible and effective solution for long-term conversational modeling.