Score Before You Speak: Improving Persona Consistency in Dialogue Generation using Response Quality Scores
Authors: Arpita Saggar, Jonathan Darling, Vania Dimitrova, Duygu Sarikaya, David Crossland Hogg


Reviewer paper summaries: This paper studies persona-based dialogue in language models, particularly in relation to following persona descriptions in system prompts. To address the potential complexity and resource-intensive nature of training pipelines, the authors propose a simpler framework called Score-Before-Speaking. Experimental results demonstrate the effectiveness of this framework.
Summary of strengths: - The paper is well-constructed.
- The proposed method is simple but likely effective.
Summary of weaknesses: - The experiment settings and baselines are inadequate for the current era of large language models. Further validation on advanced LLMs like LLaMA, Mistral, and Gemma is necessary.
- The proposed method lacks novelty, as it follows similar principles to existing prompting strategies like Chain-of-Thought and Self-Refine.
- The paper does not adequately discuss the capabilities of advanced LLMs. Given that large models already possess strong role-playing abilities, the value of this paper is unclear.
Comments, suggestions and typos: - More discussion and analysis on advanced LLMs are needed to better contextualize the contributions of this paper.
Overall assessment scores: 1.5