DIALGEN: Collaborative Human-LM Generated Dialogues for Improved
  Understanding of Human-Human Conversations
Authors: name: Bo-Ru Lu; name: Nikita Haduong; name: Chia-Hsuan Lee; name: Zeqiu Wu; name: Hao Cheng; name: Paul Koester; name: Jean Utke; name: Tao Yu; name: Noah A. Smith; name: Mari Ostendorf
Published: 2023-07-13 20:02:50

Abstract: Applications that could benefit from automatic understanding of human-human
conversations often come with challenges associated with private information in
real-world data such as call center or clinical conversations. Working with
protected data also increases costs of annotation, which limits technology
development. To address these challenges, we propose DIALGEN, a
human-in-the-loop semi-automated dialogue generation framework. DIALGEN uses a
language model (ChatGPT) that can follow schema and style specifications to
produce fluent conversational text, generating a complex conversation through
iteratively generating subdialogues and using human feedback to correct
inconsistencies or redirect the flow. In experiments on structured
summarization of agent-client information gathering calls, framed as dialogue
state tracking, we show that DIALGEN data enables significant improvement in
model performance.