Controllable Mixed-Initiative Dialogue Generation through Prompting (2023)

Abstract: Mixed-initiative dialogue tasks involve repeated exchanges of information and conversational control. Conversational agents gain control by generating responses that follow particular dialogue intents or strategies, prescribed by a policy planner. The standard approach has been fine-tuning pre-trained language models to perform generation conditioned on these intents. However, these supervised generation models are limited by the cost and quality of data annotation. We instead prompt large language models as a drop-in replacement to fine-tuning on conditional generation. We formalize prompt construction for controllable mixed-initiative dialogue. Our findings show improvements over fine-tuning and ground truth responses according to human evaluation and automatic metrics for two tasks: PersuasionForGood and Emotional Support Conversations.

## introduction mixed initiative dialogue systems allow all interacting agents to initiate actions to control the interaction. these systems dynamically adapt interaction styles to regain control and progress towards specific goals @xcite , unlike others which passively respond to users' input (e.g. some assistants like chatgpt), mixed initiative dialogue systems thus often involve complex policy planning sub-tasks to determine optimal turn-level system dialogue intents @xcite @xcite . these policies define when it is optimal for a system to regain initiative (e.g., when a moderator should interject in a conversation, or when a companion should ask questions or change a conversation topic). however, "optimal" planned dialogue intents still need to be executed through "optimal" response models. the standard practice in recent dialogue research has been to fine-tune a pretrained language model for conditional generation my girlfriend dumped me for someone else! i've been really shook up. i'm so sorry to hear that :( do you have any idea why? i think she just lost interest in me? it happened a couple months ago. i wish i could get over it. i can't get her off my mind. she's dating this total loser. ## datasets we examined esc @xcite ) and p4g @xcite . esc consists of 1053 conversations between emotional help-seekers and supporters. each conversation is annotated with the help-seeker's description of their problem, and the type of issues they are facing. each turn by the supporters is annotated with one of eight emotional support strategies (table a1 ). p4g contains 300 annotated conversations between persuaders who attempt to persuade persuadees to donate to a charity called save the children. persuader turns are annotated with one of 10 strategies (table a2 ). ## baselines in mixed-initiative dialogue, interacting parties continuously exchange control throughout the conversation. however, in order for agents to regain control, they must be able to properly execute items from their conversational agenda, e.g. generating a response that matches a desired strategy/intent. @xcite fine-tuned blenderbot @xcite on esc using input representations consisting of flattened dialogue history and the predicted emotional support strategy for a specific turn. the best-performing model in their experimental setting is "oracle-blenderbot" which conditions on the ground truth strategy for a given turn. chen et al. (2022b) proposed a persuasive dialogue system called rap, which combined targeted user response with conditional generation. the conditional generation component of rap involves fine-tuning bart (lewis et al., 2020) using a penalized loss to force the model to artificially create semantic control through dialogue intents. ## mixed-initative dialogue prompting rap required introducing a dialogue intent classifier to weakly supervise the training process, as there is not an oracle for whether the dialogue intent of a candidate response is correct. but, this confounds errors, as classifiers are imperfect. moreover, fine-tuning approaches like both rap and oracle-blenderbot involve balancing a tradeoff between response quality and semantic control accuracy. prompting llms avoids both issues as it does not involve adjusting model weights to learn representations of control codes for individual tasks. in this paper, we systematically prompt instruct-gpt "text-davinci-003." rather than requiring expert-level prompt engineering, we create general prompt templates which directly fill slots using roles and annotations from both esc and p4g. specifically, we split up prompt construction into task background and conversation history. figure 2 breaks down an example of a prompt for esc. the task background is a paragraph formed from the "emotion type," "problem type," and "situation" annotations provided by the corpus. the conversation history consists of each prior utterance, prepended by labels for each speaker. the system-side turns are also prefixed by a natural language form of the annotated emotional support strategy, derived from the annotation scheme in @xcite (e.g. "the therapist acknowledges the patient's feelings by paraphrasing their situation."). figure 2 contains the contextual dialogue turns in order, along with the three support strategies used. the p4g prompting style is similar. unlike personalized emotional support conversations, the task does not change, so the task background is fixed with relevant factual background information. the conversation history still interweaves narrative directions for each persuasive strategy (e.g. "the persuader uses a logical appeal."). example provided in figure a1 . the natural language intent mappings for both tasks are provided in tables a1 , a2 . ## experiments we evaluated prompting statically and interactively. ## discussion prompting yields strong performance in mixedinitiative tasks in the low resource regime foot_2 . promptgenerated responses are often preferable even compared to ground-truth responses in esc and p4g. from 17 paired evaluations of esc where crowdworkers rated ground truth utterances as not matching the ground truth intent annotation, the promptgenerated response was rated as correct 13 times. however, this is likely because many dialogue corpora are created or annotated by crowdworkers, so the data may vary in quality. while llms may generate "better" responses than crowdworkers, we do not expect them to be better than expert therapists. the results do indicate that prompting may be appropriate for building systems for tasks with limited data. as made evident by our ratings, annotating dialogue intents is a difficult and subjective process prone to errors which can further propagate to fine-tuned task models. this could potentially be addressed by the high semantic control demonstrated through prompting, despite not requiring downstream fine-tuning label supervision. this prompting approach could be applied to other mixed-initiative tasks, including chit-chat and task-oriented dialogue. for instance, many real-world systems such as customer service chatbots already have pre-defined policies for what systems are allowed to say, despite not necessarily having many labeled conversations. a system can be designed as long as there is a policy planner, which could simply be a hierarchical ruleset. while there is some human-effort involved in writing natural language forms of fixed dialogue intents, it is a much less costly process than annotating highquality dialogue data. ## conclusion we find encouraging results for prompting on mixed-initiative dialogue tasks, indicating that generated responses are high quality and follow semantic controls. strong low resource performance opens the possibility of future work building mixedinitiative systems around novel settings which would require subjective data annotation.