A ug ESC : Dialogue Augmentation with Large Language Models for Emotional Support Conversation (2023)

Abstract: Crowdsourced dialogue corpora are usually limited in scale and topic coverage due to the expensive cost of data curation. This would hinder the generalization of downstream dialogue models to open-domain topics. In this work, we leverage large language models for dialogue augmentation in the task of emotional support conversation (ESC). By treating dialogue augmentation as a dialogue completion task, we prompt a fine-tuned language model to complete full dialogues from available dialogue posts of various topics, which are then postprocessed based on heuristics. Applying this approach, we construct AugESC, an augmented dataset for the ESC task, which largely extends the scale and topic coverage of the crowdsourced ESConv corpus. Through comprehensive human evaluation, we demonstrate that our approach is superior to strong baselines of dialogue augmentation and that AugESC has comparable dialogue quality to the crowdsourced corpus. We also conduct human interactive evaluation and prove that post-training on AugESC improves downstream dialogue models’ generalization ability to open-domain topics. These results suggest the utility of AugESC and highlight the potential of large language models in improving data-scarce dialogue generation tasks.

## introduction current open-domain dialogue corpora are commonly curated through crowdsourcing to endow dialogue models with sophisticated skills @xcite @xcite , since the desired high-quality dialogues are usually not available in public sources. for example, the task of emotional support conversation (esc) @xcite aims to support help-seekers to reduce dailylife emotional distress. to train downstream dialogue models, @xcite crowdsourced the esconv dataset, which contains only 1.3k dialogue sessions covering 13 topic categories. the construction of esconv reveals typical limitations of crowdsourcing dialogue data. first, it is time-consuming especially when the desired dialogues should contain long multi-turn interactions (e.g., the esconv dialogues contain about 30 utterances on average). also, it usually requires laborious worker training and human screening to ensure the high dialogue quality. consequently, the expensive data curation restricts the scale and topic coverage of collected dialogues, which may hinder the generalization of downstream dialogue models to open-domain topics. in this work, we leverage large language models for large-scale dialogue augmentation in the esc task. we first present a simple yet effective approach, which treats dialogue augmentation as a dialogue completion task ( § 3), as illustrated in figure 1 . we fine-tune the 6b gpt-j model @xcite with esconv samples, prompt it to complete full dialogues on various topics using the dialogue posts of empatheticdialogues @xcite , and postprocess augmented dialogues based on heuristics. we thus construct augesc, an augmented dataset for the esc task, which largely extends the scale (45x larger) and topic coverage of the original esconv dataset ( § 4). to demonstrate the superiority of our approach, we compare with strong baselines of dialogue augmentation ( § 5) through human evaluation. we show that our approach performs substantially better and that the quality of the constructed augesc is comparable to the crowdsourced esconv. to further prove the utility of augesc, we conduct human interactive evaluation ( § 6) and show that post-training on augesc improves downstream dialogue models' generalization ability to opendomain topics. our contributions are summarized in four folds: ## methodology we treat dialogue augmentation as a dialogue completion task, as illustrated in figure 1 and 2. the dialogue augmentation procedure contains three steps: (1) fine-tuning the language model ( § 3.1), (2) prompting it to complete full dialogues on var- the following is a conversation with an ai assistant. the assistant is helpful, empathetic, clever, and very friendly. it can use various support skills to provide emotional support to human. human: i moved into a new state recently, and there's a lot to do, but i don't have any friends in the new place i stay at. ai: what's it like being away from family? human: family is all i have here. they aren't exactly close, so i haven't gotten to see them in a while. ai: that must be difficult. how do you feel about where you live? human: it's ok. i'm learning to like it a little bit. at least now i have someone who is usually around when i wake up. ai: if only you were that lucky with people in general. people move for so many different reasons. i've found that often when i move, i just need to adjust my social circle a little, and i find that i then end up liking where i am. human: that's true. maybe i should just find some people to hang out with. ai: well, of course! there are a lot of online social groups. we can start with that. human: i think that's a good idea. and i already know one person at my current location. ai: yes, it's much easier to get started doing this online than it is with people physically nearby. i know that you were actually just thinking that. i would guess that after a short amount of time you'd develop more friendships. human: yeah, i've been spending time on social media apps and just making new connections. i've found some really good friends through it. ai: that's wonderful! i'd say, you're on your way! human: what should i do first? ai: for now, try to think about some people you could go to eat lunch or coffee with, or just to chat about what's going on. human: oh yeah, that's a great idea! thank you! ai: you are welcome! of course, when you can afford it, do get together with others in real life! it'll be much more rewarding than anything you could do alone. human: do you think that will help? i was worried that i wouldn't make any friends here, but i have one friend right now. ai: it certainly will help immensely. real life friends are the best of all! human: that's good to hear! i guess i'll keep trying. thanks for the help! ai: you are welcome! and again, i wish you the best of luck with your new situation! ious topics ( § 3.2), and (3) postprocessing augmented dialogues based on heuristics ( § 3.3). ## evaluation for data utility trained on augesc? yes no fluency 47 13 identification 68 22 comforting 55 22 suggestion 58 15 overall 58 28 ## conclusion in this work, we present a simple yet effective approach for dialogue augmentation, which is formulated as a dialogue completion task. using this approach, we release an augmented dataset augesc for the task of emotional support conversation (esc), which largely extends the scale and topic coverage of the crowdsourced esconv corpus. through comprehensive empirical evaluation, we show that: (1) our approach produces augmented dialogues with higher quality than strong baselines of dialogue augmentation, (2) augesc has comparable dialogue quality to the crowdsourced es-conv corpus, and (3) post-training on augesc notably improves the generalization capability of downstream dialogue models to open-domain topics. our work demonstrates the prowess and utility of large language models in improving data-scarce tasks, especially complex open-domain dialogue tasks. it may inspire more work regarding training data augmentation with large language models. future work can explore automatic methods for further quality refinement of dialogue augmentation.