SODA : Million-scale Dialogue Distillation with Social Commonsense Contextualization (2023)

Abstract: Data scarcity has been a long standing issue in the field of open-domain social dialogue. To quench this thirst, we present SODA: the first publicly available, million-scale high-quality social dialogue dataset. By contextualizing social commonsense knowledge from a knowledge graph, we are able to distill an exceptionally broad spectrum of social interactions from a large language model. Human evaluation shows that conversations in SODA are more consistent, specific, and (surprisingly) natural than those in prior human-authored datasets. Using SODA, we train COSMO: a generalizable conversation model that is significantly more natural and consistent on unseen datasets than best-performing conversation models (e.g., GODEL, BlenderBot-1, Koala, Vicuna). Experiments reveal COSMO is sometimes even preferred to the original human-written gold responses. Additionally, our results shed light on the distinction between knowledge-enriched conversations and natural social chitchats. We plan to make our data, model, and code public.

## introduction conversations that occur in everyday spoken situations are often not recorded as data. and when they are, such as in the case of text messages, research use is rightly restricted due to privacy and legal concerns. as a result, collecting high-quality, everyday social conversations on a large scale has long been recognized as a difficult task @xcite . previous studies have relied on crowdsourcing focused on specific themes of dialogue (e.g., persona, empathy; @xcite . however, this approach is limited in scale due to its associated costs. as a result, the progress made in machine dialogues, including generation, evaluation, and understanding, has been severely hindered by the reliance on these small datasets @xcite . 1 . to alleviate this bottleneck, we introduce soda (social dialogues), a million-scale english dialogue dataset covering a wide variety of social interactions. as a result of being grounded on rich social commonsense and narratives, soda goes beyond specific skill-focused dialogues and features more general conversations. our dataset includes 1.5 million dialogues distilled from a large language model (in our case, gpt-3.5; @xcite resulting in more than 11 million utterances with 300 million tokens: soda is the largest publicly available open-domain social conversation dataset. human evaluation shows that soda surpasses existing human-authored dialogue corpora across axes like consistency, specificity, and (surprisingly, even) naturalness ( §3.2). to make soda, we propose co 3 , a framework for contextualizing commonsense for distilling conversations from a large language model (llm). illustrated in figure 1 , co 3 infuses commonsense knowledge into dialogues by transforming knowledge triples into narratives, and then into dialogues. such an approach offers two significant advantages: (1) maximizing diversity and (2) minimizing nonsensical conversations. although generating content using llms is relatively easy, determining how to cover diverse content poses a non-trivial challenge. we find that sampling from an llm without contexts results in dull conversations ( §3.3). because commonsense knowledge graphs cover a wide range of everyday situations @xcite , conditioning on them results in a broad spectrum of conversations. moreover, since llms are prone to hallucinations @xcite , the seed commonsense knowledge can help them stay on a sensible generation path. with soda, we train a conversation model, cosmo. human evaluation results demonstrate that: (1) cosmo generalizes better to unseen conversations than existing best-performing dialogue models, winning by more than 40% on average in head-to-head comparisons versus blender-bot @xcite , koala @xcite , and vicuna @xcite ( §5.1); (2) cosmo outperforms blenderbot (with the same number of parameters) on the dataset blender-bot was trained on, despite never seeing the corpus ( §5.2); and (3) cosmo responses are even preferred over human-authored, ground-truth responses in dailydialog @xcite , a dataset on which cosmo was not trained on ( §5.1). finally, the distilled dialogues in soda represent a significant resource contribution for opendomain dialogue research. most of all, soda enables the research community to train smaller dialogue agents with competitive capabilities. also, soda can help enhance the generalizability of other advancements in the dialogue field (e.g., understanding and evaluation), which have relied on existing small datasets. lastly, soda highlights a dimension where recent llm-based conversational agents (e.g., koala, vicuna, and chatgpt) struggle -i.e., the naturalness of the responses ( §5.1 and §5.3). as these models are designed to provide knowledge-based responses, they may generate responses that are informative but lack the naturalness found in social chitchat. we plan to publicly release soda, cosmo, and co 3 under the permissive license cc-by-4.0, aiming to address the data scarcity issue in open-domain dialogue. ## co 3 : a contextualization framework for conversation distillation using commonsense we propose co 3 , a framework for distilling conversations from large language models (llms) by contextualizing (i.e., adding more context information) commonsense knowledge. our goal is to obtain natural conversations covering a wide variety of social interactions. co 3 consists of three steps: (1) retrieving social commonsense from a symbolic commonsense knowledge graph ( §2.2), (2) converting it into sentence form and generating a narrative from the sentence ( §2.3), and (3) inferring the conversation participants from the narrative and derive a conversation grounded in the narrative ( §2.4). we use gpt-3.5 (i.e., text-davinci-002 foot_0 ; @xcite to implement co 3 , though in practice, a different model could be used. we use co 3 to create soda: an example is in ## soda: a million-scale social dialogue dataset we obtain soda (social dialogues), a large-scale high-quality conversation dataset covering a wide range of social interactions, by applying a series of post-processing ( §3.1) to the conversations generated from our contextualization framework ( §2). we compare soda with existing human-curated dialogue corpora ( §3.2) and analyze the effectiveness of contextualization ( §3.3). ## cosmo: a socially situated conversation model we use soda to train cosmo: a conversation model that can converse in a wide range of social situations. cosmo can take in situation narrative, along with dialogue history, and generate a next utterance according to a given role. training cosmo we use several structured components of soda during training: (1) the contextual narrative n ( §2.3), (2) the perspective/speaker instruction i (e.g., "imagine you are madeleine and speak to her coach") built with the inferred conversation participants ( §2.4), and (3) the dialogue context c. the model is trained to generate a target response r when given n, i, and c -i.e., p(r|n, i, c). we do so in a sequence-tosequence fashion, concatenating n, i, c with a @xmath0. c is made up of the previous conversation utterances concatenated with a turn @xmath1. because conversational models often agree to toxic or unethical behavior @xcite , for additional training data, we include prosocial-dialog @xcite (adapted to the same format as soda, see appendix c). prosocialdialog includes a wide range of negative constructive feedback based on social rules-of-thumb, e.g., "so i think it's best to continue being honest, and apologize that you were lying." the inclusion of this corpus assists conversation models in handling sensitive contexts (e.g., biased, harmful, unethical) without affecting the model performance on other datasets @xcite . we build cosmo on top of the lm-adapted t5 @xcite , which achieves strong benchmark performance across various classification and generation tasks. @xcite . we train two versions of the model: cosmo-3b and cosmo-11b using the t5x library @xcite . for better robustness and generalizablity to datasets that don't have contexts or dialogue starting prompts, we randomly drop narrative n and role instruction i 30% and 50% of the time, respectively. ## generalizability of cosmo we compare cosmo to other conversational agents on social conversation datasets under both out-ofdomain and in-domain settings. since automatic response evaluation is brittle, we focus on human evaluation @xcite . automatic evaluation results via gpt-4 are in appendix d. baselines we compare cosmo with four bestperforming stand-alone conversation models: blenderbot-1 @xcite , godel @xcite , koala @xcite , and vicuna @xcite . blenderbot is a transformer pretrained on 1.5b reddit comments and trained on various chitchat datasets. godel utilizes a pretrained language model t5 @xcite trained on web text data, and further trains on 551m reddit threads and 5m instruction and grounded dialogue datasets. koala and vicuna are models that finetuned llama @xcite , which is an open-source llm, using dialogue data from the web. they are both known to achieve comparable performance to @xcite , which is a model finetuned for conversational interaction based on gpt-3.5 -i.e., our teacher model. we also compare cosmo with gpt-3.5 and chatgpt; prompting details are in appendix d. ## conclusion we presented soda, the first million-scale dialogue dataset covering an exceptionally wide range of social interactions to alleviate the data scarcity issue. soda is not only orders of magnitude larger than popular dialogue datasets; it is also perceived to be significantly better than them across multiple aspects (e.g., naturalness, specificity, consistency). for making soda, we also introduced co 3 , a framework for distilling conversations from a large language model by contextualizing commonsense knowledge. with soda, we trained a conversation model cosmo that can generalize significantly better than existing models to unseen dialogues; and generate responses that are even more preferred than ground-truth responses of an existing dataset.