P rosocial D ialog: A Prosocial Backbone for Conversational Agents (2022)

Abstract: Most existing dialogue systems fail to respond properly to potentially unsafe user utterances by either ignoring or passively agreeing with them. To address this issue, we introduce ProsocialDialog, the first large-scale multi-turn dialogue dataset to teach conversational agents to respond to problematic content following social norms. Covering diverse unethical, problematic, biased, and toxic situations, ProsocialDialog contains responses that encourage prosocial behavior, grounded in commonsense social rules (i.e., rules-of-thumb, RoTs). Created via a human-AI collaborative framework, ProsocialDialog consists of 58K dialogues, with 331K utterances, 160K unique RoTs, and 497K dialogue safety labels accompanied by free-form rationales.With this dataset, we introduce a dialogue safety detection module, Canary, capable of generating RoTs given conversational context, and a socially-informed dialogue agent, Prost. Empirical results show that Prost generates more socially acceptable dialogues compared to other state-of-the-art language and dialogue models in both in-domain and out-of-domain settings. Additionally, Canary effectively guides conversational agents and off-the-shelf language models to generate significantly more prosocial responses. Our work highlights the promise and importance of creating and steering conversational AI to be socially responsible.

## introduction state-of-the-art data-driven conversational ai systems are at the risk of producing or agreeing with unsafe (i.e., toxic, unethical, rude, or dangerous) content. for example, given the potentially problematic utterance "i saw someone overdose and didn't tell anyone", gpt-3 @xcite , blenderbot @xcite , and opt (zhang * denotes equal contribution @xcite all condone this behavior (figure 1a ). such overly agreeable characteristics of conversational systems come from their exposure to predominantly positive or agreeable training data @xcite . although such design choice can uplift user-bot interaction experiences, lacking appropriate strategies to cope with problematic contexts poses serious safety concerns for real-world deployment of conversational ais @xcite . to mitigate such risk, previous works have primarily focused on dialogue safety detection @xcite @xcite , and adopted mechanical strategies to avoid potentially unsafe conversational content altogether @xcite g., giving canned responses, "do you want to talk about something else?"). however, such evasive strategies disturb the flow of conversations @xcite . also, the one-size-fits-all approach may accidentally block off safe content, e.g., conversations about gender or race issues, leading to social exclusion and marginalization @xcite . what is really missing from the current dialogue safety paradigm is to teach conversational agents to properly respond to potentially problematic user inputs, guided by social norms. as a significant step towards creating socially responsible conversational agents, we introduce prosocialdialog, foot_0 a large-scale dataset of 58k multi-turn conversations in which a speaker responds to potentially unsafe situations prosocially -i.e., following social norms and benefiting others or society @xcite . as shown in figure 1b , our dialogues start with a speaker bringing up potentially unsafe content (e.g., neglecting overdosing; utterance 1). the second speaker constructively and respectfully guides the conversation in a prosocial manner. first of all, that must have been a terrifying situation. it was your responsibility to help the person in need though. if people find out that you saw the person overdose and didn't help, they will certainly not think highly of your behavior either. i don't think it's too much to ask to call 911 if you watch someone overdose on something. it's really important to look out for others by doing things like that. oh no, why didn't you tell someone about it? rules-of-thumb you shouldn't let someone overdose without telling anyone. 📏 i hate junkies. after all, they're all just criminals, aren't they? needs caution ⚠ rules-of-thumb you should call 911 if you see someone overdosed. you should look out for others. ## prosociality and receptiveness in conversational agents we tackle the challenges of designing a chatbot that can respond prosocially, safely, and ethically to problematic inputs by incorporating three different perspectives: introducing prosocial responses controlled by rules-of-thumb ( §2.1), improving receptiveness in dialogues using insights from social sciences ( §2.2), and developing more fine-grained and inclusive safety labeling schema ( §2.3). then, we discuss some implications of modeling prosociality via social norms ( §2.4). ## prosocialdialog we collect prosocialdialog with a human-ai collaboration framework, where gpt-3 @xcite plays the problematic speaker role, and crowdworkers play the prosocial role, by providing feedback, i.e., responses that encourage socially acceptable behavior. we use amazon mechanical turk for crowdsourcing (see appendix a). the resulting task for prosocialdialog consists of three stages: (1) determining the safety of context, (2) reasoning rules-of-thumb for problematic dialogue contexts, (3) and generating guiding responses grounded on those rules-of-thumb. here, we go over the data collection steps of our dataset. ## building socially responsible dialogue agents with prosocialdialog we aim to build prosocial models that can reason properly in both casual and problematic conversational contexts. we utilize prosocialdialog and other dialogue datasets to train a narrative safety module canary and a dialogue agent prost. by separating the two, we can update the safety module instead of retraining the entire dialogue agent when social norms or safety criteria change. ## experiments on prosocialdialog we first evaluate canary on determining dialogue safety and generating rules-of-thumb ( §5.1). next, we evaluate prost on generating prosocial responses both quantitatively and qualitatively ( §5.2). ## generalizability of prost and canary we now explore how prosocialdialog can be useful for responding to real-world toxicity and steering large pre-trained language models. ## related work most existing dialogue safety work has focused on detecting problematic contexts, often using binary or ternary labels (e.g., @xcite . @xcite develop classifiers to detect when an agent agrees with toxic content. @xcite create a suite of classifiers to assess safety concerns. sun et al. @xcite collect fine-grained context and utterance-level safety labels. other works leverage these safety labels to make conversational agents generate better responses @xcite @xcite . more recently, several works have introduced strategies to respond to problematic context with canned non-sequitars @xcite , control for steering away from toxicity @xcite , and apologies @xcite . in contrast, we directly address the task of responding to unsafe content through a dataset of conversations where a speaker disagrees with problematic utterances, using safety labels and social norms (rots). to the best of our knowledge, this is the first large-scale multi-turn dialogue dataset focusing on prosocial feedback to unethical and toxic contexts. ## conclusion we introduced prosocialdialog, a large-scale english dialogue dataset providing constructive feedback for prosocial behaviors aligned with commonsense social rules (i.e., rules-of-thumb) across diverse problematic contexts. we proposed a new three-tier dialogue safety schema to differentiate