T alk U p: Paving the Way for Understanding Empowering Language (2023)

Abstract: Empowering language is important in many real-world contexts, from education to workplace dynamics to healthcare. Though language technologies are growing more prevalent in these contexts, empowerment has seldom been studied in NLP, and moreover, it is inherently challenging to operationalize because of its implicit nature. This work builds from linguistic and social psychology literature to explore what characterizes empowering language. We then crowdsource a novel dataset of Reddit posts labeled for empowerment, reasons why these posts are empowering to readers, and the social relationships between posters and readers. Our preliminary analyses show that this dataset, which we call TalkUp, can be used to train language models that capture empowering and disempowering language. More broadly, TalkUp provides an avenue to explore implication, presuppositions, and how social context influences the meaning of language.

## introduction empowerment -the act of supporting someone's ability to make their own decisions, create change, and improve their lives -is a goal in many social interactions. for instance, teachers aim to empower their students, social workers aim to empower their clients, and politicians aim to empower their supporters. a growing body of psychology and linguistics research shows how empowerment -and disempowerment -can impact people by increasing their sense of self-efficacy and self-esteem @xcite . understanding how empowerment is conveyed in language becomes more important as language technologies are increasingly being used in interactive contexts like education @xcite , workplace communication (prabhakaran figure 1 : two examples of annotated conversations in talkup. post 1 is straightforwardly empowering, but post 2 is inherently ambiguous and could either be interpreted as helpful advice or as a dismissive, belittling comment. social context can also affect post 2's implications: the post might elicit different reactions if it were written by a woman to a man or vice versa. @xcite , and healthcare @xcite . whether we are building dialogue agents for mental health support, supplementing children's education, or analyzing managers' feedback to their employees, language that empowers or disempowers the reader can have drastically different effects. with a few exceptions @xcite , prior nlp research has focused on flagging harmful text, but there has been much less investigation of what makes text helpful. other works have studied related concepts like condescension @xcite and implicit toxicity @xcite @xcite , and we build off of these to construct a dataset that complements those tasks. consider the two examples of potentially empowering interactions in figure 1 . empowerment exhibits the importance of social context in understanding the pragmatics of language: whether an exchange is interpreted as empowering or disempowering may depend on the participants' social roles and the power dynamics implied by their identities, including race, age, socioeconomic class, and many other social dimensions. furthermore, empowerment cannot be easily detected with sentiment or emotion analyzers, since interactions with negative implicatures can be empowering (e.g., you can quit!!!), and messages that are positive on the surface can be disempowering (e.g., you are so articulate for a girl!) @xcite . modern language technologies do not model social context or deeper pragmatic phenomena, and thus are unable to capture or control for empowerment. this work makes concrete steps towards understanding these linguistic phenomena by investigating the following research questions: [rq1] what makes language empowering, and how is it manifested in language? [rq2] can empowerment be detected with computational approaches? our contributions are threefold: (1) we introduce the new task of empowerment detection, grounding it in linguistic and psychology literature. ultimately, talkup aims to assist future researchers in developing models that can detect, generate, and control for empowerment, and to facilitate broader exploration of pragmatics. we have by no means covered every possible social dimension, but by focusing on a few social factors in the simplified setting of two-turn dialogues, we hope that talkup's framework can make strides toward understanding language in more complex social interactions, such as conversations involv-ing intersectionality as well as longer multi-turn dialogues. ## background we discuss empowerment following its definitions in clinical psychology @xcite . we find this most appropriate for studying language because clinical psychology practice is usually centered around dialogue between clinician and patient, and because it involves concrete implications about individuals rather than vague cultural phenomena. thus, summarizing the different characteristics of empowerment described in psychology literature, we define empowering text as text that supports the reader's rights, choices, selffulfillment, or self-esteem. incorporating empowerment in dialogue agents, mental health support chatbots, educational assistants, and other social-oriented nlp applications is clearly a desirable goal. however, empowerment is inherently challenging to operationalize for several reasons. first, it is a flexible term that describes a wide range of behaviors across many domains -empowerment in economics, for example, looks very different from empowerment in a therapy session @xcite . we follow recent literature outside of nlp in trying to distill these varied interactions into a concrete definition. second, empowerment is implicit: it is often read in between the lines rather than declared explicitly. text might be empowering by reminding someone of their range of options to choose from, encouraging them to take action, asking for and valuing their opinion, or even validating their feelings @xcite . third, empowerment is heavily dependent on social context: whether or not a person is empowered depends on who is saying what to whom. we incorporate these consideration in our data collection process described next. ## the talkup dataset we now discuss the talkup dataset's construction. annotation scheme our annotation task 2 was shaped through multiple pilot studies, where we learned that context is useful for judging a post, annotators' free-response descriptions of social roles lack consistency, and posts are often inherently ambiguous. we elaborate on these findings in appendix d. based on these insights, the final task, which is illustrated in figure 1 , consists of three main parts: data source talkup consists of english reddit posts from rtgender @xcite , a collection of 25m comments on posts from five different domains, each labeled with the genders of the commenter and the original poster. we take advantage of the fact that these conversations are already annotated for gender, which provides contextual information about who is speaking to whom and allows us to explore at least one dimension of social context. 3 though rtgender contains posts from several platforms, given our focus on conversational language, we specifically selected rtgender posts from reddit because they were the most generalizable and contained natural-sounding conversations. we manually chose five subreddits, aiming to include (1) a diverse range of topics and user demographics, and (2) discussions that are personal rather than about external events unrelated to the conversants. the subreddits are listed in we filtered data from these subreddits to exclude posts or responses that exceeded 4 sentences in length or were shorter than 5 words. additionally, we excluded posts with "redditisms", and posts that were edited after they were initially posted (marked "edit:" by the original poster) and posts that began with quoted text (marked ">") were removed. from pilot studies, we found that models can help to surface potentially empowering posts and help increase the yield of posts that were actually labeled as empowering by annotators. we trained a roberta-based regression model with the data we collected from the pilot studies to predict the level of empowerment (0 for disempowering, 0.5 for neutral, 1 for empowering) in reddit posts. we used this model to rank and select the top-k posts for annotation, and continually updated the model as we collected more data. 4 to ensure we annotate a diverse range of posts, our final annotation task was done with half model-surfaced posts and half randomly-sampled posts. annotation on amazon mechanical turk with 1k model-surfaced posts and 1k randomly-sampled posts spread evenly among the five subreddits, we collected annotations via amazon mechanical turk (amt). appendix f shows a screenshot of the user interface displayed to annotators. each example was annotated by 3 different workers. to ensure high quality annotations, we required annotators to have amt's masters qualification, foot_3 a task approval rate of at least 95%, and a minimum of 100 prior tasks completed. additionally, since our task requires english fluency, we limited annotators to those located in the us or canada. workers were compensated at $15/hour, and we calculated the reward per task based on the average time spent on each annotation in our pilot studies. ## data analysis we present preliminary analyses of talkup. empowerment is a nuanced phenomenon in pragmatics and deeper exploration of social and linguistic variables remains open for future work. the analyses we present here provide some initial, surface-level insights into what makes language empowering. ## example application: unearthing empowerment patterns on reddit as a case study, we demonstrate how talkup and the trained empowerment classifier can be used to uncover interesting patterns in how people use empowering language. specifically, we apply the trained classifier in §4.5 to generate empowerment labels of all reddit posts and responses in rtgender, to learn about how both posters and responders communicate. 9 we analyze empowering and disempowering posts in different subreddits and by different gender of the poster and responder. ## related work to our knowledge, @xcite is the only prior work exploring empowerment in nlp, but the contributions of our works are quite different. @xcite primarily focus on an algorithm for predicting rare classes and use empowerment as an example. in contrast, we focus on understanding empowering language itself, before developing automated detection tools. we explore the reasons behind empowerment, considering multiple dimensions of social context such as gender, topic, and poster-commenter alignment. @xcite use non-public data from a specific cancer support group, while talkup spans diverse topics and user bases, making our scope broader and more generalizable. as empowering language is not well understood in nlp, our work has also drawn insights from research on related concepts: power. danescu-niculescu-mizil et al. (2011) develop a framework for analyzing power differences in social interactions based on how much one conversant echoes the linguistic style of the other. prabhakaran and rambow (2014a,b) predict power levels of participants in written dialogue from the enron email corpus, and several other of their works explore power dynamics in other contexts, such as gender @xcite and political debates @xcite . our work studies empowerment rather than power. power is certainly a closely related concept, but empowerment is a distinct linguistic phenomenon -it concerns not just static power levels, but interactions that increase or decrease a person's power, and it is also a broader concept that encompasses things like self-fulfillment and self-esteem. while power has primarily been analyzed at the word level, such as by examining connotations of particular verbs @xcite , our work attempts to look at higher-level pragmatics -implications that may not be captured by word choice alone, but suggested between the lines. condescension. the closest concept to empowerment that has been more thoroughly studied in nlp is condescension. prior works have defined condescension as language that is not overtly negative, but that assumes a status difference between the speaker and listener that the listener disagrees with @xcite . intuitively, condescension can be interpreted as roughly the opposite of empowerment: it implicitly suggests that the listener has lower status or worth. our work particularly builds upon @xcite : they develop talkdown, a dataset of reddit posts labeled as "condescending" or "not condescending." specifically, they identify condescending posts by looking for replies that indicate the original post is condescending. our approach is parallel to this work: we likewise surface reddit posts whose responses indicate that the original post is empowering (thus aligning with our definition of empowerment in §2 as an effect on the listener). talkup complements talkdown by focusing on the positive aspect of such language: instead of only identifying text as condescending or not condescending, we distinguish between disempowering, empower, and neutral posts. ## future directions in this work, we focus only on empowerment classification and detection, with our primary contribution being the proposal of a new dataset to facilitate research in a new area of computational sociolinguistics. however, talkup not only can be used to detect empowerment, but also to generate more empowering language. as in @xcite , we believe a classifier trained with our data can be used to assign rewards that tailor a generation model to produce more empowering outputs. an empowerment classifier can also be used for controllable text generation with constrained decoding, as in yang and klein (2021), @xcite , and @xcite . additionally, a model that can control for empowerment could be used to suggest edits to make human-written text more empower-ing, which has potential applications in real-world dialogue settings like education and psychotherapy. talkup focuses on simple two-turn interactions with 3 social variables (gender, alignment, and topic), but its framework can extend to more complex social interactions. for example, there are many other social roles that can influence power dynamics, including occupation (e.g. manager vs. employee), race (e.g. white vs person of color), and age (e.g. old vs. young person). different combinations of these identities can result in further intersectional dynamics @xcite @xcite . additionally, since most real-world conversations are long backand-forth exchanges, we encourage future work to explore empowerment in multi-turn dialogues. ## conclusion we explore the problem of empowerment detection, grounding it in relevant social psychology and linguistics literature. to facilitate studies of empowerment, we create talkup, a high-quality dataset of reddit posts labeled for empowerment and other contextual information. our preliminary analyses demonstrate that empowerment is not captured by existing nlp methods and models, but that it can be detected with our dataset. furthermore, we demonstrate the importance of social context in understanding empowering language with different genders, poster-commenter alignments, and topics of discussion. in studying empowerment, we work towards bigger open challenges in pragmatics, implicature, and social context in nlp. the practical goal is to advance nlp technologies with positive impact through understanding and incorporating empowerment in practical applications including education, therapy, medicine, and more.