Balancing Objectives in Counseling Conversations: Advancing Forwards or Looking Backwards (2020)

Abstract: Throughout a conversation, participants make choices that can orient the flow of the interaction. Such choices are particularly salient in the consequential domain of crisis counseling, where a difficulty for counselors is balancing between two key objectives: advancing the conversation towards a resolution, and empathetically addressing the crisis situation. In this work, we develop an unsupervised methodology to quantify how counselors manage this balance. Our main intuition is that if an utterance can only receive a narrow range of appropriate replies, then its likely aim is to advance the conversation forwards, towards a target within that range. Likewise, an utterance that can only appropriately follow a narrow range of possible utterances is likely aimed backwards at addressing a specific situation within that range. By applying this intuition, we can map each utterance to a continuous orientation axis that captures the degree to which it is intended to direct the flow of the conversation forwards or backwards. This unsupervised method allows us to characterize counselor behaviors in a large dataset of crisis counseling conversations, where we show that known counseling strategies intuitively align with this axis. We also illustrate how our measure can be indicative of a conversation’s progress, as well as its effectiveness.

## introduction participants in a conversation constantly shape the flow of the interaction through their choices. in psychological crisis counseling conversations, where counselors support individuals in mental distress, these choices arise in uniquely complex and high-stakes circumstances, and are reflected in rich conversational dynamics @xcite . as such, counseling is a valuable context for computationally modeling conversational behavior (atkins when i tell my mom about the bullies she just ignores me have you confided to anyone else about this? yeah there's my sister… she just tells me to get over it that sounds so frustrating, you deserve to be listened to. et @xcite @xcite . modeling the conversational choices of counselors in this endeavor is an important step towards better supporting them. counselors are driven by several objectives that serve the broader goal of helping the individual in distress; two key objectives are exemplified in figure 1 . 1 the counselor must advance a conversation towards a calmer state where the individual is better equipped to cope with their situation @xcite : in c 1 , the counselor prompts the individual to brainstorm options for social support. the counselor must also empathetically address what was already said, "coming to an empathic understanding" of the individual @xcite : in c 2 , the counselor validates feelings that the individual has just shared. balancing both objectives is often challenging, and overshooting in one direction can be detrimental to the conversation. a counselor who leans too much on advancing forwards could rush the conversation at the expense of establishing an empathetic connection; a counselor who leans too much backwards, on addressing what was already said, may fail to make any progress. in this work, we develop a method to examine counselor behaviors as they relate to this balancing challenge. we quantify the relative extent to which an utterance is aimed at advancing the conversation, versus addressing existing content. we thus map each utterance onto a continuous backwardsforwards axis which models the balance of these objectives, and refer to an utterance's position on this axis as its orientation. at an intuitive level, our approach considers the range of content that is expected to follow or precede a particular utterance. for an utterance like c 1 that aims to advance the conversation towards an intended target, we would expect a narrow range of appropriate replies, concentrated around that target (e.g., suggestions of possible confidants). we would likewise expect an utterance like c 2 that aims to address a previously-discussed situation to only be an appropriate reply for a narrow range of possible utterances, concentrated around that specific type of situation (e.g., disclosures of negative feelings). starting from this intuition, we develop an unsupervised method to quantify and compare these expected forwards and backwards ranges for any utterance, yielding our orientation measure. using this measure, we characterize counselor behaviors in a large collection of text-message conversations from a crisis counseling service, which we accessed in collaboration with the service and with the participants' consent. we show how orientation meaningfully distinguishes between key conversational strategies that counselors are taught during their training. we also show that our measure tracks a conversation's progress and can signal its effectiveness, highlighting the importance of balancing the advancing and addressing objectives, and laying the basis for future inquiries in establishing potential causal effects. in summary, we develop an unsupervised methodology that captures how counselors balance the conversational objectives of advancing and addressing (section 4), apply and validate it in a large dataset of counseling conversations (section 5), and use it to investigate the relation between a counselor's conversational behavior and their effectiveness (section 5.4). while our method is motivated by a salient challenge in counseling, we expect similar balancing problems to recur in other conversational settings where participants must carefully direct the flow of the interaction, such as court trials and debates (section 6). ## setting: counseling conversations we develop our method in the context of crisis text line, a crisis counseling platform which provides a free 24/7 service for anyone in mental crisis-henceforth texters-to have one-onone conversations via text message with affiliated counselors. we accessed a version of this collection, with over 1.5 million conversations, in collaboration with the platform and with the consent of the participants. the data was scrubbed of personally identifiable information by the platform. 2 these conversations are quite substantive, averaging 25 messages with 29 and 24 words per counselor and texter message, respectively. in each conversation, a crisis counselor's highlevel goal is to guide the texter towards a calmer mental state. in service of this goal, all counselors first complete 30 hours of training provided by the platform, which draws on past literature in counseling to recommend best practices and conversational strategies. the first author also completed the training to gain familiarity with the domain. while the platform offers guidance to counselors, their task is inevitably open-ended, given the emotional complexity of crisis situations. as such, the counselors are motivated by an explicit goal that structures the interaction, but they face a challenging flexibility in choosing how to act. ## background and related work we now describe the conversational challenge of balancing between advancing the conversation forwards or addressing what was previously said. our description of the challenge and our computational approach to studying it are informed by literature in counseling, on the platform's training material and on informal interviews with its staff. a conversational balance. a crisis counselor must fulfill multiple objectives in their broader goal of helping a texter. one objective is guiding the texter through their initial distress to a calmer mental state @xcite , as in figure 1 , c 1 . various strategies that aim to facilitate this advancing process are taught to counselors during training: for instance, a counselor may prompt a texter to identify a goal or cop-ing mechanism @xcite . as such, they attempt to move the conversation forwards, towards its eventual resolution. the counselor must also engage with the texter's concerns @xcite , as in c 2 , via strategies that empathetically address what the texter has already shared @xcite @xcite . for instance, counselors are taught to reflect, i.e., reframe a texter's previous message to convey understanding, or draw on what was said to affirm the texter's positive qualities. in doing so, the counselor looks backwards in the conversation. past work has posited the benefits of mixing between strategies that aim at either objective @xcite . however, as the training acknowledges, striking this balance is challenging. overzealously seeking to advance could cut short the process of establishing an empathetic connection. conversely, focusing on the conversation's past may not help with eventual problem solving @xcite , and risks stalling it. a texter may start to counterproductively rehash or ruminate on their concerns @xcite ; indeed, prior psychological work has highlighted the thin line between productive reflection and rumination @xcite . orientation. to examine this balancing dynamic, we model the choices that counselors make at each turn in a conversation. our approach is to derive a continuous axis spanned by advancing and addressing. we refer to an utterance's position on this axis, representing the relative extent to which it aims at either objective, as its orientation ω. we interpret a forwards-oriented utterance with positive ω as aiming to advance the conversation, and a backwards-oriented utterance with negative ω as aiming to address what was previously brought up. in the middle, the axis reflects the graded way in which a counselor can balance between aims-for instance, using something the texter has previously said to help motivate a problem-solving strategy. related characterizations. while we develop orientation to model a dynamic in counseling, we view it as a complement to other characterizations of conversational behaviors in varied settings. prior work has similarly considered how utterances relate to the preceding and subsequent discourse @xcite . frameworks like centering theory @xcite aim at identify-ing referenced entities, while we aim to more abstractly model interlocutor choices. past work has also examined how interlocutors mediate a conversation's trajectory through taking or ceding control @xcite or shifting topic @xcite ; @xcite considers the rate at which counselors in our setting advance across stages of a conversation. while these actions can be construed as forwards-oriented, we focus more on the interplay between forwards-and backwards-oriented actions. a counselor's objectives may also cut across these concepts: for instance, the training stresses the need for empathetic reflecting across all stages and topics. orientation also complements prior work on dialogue acts, which consider various roles that utterances play in discourse @xcite @xcite @xcite @xcite . in counseling settings, such approaches have highlighted strategies like reflection and questionasking @xcite @xcite @xcite @xcite @xcite . instead of modeling a particular taxonomy of actions, we model how counselors balance among the underlying objectives; we later relate orientation to these strategies (section 5). most of these approaches use annotations or predefined labeling schemes, while our method is unsupervised. ## measuring orientation we now describe our method to measure orientation, discussing our approach at a high level before elaborating on the particular operationalization. the code implementing our approach is distributed as part of the convokit library @xcite , at http://convokit.cornell.edu . ## application to counseling data we apply our method to characterize messages from crisis counselors on the platform. we compute the orientations of the phrasings they use, represented as dependency-parse arcs. we use a training set of 351,935 texter and counselor messages each, from a random sample of conversations omitted in subsequent analyses. 5 in what follows, we analyze counselor behaviors in terms of orientation, and illustrate how the measure can be useful for examining conversations. we start by validating our method via two complementary approaches. in a subset of sentences manually annotated with the counseling strategies they exhibit, we show that orientation meaningfully reflects these strategies (section 5.1). at a larger scale, we show that the orientation of utterances over the course of a conversation aligns with domain knowledge about counseling conversation structure (section 5.2). we also find that other measures for characterizing utterances are not as rich as orientation in capturing counseling strategies and conversation structure (section 5.3). finally, we show that a counselor's orientation in a conversation is tied to indicators of their effectiveness in helping the texter (section 5.4). ## discussion and future work in this work, we sought to examine a key balance in crisis counseling conversations between advancing forwards and addressing what has already been said. realizing this balance is one of the many challenges that crisis counselors must manage, and modeling the actions they take in light of such challenges could point to policies to better support them. for instance, our method could assist human supervisors in monitoring the progress of ongoing conversations to detect instances of rushing or stalling, or enable largerscale analyses of conversational behaviors to inform how counselors are trained. the unsupervised approach we propose could circumvent difficulties in getting large-scale annotations of such sensitive content. future work could bolster the measure's usefulness in several ways. technical improvements like richer utterance representations could improve the measure's fidelity; more sophisticated analyses could better capture the dynamic ways in which the balance of objectives is negotiated across many turns. the preliminary explorations in section 5.4 could also be extended to gauge the causal effects of counselors' behaviors @xcite . we expect balancing problems to recur in conversational settings beyond crisis counseling, such as court proceedings, interviews, debates and other mental health contexts like long-term therapy. in these settings, individuals also make potentially consequential choices that span the backwardsforwards orientation axis, such as addressing previous arguments @xcite or asking leading questions @xcite . our measure is designed to be broadly applicable, requiring no domain-specific annotations; we provide exploratory output on justice utterances from the supreme court's oral arguments in the appendix and release code implementing our approach at http://convokit.cornell.edu to encourage experiments in other domains. however, the method's efficacy in the present setting is likely boosted by the relative uniformity of crisis counseling conversations; and future work could aim to better accomodate settings with less structure and more linguistic variability. with such improvements, it would be interesting to study other domains where interlocutors are faced with conversational challenges.