e- THERAPIST : I suggest you to cultivate a mindset of positivity and nurture uplifting thoughts (2023)

Abstract: The shortage of therapists for mental health patients emphasizes the importance of globally accessible dialogue systems alleviating their issues. To have effective interpersonal psychotherapy, these systems must exhibit politeness and empathy when needed. However, these factors may vary as per the user’s gender, age, persona, and sentiment. Hence, in order to establish trust and provide a personalized cordial experience, it is essential that generated responses should be tailored to individual profiles and attributes. Focusing on this objective, we propose e-THERAPIST, a novel polite interpersonal psychotherapy dialogue system to address issues like depression, anxiety, schizophrenia, etc. We begin by curating a unique conversational dataset for psychotherapy, called PsyCon. It is annotated at two levels: (i) dialogue-level - including user’s profile information (gender, age, persona) and therapist’s psychotherapeutic approach; and (ii) utterance-level - encompassing user’s sentiment and therapist’s politeness, and interpersonal behaviour. Then, we devise a novel reward model to adapt correct polite interpersonal behaviour and use it to train e-THERAPIST on PsyCon employing NLPO loss. Our extensive empirical analysis validates the effectiveness of each component of the proposed e-THERAPIST demonstrating its potential impact in psychotherapy settings.

## introduction the prevalence of psychological and mental disorders, such as depression, anxiety, stress and others, is increasing globally. approximately 5% of the adult population worldwide is estimated to experience depression @xcite . consequently, the demand for counseling services continues to rise, and the existing mental health workforce is struggling to meet the needs adequately. hence, dialogue systems possessing social influence skills like psychotherapy are crucial for expanding the application of technology to a wide range of realistic situations. politeness has shown to be effective in driving a smooth and engaging conversation during psychotherapy @xcite . however, politeness, in itself, encompasses various aspects @xcite @xcite , making it challenging to instil in a dialogue system. for instance, the degree of politeness in communication varies based on the gender and age of the person involved @xcite @xcite ; conversations with females/elders often include more polite expressions than males/youths. politeness can also be influenced by various aspects of an individual's personality traits @xcite @xcite . interpersonal behaviour advances the understanding of the interpersonal dispositions associated with psychological problems. the interpersonal circumplex (ipc) model assesses these dispositions and facilitates the comprehension of user's social cognition, motivation, and behaviour @xcite . it indicates that adults and youth facing psychological issues are more likely to seek negative feedback and criticism in their interactions compared to older individuals. further, females tend to initiate more interpersonal stressors than males, and their responses to these stressors often involve rumination and negative thinking patterns. the ipc model can also help identify patterns and relationships between personality traits, thereby fostering an understanding of how individuals interact with others @xcite . the incorporation of polite and interpersonal conduct into the dialogue agents establishes a friendly and congenial environment, thereby, providing a better personalized experience to users. the user's sentiment further facilitates the generation of contextually correct polite @xcite and appropriate interpersonal behavioural responses. for instance, in figure 1 , for the "young female openness" sample (fourth utterance), the user expresses a negative sentiment, thus agent's behaviour in the form of imposing or confrontation might make the user more angry or dissatisfied. hence, it should adapt helping, understanding or empathetic behaviour. besides, a therapeutic dialogue system needs to utilize various psychotherapeutic approaches @xcite @xcite during the ongoing conversation based on user's personal profile (gender, age and persona) to produce effective outcomes during psychotherapy as depicted in figure1. driven by these considerations, in this work, we propose e-therapist, a novel gender, age, persona and sentiment-aware polite and interpersonal dialogue system for psychotherapy. to develop e-therapist, we exploit the newly designed seven rewards in a reinforcement learning (rl) setting @xcite @xcite . this allows e-therapist to learn and enhance their performance based on the rewards received through interactions with the environment. to build e-therapist, we first curate a novel conversational dataset, named psycon by employing welldesigned prompts with manual interventions. then, we annotate the user's utterances with sentiment, and agent's utterances with politeness and interpersonal behavioural information. first, psycon is used to fine-tune a large language model (llm) in a supervised setting. then, we fine-tune this trained llm in an rl framework incorporating a novel reward function. this reward function ensures appropriate psychotherapeutic approach, politeness, and interpersonal behaviour based on gender, age, persona and sentiment of the user along with context adequacy and fluency in the generated responses. this reward is used in a natural language policy optimization (nlpo) loss (ramamurthy et al., 2022) to optimize the model fine-tuned in a supervised setting. finally, the performance of the proposed system is evaluated through both automatic metrics and human assessment. in summary, the key contributions of our current work are summarized as follows: (i) introduced e-therapist, a novel gender, age, persona and sentiment-aware polite and interpersonal dialogue system for psychotherapy, fine-tuned in an rl environment; (ii) created a novel conversational dataset for psychotherapy, psycon, and annotated it at two distinct levels -(a) dialogue-level with gender, age, persona, and psychotherapeutic approach (b) utterance-level information, viz. sentiment, politeness, and interpersonal behaviour information; (iii) devised a novel reward function incorporating five attribute-specific rewards and two responsequality rewards to generate engaging, fluent, and interactive responses tailored to the user's sentiment, gender, age, and persona; (iv) conducted extensive empirical evaluation to test the efficacy of e-therapist in terms of novel metrics, attributesuccess and response-quality with respect to the strong baselines. ## related work the issue of mental health disorders, which is a significant concern for public health @xcite , has been the focus of previous research, including computational studies. while depression has received the most attention, other mental illnesses like anxiety, schizophrenia, post-traumatic stress disorder, suicide risk, and self-harm have also been examined @xcite . a few studies have examined the posts and blogs of users on social sites to detect depression @xcite , suicidal thoughts @xcite , and other mental health issues @xcite ) using natural language processing (nlp) techniques. some researchers have also worked on developing "therapybots" and creating dialogue agents to provide therapeutic support @xcite . in recent times, generating empathetic responses in psychotherapeutic conversations has grown in popularity @xcite . in order to help mental health supporters, @xcite investigated empathy rewriting as a text generation task. the authors in @xcite focused on generating sentiment-driven motivational responses in mental health support dialogues. a few studies have explored the role of politeness in improving the sense of empathy and compassion during conversation @xcite . the agent's courteous attitude conveys a sense of concern and emotional involvement like a human companion. further, comprehending and demonstrating proper interpersonal behaviour has proved its effectiveness in psychology to study interpersonal processes, personality traits, and relationship functioning @xcite @xcite . studies have suggested that good quality interpersonal relationships and behaviour are important for peoples' social functioning and mental health @xcite . lately, the authors in @xcite have demonstrated that inculcating politeness in the agent based on the user's personal profile, such as gender and age group makes the dialogue agent capable of identifying subtle language changes while conversing with different users. likewise, taking the users' persona into account while generating responses will further enhance the personalization quotient in the dialogue systems @xcite @xcite . inspired by the significance of politeness and interpersonal relationship in psychological support, together with the subtle change in these aspects with users' profiles (gender, age and persona) and sentiment information, we propose a polite and interpersonal dialogue system for psychotherapy that generates responses in accordance to user's age, gender, persona and sentiment. to the best of our knowledge, ours is the first attempt that exploits politeness and interpersonal relationship to generate precise responses in dialogue systems for psychotherapy. furthermore, our research pioneers the exploration of how politeness and interpersonal relationships differ across individuals of varying gender, age groups, and personas within psychotherapeutic conversations. ## dataset to develop e-therapist, we create psycon, a novel high-quality conversational dataset for psychotherapy. we focus on conversations considering the gender, age and persona of the user with the ultimate purpose of enhancing mental health support in a personalized way and improving the overall outlook of people facing psychological issues. ## e-therapist we warm start with a pre-trained language model gpt-2 medium @xcite fine-tuned on a psycon denoted by t c n . each conversation in t c n can be represented by t c @xmath0, u 0 , .., t i , u i , .., t t -1 , u t -1 }; t i and u i give the therapist's and user's i th utterance in the conversation, respectively, where 0 <= @xmath1. for each t c k , the user's corresponding gender, age, and persona can be represented by g @xmath2, f }, a @xmath3, a, e} and p @xmath4, c, e, a, n }, respectively. further, for each user utterance, u i in t c k , corresponding sentiment s @xmath5, neutral, positive} is predicted by sentiment classifier sc. the concatenated u i with s i can be given as u we call the trained lm θ as supervised learning fine-tuned language model (sllm). next, to end up with proposed e-therapist, we fine-tune sllm using our novel reward model in rl setting to generate user's profile and sentiment-aware polite responses with correct interpersonal behaviour in therapeutic conversations. the architecture of the proposed system is shown in figure 2 . ## experiments baselines. we compare the performance of proposed e-therapist with seven strong baselines -lm @xcite , ardm @xcite : lms trained alternatively for both user and therapist, gpt-critic @xcite : improving lm through cloning of critic-guided selfgenerated sentences during fine-tuning, sllm: fine-tuned lm in a supervised setting with user profiles and sentiment, sllm+ppo: supervised + ppo loss based fine-tuning, e-therapist-r: e-therapist with @xmath6, e-therapist-asr: e-therapist with r @xmath7: e-therapist with r @xmath8. implementation. we experiment with different values of @xmath9, 3, 4, 5, 8}, and found that @xmath10. we use gpt-2 as the language model (lm) and context window size of 4. we employ top-k sampling with @xmath11. for rl-based models, we determine the end values of coefficients β 1 = 0.1, β 2 = 0.2, β 3 = 0.2, β 4 = 0.2, β 5 = 0.3, γ 1 = 0.5 γ 2 = 0.5, δ 1 = 0.75, and δ 2 = 0.25 empirically. due to space restrictions, we have included detailed implementations and weight optimization in sections a.3 and a.5 of the appendix, respectively. for human evaluation, we ask the same six experts to evaluate the e-therapist. initially, each evaluator engages with the system 5 times, with a different set of responses each time. these 5 human-evaluated interactions are then crossverified by psychotherapists from a governmentrun institution to ensure evaluation quality. upon passing verification, the additional 90 interactions (15 per evaluator) are evaluated, resulting in a total of 120 human-evaluated dialogues. human evaluation metrics also includes ga c , p c , ct c , p o c , ib c as attribute-success and fluency (f ), consistency (c), and non-repetitiveness (n r ). all dialogue interactions are evaluated on an integer likert scale of 1-5 foot_4 foot_5 . ## conclusion this work introduces e-therapist, a novel polite interpersonal dialogue system for psychotherapy. first, considering user profile information (gender, age, persona) and the therapist's psy-chotherapeutic approach, a new conversational dataset psycon is created by prompting gpt-j with manual interventions. further, it is annotated with user's sentiment and therapist politeness label and interpersonal behaviour. then, a psychotherapeutic dialogue system e-therapist is developed in an rl framework. to ensure the preferences of gender, age, persona and sentiment-aware polite interpersonal psychotherapeutic responses, a novel reward function is designed consisting of seven rewards. our results concludes the requirement of each of the reward to ensure polite and interpersonal psychotherapeutic responses tailored to user's profile and attributes, and eventually contributing to improved therapy experiences. in future, we would like to incorporate external knowledge to facilitate more factual conversations.