Be Helpful but Don’t Talk too Much - Enhancing Helpfulness in Conversations through Relevance in Multi-Turn Emotional Support (2024)

Abstract: For a conversation to help and support, speakers should maintain an “effect-effort” tradeoff. As outlined in the gist of “Cognitive Relevance Principle”, helpful speakers should optimize the “cognitive relevance” through maximizing the “cognitive effects” and minimizing the “processing effort” imposed on listeners. Although preference learning methods have given rise a boon of studies in pursuit of“effect-optimization”, none have delved into the critical “effort-optimiazation” to fully cultivate the awareness of “optimal relevance” into thecognition of conversation agents. To address this gap, we integrate the “Cognitive Relevance Principle” into emotional support agents in the environment of multi-turn conversation. The results demonstrate a significant and robust improvement against the baseline systems with respect to response quality, human-likedness and supportivenss. This study offers compelling evidence for the effectiveness of the “Relevance Principle” in generating human-like, helpful, and harmless emotional support conversations. The source code will be available at https://github.com/CN-Eyetk/VLESA-ORL.git

## introduction as one of the central conversation principles, "relevance principle" stipulates that speakers should preserve a sophisticated balance between "effect" @xcite and "effort" @xcite . such cognitive principle of relevance is essential to emotional support conversations, as communicative stimuli will not change listeners' emotions if they fail to achieve such a delicate balance @xcite . to elucidate, figure 1 illustrates that a helpful speaker should provide as much helpful information as possible while minimizing the cognitive effort required for the listener to process the information. to be specific, "being too concise" typically gives rise to the misperception of indifference and apathy. on the other hand, "talking too much" unavoidably generates ironic and harmful interpretations and potentially threatens the recipient's mental state @xcite . solving the problem of "optimal relevance" serves as a pivotal motivation of various dialogic actions such as lexical choice @xcite , speech act decision @xcite and emotion control @xcite . though substantial to the cognitive account of communication, the "optimal relevance" has been longlastingly ignored regardless of the staggering rise of transformer-based dialogue systems, even the llm chatbots. pertaining to such a lack of focus, it should be noted that the most recent work that incorporates "relevance principle" with dialogue systems dates back to 1991 when a "relevance"aware model was implemented to generate helpful answers @xcite . to cultivate such a principle in the cognition of emotional support agents, we need to balance the optimization of "effect" as the cognitive gain, against the potential user's "effort" as the cogni- reward reward reward effect effort utterance-level question surprised emotion: strategy: generation: word-level i recently found the coursework too difficult, hard to compelete that's terrible what happened? (future resposne) ..(listenting).. supporter well, i feel guilty that i ate... i recently found the coursework too difficult, hard to compelete interact utterance-level question surprised emotion: strategy: thatʼs -> terrible. -> what -> happenend? generation: word-level utterance-level ???? ???? emotion: strategy: ???? generation: word-level seeker 0.1 0.3 0.4 0.2 thatʼs -> terrible. -> what -> happenend? attention z z z tive cost. although the reward modeling, driven by the annotation of human preference and feedback data, successfully reinforces the cognitive effect against various conversation goals @xcite @xcite , few studies have embraced the "effort"-modeling to improve the cognitive relevance. to provide deeper insight into cognitive relevance and its linkage to automatic emotional support, we propose a novel approach named "optimal relevance learning" (fig. 2 ). based on this training paradigm, our research question is "what will the reinforcement of 'cognitive relevance' bring to the performance of esc agent." we expect the acquisition of "optimal relevance" to improve the simulation of human cognition in conversation and consequently generate human-like and helpful responses with improved positive human feedback in multi-turn interactions. the novelty and contribution of our work are highlighted below. ## method our method centers on optimal relevance learning (orl for abbreviation) and a "variational latent emotional support agent" (vlesa for abbreviation). variational inference has proven effective in modeling high-level policy in task-oriented @xcite and open-domain dialogue systems @xcite . our vlesa relies on hierarchical variational autoencoder to model the coarse-to-fine dependency between high-level (speech act, emotion) and low-level policies (word generation). algorithm 1 optimal relevance learning pretrain multi-level policies π a , π e and π w initialize dialogue history @xmath0, u s 1 , • • • , u o t ] between user o and system s for @xmath1,2,. . . ,num_episodes do encode dialogue state s u ∼ lm enc (d) sample latent variables z a and z e from s u run policy a ∼ π a and u o t +1 from user simulator update d with u s t , u o t +1 , get d ′ and collect(s u , a, r, s ′ u ) and (s u , e, r, s ′ u ) assign per-word reward according to the importance of each word a [cls]->wt , collect {(s t w , w t , r t , s t+1 w )} l @xmath2, minimize value loss and policy loss 3.1 optimal relevance learning (orl) the proposed "optimal relevance learning" (orl) is inspired by hierarchical reinforcement learning in the task-oriented dialogue systems (rohmatillah and chien, 2023). comparable with saleh et al. (2020), policies on two levels share an identical reward source. the implementation of orl is outlined in algorithm 1 for clarity. ## conclusion the current work represents a pioneering effort that integrates the cognitive relevance theory in the field of systems of emotional support conversation. our results demonstrate the efficacy of the effect-effort" trade-off in boosting the general quality, especially in terms of coherence, human likeness, and harmlessness, as well as the helpfulness of emotional support agents. specifically, the joint optimization of effect and effort provides a reliable framework to customize conversation agents to human taste in a non-toxic, human-like, and helpful manner. our work also highlights the importance of integrating multi-level actions with human subjective judgment. the empirical findings recommend future studies to explore various human judgment and processing effort norms to build a human-like, helpful, and harmless conversation agent.