Joint Semantic and Strategy Matching for Persuasive Dialogue (2023)

Abstract: Persuasive dialogue aims to persuade users to achieve some targets by conversations. While previous persuasion models have achieved notable successes, they mostly base themselves on utterance semantic matching, and an important aspect has been ignored, that is, the strategy of the conversations, for example, the agent can choose an emotional-appeal strategy to impress users. Compared with utterance semantics, conversation strategies are high-level concepts, which can be informative and provide complementary information to achieve effective persuasions. In this paper, we propose to build a persuasion model by jointly modeling the conversation semantics and strategies, where we design a BERT-like module and an auto-regressive predictor to match the semantics and strategies, respectively. Experimental results indicate that our proposed approach can significantly improve the state-of-the-art baseline by 5% on a small dataset and 37% on a large dataset in terms of Recall@1. Detailed analyses show that the auto-regressive predictor contributes most to the final performance.

## introduction persuasive dialogue has recently attracted increasing attention from the research community and shown great potential in real-world scenarios. it aims to simulate the persuaders to communicate with the users for achieving a given goal (e.g., raising charitable donations). as demonstrated in figure 1 , the agent (left side) must not only introduce the charitable organization, but also consider the user's emotions and encourage a reasonable donation amount in order to persuade the user (right side) to donate. compared with the other conversation tasks, persuasive dialogue can be particularly challenging as it involves not only keeping the user this work was performed when chuhao jin and lingzhen kong were visiting meituan as research interns. to realize effective persuasive dialogue systems, recent years have witnessed several promising persuasion models. for example, @xcite utilizes reinforcement learning methods to avoid repetition and inconsistency in the persuader's responses; @xcite integrates factual information and social content into persuasive dialogue systems; @xcite developed an empathetic persuasive dialogue system, etc. although these models have achieved remarkable success, they primarily focus on designing improved algorithms from a semantic perspective to generate or match responses, rather than efficiently capitalizing on the vital role of conversational strategies. skilled persuaders may use high-level strategies to attract and guide the users to achieve the predefined goals. as shown in figure 1 , when continuing the persuasive conversation at the fourth turn, the agent may improve the effectiveness of the conversation by first using the phrase "i understand" to address the user's emotions. subsequently, utilizing a logical-appeal strategy before foot-in-door is more likely to result in the person accepting a donation proposal, rather than solely relying on a foot-in-door strategy by asking "think about donating a little bit of 50 cents". the conversation strategy and utterance play different yet complementary roles in persuasion modeling. the strategy abstracts the conversations in a more direct and global manner. successful persuasions are usually determined by effective transition patterns among different strategies. the utterances are the specific outcomes leveraged to interact with the users. they contain more detailed information that is not covered in the strategies. as a result, it is essential to consider both conversation strategies and utterances jointly to promote better persuasion models. inspired by the above analysis, we propose to build a persuasion dialogue model by simultaneously matching strategies and semantic. specifically, we propose using an auto-regressive decoder to generate next strategies sequentially. then we propose a new framework that combines both semantic and strategy matching scores as the final score for response candidates. we evaluate our proposed methods and state-of-the-art baselines on a small publicly available dataset @xcite and a newly collected dataset that are 30 times larger. experimental results indicate that our proposed approach significantly outperforms all baselines in terms of recall@1, by 5% on the small dataset and 37% on the large dataset. the main contributions of this paper are summarized as follows: (1) we propose to build persuasion models by jointly considering the conversation strategies and semantic, which, to our best knowledge, is the first time in this domain. (2) to achieve the above goal, we propose an autoregressive model to generate next conversation strategy from previous strategies and utterances. ## problem formulation this paper investigates the persuasive dialogue between two interlocutors, referred to as the persuader or "+" party and the persuadee or "-" party. the task is formulated as a retrieval-based problem, in which the model should select an appropriate response for the persuader in order to continue the conversation and try to achieve a persuasion goal. formally, we define a session of persuasive dialogue as a pair (d, r). d is the dialogue content, and r is a binary return indicating whether the persuasion is ultimately successful (@xmath0. the signal r is only available during the training stage. the dialogue content is given by @xmath1, u - 1 , . . . , u + n , u - n }, where u + t and u - t are the persuader's and persuadee's utterances at the t-th turn, respectively. each u t contains l t tokens. there are k atomic strategies that can be used by the persuader. note that the persuader can use multiple strategies in a single utterance. in this paper, we train multi-label classifiers from the data with strategy labels (see section 4.1) and use them to automatically label all utterances. we further denote the strategy set at the t-th turn as a binary vector a @xmath2, where a t,@xmath3, 1} indicating whether the i-th strategy is used or not. based on the above notations, we learn a matching model f with the following input and output: input: a dialogue context {u + 1 , u - 1 , ..., u + t , u - t } associated with a strategy set {a 1 , ..., a t }, and a response-strategy pair (u + c , a c ) output: a matching score between (0, 1). based on f , we select the next response with the largest matching score. ## our approach effective persuasive conversation requires a next utterance that is fluent, appealing and presents strategies that are successful in influencing users to achieve the persuasion goal. as shown in figure 2 , we propose a new framework called sara (semantic and strategy matching) that assesses the quality of response candidates from both utterance and strategy perspectives. for the utterances, we calculate the matching score between the candidate u + c and context c by a fine-tuned bert and two auxiliary tasks. as for the strategies, we design an auto-regressive model to generate the next strategies, which are expected to capture the strategy transition patterns, and estimate the matching score between the strategies of candidate a c and the generated strategies. finally, we aggregate the two scores as the final score for ranking candidates and select the most appropriate response. in the following, we introduce our model more in detail. transformer decoder 𝐴 e - 𝑡 -1 e + 𝐺 𝑡 -1 𝑡 -1 𝑡 -1 bert encoder 𝑆𝐸𝑃 𝑢 𝑐 + 𝐶 𝐶𝐿𝑆 𝑆𝐸𝑃 nup csp rp aggregation s 𝑁𝑈𝑃 s 𝑝 𝑢 𝑡-1 + our bert encoder semantic m atching strategy m atching 𝐴 e - e + 𝐺 𝑡 𝐴 𝑡 𝐴 𝑡 + 1 𝐶 𝑝𝑜𝑠𝑡 𝐶 𝑝𝑟𝑒 csp 𝐶 𝑠𝑢𝑏 bert encoder 𝑆𝐸𝑃 𝐶𝐿𝑆 𝑆𝐸𝑃 rp bert encoder 𝑆𝐸𝑃 𝐶𝐿𝑆 𝑆𝐸𝑃 𝐵𝑂𝑅 𝐶𝐿𝑆 𝑢 𝑡-1 -our bert encoder 𝑆𝐸𝑃 𝐵𝑂𝐸 𝐶𝐿𝑆 𝐾 1 0 0 ⋮ 1 𝑖=𝑡-1 𝑁 𝑟 𝑖 pos. enc. 𝑡 𝑡 𝑡 𝐴 𝑐 cosine similarity 𝑢 𝑐 + 𝑠 𝑐 strategy classifier auxiliary tasks final m atching score figure 2: overview of our sara framework. at the t-th turn, u + t and u - t are persuader's and persuadee's utterances, e + t and e - t are the semantic representation of u + t and u - t , a t is the used strategies, g t is the expect gain. ## experiments we conduct experiments to compare our model with two categories of retrieval-based dialogue models, and compare our strategy matching module with strategy selection baselines. moreover, we do analyses to gain in-depth insights. we also conduct an online evaluation by deploying our sara model to real users. implementation details of sara are described in section a.1 of appendix. ## related work retrieval-based dialogue systems aim to select proper responses from a large-scale response repository by estimating the matching degree between context and response @xcite . pre-trained methods have become the dominant approach @xcite @xcite . directly fine-tuning the pre-trained model as a dialogue model is less effective because of a domain gap. a common practice is to further train the model with the language modeling objectives using texts from the target domain to reduce the negative impact @xcite . as a preliminary study of modeling persuasion strategy sequentially in persuasive dialogue, we follow the retrieval-based paradigm. in addition to being easier to evaluate, it is more controllable and thus can respond ethically in real applications. persuasive dialogue systems aim to influence the views, attitudes, or behaviors of people through conversation. compared to task-oriented and chitchat-oriented dialogue, identifying and applying persuasion strategies are unique and essential in developing persuasive dialogue systems @xcite @xcite . some work @xcite assumes that persuasion strategies are hidden in the persuader's utterances, and thus a separate conversation model is learned from the persuader's content, which is reinforced by the feedback from a model learned from the other person. some works @xcite build a persuasion strategy classifier and add the loss of matching the strategies of generated and ground-truth responses during the training, but they do not explicitly use strategies in inference. some work @xcite assumes that the proper strategy is given and investigate how to take the strategy as a condition to generate better response. different from the previous works, to our best knowledge, we are the first to explicitly predict the proper strategies by the whole sequence of previous strategies, utterances from the persuader and the other person, and explicitly integrate the predicted strategies in scoring the best response. offline reinforcement learning (rl) approaches provide an offline learning paradigm for constructing sequential decision-making engines based on the fixed dataset without online environment interaction. recently, offline rl approaches have been used in task-oriented dialogue systems, which were used to optimize the policy of response selection to achieve user goals @xcite @xcite . since some representative offline rl algorithms, such as decision transformer @xcite can conduct sequential action selection conditioned on a preset target, they provide a promising way to model the sequential strategy generation process and inspire our ideas to improve persuasive dialogue. ## conclusion in this paper, we propose a new framework called sara that jointly considers semantic and strategy matching in persuasive dialogue. for strategy matching, we propose using an auto-regressive generator to sequentially predict the next appropriate strategies based on all previous utterances, strategies, and expected gain to return. we also collect a new large-scale dataset from a real application. experimental results on the dataset and a small public dataset indicate that integrating strategy matching can significantly improve the state-of-the-art dialogue baselines in terms of r@1. our proposed strategy generation model performs the best among strategy selection baselines. online evaluation also indicates the effectiveness of our model in achieving the ultimate goal of persuasion. the big impact of training data scale suggests using enough data is necessary in future work. we plan to improve our persuasive dialogue systems by supervised data, e.g.differentiating high-quality training data from others by the average return of a caller.