Injecting Salesperson’s Dialogue Strategies in Large Language Models with Chain-of-Thought Reasoning (2024)

Abstract: Recent research in dialogue systems focuses on two main categories: task-oriented (TOD) and open-domain (chit-chat) dialogues. TOD systems help users complete specific tasks, while open-domain systems aim to create engaging conversations. However, user intents often emerge during interactions. A recent study introduced SalesBot, simulating dialogues that transition from chit-chat to task-oriented scenarios to train sales agents. Unfortunately, the initial data lacked smooth transitions and coherent long dialogues, resulting in unnatural interactions. This paper presents SalesBot 2.0, an improved dataset leveraging commonsense knowledge from large language models (LLMs) through strategic prompting. Additionally, we introduce SalesAgent, a novel model trained on salesperson interactions using chain-of-thought (CoT) reasoning. This model excels in transitioning topics, understanding user intents, and selecting appropriate strategies.Experiments with diverse user simulations validate our method’s effectiveness in controlling dialogue strategies in LLMs. SalesBot 2.0 enhances coherence and reduces aggression, improving model learning for sales-customer interactions.

## introduction dialogue systems have undergone significant advancements due to improvements in modeling techniques and computing power. however, most research in this field has focused on two distinct areas: task-oriented dialogues (tod) and open-domain dialogues, also known as chitchat systems. benchmark large-scale datasets for tod include sgd (schema-suided dialogue) @xcite and multiwoz @xcite , which contain annotated information on the user intents and dialogue states. in tod, the agent's goal is to identify the user's intention and fulfill their task by the end of the dialogue @xcite . meanwhile, research on open-domain chitchat systems and datasets @xcite @xcite aims to build models capable of engaging in free-form conversations. as pre-trained language models continue to improve, larger sets of dialogues are being used to train models with the ability to engage in free-form chatting @xcite . despite significant advancements in both areas, there has been a lack of integration between them, which is crucial for real-world applications. recently, efforts have been made to integrate tod and open-domain dialogues. for instance, @xcite incorporated chitchat responses into existing tod datasets to enhance the conversation's naturalness and social engagement. furthermore, there have been attempts to develop datasets and models capable of handling both tod and chitchat scenarios. @xcite developed the pivotbot model, capable of handling three predefined scenarios. one scenario involves adding chit-chat dialogues as context to a tod, while another includes chitchat dialogues to facilitate domain transition. in contrast, the third scenario involves incorporating chitchat content to enhance a tod, similar to the approach taken in the accen-tor model @xcite . however, these approaches assume that the user has an explicit goal to accomplish before starting the conversation, and chitchat responses merely enrich the conversation. in real scenarios, user intents or preferences are gradually revealed during the course of conversations @xcite . therefore, our work focuses on identifying potential intents, which may be explicitly or implicitly expressed by the user, and smoothly pivoting the dialogue to the associated tasks. with this idea, @xcite first introduced a framework for generating data that transitions from chit-chat to tod dialogues. they utilized two open-domain blenderbots to chat with each other and generate chit-chat dialogues, followed by a task-oriented intent detector to determine whether a transition to the tod system should be made. however, the salesbot dataset has some limitations, such as the absence of proper social engagement in 20% of the data, nonsensical detected intents due to short chit-chat context, and unnatural transition turns. our paper aims to improve the dataset by leveraging commonsense knowledge encoded in large language models (llms) to generate more human-like chitchat dialogues, along with smooth intent-guided transition turns. moreover, we propose a novel approach, salesagent, to inject dialogue strategies in llms by fine-tuning understanding and policy results from the improved data, salesbot 2.0. by incorporating chain-of-thought reasoning @xcite for training llms, our model can follow the desired dialogue strategies and provide controllable and explainable interactions. the experiments demonstrate that our model trained on salesbot 2.0 demonstrates superior interaction performance compared to a model trained on salesbot 1.0 across all evaluated perspectives. our contributions can be summarized below: our study focuses on a conversation scenario where a conversational agent attempts to steer the discussion towards determining whether the user is interested in receiving recommendations. similar scenarios have been explored in various domains. persuasive dialogue dataset prior studies @xcite @xcite focused on persuasive dialogue construction in different scenarios. @xcite annotated 34 dialogues in which a salesperson with expertise attempted to persuade a customer to purchase a camera. @xcite generated 200 dialogues through crowdsourcing, where one participant persuades the other to adopt a suggestion, such as cleaning a room. in comparison, @xcite collected 1,017 dialogues, where one participant was convinced to donate to a specific charity. while all of these datasets are limited to specific scenarios, our framework can generate dialogues with any potential intent, making it more versatile. additionally, our dataset is much larger, two to three times bigger than the previous ones, which makes it a valuable resource for training and evaluating non-collaborative conversational agents. ## salesbot 2.0 framework this work aims to improve the dataset, salesbot 1.0 @xcite , due to its higher aggres- salesbot 1.0 u: i was in a car accident when i was a teenager. it was a long time ago. a: i am so sorry to hear that. i hope you are doing better now. i have been in a few car accidents myself. u: i am doing a lot better now, thank you. what kind of accidents have you been in? a: i was involved in a motorcycle accident a few years ago. luckily i was not hurt. u: i am glad to hear you are okay. motorcycles are a lot of fun. a: yes they are. do you have any hobbies that you like to do in your spare time? u: i like to watch movies and play video games. i also like to read. what about you? a: what movie do you want to play? u: i want to check out a film while i'm over there. → proceed to tod (findmovie) ## salesagent: injecting dialogue strategies in llms the operational process of our model, as illustrated in figure 2 , begins by checking whether the user's intent is implicitly or explicitly expressed within their utterance. if such intent is detected, the model smoothly redirects the conversation toward topics aligned with the identified user intent. in cases where explicit intent is absent, the model gracefully maintains chit-chat engagement with the user. subsequently, the model crafts a response grounded in its prior thought process, encompassing both comprehension and policy considerations. with the enrichment of the salesbot dataset to include more realistic and immersive chit-chat dialogues, along with intent-guided transition exchanges, we leverage the dataset annotations to adapt an llm into a sales-oriented conversational agent that adheres to our designated strategies. instead of direct end-to-end training, we introduce salesagent, which is based on a large pretrained language model with a novel reasoning strategy inspired by the chain-of-thought (cot) approach @xcite . this enhancement empowers the model to identify potential user intents, select an appropriate policy based on the prevailing context, and generate a fitting response. hence, we can better control the fine-tuned llm to follow our designed strategies. to our knowledge, this is the first model that integrates intent detection, policy selection, and response generation capabilities as dialogue strategies in an end-to-end manner. as previously noted, salesbot 2.0 features comprehensive annotations, encompassing potential user intent, chit-chat context, transition dialogues, and transition boundaries. leveraging these meticulous annotations, we are capable of constructing a model that not only excels in generating responses but also exhibits the prowess to detect potential user intent and determine response policies prior to generating a response for better controllability and explainability. in the subsequent subsection, we delve into the integration of these capabilities into our salesagent. ## evaluation evaluating the performance of a salesperson encompasses a range of complexities, including variations in customer profiles, industry dynamics, and individual preferences. to facilitate a more consistent and standardized evaluation process, we establish two baseline models: the first one tuning on the previously developed salesbot 1.0 @xcite , and the second employs detailed instructions without any further tuning. our experiments reveal that the untuned model lacks controllability when responding to system-generated prompts in our specific environment, probably because of the inherent difficulty in managing the nuanced behaviors of a salesperson in llm responses. as a result, we have omitted the evaluation scores of this model from later discussions (see to thoroughly assess model quality, evaluations are conducted at both the turn-level and dialoguelevel using gpt-4 (openai, 2023), considering the llms' capabilities to provide comprehensive measurement of dialogue quality across various dimensions (lin and chen, 2023). ## conclusion this paper introduces a novel framework for generating intent-oriented dialogues, leveraging the commonsense knowledge of large language models (llms). our proposed salesbot 2.0 dataset encompasses thousands of human-like dialogues, demonstrating smoother transitions, improved naturalness, and enhanced consistency compared to existing datasets. this dataset represents a substantial contribution to the advancement of more sophisticated and effective conversational agents capable of aligning with users' preferences and requirements. additionally, we develop a model, salesagent, through fine-tuning on the proposed dataset. employing the innovative chain-of-thought (cot) reasoning technique, the model not only generates human-like responses but also formulates thoughts about the user's potential intent and determines the response policy. in the evaluation phase, our model consistently outperforms its counterpart trained on previous datasets, salesbot 1.0, across various perspectives. we also find out that the model without tuning lacks the ability to understand complex scenarios and react properly with mere instructions in the system prompt. this work serves as inspiration for future research in this domain, paving the way for the development of more intelligent conversational agents capable of better understanding and responding to users' needs.