P oli S e: Reinforcing Politeness Using User Sentiment for Customer Care Response Generation (2022)

Abstract: The interaction between a consumer and the customer service representative greatly contributes to the overall customer experience. Therefore, to ensure customers’ comfort and retention, it is important that customer service agents and chatbots connect with users on social, cordial, and empathetic planes. In the current work, we automatically identify the sentiment of the user and transform the neutral responses into polite responses conforming to the sentiment and the conversational history. Our technique is basically a reinforced multi-task network- the primary task being ‘polite response generation’ and the secondary task being ‘sentiment analysis’- that uses a Transformer based encoder-decoder. We use sentiment annotated conversations from Twitter as the training data. The detailed evaluation shows that our proposed approach attains superior performance compared to the baseline models.

## introduction human-machine interactions have increased rapidly assisting humans in their everyday lives. with the growth in artificial intelligence (ai) and natural language processing (nlp), chatbots and personal assistants, such as microsoft's cortana, apple's siri, etc., have predominantly become a part of our daily lives. thus, research in recent years has been on modulating biases, styles, and control in text generation to enhance these interactions. customer care is an essential tool used by companies to provide guidance, and assistance and in building stable customer relations. the ease of access, ease of following up, and immediacy of social media has made it a strong platform for companies and applications to interact with their customers. in this regard, conversational agents such as shopping agents, customer service agents, and personal assistants have become extremely famous for handling the various queries of a customer and providing suitable responses to them. assisting the customer through social media channels is attaining high popularity. the main reason behind this is the fact that the important elements of social media are its immediacy, transparency, ease of following up, and providing a human-like feel to the company/brand. in this platform, we see the usage of polite and emphatic language, which is the center of our current study. for the growth of any company or application, customer care agents must be cordial and amicable to the customer. thus along with handling queries, the agents need to provide customer satisfaction by greeting, empathizing, appreciating feedback, apologizing at the right time, and thus building a strong relationship with the customer. recently, incorporating politeness in responses has been investigated @xcite @xcite to make the dialogue agent more human-like and interactive. also, the sentiments of the user are important for properly addressing the customer needs @xcite @xcite to assist in creating smooth and cordial conversations. therefore, for politely responding to the user it is important to have knowledge about user sentiment to avoid frustrating experiences and help build better systems. the usage of the user feedback in the form of sentiments is crucial to get contextually correct polite responses as presented in due to the unavailability of sentiment-annotated customer care conversations, we annotate the courteously yours customer care dataset (cyccd) @xcite with sentiment labels. to address the task of sentiment-guided polite response generation we design an end-to-end framework that identifies the sentiment of the user and uses the sentiment knowledge for generating polite responses. we employ a hierarchical transformer network that captures the utterance as well as contextual information for generating responses. while we utilize a bert-based classifier for predicting the sentiments. for incorporating politeness we use task-specific rewards that reinforce polite behavior in the generated customer care responses. the key contributions of our current work are: ## related work natural language generation (nlg) module has been gaining importance in several applications such as dialogue systems @xcite @xcite @xcite , question answering systems @xcite , and many other natural language interfaces. to help the users achieve their desired goals, response generation provides the medium through which a conversational agent can communicate with its user. in @xcite , the authors have proposed a hierarchical encoder-decoder model for capturing the dependencies in the utterances of a dialogue. emotion classification and analysis @xcite in customer support dialogue is important for a better understanding of the customer and to provide better customer support. lately, several works have been carried out on controlled text generation @xcite @xcite @xcite to generate responses with desired attributes. style transfer has been an emerging field in natural language processing (nlp). in @xcite , a dataset has been introduced for formality style transfer. unsupervised text style transfer has encouraged in transforming a given text without parallel data @xcite @xcite @xcite . one of the early works in politeness @xcite was based upon making the conversational agents more affective and socially intelligent by incorporating different politeness strategies in the responses. recently in @xcite , the authors proposed a neural framework that could induce politeness in chit-chat conversations in the absence of parallel data. lately, @xcite presented a method for increasing user satisfaction by inducing courteous phrases in the customer-care responses by exploiting reinforced pointer networks. one of the recent studies presented in @xcite devised a tag and generate framework for converting non-polite sentences into polite ones. recent works on politeness focuses on modeling politeness across languages @xcite , gender and age group @xcite , predicting variations in politeness @xcite and building a politeness adaptive system @xcite . from the existing literature on politeness, we can conclude that politeness in conversational agents is essential for increasing the social and affective understanding of conversational agents. our current work differs from the existing baselines as we include user feedback in the form of sentiment information to enhance the quality of generation and to make the responses contextually coherent with the dialog. ## methodology in this section, we formally define the problem statement and give a detailed description of our proposed methodology. ## datasets and experiments in this section we explain the dataset used for experimentation and briefly provide the implementation details and evaluation metrics. ## result and analysis in this section, we provide the experimental results for both sentiment classification and polite re-sponse generation. the proposed model performs significantly better than the other baselines for all the evaluation metrics and the improvement in each model is statistically significant compared to the other models for both the tasks foot_2 . the sentiment classification results are provided in ## conclusion in our current work, we propose the task of transforming a generic customer care response into a polite response according to the sentiment information of the user and consistent to the context history. for the proposed task, we design a hierarchical transformer network that captures the user and dialog context simultaneously. we create a multi-task network that identifies the sentiment information and uses the predicted sentiment information for generating the corresponding polite response. we use reinforcement learning based rewards to incorporate politeness in the customer care responses. experiments on the sentiment annotated cy-ccd dataset proves that the proposed network not only identifies the sentiment information but also generate the contextually correct polite responses in accordance to the user sentiments. hence, it can be concluded that the polite generation of responses is dependent on the sentiments of the user to ensure that the generation is correct and relevant.