Self-Supervised Contrastive Learning for Efficient User Satisfaction Prediction in Conversational Agents (2021)

Abstract: Turn-level user satisfaction is one of the most important performance metrics for conversational agents. It can be used to monitor the agent’s performance and provide insights about defective user experiences. While end-to-end deep learning has shown promising results, having access to a large number of reliable annotated samples required by these methods remains challenging. In a large-scale conversational system, there is a growing number of newly developed skills, making the traditional data collection, annotation, and modeling process impractical due to the required annotation costs and the turnaround times. In this paper, we suggest a self-supervised contrastive learning approach that leverages the pool of unlabeled data to learn user-agent interactions. We show that the pre-trained models using the self-supervised objective are transferable to the user satisfaction prediction. In addition, we propose a novel few-shot transfer learning approach that ensures better transferability for very small sample sizes. The suggested few-shot method does not require any inner loop optimization process and is scalable to very large datasets and complex models. Based on our experiments using real data from a large-scale commercial system, the suggested approach is able to significantly reduce the required number of annotations, while improving the generalization on unseen skills.

## introduction nowadays automated conversational agents such as alexa, siri, google assistant, cortana, etc. are widespread and play an important role in many different aspects of our lives. their applications vary from storytelling and education for children to assisting the elderly and disabled with their daily activities. any successful conversational agent should be able to communicate in different languages and accents, understand the conversation context, analyze the query paraphrases, and route the requests to various skills available for handling the user's request @xcite . in such a large-scale system with many components, it is crucial to understand if the human user is satisfied with the automated agent's response and actions. in other words, it is desirable to know if the agent is communicating properly and providing the service that is expected by the user. in the literature, it is referred to as targeted turn-level satisfaction as we are only interested in the user's satisfaction for a certain conversation turn given the context of the conversation, and not the overall satisfaction for the whole conversation @xcite . perhaps the most basic use of a user satisfaction model would be to monitor the performance of an agent and to detect defects as a first step to fix issues and improve the system. anticipating user dissatisfaction for a certain turn in a conversation, an agent would be able to ask the user for repeating the request or providing more information, improving the final experience. also, a powerful user satisfaction model can be used as a ranking or scoring measure to select the most satisfying response among a set of candidates and hence guiding the conversation. the problem of user satisfaction modeling has recently attracted significant research attention @xcite @xcite @xcite . these methods either rely on annotated datasets providing ground-truth labels to train and evaluate @xcite or rely on ad hoc or human-engineered metrics that do not necessarily model the true user satisfaction @xcite . access to reliable annotations to be used in building satisfaction models has been very challenging partly due to the fact that a large-scale conversation system supports many different devices as well as voice, language, and application components, providing access to a wide variety of skills. the traditional approach of collecting samples from the live system traffic and tasking human annotators to label samples would not be scalable due to the cost of annotations as well as the turn-around time required to collect and annotate data for a new skill or feature. note that onboarding new skills in a timely manner is a crucial to ensure active skill developer engagement. to address this problem, we propose a novel training objective and transfer learning scheme that significantly improves not only the data efficiency but also the model generalization to unseen skills. in summary, we make the following contributions: ## conclusion this paper suggested a self-supervised objective to learn user-agent interactions leveraging large amounts of unlabeled data available. in addition to the standard fine-tuning approach, this paper presented a novel few-shot transfer learning method based on adjusting the rbcd block selection distribution to favor layer parameters with source and target gradients pointing in similar directions. according to the experiments using real-world data, the proposed approach not only requires significantly less number of annotations, but also generalizes better for unseen out-of-domain skills.