Contextual Rephrase Detection for Reducing Friction in Dialogue Systems (2021)

Abstract: For voice assistants like Alexa, Google Assistant, and Siri, correctly interpreting users’ intentions is of utmost importance. However, users sometimes experience friction with these assistants, caused by errors from different system components or user errors such as slips of the tongue. Users tend to rephrase their queries until they get a satisfactory response. Rephrase detection is used to identify the rephrases and has long been treated as a task with pairwise input, which does not fully utilize the contextual information (e.g. users’ implicit feedback). To this end, we propose a contextual rephrase detection model ContReph to automatically identify rephrases from multi-turn dialogues. We showcase how to leverage the dialogue context and user-agent interaction signals, including the user’s implicit feedback and the time gap between different turns, which can help significantly outperform the pairwise rephrase detection models.

## introduction large-scale conversational ai based dialogue systems like alexa, siri, and google assistant, are getting more and more prevalent in real-world applications to help users across the globe. natural language understanding (nlu) technology is an established component that produces semantic interpretations of a user request. improving the accuracy of the nlu component is a key consideration for satisfactory end-to-end user experience, especially when the nlu component misinterprets the semantics due to ambiguity or errors that come from the previous component (e.g., automatic speech recognition). for instance, the asr system may incorrectly recognize "play jacking the ball" as " play jack in the fall". these errors accumulate and introduce friction in the dialogue figure 1 : difference between the contextual rephrase detection and pairwise approach. pair-wise rephrase detection model computes the similarity score for each pair that appears in the multi-turn dialogue, and selects the maximum score among them for the rephrase prediction. in this case, the pair-wise model without considering context information incorrectly predicts "play tyler hero explicit by jack harlow" as the rephrase of user's defective request "play tyler hero explicit" since it has highest similarity score. previous works @xcite @xcite focus on friction reduction in the asr and nlu components using query rewriting (qr) @xcite . these approaches reformulate the asr transcription of user's query, such that it conveys the same meaning/intent, to minimize user dissatisfaction. an important aspect of the qr approaches is to detect user rephrase of a previous query that leads to a satisfactory response. however, these approaches focus only on the pairwise semantic similarity of queries, which does not consider the corresponding user feedback, with proper dialogue context. as shown in fig. 1 , dissatisfied users might provide implicit feedback, i.e., they rephrase the previous query (e.g. the first user request "play tyler hero explicit" in the left of fig. 1 ) multiple times unless the agent does the needful. if we only consider the semantic similarity between different queries from pairwise-based models, the system may choose an unsuitable query "play tyler hero explicit by jack harlow" to correct the problematic request. the dialogue context includes additional information like previous turns, the responses of the dialogue agent, and time differences between user queries. by leveraging this context, we can detect the correct rephrase -"play tyler hero by jack harlow", with a much higher probability. in this paper, we propose an automatic user rephrase detection approach contreph, which leverages implicit user feedback and dialogue context in a multi-turn dialogue setting. contreph detects if any of the user queries in the dialogue session is rephrased, and then extracts the most probable rephrase span that led to a satisfactory response from the dialogue agent. specifically, we input the full dialogue session to the model, including agent's responses and capture time gaps between user queries using a novel time-difference encoding scheme. we evaluate the performance of our proposed framework by conducting an extensive set of experiments on production data of a large scale dialogue agent and showcase the effectiveness of our approach against existing methods. although, in this work, we focus on rephrase detection only, the rephrases identified by our approach can directly be used as query rewrites for reducing friction in dialogue systems. ## conclusion and future work in this paper, we presented a novel approach for detecting user rephrases in multi-turn dialogue systems. users tend to rephrase their queries until they get the desired response from ai agents. our system can detect these rephrases with a high accuracy using the dialogue context and significantly outperforms other approaches that consider queries in a pair-wise manner only. the output of our model is a crucial step towards building self-learning mechanisms in dialogue agents to fix issues with minimal human intervention. for future work, we plan to leverage contrastive learning strategies as a post-training step, which could help us obtain better query representations, before we do fine-tuning for rephrase detection. we also want to deploy the detected rephrases as query rewrites to gauge how much we can improve the ux of a real world dialogue system.