Voice Query Auto Completion (2021)

Abstract: Query auto completion (QAC) is the task of predicting a search engine user’s final query from their intermediate, incomplete query. In this paper, we extend QAC to the streaming voice search setting, where automatic speech recognition systems produce intermediate transcriptions as users speak. Naively applying existing methods fails because the intermediate transcriptions often don’t form prefixes or even substrings of the final transcription. To address this issue, we propose to condition QAC approaches on intermediate transcriptions to complete voice queries. We evaluate our models on a speech-enabled smart television with real-life voice search traffic, finding that this ASR-aware conditioning improves the completion quality. Our best method obtains an 18% relative improvement in mean reciprocal rank over previous methods.

## introduction query auto completion (qac) is the task of predicting a user's complete query given the present, incomplete prefix of the query. for example, suppose a user types "covid vaccine" into google. then, a qac system proposes the most likely completions for that prefix, e.g., "covid vaccine near me," saving the user time when the prediction is correct. existing state-of-the-art approaches generate completions using language models conditioned on the prefix @xcite , with simple prefix trees serving as a strong baseline. in the streaming voice search setting, such as on the google voice assistant, naïvely adapting these existing approaches fails because the key assumptions differ. most glaringly, incomplete queries comprise partial speech, not text. in place of human users, an automatic speech recognition (asr) system produces the textual transcripts, resulting in intermediate queries that often don't form prefixes or even substrings of the final query. consider * work done while employed at comcast ai. the voice query "hulu" as an example. passing it through a streaming asr system, we observe the transcripts "who," then "hulu." traditional qac approaches fail to complete "hulu" from "who," since they use orthographic prefixes and substrings @xcite instead of the true phonetic prefix, which is generally unavailable at training time. nevertheless, the intermediate transcript "who" is still informative toward predicting "hulu" because it frequently precedes "hulu" in the sample. based on this observation, we hypothesize that, for improved qac quality, we must additionally model the dynamics between the intermediate and the final transcripts from the asr system. in this paper, we precisely design and evaluate asr-aware qac models. the main contributions of our work are as follows: first, we are the first to describe the task of qac for streaming, bidirectional asr systems in voice search. second, we propose and evaluate novel, asr system-aware qac models for the task, showing that incorporating context from intermediate transcripts helps. on the xfinity x1, a voice-enabled smart tv serving more than twenty million american customers, our best approach attains an 18% relative improvement in mean reciprocal rank over the previous best. ## voice query auto completion our novel task is to predict the final voice queries that users issue, given some mid-utterance, intermediate transcripts of their incomplete speech from the asr system. as is typical, these systems are streaming, with the speech being transcribed to text in real time. concretely, for some utterance, we are given a k-tuple of string transcripts x := (x (1) , x (2) , . . . , x (k) ) representing the streaming outputs of the asr system across the utterance, where x (i) is a string of words. we index x in chronological order, e.g., ("who", "hulu", "hulu now") for the utterance "hulu now." we wish c input output 1 ("a") → ("a [@xmath0] c", "b [@xmath1] c") 2 ("a", "b", "c") → ("a [@xmath2] c", "b [|] c [@xmath3] c") 3 ("a", "b", "c") → ("a [@xmath4] c", "b [|] c [@xmath5] c") 3 ("a", "b", "c", "d") → ("a [@xmath6] for [sep] and [eos]. to predict x (k) from x (j) for each 1 ≤ @xmath7; that is, we model p(x (k) |x (1) , . . . , x (j) ). in keyboard-input qac, there is only a single transcript, the submitted query, which researchers model with prefix trees @xcite and autoregressive language models (park and chiba, 2017) to generate completions from prefixes. a tacit assumption is that partial queries are prefixes (or within a small edit distance of an observed prefix; @xcite of the final query. since this property is invalid for us, we propose voice query-oriented flavors of two state-of-the-art qac approaches, most popular completion (mpc) and neural query language models (nqlms), representing a statistical and a neural approach, respectively. ## conclusions and future work we study the task of qac for voice queries on bidirectional asr systems. along with an improved language modeling objective for query logs, we propose several novel methods which relate the mid-utterance transcripts to the final one, attaining relative gains of 18% over the previous best. voice qac lends itself to a variety of end applications: for one, on voice-controlled smart televisions, it can guide viewers toward final queries in real time, much like the now-retired google instant feature. for another, in general voice query processing pipelines, it can serve as part of a latency reduction method, where the most likely voice queries are speculatively processed and their responses precomputed as the user speaks. if we know what the user is going to say before they finish speaking, then we can speculatively send those predicted final queries to the rest of the information retrieval system, while the user is speaking. we plan to explore these lines of research in future work.