Follow-on Question Suggestion via Voice Hints for Voice Assistants (2023)

Abstract: The adoption of voice assistants like Alexa or Siri has grown rapidly, allowing users to instantly access information via voice search. Query suggestion is a standard feature of screen-based search experiences, allowing users to explore additional topics. However, this is not trivial to implement in voice-based settings. To enable this, we tackle the novel task of suggesting questions with compact and natural voice hints to allow users to ask follow-up questions. We define the task, ground it in syntactic theory and outline linguistic desiderata for spoken hints. We propose baselines and an approach using sequence-to-sequence Transformers to generate spoken hints from a list of questions. Using a new dataset of 6681 input questions and human written hints, we evaluated the models with automatic metrics and human evaluation. Results show that a naive approach of concatenating suggested questions creates poor voice hints. Our approach, which applies a linguistically-motivated pretraining task was strongly preferred by humans for producing the most natural hints.

## introduction voice assistants, like alexa or google assistant provide ubiquitous services through a variety of devices (e.g. smart speakers, phones, tvs, etc.). users interact with voice assistants for different purposes @xcite such as question answering, e-commerce, or entertainment. with increasing adoption, user expectations also grow and related content recommendation is a valued feature @xcite . the question of how to present proactive suggestions is an open one, and recent work has examined how content such as news articles can be recommended over voice @xcite . query and question recommendation (see fig. and are well integrated in screen-based web search experiences (i.e., those from google/bing). however, such functionality does not exist for voicebased systems. suggestions enable highly useful exploratory search capabilities, and we aim to provide a similar experience over voice (see fig. 1 (b) ), where through a follow-on hint we suggest related topics they can ask about. contrary to suggestions on web search, integrating recommendations in voice assistants poses unique challenges @xcite , such as (i) modality: voice lacks the advantages of visual interfaces used on the web (e.g., showing a list), (ii) transmitted information: to ensure comprehension, the amount of transmitted information in an utterance is limited in terms of time and number of words, and (iii) shape: simply reading out a list of questions is not natural over voice. we propose a new approach on how to deliver voice-based question suggestions using hints. we do not consider what to suggest as this is widely explored in existing work. figure 1 provides an overview. for an input question, we assume the voice assistants can retrieve related questions 1 from which a suggestion hint is generated. differently from questions recommendation in web search (fig. 1 (a) ) where new related questions are listed, we aim to synthesize a natural utterance (fig. 1 (b )) suggesting the same questions. the hint does not contain questions, rather, it contains several subordinate clauses describing facts or knowledge that the user can ask about. our overarching contribution is a framework for generating voice-friendly hints. we begin with a grounded linguistic description of the task, outlining the characteristics of a good hint (e.g., cohesion, length), and the syntactic transformations needed to construct such utterances. next, we frame the task as a seq2seq approach @xcite , where for an input question and its top-3 related questions, covering a diverse set of topics (unrelated topics to the initial question's topic), a voice hint is synthesized to meet the desiderata in we create a dataset of voice-friendly hints, consisting of the triple: initial question, related questions, follow-on hint, in 9 different domains. we evaluate hint generation on our dataset by means of automated metrics and human evaluation studies. to summarize, our contributions are: ## linguistic task and background to generate a spoken hint, our objective is to take a set of standalone questions (interrogative sentences), and convert them into a single sentence that informs the listener about the different pieces of information available. figure 2 shows the overview of the linguistic tasks that are needed to be performed in order for a set of input questions to generate a voice-friendly hint. direct questions ("can a dog eat peanuts?") can be presented as an indirect question ("alice asked if dogs can eat peanuts.") @xcite . all direct questions can have an indirect equivalent, and the embedded clause of the indirect version is said to refer to the direct question @xcite . while both direct and indirect questions can be used to ask, when an indirect question's main clause reports information (e.g. "i know . . ."), their pragmatic purpose is to provide information (puigdollers, 1999). our task requires transforming independent questions into subordinate clauses, and then embedding them into a new sentence whose main verb is one of cognition or reporting, and takes the clauses as direct objects (appendix a). the most interesting syntactic transformation is that of converting a question to a dependent clause. in english, this can be done using content clauses (also known as noun clauses), which describe the inquired information in a main clause. the contents of a question can be framed as an interrogative content clause which represents the knowledge or entity that is being interrogated in the question. the syntactic transformations needed to construct the content clause vary depending on the question type and its complexity. in general, these are the same changes used to generate reported or indirect speech, and can include subject-auxiliary inversion, changes in tense, and other lexical substitutions. this resulting subordinate clause is a syntactic unit which can be used as a direct object in a declarative sentence. multiple subordinates can be combined to compose a single sentence. since these transformations between direct and reported speech are commonly used in english, representing our questions this way sounds very natural, and allows listeners to effortlessly convert any of the clauses into a fully formed question. ## vofh -voice-friendly question suggestion via hints dataset we now describe the process of generating a new voice-friendly hints dataset. 4 we first construct tuples of input questions and related questions ⟨q, q rel ⟩. we then annotate spoken hints for each tuple, creating a dataset of 6, 681 samples composed of the triples @xmath0, q rel , h⟩ i . . .}. the input question q and related questions q rel datasets are described in appendix b. ## experimental setup we evaluate different models and assess hint quality using automatic and human evaluation metrics. ## evaluation on automated metrics baseline performance: tb achieves the lowest scores across all metrics (except for bertscore). this is expected, since concatenated questions are compared w.r.t the ground-truth hints, written by annotators. rsb obtains a consistent improvement across all metrics. it rewrites individual questions into content clauses, which then are concatenated using the conjunction "or". however, rsb does not reduce lexical repetition via anaphora, and simple concatenation results in lower coherence. overall tb and rsb, achieve low scores as expected. more insights are provided by the human evaluation studies, which capture hint voice friendliness. approach performance: our approaches, dhg and ptg, show a consistent improvement over tb and rsb across all automated metrics. this is intuitive since they are optimized to generate hints. comparing ptg and dhg in the difference in performance between ptg and dhg, demonstrates that for complex rewrit-ing tasks, end-to-end training may be sub-optimal. decomposing the problem into specific pretraining subtasks before fine-tuning in an end-to-end manner yields significant improvements. similar finding are reported in @xcite . for rouge metrics, only ptg-t5 obtains significantly better results than dhg-t5 for rouge1. for the rest, although ptg has higher rouge scores, the differences are not significant. finally, for bertscore the differences are significant between ptg-bart over dhg-bart. robustness: ## conclusions we presented a novel approach for question suggestion using spoken hints. our work enables the creation of new voice-based experiences where users can receive compact and natural hints about additional questions they can ask. question suggestion is a standard feature in screen-based search experiences, and our work takes an important first step in bringing this capability to voice interfaces. our contributions are manifold: (i) a novel task of suggesting questions with voice hints; (ii) outlined the linguistic desiderata and processes to decompose questions into interrogative content clauses, and recompose them into declarative hints; and (iii) a new dataset of over 14k input questions and hints, using carefully constructed annotation guidelines and quality checks. we defined seq2seq models to generate hints. using both automatic metrics and human evaluations, we conclusively showed that our most sophisticated approach ptg, which utilizes a linguistically motivated pretraining task was strongly preferred by humans with most natural hints.