A Textual Dataset for Situated Proactive Response Selection (2023)

Abstract: Recent data-driven conversational models are able to return fluent, consistent, and informative responses to many kinds of requests and utterances in task-oriented scenarios. However, these responses are typically limited to just the immediate local topic instead of being wider-ranging and proactively taking the conversation further, for example making suggestions to help customers achieve their goals. This inadequacy reflects a lack of understanding of the interlocutor’s situation and implicit goal. To address the problem, we introduce a task of proactive response selection based on situational information. We present a manually-curated dataset of 1.7k English conversation examples that include situational background information plus for each conversation a set of responses, only some of which are acceptable in the situation. A responsive and informed conversation system should select the appropriate responses and avoid inappropriate ones; doing so demonstrates the ability to adequately understand the initiating request and situation. Our benchmark experiments show that this is not an easy task even for strong neural models, offering opportunities for future research.

## introduction conversational assistant systems have recently shown significant improvements for understanding users' inquiries along with background knowledge, conducting requested operations, and returning natural language responses. yet, typical systems are likely to be passive and only process user-initiated requests or merely ask values for domain-specific slots @xcite . in contrast, human assistants like hotel concierges are more proactive, acting to address unmentioned needs and expected future events @xcite . they do not only make the room temperature is hot. ## task and data the goal of this study is to provide a resource for developing a system that can observe situational information and return a proactive response to a user. we consider six categories of observable situational statements ( figure 2: pipeline for data collection. we start with existing common-sense knowledge bases (atomic and conceptnet) and extract utterance and goal events as a seed (1). we collect responses and situational statements for each seed by crowdsourcing (2-5), acquire more situational statements semi-automatically (6), and select distractor responses and situations to form response selection examples (7). we finally validate the examples manually (8). steps ( 2 ) and (3-4) are executed in parallel. ## experiments we evaluate several baseline models on sugar to explore two questions concerned with the nature of the proposed task and dataset: (1) is understanding of situational information required to identify proactive responses in sugar? (2) can standard matching-based systems capture relevant situational information and solve the task? ## conclusion and future work we proposed a task of situated proactive response selection for developing and evaluating conversational assistants that can help users proactively in various help-seeking scenarios. we constructed a dataset of 1.7k examples by crowdsourcing and semi-automatic generation. there are several interesting directions for future research. first, as shown in our experiments, it is challenging to pick up relevant situational information and use it to reason about user requests and potential assistance. to achieve this, conversational systems will need to be equipped with world knowledge to effectively align situation information with an interaction. one promising approach is knowledge-based response models such as graph neural networks, which recently has shown to be effective in various nlp tasks @xcite @xcite . second, although we leveraged implicit goals only for soliciting proactive responses in data collection in this study, understanding of goals should be necessary for building better conversation engines as claimed in early studies @xcite ; inter alia). we believe sugar can facilitate future research in this direction.