Target-Guided Open-Domain Conversation Planning (2022)

Abstract: Prior studies addressing target-oriented conversational tasks lack a crucial notion that has been intensively studied in the context of goal-oriented artificial intelligence agents, namely, planning. In this study, we propose the task of Target-Guided Open-Domain Conversation Planning (TGCP) task to evaluate whether neural conversational agents have goal-oriented conversation planning abilities. Using the TGCP task, we investigate the conversation planning abilities of existing retrieval models and recent strong generative models. The experimental results reveal the challenges facing current technology.

## introduction neural conversational agents have achieved great successes in recent years, and various methods have been proposed to generate informative responses, e.g., the use of knowledge (zhao et al., 2020; wu et al., 2020), personality (li et al., 2016; zhang et al., 2018), emotional considerations (rashkin et al., 2019; zhong et al., 2020), and large-scale models (zhang et al., 2020; adiwardana et al., 2020; roller et al., 2021; thoppilan et al., 2022). one hot topic in this research area is to develop proactive behavior in agents. for example, tang et al. (2019) proposed the task of target-guided open-domain conversation, in which an agent is required to actively lead a conversation to a predefined target word. wu et al. (2019) proposed a task that uses a knowledge graph to actively lead a conversation to a target entity. several studies have implemented these target-oriented task settings (dai et al., 2019; qin et al., 2020; yuan and an, 2020; zhong et al., 2021; zhu et al., 2021). however, these prior studies all lack planning, a crucial notion that has been intensively studied in the context of goal-oriented artificial intelligence (ai) agents (norvig and russell, 1995; kuijpers and dockx, 1998; stent et al., 2004; walker et al., 𝑢 ! : hi, what do you do for living? 𝑔 ! : book 𝑢 " : i work as an engineer. 2007, etc.) and has also been introduced in neural conversational agents (botea et al., 2019; jiang et al., 2019a,b). in other words, these studies do not explicitly consider the generation of a multiple-step plan to achieve a target. given this background, in this study, we propose the target-guided open-domain conversation planning (henceforth, tgcp) task such that an agent's planning ability in goal-oriented conversations can be assessed. the tgcp task is to produce a plan that leads a conversation to a given target, as illustrated in figure 1 . the point is to consider the task of producing a conversation plan for several utterances ahead, which we first address in the aforementioned context of target-guided open-domain conversation. furthermore, we also propose modeling the planning process by simulating the user's succeeding utterances using the model of the agent itself; namely, the agent converses with itself (i.e., self-conversation) to search for potential conversation paths that achieve the goal. this task setting is not the same as a real-world setting, in which an agent is required to plan a conversation while uncertain of the user's future utterances. however, planning in the self-conversation setting can be considered a prerequisite capability for a planning-aware goal-oriented conversational agent. tgcp works as a framework to evaluate an agents' prerequisite ability for conversation planning without employing human subjects; this can abstract away the hard-to-control human factors from experiments (e.g., some human subjects may not be as cooperative as others). this paper has three major contributions: (1) we propose the tgcp task as a framework to assess the prerequisite ability of a model for goaloriented conversation planning. (2) we conduct a set of experiments on the tgcp framework using several existing retrieval-based neural models and recently proposed strong generative neural models of conversational agents. (3) our experimental results reveal the challenges facing current technology. the evaluation codes and the test set used in the experiments are available. foot_0 2 target-guided open-domain ## conclusion we have proposed the tgcp task as a platform for assessing the conversation planning ability of @xcite we counted the number of turns of the conversations where the target words are mentioned. @xcite a generated example is provided in appendix d.1. a dialogue model. through this task setting, we have presented a first study for assessing the present neural conversational models' abilities for multipleutterance planning, abstracting away the hard-tocontrol potential human factors. while the reported experiments cover only the task of target-guided open-domain conversation (tang et al., 2019), the idea of tgcp is expected to be applicable to a wider range of goal-oriented conversation tasks. using tgcp, we revealed that the dialogue models with current technology have difficulty planning conversations to achieve given goals while ensuring the naturalness of the conversation. the experimental results also showed that refining the subgoal strategies for generative models might be an effective method to overcome this trade-off. we plan to research methods to solve this task setting with higher performance.