SlowGuess/NAACL_2022_OCR huggingface dataset:
Prakhar Gupta\* Harsh Jhamtani\* Jeffrey P. Bigham\*,
$\clubsuit$ Language Technologies Institute, Carnegie Mellon University  $\diamond$ Human-Computer Interaction Institute, Carnegie Mellon University
prakharg@cs.cmu.edu, jharsh@alumni.cmu.edu, jbigham@cs.cmu.edu
Target-guided response generation enables dialogue systems to smoothly transition a conversation from a dialogue context toward a target sentence. Such control is useful for designing dialogue systems that direct a conversation toward specific goals, such as creating nonobtrusive recommendations or introducing new topics in the conversation. In this paper, we introduce a new technique for target-guided response generation, which first finds a bridging path of commonsense knowledge concepts between the source and the target, and then uses the identified bridging path to generate transition responses. Additionally, we propose techniques to re-purpose existing dialogue datasets for target-guided generation. Experiments reveal that the proposed techniques outperform various baselines on this task. Finally, we observe that the existing automated metrics for this task correlate poorly with human judgement ratings. We propose a novel evaluation metric that we demonstrate is more reliable for target-guided response evaluation. Our work generally enables dialogue system designers to exercise more control over the conversations that their systems produce.<sup>1</sup>
Open-domain conversational systems have made significant progress in generating good quality responses driven by strong pre-trained language models (Radford et al., 2019; Devlin et al., 2019) and large-scale corpora available for training such models. However, instead of passively responding to a user, dialogue systems can take on a more proactive role to make recommendations, help users discover new services, or introduce interesting new topics to users to improve user experience. Furthermore, a proactive or target-guided system can guide the conversation towards safer conversational topics in
Figure 1: Given a dialogue context and a target sentence, our goal is to generate a dialogue response that smoothly transitions the conversation from context towards the target. Our proposed approach involves identifying a bridging path of entities to link the context and the target.
case a conversation goes awry or a user becomes abusive towards the system, and direct the users towards topic areas that the system knows how to talk about. Prior work has used mechanisms such as emotion labels (Zhong et al., 2019), persona (Song et al., 2019), and politeness (Niu and Bansal, 2018) to control conversations. However, such approaches require labeled training data for a set of pre-determined labels, making it harder to incorporate new goals into a system. In this work, we study the problem of proactive response generation based on a target sentence. For example in Figure 1, given the context 'I enjoy swimming', the system guides the conversation towards the target 'I like to travel to new places' by mentioning 'I like to swim at beaches when I go on vacation'. Using target sentences for proactive control is an intuitive and flexible control mechanism for dialogue developers, free of domain-specific handcrafting and annotations.
Existing publicly available dialogue corpora generally consists of free-flow conversations where the speakers move the conversation forward based on the dialogue history alone, with no particular agenda. We build upon the recently released Otters
1Code available at www.github.com/prakhargupt az/target-guided-dialogue-coda
Findings of the Association for Computational Linguistics: NAACL 2022, pages 1301 - 1317 July 10-15, 2022 ©2022 Association for Computational Linguistics
dataset (Sevegnani et al., 2021) with one-turn topic transitions for mixed-initiative in open-domain conversations. Given a source sentence from a speaker, the task is to generate a topic transition sentence with "bridging" strategies to a target sentence from another speaker. The task is challenging on several fronts. First, the system needs to balance the trade-off between coherence with the context while smoothly transitioning towards the target. Second, the Otters training dataset is relatively small (less than 2000 training instances), making it a low-resource setting. Finally, we show that standard word-overlap metrics are insufficient for this task.
In this work, we propose methods to leverage commonsense knowledge from ConceptNet (Speer et al., 2017a) to improve the quality of transition responses. Our technique decomposes the response generation process into first generating explicit commonsense paths between the source and target concepts, followed by conditioning on the generated paths for the response generation. This is intended to mimic how humans might bridge concepts for creating transitions in conversations using commonsense knowledge. This technique offers two benefits: 1) Leveraging external ConceptNet knowledge solves the data scarcity issue and improves the model's capability to generate logical transitions; 2) Since the transition response is grounded on commonsense knowledge paths, the explicit paths used by the model can provide explanations for the concepts used by the model, as well as provide control over the generation process. Furthermore, we propose a data augmentation mechanism to help with the data scarcity issue by re-purposing training data from DailyDialog, an open-domain dialogue dataset. Both these approaches are complementary and outperform existing baselines in response quality and transition smoothness. We demonstrate how the proposed approach of using explicit bridging paths enables improved quality of transitions through qualitative and human studies.
Automated evaluation is a challenging aspect of dialogue response generation tasks (Zhao et al., 2017). We show that the existing word-overlap metrics such as BLEU can be easily fooled to assign high scores to poor responses just based on high n-gram overlap with reference responses. We propose a metric TARGET-COHERENCE which is trained using hard adversarial negative instances and achieves a high correlation with human judge
ment ratings of system outputs. As part of this work, we collect and release a dataset of human ratings of various system outputs for this task.
We discuss the broader impact and potential uses of the proposed system, its limitations and potential ethical issues related to this task in Section 8.
Target Guided Dialogue Response Generation:
Sevegnani et al. (2021) is perhaps the closest to our work described in this paper. They work on the task of generating a new utterance which can achieve a smooth transition between the previous turn's topic and the given target topic. Past work in controllable text generation has explored steering neural text generation model outputs to contain a specific keyword (Keskar et al., 2019), a knowledge graph (Wu et al., 2019), or a topic (Ling et al., 2021). Steering dialogue towards a given keyword has also been explored in past work (Tang et al., 2019; Qin et al., 2020a; Zhong et al., 2021), albeit as a retrieval task. In contrast, our goal is to generate a next utterance in a dialogue setup which can steer a conversation towards target sentence in a smooth fashion rather than generating a response for a given keyword or topic. Our work is also related to prior work on text infilling (Donahue et al., 2020; Qin et al., 2020b), though compared to them we work in a dialogue setup and utilize commonsense knowledge to perform the infilling.
Commonsense for Dialogue Generation: Commonsense knowledge resources (Speer et al., 2017b; Malaviya et al., 2020) have been used in dialogue response generation for tasks such as person-grounded dialogue (Majumder et al., 2020) and open-domain dialogue generation (Ghazvininejad et al., 2018; Hedayatnia et al., 2020; Zhou et al., 2021c). Zhou et al. (2021a) created a dataset focusing on social commonsense inferences in dialogue and Arabshahi et al. (2020) designed a theorem prover for if-then-because reasoning. A concurrent work (Zhou et al., 2021b) proposed to train a model to explicitly generate implicit knowledge and use this knowledge to generate a response. Compared to their work, we focus on target-guided response generation, suggest mechanism for knowledge alignment with the transition response during training, and focus on multi-hop knowledge paths. More broadly, commonsense knowledge has been used in text generation tasks such as story and essay generation (Guan et al., 2019a; Yang et al., 2019).
Automated Metrics for Evaluating Dialogue Quality: Automated metrics such as BLEU (Papineni et al., 2002), METEOR (Banerjee and Lavie, 2005), and BertScore (Zhang et al., 2020) are widely used to evaluate quality of machine-generated text. However, such metrics often correlate poorly with human judgement ratings of generated text quality (Sai et al., 2020). Past work has explored trained model-based metrics such as ADEM (Lowe et al., 2017) and RUBER (Tao et al., 2017). However, training such model-based metrics often relies on tagged training data. Gupta et al. (2021a) propose ways to mitigate the need for such labelled data by automatically synthesizing negative examples. Our proposed metric is along similar lines, though we utilize different techniques for synthetic negative example generation.
3 Task Overview
We first formalize the task of target-guided response generation. Given a conversation context  $c$  between two speakers A and B, and a target utterance  $t$  for speaker B, the task is to generate a transition sentence  $s$  which serves as a smooth link between the context and the target. The target is a phrase or a sentence. Otters dataset (Sevegnani et al., 2021) consists of a simplified setting of one-turn topic transitions, where the conversation history consists of a single utterance  $u_{a}$  from speaker A, and a target utterance  $u_{b}$  for speaker B, and the task is to generate a transition utterance  $s$  for speaker B to serve as a smooth link between  $u_{a}$  and  $u_{b}$ . The task is challenging since a system needs to devise a strategy that balances the competitive objectives of generating a response which is coherent to the context, while smoothly driving the conversation towards the target.
In this work, we propose two approaches for the transition response generation task: 1) Commonsense-guided response generation (section 4), and 2) Data augmentation to tackle data sparsity (section 5). We refer to the proposed method as CODA (Commonsense Path and Data Augmentation). We also propose a novel metric TARGET-COHERENCE to automatically evaluate the smoothness of response transitions (section 6).
4 Commonsense-Guided Response Generation
We frame the target-guided response generation task as follows. Given a conversation context  $c$
and a target  $t$ , a conditional language model learns to predict the transition response  $s$ . Target-guided generation can potentially benefit by incorporating commonsense reasoning by identifying rich connections between a pair of entities which enable us to generate logical transition responses connecting the two. Pre-trained language models are known to suffer in cases where commonsense knowledge is required during generation (Zhou et al., 2018; Guan et al., 2019b), especially in tasks where there is not enough data available for learning commonsense patterns from the text, which is true for our case. In contrast, Commonsense Knowledge Graphs like ConceptNet (Speer et al., 2017a) provide structured knowledge about entities, which enables higher-level reasoning about concepts.
In this work we use commonsense knowledge from ConceptNet for planning a transition response. ConceptNet is a large-scale semantic graph that has concepts as nodes and has commonsense relationships between them, such as 'IsA' and 'At-Location'. However, ConceptNet suffers from severe sparsity issues (Malaviya et al., 2020; Bosselut et al., 2019). Therefore, it is not always possible to find the concepts and relationships between context and target concepts. To address the sparsity issue, we develop Knowledge Path Generator (KPG), a language model trained on paths sampled from ConceptNet. The model takes a pair of entities or concepts as input and generates a multi-hop path connecting the two. Since the knowledge paths are sampled from a generative model rather than retrieved from a fixed knowledge base, we are no longer limited by the entities and paths present in the ConceptNet knowledge base.
To generate commonsense based responses, we train a Commonsense Response Generator (CRG) model to generate the transition response conditioned on the paths generated by the KPG model (Figure 2). Conditioning the response generation on commonsense paths improves the reasoning capabilities of the CRG model and provides the added benefits of interpretability and control over the generation process.
4.1 Commonsense path generator
The KPG models attempts to connect a concept or entity phrase from the context to a concept from the target by creating knowledge paths between them.
Path Sampling: To create training data for the KPG models, we sample paths between entity
Figure 2: Model illustrations for KPGs - Knowledge Path Generators (top) and CRG - Commonsense Response Generator (bottom). Base architecture for all models is GPT-2. Given a path sampled from ConceptNet, KPG-wc learns to predict the path given the head, tail and intermediate entities of the path while KPG-ht learns to predict the path given only the head and tail entities. For the CRG model, during training, a head entity from the context, a tail entity from the target and intermediate entities from the gold transition response are fed into KPG-wc and its output path is used as input to the CRG model. During inference, a head entity from the context and a tail entity from the target are fed into the KPG-ht model. KPG-ht then generates a path with new concepts such as "go on vacation". CRG model conditions on this path for transition response generation.
phrases from ConceptNet using random walks. This step builds upon past work of Wang et al. (2020). Given nodes  $N$  and edges  $E$  from ConceptNet, we perform random walks on the graph to sample a set of paths  $P$  of the form  $p = \{n_0, e_0, n_1, e_1, \dots, e_{k-1}, n_k\} \in P$ . Here, a path  $p$  connects a head entity phrase  $n_0$  with the tail entity phrase  $n_k$  via intermediate entities and edges (or relations)  $n_i, e_i$ . To sample paths, the random walk begins with a random entity node  $n_0$  and samples a path of random length  $k \in \{1, 2, \dots, K\}$ , where we have set  $K = 6$  in this work. To sample paths that are useful for our task, we prevent sampling certain edges types such as Synonym (Appendix A.1).
KPG-head-tails (KPG-ht): KPG-ht is a GPT-2 (Radford et al., 2019) based model which is trained to predict a knowledge path  $p$  which links a head entity  $n_h$  to a tail entity  $n_t$ . For a sample path  $p = \{n_h, e_0, n_1, e_1, \dots, e_{k-1}, n_t\}$  from ConceptNet, the path is formatted into the following sequence "[target]  $n_t$  [sep]  $n_h$ $e_0$ $n_1$ $e_1, \dots, e_{k-1}$ $n_t$ ". KPG-ht is only used during CRG inference where the head entity is extracted from the context and tail entity from the target (Figure 2).
KPG-will-containing (KPG-wc): A large number of
possible paths can exist for a given head-tail entity pair. Training the CRG model by conditioning on paths which are irrelevant to the gold transition response might discourage the CRG model from conditioning on the provided commonsense path. Since we do not have gold paths for a response, we instead train a model KPG-wc to generate paths which are more aligned to the gold response by enforcing the generated path to contain entities from the gold response. KPG-wc is trained to predict a path which contains a pre-specified entity set  $E_{p} = \{k_{1},\dots,k_{n}\}$  in the generated path by formatting paths sampled from ConceptNet as the following sequence: "[wc]  $k1$  [wc]  $k2\dots$  [target]  $n_t$  [sep]  $n_h$ $e_0$ $n_1$ $e_1$ , ...,  $e_{k - 1}$ $n_t$ " (Figure 2). The entity set  $E_{p}$  is a randomly permuted sequence of entities  $n_1, n_2, \ldots, n_{k - 1}$  from the sampled path. Here "wc" symbolizes "will contain". Training with this sequence indicates to the model that the path generated between  $n_h$  and  $n_t$  should contain the entities from the set  $E_{p}$  in a sensible order. Specifying the special token "[target]" followed by the tail entity  $n_t$  informs the model about the last entity it should output when generating a path. We discuss how the set  $E_{p}$  is constructed for CRG model training
in the next section.
In practice, we train a single common GPT-2 based model for KPG-wc and KPT-ht. The model at test time is able to generate knowledge paths for either case, whether in-path entities from  $E_{p}$  are present (KPG-wc) in the input or not (KPG-ht).
4.2 Response generator
The Commonsense response generator conditions on the commonsense paths generated from the KPG models to generate the transition responses.
Entity extraction. We extract a set of entities  $E_{h}, E_{t}$  and  $E_{r}$  from the context, target and gold transition response respectively using NLTK. We designed simple grammar rules (details in Appendix A.1) to convert phrases to concise forms that match the nodes present in ConceptNet, e.g., "watching the star" is converted to "watch stars".
Sampling and filtering paths: In this step, for every pair of head and tail entity from  $E_{h}$  and  $E_{t}$ , we sample multiple paths from the KGP models using topk sampling and chose one or more of these paths for training and inference. For training the CRG models with the commonsense paths, we need to curate paths that are relevant to and aligned with the gold response so that they are not ignored by the CRG model during inference. We achieve this by first sampling paths which are relevant to the gold response, and then apply filtering mechanisms to curate the final set of paths. For training data path sampling, we use the KPG-wc model (Figure 2). The input to the model is a head and tail entity pair  $n_{h}$  and  $n_{t}$ , and the entity set  $E_{p}$  that consists of the set of entities  $E_{r}$  from the gold transition response. The model then generates a set of paths that contain the head and tail entities as well as the gold response keywords. Thus, the sampled path is inherently relevant to the gold response due to the conditioning on gold keyword entities. During inference, the set  $E_{r}$  is not available, so we leverage the KPG-ht model that takes just the head and tail entity pair  $n_{h}$  and  $n_{t}$  as input to generate a commonsense path.
Assuming the context and target consists of  $m$  and  $n$  entities each, and we generate  $q$  number of paths per pair, we get a total of  $m \times n \times q$  number of paths for each data instance. Since  $m \times n \times q$  can be a large number, we use simple methods to sub-select entity pairs and paths. (1) Sub-selecting Entity Pairs: We score an entity pair by calculating the inverse document frequencies (computed using
Gutenberg English corpus) of the entity tokens and summing up the maximum value found for a token in each entity in the pair. For training phase, we keep the top D pairs of entities, and for testing phase we keep only the highest-scoring pair. (2) Sub-selecting paths: We apply the following strategies to prune the set of paths for each entity pair: 1) Perplexity - We filter out all the paths whose perplexity values (from the KGP models) are more than double the average perplexity values of all paths between an entity pair. 2) We remove all the paths which have repetition of entities since repetition often leads to degeneration during decoding. 3) For paths in training data, we filter out paths which contain entities not present in the gold response. The final set of paths  $P$  are converted into natural language by converting the relation and inverse relations into textual format. For example, "art gallery UsedFor for art" is converted to "art gallery is used for art".
Training and inference in CRG model. The CRG model (GPT-2 based) is trained as a conditional model with the following input sequence: "knowledge path [target] target sentence [context] context sentence [response] transition response" for each knowledge path from the set  $P$ . We train the CRG model by minimizing the log-likelihood loss of the transition response. For inference, we create the set of paths  $P$  by entity extraction, path sampling and filtering and choose a random path  $p$  from the final set  $P$ . The model generates the transition response conditioned on the sequence of  $c, t,$  and  $p$ .
5 Data Augmentation
The task of target-guided response generation is still a relatively unexplored task, and Otters (Sevegnani et al., 2021) is the only suitable dataset for this task to the best of our knowledge. However, Otters is small and consists of only a few hundred context-target pairs. This makes learning transition concepts and strategies challenging in this low-resource setup. On the other hand, there are many publicly available dialogue datasets for training response generation models. Such datasets contain free-flow conversations, where although the speakers generate context coherent responses, they do not condition their responses on any target. We propose a technique to leverage and re-purpose such datasets for the task of target-guided response generation. We pick the DailyDialog (Li et al., 2017) dataset for experimentation and convert its conver
Context	the restaurant looks authentic european.
Response	the chef trained in florence. the pasta tastes nice here.
SRL Output	predicate = tastes, arguments= the pasta; nice here
Target clause	the pasta tastes nice here.
sations to target-guided conversations in two steps: 1) Target creation, and 2) Data filtering.
For target creation, we run Semantic Role Labelling (SRL) to predict predicate and arguments in a response. For each predicate identified, we create a clause by putting together the predicate and arguments in a textual sequence. Finally, we only use the clause occurring towards the end of the response as a target. An example for target creation is shown in Figure 3 (More details about clause identification are in Appendix A.2).
The target creation step does not guarantee that a candidate response transitions smoothly towards the target clause. In the data filtering step, we introduce a TARGET-COHERENCE metric to score a transition response in terms of its coherence to the context and smoothness towards the target. The metric is described in more detail in section 6. The metric assigns a score between 0-1 for a transition response and we remove instances with a score less than a threshold  $k$  (set to 0.7) from consideration. The remaining instances are used for pretraining response generation models which are finally finetuned on the Otters dataset.
6 Target-Coherence Metric
Evaluating target-guided responses is a challenging task as a good transition response needs to be both - coherent to the context and smoothly transition towards the target. Furthermore, since the task is open-domain and open-ended, there are many possible correct responses which may not match with a reference response (Celikyilmaz et al., 2020). To tackle these challenges, we propose an automatic metric for this task that does not use human references. The proposed metric TARGET-COHERENCE is based on a classification model trained to classify a transition response as either positive, that is, it is coherent to the context and smoothly transitions towards the target, or negative, that is, the response is either not coherent to the context or does not transition towards the target.