A Systematic Evaluation of Response Selection for Open Domain Dialogue (2022)

Abstract: Recent progress on neural approaches for language processing has triggered a resurgence of interest on building intelligent open-domain chatbots. However, even the state-of-the-art neural chatbots cannot produce satisfying responses for every turn in a dialog. A practical solution is to generate multiple response candidates for the same context, and then perform response ranking/selection to determine which candidate is the best. Previous work in response selection typically trains response rankers using synthetic data that is formed from existing dialogs by using a ground truth response as the single appropriate response and constructing inappropriate responses via random selection or using adversarial methods. In this work, we curated a dataset where responses from multiple response generators produced for the same dialog context are manually annotated as appropriate (positive) and inappropriate (negative). We argue that such training data better matches the actual use case examples, enabling the models to learn to rank responses effectively. With this new dataset, we conduct a systematic evaluation of state-of-the-art methods for response selection, and demonstrate that both strategies of using multiple positive candidates and using manually verified hard negative candidates can bring in significant performance improvement in comparison to using the adversarial training data, e.g., increase of 3% and 13% in Recall@1 score, respectively.

## introduction building an open-domain dialog system to interact with users on a variety of topics can involve building multiple response generators (rg) with different functions @xcite . these rgs can be a mixture of generative, retrieval and template based methods. a response selector is then built to re-rank response candidates produced by different applicable rgs to determine the best response for a given turn. these response selectors are based on either rule-based or model-based architectures @xcite @xcite . rule-based systems typically consist of manuallydesigned logic to rank hypotheses, whereas modelbased approaches can either be conventional machine learning models or recent neural models that learn to rank candidates. as the number of rgs grows, a rule-based system can become cumbersome to maintain, whereas model-based methods can simplify the selection process as well as achieve better performance. latest work in model-based response selectors involves leveraging pretrained transformer models such as bert @xcite and di-alogpt @xcite . these selection models are often trained using existing dialog datasets that typically contain ground truth responses. thus a focus of past response selection work is on the construction of inappropriate/negative responses, using methods such as random selection, utterance manipulation or leveraging user feedback @xcite @xcite @xcite @xcite @xcite . however, such synthesized datasets for response selection have the following known drawbacks. first of all, their claimed incorrect responses are not verified if they are actually incorrect. second, these negative responses are easy to differentiate from positive ones since it is very likely that they will be on different topics from the context. therefore, models trained on such easy negative responses will not be able to generalize to real-world settings, where multiple responses are generated given the same dialog context and many of them are strong candidates. to resolve the aformentioned issues, we construct a new dataset (named rsd) for response selection by showing human annotators multiple response candidates produced by different rgs for a given turn and dialog context, and asking them to annotate all responses that are appropriate for that specific dialog context. we leverage rsd to conduct a systematic evaluation of state-of-the-art methods for response selection, including existing trained models, dialogrpt @xcite and bert-fp @xcite , and a bert based ranker that we trained. our experimental results show the following findings: (1) models trained on rsd significantly outperform those trained on existing datasets, e.g., reddit and ubuntu, showing the benefit of bringing in human annotated data for this task; (2) using manually verified hard negatives greatly outperforms using adversarial negatives; (3) training on multiple positive candidates improves performance in comparison to a single positive candidate. though these findings are most expected, this is the first empirical study that clearly shows that constructing a more realistic dataset benefits strongly over generating synthetic examples for response selection, and we hope such results can guide future research in this direction and deployment of open domain dialog systems. ## related work previous work in response selection has been conducted in different domains, such as chatlogs @xcite , e-commerce @xcite , and open-domain dialog @xcite @xcite . our work focuses on opendomain dialog, where current systems typically consist of multiple response generators, each of which is designed to deal with a certain domain. for example, in the alexa prize challenge @xcite , most of the participating socialbots built by university teams consist of a variety of responders that are based on retrievalbased methods, template-based methods, or generative models @xcite @xcite . in order to select the final response to present to users, both rule-based or model-based ranking models have been proposed @xcite @xcite @xcite . this approach is also common in other real-world systems such as xiaoice that employs a manually-designed set of features to rank hypotheses @xcite . for training response selection models, typically human-human dialogs are used, where positive examples are the ground truth responses and negative responses are often randomly selected or synthetically created since there are no labeled negative responses. han et al. (2021) randomly selected responses from other dialogs or within the same dialog session. whang et al. (2021) corrupted utterances by inserting, substituting and deleting random tokens. xu et al. (2020) masked and shuffled utterances within a dialog. li et al. (2019) selected negative responses from a batch based on their similarity scores from the positive response score. gupta et al. (2021) used automatic methods such as replacing random tokens in a positive examples using a mask-and-fill approach to create adversarial negative examples. however, these sampling strategies do not ensure the selected negative responses are hard examples. in this work, rather than relying on approximation for negative responses, we perform turn level annotation of multiple response candidates for response appropriateness for a given dialog context. see and manning (2021b); gao et al. (2020) did construct hard negative examples by annotating responses from a single generative model for appropriateness; however, our work contains responses from a mixture of various rg methods. on the other hand, open-domain dialogs can have multiple appropriate responses for a given dialog context. previous work has augmented dialog datasets with multiple positive examples @xcite @xcite @xcite . within open-domain dialogs, @xcite ; sai et al. (2020) augmented the dai-lydialog dataset @xcite with multiple positive human written responses. in contrast, our dataset has multiple positive responses generated from models, which reduces the cost of human annotation significantly. the closest work to ours is (sai et al., 2020) that constructed negative examples by asking annotators to copy information from the dialog context. we do not restrict the definition of negative examples to be copying information from the dialog context, since incorrect responses in open-domain dialog can have different issues, e.g., off-topic, contradicting or repetitive responses. ## datasets as described earlier, most previous work in response selection has constructed test sets that typically contain only one positive candidate and one or more synthetically created negative candidates. however, such negative responses may be easy for a model to detect. additionally, in real-world open-domain dialogs there can be more than one positive response per turn. therefore, in this work we constructed a more realistic dataset consisting of annotations for real response candidates. our dataset consists of spoken interactions between a dialog system and real users. ## response selection models we have adopted two state-of-the-art methods for response selection and adapted them to our new dataset for a comprehensive empirical evaluation. ## conclusion in this work, we have curated a new dataset for response selection, which contains multiple positive responses and human verified hard negatives. we conducted a comprehensive evaluation of sota response selection models and various techniques to construct negative candidates to demonstrate the benefit of the dataset. even though rsd requires manual annotation we see that training on our dataset greatly outperforms methods that use only one positive example and generate adversarial negative candidates. gpt2-xl model where the dialog context is truncated to 256 tokens and a single knowledge sentence is truncated to 32 tokens. the dialog context and knowledge sentence are concatenated together to be used as input into the model.