Dialogue Response Selection with Hierarchical Curriculum Learning (2021)

Abstract: We study the learning of a matching model for dialogue response selection. Motivated by the recent finding that models trained with random negative samples are not ideal in real-world scenarios, we propose a hierarchical curriculum learning framework that trains the matching model in an “easy-to-difficult” scheme. Our learning framework consists of two complementary curricula: (1) corpus-level curriculum (CC); and (2) instance-level curriculum (IC). In CC, the model gradually increases its ability in finding the matching clues between the dialogue context and a response candidate. As for IC, it progressively strengthens the model’s ability in identifying the mismatching information between the dialogue context and a response candidate. Empirical studies on three benchmark datasets with three state-of-the-art matching models demonstrate that the proposed learning framework significantly improves the model performance across various evaluation metrics.

## introduction building intelligent conversation systems is a longstanding goal of artificial intelligence and has attracted much attention in recent years @xcite . an important challenge for building such conversation systems is the response selection problem, that is, selecting the best response to a given dialogue context from a set of candidate responses @xcite . to tackle this problem, different matching models are developed to measure the matching degree between a dialogue context and a response candidate @xcite @xcite . despite their differences, most prior works train the model with data constructed by a simple heuristic. for each context, the human-written response is considered as positive (i.e., an appropriate response) and the responses from other dialogue contexts are considered as negatives (i.e., inappropriate responses). in practice, the negative responses are often randomly sampled and the training objective ensures that the positive response scores are higher than the negative ones. recently, some researchers @xcite have raised the concern that randomly sampled negative responses are often too trivial (i.e., totally irrelevant to the dialogue context). models trained with trivial negative responses may fail to handle strong distractors in real-world scenarios. essentially, the problem stems from the ignorance of the diversity in context-response matching degree. in other words, all random responses are treated as equally negative regardless of their different distracting strengths. for example, ta-ble 1 shows a conversation between two speakers and two negative responses (n1, n2) are presented. for n1, one can easily dispel its appropriateness as it unnaturally diverges from the tv show topic. on the other hand, n2 is a strong distractor as it overlaps significantly with the context (e.g., fantasy series and game of thrones). only with close observation we can find that n2 does not maintain the coherence of the discussion, i.e., it starts a parallel discussion about an actor in game of thrones rather than elaborating on the enjoyable properties of the tv series. in addition, we also observe a similar phenomenon on the positive side. for different training context-response pairs, their pairwise relevance also varies. in inspired by the above observations, in this work, we propose to employ the idea of curriculum learning (cl) @xcite . the key to applying cl is to specify a proper learning scheme under which all training examples are learned. by analyzing the characteristics of the concerned task, we tailor-design a hierarchical curriculum learning (hcl) framework. specifically, our learning framework consists of two complementary curriculum strategies, corpus-level curriculum (cc) and instance-level curriculum (ic), covering the two distinct aspects of response selection. in cc, the model gradually increases its ability in finding matching clues through an easy-to-difficult arrangement of positive context-response pairs. in ic, we sort all negative responses according to their distracting strength such that the model's capability of identifying the mismatching information can be progressively strengthened. notably, our learning framework is independent to the choice of matching models. for a comprehensive evaluation, we evaluate our approach on three representative matching models, including the current state of the art. results on three benchmark datasets demonstrate that the proposed learning framework leads to remarkable performance improvements across all evaluation metrics. in a nutshell, our contributions can be summa-rized as: (1) we propose a hierarchical curriculum learning framework to tackle the task of dialogue response selection; and (2) empirical results on three benchmark datasets show that our approach can significantly improve the performance of various strong matching models, including the current state of the art. ## background given a dataset @xmath0, r i )} |d| @xmath1, the learning of a matching model s(•, •) is to correctly identify the positive response r i conditioned on the dialogue context c i from a set of negative responses r - i . the learning objective is typically defined as in most existing studies @xcite @xcite , the training negative responses r - i are randomly selected from the dataset d. recently, @xcite and @xcite proposed different approaches to strengthen the training negatives. in testing, for any contextresponse (c, r), the models give a score s(c, r) that reflects their pairwise matching degree. therefore, it allows the user to rank a set of response candidates according to the scores for response selection. ## end 6 as input to optimize the model using eq. ( 1 ). fast ranking model. as described in eq. ( 2 ) and ( 3 ), our framework requires a ranking model g(•, •) that efficiently measures the pairwise relevance of millions of possible context-response combinations. in this work, we construct g(•, •) as an non-interaction matching model with dualencoder structure such that we can precompute all contexts and responses offline and store them in cache. for any context-response pair (c, r), its pairwise relevance g(c, r) is defined as offline index. after training the ranking model on the same response selection dataset d using the in-batch negative objective @xcite , we compute the dense representations of all contexts and responses contained in d. then, as described in eq. ( 4 ), the relevance scores of all possible combinations of the contexts and responses in d can be easily computed through the dot product between their representations. after this step, we can compute the corpus-level and instance-level difficulty of all possible combinations and cache them in memory for a fast access in training. ## conclusion in this work, we propose a novel hierarchical curriculum learning framework for training response selection models for multi-turn conversations. during training, the proposed framework simultaneously employs corpus-level and instance-level curricula to dynamically select suitable training data based on the state of the learning process. extensive experiments and analysis on two benchmark datasets show that our approach can significantly improve the performance of various strong matching models on all evaluation metrics.