Efficient Dynamic Hard Negative Sampling for Dialogue Selection (2024)

Abstract: Recent studies have demonstrated significant improvements in selection tasks, and a considerable portion of this success is attributed to incorporating informative negative samples during training. While traditional methods for constructing hard negatives provide meaningful supervision, they depend on static samples that do not evolve during training, leading to sub-optimal performance. Dynamic hard negative sampling addresses this limitation by continuously adapting to the model’s changing state throughout training. However, the high computational demands of this method restrict its applicability to certain model architectures. To overcome these challenges, we introduce an efficient dynamic hard negative sampling (EDHNS). EDHNS enhances efficiency by pre-filtering easily discriminable negatives, thereby reducing the number of candidates the model needs to compute during training. Additionally, it excludes question-candidate pairs where the model already exhibits high confidence from loss computations, further reducing training time. These approaches maintain learning quality while minimizing computation and streamlining the training process. Extensive experiments on DSTC9, DSTC10, Ubuntu, and E-commerce benchmarks demonstrate that EDHNS significantly outperforms baseline models, proving its effectiveness in dialogue selection tasks.

## introduction the problem of selecting the most suitable answer from multiple candidates has been extensively explored in the field of natural language processing, particularly within selection tasks @xcite @xcite @xcite . typically, these tasks involve one positive candidate and multiple negative candidates associated with a given question. training on all negative samples can be time-consuming, so it is common practice to randomly select a subset of negative samples for training. however, random negative sampling may not provide meaningful supervision, as models updated with easily discriminable negative samples contribute minimally to gradient updates @xcite . to address this issue, various strategies for hard negative sampling have been proposed and have demonstrated their effectiveness @xcite @xcite . heuristic and data-dependent methods @xcite utilize the unique characteristics of datasets but are constrained by their limited generalizability, making them less effective for other datasets. @xcite ; @xcite have enhanced these approaches with model-based strategies. however, these approaches still face challenges, as they rely on static (fixed) hard negative samples that do not dynamically adapt during training. recently, dynamic hard negative sampling @xcite has been introduced to overcome these limitations by adaptively selecting hard negatives for learning in response to model updates, effectively aligning with changes in model behavior. however, it requires continual recalculations of matching scores for all negative candidates throughout training, significantly increasing computational costs. this restriction predominantly confines its application to fast dense retrieval models @xcite @xcite , and poses implementation challenges in models with slower inference speeds. to mitigate these challenges, we propose an efficient dynamic hard negative sampling (edhns) method applicable to various model architectures. like traditional approaches, our method computes matching scores for negative candidates at each training step. however, it alleviates the computational burden through two main strategies: shortlisting and selective update. in shortlisting, we compute scores only for a filtered subset of candidates by removing easily discriminable negative candidates from the pool, enabling the selection of sufficiently hard negatives from a smaller set. in the selective update, we measure confidence scores for question-candidate pairs and exclude those with high scores from training, further save training time. these strategies enable meaningful learning with reduced computational demands, enhancing overall performance. notably, for the first time, we have applied dynamic hard negative sampling to the cross encoder, which has demonstrated strong performance in selection tasks, leading to significant performance improvements. we empirically demonstrate the efficacy of our method through extensive experiments on two key tasks. the first task, knowledge selection, focuses on choosing relevant knowledge for a given conversation. we evaluate the performance of this task using the dstc9 @xcite and dstc10 @xcite benchmarks. the second task, response selection, requires choosing the most appropriate response for a given dialogue context. we assess this task using the ubuntu @xcite and e-commerce @xcite benchmarks. our experiments show that models using edhns significantly outperform baseline models across all four benchmarks. specifically, edhns achieves top performance in most evaluation metrics for dstc9 and dstc10, and also demonstrates superior performance in the ubuntu and e-commerce benchmarks. ## related work previous studies have introduced various hard negative sampling approaches, resulting in notable enhancements in various nlp tasks. these strategies can be categorized into two types: static hard negative sampling and dynamic hard negative sampling @xcite . static hard negative sampling pre-defines fixed hard negative samples before the training process. this method selects hard negative samples based on data characteristics or by retrieving or generating them using a model. in the knowledge selection task, @xcite introduce a data-dependent negative sampling strategy by categorizing given knowledge into different groups. @xcite adopt a model-based negative sampling method to sample fixed hard negatives. in the response selection task, @xcite use retrieval and generation models to diversify negative samples, while @xcite generate adversarial examples using gpt-3. in text retrieval tasks, since negative samples are derived from text candidates recalled by the retrieval module, previous works @xcite focus on jointly optimizing the retriever and reranker modules. dynamic hard negative sampling, in contrast, selects hard negative samples dynamically during the training process, considering the evolving state of the model. in response selection, @xcite adapt negative examples to matching models during the learning process, exploring various sampling strategies. particularly, this approach has been extensively studied in the training of dense retrieval models. @xcite al. (2020) and xiong et al. (2021) use dense retrieval models to pre-retrieve the top documents as hard negatives during training, periodically rebuilding the index and refreshing the hard negatives. @xcite propose a queryside training algorithm that directly optimizes the dense retrieval model using dynamic hard negative sampling. however, applying dynamic hard negative sampling to most model architectures-except for the bi-encoder structure commonly used in dense retrieval-poses challenges due to the slower speeds and high computational demands. this limitation is especially evident in cross-encoder-based models, which, despite their superior performance in selection tasks, require extensive computations for token-level interactions and cannot pre-compute candidate embeddings. to overcome these challenges, we propose a novel and efficient dynamic hard negative sampling method.