Pneg: Prompt-based Negative Response Generation for Dialogue Response Selection Task (2022)

Abstract: In retrieval-based dialogue systems, a response selection model acts as a ranker to select the most appropriate response among several candidates. However, such selection models tend to rely on context-response content similarity, which makes models vulnerable to adversarial responses that are semantically similar but not relevant to the dialogue context. Recent studies have shown that leveraging these adversarial responses as negative training samples is useful for improving the discriminating power of the selection model. Nevertheless, collecting human-written adversarial responses is expensive, and existing synthesizing methods often have limited scalability. To overcome these limitations, this paper proposes a simple but efficient method for generating adversarial negative responses leveraging a large-scale language model. Experimental results on dialogue selection tasks show that our method outperforms other methods of synthesizing adversarial negative responses. These results suggest that our method can be an effective alternative to human annotators in generating adversarial responses. Our code and dataset will be released if the paper is accepted.

## introduction in retrieval-based dialogue systems, the response selection model aims to predict the most appropriate response among multiple candidates retrieved for a given conversation context @xcite @xcite . the selection model is generally trained to distinguish a related (positive) response from randomly sampled negative responses on training datasets, but such a model generally poses the following problems. first, randomly selected negatives are often too easy to distinguish because they are totally irrelevant to the dialogue context @xcite . in this case, the model is more likely dialogue context positive response pneg negative responses (generated) negative response generation by using gpt-3 few human-written examples training response selection model with generated negative responses original training sample yes! what would you like me to translate? can you help me to translate these sentences into german? i can speak german, but can't speak french. how well can you speak them? i can speak french and german. how many languages can you speak? i speak three languages, but my english is poor. can you give me a quick translation of everything? it's urgent. i am going on a trip to germany next month to predict the response only by relying on the superficial content similarity of the context-response pairs @xcite @xcite . these models are vulnerable to adversarial responses with high content similarity to the dialogue context, and fail to distinguish subtle differences in various contexts in real-world scenarios. second, random sampling causes sampling bias, such as containing false negatives for a given dialogue context @xcite . these biases inherent in the training datasets hinder the accurate prediction of the selection model, resulting in performance degradation. to mitigate this problem, recent studies have proposed various methods to synthesize and leverage adversarial negative training samples so that the selection model can learn features beyond content similarity @xcite . however, existing methods for synthesizing adversarial negative responses @xcite al., 2018; alzantot et al., 2018; zhang et al., 2019; qiu et al., 2021; gupta et al., 2021) still have limitations in creating human-like responses. the most reliable method is to collect human-written adversarial negatives (sai et al., 2020), but it is not scalable because it is expensive and time consuming. to overcome these limitations, we note that large-scale language models such as gpt3 can be utilized as a low-cost data labeler @xcite . in this paper, we present pneg, a promptbased negative response generation method leveraging a large-scale language model (figure 1 ). in p-neg, we can cheaply collect human-like adversarial negatives by using an in-context learning-based data augmentation method with human-written samples as demonstration examples. experimental results on the dialogue response selection task show that the selection model that trained the negative samples from pneg has better performance than other baselines. we then conduct quality evaluation and ablation studies to analyze the validity of pneg. consequently, we argue that pneg can be an efficient alternative to human annotators in generating adversarial responses. our contributions are as follows: ## pneg: prompt-based negative response generation large-scale language models such as gpt-3 @xcite al., 2020) can augment fluent text training samples using natural language prompts and in-context examples (yoo et al., 2021; schick and schütze, 2021; bae et al., 2022; liu et al., 2022). by extending such direction to the response selection task, we propose pneg, a prompt-based negative response generation method for robust response selection models. our method consists of three steps: (1) selecting demonstration examples, (2) constructing a prompt containing examples and target dialogue context, and (3) generating adversarial negative responses by inputting the prompt into gpt-3. the generated negative responses are used as training samples for response selection task. ## conclusion in this paper, we present pneg, a prompt-based adversarial negative response generation method for response selection task. pneg can collect highquality adversarial negative responses at a lower cost than human annotators. our experiments on dialogue response selection tasks show that negative responses generated by pneg can improve the discriminating power of the selection models. in future work, we are interested in designing more confused adversarial responses and how to effectively utilize them for training the selection models.