R e ^3 D ial: Retrieve, Reorganize and Rescale Conversations for Long-Turn Open-Domain Dialogue Pre-training (2023)

Abstract: Pre-training on large-scale open-domain dialogue data can substantially improve the performance of dialogue models. However, the pre-trained dialogue model’s ability to utilize long-range context is limited due to the scarcity of long-turn dialogue sessions. Most dialogues in existing pre-training corpora contain fewer than three turns of dialogue. To alleviate this issue, we propose the Retrieve, Reorganize and Rescale framework (Re ^3 Dial), which can automatically construct billion-scale long-turn dialogues by reorganizing existing short-turn ones. Given a short-turn session, Re ^3 Dial first employs a session retriever to retrieve coherent consecutive sessions. To this end, we train the retriever to capture semantic and discourse relations within multi-turn dialogues through contrastive training. Next, Re ^3 Dial samples a session from retrieved results following a diversity sampling strategy, which is designed to penalize repetitive or generic sessions. A longer session is then derived by concatenating the original session and the sampled session. By repeating the above process, Re ^3 Dial can yield a coherent long-turn dialogue. Extensive experiments on multiple multi-turn dialogue benchmarks demonstrate that Re ^3 Dial significantly improves the dialogue model’s ability to utilize long-range context and thus generate more sensible and informative responses. Finally, we build a toolkit for efficiently rescaling conversations with Re ^3 Dial, which enables us to construct a corpus containing 1B Chinese dialogue sessions with 11.3 turns on average (5X longer than the original corpus). We will release our retriever model, toolkit, and data for public use.

## introduction building intelligent open-domain dialogue systems that can generate coherent and engaging multi-turn dialogues with humans has been one of the long- ## methodology an overview of re 3 dial is shown in figure 2 . let @xmath0. for each session s i , re 3 dial constructs longturn dialogues automatically in four steps: (1) initialize the constructed session s out with s i : s @xmath1. (2) use udsr to retrieve top-k coherent sessions from d: udsr( we observe that previous retrievers either rely solely on local term matching and fail to capture global semantic relations (e.g., bm25 @xcite ) or struggle to capture discourse coherence within multi-turn dialogues @xcite (e.g., contriever @xcite ). therefore, these retrievers exhibit unsatisfactory performance in our dialogue session retrieval task. to remedy these problems, we train an unsupervised dense session retriever (udsr). by using consecutive dialogue segments from the same dialogue as positive pairs and those from different dialogues as negative pairs, udsr demonstrates superior capability in capturing global semantic relevance and discourse coherence within multi-turn dialogues. model structure given two dialogue sessions s q and s c , we encode them using two encoder models, e q and e c . the similarity score is defined as the dot product of their representations: contrastive training we adopt contrastive training to train the dialogue session encoder e q and e c . for each training instance {s q , s + c , s - c 1 , • • • , s - cn }, which contains one query session s q , one coherent session s + c , and n incoherent sessions {s - c i } @xmath2...n , we optimize the contrastive loss l as follows: positive and negative pairs large-scale positive and negative pairs are crucial for the effectiveness of contrastive learning @xcite . however, considering that there is no available labeled data for this task, we propose to build positive and negative pairs from unlabeled plain dialogues, which are much easier to access. let {s u i } be an unlabeled dialogue corpus, where we then obtain a positive pair (s q i , s c j ) if @xmath3. moreover, we consider two kinds of negatives: (1) easy negatives: given s q i , we randomly select a consecutive session s c j where i ̸ = j. (2) hard negatives: given s q i , we leverage the top-k sessions retrieved by bm25 as s c i , thereby improving the model's ability to differentiate those incoherent negatives that have lexical overlaps @xcite .