Venue: NAACL 2022

# Learning Dialogue Representations from Consecutive Utterances

Zhihan Zhou \({}^{*}\) Dejiao Zhang\({}^{\dagger}\) Wei Xiao\({}^{\dagger}\) Nicholas Dingwall\({}^{\dagger}\)

**Xiaofei Ma\({}^{\dagger}\) Andrew O. Arnold\({}^{\dagger}\) Bing Xiang\({}^{\dagger}\)**

\({}^{*}\)Northwestern University AWS AI Labs

Work done during an internship at AWS AI Labs1Throughout this paper, we use _utterance_ to refer to all the sentences that belong to the same dialogue turn.2The code and pre-trained models are publicly available at [https://github.com/amazon-research/dse](https://github.com/amazon-research/dse).

Footnote 1: footnotemark:

###### Abstract

Learning high-quality dialogue representations is essential for solving a variety of dialogeuf oriented tasks, especially considering that dialogue systems often suffer from data scarcity. In this paper, we introduce Dialogue Sentence Embedding (DSE), a self-supervised contrastive learning method that learns effective dialogue representations suitable for a wide range of dialogue tasks. DSE learns from dialogues by taking consecutive utterances2 of the same dialogue as positive pairs for contrastive learning. Despite its simplicity, DSE achieves significantly better representation capability than other dialogue representation and universal sentence representation models. We evaluate DSE on five downstream dialogue tasks that examine dialogue representation at different semantic granularities. Experiments in few-shot and zero-shot settings show that DSE outperforms baselines by a large margin. For example, it achieves \(13\%\) average performance improvement over the strongest unsupervised baseline in 1-shot intent classification on 6 datasets. 3 We also provide analyses on the benefits and limitations of our model.

Footnote 2: The code and pre-trained models are publicly available at [https://github.com/amazon-research/dse](https://github.com/amazon-research/dse).

+
Footnote †: 


dialogue-NLI dataset Welleck et al. (2018) that consists of both semantically entailed and contradicted pairs. However, due to its relatively limited scale and diversity, we found learning from this dataset leads to less satisfying performance, while the high cost of collecting additional human annotations precludes its scalability. On the other extreme, unsupervised representation learning has achieved encouraging results recently, among which SimCSE Gao et al. (2021) and TOD-BERT Wu et al. (2020) set new state-of-the-art results on general texts and dialogues, respectively.

SimCSE uses Dropout Srivastava et al. (2014) to construct positive pairs from any text by passing a sentence through the encoder twice to generate two different embeddings. Although SimCSE outperforms common data augmentations that directly operate on discrete text, we find it performs poorly in the dialogue domain (see Sec. 4.3). This motivates us to seek better positive pair constructions by leveraging the intrinsic properties of dialogue data. On the other hand, TOD-BERT takes an utterance and the concatenation of all the previous utterances in the dialogue as a positive pair. Despite promising performance on same tasks, we found TOD-BERT struggles on many other dialogue tasks where the semantic granularities or data statistics are different from those evaluated in their paper.

In this paper, inspired by the fact that dialogues consist of consecutive utterances that are often semantically related, we use consecutive utterances within the same dialogue as positive pairs for contrastive learning (See Figure 2). This simple strategy works surprisingly well. We evaluate DSE on a wide range of task-oriented dialogue applications, including intent classification, out-of-scope detection, response selection, and dialogue action prediction. We demonstrate that DSE substantially outperforms TOD-BERT, SimCSE, and some other sentence representation learning models in most scenarios. We assess the effectiveness of our approach by comparing DSE against its variants trained on other types of positive pairs (e.g., Dropout and Dialogue-NLI). We also discuss the trade-off in learning dialogue representation for tasks focusing on different semantic granularities and provide insights on the benefits and limitations of the proposed method. Additionally, we empirically demonstrate that using consecutive utterances as positive pairs can effectively improve the training stability (Appendix A.3).

## 2 Why Contrastive Learning on Consecutive Utterances?

When performing contrastive learning on consecutive utterances, we encourage the model to treat an utterance as similar to its adjacent utterances and dissimilar to utterances that are not consecutive to it or that belong to other dialogues.

On the one hand, this training process directly increases an utterance's similarity with its _true_ response and decreases its similarities with other randomly sampled utterances. The ability to identify the appropriate response from many similar utterances is beneficial for dialogue ranking tasks (e.g., response selection). On the other hand, consecutive utterances also contain implicit categorical information, which benefits dialogue classification tasks (e.g., intent classification and out-of-scope detection). Consider pairs 1 and 4 in Figure 2: we implicitly learn similar representations of _I am looking for restaurants_ and _Find me some restaurants_, since they are both consecutive with _What type of food do you like?_.

In contrast, SimCSE does not enjoy these benefits by simply using Dropout as data augmentation. Although TOD-BERT also leverages the intrinsic dialogue semantics by combining an utterance with its dialogue context as positive pair, the context is often the concatenation of \(5\) to \(15\) utterances.

Figure 2: Illustration of the positive pair construction from dialogues.

 

Due to the large discrepancy in both semantics and data statistics between each utterance and its context, simply optimizing the similarity between them leads to less satisfying representations on many dialogue tasks. As shown in Section 4, TODBERT can even lead to degenerated representations on some downstream tasks when compared to the original BERT model.

## 3 Model

### Notation

Let be a batch of positive pairs, where is the batch size. In our setting, each denotes a pair of consecutive utterances sampled from a dialogue. Let denote the representation of the text instance that is obtained through an encoder. In this paper, we use mean pooling to obtain representations.

### Training Target

Contrastive learning aims to maximize the similarity between positive samples and minimize the similarity between negative samples. For a contrastive anchor , the contrastive loss aims to increase its similarity with its positive sample and decrease its similarity with the other negative samples within the same batch.

We adopt the _Hard-Negative_ sampling strategy proposed by Zhang et al. (2021), which puts higher weights on the samples that are close to the anchor in the representation space. The underlying hypothesis is that hard negatives are more likely to occur among those that are located close to the anchor in the representation space. Specifically, the Hard-Negative sampling based contrastive loss regarding anchor is defined as follows:

As mentioned above, here and represent the indices of the anchor and its positive sample. We use to denote the temperature hyperparameter and represent the cosine similarity of and . In the above loss, is defined as follows,

(2)

Noted here, the denominator is averaged over all the other -2 negatives of . Intuitively, samples that are close to the anchor in the representation space are assigned with higher weights. In other words, denotes the relative importance of instance for optimizing the contrastive loss of the anchor among all -2 negatives. For every positive pair , we respectively take and as the contrastive anchor to calculate the contrastive loss. Thereby, the contrastive loss over the batch is calculated as:

(3)

Here is defined by exchanging the roles of instances and in Equation (1), respectively.

## 4 Experiments

We run experiments with five different backbones: \({\text{BERT}}_{\text{base}}\), \({\text{BERT}}_{\text{large}}\)Devlin et al. (2018), RoBERTa, RoBERTa[Liu et al., 2019], DistilBERTb,Dasu et al. (2019). Due to the space limit, we only present the results on in the main text. The results of other models are summarized in Appendix D. We use the same training data as TOD-BERT for a fair comparison. We summarize the implementation details and data statistics of both pre-training and evaluation in Appendices A and B, respectively.

### Baselines

We compare DSE against several representation learning models that attain state-of-the-art results on both general text and dialogue languages. We categorize them into the following two categories.

Supervised LearningSimCSE-supGao et al. (2021) is the supervised version of SimCSE, which uses entailment and contradiction pairs in the NLI datasets Bowman et al. (2015); Williams et al. (2018) to construct positive pair and hard negative pair accordingly. In a similar vein, PairSupConZhang et al. (2021) leverages the entailment pairs as positive pairs only while proposing an unsupervised hard negative sampling strategy that we summarized in Section 3.2. Following this line, we also evaluate DSE against its variant DSE-dianli trained on the Dialogue Natural Language Inference dataset Welleck et al. (2018) by taking all the _entail_ pairs as positive pairs 

Unsupervised LearningTOD-BERTWu et al. (2020) optimizes a contrastive response selection objective by treating an utterance and its dialogue context as positive pair. DialoGPTZhang et al. (2019) is a dialogue generation model that learns from consecutive utterance by optimizing a language modeling target.3 SimCSE-unsup Gao et al. (2021) uses Dropout Srivastava et al. (2014) to construct positive pairs. In the general text domain, SimCSE-unsup has attained impressive performance over several explicit data augmentation strategies that directly operate on the discrete texts. To test its effectiveness in the dialogue domain, we compare DSE against its variant DSE-dropout where augmentations of every single utterance are obtained through Dropout.

Footnote 3: We use mean pooling of its hidden states as sentence representation, which leads to better performance than using only the last token. We use its _Medium_ version that has twice as many parameters as \({\bf BERT}_{\texttt{base}}\), since we found its _Small_ version performs dramatically worse under our settings.

The evaluations on DSE-dropout and DSE-dianli allow us to fairly compare our approach against the state-of-the-art approaches in both the supervised learning and the unsupervised learning regimes.

### Evaluation Setting

To accommodate the fact that obtaining a large number of annotations is often time-consuming and expensive for solving the task-oriented dialogue applications, especially considering the variety of domains and certain privacy concerns, we mainly focus on few-shot or zero-shot based evaluations.

#### 4.2.1 Evaluation Methods

Considering that only a few annotations are available in our setting, we mainly focus on the **similarity-based** evaluations, where predictions are made based on different similarity metrics applied in the embedding space without requiring updating the model.

We use different random seeds to independently construct multiple (See Table 1) few-shot train and validation sets from the original training data and use the original test data for performance evaluation. To examine whether the performance gap reported in the similarity-based evaluations is consistent with the associated fine-tuning approaches, we also report the **fine-tuning** results. We perform early stopping according to the validation set and report the testing performance averaged over different data splits.

#### 4.2.2 Tasks and Metrics

We evaluate all models considered in this paper on two types of tasks: **utterance-level** and **dialogue-level**. The utterance-level tasks take a single dialogue utterance as input, while the dialogue-level tasks take the dialogue history as input. These two types of tasks assess representation quality on dialogue understanding at different semantic granularities, which are shared across a variety of downstream tasks.

Intent Classificationis an utterance-level task that aims to classify user utterances into one of the pre-defined intent categories. We use Prototypical Networks Snell et al. (2017) to perform the similarity-based evaluation. Specifically, we calculate a prototype embedding for each category by averaging the embedding of all the training samples that belong to this category. A sample is classified into the category whose prototype embedding is the most similar to its own. We report the classification accuracy for this task.

Out-of-scope Detectionadvances intent classification by detecting whether the sample is out-of

\begin{table}
\begin{tabular}{l l c r} \hline \hline
**Task** & **Dataset** & **Evaluation Setting** & **Num.** \\ \hline
**Intent Classification** & Clin50, Snips, Hwu64, & 1-shot \& 5-shot fine-tune & 10 \\  & Bank77, Appen-A, Appen-H & 1-shot \& 5-shot similarity & 10 \\ \hline
**Out-of-scope Detection** & Clin150 & 1-shot \& 5-shot similarity & 10 \\ \hline
**Utterance-level Response Selection** & AmazonQA & 0-shot similarity & 1 \\  & & 500-shot \& 1000-shot fine-tune & 5 \\ \hline
**Dialogue-level Response Selection** & DSTC7–Ubuntu & 0-shot similarity & 1 \\ \hline
**Dialogue Action Prediction** & DSTC2, GSIM & 10-shot \& 20-shot fine-tune & 5 \\ \hline \hline \end{tabular}
\end{table}
Table 1: Summarization of all the experimental settings. Please see Appendix B.2 for details of each dataset. The last column (**Num**) indicates the number of independent experiments with different random seeds (we report the averaged results). Since there is no randomness in 0-shot evaluations, we only run them once.

 

scope, _i.e.,_ does not belong to any pre-defined categories. We adapt the aforementioned Prototypical Networks to solve it. For a test sample, if its similarity with its most similar category is lower than a threshold, we classify it as out-of-scope. Otherwise, we assign it to its most similar category. For each model, we calculate the mean and std (standard deviation) of the similarity scores between every sample and its most similar prototype embedding, and take mean\(-\)std and mean as the threshold, respectively. The evaluation set contains both in-scope and out-of-scope examples. We evaluate this task with four metrics: 1) Accuracy: accuracy of both in-scope and out-of-scope detection. 2) In-Accuracy: accuracy reported on 150 in-scope intents. 3) OOS-Accuracy: out-of-scope detection accuracy. 4) OOS-Recall: recall of OOS detection.

**Utterance-level Response Selection** is an utterance-level task that aims to find the most appropriate response from a pool of candidates for the input user query, where both the query and response are single dialogue utterances. We formulate it as a ranking problem and evaluate it with Top-k-100 accuracy (a.k.a., k-to-100 accuracy), a standard metric for this ranking problem Wu et al. (2020). For every query, we combine its ground-truth response with 99 randomly sampled responses and rank these 100 responses based on their similarities with the query in the embedding space. The Top-k-100 accuracy represents the ratio of the ground-truth response being ranked at top-k, where k is an integer between 1 and 100. We report the Top-1, Top-3, and Top-10 accuracy of the models.

**Dialogue-Level Response Selection** is a dialogue-level task. The only difference with the _Utterance-level Response Selection_ is that query in this task is dialogue history (e.g., concatenation of multiple dialogue utterances from different speakers). We also report the Top-1, Top-3, and Top-10 accuracy for this task.

**Dialogue Action Prediction** is a dialogue-level task that aims to predict the appropriate system action given the most recent dialogue history. We formulate it as a multi-label text classification problem and evaluate it with model fine-tuning. We report the Macro and Micro F1 scores for this task.

### Main Results

**Intent Classification & Out-of-scope Detection** Tables 2 and 3 show the results of similarity-based intent classification and out-of-scope detection. The fine-tuning based results are presented in Appendix C. As we can see, DSE substantially outperforms all the baselines. In intent classification, it attains \(13\%\) average accuracy improvement over the strongest unsupervised baseline. More impor

\begin{table}
\begin{tabular}{l l c c c c c c c} \hline \hline  & **BERT\({}^{\diamondsuit}_{base}\)** & **Clinc150** & **Bank77** & **Snips** & **Hwu64** & **Appen-A** & **Appen-H** & **Ave.** \\ \hline \multirow{8}{*}{
\begin{tabular}{c} **SimCSE-sup\({}^{\clubsuit}_{\vspace}\)** \\ **PairSupCon\({}^{\spadesuit}\)** \\ **Desé-diani\({}^{\clubsuit}\)** \\ \end{tabular} } & 52.30 & 38.05 & 65.98 & 40.79 & 35.35 & 44.81 & 46.21 \\  & **PairSupCon\({}^{\clubsuit}\)** & 55.34 & 41.30 & 65.20 & 41.43 & 37.55 & 47.55 & 48.06 \\  & **DSE-diani\({}^{\clubsuit}\)****(ours)** & 45.91 & 38.33 & 58.23 & 34.95 & 33.87 & 42.26 & 42.26 \\ \cline{2-10}  & **BERT\({}^{\diamondsuit}\)** & 36.98 & 22.05 & 62.51 & 27.74 & 13.19 & 18.74 & 30.20 \\  & **SimCSE-unsup\({}^{\diamondsuit}\)** & 46.44 & 37.51 & 59.58 & 34.34 & 27.10 & 36.00 & 40.16 \\  & **DialoGPT\({}^{\diamondsuit}\)** & 42.23 & 28.08 & 63.10 & 30.45 & 18.90 & 24.48 & 34.54 \\  & **TOD-BERT\({}^{\diamondsuit}\)****(ours)** & 36.67 & 27.11 & 62.52 & 29.52 & 20.61 & 26.68 & 33.85 \\  & **DSE-dropout\({}^{\diamondsuit}\)****(ours)** & 46.48 & 30.02 & 65.03 & 33.25 & 16.94 & 21.77 & 35.58 \\  & **DSE-’(ours)** & **62.53** & **43.12** & **79.57** & **44.31** & **37.97** & **48.71** & **52.70** \\  & **SimCSE-sup\({}^{\clubsuit}\)** & 71.11 & 56.38 & 79.98 & 56.52 & 49.71 & 59.42 & 62.18 \\  & **PairSupCon\({}^{\clubsuit}\)** & 73.88 & 60.07 & 76.14 & 55.75 & **52.71** & 62.23 & 63.46 \\  & **DE-diani\({}^{\clubsuit}\)****(ours)** & 60.65 & 49.78 & 73.80 & 46.65 & 46.52 & 54.39 & 55.30 \\  & **BERT\({}^{\diamondsuit}\)****(ours)** & 59.48 & 38.73 & 78.65 & 43.15 & 21.39 & 27.61 & 44.83 \\  & **SimCSE-unsup\({}^{\diamondsuit}\)** & 65.37 & 55.03 & 77.01 & 48.79 & 43.35 & 51.55 & 56.85 \\  & **DialoGPT\({}^{\diamondsuit}\)****(data)** & 64.53 & 46.56 & 82.15 & 45.67 & 33.67 & 39.61 & 52.03 \\  & **TOD-BERT\({}^{\diamondsuit}\)****(ours)** & 57.74 & 42.98 & 79.68 & 42.32 & 33.58 & 42.52 & 49.80 \\  & **DSE-dropout\({}^{\diamondsuit}\)****(ours)** & 70.46 & 49.95 & 80.10 & 52.16 & 30.00 & 37.48 & 53.36 \\  & **DSE\({}^{\diamondsuit}\)****(ours)** & **78.73** & **61.65** & **88.62** & **60.87** & 52.32 & **62.68** & **67.48** \\ \hline \hline \end{tabular}
\end{table}
Table 2: Results on similarity-based 1-shot and 5-shot Intent Classification. Predictions are made purely based on the embeddings provided by each model without any parameter tuning. All the models use BERT\({}_{\texttt{base}}\) as the backbone model. ▲: Supervised models. \(\diamondsuit\): Unsupervised models.

 

tantly, DSE achieves a \(5\%\)-\(10\%\) average accuracy improvement over the supervised baselines that are trained on a large amount of expensively annotated data. The same trend was observed in out-of-scope detection, where DSE achieves \(13\%\)-\(20\%\) average performance improvement over the strongest unsupervised baseline. The comparison between DSE, DSE-dropout, and DSE-dianli further demonstrates the effectiveness of using consecutive utterances as positive pairs in learning dialogue embeddings.

The left panel of Figure 1 visualizes the embeddings on the Clinc150 dataset given by TODBERT, SimCSE, and DSE, which provides more intuitive insights into the performance gap. As shown in the figure, with the DSE embeddings, in-scope samples belonging to the same category are closely clustered together. Clusters of different categories are clearly separated with a large margin, while the out-of-scope samples are far away from those in-scope clusters.

Response SelectionTable 4 shows the the results of similarity-based 0-shot response selection on utterance-level (AmazonQA) and dialogue-level (DSTC7-Ubuntu). Results of finetune-based evaluation on AmazonQA show similar trend and we summarize in Table 9 in Appendix. In Table 4, the large improvement attained by DSE over the baselines indicate our model's capability in dialogue response selection, in presence of both single-utterance query or using long dialogue history as query. The right panel of Figure 1 further illustrates this. It visualizes the embeddings of questions and

\begin{table}
\begin{tabular}{l c c c c c c} \hline \hline  & \multicolumn{3}{c}{**AmazonQA**} & \multicolumn{3}{c}{**DSTC7-Ubuntu**} \\ \cline{2-8}  & **Top-1 Acc.** & **Top-3 Acc.** & **Top-10 Acc.** & **Top-1 Acc.** & **Top-3 Acc.** & **Top-10 Acc.** \\ \hline
**SimCSE-sup****[FOOTNOTE:]Footnote : Supervised models. \(\diamond\): Unsupervised models.[ENDFOOTNOTE]** & 47.03 & 62.40 & 76.80 & 11.37 & 19.40 & 33.53 \\
**PairSupCon****[FOOTNOTE:]Footnote : Supervised models. \(\diamond\): Unsupervised models.[ENDFOOTNOTE]** & 52.22 & 65.09 & 76.85 & **15.00** & 23.02 & **35.73** \\
**DSE-dianli****[FOOTNOTE:]Footnote : Supervised models. \(\diamond\): Unsupervised models.[ENDFOOTNOTE]** (ours)** & 49.16 & 63.36 & 76.66 & 14.92 & 22.73 & 34.72 \\ \hline
**BERT**\({}^{\diamond}\) & 29.70 & 43.86 & 60.36 & 6.75 & 12.97 & 24.20 \\
**SimCSE-unsp****[FOOTNOTE:]Footnote : Supervised models. \(\diamond\): Unsupervised models.[ENDFOOTNOTE]** & 48.02 & 62.45 & 76.00 & 10.03 & 17.13 & 29.37 \\
**DialoGPT**\({}^{\diamond}\) & 35.96 & 49.52 & 64.44 & 10.20 & 17.60 & 29.82 \\
**TOD-BERT**\({}^{\diamond}\) & 27.25 & 40.26 & 56.63 & 5.52 & 10.55 & 22.30 \\
**DSE-dropout**\({}^{\diamond}\) **(ours)** & 37.80 & 51.64 & 66.58 & 9.55 & 16.97 & 28.80 \\
**DSE\({}^{\diamond}\) **(ours)** & **56.62** & **70.54** & **81.90** & 14.78 & **23.10** & **35.73** \\ \hline \hline \end{tabular}
\end{table}
Table 4: Results on 0-shot response selection on AmazonQA (utterance-level) and DSTC7–Ubuntu (dialogue-level).

 

answers in the AmazonQA dataset calculated by DSE, SimCSE, and TOD-BERT. With the DSE embedding, each question is placed close to its real answer while far away from other candidates.

Dialogue Action PredictionTable 5 shows that DSE outperforms all baselines except TOD-BERT, which indicates its capability in capturing dialogue-level semantics. To better understand TOD-BERT's superiority over DSE on this task, we further investigate this task and find its data format is special. Concretely, here each input consists of multiple utterances explicitly concatenated by using two special tokens [SYS] and [USR] to indicate the system and user inputs, respectively. For example, ([SYS] hi [USR] how are you? [SYS] I'm good). It follows the same format as the queries4 used for training TOD-BERT, while DSE uses a single utterance as the query.

Footnote 4: We use _query_ to refer the first utterance in a positive pair and use _response_ to refer the other one

### Trade-off in Query Construction

To understand the impact of using multiple utterances as queries, we train three new variants of DSE. Specifically, we construct positive pairs as: \((u_{1}\) [SEP] \(u_{2},u_{3})\), \((u_{2}\) [SEP] \(u_{3},u_{4})\), where \(u_{i}\) represents the \(i\)-th utterance in a dialogue. We use the [SEP] token to concatenate two consecutive utterances as query. We refer DSE trained with this data as DSE\({}_{2-1}\) since it uses \(2\) utterances as the query and \(1\) utterance as the response. Similarly, we train another variant DSE\({}_{3-1}\). Lastly, we also combine the positive pairs constructed for training DSE, DSE\({}_{2-1}\), and DSE\({}_{3-1}\) together to train another variant named DSE\({}_{123-1}\).

As shown in Table 5, by simply increasing the number of utterances within each query to three, DSE again outperforms TOD-BERT, and the improvement further expands when trained with the combined set, _i.e.,_ DSE\({}_{123-1}\). Our results demonstrate that using long queries that consist of \(5\) to \(15\) utterances as what TOD-BERT does is not necessary even for dialogue action prediction. We further demonstrate this by evaluating DSE and its variants on all the other four tasks in Table 6, where our model outperforms TOD-BERT by a large margin. As it indicates, by using a single utterance as a query, DSE achieves a good balance among different dialogue tasks. In cases where dialogue action prediction is of great importance, augmenting the original training set of DSE with positive pairs constructed by using query consisting of 2 to 3 utterances is good enough to attain better performance while only incurring a slight performance drop on other tasks.

### Potential Limitation

Considering the effectiveness of using consecutive utterances as positive pairs, a natural yet important question is: what are the potential limitations of

\begin{table}
\begin{tabular}{l c c c c c} \hline \hline
**BERT\({}_{base}\)** & **IC** & **OOS** & **u-RS** & **d-RS** & **DA** \\ \hline
**TOD-BERT** & 41.83 & 47.25 & 41.38 & 12.79 & 63.60 \\
**DSE** & **60.09** & **62.43** & **69.69** & **24.54** & 61.03 \\
**DSE\({}_{2-1}\)** & 56.56 & 61.55 & 59.88 & 19.36 & 62.22 \\
**DSE\({}_{S-1}\)** & 57.26 & 61.19 & 61.94 & 22.04 & 63.67 \\
**DSE\({}_{123-1}\)** & 59.60 & 61.59 & 63.67 & 22.63 & **65.19** \\ \hline \hline \end{tabular}
\end{table}
Table 6: Performance of TOD-BERT, DSE, and its variants on intent classification (IC), out-of-scope detection(OOS), response selection on utterance-level (u-RS) and dialogue-level (d-RS), dialogue action prediction (DA).

 

our proposed approach? When using consecutive utterances as positive pairs for contrastive learning, an assumption is that responses to the same query are semantically similar. Vice versa, queries that prompt the same answer are similar. This assumption holds in many scenarios, yet it fails sometimes.

It may fail when answers have different semantic meanings. Take the pairs 2 and 5 in Figure 2 as an example. Through our data construction, we implicitly consider _I want some pizza_ and _Korean food, please_ to be semantically similar since they are both positively paired with _What type of food do you like_. Although this may be correct in some coarse-grained classification tasks since these two sentences generally represent the same intent (e.g., order food), using them as positive pairs can introduce some noise when considering more fine-grained semantics. This problem is further elaborated when answers are general and ubiquitous, _e.g., Thank you_. Since these utterances can be used to respond to countless types of dissimilar queries, _e.g., I have booked a ticket for you_ v.s. _Happy birthday_, we may implicitly increase the similarities among highly dissimilar utterances when training on these samples, which is undesirable.

We verify this on the NLI datasets, where the the task is to identify whether one sentence semantically entails or contradicts the anchor sentence. For each anchor sentence, we calculate its cosine similarities with both the true _entailment_, _contradiction_ sentences in the representation space. We classify the sentence with higher cosine similarity with the anchor as entailment and the other as the contradiction. Despite DSE achieves better classification accuracy (76.62) than BERT (69.40) and TOD-BERT (70.51), it underperforms SimCSE-unsup (80.31). Although using dropout to construct positive pairs is not as effective as ours in many dialogue scenarios, this method better avoids introducing fine-grained semantic noise.

Despite the limitations, using consecutive utterances as positive pairs still leads to better dialogue representation than the elaborately labeled NLI datasets, indicating the great value of the information contained in dialogue utterances.

## 5 Related Work

Positive Pair ConstructionPopular supervised sentence representation learning often takes advantage of the human-annotated natural language inference (NLI) datasets Bowman et al. (2015); Williams et al. (2018) for contrastive learning Gao et al. (2021); Zhang et al. (2021a); Reimers and Gurevych (2019); Cer et al. (2018). These sentence pairs either entail or contradict each other, making them the great choice for constructing positive and negative training pairs. Unsupervised sentence representation learning often relies on variant data augmentation strategies. Logeswaran and Lee (2018) and Giorgi et al. (2020) propose using sentences and their surrounding context as positive pairs. Other works resort to popular NLP augmentation methods such as word permutation Wu et al. (2020b) and back-translation Fang et al. (2020). Recently, Gao et al. (2021) demonstrates the superiority of using Dropout over other data augmentations that directly operate on the discrete texts.

Contrastive Learning MethodsContrastive learning is key to recent advances in learning sentence embeddings. Many contrastive learning approaches utilize memory-based methods, which draw negative samples from a memory bank of embeddings Hjelm et al. (2018); Bachman et al. (2019); He et al. (2020). On the other hand, Chen et al. (2020) introduces a memory-free contrastive framework, SimCLR, that takes advantage of negative sampling within large mini-batches. Promising results were also reported in the NLP domain. To name a few, Gao et al. (2021) leverages both within batch negatives and the 'contradiction' annotations in NLI; and Zhang et al. (2021a) propose an unsupervised hard-negative sampling strategy.

Dialogue Language ModelLearning dialogue-specific language models has attracted a lot of attention. Along this line, Zhang et al. (2019) adapts the pre-trained GPT-2 model Radford et al. (2019) on Reddit data to perform open-domain dialogue response generation. Bao et al. (2019) evaluates multiple dialogue generation tasks after training on Twitter and Reddit data Wolf et al. (2019); Peng et al. (2020). For dialogue understanding, Henderson et al. (2019) propose a response selection approach using a dual-encoder model. They pretrain the response selection model on Reddit and then fine-tune it for different response selection tasks. Following this, Henderson et al. (2019) introduces a more efficient conversational model that is pre-trained with a response selection target on the Reddit corpus. However, they did not release code or pre-trained models for comparison. Wu 

et al. (2020) combines nine dialogue datasets to obtain a large and high-quality task-oriented dialogue corpus. They introduce the TOD-BERT model by further pre-training BERT on this corpus with both the masked language modeling loss and the contrastive response selection loss.

## 6 Conclusion

In this paper, we introduce a simple contrastive learning method DSE that learns dialogue representations by leveraging consecutive utterances in dialogues as positive pairs. We conduct extensive experiments on five dialogue tasks to show that the proposed method greatly outperforms other state-of-the-art dialogue representation models and universal sentence representation methods. We provide ablation study and analysis on our proposed data construction from different perspectives, investigate the trade-off between different data construction variants, and discuss the potential limitation to motivate further exploration in representation learning on unlabeled dialogues. We believe DSE can serve as a drop-in replacement of the dialogue representation model (e.g., the text encoder) for a wide range of dialogue systems.

## References

* L. El Asri, H. Schulz, S. Sharma, J. Zumer, J. Harris, E. Fine, R. Mehrotra, and K. Suleman (2017)Frames: a corpus for adding memory to goal-oriented dialogue systems. arXiv preprint arXiv:1704.00057. Cited by: SS3.
* P. Bachman, R. D. Hjelm, and W. Buchwalter (2019)Learning representations by maximizing mutual information across views. arXiv preprint arXiv:1906.00910. Cited by: SS1, SS2.
* S. Bao, H. He, F. Wang, H. Wu, and H. Wang (2019)Plato: pre-trained dialogue generation model with discrete latent variable. arXiv preprint arXiv:1910.07931. Cited by: SS1, SS2.
* S. R. Bowman, G. Angeli, C. Potts, and C. D. Manning (2015)A large annotated corpus for learning natural language inference. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, Lisbon, Portugal, pp. 632-642. External Links: Link Cited by: SS1, SS2.
* P. Budzianowski, T. Wen, B. Tseng, I. Casanueva, S. Ultes, O. Ramadan, and M. Gasic (2018)MultiwoZ-a large-scale multi-domain wizard-of-oz dataset for task-oriented dialogue modelling. arXiv preprint arXiv:1810.00278. Cited by: SS1, SS2.
* B. Byrne, K. Krishnamoorthi, C. A. Sankar, A. Neelakantan, D. Duckworth, S. Yavuz, B. Goodrich, A. Dubey, A. Cedilnik, and Kyu-Young Kim (2019)Taskmaster-1: toward a realistic and diverse dialog dataset. arXiv preprint arXiv:1909.05358. Cited by: SS1, SS2.
* I. Casanueva, T. Temcinas, D. Gerz, M. Henderson, and I. Vulic (2020)Efficient intent detection with dual sentence encoders. arXiv preprint arXiv:2003.04807. Cited by: SS1, SS2.
* D. Cer, Y. Yang, S. Kong, N. Hua, N. Limtiaco, R. St John, N. Constant, M. Guajardo-Cespedes, S. Yuan, C. Tar, et al. (2018)Universal sentence encoder. arXiv preprint arXiv:1803.11175. Cited by: SS1, SS2.
* Ting Chen, S. Kornblith, M. Norouzi, and G. Hinton (2020)A simple framework for contrastive learning of visual representations. In International conference on machine learning, pp. 1597-1607. Cited by: SS1, SS2.
* Alice Coucke, Alaa Saade, Adrien Ball, Theodore Bluche, Alexandre Caulier, David Leroy, C. Doumorno, Thibautl Gisselbrecht, Francesco Caltagirone, Thibaut Lavril, et al. (2018)Snips voice platform: an embedded spoken language understanding system for private-by-design voice interfaces. arXiv preprint arXiv:1805.10190. Cited by: SS1, SS2.
* J. Devlin, M. Chang, K. Lee, and K. Toutanova (2018)Bert: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805. Cited by: SS1, SS2.
* Mihail Eric and C. D. Manning (2017)Key-value retrieval networks for task-oriented dialogue. arXiv preprint arXiv:1705.05414. Cited by: SS1, SS2.
* Hongchao Fang, S. Wang, M. Zhou, J. Ding, and P. Xie (2020)CFT: contrastive self-supervised learning for language understanding. arXiv preprint arXiv:2005.12766. Cited by: SS1, SS2.
* Tianyu Gao, X. Yao, and D. Chen (2021)SimCSE: simple contrastive learning of sentence embeddings. arXiv preprint arXiv:2104.08821. Cited by: SS1, SS2.
* J. M. Giorgi, O. Nitski, G. D. Bader, and Bo Wang (2020)DecLUTr: deep contrastive learning for unsupervised textual representations. arXiv preprint arXiv:2006.03659. Cited by: SS1, SS2.
* K. He, H. Fan, Y. Wu, S. Xie, and R. Girshick (2020)Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9729-9738. Cited by: SS1, SS2.
* Matthew Henderson, I. Casanueva, N. Mrksic, P. Su, T. Wen, and I. Vulic (2019)Convert: efficient and accurate conversational representations from transformers. arXiv preprint arXiv:1911.03688.

 

Matthew Henderson, Blaise Thomson, and Jason D Williams. 2014. The second dialog state tracking challenge. In _Proceedings of the 15th annual meeting of the special interest group on discourse and dialogue (SIGDIAL)_, pages 263-272.
* Henderson et al. [2019b] Matthew Henderson, Ivan Vulic, Daniela Gerz, Inigo Casanueva, Pawel Budzianowski, Sam Coope, Georgios Spithourakis, Tsung-Hsien Wen, Nikola Mrksic, and Pei-Hao Su. 2019b. Training neural response selection for task-oriented dialogue systems. _arXiv preprint arXiv:1906.01543_.
* Hjelm et al. [2018] R Devon Hjelm, Alex Fedorov, Samuel Lavoie-Marchildon, Karan Grewal, Phil Bachman, Adam Trischler, and Yoshua Bengio. 2018. Learning deep representations by mutual information estimation and maximization. _arXiv preprint arXiv:1808.06670_.
* Hou et al. [2020] Yutai Hou, Wanxiang Che, Yongkui Lai, Zhihan Zhou, Yijia Liu, Han Liu, and Ting Liu. 2020. Few-shot slot tagging with collapsed dependency transfer and label-enhanced task-adaptive projection network. _arXiv preprint arXiv:2006.05702_.
* Kingma and Ba [2014] Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. _arXiv preprint arXiv:1412.6980_.
* Krone et al. [2020] Jason Krone, Yi Zhang, and Mona Diab. 2020. Learning to classify intents and slot labels given a handful of examples. _arXiv preprint arXiv:2004.10793_.
* Larson et al. [2019] Stefan Larson, Anish Mahendran, Joseph J Peper, Christopher Clarke, Andrew Lee, Parker Hill, Jonathan K Kummerfeld, Kevin Leach, Michael A Laurenzano, Lingjia Tang, et al. 2019. An evaluation dataset for intent classification and out-of-scope prediction. _arXiv preprint arXiv:1909.02027_.
* Lee et al. [2020] Kubok Lee, Yuan Zhu, Kihyuk Sohn, Chun-Liang Li, Jinwoo Shin, and Honglak Lee. 2020. I-mix: A domain-agnostic strategy for contrastive representation learning. _arXiv preprint arXiv:2010.08887_.
* Lee et al. [2019] Sungjin Lee, Hannes Schulz, Adam Atkinson, Jianfeng Gao, Kaheer Suleman, Layla El Asri, Mahmoud Adada, Minlie Huang, Shikhar Sharma, Wendy Tay, and Xiujun Li. 2019. Multi-domain task-completion dialog challenge. In _Dialog System Technology Challenges 8_.
* Li et al. [2018] Xiujun Li, Sarah Panda, JJ (Jingjing) Liu, and Jianfeng Gao. 2018. Microsoft dialogue challenge: Building end-to-end task-completion dialogue systems. In _SLT 2018_.
* Liu et al. [2019a] Xingkun Liu, Arash Eshghi, Pawel Swietojanski, and Verena Rieser. 2019a. Benchmarking natural language understanding services for building conversational agents.
* Liu et al. [2019b] Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019b. Roberta: A robustly optimized bert pretraining approach. _arXiv preprint arXiv:1907.11692_.
* Logeswaran and Lee [2018] Lajanugen Logeswaran and Honglak Lee. 2018. An efficient framework for learning sentence representations. _arXiv preprint arXiv:1803.02893_.
* Lowe et al. [2017] Ryan Lowe, Nissam Pow, Iulian Vlad Serban, Laurent Charlin, Chia-Wei Liu, and Joelle Pineau. 2017. Training end-to-end dialogue systems with the ubuntu dialogue corpus. _Dialogue & Discourse_, 8(1):31-65.
* Mrksic et al. [2016] Nikola Mrksic, Diarmuid O Seaghdha, Tsung-Hsien Wen, Blaise Thomson, and Steve Young. 2016. Neural belief tracker: Data-driven dialogue state tracking. _arXiv preprint arXiv:1606.03777_.
* Peng et al. [2020] Baolin Peng, Chengquang Zhu, Chunyuan Li, Xiujun Li, Jinchao Li, Michael Zeng, and Jianfeng Gao. 2020. Few-shot natural language generation for task-oriented dialog. _arXiv preprint arXiv:2002.12328_.
* Radford et al. [2019] Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners.
* Rastogi et al. [2020] Abhinav Rastogi, Xiaoxue Zang, Srinivas Sunkara, Raghaw Gupta, and Pranav Khatian. 2020. Towards scalable multi-domain conversational agents: The schema-guided dialogue dataset. In _Proceedings of the AAAI Conference on Artificial Intelligence_, volume 34, pages 8689-8696.
* Reimers and Gurevych [2019] Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks. _arXiv preprint arXiv:1908.10084_.
* Sanh et al. [2019] Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019. DistilBERT, a distilled version of bert: smaller, faster, cheaper and lighter. _arXiv preprint arXiv:1910.01108_.
* Shah et al. [2018] Pararth Shah, Dilek Hakkani-Tur, Bing Liu, and Gokhan Tur. 2018. Bootstrapping a neural conversational agent with dialogue self-play, crowdsourcing and on-line reinforcement learning. In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 3 (Industry Papers)_, pages 41-51.
* Snell et al. [2017] Jake Snell, Kevin Swersky, and Richard S Zemel. 2017. Prototypical networks for few-shot learning. _arXiv preprint arXiv:1703.05175_.
* Srivastava et al. [2014] Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: a simple way to prevent neural networks from overfitting. _The journal of machine learning research_, 15(1):1929-1958.
* Vinyals et al. [2016] Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Daan Wierstra, et al. 2016. Matching networks for one shot learning. _Advances in neural information processing systems_, 29:3630-3638 

Mengting Wan and Julian McAuley. 2016. Modeling ambiguity, subjectivity, and diverging viewpoints in opinion question answering systems. In _2016 IEEE 16th international conference on data mining (ICDM)_, pages 489-498. IEEE.
* Welleck et al. (2018) Sean Welleck, Jason Weston, Arthur Szlam, and Kyunghyun Cho. 2018. Dialogue natural language inference. _arXiv preprint arXiv:1811.00671_.
* Wen et al. (2017) Tsung-Hsien Wen, David Vandyke, Nikola Mrksic, Milica Gasic, Lina M. Rojas-Barahona, Pei-Hao Su, Stefan Ultes, and Steve Young. 2017. A network-based end-to-end trainable task-oriented dialogue system. In _Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers_, pages 438-449, Valencia, Spain. Association for Computational Linguistics.
* Williams et al. (2018) Adina Williams, Nikita Nangia, and Samuel Bowman. 2018. A broad-coverage challenge corpus for sentence understanding through inference. In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume I (Long Papers)_, pages 1112-1122, New Orleans, Louisiana. Association for Computational Linguistics.
* Wolf et al. (2019) Thomas Wolf, Victor Sanh, Julien Chaumond, and Clement Delangue. 2019. Transfertransfo: A transfer learning approach for neural network based conversational agents. _arXiv preprint arXiv:1901.08149_.
* Wu et al. (2020a) Chien-Sheng Wu, Steven Hoi, Richard Socher, and Caiming Xiong. 2020a. Tod-bert: pre-trained natural language understanding for task-oriented dialogue. _arXiv preprint arXiv:2004.06871_.
* Wu et al. (2020b) Zhuofeng Wu, Sinong Wang, Jiatao Gu, Madian Khabsa, Fei Sun, and Hao Ma. 2020b. Clear: Contrastive learning for sentence representation. _arXiv preprint arXiv:2012.15466_.
* Yu et al. (2021) Dian Yu, Luheng He, Yuan Zhang, Xinya Du, Panupong Pasupat, and Qi Li. 2021. Few-shot intent classification and slot filling with retrieved examples. _arXiv preprint arXiv:2104.05763_.
* Zhang et al. (2021a) Dejiao Zhang, Shang-Wen Li, Wei Xiao, Henghui Zhu, Ramesh Nallapati, Andrew O Arnold, and Bing Xiang. 2021a. Pairwise supervised contrastive learning of sentence representations. _arXiv preprint arXiv:2109.05424_.
* Zhang et al. (2021b) Dejiao Zhang, Wei Xiao, Henghui Zhu, Xiaofei Ma, and Andrew O Arnold. 2021b. Virtual augmentation supported contrastive learning of sentence representations. _arXiv preprint arXiv:2110.08552_.
* Zhang et al. (2019) Yizhe Zhang, Siqi Sun, Michel Galley, Yen-Chun Chen, Chris Brockett, Xiang Gao, Jianfeng Gao, Jingjing Liu, and Bill Dolan. 2019. Dialogpt: Large-scale generative pre-training for conversational response generation. _arXiv preprint arXiv:1911.00536_.

## Appendix A Pre-train

In this section, we present the training data, implementation details, and training stability of model pre-training.

### Data

We utilize the corpus collected by TOD-BERT Wu et al. (2020a) to construct positive pairs. This dataset is the combination of \(9\) publicly available task-oriented datasets: MetaLWOZ Lee et al. (2019), Schema Rastogi et al. (2020), Taskmaster Byrne et al. (2019), MWOZ Budzianowski et al. (2018), MSR-E2E Li et al. (2018), SMD Eric and Manning (2017), Frames Asri et al. (2017), WOZ Mrksic et al. (2016), CamRest676 Wen et al. (2017). The combined dataset contains 100707 dialogues with 1388152 utterances over 60 domains. We filter out sentences with less or equal to 3 words and end up with 892835 consecutive utterances (for DSE) and 879185 unique sentences (for DSE-dropout). Note that, the training data of SimCSE-unsup consists of 1 million sentences from Wikipedia. That says, on the one hand, we use the same dataset as TOD-BERT but with our proposed data construction. On the other hand, we use a similar number of training samples as SimCSE-unsup. We believe such data construction makes the comparisons fair enough.

### Hyperparameters

We add a contrastive head after the Transformer model and use the outputs of the contrastive head to perform contrastive learning. We use a two-layer MLP with size (\(d\times d,d\times 128\)) as the contrastive head. We use Adam Kingma and Ba (2014) with a batch size of \(1024\) and a constant learning rate as the optimizer. We set the learning rate for contrastive head as \(3e-4\) and the learning rate for the Transformer model as \(3e-6\). The temperature hyperparameter \(\tau\) is set as 0.05. We train the model for \(15\) epochs (see Appendix A.3 for more details) and save the model at the end for evaluation. We use the same hyperparameters across all the experiments for BERTbase, RoBERTbase, and DistilBERTbase models. For BERlarge and RoBERTlanguage, we change the batch size to \(512\) to fit it into the GPUs. Pretraining of the DistilBERTbase, BERTbase, and BERTlarge model respectively takes \(3\), \(4\), and \(13\) hours on \(8\) NVIDIA@ V\(100\) GPUs5.

 



### Training Stability

In this section, we analyze the model's stability in terms of training steps when training with different type of positive pairs. We compare two data construction methods: consecutive utterances (DSE) and dropout (DSE-dropout). We first train each model for \(15\) epochs, save the checkpoint at the end of each epoch and evaluate each checkpoint with similarity-based methods. Figure 3 shows the two model's average performances on intent classification, out-of-scope detection, utterance-level response selection and dialogue-level response selection.

This result further illustrates the effectiveness of using consecutive utterances as positive pairs for learning dialogue representation. As shown in the figure, DSE's performance on all the tasks consistently improves during the training process, while DSE-dropout achieves the best performance at the first epoch and significantly loses performance afterwards. Besides, DSE's performance is less sensitive to the training steps. It achieves stable performance after about 5 epochs. In contrast, DSE-dropout's performance drops dramatically during the training process, yet it never surpasses DSE's performance. Therefore, we report DSE-dropout's performance at the first epoch in all the tables.

## Appendix B Evaluation Setup

In this section, we present evaluation details and introduction to the evaluation dataset. Throughout this paper, we use _cosine similarity_ as the similarity metric and _mean pooling of token embeddings_ as the sentence representation. For baseline models, we report the better results of using its default setting (e.g., last hidden state of the [CLS] token as sentence embedding for SimCSE) and mean pooling.

### Hyperparameters

We use the same hyperparameters for all the models. For similarity-based methods, the only hyperparameter is the max sequence length, we empirically choose a number that can fit at least \(99\%\) of the samples. We respectively set it as \(64\), \(64\), \(128\), and \(128\) for intent classification, out-of-scope detection, utterance-level response selection and dialogue-level response selection. Hyperparameters for fine-tune evaluations as listed as follows:

Intent ClassificationWe fine-tune all the models for \(50\) epochs with a batch size of \(16\) and learning rate of \(3e{}\)-\(05\). We evaluate the model on the few-shot validation set after every \(10\) steps. Early stopping is applied based on the model's validation results. The max sequence length is set as \(64\) and the dropout at the classification layer is set as \(0.1\).

Utterance-level Response SelectionIn this task, we set the max sequence length as \(128\) and batch size as \(100\). Other hyperparameters are same as those in Intent Classification. We use the original SimCLR loss Chen et al. (2020) to optimize the model.

Dialogue Action PredictionIn this task, we finetune all the models for \(100\) epochs with a batch size of \(32\) and learning rate of \(5e{}\)-\(05\). We evaluate the model on the few-shot validation set after every \(30\) steps. Early stopping is also applied. The max sequence length is set as \(32\) since we find shorter inputs leads to much better performance for all the models. We truncate sentences from the head to keep the most recent dialogue utterances as model input. We set the dropout at the classification layer as \(0.2\).

### Datasets

Intent ClassificationWe use 4 popular publicly available datasets: Clinc150 Larson et al. (2019) with \(150\) categories and \(4500\) test sample, Bank77 Casanueva et al. (2020) with \(77\) categories and \(3080\) test sample, Snips Coucke et al. (2018) with \(7\) categories and \(1447\) test sample, and Hwu64 Liu et al. (2019) with \(64\) categories and \(3853\) test sample. To apply the Clinc150 dataset

Figure 3: DSE and DSE–dropout’s performance on each task at every epoch. The dashed lines represent the results of DSE-dropout.



in intent classification, we remove all the out-of-scope samples. We also use an internal dataset named Appen, whose texts are transcribed from customer recording. This dataset contains \(30\) categories and \(310\) test samples. There are two versions of each sentence. One is transcribed by Automatic Speech Recognition (ASR), which includes some ASR noise (e.g., transcribe errors). The other is transcribed by human annotator. We refer them respectively as Appen-A and Appen-H.

Out-of-scope DetectionWe use the entire Clinc150 dataset, which contains \(150\) in-scope intents and one out-of-scope intent. There are \(5500\) test samples in total (\(4500\) in-scope and \(1000\) out-of-scope).

Utterance-level Response SelectionWe use the AmazonQA dataset Wan and McAuley (2016), which contains 5334606 question-answer pairs about different products. Following Henderson et al. (2019a), we randomly select 300K pairs for model evaluation.

Dialogue-Level Response SelectionWe use the DSTC7-Ubuntu dataset Lowe et al. (2017), which contains conversations about the Ubuntu system. Each query of this dataset comes together with one ground-truth response and \(100\) candidate responses. We combine the validation and test sets together for evaluation, which results in \(6000\) evaluation samples.

Dialogue Action PredictionWe use the DSTC2 Henderson et al. (2014) and GSIM Shah et al. (2018) dataset processed by Wu et al. (2020a). These two datasets respectively contains \(13/19\) actions and \(1117\)/\(1039\) test samples. The average number of samples in \(10\)-shot and \(20\)-shot training is \(79\) and \(149\) for DSTC2; \(60\) and \(120\) for GSIM.

## Appendix C Results of BERTbase

In this section, we present other evaluation results on the BERTbase model, including 1-shot and 5-shot fine-tune on intent classification (Table 7), 5-shot similarity-based out-of-scope detection (Table 8), and 500-shot and 1000-shot fine-tune on AmazonQA response selection (Table 9).

 



\begin{table}
\begin{tabular}{l l c c c c c c c} \hline \hline  & BERT\({}_{base}\) & **Clinc150** & **Bank77** & **Snips** & **Hwu64** & **Appen-A** & **Appen-H** & **Ave.** \\ \hline  & SimCSE-sup\({}^{\clubsuit}\) & 48.30 & 35.29 & 49.77 & 34.09 & 30.16 & 38.42 & 39.34 \\  & PairSupCon\({}^{\clubsuit}\) & 50.33 & 37.59 & 52.53 & 34.21 & **33.58** & 41.65 & 41.65 \\  & DSE-dianni\({}^{\clubsuit}\) (ours) & 43.07 & 38.02 & 46.57 & 30.98 & 29.39 & 36.77 & 37.47 \\ \multirow{-7}{*}{\(\downarrow\)} & BERT\({}^{\diamond}\) & 37.01 & 24.28 & 52.05 & 26.36 & 17.87 & 19.71 & 29.55 \\  & SimCSE-unsup\({}^{\diamond}\) & 42.72 & 33.56 & 47.13 & 30.19 & 24.00 & 32.68 & 35.05 \\  & TOD-BERT\({}^{\diamond}\) & 39.48 & 26.12 & 46.13 & 26.81 & 13.45 & 23.26 & 29.21 \\  & DSE-dropout\({}^{\diamond}\) (ours) & 41.89 & 30.10 & 44.46 & 28.06 & 18.68 & 20.48 & 30.61 \\  & DSE (ours) & **55.67** & **38.10** & **76.67** & **37.93** & 32.68 & **45.03** & **46.68** \\  & SimCSE-sup\({}^{\clubsuit}\) & 85.49 & 70.16 & 88.90 & 68.86 & 61.71 & 73.29 & 74.74 \\  & PairSupCon\({}^{\clubsuit}\) & 85.15 & 71.00 & 86.01 & 68.12 & **63.94** & 73.68 & 74.65 \\  & DSE-dianni\({}^{\clubsuit}\) (ours) & 81.87 & 69.33 & 82.49 & 64.40 & 58.90 & 68.52 & 70.92 \\ \multirow{-7}{*}{\(\downarrow\)} & BERT\({}^{\diamond}\) & 84.00 & 68.51 & 85.72 & 64.79 & 55.00 & 65.52 & 70.59 \\  & SimCSE-unsup\({}^{\diamond}\) & 83.35 & 70.08 & 86.82 & 65.61 & 59.68 & 70.03 & 72.60 \\  & TOD-BERT\({}^{\diamond}\) & 83.15 & 65.29 & 88.49 & 66.29 & 56.32 & 67.13 & 71.11 \\  & DSE-dropout\({}^{\diamond}\) (ours) & 84.14 & 69.74 & 87.39 & 66.47 & 57.74 & 68.13 & 72.27 \\  & DSE\({}^{\diamond}\) (ours) & **86.67** & **71.52** & **92.56** & **70.71** & 63.71 & **75.10** & **76.71** \\ \hline \hline \end{tabular}
\end{table}
Table 7: Results of fine-tuning all the models for 1-shot and 5-shot Intent Classification for BERT\({}_{\text{base}}\) models. \(\clubsuit\): Supervised models

\begin{table}
\begin{tabular}{c l c c c c c} \hline \hline  & \multicolumn{2}{c}{**BERT\({}_{base}\)**} & **Accuracy** & **In-Accuracy** & **OOS-Accuracy** & **OOS-Recall** & **Ave.** \\ \hline  & SimCSE-sup\({}^{\clubsuit}\) & 59.65 & 69.04 & 80.35 & 17.40 & 56.61 \\  & PairSupCon\({}^{\clubsuit}\) & 68.37 & 71.03 & 85.62 & 56.43 & 70.36 \\  & DSE-daniI\({}^{\clubsuit}\) (ours) & 56.22 & 56.92 & 83.38 & 53.07 & 62.40 \\ \multirow{-7}{*}{\(\downarrow\)} & BERT\({}^{\diamond}\) & 53.92 & 57.25 & 81.94 & 38.95 & 58.01 \\  & SimCSE-unsup\({}^{\diamond}\) & 55.68 & 63.06 & 79.58 & 22.47 & 55.20 \\  & DialoPT\({}^{\diamond}\) & 58.53 & 63.25 & 82.62 & 37.25 & 60.41 \\  & TOD-BERT\({}^{\diamond}\) & 53.49 & 56.12 & 81.75 & 41.64 & 58.25 \\  & DSE-dropout\({}^{\diamond}\) (ours) & 64.21 & 67.39 & 83.59 & 49.92 & 66.28 \\  & DSE\({}^{\diamond}\) (ours) & **72.62** & **74.77** & **87.16** & **62.95** & **74.38** \\  & SimCSE-sup\({}^{\clubsuit}\) & 42.50 & 34.95 & 48.26 & 76.50 & 50.55 \\  & PairSupCon\({}^{\clubsuit}\) & 56.19 & 47.53 & 62.79 & **95.17** & 65.42 \\  & DSE-dianli\({}^{\clubsuit}\) (ours) & 46.87 & 37.75 & 59.96 & 87.92 & 58.13 \\ \multirow{-7}{*}{\(\downarrow\)} & BERT\({}^{\diamond}\) & 47.79 & 38.40 & 57.86 & 90.07 & 58.53 \\  & SimCSE-unsup\({}^{\diamond}\) & 45.13 & 36.82 & 52.01 & 82.51 & 54.12 \\  & DialoGPT\({}^{\diamond}\) & 49.64 & 40.26 & 57.49 & 91.81 & 59.80 \\  & TOD-BERT\({}^{\diamond}\) & 47.88 & 38.46 & 58.43 & 90.31 & 58.77 \\  & DSE-dropout\({}^{\diamond}\) (ours) & 54.45 & 46.00 & 61.31 & 92.50 & 63.56 \\  & DSE\({}^{\diamond}\) (ours) & **59,20** & **51.35** & **64.79** & 94.53 & **67.47** \\ \hline \hline \end{tabular}
\end{table}
Table 8: Results on similarity-based 5-shot out-of-scope detection on Clinc150 dataset. The out-of-scope threshold is respectively set as _mean_ (m) and _mean-std_ (m-d) of each sample’s similarity with its closest category. See Sec. 4.2.2 for details. All the models use BERT\({}_{\text{base}}\) as the backbone model. \(\clubsuit\): Supervised models. \(\diamond\): Unsupervised models.





\begin{table}
\begin{tabular}{l l c c c c c} \hline \hline  & & **Clinc150** & **Bank77** & **Snips** & **Hwu64** & **Ave.** \\ \hline  & \(\textbf{BERT_{large}}\) & 31.71 & 20.47 & 54.31 & 25.24 & 32.93 \\  & \(\textbf{BERT_{large-DSE}}\) & 65.57 & 45.45 & 78.52 & 46.37 & 58.97 \\ \cline{2-7}  & \(\textbf{RoBERT_{base}}\) & 34.58 & 20.58 & 52.25 & 24.24 & 32.91 \\  & \(\textbf{RoBERT_{base-DSE}}\) & 66.05 & 45.01 & 80.58 & 43.98 & 58.90 \\ \cline{2-7}  & \(\textbf{RoBERT_{large}}\) & 35.72 & 20.84 & 54.80 & 23.57 & 33.73 \\  & \(\textbf{RoBERT_{large-DSE}}\) & 69.23 & 45.42 & 73.72 & 44.29 & 58.16 \\ \cline{2-7}  & \(\textbf{DistilBERT_{base}}\) & 39.48 & 23.96 & 63.00 & 30.25 & 39.17 \\  & \(\textbf{DistilBERT_{base-DSE}}\) & 60.47 & 43.52 & 76.38 & 44.63 & 56.25 \\ \cline{2-7}  & \(\textbf{BERT_{large}}\) & 46.78 & 33.53 & 70.89 & 37.06 & 47.06 \\  & \(\textbf{BERT_{large-DSE}}\) & 80.40 & 64.49 & 89.08 & 63.00 & 74.24 \\ \cline{2-7}  & \(\textbf{RoBERT_{base}}\) & 53.58 & 32.40 & 68.90 & 34.98 & 47.46 \\  & \(\textbf{RoBERT_{base-DSE}}\) & 81.73 & 64.92 & 89.67 & 62.81 & 74.78 \\ \cline{2-7}  & \(\textbf{RoBERT_{large}}\) & 55.43 & 33.25 & 78.01 & 36.25 & 50.73 \\  & \(\textbf{RoBERT_{large-DSE}}\) & 82.52 & 62.93 & 86.64 & 61.04 & 73.28 \\ \cline{2-7}  & \(\textbf{DistilBERT_{base}}\) & 61.00 & 39.45 & 78.90 & 45.00 & 56.08 \\  & \(\textbf{DistilBERT_{base}}\) & 77.16 & 60.39 & 86.48 & 60.81 & 71.21 \\ \hline \hline \end{tabular}
\end{table}
Table 10: Results on similarity-based 1-shot and 5-shot Intent Classification with different model as the backbone. DSE leads to significant and consistent performance boost for all the models.

\begin{table}
\begin{tabular}{l c c c c c c} \hline \hline  & & **AmazonQA** & \multicolumn{3}{c}{**DSTC7-Ubuntu**} \\ \cline{2-7} \multirow{2}{*}{\(\mathbf{BERT_{large}}\)} & **Top-1 Acc.** & **Top-3 Acc.** & **Top-10 Acc.** & **Top-1 Acc.** & **Top-3 Acc.** & **Top-10 Acc.** \\ \hline \(\textbf{BERT_{large}}\) & 27.97 & 41.30 & 57.04 & 6.10 & 11.08 & 22.31 \\ \(\textbf{BERT_{large-DSE}}\) & 59.63 & 73.46 & 84.12 & 16.40 & 24.56 & 36.51 \\ \cline{2-7} \(\textbf{RoBERT_{base}}\) & 19.60 & 29.67 & 44.70 & 4.86 & 9.80 & 20.70 \\ \(\textbf{RoBERT_{base-DSE}}\) & 55.69 & 70.01 & 81.68 & 15.86 & 24.25 & 37.38 \\ \cline{2-7} \(\textbf{RoBERT_{large}}\) & 26.68 & 37.73 & 51.70 & 7.65 & 14.50 & 26.10 \\ \(\textbf{RoBERT_{large-DSE}}\) & 58.13 & 71.65 & 82.20 & 18.66 & 27.70 & 40.93 \\ \cline{2-7} \(\textbf{DistilBERT_{base}}\) & 31.73 & 46.47 & 63.23 & 6.65 & 12.46 & 24.98 \\ \(\textbf{DistilBERT_{base}}\)**-DSE** & 56.36 & 70.11 & 81.51 & 14.56 & 22.78 & 35.63 \\ \hline \hline \end{tabular}
\end{table}
Table 11: Results on 0-shot response selection on AmazonQA (utterance-level) and DSTC7–Ubuntu (dialogue-level). DSE leads to significant and consistent performance improvements on all the models.