Venue: NAACL 2021

# Adding Chit-Chat to Enhance Task-Oriented Dialogues

Kai Sun 1, Seungwhan Moon2, Paul Crook2, Stephen Roller3, Becka Silvert2, Bing Liu2, Zhiguang Wang2, Honglei Liu2, Eunjoon Cho2, and Claire Cardie1

Correspond University

Footnote 1: Work done as a research intern at Facebook. The code and data are available at [https://github.com/facebookresearch/accentor](https://github.com/facebookresearch/accentor).

###### Abstract

Existing dialogue corpora and models are typically designed under two disjoint motives: while task-oriented systems focus on achieving functional goals (e.g., booking hotels), open-domain chatbots aim at making socially engaging conversations. In this work, we propose to integrate both types of systems by Adding Chit-Chat to ENhance Task-ORiented dialogues (accentor), with the goal of making virtual assistant conversations more engaging and interactive. Specifically, we propose a Human \(\leftrightarrow\) AI collaborative data collection approach for generating diverse chit-chat responses to augment task-oriented dialogues with minimal annotation effort. We then present our new chit-chat-based annotations to \(23.8\)K dialogues from two popular task-oriented datasets (Schema-Guided Dialogue and MultiWOZ 2.1) and demonstrate their advantage over the originals via human evaluation. Lastly, we propose three new models for adding chit-chat to task-oriented dialogues, explicitly trained to predict user goals _and_ to generate contextually relevant chit-chat responses. Automatic and human evaluations show that, compared with the state-of-the-art task-oriented baseline, our models can codeword between task and chit-chat to be more engaging, interesting, knowledgeable, and humallike, while maintaining competitive task performance.

## 1 Introduction

With modeling innovations, increasing computing power, and a growing number of datasets, recent years have witnessed significant improvements in the performance of both task-oriented dialogue systems and chit-chat systems Adiwardana et al. (2020); Roller et al. (2020); Hosseini-Asl et al. (2020); Peng et al. (2020). Most research on dialogue systems focuses on a particular type of dialogue system. Work on task-oriented dialogue systems typically aims to track user goals with higher accuracy to better achieve functional goals Rastogi et al. (2020) with the sacrifice of not paying explicit attention to user experience, such as making the conversation more engaging, while the latter is usually the target of research on chit-chat systems Li et al. (2019). In this work, we step forward and propose to integrate both types of systems by **A**dding **C**hit-**C**hat to **E**N**hance **T**ask-**O**R**iented dialogues (accentor), aiming to have a virtual assistant capable not only of performing various complex tasks such as checking the weather, booking hotels, and finding restaurants, but also incorporating casual and contextually relevant chit-chat. We hypothesize that the added chit-chat can make the assistant appear more social, personable, and engaging, without being misleading or inappropriate, compared with existing task-oriented dialogue systems.

To show the feasibility of acccentor and gather supervisory data for follow-up research, we propose a Human\(\leftrightarrow\)AI collaborative data construction approach that can effectively add suitable chi-chat to the beginning or end of system responses in existing task-oriented dialogue datasets. Specifically, we first generate chit-chat candidates for augmentation using off-the-shelf pre-trained language models and open-domain chatbots (Section 2.1). Next, we automatically filter out candidates that

Figure 1: A sample task-oriented dialogue snippet augmented by _chit-chat_.



are unlikely to be of good quality using a filter model (Section 2.2). Finally, human annotators label each of the remaining candidates as good or bad, with justifications (Section 2.3). We augment the Schema-Guided Dialogue (SGD) Rastogi et al. (2020) and MultiWOZ 2.1 Eric et al. (2020) corpora using the proposed approach. (See Figure 1 or Appendix A.4 for examples.) We employ ACUTE-Eval Li et al. (2019) to compare the augmented versions with the originals along four axes: engagingness, interestingness, knowledge, and humanness. We find that the augmented dialogues are consistently preferred by human judges across the four axes for both datasets (Section 4.1).

In addition, we propose and evaluate three models for adding chit-chat to task-oriented dialogues, including an end-to-end model and two code-switcher models built upon off-the-shelf task-oriented and chit-chat systems (Section 3). Compared with the baseline model trained with the original unaugmented data, our models trained with the augmented version can generate significantly higher-rated responses in terms of human preference while maintaining competitive task performance in goal tracking accuracy and action decision F1 (Section 4.2).

**Our main contributions** are: we propose (1) a data augmentation approach for generating diverse chit-chat supervisory data for task-oriented dialogues, leveraging pre-trained generative models and a custom filter model to minimize human annotation effort; (2) new versions of the popular task-oriented datasets, SGD and MultiWOZ 2.1, with newly added chit-chat annotations to \(23.8\)K dialogues; and (3) three integrated chit-chat and task-oriented neural dialogue models for the above, substantially outperforming the state-of-the-art approach in terms of human evaluation of engagingness, interestingness, knowledge, and humanness. To our knowledge, we are the first to propose an annotated dataset and models that study explicit code-switching between full-stack task-oriented dialogues and free-form chit-chat responses.

## 2 Data Construction

In this section, we describe an approach to gather supervisory data for adding contextually relevant chit-chat to task-oriented dialogues. Our approach needs minimal annotation effort to augment suitable and diverse chit-chat add-ons that are not available in existing task-oriented datasets (Section 5.1). We primarily report results based on dialogues from the SGD dataset in this study, because it is the largest task-oriented dialogue dataset and is generally cleaner compared with most other task-oriented dialogue datasets. However, our approach is flexible and thus not limited to dialogues from a particular task-oriented dataset (Section 4.1). Figure 2 shows the overview of our approach.

### Candidate Generation

Given a task-oriented dialogue \(D=\{u_{1},s_{1},u_{2},s_{2},\ldots,u_{n},s_{n}\}\), where \(u_{1...n}\) and \(s_{1...n}\) represent user turns and system turns, respectively, we generate chit-chat candidates for augmenting \(s_{i}\) in two ways: (i) pass \(u_{1},s_{1},\dots,u_{i},s_{i}\) to an off-the-shelf pre-trained model (a language model or a chit-chat chatbot) and let the model add tokens to the end of \(s_{i}\); (ii) pass \(u_{1},s_{1},\dots,u_{i}\) to a pre-trained model and let the model generate a turn. We regard the output of (i) and (ii) as a chit-chat candidate to be appended and prepended to \(s_{i}\), respectively. If a chit-chat candidate consists of multiple sentences, we also regard each individual sentence as a chit-chat candidate. We run differently sized GPT-2 Radford et al. (2019) and BlenderBot Roller et al. (2020) with various decoding parameters as the pre-trained model and generate an average of \(175.5\) candidates for each of the dialogues from the SGD dataset. See Section A.1 for configuration details.

### Candidate Filtering

We examine the quality of the model-generated candidates from Section 2.1 by performing a pilot annotation ourselves on a small proportion of the

Figure 2: Data construction overview: (a) We generate diverse free-form chit-chat candidates using the state-of-the-art pre-trained generative models to augment original task-oriented dialogues, and (b) filter out bad candidates using the custom filter to minimize annotation effort. (c) Crowd workers annotate contextually relevant chit-chat augmentation with justifications.

 

candidates. The annotation results show that only about \({}^{1}\!/_{10}\) of the candidates are suitable. Therefore, instead of directly sending the candidates to crowd workers for annotation, we propose to build a filter model to automatically filter out candidates that are unlikely to be of good quality first to reduce potential annotation workload.

The filter is a hybrid model that consists of a RoBERTa-based binary classifier Liu et al. (2019) and a rule-based ranker. The classifier takes as input an augmented dialogue, in which we explicitly surround the added chit-chat candidate with a pair of special tokens to help the model locate the candidate. We train the classifier with \(1.7\text{K}\) candidates that are labeled as good/bad from the pilot annotation. The rule-based ranker ranks each candidate based on (i) the posterior probability output by the binary classifier, (ii) whether the candidate matches a list of bad patterns (e.g., containing an URL), (iii) the frequency of appearances of the candidate among all generated candidates, (iv) the similarity to the other candidates for the dialogue, and (v) the similarity to the system response being augmented. While (i) and (ii) directly help evaluate the quality of the candidate, (iii), (iv), and (v) additionally help create more variety (e.g., punishing high-frequency candidates such as "_You're welcome_"). We keep the top ten candidates for each of the dialogues. We present more details in Appendix A.2.

### Annotation

We ask annotators (crowd workers) to label each of the remaining candidates from Section 2.2 as _good_ or _bad_. Additionally, to guide the annotation process, improve the potential quality, and facilitate the candidate distribution analysis, we also ask annotators to choose from four justifications that we come up with based on our pilot annotation experience to support their annotations. Annotators can choose one, both, or neither of the following justifications for a _bad_ candidate:

* Inappropriate: The candidate does not fit into the context (e.g., repeating, unnatural), or it contradicts the context or the role of the assistant (Table 1). This category comprises most of the commonly found bad cases such as improper switching, providing opinions or comments that are incompatible with the context, and misusing verbal routine.
* Misleading: The candidate provides additional information that is false or cannot be verified immediately. For example, the underlined candidate in the two-turn dialogue "_U: I want to book a hotel room in San Diego with a check in on Thursday. A: There are over 10 hotels in San Diego. I would stay at Arlo NoMad if I were you._" should be marked as misleading because "_Arlo NoMad_" is newly introduced information, which the annotator would have to look up to verify that a hotel by this name exists in San Diego, even though the information may be true.

Annotators can choose one, both, or neither of the following justifications for a _good_ candidate:

* Social: The candidate keeps the conversation flowing smoothly by appropriately switching to relevant topics, asking casual follow up questions, or engaging in social pleasantries.

\begin{table}
\begin{tabular}{l l l l l} \hline \hline  & **Approportiate Behaviours** & **Examples** & **Inappropriate Behaviours** & **Examples** \\ \hline \multirow{3}{*}{Opinions} & Express general opinions about & “_“I force pasatius.”_ & Express strong personal opinions, & _“I love yous.”_ \\  & generic, impersonal, or & “_There’s a lot of fun snd to & or opinions on sensitive topics. & _“The President is an idiot.”_ \\  & non-sensitive topics. & & _“Their latest album wasn?_ & Express strong dispreferances, or preferences on personal or & _“I hated it, but you might like it.”_ \\ \cline{2-5} Preferences & Expenses preferences when & “_Their latest album wasn?_ & & _“I hated it, but you might_ \\  & making impersonal, or & as good.” & preferences on personal or & _like it.”_ \\  & non-sensitive recommendations. & “_‘Ther food is good.”_ & sensitive subjects. & _“Impute her I like here better.”_ \\ \hline \multirow{4}{*}{Physical Actions} & Use epistemic verbs to express express & ”_I hear it’s beautiful.”_ & Behave as though it could act physically, or perform tasks outside of its role. & _“I haven’t arrived there_ \\  & uncertainty or opinions, or refer through hears to actions that it & “_They say it states like chicken._ & Physically, or perform tasks outside of its role._ & _“I can drive you there.”_ \\  & may not perform. & & _“That sounds like a great trip”_ & Pretend to have experiences that & & _“We didn’s have that when I was a kid.”_ \\ \cline{2-5} Experiences & Refer to others’ experiences or persony experiences is it & “_They enjoyed reading that_ & it is incapable of. & _“My roommate used to eat there a lot.”_ \\  & capable of (e.g., reading). & _nonet.”_ & & _there a lot.”_ \\ \hline \multicolumn{5}{c}{**Who is the virtual assistant?** This digital assistant is more than just a bot that spins out facts. It has access to a wide range of information which can express not only as factual commentaries but also as opinions and preferences. However, it is not a person and should not pretend to have real experiences or be capable of physical actions. It should be personality and personlike, without appearing counterfeist.} \\ \hline \hline \end{tabular}
\end{table}
Table 1: The role of the virtual assistant and its appropriate/inappropriate behaviors with examples.

 

The design of this subcategory is inspired by the line of research that studies different social and discourse strategies in chit-chat dialogue systems (Yu et al., 2016).
* Useful: The candidate enhances the conversation by appropriately offering opinions, commentaries, or pertinent and truthful information. Truthfulness should be established by conversational context or real world knowledge. To reduce annotation workload, if annotators have to use external resources (e.g., Wikipedia, search engines, maps) to verify information, they are instructed to label the candidate as _misleading_ instead. The design of this subcategory is inspired by the line of work on knowledge-grounded dialogue systems that study contextual knowledge injections (Dinan et al., 2019).

We instruct annotators to evaluate each candidate independently as if it were the only augmentation for its associated dialogue. We discuss the additional dimension of complexity introduced by having multiple augmentations jointly in Section 4.1.

Annotation time per dialogue is \(243\)s. The Fleiss' Kappa among crowd workers is \(0.52\). We view the agreement score as reasonable since whether an added chit-chat candidate leads to improved quality of a conversation can be highly subjective in many scenarios. We denote our augmented version of the SGD dataset as acccentor-SGD and summarize the statistics in Table 2. We observe that the four provided justification categories provide adequate coverage of the justifications for most annotations. \(41.4\%\) of the candidates are good, showing the effectiveness of candidate filtering. An analysis based on linguistic features suggests that bad candidates are more personal and negative than good candidates. Specifically, \(40.0\%\) of bad candidates involve first-person pronouns, while the ratio is \(26.5\%\) for good candidates. \(81.7\%\) of good candidates have positive sentiment, measured by VADER, a lexicon and rule-based sentiment analysis tool (Hutto and Gilbert, 2014), while the ratio is \(73.0\%\) for bad candidates. Examples of the resulting dataset are presented in Appendix A.4.

## 3 Approaches

### Task Formulations

Since oracle information (i.e., oracle belief states and oracle action decisions) is not available in practical use and the SGD dataset does not have the associated database (i.e., a table of possible entities) released, we focus on exploring the end-to-end setting in which we generate _delexicalized_ task-oriented responses without using oracle information and database search results following Hosseini-Asl et al. (2020). Given dialogue history (i.e., previous turns) as context, the goal of the model for each system turn is to accurately generate belief states (i.e., a list of \((domain,slot,value)\) triplets), action decisions (i.e., a list of \((domain,action\_type,slot)\) triplets), and a corresponding system response that is functionally accurate _and_ socially engaging.

### Models

We re-implement SimpleTOD (Hosseini-Asl et al., 2020) as our main baseline model, which is a state

Figure 3: A diagram for the proposed code-switching models. Given the dialogue context (\(H_{t}\)) and the pre-generated task-oriented and chit-chat response candidates (\(T_{t},\tilde{C_{t}}\)), the **Arranger** learns the optimal code-switching sequences (discriminative), while the **Rewriter** outputs free-form paraphrases (generative).

 

of-the-art model in the end-to-end setting we explore. In addition, we propose an extension of SimpleTOD that incorporates chit-chat acts, as well as two new models (Arranger and Rewriter; Figure 3) that code-switch between chit-chat and task-oriented responses more explicitly.

SimpleTOD.It is a causal language model that models the joint probability over the concatenation of dialogue history \(H_{t}\), belief states \(B_{t}\), action decisions \(A_{t}\), and a task-oriented response \(T_{t}\) for each turn \(t\). During inference, the model takes as input \(H_{t}\) and generates \(B_{t}\), \(A_{t}\), and \(T_{t}\). We refer readers to Hosseini-Asl et al. (2020) for more details.

SimpleTOD+.We extend SimpleTOD by introducing to the construction of input sequences a special new dialogue action chit-chat and good chit-chat candidates during training. Specifically, let \(\mathbb{C}^{+}_{t}\) denote the set of good candidates for system turn \(t\). If \(\mathbb{C}^{+}_{t}\) is empty, we construct the same training sequence as SimpleTOD. Otherwise, for each \(C_{t}\in\mathbb{C}^{+}_{t}\) that is labeled as a candidate to be prepended (resp. appended) to the turn, we use the concatenation of \(H_{t}\), \(B_{t}\), [chit-chat], \(A_{t}\), \(C_{t}\), and \(T_{t}\) (resp. \(H_{t}\), \(B_{t}\), \(A_{t}\), [chit-chat], \(T_{t}\), and \(C_{t}\)) as a training sequence.

Arranger.This model arranges the output of an off-the-shelf task-oriented dialogue model and an off-the-shelf chit-chat model without intervening in the task. It outputs the belief states and action decisions generated by the task-oriented model without modification. To generate a response for each system turn \(t\), this model takes as input (i) dialogue history \(H_{t}\), (ii) a chit-chat response \(\tilde{C}_{t}\) generated by the chit-chat model based on \(H_{t}\), and (iii) a task-oriented response \(T_{t}\) generated by the task-oriented dialogue model based on \(H_{t}\). The model chooses one of the following as the response: \(\tilde{C}_{t}\) followed by \(T_{t}\), \(T_{t}\) followed by \(\tilde{C}_{t}\), and \(T_{t}\) only. Specifically, the model encodes the concatenation of \(H_{t}\) and each of these three responses by a RoBERTa encoder (Liu et al., 2019) and passes the resulting representations through a linear plus softmax layer to make the choice. To train the model, we form training instances by regarding each chit-chat candidate for turn \(t\) from the training set of accentor-SGD as \(\tilde{C}_{t}\) and the ground-truth task-oriented response as \(T_{t}\) and setting the target choice based on the label (i.e., good/bad) and position (i.e., beginning/end of the response) of the candidate.

Rewriter.This model rewrites the output of an off-the-shelf task-oriented dialogue model and an off-the-shelf chit-chat model. It directly outputs the task-oriented model's belief states without modification and generates action decisions and a system response by a causal language model. The causal language model differs from SimpleTOD+ in that it has two additional components \(T_{t}\) and \(\hat{C}_{t}\) added between \(H_{t}\) and \(B_{t}\) in each training sequence, where we form \(T_{t}\) and \(\hat{C}_{t}\) in the same way as we do for Arranger. During the inference stage, it takes as input \(H_{t}\), \(T_{t}\) output by the task-oriented dialogue model, \(\tilde{C}_{t}\) output by the chit-chat model, and \(B_{t}\) output by the task-oriented dialogue model, and generates action decisions and a system response. Note that since \(25.4\%\) of the annotated system turns in the training set of accentor-SGD have both good and bad chit-chat candidates, \(\mathbb{C}^{+}_{t}\) can be non-empty when \(\tilde{C}_{t}\) is a bad candidate, which enables the model to potentially generate a suitable chit-chat augmented response even if the output of the off-the-shelf chit-chat model is not good.

### Implementation Details

Unless specified otherwise, for causal language models, we use the \(12\)-layer GPT-2 (\(117\)M parameters) as the pre-trained language model (Radford et al., 2019) and fine-tune for ten epochs. We set the batch size to \(36\) and the learning rate to \(1\times 10^{-3}\). We employ the SimpleTOD baseline as the off-the-shelf task-oriented dialogue model for Arranger and Rewriter. We fine-tune a 90M parameter model (Shuster et al., 2020) on each of the good chit-chat candidates with the associated dialogue history as the context from the training set of accentor-SGD following hyperparameters employed by Roller et al. (2020) and employ the resulting model as the off-the-shelf chit-chat model in Arranger and Rewriter. We use RoBERTaBASE(Liu et al., 2019) as the pre-trained language model for Arranger and fine-tune for three epochs with a learning rate of \(2\times 10^{-5}\) and a batch size of \(24\).

## 4 Experiments and Discussions

### Data Evaluations

accentor-Sgd.We first evaluate accentor at the dataset level, aiming to answer two questions: Q1. Are task-oriented dialogues augmented with good chit-chat more preferred by human judges than the unaugmented? Q2. Does the answer to Q1 depend on how frequently we augment system 

responses with chit-chat? To answer these questions, we randomly sample \(100\) dialogues from acccentor-SGD, each having at least \(8\) turns and enough candidates labeled as good for augmenting over \(40\%\) of system responses so that we can compare the same task-oriented dialogue with different chit-chat injection frequencies that fall into each of the following four intervals: \((0.1,0.2]\), \((0.2,0.3]\), \((0.3,0.4]\), and \((0.4,1]\). Particularly, for the last interval, we augment all system responses that have chit-chat candidates labeled as good, while for the first three intervals, we only augment a randomly selected fraction to fit the interval. We employ ACUTE-Eval Li et al. (2019) for evaluation, whereby we ask human evaluators to make pairwise comparisons of complete dialogues over four axes: engagingness, interestingness, knowledge, and humanness. We provide the wording of the questions in Appendix A.3.

As shown in Figure 4, the chit-chat augmented dialogues from acccentor-SGD are more preferred by human judges than the originals over all ACUTE-Eval metrics, regardless of the injection frequency (all p-values \(<0.05\)). Among different injection frequency ranges, \((0.2,0.3]\) is the best. We offer three hypotheses to explain this finding: (i) \((0.2,0.3]\) best balances being engaging and not too talkative. (ii) There are inevitable annotation errors, and scenarios where whether a candidate is good or bad is subjective. A higher injection frequency means a higher chance of being affected by these factors. (iii) Since candidates are labeled independently, inter-candidate incompatibility may arise (e.g., expressing contradicted preferences), especially when we have a high injection frequency. Table 4 shows a real example to support our hypotheses. Specifically, 3 is labeled as good but is indeed not a suitable (or at least a questionable) candidate, supporting the hypothesis (ii). While 2 and 4 are good candidates when we evaluate them separately, they may be less preferred if we assess them jointly because they convey the same meaning: "_Long Shot is a good comedy._" Having them together may appear incompatible (i.e., repetition) or sound verbose to the user, supporting the hypothesis (i) and (iii).

#### accentor-MultiWOZ.

To investigate the flexibility of our data construction approach, we aug

Figure 4: Comparisons between SGD and acccentor-SGD with different injection frequencies at the dataset level using ACUTE-Eval.

\begin{table}
\begin{tabular}{l l l l l l l l l l l} \hline \hline  & **Joint GA** & \multicolumn{2}{c}{**Avg GA**} & \multicolumn{2}{c}{**Act-Stot F1**} & \multicolumn{2}{c}{**BLEU-4rgla**} & \multicolumn{2}{c}{**BLEU-4rgn**} \\  & All & Seen & All & Seen & All & Seen & All & Seen & All & Seen \\ \hline SimpleTOD & \(29.4\) & \(79.0\) & \(46.9\) & \(90.3\) & \(61.7\) & \(88.9\) & \(12.3\) & \(17.0\) & \(8.0\) & \(9.9\) \\ SimpleTOD+ & \(29.3\) & \(77.4\) & \(47.5\) & \(90.0\) & \(61.5\) & \(88.1\) & \(11.2\) & \(15.3\) & \(10.8\) & \(12.8\) \\ Arranger & \(29.4\) & \(79.0\) & \(46.9\) & \(90.3\) & \(61.7\) & \(88.9\) & \(9.6\) & \(13.5\) & \(12.2\) & \(14.4\) \\ Rewitter & \(29.4\) & \(79.0\) & \(46.9\) & \(90.3\) & \(60.8\) & \(86.6\) & \(10.4\) & \(14.5\) & \(11.3\) & \(13.3\) \\ \hline \hline \end{tabular}
\end{table}
Table 3: Automatic evaluation results on the test set of ACCENTOR-SGD.

 

ment about \(1\)K randomly sampled dialogues from another task-oriented dataset, MultiWOZ 2.1 Eric et al. (2020) following the same steps as described in Section 2. Crowd workers label \(30.0\%\) of the candidates as good, which is lower compared with accentor-SGD (\(41.4\%\) in Table 2). We attribute the difference to (i) the performance downgrade of the filter model since we do not re-train the model for MultiWOZ 2.1, and (ii) a higher chance of a chit-chat augmented response being too verbose to be good since the average number of tokens per system turn in MultiWOZ 2.1 is larger than that of SGD (\(17.3\) vs. \(13.1\)). Nevertheless, the augmented version (denoted as accentor-MultiWOZ) is significantly more preferred than the original, as shown in Figure 6, where we randomly sample \(100\) dialogues from accentor-MultiWOZ, augment all of their system responses that have chit-chat candidates labeled as good, and compare these augmented dialogues with the corresponding original dialogues.

### Model Evaluations

Automatic Evaluations.We consider joint goal accuracy (Joint GA) and average goal accuracy (Avg GA) for evaluating belief states, act-slot F1 for evaluating action decisions, and two BLEU-4 scores (BLEU-4orig, BLEU-4aug) for evaluating system responses, where we use **original** (resp. **augmented**) system responses as references for BLEU-4orig (resp. BLEU-4aug). Table 3 summarizes the evaluation results. Since the test set of SGD contains unseen services (i.e., services not seen during training) designed to evaluate the model's generalizability, we report the results on all services (All) and seen services only (Seen) following Rastogi et al. (2020). Our proposed models generally achieve a similar task performance level compared with the SimpleTOD baseline. Unsurprisingly, the proposed models achieve lower BLEU-4orig and higher BLEU-4aug.

Human Evaluations.We turn to human evaluations for a more comprehensive measure of the response generation performance. We employ the same ACUTE-Eval metrics as we do in data evaluations. We randomly sample \(100\) dialogues from the test set of accentor-SGD. For each sampled dialogue \(D=\{u_{1},s_{1},u_{2},s_{2},\ldots,u_{n},s_{n}\}\), we pass \(u_{1},s_{1},\dots,u_{i}\) to each model \(\mathcal{M}\in\{\text{SimpleTOD},\) SimpleTOD+, Arranger, Rewwriter} to obtain its system response \(s^{\mathcal{M}}_{i}\) for the \(i\)-th system turn (\(1\leq i\leq n\)). Let \(D^{\mathcal{M}}\) represent \(\{u_{1},s^{\mathcal{M}}_{1},\dots,u_{n},s^{\mathcal{M}}_{n}\}\). We ask evaluators to compare each pair of \(D^{\mathcal{M}_{1}}\) and \(D^{\mathcal{M}_{2}}\), where \(\mathcal{M}_{1},\mathcal{M}_{2}\in\{\text{SimpleTOD},\) SimpleTOD+, Arranger, Rewriter} and \(\mathcal{M}_{1}\neq\mathcal{M}_{2}\). As shown in Figure 5, all of the chit-chat augmented models outperform the SimpleTOD baseline over four ACUTE-Eval metrics. Among the chit-chat augmented models, no one shows a clear win over the other two on the quantitative level. We show a full dialogue example comparing responses gener

Figure 5: Human evaluation results on the test set of accentor-SGD using ACUTE-Eval (*: p-value \(<0.05\), **: p-value \(<0.005\)).

Figure 6: Comparisons between MultiWOZ 2.1 and accentor-MultiWOZ at the dataset level using ACUTE-Eval (**: p-value \(<0.005\)).

 

ated by different models along with supplementary discussions in Appendix A.5.

Considering that the injection frequency affects human evaluations (Section 4.1) and that all our models do not explicitly control the injection frequency, we experiment with controlling the injection frequency by modifying Arranger to consider including chit-chat into the current turn only when the injection frequency from the first turn to the current turn is less than \(0.3\). Compared with the original Arranger, the modified Arranger achieves a higher win percentage over SimpleTOD, as shown in Figure 7. We leave further exploration of injection frequency for future work.

### Limitations and Further Discussions

ApproachOur proposed strategy to augment task-oriented dialogue system responses with chitcht is simple, compared with how it emerges in human conversations, where both functionality and engagingness structurally intertwine with each other in a more complex fashion. Our proposed Rewriter model does have a modeling capability to compose both functions organically but is limited due to the dataset's target arrangement (i.e., concatenation of two separate components). Despite the limitation, our chosen design of "code-separation" has practical merits: we can easily extend the proposed approach to an existing production-level virtual assistant system as a modularized solution, and it has minimal interference to the user-perceived _task success rate_, a core metric widely adapted in virtual assistant systems. Another limitation of our work is that we only augment responses on the system side in our dataset, and the augmentations are independent of each other, whereas in real-life situations, users are also likely to make chit-chat, and the chit-chat between the user and the system should ideally be related to each other. We leave for future research addressing these limitations.

EvaluationWe follow the previous literature on evaluation and regard the four ACUTE-Eval metrics as the primary measure of the response generation performance in this work. However, there is a large overlap between the desired quality measured by different human judgment categories used in ACUTE-Eval. The four ACUTE-Eval metrics favor the same dialogue \(84.4\%\) of the time in our evaluation, indicating high correlations between these metrics. We leave the study of addressing this issue for future work.

## 5 Related Work

### Dialogue Datasets

Dialogue system research has been consistently supported by the development of new datasets. The Dialog State Tracking Challenge (DSTC) series Williams et al. (2013); Henderson et al. (2014a, b); Williams et al. (2014); Kim et al. (2016, 2017); Moon et al. (2020) provide common testbeds for task-oriented dialogues. Following DSTC, researchers have created a variety of publicly available task-oriented dialogue datasets El Asri et al. (2017); Shah et al. (2018); Budzianowski et al. (2018); Rastogi et al. (2020). Another line of work seeks to facilitate open-domain chatbot development with large amounts of human-created text data generated in a social context Baumgartner et al. (2020) and supervision for a variety of desirable general qualities such as being engaging, personable, knowledgeable, and empathetic Zhang et al. (2018); Dinan et al. (2019); Rashkin et al. (2019); Moon et al. (2019); Wang et al. (2019); Smith et al. (2020). Our work bridges the two lines. We compare accentor-SGD and accentor-MultiWOZ with relevant and representative dialogue datasets in Table 5.

Note that very few dialogue corpora contain explicit annotations for both task-oriented and chit-chat utterances. For example, task-oriented dialogue corpora constructed by Rastogi et al. (2020) and Moon et al. (2020) contain annotations for a few chit-chat dialogue acts, but they are limited to light social greetings (e.g., "_Thank you!_", "_Good Bye._") typically at the end of each dialogue session. Zhao et al. (2017) propose to artificially augment task-oriented dialogues with randomly sampled utterances from a chit-chat corpus, mainly to improve the out-of-domain recovery performance. Akasaki and Kaji (2017) annotate user utterances with chat/non-chat binary labels. Still, they do not study the contextual combination of these two

Figure 7: Human evaluation results of the modified Arranger with controlled injection frequency (**: p-value \(<0.005\), †\(\downarrow\): increased/decreased win % compared with the original Arranger).

 

to make conversations more engaging, and their corpus does not contain goal labels like typical task-oriented dialogue corpora. In contrast, our work drastically increases the diversity and contextual coverage of chit-chat additions for any task-oriented dialogue corpus (e.g., _"It's a great way to kick off the summer!"_, _"I hear it's beautiful."_).

Compared with other approaches of creating a high-quality dialogue corpus (e.g., via human-to-human "Wizard-of-Oz" collection Eric et al. (2020), dialogue self-play and paraphrase Shah et al. (2018)), the annotation cost of the proposed model-based dialogue generation approach combined with the quality control mechanisms is lower, as our work does not involve authoring new sentences by human annotators.

### Task-Oriented Dialogue Systems

Over the past few years, neural models have achieved remarkable success in the development of the main components of task-oriented dialogue systems, including understanding user intent, tracking dialogue states, determining system actions, and generating system responses Henderson et al. (2013); Sun et al. (2014); Wen et al. (2015); Liu and Lane (2016); Mrksic et al. (2017); Wen et al. (2017); Nouri and Hosseini-Asl (2018); Heck et al. (2020); Chen et al. (2020). Recently, connecting separate components and building end-to-end task-oriented neural dialogue systems have attracted increasing interest Bordes et al. (2017); Peng et al. (2020). The most recent thread is to unify all components in a single end-to-end neural model by fine-tuning a pre-trained deep language model on multiple tasks, which leads to state-of-the-art performance Hosseini-Asl et al. (2020); Peng et al. (2020). We follow this thread and further enhance the ability to generate appropriate non-task-oriented add-ons, on top of the ability to achieve functional goals that existing systems are typically narrowly tailored to. A few work have studied training a dialogue model leveraging multiple chit-chat and task-oriented dialogues Madotto et al. (2019, 2020), which allows the model to attend on a relevant task for a given user utterance and respond accordingly, thus increasing the skill coverage of the model. Our proposed models are trained on the newly collected accentor-SGD dataset with the turn-level supervision signals, allowing for contextual and flexible code-switching between chit-chat and functional tasks in a single system turn.

## 6 Conclusion

We propose adding chit-chat to enhance task-oriented dialogues (accentor) in this study. We present a general Human\(\leftrightarrow\)AI collaborative data construction approach for accentor, with which we create a dataset consisting of \(23.8\)K chit-chat augmented task-oriented dialogues. We show via human evaluation that chit-chat augmented dialogues are preferred than the unaugmented. In addition, we propose three models for accentor. Evaluation results show that compared with the baseline trained on the original unaugmented data, our proposed models trained on the chit-chat augmented counterpart achieve a similar task performance level and higher human evaluation scores.

## Acknowledgements

We thank Gerald Demenynck for helping with the data annotation process. We would also like to thank the anonymous NAACL reviewers for their constructive and insightful feedback.

\begin{table}
\begin{tabular}{p{113.8pt} p{56.9pt} p{56.9pt} p{56.9pt} p{56.9pt}} \hline \hline
**Dataset** & **Construction Method** & **\# Dialogues** & **Task-Oriented** & **Chit-Chat** \\ \hline DSTCE Henderson et al. (2014a) & crowdsourcing & \(3,235\) & ✓ & ✗ \\ MultiWOZ 2.1 Eric et al. (2020) & crowdsourcing & \(10,438\) & ✓ & ✗ \\ Schema-Guided Dialogue Rastogi et al. (2020) & crowdsourcing & \(22,825\) & ✓ & ✗ \\ SIMMC Moon et al. (2020) & crowdsourcing & \(12,948\) & ✓ & ✗ \\ \hline PersonaChat Zhang et al. (2018) & crowdsourcing & \(10,907\) & ✗ & ✓ \\ Wizard of Wikipedia Dinan et al. (2019) & crowdsourcing & \(22,311\) & ✗ & ✓ \\ EmpatheticDialogues Rashkin et al. (2019) & crowdsourcing & \(24,850\) & ✗ & ✓ \\ BlendedSkillTalk Smith et al. (2020) & crowdsourcing & \(6,808\) & ✗ & ✓ \\ Pushshift Reddit Baumgartner et al. (2020) & crawling \& scraping & \(651,778,198^{\dagger}\) & ✗ & ✓ \\ \hline
**accentor-SGD (this work)** & crowdsourcing & \(22,825\) & ✓ & ✓ \\
**accentor-MultiWOZ (this work)** & crowdsourcing & \(997\) & ✓ & ✓ \\ \hline \hline \end{tabular}
\end{table}
Table 5: Statistics of dialogue datasets (\({}^{\dagger}\): regarding each thread (i.e., a post and its comments) as a dialogue).

 



## References

* A. Adiwardana, M. Luong, D. R. So, J. Hall, N. Fiedel, R. Thoppilan, Z. Yang, A. Kulshreshtha, G. Nemade, Y. Lu, and Q. V. Le (2020)Towards a human-like open-domain chatbot. arXiv preprintabs/cs.CL/2001.09977v3 Cited by: SS4.1.
* S. Akasaki and N. Kaji (2017)Chat detection in an intelligent assistant: combining task-oriented and non-task-oriented spoken dialogue systems. In Proceedings of the ACL, Vancouver, Canada, pp. 1308-1319. External Links: Link Cited by: SS5.
* J. Baumgartner, S. Zannettou, B. Keegan, M. Squire, and J. Blackburn (2020)The pushshift reddit dataset. In Proceedings of the ICWSM, Atlanta, GA, pp. 830-839. External Links: ISBN 978-0-6450-3213-1, Document, Link Cited by: SS1, SS2.
* A. Bordes, Y. Boureau, and J. Weston (2017)Learning end-to-end goal-oriented dialog. In Proceedings of the ICLR, Toulon, France, Cited by: SS1, SS2.
* a large-scale multi-domain wizard-of-oz dataset for task-oriented dialogue modelling. In Proceedings of the EMNLP, Brussels, Belgium, pp. 5016-5026. External Links: Link Cited by: SS1, SS2.
* L. Chen, B. Lv, C. Wang, S. Zhu, B. Tan, and K. Yu (2020)Schema-guided multi-domain dialogue state tracking with graph attention neural networks. In Proceedings of the AAAI, New York, NY, pp. 7521-7528. External Links: Link Cited by: SS1, SS2.
* E. Dinan, S. Roller, K. Shuster, A. Fan, M. Auli, and J. Weston (2019)Wiizard of Wikipedia: knowledge-powered conversational agents. In Proceedings of the ICLR, New Orleans, LA, Cited by: SS1, SS2.
* L. El Asri, H. Schulz, S. Sharma, J. Zumer, J. Harris, E. Fine, R. Mehrotra, and K. Suleman (2017)Frames: a corpus for adding memory to goal-oriented dialogue systems. In Proceedings of the SIGDIAL, Saarbrucken, Germany, pp. 207-219. External Links: Link Cited by: SS1, SS2.
* M. Eric, R. Goel, S. Paul, A. Sethi, S. Agarwal, S. Gao, A. Kumar, A. Goyal, P. Ku, and D. Hakkani-Tur (2020)MultiWOZ 2.1: a consolidated multi-domain dialogue dataset with state corrections and state tracking baselines. In Proceedings of the LREC, Marseille, France, pp. 422-428. External Links: Link Cited by: SS1, SS2.
* M. Heck, C. van Niekerk, N. Lubis, C. Geishauser, H. Lin, M. Moresi, and M. Gasic (2020)TriPPy: a triple copy strategy for value independent neural dialog state tracking. In Proceedings of the SIGDIAL, pp. 35-44. External Links: Link Cited by: SS1, SS2.
* M. Henderson, B. Thomson, and J. D. Williams (2014a)The second dialog state tracking challenge. In Proceedings of the SIGDIAL, Philadelphia, PA, pp. 263-272. External Links: Link Cited by: SS1, SS2.
* M. Henderson, B. Thomson, and J. D. Williams (2014b)The third dialog state tracking challenge. In Proceedings of the SLT, South Lake Tahoe, NV, pp. 324-329. External Links: Link Cited by: SS1, SS2.
* M. Henderson, B. Thomson, and S. Young (2013)Deep neural network approach for the dialog state tracking challenge. In Proceedings of the SIGDIAL, Metz, France, pp. 467-471. External Links: Link Cited by: SS1, SS2.
* E. Hosseini-Asl, B. McCann, C. Wu, S. Yavuz, and R. Socher (2020)A simple language model for task-oriented dialogue. arXiv preprintabs/2005.00796v3 Cited by: SS1, SS2.
* C. Hutto and E. Gilbert (2014)VADER: a parsimonious rule-based model for sentiment analysis of social media text. In Proceedings of the ICWSM, Ann Arbor, MI, Cited by: SS1, SS2.
* S. Kim, L. Fernando D'Haro, R. E. Banchs, J. D. Williams, M. Henderson, and K. Yoshino (2016)The fifth dialog state tracking challenge. In Proceedings of the SLT, San Diego, CA, pp. 511-517. External Links: Link Cited by: SS1, SS2.
* S. Kim, L. Fernando D'Haro, R. E. Banchs, J. D. Williams, and M. Henderson (2017)The fourth dialog state tracking challenge. In Dialogues with Social Robots, pp. 435-449. External Links: Link Cited by: SS1, SS2.
* M. Li, J. Weston, and S. Roller (2019)ACUTE-EVAL: improved dialogue evaluation with optimized questions and multi-turn comparisons. In NeurIPS workshop on Conversational AI, Vancouver, Canada, Cited by: SS1, SS2.
* B. Liu and I. Lane (2016)Attention-based recurrent neural network models for joint intent detection and slot filling. In Proceedings of the Interspeech, San Francisco, CA, pp. 685-689. External Links: Link Cited by: SS1, SS2.
* Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V. Stoyanov (2019)RoBERTa: a robustly optimized bert pretraining approach. arXiv preprintabs/1907.11692v1 Cited by: SS1, SS2.
* Andrea Madotto, Z. Lin, X. Bang, and P. Fung (2020)The adapter-bot: all-in-one controllable conversational model. arXiv preprintabs/2008.12579v2 Cited by: SS1, SS2.
* A. Madotto, Z. Lin, C. Wu, J. Shin, and P. Fung (2019)Attention over parameters for dialogue systems. In NeurIPS workshop on Conversational AI, Vancouver, Canada, Cited by: SS1, SS2.
* A. Miller, W. Feng, D. Batra, A. Bordes, A. Fisch, J. Lu, D. Parikh, and J. Weston (2017)ParAI: a dialog research software platform. In Proceedings of the EMNLP, Copenhagen, Denmark, pp. 79-84. External Links: Link Cited by: SS1, SS2.



Seungwhan Moon, Satwik Kottur, Paul A Crook, Ankita De, Shivani Poddar, Theodore Levin, David Whitney, Daniel Difranco, Ahmad Beirami, Eunjoon Cho, Rajen Subba, and Alborz Germinafard. 2020. Situated and interactive multimodal conversations. _arXiv preprint_, cs.CL/2006.01460v1.
* Moon et al. (2019) Seungwhan Moon, Pararth Shah, Anuj Kumar, and Rajen Subba. 2019. OpenDialKG: Explainable conversational reasoning with attention-based walks over knowledge graphs. In _Proceedings of the ACL_, pages 845-854, Florence, Italy.
* Mrksic et al. (2017) Nikola Mrksic, Diarmuid O Seaghdha, Tsung-Hsien Wen, Blaise Thomson, and Steve Young. 2017. Neural belief tracker: Data-driven dialogue state tracking. In _Proceedings of the ACL_, pages 1777-1788, Vancouver, Canada.
* Nouri and Hosseini-Asl (2018) Elnaz Nouri and Ehsan Hosseini-Asl. 2018. Toward scalable neural dialogue state tracking model. In _NeurIPS workshop on Conversational AI_, Montreal, Canada.
* Peng et al. (2020a) Baolin Peng, Chunyuan Li, Jinchao Li, Shahin Shayandeh, Lars Liden, and Jianfeng Gao. 2020a. SOLOIST: Few-shot task-oriented dialog with a single pre-trained auto-regressive model. _arXiv preprint_, cs.CL/2005.05298v3.
* Peng et al. (2020b) Baolin Peng, Chenguang Zhu, Chunyuan Li, Xiujun Li, Jinchao Li, Michael Zeng, and Jianfeng Gao. 2020b. Few-shot natural language generation for task-oriented dialog. _arXiv preprint_, cs.CL/2002.12328v1.
* Radford et al. (2019) Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. _Preprint_, available at [https://openai.com/blog/better-language-models/](https://openai.com/blog/better-language-models/).
* Rashkin et al. (2019) Hannah Rashkin, Eric Michael Smith, Margaret Li, and Y-Lan Boureau. 2019. Towards empathetic open-domain conversation models: A new benchmark and dataset. In _Proceedings of the ACL_, pages 5370-5381, Florence, Italy.
* Rastogi et al. (2020) Abhinav Rastogi, Xiaoxue Zang, Srinivas Sunkara, Raghav Gupta, and Pranav Khainta. 2020. Towards scalable multi-domain conversational agents: The schema-guided dialogue dataset. In _Proceedings of the AAAI_, pages 8689-8696, New York, NY.
* Roller et al. (2020) Stephen Roller, Emily Dinan, Naman Goyal, Da Ju, Mary Williamson, Yinhan Liu, Jing Xu, Myle Ott, Kurt Shuster, Eric M Smith, et al. 2020. Recipes for building an open-domain chatbot. _arXiv preprint_, cs.CL/2004.13637v2.
* Shah et al. (2018) Pararth Shah, Dilek Hakkani-Tur, Gokham Tur, Abhinav Rastogi, Ankur Bapna, Neha Nayak, and Larry Heck. 2018. Building a conversational agent overnight with dialogue self-play. _arXiv preprint_, cs.CL/1801.04871v1.
* Shuster et al. (2020) Kurt Shuster, Da Ju, Stephen Roller, Emily Dinan, Y- Lan Boureau, and Jason Weston. 2020. The dialogue dodecathlon: Open-domain knowledge and image grounded conversational agents. In _Proceedings of the ACL_, pages 2453-2470.
* Smith et al. (2020) Eric Michael Smith, Mary Williamson, Kurt Shuster, Jason Weston, and Y-Lan Boureau. 2020. Can you put it all together: Evaluating conversational agents' ability to blend skills. In _Proceedings of the ACL_, pages 2021-2030.
* Sun et al. (2014) Kai Sun, Lu Chen, Su Zhu, and Kai Yu. 2014. The SJTU system for dialog state tracking challenge 2. In _Proceedings of the SIGDIAL_, pages 318-326, Philadelphia, PA.
* Wang et al. (2019) Xuewei Wang, Weiyan Shi, Richard Kim, Yoojing Oh, Sijia Yang, Jingwen Zhang, and Zhou Yu. 2019. Persusasion for good: Towards a personalized persuasive dialogue system for social good. In _Proceedings of the ACL_, pages 5635-5649, Florence, Italy.
* Wen et al. (2015) Tsung-Hsien Wen, Milica Gasic, Nikola Mrksic, Pei-Hao Su, David Vandyke, and Steve Young. 2015. Semantically conditioned LSTM-based natural language generation for spoken dialogue systems. In _Proceedings of the EMNLP_, pages 1711-1721, Lisbon, Portugal.
* Wen et al. (2017) Tsung-Hsien Wen, Yishu Miao, Phil Blunsom, and Steve Young. 2017. Latent intention dialogue models. In _Proceedings of the ICML_, volume 70, pages 3732-3741, Sydney, Australia.
* Williams et al. (2013) Jason Williams, Antoine Raux, Deepak Ramachandran, and Alan Black. 2013. The dialog state tracking challenge. In _Proceedings of the SIGDIAL_, pages 404-413, Metz, France.
* Williams et al. (2014) Jason D Williams, Matthew Henderson, Antoine Raux, Blaise Thomson, Alan Black, and Deepak Ramachandran. 2014. The dialog state tracking challenge series. _AI Magazine_, 35(4):121-124.
* Yu et al. (2016) Zhou Yu, Ziyu Xu, Alan W Black, and Alexander Rudnicky. 2016. Strategy and policy learning for non-task-oriented conversational systems. In _Proceedings of the SIGDIAL_, pages 404-412, Los Angeles, CA.
* Zhang et al. (2018) Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. 2018. Personalizing dialogue agents: I have a dog, do you have pets too? In _Proceedings of the ACL_, pages 2204-2213, Melbourne, Australia.
* Zhao et al. (2017) Tiancheng Zhao, Allen Lu, Kyusong Lee, and Maxine Eskenzi. 2017. Generative encoder-decoder models for task-oriented spoken dialog systems with chatting capability. In _Proceedings of the SIGDIAL_, pages 27-36, Saarbrucken, Germany 

Appendix

### Details of Candidate Generation

We summarize model configurations in Table 6, which are employed together for candidate generation in Section 2.1. Our implementation is based on ParlAI Miller et al. (2017), and all unspecified parameters take the default values set in the interactive mode of ParlAI.

### Details of Candidate Filtering

The ranker initially ranks each candidate according to the posterior probability output by the binary classifier. It then lowers the ranks of candidates that match a list of bad patterns. Most bad patterns are about newly introduced counterfeit information (e.g., containing an URL/email address, a phone number, time, or amount of money). The rest of the bad patterns are mainly about text genre (e.g., containing email sign-offs such as "best regards") and format (e.g., misuse of punctuation marks). Lastly, the ranker raises the ranks of (i) uncommon candidates and (ii) candidates that are dissimilar to the other candidates for the dialogue and the system response being augmented. We measure the similarity by Levenshtein distance.

Note that we do not explore the optimal settings for candidate filtering, as it is not the primary focus of this paper. For instance, how much the rule-based ranker lowers or raises the ranks of candidates is set manually based on engineering intuition rather than rigorous analysis; we do not exhaustively investigate how much labeled data is required to obtain a good enough binary classifier; the \(1.7\text{K}\) examples from the pilot annotation are randomly sampled. Tuning the procedure (e.g., the number and selection of training examples) may lead to a better resulting candidate set.

### Human Evaluation Questions

* Engaging: _Who would you prefer to talk to? Which version is more likely to hold your attention and make you want to hear more?_
* Interesting: _Who would you say is more interesting? Which versionouses your curiosity or tells you something new or useful?_
* Humanlike: _Who would you say sounds more human? Which version is more natural and personable?_
* Knowledgeable: _Who would you say is more knowledgeable? Which version seems more well informed and confident in the information?_

\begin{table}
\begin{tabular}{c c r} \hline \hline
**Generative Model** & **Beam Size** & **Minimum Beam Length** \\ \hline BlenderRot (90M) & 10 & 1 \\ BlenderRot (90M) & 10 & 5 \\ BlenderRot (90M) & 10 & 20 \\ BlenderRot (90M) & 30 & 20 \\ BlenderRot (2.7B) & 10 & 1 \\ BlenderRot (2.7R) & 10 & 5 \\ BlenderRot (2.78) & 10 & 20 \\ BlenderRot (2.78) & 30 & 20 \\ BlenderRot (9.4B) & 10 & 1 \\ GPT+2 (11TM) & 10 & 1 \\ GPT-2 (34CM) & 1 & 1 \\ GPT-2 (34CM) & 10 & 1 \\ GPT-2 (34CM) & 10 & 5 \\ GPT-2 (345M) & 10 & 20 \\ GPT-2 (345M) & 30 & 1 \\ GPT-2 (762M) & 10 & 1 \\ \hline \hline \end{tabular}
\end{table}
Table 6: Employed models and decoding parameters for candidate generation.

 



### Example Dialogues

\begin{table}
\begin{tabular}{l l p{142.3pt}} \hline \hline  & Utterance & Task Labels \\ \hline U: & I’m starting to get hungry. I want to go eat at a nice restaurant. A: Sure. What city would live to eat in? U: Mitiplas. A: (_O_) That sounds delicious. D: Do you have a particular type of food you are craving? Mexican? American? Smoking else. U: I feel like Cambodian tonight. A: (_A_) If heaf the food is amazing. There is a Cambodian restaurant in Mipitals called Anh Hong. (_A_) It's a great place so out. U: That’s exactly what IT looking for. A: Do you want me to make a reservation for you? (_A_) can also recommend some other restaurants in the area. U: Yes, I take to try this place out. A: Okay. At what time would you like to eat there? U: G:30 pm. Please make the reservation for four people. A: (_G_) Only. Sure: A reservation at Ash Hong in Mitigats for 4 e:30 pm today. Is that correct? U: Perfect. What are the prices at Ahn Hong like? I: They have moderate pricing. Your reservation has been made. (_A_) Thank you. (_B_) You your restart direct? U: Thanks. You be a big help. A: (_A_) For one welcome. I hope you enjoy your meal._ Have a day. (_A_) _I hope you enjoy your meal. Do you know any plans for the rest of the day?_ U: Can you find me some songs? Any album is fine. A: I have found 10 songs you may like. How does A Little Bit Stronger by Sara Evans in the album Stronger sound? U: That’s great. A: (_G_) If they to check it out. A: (_G_) If all you like it. What else can I help with? (_A_) I love helping people find new music. U: I want to play the song. A: Should I play A Little Bit Stronger on the TV? (_A_) I think it would be a great song to play on the television. What do you think? (_A_) That’s my favorite song by Sarah Evans. U: Yes, This carrot. A: Started playing the song. A: (_G_) Let us now about the lyrics. Winter che dye you want to know about it? U: Thanks. ’that’s all need. A: (_G_) Two we welcome. Hava a good day. (_A_) What do you like? (_A_) They two find some good music to listen to. I like all kinds of music. What do you look? (_A_)



### Generation Samples

As shown in Table 8, we observe that compared with SimpleTOD+, both Arranger and Rewriter tend to add chit-chat to the beginning of task-oriented responses. This is perhaps because the underlying off-the-shelf chit-chat model takes only \(u_{1},s_{1},\ldots,u_{i}\) as input, making it more likely to generate a suitable chit-chat to start, rather than end the \(i\)-th system turn. The responses generated by Arranger and Rewriter are similar because Rewriter generates responses by copying contents from the responses output by the underlying off-the-shelf models without modification for most of the time (\(87.0\%\) of dialogues on the test set).

\begin{table}
\begin{tabular}{p{34.2pt} p{56.9pt} p{34.2pt}} \hline \hline
**U:** & I’m looking for something interesting to do, please find an activity from the chit of this month. \\ \hline
**U:** & T d like to search for an activity around Seattle, WA. \\ \hline

[MISSING_PAGE_POST]

ike to smeting.