A.4 Example Dialogues
Utterance	Task Labels
U:	I'm starting to get hungry. I want to go eat at a nice restaurant.	INFORM_INTENT [intent: FindRestaurants]
A:	Sure. What city would you like to eat in?	REQUEST [slot: city]
U:	Milpitas.	INFORM [city: Milpitas]
A:	(X) That sounds delicious.
Do you have a particular type of food you are craving? Mexican? American? Something else?	REQUEST [cuisine: Mexican, American]
U:	I feel like Cambodian tonight.	INFORM [cuisine: Cambodian]
A:	(✓) I hear the food is amazing.	OFFER [restaurant_name: Anh Hong, city: Milpitas]
There is a Cambodian restaurant in Milpitas called Anh Hong.
(✓) It's a great place to eat.
U:	That's exactly what I'm looking for.	SELECT
A:	Do you want me to make a reservation for you?	OFFER_INTENT [intent: ReserveRestaurant]
(X) I can also recommend some other restaurants in the area.
U:	Yes, I'd like to try this place out.	AFFIRM_INTENT
A:	Okay. At what time would you like to eat there?	REQUEST [slot: time]
U:	6:30 pm. Please make the reservation for four people.	INFORM [time: 6:30 pm, party_size: 4]
A:	(X) Okay.	CONFIRM [restaurant_name: Ahn Hong, city: Milpitas, time: 6:30 pm, party_size: 4]
Sure. A reservation at Anh Hong in Milpitas for 4 at 6:30 pm today. Is that correct?	REQUEST [slot: price_range]
U:	Perfect. What are the prices at Anh Hong like?	INFORM [price_range: moderate]
A:	They have moderate pricing. Your reservation has been made.
(✓) Thank you.
(X) Do you want dessert?
U:	Thanks. You've been a big help.	THANK_YOU
A:	(✓) You are welcome. I hope you enjoy your meal.	GOODBYE
Have a nice day.
(X) I hope you enjoy your meal. Do you have any plans for the rest of the day?
U:	Can you find me some songs? Any album is fine.	INFORM_INTENT [intent: LookupMusic, album: dont-care]
A:	I have found 10 songs you may like. How does A Little Bit Stronger by Sara Evans in the album Stronger sound?	INFORM_COUNT [count: 10], OFFER [artist: Sara Evans, song_name: A Little Bit Stronger]
U:	That's great.	SELECT
A:	(X) I'll have to check it out.	REQ_MORE
(✓) I'm glad you like it.
What else can I help with?
(✓) I love helping people find new music.
U:	I want to play the song.	INFORM_INTENT [intent: PlayMedia]
A:	Should I play A Little Bit Stronger on the TV?	CONFIRM [playback_device: TV, song_name: A Little Bit Stronger]
(✓) I think it would be a great song to play on the television. What do you think?
(✓) That's my favorite song by Sarah Evans.
U:	Yes, that's correct.	AFFIRM
A:	Started playing the song.	NOTIFY_SUCCESS
(X) I can tell you about the lyrics.
(X) What else do you want to know about it?
U:	Thanks, that's all I need.	THANK_YOU
A:	(✓) You're welcome.	GOODBYE
Have a good day.
(X) What do you like?
(X) I hope you find some good music to listen to. I like all kinds of music. What do you like?
Table 7: Example dialogues from ACCENTOR-SGD (U: user; A: assistant). All chit-chat candidates for augmentation, generated with the state-of-the-art pre-trained language models, are annotated by the crowd workers with good  $(\checkmark)$  and  $(X)$  bad labels. Note that while most of the bad chit-hat candidates are fluent, they are often contextually inappropriate or inconsistent with the rest of the dialogue. The annotation guideline is highlighted in Section 2.3.