Guideline Bias in W izard-of- O z Dialogues (2021)

Abstract: NLP models struggle with generalization due to sampling and annotator bias. This paper focuses on a different kind of bias that has received very little attention: guideline bias, i.e., the bias introduced by how our annotator guidelines are formulated. We examine two recently introduced dialogue datasets, CCPE-M and Taskmaster-1, both collected by trained assistants in a Wizard-of-Oz set-up. For CCPE-M, we show how a simple lexical bias for the word like in the guidelines biases the data collection. This bias, in effect, leads to poor performance on data without this bias: a preference elicitation architecture based on BERT suffers a 5.3% absolute drop in performance, when like is replaced with a synonymous phrase, and a 13.2% drop in performance when evaluated on out-of-sample data. For Taskmaster-1, we show how the order in which instructions are resented, biases the data collection.

## introduction sample bias is a well-known problem in nlp -discussed from @xcite to @xcite -and annotator bias has been discussed as far back as @xcite . this paper focuses on a different kind of bias that has received very little attention: guideline bias, i.e., the bias introduced by how our annotator guidelines are formulated. annotation guidelines are used to train annotators, and guidelines are therefore in some sense intended to and designed to prime annotators. what we will refer to in our discussion of guideline bias, is rather the unintended biases that result from how guidelines are formulated, and the examples used in those guidelines. if a treebank annotation guideline focuses overly on parasitic gap constructions, for example, inter-annotator agreement may be higher on those, and annotators may be biased to annotate similar phenomena by analogy with parasitic gaps. figure 1 : the percentage of sentences with the word like in the ccpe-m annotation guidelines (guidelines), the suggested questions to ask users, in the guidelines (suggestions), (c) the actual first turns by the assistants (1st turn), and (d) the actual replies by the users (2nd turn). in all cases, more than half of the sentences contain the word like. we focus on two recently introduced datasets, the coached conversational preference elicitation corpus (ccpe-m) from @xcite , related to the task of conversational recommendation @xcite , and taskmaster-1 @xcite , which is a multipurpose, multi-domain dialogue dataset. ccpe-m consists of conversations about movie preferences, and the part of taskmaster-1, we focus on here, conversations about theatre ticket reservations. both corpora were collected by having a team of assistants interact with users in a wizard-of-oz (woz) set-up, i.e. a human plays the role of a digital assistant which engages a user in a conversation about their movie preferences. the assistants were given a set of guidelines in advance, as part of their training, and it is these guidelines that induce biases. in ccpe-m, it is the overwhelming use of the verb like (see figure 5 ) and its trickle-down effects, we focus on; in taskmaster-1, the order of the instructions. in fact, the ccpe-m guidelines consist of 324 words, of which 20 (6%) are inflections or derivations of the lemma like: as shown in figure 5 in the appendix, more than 50% of the sentences in the guidelines include forms of like! this very strong bias in the guidelines has a clear downstream effect on the assistants that are collecting the data. in their first dialogue turn, the assistants use the word like in 72% of the dialogues. this again biases the users responding to the assistants in the woz set-up: in 58% of their first turns, given that the assistant uses a form of the word like, they also use the verb like. we show that this bias leads to overly optimistic estimates of performance. additionally, we also demonstrate how the guideline affects the user responses through a controlled priming experiment. for taskmaster-1, we show a similar effect of the guidelines on the collected dialogues. contributions we introduce the notion of guideline bias and present a detailed analysis of guideline bias in two recently introduced dialogue corpora (ccpe-m and taskmaster-1). our main experiments focus on ccpe-m: we show how a simple bias toward the verb like easily leads us to overestimate performance in the wild by showing performance drops on semantically innocent perturbations of the test data, as well as on a new sample of movie preference elicitations that we collected from reddit for the purpose of this paper. we also show that debiasing the data, improves performance. the ccpe-m provides a very clear example of guideline bias, but other examples can be found, e.g., in taskmaster-1, which we discuss in §3. we discuss more examples in §4. we first examine the ccpe-m dataset of spoken dialogues about movie preferences. the dialogues in ccpe-m are generated in a wizard-of-oz set-up, where the assistants type their input, which is then translated into speech using text-to-speech technologies, at which point users respond by speech. the dialogues were transcribed and annotated by the authors of @xcite . ## discussion & conclusion in this work, we examined guideline bias in two newly presented woz style dialogue corpora: we showed how a lexical bias for the word like in the annotation guidelines of ccpe-m, through a controlled priming experiment leads to a bias for this word in the dialogues, and that models trained on this corpus are sensitive to the absence of this verb. we provided a new test dataset for this task, collected from reddit, and show how a debiased model performs better on this dataset, suggesting the 13% drop is in part the result of guideline bias. we showed a similar bias in taskmaster-1.