Follow the Knowledge: Structural Biases and Artefacts in Knowledge Grounded Dialog Datasets (2023)

Abstract: Crowd-sourcing has been one of the primary ways to curate conversational data, specially for certain scenarios like grounding in knowledge. In this setting, using online platforms like AMT, non-expert participants are hired to converse with each other, following instructions which try to guide the outcome towards the desired format. The resulting data then is used for different parts of dialog modelling like knowledge selection and response selection/generation. In this work, we take a closer look into two of the most popular knowledge grounded dialog (KGD) datasets. Investigating potential biases and artefacts in knowledge selection labels, we observe that in many cases the ‘knowledge selection flow’ simply follows the order of presented knowledge pieces. In Wizard of Wikipedia (the most popular KGD dataset) we use simple content-agnostic models based on this bias to get significant knowledge selection performance. In Topical-Chat we see a similar correlation between the knowledge selection sequence and the order of entities and their segments, as provided to crowd-source workers. We believe that the observed results, question the significance and origin of the presumed dialog-level attributes like ‘knowledge flow’ in these crowd-sourced datasets.

## introduction since the introduction of data hungry methods into dialog modeling, sizeable datasets have become an essential asset for researchers of the field. while generic conversational data can be harvested in large quantities from already existing resources like movie subtitles @xcite or website forums @xcite , more specific datasets usually need to be curated under supervision. knowledge grounded dialog is one of the fields that has remarkably benefited from crowdsourced datasets like wizard of wikipedia or wow @xcite , holl-e @xcite and topical-chat @xcite , which offer grounded utterances generated by nonexpert annotators or turkers. during the curation, participants are commonly asked to first choose a knowledge piece (or no knowledge) from a provided pool and then use the selected piece to ground their next utterance on. one common attribute in these datasets is the sizeable difference between the knowledge selection performance with and without access to the uttered response. the phenomenon -referred to as the prior-posterior gap-is demonstrated in in this work we investigate the potential spurious origins of 'knowledge flow' in crowd-sourced kgd datasets. focusing on the most popular resources in the field, i.e. wow, we show that competitive results can be obtained in the knowledge selection task, using very simple structural heuristics. we also show that these rudimentary patterns are not an isolated case and can be found in other knowledge grounded dialog datasets like topicalchat, which -in our opinion-connects knowledge selection patterns to dataset curation choices and design. while dataset artifacts and their relation to the curation process have been widely studied in nlu tasks and especially nli @xcite , it is an under-studied topic in dialog modeling. we hope our work draws attention to the issue and contributes to having better dialog datasets, which we believe is necessary for properly modeling higher-order dialog attributes. ## knowledge selection biases and artefacts our main objective in this work is to explore the structural biases and artefacts in the knowledge selection labels of popular kgd datasets. for this, we use different methods depending on the way knowledge pools are constructed and presented to crowd-source workers, but in both cases, we essentially investigate the same hypothesis: crowd-source workers often base their knowledge selection on the structure and order of the knowledge pool, as presented to them. in other words, when selecting the knowledge piece for the next utterance, they tend to just follow the knowledge document and pick the 'next' item, instead of coming up with a more sophisticated 'flow'. in the following sections, we explore this hypothesis separately for wizard of wikipedia and topical-chat. ## discussion and conclusion in this work we investigated the potential knowledge selection biases and artefacts in two popular kgd datasets. our central governing hypothesis was that crowd-source workers tend to simply follow the structure and order of knowledge pieces, as presented to them. for the wow dataset, we showed that using this hypothesis, content-agnostic models can achieve noticeable knowledge selection performance, and combined with simple sequence classification training are able to compete with sophisticated solutions. for topical-chat we observed a noisy alignment between the ks sequence and the order of entities and their segments, as provided to crowd-sources. although following the existing order of knowledge pieces is not strange or unexpected (at least within one document), we believe that the way knowledge options are curated and presented to crowd-source workers can be an exacerbating factor. all 4 datasets provide a large number of retrieved knowledge pieces at each turn (usually more than 60) which is statistically beneficial to the dataset, but it could also encourage an 'easy solution' regime in which annotators opt for the safe and convenient choice of following the already existing structure of knowledge articles, instead of trying to create and maintain a novel 'flow'. in its extreme case, this leads to conversations similar to reciting an article line by line 7 . in terms of dialog modeling, these results can suggest that the origin and significance of higherorder attributes in the dataset can be questioned. in particular, the concept of 'flow' as governing the dialog-level pattern of knowledge selection seems to be rooted substantially in the structure of knowledge documents. this does not rule out the ex-istence or learnability of genuine patterns/flows, but the very low human performance for this task ( 17%; although the ultimate goal in kgd modeling is generating proper responses (and not mastering the knowledge selection part), but in order to model higher-order and dialog-level conversational phenomena, we probably need better datasets. one important factor in producing such resources is considering the process from annotators' point of view, and how design choices (e.g. annotation interface and instructions, size of knowledge pool, etc.) can persuade them towards or away from 'easy solution' regimes which are prone to artifacts. another approach is providing explicit 'scenarios' for the way dialogs are supposed to unfold. this is how duconv @xcite and naturalconv @xcite datasets (both chinese) have been curated, but whether this mitigates the problem or introduces new artifacts should be studied.