The Interplay of Task Success and Dialogue Quality: An in-depth Evaluation in Task-Oriented Visual Dialogues (2021)

Abstract: When training a model on referential dialogue guessing games, the best model is usually chosen based on its task success. We show that in the popular end-to-end approach, this choice prevents the model from learning to generate linguistically richer dialogues, since the acquisition of language proficiency takes longer than learning the guessing task. By comparing models playing different games (GuessWhat, GuessWhich, and Mutual Friends), we show that this discrepancy is model- and task-agnostic. We investigate whether and when better language quality could lead to higher task success. We show that in GuessWhat, models could increase their accuracy if they learn to ground, encode, and decode also words that do not occur frequently in the training set.

## introduction a good dialogue model should generate utterances that are indistinguishable from human dialogues @xcite . this holds for both chit-chat, open-domain, and task-oriented dialogues. while chit-chat dialogue systems are usually evaluated by analysing the quality of their dialogues @xcite , task-oriented dialogue models are evaluated on their task success and it is common practice to choose the best model based only on the task success metric. we explore whether this choice prevents the system from learning better linguistic skills. important progress has been made on the development of such conversational agents. the boost is mostly due to the introduction of the encoderdecoder framework @xcite which allows learning directly from raw data to both understand and generate utterances. the framework has been found to be promising both for chit-chat @xcite and task-oriented dialogues @xcite , and it has been further extended to develop agents that can communicate through natural language about visual content @xcite @xcite . several dialogue tasks have been proposed as referential guessing games in which an agent (the q-bot) asks questions to another agent (the a-bot) and has to guess the referent (e.g., a specific object depicted in the image) they have been speaking about @xcite @xcite @xcite . we are interested in understanding the interplay between the learning processes behind these two sub-tasks: generating questions and guessing the referent. @xcite have compared models on guesswhat and have shown that task success (ts) does not correlate with the quality of machine-generated dialogues. first of all, we check whether this result is task-agnostic by carrying out a comparative analysis of models playing different referential games. we choose a task in which visual grounding happens during question generation @xcite ; a task in which it happens only in the guessing phase @xcite , and a task that is only based on language (mutualfriends, @xcite . we introduce a linguistic metric, linguistic divergence (ld), that, by assembling various metrics used in the literature @xcite @xcite , measures how much the language generated by computational models differs, on the surface level, from the one used by humans. we consider ld to be a proxy of the quality of machine-generated dialogues. for each task, we compare state-of-the-art (sota) models against their ts and ld. in the core part of the paper, we study the relationship between the learning process behind ts and ld by comparing model performance across epochs and by downsizing the training set. finally, we study whether and when a lower ld (i.e., the generated dialogues are more similar to humans) could help reach a higher ts. our results confirm that models performing similarly on ts differ quite a lot on their conversational skills, as claimed in @xcite for models evaluated on the guesswhat game. furthermore, we show that: ## related work task-oriented models can be evaluated based on their task success, but this is not enough to know whether the generated dialogues are human-like. the development of quantitative metrics to evaluate the quality of dialogues generated by conversational agents is a difficult challenge @xcite , and it is under investigation for chit-chat dialogue systems. for instance, @xcite study topic diversity in the conversational flow, which is rather important in chit-chat and opendomain dialogues, but less so for task-oriented ones; @xcite , @xcite , @xcite propose to use adversarial evaluation, whereas @xcite , @xcite propose automatic systems that build upon human evaluation. all these efforts are still preliminary and are not easily employable for new datasets or new models. since no standard and unique metric has been proposed to evaluate the quality of taskoriented (grounded) conversational dialogues, we consider a mixture of metrics used independently in various studies, and we provide a comparative analysis across models and tasks based on the same set of linguistic metrics. neural networks have been shown to generate text that sounds unnatural due to the presence of repeated utterances, poor vocabulary, and inconsistency in word usage @xcite . various improvements have been proposed to mitigate these weaknesses. to prevent the decoder from choosing words based simply on their frequency, @xcite replace its maximum likelihood estimation objective, while others change the sampling search strategy @xcite @xcite ; these changes aim to reduce the number of repeated questions, to increase the variety of words and their distribution. attempts have been made to provide the conversational models with a reasoning module based on bayesian inference @xcite or rational speech act @xcite frameworks that should lead to more informative and coherent questions. here, we do not propose new models, but rather aim to better understand the strengths and weaknesses of current models. ## games and metrics our focus is on task-oriented dialogues. we consider a task that relies on grounding language into vision during question generation, i.e. guesswhat @xcite , a task that requires grounding only at the guessing phase, i.e. guesswhich @xcite , and a task based only on language, i.e. mutualfriends, @xcite . games as illustrated by the snippets reported in guesswhat @xcite is an asymmetric game. foot_0 a questioner (q-bot) has to ask yes/no questions to guess which is the target object among a set of maximum 20 candidates; while asking questions, it sees the image containing the guesswhich @xcite is also an asymmetric game. unlike the task described above, the q-bot has to ask questions without seeing the candidate images, but it has access to captions describing the images. q-bot can ask any type of question; the target image has to be selected among 2k candidates at the end of the dialogue. the a-bot instead sees both the caption and the target image. human dialogues are from the visdial dataset 2 and were collected as chitchat dialogues @xcite . both humans and models have to ask exactly 10 questions. mutualfriends @xcite is a symmetric game based only on text: two agents, each given a private list of friends described by a set of attributes/labels, try to identify their mutual friend based on the friend's attributes. metrics: since we are interested in the interplay between the downstream task and the quality of the generated dialogues, we consider two types of metrics. task success: we use the task success (ts) metrics used in the literature to evaluate models against these tasks, namely accuracy (acc) for 2 visdial is available from https:// visualdialog.org/data . guesswhat and mutualfriends, and mean percentile rank (mpr) for guesswhich. the latter is computed from the mean rank position (mr) of the target image among all the candidates. an mpr of e.g., 96% means that, on average, the target image is closer to the one chosen by the model than the 96% of the candidate images. hence, in the visdial test set with 2k candidates, 96% mpr corresponds to an mr of 80, and a difference @xmath0. the task success chance levels are: 5% accuracy (guess-what), 50% mpr (guesswhich) and 11.76% accuracy (mutualfriends). linguistic metrics: it has been shown that the quality of the dialogues generated by computational agents is not satisfactory. the main weaknesses of these models consist of poor lexical diversity, a high number of repetitions, and the use of a limited vocabulary. to evaluate the quality of the generated dialogues (defined as the closeness to human dialogues according to surfacelevel cues), we use several metrics that have been proposed in the literature. as in @xcite , we compute unigram entropy (h), which measures the entropy of unique unigrams in the generated dialogues normalized by the total number of tokens used by the model. from @xcite , we take the mutual overlap (mo) metric, which evaluates the question diversity within a dialogue by computing the average of the bleu-4 score obtained by comparing each question with the other questions within the same dialogue. 3 moreover, following @xcite , we report the percentage of games with one question repeated verbatim (grq) within a dialogue. finally, we compare models with respect to their ability on lexical acquisition by calculating the global recall (gr) introduced by @xcite to evaluate image captioning: it is defined as the overall percentage of learnable words (from the training set) that the models recall (use) during generation. furthermore, taking inspiration from the local recall introduced in the same work, we propose a similar metric tailored to dialogues, i.e., local recall-d (lrd), which measures how many content words the generated dialogue shares with the corresponding human dialogue for the same game. given a human dialogue d h about an image and a generated dialogue d g about the same image, we compute lrd as the normalized lexical overlap (considering only content words) between d h and d g . we sum up all these linguistic metrics used in the literature so far into one which we take as a proxy of the quality of dialogues: it shows the linguistic divergence (ld) of the dialogues generated by a model from human dialogues. to this end, we normalize each metric so that all values lie between 0 and 1: 0 stands for human performance for the "lower is better" metrics and 1 stands for human performance for "higher is better" metrics. we compute ld by averaging all the scaled values v for each model; we take 1 -v for "higher is better" metrics to obtain a "divergence" value. all the metrics are equally weighted. by definition, ld is 0 for human dialogues. ld captures three main surface-level aspects: overall vocabulary usage (h, gr), diversity of questions/phrases within a dialogue (mo, grq), and similarity of content word usage with respect to human dialogues (lrd). there could be some correlation between metrics capturing similar aspects of language quality, but this does not affect the validity of the proposed ld metric. ## models for both visual dialogue games, guesswhat and guesswhich, supervised learning has been compared with other learning paradigms. after the random questions taken from other dialogues. embeddings are obtained by using universal sentence encoder-use @xcite . we found that novel and low-mo questions are more similar to their dialogue than the random ones, confirming the effectiveness of these metrics. guesswhat we use the a-bot introduced in de @xcite , which is trained in a supervised learning (sl) fashion. for the q-bot, we compare models based on different learning paradigms: supervised and cooperative learning (gdse-sl and gdse-cl, respectively) proposed in @xcite and reinforcement learning (rl) proposed in @xcite . in rl, the reinforce paradigm used aims at optimizing the task accuracy of the game. besides using different learning paradigms, these models differ in their architecture. in particular, while in rl the question generator (qgen) and the guesser are trained independently, in gdse a common visually-grounded dialogue state encoder is used and the two modules are trained jointly. in both cases, the guesser receives as input the candidate object's categories and their spatial coordinates, and during training it is updated only at the end of the dialogue. 4 guesswhich we use the a-bot introduced in @xcite . for the q-bot, we compare diverse @xcite and recap @xcite . diverse and recap have similar architectures: several encoders incrementally process the linguistic inputs to produce the dialogue hidden state. this state is used to condition a decoder that generates a new question at each turn, and a guesser that is trained to produce the visual representation of the target image through a feature regression module. the two models differ in the encoders used and in the training paradigm. while diverse encodes the caption together with the dialogue history through a hierarchical lstm, recap has two independent lstms that produce the linguistic features of the caption and of the dialogue history, merged together to produce the dialogue hidden state. secondly, in diverse, an auxiliary objective on the dialogue state embedding (huber loss) is used to incentivize the bot to ask more diverse questions with respect to the immediate previous turn. in recap, the guesser sees the ground-truth image only at the end of the game while in diverse the guesser is updated at each turn. recap has been trained only by sl, diverse both by sl (d-sl) and sl plus rl (d-rl). further details can be found in the respective papers @xcite . 5 mutualfriends we evaluate the model proposed in @xcite , dynonet (dynamic knowledge graph network), in which entities are structured as a knowledge graph and the utterance generation is driven by an attention mechanism over the node embeddings of such graph. the model is trained via supervised learning and at test time it plays with itself. dynonet consists of three components: a dynamic knowledge graph (which represents the agent's private kb and shared dialogue history as a graph), and two lstms that map the graph embedding over the nodes and generate utterances or guess the entity. 6 5 experiments and results @xcite has shown that in guesswhat task success (ts) does not correlate with the quality of the dialogues. first of all, we check to what extent this result is task and model agnostic by taking guesswhat, guesswhich and mutualfriends as case-studies and compare the behaviour of the models described above. first, we evaluate the impact of the number of epochs and the size of the training set; then, we study whether and when a lower ld could help to reach a higher ts. ## conclusion our work highlights the different complexity of two sub-tasks involved in referential guessing (visual) games: guessing the target and asking questions. we have shown that while learning to win the game can be reached in a rather short time (with a few epochs), learning to generate rich human-like dialogues takes much longer. this holds for all three tasks and models we have scrutinized independently of the size of the vocabulary, the task, and the learning paradigm used. therefore, choosing the best model only on the base of the task success could prevent the model from generating more human-like dialogues. we have shown that in guesswhich decreasing the size of the training set does not bring a drop in either ts (task success, higher is better) or in ld (linguistic divergence, lower is better) and, moreover, the dialogues play a minor role on ts. instead, for guesswhat, decreasing the size of the training dataset brings a decrease in ts and an increase in ld, and, through dialogues, models accumulate information to succeed in the task. hence, we have focused our in-depth analysis on guess-what. furthermore, we have investigated whether and when higher language quality could lead to higher task success. we have shown that if models are given human dialogues, they can reach a higher ts. hence, ld could boost ts. we have shown that this boost could help more in difficult games, i.e. those for which humans ask longer dialogues. these games contain images with more distractors and humans use less frequent words while playing them. hence, we claim that in guesswhat models could increase their accuracy if they learn to ground, encode and decode words that do not occur frequently occur in the training set. in the paper, we propose the ld metric that, despite its limitations (i.e., being based only on surface cues) represents a proxy of the quality of dialogues. we believe ld effectively captures the most common deficiencies of current models and it allows a straightforward comparison between different models. as future work, ld can be used as a training signal to improve the quality of generated dialogues. moreover, a comparison between human quality judgments and ld may shed some light on the strengths and weaknesses of this metric. further work is needed to design new metrics that capture more fine-grained phenomena and better evaluate the quality of generated dialogues.