Jointly Learning to See, Ask, Decide when to Stop, and then GuessWhat
Authors: Ravi Shekhar, Alberto Testoni, Raquel Fernández, Raffaella Bernardi (Università di Trento, University of Amsterdam, 2019)
Topic: Chatbots and Dialogue Systems

Abstract
We augment a task-oriented visual dialogue model with a decision-making module that decides which action needs to be performed next given the current dialogue state, i.e. whether to ask a follow-up question or stop the dialogue. We show that, on the GuessWhat?! game, the new module enables the agent to succeed at the game with shorter and hence less error-prone dialogues, despite a slightly decrease in task accuracy. We argue that both dialogue quality and task accuracy are essential features to evaluate dialogue systems.

Introduction
The development of conversational agents that ground language in visual information is a challenging problem that requires the integration of dialogue management skills with multimodal understanding. A common test-bed to make progress in this area are guessing tasks where two dialogue participants interact with the goal of letting one of them guess a visual target (Das et al., 2017a; de Vries et al., 2017; Das et al., 2017b). We focus on the GuessWhat?! game, which consists in guessing a target object within an image which is visible to both participants. One participant (the Questioner) is tasked with identifying the target object by asking yes-no questions to the other participant (the Oracle), who is the only one who knows the target. Participants are free to go on with the task for as many turns as required. Most models of the Questioner agent in the GuessWhat?! game consist of two disconnected modules, a Question Generator and a Guesser, which are trained independently with Supervised Learning or Reinforcement Learning (de Vries et al., 2017; Strub et al., 2017). In contrast, Shekhar et al. (2019) model these two modules jointly. They show that thanks to its joint architecture, their Questioner model leads to dialogues with higher linguistic quality in terms of richness of the vocabulary and variability of the questions, while reaching a performance similar to the state of the art with Reinforcement Learning. They argue that achieving high task success is not the only criterion by which a visually-grounded conversational agent should be judged. Crucially, the dialogue should be coherent, with no unnatural repetitions nor irrelevant questions. We claim that to achieve this, a conversational agent needs to learn a strategy to decide how to respond at each dialogue turn, based on the dialogue history and the current context. In particular, the Questioner model has to learn when it has gathered enough information and it is therefore ready to guess the target. In this work, we extend the joint Questioner architecture proposed by Shekhar et al. (2019) with a decision-making component that decides whether to ask a follow-up question to identify the target referent, or to stop the conversation to make a guess. Shekhar et al. (2018) had added a similar module to the baseline architecture by de Vries et al. (2017). Here we show that the novel joint architecture by Shekhar et al. (2019) can also be augmented with a decision-making component and that this addition leads to further improvements in the quality of the dialogues. Our extended Questioner agent reaches a task success comparable to Shekhar et al. (2019), but it asks fewer questions, thus significantly reducing the number of games with repetitions.

Conclusion
We have enriched the Questioner agent in the goal-oriented dialogue game GuessWhat?! with a Decision Making (DM) component. Based on the visually grounded dialogue state, our Questioner model learns whether to ask a follow-up question or to stop the conversation to guess the target object. We show that the dialogue produced by our model has less repetitions and less unnecessary questions, thus potentially leading to more efficient and less unnatural interactions – a well-known limitation of current visual dialogue systems. As in Shekhar et al. (2018), where a simple baseline model was extended with a DM component, task accuracy slightly decreases while the quality of the dialogues increases. A first attempt to partially tackle the issue within the GuessWhat?! game was made by Strub et al. (2017), who added a <stop> token to the vocabulary of the question generator module to learn when to stop asking questions using Reinforcement Learning. This is a problematic approach as it requires the QGen to generate probabilities over a non-linguistic token; further, the decision to ask more questions or guess is a binary decision and thus it is not desirable to incorporate it within the large softmax output of the QGen. Jiaping et al. (2018) propose a hierarchical RL-based Questioner model for the GuessWhich image-guessing game introduced by Chattopadhyay et al. (2017) using the VisDial dataset (Das et al., 2017a). The first RL layer is a module that learns to decide when to stop asking questions. We believe that a decision making component for the GuessWhich game is an ill-posed problem. In this game, the Questioner does not see the pool of candidate images while carrying out the dialogue. Hence, it will never know when it has gathered enough information to distinguish the target image from the distractors. In any case, our work shows that a simple approach can be used to augment visually-grounded dialogue systems with a DM without having to use the high complexity of RL paradigms. Task accuracy and dialogue quality are equally important aspects of visually-grounded dialogue systems. It remains to be seen how such systems can reach higher task accuracy while profiting from the better quality that DM-based models produce.