Enhancing Visual Dialog Questioner with Entity-based Strategy Learning and Augmented Guesser (2021)

Abstract: Considering the importance of building a good Visual Dialog (VD) Questioner, many researchers study the topic under a Q-Bot-A-Bot image-guessing game setting, where the Questioner needs to raise a series of questions to collect information of an undisclosed image. Despite progress has been made in Supervised Learning (SL) and Reinforcement Learning (RL), issues still exist. Firstly, previous methods do not provide explicit and effective guidance for Questioner to generate visually related and informative questions. Secondly, the effect of RL is hampered by an incompetent component, i.e., the Guesser, who makes image predictions based on the generated dialogs and assigns rewards accordingly. To enhance VD Questioner: 1) we propose a Related entity enhanced Questioner (ReeQ) that generates questions under the guidance of related entities and learns entity-based questioning strategy from human dialogs; 2) we propose an Augmented Guesser that is strong and is optimized for VD especially. Experimental results on the VisDial v1.0 dataset show that our approach achieves state-of-the-art performance on both image-guessing task and question diversity. Human study further verifies that our model generates more visually related, informative and coherent questions.

## introduction visual dialog (vd), which expects ai agents to conduct visually related dialog, has attracted growing interests due to its research significance and application prospects. most of the work @xcite @xcite @xcite @xcite pays attention to modeling an answerer agent. however, it is also important to model a vd questioner agent that can constantly ask visually related and informative questions. previous researches @xcite @xcite have explored building open-domain vd questioner under a q-bot-a-bot image-guessing game setting, namely guesswhich @xcite . given an undisclosed image, guesswhich can be regarded to have two stages: 1) dialog generation stage: q-bot (questioner, who only knows a caption of the image at first) successively asks questions to collect information about the image, and a-bot (answerer, who can see the image) answers the questions. 2) guess stage: q-bot guesses the target image based on the generated dialog. corresponding to the two stages, q-bot has two roles, i.e, question generator (qgen) and guesser foot_0 . besides supervised learning (sl), previous methods @xcite @xcite introduce reinforcement learning (rl) to further fine-tune the agent. though progress has been made, issues still exist. firstly, previous work does not provide explicit and effective guidance to generate visually related and informative questions. to encourage diverse questions, @xcite penalize the similarity in successive textual dialog hidden states. but this method can not promise the diverse questions are visually related. to ask visually related questions, @xcite retrieve the most likely image at each round to provide questioner with visual information. yet, an image contains many contents while the method does not provide explicit guidance for questioner to ask about which one. secondly, the reward in rl is not efficient due to an incompetent guesser, hampering the effect of rl optimization. at each round of the dialog, guesser makes an image feature prediction based to remedy above issues, we propose a related entity enhanced questioner (reeq) and an augmented guesser (augg) to enhance the vd questioner in both sl and rl. reeq is a questioner that explicitly uses related entities as guidance to generate questions and learns entity-based questioning strategy through large-scale human dialogs. in concrete, reeq firstly uses the image caption to retrieve related entities, which are preprocessed to be related to the target image; then at each round of the dialog, it selects which entity to ask about according to current dialog condition; lastly, it uses the selected entity as a hint to guide question generation. the related entities help reeq to ask visually-related questions while questioning strategy-learning enables it to ask constantly informative questions. augg is a strong guesser that is optimized with a special consideration for the vd setting. specifically, we separately train the augg with a hinge loss that incorporates hard negative samples. in particular, we introduce the competitive vd-oriented negative samples, which are images that contain alike visual contents related to the caption of target image, so as to enforce more distinguishable image feature predictions from the model, especially when dialog contexts are similar. in rl, we use augg to assign reliable rewards to further improve the questioner. we evaluate our method on the visdial v1.0 dataset @xcite . experimental results show that our approach achieves state-of-the-art (sota) performance on both the image-guessing task and question diversity. human study indicates that our questioner generates more visually related, informative and coherent questions as compared to previous strong baselines. our main contributions foot_3 are concluded as follows: ## background guesswhich @xcite ) is an interactive q-bot-a-bot image-guessing task. q-bot, who only knows the caption of an undisclosed image i at first, needs to ask a series of questions and guess the target image. a-bot, who can see the image, answers accordingly. in this section, we formally introduce the modeling of q-bot and a-bot in previous methods @xcite , as well as the training paradigm. ## approach in this section, we introduce the related entity enhanced questioner (reeq), augmented guesser (augg) and training approach. reeq generates questions under the guidance of related entities and learns entity-based questioning strategy from human dialogs. augg is a strong guesser and assigns rewards during the rl optimization process. ## discussion entity-selection accuracy. considering the diverse questioning strategies in real-world scenes, "correct" entity is hard to define. therefore, we conduct human study. we sample 50 generated dialogs (selected entities highlighted) in val-set and ask 3 evaluators to judge whether entities are relevant to the image and caption (i.e., are correct in the current context). 92.5% selected entities are regarded as relevant (qualitatives in appendix e). additionally, we study the ability to select entities just the same as entities in ground-truth human dialogs. evaluation result on val-set shows 14% selected entities are same, indicating the model has learned from human dialogs since there are 100 candidates.14% is not high, but it is reasonable considering the rich visual scenes and various questioning paths. computational cost. between reeq and dasq, the ratio of time to get the best performed model is 1.47 (11h vs. 7.5h) in sl and 1.38 (14.5h vs. 10.5h) in rl. at inference, reeq spends 1.1 times the baseline (72s vs. 64s). we conclude reeq costs more for estimating the entities to ask and generating entity-guided questions. despite additional time cost, generation results of reeq are inspiring. ## related work our work is mostly related to building opendomain visual dialog questioner in the imageguessing task setting. @xcite propose the task and generate questions in a sequence-tosequence fashion. @xcite propose to reduce repetition by penalizing the similarity in successive dialog hidden states. @xcite retrieve the most-likely image, encode the image into a multi-modal context vector and use it to decode questions. these methods follow a sequenceto-sequence fashion while reeq explicitly uses related-entities as guidance to generate questions following a learned strategy. our work is also relevant to the works @xcite @xcite @xcite ) that focus on vd questioner for guess-what?! @xcite , where the goal is to locate a target object in the image and the answers can only be "yes/no/not available". compared to them, building a questioner in a more open-domain vd setting is of more difficulty. moreover, q-bot in guesswhich has no access to visual information, making it harder to ask visually related questions. ## conclusion in this paper, we propose related entity enhanced questioner (reeq) and augmented guesser (augg) to enhance visual dialog questioner in both sl and rl. reeq generates questions with related entities as guidance and learns an entity-based questioning strategy from human dialogs. augg is a strong guesser that is optimized for vd especially. we use augg to assign reliable rewards in rl. experimental results on visdial v1.0 show our method outperforms priors on multiple criteria.