Large-Scale Answerer in Questioner's Mind for Visual Dialog Question Generation

1 INTRODUCTION . Recent advances in deep learning have led an end-to-end neural approach to task-oriented dialog problems that can reduce a laborious labeling task on states and intents ( Bordes & Weston , 2017 ) . Many researchers have applied sequence-to-sequence models ( Vinyals & Le , 2015 ) that are trained in a supervised learning ( SL ) and a reinforcement learning ( RL ) fashion to generate an appropriate sentence for the task . In SL approaches , given the dialog histories so far , the model predicts the distribution of the responses from the task-oriented system ( Eric & Manning , 2017 ; de Vries et al. , 2017 ; Zhao et al. , 2018 ) . However , the SL approach typically requires a lot of training data to deal with unseen scenarios and cover all trajectories of the vast action space of dialog systems ( Wen et al. , 2016 ) . Furthermore , because the SL-based model does not consider the sequential characteristic of the dialog , the error may propagate over time that causes an inconsistent dialog ( Li et al. , 2017 ; Zhao & Eskenazi , 2016 ) . To address this issue , RL has been applied to the problem ( Strub et al. , 2017 ; Das et al. , 2017b ) . By learning the intrinsic planning policy and the reward function , RL approach enables the models to generate a consistent dialog and generalize better on unseen scenarios . However , these methods struggle to find a competent RNN model that uses backpropagation , owing to the complexity of learning a series of sentences ( Lee et al. , 2018 ) . As an alternative , Lee et al . ( 2018 ) have recently proposed Answerer in Questioners Mind ( AQM ) algorithm that does not depend on a limited capacity of RNN models to cover an entire dialog . AQM treats the problems as twenty question games and selects the question that gives a maximum information gain . Unlike the other approaches , AQM benefits from explicitly calculating the posterior distribution and finding a solution analytically . The authors showed promising results in the task-oriented dialog problem , such as GuessWhat ( de Vries et al. , 2017 ) , where a questioner tries to find an object that is in answerer ’ s mind via a series of Yes/No questions . The candidates are confined to the objects that are presented in the given image ( less than ten on average ) . However , this simplified task may not be general enough to practical problems where the number of objects , questions and answers are typically unrestricted . For example , GuessWhich is a generalized version of GuessWhat that has a greater number of class candidates ( 9,628 images ) and a dialog that consists of sentences beyond yes or no ( Das et al. , 2017b ) . Because the computational complexity vastly increases to explicitly calculate the information gain over the size of the entire search space , the original AQM algorithm is not scalable to a large scale problem . More specifically , the number of the unit calculation for information gain in GuessWhat is 10 ( number of objects ) × 2 ( Yes/No ) , while that of GuessWhich is 10 , 000 ( number of images ) × ∞ ( answer is a sentence ) which makes the computation intractable . One of the interesting ideas Lee et al . ( 2018 ) suggested is to retrieve an appropriate question from the training set . Retrieval-based models , which are basically discriminative models that select a response from a predefined candidate set of system responses , are often used in task-oriented dialog tasks ( Bordes & Weston , 2017 ; Seo et al. , 2017a ; Liu & Perez , 2017 ) . It is critical not to generate sentences that are ill-structured or irrelevant to the task . However , such a discriminative approach does not fit well with complicated task-oriented visual dialog tasks , because asking an appropriate question considering the visual context is crucial to successfully tackle the problem . It is noticeable that AQM achieved high performance even with a retrieval-based approach in GuessWhat by making the candidate set of questions form the training set . However , Han et al . ( 2017 ) pointed out that there exist dominant questions in GuessWhat which can be generally applied to all images ( contexts ) , such as “ is it left ? or “ is it human ? . Since GuessWhich is a more complicated task where questions dominant for the game are less likely to exist , it is another reason why the original AQM is difficult to be applied . To address this , we propose a more generalized version of AQM , dubbed AQM+ . Compared to the original AQM , the proposed AQM+ can easily handle the increased number of questions , answers , and candidate classes by employing an approximation based on subset sampling . Particularly , unlike AQM , AQM+ generates candidate questions and answers at every turn , and then selects one of them to ask a question . Because our algorithm considers the previous history of the dialog , AQM+ can generate a more contextual question . To understand the practicality and demonstrate the superior performance of our method , we conduct extensive experiments and quantitative analysis on GuessWhich . Experimental results show that our model could successfully deal with the answers in sentence and significantly decrease 61.5 % of the error while the SL and RL methods decrease less than 6 % of the error . The ablation study shows that our information gain approximation is reasonable . Increasing the number of sampling by eight times brought only a marginal improvement of percentile mean rank ( PMR ) from 94.63 % to 94.79 % , which indicates that our model can effectively approximate the distribution over the large search space with a small number of sampling . Overall , our experimental results provide meaningful insights on how AQM framework can further provide an additional improvement on top of the SL and RL approaches . Our main contributions are summarized as follows : • We propose AQM+ that extends the AQM framework toward the more general and complicated tasks . AQM+ can handle a more complicated problem where the number of candidate classes is extremely large . • At every turn , AQM+ generates a question considering the context of the previous dialog , which is desirable in practice . In particular , AQM+ generates candidate questions and answers at every turn to ask an appropriate question in the context . • AQM+ outperforms comparative deep learning models by a large margin in Guesswhich , a challenging task-oriented visual dialog task . 2 RELATED WORKS . A task-oriented visual dialog problem has recently been paid attention in the field of computer vision and natural language processing ( Kim et al. , 2017 ) . GuessWhat is one of the famous taskoriented dialog tasks , where the goal is to figure out a target object in the image through a dialog that the answerer has in mind ( de Vries et al. , 2017 ) . However , GuessWhat is relatively an easy task because it only allows the answer form of yes or no . The baseline visual question answering ( VQA ) model achieves 78.5 % . In the object guessing task ( i.e. , GuessWhat task itself ) , the state-of-the-art averaged accuracy of SL , RL ( Zhang et al. , 2018b ) , and AQM ( Lee et al. , 2018 ) reached 44.6 % and 60.8 % , and 72.9 % at the 5th round , respectively . Random guessing baseline has an accuracy of 16.0 % ( Han et al. , 2017 ) , thus RL algorithms achieve 53.3 % error decrease , whereas AQM achieves 67.7 % . GuessWhich is a cooperative two-player game that one player tries to figure out an image out of 9,628 that another has in mind ( Das et al. , 2017b ) . GuessWhich uses Visual Dialog dataset ( Das et al. , 2017a ) which includes human dialogs on MSCOCO images ( Lin et al. , 2014 ) as well as the captions that are generated . Although GuessWhich is similar to GuessWhat , it is more challenging in every task including asking a question , giving an answer , and guessing the target class . For example , unlike GuessWhat that can be answered in yes or no , the answer can be an arbitrary sentence in GuessWhich . Therefore , the VQA task in the Visual Dialog dataset is much studied than the GuessWhat dataset ( Lu et al. , 2017 ; Seo et al. , 2017b ) . Similar to GuessWhat , SL and RL approaches have been applied to solve the GuessWhich task and they showed a moderate increase in performance ( Das et al. , 2017b ; Jain et al. , 2018 ; Zhang et al. , 2018a ) . However , based on the authors ’ recent Github implementation1 of the papers in ICCV ( Das et al. , 2017b ) , SL and RL methods have shown that only 6 % of error is diminished through the dialog compared to the zeroth turn baselines which only use generated caption . 3 ALGORITHM : AQM+ . 3.1 PROBLEM SETTING . In our experiments , a questioner bot ( Qbot ) and an answerer bot ( Abot ) cooperatively communicate to achieve the goal via natural language . Under the AQM framework , at each turn t , Qbot generates an appropriate question qt and guesses the target class c given a previous history of the dialog ht−1 = ( q1 : t−1 , a1 : t−1 , h0 ) . Here , at is the t-th answer and h0 is an initial context that can be obtained before the start of the dialog . We refer to the random variables of target class and the t-th answer as C and At , respectively . Note that the t-th question is not a random variable in our information gain calculation . To distinguish from the random variables , we use a bold face for a set notation of target class , question , and answers ; i.e . C , Q , and A . Figure 1 explains the AQM+ algorithm applied to GuessWhich game . In Figure 1 , c is the image with three elephants , q1 is “ Are there many people ? ” , a1 is “ Yes it is. ” , a2 is “ How many elephants ? ” , and h0 is “ There are elephants walking in the zoo. ” In GuessWhich game , C is the set of test images whose size is 9,628 . The size of Q and A is theoretically infinity as questions and answers can be more than one word . 3.2 PRELIMINARY : SL , RL , AND AQM APPROACHES . In SL and RL approaches ( Das et al. , 2017b ; Jain et al. , 2018 ; Zhang et al. , 2018a ) , Qbot consists of two RNN modules . One is “ Qgen ” , a question generator finding the solution that maximizes its distribution p† ; i.e . q∗t = argmax p † ( qt|ht−1 ) . The other is a “ Qscore ” , a class guesser using score function for each class f‡ ( c|ht ) . Two RNN modules can either be fully separated two RNNs ( Strub et al. , 2017 ) , or share some recurrent layers but have a different output layer for each ( Das et al. , 2017b ) . On the other hand , in the previous AQM approach ( Lee et al. , 2018 ) , these two RNN-based models are substituted to the calculation that explicitly finds an analytic solution . It finds a question that 1https : //github.com/batra-mlp-lab/visdial-rl maximizes information gain or mutual information Ĩ , i.e . q∗t = argmaxqt∈Qfix Ĩ [ C , At ; qt , ht−1 ] , where Ĩ [ C , At ; qt , ht−1 ] = ∑ c∈C ∑ at∈A p̂ ( c|ht−1 ) p̃ ( at|c , qt , ht−1 ) ln p̃ ( at|c , qt , ht−1 ) p̃′ ( at|qt , ht−1 ) . ( 1 ) Here , a posterior function p̂ can be calculated with a following equation in a sequential way , where p̂′ is a prior function given h0 . p̂ ( c|ht ) ∝ p̂′ ( c|h0 ) t∏ j=1 p̃ ( aj |c , qj , hj−1 ) = p̂ ( c|ht−1 ) p̃ ( at|c , qt , ht−1 ) ( 2 ) In AQM , Equation 1 and Equation 2 can be explicitly calculated from the model . For ease of reference , let us name every component one by one . A module that calculates an information gain Ĩ is referred to as “ Qinfo ” and a module that finds an approximated answer distribution p̃ ( at|c , qt , ht−1 ) is referred to as “ aprxAgen ” . In AQM , aprxAgen is a model distribution that Qbot has in mind where the target is the true distribution of an answer generator p̄ ( at|c , qt , ht−1 ) , which is referred to as “ Agen ” . Finally , “ Qpost ” denotes a posterior p̂ calculation module for guessing a target class . As AQM uses full set of C and A , the complexity depends on the size of C and A . For the question selection , AQM uses a predefined set of candidate questions ( Qfix ) , which is not changed for a different turn .

Summary: The paper proposes an improvement over the AQM approach for an information-theoretic framework for task-oriented dialog systems. Specifically, the paper tries to circumvent the problem of explicitly calculating the information gain while asking a question in the AQM setting. While the original AQM approach sweeps over all possible guesses and answers while estimating information gain, this is rendered impractical in scenarios where this space cannot be tractably enumerated. As a solution, AQM+ proposes sweeping over only some top-k relevant instantiations of answers and guesses in this space by normalizing the probabilities of the subset of the space in consideration. In addition, unlike AQM, AQM+ can ask questions which are relevant to the dialog context so far. Consequentially, this is generalizable and applicable for dialog systems with non ‘yes/no’ answers. Empirical observations demonstrate improvements over the existing approaches for such task-oriented dialog systems. The paper is not very well-written and at times is hard to understand. The contributions seem incremental as well in addition to the concerns mentioned below.