Learning to Generate Visual Questions with Noisy Supervision
Authors: Kai Shen
Lingfei Wu
Siliang Tang
Yueting Zhuang
zhen he
Zhuoye Ding
Yun Xiao
Bo Long
Venue: NeurIPS 2021 Poster (2021)
Keywords: Noisy Supervision
Generative Adversarial Networks
Visual Question Generation

Abstract: The task of visual question generation (VQG) aims to generate human-like neural questions from an image and potentially other side information (e.g., answer type or the answer itself). Existing works often suffer from the severe one image to many questions mapping problem, which generates uninformative and non-referential questions. Recent work has demonstrated that by leveraging double visual and answer hints, a model can faithfully generate much better quality questions. However, visual hints are not available naturally. Despite they proposed a simple rule-based similarity matching method to obtain candidate visual hints, they could be very noisy practically and thus restrict the quality of generated questions. In this paper, we present a novel learning approach for double-hints based VQG, which can be cast as a weakly supervised learning problem with noises. The key rationale is that the salient visual regions of interest can be viewed as a constraint to improve the generation procedure for producing high-quality questions. As a result, given the predicted salient visual regions of interest, we can focus on estimating the probability of being ground-truth questions, which in turn implicitly measures the quality of predicted visual hints. Experimental results on two benchmark datasets show that our proposed method outperforms the state-of-the-art approaches by a large margin on a variety of metrics, including both automatic machine metrics and human evaluation. 

Learning to Generate Visual Questions with Noisy Supervision
Kai Shen", Lingfei Wu Siliang Tang", Yueting Zhuang,, Zhen He Zhuoye Dingt, Yun Xïao, and Bo Long*
ZZejiang University JD.COM
shenkai@zju. edu, cn, lwu@email. um, edu, {siliang, fsiliang,ythuangll yzhuang) edu edu. cn, [b (bjhezhen, dingzuuoye xiaoyun1, bo long]@jd..com
Abstract
The task of visual question generation (VQG) aims to generate human-like neural questions from an image and potentially other side information (e.g.. answer type or the answer itself). Existing works often suffer from the severe image to many questions mapping problem, which generates uninformative and non-referential questions. Recent work has demonstrated that by leveraging double visual and an- swer hints, a model can faithfully generate much better quality questions. However, visual hints not available naturally. Despite they proposed simple rule-based similarity matching method to obtain candidate visual hints, they could be very noisy practically and thus restrict the quality of generated questions. In this paper, we present novel learning approach for double-hints based VQG, which can be weakly supervised learning problem with noises. The key rationale is that the salient visual regions of interest can be viewed as constraint to improve the generation procedure for producing high-quality questions. As result, given the predicted salient visual regions of interest, we can focus on estimating the probability of being ground-truth questions, which in turn implicitly measures the quality of predicted visual hints. Experimental results two benchmark datasets show that our proposed method outperforms the state-of-the-ar approaches by a large margin on variety of metrics, including both automatic machine metrics and human evaluation.
1 Introduction
Recent years have witnessed a surge of interest in the Visual Question Generation (VQG) task. VQG aims generate human-like neural questions from image and potentially other side information (e.g., answer type or the answer itself). VQG is one of the effective data augmentation approaches for providing high-quality synthetic training data for visual question answering (VQA) (28 and visual dialog system [19). Moreover, VQG models are also particularly useful for the few-shot learning or zero-shot learning 36 44 Conceptually, VQG very challenging task since the generated questions are not only required to be consistent with the image content but also meaningful and answerable to humans.
To handle the VQG task, recent approaches explore three directions mainly. One direction is focusing on VQG without any hints. Although more diverse questions be generated, often leads to non-informative questions and hard to find any corresponding answers. Furthermore, in view of data augmentation, successful VQG system should be goal-driven in order to provide valuable additional QA pairs to further enhance generalization capability of its dual VQA task 22).
Both Authors Contributed Equally. This work was conducted when the first author was an intern in JD.COM 8 Corresponding Author
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

Another direction utilizes answer-type hints for VQG in order to generate more referential questions while exploiting diverse questions. However, given one image. there are often multiple questions with the same question types (i.e., what, how, and how many). If only answer-type is used, the generated questions are often either not referred to any existing answers or not informative questions. As result, these generated questions either cannot be used for training VQA or can be too noisy to degenerate the generalizability of VQA.
The last direction exploits the textual answer information as answer hints for VQG. This is particularly useful when two questions are the same question types but different questions for an image. As result, the answer hint can often greatly reduce the ambiguities when learning to generate visual questions. Therefore, it can train a better question generator for more diverse questions instead of specific questions. Unfortunately, there are still many cases where answer hints are not informative enough to reduce the ambiguities for generating a question, especially when there are multiple visual objects that are associated with answer hint.
Recently, Kai et al. (20) presented novel double-hints approach for VQG, which showed that the model can faithfully generate much better quality questions by leveraging double visual and answer hints. However, visual hints are not available naturally, which requires highly expensive human annotations. To handle this issue, they proposed a simple rule-based similarity matching method to obtain a set of visual hints, which are often too noisy to generate high-quality questions. As result, the performance of VQG could be highly restricted by these low-quality visual hints.
Therefore, an important question is, without any expensive human annotation, can we identify important salient visual regions of interest associated with answer hints to overcome the above deficiencies? In this paper, we present novel learning approach for double-hints based VQG, which can be cast as a weakly supervised learning problem with noises. The key rationale is that, the salient visual regions of interest can be viewed as a constraint to improve the generation procedure for producing high-quality questions. As result, given the predicted salient visual regions of interest, we can focus on estimating the probability of being ground-truth questions, which in turn implicitly measures the quality of predicted visual hints.
To this end, propose a Double- Hints guided Generative Adversarial Networks (DH-GAN) con- sisting of a double-hints based question generator and a queestio--answeraave discriminator. In particular, our generator has visual hints predictor with cross- modsl reasoning module to deter- mine the salient visual regions associated to the corresponding question, and a question generation module with predicted visual hints and textual answer hints. Our discriminator is to distinguish whether sample triplet (i.e., the image, answer, and question) is generated from the generator or ground-truth. Moreover, we design novel hybrid reward function that combines the generator and discriminator so that it encourages the model to perform both exploration of question sample and the visual hints via policy gradient. Experimental results on two benchmark datasets show that our proposed DH-GAN outperforms the state-of- metrics, including both automatic machine metries and human evaluation.
approaches by a large margin on a variety of
2 Double-hints Guided Generative Adversarial Networks for VQG
In this section, we first introduce the double-hints based visual question generation (VQG) problem and then discuss the overall model architecture. Next, we detail each key components of our proposed DH-GAN model for double-hints based VQG in the subsequent sections.
2.1 Problem Formulation and Overall Model Architecture
Problem Formulation. Conceptually, the goal of VQG is to generate natural questions based given form of visual data (i.e., images (22] and videos (46)), together with other side information (i.c., answers 28), answer-types [22], or both textual answers and visual regions 20)). In this paper, we focus on the task of generating high-quality questions from images with double text and visual hints.
Given an image I, we assume an image is represented by a set of visual regions R = fri,r2..... .... T2...., PN with initial embedding number of visual regions. Similarly, we assume the answer A = {ai, a22 .... .....m] with initial embedding (al, a2; ... am) and the target question Q = {q1, -.. 4n with initial embedding {qi, 92. qn } are collection of words, respectively. Next, the visual hints V = {r1, ..., PT)
[Vi, V2,... vN] via an object detection method (40], where N is the
2

---

Ground truth Question (@) color being thrown... ...
Q
Discriminator
Ground truth Question (0)
Attention
Answer: White
Answer
Quention
Hybrid
Real? Fake?
Generation Quality
Semantic Quality
Sampling Decoding
Greedy Decoding
Faster RCNN
Visual Hint Prodictoo
Predicted Visual Hints
Language LSTM
Cord
Conul
Double- Attention
Dlated-Cana
Vision
Ansuwer:
Policy Gradient
Figure 1: The proposed DH-GAN Model Architecture with Noisy Supervision.
are the visual regions of interest, which are (semantically) associated with the corresponding textual answer. Note that, V is the subset of R (V S R). The double-hints based VQG task aims to generate not only answerable but also referential questions based on the given image I and the double hints A and V, which maximizes the following conditional likelihood
Q = argmax P(Q)I, =
(1)
where the visual hints V are obtained through the alignments between the image I and the answer A. Note that, since there are no additional human annotations when obtaining target visual hints during pre-processing, is not surprising that they are very noisy, which leads to the non-accurate prediction of V during inference.
The proposed DH-GAN Framework. Despite the potentially useful additional information the visual hints may carry, how identify the salient visual regions of interest from set of visual region candidates without any human annotations a very challenging task. Therefore, it is crucial that proposed model is able learn generate visual questions with noisy supervision. In order to effectively address this challenge, we propose the DH-GAN model, which consists of two key generative model G and discriminative model as shown in Fig.
components:
Formally, we denote the sample triplet as x A1 (I for image, Q for question, and A for answer). We further denote x+ {I, Q f0+ A} as the annan-en sample ("real") and = [I, Q A} as the sample generated by the system ("fake"). Generally speaking. the goal the generator G is to first select the salient visual hints from the input image and the textual answer and then generate the question based on the predicted visual hints and the corresponding textual answers. Since there is direct alignment information between the visual hints and answer, the goal of the discriminator is to implicitly measure the quality of predicted visual hints by judging whether the current sample triplet is "real" or "fake". In other words, the generator G and discriminator D undergo an adversarial process to encourage the generator to identify the salient visual region of interest that the high-quality question can be generated, which indistinguishable from the "real" ones by the discriminator D. Once the Nash Equilibrium is achieved via GAN, the generated question Q would be very close to Q* in the discriminator's embedding space semantically. In what follows, we will discuss each component of the proposed DH-GAN in detail.
2.2 G-step: Double-hints Guided Question Generator
Since visual hints are not available naturally, is unclear how effectively identify the salient visual hints and leverage them for generating the high-quality question. In this section, will introduce two important steps of G-step: identifying visual hints and ii) generating questions with double visual and text hints.

---

2.2.1 Identifying Visual Hints
To exploit the rich correlations of the visual regions, inspired by ResNet 17, we utilize the following relation-aware encoder module:
V out (DiniiiieaateeCnsssCoo (V)) + V
(2)
where V E IR-xxd Vour E xg are input and output features for the visual regions, Conv (-) the CNN layer with kernel size 1, DilatedConvs(-)) is the dilated layer with kernel size 3. Each convolution layer is followed by the batch normalization with a ReLU activation. Empirically, stacking multiple modules typically capture more concise relations between regions. We thus stack k modules to obtain & + number of representations of the visual regions in different subspaces, denoted as {V(0), v(), .... v(k),, J. where V(O) is the original visual features (V) and V is the output of layer I. We assume that each layer can learn the specific visual meaning, which will be used later to align with the corresponding textual answers.
We adopt the bidirectional GRU network to encode the input answer A {[1, 02 .... am } with initial embedding [a1, ag...., am J. We then concatenate the last hidden state of each directions to present the answer
single vector a E Rdd Similarly, to better align with visual regions, we also utilize multiple linear projections to project answer representation a into k + 1 number latent subspaces, denoted as {a("), (1),, a (a)), j. Thus, for each subspace E [0, kd), we employ attention mechanism learn the relevance between each visual region !! (v (!) is the i-th row of v(O) and the answer (!) as follows:
$ T a (W + a1) + b),
where W; Rdxd W RR b RR are the learnable weights, a-)) is the Tanh(.) activation function, and 8 is the similarity score capturing the semantic relevance between the region r; and the answer A. Then we take the average of relevance score in each layer and caleulate the probability of being
visual hint:
Ee Li=0
Puh,i o
k+1
where (-) is the Sigmoid(-) function. Given the target visual hints V, we apply focal loss 32 to optimize this module, which is denoted as Luh as follows:
Lun -1 Ereh,
pens log(pen))) - Σ (1 pon,a) log(1 - Pon.),
(5)
rv
where 1), À are hyper-parameters.
2.2.2 Generating Question with Double Hints
After obtaining the visual hints, the next step is to generate the high-quality question under the guidance of double hints. We employ the attention- based hierarchical sequence decoder from [34] for the double-hints based question generation step. This module consists of two LSTMs: 1) vision LSTM and 2) language LSTM. The vision LSTM is used encode the global visual features, the input word embedding, and the answer embedding into hidden state hf E Rd. and the language LSTM is used to generate question text sequence. Note that the starting states are initialized by answer embedding. Between the two LSTMs, the attention module is employed guided by the predicted visual hints and the answer hints. Specifically, we first apply a simple pruning strategy to select the candidate visual regions and then identify the predicted visual hints to obtain Voh. E R xd using the predictive model mentioned in the previous section. Next, employ attention visual hints V with h (textural answer hint) to obtain the importance of the salient visual clues for generating the current word token in question. We refer the reader Appendix C for more details.
the predicted
We train with the cross-entropy loss function denoted as Lim- As a result, the supervised loss function for the generator G consists of visual hints prediction loss Loh and language generation loss Lim shown as follows:
L.up = Lim + ahheth
(6)
where a is the coefficient hyper-paramette

---

2.3 D-step: Question-Answer Discriminator
Although the salient visual hints are predicted by our Generator, there is no clear way how to evaluate the quality of selected salient visual hints. In this section, we will introduce the design protocol for an effective discriminator in D-step by leveraging the signals from the generated questions.
In particular, the proposed discriminator is a binary classifier that takes the sample triplet x = (T,Q, A), where Q could be either the generated question the ground-truth question, and then outputs a class label indicating whether the input triplet is from the ground-truit or the proposed generator. Therefore, the discriminator rewards the generator for identifying the most meaningful region of interest in order to leverage these visual hints to generate the best quality of questions.
Specifically, employ bidirectional GRU to encode both question Q and answer A to its vector representations q E R" and a E R', respectively. Then we attend on the visual region features V E R xdd using question vector embeddings q I to obtain the question-ave embeddings Vattn. Next, we concatenate Vattn and answer vector a to generate the probability of being human-annotated triples follows:
visual vector
p= o=o(FVaatto a|a)),
where F(.) : Rdd + R is the linear projection, at-) is the sigmoid function and the simple concatenation. We then apply binary eross-entropy function to calculate the loss denoted as LD.
2.4 Training DH-GAN via Policy Gradient
Our model is not fully differentiable due to the procedure of choosing the next token based on the probability distribution, which makes it infeasible to propagate the gradients from the discriminator the generator. Inspired by the sequence GAN 24 53 8. we adopt the policy gradient mechanism to handle this issue. Concretely, the reward score of the generated sample x=, which is formulated by the discriminator, is used reward signal for the generator. Formally, given sample æ, we define the reward function as R(x)). We adopt the reinforcement learning (RL. algorithm [47 to maximize the expected reward of generated samples x
J(0) = Eq - (1,)) (R(x )(0).
where Q is the question generated by the generator and is the model parameter. The equation can be optimized by the likelihood ratio trick [47 10)
24
g
VJ ~ [
B(x Vog IIQ-
where log P(Q-17, A) is the log-likelihood of the question generator, R(a=) ) is the reward of being "real" given a generated sample x=, and (xx)) ) is the baseline which can reduce the variance of RI. system while keeping unbiased. Obviously, how to select appropriate baseline is one of the keys to learn a useful reward.
we utilize the output of the test-time inference normalize the rewards the generator meets. At each iteration, the generator will output two results: 1) the baseline output Q which is obtained by greedy search, that is, by maximizing the output probability distribution each decoding step, 2) the sampled output Q, generated by multinomial sampling, that is, each word token n. Q is sampled according to the likelihood Pl Ter) predicted by the generator. Following that, the Eq. can be written as:
Inspired by the self-critical sequence training (SCST) 5 42
VJ R [R(U, Q,) R(L1, Q A))J log A)
(10)
Equally important, the exploration of the visual hints could lead to a better question generation. Therefore, for an individual region rú, we determine whether it could be a visual hint according to the probability of Pus.i in Eq. 4 We denote the sampled visual hints as V. It is worth noting that to make the training stable, we apply the temperature on the logits before sigmoid in Eq. 4 Intuitively, a good exploration of the visual hints would be beneficial to improve the quality of the generated questions. Thus we apply the same reward gain in Eq. 10 which can be written as:
R([T, Q A)) log P(Q)T,A) VJ TR([T
(11)
= [R( - R(U, [)) P(Q),, 1 A, D) + log P()),
5

---

2.4.1 Reward Function
One of the key factors of Reinforcement Learning is to design a proper reward function. In order to generate syntactically and semantically meaningful questions, we first consider adopting one of the automatic evaluation metrics (i.e. BLEU@4 (77) for measuring the syntactic quality of the generated questions as part of the optimization target. Since well known that the BLEU score cannot faithfully reflect the semantic quality, we further select the discriminator'ss probability output as a reward to measure the semantic quality, Note that, the larger (xx is, the more "real" question Q is. Therefore, define the hybrid reward function follows:
R(x)= BLEU(Q) + cD(z).
(12)
where BI.E(()) is the generation reward, and D(-) is the semantic reward given the ground-truth question, respectively. Moreover, is the coefficient balancing these rewards.
2.4.2 The Loss Functions of DH-GAN for VQG
Generator Loss. The loss function of the generator derived from Eq. 11 can be written as:
F - - [R([), R([I, A))loo P(Q))) P,Q),, A, + 3 log A()],,A)),
(13)
where 9 is the hyper-parameter: log (, the question generation loss with target question Q in Sec. 2.2.2 and log (( the visual hints prediction loss given target visual hints V in Sec. [.2.1 Practically, we find that it is unstable to update the generator by minimizing the loss C... Thus we combine both the teacher-forcing loss Ssup in Eq. and the reinforcement loss as:
LG A,nt + (1 Ceups
(14)
where y is scaling factor controlling the trade-off between teacher-forcing loss and RL loss.
Discriminator Loss After updating the generator for some steps, we will re-train the discriminator to classify the generated and the original samples better as follows:
LD - log D(x )
log(1 D(x ))
(15)
3 Experiments
In this section, we conduct an extensive set of experiments to demonstrate the effectiveness of the proposed method compared with existing state-of-the-art !
Dataset. We conduct the experiments on the VQA2.0 3 and COCO-QA (39 datasets. Following 20). the visual hints are generated by rule-based similarity matching (see Appendix 1.1.1 for more details). Clearly, the original visual hints are too noisy to guide the question generation. To alleviate this issue, introduce simple pruning mechanism to decrease the number of candidates while preserving the key information. After pruning, we will keep at most m visual regions to be visual hints (see Appendix 1.2). After pre-procd proppecesssngg the VQA2.0 has 278707/1 355544, and the COCO-QA dataset has 58979/29017 examples for training/validation split, respectively.
Pre-processing. For each input image, we employ pre-trained Faster-RCNI 44 for visual region detection and feature extraction (fc6) (see Appendix E.1.3). For text data, we truncate the questions longer than 20 words and build the vocabulary the words with least 3 occurrences. Since the test splits (for these two datasets) are not open for the public, we divide the validation set to 10% validation split and 90% test split.
Baseline Methods. We compare against the following baselines our experiments: i) 12Q [49 standard CNN-RNN model in image caption that maps the image to the questions directly. ii IT2Q [22]: a CNN-RNN based model which uses answer type as side information. iii) IMVQG* [2]] variational method that maximizes the mutual information between question, image, and the expected answer. iv) Dual (8]: an advanced model that combines the VQA and VQG tasks by
'The code and data for model are provided for research purposes: DH-GAN for VQG Github Repo

---

Table 1: Results VQA2.0 and COCO-QA val set. All metrics in % and B@4.BLEU@4, C-CIDEr, M-METEOR, R- R-ROUGE, S-SPICE.
Dataset
VQA 2.0
COCO-QA
Method
B@4
C
M
R
@@4
C
M
R
S
12Q
9.02
63.21
13.89
35.33
18.04
14.71
107.90
13.71
38.32
18.65
IT2Q
18.41
134.88
19.90
45.71
22.90
18.04
135.23
17.34
46.76
22.21
IMVQG
19.72
149.28
20.43
47.20
23.10
21.16
156.76
18,93
46.89
24.21
Dual
19.90
151.60
20.60
47.00
23.21
21.48
153.32
18.93
47.03
24.34
Radial
21.87
162.92
22.22
48.65
25.34
22.63
168.29
19.73
47.71
26.71
DH-VQG
22.43
180.55
22.57
49.36
27.40
23.15
175.18
20.04
47.84
27.63
DH-GAN
23.71
191.06
22.91
50.53
28.18
23.52
186.65
20.44
48.61
28.32
dual learning. v) Radial* [50] a strong baseline that evolves an answer-centric radial graph with
GNN encoding to ask questions. vi) DH-VQG [20]: recently proposed baseline that uses very noisy double-hints as side information to generate referential questions. The detailed descriptions of these baselines are provided in Appendix D. Experiments on baselines followed by are conducted using the codes originally released by authors.
Evaluation Metrics. Following previous works [22 50 20
adopt the standard linguistic
measures including BLEU (37], CIDEr [45]. METEOR 4 ROUGE-L [31] and SPICE (1) For visual hints prediction, adopt Fl metric. These scores calculated by officially released evaluation scripts'.
3.1 Results Analysis and Human Evaluation
Table T shows the results comparing our proposed method against other state-of-art baselines on various automatic evaluation metrics. Note that, due limited space, omit the reports of the result performance of the visual-hints only (please see these details in Appendix F). There are two observations worth noting here. First, our proposed DH-GAN method for VQG consistently outperforms other state-of-the-ar methods by a large margin, highlighting the importance of double visual hints and textual answer hints. Second, compared with recently proposed DH-VQG, DH-GAN improves over DH-VQG by a noticeable margin thanks to the proposed GAN method that could effectively identify the salient visual regions associated with the answer hints.
Furthermore, we conduct a human evalua- tion study assess the quality of the ques- tions generated by our proposed DH-GAN method, the ground truth (GT), and the Daseline-Radial on three different dimen- sions in terms of syntax, semanties, and rel- evance metrics, respectively. In addition, to investigate the effectiveness of DH-GAN, we also compare the results with only pre-trained generator without the proposed discriminator (abbr: that our model outperforms all strong
Table 2: Human-study Results VQA2.0.
Method
Syntax
Semantics
Relevance
Radial
4.43 (0.48)
4.44 (0.6)
3.48 (0.59)
Generator
4.67 (0.37)
4.66 (0.48)
3,86 (0.6)
DHI-GAN
4.68 (0.32)
4.65 (0.42)
4 (0.58)
GT
4.7 (0.33)
4.67 (0.44)
4.31 (0.53)
Generator). The results are shown in Table is clear baseline methods on almost all three metrics, especially on relevance score (the closest one to the ground- ground-truth), which highlights how important identify the most important visual region of interest in order to reduce the noises. Our human evaluation details are discussed in Appendix H.
3.2 Ablation Study
We further conduct the ablation study to evaluate the importance of each key component of the proposed DH-GAN method on the VQA 2.0, including visual hints, visual hints sampling, textual answer hints and answer type. Note that, in order to make the comparison consistent and clear, we use the Generator model (from our DH-GAN) as baseline (abbr: Generator) and investigate the performance effect of each component by turning off or adding modules. The ablation study results on the VQA2.0 validation set are shown in Table 3
7

---

Original image
Raw visual hints
Answer:
Detection error 7%
Others
GT: How many people can you see the boat?
9%
Syntactic etror 9%
Radial: How many boats in the picture?
Visual prediction
Generator
Ours
Reasoning error
Generator: How many boats are in the water?
33%
42%
Ours: many people
the boat?
Figure 2: Case study (left) and Error analysis (right)
There several important observations worth highlighting here. First, by turning off the visual hints (- visual hints), the performance of model drops nearly 3.2% (BLEU@4). Clearly, the visual hints are indeed helpful for generating high-quality questions. When we further replace the answer hints with only answer type, the performance continues to drop rapidly by 17.5% (BLEU@4) which indi- cates that the answer hints carry much more helpful information than the answer type.
Table 3: Ablation Results on VQA2.0. Note that "+" denotes having this component and "." means removing this component.
Method
B@4
R
DH-GAN (full model)
23.71
22.91
50.53
GAN
23.53
22.87
50.45
visual hints sampling
Generator
22.60
22.69
49.77
viisual hints
21.87
22.05
49.14
Second, comparing our DH-GAN (full model) with
the baseline Generator, we can see that the perfor- mance improves significantly by 4.9% BLEU@4)) which demonstrates the effectiveness of the GAN framework for generating informative and referen-
visual/answer hints
18.63
20.01
45.43
answer type
tial questions by identifying the most important salient visual regions. Furthermore, we notice that by removing the visual hints sampling module (+GAN visual hints sampling), the performance drops from 23.71 to 23.53 (BLEU@4), which indicates that the visual hints sampling strategy is beneficial the VQG task by encouraging the model to explore more visual hints.
3.3 Case Analysis and Error Analysis
Next, we perform the case study to illustrate the superiority our method compared with other baselines: 1) Radial, 2) generator without DH-GAN (abbr: Generator), Specifically, we visualize the visual hints: 1) generated with pre-processing visual hints (abbr: Raw visual hints), 2) predicted with only Generator (abbr: Generator), and 3) predicted with full DH-GAN (abbr: Ours). As shown in Fig. 2 (left), we can find that the raw visual hints obtained in pre-processing are quite noisy. Such noisy visual hints fail to provide the clear supervised signal to guide the generator, leading to the imprecise prediction of the generator. In contrast, with the help of DH-GAN, the generator can predict more accurate and referential ones, which faithfully guides the generator predict precise and high-quality questions. Please refer to Appendix for more detailed case studies.
Moreover, we further investigate the failures of our model by pinpointing various bad cases. The results are shown Fig. (right). Since it is very time-consuming to analyze the errors of every example, we qualitatively categorize them into five different error groups. Clearly, even with our DH-GAN method, most the bad cases are caused by the first error type visual hint prediction error, which is not surprising since we have no human annotations on visual hints. The second- largest error group about reasoning error (i.e., yes/no), which indicates the considerable improvement potential by truly understanding the logical polarity in the answer. The remaining errors are mainly from detection error and syntactic error, which can improved with better object detection tools and language models. More detailed examples and analyses are provided in Appendix J.
3.4 Data Augmentation and Zero-shot VQA
To investigate the effect of VQG for real world applications, we apply the VQG to the visual question answering (VQA) under two settings: 1) as a data augmentation method, 2) as a solution for zero-shot VQA.

---

Data augmentation is one of the most important applications of VQG. In this section, we use our proposed DH- BQG method to generate more ques- tions for training a better VQA model. We em- ploy two existing VQA models: 1) Bottom-Up Top- Down (abbr: BUTD (2) and 2) the Bilinear Attention Network (abbr: BAN [2I]). We compare our DH- VQG with Radial (a strong baseline) to demonstrate the effectiveness of the QG-driven data augmentation.
45911
Clussification Accuracy
64.22
43.03
<2991
Specifically, we split the VQA2.0 train-set (the same as the VQG dataset) z E {0.3, 0.5, 0.7). The VQA models are trained only on the part, while the other VQA variants are trained on the combination of the golden x part and the questions generated by VQG models. The results are shown in Fig. 3 Surprisingly, we observe that the Radial augmented VQA models perform inferior to the single VQA models. We think that the generated questions contain noises, which may mislead the VQA model in some ways [50]. Notably, our DH-GAN model outperforms all the baselines consistently, demonstrating the high- quality of the generated questions.
01888
60.967
BUTD
BUTD WRadia
BUTD
88.089
BAN
BAN Radiil
BAN DHHGAN
35,000
0.3
Percentage training data
Figure Data augmentation
In addition, we introduce the VQG as an effective solution for the zero-shot VQA (ZS-VQA) problem. Following the practice of [50] [29]. we first obtain the zero-shot words and
Table 4: VQG for Zero-shot VQA
Results on VQA2.0.
then construct the zero-shot dataset for VQA 2.0. Concretely,
Method
BUTD
BAN
we first find the same words between the questions and answers in the original VQA dataset (including the training and testing) by filtering out stop words. Then we uniformly sample 10% words from them and obtain the zero-shot words. Secondly. we pick up all samples whose answers contain zero-shot words and construct the ES-VAA testing set. Finally, the ZS- VQA training set which has no overlap with the 7S-VQA testing set can be constructed. Similar to the above data augmentation experiment, we employ two VQA methods: 1) BUTD, 2) BAN, and two VQG methods: 1) Radial, 2) DH- GAN. Technically, for each variant, we use the VQG method to generate the questions and produce additional training data, which can be used to train the VQA model. Following [50), since ZS words appear in the question, they can be found in the images and thus can treat them the target answers to generate questions to help the VQA model to predict the zero-shot answer. As shown in Table 4 we observe that both BUTD and BAN achieve 0% in the ZS-VQA test set. This is because without the help of VQG, both VQA models cannot see the ZS- 2o-words during the inference. However, when we leverage various VQG methods, both the VQA models achieve remarkable improvements. which demonstrate that the VQG can be very beneficial in solving the ZS- VQA problem. Furthermore, we observe that the DH-GAN variants outperform the Radial variants, which proves the superiority of our proposed method in generating high-quality QA pairs.
w/o. VQG.
0%
0%
Radial VQG
18.15%
18.37%
DH-GAN
19.21 %
19.93%
4 Related Work
Visual Question Generation. Visual question generation 355
7 38
25
48 is a promising and
attractive task in the vision and language domain. Jain et al. [88] firstly introduced the variational autoencoder (VAE) based framework to generate diverse questions. Krishna et al. (22) introduced the answer-type as side information based on the variational model generate more goal-driven
results. Furthermore, Li et al. (28), Shah et al. 4 combined the visual question answering (VQA) and visual question generation tasks by dual learning mechanism. Liu al. [33] further formulated the VQG problem as the inverse of the VQA task and regard the type-specific partial question as
side information to guide the training process. Recently, Xu et al. (50) adopted the GNN method to generate answer-specific questions. Kai et al. (20) introduced the double-hints, leveraging both textual answers and visual regions to faithfully guided the generating process, which made a valuable step to address the one-to soc-toomaayy mapping issue.

---

The generative adversarial network has achieved great success in computer vision [2 51 53
Generative Adversarial Network for Text Generation.
6 9 However, compared with images, the language is discrete, making it infeasible for gradients propagating from discriminator to generator. Li al. (24), Yu al. [53] firstly proposed the sequence GAN, which employs the Reinforcement Learning mechanism to provide feedback obtained from the discriminator
27
54
reward g proposed the MaskGAN, which trains the model on a text fill-in- the-blank or in-filling task to reduce the training instability and model drop problem. Despite the RL-based methods [26 5 M.. Kusner and Hemández-Lobato [23 attempted to find continuous approximation of discrete sampling by the Gumbel Softmax mechanism. In addition, Zhang et al. [56] proposed the continuous kernelized discrepancy function to approximate the non-differentiable sampling function in the original GAN.
to the generator. Fedus et al.
Noisy Supervision. Learning from noisy data with noisy supervision has attracted significant Li et al. [30] proposed unified framework to distill the knowledge learned from clean dataset to learn a better model from noisy labels. Han et al. [T5] proposed the "co-teaching" learning diagram for combating with noisy labels. They employ two network and let them teacher each other simultaneously. Han et al. [14] proposed husan-assisted approach called "Masking" which conveys the external rules speculate the noise transition matrix effectively. Ren et al. [4]] further studied the text classification task with rule-induced weak labels. They designed label denoiser to estimate the the source reliability to learn from noisy data. Different from these methods which mostly focus on classification task with noisy target labels, focus on generation task under noisy supervision.
amount of interests in recent years 30
15 14
16 55 52
Conclusion
In this paper, we present novel learning approach for double-hints based VQG, which can be cast as a weakly supervised learning problem with noises. In particular, we propose a Double- Hints guided Generative Adversarial Networks (DH-GAN), which consists of double-hints based question generator and question aasseioona discriminator. Thanks to DH-GAN, the generator and discriminator undergo an adversarial process encourage the generator to identify the salient visual regions of interest so that the high-quality question can be generated, which are indistinguishable from the ground truth questions by the discriminator. Our extensive experiments demonstrate the effectiveness of our proposed model.
Acknowledge
This work has been supported in part by the National Key Research and Development Program of China (2018AAA0101900) Zhejiang NSF (LR21F000004), Chinese Knowledge Center of Engineer- ing Science and Technology (CKCEST).
10

---

References
[1] Peter Anderson, Basura Fernando, Mark Johnson, and Stephen Gould. Spice: Semantic propositional image caption evaluation. In European conference computer vision, pages 382-398. Springer, 2016.
[2] Peter Anderson, Xiaodong He, Chris Buehler, Damien Teney, Mark Johnson, Stephen Gould, and Lei Zhang. Bottom-up and top-down attention for image captioning and visual question answering. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 6077-6086, 2018.
[3] Stanislaw Antol, Aishwarya Agrawal, Jiasen Lu, Margaret Mitchell, Dhruv Batra, C. Lawrence Zitmick, and Devi Parikh. Vqa: Visual question answering. In The IEEE International Confer- ence on Computer Vision (ICCV), December 2015.
[4] Satanjeev Banerjee and Alon Lavie. Meteor: An automatic metric for mt evaluation with improved correlation with human judgments. In Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization, pages 65-72, 2005.
[5] Yu Chen, Lingfei Wu, and Mohammed J Zaki. Reinforcement learning based -e-e--eeee model for natural question generation. In The Eighth International Conference on Learning Representations (ICLR 2020), 2020.
[6] Xuefeng Du, Jingfeng Zhang, Bo Han, Tongliang Liu, Yu Rong, Gang Niu, Junzhou Huang, and Masashi Sugiyama. Learning diverse-structured networks for adversarial robustness. arXiv preprint arXiv:2102 01886, 2021.
[7] Zhihao Fan, Zhongyu Wei, Piji Li, Yanyan Lan, and Xuanjing Huang. A question type driven framework to diversify visual question generation. In I/CAI, pages 4048 4084,005,4 2018.
[8] William Fedus, Ian Goodfellow, and Andrew M Dai. Maskgan: better text generation via filling in the_. arXiv preprint arXiv:1801.07736. 2018.
[9] Ruize Gao, Feng Liu, Jingfeng Zhang, Bo Han, Tongliang Liu, Gang Niu, and Masashi Sugiyama. Maximum mean discrepancy is aware of adversarial attacks. arXiv preprint 1415, arXiv:2010.1 1415,, 2020.
[10] Peter W Glynn. Likelihood ratio gradient estimation for stochastic systems. Communications of the ACM, 33(10):75-84, 1990.
[11] Hongyu Gong, Suma Bhat, Lingfei Wu, JinJun Xiong, and Wen-mei Hwu. Reinforcement learning based text style transfer without parallel training corpus. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume I (Long and Short Papers, pages 3168-3180, 2019.
[12] Ian J Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial networks. arXiv preprint arXiv: 2014.
[13] Jiaxian Guo, Mingming Gong, Tongliang Liu, Kun Zhang, and Dacheng Tao. Ltf: A label transformation framework for correcting target shift. ICML, 2020.
[14] Bo Han, Jiangchao Yao, Gang Niu, Mingyuan Zhou, Ivor Tsang, Ya Zhang, and Masashi Sugiyama. Masking: A new perspective of noisy supervision. arXiv preprint arXiv: /00. 0000000833. 2018.
[15] Bo Han, Quanming Yao, Xingrui Yu, Gang Niu, Miao Xu, Weihua Hu, Ivor Tsang, and Masashi Sugiyama. Co-teaching: Robust training of deep neural networks with extremely noisy labels. arXiv preprint arXiv: arXiv:/804.0000 1804.0687, 2018.
[16] Bo Han, Gang Niu, Xingrui Yu, Quanming Yao, Miao Xu, Ivor Tsang, and Masashi Sugiyama. Sigua: Forgetting may make learning with noisy labels more robust. In International Conference on Machine Learning, pages 4006-4016. PMLR, 2020.
11

---

[17] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.
[18] Unnat Jain, Ziyu Zhang, and Alexander G Schwing. Creativity: Generating diverse questions using variational autoencoders. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 6485-6494, 2017.
[19] Unnat Jain, Svetlana Lazebnik, and Alexander G. Schwing. Two can play this game: Visual dialog with discriminative question generation and answering. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition (CVPR), June 2018.
[20] Shen Kai, Lingfei Wu, Siliang Tang, Fangli Xu, Zhu Zhang. Yu Qiang, and Yueting Zhuang. Ask question with double hints: Visual question generation with answer-awareness and region- reference, 2021. URL httttttt::t////tettt htttpa::/opeerviiiiie
[21] Jin-Hwa Kim, Jachyun Jun, and Byoung- Tak Zhang. Bilinear attention networks, arXiv preprint arXiv:1805.07932 2018.
[22] Ranjay Krishna, Michael Bernstein, and Li Fei-Fei. Information maximizing visual question generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2008-2011 8, 2019.
[23] Matt J Kusner and José Miguel Hernández-Lobato Gans for sequences of discrete elements with the gumbel-softmax distribution. arXiv preprint arXiv: 161 1. 04051, 2016.
[24] Jiwei Li, Will Monroe, Tianlin Shi, Sébastien Jean, Alan Ritter, and Dan Jurafsky. Adversarial learning for neural dialogue generation. arXiv preprint arXiv:1701.06544 2017.
(25] Juncheng Li, Siliang Tang, Fei Wu, and Yueting Zhuang. Walking with mind: Mental im- agery enhanced embodied qa. In Proceedings of the 27th ACM International Conference on Multimedia, pages 1211-1219, 2019.
[26] Juncheng Li, Xin Wang, Siliang Tang, Haizhou Shi, Fei Wu, Yueting Zhuang, and William Yang Wang. Unsupervised reinforcement learming of transferable meta-skills for embodied navigation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pages 12123-12132, 2020.
[27] Ya Li, Xinmei Tian, Mingming Gong, Yajing Liu, Tongliang Liu, Kun Zhang, and Dacheng Tao. Deep domain generalization via conditional invariant adversarial networks. In Proceedings of the European Conference on Computer Vision (ECCV), pages 624-639, 2018.
(28) Yikang Li, Nan Duan, Bolei Zhou, Xiao Chu, Wanli Ouyang, Xiaogang Wang, and Ming Zhou. Visual question generation as dual task of visual question answering. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 61 16-6124, 2018.
[29] Yuanpeng Li, Yi Yang, Jianyu Wang, and Wei Xu. Zero-shot transfer vqa dataset. arXiv preprint 1 2018
[30] Yuncheng Li, Jianchao Yang, Yale Song, Liangliang Cao, Jiebo Luo, and Li-Jia Li. Learning from noisy labels with distillation. In Proceedings of the IEEE International Conference Computer Vision (ICCV), Oct 2017.
[31] Chin-Yew Lin. ROUGE: A package for automatic evaluation of summaries. In Text Summariza- tion Branches Out, pages 74-81, Barcelona, Spain, July 2004. Association for Computational Linguistics, URL hhtpp::/// wwwp:://ww//wwwwbbbb///b//m/ org/ attp/ooo/oool0000----- 1013
[32] Tsung- Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. Focal loss for dense object detection. Proceedings of the IEEE international conference on computer vision, pages 2980-2988, 2017.
[33] Feng Liu, Tao Xiang, Timothy M Hospedales, Wankou Yang, and Changyin Sun. ivqa: Inverse visual question answering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 861 1-8619, 2018
12

---

[34] Jiasen Lu, Jianwei Yang, Dhruv Batra, and Devi Parikh. Neural baby talk. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 7219-7228, 2018
[35] Issey Masuda Mora, Santiago Pascual de la Puente, and X Giro-i Nieto. Towards automatic generation of question answer pairs from images. CVPRW, 2016.
[36] Liangming Pan, Wenhu Chen, Wenhan Xiong, Min- Yen Kan, and William Yang Wang. Unsuper- vised multi-hop question answering by question generation. arXiv preprint arXiv:2010. 12623, 2020.
[37] Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting on association for computational linguistics, pages 311-318. Association for Computational Linguistics. 2002.
[38] Badri N Patro, Sandeep Kumar, Vinod K Kurmi, and Vinay P Namboodiri. Multimodal differential network for visual question generation. arXiv preprint arXiv: 03986, 2018.
[39] Mengye Ren, Ryan Kiros, and Richard Zemel. Exploring models and data for image question answering. In Advances in neural information processing systems, pages 3953-296.1 2015.
[40] Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster r-cmn: Towards real-time object detection with region proposal networks. arXiv preprint arXiv:1506.01499 2015.
[41] Wendi Ren, Yinghao Li, Hanting Su, David Kartchner, Cassie Mitchell, and Chao Zhang. Denoising multi-source weak supervision for neural text classification. arXiv preprint arXiv:2010.04582, 2020.
[42] Steven Rennie, Etienne Marcheret, Youssef Mrouch, Jerret Ross, and Vaibhava Goel. Self- critical sequence training for image captioning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 7008-7024, 2017.
[43] Meet Shah, Xinlei Chen, Mareus Rohrbach, and Devi Parikh. Cycle-consistency for robust. visual question answering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 6649-6658, 2019.
[44] Siamak Shakeri, Cicero Nogueira dos Santos, Henry Zhu, Patrick Ng, Feng Nan, Zhiguo Wang. Ramesh Nallapati, and Bing Xiang. End-to-end synthetic data generation for domain adaptation of question answering systems. arXiv preprint Sktiv:2010.06022 2020.
[45] Ramakrishna Vedantam, Lawrence Zitnick, and Devi Parikh. Cider: Consensus-basse image description evaluation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4566-4575, 2015.
[46] Yu-Siang Wang, Hung-Ting Su, Chen-Hsi Chang, Zhe- Yu Liu, and Winston H Hsu. Video question generation via cross-modal self-attention networks learning. arXiv preprint 100.000907 2019.
[47] Ronald J Williams. Simple statistical gradient-following algorithms for connectionist reinforce- ment learning. Machine learning, 8(3-4):229-255 1992.
[48] Yiquan Wu, Kun Kuang, Yating Zhang, Xiaozhong Liu, Changlong Sun, Jun Xiao, Yueting Zhuang, Luo Si, and Fei Wu. De-biased court's 's view generation with causality. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 763-780, 2020.
[49] Kelvin Xu, Jimmy Ba, Ryan Kiros, Kyunghyun Cho, Aaron Courville, Ruslan Salakhudinov, Rich Zemel, and Yoshua Bengio. Show, attend and tell: Neural image caption generation with visual attention. International conference on machine learning, pages 2048-2057, 2015.
[50] Xing Xu, Tan Wang, Yang Yang, Alan Hanjalic, and Heng Tao Shen. Radial graph convolutional network for visual question generation. IEEE Transactions on Neural Networks and Learning Systems, 2020.
13

---

[51] Yanwu Xu, Mingming Gong, Junxiang Chen, Tongliang Liu, Kun Zhang, and Kayhan Bat- manghelich. anneratveediiccimnnati complementary learning. In Proceedings of the AAAI Conference Artificial Intelligence, volume 34, pages 6526-6533 2020.
[52] Jiangchao Yao, Jiajie Wang. Ivor W Tsang, Ya Zhang, Jun Sun, Chenggi Zhang, and Rui Zhang. Deep learning from noisy image labels with quality embedding. IEEE Transactions on Image Processing. 28(4):1909-199 2018.
[53] Lantao Yu, Weinan Zhang, Jun Wang, and Yong Yu. Seqgan: Sequence generative adversarial nets with policy gradient. In Proceedings of the AAAI conference on artificial intelligence, volume 31, 2017.
[54] Jingfeng Zhang, Xilie Xu, Bo Han, Gang Niu, Lizhen Cui, Masashi Sugiyama, and Mohan Kankanhalli. Attacks which do not kill training make adversarial learning stronger. In Interna- tional Conference on Machine Learning, pages 11278 1287. PMLR, 2020.
[55] Jingfeng Zhang, Jianing Zhu, Gang Niu, Bo Han, Masashi Sugiyama, and Mohan Kankanhalli. Geometry-aware instance-reweighted adversarial training. arXiv preprint arX1v:2010.01736. 2020.
[56] Yizhe Zhang, Zhe Gan, Kai Fan, Zhi Chen, Ricardo Henao, Dinghan Shen, and Lawrence Carin. Adversarial feature matching for text generation. In International Conference Machine Learning, pages 4006-4015. PMLR, 2017.
[57] Shanshan Zhao, Mingming Gong, Tongliang Liu, Huan Fu, and Dacheng Tao. Domain gener- alization via entropy regularization. Advances in Neural Information Processing Systems, 33, 2020.
14


Reviews:
[{"review_id": "MWsPbkIRpmF", "reviewer": "Reviewer_pdQY", "summary": "This paper focuses on the visual question generation (VQG) problem. Given an image and an answer, the goal is to generate a relevant and understandable question that can be answered by the given answer. A recent paper proposes to leverage visual hints, i.e., regions of interest in the given image, to improve the performance of VQG. The visual hints could be generated based on similarity matching. However, such automatically generated visual hints are noisy. To fully take advantage of the noisy visual hints, this paper proposes a Generative Adversarial Network (GAN) based framework DH-GAN to enable weakly supervision of ground truth questions. It would first identify suitable visual hints, and then generate the question based on the visual hints and the answer. Intensive experiments demonstrate the effectiveness of DH-GAN.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Positive points:\n1. The research problem, i.e., double-hint-based VQG, is new and interesting. The authors observe that automatically generated visual hints are useful but noisy, and develop a tailored GAN model to enable noisy supervision.\n\n2. The manuscript is well structured and easy to follow. The motivations could encourage readers to continue to read. The proposed framework has been explained in a structured way. The connections among different components of the proposed framework have also been explained.\n\n3. A comprehensive evaluation process has been conducted, including effectiveness study, human evaluation, ablation study, and case study. The experiments not only include four standard linguistic measures, but also human evaluation, which makes the results more convincing and interesting. The implementation details have been included. The experimental results are highly likely reproducible.\n\nComments:\n1. In the Introduction Section, we could use a toy example to illustrate the differences among the three different VQG research directions.\n\n2. We could include the visual hint prediction results (in terms of F1 metric) in the main manuscript. Predicting better visual hints is one of the key novelties of this paper. It would be more convincing to show this in the experiments.\n\n3. It should be \"Implementation Details\" instead of \"Implement Details\" in the appendix.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors have explained the limitations in Section 3.3 and Appendix J. There is no potential negative societal impact.", "review_text": "", "extra_scores": {"ethical_concerns": "No.", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "12 hours"}, "extra_text": {}}, {"review_id": "LdRE-MLK5kH", "reviewer": "Reviewer_nCHe", "summary": "How to generate high-quality visual questions is an important area across ML and CV domains. However, this problem has not been well addressed for generating informative and referential questions because of one-to-many question mapping issues. This paper presents DH-GAN that leverages double-hits for the issue. To alleviate the problem of noisy hints, the authors view the solution as a weakly supervised learning problem with noises. Extensive experiment results are presented in the manuscript and its appendix, which demonstrate the effectiveness of the proposed DH-GAN.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strength:\n1.\tThis paper studies an important problem. The overall idea of the work is reasonable, which a novel solution of leveraging double hints is proposed with a combination of the GAN model. The solution of measuring the quality of predicted visual hints is good.\n2.\tThe authors propose a good model DH-GAN, consisting of double-hints guided question generator, question-answer discriminator. As the model is not fully differentiable, they utilize RL to help model training via policy gradient with a new function. All the components are put together in a nice architecture for achieving the overall goal of VQA – generating high-quality informative and referential questions.\n3.\tAs visual hints are not available naturally and previous methods can only obtain very noisy hints, they view the visual hints signals as weak labels and constraints for the whole generation. The solution looks good.\n4.\tSufficient experiments and results are presented in the manuscript and its appendix, with appropriate settings and baselines. The results show the good empirical performance of the model, with sufficient analyses.\n\nI have the following concerns:\n1.\tIn lines 129-142, the authors explain how the visual hints are predicted. However, it is unclear to me how the target visual hints V is obtained. And it seems like the multi-head attention type of method is employed when comparing answer information and visual proposals. My concern is about how they are matched and why these functions are chosen. Could you elaborate more on this part?\n2.\tAlthough the number of pages is limited, and some details are added in the appendix, I still think necessary model details and experiments should be put in the main paper rather than the appendix. I suggest re-organizing the presentation to guarantee the important results in the main paper.\n3.\tAs answers in VQA or VQG usually has only one or two words (e.g., numbers or colors), is it really suitable to represent answer features with LSTM? Have you thought about or tried other methods?\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors have addressed some of the limitations and potential negative social impact of their work.\nPlease refer to the main review for the suggestions in detail.\n", "review_text": "", "extra_scores": {"ethical_concerns": "No", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "8"}, "extra_text": {}}, {"review_id": "B2EADXE-aQ", "reviewer": "Reviewer_Kags", "summary": "This paper proposed an interesting learning approach to extend the visual question generation task. The authors continue to address the one image to many questions limitations and follow the setting of double visual and answer hints. Unlike the previous method, which leverages the rule-based method to obtain rough visual regions and learns to generate questions under the guidance of double hints faithfully, they cast the problem as a weakly supervised problem and optimize it with noisy supervision. More precisely, they regard the visual hints as a hidden constraint instead of a certain result. The proposed method outperforms existing methods on both VQA and COCO-QA tasks.", "questions": "", "limitations": "", "rating": 7, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Originality:\nThis paper proposes a novel learning framework. Although it is a follow-up of double hints VQG, this work makes valuable progress by clearly addressing the vital limitation: the visual hints are noisy, which fails to guide the training process. Furthermore, it is the first time that a weakly supervised VQG with double hints algorithm is proposed.\n \nQuality:\nThis paper makes a valuable step towards high-quality image question generation. The authors adapt the sequence-GAN to model the visual hints as latent constraints. They estimate the probability of being a human-annotated (good) question via discriminator and provide the optimization signals to the generator. Thus, it implicitly estimates the quality of visual hints, avoiding the noise which may hurt the results. This idea makes sense, and experimental results are good. From the visualization results, the DH-GAN can generate referential visual regions and high-quality questions.\n \nClarity:\nThis paper is well organized and well written. However, there are some minor typos (not all of them):\nL165: questions question\nL182: grammar error\nL184: samples sample\nand etc.\nPlease check the typos carefully.\nAnd I have a technical problem: see Limitation 2).\n \nSignificance:\nThe most important contribution is that the authors propose a novel noisy supervision-based weakly supervised learning method for double-hints VQG to address the one-to-many problem in the VQG task. This technique can bring the researchers a new insight.\n\n----------\nThe reviewer has read the author response and review comments of other reviewers. I stick to my original score and vote for acceptance on this paper.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "1. There are some typos and errors (see Main reviews).\n2. I have a technical problem to ask. You have a sampling procedure when obtaining visual hints. Is it differentiable?\n", "review_text": "", "extra_scores": {"ethical_concerns": "I think there is no ethical concern on this paper.", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "6"}, "extra_text": {}}, {"review_id": "4_DjuMgY7bK", "reviewer": "Reviewer_6a7g", "summary": "In this paper, the authors presented a new learning paradigm for generating visual questions with noisy supervision. The authors started to analyze the significant drawbacks of existing methods for VQG: i) suffer from the severe one-many mapping issue between image-questions; ii) recent SOTA leveraging double-hints (both visual and text hints) proves effective, which however the high-quality visual hints are hard to obtain. To solve these problems, this paper proposed a double-hints guided generative adversarial networks (DH-GAN). In particular, their generator has a visual hint predictor to determine the salient visual region associated with the question, and an encoder-decoder module to generate questions with predicted visual hints and answer hints. Their discriminator is designed to take the triplet (question-visual hints-answer hints) from generated or ground truth to judge how well the generated triplet is. Their experimental results on two datasets showed clear advantages over other state-of-the-art approaches. Their ablation study, error analysis and case analysis are also well performed. ", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "**Pros:**  \n\n1.  The idea of using generator-discriminator learning framework to implicitly evaluate the quality of the hidden variable (salient visual hint prediction) seems very novel, to the best of my knowledge. The proposed solution is a quite decent solution for addressing existing method’s annotation issue (or noisy issue) for double-hints based VQG. \n\n2. I found the proposed DH-GAN is a generic learning architecture, where each component (generator and discriminator) is well motivated and designed. In addition, the proposed hybrid reward function that combines both generator and discriminator for encouraging high-quality question generation is very interesting and could be transferred to other similar tasks as well. \n\n3. The paper is well written and clearly motivated. The overall experimental and ablation study results well investigated several advantages of the method in terms of both machine accuracy and human evolution compared to other state-of-the-art methods.  \n\n4. The code and dataset are provided, which is very useful for reproducing the results. \n\n**Cons:**  \n\n1. Author discussed how the new reward function is designed, which combines BLEU score and the discriminator’s probability output as the mix reward. I understand that BLEU score is picked for ensuring the syntactic quality of generated questions. However, there are multiple well-known metrics for automatically evaluating the quality of text, such as Rouge-L, METEOR, SacreBLEU. Are there any particular reasons that BLEU score is picked? \n\n2. There are quite a few hyperparameters (e.g. in (12), (13) (14)), which seem important for final performance. However, there are few discussions regarding the sensitivity of these hyperparameters. \n\n3. In Table 1, the proposed DH-GAN significantly outperforms other baselines. I am wondering is there any good way to evaluate how useful the generated questions in some real applications. \n\n4. Grammar issues should be addressed in the revised version.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes, the authors have adequately addressed the limitations and potential negative societal impact of their work.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper focuses on learning to generate visual questions with noisy supervision. The proposed problem is double-hint-based VQG, which is new and interesting. Specifically, the authors observe that automatically generated visual hints are useful but noisy. Based on such noisy problem, this paper develop a tailored GAN model to enable noisy supervision. The philosophy behind sounds quite interesting to me, namely, using generator-discriminator learning framework to implicitly evaluate the quality of the hidden variable (salient visual hint prediction). This philosophy leads to a novel algorithm design I have never seen.

The clarity and novelty are clearly above the bar of NeurIPS. While the reviewers had some concerns on the significance, the authors did a particularly good job in their rebuttal. Thus, all of us have agreed to accept this paper for publication! Please include the additional experimental results and fix some typos and errors in the next version.
Decision: Accept (Poster)