Original: Generative Self-training Improves Pre-training for Visual Dialog
Revised: Generative Self-training Improves Pre-training for Visual Dialog

Original: Anonymous Authors

Original: Abstract
Revised: Abstract

Original: Visual dialog (VisDial) is a task of answering a series of questions grounded in an image, using the dialog history as context.
Revised: Visual dialog (VisDial) is a task of answering a series of questions grounded in an image, using the dialog history as context.

Original: Prior work has trained the dialog models solely on VisDial data via supervised learning or leveraged pre-training on related vision-and-language datasets.
Revised: Prior work has trained the dialog models solely on VisDial data via supervised learning or leveraged pre-training on related vision-and-language datasets.

Original: This paper presents a semi-supervised learning approach for
Revised: This paper presents a semi-supervised learning approach for

Original: VisDial, called Generative Self-Training (GST), to enhance the pre-training.
Revised: VisDial, called Generative Self-Training (GST), to enhance the pre-training.

Original: Specifically, GST generates synthetic dialog data for unlabeled images via multimodal conditional text generation and trains the dialog model on the synthetic and the original VisDial data.
Revised: Specifically, GST generates synthetic dialog data for unlabeled images via multimodal conditional text generation and trains the dialog model on the synthetic and the original VisDial data.

Original: Moreover, we also propose perplexity-based data selection and multimodal consistency regularization for robust training of the synthetic data.
Revised: Moreover, we also propose perplexity-based data selection and multimodal consistency regularization for robust training of the synthetic data.

Original: Evaluation on VisDial v1.0 dataset shows that GST improves the pretraining and achieves new state-of-the-art results.
Revised: Evaluation on VisDial v1.0 dataset shows that GST improves the pretraining and achieves new state-of-the-art results.

Original: 1. Introduction
Revised: 1. Introduction

Original: Recently, there has been extensive research towards developing visually-grounded dialog systems (Das et al., 2017;
Revised: Recently, there has been extensive research towards developing visually-grounded dialog systems (Das et al., 2017;

Original: De Vries et al., 2017; Kottur et al., 2019;
Revised: De Vries et al., 2017; Kottur et al., 2019;

Original: Kim et al., 2019) due to their significance in many real-world applications ( e g., helping visually impaired person).
Revised: Kim et al., 2019) due to their significance in many real-world applications ( e g., helping visually impaired person).

Original: Notably, Visual Dialog (VisDial) (Das et al., 2017) has provided a testbed for studying such systems, where a dialog agent should answer a sequence of image-grounded questions.
Revised: Notably, Visual Dialog (VisDial) (Das et al., 2017) has provided a testbed for studying such systems where a dialog agent should answer a sequence of image-grounded questions.

Original: For instance, the agent is expected to answer the open-ended question like “What color is it?” .
Revised: For instance, the agent is expected to answer the open-ended question like “What color is it?” .

Original: This task requires a holistic understanding of visual information, linguistic semantics in context ( e.g., it), and most importantly, the grounding of these two.
Revised: This task requires a holistic understanding of visual information, linguistic semantics in context ( e.g., it), and most importantly, the grounding of these two.

Original: Most of the previous approaches in VisDial (Lu et al., 2017;
Revised: Most of the previous approaches in VisDial (Lu et al., 2017;

Original: Kottur et al., 2018; Gan et al., 2019; Kang et al., 2019;
Revised: Kottur et al., 2018; Gan et al., 2019; Kang et al., 2019;

Original: Nguyen et al., 2020) have trained the dialog agents solely on VisDial data via supervised learning.
Revised: Nguyen et al., 2020) have trained the dialog agents solely on VisDial data via supervised learning.

Original: More recent studies (Murahari et al., 2020; Wang et al., 2020b; Chen et al., 2022) have employed self-supervised pre-trained models such as
Revised: More recent studies (Murahari et al., 2020; Wang et al., 2020b; Chen et al., 2022) have employed self-supervised pre-trained models such as

Original: BERT (Devlin et al., 2019) or ViLBERT (Lu et al., 2019) and finetuned them on VisDial data.
Revised: BERT (Devlin et al., 2019) or ViLBERT (Lu et al., 2019) and finetuned them on VisDial data.

Original: This pretrain-thentransfer learning strategy has shown impressive results by transferring knowledge successfully from the models pretrained on large-scale data sources (Sharma et al., 2018;
Revised: This pretrain-thentransfer learning strategy has shown impressive results by transferring knowledge successfully from the models pretrained on large-scale data sources (Sharma et al., 2018;

Original: Antol et al., 2015; Zhu et al., 2015) to VisDial.
Revised: Antol et al., 2015; Zhu et al., 2015) to VisDial.

Original: Our research question is the following: How can the dialog agent expand its knowledge beyond what it can acquire via self-supervised pre-training on the provided datasets?
Revised: Our research question is the following: How can the dialog agent expand its knowledge beyond what it can acquire via self-supervised pre-training on the provided datasets?

Original: Some recent studies have shown that semi-supervised learning and pre-training are complementary in image classification (Zoph et al., 2020) and text classification (Du et al., 2021).
Revised: Some recent studies have shown that semi-supervised learning and pre-training are complementary in image classification (Zoph et al., 2020) and text classification (Du et al., 2021).

Original: Inspired by the studies, we consider semi-supervised learning (SSL) as a way to address the above question.
Revised: Inspired by the studies, we consider semi-supervised learning (SSL) as a way to address the above question.

Original: Let us assume that large amounts of unlabeled images are available.
Revised: Let us assume that large amounts of unlabeled images are available.

Original: SSL for VisDial can be applied to generate synthetic conversations for the unlabeled images and train the agent with the synthetic data.
Revised: SSL for VisDial can be applied to generate synthetic conversations for the unlabeled images and train the agent with the synthetic data.

Original: However, there are two critical challenges for this approach.
Revised: However, there are two critical challenges for this approach.

Original: First, the target output for VisDial ( i.e., multi-turn visual QA data) is more complex than that of the studies (Zoph et al., 2020; Du et al., 2021).
Revised: First, the target output for VisDial ( i.e., multi-turn visual QA data) is more complex than that of the studies (Zoph et al., 2020; Du et al., 2021).

Original: Specifically, they have addressed the classification problems, yielding class probabilities as pseudo labels (Lee et al., 2013).
Revised: Specifically, they have addressed the classification problems, yielding class probabilities as pseudo labels (Lee et al., 2013).

Original: In contrast, SSL for VisDial should generate a sequence of pseudo queries ( i.e., visual questions) and pseudo labels ( i.e., corresponding answers) in natural language to train the answering agent.
Revised: In contrast, SSL for VisDial should generate a sequence of pseudo queries ( i.e., visual questions) and pseudo labels ( i.e., corresponding answers) in natural language to train the answering agent.

Original: It further indicates that the target output should be generated while considering the visual inputs and the sequential nature of the conversation.
Revised: It further indicates that the target output should be generated while considering the visual inputs and the sequential nature of the conversation.

Original: Next, even if SSL yields synthetic dialogs via text generation, there may be noise such as generating incorrect answers to given contexts.
Revised: Next, even if SSL yields synthetic dialogs via text generation, there may be noise such as generating incorrect answers to given contexts.

Original: A robust training method is required to leverage such noisy synthetic dialog datasets.
Revised: A robust training method is required to leverage such noisy synthetic dialog datasets.

Original: In this paper, we study the above challenges in the context of SSL, especially self-training (Zoph et al., 2020; Du et al., 2021;
Revised: In this paper, we study the above challenges in the context of SSL, especially self-training (Zoph et al., 2020; Du et al., 2021;

Original: Xie et al., 2020b; He et al., 2020; Karamanolakis et al., 2021), where a teacher model trained on labeled data predicts the pseudo labels for unlabeled data.
Revised: Xie et al., 2020b; He et al., 2020; Karamanolakis et al., 2021), where a teacher model trained on labeled data predicts the pseudo labels for unlabeled data.

Original: Then, a student model jointly learns on both the labeled and the pseudolabeled datasets.
Revised: Then, a student model jointly learns on both the labeled and the pseudolabeled datasets.

Original: Unlike existing studies in self-training that have mainly studied discriminative tasks such as im- age classification (Li et al., 2019; Zoph et al., 2020; Sohn et al., 2020) or text classification (Du et al., 2021; Karamanolakis et al., 2021), we extend the idea of self-training to the task of multimodal conditional text generation.
Revised: Unlike existing studies in self-training that have mainly studied discriminative tasks such as im- age classification (Li et al., 2019; Zoph et al., 2020; Sohn et al., 2020) or text classification (Du et al., 2021; Karamanolakis et al., 2021), we extend the idea of self-training to the task of multimodal conditional text generation.

Original: To this end, we propose a new learning strategy, called Generative
Revised: To this end, we propose a new learning strategy, called Generative

Original: Self-Training (GST), that artificially generates multi-turn visual QA data and utilizes the synthetic data for training.
Revised: Self-Training (GST), that artificially generates multi-turn visual QA data and utilizes the synthetic data for training.

Original: GST first trains the teacher model (answerer) and the visual question generation model (questioner) using VisDial data.
Revised: GST first trains the teacher model (answerer) and the visual question generation model (questioner) using VisDial data.

Original: It then retrieves a set of unlabeled images from a Web image dataset, Conceptual 12M (Changpinyo et al., 2021).
Revised: It then retrieves a set of unlabeled images from a Web image dataset, Conceptual 12M (Changpinyo et al., 2021).

Original: Next, the questioner and the teacher alternately generate a series of visual QA pairs for the retrieved images.
Revised: Next, the questioner and the teacher alternately generate a series of visual QA pairs for the retrieved images.

Original: Finally, the student is trained on the synthetic and the original VisDial data.
Revised: Finally, the student is trained on the synthetic and the original VisDial data.

Original: We also propose perplexity-based data selection (PPL) and multimodal consistency regularization (MCR) to effectively train the student with the noisy dialog data.
Revised: We also propose perplexity-based data selection (PPL) and multimodal consistency regularization (MCR) to effectively train the student with the noisy dialog data.

Original: PPL is to selectively utilize the answers whose perplexity of the teacher is below a threshold.
Revised: PPL is to selectively utilize the answers whose perplexity of the teacher is below a threshold.

Original: MCR encourages the student to yield consistent predictions even when the perturbed multimodal inputs are given.
Revised: MCR encourages the student to yield consistent predictions even when the perturbed multimodal inputs are given.

Original: Our key contributions are three-fold:
Revised: Our key contributions are three-fold:

Original: • We introduce Generative Self-Training (GST) to study the effect of semi-supervised learning on top of the pre-training method.
Revised: • We introduce Generative Self-Training (GST) to study the effect of semi-supervised learning on top of the pre-training method.

Original: • We show the efficacy of the perplexity-based data selection (PPL) and the multimodal consistency regularization (MCR) when training the synthetic data.
Revised: • We show the efficacy of the perplexity-based data selection (PPL) and the multimodal consistency regularization (MCR) when training the synthetic data.

Original: Finally, we validate GST on VisDial v1.0 dataset, and GST achieves new state-of-the-art results on both datasets.
Revised: Finally, we validate GST on VisDial v1.0 dataset, and GST achieves new state-of-the-art results on both datasets.

Original: We also show that GST and self-supervised pre-training are complementary in VisDial.
Revised: We also show that GST and self-supervised pre-training are complementary in VisDial.

Original: 2. Approach 2.1. Preliminaries
Revised: 2. Approach 2.1. Preliminaries

Original: Self-Training.
Revised: Self-Training.

Original: We have a labeled dataset L = { ( x n , y n ) }
Revised: We have a labeled dataset L = { ( x n , y n ) }

Original: Nn =1 and an unlabeled dataset U
Revised: Nn =1 and an unlabeled dataset U

Original: =
Revised: =

Original: { ˜ x m } Mm =1 .
Revised: { ˜ x m } Mm =1 .

Original: Typically, self-training first trains a teacher model P T on the labeled dataset L .
Revised: Typically, self-training first trains a teacher model P T on the labeled dataset L .

Original: The teacher then predicts the pseudo label ˜
Revised: The teacher then predicts the pseudo label ˜

Original: y for the unlabeled data ˜ x ∼ U , constructing the pseudo-labeled dataset ˜ L
Revised: y for the unlabeled data ˜ x ∼ U , constructing the pseudo-labeled dataset ˜ L

Original: = { (˜ x m , ˜ y m ) }
Revised: = { (˜ x m , ˜ y m ) }

Original: Mm =1 .
Revised: Mm =1 .

Original: Finally, a student model P S is trained on L ∪ ˜ L .
Revised: Finally, a student model P S is trained on L ∪ ˜ L .

Original: Many variants have been studied on this setup: (1) selecting the subset of the pseudo-labeled dataset (He et al., 2020; Xie et al., 2020b;
Revised: Many variants have been studied on this setup: (1) selecting the subset of the pseudo-labeled dataset (He et al., 2020; Xie et al., 2020b;

Original: Sohn et al., 2020), (2) adding noise to inputs (Zoph et al., 2020;
Revised: Sohn et al., 2020), (2) adding noise to inputs (Zoph et al., 2020;

Original: He et al., 2020), and (3) iterating the above setup multiple times (He et al., 2020; Xie et al., 2020b).
Revised: He et al., 2020), and (3) iterating the above setup multiple times (He et al., 2020; Xie et al., 2020b).

Original: Visual Dialog.
Revised: Visual Dialog.

Original: The visual dialog (VisDial) dataset (Das et al., 2017) contains an image v and a visually-grounded dialog d = { c (cid:124)(cid:123)(cid:122)(cid:125)
Revised: The visual dialog (VisDial) dataset (Das et al., 2017) contains an image v and a visually-grounded dialog d = { c (cid:124)(cid:123)(cid:122)(cid:125)

Original: d, ( q 1 , a gt 1 ) (cid:124) (cid:123)(cid:122) (cid:125) d, · · · , ( q T , a gtT ) (cid:124) (cid:123)(cid:122) (cid:125)
Revised: d, ( q 1 , a gt 1 ) (cid:124) (cid:123)(cid:122) (cid:125) d, · · · , ( q T , a gtT ) (cid:124) (cid:123)(cid:122) (cid:125)

Original: d T } where c denotes an image caption.
Revised: d T } where c denotes an image caption.

Original: T is the number of rounds for each dialog.
Revised: T is the number of rounds for each dialog.

Original: At round t , a dialog agent is given a triplet ( v, d <t , q t ) as context, consisting of the image, the dialog history, and a visual question.
Revised: At round t , a dialog agent is given a triplet ( v, d <t , q t ) as context, consisting of the image, the dialog history, and a visual question.

Original: d <t denotes all dialog rounds before t -th round.
Revised: d <t denotes all dialog rounds before t -th round.

Original: The agent should predict a ground-truth answer a gtt .
Revised: The agent should predict a ground-truth answer a gtt .

Original: 2.2. Generative Self-Training (GST)
Revised: 2.2. Generative Self-Training (GST)

Original: Overview.
Revised: Overview.

Original: An overview of GST is shown in Figure 1.
Revised: An overview of GST is shown in Figure 1.

Original: We have a human-labeled VisDial dataset L = { ( v n , d n ) }
Revised: We have a human-labeled VisDial dataset L = { ( v n , d n ) }

Original: Nn = where v n is a given image and each dialog d n = {c n
Revised: Nn = where v n is a given image and each dialog d n = {

Original: [Equation]
Revised: [Equation]

Original:  } consists of an image caption c and T rounds of QA pairs.
Revised: n [Equation] [Equation] ) (cid:124) (cid:123)(cid:122) (cid:125) d n,T } consists of an image caption c and T rounds of QA pairs.

Original: In the following, we omit the superscript gt in the ground-truth answer for brevity.
Revised: In the following, we omit the superscript gt in the ground-truth answer for brevity.

Original: GST first trains a teacher P T and a questioner P Q with the labeled dataset L via supervised learning.
Revised: GST first trains a teacher P T and a questioner P Q with the labeled dataset L via supervised learning.

Original: It then retrieves unlabeled images U
Revised: It then retrieves unlabeled images U

Original: =
Revised: =

Original: { ˜ v m } M m =1 from the image dataset using a simple outlier detection model.
Revised: { ˜ v m } M m =1 from the image dataset using a simple outlier detection model.

Original: Next, the questioner and the teacher generate the visually-grounded dialog ˜ d for the unlabeled image ˜ v via multimodal conditional text generation, finally yielding a synthetic dialog dataset ˜ L = { (˜ v m , ˜ d m ) }
Revised: Next, the questioner and the teacher generate the visually-grounded dialog ˜ d for the unlabeled image ˜ v via multimodal conditional text generation, finally yielding a synthetic dialog dataset ˜ L = { (˜ v m , ˜ d m ) }

Original: Mm =1 .
Revised: Mm =1 .

Original: We call this dataset the machine VisDial data to distinguish it from the human-labeled
Revised: We call this dataset the machine VisDial data to distinguish it from the human-labeled

Original: VisDial dataset (Das et al., 2017) (short for the human VisDial data).
Revised: VisDial dataset (Das et al., 2017) (short for the human VisDial data).

Original: Finally, a student P S is trained on the human and the machine VisDial data by applying perplexity-based data selection (PPL) and multimodal consistency regularization (MCR) to the machine VisDial data.
Revised: Finally, a student P S is trained on the human and the machine VisDial data by applying perplexity-based data selection (PPL) and multimodal consistency regularization (MCR) to the machine VisDial data.

Original: Teacher & questioner training.
Revised: Teacher & questioner training.

Original: GST first trains the answer generator, the teacher model P T , on the human VisDial dataset.
Revised: GST first trains the answer generator, the teacher model P T , on the human VisDial dataset.

Original: Specifically, the teacher is optimized by minimizing the negative log-likelihood of the ground-truth answer, given the context c n,t ≜ ( v n , d n,<t , q n,t ) , consisting of the image, the dialog history, and the question.
Revised: Specifically, the teacher is optimized by minimizing the negative log-likelihood of the ground-truth answer, given the context c n,t ≜ ( v n , d n,<t , q n,t ) , consisting of the image, the dialog history, and the question.

Original: Formally,
Revised: Formally,

Original: [Equation]
Revised: [Equation]

Original: where N and T denote the number of data tuples in human
Revised: where N and T denote the number of data tuples in human

Original: VisDial data and dialog rounds, respectively.
Revised: VisDial data and dialog rounds, respectively.

Original: Similarly to the teacher, the questioner is trained to generate the question at round t for n -th dialog data, given the image and the dialog history until round t − 1 ( i.e., P Q ( q n,t | v n , d n,<t ) ).
Revised: Similarly to the teacher, the questioner is trained to generate the question at round t for n -th dialog data, given the image and the dialog history until round t − 1 ( i.e., P Q ( q n,t | v n , d n,<t ) ).

Original: Both the teacher and the questioner are based on encoder-decoder architecture, where an encoder aggregates the context, and a decoder generates the target sentence.
Revised: Both the teacher and the questioner are based on encoder-decoder architecture, where an encoder aggregates the context, and a decoder generates the target sentence.

Original: We implement the models by integrating a pretrained vision-and-language encoder, VisDial-BERT (Murahari et al., 2020), with the transformer decoder (Rothe et al., 2020).
Revised: We implement the models by integrating a pretrained vision-and-language encoder, VisDial-BERT (Murahari et al., 2020), with the transformer decoder (Rothe et al., 2020).

Original: Unlabeled image retrieval.
Revised: Unlabeled image retrieval.

Original: GST selects in-domain image data from the Conceptual 12M dataset (Changpinyo et al., 2021) with an out-of-distribution (OOD) detection model.
Revised: GST selects in-domain image data from the Conceptual 12M dataset (Changpinyo et al., 2021) with an out-of-distribution (OOD) detection model.

Original: Specifically, we extract the D -dimensional feature vector for each image in the human VisDial dataset by using the Vision Transformer (ViT)
Revised: Specifically, we extract the D -dimensional feature vector for each image in the human VisDial dataset by using the Vision Transformer (ViT)

Original: (Dosovitskiy et al., 2021), yielding a feature matrix for the entire images
Revised: (Dosovitskiy et al., 2021), yielding a feature matrix for the entire images

Original: X =
Revised: X =

Original: ( X 1 , · · · , X N ) ⊤ ∈ R N × D .
Revised: ( X 1 , · · · , X N ) ⊤ ∈ R N × D .

Original: Based on the feature matrix, we build the multivariate normal distribution whose dimension is D , i.e., X ∼ N D ( µ, Σ) .
Revised: Based on the feature matrix, we build the multivariate normal distribution whose dimension is D , i.e., X ∼ N D ( µ, Σ) .

Original: We regard this normal distribution as the empirical distribution of the human
Revised: We regard this normal distribution as the empirical distribution of the human

Original: VisDial images and perform the OOD detection by identifying the probability of each feature vector for the unlabeled image.
Revised: VisDial images and perform the OOD detection by identifying the probability of each feature vector for the unlabeled image.

Original: Consequently, the top- M unlabeled images are retrieved out of 12 million Web images.
Revised: Consequently, the top- M unlabeled images are retrieved out of 12 million Web images.

Original: Visually-grounded dialog generation.
Revised: Visually-grounded dialog generation.

Original: Given the retrieved images U =
Revised: Given the retrieved images U =

Original: { ˜ v m } Mm =1 , our goal is to generate the visuallygrounded dialogs { ˜ d m } Mm =1 where each dialog ˜ d consists of the image caption and T rounds of QA pairs.
Revised: { ˜ v m } Mm =1 , our goal is to generate the visuallygrounded dialogs { ˜ d m } Mm =1 where each dialog ˜ d consists of the image caption and T rounds of QA pairs.

Original: In an actual implementation, we use the image captions in the Conceptual 12M dataset (Changpinyo et al., 2021) and thus do not generate the captions.
Revised: In an actual implementation, we use the image captions in the Conceptual 12M dataset (Changpinyo et al., 2021) and thus do not generate the captions.

Original: The QA pairs are sequentially generated.
Revised: The QA pairs are sequentially generated.

Original: Specifically, the image ˜ v , the caption ˜ c , and the generated
Revised: Specifically, the image ˜ v , the caption ˜ c , and the generated

Original: QA pairs until round t − 1 are used as inputs when the questioner generates the question at round t .
Revised: QA pairs until round t − 1 are used as inputs when the questioner generates the question at round t .

Original: After then, the teacher produces the answer ˜ a t based on the image ˜ v , the dialog history ˜ d <t , and the question ˜ q t .
Revised: After then, the teacher produces the answer ˜ a t based on the image ˜ v , the dialog history ˜ d <t , and the question ˜ q t .

Original: Finally, GST produces the machine VisDial dataset ˜ L = { (˜ v m , ˜ d m ) } Mm =1 .
Revised: Finally, GST produces the machine VisDial dataset ˜ L = { (˜ v m , ˜ d m ) } Mm =1 .

Original: Student training with noisy data.
Revised: Student training with noisy data.

Original: As shown in Figure 1, the student P S is trained on the combination of the machine and the human VisDial data.
Revised: As shown in Figure 1, the student P S is trained on the combination of the machine and the human VisDial data.

Original: According to the studies (Xie et al., 2020b; He et al., 2020) in self-training, selectively utilizing the samples in the pseudo-labeled dataset is a common strategy.
Revised: According to the studies (Xie et al., 2020b; He et al., 2020) in self-training, selectively utilizing the samples in the pseudo-labeled dataset is a common strategy.

Original: To this end, we introduce a simple data selection method called perplexity-based data selection (PPL) to utilize the answers whose perplexity of the teacher is below a certain threshold.
Revised: To this end, we introduce a simple data selection method called perplexity-based data selection (PPL) to utilize the answers whose perplexity of the teacher is below a certain threshold.

Original: Perplexity is defined as the exponentiated average negative log-likelihood of a sequence; the lower, the better.
Revised: Perplexity is defined as the exponentiated average negative log-likelihood of a sequence; the lower, the better.

Original: Furthermore, inspired by the consistency regularization (Xie et al., 2020a; Sohn et al., 2020) widely utilized in recent SSL algorithms, we also propose the multimodal consistency regularization (MCR) which encourages the student to yield the predictions similar to the teacher’s predictions even when the student is provided with perturbed multimodal inputs.
Revised: Furthermore, inspired by the consistency regularization (Xie et al., 2020a; Sohn et al., 2020) widely utilized in recent SSL algorithms, we also propose the multimodal consistency regularization (MCR) which encourages the student to yield the predictions similar to the teacher’s predictions even when the student is provided with perturbed multimodal inputs.

Original: Finally, we design a loss function:
Revised: Finally, we design a loss function:

Original: [Equation]
Revised: [Equation]

Original: where M , 1 , and τ denote the number of data tuples in machine VisDial data, indicator function, and selection threshold, respectively.
Revised: where M , 1 , and τ denote the number of data tuples in machine VisDial data, indicator function, and selection threshold, respectively.

Original: ˜ c m,t ≜ (˜ v m , ˜ d m,<t , ˜ q m,t ) denotes the context for the machine VisDial data.
Revised: ˜ c m,t ≜ (˜ v m , ˜ d m,<t , ˜ q m,t ) denotes the context for the machine VisDial data.

Original: The loss function is the sum of the losses for the machine and the human VisDial data.
Revised: The loss function is the sum of the losses for the machine and the human VisDial data.

Original: PPL and MCR are applied to computing the loss of the machine VisDial data.
Revised: PPL and MCR are applied to computing the loss of the machine VisDial data.

Original: PPL is used in the indicator function above, selecting the synthetic answers whose perplexity of the teacher is below the threshold τ .
Revised: PPL is used in the indicator function above, selecting the synthetic answers whose perplexity of the teacher is below the threshold τ .

Original: It implies that the unselected answers are ignored during training.
Revised: It implies that the unselected answers are ignored during training.

Original: Next, M denotes the stochastic function for MCR that injects perturbations to the input space of the student.
Revised: Next, M denotes the stochastic function for MCR that injects perturbations to the input space of the student.

Original: Inspired by
Revised: Inspired by

Original: ViLBERT (Lu et al., 2019), we implement the stochastic function by randomly masking 15% of image regions and word tokens.
Revised: ViLBERT (Lu et al., 2019), we implement the stochastic function by randomly masking 15% of image regions and word tokens.

Original: Specifically, masked image regions have their image features zeroed out, and the masked word tokens are replaced with a special [MASK] token.
Revised: Specifically, masked image regions have their image features zeroed out, and the masked word tokens are replaced with a special [MASK] token.

Original: The intuition behind
Revised: The intuition behind

Original: MCR is minimizing the distance between the perturbed ( i.e., masked) predictions from the student and the unperturbed predictions ( i.e., ˜ a m,t ) from the teacher.
Revised: MCR is minimizing the distance between the perturbed ( i.e., masked) predictions from the student and the unperturbed predictions ( i.e., ˜ a m,t ) from the teacher.

Original: The student can be robust to the input noise with MCR.
Revised: The student can be robust to the input noise with MCR.

Original: We believe MCR
Revised: We believe MCR

Original: [Table]
Revised: [Table]

Original: improves [Table]
Revised: improves the generalization abilities of the student, and

Original: PPL encourages the student to maintain a low entropy ( i.e., confident) in noisy data training.
Revised: PPL encourages the student to maintain a low entropy ( i.e., confident) in noisy data training.

Original: 3. Experiments 3.1. Evaluation Protocol
Revised: 3. Experiments 3.1. Evaluation Protocol

Original: We follow the standard evaluation protocol established in the work (Das et al., 2017).
Revised: We follow the standard evaluation protocol established in the work (Das et al., 2017).

Original: Specifically, the visual dialog models have been evaluated by the retrieval-based evaluation metrics: mean reciprocal rank (MRR), recall@k (R@k), mean rank (Mean), and normalized discounted cumulative gain (NDCG).
Revised: Specifically, the visual dialog models have been evaluated by the retrieval-based evaluation metrics: mean reciprocal rank (MRR), recall@k (R@k), mean rank (Mean), and normalized discounted cumulative gain (NDCG).

Original: Specifically, all dialogs in VisDial contain a list of 100 answer candidates for each visual question and there is one ground-truth answer in the answer candidates.
Revised: Specifically, all dialogs in VisDial contain a list of 100 answer candidates for each visual question and there is one ground-truth answer in the answer candidates.

Original: The model sorts the answer candidates by the loglikelihood scores and then is evaluated by the four different metrics.
Revised: The model sorts the answer candidates by the loglikelihood scores and then is evaluated by the four different metrics.

Original: MRR, R@k, and Mean consider the rank of the single ground-truth answer, while NDCG 1 considers all relevant answers from the 100-answers list by using the densely annotated relevance scores for all answer candidates.
Revised: MRR, R@k, and Mean consider the rank of the single ground-truth answer, while NDCG 1 considers all relevant answers from the 100-answers list by using the densely annotated relevance scores for all answer candidates.

Original: The community regards MRR and NDCG as primary metrics.
Revised: The community regards MRR and NDCG as primary metrics.

Original: 3.2. Ablation Study
Revised: 3.2. Ablation Study

Original: We perform an ablation study to illustrate the effect of each component in GST.
Revised: We perform an ablation study to illustrate the effect of each component in GST.

Original: The machine VisDial data whose size is five times larger than the human VisDial data is utilized to train the student.
Revised: The machine VisDial data whose size is five times larger than the human VisDial data is utilized to train the student.

Original: We report the performance of ablative models: Student w/o PT, Student w/o PPL, Student w/o MCR, Student w/o MCR and PPL.
Revised: We report the performance of ablative models: Student w/o PT, Student w/o PPL, Student w/o MCR, Student w/o MCR and PPL.

Original: Student w/o PT is the model that does not leverage the pre-trained weights of the VisDial-BERT model.
Revised: Student w/o PT is the model that does not leverage the pre-trained weights of the VisDial-BERT model.

Original: Student w/o PPL denotes the model that utilizes all generated QA pairs without applying the perplexity-based data selection.
Revised: Student w/o PPL denotes the model that utilizes all generated QA pairs without applying the perplexity-based data selection.

Original: Student w/o MCR does not inject noises to the inputs of the student model.
Revised: Student w/o MCR does not inject noises to the inputs of the student model.

Original: [Table]
Revised: [Table]

Original: [Table]

Original: In Table 1, we observe all components play a significant role in boosting the performance.
Revised: In Table 1, we observe all components play a significant role in boosting the performance.

Original: Especially, we develop the teacher, the questioner, and the student models on top of VisDial-BERT (Murahari et al., 2020) which leverages vision-and-language pre-training (Lu et al., 2019).
Revised: Especially, we develop the teacher, the questioner, and the student models on top of VisDial-BERT (Murahari et al., 2020) which leverages vision-and-language pre-training (Lu et al., 2019).

Original: Thus, the teacher can be understood as a typical model that follows the pretrain-then-transfer learning strategy mentioned in the introduction, whereas the student leverages both pre-training and GST.
Revised: Thus, the teacher can be understood as a typical model that follows the pretrain-then-transfer learning strategy mentioned in the introduction, whereas the student leverages both pre-training and GST.

Original: By comparing the student, the student without pre-training, and the teacher in Table 1, we identify that self-supervised pre-training and GST are complementary.
Revised: By comparing the student, the student without pre-training, and the teacher in Table 1, we identify that self-supervised pre-training and GST are complementary.

Original: 3.3. Comparison with State-of-the-Art
Revised: 3.3. Comparison with State-of-the-Art

Original: We compare GST with the state-of-the-art approaches on the validation set of the VisDial v1.0 dataset, consisting of UTC (Chen et al., 2022), MITVG (Chen et al., 2021), LTMI (Nguyen et al., 2020), ReDAN (Gan et al., 2019), Primary (Guo et al., 2019), CoAtt (Wu et al., 2018),
Revised: We compare GST with the state-of-the-art approaches on the validation set of the VisDial v1.0 dataset, consisting of UTC (Chen et al., 2022), MITVG (Chen et al., 2021), LTMI (Nguyen et al., 2020), ReDAN (Gan et al., 2019), Primary (Guo et al., 2019), CoAtt (Wu et al., 2018),

Original: HCIAE (Lu et al., 2017), and MN (Das et al., 2017).
Revised: HCIAE (Lu et al., 2017), and MN (Das et al., 2017).

Original: In
Revised: In

Original: Table 2, GST significantly outperforms the state-of-the-art methods, including UTC (Chen et al., 2022), on all evaluation metrics except for Mean.
Revised: Table 2, GST significantly outperforms the state-of-the-art methods, including UTC (Chen et al., 2022), on all evaluation metrics except for Mean.

Original: GST improves NDCG 1.20% (63.86 → 65.06) and MRR 0.62% (52.22 → 52.84).
Revised: GST improves NDCG 1.20% (63.86 → 65.06) and MRR 0.62% (52.22 → 52.84).

Revised: 3.4. Is GST helpful when human-labeled data is scarce?

Revised: We investigate this question to identify the effect of GST in the low-data regime.

Revised: We assume that only a small subset of the human VisDial data (1%, 5%, 10%, 20%, and 30%) is available.

Revised: Therefore, the size of the human-labeled data is 0.01 N , 0.05 N , 0.1 N , 0.2 N , and 0.3 N , respectively.

Revised: We first train the teacher and the questioner on such scarce data, and then these two agents generate a new machine

Original: 4. Conclusion
Revised: VisDial data for unlabeled images in the Conceptual 12M dataset (Changpinyo et al., 2021) with size N . The student is then trained on the newly generated machine VisDial data [Table] [Table]

Revised: (+0.44 / +0.75) 65.06 / 52.

Revised: (+0.56 / +0.78) and the small amount of the human VisDial data.

Revised: Note that

Revised: PPL and MCR are still applied in this experiment.

Revised: In Table 3,

Revised: GST yields huge improvements on both metrics, especially

Revised: NDCG, boosting up to 9.35 absolute points compared with the teacher.

Revised: We observe that the smaller the amount of human-labeled data, the larger the performance gap between the teacher and the student on NDCG.

Revised: It implies that GST is helpful, especially when human-labeled data is scarce.

Revised: We think the results in the low-data regime are particularly remarkable in other dialog-based tasks (Thomason et al., 2020; Alamri et al., 2019; Rashkin et al., 2019;

Revised: Li et al., 2017b) since many tasks provide the human dialog data less than 0.3 N ( i.e., 360k QA pairs).

Revised: 3.5. Scaling up the size of the machine VisDial data.

Revised: We also conduct experiments on the high-data regime where the entire human VisDial data and the machine VisDial data with different number data chunks (1, 5, 10, 20, and 30 out of 30 data chunks) are used to train the student.

Revised: In Table 4, the student shows an increase in performance compared with the teacher except when using one data chunk.

Revised: Notably, we observe significant gains in the student when increasing from one to five data chunks ( i.e., 1x → 5x).

Revised: NDCG seems to be saturated when using more than five data chunks, but

Revised: MRR increases monotonically.

Revised: The student leveraging the entire machine VisDial data (x30) shows the best performance on average.

Revised: 4. Conclusion

Original: We propose a semi-supervised learning approach for VisDial, called GST, that explicitly generates a synthetic visual dialog dataset for unlabeled images via multimodal conditional text generation.
Revised: We propose a semi-supervised learning approach for VisDial, called GST, that explicitly generates a synthetic visual dialog dataset for unlabeled images via multimodal conditional text generation.

Original: We also present the perplexity-based data selection and the multimodal consistency regularization to effectively leverage the generated dialogs.
Revised: We also present the perplexity-based data selection and the multimodal consistency regularization to effectively leverage the generated dialogs.

Original: The experiments quantitatively support the effectiveness of our proposed method.
Revised: The experiments quantitatively support the effectiveness of our proposed method.

Original: Above all, we validate that GST can further improve self-supervised pre-training approach for VisDial. 
Revised: Above all, we validate that GST can further improve self-supervised pre-training approach for VisDial. Acknowledgements.

Revised: The authors thank Minjoon Jung, YuJung Heo, Woosuk Choi, Seongho Choi, and Seongjun Jung for helpful comments.

Revised: This work was partly supported by the Institute of Information & Communications Technology

Revised: Planning & Evaluation (2015-0-00310-SW.StarLab/20%, 2019-0-01371-BabyMind/20%, 2021-0-02068-AIHub/10%, 2021-0-01343-GSAI/10%, 2022-0-00951-LBA/20%, 20220-00166-PICA/20%) grant funded by the Korean government.

Original: A. Related Work
Revised: A. Related Work

Original: Visual dialog.
Revised: Visual dialog.

Original: Visual Dialog (VisDial) (Das et al., 2017) has been proposed as an extended version of Visual Question
Revised: Visual Dialog (VisDial) (Das et al., 2017) has been proposed as an extended version of Visual Question

Original: Answering (VQA) (Antol et al., 2015; Anderson et al., 2018;
Revised: Answering (VQA) (Antol et al., 2015; Anderson et al., 2018;

Original: Kim et al., 2018), where a dialog agent should answer a series of interdependent questions using an image and the dialog history.
Revised: Kim et al., 2018), where a dialog agent should answer a series of interdependent questions using an image and the dialog history.

Original: Prior work has developed a variety attention mechanisms (Lu et al., 2017; Seo et al., 2017;
Revised: Prior work has developed a variety attention mechanisms (Lu et al., 2017; Seo et al., 2017;

Original: Wu et al., 2018; Kottur et al., 2018;
Revised: Wu et al., 2018; Kottur et al., 2018;

Original: Niu et al., 2019; Schwartz et al., 2019; Guo et al., 2019; Gan et al., 2019; Kang et al., 2019; Nguyen et al., 2020) considering the interactions among the image, dialog history, and question.
Revised: Niu et al., 2019; Schwartz et al., 2019; Guo et al., 2019; Gan et al., 2019; Kang et al., 2019; Nguyen et al., 2020) considering the interactions among the image, dialog history, and question.

Original: Some studies (Zheng et al., 2019; Kang et al., 2021) have attempted to discover the semantic structures of the dialog in the context of graph neural networks (Scarselli et al., 2008) using the soft attention mechanisms (Bahdanau et al., 2014).
Revised: Some studies (Zheng et al., 2019; Kang et al., 2021) have attempted to discover the semantic structures of the dialog in the context of graph neural networks (Scarselli et al., 2008) using the soft attention mechanisms (Bahdanau et al., 2014).

Original: From the learning algorithm perspective, all of them have relied on supervised learning on VisDial data.
Revised: From the learning algorithm perspective, all of them have relied on supervised learning on VisDial data.

Original: More recently, a line of research (Murahari et al., 2020; Wang et al., 2020b; Chen et al., 2022) has employed self-supervised pre-training to leverage the knowledge of related vision-and-language datasets (Sharma et al., 2018; Antol et al., 2015; Zhu et al., 2015).
Revised: More recently, a line of research (Murahari et al., 2020; Wang et al., 2020b; Chen et al., 2022) has employed self-supervised pre-training to leverage the knowledge of related vision-and-language datasets (Sharma et al., 2018; Antol et al., 2015; Zhu et al., 2015).

Original: However, our approach is based on semi-supervised learning and produces the task-specific data ( i.e., visual dialogs) for unlabeled images to train the dialog agent.
Revised: However, our approach is based on semi-supervised learning and produces the task-specific data ( i.e., visual dialogs) for unlabeled images to train the dialog agent.

Original: Sequence generation in vision-and-language tasks.
Revised: Sequence generation in vision-and-language tasks.

Original: Many studies have generated natural language for the visual inputs such as image captioning (Xu et al., 2015; Anderson et al., 2018), video captioning (Iashin & Rahtu, 2020; Pan et al., 2017), visual question generation (VQG) (Kai et al., 2021; Krishna et al., 2019;
Revised: Many studies have generated natural language for the visual inputs such as image captioning (Xu et al., 2015; Anderson et al., 2018), video captioning (Iashin & Rahtu, 2020; Pan et al., 2017), visual question generation (VQG) (Kai et al., 2021; Krishna et al., 2019;

Original: Fan et al., 2018b; Liu et al., 2018; Patro et al., 2018; Jain et al., 2017), visual dialog (VisDial) (Das et al., 2017; Gan et al., 2019), and video dialog (Alamri et al., 2019;
Revised: Fan et al., 2018b; Liu et al., 2018; Patro et al., 2018; Jain et al., 2017), visual dialog (VisDial) (Das et al., 2017; Gan et al., 2019), and video dialog (Alamri et al., 2019;

Original: Le et al., 2019).
Revised: Le et al., 2019).

Original: Furthermore, recent studies (Yang et al., 2021; Li et al., 2022) have produced text data for vision-and-language pre-training.
Revised: Furthermore, recent studies (Yang et al., 2021; Li et al., 2022) have produced text data for vision-and-language pre-training.

Original: GST is similar to these studies in that the model generates the text data, but our focus is on studying the effect of semi-supervised learning (SSL) on top of such pre-training approaches.
Revised: GST is similar to these studies in that the model generates the text data, but our focus is on studying the effect of semi-supervised learning (SSL) on top of such pre-training approaches.

Original: To the best of our knowledge, GST is the first approach to show the efficacy of SSL throughout a wide range of visual QA tasks.
Revised: To the best of our knowledge, GST is the first approach to show the efficacy of SSL throughout a wide range of visual QA tasks.

Original: Neural dialog generation.
Revised: Neural dialog generation.

Original: In NLP literature, extensive studies have been conducted regarding neural dialogue generation for both open-domain dialogue (Zhang et al., 2020; Shang et al., 2015;
Revised: In NLP literature, extensive studies have been conducted regarding neural dialogue generation for both open-domain dialogue (Zhang et al., 2020; Shang et al., 2015;

Original: Li et al., 2016; Serban et al., 2017; Saleh et al., 2020; Li et al., 2017) and task-oriented dialogue (Wang et al., 2020a; Huang et al., 2020).
Revised: Li et al., 2016; Serban et al., 2017; Saleh et al., 2020; Li et al., 2017a) and task-oriented dialogue (Wang et al., 2020a; Huang et al., 2020).

Original: Our approach is similar to neural dialogue generation in that the model should generate a corresponding response based on the dialog history and the current utterance.
Revised: Our approach is similar to neural dialogue generation in that the model should generate a corresponding response based on the dialog history and the current utterance.

Original: However, we aim to produce visually-grounded dialogs, and thus the image-groundedness of the question and the semantic correctness of the answer are important.
Revised: However, we aim to produce visually-grounded dialogs, and thus the image-groundedness of the question and the semantic correctness of the answer are important.

Original: On the other hand, neural dialogue generation considers many different aspects: specificity, response-relatedness (See et al., 2019), interestingness, fluency (Mehri & Eskenazi, 2020), and diversity (Li et al., 2016).
Revised: On the other hand, neural dialogue generation considers many different aspects: specificity, response-relatedness (See et al., 2019), interestingness, fluency (Mehri & Eskenazi, 2020), and diversity (Li et al., 2016).

Original: B. Implementation Details
Revised: B. Implementation Details

Original: As mentioned in Section 2.2, we integrate the vision-and-language encoder for a discriminative task ( i.e., VisDial-BERT (Murahari et al., 2020)) with the transformer decoder for sequence generation ( i.e., BERT BASE (Rothe et al., 2020)) to train the teacher, the questioner, and the student.
Revised: As mentioned in Section 2.2, we integrate the vision-and-language encoder for a discriminative task ( i.e., VisDial-BERT (Murahari et al., 2020)) with the transformer decoder for sequence generation ( i.e., BERT BASE (Rothe et al., 2020)) to train the teacher, the questioner, and the student.

Original: The encoder architecture is based on the vision-and-language pre-training model,
Revised: The encoder architecture is based on the vision-and-language pre-training model,

Original: ViLBERT (Lu et al., 2019).
Revised: ViLBERT (Lu et al., 2019).

Original: The decoder has 12 layers of transformer blocks, with each block having 12 attention heads and a hidden size of 768.
Revised: The decoder has 12 layers of transformer blocks, with each block having 12 attention heads and a hidden size of 768.

Original: The maximum sequence length of the encoder and the decoder is 256 and 25, respectively.
Revised: The maximum sequence length of the encoder and the decoder is 256 and 25, respectively.

Original: We extract the feature vectors of the input images by using the Faster R-CNN (Ren et al., 2015; Anderson et al., 2018) pre-trained on
Revised: We extract the feature vectors of the input images by using the Faster R-CNN (Ren et al., 2015; Anderson et al., 2018) pre-trained on

Original: Visual Genome (Krishna et al., 2017).
Revised: Visual Genome (Krishna et al., 2017).

Original: The number of bounding boxes for each image is fixed to 36.
Revised: The number of bounding boxes for each image is fixed to 36.

Original: We set the threshold for
Revised: We set the threshold for

Original: PPL τ to 50.
Revised: PPL τ to 50.

Original: We train on one A100 GPU with a batch size of 72 for 70 epochs.
Revised: We train on one A100 GPU with a batch size of 72 for 70 epochs.

Original: Training time takes about 3 days.
Revised: Training time takes about 3 days.

Original: We use the Adam optimizer (Kingma & Ba, 2014) with an initial learning rate 1e-5.
Revised: We use the Adam optimizer (Kingma & Ba, 2014) with an initial learning rate 1e-5.

Original: The learning rate is warmed up to 2e-5 until 10k iterations and linearly decays to 1e-5.
Revised: The learning rate is warmed up to 2e-5 until 10k iterations and linearly decays to 1e-5.

Original: In visually-grounded dialog generation, the questioner and the teacher decode the sequences using the top- k sampling (Fan et al., 2018a; Holtzman et al., 2018;
Revised: In visually-grounded dialog generation, the questioner and the teacher decode the sequences using the top- k sampling (Fan et al., 2018a; Holtzman et al., 2018;

Original: Radford et al., 2019) with k = 7 and the temperature of 0.7.
Revised: Radford et al., 2019) with k = 7 and the temperature of 0.7.

Original: We use the top- k sampling since its computation is cheap yielding accurate and diverse sequences.
Revised: We use the top- k sampling since its computation is cheap yielding accurate and diverse sequences.

Original: Furthermore, we apply the 4-gram penalty (Paulus et al., 2018; Klein et al., 2017) when generating visual questions to ensure that no 4-gram appears twice in the questions for each dialog.
Revised: Furthermore, we apply the 4-gram penalty (Paulus et al., 2018; Klein et al., 2017) when generating visual questions to ensure that no 4-gram appears twice in the questions for each dialog.

Original: C. Qualitative Results and Analysis
Revised: C. Qualitative Results and Analysis

Original: For qualitative analysis, we visualize the generated conversations from our models and the ones from human.
Revised: For qualitative analysis, we visualize the generated conversations from our models and the ones from human.

Original: We excerpt the human conversation from the VisDial v1.0 validation dataset, and the questioner and the student generate the machine conversation using the image and the caption in the validation data.
Revised: We excerpt the human conversation from the VisDial v1.0 validation dataset, and the questioner and the student generate the machine conversation using the image and the caption in the validation data.

Original: As in Figure 2, diverse visual questions and correct answers are generated in the machine conversation.
Revised: As in Figure 2, diverse visual questions and correct answers are generated in the machine conversation.

Original: For instance, from D3 to D6 in the first example, the question generation model asks about “a cell phone,” which first appears in the caption, whereas the human questioner deals with different topics.
Revised: For instance, from D3 to D6 in the first example, the question generation model asks about “a cell phone,” which first appears in the caption, whereas the human questioner deals with different topics.

Original: The student responds appropriately to the given questions.
Revised: The student responds appropriately to the given questions.

Original: However, we also identify that the student sometimes fails to generate correct answers ( i.e., the red-colored text in D9 of the last example), which shows the importance of more precise vision-and-language grounding.
Revised: However, we also identify that the student sometimes fails to generate correct answers ( i.e., the red-colored text in D9 of the last example), which shows the importance of more precise vision-and-language grounding.