Original: L EARNING D ISCRIMINATORS AS E NERGY N ETWORKS IN A DVERSARIAL L EARNING
Revised: L EARNING D ISCRIMINATORS AS E NERGY N ETWORKS IN A DVERSARIAL L EARNING

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: We propose a novelframework for structured prediction via adversarial learning.
Revised: We propose a novel adversarial learning framework in this work.

Original: Existing adversarial learning methods involve two separate networks, i e., the structured prediction models and the discriminative models, in the training.
Revised: Existing adversarial learning methods involve two separate networks, i e., the structured prediction models and the discriminative models, in the training.

Original: The information captured by discriminative models complements that in the structured prediction models, but few existing researches have studied on utilizing such information to improve structured prediction models at the inference stage.
Revised: The information captured by discriminative models complements that in the structured prediction models, but few existing researches have studied on utilizing such information to improve structured prediction models at the inference stage.

Original: In this work, we propose to reﬁne the predictions of structured prediction models by effectively integrating discriminative models into the prediction.
Revised: In this work, we propose to reﬁne the predictions of structured prediction models by effectively integrating discriminative models into the prediction.

Original: Discriminative models are treated as energy-based models.
Revised: Discriminative models are treated as energy-based models.

Original: Similar to the adversarial learning, discriminative models are trained to estimate scores which measure the quality of predicted outputs, while structured prediction models are trained to predict contrastive outputs with maximal energy scores.
Revised: Similar to the adversarial learning, discriminative models are trained to estimate scores which measure the quality of predicted outputs, while structured prediction models are trained to predict contrastive outputs with maximal energy scores.

Original: In this way, the gradient vanishing problem is ameliorated and thus we are able to perform inference by following the ascent gradient directions of discriminative models to reﬁne structured prediction models.
Revised: In this way, the gradient vanishing problem is ameliorated and thus we are able to perform inference by following the ascent gradient directions of discriminative models to reﬁne structured prediction models.

Original: The proposed method is able to handle a range of tasks, e g. , multi-label classiﬁcation and image segmentation.
Revised: The proposed method is able to handle a range of tasks, e g. , multi-label classiﬁcation and image segmentation.

Original: Empirical results on these two tasks validate the effectiveness of our learning method
Revised: Empirical results on these two tasks validate the effectiveness of our learning method

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: This work focuses on applying adversarial learning (Goodfellow et al., 2014a) to solve structured prediction tasks, e.g. , multi-label classiﬁcation and image segmentation.
Revised: This work focuses on applying adversarial learning (Goodfellow et al., 2014a) to solve structured prediction tasks, e.g. , multi-label classiﬁcation and image segmentation.

Original: Adversarial learning can be formalized as a minimax two-player game between structured prediction models and discriminative models.
Revised: Adversarial learning can be formalized as a minimax two-player game between structured prediction models and discriminative models.

Original: Discriminative models are learned to distinguish between the outputs predicted by the structured prediction models and the training data, while structured prediction models are learned to predict outputs to fool discriminative models.
Revised: Discriminative models are learned to distinguish between the outputs predicted by the structured prediction models and the training data, while structured prediction models are learned to predict outputs to fool discriminative models.

Original: Though structured prediction models are trained by the gradients of discriminative models, existing methods rarely use discriminative models to improve structured prediction models at the inference stage.
Revised: Though structured prediction models are trained by the gradients of discriminative models, existing methods rarely use discriminative models to improve structured prediction models at the inference stage.

Original: A straightforward way of utilizing discriminative models for inference is to follow the ascent gradient directions of discriminative models to reﬁne the predicted outputs.
Revised: A straightforward way of utilizing discriminative models for inference is to follow the ascent gradient directions of discriminative models to reﬁne the predicted outputs.

Original: However, due to the wellknown gradient vanishing problems, the gradients of discriminative models are almost zero for all the predicted outputs.
Revised: However, due to the wellknown gradient vanishing problems, the gradients of discriminative models are almost zero for all the predicted outputs.

Original: It is difﬁcult to resolve the gradient vanishing problems, because they are caused by the training instability of the existing adversarial learning framework.
Revised: It is difﬁcult to resolve the gradient vanishing problems, because they are caused by the training instability of the existing adversarial learning framework.

Original: Consequently, most existing methods do not use the information from discriminative models to reﬁne structured prediction models.
Revised: Consequently, most existing methods do not use the information from discriminative models to reﬁne structured prediction models.

Original: Most existing adversarial learning methods take discriminative models as classiﬁers.
Revised: Most existing adversarial learning methods take discriminative models as classiﬁers.

Original: If discriminative models well separate real and predicted samples, they tend to assign the same scores to all predicted samples.
Revised: If discriminative models well separate real and predicted samples, they tend to assign the same scores to all predicted samples.

Original: Differently, energy-based models (LeCun et al., 2006; Gygli et al., 2017) usually predict different energy scores for different samples.
Revised: Differently, energy-based models (LeCun et al., 2006; Gygli et al., 2017) usually predict different energy scores for different samples.

Original: Therefore, we propose to train discriminative models as energy-based models to ameliorate the gradient vanishing problems.
Revised: Therefore, we propose to train discriminative models as energy-based models to ameliorate the gradient vanishing problems.

Original: In our framework, discriminative models are learned to assign scores to evaluate the quality of predicted outputs.
Revised: In our framework, discriminative models are learned to assign scores to evaluate the quality of predicted outputs.

Original: Structured prediction models are learned to predict outputs that are judged to have maximum scores by discriminative models.
Revised: Structured prediction models are learned to predict outputs that are judged to have maximum scores by discriminative models.

Original: In this way, discriminative models are trained to approximate continues value functions which evaluate the quality of predicted output.
Revised: In this way, discriminative models are trained to approximate continues value functions which evaluate the quality of predicted output.

Original: The gradients of discriminative models are not zero for predicted outputs.
Revised: The gradients of discriminative models are not zero for predicted outputs.

Original: Thus, we can reﬁne structured prediction models by following the ascent gradient directions of discriminative models at the inference stage.
Revised: Thus, we can reﬁne structured prediction models by following the ascent gradient directions of discriminative models at the inference stage.

Original: In this paper, we refer our method as learning discriminative models to reﬁne structured prediction models (LDRSP).
Revised: In this paper, we refer our method as learning discriminative models to reﬁne structured prediction models (LDRSP).

Original: The proposed method learns discriminative models utilizing the data generated by the structured prediction models.
Revised: The proposed method learns discriminative models utilizing the data generated by the structured prediction models.

Original: Gygli et al.
Revised: Gygli et al.

Original: (2017) found that the key to learning energy-based models is generating proper training data.
Revised: (2017) found that the key to learning deep value networks is generating proper training data.

Original: We propose to augment the training set of discriminative models by following the data generation methods proposed in (Gygli et al., 2017).
Revised: We propose to augment the training set of discriminative models by following the data generation methods proposed in (Gygli et al., 2017).

Original: At the training stage, we simultaneously run the inference algorithm to generate extra training samples utilizing models in previous iterations.
Revised: At the training stage, we simultaneously run the inference algorithm to generate extra training samples utilizing models in previous iterations.

Original: These samples are useful since they are generated along the gradient-based inference trajectory utilized at the inference stage.
Revised: These samples are useful since they are generated along the gradient-based inference trajectory utilized at the inference stage.

Original: We also augment the training set with adversarial samples (Goodfellow et al., 2014b).
Revised: We also augment the training set with adversarial samples (Goodfellow et al., 2014b).

Original: These samples are used as negative samples to train the discriminative models.
Revised: These samples are used as negative samples to train the discriminative models.

Original: To validate our method, experiments are conducted on multi-label classiﬁcation, binary image segmentation, and 3-class face segmentation tasks, and experimental results indicate that our method can learn discriminative models to effectively reﬁne structured prediction models.
Revised: To validate our method, experiments are conducted on multi-label classiﬁcation, binary image segmentation, and 3-class face segmentation tasks, and experimental results indicate that our method can learn discriminative models to effectively reﬁne structured prediction models.

Original: This work has two contributions: (1) We propose a novel adversarial learning framework for structured prediction, in which the information captured by discriminative models can be used to improve structured prediction models at the inference stage.
Revised: This work has two contributions: (1) We propose a novel adversarial learning framework for structured prediction, in which the information captured by discriminative models can be used to improve structured prediction models at the inference stage.

Original: (2) We propose to learn discriminative models to approximate continues value functions that evaluate the quality of the predicted outputs, and thus ameliorate the gradient vanishing problems.
Revised: (2) We propose to learn discriminative models to approximate continues value functions that evaluate the quality of the predicted outputs, and thus ameliorate the gradient vanishing problems.

Original:  R ELATED WORK
Revised:  R ELATED WORK

Original: Recently, adversarial learning has been well studied on producing high-resolution and high-quality images (Denton et al., 2015; Karras et al., 2017), improving the training stability and getting rid of problems like model collapse (Arjovsky et al., 2017; Gulrajani et al., 2017; Mao et al., 2017; Qi, 2017), image to image translation (Isola et al., 2017; Zhu et al., 2017; Huang et al., 2018), semantic image segmentation (Luc et al., 2016; Hwang et al., 2018) and neural dialogue generation (Li et al., 2017).
Revised: Recently, adversarial learning has been well studied on producing high-resolution and high-quality images (Denton et al., 2015; Karras et al., 2017), improving the training stability and getting rid of problems like model collapse (Arjovsky et al., 2017; Gulrajani et al., 2017; Mao et al., 2017; Qi, 2017), image to image translation (Isola et al., 2017; Zhu et al., 2017; Huang et al., 2018), semantic image segmentation (Luc et al., 2016; Hwang et al., 2018) and neural dialogue generation (Li et al., 2017).

Original: These methods learn discriminative models and generative models in an adversarial way, but discriminative models are abandoned once models are trained.
Revised: These methods learn discriminative models and generative models in an adversarial way, but discriminative models are abandoned once models are trained.

Original: Since discriminative models are learned to capture the discriminative distributions between the generated samples and the training data, we argue that they are able to be utilized to make the generated samples more real.
Revised: Since discriminative models are learned to capture the discriminative distributions between the generated samples and the training data, we argue that they are able to be utilized to make the generated samples more real.

Original: As we know, we are the ﬁrst to learn discriminative models that can be directly utilized to reﬁne structured prediction models.
Revised: As we know, we are the ﬁrst to learn discriminative models that can be directly utilized to reﬁne structured prediction models.

Original: There are some researches that utilize iterative inference methods.
Revised: There are some researches that utilize iterative inference methods.

Original: Since Denoising Autoencoders (DAEs) can estimate the gradients of energy distribution functions, Nguyen et al.
Revised: Since Denoising Autoencoders (DAEs) can estimate the gradients of energy distribution functions, Nguyen et al.

Original: (2016) propose to learn DAEs to capture probability distributions of realistic images.
Revised: (2016) propose to learn DAEs to capture probability distributions of realistic images.

Original: In the inference process, DAEs can be utilized to iteratively reﬁne generated images.
Revised: In the inference process, DAEs can be utilized to iteratively reﬁne generated images.

Original: Romero et al.
Revised: Romero et al.

Original: (2017) expand similar ideas to the image segmentation.
Revised: (2017) expand similar ideas to the image segmentation.

Original: Different from these methods, in which feed-forward networks are utilized to iteratively reﬁne the generated samples, we adopt a gradient-based inference to ﬁnd samples with the zero task loss.
Revised: Different from these methods, in which feed-forward networks are utilized to iteratively reﬁne the generated samples, we adopt a gradient-based inference to ﬁnd samples with the zero task loss.

Original: Learning discriminative models as energy-based models has been recently studied (Zhao et al., 2017;
Revised: Learning discriminative models as energy-based models has been recently studied (Zhao et al., 2017;

Original: Dai et al., 2017).
Revised: Dai et al., 2017).

Original: These works only constrain that discriminative models predict lower energy scores for the training data than the generated samples.
Revised: These works only constrain that discriminative models predict lower energy scores for the training data than the generated samples.

Original: However, our method learns discriminative models to approximate value functions that evaluate the quality of the generated samples.
Revised: However, our method learns discriminative models to approximate value functions that evaluate the quality of the generated samples.

Original: Moreover, we propose to generate extra training samples to learn discriminative models, so that discriminative models can better capture the probability distributions of the sample space.
Revised: Moreover, we propose to generate extra training samples to learn discriminative models, so that discriminative models can better capture the probability distributions of the sample space.

Original: There is a rising interest in energy-based structured prediction (Zheng et al., 2015; Chen et al., 2015;
Revised: There is a rising interest in energy-based structured prediction (Zheng et al., 2015; Chen et al., 2015;

Original: Song et al., 2016).
Revised: Song et al., 2016).

Original: Amos et al.
Revised: Amos et al.

Original: (2016) proposed to add constraints to the neural network parameters such that the output of the neural network is a convex function of (some of) the inputs.
Revised: (2016) proposed to add constraints to the neural network parameters such that the output of the neural network is a convex function of (some of) the inputs.

Original: Belanger & McCallum (2016) introduced Structured Prediction Energy Network (SPEN).
Revised: Belanger & McCallum (2016) introduced Structured Prediction Energy Network (SPEN).

Original: SPEN relies on a max-margin surrogate objective to ensure that the neural network predicts the lowest energy value for the ground-truth label.
Revised: SPEN relies on a max-margin surrogate objective to ensure that the neural network predicts the lowest energy value for the ground-truth label.

Original: Belanger et al.
Revised: Belanger et al.

Original: (2017) improved SPEN by proposing an end-to-end version of SPEN, which directly back-propagates through a computation graph that unrolls gradient-based energy minimization.
Revised: (2017) improved SPEN by proposing an end-to-end version of SPEN, which directly back-propagates through a computation graph that unrolls gradient-based energy minimization.

Original: Inspired by the reinforcement learning, Gygli et al.
Revised: Inspired by the reinforcement learning, Gygli et al.

Original: (2017) proposed a Deep
Revised: (2017) proposed a Deep

Original: Value Network (DVN) that directly learns to evaluate the quality of different output conﬁgures.
Revised: Value Network (DVN) that directly learns to evaluate the quality of different output conﬁgures.

Original: Compared with these methods, our method adversarially learns energy-based models and structured prediction models rather than learns energy-based models alone.
Revised: Compared with these methods, our method adversarially learns energy-based models and structured prediction models rather than learns energy-based models alone.

Original:  A PPROACH
Revised:  A PPROACH

Original: We propose a novel adversarial learning framework for structured prediction, in which discriminative models D ( x , y ; θ d ) can be used to reﬁne structured prediction models G ( x ; θ g ) at the inference stage.
Revised: We propose a novel adversarial learning framework for structured prediction, in which discriminative models D ( x , y ; θ d ) can be used to reﬁne structured prediction models G ( x ; θ g ) at the inference stage.

Original: In our method, discriminators are treated as energy-based models, which take both input objects x and possible outputs y as inputs and predict scores in the range of [0 , 1] .
Revised: In our method, discriminators are treated as energy-based models, which take both input objects x and possible outputs y as inputs and predict scores in the range of [0 , 1] .

Original: We assume that at the training stage, one can get access to an oracle value function v ∗ ( y , y ∗ ) , which evaluates the quality of y corresponding to x .
Revised: We assume that at the training stage, one can get access to an oracle value function v ∗ ( y , y ∗ ) , which evaluates the quality of y corresponding to x .

Original: Here y ∗ is the ground-truth label.
Revised: Here y ∗ is the ground-truth label.

Original: The discriminators are learned to mimic the behavior of the oracle value functions.
Revised: The discriminators are learned to mimic the behavior of the oracle value functions.

Original: Following (Gygli et al., 2017), we utilize intersection over union (IOU) and F 1 metrics as the oracle value functions for image segmentation and multi-label classiﬁcation, respectively, which are deﬁned on ( y , y ∗ ) ∈ { 1 , 0 } M × { 0 , 1 } M ,
Revised: Following (Gygli et al., 2017), we utilize intersection over union (IOU) and F 1 metrics as the oracle value functions for image segmentation and multi-label classiﬁcation, respectively, which are deﬁned on ( y , y ∗ ) ∈ { 1 , 0 } M × { 0 , 1 } M ,

Original: [Equation]
Revised: [Equation]

Original: Here y ∩ y ∗ denotes the number of dimension i where both y i and y ∗ i are active and y ∪ y ∗ denotes the number of dimensions where at least one of y i and y ∗ i is active.
Revised: Here y ∩ y ∗ denotes the number of dimension i where both y i and y ∗ i are active and y ∪ y ∗ denotes the number of dimensions where at least one of y i and y ∗ i is active.

Original: y i
Revised: y i

Original: and y ∗ i denote the i -th variable of y and y ∗ .
Revised: and y ∗ i denote the i -th variable of y and y ∗ .

Original: To apply v ∗ ( y , y ∗ ) to the continuous output y , the notions of intersection and union are extended by using element-wise min and max operators,
Revised: To apply v ∗ ( y , y ∗ ) to the continuous output y , the notions of intersection and union are extended by using element-wise min and max operators,

Original: [Equation]
Revised: [Equation]

Original: We propose to learn discriminators D ( x , y ; θ d ) to estimate v ∗ ( y , y ∗ ) .
Revised: We propose to learn discriminators D ( x , y ; θ d ) to estimate v ∗ ( y , y ∗ ) .

Original: The learning of discriminators can be understood from a regression setting with z = ( x , y ) as inputs and v = v ∗ ( x , y ) as the target outputs.
Revised: The learning of discriminators can be understood from a regression setting with z = ( x , y ) as inputs and v = v ∗ ( x , y ) as the target outputs.

Original: The structured prediction models are learned to predict outputs scored highly by discriminators.
Revised: The structured prediction models are learned to predict outputs scored highly by discriminators.

Original: Discriminators and structured prediction models are respectively learned by optimizing:
Revised: Discriminators and structured prediction models are respectively learned by optimizing:

Original: [Equation]
Revised: [Equation]

Original: Here v ∗ g = v ∗ ( G ( x ; θ g ) , y ∗ ) , and L g is a task-speciﬁc loss.
Revised: Here v ∗ g = v ∗ ( G ( x ; θ g ) , y ∗ ) , and L g is a surrogate loss.

Original: Equation 5 can be understood in two aspects: (1) It’s a modiﬁed version of least-square GAN (Mao et al., 2016) loss, and discriminators are learned to predict oracle values v ∗ g for the samples predicted by structured prediction models; (2) It learns an energy-based model using training samples that consist of ground-truth samples ( x , y ∗ , 1) and predicted samples ( x , G ( x ; θ g ) , v ∗ g ) .
Revised: Equation 5 can be understood in two aspects: (1) It’s a modiﬁed version of least-square GAN (Mao et al., 2016) loss, and discriminators are learned to predict oracle values v ∗ g for the samples predicted by structured prediction models; (2) It learns an energy-based model using training samples that consist of ground-truth samples ( x , y ∗ , 1) and predicted samples ( x , G ( x ; θ g ) , v ∗ g ) .

Original: The second term of Equation 6 regularizes structured prediction models such that the predicted samples tend to have higher scores.
Revised: The second term of Equation 6 regularizes structured prediction models such that the predicted samples tend to have higher scores.

Original: Once models are trained, discriminators can reﬁne structured prediction models by utilizing a gradient-based inference.
Revised: Once models are trained, discriminators can reﬁne structured prediction models by utilizing a gradient-based inference.

Original: The outputs predicted by structured prediction models are updated following the ascent gradient directions of discriminators that lead to the high scores:
Revised: The outputs predicted by structured prediction models are updated following the ascent gradient directions of discriminators that lead to the high scores:

Original: [Equation]
Revised: [Equation]

Original: Here, P Y denotes an operator that projects the predicted outputs back to the feasible set of solutions.
Revised: Here, P Y denotes an operator that projects the predicted outputs back to the feasible set of solutions.

Original: In a simple case, where Y =
Revised: In a simple case, where Y =

Original: [0 , 1] M , the P Y operator clips the predicted outputs.
Revised: [0 , 1] M , the P Y operator clips the predicted outputs.

Original: It is observed in experiments, i.e. , the learned discriminators tend to give small gradients during the gradient-based inference.
Revised: It is observed in experiments, i.e. , the learned discriminators tend to give small gradients during the gradient-based inference.

Original: One reason is that the predicted outputs of structured prediction models are already close to the ground-truth labels.
Revised: One reason is that the predicted outputs of structured prediction models are already close to the ground-truth labels.

Original: In order to further improve the predicted outputs by using the gradientbased inference and overcome the small-gradient issue, we use the normalized gradient method, i.e. ,
Revised: In order to further improve the predicted outputs by using the gradientbased inference and overcome the small-gradient issue, we use the normalized gradient method, i.e. ,

Original: [Equation]
Revised: [Equation]

Original: Gygli et al.
Revised: Gygli et al.

Original: (2017) proposed to simultaneously generate training samples to learn their energy-based model, i.e. , Deep Value Network (DVN) in the training process.
Revised: (2017) proposed to simultaneously generate training samples to learn their energy-based model, i.e. , Deep Value Network (DVN) in the training process.

Original: We follow their method to generate extra training samples to learn discriminators.
Revised: We follow their method to generate extra training samples to learn discriminators.

Original: The training samples are a set of tuples (input x , output y , oracle value v ∗ ) represented as T ≡ { ( x ( i ) , y ( i ) , v ∗ ( i ) )
Revised: The training samples are a set of tuples (input x , output y , oracle value v ∗ ) represented as T ≡ { ( x ( i ) , y ( i ) , v ∗ ( i ) )

Original: } Ni =1 .
Revised: } Ni =1 .

Original: Here N is the size of the training set, and x ( i ) , y ( i ) and v ∗ ( i ) respectively denote the i -th image, i -th output and i -th oracle value in the training set.
Revised: Here N is the size of the training set, and x ( i ) , y ( i ) and v ∗ ( i ) respectively denote the i -th image, i -th output and i -th oracle value in the training set.

Original: Similar to (Gygli et al., 2017), we utilize two different methods to generate training samples:
Revised: Similar to (Gygli et al., 2017), we utilize two different methods to generate training samples:

Original: • Inference samples.
Revised: • Inference samples.

Original: At each training iteration, structured prediction models are ﬁrst used to predict samples.
Revised: At each training iteration, structured prediction models are ﬁrst used to predict samples.

Original: Then, we take these samples as initial solutions and run a gradient-based inference to ﬁnd high-valued samples of discriminators.
Revised: Then, we take these samples as initial solutions and run a gradient-based inference to ﬁnd high-valued samples of discriminators.

Original: These samples are useful since they are generated along the inference trajectory at the inference stage.
Revised: These samples are useful since they are generated along the inference trajectory at the inference stage.

Original: These samples can be generated during training by using models in a previous iteration (Gygli et al., 2017).
Revised: These samples can be generated during training by using models in a previous iteration (Gygli et al., 2017).

Original: • Adversarial samples.
Revised: • Adversarial samples.

Original: Maximize the loss: 12 ( D ( x , y ; θ d ) − v ∗ ) 2 with respect to y using a gradient-based optimizer (Goodfellow et al., 2014b).
Revised: Maximize the loss: 12 ( D ( x , y ; θ d ) − v ∗ ) 2 with respect to y using a gradient-based optimizer (Goodfellow et al., 2014b).

Original: These samples serve as negative samples to learn discriminators.
Revised: These samples serve as negative samples to learn discriminators.

Original: Similar to the inference samples, the adversarial samples are also generated during training.
Revised: Similar to the inference samples, the adversarial samples are also generated during training.

Original: To utilize these training samples to learn discriminators, we add another loss term to the Equation 5, and the new objective function is as follows:
Revised: To utilize these training samples to learn discriminators, we add another loss term to the Equation 5, and the new objective function is as follows:

Original: [Equation]
Revised: [Equation]

Original: We utilize the Adam optimizer (Kingma & Ba, 2014) with the momentum term β 1 = 0 .
Revised: We utilize the Adam optimizer (Kingma & Ba, 2014) with the momentum term β 1 = 0 .

Original: 5 to train structure prediction models and discriminators.
Revised: 5 to train structure prediction models and discriminators.

Original: At each training iteration, we randomly generate inference or adversarial samples and update the parameters of the structured prediction models and the discriminators according to Equation (6) and Equation (10).
Revised: At each training iteration, we randomly generate inference or adversarial samples and update the parameters of the structured prediction models and the discriminators according to Equation (6) and Equation (10).

Original: The models are trained until convergence.
Revised: The models are trained until convergence.

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: Experiments are conducted on three tasks: multi-label classiﬁcation, binary image segmentation, and 3-class face segmentation.
Revised: Experiments are conducted on three tasks: multi-label classiﬁcation, binary image segmentation, and 3-class face segmentation.

Original: We compare our LDRSP to other state-of-the-art adversarial learning methods on these tasks, and results are reported in Section 4.1, 4.2 and 4.3.
Revised: We compare our LDRSP to other state-of-the-art adversarial learning methods on these tasks, and results are reported in Section 4.1, 4.2 and 4.3.

Original: The code is implemented using the deep learning framework, i.e. , Tensorﬂow (Abadi et al., 2016).
Revised: The code is implemented using the deep learning framework, i.e. , Tensorﬂow (Abadi et al., 2016).

Original: 4. P REDICTION P ERFORMANCE ON M ULTI - LABEL C LASSIFICATION
Revised: 4. P REDICTION P ERFORMANCE ON M ULTI - LABEL C LASSIFICATION

Original: We use standard benchmarks of this task, namely Bibtex and Bookmarks introduced by (Katakis et al., 2008) in this section.
Revised: We use standard benchmarks of this task, namely Bibtex and Bookmarks introduced by (Katakis et al., 2008) in this section.

Original: On these two datasets, tags need to be predicted for text inputs and multiple labels are possible for each input.
Revised: On these two datasets, tags need to be predicted for text inputs and multiple labels are possible for each input.

Original: A two-layer neural network (Belanger & McCallum, 2016) is utilized as our classiﬁer.
Revised: A two-layer neural network (Belanger & McCallum, 2016) is utilized as our structured prediction model.

Original: The same network architecture of DVN is adopted as our discriminator, which consists of one or two hidden layers with Softplus non-linearities.
Revised: The same network architecture of DVN is adopted as our discriminator, which consists of one or two hidden layers with Softplus non-linearities.

Original: Following (Belanger & McCallum, 2016), a cross-entropy loss is used as the task-speciﬁc loss.
Revised: Following (Belanger & McCallum, 2016), a cross-entropy loss is used as the surrogate loss.

Original: Besides the proposed LDRSP, different adversarial learning methods, e.g. , GAN (Goodfellow et al., 2014a), WGAN+GP (Gulrajani et al., 2017), LSGAN (Mao et al., 2016) and EBGAN (Zhao et al., 2017) are utilized to learn the classiﬁers and the discriminators.
Revised: Besides the proposed LDRSP, different adversarial learning methods, e.g. , GAN (Goodfellow et al., 2014a), WGAN+GP (Gulrajani et al., 2017), LSGAN (Mao et al., 2016) and EBGAN (Zhao et al., 2017) are utilized to learn the structured prediction models and the discriminators.

Original: These classiﬁers and discriminators have the same network architecture.
Revised: These structured prediction models and discriminators have the same network architecture.

Original: For all the methods, the hyper-parameter exploration is performed, and we follow the gradient directions of the discriminators to reﬁne the classiﬁers at the inference stage.
Revised: For all the methods, the hyper-parameter exploration is performed, and we follow the gradient directions of the discriminators to reﬁne the structured prediction models at the inference stage.

Original: The experimental results are reported in Table 1.
Revised: The experimental results are reported in Table 1.

Original: For all the adversarial learning methods, we report both the prediction results of classiﬁers and the prediction results reﬁned by the discriminators.
Revised: For all the adversarial learning methods, we report both the prediction results of structured prediction models G and the prediction results reﬁned by the discriminators D .

Original: As it shows, the proposed LDRSP successfully learns discriminators that can be utilized to reﬁne the classiﬁers and achieve state-of-the-art performance on both Bibtex and Bookmarks datasets.
Revised: As it shows, the proposed LDRSP successfully learns D that can be utilized to reﬁne the G and achieve state-of-the-art performance on both Bibtex and Bookmarks datasets.

Original: The reﬁnement improves the performance of the classiﬁers by 3.6 % on Bibtex dataset and 1. % on Bookmarks dataset.
Revised: The reﬁnement improves the performance of G by 3.6 % on Bibtex dataset and 1.1 % on Bookmarks dataset.

Original: However, for other adversarial learning methods, the reﬁnement leads to negligible improvements or even decrements in performance.
Revised: However, for other adversarial learning methods, the reﬁnement leads to negligible improvements or even decrements in performance.

Original: For example, the reﬁnement of EBGAN improves the performance by 0.2 % on Bookmarks dataset, and the reﬁnement of LSGAN decreases the performance by 0.1 % on Bibtex dataset.
Revised: For example, the reﬁnement of EBGAN improves the performance by 0.2 % on Bookmarks dataset, and the reﬁnement of LSGAN decreases the performance by 0.1 % on Bibtex dataset.

Original: Following (Belanger & McCallum, 2016), we implement aclassiﬁer baseline model by training classiﬁers via minimizing a cross-entropy loss, i.e. , the ﬁrst term of Equation (6).
Revised: Following (Belanger & McCallum, 2016), we implement a baseline model by training structured prediction models via minimizing a cross-entropy loss, i.e. , the ﬁrst term of Equation (6).

Original: Comparing the performance of theclassiﬁer baseline model and the performance of our adversarially learned classiﬁers, we notice that jointly learning classiﬁers and discriminators via the proposed method greatly improves the performance of classiﬁers.
Revised: Comparing the performance of the baseline model and the performance of our adversarially learned structured prediction models, we notice that jointly learning structured prediction models and discriminators via the proposed method greatly improves the performance of structured prediction models.

Original: The performance of state-of-the-art structured prediction methods e.g. , logistic regression, a twolayer neural network learned with a cross-entropy loss, SPEN (Belanger & McCallum, 2016),
Revised: The performance of state-of-the-art structured prediction methods e [Table] g. , logistic regression, a twolayer neural network learned with a cross-entropy loss, SPEN (Belanger & McCallum, 2016),

Original: PRLR (Lin et al., 2014), and DVN (Gygli et al., 2017) is also reported in Table 1.
Revised: PRLR (Lin et al., 2014) [Table] and DVN (Gygli et al., 2017) is also reported in Table 1.

Original: Although our method, the DVN, and the SPEN have the same energy network architecture, our method outperforms the DVN and the SPEN on both Bibtex and Bookmarks datasets.
Revised: Although our method, the DVN, and the SPEN have the same energy network architecture, our method outperforms the DVN and the SPEN on both Bibtex and Bookmarks datasets.

Original: Our method also greatly improves over feed-forward models: the logistic regression, the two-layer neural network [Table]
Revised: Our method also greatly improves over feed-forward models: the logistic regression, the two-layer neural network, and

Original: and the
Revised: the

Original: PRLP.
Revised: PRLP.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: P REDICTION P ERFORMANCE ON B INARY I MAGE S EGMENTATION
Revised: P REDICTION P ERFORMANCE ON B INARY I MAGE S EGMENTATION

Original: Compared with the multi-label classiﬁcation task, the image segmentation task is more challenging due to the high-dimensional output space and the complex correlation among variables in the segmentation masks.
Revised: Compared with the multi-label classiﬁcation task, the image segmentation task is more challenging due to the high-dimensional output space and the complex correlation among variables in the segmentation masks.

Original: We utilize the Weizmann horses dataset (Borenstein & Ullman, 2004) in this section.
Revised: We utilize the Weizmann horses dataset (Borenstein & Ullman, 2004) in this section.

Original: It is a commonly used dataset for binary image segmentation which consists of 328 left oriented horse images and their corresponding binary segmentation masks.
Revised: It is a commonly used dataset for binary image segmentation which consists of 328 left oriented horse images and their corresponding binary segmentation masks.

Original: Following (Gygli et al., 2017; Li et al., 2013), all images and segmentation masks are resized to 32 × 32 .
Revised: Following (Gygli et al., 2017; Li et al., 2013), all images and segmentation masks are resized to 32 × 32 .

Original: The segmentation of horses at this low resolution is challenging and requires models to capture strong priors of the horse shape, since some thin parts of the horse like legs, tails are almost invisible in the images.
Revised: The segmentation of horses at this low resolution is challenging and requires models to capture strong priors of the horse shape, since some thin parts of the horse like legs, tails are almost invisible in the images.

Original: We follow the experimental protocol of (Li et al., 2013) to split the Weizmann horses dataset and report results on the same testing set.
Revised: We follow the experimental protocol of (Li et al., 2013) to split the Weizmann horses dataset and report results on the same testing set.

Original: In the experiment, we adopt the fully convolutional network (FCN) (Long et al., 2015) baseline model proposed in (Gygli et al., 2017) as our segmentation network.
Revised: In the experiment, we adopt the fully convolutional network (FCN) (Long et al., 2015) baseline model proposed in (Gygli et al., 2017) as our structured prediction model.

Original: It consists of three 5 ×convolutional layers and two deconvolution layers.
Revised: It consists of three 5 ×convolutional layers and two deconvolution layers.

Original: We ﬁnd that using discriminators similar to
Revised: We ﬁnd that using discriminators similar to

Original: PatchGAN discriminators (Isola et al., 2017; Liu et al., 2017) improves the prediction performance.
Revised: PatchGAN discriminators (Isola et al., 2017; Liu et al., 2017) improves the prediction performance.

Original: Therefore, our discriminators are designed to map ( x , y ) to score matrices.
Revised: Therefore, our discriminators are designed to map ( x , y ) to score matrices.

Original: Here x ∈ R W × 3 , y ∈ R W × C , W is the number of pixels in an image, and, C is the number of object classes.
Revised: Here x ∈ R W × 3 , y ∈ R W × C , W is the number of pixels in an image, and, C is the number of object classes.

Original: We view image segmentation as pixel-level multi-label classiﬁcation.
Revised: We view image segmentation as pixel-level multi-label classiﬁcation.

Original: Instead of approximating discriminators to the intersection over union (IOU) metric, we calculate the F 1 metric for each pixel and estimate an oracle value function v ∗ ∈ [0 , 1] W ,
Revised: Instead of approximating discriminators to the intersection over union (IOU) metric, we calculate the F 1 metric for each pixel and estimate an oracle value function v ∗ ∈ [0 , 1] W ,

Original: [Equation]
Revised: [Equation]

Original: (11) where y i ∈ R C denotes the i -th row of y .
Revised: (11) where y i ∈ R C denotes the i -th row of y .

Original: Our discriminators are implemented by modifying the
Revised: Our discriminators are implemented by modifying the

Original: FCN baseline model: the number of object classes is set to 1 and a sigmoid function is added at the end of the discriminator to ensure v ( x , y ) ∈ (0 , 1) W .
Revised: FCN baseline model: the number of object classes is set to 1 and a sigmoid function is added at the end of the discriminator to ensure v ( x , y ) ∈ (0 , 1) W .

Original: Similar to the multi-label classiﬁcation, a cross-entropy loss is adopted as the task-speciﬁc loss.
Revised: Similar to the multi-label classiﬁcation, a cross-entropy loss is adopted as the surrogate loss.

Original: For both discriminators and FCNs, we adopt Adam optimizers and set the learning rates to 0.01.
Revised: For both discriminators and FCNs, we adopt Adam optimizers and set the learning rates to 0.01.

Original: Data augmentation is utilized by using random 24 × 24 patches cropped from the 32 × 32 images and by randomly mirroring the images.
Revised: Data augmentation is utilized by using random 24 × 24 patches cropped from the 32 × 32 images and by randomly mirroring the images.

Original: We empirically ﬁnd that setting the inference step size η to be 4.0 and setting the number of inference steps to be 30 achieve the best performance.
Revised: We empirically ﬁnd that setting the inference step size η to be 4.0 and setting the number of inference steps to be 30 achieve the best performance.

Original: At the inference stage, we divide each test image into 36 crops.
Revised: At the inference stage, we divide each test image into 36 crops.

Original: The proposed method is utilized to estimate segmentation masks for these crops.
Revised: The proposed method is utilized to estimate segmentation masks for these crops.

Original: The estimated segmentation masks are averaged to obtain the ﬁnal segmentation mask.
Revised: The estimated segmentation masks are averaged to obtain the ﬁnal segmentation mask.

Original: We notice that the proposed method usually converges within 20 inference steps.
Revised: We notice that the proposed method usually converges within 20 inference steps.

Original: [Table]
Revised: [Table]

Original: As commonly done in the literature
Revised: As commonly done in the literature

Original: [Table]
Revised: [Table]

Original: we report the mean image IOU as well as the IOU over the whole testing set on the Weizmann horses dataset in Table 2.
Revised: we report the mean image IOU as well as the IOU over the whole testing set on the Weizmann horses dataset in Table 2.

Original: A higher IOU score means a more accurate segmentation mask.
Revised: A higher IOU score means a more accurate segmentation mask.

Original: It is clear that the proposed LDRSP outperforms other state-of-the-art methods on both metrics.
Revised: It is clear that the proposed LDRSP outperforms other state-of-the-art methods on both metrics.

Original: It can be observed that using the discriminators of the LDRSP to reﬁne the FCN improves the performance by 4.2 % on the Mean IOU metric and 4.1 % on the Global
Revised: It can be observed that using the discriminators of the LDRSP to reﬁne the FCN improves the performance by 4.2 % on the Mean IOU metric and 4.1 % on the Global

Original: IOU metric.
Revised: IOU metric.

Original: It indicates that our LDRSP learns discriminators that are able to estimate stronger horse shape priors than the FCN.
Revised: It indicates that our LDRSP learns discriminators that are able to estimate stronger horse shape priors than the FCN.

Original: As it can be seen, the reﬁnement of discriminators learned by other adversarial learning methods, e.g. , GAN (Goodfellow et al., 2014a), WGAN+GP (Gulrajani et al., 2017), LSGAN (Mao et al., 2016) decreases the performance.
Revised: As it can be seen, the reﬁnement of discriminators learned by other adversarial learning methods, e.g. , GAN (Goodfellow et al., 2014a), WGAN+GP (Gulrajani et al., , LSGAN (Mao et al., 2016) decreases the performance.

Original: It’s due to the fact that these methods learn discriminators as classiﬁers which assign almost the same scores to all the predicted samples.
Revised: It’s due to the fact that these methods learn discriminators as classiﬁers which assign almost the same scores to all the predicted samples.

Original: Thus, it’s difﬁcult to utilize these discriminators to improve the performance of FCNs.
Revised: Thus, it’s difﬁcult to utilize these discriminators to improve the performance of FCNs.

Original: Our LDRSP outperforms the DVN (Gygli et al., 2017).
Revised: Our LDRSP outperforms the DVN (Gygli et al., 2017).

Original: It indicates that utilizing the proposed method to jointly learn energy-based models and structured prediction models advances over learning energy-based models alone.
Revised: It indicates that utilizing the proposed method to jointly learn energy-based models and structured prediction models advances over learning energy-based models alone.

Original: The qualitative results on the Weizmann horses dataset are shown in Figure 1.
Revised: The qualitative results on the Weizmann horses dataset are shown in Figure 1.

Original: It can be seen that
Revised: It can be seen that

Original: FCN shows poor performance in segmenting thin parts like legs and generating single-connected segmentation masks.
Revised: FCN shows poor performance in segmenting thin parts like legs and generating single-connected segmentation masks.

Original: The proposed method utilizes discriminators to reﬁne the predictions of the
Revised: The proposed method utilizes discriminators to reﬁne the predictions of the

Original: FCN by ﬁlling the missing part ( e.g. , Figure 1, seventh and eighth row, far left images), generating legs to connect disconnected parts ( e.g. , Figure 1, seventh and eighth row, ﬁrst and second images from the right).
Revised: FCN by ﬁlling the missing part ( e.g. , Figure 1, seventh and eighth row, far left images), generating legs to connect disconnected parts ( e.g. , Figure 1, seventh and eighth row, ﬁrst and second images from the right).

Original: [Figure]
Revised: [Figure]

Original: 4. P REDICTION P ERFORMANCE ON 3- CLASS F ACE S EGMENTATION
Revised: 4. P REDICTION P ERFORMANCE ON 3- CLASS F ACE S EGMENTATION

Original: We utilize the Labeled Faces on the Wild (LFW) dataset (Huang et al., 2007) to evaluate our method on 3-class face segmentation.
Revised: We utilize the Labeled Faces on the Wild (LFW) dataset (Huang et al., 2007) to evaluate our method on 3-class face segmentation.

Original: This dataset contains more than 13,000 images, which were ﬁrst introduced for face recognition and later were annotated on a subset of 2,927 images for face segmentation.
Revised: This dataset contains more than 13,000 images, which were ﬁrst introduced for face recognition and later were annotated on a subset of 2,927 images for face segmentation.

Original: The annotations provide superpixel-level labels which consist of three classes: face, hair, and background.
Revised: The annotations provide superpixel-level labels which consist of three classes: face, hair, and background.

Original: Since our method generates pixel-level labels, we map pixel-level labels to superpixel-level labels by using the most frequent labels in a superpixel as the superpixel’s label following (Tsogkas et al., 2015; Gygli et al., 2017).
Revised: Since our method generates pixel-level labels, we map pixel-level labels to superpixel-level labels by using the most frequent labels in a superpixel as the superpixel’s label following (Tsogkas et al., 2015; Gygli et al., 2017).

Original: We adopt the training, validation, and testing splits proposed in (Kae et al., 2013; Tsogkas et al., 2015; Gygli et al., 2017).
Revised: We adopt the training, validation, and testing splits proposed in (Kae et al., 2013; Tsogkas et al., 2015; Gygli et al., 2017).

Original: The network architecture and the data augmentation are the same as those utilized on the Weizmann horses dataset.
Revised: The network architecture and the data augmentation are the same as those utilized on the Weizmann horses dataset.

Original:  We compare the proposed method with other methods on the LFW dataset in Table 3.
Revised: [Table] [Figure] We compare the proposed method with other methods on the LFW dataset in Table 3.

Original: It can be observed that the FCN baseline outperforms the DVN (Gygli et al., 2017) by 2.92 % on the superpixel[Table] [Figure] accuracy metric, while our method outperforms the FCN baseline by 1.11 % .
Revised: It can be observed that the FCN baseline outperforms the DVN (Gygli et al., 2017) by 2.92 % on the superpixel accuracy metric, while our method outperforms the FCN baseline by 1.11 % .

Original: The performance improvement between our method and the DVN (Gygli et al., 2017) on the LFW dataset is more signiﬁcant than the performance improvement on the Weizmann horses dataset.
Revised: The performance improvement between our method and the DVN (Gygli et al., 2017) on the LFW dataset is more signiﬁcant than the performance improvement on the Weizmann horses dataset.

Original: It indicates that utilizing the proposed method to jointly learn an energy-based model and a FCN greatly improves the performance of the energy-based model, when the output space is large.
Revised: It indicates that utilizing the proposed method to jointly learn an energy-based model and a FCN greatly improves the performance of the energy-based model, when the output space is large.

Original: The qualitative results of the LDRSP on the LFW dataset are shown in Figure 2.
Revised: The qualitative results of the LDRSP on the LFW dataset are shown in Figure 2.

Original: As it can be observed, our method can generate high-quality hair and face segmentation masks that are close to the ground-truth labels.
Revised: As it can be observed, our method can generate high-quality hair and face segmentation masks that are close to the ground-truth labels.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: This paper proposes a novel learning framework, in which discriminative models are learned to reﬁne structured prediction models.
Revised: This paper proposes a novel learning framework, in which discriminative models are learned to reﬁne structured prediction models.

Original: Discriminative models are trained as energy- based models to estimate scores that measure the quality of generated samples.
Revised: Discriminative models are trained as energy-based models to estimate scores that measure the quality of generated samples.

Original: Structured prediction models are trained to predict contrastive samples with maximum energy scores.
Revised: Structured prediction models are trained to predict contrastive samples with maximum energy scores.

Original: Once models are learned, we perform inference by following the ascent gradient directions of discriminative models to reﬁne structured prediction models.
Revised: Once models are learned, we perform inference by following the ascent gradient directions of discriminative models to reﬁne structured prediction models.

Original: We apply the proposed method to multi-label classiﬁcation and image segmentation tasks.
Revised: We apply the proposed method to multi-label classiﬁcation and image segmentation tasks.

Original: The experimental results indicate that discriminative models learned by the proposed methods can effectively reﬁne generative models.
Revised: The experimental results indicate that discriminative models learned by the proposed methods can effectively reﬁne generative models.

Original: As the future work, we will explore different ways to generate extra training samples and apply our method to more challenging tasks.
Revised: As the future work, we will explore different ways to generate extra training samples and apply our method to more challenging tasks.