Original: CaptainGAN: Navigate Through Embedding Space For Better Text Generation
Revised: CaptainGAN: Navigate Through Embedding Space For Better Text Generation

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: Abstract
Revised: Abstract

Original: Score-function-based text generation approaches such as REINFORCE, in general, suffer from high computational complexity and training instability problems.
Revised: Score-function-based text generation approaches such as REINFORCE, in general, suffer from high computational complexity and training instability problems.

Original: This is mainly due to the non-differentiable nature of the discrete space sampling and thus these methods have to treat the discriminator as a reward function and ignore the gradient information.
Revised: This is mainly due to the non-differentiable nature of the discrete space sampling and thus these methods have to treat the discriminator as a reward function and ignore the gradient information.

Original: In this paper, we propose a novel approach, CaptainGAN, which adopts the straight-through gradient estimator and introduces a ”re-centered” gradient estimation technique to steer the generator toward better text tokens through the embedding space.
Revised: In this paper, we propose a novel approach, CaptainGAN, which adopts the straight-through gradient estimator and introduces a ”re-centered” gradient estimation technique to steer the generator toward better text tokens through the embedding space.

Original: Our method is stable to train and converges quickly without maximum likelihood pre-training.
Revised: Our method is stable to train and converges quickly without maximum likelihood pre-training.

Original: On multiple metrics of text quality and diversity, our method outperforms existing GAN-based methods on natural language generation.
Revised: On multiple metrics of text quality and diversity, our method outperforms existing GAN-based methods on natural language generation.

Original:  Introduction
Revised:  Introduction

Original: Generative Adversarial Networks (GAN) (Goodfellow et al., 2014) have led to many advances in image generation, image editing, style transfer, and representation learning (Karras et al., 2017;
Revised: Generative Adversarial Networks (GAN) (Goodfellow et al., 2014) have led to many advances in image generation, image editing, style transfer, and representation learning (Karras et al., 2017;

Original: Brock et al., 2018; Karras et al., 2019).
Revised: Brock et al., 2018; Karras et al., 2019).

Original: Unsurprisingly, much effort has been devoted to adopting the GAN framework for unsupervised text generation (Yu et al., 2017; Che et al., 2017; Balagopalan et al., 2018; Fedus et al., 2018; Guo et al., 2018; de Masson d’Autume et al., 2019;
Revised: Unsurprisingly, much effort has been devoted to adopting the GAN framework for unsupervised text generation (Yu et al., 2017; Che et al., 2017; Balagopalan et al., 2018; Fedus et al., 2018; Guo et al., 2018; de Masson d’Autume et al., 2019;

Original: Zhang et al., 2017; Nie et al., 2019).
Revised: Zhang et al., 2017; Nie et al., 2019).

Original: However, as finding a Nash equilibrium is not as straightforward as finding a local optima, researchers have been forced to develop many ad-hoc tricks and techniques to make GAN training well-behaved.
Revised: However, as finding a Nash equilibrium is not as straightforward as finding a local optima, researchers have been forced to develop many ad-hoc tricks and techniques to make GAN training well-behaved.

Original: In the text generation setting, they are also faced with the additional obstacle of passing discrete tokens through a non-differentiable operation, which prohibits back-propagating the gradient signal to the generator.
Revised: In the text generation setting, they are also faced with the additional obstacle of passing discrete tokens through a non-differentiable operation, which prohibits back-propagating the gradient signal to the generator.

Original: To address the issue of non-differentiablity, researchers and practitioners use score function gradient estimators such as REINFORCE to train GANs for text generation, where the discriminator is cast as a reward function for the generator.
Revised: To address the issue of non-differentiablity, researchers and practitioners use score function gradient estimators such as REINFORCE to train GANs for text generation, where the discriminator is cast as a reward function for the generator.

Original: However, these methods still suffer from poor sample efficiency due to the credit assignment problem.
Revised: However, these methods still suffer from poor sample efficiency due to the credit assignment problem.

Original: We argue that it is disadvantageous to utilize the discriminator as simply a reward function when it is known that gradient-based backpropagation is a far more efficient way to perform credit assignment.
Revised: We argue that it is disadvantageous to utilize the discriminator as simply a reward function when it is known that gradient-based backpropagation is a far more efficient way to perform credit assignment.

Original: In this paper, we propose a novel unsupervised text generation technique, called CaptainGAN, which propagates a modified gradient signal from discriminator to generator in order to improve the efficiency and accuracy of the estimator.
Revised: In this paper, we propose a novel unsupervised text generation technique, called CaptainGAN, which propagates a modified gradient signal from discriminator to generator in order to improve the efficiency and accuracy of the estimator.

Original: Our contributions are as follows:
Revised: Our contributions are as follows:

Original: • An update procedure for the generator to incorporate gradient information from the discriminator during generator training.
Revised: • An update procedure for the generator to incorporate gradient information from the discriminator during generator training.

Original: • Lower memory and computational requirements than other RL-based counterparts.
Revised: • Lower memory and computational requirements than other RL-based counterparts.

Original: • Near SOTA results without maximum likelihood pretraining.
Revised: • Near SOTA results without maximum likelihood pretraining.

Original: Please see Appendix A for a detailed description of the notation used in this paper.
Revised: Please see Appendix A for a detailed description of the notation used in this paper.

Original:  Background
Revised:  Background

Original: The Generative Adversarial Network (GAN) proposed in Goodfellow et al.
Revised: The Generative Adversarial Network (GAN) proposed in Goodfellow et al.

Original: (2014) is an innovative approach to the generative modeling problem.
Revised: (2014) is an innovative approach to the generative modeling problem.

Original: Rather than using the maximum likelihood estimation (MLE) directly to learn a probabilistic model, the GAN is a two-player minimax game in which the goal of one player, the generator G (cid:18) , is to generate samples ^ x from p (cid:18) , and the goal of the other player, the discriminator D ϕ , is to learn to classify whether or not a sample was generated from real data p data or the generator.
Revised: Rather than using the maximum likelihood estimation (MLE) directly to learn a probabilistic model, the GAN is a two-player minimax game in which the goal of one player, the generator G (cid:18) , is to generate samples ^ x from p (cid:18) , and the goal of the other player, the discriminator D ϕ , is to learn to classify whether or not a sample was generated from real data p data or the generator.

Original: [Equation]
Revised: [Equation]

Original: where V D and V G are respectively the objective functions of the discriminator D ϕ and the generator G (cid:18) .
Revised: where V D and V G are respectively the objective functions of the discriminator D ϕ and the generator G (cid:18) .

Original: Equation 2 is the alternative generator loss suggested by the original work as its gradient does not vanish when D ϕ (^ x ) is small.
Revised: Equation 2 is the alternative generator loss suggested by the original work as its gradient does not vanish when D ϕ (^ x ) is small.

Original: In the standard GAN architecture, the generator’s output is directly connected as the input to the discriminator in a fully differentiable manner, which means the gradients from the discriminator’s loss function can be back-propagated to the parameters of the generator.
Revised: In the standard GAN architecture, the generator’s output is directly connected as the input to the discriminator in a fully differentiable manner, which means the gradients from the discriminator’s loss function can be back-propagated to the parameters of the generator.

Original: However, it requires sampling a sequence of tokens from a discrete distribution in text generation, which is essentially non-differentiable.
Revised: However, it requires sampling a sequence of tokens from a discrete distribution in text generation, which is essentially non-differentiable.

Original: In order to avoid the intractability of the gradients, text GANs resort to some sort of estimation.
Revised: In order to avoid the intractability of the gradients, text GANs resort to some sort of estimation.

Original: 2.1 Continuous Relaxation
Revised: 2.1 Continuous Relaxation

Original: Continuous relaxation approaches such as Gumbel-Softmax (Jang et al., 2016) approximate a stochastic categorical distribution in terms of a deterministic continuous function.
Revised: Continuous relaxation approaches such as Gumbel-Softmax (Jang et al., 2016) approximate a stochastic categorical distribution in terms of a deterministic continuous function.

Original: While this apparently allows us to remove the non-differentiable discrete sampling altogether (Kusner & Hernández-Lobato, 2016), it creates several serious issues.
Revised: While this apparently allows us to remove the non-differentiable discrete sampling altogether (Kusner & Hernández-Lobato, 2016; Nie et al., 2019), it creates several serious issues.

Original: The continuous distribution interpolates between word embeddings and generates ”pseudo-embeddings” - a weighted sum of embeddings with no direct correspondence to an actual word or token.
Revised: The continuous distribution interpolates between word embeddings and generates ”pseudoembeddings” - a weighted sum of embeddings with no direct correspondence to an actual word or token.

Original: This means all the discriminator has to do is spot the difference the actual word embeddings and ”pseudoembeddings”.
Revised: This means all the discriminator has to do is spot the difference the actual word embeddings and ”pseudo-embeddings”.

Original: In turn, the generator will try to compensate by producing extremely ”spiky” predictions.
Revised: In turn, the generator will try to compensate by producing extremely ”spiky” predictions.

Original: Furthermore, this way of generating data creates a major inconsistency in that during training, the discriminator is given a pseudo-embedding (a mixture of real word embeddings) whereas during inference, the pseudo-embedding is used to inform a sampling procedure which produce a single token, which corresponds to an actual word embedding and not to any pseudo-embedding seen during training.
Revised: Furthermore, this way of generating data creates a major inconsistency in that during training, the discriminator is given a pseudo-embedding (a mixture of real word embeddings) whereas during inference, the pseudo-embedding is used to inform a sampling procedure which produce a single token, which corresponds to an actual word embedding and not to any pseudo-embedding seen during training.

Original: 2.2 Score-Function Gradient Estimator
Revised: 2.2 Score-Function Gradient Estimator

Original: The score function gradient estimator (Fu, 2006; Glynn, 1990), also known as the REINFORCE (Williams, 1992) is a common solution for non-differentiable issue as mentioned above.
Revised: The score function gradient estimator (Fu, 2006; Glynn, 1990), also known as the REINFORCE (Williams, 1992) is a common solution for non-differentiable issue as mentioned above.

Original: Applying the REINFORCE algorithm, the gradient of the expectation of reward function f ϕ can be written as
Revised: Applying the REINFORCE algorithm, the gradient of the expectation of reward function f ϕ can be written as

Original: [Equation]
Revised: [Equation]

Original: Since it does not require f ϕ to be differentiable or even continuous as a function of x , the gradient of E ^ x (cid:24) p (cid:18) [ f ϕ (^ x )] can be back-propagated to the generator G (cid:18) .
Revised: Since it does not require f ϕ to be differentiable or even continuous as a function of x , the gradient of E ^ x (cid:24) p (cid:18) [ f ϕ (^ x )] can be back-propagated to the generator G (cid:18) .

Original: In the context of
Revised: In the context of

Original: GANs, E ^ x (cid:24) p (cid:18) [ f ϕ (^ x )] can be seen as the objective function of the generator and the reward function f ϕ can be replaced with D ϕ .
Revised: GANs, E ^ x (cid:24) p (cid:18) [ f ϕ (^ x )] can be seen as the objective function of the generator and the reward function f ϕ can be replaced with D ϕ .

Original: Although REINFORCE is an unbiased estimator, it still has a number of disadvantages such as high variance, low sample efficiency and the credit assignment problem.
Revised: Although REINFORCE is an unbiased estimator, it still has a number of disadvantages such as high variance, low sample efficiency and the credit assignment problem.

Original: Therefore, much effort is devoted to reducing the variance using special methods (Gu et al., 2016; Grathwohl et al., 2017), or to providing more dense rewards such as in Yu et al.
Revised: Therefore, much effort is devoted to reducing the variance using special methods (Gu et al., 2016; Grathwohl et al., 2017), or to providing more dense rewards such as in Yu et al.

Original: ; Che et al.
Revised: ; Che et al.

Original: (2017); Fedus et al.
Revised: (2017); Fedus et al.

Original: (2018); de Masson d’Autume et al.
Revised: (2018); de Masson d’Autume et al.

Original: (2019), where Monte-Carlo roll-outs are used to obtain per-word rewards.
Revised: (2019), where Monte-Carlo roll-outs are used to obtain per-word rewards.

Original: 2.3 Straight-Through Gradient Estimator
Revised: 2.3 Straight-Through Gradient Estimator

Original: Another approach is using a straight-through gradient estimator (Bengio et al., 2013; Jang et al., 2016).
Revised: Another approach is using a straight-through gradient estimator (Bengio et al., 2013; Jang et al., 2016).

Original: The basic idea is to perform non-differentiable operation during the forward pass, but approximate it with a differentiable proxy during the backward pass.
Revised: The basic idea is to perform non-differentiable operation during the forward pass, but approximate it with a differentiable proxy during the backward pass.

Original: Consider sampling from a categorical distribution (notation borrowed from Jang et al.
Revised: Consider sampling from a categorical distribution (notation borrowed from Jang et al.

Original: The following operation is performed during the forward pass
Revised: The following operation is performed during the forward pass

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: ) where a categorical sample ^ x is encoded as a v -dimensional one-hot vector z ( v being the given vocabulary size).
Revised: ) where a categorical sample ^ x is encoded as a v -dimensional one-hot vector z ( v being the given vocabulary size).

Original: During the backward pass, the corresponding straight-through estimator of function f with respect to (cid:18) is
Revised: During the backward pass, the corresponding straight-through estimator of function f with respect to (cid:18) is

Original: [Equation]
Revised: [Equation]

Original: derived by using the approximation @
Revised: derived by using the approximation @

Original: z @(cid:18) (cid:25) @ m (cid:18) @(cid:18) , where m (cid:18) is a differentiable proxy for the non-differentiable one-hot encoding vector z
Revised: z @(cid:18) (cid:25) @ m (cid:18) @(cid:18) , where m (cid:18) is a differentiable proxy for the non-differentiable one-hot encoding vector z

Original: (Jang et al., 2016).
Revised: (Jang et al., 2016).

Original: Moreover, Jang et al.
Revised: Moreover, Jang et al.

Original: (2016) suggest the choice of
Revised: (2016) suggest the choice of

Original: [Equation]
Revised: [Equation]

Original: 6) as proxy, where f x 1 ; : : : ; x v g are all possible tokens in vocabulary V .
Revised: 6) as proxy, where f x 1 ; : : : ; x v g are all possible tokens in vocabulary V .

Original: Compared to the former method, this approach updates all the possible tokens’ probability, even if the tokens were not sampled.
Revised: Compared to the former method, this approach updates all the possible tokens’ probability, even if the tokens were not sampled.

Original:  Proposed Method
Revised:  Proposed Method

Original: The non-differentiable connection between the generator G (cid:18) and the discriminator D ϕ is our primary focus in this paper (Figure 1).
Revised: The non-differentiable connection between the generator G (cid:18) and the discriminator D ϕ is our primary focus in this paper (Figure 1).

Original: Starting from the straight through gradient estimator, we derive a re-centered estimator which helps the generator learn to select better tokens.
Revised: Starting from the straight through gradient estimator, we derive a re-centered estimator which helps the generator learn to select better tokens.

Original: 3.1 Gradient Estimation in the Generator
Revised: 3.1 Gradient Estimation in the Generator

Original: Consider a discriminator whose first layer is an embedding lookup layer.
Revised: Consider a discriminator whose first layer is an embedding lookup layer.

Original: During forward propagation, the operation below is performed first:
Revised: During forward propagation, the operation below is performed first:

Original: [Equation]
Revised: [Equation]

Original: where E = [ e 1 ; : : : ; e v ] and z is one-hot encoding of vocabulary.
Revised: where E = [ e 1 ; : : : ; e v ] and z is one-hot encoding of vocabulary.

Original: As described in Section 2.3, this is a non-differentiable categorical sampling operation with a straight-through estimator of the gradient of V G with respect to (cid:18) , formulated as follows:
Revised: As described in Section 2.3, this is a non-differentiable categorical sampling operation with a straight-through estimator of the gradient of V G with respect to (cid:18) , formulated as follows:

Original: [Equation]
Revised: [Equation]

Original: where ^ x t is the selected token at time-step t , T is the maximum time-step and V is the given vocabulary.
Revised: where ^ x t is the selected token at time-step t , T is the maximum time-step and V is the given vocabulary.

Original: The first term of Equation 8 can be derived from Equation 7 by applying the chain rule.
Revised: The first term of Equation 8 can be derived from Equation 7 by applying the chain rule.

Original: 3.2 Re-centered Estimator
Revised: 3.2 Re-centered Estimator

Original: We turn our attention to the first term of Equation 9:
Revised: We turn our attention to the first term of Equation 9:

Original: [Equation]
Revised: [Equation]

Original: Denoting it as (cid:14) ^ x t !
Revised: Denoting it as (cid:14) ^ x t !

Original: x [ V G ] , Equation 9 becomes
Revised: x [ V G ] , Equation 9 becomes

Original: [Equation]
Revised: [Equation]

Original: The coefficient (cid:14) ^ x t !
Revised: The coefficient (cid:14) ^ x t !

Original: x [ V G ] can be considered as an affinity factor indicating how much the discriminator is ‘attracted’ to the token x .
Revised: x [ V G ] can be considered as an affinity factor indicating how much the discriminator is ‘attracted’ to the token x .

Original: Recall that the gradient @
Revised: Recall that the gradient @

Original: V G @
Revised: V G @

Original: e ( e ^ x ) has the direction of greatest increase of the objective V G at e ^ x .
Revised: e ( e ^ x ) has the direction of greatest increase of the objective V G at e ^ x .

Original: Since this direction is relative to e ^ x , we suggest taking the inner product with the vector e x (cid:0) e ^ x which has been re-centered relative to e ^ x (Figure 2).
Revised: Since this direction is relative to e ^ x , we suggest taking the inner product with the vector e x (cid:0) e ^ x which has been re-centered relative to e ^ x (Figure 2).

Original: [Equation]
Revised: [Equation]

Original: This inner product approximates the increase of V G when ^ x t is replaced by x .
Revised: This inner product approximates the increase of V G when ^ x t is replaced by x .

Original: Because this approximation only holds in the neighborhood of e ^ x t , j e x (cid:0) e ^ x t j ≪ 1 , we also add a kernel to dampen the affinity factor for e x that are far away.
Revised: Because this approximation only holds in the neighborhood of e ^ x t , j e x (cid:0) e ^ x t j ≪ 1 , we also add a kernel to dampen the affinity factor for e x that are far away.

Original: That is,
Revised: That is,

Original: [Equation]
Revised: [Equation]

Original: where
Revised: where

Original: [Equation]
Revised: [Equation]

Original: See Appendix B for more details.
Revised: See Appendix B for more details.

Original: As a result, Equation 9 becomes
Revised: As a result, Equation 9 becomes

Original: [Equation]
Revised: [Equation]

Original: In score function-based approaches, the generator is only given feedback on the tokens it samples, so it requires a large amount of trial-and-error in order to figure out how to write realistic sentences.
Revised: In score function-based approaches, the generator is only given feedback on the tokens it samples, so it requires a large amount of trial-and-error in order to figure out how to write realistic sentences.

Original: To use an analogy, imagine a captain at sea (embedding space) searching for treasure.
Revised: To use an analogy, imagine a captain at sea (embedding space) searching for treasure.

Original: Each iteration of the game, the captain chooses random locations in the space and receives feedback regarding the choices.
Revised: Each iteration of the game, the captain chooses random locations in the space and receives feedback regarding the choices.

Original: Score function-based approaches tell us how good the location is but say nothing about how to improve it.
Revised: Score function-based approaches tell us how good the location is but say nothing about how to improve it.

Original: In contrast, our method, gives the captain a navigational direction to the next best location.
Revised: In contrast, our method, gives the captain a navigational direction to the next best location.

Original: Even if the direction is biased, it should allow the captain to find the treasures with fewer iterations.
Revised: Even if the direction is biased, it should allow the captain to find the treasures with fewer iterations.

Original: [Figure]
Revised: [Figure]

Original: (a) The inner products of A and B with ∇ e V G are both positive even though B leads to a lower V G .
Revised: (a) The inner products of A and B with ∇ e V G are both positive even though B leads to a lower V G .

Original: (b) After applying re-centering in the inner product, the affinity factors (cid:14) RC
Revised: (b) After applying re-centering in the inner product, the affinity factors (cid:14) RC

Original: [Figure]
Revised: [Figure]

Original: B [ V G ] ,
Revised: B [ V G ] ,

Original: [Figure]
Revised: [Figure]

Original: A [ V G ] have opposite signs, and thus better at reflecting the relative suitability of A and B as judged by the discriminator
Revised: A [ V G ] have opposite signs, and thus better at reflecting the relative suitability of A and B as judged by the discriminator

Original:  Evaluation Metrics
Revised:  Evaluation Metrics

Original: We evaluate our model’s ability to generate realistic and diverse texts with BLEU-based metric,
Revised: We evaluate our model’s ability to generate realistic and diverse texts with BLEU-based metric,

Original: Fréchet Embedding Distance and language model based metric.
Revised: Fréchet Embedding Distance and language model based metric.

Original: 4.1 BLEU and Self-BLEU
Revised: 4.1 BLEU and Self-BLEU

Original: BLEU (Papineni et al., 2002) and Self-BLEU (Zhu et al., 2018) capture text quality and diversity respectively.
Revised: BLEU (Papineni et al., 2002) and Self-BLEU (Zhu et al., 2018) capture text quality and diversity respectively.

Original: BLEU, a modified form of n-gram precision, measures local consistency between a set of reference and a set of candidate texts.
Revised: BLEU, a modified form of n-gram precision, measures local consistency between a set of reference and a set of candidate texts.

Original: Self-BLEU is a version of BLEU in which both the reference and candidate texts are drawn from the same corpus.
Revised: Self-BLEU is a version of BLEU in which both the reference and candidate texts are drawn from the same corpus.

Original: A high
Revised: A high

Original: Self-BLEU means that members within the corpus are too highly correlated, indicating a lack of diversity.
Revised: Self-BLEU means that members within the corpus are too highly correlated, indicating a lack of diversity.

Original: 4.2 Fréchet Embedding Distance
Revised: 4.2 Fréchet Embedding Distance

Original: Fréchet Embedding Distance (FED) (de Masson d’Autume et al., 2019) measures sentimental similarity, global consistency and diversity between reference and candidate texts.
Revised: Fréchet Embedding Distance (FED) (de Masson d’Autume et al., 2019) measures sentimental similarity, global consistency and diversity between reference and candidate texts.

Original: Semeniuta et al.
Revised: Semeniuta et al.

Original: (2018) claims that Fréchet Distance (FD) is not sensitive to the choice of embedding model.
Revised: (2018) claims that Fréchet Distance (FD) is not sensitive to the choice of embedding model.

Original: However, we notice significant discrepancies between FD scores calculated using different embedding models.
Revised: However, we notice significant discrepancies between FD scores calculated using different embedding models.

Original: The details are discussed in Appendix G.
Revised: The details are discussed in Appendix G.

Original: 4.3 Language Model Scoring Methods
Revised: 4.3 Language Model Scoring Methods

Original: Following Caccia et al.
Revised: Following Caccia et al.

Original: ; Zhao et al.
Revised: ; Zhao et al.

Original: (2017), we also evaluate the quality and the diversity of generated samples by Language Model Perplexity (LM) and Reverse Language
Revised: (2017), we also evaluate the quality and the diversity of generated samples by Language Model Perplexity (LM) and Reverse Language

Original: Model Perplexity (RLM).
Revised: Model Perplexity (RLM).

Original: Language Model Score (LM) .
Revised: Language Model Score (LM) .

Original: We use a language model trained on real data to estimate the negative log likelihood per word of generated sentences.
Revised: We use a language model trained on real data to estimate the negative log likelihood per word of generated sentences.

Original: Samples which the language model considers improper, such as ungrammatical or nonsense sentences, have high perplexity.
Revised: Samples which the language model considers improper, such as ungrammatical or nonsense sentences, have high perplexity.

Original: Reverse Language Model Score (RLM) .
Revised: Reverse Language Model Score (RLM) .

Original: If we instead train a language model on generated data while evaluating on real data, we can judge the diversity of the generated samples.
Revised: If we instead train a language model on generated data while evaluating on real data, we can judge the diversity of the generated samples.

Original: Low-diversity generated data leads to an overfitted model which generalizes poorly on real data, as indicated by a high perplexity.
Revised: Low-diversity generated data leads to an overfitted model which generalizes poorly on real data, as indicated by a high perplexity.

Original:  Results & Discussion
Revised:  Results & Discussion

Original: In this section, we use two datasets to evaluate CaptainGAN: COCO Image Captions 1 and
Revised: In this section, we use two datasets to evaluate CaptainGAN: COCO Image Captions 1 and

Original: EMNLP 2017 News 2 .
Revised: EMNLP 2017 News 2 .

Original: We discuss our results from different perspectives via the evaluation metrics presented in the last section.
Revised: We discuss our results from different perspectives via the evaluation metrics presented in the last section.

Original: Furthermore, an ablation study is performed in order to know the contribution of each technique.
Revised: Furthermore, an ablation study is performed in order to know the contribution of each technique.

Original: More details about our experiment setup, datasets, architecture and training techniques such as temperature control are described in
Revised: More details about our experiment setup, datasets, architecture and training techniques such as temperature control are described in

Original: Appendix D.
Revised: Appendix D.

Original: [Figure]
Revised: [Figure]

Original: 5.1 Quality and Diversity
Revised: 5.1 Quality and Diversity

Original: CaptainGAN significantly outperforms prior GAN-based models on metrics of local text consistency and diversity.
Revised: CaptainGAN significantly outperforms prior GAN-based models on metrics of local text consistency and diversity.

Original: The BLEU/Self-BLEU temperature sweeps shown in Figure 3a for CaptainGAN indicates that CaptainGAN approaches the performance of a language model.
Revised: The BLEU/Self-BLEU temperature sweeps shown in Figure 3a for CaptainGAN indicates that CaptainGAN approaches the performance of a language model.

Original: On the other hand, in Figure 3b, the language model scores show similar results for
Revised: On the other hand, in Figure 3b, the language model scores show similar results for

Original: CaptainGAN and MLE model.
Revised: CaptainGAN and MLE model.

Original: Results for COCO are shown in Appendix E.
Revised: Results for COCO are shown in Appendix E.

Original: 5.2 Temperature Sensitivity
Revised: 5.2 Temperature Sensitivity

Original: CaptainGAN is robust to temperature variations.
Revised: CaptainGAN is robust to temperature variations.

Original: This is demonstrated in Figure 4a, which depicts how the softmax temperature influences the FED score.
Revised: This is demonstrated in Figure 4a, which depicts how the softmax temperature influences the FED score.

Original: Regardless of model, changes in softmax temperature are certain to affect the FED, however, CaptainGAN shows the least variation.
Revised: Regardless of model, changes in softmax temperature are certain to affect the FED, however, CaptainGAN shows the least variation.

Original: 5.3 Global Consistency
Revised: 5.3 Global Consistency

Original: CaptainGAN improves global consistency comparing to prior works and language model on
Revised: CaptainGAN improves global consistency comparing to prior works and language model on

Original: LM perplexity and FED score, respectively.
Revised: LM perplexity and FED score, respectively.

Original: As shown in Table 1, our method significantly reduces the gap of perplexity between GAN-based method and language model, which is directly trained to minimize the perplexity.
Revised: As shown in Table 1, our method significantly reduces the gap of perplexity between GAN-based method and language model, which is directly trained to minimize the perplexity.

Original: Besides, we show that CaptainGAN outperforms
Revised: Besides, we show that CaptainGAN outperforms

Original: MLE trained model on FED score as shown in Table 1.
Revised: MLE trained model on FED score as shown in Table 1.

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: 5.4 Matching N-grams of Training Data
Revised: 5.4 Matching N-grams of Training Data

Original: Text generating models are often criticized for simply memorizing texts seen during training instead of learning how to model the true data distribution.
Revised: Text generating models are often criticized for simply memorizing texts seen during training instead of learning how to model the true data distribution.

Original: We show the longest matching ngrams between text from the training data and samples in Figure 4b.
Revised: We show the longest matching ngrams between text from the training data and samples in Figure 4b.

Original: CaptainGAN generates more samples with small matching n-grams (n < 5) comparing to MLE model, implying that our model can generate novel sentences without resorting to memorizing phrases from the training data.
Revised: CaptainGAN generates more samples with small matching n-grams (n < 5) comparing to MLE model, implying that our model can generate novel sentences without resorting to memorizing phrases from the training data.

Original: 5.5 CaptainGan As A Language Model
Revised: 5.5 CaptainGan As A Language Model

Original: CaptainGAN’s strong performance on the evaluation metrics described earlier shows that it is capable of generating realistic text.
Revised: CaptainGAN’s strong performance on the evaluation metrics described earlier shows that it is capable of generating realistic text.

Original: Since the generator in our case is a RNN which can produce conditional probabilities at each timestep, this raises the question of whether or not an explicit language model is being learned.
Revised: Since the generator in our case is a RNN which can produce conditional probabilities at each timestep, this raises the question of whether or not an explicit language model is being learned.

Original: We feed real data to our generator and calculate the perplexity to measure the generator’s suitability as a language model.
Revised: We feed real data to our generator and calculate the perplexity to measure the generator’s suitability as a language model.

Original: Surprisingly, as shown in Table 2, the perplexity of the generator on both training and validation data is unusually high.
Revised: Surprisingly, as shown in Table 2, the perplexity of the generator on both training and validation data is unusually high.

Original: A similar phenomenon was noted in de Masson d’Autume et al.
Revised: A similar phenomenon was noted in de Masson d’Autume et al.

Original: (2019), though not as extreme.
Revised: (2019), though not as extreme.

Original: We explain the phenomenon by the following reasons.
Revised: We explain the phenomenon by the following reasons.

Original: Minimizing alternative loss (Equation 2) is similar to minimizing reverse KL divergence (Arjovsky & Bottou, 2017).
Revised: Minimizing alternative loss (Equation 2) is similar to minimizing reverse KL divergence (Arjovsky & Bottou, 2017).

Original: This objective
Revised: This objective

Original: [Table]
Revised: [Table]

Original: assigns an extremely low cost on mode dropping and does not force the generator to mimic every part of real data.
Revised: assigns an extremely low cost on mode dropping and does not force the generator to mimic every part of real data.

Original: Following 5.4, we conclude that our model’s ability to generate realistic text cannot be the result of simply plagiarizing training samples.
Revised: Following 5.4, we conclude that our model’s ability to generate realistic text cannot be the result of simply plagiarizing training samples.

Original: 5.6 Comparison to score function-based methods
Revised: 5.6 Comparison to score function-based methods

Original: Our method is able to outperform score function-based approaches using small batch sizes.
Revised: Our method is able to outperform score function-based approaches using small batch sizes.

Original: Score function-based approaches often rely on large batch sizes for variance reduction (de Masson d’Autume et al., 2019), but this does not appear to be necessary for our recentered estimator.
Revised: Score function-based approaches often rely on large batch sizes for variance reduction (de Masson d’Autume et al., 2019), but this does not appear to be necessary for our recentered estimator.

Original: This confirms the reasoning at the end of Section 3.2, and greatly reduces our memory requirements during training.
Revised: This confirms the reasoning at the end of Section 3.2, and greatly reduces our memory requirements during training.

Original: In addition, since gradients are naturally calculated for every timestep, there is no need for
Revised: In addition, since gradients are naturally calculated for every timestep, there is no need for

Original: Monte-Carlo estimation of rewards like in score function-based approaches.
Revised: Monte-Carlo estimation of rewards like in score function-based approaches.

Original: Because our generator receives a richer learning signal, we use a 5:1 ratio for discriminator-generator updating.
Revised: Because our generator receives a richer learning signal, we use a 5:1 ratio for discriminator-generator updating.

Original: This greatly lessens our computational requirements.
Revised: This greatly lessens our computational requirements.

Original: 5.7 Ablation Study
Revised: 5.7 Ablation Study

Original: To understand the impact of each component of the CaptainGAN, we conduct an ablation study.
Revised: To understand the impact of each component of the CaptainGAN, we conduct an ablation study.

Original: In Table 3, we show the influence of adding new features, namely spectral normalization, re-center estimator, pretrained embeddings and discriminator regularization decoupled weight decay (Loshchilov & Hutter, 2018) and one-sided label smoothing (Salimans et al., 2016), by scoring FED on validation data.
Revised: In Table 3, we show the influence of adding new features, namely spectral normalization, re-center estimator, pretrained embeddings and discriminator regularization decoupled weight decay (Loshchilov & Hutter, 2018) and one-sided label smoothing (Salimans et al., 2016), by scoring FED on validation data.

Original: We observe that:
Revised: We observe that:

Original: • As noted earlier, spectral normalization is critical for our method as without it the model cannot converge.
Revised: • As noted earlier, spectral normalization is critical for our method as without it the model cannot converge.

Original: • Rather unexpectedly, pretrained embeddings do not lead to any significant improvement, which is surprising given our method’s dependence on the discriminator’s embedding space.
Revised: • Rather unexpectedly, pretrained embeddings do not lead to any significant improvement, which is surprising given our method’s dependence on the discriminator’s embedding space.

Original: The embeddings which the discriminator learns from scratch are capable of giving the generator an effective learning signal.
Revised: The embeddings which the discriminator learns from scratch are capable of giving the generator an effective learning signal.

Original: • The application of the re-centered estimator is responsible for a 20% improvement in FED, confirming its effectiveness.
Revised: • The application of the re-centered estimator is responsible for a 20% improvement in FED, confirming its effectiveness.

Original: • Equation 14 is dependent on the norm of the embeddings.
Revised: • Equation 14 is dependent on the norm of the embeddings.

Original: This dependency explains the effectiveness of discriminator weight regularization at improving FED score.
Revised: This dependency explains the effectiveness of discriminator weight regularization at improving FED score.

Original:  Conclusion and Future Work
Revised:  Conclusion and Future Work

Original: In this work we have presented CaptainGAN, an effective gradient-based method for quickly training a text generating GAN without pretraining.
Revised: In this work we have presented CaptainGAN, an effective gradient-based method for quickly training a text generating GAN without pretraining.

Original: Starting from the straight-through estimator, we derive a re-centered gradient estimator that improves the quality/diversity of generated texts as measured by various standard metrics.
Revised: Starting from the straight-through estimator, we derive a re-centered gradient estimator that improves the quality/diversity of generated texts as measured by various standard metrics.

Original: In future work, we plan to investigate our estimator in more theoretical detail and further decrease its bias.
Revised: In future work, we plan to investigate our estimator in more theoretical detail and further decrease its bias.

Original: [Table]
Revised: [Table]

Original: Appendix A Notation Symbol
Revised: Appendix A Notation Symbol

Original: Meaning
Revised: Meaning

Original: V =
Revised: V =

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: a predefined vocabulary x a token belongs to V ^ x a token sampled from V x a sequence of tokens belong to V ^ x a sequence of tokens sampled from V ⊤ a transpose operation
Revised: a predefined vocabulary x a token belongs to V ^ x a token sampled from V x a sequence of tokens belong to V ^ x a sequence of tokens sampled from V ⊤ a transpose operation

Original: Appendix B The Importance of Kernel
Revised: Appendix B The Importance of Kernel

Original: To verify the necessity of the kernel term (Equation 14), we directly apply Equation 12 to
Revised: To verify the necessity of the kernel term (Equation 14), we directly apply Equation 12 to

Original: Equation 11 as follows:
Revised: Equation 11 as follows:

Original: [Equation]
Revised: [Equation]

Original: (16) where ^ x is the selected token and x is a tokens in the vocabulary V.
Revised: (16) where ^ x is the selected token and x is a tokens in the vocabulary V.

Original: Let f ( x ) = ⟨ e x ; @ V G @
Revised: Let f ( x ) = ⟨ e x ; @ V G @

Original: e ( e ^ x ) ⟩ and b = ⟨ e ^ x ; @ V G @
Revised: e ( e ^ x ) ⟩ and b = ⟨ e ^ x ; @ V G @

Original: e ( e ^ x ) ⟩ .
Revised: e ( e ^ x ) ⟩ .

Original: b does not depend on x and is constant in the summation.
Revised: b does not depend on x and is constant in the summation.

Original: ∑ x 2 V ( f ( x ) + b ) @p (cid:18) ( x ) @(cid:18) = @
Revised: ∑ x 2 V ( f ( x ) + b ) @p (cid:18) ( x ) @(cid:18) = @

Original: @(cid:18)
Revised: @(cid:18)

Original: [ ∑ x 2 V ( f ( x ) + b ) p (cid:18) ( x )]
Revised: [ ∑ x 2 V ( f ( x ) + b ) p (cid:18) ( x )]

Original: = @
Revised: = @

Original: @(cid:18)
Revised: @(cid:18)

Original: [( ∑ x 2 V f ( x ) p (cid:18) ( x ))
Revised: [( ∑ x 2 V f ( x ) p (cid:18) ( x ))

Original: + b ( ∑ x 2 V p (cid:18) ( x ))]
Revised: + b ( ∑ x 2 V p (cid:18) ( x ))]

Original: = @
Revised: = @

Original: @(cid:18)
Revised: @(cid:18)

Original: [( ∑ x 2 V f ( x ) p (cid:18) ( x ))
Revised: [( ∑ x 2 V f ( x ) p (cid:18) ( x ))

Original: + b (cid:3) 1] = @
Revised: + b (cid:3) 1] = @

Original: @(cid:18)
Revised: @(cid:18)

Original: ∑ x 2 V
Revised: ∑ x 2 V

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: Without the kernel, the update to (cid:18) will be the same as prior work and thus carries the same drawback.
Revised: Without the kernel, the update to (cid:18) will be the same as prior work and thus carries the same drawback.

Original: Appendix C Model Structure
Revised: Appendix C Model Structure

Original: We use a pre-trained word embedding of 300 dimensions to initialize all embedding weights.
Revised: We use a pre-trained word embedding of 300 dimensions to initialize all embedding weights.

Original: The embeddings are pre-trained using the fastText library Bojanowski et al.
Revised: The embeddings are pre-trained using the fastText library Bojanowski et al.

Original: (2016) on the corresponding training data of each task.
Revised: (2016) on the corresponding training data of each task.

Original: [Table]
Revised: [Table]

Original: C.1 Generator
Revised: C.1 Generator

Original: The purpose of the first dense layer is to project the GRU cell outputs into the embedding space.
Revised: The purpose of the first dense layer is to project the GRU cell outputs into the embedding space.

Original: The second dense layer converts the projected embedding vectors to logits over the tokens and is initialized as the transpose of the embedding weights.
Revised: The second dense layer converts the projected embedding vectors to logits over the tokens and is initialized as the transpose of the embedding weights.

Original: At the first timestep, the input to the embedding layer is the start-of-sentence token.
Revised: At the first timestep, the input to the embedding layer is the start-of-sentence token.

Original: The vocabulary also contains an end-of-sentence token.
Revised: The vocabulary also contains an end-of-sentence token.

Original: If the generator outputs this token at any timestep, the sentence ends and its length is set to this timestep.
Revised: If the generator outputs this token at any timestep, the sentence ends and its length is set to this timestep.

Original: C.2 Discriminator
Revised: C.2 Discriminator

Original: [Table]
Revised: [Table]

Original: To perform consistent convolution on sentences of different lengths, we use a special masking mechanism on all layers.
Revised: To perform consistent convolution on sentences of different lengths, we use a special masking mechanism on all layers.

Original: For all convolution and pooling layers, we mask out input features where the receptive field is out-of-bounds with respect to the unpadded input sentence.
Revised: For all convolution and pooling layers, we mask out input features where the receptive field is out-of-bounds with respect to the unpadded input sentence.

Original: Appendix D Experimental details D.1 Setup D.1.1 Datasets
Revised: Appendix D Experimental details D.1 Setup D.1.1 Datasets

Original: We use two datasets COCO Image Caption dataset Chen et al.
Revised: We use two datasets COCO Image Caption dataset Chen et al.

Original: and EMNLP
Revised: and EMNLP

Original: News dataset 3 .
Revised: News dataset 3 .

Original: Results are reported on EMNLP 2017 News if not specified.
Revised: Results are reported on EMNLP 2017 News if not specified.

Original: For each datasets, 10k sentences are set aside as validation data.
Revised: For each datasets, 10k sentences are set aside as validation data.

Original: COCO Image Captions.
Revised: COCO Image Captions.

Original: Sentences are limited in length to 24 tokens.
Revised: Sentences are limited in length to 24 tokens.

Original: The vocabulary size is 4.6k tokens.
Revised: The vocabulary size is 4.6k tokens.

Original: Training data consists of 10k sentences 4 .
Revised: Training data consists of 10k sentences 4 .

Original: EMNLP 2017 News.
Revised: EMNLP 2017 News.

Original: Sentences in this dataset are much longer and more complicated, with a maximum length of 50 tokens and a vocabulary of 5.7k words.
Revised: Sentences in this dataset are much longer and more complicated, with a maximum length of 50 tokens and a vocabulary of 5.7k words.

Original: Training data consists of 300k sentences.
Revised: Training data consists of 300k sentences.

Original: D.1.2 Architecture and Techniques
Revised: D.1.2 Architecture and Techniques

Original: Our architecture is described in Appendix C. Spectral normalization of the discriminator
Revised: Our architecture is described in Appendix C. Spectral normalization of the discriminator

Original: ’s weights Miyato et al.
Revised: ’s weights Miyato et al.

Original: (2018) is critical to our method as without it the gradients are too unstable and convergence becomes impossible (Section 5.7).
Revised: (2018) is critical to our method as without it the gradients are too unstable and convergence becomes impossible (Section 5.7).

Original: D.1.3 Temperature Control
Revised: D.1.3 Temperature Control

Original: Following Caccia et al.
Revised: Following Caccia et al.

Original: (2018), we adjust the softmax temperature parameter at sampling time to measure the trade-off between quality and diversity.
Revised: (2018), we adjust the softmax temperature parameter at sampling time to measure the trade-off between quality and diversity.

Original: Increasing the temperature lowers the differences between softmax probabilities, which leads to diverse but low-quality samples.
Revised: Increasing the temperature lowers the differences between softmax probabilities, which leads to diverse but low-quality samples.

Original: On the other hand, reducing the temperature leads to high-quality yet low-diversity samples.
Revised: On the other hand, reducing the temperature leads to high-quality yet low-diversity samples.

Original: D.2 training details
Revised: D.2 training details

Original: The discriminator and generator updates are performed with a 5:1 ratio.
Revised: The discriminator and generator updates are performed with a 5:1 ratio.

Original: We use Adam
Revised: We use Adam

Original: Kingma & Ba (2014) with learning rate = 5 : 0 (cid:1) 10 (cid:0) 5 , (cid:12) 1 = 0 : 5 , and (cid:12) 2 = 0 : 999 .
Revised: Kingma & Ba (2014) with learning rate = 5 : 0 (cid:1) 10 (cid:0) 5 , (cid:12) 1 = 0 : 5 , and (cid:12) 2 = 0 : 999 .

Original: Decoupled weight decay regularization Loshchilov & Hutter (2018) is applied to all variables in the discriminator, and (cid:21) = 0 : 05 (cid:1) learning rate.
Revised: Decoupled weight decay regularization Loshchilov & Hutter (2018) is applied to all variables in the discriminator, and (cid:21) = 0 : 05 (cid:1) learning rate.

Original: All our models are trained using an NVIDIA
Revised: All our models are trained using an NVIDIA

Original: GTX 1080 Ti.
Revised: GTX 1080 Ti.

Original: To evaluate samples on BLEU-5, LM versus RLM, and FED, we use 10k validation sentences as reference texts for both COCO and EMNLP 2017 News.
Revised: To evaluate samples on BLEU-5, LM versus RLM, and FED, we use 10k validation sentences as reference texts for both COCO and EMNLP 2017 News.

Original: For Self-BLEU, we randomly select half of the 10k samples as reference texts and leave the remainder as target texts.
Revised: For Self-BLEU, we randomly select half of the 10k samples as reference texts and leave the remainder as target texts.

Original: A smoothing function is used on BLEU-based metrics.
Revised: A smoothing function is used on BLEU-based metrics.

Original: See Appendix D.4 for more details.
Revised: See Appendix D.4 for more details.

Original: The maximum likelihood model architecture in this paper follows the setting of de Masson d’Autume et al.
Revised: The maximum likelihood model architecture in this paper follows the setting of de Masson d’Autume et al.

Original: We could not reproduce the exact results reported in de Masson d’Autume et al.
Revised: We could not reproduce the exact results reported in de Masson d’Autume et al.

Original: (2019), which we believe stems from discrepancies between our trained language model and theirs.
Revised: (2019), which we believe stems from discrepancies between our trained language model and theirs.

Original: D.3 One-sided label smoothing
Revised: D.3 One-sided label smoothing

Original: One-sided label smoothing has been shown to reduce the vulnerability of neural networks to adversarial examples and it is strongly recommended for GAN training Salimans et al.
Revised: One-sided label smoothing has been shown to reduce the vulnerability of neural networks to adversarial examples and it is strongly recommended for GAN training Salimans et al.

Original: Therefore, the first term of Equation 1 is reformulated as
Revised: Therefore, the first term of Equation 1 is reformulated as

Original: [Equation]
Revised: [Equation]

Original: where (cid:11) = 0 : 9 in our experiments.
Revised: where (cid:11) = 0 : 9 in our experiments.

Original: D.4 BLEU smoothing
Revised: D.4 BLEU smoothing

Original: One of the issue with BLEU is that in the case that a higher order n-gram precision of a sentence is 0, then the BLEU score will be 0, resulting in severely underestimation.
Revised: One of the issue with BLEU is that in the case that a higher order n-gram precision of a sentence is 0, then the BLEU score will be 0, resulting in severely underestimation.

Original: This is due to the fact that BLEU is calculated by the geometric mean of precision.
Revised: This is due to the fact that BLEU is calculated by the geometric mean of precision.

Original: To solve this, we applied the smoothing technique as follows:
Revised: To solve this, we applied the smoothing technique as follows:

Original: [Equation]
Revised: [Equation]

Original: 19) where m n is the original match count, m ′ n is the one used during BLEU calculation.
Revised: 19) where m n is the original match count, m ′ n is the one used during BLEU calculation.

Original: We’ve picked the smoothing factor ϵ = 0 : 1 as proposed by Chen & Cherry (2014).
Revised: We’ve picked the smoothing factor ϵ = 0 : 1 as proposed by Chen & Cherry (2014).

Original: Appendix E COCO Results
Revised: Appendix E COCO Results

Original: [Figure]
Revised: [Figure]

Original: See Figure 6.
Revised: See Figure 6.

Original: Appendix F Sentence Length Consistency
Revised: Appendix F Sentence Length Consistency

Original: The sentence length of samples which are generated by CaptainGAN is highly related to the training data, as shown in Figure 7.
Revised: The sentence length of samples which are generated by CaptainGAN is highly related to the training data, as shown in Figure 7.

Original: This suggests that CaptainGAN is able to fully learn the distribution of sentence length by gradient estimator without additional mechanism like positional encoding or dense reward.
Revised: This suggests that CaptainGAN is able to fully learn the distribution of sentence length by gradient estimator without additional mechanism like positional encoding or dense reward.

Original: Appendix G Details on Fréchet Embedding Distance
Revised: Appendix G Details on Fréchet Embedding Distance

Original: Calculation of the FED score depends on the Universal Sentence EncoderCer et al.
Revised: Calculation of the FED score depends on the Universal Sentence EncoderCer et al.

Original: However, the version of Universal Sentence Encoder 6 used in de Masson d’Autume et al.
Revised: However, the version of Universal Sentence Encoder 6 used in de Masson d’Autume et al.

Original: (2019) is incompatible with TensorflowAbadi et al.
Revised: (2019) is incompatible with TensorflowAbadi et al.

Original: (2015) version 1.7 or newer.
Revised: (2015) version 1.7 or newer.

Original: Furthermore, it would be infeasible to limit the experimentation environment to such ancient version of
Revised: Furthermore, it would be infeasible to limit the experimentation environment to such ancient version of

Original: Tensorflow.
Revised: Tensorflow.

Original: Therefore, as a work around, we decide to report our FED score, as shown in Table 6, with Universal Sentence Encoder
Revised: Therefore, as a work around, we decide to report our FED score, as shown in Table 6, with Universal Sentence Encoder

Original: Large 7 instead, which is compatible with the current version of Tensorflow (1.14 as of Jul 2019).
Revised: Large 7 instead, which is compatible with the current version of Tensorflow (1.14 as of Jul 2019).

Original: While there are many benefits to using the FED metric, it is not without drawback.
Revised: While there are many benefits to using the FED metric, it is not without drawback.

Original: A drawback that we’ve observed is that the FED score will be significantly underestimated
Revised: A drawback that we’ve observed is that the FED score will be significantly underestimated

Original: [Table]
Revised: [Table]

Original: whenever we are using a small number of samples, as shown in Figure 8.
Revised: whenever we are using a small number of samples, as shown in Figure 8.

Original: Therefore, we’ve ensured that sufficient samples are used in our evaluation.
Revised: Therefore, we’ve ensured that sufficient samples are used in our evaluation.

Original: Appendix H Generated samples
Revised: Appendix H Generated samples

Original: Training samples of COCO and EMNLP 2017 News can be found in Table 7.
Revised: Training samples of COCO and EMNLP 2017 News can be found in Table 7.

Original: Randomly picked samples from MLE model and CaptainGAN are available in Table 8 and Table
Revised: Randomly picked samples from MLE model and CaptainGAN are available in Table 8 and Table

Original: [Table]
Revised: [Table]

Original: COCO
Revised: COCO

Original: - a person standing in a boat on the water .
Revised: - a person standing in a boat on the water .

Original: - three sheep are grazing on the city sidewalk .
Revised: - three sheep are grazing on the city sidewalk .

Original: - kenya airways airplane wing , engine and cabin on the tarmac . - light poles in the snow with yellow traffic lights mounted on them .
Revised: - kenya airways airplane wing , engine and cabin on the tarmac . - light poles in the snow with yellow traffic lights mounted on them .

Original: - we are looking at the floor between the toilet and the wall . -
Revised: - we are looking at the floor between the toilet and the wall . -

Original: someone taking a photo of a small residential bathroom .
Revised: someone taking a photo of a small residential bathroom .

Original: - a cat sits on the seat of a bicycle and looks down at another cat on a snowy day .
Revised: - a cat sits on the seat of a bicycle and looks down at another cat on a snowy day .

Original: - large set of motorcycles all lined up down a street .
Revised: - large set of motorcycles all lined up down a street .

Original: EMNLP 2017 News
Revised: EMNLP 2017 News

Original: - apple clearly doesn ’ t want to see the dollar value of its u .
Revised: - apple clearly doesn ’ t want to see the dollar value of its u .

Original: k .
Revised: k .

Original: earnings fall by a similar amount .
Revised: earnings fall by a similar amount .

Original: - we do look at the number of hours we produce , and measure that against the religious make - up of society . - the u .
Revised: - we do look at the number of hours we produce , and measure that against the religious make - up of society . - the u .

Original: s . has struggled to find an effective ground force to take on isis in syria , where president obama has ruled out a u .
Revised: s . has struggled to find an effective ground force to take on isis in syria , where president obama has ruled out a u .

Original: s .
Revised: s .

Original: ground combat role .
Revised: ground combat role .

Original: - it is a happy and kind world that we live in on this show and that is where i hope we can live in real life . - while men and women may end up earning roughly the same amount in the same jobs , men are more likely to end up in higher - paying roles in the tech industry . -
Revised: - it is a happy and kind world that we live in on this show and that is where i hope we can live in real life . - while men and women may end up earning roughly the same amount in the same jobs , men are more likely to end up in higher - paying roles in the tech industry . -

Original: but
Revised: but

Original: while
Revised: while

Original: they were beaten by a better side , the tie did reveal what i think has been city ’
Revised: they were beaten by a better side , the tie did reveal what i think has been city ’

Original: s biggest
Revised: s biggest

Original: problem
Revised: problem

Original: this season :
Revised: this season :

Original: they have
Revised: they have

Original: lost
Revised: lost

Original: the ability to
Revised: the ability to

Original: score
Revised: score

Original: against good
Revised: against good

Original: teams
Revised: teams

Original: . - according to facebook ’ s policies , accounts can be suspended if law enforcement believe individuals are at risk of harm . -
Revised: . - according to facebook ’ s policies , accounts can be suspended if law enforcement believe individuals are at risk of harm . -

Original: she has to take personal responsibility for this -
Revised: she has to take personal responsibility for this -

Original: when she was health
Revised: when she was health

Original: secretary
Revised: secretary

Original: she
Revised: she

Original: was told by those
Revised: was told by those

Original: who
Revised: who

Original: know best that her
Revised: know best that her

Original: decision
Revised: decision

Original: to cut student places would have a damaging impact
Revised: to cut student places would have a damaging impact

Original: COCO
Revised: COCO

Original: - the bathroom bowl with focus on the vanity
Revised: - the bathroom bowl with focus on the vanity

Original: - a man racer in a car brushing a carved cake . -
Revised: - a man racer in a car brushing a carved cake . -

Original: a lady riding a motorcycle and a street while sitting on a pedestal track .
Revised: a lady riding a motorcycle and a street while sitting on a pedestal track .

Original: a
Revised: a

Original: commercial man
Revised: commercial man

Original: are standing
Revised: are standing

Original: in the bicycle next to the back of an orange computer an
Revised: in the bicycle next to the back of an orange computer an

Original: airplane
Revised: airplane

Original: the
Revised: the

Original: bus
Revised: bus

Original: is headed as something
Revised: is headed as something

Original: in
Revised: in

Original: the sky
Revised: the sky

Original: . - people riding on a bike on a surfer surfing doing the ocean .
Revised: . - people riding on a bike on a surfer surfing doing the ocean .

Original: - a plane propped up in front of a vehicle .
Revised: - a plane propped up in front of a vehicle .

Original: - a man wearing a cat in front of a motorcycle
Revised: - a man wearing a cat in front of a motorcycle

Original: EMNLP 2017 News
Revised: EMNLP 2017 News

Original: - a one - game event and a limited number this dropped by 38 per cent to find out what we could want , to be as much done as we finish as the games .
Revised: - a one - game event and a limited number this dropped by 38 per cent to find out what we could want , to be as much done as we finish as the games .

Original: - more than 30 per cent of calais camps at risk were taken between children and child refugees crossing the mediterranean to fall apart .
Revised: - more than 30 per cent of calais camps at risk were taken between children and child refugees crossing the mediterranean to fall apart .

Original: - the uk government got on the deal when it came to other eu countries that allowed workplace checks for 26 .
Revised: - the uk government got on the deal when it came to other eu countries that allowed workplace checks for 26 .

Original: - so women i didn ’ t want my parents to work hard with me at the time of the task of making it presents .
Revised: - so women i didn ’ t want my parents to work hard with me at the time of the task of making it presents .

Original: - black voters feel there may not be enough choice between former president george w .
Revised: - black voters feel there may not be enough choice between former president george w .

Original: bush and i have to go along with problems and support the people that are in a swing state for president .
Revised: bush and i have to go along with problems and support the people that are in a swing state for president .

Original: - he sold out to its eighth list , but a stunning showing that what cost the ability to start the league failed ? - let ’ s catch him down his back phone one morning john f kennedy , where he is in a us meeting group with supporters of love not being successful in the fields .
Revised: - he sold out to its eighth list , but a stunning showing that what cost the ability to start the league failed ? - let ’ s catch him down his back phone one morning john f kennedy , where he is in a us meeting group with supporters of love not being successful in the fields .

Original: - then there ’ s a couple of chances that they won competition - - the result from what is of me going out there that i really will play for the foundation .
Revised: - then there ’ s a couple of chances that they won competition - - the result from what is of me going out there that i really will play for the foundation .

Original: COCO
Revised: COCO

Original: - a black motorcycle parked next to a river .
Revised: - a black motorcycle parked next to a river .

Original: - a bathroom with a mirror , a shower and a toilet .
Revised: - a bathroom with a mirror , a shower and a toilet .

Original: - two people sit on a building in front of a building .
Revised: - two people sit on a building in front of a building .

Original: - a group of people walking around the corner of a building .
Revised: - a group of people walking around the corner of a building .

Original: - people are riding bikes on a city street .
Revised: - people are riding bikes on a city street .

Original: - there is a picture of a bathroom in a bathroom . -
Revised: - there is a picture of a bathroom in a bathroom . -

Original: an empty city street with cars parked on the side of the
Revised: an empty city street with cars parked on the side of the

Original: road .
Revised: road .

Original: - a man taking a picture of a white bathroom .
Revised: - a man taking a picture of a white bathroom .

Original: EMNLP 2017 News
Revised: EMNLP 2017 News

Original: - if i go to the party , it ’ s a disaster , and i ’ m going to keep it right .
Revised: - if i go to the party , it ’ s a disaster , and i ’ m going to keep it right .

Original: - i think i ’ ve had some great strength in my life and i want that to be here for me .
Revised: - i think i ’ ve had some great strength in my life and i want that to be here for me .

Original: - obama pointed to reporters claiming he was willing to discuss the situation , but the u .
Revised: - obama pointed to reporters claiming he was willing to discuss the situation , but the u .

Original: s .
Revised: s .

Original: have been military concern by american troops , including a nuclear deal in march .
Revised: have been military concern by american troops , including a nuclear deal in march .

Original: - a 16 - year - old man was arrested after the death of a man shot and arrested him on suspicion on his behalf .
Revised: - a 16 - year - old man was arrested after the death of a man shot and arrested him on suspicion on his behalf .

Original: - the problem is that they are hard to find out what they want to see where they are and what they can achieve their past two years .
Revised: - the problem is that they are hard to find out what they want to see where they are and what they can achieve their past two years .

Original: - now it ’ s important to me and i am thinking so hard to make it the best for a long time .
Revised: - now it ’ s important to me and i am thinking so hard to make it the best for a long time .

Original: - great britain is suggesting the u .
Revised: - great britain is suggesting the u .

Original: s . - led coalition figures at the australian foreign investor have raised more than $ 250 , 000 in the construction of operations .
Revised: s . - led coalition figures at the australian foreign investor have raised more than $ 250 , 000 in the construction of operations .

Original: - it is a simple way to believe that if it can happen , then i ’ m not the right choice .
Revised: - it is a simple way to believe that if it can happen , then i ’ m not the right choice .