Title: DELIBGAN: COARSE-TO-FINE TEXT GENERATION VIA ADVERSARIAL NETWORK

Abstract: In this paper, we propose a novel adversarial learning framework, namely Delib-GAN, for generating high-quality sentences without supervision. Our framework consists of a coarse-to-fine generator, which contains a first-pass decoder and a second-pass decoder, and a multiple instance discriminator. And we propose two training mechanisms DelibGAN-I and DelibGAN-II. The discriminator is used to fine-tune the second-pass decoder in DelibGAN-I and further evaluate the importance of each word and tune the first-pass decoder in DelibGAN-II. We compare our models with several typical and state-of-the-art unsupervised generic text generation models on three datasets (a synthetic dataset, a descriptive text dataset and a sentimental text dataset). Both qualitative and quantitative experimental results show that our models produce more realistic samples, and DelibGAN-II performs best.

1. INTRODUCTION:

Building a good text generative model has always been a fundamental problem in the natural language processing field. The most common generative model is Recurrent Neural Networks (RNN) (Mikolov et al., 2011) , which predicts each word of a sentence conditioned on the previous word and an evolving hidden state. However, it suffers from two main drawbacks: First, RNN based models are always trained through the maximum likelihood approach, which suffers from exposure bias (Bengio et al., 2015) ; Second, the loss function used to train the model is at word level but the performance is typically evaluated at sentence level (Wang & Wan, 2018a) . Some recent studies have tried to solve these problems. Some approaches work indirectly by making the hidden state dynamics predictable (Professor Forcing (Goyal et al., 2016) ) or by randomly adjusting the sampling words during training time (Scheduled Sampling (Bengio et al., 2015) ). But they do not directly specify the cost function on the RNN output to encourage high sample quality. Later works used Generative Adversarial Networks (GAN) (Goodfellow et al., 2014) or reinforcement learning to directly affect the decoding sequence, such as Gumbel-softmax distribution (Kusner & Hernández-Lobato, 2016) , SeqGAN (Yu et al., 2017) , MaskGAN (Fedus et al., 2018) , etc. However, they all adopt an one-pass forward decoding , which can only see words that have been decoded. Recently, coarse-to-fine generator with multiple decoders has achieved great success on some tasks. But they either do not define the meaning of the sketch (Xia et al., 2017) , or are limited to specific tasks such as logical form parsing, code generation, SQL query generation, etc (Lapata & Dong, 2018) . And they all are supervised learning methods.

Inspired by Deliberation Networks (Xia et al., 2017) , we propose a novel adversarial learning framework, namely DelibGAN, for generating high-quality sentences without supervision. Our framework consists of a coarse-to-fine generator and a multiple instance discriminator, and the coarseto-fine generator contains a first-pass decoder and a second-pass decoder. Further, we propose two training mechanisms, named DelibGAN-I and DelibGAN-II. The former one uses the sentencelevel penalty predicted by the discriminator to fine-tune the second-pass decoder. And the latter one imposes the influence of word importance on the first-pass decoder, with the purpose of decoding important words as much as possible in the first pass, and thus better guiding the decoding of the second-pass decoder. The word importance is obtained through the multiple instance learning mechanism of the discriminator, which is used to directly tune the first-pass decoder and later guide the second-pass decoding. In short, for DelibGAN-I, the discriminator is used to fine-tune the second-pass decoder G 2 , while for DelibGAN-II, the discriminator is further used to evaluate the importance of each word and tune G 1 .

Our motivation is two-fold: (1) We use a coarse-to-fine generator to generate important words (i.e., sketch) as much as possible during the first decoding and utilize the global information (i.e., the sentence decoded in the first pass) at the time of final decoding; (2) The discriminator can evaluate the sentence-level quality to guide the final generation via policy gradients, moreover, it can provide word-level importance, which can be used to adjust the sketches in the first-pass generation to make sense.

We compare our models with several typical and state-of-the-art unsupervised generic text generation models, including RNNLM (Mikolov et al., 2011) , SeqGAN (Yu et al., 2017) , SentiGAN (Wang & Wan, 2018a) , MaskGAN (Fedus et al., 2018) . Experimental results on three datasets (a synthetic dataset, a descriptive text dataset and a sentimental text dataset) show that our models can produce more realistic samples. The major contributions of this paper are summarized as follows:

• We propose a novel adversarial learning framework -DelibGAN, which consists of a coarse-to-fine generator and a multiple instance discriminator, to generate high-quality sentences without supervision. 

2. FRAMEWORK:



2.1. ARCHITECTURE:

The architecture of our proposed DelibGAN is show in Figure1. The entire model consists of two modules: a coarse-to-fine generator and a multiple instance discriminator. The coarse-to-fine generator has a first-pass decoder G 1 and a second-pass decoder G 2 . The multiple instance discriminator D can evaluate both a whole sentence and each individual word in the sentence by using the multiple instance learning mechanism, and thus the two decoders can be influenced by the discriminator D in different ways. In our first training mechanism DelibGAN-I, the sentence-level penalty predicted by the discriminator is used to fine-tune the second-pass decoder G 2 . In our second training mechanism DelibGAN-II, in addition to the fine-tuning of G 2 , the word-level importance is predicted and used to tune G 1 , to make G 1 decode important words as much as possible, and thus better guide the decoding of G 2 . 

2.2. COARSE-TO-FINE GENERATOR:

Inspired by Deliberation Networks (Xia et al., 2017) , we propose a coarse-to-fine generator with the following two purposes: The first is to enable the generator to utilize the global information (i.e., the sentence decoded in the first pass) at the time of final decoding, rather than only the sequence that has been decoded; The second is to force the generator to generate important words (i.e., sketch) as much as possible during the first-pass decoding, so as to provide more useful clues for decoding in the second pass.

Our coarse-to-fine generator contains a first-pass decoder G 1 and a second-pass decoder G 2 . Both decoders are RNN and can be easily replaced with its variants such as LSTM and GRU (In this work, we use LSTM). The prior input noise z sampled from a distribution P z (e.g., a normal distribution) is used to initialize the input of G 1 , then G 1 will generate a first-pass sequence

x = { x1 , x2 • • • xl }, and a series of hidden states Ĥ = { ĥ1 , ĥ2 • • • ĥl },

where l is the length of the generated sequence. At step t, ĥt is calculated as ĥt = RNN(f (x t-1 ), ĥt-1 ), where f is a word embedding representation function. Then ĥt is fed into a softmax layer, and xt is sampled out from the obtained multinomial distribution. Finally, the first-pass output sequence x is generated by G 1 .

In the second-pass decoding, we take the hidden state of the last moment of G 1 as the initial hidden state of G 2 . And we use an attention model in G 2 , Specifically, at step t, the attention model in G 2 first generates a context c t defined as follows:

EQUATION

After receiving c t , we calculate the hidden state h t as h t = RNN([f (x t-1 ); c t ], h t-1 ). Then x t is sampled in the same way of sampling xt , and the second-pass output sequence x is generated and its length is m.

2.3. MULTIPLE INSTANCE DISCRIMINATOR:

Like Yu et al. (2017) and Wang & Wan (2018a) , in order to evaluate the performance of a generator at the sentence-level, we use a discriminator to calculate rewards. Further, we want to get the importance of each word in the sentence to adjust the generator to decode the important words as much as possible in the first-pass decoding. Thus we adopt a multi-instance learning (MIL) mechanism in our discriminator. MIL deals with problems where labels are associated with groups of instances or bags (sentences in our case), while instance labels (word-level importance) are unobserved (Keeler & Rumelhart, 1991; Zhang et al., 2002; Wang & Wan, 2018b) .

Our multiple instance discriminator D uses a structure similar to a bidirectional RNN. Supposing the input to D is a sentence w = {w 1 , w 2 , • • • , w n }, where n is the length of w. We apply a bidirectional LSTM on the sentence, and for each word w t (t ∈ n), we concatenate its corresponding forward and backward hidden state vectors:

EQUATION

where ----→ LSTM(w t ) and ← ----LSTM(w t ) represent the forward and backward LSTM hidden state vectors for w t . Then we use a softmax classifier to get the word-level probability distribution over target labels (i.e., real and f ake).

p t = softmax(tanh(W d s t + b d ), where p t = [p (t) real , p (t) f ake ]; p (t) real + p (t) f ake = 1.

(3)

p (t)

real and p

(t)

f ake represent the probability that the t-th word is predicted to be real and fake, respectively; W d and b d are parameters, shared across all words. Finally, we use the average of the word-level distribution as the sentence-level distribution over the target labels:

EQUATION

2.4. TRAINING:

Here we introduce the adversarial training of the generator and the discriminator in detail. In this paper, we formalize the text generation problem as a sequential decision making process (Bachman & Precup, 2015) , which solves the problem that the gradient cannot pass back to the generative model when the output is discrete. Moreover, we use policy gradients to update parameters of the generator.

We propose two loss functions for the generator, named DelibGAN-I and DelibGAN-II. DelibGAN-I just forces the first-pass decoder G 1 to maintain the same decoding result, while DelibGAN-II utilizes the result of the discriminator at the word level, forcing G 1 to decode important words (evaluated by the discriminator) as much as possible. But for the second-pass decoder G 2 , both of them use the penalty-based objective function (Wang & Wan, 2018a) to update the parameters θ G2 of G 2 . Specifically, at each timestep t, the existing sequence generated by G 2 is x 1:t-1 = {x 1 , ..., x t-1 }, and the next token x t selected in the next step is an action sampling from the policy G 2 (x t |x 1:t-1 ; θ G2 ), and we use the Monte Carlo rollouts method (Dai et al., 2017; Yu et al., 2017) to simulate intermediate penalty

R θ G 2 ,θ D (x t )

, where θ D is the parameters of the discriminator D.

Then, the total penalty of G 2 can be computed by:

E x∼G2 (x) = m t=1 [(R θ G 2 ,θ D (x t ) -b)G 2 (x t |x 1:t-1 ; θ G2 )] = m t=1 [( 1 q ( q j=1

(1 -D(x t |x (j)

EQUATION

where b is the bias, x

1:t-1 is the q-time Monte Carlo search sampled based on the roll-out policy G 2 (x t |x 1:t-1 ; θ G2 ), and D(x t |x (j) 1:t-1 ; θ D ) is the sentence probability p real given by the discriminator for sentence x (j) 1:t-1 ||x t . DelibGAN-I: For DelibGAN-I, we just force the first-pass decoder G 1 to maintain the same decoding result. Specifically, after we use (G 1 , G 2 ) to sample a pair of sentences ( x, x), we first use the discriminator to calculate the sentence-level penalties. Then in the process of using penalties to update the generator, we force the first-pass decoder G 1 to decode the same result, which is x. The loss of generator G 1 (the parameters are denoted as θ G1 ) is defined as follows:

EQUATION

we propose it with the purpose of keeping the results of G 1 as constant as possible, and just finetuning G 2 based on the sentence-level penalty mentioned above. Finally, the objective of the generator is to minimize:

J (I) G1,G2 (θ G1 , θ G2 ) = E (I) x∼G1 ( x) + E x∼G2 (x) = 1 l l t=1 -log(G 1 (x t | x1:t-1 ; θ G1 )) + m t=1 [( 1 q ( q j=1

(1 -D(x t |x (j)

EQUATION

DelibGAN-II: For DelibGAN-II, we want to impose direct influence to G 1 , forcing it to decode important words as much as possible in the first-pass decoding, to better guide the decoding of G 2 . We tried different ways of influencing, including using sentence-level penalties like Eq5, but finally we introduced and leveraged a multi-instance learning mechanism in the discriminator. This is because it can provide the importance of words in a sentence, and we add the word-level importance to the loss as weights. So the loss of generator G 1 is changed as follows:

EQUATION

where p

(t):

real is the probability that the discriminator predicts xt as real. Therefore, the objective of the generator is to minimize the total loss:

J (II) G1,G2 (θ G1 , θ G2 ) = E (II) x∼G1 ( x) + E x∼G2 (x) = 1 l l t=1 -p real (x) log(G 1 (x t | x1:t-1 ; θ G1 )) + m t=1 [( 1 q ( q j=1

(1 -D(x t |x (j)

EQUATION

For discriminator D, the goal is to distinguish between real text and generated text as much as possible. It is worth noting that we only have the target label for the sentence, but do not have labels on words. The objective function of the discriminator is to minimize:

EQUATION

)

where S is real sentences in the corpus. We perform the adversarial training of the generator and the discriminator, and train them alternately, as shown in Algorithm 1.

Algorithm 1 The adversarial training process in DelibGAN:

Input: Input noise, z; Generator, (G1 , G2); Discriminator, D; Real text dataset, S; Output: Well trained generator, (G1 , G2); 1: Initialize (G1 , G2),D with random weights; 2: Pre-train (G1 , G2) using MLE on S; 3: repeat 4: for d-steps do 5:

Generate fake texts ({ x}, {x}) using (G1, G2); 6:

Update D using ({x}, S) by minimizing Eq 10; 7:

end for 8:

for g-steps do 9:

Generate fake texts ({ x}, {x}) using (G1, G2); 10:

Calculate penalty Ex∼G 2 (x) by Eq (5) ; 11:

For DelibGAN-I, Update (G1, G2) by minimizing Eq (7); 12:

For DelibGAN-II, Update (G1, G2) by minimizing Eq (9); 13:

end for 14: until Model converges 15: return ;

3. EXPERIMENTS:



3.1. SETUP:

In this study, we hope that our unsupervised models can generate sentences with higher quality. Without loss of generality, we evaluate our model1 on three benchmark datasets: a synthetic dataset, a descriptive text dataset and a sentimental text dataset. We compare with several typical and stateof-the-art unsupervised generic text generation models, including RNNLM (Mikolov et al., 2011) , SeqGAN (Yu et al., 2017) , SentiGAN (Wang & Wan, 2018a) , MaskGAN (Fedus et al., 2018) . It is worth noting that pre-training was used for all selected baselines.

After training models, we let them generate 1k sentences each. Measuring the quality of generated sentences has always been a difficult problem. For synthetic data, we have an oracle model to measure the negative log-likelihood (NLL) scores. But for real texts, we use automatic quantitative indicators (e.g., Fluency, Novelty and Diversity) and human evaluation methods (e.g., Grammaticality, Topicality and Overall) to evaluate the quality of generated sentences.

For automatic quantitative indicators, same as Wang & Wan (2018a) , we use a language modeling toolkit -SRILM (Stolcke, 2002) to test the fluency of generated sentences, which calculates the perplexity of generated sentences using the language model trained on respective corpus. In addition, given the generated sentence set A and its training corpus set B, the novelty N ovelty(A|B) and the diversity Diversity(A) are defined as follows:

EQUATION

EQUATION

ϕ is the Jaccard similarity function. It is worth noting that in terms of the fluency indicator, a smaller value is better, but the novelty and diversity indicators are just the opposite.

For human evaluation, we randomly extracted 100 sentences from the generated sentences, and then let five experts rate each sentence according to its "grammaticality", "topicality" and "overall" aspects, where "topicality" indicates whether the sentence accords with the topic/genre of the dataset (i.e., happy moment and sentimental text for the two real datasets, respectively). Scores range from 1 to 5, with 5 being the best. We finally calculate the average of the scores.

3.2. SIMULATION ON SYNTHETIC DATA:

Here we use the synthetic dataset2 used by Yu et al. (2017) , which consists of a set of sequential tokens which can be seen as the simulated data comparing to the real-word language data. Moreover, it automatically evaluates the negative log-likelihood (NLL) scores of our generated sequences, which brings us convenience. We compare our model with various models on this dataset, as shown in Table 1 and Figure 2 . It is worth noting that the DelibMLE method is just using our coarse-to-fine generator. From the results, we can see that: (1) Our coarse-to-fine generator (DelibMLE) performs better and even outperforms seqGAN. (2) Our proposed methods DelibGAN-I and DelibGAN-II outperform all other competitors with a large margin which means the framework we proposed is better than the other models in capturing the dependency of the sequential tokens.

(3) DelibGAN-II is better than DelibGAN-I, which indicates that it is useful to use the word-level information given by the discriminator during the first-pass decoding.

3.3. RESULTS ON DESCRIPTIVE TEXT DATA:

In this section, we validate our model on the descriptive text corpus: HappyDB (Asai et al., 2018) .

HappyDB is a corpus of 100,000+ crowd-sourced happy moments. Its content is mainly about events that make people happy in the past, and 83% of corpora contain only one sentence. We only use those instances that contain one sentence, and we get a total of 83711 happy moments.

Table 2 shows the results on the automatic quantitative indicators, and we can see that DelibGAN-I and DelibGAN-II perform exceptionally well, with the ability of keeping the fluency of sentences, generating sentences different from that in the training corpus and generating a variety of sentences.

Further, we use human evaluation to evaluate generated sentences. Table 3 shows the results, and we can see DelibGAN-I and DelibGAN-II outperform other models, especially in topicality.

3.4. RESULTS ON SENTIMENTAL TEXT DATA:

Here we validate our model on the sentimental text corpus: Stanford Sentiment Treebank (SSTB, Socher et al. (2013) ), which contains 9613 sentimental reviews of movies. Tables 4 and 5 show the results of automatic quantitative indicators and human evaluation, respectively. Both results show that our models DelibGAN-I and DelibGAN-II produce more realistic samples of sentimental texts than other models, and DelibGAN-II performs best.

3.4.1. CASE STUDY:

In Table 6 , we show example sentences generated by DelibMLE, DelibGAN-I and DelibGAN-II.

From the examples, we can see that: (1) In DelibMLE, there is no significant correlation between the first-pass decoding result G 1 and the second-pass decoding result G 2 . (2) Compared with DelibGAN-I, DelibGAN-II is more likely to decode important words in G 1 , and those words behave like keywords in HappyDB and sentiment words in SSTB. 

4. RELATED WORK:

Unsupervised text generation models are usually based on RNN (Mikolov et al., 2011) , which easily suffers from exposure bias (Bengio et al., 2015) and the inconsistency between word-level loss function and sentence-level evaluation (Wang & Wan, 2018a) . Several strategies have been exploited later, including Professor Forcing (Goyal et al., 2016 ), Scheduled SamplingBengio et al. (2015) and MaskMLE (Fedus et al., 2018) .

Generative Adversarial Networks (GAN) (Goodfellow et al., 2014) has also been applied for text generation. Due to the non-differentiable problem and more complex modes of languages, the quality of generated texts are usually not very satisfactory. Some studies apply GAN in text generation by modifying the word prediction function, such as Gumbel-softmax distribution (Kusner & Hernández-Lobato, 2016) . Some other works apply reinforcement learning to solve the non-differentiable problem, including SeqGAN (Yu et al., 2017) , LeakGAN (Guo et al., 2018) , RankGAN (Lin et al., 2017) , SentiGAN (Wang & Wan, 2018a) and MaskGAN (Fedus et al., 2018) . However, they all adopt an one-pass forward decoding , which can only see words that have been decoded.

Recently, Deliberation networks (Xia et al., 2017) , which has two decoders and produces better sequences by modifying sketches, has achieved good results on machine translation tasks. Later, Lapata & Dong (2018) also propose a similar structure and make meaningful settings for the sketches. But this is limited to specific tasks such as logical form parsing, code generation and SQL query generation. Moreover, these methods are supervised learning methods.

Multiple Instance Learning (MIL) (Keeler & Rumelhart, 1991) deals with problems where labels are associated with groups of instances or bags (sentences in our case), while instance labels (word-level importance in our case) are unobserved. The initial MIL makes a strong assumption that a bag is negative only if all of its instances are negative, and positive otherwise (Dietterich et al., 1997; Maron & Ratan, 1998; Zhang et al., 2002) , and subsequent works relax this assumption and make it more suitable for the task at hand (Cour et al., 2011; Kotzias et al., 2015; Wang & Wan, 2018b) .

5. CONCLUSIONS AND FUTURE WORK:

In this paper, we propose a novel adversarial learning framework DelibGAN to generate high-quality sentences without supervision, by making use of a coarse-to-fine generator and a multiple instance discriminator. Evaluation results on various datasets verifies the efficacy of our proposed models.

In future work, we will try to make the sketch more meaningful and apply our framework to more tasks.

:

We will release the source codes of our methods upon the acceptance of this paper.

The synthetic data and the oracle model (LSTM model) are publicly available at https://github.com/LantaoYu/SeqGAN

Figures and Tables:

Figure 1: The architecture of DelibGAN. Two training mechanisms DelibGAN-I and DelibGAN-II are detailed in section 2.4.

Figure 2: The illustration of learning curves. Dotted line is the end of pre-training.

We propose two training mechanisms, DelibGAN-I and DelibGAN-II, to make the generator generate higher-quality sentences.• Both qualitative and quantitative results show that DelibGAN-I and DelibGAN-II can produce more realistic samples, and DelibGAN-II performs best.

Table content: <table/>

The performance comparison on the synthetic data in terms of the NLL scores.

Table content: <table><tr><td colspan="7">Methods MLE SeqGAN SentiGAN DelibMLE DelibGAN-I DelibGAN-II</td></tr><tr><td>NLL</td><td>9.038</td><td>8.736</td><td>6.924</td><td>8.604</td><td>5.594</td><td>5.250</td></tr></table>

Comparison of automatic quantitative indicators of generated sentences on HappyDB corpus. ↓ means that the smaller the better, and ↑ is the opposite. Methods

Table content: <table><tr><td/><td colspan="6">SeqGAN SentiGAN MaskGAN DelibMLE DelibGAN-I DelibGAN-II</td></tr><tr><td colspan="3">Fluency(↓) 156.212 130.655</td><td>121.273</td><td>138.353</td><td>105.593</td><td>98.348</td></tr><tr><td>Novelty(↑)</td><td>0.275</td><td>0.324</td><td>0.267</td><td>0.305</td><td>0.359</td><td>0.399</td></tr><tr><td>Diversity(↑)</td><td>0.694</td><td>0.732</td><td>0.753</td><td>0.723</td><td>0.765</td><td>0.767</td></tr></table>

Comparison of human evaluation of generated sentences on HappyDB corpus. Methods

Table content: <table><tr><td/><td colspan="6">SeqGAN SentiGAN MaskGAN DelibMLE DelibGAN-I DelibGAN-II</td></tr><tr><td>Grammaticality</td><td>2.256</td><td>2.694</td><td>3.548</td><td>2.438</td><td>3.864</td><td>3.871</td></tr><tr><td>Topicality</td><td>2.331</td><td>2.652</td><td>3.361</td><td>3.237</td><td>3.763</td><td>4.026</td></tr><tr><td>Overall</td><td>2.384</td><td>2.698</td><td>3.525</td><td>2.752</td><td>3.644</td><td>3.836</td></tr></table>

Comparison of automatic quantitative indicators of generated sentences on SSTB corpus. ↓ means that the smaller the better, and ↑ is the opposite. Methods

Table content: <table><tr><td/><td colspan="6">SeqGAN SentiGAN MaskGAN DelibMLE DelibGAN-I DelibGAN-II</td></tr><tr><td>Fluency(↓)</td><td>98.253</td><td>82.554</td><td>83.967</td><td>89.365</td><td>75.573</td><td>71.365</td></tr><tr><td>Novelty(↑)</td><td>0.298</td><td>0.344</td><td>0.342</td><td>0.348</td><td>0.387</td><td>0.393</td></tr><tr><td>Diversity(↑)</td><td>0.641</td><td>0.711</td><td>0.723</td><td>0.673</td><td>0.734</td><td>0.740</td></tr></table>

Comparison of human evaluation of generated sentences on SSTB corpus. Methods

Table content: <table><tr><td/><td colspan="6">SeqGAN SentiGAN MaskGAN DelibMLE DelibGAN-I DelibGAN-II</td></tr><tr><td>Grammaticality</td><td>2.534</td><td>3.712</td><td>3.826</td><td>3.529</td><td>3.684</td><td>4.037</td></tr><tr><td>Topicality</td><td>2.164</td><td>3.231</td><td>3.567</td><td>3.471</td><td>3.973</td><td>4.304</td></tr><tr><td>Overall</td><td>2.865</td><td>3.284</td><td>3.483</td><td>3.197</td><td>4.082</td><td>4.243</td></tr></table>

Examples sentences generated by our models trained on HappyDB and SSTB.DelibGAN-IIG 1 i friends to to visit a games . weekend G 2 our friends came over to play video game last weekend .DelibGAN-IIG 1 it funny . <UNK> us . G 2 a funny story for the former child star in all of us

Table content: <table><tr><td/><td>DelibMLE</td><td>G 1 i got out . had a money money . . G 2 i worked extra and well donut with really well recently .</td></tr><tr><td>HappyDB Corpus</td><td>DelibGAN-I</td><td>G 1 i got a new car . . . G 2 i purchased a new car for my family .</td></tr><tr><td/><td>DelibMLE</td><td>G 1 my &lt;UNK&gt;review : deeply deeply romantic film that cynicism G 2 fantastic summer thrill ride !</td></tr><tr><td>SSTB Corpus</td><td>DelibGAN-I</td><td>G 1 it was &lt;UNK&gt;. G 2 it was a very important classic !</td></tr></table>

Peer review (iclr2019):
 Summary: The authors propose a novel adversarial learning framework consisting of a coarse-to-fine generator and a multiple instance discriminator to generate high-quality sentences without supervision, along with two training mechanisms to make the generator produce higher-quality sentences. The coarse-to-fine generator is implemented as a two-stage decoder where the first stage decoder is initialized with a noise vector z and produces an initial sequence by sampling from its distribution at each step. The second stage decoder is able to attend to first stage decoder hidden states when generating its sequence and is initialized with the last hidden state of the first state decoder Two training frameworks are defined to help the coarse-to-fine generator learn to produce coherent text. In both frameworks, a multi-instance learner is used as a discriminator and the overall score for the sequence is an average of word-level scores. In DelibGAN-I, the first stage decoder is just trained to minimize the negative log likelihood of producing the sampled sequence. I’m not sure I understand how G1 learns to generate coherent text by just maximizing the probability of its samples. In DelibGAN-2, the word-level scores from passing the first stage decoder’s output through the discriminator is used to score the sequence. This makes more sense to me w.r.t to why the first stage generator could learn to produce coherent text. Review: The writing could be clearer. Spacing is consistently an issue in w.r.t. to citations, referencing equations an figures, and using punctuations. Equations are difficult to understand, as well. See Equation 5. I’m confused by how the discriminator score of x_t is computed in equation 5. It seems to be the score of the multi-instance learner when applied to the concatenation of the previous generated tokens and the most recently generated token. This isn’t really a roll-out policy, however, since only a single step is taken in the future. It’s just scoring the next token according to the discriminator. In this case, I’m not sure what the summation over j is supposed to represent. It seems to index different histories when scoring word x_t. The authors should clarify exactly what the scoring procedure is for sequences generated by G2 and make sure that clarification matches what is written in the equation. Because deliberation has previous been explored in Xia et al., 2017, the novelty of this work rests on the application of deliberation to GAN training of generative text models, as well as the development of the multi-instance discriminator for assigning word-level scores to the first-stage decoder*s actions. However, it’s difficult to know how much of the improvement in performance is due to the modeling improvement because the evaluations are missing key details. First, no information about the model’s hyperparameter setting were provided. Naturally, having two generators would double the number of parameters of the total model. Was an ablation run that looked at doubling the parameters of a single-generator? How powerful are the models being used in this new setup. With powerful models such as the OpenAI GPT being available for generative modeling, how powerful are the base units in this framework? Second, I don’t have much intuition for how difficult the tasks being performed are. Because the authors aren’t evaluating on particularly difficult language datasets, they should provide statistics about the datasets w.r.t to vocabulary size, average length, etc. Consequently, a lot of information that is necessary for evaluating the strengths and weaknesses of this paper are missing from the write-up. Questions: Equation 2 is slightly confusing. Is the representation of the word at that time step not conditioned on previous and future hidden states? Is the multi-instance discriminator used in DelibGAN-II or is G2 only scored using the sentence level discriminator score? Small: -Wang & Wan, 2018a are definitely not the first work to discover that using word-level losses to train models for sentence-level evaluations is suboptimal -Please fix issues with spacing. There are issues when citing papers, or referencing equations, or using punctuation. -Equation 3 is missing a closing parenthesis on the softmax