Title: PLANNER: Generating Diversified Paragraph via Latent Language Diffusion Model
Venue: NeurIPS 2023
Accepted: yes
Keywords: Text generation

diffusion model

NLP
Abstract: Autoregressive models for text sometimes generate repetitive and low-quality output because errors accumulate during the steps of generation. This issue is often attributed to exposure bias -- the difference between how a model is trained, and how it is used during inference. Denoising diffusion models provide an alternative approach in which a model can revisit and revise its output. However, they can be computationally expensive and prior efforts on text have led to models that produce less fluent output compared to autoregressive models, especially for longer text and paragraphs. In this paper, we propose PLANNER, a model that combines latent semantic diffusion with autoregressive generation, to generate fluent text while exercising global control over paragraphs. The model achieves this by combining an autoregressive "decoding" module with a "planning" module  that uses latent diffusion to generate semantic paragraph embeddings in a coarse-to-fine manner. The proposed method is evaluated on various conditional generation tasks, and results on semantic generation, text completion and summarization show its effectiveness in generating high-quality long-form text in an efficient manner.

PLANNER: Generating Diversified Paragraph via Latent Language Diffusion Model

Abstract

Autoregressive models for text sometimes generate repetitive and low-quality output because errors accumulate during the steps of generation. This issue is often attributed to exposure bias - the difference between how a model is trained, and how it is used during inference. Denoising diffusion models provide an alternative approach in which a model can revisit and revise its output. However, they can be computationally expensive and prior efforts on text have led to models that produce less fluent output compared to autoregressive models, especially for longer text and paragraphs. In this paper, we propose PLANNER, a model that combines latent semantic diffusion with autoregressive generation, to generate fluent text while exercising global control over paragraphs. The model achieves this by combining an autoregressive 'decoding' module with a 'planning' module that uses latent diffusion to generate semantic paragraph embeddings in a coarse-to-fine manner. The proposed method is evaluated on various conditional generation tasks, and results on semantic generation, text completion and summarization show its effectiveness in generating high-quality long-form text in an efficient manner.

1 Introduction

Autoregressive models trained with a teacher forcing strategy (Williams & Zipser, 1989) are considered the gold standard for text generation. However, a significant drawback of this approach is that it lacks the ability to correct any mistakes made during the generation process which can lead to errors that accumulate as the generation progresses. Previous work (Ott et al., 2018; Holtzman et al., 2019; Welleck et al., 2019; Xu et al., 2022) has observed that deterministic decoding methods have a

tendency to generate consecutive repetitions at the word, phrase and sentence levels. For example, with repetitive prompt, the model can enter an absorbing state where it produces repetitive outputs with higher and higher confidence by self-reinforcing the pattern (Xu et al., 2022) (Fig. 1). Through our experiments, we have observed that such degeneration is more prevalent in open-ended tasks that allow the model greater freedom for creativity. Even for large language models, the generation can drift away from the desired semantics, especially when the model is poorly prompted or has high initial probabilities (Xu et al., 2022).

Why does using maximum likelihood decoding lead to repetitions during generation which is significantly different from the training data distribution? One possible explanation for this is 'exposure bias' (Bengio et al., 2015) arising from the discrepancy between the training and inference phases in the teacher forcing training strategy. During training phase, the model focuses only on predicting the next token. However, during inference, predicting the next token alone can be myopic because the model may not have enough foresight to anticipate its impact on future generation. This can also be seen as the 'distribution shift' issue of behavior cloning (de Haan et al., 2019), where the model is trained to mimic the expert's actions on the states encountered by the expert in the training data. However, small differences between the model and the expert can compound over multiple steps, leading the model to states it never encountered during training, rendering unreliable and undesirable predictions.

Although many approaches have been proposed to address this issue, such as adversarial models (Yu et al., 2017; Lamb et al., 2016; Zhang et al., 2017), reinforcement learning (Li et al., 2016b) or repetition penalties (Xu et al., 2022) , they attempt to improve the global aspects of the generation by making local adjustments which still follow the autoregressive generation recipe. Diffusion models provide an alternative solution - the model can revisit and revise its output iteratively, potentially rendering more global control of the generation in a non-autoregressive manner. However, these text diffusion models can generate less fluent text compared to autoregressive ones (Gong et al., 2023). Also, when generating long text, the diffusion process involves multiple passes of the underlying denoising model over a long generation length, making it computationally expensive. The discrete nature of text also presents a challenge for diffusion models, which can suffer from 'rounding errors' when converting between the text token and its embedding (Li et al., 2022; Lin et al., 2022).

Instead of performing diffusion on the original text or the corresponding word embeddings, we propose to apply diffusion techniques to the latent semantic space (Rombach et al., 2022; Lovelace et al., 2022). To achieve this, we learn a fixed number of continuous semantic tokens that encode salient information at the paragraph level. These tokens can then be used to reconstruct the original text. The latent diffusion can be additionally conditioned on an external signal to generate the semantic tokens. Finally, a decoder maps the obtained semantic tokens back to the raw text space. This process combines a non-autoregressive semantic diffusion approach with an autoregressive decoding technique. The semantic diffusion process handles the 'planning', enabling the modification of semantics in a coarse-to-fine manner, while the decoder handles the 'decoding' by translating the semantics into raw text, with less flexibility in controlling the meaning. We call our proposed method PLANNER ( P aragraph-leve L Diffusio N model for E mbedding R epresentation).

Our contributions include: ( i ) We propose a latent semantic diffusion model for paragraphs that incorporates both non-autoregressive semantic diffusion and autoregressive generation. This allows us to generate fluent text while being able to exercise global control inherited from a diffusion model. ( ii ) We study the essential requirements for a good latent space for paragraph diffusion models. ( iii ) We evaluate the effectiveness of our proposed method on various conditional generation tasks. Thanks to the iterative refinement of desnoising diffusion, our method enjoys less repetitive and more diverse generation, while maintaining good fluency and relevancy, comparing with autoregressive and text diffusion baselines (Li et al., 2022; Lin et al., 2022).

2 Preliminary

Diffusion Probabilistic Models The standard diffusion model (DM) (Ho et al., 2020; Song & Ermon, 2019) learns the data distribution p ( x ) by gradually denoising a normally distributed variable in a Markov chain of length T . The diffusion process can be viewed as a continuous-time stochastic process (Song et al., 2021b; Kingma et al., 2021) where the initial data point x ∈ R N is progressively corrupted by noise according to a predefined signal-noise schedule { α t , σ t } , resulting in time-

dependent corrupted data { x t ∣ t ∈ 0 , 1 , x 0 x . The transition distribution is given by:

( ∣ ) = N ( ) where α t ∣ s = α t / α s , σ 2 t ∣ s = σ 2 t -α 2 t ∣ s σ 2 s , and s < t . When x s = x , the marginal distribution q ( x t ∣ x ) is given as q ( x t ∣ x ) = N ( x t ; α t x , σ 2 t I ) . The diffusion model relies on a parametric function θ optimized to reverse the diffusion process by denoising x t to the clean input x . The model is trained using a weighted reconstruction loss:

where ω t = α 2 t / σ 2 t , ( s.t. α 2 t + σ 2 t = 1 ) is the signal-to-noise-ratio (SNR) and F θ ( ⋅ ) denotes the backbone denoising function. Sampling from the learned model can be performed using either ancestral sampling (DDPM) (Ho et al., 2020) or a deterministic DDIM sampler (Song et al., 2021a). While the DM is capable of generating high-quality samples, the fact that the corrupted data x t shares the same space as the input x results in inefficient training (Jing et al., 2022) and difficulty in learning abstract and semantically meaningful latent spaces (Preechakul et al., 2022).

Latent Diffusion Models To improve the efficiency, the Latent Diffusion Model (LDM) (Rombach et al., 2022) introduces an explicit separation between the compressive and generative learning phases of training diffusion models. It employs an autoencoding model consisting of an encoder E ( ⋅ ) and a decoder D ( ⋅ ) to learn a low-dimensional latent space that is perceptually equivalent to the image space when decoded, but with reduced computational complexity, while retaining the perceptual quality of generated samples. The reweighted objective for training LDM is given by:

3 Related Work

where z is obtained from E during training. The generated z can be decoded to image using D .

Text diffusion models Early attempts on using diffusion models for discrete data used a noising processes which masked or randomly mutated the discrete tokens (Austin et al., 2021; Hoogeboom et al., 2021). Recently, Diff-LM (Li et al., 2022) and DiffuSeq (Gong et al., 2023) have instead used a continuous token embedding space, converting the continuous token embeddings to text via "rounding". Analog Bits (Chen et al., 2022) converts raw text into a set of bits and models them as analog bits with a continuous diffusion model. (Lovelace et al., 2022) performed diffusion model on the contextualized BART embeddings rather than on the word embedding space. (Zhu et al., 2022) has applied text diffusion to image-captioning and achieved good performance.

However, existing text diffusion models present several issues: (i) The varying length of the input text necessitates the prediction of additional length or superfluous paddings, and (ii) token generation in parallel may result in disfluent text and/or frequent repetitions especially when the generation is long. We instead employ the diffusion model to learn paragraph embeddings that contain fewer fixed-sized tokens, which allows for computational benefits and improved fluency.

Text Variational Autoencoders Text VAEs (Bowman et al., 2016; Kim et al., 2018; Li et al., 2020) have been particularly useful for learning a smooth and interpretable representation space, as well as for generating diverse text. However, one of the challenges is the KL vanishing problem (Bowman et al., 2016), which results in the decoder disregarding the latent code sampled from the prior distribution during the inference stage. Our approach can be perceived as to address this issue by leveraging a more flexible prior distribution to ensure the codes can strongly influence the output text distribution.

4 PLANNER: A Language Diffusion Model on Paragraph Embeddings

We use latent diffusion to improve the diversity and fluency of paragraphs generated from the model. Our model comprises two parts (Fig. 2) - a paragraph embedder via variational autoencoder (VAE) that learns a meaningful and smooth latent space that corresponds to the original text space, and a diffusion model that generates latent codes corresponding to the semantics of longer text.

<latexi sh

1_b

64="V

dE8

MF

9S

05

T

>A

B

H

c

DL

N

Oz

G

r

3

Q

Rj

PC

J

ky

7v

Ip

/

m

X

W

2

U

q

K

o

+

f

<latexi sh

1_b

64="ZN5pF8

2Ocw

VC

E

A

>

B

n

DLSgM

3U

vqk

K

jP

W

+

J

yR

0X

/

9

T

oI

r

d

G

Q

m

f

H

<latexi sh

1_b

64="V

dE8

MF

9S

05

T

>A

B

H

c

DL

N

Oz

G

r

3

Q

Rj

PC

J

ky

7v

Ip

/

m

X

W

2

U

q

K

o

+

f

<latexi sh

1_b

64="Mp+fy

n

KB

T

rEZouW

Y

>A

8

c

VDLSg

F

3U

2vqk

w

jP

9

H

CG

dm

7

Q

0

N

X

/

J

IR

O

z

5

<latexi sh

1_b

64="

WJ

o

O2A

d

u

I

>

B

3

c

V

N

E

Ur q/

vg

T

7H

MF

w

K

p

m

G

5

0+

Q

D

f

Z

CY

R

Pz

k

<latexi sh

1_b

64="Mp+fy

n

KB

T

rEZouW

Y

>A

8

c

VDLSg

F

3U

2vqk

w

jP

9

H

CG

dm

7

Q

0

N

X

/

J

IR

O

z

5

<latexi sh

1_b

64="SRn qC

IFQX0ZrL

9

Ew

g

8

>A

B

c

VD

M

3U

2v

k

K

jP

o

d

5J

m

O

+

u

Y

W

H

f

7

/

N

p

z

T

y

G

<latexi sh

1_b

64="

n

u5

z+

C23OLF/

B

>A

8

VD

S

N

J

U

qk g0

w

R

y

P

TKd

EM

Hm

G

o

9

W

r

7

I

f

j

Y

Q

<latexi sh

1_b

64="

WRQEG

Mp

c

zI

T9D7/

A

>

B

H

V

LSgN

Oy

r

fUY

oP

K

2

8

J

j

u

CX

w

k

q

n

3

m+

Z

v

F

5

d

0

<latexi sh

1_b

64="7/Z

Pqp

Vu

f

OR+B

>A

n

c

FI

3W

CE

d

T

9g

M

2

X0U

y

z

G

r

k

o

Q8

J

H

v

j

m

<latexi sh

1_b

64="CnN

O

3m

Xy5r

2M

>

c

ZDLS

FIY

u9p

0

gE

8

wV

k

Hz

R

qKU

d

J

G

v

f

j

T

W

o

<latexi sh

1_b

64="A

TYu58g3

0

zyK/P

OE

Qo qRk

>

B7

c

VDLS

N

G

Mr

H

d

j

C

U

J

n

m

X

p

w

I

vZ2

F

W

f

+

9

<latexi sh

1_b

64="TY

UROkd5Vo2

LN

rqc

7+

8

>A

B

3

D

E

y

f

9

P

K

Q

I

HJ

n

m

Z

C

w

Fv

G

j

p

X

W/

z

M

u

0

z

(1)

…

z

(k)

x

1

x

2

x

3

…

x

L

BOS

x

1

…

x

L

x

1

…

x

L

EOS

Plan

Transformer Model

Transformer Model

y

1

…

The Bank of Scotland has announced

it is to close 23 branches in Scotland.

The bank announced

the closures … in UK.

z

T

(1)

z

T

(2)

…

z

T

(k)

z

t

(1)

z

t

(2)

…

z

t

(k)

z

0

(1)

z

0

(2)

…

z

0

(k)

E

D

D

τ

N

z

Decode

ˆ

x

Layer Norm

Scale, Shift

Multi-head Attention

Scale

Layer Norm

Scale, Shift

Pointwise FFN

Scale

+

Input

+

y

t

t=0.9

Time Embed

t=0.1

β

1

,

γ

1

β

2

,

γ

2

α

1

α

2

˜

x

˜

x

Variational Paragraph Embedder

Latent Diffusion Model

4.1 Learning a Variational Paragraph Embedder

Instead of applying diffusion to tokens directly to generate long text, we propose to learn a set of latent codes z = { z ( 1 ) , /uni22EF , z ( k ) } ∈ R k × h , which we call paragraph embeddings , that capture the semantics in the target text (of length up to 512 tokens), where h denotes the embedding dimension. These paragraph embeddings have shorter length, such as k = 16 , than the original text.

Conversion error Ideally, the original input x can be perfectly reconstructed via ˆ x = D ( z ) , z = E ( x ) , and modeling the lower-dimensional continuous space p ( z ) is equivalent to modeling p ( x ) . However, in practice a loss of information can occur when converting raw text into paragraph embeddings or when doing the reverse. We assess the conversion loss by computing the BLEU score ( BLEU clean ) between the input x and the reconstruction ˆ x .

To obtain such embeddings z , we train a transformer-based encoder-decoder model. The architecture used for the autoencoder is shown in Fig. 2. The encoder E and decoder D construct a bidirectional mapping between the discrete data space and the latent code space. The paragraph embeddings z are extracted by taking the first k hidden state vectors of dimension h from the final layer of E , which are fed into the initial steps of the decoder which is trained to reconstruct the original text. It's worth noting that the paragraph embeddings share the same hidden dimension h as the word embeddings, and forming a manifold in the word embedding space. Pretrained BERT and GPT-2 models are used to initialize E and D , respectively. The manifold of the learned embeddings ideally possesses several desirable properties, including low conversion error , local smoothness and distributional smoothness .

Local smoothness To generate target text that is fluent and consistent with the corresponding paragraph embeddings, it is essential to achieve a certain level of local smoothness in the paragraph embeddings space. Ideally, a slight variation in the input vector x would not cause a significant change in the resulting encoded vector E ( x ) . Similarly, a small perturbation in the latent vector z should not lead to a significant change in the decoded vector D ( z ) . Otherwise, the error accumulated in the diffusion process when generating z could result in an inaccurate realization of the desired semantics. To accomplish this, the denoising autoencoder is trained by substituting ( Sub ) input tokens with random tokens with probability p . The local smoothness is measured using the BLEU score ( BLEU robust ) between the input x and the denoising output from corrupted input D ( E ( ˜ x )) , where ˜ x = Sub ( x , p = 0 . 3 ) . The level of injected noise will affect both the conversion error and the local smoothness, and it is important to strike a balance between the two.

Distributional Smoothness The diffusion model may face difficulty in learning a distribution, p ( z ) , that is highly multimodal, or has density that are associated with a large Lipchitz constant ( i.e. , has abrupt changes). Therefore, we employ a text V AE (Bowman et al., 2016; Li et al., 2020) to encourage the posterior to take on a form close to a Gaussian distribution. Specifically, we parameterize q ( z ∣ x ) to be N ( E µ , E ν ) and maximize the objective L ( E , D ; x ) = E q ( z ∣ x ) [ log p ( x ∣ z )] -β ⋅ KL ( q ( z ∣ x ) ∣∣ p ( z )) . Here E µ and E ν represent the posterior mean and variance predictions of the encoder E , while the

hyperparameter β controls the strength of regularization. It is typically set to a small value to alleviate the notorious posterior collapsing issue (Bowman et al., 2016) in text V AE. To gauge the distributional smoothness of the paragraph embedding space, we select two examples, x and x ′ at random from the training set and interpolate their embeddings to compute z INT = 1 2 E ( x ) + 1 2 E ( x ′ ) . We then evaluate the perplexity ( PPL int ) of the decoded interpolation D ( z INT ) using a GPT-2 model.

4.2 Planning then Decoding: A Latent Diffusion Model for Paragraph Embeddings

Training phase We now use the learned mean paragraph embeddings z = E µ ( x ) to train a continuous-time latent diffusion model as in Fig. 2 while keeping E and D frozen. We conducted experiments using two types of conditioning signal: ( i ) class labels, such as positive or negative sentiment labels, and ( ii ) raw text, such as preceding context or the document to be summarized. For class labels, we learned a label embedding y ∈ R h to represent each class. For the raw text, we applied a conditional feature encoder τ to the input and used the hidden states at the last layer as y ∈ R c × h , where c represents the number of feature embeddings.

For the adaLN, we flattened and projected y to a vector of R h using a linear projection layer. We then added the projected y to the time embedding t . Instead of directly learning dimension-wise scale and shift parameters ( β and γ ) in the standard Layer Norm (LN), these parameters are regressed from the sum of the embeddings. In addition, dimension-wise scaling parameters α are regressed and applied immediately before any residual connections within the transformer block. This has been shown to be efficient and effective in image diffusion models (Peebles & Xie, 2022).

During training, we gradually add noise to z via a cosine scheduler (Ho et al., 2020), and use a signal prediction scheme as the training objective (Kingma et al., 2021). For our denoising backbone model F ( ⋅ ) , we use a transformer block similar to the one in the DiT (Peebles & Xie, 2022) model. Specifically, we fed y and the time embedding t ∈ R h into the model through two channels, namely cross-attention and adaptive layer norm (adaLN) (Peebles & Xie, 2022). For the cross-attention, the conditional embeddings t and y are concatenated into a sequence of length c + 1 . The transformer block is modified to enable multi-head cross-attention to the conditional embeddings.

Inference phase During the inference process, we start with random Gaussian embeddings and use a fixed number of steps T to generate the final z . The resulting embeddings are then used as inputs for D to generate the text using a deterministic decoding method like greedy decoding 1 . We provide discussion and ablation study on using stochastic decoding in the App. C

We applied the classifier-free guidance (CFG) (Ho & Salimans, 2021) during the inference steps. After each step, we apply a dynamic thresholding technique that was introduced in Imagen (Saharia et al., 2022) for post-processing. However, we do not use the rescaling step in Imagen because rescaling step can completely alter the underlying semantics, as we have not imposed any constraints to ensure that the generated output remains the same after rescaling the latent code. In contrast, for Imagen, where the generation takes place in the raw pixel space, rescaling will predominantly retain the shape information while altering only the contrast and brightness.

5 Experimental Setups

We tested the effectiveness of our model in three different conditional generation tasks including sentiment-guided generation, long-form text completion, and summarization. The tasks can require generating text of hundreds of tokens in length, making them suitable to assess model performance.

Datasets For the Sentiment-guided generation task, we used the TripAdvisor dataset provided by (Li et al., 2014). By exclusively selecting reviews with a rating of 1 or 5 and balancing the two ratings via subsampling, we acquired 218,509 reviews. For the text completion task, our model was assessed on two datasets: 1) the aforementioned TripAdvisor review dataset with postprocessing to remove reviews that are less than 20 or more than 512 tokens, result in 690,862 samples, and 2) one-tenth of the overall C4 datasets (Raffel et al., 2020), which contains 36.5M samples. For each sample, we extracted the initial two sentences from a paragraph as source context, and predicted the remainder of the text as target. The datasets were partitioned into training, validation, and test in the ratios of

( 0 . 96 , 0 . 02 , 0 . 02 ) . For the summarization task, we use CNN/DailyMail (Hermann et al., 2015) and XSum (Narayan et al., 2018).

Automatic Evaluation Metrics Following previous work (Gong et al., 2023), we assess the fluency of the generation by computing the perplexity ( PPL ) under a GPT-2 large model. We use Ent-n (Zhang et al., 2018) and DIST-n (Li et al., 2016a) and self-BLEU ( S-BL ) (Zhu et al., 2018) to evaluate lexical diversity. We present DIST-n and Ent-n metrics solely at n = 1 owing to their strong correlation of the varying n values. We use REP-n to assess the extent of repetition in the generation following previous work (Welleck et al., 2019; Xu et al., 2022). For relevancy we use standard metrics following (Gong et al., 2023), including SacreBLEU ( BL ) (Post, 2018), ROUGE-L ( R-L ) (Lin, 2004) and BERTScore ( Score ) (Zhang et al., 2019). Details are provided in App. F.

AuBLEU: Evaluating Denoising Capability Our proposed model is a latent diffusion model, which differs from text diffusion models that operate directly on text or text embedding space. To comparing the denoising ability across different text diffusion models, we introduce a novel metric, named AuBLEU ( AuBL ). To compute the AuBLEU score, we first add varying levels of noise to each input text x by performing diffusion at T different time steps t 0 < t 1 < /uni22EF < t T , corresponding to a series of SNR ω t 0 > /uni22EF > ω t T . Next, we pass each corrupted input under different ω to the denoising backbone model and obtain the predicted output ˆ x i = F ( x t i ) . We then compute the BLEU score between each ( ˆ x i , x ) pairs and plot a curve with the x-axis representing α 2 = ω 1 + ω , where α 2 is monotonically increasing with ω and ranges from ( 0 , 1 ) , and the y-axis indicating the corresponding BLEU score. Finally, we compute the area under curve to obtain the AuBLEU score (see App. D for more details).

Model Setups We used the BERT-large and GPT-medium models as initialization for the encoder E and decoder D respectively. The embedding dimension h was 1024, and the number of paragraph embeddings k was set to 16, as increasing the number did not result in significant improvement in performance. We provide more analysis on the impact of k in App. A.2 The learning rate was set to 2 e -4 , and β was set to 5 e -6 . For the latent diffusion model, the channel size was set to 1024 to match the embedding dimension h , and the number of heads was set to 16 with 28 transformer layers. The total size of the latent diffusion model was 533M. The feature encoder τ was also jointly learned, and was initialized with a T5-large encoder. We use DDIM throughout our experiments as it shows better performance than DDPM. In all our experiments, we use 30 diffusion steps to generate the final z , which strikes a good balance among the efficiency, diversity and relevance. In comparison, Diff-LM (Li et al., 2022) and Genie (Lin et al., 2022) report to use 200 steps and 2000 steps respectively to generate high-quality text. We set the CFG weights to be 2 and 5 for text completion and summarization tasks, respectively, based on generation performance on validation set. For summarization task, we also incorporate a shift noise scheduler based on (Hoogeboom et al., 2023). More details, including ablations on DDPM, number of diffusion steps and noise scheduler, are provided in App. F.

Baselines We compare our method with several baseline methods trained under Teacher Forcing scheme, including decoder-only Autoregressive LM finetuned on GPT-2 ( FT ), encoder-decoder ( Enc-Dec ) transformer model, and Varitional Information Bottleneck ( VIB ) (Alemi et al., 2016). We initialized the FT model using GPT-2 large (774M), whereas encoder and decoder in the Enc-Dec/VIB models (695M/697M) are initialized with bert-large and GPT-medium, respectively. All the considered models are finetuned on the target datasets. We follow Li et al. (2022) to report the FT baselines with two decoding strategies, top-p sampling (K=50, p=0.92) and beam search (beam width 4), denoted as FT-sample and FT-search. We use top-p sampling for Enc-Dec/VIB generation. For summarization tasks, we finetune a T5-large model (770M) on the target datasets as baselines. We also compared two text diffusion models Diff-LM and Genie using their suggested configuration from the official repository. More details are in App. F.

6 Results

6.1 Paragraph Representation Learning

It is essential to learn a paragraph embedding space that is both accurate and smooth . To this end, we examined the effect of various substitution probabilities p to the input tokens x . Our findings, presented in Fig. 3, reveal that a smaller p results in a lower conversion error, as indicated by a higher reconstruction BLEU ( BLEU clean ), albeit at the expense of local smoothness ( BLEU robust ) and distributional smoothness ( PPL int ). Weperformed a grid search of p with 0 . 1 increment based on 512 samples. Empirically, we observed that a weighted score S overall = 0 . 5 BLEU clean + 0 . 8 BLEU robust -0 . 3 PPL int correlate well with downstream diffusion performance, leading to fluent and accurate generation for PLANNER (see App. A for more details). We finally opted for p = 0 . 3 for most datasets 2 , which strike a balance between conversion error and smoothness.

It is worth noting that there is an inevitable conversion loss, indicated by the fact that the BLEU clean is between 77 ∼ 87 when generating hundreds of words (App. D). We observd that most of the lexical mismatch still maintain the similar semantics, with the exception of some name entity swapping. We show some paragraph reconstruction and denoising examples from our paragraph embedder in the App. A. We also include examples of interpolated generation from random paragraph pairs in the App. A. In general the transition of semantics is natural, indicating reasonable distributional smoothness of the paragraph embedding space.

6.2 Sentiment-Guided Generation

For sentiment-guided generation experiments, following previous works (Li et al., 2022; Hu et al., 2017; Keskar et al., 2019), we use a trained classifier to assess if the given sentiment is well-controlled in the generation. The trained classifier is initialized with BERT-

large and finetuned on the training set, which yields an accuracy of 96 . 75% on the held-out test set. The results are provided in in Tab. 1. PLANNER outperforms the baseline approaches in generating long reviews at higher levels of accuracy. Although PLANNER using a greedy decoding mode is at least comparable with FT with top-p sampling in terms of diversity, and has lower repetition as assessed by Rep-4 in generation.

The perplexity of the text generated by PLANNER is close to that of human-written text. We provide examples of the generated text in App. E. Interestingly, as shown in App. E, with the same random seed but different controlling sentiment, PLANNER generates text with similar contents but different sentiments, suggesting the diffusion model may be able to disentangle the semantic space to certain extent. Unlike the autoregressive generation, the nature of the diffusion model allows the model to 'regret' and iteratively refine on the current generations. In App. B, we demonstrate how the generation evolves over multiple time steps in a coarse-to-fine manner in PLANNER.

6.3 Long-form Text Completion

We further evaluate our model on the long-form text completion tasks. For text diffusion baseline, we compared our method with Diff-LM (Li et al., 2022) on hotel review dataset. We could not perform a comparison on the C4 dataset with Diff-LM due to the significant amount (thousands) of GPU hours required to train Diff-LM adequately. A Diff-LM running time estimation is available in App. F. The results are provided in Tab. 2. FT-search performed poorly in this open-ended generation task as its generation exhibited high repetition levels, consistent with findings in previous research (Holtzman et al., 2019; Xu et al., 2022). Although our approach also employs a deterministic decoding method, we observed that it produces text with low Rep-4 metric, signifying that PLANNER is effective in

reducing repetition through holistic iterative refinement throughout the inference steps in the diffusion process. Comparing with Diff-LM and other baselines, PLANNER achieved better diversity scores while maintaining comparable relevance scores. We also observe higher AuBLEU of PLANNER comparing with Diff-LM, indicating a potentially higher overall denoising strength of PLANNER (See App. D for more details). Some examples of the generated text are available in the App. E. We also observed PLANNER exhibits robustness towards prompts that are either repetitive or ill-composed, where FT failed (Fig. 1, App. G).

We further performed pairwise human evaluation on 300 examples of hotel review generation from each system on our internal crowd-source annotation platform. Each pair of text being presented to 3 judges in random order. The judges ranked the pairs for relevance, informativeness and human-like properties using a 3-point Likertlike scale. Overall judge preferences are shown in Table 3. A moderate preference can be observed for PLANNER over FT and VIB, except for human-like between PLANNER and VIB. We also observe that judges still prefer human responses over system generations in this task. Further details, including the human evaluation template used and interrater agreement analysis, are provided in the App. H.

6.4 Summarization

We further conducted evaluation on summarization and present the results in Tab. 4. Summarization is less open-ended than the text completion task, thus a deterministic decoding approach like T5-search can produce high-quality text. Our evaluation shows that in comparison with T5-sample and Genie (Lin et al., 2022), PLANNER exhibits comparable Rouge-L scores, while improves other metrics. PLANNER achieves higher AuBLEU than Genie (See App. D for more details).

Owing to the sampling nature of the diffusion model, PLANNER and Genie yielded lower Rouge-L scores in comparison with T5-search, with single summary. To align with Genie's evaluation, we provide the results with 10 random runs in Tab. 4, where for each document it generates 10 summaries and the one with the highest Rouge score is used. However, we note that these summaries with

best Rouge-L cannot be predetermined without an oracle summary. Comparing with T5-search, PLANNER generates more diverse and less repetitive summaries. However, the improvement is less conspicuous comparing with the results observed in open-ended text completion tasks. We show some generations in the App. E (Tab. 12).

Notably, the generated content may occasionally include hallucinations or errors, especially for name entities and digits (App. E, Tab. 13). Such occurrences can be attributed to either the conversion errors in D or errors during the generation of paragraph embeddings, and requires further investigation.

6.5 Analysis

Running time We conducted inference time benchmarks of each method on a single Nvidia A100. For the sentiment-guided generation task, the autoregressive baseline is 5x faster than our method as the generation for all methods can be batched. For all other tasks, the varying input lengths make direct batchification for the FT baseline not straightforward. In these scenarios, the latent diffusion over a fixed number of latent codes offers computational advantages over a naive decoding of the FT baseline as the latent codes in our method can be conveniently batched.

For the hotel review completion task, the generation of 256 samples took 378 seconds to complete, including 83 seconds for decoding and 295 seconds for diffusion generation with 30 generation steps. The unbatched FT baseline took 1,693 seconds to complete 256 generations. Sorting input text by length and maximally batchifying them as possibley reduce the (batched) FT inference time to 338 seconds. The Diff-LM algorithm required 397 seconds to produce 256 samples using 200 generation steps, which is comparable to ours. Although our method is slower than the autoregressive ones, PLANNER enjoys the convenience of arranging input into the same length vectors without further length bucketing. On the CNN-DM summarization tasks, our method took 8.4 GPU hours to generate 11392 summaries. Genie's generation took 47.2 GPU hours. XSum gives similar inference running time benchmark to the results on CNN-DM.

Generations over diffusion steps In App. B we provide generation examples for both summarization and sentiment-guided generation over different diffusion steps, which progress in a coarse-to-fine manner. The generation from early time step tends to be less fluent and generic. As the time approaches 0, the generation becomes more detailed. We presented quantitative results characterizing the evolution of the metrics over generation steps in App. B, Fig. 5. It revealed a clear trend of improvement in the majority of the metrics as the generation proceeds. Notably, most hallucinations

occur during the late phase when more details are being incorporated. The model may excessively emphasize certain aspects, resulting in the correct generation being altered to an erroneous one (App. B, Tab. 10).

7 Conclusion

We present a two-stage latent text diffusion model that uses an autoencoder to condense lengthy texts into a limited number of paragraph embeddings, and a continous time diffusion model that learns the distribution of these embeddings. Our proposed model alleviates the issue of repetition and advances generation diversity across different tasks. Compared to text diffusion models that perform diffusion solely on token or token embedding space, our method generates fluent text with improved diversity and reduced repetition. There may be toxicity or fairness issues in the dateset we used that we have not been able to identify. There are several limitations that warrant further investigation. Our work relies on an autoregressive decoder for converting latent representation into coherent text. It is possible to explore the feasibility of non-autoregressive decoders to bolster efficiency while minimizing conversion errors and hallucination in the generation. Furthermore, the classifier-free guidance approach results in a discrepancy between training and inference data distribution when feeding to the diffusion backbone. It would be interesting to investigate a 'calibration' strategy for the latent code to better fit the data distribution during training.

References

Alexander A Alemi, Ian Fischer, Joshua V Dillon, and Kevin Murphy. Deep variational information bottleneck. In ICLR , 2016.

Jacob Austin, Daniel D Johnson, Jonathan Ho, Daniel Tarlow, and Rianne van den Berg. Structured denoising diffusion models in discrete state-spaces. NeurIPS , 34:17981-17993, 2021.

Samy Bengio, Oriol Vinyals, Navdeep Jaitly, and Noam Shazeer. Scheduled sampling for sequence prediction with recurrent neural networks. In NeurIPS , volume 28, 2015.

Samuel Bowman, Luke Vilnis, Oriol Vinyals, Andrew Dai, Rafal Jozefowicz, and Samy Bengio. Generating sentences from a continuous space. In CONLL , pp. 10-21, 2016.

Ting Chen, Ruixiang Zhang, and Geoffrey Hinton. Analog bits: Generating discrete data using diffusion models with self-conditioning. arXiv preprint arXiv:2208.04202 , 2022.

Pim de Haan, Dinesh Jayaraman, and Sergey Levine. Causal confusion in imitation learning. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett (eds.), Advances in Neural Information Processing Systems , volume 32. Curran Associates, Inc., 2019. URL https://proceedings.neurips. cc/paper_files/paper/2019/file/947018640bf36a2bb609d3557a285329-Paper.pdf .

Shansan Gong, Mukai Li, Jiangtao Feng, Zhiyong Wu, and LingPeng Kong. Diffuseq: Sequence to sequence text generation with diffusion models. In ICLR , 2023.

Karl Moritz Hermann, Tomas Kocisky, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. Teaching machines to read and comprehend. Advances in neural information processing systems , 28, 2015.

Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. In NeurIPS 2021 Workshop on Deep Generative Models and Downstream Applications , 2021.

Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In NeurIPS , volume 33, 2020.

Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. The curious case of neural text degeneration. In ICLR , 2019.

Emiel Hoogeboom, Alexey A Gritsenko, Jasmijn Bastings, Ben Poole, Rianne van den Berg, and Tim Salimans. Autoregressive diffusion models. In ICLR , 2021.

Emiel Hoogeboom, Jonathan Heek, and Tim Salimans. simple diffusion: End-to-end diffusion for high resolution images. arXiv preprint arXiv:2301.11093 , 2023.

Zhiting Hu, Zichao Yang, Xiaodan Liang, Ruslan Salakhutdinov, and Eric P Xing. Toward controlled generation of text. In International conference on machine learning , pp. 1587-1596. PMLR, 2017.

Bowen Jing, Gabriele Corso, Renato Berlinghieri, and Tommi Jaakkola. Subspace diffusion generative models. In Computer Vision-ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part XXIII , pp. 274-289. Springer, 2022.

Nitish Shirish Keskar, Bryan McCann, Lav R Varshney, Caiming Xiong, and Richard Socher. Ctrl: A conditional transformer language model for controllable generation. arXiv preprint arXiv:1909.05858 , 2019.

Yoon Kim, Sam Wiseman, and Alexander M Rush. A tutorial on deep latent variable models of natural language. arXiv preprint arXiv:1812.06834 , 2018.

Diederik Kingma, Tim Salimans, Ben Poole, and Jonathan Ho. Variational diffusion models. NeurIPS , 34: 21696-21707, 2021.

Alex M Lamb, Anirudh Goyal ALIAS PARTH GOYAL, Ying Zhang, Saizheng Zhang, Aaron C Courville, and Yoshua Bengio. Professor forcing: A new algorithm for training recurrent networks. Advances in neural information processing systems , 29, 2016.

Chunyuan Li, Xiang Gao, Yuan Li, Baolin Peng, Xiujun Li, Yizhe Zhang, and Jianfeng Gao. Optimus: Organizing sentences via pre-trained modeling of a latent space. In EMNLP , 2020.

Jiwei Li, Myle Ott, Claire Cardie, and Eduard Hovy. Towards a general rule for identifying deceptive opinion spam. In ACL , pp. 1566-1576, 2014.

Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. A diversity-promoting objective function for neural conversation models. In NAACL , 2016a.

Jiwei Li, Will Monroe, Alan Ritter, Dan Jurafsky, Michel Galley, and Jianfeng Gao. Deep reinforcement learning for dialogue generation. In EMNLP , pp. 1192-1202, 2016b.

Xiang Li, John Thickstun, Ishaan Gulrajani, Percy S Liang, and Tatsunori B Hashimoto. Diffusion-lm improves controllable text generation. NeurIPS , 35:4328-4343, 2022.

Chin-Yew Lin. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out , pp. 74-81, 2004.

Zhenghao Lin, Yeyun Gong, Yelong Shen, Tong Wu, Zhihao Fan, Chen Lin, Weizhu Chen, and Nan Duan. Genie: Large scale pre-training for text generation with diffusion model. arXiv preprint arXiv:2212.11685 , 2022.

Justin Lovelace, Varsha Kishore, Chao Wan, Eliot Shekhtman, and Kilian Weinberger. Latent diffusion for language generation. arXiv preprint arXiv:2212.09462 , 2022.

Shashi Narayan, Shay B. Cohen, and Mirella Lapata. Don't give me the details, just the summary! topicaware convolutional neural networks for extreme summarization. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing , pp. 1797-1807, Brussels, Belgium, October-November 2018. Association for Computational Linguistics. doi: 10.18653/v1/D18-1206. URL https://aclanthology.org/D18-1206 .

Myle Ott, Michael Auli, David Grangier, and Marc'Aurelio Ranzato. Analyzing uncertainty in neural machine translation. In ICML , pp. 3956-3965. PMLR, 2018.

William Peebles and Saining Xie. Scalable diffusion models with transformers. arXiv preprint arXiv:2212.09748 , 2022.

Matt Post. A call for clarity in reporting BLEU scores. In Proceedings of the Third Conference on Machine Translation: Research Papers , October 2018.

Konpat Preechakul, Nattanat Chatthee, Suttisak Wizadwongsa, and Supasorn Suwajanakorn. Diffusion autoencoders: Toward a meaningful and decodable representation. In CVPR , pp. 10619-10629, 2022.

Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. JMLR , 2020. URL http://jmlr.org/papers/v21/20-074.html .

Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In CVPR , pp. 10684-10695, 2022.

Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. In NeurIPS , volume 35, pp. 36479-36494, 2022.

Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In ICLR , 2021a.

Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. NeurIPS , 32, 2019.

Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In ICLR , 2021b.

Sean Welleck, Ilia Kulikov, Stephen Roller, Emily Dinan, Kyunghyun Cho, and Jason Weston. Neural text generation with unlikelihood training. arXiv preprint arXiv:1908.04319 , 2019.

Ronald J Williams and David Zipser. A learning algorithm for continually running fully recurrent neural networks. Neural computation , 1(2):270-280, 1989.

Jin Xu, Xiaojiang Liu, Jianhao Yan, Deng Cai, Huayang Li, and Jian Li. Learning to break the loop: Analyzing and mitigating repetitions for neural text generation. In NeurIPS , 2022.

Lantao Yu, Weinan Zhang, Jun Wang, and Yong Yu. Seqgan: Sequence generative adversarial nets with policy gradient. In AAAI , volume 31, 2017.

Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. Bertscore: Evaluating text generation with bert. In ICLR , 2019.

Yizhe Zhang, Zhe Gan, Kai Fan, Zhi Chen, Ricardo Henao, Dinghan Shen, and Lawrence Carin. Adversarial feature matching for text generation. In ICML , pp. 4006-4015. PMLR, 2017.

Yizhe Zhang, Michel Galley, Jianfeng Gao, Zhe Gan, Xiujun Li, Chris Brockett, and Bill Dolan. Generating informative and diverse conversational responses via adversarial information maximization. In NeurIPS , 2018.

Yaoming Zhu, Sidi Lu, Lei Zheng, Jiaxian Guo, Weinan Zhang, Jun Wang, and Yong Yu. Texygen: A benchmarking platform for text generation models. In The 41st international ACM SIGIR conference on research & development in information retrieval , pp. 1097-1100, 2018.

Zixin Zhu, Yixuan Wei, Jianfeng Wang, Zhe Gan, Zheng Zhang, Le Wang, Gang Hua, Lijuan Wang, Zicheng Liu, and Han Hu. Exploring discrete diffusion models for image captioning. arXiv preprint arXiv:2211.11694 , 2022.

Peer reviews:
comments: SUMMARY:
The paper presents a two-stage latent text diffusion model that uses an autoencoder to condense lengthy texts into a limited number of paragraph embeddings, and a continuous time diffusion model that359 learns the distribution of these embeddings. The paper presents detailed experiments on open-ended generation tasks and shows that the proposed model alleviates the issue of repetition and advances generation diversity across different tasks.

STRENGTHS:
The paper is well-motivated. The examples in Figure 1 with the self-reinforcing effect are impressive. The proposed method first summarizes the paragraph information with an autoencoder and then applies the diffusion process to control the token generation process, which is novel from my point. The paper conducts extensive experiments on open-ended generation tasks and provides some analyses of the running time.

WEAKNESSES:
1. Could you provide more experimental analyses on the learned paragraph embedding? I think that paragraph embedding is used to learn a high-level concept for planning but ignores the details of sentence/phrase structure to avoid copying similar phrases from previous prompts and repeating them.  Could you provide more experiments to show the quality of learned paragraph embedding? Directly extracting the first K tokens from the encoder is not the best choice for me.

2. In experiments such as Table 1, the proposed method with a greedy decoding mode is comparable to baselines with top-p sampling in terms of diversity. A question is whether PLANNER combined with top-p can lead to better results. Are the proposed methods compatible with stochastic decoding methods?

QUESTIONS:
Please see the weakness.

LIMITATIONS:
Please see the weakness.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-05

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a 2-stage generative model for text. The first stage involves training a VAE over the data to obtain an effective encoder and decoder modules. The second stage uses the output of the frozen VAE encoder as the hidden state of the input text and learns a regular continuous diffusion model over the encoded hidden states of training data instances. For generation, the diffusion model produces a hidden state from noise and this hidden state is fed into the VAE decoder to generate text. This model is evaluated on sentiment-guided generation, text completion, and summarization and compared against baselines that include token-based diffusion models (instead of hidden state text diffusion models in this work), VAE/Enc-dec models, and standard autoregressive models (like fine tuned GPT-2).\n\nSTRENGTHS:\n– Although the approach seems straightforward, to the best of my knowledge, surprisingly I am not aware of other latent sequence/paragraph embedding based diffusion models for text generation. Hence, this fills a gap in the research on diffusion models for text.\n\n– The experimental design is reasonable and informative. \n\n– This approach seems to be more fluent, diverse, and effective than popular token-based diffusion models following the results.\n\n– Although typically less fluent than the autoregressive models, it still performs competitively when considering other metrics like diversity, reference overlap, and control.\n\nWEAKNESSES:\n– I have some concerns about evaluation, particularly related to the quality of baseline models. For example, on CNN-DM summarization task, publicly available results (https://paperswithcode.com/sota/abstractive-text-summarization-on-cnn-daily) show that T5 achieve R-L that is ~7-10 points better than reported in this paper (and is hence better than the R-L proposed model achieves). Therefore, I am doubtful about the quality of training/tuning of the baseline models reported in the paper.\n\n– Choosing “first k hidden state vectors” from the encoder in the VAE seems arbitrary. What is the motivation behind this hyperparameter? What is the effect of this k?\n\n– The paper alludes to learning “paragraph embeddings” which capture high level semantic properties which enables “coarse-to-fine” generation and allows for “planning”. However, no evidence is presented in this paper about compositional/planning capabilities of the model. While it is true that paragrpahs/tokens are generated from a lower dimensional manifold, it doesn’t automatically imply that these embeddings are linguistically interesting or allow for controlled planning-oriented generation. At an abstract conceptual level, how different are these representations from VAE based representation for example?\n\n– Although this is a generative model, the training objective is arbitrary and doesn’t seem linked to maximizing the likelihood/ learning the distribution of the training data. This is because the VAE and the diffusion components are separately trained and hence preclude a clean interpretation of the training objective.\n\n– I am not convinced that the “Distributional smoothness metric” reflect the nature of the manifold as is claimed in the paper.\n\n– Runtime analysis seems to compare unbatched autoregressive models with batched diffusion models. Sorting by length and then padding for batching is a standard practice for autoregressive models.\n\nQUESTIONS:\nSee above.\n\nLIMITATIONS:\nSee above.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this paper, the authors propose PLANNER, a model that combines latent semantic diffusion with autoregressive generation to generate fluent text while exercising global control over paragraphs. The proposed method is evaluated on various conditional generation tasks, and the results show its effectiveness in generating high-quality text.\n\nSTRENGTHS:\n1. The paper addresses the issue of repetitive and low-quality output generated by autoregressive models and proposes a novel approach using latent semantic diffusion.\n2. The combination of autoregressive decoding and latent diffusion allows for efficient generation over paragraph generation.\n3. The proposed method is evaluated on various tasks and shows improved generation quality compared to autoregressive and text diffusion baselines.\n\nWEAKNESSES:\n1. I understand that revisiting and revising the generated sentences can alleviate exposure bias, as errors can be reduced through further editing at the token level. However, revisiting and revising the latent space does not seem reasonable to me. From my perspective, the exposure bias occurs in the process of picking words out, but the authors merely employ GPT-2 without making any changes in the phase.\n2. The paper claims that they can generate longer text and paragraphs, but there is no further analysis about relationship between length of generated sentence and the performance. \n3. The evaluation of CNN/Daily Mail and XSum is performed on 256 subsampled examples from the test set. This manner is not convenient for the subsequent works to follow, and this also hinders reviewers from making fair comparisons between this work and others.\n4. I have reservations about the ability of the variational paragraph embedder to learn effective representations of sentences with different lengths. What would happen if there is a significant difference in sentence lengths, such as one being very short (e.g., 5 tokens) and the other being much longer (e.g., 512 tokens or more)?\n\nQUESTIONS:\n1. Unlike autoregressive models based on conditional probabilities, diffusion models are unable to ascertain the optimal sentence from the generated set. How did you address this problem in your paper?\n2. As the reverse diffusion goes on in latent space, does the latent representation z get closer to the representation of the ground-truth one in cosine distance or other distances? \n3. What is the influence of paragraph embeddings number, k in line 119?", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes to combine latent semantic diffusion with autoregressive generation to alleviate the issues of exposure bias in training / inference of text based language models, and computational and performance cost of purely diffusion approaches. They improve the diffusion process by applying it to the latent semantic space instead of the token / embedding space. For this, they learn some \"semantic tokens\" for encoding paragraph level information and then use a decoder to map these to the raw text space.\n\nSTRENGTHS:\nWorking with the diffusion model in the semantic spaces opens up the door for controllable generation.\nThey also provide an extensive study of the requirements for a good latent space for paragraph diffusion models.\nThe paper is well written and easy to follow.\nThe ideas employed to fix / ensure local smoothness (by perturbing data) and distributional smoothness (by using VAE) are simple and useful.\nThey propose a novel (?) metric called AuBLEU to evaluate the denoising capabilities of the model. I believe is this generally suitable for other works and could be impactful. However, it does not feel to be properly justified / grounded.\nHparams are provided.\n Human eval results are significant. The other results are sound and robust evaluation is performed. The paper is just lacking ablations on the design choices.\nThe analysis is complete and justifies the main claims made by the authors.\n\nWEAKNESSES:\nThe changes employed by the authors, especially during the training stage are not properly ablated.\nIt is not clear if the proposed fixes provide benefits.\nSome of the design choices are not experimentally justified (eg: line 182)\nEvaluation is performed on just a sample of the test set (this is the first time I have seen something like this in a paper and I'm not sure how to take it - I'm not super comfortable as this makes your technique essentially un-comparable). This also might not be robust. (line 196).\nNo intention of providing code and it might be very hard to reproduce because of the many changes especially in the training setup.\n\n\nNitpicks:\ntypo line 181\n\nQUESTIONS:\nCan you talk a bit more about the design choices and how much impact they had on the results? For eg: how much impact does fixing \"distributional smoothness\" have?\nHave you performed other ablation experiments to justify claims?\nWhy did you subsample the test sets?\n\nLIMITATIONS:\nAddressed in appendix.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-15"}