Title: Meta-Diffu$B$: A Contextualized Sequence-to-Sequence Text Diffusion Model with Meta-Exploration
Venue: NeurIPS 2024
Accepted: yes
Keywords: Diffusion Models

Sequence-to-Sequence

Text Generation

Meta-Exploration

Noise Scheduling
Abstract: The diffusion model, a new generative modeling paradigm, has achieved significant success in generating images, audio, video, and text. It has been adapted for sequence-to-sequence text generation (Seq2Seq) through DiffuSeq, termed the S2S-Diffusion model. Existing S2S-Diffusion models predominantly rely on fixed or hand-crafted rules to schedule noise during the diffusion and denoising processes. However, these models are limited by non-contextualized noise, which fails to fully consider the characteristics of Seq2Seq tasks. In this paper, we propose the Meta-Diffu$B$ framework—a novel scheduler-exploiter S2S-Diffusion paradigm designed to overcome the limitations of existing S2S-Diffusion models. We employ Meta-Exploration to train an additional scheduler model dedicated to scheduling contextualized noise for each sentence. Our exploiter model, an S2S-Diffusion model, leverages the noise scheduled by our scheduler model for updating and generation. Meta-Diffu$B$ achieves state-of-the-art performance compared to previous S2S-Diffusion models and fine-tuned pre-trained language models (PLMs) across four Seq2Seq benchmark datasets. We further investigate and visualize the impact of Meta-Diffu$B$'s noise scheduling on the generation of sentences with varying difficulties. Additionally, our scheduler model can function as a "plug-and-play" model to enhance DiffuSeq without the need for fine-tuning during the inference stage.

Meta-Diffu B : A Contextualized Sequence-to-Sequence Text Diffusion Model with Meta-Exploration

Abstract

The diffusion model, a new generative modeling paradigm, has achieved significant success in generating images, audio, video, and text. It has been adapted for sequence-to-sequence text generation (Seq2Seq) through DiffuSeq, termed the S2S-Diffusion model. Existing S2S-Diffusion models predominantly rely on fixed or hand-crafted rules to schedule noise during the diffusion and denoising processes. However, these models are limited by non-contextualized noise, which fails to fully consider the characteristics of Seq2Seq tasks. In this paper, we propose the Meta-Diffu B framework-a novel scheduler-exploiter S2S-Diffusion paradigm designed to overcome the limitations of existing S2S-Diffusion models. We employ Meta-Exploration to train an additional scheduler model dedicated to scheduling contextualized noise for each sentence. Our exploiter model, an S2SDiffusion model, leverages the noise scheduled by our scheduler model for updating and generation. Meta-Diffu B achieves state-of-the-art performance compared to previous S2S-Diffusion models and fine-tuned pre-trained language models (PLMs) across four Seq2Seq benchmark datasets. We further investigate and visualize the impact of Meta-Diffu B 's noise scheduling on the generation of sentences with varying difficulties. Additionally, our scheduler model can function as a "plugand-play" model to enhance DiffuSeq without the need for fine-tuning during the inference stage. 1

1 Introduction

The diffusion model, a novel generative approach, operates through a two-step process: it first introduces noise to real data and then systematically removes this noise to facilitate data generation [12, 40, 30]. This model has demonstrated significant efficacy across several domains, including image [13, 29, 38], audio [36, 18], video [18, 14], and text generation [1, 15, 21, 3, 24, 34]. The diffusion model utilizes a technique known as noise scheduling to control the amount of noise imposed at each diffusion step [12]. DiffuSeq [8] has adapted this model to discrete generation tasks like sequence-to-sequence text generation (Seq2Seq), under a framework termed S2S-Diffusion. However, DiffuSeq employs fixed noise scheduling and does not accommodate the specific characteristics of Seq2Seq tasks [45, 44].

Seq2Seq is a foundational technique in natural language processing (NLP) that generates target sentences from specified conditional sentences. It supports a range of downstream tasks, including

,

language translation [41], image captioning [35], conversational modeling [39], and text summarization [28]. For Seq2Seq tasks, it is more reasonable to impose different levels of noise to each sentence in S2S-Diffusion models to address the varying semantic and contextual difficulties of generating sentences. This noise scheduling strategy can better adapt to the semantic characteristics and generation difficulties of each sentence, thereby improving the model's performance in various generation tasks. To meet the unique demands of S2S Diffusion, we introduce a contextualized noise-scheduling strategy that accounts for the semantics of each conditional sentence and adapts to different training epochs. Existing S2S-Diffusion models, such as DiffuSeq, lack flexibility due to their reliance on fixed, non-contextualized noise-scheduling strategies. Furthermore, models like SeqDiffuSeq [45] and Dinoiser [44], which propose adaptive noise scheduling, are also limited by their non-contextualized approach.

To address the semantics of discrete conditional sentences for contextualized noise scheduling, we introduce a novel scheduler-exploiter framework, Meta-Diffu B , which achieves trainable noisescheduling inspired by Meta-Exploration [43]. Within this framework, our scheduler model dynamically schedules noise to train our exploiter model, which is updated based on the performance rewards it generates. Our exploiter model, an S2S-Diffusion model, leverages the noise scheduled by the scheduler model for updates and generation. By design, Meta-Diffu B naturally implements contextualized noise scheduling. It achieves state-of-the-art performance on four Seq2Seq benchmark datasets, outperforming existing S2S-Diffusion models [8, 45, 44] and fine-tuned pre-trained language models (PLMs) [10, 33].

In summary, we make three primary contributions with Meta-Diffu B :

We introduce and demonstrate the application of Meta-Exploration to diffusion models in Section 3, proposing Meta-Diffu B as a strategy to enhance S2S-Diffusion models. Our main results, presented in Section 6.1, confirm that Meta-Diffu B achieves state-of-the-art performance across four benchmark datasets.

We detail the operation of our scheduler model in Section 6.2, highlighting its capability to schedule noise. The noise scheduling approach of our scheduler model-applying less noise to the harder sentences and more to the easier ones-enhances the diversity and quality of the generated text.

Wereveal that our scheduler model can function as a "plug-and-play" model, easily integrated into existing S2S-Diffusion models to enhance inference performance, as detailed in Section 6.3.

2 Problem Statement, Preliminary

2.1 Problem Statement

In this work, we focus on sequence-to-sequence text generation tasks. Given a conditioning sentence of length m , w x = { w x 1 , . . . , w x m } , our objective is to train a diffusion model capable of generating a target sentence of length n , w y = { w y 1 , . . . , w y n } , based on the conditional sentence. Here, w x and w y represent the conditional and target sentences, respectively.

2.2 Preliminary

DiffuSeq [8] primarily follows the transformation method of Diffusion-LM [21] and incorporates the diffusion and denoising processes from [12]. In the diffusion process, Diffusion-LM transforms discrete sentences into a continuous space. Given the real-world training sentence pair w x ⊕ y , concatenated by w x and w y , Diffusion-LM uses an embedding function emb to transform w x ⊕ y into continuous space, thereby obtaining the distribution z 0 ∼ q ( z ) , where q represents the diffusion process. Then, z 0 is subjected to imposed noise, diffusing into a standard Gaussian distribution z T ∼ N (0 , I ) . At each diffusion step t ∈ [1 , 2 , . . . , T ] , the noise is regulated by q ( z t | z t -1 ) = N ( z t ; √ 1 -β t z t -1 , β t I ) , where β t ∈ (0 , 1) controls the amount of noise imposed at each diffusion step. We denote β as containing a set of noise values β t , where a larger β t indicates more Gaussian noise imposed at that diffusion step. When t is large enough, z 0 gradually evolves into a standard Gaussian noise distribution. The random distribution is gradually reduced in noise during the denoising process to regenerate target sentences. The denoising process, which recovers z 0 by

Fixednoise

DiffusionProcess

DenoisingProcess

conditionalsentence

Bsqrt

targetsentence

Zo

Zt

ZT

(a)PreviousS2s-Diffusionmodel

Schedule contextualized noisewith the schedulermodelby Eq.(3).

DiffusionProcess

DenoisingProcess

schedulermodel

Encoder

B1

B2

Decoder

B3

Zo

Zt

(b) Meta-DiffuB

reducing the noise in z t , can be defined as follows:

Diffusion-LM employs a trained, parameterized denoising distribution z t -1 ∼ p θ ( z t -1 | z t ) to gradually recover z t from noise. This denoising distribution, parameterized by θ , is tailored to fit the posterior distribution q ( z t -1 | z t , z 0 ) of the forward process. The key difference between DiffuSeq [8] and Diffusion-LM [21] is that DiffuSeq imposes noise only on the target sentence part of z t to achieve classifier-free S2S Diffusion, termed Partial Noise [8]. Due to the implementation of Partial Noise in the diffusion process, conditional denoising is inherently classifier-free. To transform the continuous z 0 target sentences back into discrete sentences w y , previous S2S-Diffusion models use a Rounding Operation [21] to map the target sentence part of z 0 into w y . The Rounding Operation is a method for choosing the most probable word for each position [21]. The denoising process primarily utilizes the variational lower bound ( L VLB ) to optimize the negative log-likelihood [12]. Through the simplification and derivation from DiffuSeq [8], the training objective function for S2S-Diffusion models can be defined as:

where learning process p θ ( z t -1 | z t ) is modeled as Transformer model f θ . Previous diffusion models deploy β by dividing the interval between the minimum value β 1 and the maximum value β T using a mathematical function to determine the fixed noise sequence { β 1 , ..., β T } ∈ β , as described in [12]. The mathematical function used by Diffusion-LM and DiffuSeq [21] is the sqrt function, which has demonstrated superior performance in text generation compared to other fixed mathematical functions. However, DiffuSeq's noise scheduling is constrained by its non-contextual approach; it does not account for the semantics of each conditional sentence nor does it adapt to different training epochs.

3 Methodology

In this work, we propose a scheduler-exploiter framework named Meta-Diffu B for training S2SDiffusion models with contextualized noise. Inspired by [43], our Meta-Diffu B includes a scheduler model, B ψ , parameterized by ψ , and an exploiter model, D θ , parameterized by θ . B ψ , a simple Seq2Seq model, considers the semantics of conditional sentences to schedule contextualized β for updating D θ and is also updated based on the learning effectiveness of D θ -which refers to how well the exploiter learns. Our exploiter, D θ , an S2S-Diffusion model, leverages the noise scheduled by B ψ for its updating and generation. The framework of our Meta-Diffu B , compared with DiffuSeq, is visualized in Figure 1.

3.1 Noise Scheduling in the Scheduler Model

In this work, we propose a simple two-step approach for our scheduler B ψ , which is a Seq2Seq model, to schedule β -a set of noise values β t . Here, a larger β t indicates more noise imposed on the data. The input to B ψ is consistently w x across both training and inference stages. Instead of directly scheduling the values of β , B ψ outputs a series of Meta-Instructions, simplifying the training into a time-series binary classification problem. In the first step, B ψ samples a series of Meta-Instructions ι x = { ι 1 , . . . , ι t , . . . , ι T } from w x , where each ι t is labeled either True or False. We propose a 'skipping' method: a True label directs B ψ to increase the noise by selecting β t +1 for the next diffusion step, whereas a False label maintains the same noise level β t . In the second step, we transform ι x using the fixed noise sqrt -function β sqrt = { β 1 , . . . , β T } , as deployed by [21, 8], through the 'skipping' method to generate the new noise values β x = { β x 1 , . . . , β x T } . For example, with the continuous Meta-Instructions ι x = { T, F, T } and fixed noise values { 1 , 2 , 3 } , our new scheduling of noise values will be { 1 , 1 , 2 } . If consecutive scheduling noise values are the same, no additional noise is introduced at that diffusion step [12]. Our two-step approach maintains the same diffusion steps for parallel operations and contextualized β x in the diffusion process. We utilize a Policy Gradient to update our scheduler model following Meta-Exploration, addressing the non-differentiability of our two-step approach. The noise-scheduling mechanism of our scheduler model can be defined by the following equations:

3.2 Training the Exploiter

Unlike previous S2S-Diffusion models [8, 45, 44] that employ fixed or hand-crafted noise scheduling, we utilize contextualized β x to impose noise during the diffusion process. We also implement Partial Noise to achieve classifier-free S2S Diffusion [8]. In the denoising process, our exploiter model D θ restores the diffused data to generate the target sentences. During the diffusion process, we adopt the transformation method of Diffusion-LM to obtain emb( w x ⊕ y ) , as described in Section 2. We extend the original diffusion chain to a new Markov transition with our β x : q ϕ ( z 0 | w x ⊕ y ) = N (emb( w x ⊕ y ) , β x 0 I ) [21, 8]. Consequently, we can implement the objective function indicated in Section 2, derived from previous classifier-free S2S-Diffusion methods, to update our exploiter model D θ [8]. The training objective function for our exploiter model D θ in collaboration with B ψ can be defined as follows:

Since z 0 is not diffused, there is no need to add the superscript of B ψ . J ( θ ) is denoted as the gradient for updating exploiter model D θ . Then, we can update our exploiter model D θ 's network weights:

3.3 Contextualized Inference with Meta-Diffu B

In the inference stage, if our goal is to generate outputs based on w x , B ψ predicts contextualized β x using w x , as demonstrated in Section 3.1. We then concatenate emb( w x ) -transformed from

Algorithm 1 Meta-Diffu B

Require: exploiter model D θ ; scheduler model B ψ ; conditional sentences w x and target sequences w y from dataset.

Initialize exploiter model D θ , scheduler model B ψ with random weights θ and ψ .

repeat

until w x -with a randomly sampled y T ∼ N (0 , I ) to form z T . Our D θ predicts z 0 directly from z t and uses β x to convert the predicted z 0 into z t -1 . This step-by-step denoising process progressively recovers z t back to z 0 , following the methodologies outlined in [21, 45, 44]. Finally, we use a Rounding Operation to convert the target sentence part of z 0 into discrete target sentences.

3.4 Estimating the Meta-Reward of the Scheduler Model

In this section, we estimate the Meta-Reward of our scheduler model, which reflects the learning effectiveness of D θ . We let D θ generate Y D θ and D θ ′ generate Y D θ ′ , respectively, where Y denotes the generated w y [21]. We assess the rewards for Y D θ and Y D ′ θ , denoted as R D θ and R D θ ′ respectively, which represent the rewards for D θ and D θ ′ . In this study, we utilize the BLEU score to quantify these rewards. Consequently, the reward for the scheduler model (i.e., Meta-Reward) is defined as R B ψ = R D θ ′ -R D θ .

3.5 Training the Scheduler Model with Meta-Reward

Since B ψ generates Meta-Instructions to diffuse sentences using our two-step approach described in Section 3.1, we update the scheduler via policy gradients, incorporating both Meta-Instructions and the calculated Meta-Rewards [43]. The training objective function for our scheduler model is defined as follows:

After we obtain ∇ ψ J ( ψ ) , we can update B ψ 's network weights:

3.6 Exploration Epochs

Inspired by the exploration epochs of Meta-Exploration [43, 5, 19], we iteratively execute the procedures from Section 3.1 to Section 3.4 to collect various indicators of learning effectiveness from D θ for updating B ψ . In practice, we keep the network weights of D θ fixed until the exploration epochs are completed. This approach ensures that the scheduler model schedules noise to D θ with consistent network weights, promoting stable training [5]. Additionally, we can conduct the exploration epochs in parallel to save time by collecting learning effectiveness from D θ under consistent network weights. After accumulating the gradients for B ψ from these exploration epochs, we update B ψ to B ψ ′ , which in turn schedules new noise to update D θ to D θ ′ . In summary, we present Algorithm (1) to detail the full training process of the proposed Meta-Diffu B . The number of exploration epochs is denoted by E , with e ∈ { 1 , ..., E} indexing the exploration epochs.

4 Experiments

In this section, we conduct experiments to verify the performance of our Meta-Diffu B on four benchmark Seq2Seq datasets [48, 6, 17, 8]. We benchmark Meta-Diffu B against previous S2SDiffusion models and fine-tuned pre-trained language models (PLMs), using the same datasets and training settings as employed by DiffuSeq [8].

4.1 Datasets

In our experiment, we use four datasets: the Commonsense Conversation dataset (CC) [48], the Quasar-T dataset (QT) [6], the Wiki-Auto dataset (WA) [17], and the Quora Question Pairs dataset (QQP) [8]. These datasets consider a variety of tasks, including open-domain dialogue generation, question generation, text simplification, and paraphrase generation tasks, all within Seq2Seq contexts. For a fair comparison, we employ the same datasets with identical settings for training all mentioned models, as outlined in [8, 45]. Detailed settings of these datasets are provided in Appendix A.

4.2 Baselines

We compare the proposed Meta-Diffu B with previous S2S-Diffusion models, including DiffuSeq [8], Dinoiser [44], and SeqDiffuSeq [45]. DiffuSeq employs a fixed noise pattern in the training and inference stages using a sqrt function and has been successfully introduced to the Seq2Seq task as the basic diffusion model. We also compare Meta-Diffu B with Dinoiser and SeqDiffuSeq, which are existing S2S-Diffusion models that focus on noise scheduling. Dinoiser and SeqDiffuSeq utilize handcrafted rules that provide adaptive but not contextualized noise scheduling. Additionally, following [8, 45], we compare our Meta-Diffu B with three PLMs on Seq2Seq tasks. These PLMs include the fine-tuned GPT-2-base (GPT2-base) [33], fine-tuned GPT-2-large (GPT2-large), and fine-tuned Levenshtein Transformer (LevT) [10]. We detail these baselines in Appendix B.

4.3 Training Setting

Our exploiter model employs the same network architecture and settings as DiffuSeq [8]. Our scheduler model uses the same network architecture as described in [4]. The exploiter model and scheduler architectures are detailed in Appendix C. For consistent comparison, all S2S-Diffusion models [8, 44, 45] follow the experimental settings of prior research [8] and are trained from scratch. The diffusion step count is set at 2,000, and the maximum sequence length is 128. The Minimum Bayes risk (MBR) [23] decoding size, denoted as | S | , is 10; this involves generating sentences from 10 random seeds and selecting the best output sequence. Details on the implementation of MBR for all S2S-Diffusion models can be found in Appendix 6. The total batch size for both training and testing phases is 2048. Experiments are conducted on NVIDIA A100 Tensor Core GPUs, utilizing 4 GPUs for training and a single GPU for inference.

4.3.1 Discussion of Computational Intensity

To ensure a fair comparison during parallel exploration epochs, we avoid increasing the total batch size. Instead, we reduce the batch size by dividing the total batch size by the number of exploration epochs deployed. In this work, we set the number of exploration epochs to 32 and the batch size to 64. To update our scheduler, we run parallel exploration epochs every 100 training epochs with a total batch size of 2048. The increased computational complexity of applying Meta-Diffu B to DiffuSeq is presented in Table 1.

4.4 Evaluation Metrics

To ensure a fair comparison, we follow the same evaluation metric settings as those used in previous S2S-Diffusion models [8, 45]. For quality assessment, we utilize standard text generation metrics such

as BLEU [27], ROUGE-L [2], and BERTScore [46], where higher scores indicate better performance. For diversity assessment, we apply general text generation diversity metrics, including Distinct Unigram (Dist-1)[2] and Self-BLEU[27], where lower scores of Self-BLEU and higher scores of Dist1 signify better performance. Due to the application of multiple evaluation metrics (such as BLEU, ROUGE-L, BERTScore, Dist-1, and Self-BLEU), we also use Mean-Rank (M-R) to measure whether each model performs the best across multiple metrics [20]. A lower Mean-Rank score indicates consistently better performance across various metrics in the dataset. Details on the evaluation metric settings and their explanations are provided in Appendix D.

5 Model-Agnostic Characteristics of Meta-Diffu B

We conduct experiments on applying our Meta-Diffu B to other S2S-Diffusion models. Specifically, we use Meta-Diffu B to modify the handcrafted noise-scheduling strategies of Dinoiser [44] and SeqDiffuSeq [45] on the WA and QQP datasets. The results, shown in Table 2, demonstrate that Meta-Diffu B can be considered a model-agnostic method for enhancing the performance of other S2S-Diffusion models. Additionally, we provide results for applying our Meta-Diffu B to RDM [47] (based on D3PM [47]) and other recent S2S-Diffusion models [42, 7, 22, 9], which are based on DiffuSeq [8] on machine translation datasets [31, 26] in Appendix E.

6 Experiments of Minimum Bayes Risk Decoding

Diffusion-LM proposes using Minimum Bayes Risk (MBR) to improve generation. Following the methods described in [45, 8], we allow all S2S-Diffusion models to generate a set of candidate sentences from 10 random seeds and select the best output sequence that achieves the minimum expected risk under a meaningful loss function. Specifically, in this work, we employ the BLEU score as our loss function to evaluate performance, following the approach used in DiffuSeq [8]. We compare our Meta-Diffu B ( D θ = DiffuSeq) with DiffuSeq [8] and GPT-2 [33], using MBR decoding [21, 8, 45] on the WA and QQP datasets as described in DiffuSeq [8]. We specifically select GPT2-large and GPT2-base for comparison based on their superior performance on these datasets [8]. In this experiment, we apply MBR decoding to all three models while gradually increasing the candidate sentence size | S | . The results of the MBR decoding are presented in Figure 2.

Figure 2 shows that our Meta-Diffu B ( D θ = DiffuSeq) can generate a more diverse array of candidate sentences, achieving better results as the candidate size | S | increases. The diversity of these candidate sentences determines the upper bound of MBR performance [21, 8]. Our Meta-Diffu B ( D θ = DiffuSeq) consistently outperforms both GPT2-base and DiffuSeq across all candidate size settings

1

3

5

7

10

15

20

candidate size |

S

|

0.19

0.20

0.21

0.22

0.23

0.24

0.25

0.26

BLEU

DiffuSeq

GPT2-large

Meta-Diffu

B

(a) QQP dataset

1

3

5

7

10

15

20

candidate size |

S

|

0.30

0.32

0.34

0.36

0.38

BLEU

DiffuSeq

Meta-Diffu

B

(b) WA dataset

on the QQP dataset. As the candidate size grows, Meta-Diffu B ( D θ = DiffuSeq) also surpasses GPT2-base on the WA dataset. Notably, Meta-Diffu B ( D θ = DiffuSeq) exhibits diverse generation capabilities and achieves significantly better performance than DiffuSeq in MBR experiments.

6.1 Experiment with Seq2Seq Benchmark Datasets

We demonstrate the performance of our Meta-Diffu B ( D θ = DiffuSeq) in Table 3. This table shows that Meta-Diffu B ( D θ = DiffuSeq) outperforms other PLMs [33, 10] and S2S-Diffusion models [8, 44, 45] in terms of generation quality and diversity, achieving the lowest M-R scores across four datasets. Moreover, Meta-Diffu B ( D θ = DiffuSeq) demonstrates significant improvements over previous S2S-Diffusion models [8, 45, 44] on all evaluation metrics considered.

6.2 Contextualized Noise Scheduling of Meta-Diffu B

For the experiment involving contextualized noise of Meta-Diffu B ( D θ = DiffuSeq), we selected the 200 hardest and 200 easiest generated sentences, labeled as (H) and (E), respectively. All models listed in Table 3 assessed the generation difficulty of each sentence using BLEU scores, with lower BLEU indicating higher difficulty. The performance of generating (H) and (E) is evaluated in terms of BLEU and Self-BLEU, as shown in Table 4. We also detail the results of Table 4 evaluated by other metrics in Appendix G. We tasked all S2S-Diffusion models with scheduling noise for sentences (H) and (E), as shown in Figure 3. Since our Meta-Diffu B ( D θ = DiffuSeq) assigns specific noise to each sentence, we averaged the noise values for clearer visualization. Figure 3 displays the last 10 diffusion steps, as the noise differences in the initial steps are minimal. In Table 4, Meta-Diffu B ( D θ = DiffuSeq) consistently outperforms other S2S-Diffusion models [44, 45, 8] in terms of generation quality and diversity for sentences (E) and (H). Notably, when generating the more challenging sentences (H), Meta-Diffu B Meta-Diffu B ( D θ = DiffuSeq) maintains its performance, whereas other S2S-Diffusion models [44, 45, 8] experience a decline in both quality and diversity. Figure 3 shows the benefits of Meta-Diffu B ( D θ = DiffuSeq), which strategically imposes different noise levels for sentences (E) and (H). This noise-scheduling approach-applying less noise to the harder sentences (H) and more to the easier sentences (E)-enhances the diversity and quality of the generated text. The superior example of Meta-Diffu B in generating the hardest sentences (H) is further showcased in Appendix F, demonstrating its performance relative to other S2S-Diffusion models [44, 45, 8]. We also provide a more detailed discussion about the noise strategy of our Meta-Diffu B in Appendix H.

6.3 Plug-and-Play Experiments with the Scheduler Model

Our pre-trained scheduler model, trained under Meta-Diffu B ( D θ = DiffuSeq), which incorporates the semantics of discrete sentences, demonstrates its effectiveness by scheduling noise for pre-trained DiffuSeq [8] models across various datasets. The results, presented in Table 5, show that our pre-trained scheduler model, when applied across different datasets, enhances the performance of pre-trained DiffuSeq models without any fine-tuning during the inference stage. This confirms that our scheduler model can function as a plug-and-play model across these datasets. Additionally, we provide further experiments on different pre-trained schedulers under various S2S-Diffusion settings, as well as results on additional datasets in Appendix I.

7 Related Works

7.1 Text Diffusion

[11, 1] define an absorbing state for generating discrete data. Diffusion-LM [21] and AnalogBits [3] propose imposing noise on continuous latent representations, using transformation functions to bridge the discrete and continuous spaces of texts for both unconditional and controlled text generation.

7.2 Meta-Exploration

To transcend the limitations imposed by human-crafted rules in noise scheduling, we developed an additional model trained through Meta-Exploration, as inspired by [43]. Meta-Exploration is a Reinforcement Learning (RL) training method that utilizes learning effectiveness to devise sampling strategies that enhance model performance. Numerous studies [5, 37, 25, 19, 16] have employed Meta-Exploration to meta-learn scheduling strategies for applying additive Gaussian noise on actions and for sampling effective training data in RL tasks. We have adopted the Meta-Exploration concept [43] to train an additional model specifically for noise scheduling in S2S-Diffusion.

8 Broader Impact

In this work, our Meta-Diffu B demonstrates significant performance improvements over previous S2S-Diffusion models across four Seq2Seq tasks, as detailed in Section 4.1. Meta-Diffu B implements

1992

1994

1996

1998

2000

diffusion steps

0.2

0.4

0.6

0.8

1.0

t

DiffuSeq

Dinoiser

SeqDiffuSeq

Meta-Diffu

B

(E)

Meta-Diffu

B

(H)

(a) QQP dataset

1992

1994

1996

1998

2000

diffusion steps

0.2

0.4

0.6

0.8

1.0

t

DiffuSeq

Dinoiser

SeqDiffuSeq

Meta-Diffu

B

(E)

Meta-Diffu

B

(H)

(b) WA dataset

learnable, contextualized noise scheduling for Seq2Seq tasks. It not only shows enhanced generation quality and diversity but also has the potential to be applied to other diffusion models that require conditional data learning to generate target data. However, it is important to note that using MetaDiffu B to create fake news or other forms of misinformation is strongly discouraged.

9 Conclusions

We propose integrating Meta-Exploration into S2S-Diffusion models through our newly developed Meta-Diffu B . By utilizing Meta-Exploration to schedule contextualized noise, our Meta-Diffu B model demonstrates significant performance improvements on four Seq2Seq benchmark datasets compared to previous S2S-Diffusion models and PLMs. We have conducted a comprehensive investigation of the noise-scheduling capabilities of Meta-Diffu B and have visualized the results. Importantly, Meta-Diffu B has the potential to act as a plug-and-play model, providing a promising approach for enhancing other S2S-Diffusion models during the inference stage without the need for fine-tuning.

10 Acknowledgments

We would like to express our sincere gratitude to Professor Hung-yi Lee from NTU Speech Lab for his invaluable guidance and insightful advice throughout this work. We are also deeply grateful to Professor Ray-I Chang from NTU ICAN Lab for his mentorship and constructive feedback. Additionally, we would like to thank the reviewers for their positive evaluation and valuable suggestions. Finally, we extend our appreciation to Maxora AI for providing the computational resources and environment that made this research possible, enabling us to make meaningful contributions to the field.

References

Jacob Austin, Daniel D Johnson, Jonathan Ho, Daniel Tarlow, and Rianne Van Den Berg. Structured denoising diffusion models in discrete state-spaces. Advances in Neural Information Processing Systems , 34:17981-17993, 2021.

Asli Celikyilmaz, Elizabeth Clark, and Jianfeng Gao. Evaluation of text generation: A survey. arXiv preprint arXiv:2006.14799 , 2020.

Ting Chen, Ruixiang ZHANG, and Geoffrey Hinton. Analog bits: Generating discrete data using diffusion models with self-conditioning. In The Eleventh International Conference on Learning Representations , 2022.

Kyunghyun Cho, Bart Van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. Learning phrase representations using rnn encoderdecoder for statistical machine translation. arXiv preprint arXiv:1406.1078 , 2014.

Yun-Yen Chuang, Hung-Min Hsu, Kevin Lin, Ray-I Chang, and Hung-Yi Lee. Metaex-gan: Meta exploration to improve natural language generation via generative adversarial networks. IEEE/ACM Transactions on Audio, Speech, and Language Processing , 2023.

Bhuwan Dhingra, Kathryn Mazaitis, and William W Cohen. Quasar: Datasets for question answering by search and reading. arXiv preprint arXiv:1707.03904 , 2017.

Zhujin Gao, Junliang Guo, Xu Tan, Yongxin Zhu, Fang Zhang, Jiang Bian, and Linli Xu. Empowering diffusion models on the embedding space for text generation. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) , pages 4664-4683, 2024.

Shansan Gong, Mukai Li, Jiangtao Feng, Zhiyong Wu, and Lingpeng Kong. Diffuseq: Sequence to sequence text generation with diffusion models. In The Eleventh International Conference on Learning Representations (ICLR) , 2023.

Shansan Gong, Mukai Li, Jiangtao Feng, Zhiyong Wu, and Lingpeng Kong. Diffuseq-v2: Bridging discrete and continuous text spaces for accelerated seq2seq diffusion models. In Findings of the Association for Computational Linguistics: EMNLP 2023 , pages 9868-9875, 2023.

Jiatao Gu, Changhan Wang, and Junbo Zhao. Levenshtein transformer. Advances in neural information processing systems , 32, 2019.

Zhengfu He, Tianxiang Sun, Qiong Tang, Kuanning Wang, Xuan-Jing Huang, and Xipeng Qiu. Diffusionbert: Improving generative masked language models with diffusion models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages 4521-4534, 2023.

Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems , 33:6840-6851, 2020.

Jonathan Ho, Chitwan Saharia, William Chan, David J Fleet, Mohammad Norouzi, and Tim Salimans. Cascaded diffusion models for high fidelity image generation. Journal of Machine Learning Research , 23(47):1-33, 2022.

Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J Fleet. Video diffusion models. Advances in Neural Information Processing Systems , 35:8633-8646, 2022.

Emiel Hoogeboom, Didrik Nielsen, Priyank Jaini, Patrick Forré, and Max Welling. Argmax flows and multinomial diffusion: Learning categorical distributions. Advances in Neural Information Processing Systems , 34:12454-12465, 2021.

Timothy Hospedales, Antreas Antoniou, Paul Micaelli, and Amos Storkey. Meta-learning in neural networks: A survey. IEEE transactions on pattern analysis and machine intelligence , 44(9):5149-5169, 2022.

Chao Jiang, Mounica Maddela, Wuwei Lan, Yang Zhong, and Wei Xu. Neural crf model for sentence alignment in text simplification. arXiv preprint arXiv:2005.02324 , 2020.

Zhifeng Kong, Wei Ping, Jiaji Huang, Kexin Zhao, and Bryan Catanzaro. Diffwave: A versatile diffusion model for audio synthesis. arXiv preprint arXiv:2009.09761 , 2020.

Kwei-Herng Lai, Daochen Zha, Yuening Li, and Xia Hu. Dual policy distillation. arXiv preprint arXiv:2006.04061 , 2020.

Linjie Li, Jie Lei, Zhe Gan, Licheng Yu, Yen-Chun Chen, Rohit Pillai, Yu Cheng, Luowei Zhou, Xin Eric Wang, William Yang Wang, et al. Value: A multi-task benchmark for video-andlanguage understanding evaluation. arXiv preprint arXiv:2106.04632 , 2021.

Xiang Li, John Thickstun, Ishaan Gulrajani, Percy S Liang, and Tatsunori B Hashimoto. Diffusion-lm improves controllable text generation. Advances in Neural Information Processing Systems , 35:4328-4343, 2022.

Zihao Li, Aixin Sun, and Chenliang Li. Diffurec: A diffusion model for sequential recommendation. ACM Transactions on Information Systems , 42(3):1-28, 2023.

Chin-Yew Lin. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out , pages 74-81, 2004.

Zhenghao Lin, Yeyun Gong, Yelong Shen, Tong Wu, Zhihao Fan, Chen Lin, Nan Duan, and Weizhu Chen. Text generation with diffusion language models: A pre-training approach with continuous paragraph denoise. In International Conference on Machine Learning , pages 21051-21064. PMLR, 2023.

Iou-Jen Liu, Unnat Jain, Raymond A Yeh, and Alexander Schwing. Cooperative exploration for multi-agent deep reinforcement learning. In International Conference on Machine Learning , pages 6826-6836. PMLR, 2021.

Jan Niehues Mauro Cettolo, Luisa Bentivogli Sebastian Stüker, and Marcello Federico. Report on the 11th iwslt evaluation campaign. Proceedings of the 11th International Workshop on Spoken Language Translation: Evaluation Campaign , pages 2-17, 2014.

Ehsan Montahaei, Danial Alihosseini, and Mahdieh Soleymani Baghshah. Jointly measuring diversity and quality in text generation models. arXiv preprint arXiv:1904.03971 , 2019.

Ramesh Nallapati, Bowen Zhou, Cicero dos Santos, Çaglar Gulçehre, and Bing Xiang. Abstractive text summarization using sequence-to-sequence rnns and beyond. CoNLL 2016 , page 280, 2016.

Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen. Glide: Towards photorealistic image generation and editing with text-guided diffusion models. arXiv preprint arXiv:2112.10741 , 2021.

Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In International conference on machine learning , pages 8162-8171. PMLR, 2021.

Christian Buck Ondrej Bojar, Barry Haddow Christian Federmann, Johannes Leveling Philipp Koehn, Pavel Pecina Christof Monz, Herve Saint-Amand Matt Post, Lucia Specia Radu Soricut, and Ale s Tamchyna. Findings of the 2014 workshop on statistical machine translation. Proceedings of the ninth workshop on statistical machine translation , pages 12-58, 2014.

Matt Post. A call for clarity in reporting bleu scores. Proceedings of the Third Conference on Machine Translation: Research Papers , pages 186-191, 2018.

Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAI blog , 1(8):9, 2019.

Machel Reid, Vincent J Hellendoorn, and Graham Neubig. Diffuser: Discrete diffusion via edit-based reconstruction. arXiv preprint arXiv:2210.16886 , 2022.

Steven J Rennie, Etienne Marcheret, Youssef Mroueh, Jerret Ross, and Vaibhava Goel. Selfcritical sequence training for image captioning. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 7008-7024, 2017.

Ludan Ruan, Yiyang Ma, Huan Yang, Huiguo He, Bei Liu, Jianlong Fu, Nicholas Jing Yuan, Qin Jin, and Baining Guo. Mm-diffusion: Learning multi-modal diffusion models for joint audio and video generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 10219-10228, 2023.

Baturay Saglam and Suleyman S Kozat. Deep intrinsically motivated exploration in continuous control. Machine Learning , 112(12):4959-4993, 2023.

Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. Advances in neural information processing systems , 35:36479-36494, 2022.

Yuanlong Shao, Stephan Gouws, Denny Britz, Anna Goldie, Brian Strope, and Ray Kurzweil. Generating high-quality and informative conversation responses with sequence-to-sequence models. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing , pages 2210-2219, 2017.

Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In International Conference on Learning Representations , 2020.

Ilya Sutskever, Oriol Vinyals, and Quoc V Le. Sequence to sequence learning with neural networks. Advances in neural information processing systems , 27, 2014.

Zecheng Tang, Pinzheng Wang, Keyan Zhou, Juntao Li, Ziqiang Cao, and Min Zhang. Can diffusion model achieve better performance in text generation? bridging the gap between training and inference! In Findings of the Association for Computational Linguistics: ACL 2023 , pages 11359-11386, 2023.

Tianbing Xu, Qiang Liu, Liang Zhao, and Jian Peng. Learning to explore with meta-policy gradient. arXiv preprint arXiv:1803.05044 , 2018.

Jiasheng Ye, Zaixiang Zheng, Yu Bao, Lihua Qian, and Mingxuan Wang. Dinoiser: Diffused conditional sequence learning by manipulating noises. Transactions of the Association for Computational Linguistics (TACL) , 2024.

Hongyi Yuan, Zheng Yuan, Chuanqi Tan, Fei Huang, and Songfang Huang. Text diffusion model with encoder-decoder transformers for sequence-to-sequence generation. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) , pages 22-39, 2024.

Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. Bertscore: Evaluating text generation with bert. arXiv preprint arXiv:1904.09675 , 2019.

Lin Zheng, Jianbo Yuan, Lei Yu, and Lingpeng Kong. A reparameterized discrete diffusion model for text generation. Conference on Language Modeling: COLM 2024 , 2024.

Yaoming Zhu, Sidi Lu, Lei Zheng, Jiaxian Guo, Weinan Zhang, Jun Wang, and Yong Yu. Texygen: A benchmarking platform for text generation models. In The 41st international ACM SIGIR conference on research & development in information retrieval , pages 1097-1100, 2018.

A Details of Datasets

There are four datasets in our experiment. For a fair comparison, we use the same datasets with the same settings as those described in [8, 45]. Below, we detail the datasets used:

Commonsense Conversation Dataset (CC) [48]: CC requires models to generate informative responses given a dialogue context, an open-domain dialogue generation task. Extracted from Reddit single-round dialogues, it includes over 3 million conversational pairs. The training set contains 3,382,137 pairs, the development set has 2,048, and the test set includes 10,000 pairs. We train all S2S-Diffusions for 140,000 epochs on this dataset.

Quasar-T Dataset (QT) [6]: QT requires models to generate questions given a context, a question-generation task. Extracted from Quasar-T, it consists of 119K training samples, with 116,953 in the training set, 2,048 in the development set, and 10,000 in the test set. We train all S2S-Diffusions for 40,000 epochs on this dataset.

Wiki-Auto Dataset (WA) [17]: Wiki-Auto requires models to revise complex text into sequences with simplified grammar and vocabulary, a text simplification task. Extracted from Wikipedia, it includes 677K complex-simple sentence pairs with revision alignment. The training set contains 677,751 pairs, the development set has 2,048, and the test set has 80,000.

Quora Question Pairs Dataset (QQP) [8]: QQP requires models to generate an alternative phrasing in the same language that conveys the same semantic content, a paraphrase generation task. Extracted from the Quora forum, it features 147K question-answering pairs. The training set contains 144,715 pairs, the development set has 2,048, and the test set has 2,500. We train all S2S-Diffusions for 140,000 epochs on this dataset.

B Detailed Information on Baselines

We provide details on the PLMs [33, 10] proposed by [8] and other S2S-Diffusion models [44, 45]. DiffuSeq fine-tunes PLMs to achieve optimal performance, balancing the trade-off between quality and diversity on the development set.

Dinoiser [44]: In the training stage, Dinoiser proposes Clipping Threshold, a predefined rule to calculate the minimum value and then uses the sqrt function to determine { β 1 , ..., β T } for diffusing a batch of sentences in a training epoch. In the inference stage, Dinoiser deploys the noise from its latest training epoch to generate sentences.

SeqDiffuSeq [45]: In the training stage, SeqDiffuSeq proposes imposing an increasing amount of noise with increasing training epochs. SeqDiffuSeq follows the pre-defined formula indicated in its paper to determine { β 1 , ..., β T } for diffusing a batch of sentences in each training epoch. In the inference stage, SeqDiffuSeq deploys the noise from its latest training epoch to generate sentences.

Fine-Tuned GPT-2 Models [33]: GPT2-base and GPT2-large are fine-tuned large pretrained language models (PLMs) GPT2. GPT2-base's model parameter is 117M. GPT2large's model parameter is 774M. DiffuSeq makes these two models inference with Beam Search and tunes the temperature to achieve better diversity.

LevT [10]: LevT, a widely used, strong iterative NAR model. DiffuSeq sets the max iteration to 9 and follows the termination condition mentioned in the original paper. LevT's model parameterer is 80M.

C Network Architecture of Meta-Diffu B

Our Meta-Diffu B framework comprises a scheduler model and an exploiter model. The exploiter model is an S2S-Diffusion model that adopts DiffuSeq's architecture, featuring a 12-layer Transformer with 12 attention heads. It incorporates time step embedding similarly to position embedding. The maximum sequence length is 128, and it operates over 2,000 diffusion steps. The scheduler model is an autoregressive model with a one-layer long short-term memory (LSTM) encoder-decoder architecture. It has the same embedding size as our exploiter model, with a maximum encoder length of 128 and a decoder length of 2,000.

D Details of Evaluation Metrics

We demonstrate the details of the evaluation metrics used in this work. To ensure a robust comparison, we adhere to the same evaluation metric settings as previous S2S-Diffusion models [8, 45].

BLEU [27]: BLEU is widely used to measure the quality of text generation. In this work, we use a smoothed BLEU score ranging from BLEU-1 to BLEU-4, where higher scores indicate better quality.

ROUGE-L [2]: ROUGE-L measures text generation quality by calculating the longest common subsequence. Higher ROUGE-L scores indicate better quality.

BERTScore [46]: BERTScore assesses text generation quality by calculating the embedding similarity between generated sentences and reference sentences. It utilizes embeddings from a BERT model, with higher scores indicating better quality.

Dist-1 [2]: Dist-1 measures the diversity of generated text by calculating the uniqueness of words within a single generated target sentence.

Self-BLEU [27]: Self-BLEU assesses the diversity of text generation by calculating the ratio of unique 4-grams in a set of generated sentences. Lower Self-BLEU scores indicate better diversity.

E Experiments of Meta-Diffu B on Machine Translation and Other Datasets

Following Section 5, we conduct experiments on machine translation datasets, including IWSLT14 DE-EN [26] and WMT14 DE-EN [31], using the same dataset and evaluation metric settings as other S2S-Diffusion models [45, 44]. Specifically, we adopt SacreBLEU [32] as the evaluation metric. As shown in Table 7, our Meta-Diffu B improves the performance of DiffuSeq, Dinoiser, and SeqDiffuSeq on these machine translation tasks. Additionally, we provide experiments on DiffuSeqbased S2S-Diffusion models and discrete S2S-Diffusion models (RDM [47] based on D3PM [1]) on the QQP and QG datasets. As shown in Table 7, our Meta-Diffu B consistently improves performance across both discrete and DiffuSeq-based S2S-Diffusion models.

F Showcase of Generated Sentences

Our Meta-Diffu B ( D θ = DiffuSeq) achieves better generation diversity and quality on the hardest generated sentences (H), as evidenced by the examples provided in Table 8 and Table 9. These tables illustrate that Meta-Diffu B can generate more effective sentences (H) from both the WA and QQP datasets than other S2S-Diffusion models. Table 8 shows the performance of our Meta-Diffu B ( D θ

Conditional sentence : ostersunds bs is a bandy club in ostersund, sweden, established on 5 september 1974 when ope ifs bandy section was disestablished.

= DiffuSeq). It consistently generates the hardest sentences (H) with superior quality and diversity, providing a reliable solution. In contrast, other S2S-Diffusion models [8, 44, 45] often produce repetitive sentences, failing to ensure both quality and diversity. Our findings, as illustrated in Table 9, show that Meta-Diffu B ( D θ = DiffuSeq) also outperforms other models, generating sentences (H) with superior diversity and quality on the QQP dataset.

G More Metrics on Contextualized Noise Scheduling of Meta-Diffu B

We present the results of Table 4 evaluated by other metrics in Table 10. Table 10 illustrates that our Meta-Diffu B ( D θ = DiffuSeq) outperforms other S2S-Diffusion models in generating sentences (H) and (E) across all evaluation metrics in this study.

H Context-Aware Noise Generation in Meta-Diffu B : Analysis and Insights

In this section, we further discuss the noise generated by our Meta-Diffu B . As shown in Figure 4, the total noise per epoch produced by Meta-Diffu B exhibits less fluctuation compared to other rule-based methods. While the noise variance is smaller than that of SeqDiffuSeq, it is larger than Dinoiser. From Table 3, we can see that the magnitude or variability of the noise does not necessarily correlate with

better performance for S2S-Diffusion models. The noise generated by Meta-Diffu B is context-aware, meaning it is learned and tailored to each sentence, thereby achieving better performance.

0

10

20

30

40

50

kepochs

2

3

4

5

6

7

8

9

Meta-Diffu

B

DiffuSeq

Dinoiser

SeqDiffuSeq

(a) QQP dataset

0

20

40

60

80

kepochs

2

4

6

8

10

12

Meta-Diffu

B

DiffuSeq

Dinoiser

SeqDiffuSeq

(b) WA dataset

I Additional Plug-and-Play Experiments with the Scheduler Model

Wepresent the results of the plug-and-play experiment featuring our scheduler model and DiffuSeq [8] trained on different datasets in Table 11. This table demonstrates that our scheduler model can enhance the performance of DiffuSeq across four datasets during the inference stage without the need for fine-tuning.

We also provide experiments with our scheduler on various other S2S-Diffusion models. As shown in Table 12 and Table 13, our scheduler not only improves performance across datasets but also enhances the performance across different models.

NeurIPS Paper Checklist

The checklist is designed to encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and precede the (optional) supplemental material. The checklist does NOT count towards the page limit.

Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:

You should answer [Yes] , [No] , or [NA] .

[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.

Please provide a short (1-2 sentence) justification right after your answer (even for NA).

The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include it (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.

The reviewers of your paper will be asked to use the checklist as one of the factors in their evaluation. While "[Yes] " is generally preferable to "[No] ", it is perfectly acceptable to answer "[No] " provided a proper justification is given (e.g., "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering "[No] " or "[NA] " is not grounds for rejection. While the questions are phrased in a binary way, we acknowledge that the true answer is often more nuanced, so please just use your best judgment and write a justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.

IMPORTANT, please:

Delete this instruction block, but keep the section heading 'NeurIPS paper checklist" ,

Keep the checklist subsection headings, questions/answers and guidelines below.

Do not modify the questions and only use the provided macros for your answers .

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: We clearly delineate our contributions and the scope of the study in both the abstract and the introduction.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: We specify that this work focuses on the noise scheduling of diffusion models for Seq2Seq tasks in Section 1. Additionally, we discuss the computational intensity of introducing Meta-Exploration in Section 4.3.1.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: We present the full set of assumptions and a complete proof of our MetaDiffu B in Section 2 and Section 3.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: We provide details of our dataset settings in Section 4.1. The training settings for the baselines are discussed in Section 4.2 and Appendix B. Details on the network architecture and training settings of our Meta-Diffu B can be found in Section 4.3 and Appendix C.

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: Code and Datasets for our experiments of this work are available at https: //github.com/Meta-DiffuB/Meta-DiffuB .

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: Details of the datasets are provided in Appendix A. The network architecture and training settings of our Meta-Diffu B are detailed in Section 4.3 and Appendix C.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: We employ Minimum Bayes risk (MBR) decoding, following the approach used in DiffuSeq, to represent the error bar, as detailed in Section 4.3.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: We demonstrate the compute resources in Section 4.3.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: We use public datasets from previous studies for our experiments. Additionally, we provide a Broader Impact section in Section 8 to outline the appropriate use of our publicly available code.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: We have included a Broader Impact section in Section 8 to discuss the potential positive societal impacts of our work, outline the appropriate use of our publicly available code, and prevent misuse of our resources.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [No]

Justification: All datasets used in this study are standard Seq2Seq datasets and are publicly available through previous research, rather than being unorganized content sourced from the web.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: For all datasets and research codes used in this study, we have cited the references and sources within the manuscript.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: We exclusively use existing open datasets in this work.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: Our experiments do not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: Our experiments do not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a novel approach to improving sequence-to-sequence (Seq2Seq) text generation models using diffusion models. The authors identify limitations in existing Seq2Seq-Diffusion models, which typically rely on fixed or hand-crafted noise scheduling rules that do not account for the specific characteristics of Seq2Seq tasks. To address these limitations, they propose the Meta-DiffuB framework, which introduces a scheduler-exploiter paradigm designed to provide contextualized noise scheduling. The scheduler model dynamically schedules contextualized noise based on the semantics of each sentence. The scheduler model is trained using Meta-Exploration techniques to optimize noise scheduling and can function as a \"plug-and-play\" model. The exploiter model (an S2S-Diffusion model) utilizes the noise scheduled by the scheduler for updating and text generation.\n\nSTRENGTHS:\n- The introduction of a scheduler-exploiter framework to schedule contextualized noise for each sentence in Seq2Seq tasks is a departure from existing methods that use fixed or hand-crafted noise scheduling rules. This idea is intuitive and straightforward.\n- Experimentation on four benchmark Seq2Seq datasets somehow validates the effectiveness of the proposed Meta-DiffuB framework.\n- The paper follows a logical structure, starting with the introduction and problem statement, followed by the methodology, experiments, results, and conclusions.\n\nWEAKNESSES:\n# More Comprehensive Comparison\nMy first concern is that this paper mainly compares the proposed method with Diffuseq, SeqDiffuSeq, and Dinoiser. There are many other diffusion models tailored for general seq2seq tasks [1-6]. I would suggest the authors carefully review the existing literature and select more recent baselines to compare.\n\n1. Empowering Diffusion Models on the Embedding Space for Text Generation. *NAACL*\n2. Latent Diffusion for Language Generation. *NeurIPS*\n3. DiffuSeq-v2: Bridging Discrete and Continuous Text Spaces for Accelerated Seq2Seq Diffusion Models. *EMNLP*\n4. AR-Diffusion: Auto-Regressive Diffusion Model for Text Generation. *NeurIPS*\n5. Can Diffusion Model Achieve Better Performance in Text Generation? Bridging the Gap between Training and Inference! *EMNLP*\n6. TESS: Text-to-Text Self-Conditioned Simplex Diffusion. *EACL*\n\nThe authors may also find more related works in:\n- https://github.com/StevenYuan666/Awesome-Diffusion-Models-for-NLP\n- https://github.com/westfish/Awesome-NLP-Diffusion-Models\n- https://github.com/AoiDragon/Awesome-Text-Diffusion-Models\n\nAlso, some more recent works focus on the noise schedule:\n\n7. Effective Integration of Text Diffusion and Pre-Trained Language Models with Linguistic Easy-First Schedule. *LREC-COLING*\n\n# Presentation\n- As the paper mentioned, parts of the methodology are motivated by the Meta-Exploration. I would suggest the authors add a section in the preliminary to provide more necessary details about the meta-exploration techniques, which will benefit readers who are not familiar with Meta-Exploration.\n\n# Analysis\n- The authors claim that the proposed method results in a contextualized noise schedule, which makes sense to me in terms of the scheduler generating Meta-Instructions conditioned on $\\boldsymbol{w}^x$. However, from Figure 3, even though the total amount of noise indeed fluctuates across different training epochs, it always keeps at the same level and the differences seem insignificant. I'm wondering why this makes the proposed methods beat existing baselines.\n\nQUESTIONS:\n1. Following my argument above, can you provide a deeper interpretation of the noise scheduling visualizations and their implications for different types (easy or hard as mentioned in the paper) of sentences? How do different noise schedules affect the generation quality and diversity for various types of sentences? Any patterns or insights observed from these visualizations?\n2. Could you provide an intuitive explanation of why the scheduler can function as a plug-and-play model? From the analysis in section 4.7, I'm suspecting it is because the learned schedule is not quite different from the pre-defined noise schedule. \n3. Did you run the empirical evaluation for several runs by setting different random seeds? If so, could you also highlight which results are statistically significant than the baseline method?\n\nLIMITATIONS:\nThe proposed method is in the domain of text generation and does not directly have any negative impact by itself. The authors have provided a paragraph to discuss the negative potential of misinformation created by the text generation model.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-06-28"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nComprehensive Evaluation of the new S2S Diffusion framework Meta-Diffu$\\beta$.\n\nSTRENGTHS:\n- Clear and well-written presentation of the method\n- It provides a new framework that uses an additional scheduler based on Meta-Exploration to schedule contextualized noise, which performs well in the four given benchmarks.\n\nWEAKNESSES:\n- **Incremental Innovation**: In terms of the scheduler, the use of the Meta-Exploration method for scheduling contextualized noise is noted. However, I concern that the exploiters fully follow the DiffuSeq work, resulting in a lack of significant innovation.\n- **Missing Experimental Baselines**: The cited baselines are all from before February 2023. It would be beneficial to include more recent baseline results [1] [2] [3] and take both generation efficiency and training speed into evaluation metrics.\n- **Dataset Limitations**: The selected dataset completely follows DiffuSeq and does not take into account machine translation tasks and related datasets from SeqDiffuSeq and DINOISER. It would be advantageous to include test results on IWSLT14 and WMT14.\n- **Plug-and-Play Capability**: The study only demonstrates the plug-and-play model and its corresponding effects on DiffuSeq. It's recommended to show the plug-and-play capability in other S2S Diffusion models like SeqDiffuSeq, DINOISER and other baselines mentioned to prove the claim in the conclusion.\n\nMinor: \n\n- The main text introduces the datasets but does not provide information about the downstream Seq2Seq tasks. Consider moving the introduction of these tasks from the appendix to the main text to enhance readability.\n\n- The pre-trained model and environment dependencies are missing from your GitHub repository. It would be helpful to include these components to facilitate reproducibility.\n\n[1] Ding, Y., Tian, J., Mei, S., Zhou, Y., Dong, Y., He, H. and Hu, W., 2023, December. LDSeq: Latent Diffusion Models for Sequence to Sequence Text Generation. In *Proceedings of the 2023 7th International Conference on Computer Science and Artificial Intelligence*\n\n[2] Gong, S., Li, M., Feng, J., Wu, Z. and Kong, L., 2023. Diffuseq-v2: Bridging discrete and continuous text spaces for accelerated seq2seq diffusion models. *arxiv preprint arxiv:2310.05793*.\n\n[3] Tang, Z., Wang, P., Zhou, K., Li, J., Cao, Z. and Zhang, M., 2023. Can Diffusion Model Achieve Better Performance in Text Generation? Bridging the Gap between Training and Inference!. *arxiv preprint arxiv:2305.04465*.\n\nQUESTIONS:\n- Regarding the selection of fixed noise, have you attempted to use different mathematical functions to obtain the $\\beta$ values as inputs for the skipping function? How is the performance using different $\\beta$ in the scheduler?\n\nLIMITATIONS:\nThe authors have addressed the limitations.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-12"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces the Meta-DiffuB, a scheduler-exploiter diffusion framework that focuses on sequence-to-sequence (Seq2Seq) setting. Its novel trainable contextualized noise scheduler, inspired by Meta-exploration, is also flexible and plug-and-play with other models like DiffuSeq without re-training. This approach dynamically schedules noise levels based on the characteristics of each sentence, improving the performance of text generation tasks. Meta-DiffuB outperforms existing models and fine-tuned pre-trained language models across four Seq2Seq benchmark datasets, demonstrating the effectiveness of its adaptive noise scheduling.\n\nSTRENGTHS:\n- The paper is well-motivated and well-structured.\n- The introduction of a contextualized noise scheduling strategy using Meta-Exploration is a significant advancement, addressing limitations of fixed or non-contextualized noise schedules in existing diffusion models.\n- Meta-DiffuB shows superior performance across four Seq2Seq benchmark datasets.\n- The ability to integrate the scheduler model into existing Seq2Seq diffusion models without the need for fine-tuning during inference enhances the practicality and usability of the approach.\n- The paper provides details of the proposed framework, pseudo-code with actual code for ease of reproduction.\n\nWEAKNESSES:\n- While the model performs well on benchmark datasets, there is limited discussion on its scalability to larger, real-world datasets and more complex text generation tasks.\n- The dynamic noise scheduling process might introduce additional computational overhead during inference, which is not thoroughly analyzed in the paper.\n- Meta-DiffuB is built upon and compared with S2S-Diffusion models known as continuous-based text diffusion models. It lacks a detailed comparison with other discrete-based text diffusion models such as D3PM [1] or RDM [2]. RDM claimed to achieve better performance and inference speed compared to DiffuSeq.\n- The effects of the dynamic noise scheduling on the overall training dynamics and convergence rate of the model are not explored in detail, which could be crucial for understanding the method’s effectiveness.\n\nQUESTIONS:\n- L120, 123: the quote for 'skipping' is wrongly formatted\n- L202-203: it should be either \"The maximum Minimum Bayes Risk (MBR)...\" or \"The Minimum Bayes Risk (MBR)....\", also it cites paper [20] (the Rouge package) which I think is incorrect for MBR.\n- Besides the increased training time compared to DiffuSeq (L217-218), it would be beneficial to know the increased inference time too.\n- How well does the Meta-DiffuB model generalize to other types of text generation tasks beyond the Seq2Seq framework? Have experiments been conducted to evaluate its performance in different text generation scenarios?\n- How does the model scale with larger datasets or more complex text generation tasks? Are there any specific optimizations or modifications required to handle real-world data volumes?\n\n[1] https://arxiv.org/pdf/2107.03006\n\n[2] https://arxiv.org/pdf/2302.05737\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-13"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a meta-learning based noise scheduler to incoporate contextualisation in texts. The scheduler is a plug-and-play module which could be applied to other similar sequential setting. The experimental results demonstrate its superiority comparing to the baselines, in terms of generating quality and diversity. Besides, the ablation study investigates into and provide insights about the relationship of the difficulty levels and noise schedule, which is quite interesting. The paper is well organised and easy to follow.\n\nSTRENGTHS:\nIt is the first work to take the contextilisation into consideration when doing adaptive noise schedule for text generation. Its effectiveness has been demonstrated, upon the questions listed below are addressed. \n\nThe experiments are well planned to support the arguments proposed. The presentation is clear and easy to follow.\n\nWEAKNESSES:\nWhile the diffusion process is now designed to incoporate a time variable noise schedule, the denoising process is still supposed to handle fixed denoising steps. More insights on how this will be handled effectively would be expected. \n\nSome statements in the text seems not align well with the information disclosing by the figures. For example, Figure 3, the noise level is quite stable for the proposed method but in the text below, it claims 'Meta-DiffuB applies adaptive noise at varying training epchos'. I am wondering if there are any typos involved. Another example is in Figure 4 at least for QQP dataset - Meta-DiffuB applies less noise than the other methods as its curves lay below the curves of the other methods. thus, the claim 'This noise-scheduling approach—applying more noise to the harder sentences (H) and less to the easier sentences' looks conflict with such observation from the figure. \n\nBesides, as algorithm 1 alternates the training of scheduler and exploiter, It would be guideful if some insights about the convergence rate are provided.\n\nQUESTIONS:\nI have three major concerns as listed in the weakness section about the reasonability, the consistence of figures and in-text description about some experiemtnal results, and the time complexity of training algorithm.\n\nLIMITATIONS:\nI suspect the main limitation would be the converge speed but it was not mentioned in the paper. \n\nThe schedular is implemented with a Seq2Seq structure, and other options could be provided as well to inspire other potential application scenarios.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-13"}