Meta-Diffu$B$: A Contextualized Sequence-to-Sequence Text Diffusion Model with Meta-Exploration
Authors: Yunyen Chuang
Hung-Min Hsu
Kevin Lin
Chen-Sheng Gu
Ling Zhen Li
Ray-I Chang
Hung-yi Lee
Venue: NeurIPS 2024 poster (2024)
Keywords: Diffusion Models
Sequence-to-Sequence
Text Generation
Meta-Exploration
Noise Scheduling
Primary area: diffusion_based_models
TL;DR: We propose a novel scheduler-exploiter framework, Meta-Diffu$B$, to achieve contextualized noise scheduling inspired by Meta-Exploration.

Abstract: The diffusion model, a new generative modeling paradigm, has achieved significant success in generating images, audio, video, and text. It has been adapted for sequence-to-sequence text generation (Seq2Seq) through DiffuSeq, termed the S2S-Diffusion model. Existing S2S-Diffusion models predominantly rely on fixed or hand-crafted rules to schedule noise during the diffusion and denoising processes. However, these models are limited by non-contextualized noise, which fails to fully consider the characteristics of Seq2Seq tasks. In this paper, we propose the Meta-Diffu$B$ framework—a novel scheduler-exploiter S2S-Diffusion paradigm designed to overcome the limitations of existing S2S-Diffusion models. We employ Meta-Exploration to train an additional scheduler model dedicated to scheduling contextualized noise for each sentence. Our exploiter model, an S2S-Diffusion model, leverages the noise scheduled by our scheduler model for updating and generation. Meta-Diffu$B$ achieves state-of-the-art performance compared to previous S2S-Diffusion models and fine-tuned pre-trained language models (PLMs) across four Seq2Seq benchmark datasets. We further investigate and visualize the impact of Meta-Diffu$B$'s noise scheduling on the generation of sentences with varying difficulties. Additionally, our scheduler model can function as a "plug-and-play" model to enhance DiffuSeq without the need for fine-tuning during the inference stage.

Meta-DiffuB: A Contextualized Sequence-to-Sequenee Text Diffusion Model with Meta-Exploration
Yun- Yen Chuang 12, Hung-Min Hsu Kevin Lin', Chen-Sheng Gu Ling Zhen Li Ray-I Chang", Hung-yi Lee"
Maxora AI National Taiwan University
University of Washington *Microsoft
yunyenchuang@maxora. ai, hnhsuQus. keli@microsoft.com
chenshengguumaxora.. lingzhenli amaxora..aiaaaaa.a....
rayichangentu.cm edu. hungyileeentu.ce edu.
Abstract
The diffusion model, a new generative modeling paradigm, has achieved significant success in generating images, audio. video, and text. It has been adapted for -e-nue--------eeceeee text generation (Seq2Seq) through DiffuSeq. termed the S2S-Diffusion model. Existing S2S-Diffusice models predominantly rely fixed or hand-crafted rules to schedule noise during the diffusion and denoising processes. However, these models limited by non-contextualized noise, which fails to fully consider the characteristics of Seq2Seq tasks. In this paper, we propose the Meta-DiffuB framework novel scheduler-exploiter S2S-Diffusion paradigm designed to overcome the limitations of existing S2S-Diffusion models. We employ Meta-Ex Exptortipooo train an additional scheduler model dedicated scheduling contextualized noise for each sentence. Our exploiter model, an S2S- Diffusion model, leverages the noise scheduled by our scheduler model for updating and generation. Befa-- achieves state-of-the-un performance compared to previous S2S 2S--ifffusion models and fine-tuned pre- pre-trained language models (PLMs) across four Seq2Seq benchmark datasets. We further investigate and visualize the impact of Meta-DiffuB's noise scheduling on the generation of sentences with varying difficulties. Additionally, our scheduler model can function as "plug- and-play" model enhance DiffuSeq without the need for fine-tuning during the inference stage.
Introduction
The diffusion model, novel generative approach, operates through a two-step process: it first introduces noise real data and then systematically removes this noise facilitate data generation
12
40
30
This model has demonstrated significant efficacy across several domains, including
image
13
29
38
audio [6 [8], video [18 14], and text generation [
15
21
B
24
34). The
diffusion model utilizes technique known as noise scheduling to control the amount of noise imposed
each diffusion step [12]. DiffuSeq g has adapted this model to discrete generation tasks like text generation (Seq2Seq), under framework termed RSS-Diffssion. However, DiffuSeq employs fixed noise scheduling and does not accommodate the specific characteristics of
Seq2Seq tasks 44 44
Seq2Seq is a foundational technique in natural language processing (NLP) that generates target sentences from specified conditional sentences. It supports a range of downstream tasks, including
"Code and datascts for Meta-DiffuB3 are available at: https://github. com/Neta -DiffBB Meta-DiffuB
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

language translation 41 image captioning [35,, conversational modeling [3)] and text summa- rization 288] For Seq2Seq tasks, more reasonable impose different levels of noise to each sentence in S2S- Diffusion models to address the varying semantic and contextual difficulties of generating sentences. This noise scheduling strategy can better adapt to the semantic characteristics and generation difficulties of each sentence, thereby improving the model's performance in various generation tasks. To meet the unique demands of S2S Diffusion, we introduce a contextualized noise-scheduling strategy that accounts for the semantics of each conditional sentence and adapts to different training epochs. Existing 22S-Diffusion models, such as DiffuSeq, lack flexibility due to their reliance on fixed, non-contextualizze noise-scheduling strategies. Furthermore, models like SeqDiffuSeq [45] and Dinoiser [44], which propose adaptive noise scheduling, are also limited by their conteeeextualize approach.
To address the semantics of discrete conditional sentences for contextualized noise scheduling, we introduce a novel scheduler-exploiie framework, Meta-Diffußß, which achieves trainable noise- scheduling inspired by Meta- Meta--xxloration 44 Within this framework, our scheduler model dy- namically schedules noise train our exploiter model, which is updated based on the performance rewards generates. Our exploiter model, an S2S-Diffusion model, leverages the noise scheduled by the scheduler model for updates and generation. By design, Meta-Diffu naturally implements contextualized noise scheduling. achieves state-of-the-as performance on four Seq2Seq benchmark datasets, outperforming existing S2S DSS-iiffusion models 8 45 44 and fine-tuned pre- rre-triine language models (PLMs) 10 333.
In summary, we make three primary contributions with Meta-DiffuB:
We introduce and demonstrate the application of Meta- veta-Exploration to diffusion models in Section 3 proposing Meta-Diffu as strategy to enhance S2S-Diffusion models. Our main results, presented in Section 6.I] confirm that Meta-Diffußß achieves state-of-the-ar performance across four benchmark datasets.
We detail the operation of our scheduler model in Section 6.2
highlighting its capability to
schedule noise. The noise scheduling approach our scheduler applying less noise to the harder sentences and more the easier ones enhances the diversity and quality of the generated text.
We reveal that our scheduler model can function as a "plug-and sase-and-pppyy playppplay model, easily integrated into existing 2S--iffusion models to enhance inference performance, as detailed in Section
2 Problem Statement, Preliminary
2.1 Problem Statement
In this work, we focus on sequence-tor toaiicee-t--ceeee tosueuneee text generation tasks. Given conditioning sentence of length m, w = {wi.... w J. our objective to train a diffusion model capable of generating a target sentence of length w = {w", ... W }. based on the conditional sentence. Here, w and w represent the conditional and target sentences, respectively.
2.2 Preliminary
DiffuSeq
{ primarily follows the transformation method of Diffusion-LM [2 and incorporates the diffusion and denoising processes from [12]. In the diffussion process, Diffusion-LLM transforms discrete sentences into a continuous space. Given the real-world training sentence pair w concatenated by w and w Diffusion-LM uses an embedding function emb to transform wry into continuous space, thereby obtaining the distribution Zn ~ g(z). where q represents the diffusion process. Then, Zo is subjected to imposed noise, diffusing into standard Gaussian distribution Zr (O. I). At each diffusion step
21, T), the noise is regulated by q(z (2-1) = Nlzz VI- 3 B,I), where 3 (0, 1) controls the amount of noise imposed at each diffusion step. We denote 3 as containing set of noise values 3. where larger 33 indicates more Gaussian noise imposed at that diffusion step. When / is large enough, Zn gradually evolves into standard Gaussian noise distribution. The random distribution is gradually reduced in noise during the denoising process regenerate target sentences. The denoising process, which recovers zo by

---

Diffusion Process
Denolsing
gryr
target
Zo
Z
Zr
(a) Previous 525- Diffusion model
Schedule contextuallzed noise with the scheduler mocel (3).
Diffusion Process
Dencising Process
scheduler model
Encoder
Decoder
Zg
Zx
Z5
Update the schedular model with the rewardss of target sentences using Polley Gradient shown in Eqp (6).
[b]
Diffa-
Figure 1:
Comparison between S2S- 22S-Diffusion model (i.e., DiffuSeq (II) and the proposed Meta- DiffuBB. The shades of color represent different amounts of noise being imposed. Different from prior works that use a fixed noise, we introduce novel scheduler-exploiter framework, Meta-Diffu B, which achieves trainable noise scheduling inspired by Meta Exploration. Our scheduler model schedules contextualized noise, enhancing the training and generation the S2S-Diffusion model, resulting in state-of-the-art (SOTA) performance compared previous S2S-Diffusion models, as detailed in Section
reducing the noise in Z,, can be defined follows:
T po(2--122))
(o(za)) p(zr) II
(1)
t=1
Diffusion-LM employs a trained, parameterized denoising distribution 2:-1 Po(2:-11 l/z)) to grad- ually recover 2, from noise. This denoising distribution, parameterized by 0, is tailored fit the posterior distribution ((2-1 1 Zn) of the forward process. The key difference between DiffuSeq [ and Diffusion-I.M 21 that DiffuSeq imposes noise only the target sentence part of 2, to achieve classifier-free free S2S Diffusion, termed Partial Noise [8]. Due to the implementation of Partial Noise in the diffusion process, conditional denoising is inherently classifier-free. To transform the continuous target sentences back into discrete sentences w previous S2S-Diffusion models use Rounding Operation 21 map the target sentence part of zg into w The Rounding Operation is method for choosing the most probable word for each position [21]. The denoising process primarily utilizes the variational lower bound (LVLB) optimize the negative log-likelihood [12]. Through the simplification and derivation from DiffuSeq [8), the training objective function for S2S-Diffusion models can be defined as:
min LVLB min
Iz0
fo(z,, 2] + embbw - fo(z1, R([1zo
(2)
where learning process Po (2.) is modeled Transformer model fo. Previous diffusion models
deplay /3 by dividing the interval between the minimum value 3n and the maximum value By using
mathematical function determine the fixed noise sequence {S1.... 3r E B, as described in [12). The mathematical function used by Diffusion-LLN and DiffuSeq [21] is the sqrt function, which
has demonstrated superior performance in text generation compared to other fixed mathematical functions. However, DiffuSeq's 's noise scheduling is constrained by its non-contextual approach; it
does not account for the semantics of each conditional sentence nor does adapt to di different training epochs.

---

3 Methodology
In this work, propose scheduler-exploiter framework named Meta-Diffuß for training S2S- Diffusion models with contextualized noise. Inspired by [43), our Meta-Diffußß includes a scheduler model, Bi. parameterized by v. and an exploiter model, De. parameterized by 0. Be, a simple Seq2Seq model, considers the semantics of conditional sentences to schedule contextualized / for updating Dg and also updated based on the learning effectiveness of D3 D0-which refers to how well the exploiter learns. Our exploiter, Dø, an D2S-Diffusion model, leverages the noise scheduled by By for its updating and generation. The framework of our Meta- Meta-DiffuBB, compared with DiffuSeq. is visualized in Figure I
3.1 Noise Scheduling in the Scheduler Model
In this work, propose simple two-step approach for scheduler By, which is Seq2Seq model, to schedule B-a set of noise values St. Here, a larger 3r indicates more noise imposed on the data. The input B is consistently w across both training and inference stages. Instead of directly scheduling the values of B. By outputs a series of Meta-Instructions, simplifying the training into time-series binary classification problem. In the first step, By samples series of Instrtnnnssssionss ,. = (41....,4,, .4,..,,rr? from w where each is labeled either True or False. We propose 'skipping' method: True label directs By to increase the noise by selecting 3 for the next diffusion step, whereas a False label maintains the same noise level Sr. In the second step, we transform xx using the fixed noise sqrt-function 3eart {BI..... BT as deployed by ( For
22 8) through the "skipping' method to generate the new noise values 3 = {8T example, with the continuous Meta- leta-Inntructions 1x {T, F) and fixed noise values f1, 3). our new scheduling of noise values will be {1, 1.2). If consecutive scheduling noise values are the same, no additional noise is introduced that diffusion step [22. Our two-step approach maintains the same diffusion steps for parallel operations and contextualized 3 in the diffusion process. We utilize a Policy Gradient to update our scheduler model following Meta-Exploration, addressing the non-differentiabilit of our two-step approach. The noise-scheduling mechanism of our scheduler model can be defined by the following equations:
tx (ww)))
B skipping((", goortt).
3.2 Training the Exploiter
that employ fixed or hand-crafted noise scheduling. we utilize contextualized 3 to impose noise during the diffusion process. We also implement Partial Noise to achieve classifier-free S2S Diffusion [8]. In the denoising process, exploiter model Do restores the diffused data to generate the target sentences. During the diffusion process, we adopt the transformation method of Diffusion-LM to obtain emb(w=@*), described in Section 2 We extend the original diffusion chain to a new Markov transition with our 3": go Zu (ia)
Unlike previous S2S-Diffusion models 8 45
(cmmbww**)) :6 I) (21 8). Consequently, we can implement the objective function indicated in Section
2] derived from previous classifier-free S2S- DDs-Diffusion methods, to update our exploiter model
Da H. The training objective function for our exploiter model Do in collaboration with By can be defined as follows:
T
VoJ = min
By 1)]12 + R(11zo1))) Izo - VoDolz Be ,1]1 + emmbbw VoDo(z
Since Zn is not diffused, there is no need to add the superscript of Bg. J(0) is denoted as the gradient for updating exploiter model Do. Then, we can update our exploiter model Dø's network weights:
0 + Vo.
(5)
3.3 Contextualized Inference with Meta-DiffuB
In the inference stage, if our goal is to generate outputs based on w", Bi predicts contextualized 3d using w", demonstrated in Section 3.1 We then concatenate emb(w") -rnns"")-trassfomnee transformed from

---

Algorithm 1 Meta-DiffuB
Require:
exploiter model Do; scheduler model Byi conditional sentences w and target sequences
w from dataset.
Initialize exploiter model Da, scheduler model By with random weights and vb. repeat
for in : & exploration epochs do
By schedules noise 3 by Eq. 3
8 t- 0 V (f)) by Eq. (4 and Eq. 5
Estimate the Meta-Reward R. as descrbbed Section 3.4
Compute the gradient (15)) by Eq. (6).
end for
Ve' < th + - - VJ by Eq.
Scheduler Update
Byr schedules noise 8 by Eq. 33.
0' 0 + V0J by Eq. (4) and Eq.
Exploiter Update
until Meta-DiffulB converges
with a randomly sampled yr ~ .V I) to form zy. Our Da predicts Zo directly from z, and uses 3 to convert the predicted Zg into Zı-1- This step-by-step denoising process progressively recovers 2, back to 2o. following the methodologies outlined in [2] 45 44 Finally, we use a Rounding Operation to convert the target sentence part of 2o into discrete target sentences.
3.4 Estimating the Meta-Reward of the Scheduler Model
In this section, we estimate the Meta-Reward Reward of our scheduler model, which reflects the learning effectiveness of Dg- We let Dg generate YD, and Dør generate Y respectively, where Y denotes the generated w 21. We assess the rewards for YD, and D denoted as Rp, and RD respectively, which represent the rewards for Dg and Dør, In this study, we utilize the BLEU score to quantify these rewards. Consequently, the reward for the scheduler model (i.c., Meta-Reearde is defined as RD, RD.
3.5 Training the Scheduler Model with Meta-Reward
to diffuse sentences using our two-step approach described in Section 3.1] update the scheduler via policy gradients, incorporating both Meta-Inste Instructions and the calculated Meta-Rewards [43]. The training objective function for our scheduler model is defined follows:
Since By generates
V+J( Σ Balf w²) Rg=
(6)
After we obtain (V)), we can update
By 's network weights:
w = し v.)())
(7)
3.6 Exploration Epochs
we iteratively execute the (..tion[3.4|oa collect various indicators of learning effectiveness from Do for updating B.. practice, we keep the network weights of D, fixed until the exploration epochs are completed. This approach ensures that the scheduler model schedules noise to Do with consistent network weights, promoting stable training [5]. Additionally, we can conduct the exploration epochs in parallel to save time by collecting learning effectiveness from Do under consistent network weights. After accumulating the gradients for B& from these exploration epochs, we update B to Bgr, which in turn schedules new noise update Dg to Dar In summary, we present Algorithm (I) to detail the full training process of the proposed Meta-Diffulh The number of exploration epochs denoted by E, with E {1,..., E} indexing the exploration epochs.
Inspired by the exploration epochs of Meta-Exploration procedures from Section
43
19
5

---

Experiments
In this section, we conduct experiments verify the performance of our Meta-DiffuB on four benchmark Seq2Seq datasets [48
6
17
g We benchmark Meta-DiffuB against previous S2S-
Diffusion models and fine-tuned pre-trained language models (PLMs), using the same datasets and
training settings as employed by DiffuSeq [8].
4.1 Datasets
In our experiment, we use four datasets: the Commonsense Conversation dataset (CC) 48 the Quasar-T T dataset (QT) [6], the Wiki-Auto dataset (WA) [77. and the Quora Question Pairs dataset (QQP) g These datasets consider a variety of tasks, including open-domain dialogue generation, question generation, text simplification, and paraphrase generation tasks, all within Seq2Seq contexts. For fair comparison, we employ the same datasets with identical settings for training all mentioned models, as outlined in g 45]. Detailed settings of these datasets are provided in Appendix
4.2 Baselines
We compare the proposed Meta-Diffu.B with previous S2S-Diffusion models, including DiffuSeq [ Dinoiser [44], and SeqDiffuSeq 45). DiffuSeq employs a fixed noise pattern in the training and inference stages using sqrt function and has been successfully introduced the Seq2Seq task as the basic diffusion model. We also compare Miea--Difful with Dinoiser and SeqDiffuSeq, which are existing S2S-Diffusion models that focus on noise scheduling. Dinoiser and SeqDiffuSeq utilize hand- crafted rules that provide adaptive but not contextualized noise scheduling. Additionally, following 8 [45,, we compare our Meta-Diffuß with three PL.Ms on Seq2Seq tasks. These PLMMs include the fine-tumed GPT-2-base (GPT2-base) 33 fine-tuned GPT-22large (GPT2-large), and fine-tuned
Levenshtein Transformer (Lev')) 10. We detail these baselines in Appendix B
4.3 Training Setting
Our exploiter model employs the same network architecture and settings as DiffuSeq g Our scheduler model uses the same network architecture as described in [4). The exploiter model and scheduler architectures are detailed in Appendix [ For consistent comparison, all S2S-Diffusion 4 follow the experimental settings prior research K and are trained from scratch. The diffusion step count is set at 2,000, and the maximum sequence length is 128. The Minimum Bayes risk (MBR) [23] decoding size, denoted as [S], is 10; this involves generating sentences from 10 random seeds and selecting the best output sequence. Details on the implementation of MBR for all D2S-Diffusion models can be found in Appendix 6 The total batch size for both training and testing phases is 2048. Experiments are conducted on NVIDIA A100 Tensor Core GPUs, utilizing GPUs for training and single GPU for inference.
models 8
4.3.1 Discussion of Computational Intensity
To ensure fair comparison during parallel exploration epochs, avoid increasing the total batch size. Instead, we reduce the batch size by dividing the total batch size by the number of exploration epochs deployed. In this work, set the number of exploration epochs to 32 and the batch size to 64. To update our scheduler, we run parallel exploration epochs every 100 training epochs with a total batch size of 2048. The increased computational complexity of applying Meta-Difful to DiffuSeq is presented in Table
Table I: Computational complexity increase when applying Meta-Diffu/B to DiffuSeq.
Method
Increased Parameters (%) Increased Training Time (%) Increased Inference Time (%)
Bifl--iifuB
2.2%
5%
0.5%
4.4 Evaluation Metrics
To ensure fair comparison, we follow the same evaluation metric settings as those used previous S2S-Di Diffusion models [8 45). For quality assessment, we utilize standard text generation metrics such

---

as BLBU (27], ROUGE-L 2] and BERTScore 46], where higher scores indicate better performance. For diversity assessment, we apply general text generation diversity metrics, including Distinct Unigram (Dist-1) 1) 22 and Self-BI [U where lower scores of Self-BLEU and higher scores of Dist- 1 signify better performance. Due the application of multiple evaluation metrics (such as BLEU, ROUGE-L, BERT'Score, Dist-1, and Self-BLEU), we also use Mean-Rank (M-R) to measure whether each model performs the best across multiple metrics [20). A lower Mean-Rank score indicates consistently better performance across various metrics in the dataset. Details the evaluation metric settings and their explanations are provided Appendix
Model-Agnostic Characteristics of Meta-DiffuB
We conduct experiments on applying our Meta-DiffuBB other S2S-Diffusion models. Specifically, we use Meta-DiffuB to modify the handcrafted noisesccheduling strategies of Dinoiser 44 and SeqDiffuSeq 45 on the WA and QQP datasets. The results, shown in Table 2 demonstrate that Meta-Diffu can be considered model-agnostic method for enhancing the performance of other S2S-Diffusion models. Additionally, we provide results for applying our Meta-Diffuß to RDM [47]
(based on D3PM [47]) and other recent S2S- Disfuusssi models 42 7
9
which are based on
DiffuSeq on machine translation datasets
26 in Appendix
Table 2: Results of applying our Meta-DiffuB (Da = specific S2S-Diffusion model) to other S2S-Diffusion models 8 45 44 The specific S2S-Diffusion model used the exploiter model indicated by the assignment of Da- Outcomes where Meta-Diffu outperforms previous S2S- Diffusion models are highlighted bold. star (x) indicates results reported directly from previous studies, while a dagger (f) signifies that we reproduced the results because the original studies did report them using the same metries on these datasets.
Tasks
Methods
BLEU (t)
BERTScore (t)
Dist-1 (1)
Diffuueq
0.2413
0.8365
0.9807
Meta-DiffuB (D, DiffuSeq)
0.2552
0.8821
0.9922
QQP
SeqDiffuSeq
0.2434
0.8400
0.9807
Meta-DiffuBB (D, = SegDiffuSeq)
0.2632
0.8919
0.9902
Dinoiser
0. 0919
0.8036
0.9723
Meta-DiffuBB (D, Dinoiser)
0.2271
0.8525
0.9752
DiffuSeq
0.3622
0.8126
0.9264
Meta-DifluB (D, DiffuSeg)
0.3877
0.8233
0.9355
WA
SeqDiffuSeq
0.3712
0.8214
0.9077
Meta-DiffuBB (D, SegDiffuSeq)
0.3957
0.8451
0.9412
Dinoiser
0.2388
0.6787
0.8421
Meta-Diffu/B (D, Dinoiser)
0.2471
0.7285
0.8694
6 Experiments of Minimum Bayes Risk Decoding
Diffusion- LM proposes using Minimum Bayes Risk (MBR) to improve generation. Following the we allow all S2S-Diffusion models to generate set of candidate sentences from 10 random seeds and select the best output sequence that achieves the minimum expected risk under a meaningful loss function. Specifically, in this work, we employ the BLEU score as our loss function to evaluate performance, following the approach used in DiffuSeq [8]. We compare
methods described in 45 g
Mcta-Diffull (D9 DiffuSeq) with DiffuSeq (8) and GPT-2 [33 using MBR decoding 21 S 45) on the WA and QQP datasets as described DiffuSeq R. We specifically select Garge and GPT2-base for comparison based on their superior performance on these datasets (8). In this experiment, we apply MBR decoding to all three models while gradually increasing the candidate sentence size |S]. The results of the MBR decoding are presented in Figure 2
Figure 2 shows that our Meta-Diffu ( (D, DiffiuSe) can generate more diverse array of candidate sentences, achieving better results as the candidate size |S] increases. The diversity of these candidate sentences determines the upper bound of MBR performance [2I g.. Our Meta-Difful (Dg DiffuSeq) consistently outperforms both GPT2-basc and DiffuSeq across all candidate size settings
7

---

0.26
6,25
0.36
0.23
0.34
0.21
Dillow
Diflube
GPTz-lange
GPT2-0a5e
0.19
DimTus
0.30
Diftu
candidate
candidate
(a) QQP dataset
(h) WA dataset
Figure 2: Increase in BLEU score with varying candidate
sizes |S| the QQP and WA datasets.
on the QQP dataset. As the candidate size grows, Meta-Diffu (Dg = DiffuSeq) also surpasses GPT2-base the WA dataset. Notably, Meta-Diffull (Dg DiffuSeq) exhibits diverse generation capabilities and achieves significantly better performance than DiffuSeq in MBR experiments.
6.1 Experiment with Seq2Seq Benchmark Datasets
We demonstrate the performance of our Meta- Beta-DiffuB (D₂ DiffuSeq) in Table 3 This table shows that Meta-DiffuB (Dp = DiffuSeq) outperforms other PLMSs 33 10] and S2S-Diffusion models 8
44 45 in terms of generation quality and diversity, achieving the lowest M-R scores across four datasets. Moreover, Meta-Diffu B (Do DiffuSeq) demonstrates significant improvements
previous S2S-Diffusion models g 45
44
on all evaluation metrics considered.
6.2 Contextualized Noise Scheduling of Meta-Diffu.2
For the experiment involving contextualized noise of Meta-DiffuB (Dg = DiffuSeq), we selected the 200 hardest and 200 easiest generated sentences, labeled (H) and (E), respectively. All models listed Table 3 assessed the generation difficulty of each sentence using BLEU scores, with lower BLEU indicating higher difficulty. The performance of generating (H) and (E) is evaluated in terms of BLEU and Self-BI BeE,, as shown in Table 4
We also detail the results of Table evaluaated by other metrics in Appendix G We tasked all S2S-Diffusion models with scheduling noise for sentences (H) and (E), as shown in Figure 3 Since our Meta-Diffuß (D, = DifffSeq) assigns specific noise to each sentence, we averaged the noise values for clearer visualization. Figure 3 displays the last 10 diffusion steps, as the noise differences in the initial steps are minimal. In Table 4 Meta-Diffu (Da DiffuSeq) consistently outperforms other S2S-Diffusion models 44 45
terms of generation quality and diversity for sentences (E) and (H). Notably, when generating the more challenging sentences (H), -eta-Diffuß Meta- Diff-Di (Dg DiffuSeq) maintains its performance, whereas other S2S-Diffusion models 44 45 8 experience decline in both quality and diversity. F shows the benefits of Mifa-DiffuB (D, DiffuSeq), which strategically imposes different noise levels for sentences (E) and (H). This noise- scheeusche approach applying less noise to the harder sentences (H) and more to the easier sentences (E) enhances the diversity and quality of the generated text. The superior example of Meta-DiffuB in generating the hardest sentences (H) is further showcased Appendix F demonstrating its performance relative to other S2S-Diffusion models [44 g We
also provide more detailed discussion about the noise strategy of our Meta-DiffuB in Appendix
6.3 Blug-and-Play Experiments with the Scheduler Model
Our pre-trained scheduler model, trained under Meta-DiffuJB (D, DiffuSeq), which incorporates the semantics of discrete sentences, demonstrates its effectiveness by scheduling noise for pre-trained DiffuSeq models across various datasets. The results, presented in Table [5] show that our pre- treitra scheduler model, when applied across different datasets, enhances the performance of pre-trained DiffuSeq models without any fine-tuning during the inference stage. This confirms that our scheduler model can function as plug-and-play model across these datasets. Additionally, we provide further experiments on different pre-trained schedulers under various $2f-Diffusion settings. well results additional datasets in Appendix

---

Table We present the results our Meta-DiffuB (D₂ = Diffuueq) compared with other models across four Seq2Seq datasets. We report the scores of DiffuSeq and PLMs from [8] A star (+) indi- cates results reported directly from previous studies, while a dagger (i) signifies that we reproduced the results because the previous studies did not report them using the same metrics on these datasets.
The best results among S2S- Diffusion models are underlined, and the overall best results are bold.
Tasks
Methods
BLEU (t)
ROUGH-L (1)
BERTScore (t)
Dist-I (f)
Belf-BLEU (1)
M-R (4)
GPT2-hase
0.1980
0.52
0.8246
0.9798
0.5480
5.20
GPT2-large
0,2059
0,5415
0,8363
0,9819
0.7325
3.80
LevT
0.2268
0,5795
0.8344
0,9790
0,9995
4.80
QQP
DiffuSeq
0.2413
0,5880
0.8365
0,9807
0.2732
2.60
SeqDiffuSe
0.2434
0.8400
0,9807
2.33
1 Dinoiser
0.1949
0.5316
0.8036
0,9723
0.8643
6.20
Meta-DiffuBB
0.2632
0.5933
0.8519
0.9902
0.2595
1.00
GPT2-base
0,3083
0.5461
0.8021
0.9439
0,5444
3.40
GPP12-lare
0,2693
0.5111
0.7882
0.9464
0,6042
4.00
* LevT
0.2052
0.4402
0.7254
0.9715
0.9907
5.00
WA
DiffuScq
0,3622
0.5849
0,8126
0.9264
0.4642
3.00
SeqDitfuSeq
0.3712
0.8214
0,9077
3.33
Dinniser
0.2388
0.4821
0.6787
0.8421
0.9132
6.20
Meta-DiffuB
0.3877
0.6047
0.8233
0,9355
0.3888
1.60
GPT2-base
0.0741
0.2714
0.6052
0.9602
0.1403
3.80
GPT2-large
0.1110
0.3215
0.6346
0.9670
0.2910
2.60
* LevT
0.0930
0.2893
0.5491
0,8914
0.9830
5.40
QT
DiffuSeq
0.1731
0.3665
0.6123
0.9056
0.2789
3.20
ScqDitfuSeq
0.1746
0.6174
0.9248
3.33
Dinoiser
0.0477
0.1872
0.4690
0.8191
0,5273
6.40
Meta-DiffufB
0.1820
0,3870
0.6286
0,9323
0.2527
1.80
GPT2-base
0.0108
0.1508
0,5279
0,9194
0.0182
4.00
Gar2-large
0.0125
0. 1002
0,5293
0.9244
0.0213
4.00
LeyT
0.0158
0.0550
0.4760
0.9726
0.7103
3.80
CC
DiffuSeq
0.0139
0.1056
0.5131
0.9467
0.0144
3.40
SeqDitfuSeq
0.0112
0.4425
0,9608
2.80
Dinoiser
0.0096
0.1166
0.3545
0,2485
0.9994
6.00
Meta-DiffufB
0.0220
0.1528
0.5316
0.9670
0.0133
1.20
7 Related Works
7.1
Text Diffusion
define an absorbing state for generating discrete data. Diffusion-LMM (1 and AnalogBits 3 propose imposing noise on continuous latent representations, using transformation functions to bridge the discrete and continuous spaces of texts for both unconditional and controlled text generation.
7.2 Meta-Exploration
To transcend the limitations imposed by human-crafted rules in noise scheduling, we developed additional model trained through Neta-Explorrtion. inspired by [43. Meta-Exploration is
Reinforcement Learning (RL) training method that utilizes learning effectiveness to devise sampling
strategies that enhance model performance. Numerous studies [5] 37
25
19 16 have employed
Meta-Exploration to meta-learn scheduling strategies for applying additive Gaussian noise on actions and for sampling effective training data in RL tasks. We have adopted the Meta-Exploration concept 43 to train an additional model specifically for noise scheduling in S2S-Diffusiia DS---ffffsion
8 Broader Impact
In this work, our Meta-Diffu demonstrates significant performance improvements over previous S2S- SS-Difffusion models across four Seq2Seq tasks, detailed in Section 4.1 Meta-DiffuB implements

---

10
Diftulen
Dinoher
Drneter
Segp musse
0.5
SesSefi-Sez
Table The results of our Meta-DiffuB (Dg DiffuSeqq and other S2S-Diffusion models for generating sentences (E) and (H) on the WA dataset. The best result in each group is high- lighted bold.
Nata Dific
0.6
0.
02
Methods
BLEU (1)
Self-BLEU (4)
difusion
diftusion
DiffuSeq (E)
0.3721
0,4345
(a) QQP dataset
(b) WA datasct
SeqDiffuSeq (E)
0.3752
0.4652
Dinoiser (E)
0.2892
0.8852
Figure 3: Visualization of noise scheduling for each S2S-Diffusion model on the QQP and WA datasets, 3, represents the average noise imposed on sentences at diffusion step t. Unlike other models, which impose the same noise on all sen- tences, our Meta-DiffuB (D, DiffuSeq) varies the noise levels.
Meta-DiffuJB (E)
0.3997
0.3688
DiffuSeq (H)
0.3216
0.5085
SeqDiffuSeq (H)
0.3282
0.6251
Dinoiser (H)
0.2092
0.9528
Meta-DiffuFB (H)
0.3724
0.4056
Table 5: Results of the plug-and-play experiment for our scheduler model. The 'Scheduler' field indicates the dataset used train our scheduler model, while the 'DiffuSeq' field indicates the dataset used to train DiffuSeq. If the 'DiffuSeq' field is 'Null', DiffuSeq generates sentences using its own
noise. Results that outperform those where DiffuSeq uses its own noise scheduling are highlighted in bold.
Scheduler
Diffuseq
BLEU (t)
ROUGH-L (t)
BERTScore (t)
Dist-1 (t)
Self-BLEU (4)
WA QT Null
0.2594
0.5912
0.8459
0.9834
0.2653
QQP
0.2603
0.5947
0.8503
0.9812
0.2649
0.2413
0.5880
0.8365
0.9807
0.2732
learnable, contextualized noise scheduling for Seq2Seq tasks. It not only shows enhanced generation quality and diversity but also has the potential to be applied to other diffusion models that require conditional data learning to generate target data. However, it is important to note that using Meta- Diffu to create fake news or other forms of misinformation strongly discouraged.
9 Conclusions
We propose integrating Reta-Exploration into S2S-Diffusion models through our newly developed Meta-DiffuB. By utilizing Meta- eeta-axploration to schedule contextualized noise, our Meta-DiffuB model demonstrates significant performance improvements on four Seq2Seq benchmark datasets compared to previous 22S-Diffusion models and PLMs. We have conducted a comprehensive investigation of the noise-scheduling capabilities of Meta-Diffuß and have visualized the results. Importantly, Meta-DiffuBB has the potential act as plug-and-play model, providing a promising approach for enhancing other S2S- SS-Diffusion models during the inference stage without the need for fine-tuning.
10
Acknowledgments
We would like to express our sincere gratitude to Professor Hung-yi Lee from NTU Speech Lab for his invaluable guidance and insightful advice throughout this work. We also deeply grateful to Professor Ray-I Chang from NTU ICAN Lab for his mentorship and constructive feedback. Addi- tionally, we would like to thank the reviewers for their positive evaluation and valuable suggestions. Finally, we extend our appreciation to Maxora Al for providing the computational resources and environment that made this research possible, enabling us to make meaningful contributions to the field.
10

---

References
[1] Jacob Austin, Daniel D Johnson, Jonathan Ho, Daniel Tarlow, and Rianne Van Den Berg. Structured denoising diffusion models in discrete state-spaces. Advances in Neural Information Processing Systems, 34:17981-17999 2021.
[2] Asli Celikyilmaz, Elizabeth Clark, and Jianfeng Gao. Evaluation of text generation: A survey. arXiv preprint arXiv:2006. /4799, 2020.
[3] Ting Chen, Ruixiang ZHANG, and Geoffrey Hinton. Analog bits: Generating discrete data using diffusion models with self-conditionigg In The Eleventh International Conference on Learning Representations, 2022.
[4] Kyunghyun Cho, Bart Van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. Learning phrase representations using rnn encoder- decoder for statistical machine translation. arXiv preprint arXiv: 1078, 2014.
[5] Yun-Yen Chuang, Hung-Min Hsu, Kevin Lin, Ray-I Chang, and Hung-Y Yi Lee. Metaex-gan Meta exploration to improve natural language generation via generative adversarial networks. JEEE/ACM Transactions on Audio, Speech, and Language Processing, 2023.
[6] Bhuwan Dhingra, Kathryn Mazaitis, and William W Cohen. Quasar: Datasets for question answering by search and reading. arXiv preprint arXiv: 1xXiv:777700300 2017.
[7] Zhujin Gao, Junliang Guo, Xu Tan, Yongxin Zhu, Fang Zhang, Jiang Bian, and Linli Xu. Empowering diffusion models on the embedding space for text generation. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume I: Long Papers), pages 4664 4683, 2024.
[8] Shansan Gong, Mukai Li, Jiangtao Feng. Zhiyong Wu, and Lingpeng Kong, Diffuseq: Sequence to sequence text generation with diffusion models. The Eleventh International Conference on Learning Representations (ICLR), 2023.
[9] Shansan Gong, Mukai Li, Jiangtao Feng, Zhiyong Wu, and Lingpeng Kong. Diffuseq-v2 Bridging discrete and continuous text spaces for accelerated seq2seq diffusion models. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 2023.
[10] Jiatao Gu, Changhan Wang, and Junbo Zhao Levenshtein transformer. Advances in neural information processing systems, 32, 2019.
[11] Zhengfu He, Tianxiang Sun, Qiong Tang, Kuanning Wang, Xuan-Jing Huang, and Xipeng Qiu. Diffusionbert: Improving generative masked language models with diffusion models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 4521 1521-4534, 2023.
[12] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in nearal information processing systems, 33:684006851,, 2020.
[13] Jonathan Ho, Chitwan Saharia, William Chan, David Fleet, Mohammad Norouzi, and Tim Salimans. Cascaded diffusion models for high fidelity image generation. Journal of Machine Learning Research, 23(4)):1-33, 2022.
[14] Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J Fleet. Video diffusion models. Advances in Neural Information Processing Systems, 35:8633-8646, 2022.
[15] Emiel Hoogeboom, Didrik Nielsen, Priyank Jaini, Patrick Forré, and Max Welling, Argmax flows and multinomial diffusion: Learning categorical distributions. Advances in Neural Information Processing Systems, 34:12454-1244 2021.
[16) Timothy Hospedales, Antreas Antoniou, Paul Micaelli, and Amos Storkey. Meta-learning in neural networks: survey. IEEE transactions on pattern analysis and machine intelligence, 4499)95149-5169, 2022.
11

---

[17] Chao Jiang, Mounica Maddela, Wuwei Lan, Yang Zhong, and Wei Xu. Neural crf model for sentence alignment in text simplification. arXiv preprint arXiv:2005.02324, 2020.
[18] Zhifeng Kong. Wei Ping. Jiaji Huang, Kexin Zhao, and Bryan Catanzaro. Diffwave: A versatile diffusion model for audio synthesis. arXiv preprint arXiv:2009.09761, 2020.
[19] Kwei-Herng Lai, Daochen Zha, Yuening Li, and Xia Hu. Dual policy distillation. arXiv preprint wwXi:2200600466, 2020.
[20] Linjie Li, Jie Lei, Zhe Gan, Licheng Yu, Yen-Chun Chen, Rohit Pillai, Yu Cheng, Luowei Zhou, Xin Eric Wang, William Yang Wang, et al. Value: A multi-task benchmark for video-and- language understanding evaluation. arXiv preprint 1.Xi::206.004663.. 2021.
[21] Xiang Li, John Thickstun, Ishaan Gulrajani, Percy S Liang, and Tatsunori B Hashimoto. Diffusion-Im improves controllable text generation. Advances in Neural Information Processing Systems, 0114328-43443 2022.
[22] Zihao Li, Aixin Sun, and Chenliang Li. Diffurec: A diffusion model for sequential recommen- dation. ACM Transactions on Information Systems, 42(3):1-28, 2023.
[23] Chin-Yew Lin. Rouge: A package for automatic evaluation of summaries. In Text summarization branches oul, pages 74-81, 2004.
[24] Zhenghao Lin, Yeyun Gong, Yelong Shen, Tong Wu, Zhihao Fan, Chen Lin, Nan Duan, and Weizhu Chen. Text generation with diffusion language models: A pre-training approach with continuous paragraph denoise. In International Conference on Machine Learning, pages 21051-21064. PMLR, 2023.
[25] Iou-Jen Liu, Unnat Jain, Raymond A Yeh, and Alexander Schwing, Cooperative exploration for multi-agent deep reinforcement learning. In International Conference on Machine Learning, pages 6826-6836 PMLR, 2021.
[26] Jan Niehues Mauro Cettolo, Luisa Bentivogli Sebastian Stüker, and Marcello Federico. Report on the 11th iwslt evaluation campaign. Proceedings of the 1 11th International Workshop on Spoken Language Translation: Evaluation Campaign, pages 2-17, 2014.
[27] Ehsan Montahaei, Danial Alihosseini, and Mahdieh Soleymani Baghshah. Jointly measuring diversity and quality in text generation models. arXiv preprint arXiv: 190.00399 2019.
[28) Ramesh Nallapati, Bowen Zhou, Cicero dos Santos, Çaglar Gulçehre, and Bing Xiang. Abstrac- tive text summarization using 2016.
rnns and beyond. CoNLL 2016, page 280,
[29] Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen. Glide: Towards photorealistic image generation and editing with text-guided diffusion models. arXiv preprint arXiv 2021.
[30] Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In International conference on machine learning, pages 8162-88888 8171. PMLR, 2021.
[31] Christian Buck Ondrej Bojar, Barry Haddow Christian Federmann, Johannes Leveling Philipp Koehn, Pavel Pecina Christof Monz, Herve Saint-Amand Matt Post, Lucia Specia Radu Soricut, and Ale s Tamchyna. Findings of the 2014 workshop on statistical machine translation. Proceedings of the ninth workshop on statistical machine translation, pages 12-58, 2014.
[32] Matt Post. A call for clarity in reporting bleu scores. Proceedings of the Third Conference on Machine Translation: Research Papers, pages 186-191, 2018.
[33] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAl blog, 1(8):9 2019.
[34] Machel Reid, Vincent J Hellendoorn, and Graham Neubig. Diffuser: Discrete diffusion via edit-based reconstruction. arXiv preprint arXiv:2210.16886, 2022.
12

---

[35] Steven J Rennie. Etienne Marcheret, Youssef Mroueh, Jerret Ross, and Vaibhava Goel. Self- critical sequence training for image captioning. Proceedings of the IEEE conference on computer vision and pattern recognition, pages 7008-7024, 2017.
[36] Ludan Ruan, Yiyang Ma, Huan Yang, Huiguo He, Bei Liu, Jianlong Fu, Nicholas Jing Yuan, Qin Jin, and Baining Guo. Mm-diffusion: Learning multi-modal diffusion models for joint audio and video generation. In Proceedings of the IEEE/CVF Conference Computer Viston and Pattern Recognition, pages 10219-10228, 2023.
[37] Baturay Saglam and Suleyman S Kozat. Deep intrinsically motivated exploration in continuous control. Machine Learning, 1 12(12):4959-4999 2023.
[38] Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to- to-image diffusion models with deep language understanding, Advances in neural information processing systems, 35:36479-364433 2022.
[39] Yuanlong Shao, Stephan Gouws, Denny Britz, Anna Goldie, Brian Strope, and Ray Kurzweil. Generating high-quality and informative conversation responses with sequence-to-sequee models. In Proceedings of the 2017 Conference Empirical Methods in Natural Language Processing, pages 2210-2219 2017.
[40] Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In International Conference on Learning Representations, 2020.
[41] lya Sutskever, Oriol Vinyals, and Quoc V Le. Sequence to sequence learning with neural networks. Advances in neural information processing systems, 27, 2014.
[42] Zecheng Tang, Pinzheng Wang, Keyan Zhou, Juntao Li, Ziqiang Cao, and Min Zhang. Can diffusion model achieve better performance text generation? bridging the gap between training and inference! In Findings of the Association for Computational Linguistics: ACI. 2023, pages 11359-1133 11386, 2023.
[43] Tianbing Xu, Qiang Liu, Liang Zhao, and Jian Peng. Learning to explore with meta-policy gradient. arXiv preprint arXiv: 1803000440000 2018.
[44] Jiasheng Ye, Zaixiang Zheng. Yu Bao, Lihua Qian, and Mingxuan Wang. Dinoiser: Diffused conditional sequence learning by manipulating noises. Transactions of the Association for Computational Linguistics (TACL), 2024.
[45] Hongyi Yuan, Zheng Yuan, Chuanqi Tan, Fei Huang, and Songfang Huang. Text diffusion model with encoder-decode transformers for sruuence-to-eqqeene generation. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Technologies (Volume I: Long Papers), pages 22-39, 2024.
[46] Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi Bertscore: Evaluating text generation with bert. arXiv preprint
2019.
[47] Lin Zheng, Jianbo Yuan, Lei Yu, and Lingpeng Kong. A reparameterized discrete diffusion model for text generation. Conference on Language Modeling: COLM 2024, 2024.
[48] Yaoming Zhu, Sidi Lu, Lei Zheng, Jiaxian Guo, Weinan Zhang, Jun Wang, and Yong Yu. Texygen: A benchmarking platform for text generation models. In The 41st international ACM SIGIR conference on research & development in information retrieval, pages 1097-1100, 2018
13

---

A Details of Datasets
There are four datasets in experiment. For fair comparison,
the same datasets with the same settings as those described in 8 45 Below, we detail the datasets used:
Commonsense Conversation Dataset (CC) 48 CC requires models to generate informa- tive responses given a dialogue context, an open-domain dialogue generation task. Extracted from Reddit single-round dialogues, it includes over 3 million conversational pairs. The training set contains 3,382,13 137 pairs, the development has 2,048, and the test set includes 10,000 pairs, We train all S2S- S2S-Diffusions for 140,000 epochs on this dataset.
Quasar-T Dataset (QT) 6 QT requires models to generate questions given a context, a task. Extracted from Quasar-TT, it consists of 1 119K training samples, with 116,953 the training set, 2,048 in the development set, and 10,000 in the test set. We train all S2S-Diffusions for 40,000 epochs on this dataset.
Wiki-Auto Dataset (WA) [17]: Wiki-Auto requires models to revise complex text into sequences with simplified grammar and vocabulary, a text simplification task. Extracted from Wikipedia, it includes 677K complex-simple sentence pairs with revision alignment. The training contains 677,751 pairs, the development set has 2,048, and the test set has 80,000.
Quora Question Pairs Dataset (QQP) [8]: QQP requires models to generate an alternative phrasing in the same language that conveys the same semantic content, paraphrase genera- tion task. Extracted from the Quora forum, it features 147K question-answering pairs. The training set contains 144,715 pairs, the development set has 2,048, and the test set has 2,500, We train all S2S- DSS--iffussons for 140,000 epochs on this dataset.
B Detailed Information on Baselines
We provide details on the PLMs 33 10 proposed by 8 and other S2S-Diffusion models 44 45 DiffuSeq fine-tunes PLMs achieve optimal performance, balancing the trade-off between quality and diversity on the development set.
Dinoiser
In the training stage, Dinoiser proposes Clipping Threshold, a pre- defined rule calculate the minimum value and then uses the sqrt function to determine }r for diffusing a batch of sentences in a training epoch. In the inference stage, Dinoiser deploys the noise from its latest training epoch to generate sentences.
SeqDiffuSeq [55]: In the training stage, SeqDiffuSeq proposes imposing an increasing amount of noise with increasing training epochs. SeqDiffuSeq follows the pre-defined formula indicated in its paper to determine {BI,... 3r} for diffusing batch of sentences in each training epoch. In the inference stage, SeqDiffuSeq deploys the noise from its latest training epoch to generate sentences.
Fine-Tuned GPT-2 Models (33] GPT2-base and GPT2-large are fine-tuned large pre- trained language models (PLMs) GPT2. GPT2-base's model parameter is 117M. GPT2- large's model parameter is 774M. DiffuSeq makes these two models inference with Beam Search and tunes the temperature to achieve better diversity.
LevT, a widely used, strong iterative NAR model. DiffuSeq sets the max LevT [10]
iteration to and follows the termination condition mentioned in the original paper. LevT's model parameterer is 80M.
C Network Architecture of Meta-DiffuB
Our Deft-Di framework comprises a scheduler model and an exploiter model. The exploiter model is an S2S-Diffusion model that adopts DiffuSeq's architecture, featuring 12-layer Transformer with 12 attention heads. It incorporates time step embedding similarly to position embedding. The maximum sequence length 128, and it operates over 2,000 diffusion steps. The scheduler model is an autoregressive model with a one-layer long short-term memory (LSTM) encoder-decoder architecture. It has the same embedding size as our exploiter model, with maximum encoder length of 128 and decoder length of 2,000.
14

---

D Details of Evaluation Metries
We demonstrate the details of the evaluation metrics used in this work. To ensure robust comparison, we adhere to the same evaluation metric settings as previous S2S-Diffusion models 8 45
22 BLEU is widely used measure the quality of text generation. In this work, we use smoothed BLEU score ranging from BLEU-1 to BLEU-4, where higher scores indicate better quality.
ROUGE-L [2]: ROUGE-L measures text generation quality by calculating the longest common subsequence. Higher ROUGE-L scores indicate better quality.
BERTScore [4]]
BERTScore assesses text generation quality by calculating the embedding similarity between generated sentences and reference sentences. It utilizes embeddings from a BERT model, with higher scores indicating better quality.
Dist-1 measures the diversity of generated text by calculating the uniqueness of words within single generated target sentence.
Dist-1 [2:
Self-BLEU (27] Self-BLEU assesses the diversity of text generation by calculating the ratio of unique 4-grams set of generated sentences. Lower Self-BLEU scores indicate better diversity.
E Experiments of Meta-DiffuB on Machine Translation and Other Datasets
Following Section 5 we conduct experiments on machine translation datasets, including IWSLT14 DE-EN [26] and WMT14 DE-EN (3]] using the same dataset and evaluation metric settings as other S2S-Diffusion models 45
Specifically, we adopt SacreBLEU [32 as the evaluation metric. As shown in Table our Meta-Diffuzl improves the performance of DiffuSeq, Dinoiser, and SeqDiffuSeg on these machine translation tasks. Additionally, we provide experiments on DiffuSeq- based D2S-Diffusion models and discrete S2S-Diffusion models (RDM 47 based on D3PM (T) on the QQP and QG datasets. As shown in Table 7 our Meta-Diffuß consistently improves performance across both discrete and DiffuSeq-basee based S2S- S2ffDuuuusion models.
Table 6: Results of Meta-DiffuB on Machine Translation datasets (DE-EN), Results where Meta- DiffuB combined with different models show improved performance indicated in bold.
Methods
SacreBLEU 7 (IWSLT14 DE-EN) SacreBLEU ī (WMT14 DE-EN)
DiffuSeq
29.43
22.72
Meta-Diffuß (D, DiffuSeq)
31.71
26.17
SeqDifluSeq
30.16
23.28
Meta-DiffuB (D, SeqDiffuSeq)
32.41
26.14
Dinoiser
31.61
30.30
Meta-Diffull (D, Dinoiser)
33.82
32.09
Table 7: Comparison of Meta-DiffuBB the QG and QQP datasets. Results where Meta-DiffuJ3 combined with different models show improved performance are indicated in bold.
Methods
BLEU (QQP) BERTScore (QQP)
BLEU (QG)
BERTScore 1 (QG)
DiffuSeq 8
0,2413
0.8368
0.1731
0.6123
Meta-DifluB (D, = DiffuSey)
0.2552
0.8821
0.1826
0.6357
DilfuSeq-v2 9
0,2411
0.8.993
Meta-DiffuJ (D.- DiffuSeq-v2)
0.2556
BG-DiluSee
0,2619
0.8427
0.1744
0.6280
Meta-DiflufB (D, BG-DilluSeg)
0.2790
0,8757
0.1838
0.6571
TESS 22
0.3020
0.8570
0.1950
0.6580
Mitt-Di (D, TESS)
0.3142
0,8975
0.2055
0.6761
RDM [47]
0.2510
0.8472
0.1802
0.6310
Meta-Diffun (D, = RDM)
0.2684
0.8724
0.2271
0.6542
15

---

F Showcase of Generated Sentences
Our Meta-DiffuuB (Dg Diffuueeq) achieves better generation diversity and quality on the hardest generated sentences (H), as evidenced by the examples provided in Table and Table 9 These tables illustrate that Meta-DiffuB can generate more effective sentences (H) from both the WA and QQP datasets than other ESS-Diffusion models. Table 8 shows the performance of our Meta- Meta-DiffuB (Do
Table 8: The sample output of our Beta-DiffuBB (D, DiffuSee) and other
models g 45 on hardest generated sentences (H) WA dataset. The conditional sentence is the same.
Conditional sentence:
ostersunds bs is bandy club in ostersund, sweden, cstablished on 5 september 1974
when ope bandy section was disestablished.
Real target sentence
Meta-Diffu.i
DiffuSeq
Dinoiser
SeqDiffuSeq
it was established on september 1974 when upe ilss bandy section was disestablished.
was established on september 1974 when ope if ifs bandy sec- tion was disestablished.
ostersunds bs a handy club in the town ostersund in swe-
The club was dis- established on 5 September 1974.
henry bandy club in the town
of sweden.
ostersunds bs is a bandy club started ostersund, sweden, es- tablished in 5 september 1974, the ifs bandy section was disestab- lished.
ostersunds
a bundy club town of ostersund in den.
The club was dis- established 5 Septemher 1974.
henry er bandy club in town of sweden.
was founded on 5 september 1974 when ope ifs bandy section was disestablished.
ostersunds bs is a bundy club in the town ostersund in swe- den.
The club was dis- established on 5 September 1974.
henry er bandy club in the town sweden.
DifffuSeg). It consistently generates the hardest sentences (H) with superior quality and diversity, providing a reliable solution. In contrast, other 52S-Diffusion models [ repetitive sentences, failing to ensure both quality and diversity. Our findings, as illustrated in Table 9 show that Meta-Diffu.B (D, DiffuSeq) also outperforms other models, generating sentences (H) with superior diversity and quality on the QQP dataset.
44 45 often produce
Table 9:
The sample output of Meta-Difful (Dg DiffuSeq) and other S2S-Diffusion models g
44 45) on hardest generated sentence (H) of QQP dataset. The conditional sentence the same.
Conditional sentence: is possible to invent the time machine?
Real target sentence
Meta-Diffu/F
DiffuSeg
Dinoiser
SeqDiffuSeq
create time machine?
is really possible to create time ma- chine?
it possible to in our time machine?
Is possible to make a time machine?
possible to have time machine?
is it possible that we create time machine?
it possible to in our time machine?
Is possible to make a time machine?
is possible to have a time machine?
can we create time machine?
it possible to in our time machine?
Is possible to make time machine?
possible to have a time machine?
G More Metrics on Contextualized Noise Scheduling of Meta-Diffuß3
We present the results of Table I evaluated by other metrics in Table 10 Table 10 illustrates that our Meta-DiffulF (Dg DiffuSeq) outperforms other S2S- Diffuuffnio models in generating sentences (H) and (E) across all evaluation metrics in this study.
H Context-Aware Noise Generation in Meta-DiffuB:: Analysis and Insights
In this section, we further discuss the noise generated by our Meta-DiffuB. As shown Figure 4 the total noise per epoch produced by Meta-Diffu/3 exhibits less fluctuation compared to other rule-based methods. While the noise variance smaller than that of SeqDiffuSeq
larger than Dinoiser. From Table 3 can that the magnitude or variability of the noise does not necessarily correlate with
16

---

Table 10:
The results of our Meta-DiffuB and other S2S-Diffusion models 8
44
45
for generating
sentences (E) and (H) on the WA dataset. The best result in each group is highlighted in bold.
Methods
BLEU (1)
Self-BLEU (4)
ROUGH-L (1)
BERTScore (t)
Dist-1 (t)
DiffuSeq (B)
0.3721
0,4345
0,5962
0,8232
0.9285
SeqDiftuSeq (E)
0.3752
0,4652
0.6021
0.8286
0.9273
Dinoiser (E)
0.2892
0.8852
0,4937
0.6882
0.8574
Meta-DiffußB (E)
0.3997
0.3688
0.6359
0.8452
0.9462
DiffuSeq (H)
0.3216
0.5085
0,5514
0.7586
0.8828
SeqDiffuSeq (H)
0.3282
0.6251
0.5621
0.7479
0.8974
Dinoiser (H)
0.2092
0.9528
0.4375
0.6345
0.8396
Meta-Diffu (H)
0.3724
0.4056
0.5741
0.8026
0.9216
hetter performance for S2S-Diffusion models. The noise generated by Meta-Diffu context-aware, meaning is learned and tailored each sentence, thereby achieving better performance.
Merai Diiiud
Mite
pitfuiaa
DffiSca
Dinokkorr
Dissser
Scabiilussa
SagDimuGee
20
50
20
(a) QQP dataset
(b) WA dataset
Figure 4: Adaptive noise scheduling for each S2S- Diffusion model the QQP and WA datasets. E represents the total amount of noise imposed in each training epoch.
I Additional Plug-and-Play Experiments with the Scheduler Model
We present the results the plug-and-play experiment featuring our scheduler model and DiffuSeq trained different datasets in [ab This table demonstrates that our scheduler model can enhance the performance of DiffuSeq across four datasets during the inference stage without the need for fine-tuning.
Table 11: Results of the plug-and-play experiment for our scheduler model. The 'Scheduler' field indicates the dataset used to train the scheduler model, while the 'DiffuSeq' field indicates the dataset used train DiffuSeq. If the 'DiffuSeq' field is 'Null', DiffuSeq generates sentences using its
noise. Results that outperform those where DiffuSeq uses its own noise scheduling are highlighted in bold.
Scheduler
DiffuSeq
BLEU (t)
ROUGH-L (f)
BERTScore (t)
Dist-1 (t)
Self-BLEU (1)
WA
0.2594
0.5912
0.8459
0.9834
0.2653
QT
QQP
0.2603
0.5905
0.8503
0.9812
0.2649
Null
0,2413
0.5880
0.8365
0.9807
0.2732
WA
0.1804
0.3761
0.6234
0.9147
0.3848
QQP
QT
0.1769
0.3729
0.6215
0,9104
0.4485
Null
0.1731
0.3665
0.6123
0.9056
0.2789
QT
0.3666
0,5945
0.8217
0.9297
0.4052
QQP
WA
0.3711
0.5985
0.8204
0.9302
0.3996
Null
0.3622
0.5849
0.8126
0.9264
0.4642
17

---

We also provide experiments with our scheduler on various other S2S- Difffussoo models. As shown in Table 12 and Table [3] our scheduler not only improves performance across datasets but also enhances the performance across different models.
Table 12: Plug-and-play experiments on SeqDiffeeeq integrated with our scheduler. The field "SeqDiffuSeq" indicates which dataset this model is trained on. When the 'Scheduler' field is 'Null', it indicates the of the model's noise scheduling. Results where the model performs better with its own noise are indicated in bold.
Scheduler
SeqDiffuSeq
BLEU
BERTScore t
Dist-1
WA
QQP
0.2627
0.8481
0.9814
Null
QQP
0,2434
0.8400
0.9807
WA
QT
0.1834
0.6226
0.9369
Null QT
0.1746
0.6174
0.9248
Table 13: Plug-and-play experiments on Dinoiser integrated with our scheduler. The field 'Dinoiser' indicates which dataset this model is trained on. When the 'Scheduler' field is 'Null', it indicates the use of the model's own noise scheduling. Results where the model performs better with its own noise are indicated in bold.
Scheduler
Dinoiser
BLEU f
BERTSScore
Dist-1 1
WA
QQP
0.2079
0.8121
0.9765
Null
QQP
0.1949
0.8036
0.9723
WA
QT
0.0495
0.4740
0,8289
Null
QT
0.0477
0.4690
0.8191
18

---

NeurIPS Paper Checklist
The checklist is designed to encourage best practices for responsible machine learning research, addressing issues of reproducibility. transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and precede the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No], or [NA].
[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justifiiaton right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.
The reviewers of your paper will be asked to use the checklist as one of the factors in their evaluation.
While "[Yes] " is generally preferable to "[No]", ", it is perfectly acceptable to answer "[No] provided proper justification is given (e.g., "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering "[No] or "[NA] is not grounds for rejection. While the questions are phrased in binary way, we acknowledge that the true answer often more nuanced, so please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS paper checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
Justification: We clearly delineate our contributions and the scope of the study in both the abstract and the introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes]
19

---

Justification: We specify that this work focuses on the noise scheduling of diffusion models for Seq2Seq tasks in Section| [ Additionally, we discuss the computational intensity of introducing Meta-Exploratie in Section 4.3.1
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [Yes]
Justification: We present the full set of assumptions and a complete proof of our Meta-
DiffuB in Section 2lad Section
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
20

---

Justification: We provide details of our dataset settings in Section 4.1 The training settings for the baselines are discussed in Section 4.2 and Appendix E Details on the network architecture and training settings of our Meta-Diffu can be found in Section 4.3 and
Appendix C
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel urchitecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to deseribe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g.. to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: Code and Datasets for our experiments of this work are available at hhttp
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.c cc/
sublic/guides httpa//ggiiies/ooouuuiiiiseee
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https //nips.cc/pubbe httpccccppulcgggguuu httpg//ddeee/oddubiissssssssssse
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
21

---

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification: Details of the datasets are provided Appendix A The network architecture and training settings of our Meta-Diffu/3 are detailed in Section 4.3 and Appendix
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: We employ Minimum Bayes risk (MBR) decoding, following the approach used in DiffuSeq. represent the error bar, as detailed in Section 4.3
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
22

---

Justification:
We demonstrate the compute resources in Section 4.3
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e-g-, preliminary failed experiments that didn't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification: We use public datasets from previous studies for our experiments. Additionally, we provide Broader Impact section in Section g to outline the appropriate use of our publicly available code.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: We have included a Broader Impact section in Section 8 to discuss the potential positive societal impacts of our work, outline the appropriate use of our publicly available code, and prevent misuse of our resources.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (c.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
23

---

11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [No]
Justification: All datasets used in this study standard Seq2Seq datasets and are publicly available through previous research, rather than being unorganized content sourced from the web.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual- dsal-use should be released with necessary safeguards to allow for controlled use of the model, for example requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: For all datasets and research codes used in this study, we have cited the references and sources within the manuscript.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g.. website), the copyright and terms of service of that source should be provided.
If assets released, the license, copyright information, and terms of in the package should be provided. For popular datasets, papprssi www.wwtehho....mm com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re- re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's is creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
We exclusively use existing open datasets in this work.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
24

---

The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: Our experiments do not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: Our experiments do not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
25


Reviews:
[{"review_id": "qUPpql6Jne", "reviewer": "Reviewer_SoCC", "summary": "This paper proposes a meta-learning based noise scheduler to incoporate contextualisation in texts. The scheduler is a plug-and-play module which could be applied to other similar sequential setting. The experimental results demonstrate its superiority comparing to the baselines, in terms of generating quality and diversity. Besides, the ablation study investigates into and provide insights about the relationship of the difficulty levels and noise schedule, which is quite interesting. The paper is well organised and easy to follow.", "questions": "I have three major concerns as listed in the weakness section about the reasonability, the consistence of figures and in-text description about some experiemtnal results, and the time complexity of training algorithm.", "limitations": "I suspect the main limitation would be the converge speed but it was not mentioned in the paper. \n\nThe schedular is implemented with a Seq2Seq structure, and other options could be provided as well to inspire other potential application scenarios.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "It is the first work to take the contextilisation into consideration when doing adaptive noise schedule for text generation. Its effectiveness has been demonstrated, upon the questions listed below are addressed. \n\nThe experiments are well planned to support the arguments proposed. The presentation is clear and easy to follow.", "weaknesses": "While the diffusion process is now designed to incoporate a time variable noise schedule, the denoising process is still supposed to handle fixed denoising steps. More insights on how this will be handled effectively would be expected. \n\nSome statements in the text seems not align well with the information disclosing by the figures. For example, Figure 3, the noise level is quite stable for the proposed method but in the text below, it claims 'Meta-DiffuB applies adaptive noise at varying training epchos'. I am wondering if there are any typos involved. Another example is in Figure 4 at least for QQP dataset - Meta-DiffuB applies less noise than the other methods as its curves lay below the curves of the other methods. thus, the claim 'This noise-scheduling approach—applying more noise to the harder sentences (H) and less to the easier sentences' looks conflict with such observation from the figure. \n\nBesides, as algorithm 1 alternates the training of scheduler and exploiter, It would be guideful if some insights about the convergence rate are provided.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Lk3ImRuHNR", "reviewer": "Reviewer_YnwD", "summary": "The paper introduces the Meta-DiffuB, a scheduler-exploiter diffusion framework that focuses on sequence-to-sequence (Seq2Seq) setting. Its novel trainable contextualized noise scheduler, inspired by Meta-exploration, is also flexible and plug-and-play with other models like DiffuSeq without re-training. This approach dynamically schedules noise levels based on the characteristics of each sentence, improving the performance of text generation tasks. Meta-DiffuB outperforms existing models and fine-tuned pre-trained language models across four Seq2Seq benchmark datasets, demonstrating the effectiveness of its adaptive noise scheduling.", "questions": "- L120, 123: the quote for 'skipping' is wrongly formatted\n- L202-203: it should be either \"The maximum Minimum Bayes Risk (MBR)...\" or \"The Minimum Bayes Risk (MBR)....\", also it cites paper [20] (the Rouge package) which I think is incorrect for MBR.\n- Besides the increased training time compared to DiffuSeq (L217-218), it would be beneficial to know the increased inference time too.\n- How well does the Meta-DiffuB model generalize to other types of text generation tasks beyond the Seq2Seq framework? Have experiments been conducted to evaluate its performance in different text generation scenarios?\n- How does the model scale with larger datasets or more complex text generation tasks? Are there any specific optimizations or modifications required to handle real-world data volumes?\n\n[1] https://arxiv.org/pdf/2107.03006\n\n[2] https://arxiv.org/pdf/2302.05737", "limitations": "Yes", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper is well-motivated and well-structured.\n- The introduction of a contextualized noise scheduling strategy using Meta-Exploration is a significant advancement, addressing limitations of fixed or non-contextualized noise schedules in existing diffusion models.\n- Meta-DiffuB shows superior performance across four Seq2Seq benchmark datasets.\n- The ability to integrate the scheduler model into existing Seq2Seq diffusion models without the need for fine-tuning during inference enhances the practicality and usability of the approach.\n- The paper provides details of the proposed framework, pseudo-code with actual code for ease of reproduction.", "weaknesses": "- While the model performs well on benchmark datasets, there is limited discussion on its scalability to larger, real-world datasets and more complex text generation tasks.\n- The dynamic noise scheduling process might introduce additional computational overhead during inference, which is not thoroughly analyzed in the paper.\n- Meta-DiffuB is built upon and compared with S2S-Diffusion models known as continuous-based text diffusion models. It lacks a detailed comparison with other discrete-based text diffusion models such as D3PM [1] or RDM [2]. RDM claimed to achieve better performance and inference speed compared to DiffuSeq.\n- The effects of the dynamic noise scheduling on the overall training dynamics and convergence rate of the model are not explored in detail, which could be crucial for understanding the method’s effectiveness.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "hSMAqxgy03", "reviewer": "Reviewer_5aL8", "summary": "Comprehensive Evaluation of the new S2S Diffusion framework Meta-Diffu$\\beta$.", "questions": "- Regarding the selection of fixed noise, have you attempted to use different mathematical functions to obtain the $\\beta$ values as inputs for the skipping function? How is the performance using different $\\beta$ in the scheduler?", "limitations": "The authors have addressed the limitations.", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- Clear and well-written presentation of the method\n- It provides a new framework that uses an additional scheduler based on Meta-Exploration to schedule contextualized noise, which performs well in the four given benchmarks.", "weaknesses": "- **Incremental Innovation**: In terms of the scheduler, the use of the Meta-Exploration method for scheduling contextualized noise is noted. However, I concern that the exploiters fully follow the DiffuSeq work, resulting in a lack of significant innovation.\n- **Missing Experimental Baselines**: The cited baselines are all from before February 2023. It would be beneficial to include more recent baseline results [1] [2] [3] and take both generation efficiency and training speed into evaluation metrics.\n- **Dataset Limitations**: The selected dataset completely follows DiffuSeq and does not take into account machine translation tasks and related datasets from SeqDiffuSeq and DINOISER. It would be advantageous to include test results on IWSLT14 and WMT14.\n- **Plug-and-Play Capability**: The study only demonstrates the plug-and-play model and its corresponding effects on DiffuSeq. It's recommended to show the plug-and-play capability in other S2S Diffusion models like SeqDiffuSeq, DINOISER and other baselines mentioned to prove the claim in the conclusion.\n\nMinor: \n\n- The main text introduces the datasets but does not provide information about the downstream Seq2Seq tasks. Consider moving the introduction of these tasks from the appendix to the main text to enhance readability.\n\n- The pre-trained model and environment dependencies are missing from your GitHub repository. It would be helpful to include these components to facilitate reproducibility.\n\n[1] Ding, Y., Tian, J., Mei, S., Zhou, Y., Dong, Y., He, H. and Hu, W., 2023, December. LDSeq: Latent Diffusion Models for Sequence to Sequence Text Generation. In *Proceedings of the 2023 7th International Conference on Computer Science and Artificial Intelligence*\n\n[2] Gong, S., Li, M., Feng, J., Wu, Z. and Kong, L., 2023. Diffuseq-v2: Bridging discrete and continuous text spaces for accelerated seq2seq diffusion models. *arxiv preprint arxiv:2310.05793*.\n\n[3] Tang, Z., Wang, P., Zhou, K., Li, J., Cao, Z. and Zhang, M., 2023. Can Diffusion Model Achieve Better Performance in Text Generation? Bridging the Gap between Training and Inference!. *arxiv preprint arxiv:2305.04465*.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "c1YGZCuPPs", "reviewer": "Reviewer_qgkV", "summary": "This paper presents a novel approach to improving sequence-to-sequence (Seq2Seq) text generation models using diffusion models. The authors identify limitations in existing Seq2Seq-Diffusion models, which typically rely on fixed or hand-crafted noise scheduling rules that do not account for the specific characteristics of Seq2Seq tasks. To address these limitations, they propose the Meta-DiffuB framework, which introduces a scheduler-exploiter paradigm designed to provide contextualized noise scheduling. The scheduler model dynamically schedules contextualized noise based on the semantics of each sentence. The scheduler model is trained using Meta-Exploration techniques to optimize noise scheduling and can function as a \"plug-and-play\" model. The exploiter model (an S2S-Diffusion model) utilizes the noise scheduled by the scheduler for updating and text generation.", "questions": "1. Following my argument above, can you provide a deeper interpretation of the noise scheduling visualizations and their implications for different types (easy or hard as mentioned in the paper) of sentences? How do different noise schedules affect the generation quality and diversity for various types of sentences? Any patterns or insights observed from these visualizations?\n2. Could you provide an intuitive explanation of why the scheduler can function as a plug-and-play model? From the analysis in section 4.7, I'm suspecting it is because the learned schedule is not quite different from the pre-defined noise schedule. \n3. Did you run the empirical evaluation for several runs by setting different random seeds? If so, could you also highlight which results are statistically significant than the baseline method?", "limitations": "The proposed method is in the domain of text generation and does not directly have any negative impact by itself. The authors have provided a paragraph to discuss the negative potential of misinformation created by the text generation model.", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "- The introduction of a scheduler-exploiter framework to schedule contextualized noise for each sentence in Seq2Seq tasks is a departure from existing methods that use fixed or hand-crafted noise scheduling rules. This idea is intuitive and straightforward.\n- Experimentation on four benchmark Seq2Seq datasets somehow validates the effectiveness of the proposed Meta-DiffuB framework.\n- The paper follows a logical structure, starting with the introduction and problem statement, followed by the methodology, experiments, results, and conclusions.", "weaknesses": "# More Comprehensive Comparison\nMy first concern is that this paper mainly compares the proposed method with Diffuseq, SeqDiffuSeq, and Dinoiser. There are many other diffusion models tailored for general seq2seq tasks [1-6]. I would suggest the authors carefully review the existing literature and select more recent baselines to compare.\n\n1. Empowering Diffusion Models on the Embedding Space for Text Generation. *NAACL*\n2. Latent Diffusion for Language Generation. *NeurIPS*\n3. DiffuSeq-v2: Bridging Discrete and Continuous Text Spaces for Accelerated Seq2Seq Diffusion Models. *EMNLP*\n4. AR-Diffusion: Auto-Regressive Diffusion Model for Text Generation. *NeurIPS*\n5. Can Diffusion Model Achieve Better Performance in Text Generation? Bridging the Gap between Training and Inference! *EMNLP*\n6. TESS: Text-to-Text Self-Conditioned Simplex Diffusion. *EACL*\n\nThe authors may also find more related works in:\n- https://github.com/StevenYuan666/Awesome-Diffusion-Models-for-NLP\n- https://github.com/westfish/Awesome-NLP-Diffusion-Models\n- https://github.com/AoiDragon/Awesome-Text-Diffusion-Models\n\nAlso, some more recent works focus on the noise schedule:\n\n7. Effective Integration of Text Diffusion and Pre-Trained Language Models with Linguistic Easy-First Schedule. *LREC-COLING*\n\n# Presentation\n- As the paper mentioned, parts of the methodology are motivated by the Meta-Exploration. I would suggest the authors add a section in the preliminary to provide more necessary details about the meta-exploration techniques, which will benefit readers who are not familiar with Meta-Exploration.\n\n# Analysis\n- The authors claim that the proposed method results in a contextualized noise schedule, which makes sense to me in terms of the scheduler generating Meta-Instructions conditioned on $\\boldsymbol{w}^x$. However, from Figure 3, even though the total amount of noise indeed fluctuates across different training epochs, it always keeps at the same level and the differences seem insignificant. I'm wondering why this makes the proposed methods beat existing baselines.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper presents a novel approach for contextualized noise scheduling in sequence-to-sequence diffusion models. While initially there were some concerns on the novelity side and the up-to-date experiments, I think the authors have addressed these well in their rebuttal. They provided additional experiments comparing to recent baselines including both new continous diffusion baselines and discrete diffusion models, demonstrated applicability to machine translation tasks, and showed plug-and-play capabilities with other models. The core idea is innovative and results show consistent improvements over existing methods across multiple datasets and tasks. Given the thorough response and additional results, I recommend accepting this paper despite its borderline status.
Decision: Accept (poster)