Meta-Diffu$B$: A Contextualized Sequence-to-Sequence Text Diffusion Model with Meta-Exploration

Authors: yun-yen  chuang
hung-min  hsu
kevin  lin
chen-sheng  gu
ling zhen li
ray-i  chang
hung-yi  lee
maxora  ai
Venue: NeurIPS_cc_2024_Conference
Published: 2024-04-30 07:15:51

Abstract: The diffusion model, a new generative modeling paradigm, has achieved significant success in generating images, audio, video, and text. It has been adapted for sequence-to-sequence text generation (Seq2Seq) through DiffuSeq, termed the S2S-Diffusion model. Existing S2S-Diffusion models predominantly rely on fixed or hand-crafted rules to schedule noise during the diffusion and denoising processes. However, these models are limited by non-contextualized noise, which fails to fully consider the characteristics of Seq2Seq tasks. In this paper, we propose the Meta-Diffu$B$ framework—a novel scheduler-exploiter S2S-Diffusion paradigm designed to overcome the limitations of existing S2S-Diffusion models. We employ Meta-Exploration to train an additional scheduler model dedicated to scheduling contextualized noise for each sentence. Our exploiter model, an S2S-Diffusion model, leverages the noise scheduled by our scheduler model for updating and generation. Meta-Diffu$B$ achieves state-of-the-art performance compared to previous S2S-Diffusion models and fine-tuned pre-trained language models (PLMs) across four Seq2Seq benchmark datasets. We further investigate and visualize the impact of Meta-Diffu$B$'s noise scheduling on the generation of sentences with varying difficulties. Additionally, our scheduler model can function as a "plug-and-play" model to enhance DiffuSeq without the need for fine-tuning during the inference stage.

Main paper: 
 introduction: 
 The diffusion model, a novel generative approach, operates through a two-step process: it first introduces noise to real data and then systematically removes this noise to facilitate data generation [12, 40, 30] . This model has demonstrated significant efficacy across several domains, including image [13, 29, 38] , audio [36, 18] , video [18, 14] , and text generation [1, 15, 21, 3, 24, 34] . The diffusion model utilizes a technique known as noise scheduling to control the amount of noise imposed at each diffusion step [12] . DiffuSeq [8] has adapted this model to discrete generation tasks like sequence-to-sequence text generation (Seq2Seq), under a framework termed S2S-Diffusion. However, DiffuSeq employs fixed noise scheduling and does not accommodate the specific characteristics of Seq2Seq tasks [45, 44] .Seq2Seq is a foundational technique in natural language processing (NLP) that generates target sentences from specified conditional sentences. It supports a range of downstream tasks, including language translation [41] , image captioning [35] , conversational modeling [39] , and text summarization [28] . For Seq2Seq tasks, it is more reasonable to impose different levels of noise to each sentence in S2S-Diffusion models to address the varying semantic and contextual difficulties of generating sentences. This noise scheduling strategy can better adapt to the semantic characteristics and generation difficulties of each sentence, thereby improving the model's performance in various generation tasks. To meet the unique demands of S2S Diffusion, we introduce a contextualized noise-scheduling strategy that accounts for the semantics of each conditional sentence and adapts to different training epochs. Existing S2S-Diffusion models, such as DiffuSeq, lack flexibility due to their reliance on fixed, non-contextualized noise-scheduling strategies. Furthermore, models like SeqDiffuSeq [45] and Dinoiser [44] , which propose adaptive noise scheduling, are also limited by their non-contextualized approach.To address the semantics of discrete conditional sentences for contextualized noise scheduling, we introduce a novel scheduler-exploiter framework, Meta-DiffuB, which achieves trainable noisescheduling inspired by Meta-Exploration [43] . Within this framework, our scheduler model dynamically schedules noise to train our exploiter model, which is updated based on the performance rewards it generates. Our exploiter model, an S2S-Diffusion model, leverages the noise scheduled by the scheduler model for updates and generation. By design, Meta-DiffuB naturally implements contextualized noise scheduling. It achieves state-of-the-art performance on four Seq2Seq benchmark datasets, outperforming existing S2S-Diffusion models [8, 45, 44] and fine-tuned pre-trained language models (PLMs) [10, 33] .In summary, we make three primary contributions with Meta-DiffuB:• We introduce and demonstrate the application of Meta-Exploration to diffusion models in Section 3, proposing Meta-DiffuB as a strategy to enhance S2S-Diffusion models. Our main results, presented in Section 6.1, confirm that Meta-DiffuB achieves state-of-the-art performance across four benchmark datasets.• We detail the operation of our scheduler model in Section 6.2, highlighting its capability to schedule noise. The noise scheduling approach of our scheduler model-applying less noise to the harder sentences and more to the easier ones-enhances the diversity and quality of the generated text.• We reveal that our scheduler model can function as a "plug-and-play" model, easily integrated into existing S2S-Diffusion models to enhance inference performance, as detailed in Section 6.3.2 Problem Statement, Preliminary 
problem statement: 
 In this work, we focus on sequence-to-sequence text generation tasks. Given a conditioning sentence of length m, w x = {w x 1 , . . . , w x m }, our objective is to train a diffusion model capable of generating a target sentence of length n, w y = {w y 1 , . . . , w y n }, based on the conditional sentence. Here, w x and w y represent the conditional and target sentences, respectively.DiffuSeq [8] primarily follows the transformation method of Diffusion-LM [21] and incorporates the diffusion and denoising processes from [12] . In the diffusion process, Diffusion-LM transforms discrete sentences into a continuous space. Given the real-world training sentence pair w x⊕y , concatenated by w x and w y , Diffusion-LM uses an embedding function emb to transform w x⊕y into continuous space, thereby obtaining the distribution z 0 ∼ q(z), where q represents the diffusion process. Then, z 0 is subjected to imposed noise, diffusing into a standard Gaussian distribution z T ∼ N (0, I). At each diffusion step t ∈ [1, 2, . . . , T ], the noise is regulated by q(z t |z t-1 ) = N (z t ; √ 1 -β t z t-1 , β t I), where β t ∈ (0, 1) controls the amount of noise imposed at each diffusion step. We denote β as containing a set of noise values β t , where a larger β t indicates more Gaussian noise imposed at that diffusion step. When t is large enough, z 0 gradually evolves into a standard Gaussian noise distribution. The random distribution is gradually reduced in noise during the denoising process to regenerate target sentences. The denoising process, which recovers z 0 by reducing the noise in z t , can be defined as follows:EQUATIONDiffusion-LM employs a trained, parameterized denoising distribution z t-1 ∼ p θ (z t-1 |z t ) to gradually recover z t from noise. This denoising distribution, parameterized by θ, is tailored to fit the posterior distribution q(z t-1 |z t , z 0 ) of the forward process. The key difference between DiffuSeq [8] and Diffusion-LM [21] is that DiffuSeq imposes noise only on the target sentence part of z t to achieve classifier-free S2S Diffusion, termed Partial Noise [8] . Due to the implementation of Partial Noise in the diffusion process, conditional denoising is inherently classifier-free. To transform the continuous z 0 target sentences back into discrete sentences w y , previous S2S-Diffusion models use a Rounding Operation [21] to map the target sentence part of z 0 into w y . The Rounding Operation is a method for choosing the most probable word for each position [21] . The denoising process primarily utilizes the variational lower bound (L VLB ) to optimize the negative log-likelihood [12] . Through the simplification and derivation from DiffuSeq [8] , the training objective function for S2S-Diffusion models can be defined as:EQUATIONwhere learning process p θ (z t-1 |z t ) is modeled as Transformer model f θ . Previous diffusion models deploy β by dividing the interval between the minimum value β 1 and the maximum value β T using a mathematical function to determine the fixed noise sequence {β 1 , ..., β T } ∈ β, as described in [12] . The mathematical function used by Diffusion-LM and DiffuSeq [21] is the sqrt function, which has demonstrated superior performance in text generation compared to other fixed mathematical functions. However, DiffuSeq's noise scheduling is constrained by its non-contextual approach; it does not account for the semantics of each conditional sentence nor does it adapt to different training epochs. 
methodology: 
 In this work, we propose a scheduler-exploiter framework named Meta-DiffuB for training S2S-Diffusion models with contextualized noise. Inspired by [43] , our Meta-DiffuB includes a scheduler model, B ψ , parameterized by ψ, and an exploiter model, D θ , parameterized by θ. B ψ , a simple Seq2Seq model, considers the semantics of conditional sentences to schedule contextualized β for updating D θ and is also updated based on the learning effectiveness of D θ -which refers to how well the exploiter learns. Our exploiter, D θ , an S2S-Diffusion model, leverages the noise scheduled by B ψ for its updating and generation. The framework of our Meta-DiffuB, compared with DiffuSeq, is visualized in Figure 1 .In this work, we propose a simple two-step approach for our scheduler B ψ , which is a Seq2Seq We propose a 'skipping' method: a True label directs B ψ to increase the noise by selecting β t+1 for the next diffusion step, whereas a False label maintains the same noise level β t . In the second step, we transform ι x using the fixed noise sqrt-function β sqrt = {β 1 , . . . , β T }, as deployed by [21, 8] , through the 'skipping' method to generate the new noise values β x = {β x 1 , . . . , β x T }. For example, with the continuous Meta-Instructions ι x = {T, F, T } and fixed noise values {1, 2, 3}, our new scheduling of noise values will be {1, 1, 2}. If consecutive scheduling noise values are the same, no additional noise is introduced at that diffusion step [12] . Our two-step approach maintains the same diffusion steps for parallel operations and contextualized β x in the diffusion process. We utilize a Policy Gradient to update our scheduler model following Meta-Exploration, addressing the non-differentiability of our two-step approach. The noise-scheduling mechanism of our scheduler model can be defined by the following equations:ι x = B ψ (w x ) β x = skipping(ι x , β sqrt ).(3)Unlike previous S2S-Diffusion models [8, 45, 44] that employ fixed or hand-crafted noise scheduling, we utilize contextualized β x to impose noise during the diffusion process. We also implement Partial Noise to achieve classifier-free S2S Diffusion [8] . In the denoising process, our exploiter model D θ restores the diffused data to generate the target sentences. During the diffusion process, we adopt the transformation method of Diffusion-LM to obtain emb(w x⊕y ), as described in Section 2. We extend the original diffusion chain to a new Markov transition with our β x : q ϕ (z 0 |w x⊕y ) = N (emb(w x⊕y ), β x 0 I) [21, 8] . Consequently, we can implement the objective function indicated in Section 2, derived from previous classifier-free S2S-Diffusion methods, to update our exploiter model D θ [8] . The training objective function for our exploiter model D θ in collaboration with B ψ can be defined as follows:EQUATIONSince z 0 is not diffused, there is no need to add the superscript of B ψ . J(θ) is denoted as the gradient for updating exploiter model D θ . Then, we can update our exploiter model D θ 's network weights:θ ′ → θ + ∇ θ J(θ).(5)In the inference stage, if our goal is to generate outputs based on w x , B ψ predicts contextualized β x using w x , as demonstrated in Section 3.1. We then concatenate emb(w x )-transformed from 3). θ e ← θ + ∇ ψ J(θ) e by Eq. ( 4) and Eq. ( 5) Estimate the Meta-Reward R e B ψ as described in Section 3.4. Compute the gradient ∇J(ψ) e by Eq. (6) . end for ψ ′ ← ψ + E e=1 ∇J(ψ) e by Eq. (7) . { Scheduler Update } B ψ ′ schedules noise β x by Eq. (3). θ ′ ← θ + ∇ θ J(θ) ′ by Eq. ( 4) and Eq. ( 5). { Exploiter Update } until Meta-DiffuB converges w x -with a randomly sampled y T ∼ N (0, I) to form z T . Our D θ predicts z 0 directly from z t and uses β x to convert the predicted z 0 into z t-1 . This step-by-step denoising process progressively recovers z t back to z 0 , following the methodologies outlined in [21, 45, 44] . Finally, we use a Rounding Operation to convert the target sentence part of z 0 into discrete target sentences.In this section, we estimate the Meta-Reward of our scheduler model, which reflects the learning effectiveness of D θ . We let D θ generate Y D θ and D θ ′ generate Y D θ ′ , respectively, where Y denotes the generated w y [21] . We assess the rewards for Y D θ and Y D ′ θ , denoted as R D θ and R D θ ′ respectively, which represent the rewards for D θ and D θ ′ . In this study, we utilize the BLEU score to quantify these rewards. Consequently, the reward for the scheduler model (i.e., Meta-Reward) is defined asRB ψ = R D θ ′ -R D θ .Since B ψ generates Meta-Instructions to diffuse sentences using our two-step approach described in Section 3.1, we update the scheduler via policy gradients, incorporating both Meta-Instructions and the calculated Meta-Rewards [43] . The training objective function for our scheduler model is defined as follows:EQUATIONAfter we obtain ∇ ψ J(ψ), we can update B ψ 's network weights:EQUATIONInspired by the exploration epochs of Meta-Exploration [43, 5, 19] , we iteratively execute the procedures from Section 3.1 to Section 3.4 to collect various indicators of learning effectiveness from D θ for updating B ψ . In practice, we keep the network weights of D θ fixed until the exploration epochs are completed. This approach ensures that the scheduler model schedules noise to D θ with consistent network weights, promoting stable training [5] . Additionally, we can conduct the exploration epochs in parallel to save time by collecting learning effectiveness from D θ under consistent network weights. After accumulating the gradients for B ψ from these exploration epochs, we update B ψ to B ψ ′ , which in turn schedules new noise to update D θ to D θ ′ . In summary, we present Algorithm (1) to detail the full training process of the proposed Meta-DiffuB. The number of exploration epochs is denoted by E, with e ∈ {1, ..., E} indexing the exploration epochs.Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?Answer: [Yes] Justification: We present the full set of assumptions and a complete proof of our Meta-DiffuB in Section 2 and Section 3.Guidelines:• The answer NA means that the paper does not include theoretical results.• All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced. • All assumptions should be clearly stated or referenced in the statement of any theorems.• The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition. • Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material. • Theorems and Lemmas that the proof relies upon should be properly referenced. 
experiments: 
 In this section, we conduct experiments to verify the performance of our Meta-DiffuB on four benchmark Seq2Seq datasets [48, 6, 17, 8] . We benchmark Meta-DiffuB against previous S2S-Diffusion models and fine-tuned pre-trained language models (PLMs), using the same datasets and training settings as employed by DiffuSeq [8] .In our experiment, we use four datasets: the Commonsense Conversation dataset (CC) [48] , the Quasar-T dataset (QT) [6] , the Wiki-Auto dataset (WA) [17] , and the Quora Question Pairs dataset (QQP) [8] . These datasets consider a variety of tasks, including open-domain dialogue generation, question generation, text simplification, and paraphrase generation tasks, all within Seq2Seq contexts. For a fair comparison, we employ the same datasets with identical settings for training all mentioned models, as outlined in [8, 45] . Detailed settings of these datasets are provided in Appendix A.We compare the proposed Meta-DiffuB with previous S2S-Diffusion models, including DiffuSeq [8] , Dinoiser [44] , and SeqDiffuSeq [45] . DiffuSeq employs a fixed noise pattern in the training and inference stages using a sqrt function and has been successfully introduced to the Seq2Seq task as the basic diffusion model. We also compare Meta-DiffuB with Dinoiser and SeqDiffuSeq, which are existing S2S-Diffusion models that focus on noise scheduling. Dinoiser and SeqDiffuSeq utilize handcrafted rules that provide adaptive but not contextualized noise scheduling. Additionally, following [8, 45] , we compare our Meta-DiffuB with three PLMs on Seq2Seq tasks. These PLMs include the fine-tuned GPT-2-base (GPT2-base) [33] , fine-tuned GPT-2-large (GPT2-large), and fine-tuned Levenshtein Transformer (LevT) [10] . We detail these baselines in Appendix B.Our exploiter model employs the same network architecture and settings as DiffuSeq [8] . Our scheduler model uses the same network architecture as described in [4] . The exploiter model and scheduler architectures are detailed in Appendix C. For consistent comparison, all S2S-Diffusion models [8, 44, 45] follow the experimental settings of prior research [8] and are trained from scratch. The diffusion step count is set at 2,000, and the maximum sequence length is 128. The Minimum Bayes risk (MBR) [23] decoding size, denoted as |S|, is 10; this involves generating sentences from 10 random seeds and selecting the best output sequence. Details on the implementation of MBR for all S2S-Diffusion models can be found in Appendix 6. The total batch size for both training and testing phases is 2048. Experiments are conducted on NVIDIA A100 Tensor Core GPUs, utilizing 4 GPUs for training and a single GPU for inference.To ensure a fair comparison during parallel exploration epochs, we avoid increasing the total batch size. Instead, we reduce the batch size by dividing the total batch size by the number of exploration epochs deployed. In this work, we set the number of exploration epochs to 32 and the batch size to 64.To update our scheduler, we run parallel exploration epochs every 100 training epochs with a total batch size of 2048. The increased computational complexity of applying Meta-DiffuB to DiffuSeq is presented in Table 1 . To ensure a fair comparison, we follow the same evaluation metric settings as those used in previous S2S-Diffusion models [8, 45] . For quality assessment, we utilize standard text generation metrics such as BLEU [27] , ROUGE-L [2] , and BERTScore [46] , where higher scores indicate better performance.For diversity assessment, we apply general text generation diversity metrics, including Distinct Unigram (Dist-1) [2] and Self-BLEU [27] , where lower scores of Self-BLEU and higher scores of Dist-1 signify better performance. Due to the application of multiple evaluation metrics (such as BLEU, ROUGE-L, BERTScore, Dist-1, and Self-BLEU), we also use Mean-Rank (M-R) to measure whether each model performs the best across multiple metrics [20] . A lower Mean-Rank score indicates consistently better performance across various metrics in the dataset. Details on the evaluation metric settings and their explanations are provided in Appendix D.Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)? • The answer NA means that the paper does not include experiments.• If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not. • If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable. • Depending on the contribution, reproducibility can be accomplished in various ways.For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed. • While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. , with an open-source dataset or instructions for how to construct the dataset). (d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility.In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 10. Broader Impacts Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed? Answer: [Yes] Justification: We have included a Broader Impact section in Section 8 to discuss the potential positive societal impacts of our work, outline the appropriate use of our publicly available code, and prevent misuse of our resources. Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).• The paper should discuss whether and how consent was obtained from people whose asset is used. • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
model-agnostic characteristics of meta-diffub: 
 We conduct experiments on applying our Meta-DiffuB to other S2S-Diffusion models. Specifically, we use Meta-DiffuB to modify the handcrafted noise-scheduling strategies of Dinoiser [44] and SeqDiffuSeq [45] on the WA and QQP datasets. The results, shown in Table 2 , demonstrate that Meta-DiffuB can be considered a model-agnostic method for enhancing the performance of other S2S-Diffusion models. Additionally, we provide results for applying our Meta-DiffuB to RDM [47] (based on D3PM [47] ) and other recent S2S-Diffusion models [42, 7, 22, 9] , which are based on DiffuSeq [8] on machine translation datasets [31, 26] in Appendix E. Diffusion-LM proposes using Minimum Bayes Risk (MBR) to improve generation. Following the methods described in [45, 8] , we allow all S2S-Diffusion models to generate a set of candidate sentences from 10 random seeds and select the best output sequence that achieves the minimum expected risk under a meaningful loss function. Specifically, in this work, we employ the BLEU score as our loss function to evaluate performance, following the approach used in DiffuSeq [8] . We compare our Meta-DiffuB (D θ = DiffuSeq) with DiffuSeq [8] and GPT-2 [33] , using MBR decoding [21, 8, 45] on the WA and QQP datasets as described in DiffuSeq [8] . We specifically select GPT2-large and GPT2-base for comparison based on their superior performance on these datasets [8] .In this experiment, we apply MBR decoding to all three models while gradually increasing the candidate sentence size |S|. The results of the MBR decoding are presented in Figure 2 . on the QQP dataset. As the candidate size grows, Meta-DiffuB (D θ = DiffuSeq) also surpasses GPT2-base on the WA dataset. Notably, Meta-DiffuB (D θ = DiffuSeq) exhibits diverse generation capabilities and achieves significantly better performance than DiffuSeq in MBR experiments. 
experiment with seq2seq benchmark datasets: 
 We demonstrate the performance of our Meta-DiffuB (D θ = DiffuSeq) in Table 3 . This table shows that Meta-DiffuB (D θ = DiffuSeq) outperforms other PLMs [33, 10] and S2S-Diffusion models [8, 44, 45] in terms of generation quality and diversity, achieving the lowest M-R scores across four datasets. Moreover, Meta-DiffuB (D θ = DiffuSeq) demonstrates significant improvements over previous S2S-Diffusion models [8, 45, 44] on all evaluation metrics considered.For the experiment involving contextualized noise of Meta-DiffuB (D θ = DiffuSeq), we selected the 200 hardest and 200 easiest generated sentences, labeled as (H) and (E), respectively. All models listed in Table 3 assessed the generation difficulty of each sentence using BLEU scores, with lower BLEU indicating higher difficulty. The performance of generating (H) and (E) is evaluated in terms of BLEU and Self-BLEU, as shown in Table 4 . We also detail the results of Table 4 evaluated by other metrics in Appendix G. We tasked all S2S-Diffusion models with scheduling noise for sentences (H) and (E), as shown in Figure 3 . Since our Meta-DiffuB (D θ = DiffuSeq) assigns specific noise to each sentence, we averaged the noise values for clearer visualization. Figure 3 displays the last 10 diffusion steps, as the noise differences in the initial steps are minimal. In Table 4 , Meta-DiffuB (D θ = DiffuSeq) consistently outperforms other S2S-Diffusion models [44, 45, 8] in terms of generation quality and diversity for sentences (E) and (H). Notably, when generating the more challenging sentences (H), Meta-DiffuB Meta-DiffuB (D θ = DiffuSeq) maintains its performance, whereas other S2S-Diffusion models [44, 45, 8] experience a decline in both quality and diversity. Figure 3 shows the benefits of Meta-DiffuB (D θ = DiffuSeq), which strategically imposes different noise levels for sentences (E) and (H). This noise-scheduling approach-applying less noise to the harder sentences (H) and more to the easier sentences (E)-enhances the diversity and quality of the generated text.The superior example of Meta-DiffuB in generating the hardest sentences (H) is further showcased in Appendix F, demonstrating its performance relative to other S2S-Diffusion models [44, 45, 8] . We also provide a more detailed discussion about the noise strategy of our Meta-DiffuB in Appendix H.Our pre-trained scheduler model, trained under Meta-DiffuB (D θ = DiffuSeq), which incorporates the semantics of discrete sentences, demonstrates its effectiveness by scheduling noise for pre-trained DiffuSeq [8] models across various datasets. The results, presented in Table 5 , show that our pre-trained scheduler model, when applied across different datasets, enhances the performance of pre-trained DiffuSeq models without any fine-tuning during the inference stage. This confirms that our scheduler model can function as a plug-and-play model across these datasets. Additionally, we provide further experiments on different pre-trained schedulers under various S2S-Diffusion settings, as well as results on additional datasets in Appendix I.Table 3 : We present the results of our Meta-DiffuB (D θ = DiffuSeq) compared with other models across four Seq2Seq datasets. We report the scores of DiffuSeq and PLMs from [8] . A star (⋆) indicates results reported directly from previous studies, while a dagger ( †) signifies that we reproduced the results because the previous studies did not report them using the same metrics on these datasets.The best results among S2S-Diffusion models are underlined, and the overall best results are in bold. [21] and AnalogBits [3] propose imposing noise on continuous latent representations, using transformation functions to bridge the discrete and continuous spaces of texts for both unconditional and controlled text generation.Tasks Methods BLEU (↑) ROUGH-L (↑) BERTScore (↑) Dist-1 (↑) Self-BLEU (↓) M-R (↓) QQP ⋆ GPT2- 
meta-exploration: 
 To transcend the limitations imposed by human-crafted rules in noise scheduling, we developed an additional model trained through Meta-Exploration, as inspired by [43] . Meta-Exploration is a Reinforcement Learning (RL) training method that utilizes learning effectiveness to devise sampling strategies that enhance model performance. Numerous studies [5, 37, 25, 19, 16] have employed Meta-Exploration to meta-learn scheduling strategies for applying additive Gaussian noise on actions and for sampling effective training data in RL tasks. We have adopted the Meta-Exploration concept [43] to train an additional model specifically for noise scheduling in S2S-Diffusion. 
broader impact: 
 In this work, our Meta-DiffuB demonstrates significant performance improvements over previous S2S-Diffusion models across four Seq2Seq tasks, as detailed in Section 4.1. Meta-DiffuB implements learnable, contextualized noise scheduling for Seq2Seq tasks. It not only shows enhanced generation quality and diversity but also has the potential to be applied to other diffusion models that require conditional data learning to generate target data. However, it is important to note that using Meta-DiffuB to create fake news or other forms of misinformation is strongly discouraged. 
conclusions: 
 We propose integrating Meta-Exploration into S2S-Diffusion models through our newly developed Meta-DiffuB. By utilizing Meta-Exploration to schedule contextualized noise, our Meta-DiffuB model demonstrates significant performance improvements on four Seq2Seq benchmark datasets compared to previous S2S-Diffusion models and PLMs. We have conducted a comprehensive investigation of the noise-scheduling capabilities of Meta-DiffuB and have visualized the results. Importantly, Meta-DiffuB has the potential to act as a plug-and-play model, providing a promising approach for enhancing other S2S-Diffusion models during the inference stage without the need for fine-tuning. 
acknowledgments: 
 We would like to express our sincere gratitude to Professor Hung-yi Lee from NTU Speech Lab for his invaluable guidance and insightful advice throughout this work. We are also deeply grateful to Professor Ray-I Chang from NTU ICAN Lab for his mentorship and constructive feedback. Additionally, we would like to thank the reviewers for their positive evaluation and valuable suggestions. Finally, we extend our appreciation to Maxora AI for providing the computational resources and environment that made this research possible, enabling us to make meaningful contributions to the field. 
 
 Appendix: There are four datasets in our experiment. For a fair comparison, we use the same datasets with the same settings as those described in [8, 45] . Below, we detail the datasets used:• Commonsense Conversation Dataset (CC) [48] : CC requires models to generate informative responses given a dialogue context, an open-domain dialogue generation task. Extracted from Reddit single-round dialogues, it includes over 3 million conversational pairs. The training set contains 3,382,137 pairs, the development set has 2,048, and the test set includes 10,000 pairs. We train all S2S-Diffusions for 140,000 epochs on this dataset. • Quasar-T Dataset (QT) [6] We train all S2S-Diffusions for 140,000 epochs on this dataset.We provide details on the PLMs [33, 10] proposed by [8] and other S2S-Diffusion models [44, 45] . DiffuSeq fine-tunes PLMs to achieve optimal performance, balancing the trade-off between quality and diversity on the development set.• Dinoiser [44] We demonstrate the details of the evaluation metrics used in this work. To ensure a robust comparison, we adhere to the same evaluation metric settings as previous S2S-Diffusion models [8, 45] .• BLEU [27] : BLEU is widely used to measure the quality of text generation. In this work, we use a smoothed BLEU score ranging from BLEU-1 to BLEU-4, where higher scores indicate better quality.• ROUGE-L [2] : ROUGE-L measures text generation quality by calculating the longest common subsequence. Higher ROUGE-L scores indicate better quality.• BERTScore [46] : BERTScore assesses text generation quality by calculating the embedding similarity between generated sentences and reference sentences. It utilizes embeddings from a BERT model, with higher scores indicating better quality.• Dist-1 [2] : Dist-1 measures the diversity of generated text by calculating the uniqueness of words within a single generated target sentence.• Self-BLEU [27] : Self-BLEU assesses the diversity of text generation by calculating the ratio of unique 4-grams in a set of generated sentences. Lower Self-BLEU scores indicate better diversity.Following Section 5, we conduct experiments on machine translation datasets, including IWSLT14 DE-EN [26] and WMT14 DE-EN [31] , using the same dataset and evaluation metric settings as other S2S-Diffusion models [45, 44] . Specifically, we adopt SacreBLEU [32] as the evaluation metric. As shown in Table 7 , our Meta-DiffuB improves the performance of DiffuSeq, Dinoiser, and SeqDiffuSeq on these machine translation tasks. Additionally, we provide experiments on DiffuSeqbased S2S-Diffusion models and discrete S2S-Diffusion models (RDM [47] based on D3PM [1] ) on the QQP and QG datasets. As shown in Table 7 , our Meta-DiffuB consistently improves performance across both discrete and DiffuSeq-based S2S-Diffusion models. = DiffuSeq). It consistently generates the hardest sentences (H) with superior quality and diversity, providing a reliable solution. In contrast, other S2S-Diffusion models [8, 44, 45] often produce repetitive sentences, failing to ensure both quality and diversity. Our findings, as illustrated in Table 9 , show that Meta-DiffuB (D θ = DiffuSeq) also outperforms other models, generating sentences (H) with superior diversity and quality on the QQP dataset. We present the results of Table 4 evaluated by other metrics in Table 10 . Table 10 illustrates that our Meta-DiffuB (D θ = DiffuSeq) outperforms other S2S-Diffusion models in generating sentences (H) and (E) across all evaluation metrics in this study.In this section, we further discuss the noise generated by our Meta-DiffuB. As shown in Figure 4 , the total noise per epoch produced by Meta-DiffuB exhibits less fluctuation compared to other rule-based methods. While the noise variance is smaller than that of SeqDiffuSeq, it is larger than Dinoiser. From Table 3 , we can see that the magnitude or variability of the noise does not necessarily correlate with We present the results of the plug-and-play experiment featuring our scheduler model and DiffuSeq [8] trained on different datasets in Table 11 . This table demonstrates that our scheduler model can enhance the performance of DiffuSeq across four datasets during the inference stage without the need for fine-tuning. We also provide experiments with our scheduler on various other S2S-Diffusion models. As shown in Table 12 and Table 13 , our scheduler not only improves performance across datasets but also enhances the performance across different models. Justification: We specify that this work focuses on the noise scheduling of diffusion models for Seq2Seq tasks in Section 1. Additionally, we discuss the computational intensity of introducing Meta-Exploration in Section 4.3.1.Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.Code and datasets for Meta-DiffuB are available at: https://github.com/Meta-DiffuB/ Meta-DiffuB. 


Decision: This paper presents a novel approach for contextualized noise scheduling in sequence-to-sequence diffusion models. While initially there were some concerns on the novelity side and the up-to-date experiments, I think the authors have addressed these well in their rebuttal. They provided additional experiments comparing to recent baselines including both new continous diffusion baselines and discrete diffusion models, demonstrated applicability to machine translation tasks, and showed plug-and-play capabilities with other models. The core idea is innovative and results show consistent improvements over existing methods across multiple datasets and tasks. Given the thorough response and additional results, I recommend accepting this paper despite its borderline status.

Reviews: review: paper_summary: paper_summary: This paper proposes a meta-learning based noise scheduler to incoporate contextualisation in texts. The scheduler is a plug-and-play module which could be applied to other similar sequential setting. The experimental results demonstrate its superiority comparing to the baselines, in terms of generating quality and diversity. Besides, the ablation study investigates into and provide insights about the relationship of the difficulty levels and noise schedule, which is quite interesting. The paper is well organised and easy to follow.
strength_weakness: strength_weakness: While the diffusion process is now designed to incoporate a time variable noise schedule, the denoising process is still supposed to handle fixed denoising steps. More insights on how this will be handled effectively would be expected. 

Some statements in the text seems not align well with the information disclosing by the figures. For example, Figure 3, the noise level is quite stable for the proposed method but in the text below, it claims 'Meta-DiffuB applies adaptive noise at varying training epchos'. I am wondering if there are any typos involved. Another example is in Figure 4 at least for QQP dataset - Meta-DiffuB applies less noise than the other methods as its curves lay below the curves of the other methods. thus, the claim 'This noise-scheduling approach—applying more noise to the harder sentences (H) and less to the easier sentences' looks conflict with such observation from the figure. 

Besides, as algorithm 1 alternates the training of scheduler and exploiter, It would be guideful if some insights about the convergence rate are provided.
questions: questions: I have three major concerns as listed in the weakness section about the reasonability, the consistence of figures and in-text description about some experiemtnal results, and the time complexity of training algorithm.
limitations: limitations: I suspect the main limitation would be the converge speed but it was not mentioned in the paper. 

The schedular is implemented with a Seq2Seq structure, and other options could be provided as well to inspire other potential application scenarios.
paper_summary: paper_summary: The paper introduces the Meta-DiffuB, a scheduler-exploiter diffusion framework that focuses on sequence-to-sequence (Seq2Seq) setting. Its novel trainable contextualized noise scheduler, inspired by Meta-exploration, is also flexible and plug-and-play with other models like DiffuSeq without re-training. This approach dynamically schedules noise levels based on the characteristics of each sentence, improving the performance of text generation tasks. Meta-DiffuB outperforms existing models and fine-tuned pre-trained language models across four Seq2Seq benchmark datasets, demonstrating the effectiveness of its adaptive noise scheduling.
strength_weakness: strength_weakness: - While the model performs well on benchmark datasets, there is limited discussion on its scalability to larger, real-world datasets and more complex text generation tasks.
- The dynamic noise scheduling process might introduce additional computational overhead during inference, which is not thoroughly analyzed in the paper.
- Meta-DiffuB is built upon and compared with S2S-Diffusion models known as continuous-based text diffusion models. It lacks a detailed comparison with other discrete-based text diffusion models such as D3PM [1] or RDM [2]. RDM claimed to achieve better performance and inference speed compared to DiffuSeq.
- The effects of the dynamic noise scheduling on the overall training dynamics and convergence rate of the model are not explored in detail, which could be crucial for understanding the method’s effectiveness.
questions: questions: - L120, 123: the quote for 'skipping' is wrongly formatted
- L202-203: it should be either "The maximum Minimum Bayes Risk (MBR)..." or "The Minimum Bayes Risk (MBR)....", also it cites paper [20] (the Rouge package) which I think is incorrect for MBR.
- Besides the increased training time compared to DiffuSeq (L217-218), it would be beneficial to know the increased inference time too.
- How well does the Meta-DiffuB model generalize to other types of text generation tasks beyond the Seq2Seq framework? Have experiments been conducted to evaluate its performance in different text generation scenarios?
- How does the model scale with larger datasets or more complex text generation tasks? Are there any specific optimizations or modifications required to handle real-world data volumes?

[1] https://arxiv.org/pdf/2107.03006

[2] https://arxiv.org/pdf/2302.05737
limitations: limitations: Yes
paper_summary: paper_summary: Comprehensive Evaluation of the new S2S Diffusion framework Meta-Diffu$\beta$.
strength_weakness: strength_weakness: - **Incremental Innovation**: In terms of the scheduler, the use of the Meta-Exploration method for scheduling contextualized noise is noted. However, I concern that the exploiters fully follow the DiffuSeq work, resulting in a lack of significant innovation.
- **Missing Experimental Baselines**: The cited baselines are all from before February 2023. It would be beneficial to include more recent baseline results [1] [2] [3] and take both generation efficiency and training speed into evaluation metrics.
- **Dataset Limitations**: The selected dataset completely follows DiffuSeq and does not take into account machine translation tasks and related datasets from SeqDiffuSeq and DINOISER. It would be advantageous to include test results on IWSLT14 and WMT14.
- **Plug-and-Play Capability**: The study only demonstrates the plug-and-play model and its corresponding effects on DiffuSeq. It's recommended to show the plug-and-play capability in other S2S Diffusion models like SeqDiffuSeq, DINOISER and other baselines mentioned to prove the claim in the conclusion.

Minor: 

- The main text introduces the datasets but does not provide information about the downstream Seq2Seq tasks. Consider moving the introduction of these tasks from the appendix to the main text to enhance readability.

- The pre-trained model and environment dependencies are missing from your GitHub repository. It would be helpful to include these components to facilitate reproducibility.

[1] Ding, Y., Tian, J., Mei, S., Zhou, Y., Dong, Y., He, H. and Hu, W., 2023, December. LDSeq: Latent Diffusion Models for Sequence to Sequence Text Generation. In *Proceedings of the 2023 7th International Conference on Computer Science and Artificial Intelligence*

[2] Gong, S., Li, M., Feng, J., Wu, Z. and Kong, L., 2023. Diffuseq-v2: Bridging discrete and continuous text spaces for accelerated seq2seq diffusion models. *arxiv preprint arxiv:2310.05793*.

[3] Tang, Z., Wang, P., Zhou, K., Li, J., Cao, Z. and Zhang, M., 2023. Can Diffusion Model Achieve Better Performance in Text Generation? Bridging the Gap between Training and Inference!. *arxiv preprint arxiv:2305.04465*.
questions: questions: - Regarding the selection of fixed noise, have you attempted to use different mathematical functions to obtain the $\beta$ values as inputs for the skipping function? How is the performance using different $\beta$ in the scheduler?
limitations: limitations: The authors have addressed the limitations.
paper_summary: paper_summary: This paper presents a novel approach to improving sequence-to-sequence (Seq2Seq) text generation models using diffusion models. The authors identify limitations in existing Seq2Seq-Diffusion models, which typically rely on fixed or hand-crafted noise scheduling rules that do not account for the specific characteristics of Seq2Seq tasks. To address these limitations, they propose the Meta-DiffuB framework, which introduces a scheduler-exploiter paradigm designed to provide contextualized noise scheduling. The scheduler model dynamically schedules contextualized noise based on the semantics of each sentence. The scheduler model is trained using Meta-Exploration techniques to optimize noise scheduling and can function as a "plug-and-play" model. The exploiter model (an S2S-Diffusion model) utilizes the noise scheduled by the scheduler for updating and text generation.
strength_weakness: strength_weakness: # More Comprehensive Comparison
My first concern is that this paper mainly compares the proposed method with Diffuseq, SeqDiffuSeq, and Dinoiser. There are many other diffusion models tailored for general seq2seq tasks [1-6]. I would suggest the authors carefully review the existing literature and select more recent baselines to compare.

1. Empowering Diffusion Models on the Embedding Space for Text Generation. *NAACL*
2. Latent Diffusion for Language Generation. *NeurIPS*
3. DiffuSeq-v2: Bridging Discrete and Continuous Text Spaces for Accelerated Seq2Seq Diffusion Models. *EMNLP*
4. AR-Diffusion: Auto-Regressive Diffusion Model for Text Generation. *NeurIPS*
5. Can Diffusion Model Achieve Better Performance in Text Generation? Bridging the Gap between Training and Inference! *EMNLP*
6. TESS: Text-to-Text Self-Conditioned Simplex Diffusion. *EACL*

The authors may also find more related works in:
- https://github.com/StevenYuan666/Awesome-Diffusion-Models-for-NLP
- https://github.com/westfish/Awesome-NLP-Diffusion-Models
- https://github.com/AoiDragon/Awesome-Text-Diffusion-Models

Also, some more recent works focus on the noise schedule:

7. Effective Integration of Text Diffusion and Pre-Trained Language Models with Linguistic Easy-First Schedule. *LREC-COLING*

# Presentation
- As the paper mentioned, parts of the methodology are motivated by the Meta-Exploration. I would suggest the authors add a section in the preliminary to provide more necessary details about the meta-exploration techniques, which will benefit readers who are not familiar with Meta-Exploration.

# Analysis
- The authors claim that the proposed method results in a contextualized noise schedule, which makes sense to me in terms of the scheduler generating Meta-Instructions conditioned on $\boldsymbol{w}^x$. However, from Figure 3, even though the total amount of noise indeed fluctuates across different training epochs, it always keeps at the same level and the differences seem insignificant. I'm wondering why this makes the proposed methods beat existing baselines.
questions: questions: 1. Following my argument above, can you provide a deeper interpretation of the noise scheduling visualizations and their implications for different types (easy or hard as mentioned in the paper) of sentences? How do different noise schedules affect the generation quality and diversity for various types of sentences? Any patterns or insights observed from these visualizations?
2. Could you provide an intuitive explanation of why the scheduler can function as a plug-and-play model? From the analysis in section 4.7, I'm suspecting it is because the learned schedule is not quite different from the pre-defined noise schedule. 
3. Did you run the empirical evaluation for several runs by setting different random seeds? If so, could you also highlight which results are statistically significant than the baseline method?
limitations: limitations: The proposed method is in the domain of text generation and does not directly have any negative impact by itself. The authors have provided a paragraph to discuss the negative potential of misinformation created by the text generation model.

Comments: title: Thank you for your suggestion. To address your concerns, we have added more experiments.
Thank you very much for your suggestion. We have also supplemented the experiment you mentioned.
Thank you
Thank you for your valuable suggestions. We have included additional baseline model experiments and addressed each of your questions to clarify any doubts. We assure you that these experiments will be included in the final version.
Thank you for your valuable suggestions. We have included additional baseline model experiments, machine translation experiments, and a variety of different baselines. We have addressed each of your questions to clarify any doubts. We assure you that these experiments will be included in the final version.
Thank you for your valuable suggestions. We have included the inference times for the models and added machine translation experiments. In the final version, we will also address and revise the format and references you mentioned.
Thank you for your valuable suggestions. We have included the training curves and addressed your comments as below.
General Response
comment: >Q1: I still have concerns about its generalizability, as the authors only consider continuous-based diffusion models and do not consider discrete-based models.

In the discrete S2S-Diffusion model, a set of $\beta$ values is still required to control the magnitude of the noise. Our scheduler inherently generates these $\beta$ values to regulate the noise. In the discrete S2S-Diffusion model, by multiplying the $\beta$ values generated by the scheduler with a Bernoulli random vector instead of Gaussian noise, our Meta-Diffu$B$ framework can be seamlessly integrated. As a result, we also include a comparison with the Reparameterized Diffusion Model (RDM) in the Table below.

>Q2: While comparing S2S-Diffusion models before and after applying Meta-Diffu
 might be sufficient, the authors did not compare with other noise schedule baselines (as mentioned by reviewer qgkV).

In our study, we compared the noise scheduling from Denoiser, SeqDiffuSeq, and other S2S-Diffusion models as described in their papers. When paired with our scheduler, the noise scheduling consistently produced better results. The Diffusion-LEF paper (which proposed the Easy-First Schedule as noted by reviewer qgkV) used different datasets from ours. Therefore, we tested the Easy-First Schedule with DiffuSeq on the QQP and QG datasets. For a fair comparison, we used the Diffusion-LEF results without BERT. As shown in the table below, while the Easy-First Schedule improves DiffuSeq, it doesn't achieve the same level of enhancement as our Meta-Diffu$B$.


**We compared more recent baselines with our Meta-Diffu$B$ on the QG and QQP datasets. These baselines include both discrete and continuous S2S-Diffusion models.**
|Methods|BLEU ↑ (QQP)|BERTScore ↑ (QQP)|BLEU↑ (QG)| BERTScore ↑ (QG)|
|-|-|-|-|-|
|DiffuSeq|0.2413|0.8365|0.1731|0.6123|
|Diffusion (Easy-First Schedule)|0.2503|0.8692|0.1812|0.6253|
|Meta-Diffu$B$ (exploiter=DiffuSeq)|**0.2552**| **0.8821**|**0.1826**|**0.6357**|
|DiffuSeq-v2|0.2411|0.8393|-|-|
|Meta-Diffu$B$ (exploiter=DiffuSeq-v2)|**0.2556**| **0.8829**|-|-|
|BG-DiffuSeq|0.2619|0.8427|0.1744|0.6280|
|Meta-Diffu$B$ (exploiter=BG-DiffuSeq)|**0.2790**| **0.8757**|**0.1838**|**0.6571**|
|TESS|0.3020|0.8570|0.1950|0.6580|
|Meta-Diffu$B$ (exploiter=TESS)|**0.3142**|**0.8975**| **0.2055**|**0.6761**|
|RDM (Discrete Diffusion)|0.2510|0.8472|0.1802|0.6310|
|Meta-Diffu$B$ (exploiter=RDM)|**0.2684**|**0.8724**|**0.2271**|**0.6542**|
In addition to the baselines we previously supplemented, we have also included the easy-first schedule and the discrete S2S-Diffusion model you mentioned. The discrete S2S-Diffusion models, which rely on absorbing states, still require a set of $\beta$ values. Therefore, our Meta-Diffu$B$ can still be directly applied to the discrete S2S-Diffusion model. The paper on the easy-first schedule, Diffusion-LEF, did not implement the method on the same dataset as ours. Hence, we applied this noise schedule to DiffuSeq without using a pre-trained BERT model to ensure fairness. Below is the table summarizing the additional experiments we conducted.

**We compared more recent baselines with our Meta-Diffu$B$ on the QG and QQP datasets. These baselines include both discrete and continuous S2S-Diffusion models.**
|Methods|BLEU ↑ (QQP)|BERTScore ↑ (QQP)|BLEU↑ (QG)| BERTScore ↑ (QG)|
|-|-|-|-|-|
|DiffuSeq|0.2413|0.8365|0.1731|0.6123|
|Diffusion (Easy-First Schedule)|0.2503|0.8692|0.1812|0.6253|
|Meta-Diffu$B$ (exploiter=DiffuSeq)|**0.2552**| **0.8821**|**0.1826**|**0.6357**|
|DiffuSeq-v2|0.2411|0.8393|-|-|
|Meta-Diffu$B$ (exploiter=DiffuSeq-v2)|**0.2556**| **0.8829**|-|-|
|BG-DiffuSeq|0.2619|0.8427|0.1744|0.6280|
|Meta-Diffu$B$ (exploiter=BG-DiffuSeq)|**0.2790**| **0.8757**|**0.1838**|**0.6571**|
|TESS|0.3020|0.8570|0.1950|0.6580|
|Meta-Diffu$B$ (exploiter=TESS)|**0.3142**|**0.8975**| **0.2055**|**0.6761**|
|RDM (Discrete Diffusion)|0.2510|0.8472|0.1802|0.6310|
|Meta-Diffu$B$ (exploiter=RDM)|**0.2684**|**0.8724**|**0.2271**|**0.6542**|

As shown in the Table, when combined with the discrete S2S-Diffusion model RDM, we also achieve satisfactory results. Additionally, the noise generated by the scheduler through learning, compared to the noise produced by the easy-first schedule that relies on many rules, can further enhance the performance of DiffuSeq.
Thank you to the authors for their response. Since one of the key contributions of this paper is a strategy (noise scheduler) that enhances S2S-Diffusion models:
- I still have concerns about its generalizability, as the authors only consider continuous-based diffusion models and do not consider discrete-based models.
- While comparing S2S-Diffusion models before and after applying Meta-Diffu$\beta$ might be sufficient, the authors did not compare with other noise schedule baselines (as mentioned by reviewer qgkV).

Given these points, I will maintain my score.
Thank you for your detailed response. I have read through most of the comments by other reviewers, as well as the rebuttal. I decide to keep my original score.
>Q1: My first concern is that this paper mainly compares the proposed method with Diffuseq, SeqDiffuSeq, and Dinoiser. There are many other diffusion models tailored for general seq2seq tasks [1-6]. I would suggest the authors carefully review the existing literature and select more recent baselines to compare. 

**Response:**: Thank you for the insightful feedback. We have included additional baselines in the following Table, specifically those that are evaluated on the same dataset as ours and have open-source implementations (Diffuseq-v2, BG-DiffuSeq and TESS). We demonstrate that our Meta-Diffu$B$ framework can integrate various baselines as exploiter models, achieving superior performance compared to the original models. This is consistent with the experiments and results presented in Appendix F.

**More recent baselines compared with our Meta-Diffu$B$ on QG and QQP datasets**
|Methods|BLEU ↑ (QQP)|BERTScore ↑ (QQP)|BLEU↑ (QG)|BERTScore ↑ (QG)|
|-|-|-|-|-|
|DiffuSeq|0.2413|0.8365|0.1731|0.6123|
|Meta-Diffu$B$ (exploiter=DiffuSeq)|**0.2552**|**0.8821**|**0.1826**|**0.6357**|
|DiffuSeq-v2 |0.2411|0.8393|-|-|
|Meta-Diffu$B$ (exploiter=DiffuSeq-v2)|**0.2556**|**0.8829**|-|-|
|BG-DiffuSeq |0.2619|0.8427|0.1744|0.6280|
|Meta-Diffu$B$ (exploiter=BG-DiffuSeq)|**0.2790**|**0.8757**|**0.1838**|**0.6571**|
|TESS |0.3020|0.8570|0.1950|0.6580|
|Meta-Diffu$B$ (exploiter=TESS)|**0.3142**|**0.8975**|**0.2055**|**0.6761**|
    
>Q2: As the paper mentioned, parts of the methodology are motivated by Meta-Exploration. I would suggest the authors add a section in the preliminary to provide more necessary details about the meta-exploration techniques, which will benefit readers who are not familiar with Meta-Exploration.

**Response:** To assist readers in understanding the foundational concepts, we will move the discussion of Meta-Exploration from the Related Work section to the Preliminary section in our final version.

>Q3: The authors claim that the proposed method results in a contextualized noise schedule, which makes sense to me in terms of the scheduler generating Meta-Instructions conditioned on. However, from Figure 3, even though the total amount of noise indeed fluctuates across different training epochs, it always keeps at the same level and the differences seem insignificant. I'm wondering why this makes the proposed methods beat existing baselines.

**Response:** 
In Figure 3, our method differs from approaches like SeqDiffuSeq or Dinoiser, which rely on predefined rules. Instead, our approach is data-driven, with the model learning the underlying patterns. As a result, the degree of variation—whether large or small—is not the primary focus. Additionally, the learned noise can be flexibly integrated into other systems in a plug-and-play manner.

>Q4: Can you provide a deeper interpretation of the noise scheduling visualizations and their implications for different types (easy or hard as mentioned in the paper) of sentences? How do different noise schedules affect the generation quality and diversity for various types of sentences? Any patterns or insights observed from these visualizations?

**Response:** From the visualizations, we observe that the contextualized noise scheduling mechanism helps in maintaining a delicate balance between exploration (diversity) and exploitation (quality). The patterns indicate that our model has learned different strategies for different types of sentences. These insights demonstrate the effectiveness of our Meta-Diffu$B$ framework in handling various sentence complexities, resulting in superior performance across multiple datasets.

>Q5: Could you provide an intuitive explanation of why the scheduler can function as a plug-and-play model? From the analysis in section 4.7, I'm suspecting it is because the learned schedule is not quite different from the pre-defined noise schedule.

Our scheduler model learns directly from sentence semantics, allowing it to provide learnable noise across different text datasets. Figure 3 shows that Dinoiser and SeqDiffuSeq rely on rule-based noise variations, which do not necessarily lead to better performance.

    
>Q6: Did you run the empirical evaluation for several runs by setting different random seeds? If so, could you also highlight which results are statistically significant than the baseline method?

**Response:**  Thank you for your question regarding the empirical evaluation with different random seeds. We followed the evaluation protocols of DiffuSeq and Diffusion-LM, running multiple experiments with different random seeds. In Tables 1, 2, and 3, we used MBR, so the results in bold are all statistically significant compared to the baseline methods.
>Q1: Incremental Innovation: In terms of the scheduler, the use of the Meta-Exploration method for scheduling contextualized noise is noted. However, I concern that the exploiters fully follow the DiffuSeq work, resulting in a lack of significant innovation.

**Response**: Thank you for your feedback. As noted in Appendix F, our framework can utilize different diffusion models as exploiters, demonstrating its versatility. The key innovation is integrating these models with our scheduler, significantly enhancing performance. We propose a general method that strengthens other S2S-diffusion models, providing more than incremental improvements. In Sections 4.5 to 4.7, we compare our Meta-Diffu$B$, using DiffuSeq as the exploiter model, with other S2S-Diffusion models that incorporate adaptive noise to highlight its effectiveness.

>Q2: Missing Experimental Baselines: The cited baselines are all from before February 2023. It would be beneficial to include more recent baseline results [1] [2] [3] and take both generation efficiency and training speed into evaluation metrics.

**Response:** Thank you for the insightful feedback. We have included additional baselines in the following Table, specifically those that are evaluated on the same dataset as ours and have open-source implementations (Diffuseq-v2, BG-DiffuSeq and TESS). Results where Meta-Diffu$B$ combined with different models show improved performance are indicated in bold. Our Meta-Diffu$B$ framework can integrate various baselines as exploiter models, achieving superior performance compared to the original models. This is consistent with the experiments and results presented in Appendix F.

**More recent baselines compared with our Meta-Diffu$B$ on QG and QQP datasets**
|Methods|BLEU ↑ (QQP)|BERTScore ↑ (QQP)|BLEU↑ (QG)|BERTScore ↑ (QG)|
|-|-|-|-|-|
|DiffuSeq|0.2413|0.8365|0.1731|0.6123|
|Meta-Diffu$B$ (exploiter=DiffuSeq)|**0.2552**|**0.8821**|**0.1826**|**0.6357**|
|DiffuSeq-v2|0.2411|0.8393|-|-|
|Meta-Diffu$B$ (exploiter=DiffuSeq-v2)|**0.2556**|**0.8829**|-|-|
|BG-DiffuSeq|0.2619|0.8427|0.1744|0.6280|
|Meta-Diffu$B$ (exploiter=BG-DiffuSeq)|**0.2790**|**0.8757**|**0.1838**|**0.6571**|
|TESS|0.3020|0.8570|0.1950|0.6580|
|Meta-Diffu$B$ (exploiter=TESS)|**0.3142**|**0.8975**|**0.2055**|**0.6761**|

>Q3: Dataset Limitations: The selected dataset completely follows DiffuSeq and does not take into account machine translation tasks and related datasets from SeqDiffuSeq and DINOISER. It would be advantageous to include test results on IWSLT14 and WMT14.

**Response:** Thank you for the valuable feedback. We supplement additional machine translation datasets in the following table and employed the same evaluation metrics used by SeqDiffuSeq and Dinoiser for these tasks. Results where Meta-Diffu$B$ combined with different models show improved performance are indicated in bold. Furthermore, we demonstrate that our Meta-Diffu$B$ framework, when combined with different S2S-Diffusion models, achieves superior performance in machine translation tasks as well. 

**Experiment of Meta-Diffu$B$ on Machine Translation dataset** 
|Methods|SacreBLEU ↑ (IWSLT14 DE-EN)|SacreBLEU ↑ (WMT14 DE-EN)|
|-|-|-|
|DiffuSeq|29.43|22.72|
|Meta-Diffu$B$ (exploiter=DiffuSeq)|**31.71**|**26.17**|
|SeqDiffuSeq|30.16|23.28|
|Meta-Diffu$B$ (exploiter=SeqDiffuSeq)|**32.41**|**26.14**|
|Dinoiser|31.61|30.30|
|Meta-Diffu$B$ (exploiter=Dinoiser)|**33.82**|**32.09**|

>Q4: Plug-and-Play Capability: The study only demonstrates the plug-and-play model and its corresponding effects on DiffuSeq. It's recommended to show the plug-and-play capability in other S2S Diffusion models like SeqDiffuSeq, DINOISER, and other baselines mentioned to prove the claim in the conclusion.

**Response:** Thank you for your suggestion. We supplement our experiments with plug-and-play capabilities on Dinoiser and SeqDiffuSeq in the following tables. Our scheduler, trained on different datasets, improves the performance of both Dinoiser and SeqDiffuSeq. The fields "Dinoiser" and "SeqDiffuSeq" indicate which dataset these two models are trained on.  When the scheduler field is null, it indicates the use of the model's own noise scheduling. Results where the model performs better with its own noise are indicated in bold.

**Plug-and-play experiments on SeqDiffuSeq integrated with our scheduler**
|Scheduler|SeqDiffuSeq|BLEU ↑|BERTScore ↑|Dist-1 ↑ |
|-|-|-|-|-|
|WA|QQP|**0.2627**|**0.8481**|**0.9814**|
|Null|QQP|0.2434|0.8400|0.9807|
|WA|QT|**0.1834**|**0.6226**|**0.9369**|
|Null|QT|0.1746|0.6174|0.9248|

**Plug-and-play experiments on Dinoiser integrated with our scheduler**
|Scheduler|Dinoiser|BLEU ↑|BERTScore ↑|Dist-1 ↑|
|-|-|-|-|-|
|WA|QQP|**0.2079**|**0.8121**|**0.9765**|
|Null|QQP|0.1949|0.8036|0.9723|
|WA|QT|**0.0495**|**0.4740**|**0.8289**|
|Null|QT|0.0477|0.4690|0.8191|
>Q1: L120, 123: the quote for 'skipping' is wrongly formatted 

**Response**: Thank you for pointing out the formatting issue. We will correct the quotation marks for 'skipping' in lines 120 and 123 to ensure proper formatting.

>Q2: L202-203 it should be either "The maximum Minimum Bayes Risk (MBR)..." or "The Minimum Bayes Risk (MBR)....", also it cites paper [20] (the Rouge package) which I think is incorrect for MBR.

**Response**: Thank you for your reminder. We will correct this in the final version by citing the Diffusion-LM and DiffuSeq papers in this reference.

>Q2: Besides the increased training time compared to DiffuSeq (L217-218), it would be beneficial to know the increased inference time too.

**Response**: Thank you for this suggestion. We will supplement the increased inference time compared to DiffuSeq in the following Table.

**Meta-DiffuB's computational complexity compared to DiffuSeq**
|Method|increased parameters (%)|increased training time (%)|increased inference time (%)|
|-|-|-|-|
|Meta-Diffu$B$|2.2%|5%|0.5%|

>Q3: How well does the Meta-Diffu$B$ model generalize to other types of text generation tasks beyond the Seq2Seq framework? Have experiments been conducted to evaluate its performance in different text generation scenarios?

**Response**: We conducted extensive experiments to evaluate the generalization capabilities of the Meta-Diffu$B$ model across various text generation task. These tasks include generating informative dialogue responses (CC dataset), question generation (QT dataset), text simplification (WA dataset), and paraphrase generation (QQP dataset), as discussed in Section 4.1 and Appendix A. Thanks to the reviewers' suggestion, we supplemented additional machine translation datasets, as shown in the following table, and employed the same evaluation metrics used by SeqDiffuSeq and Dinoiser. Results where Meta-Diffu$B$ combined with different models show improved performance are indicated in bold. Our results show that Meta-Diffu$B$, when combined with different S2S-Diffusion models, achieves superior performance on machine translation tasks as well.

**Experiment of Meta-Diffu$B$ on Machine Translation datasets** 
|Methods|SacreBLEU ↑ (IWSLT14 DE-EN)|SacreBLEU ↑ (WMT14 DE-EN)|
|-|-|-|
|DiffuSeq|29.43|22.72|
|Meta-Diffu$B$ (exploiter=DiffuSeq)|**31.71**| **26.17**|
|SeqDiffuSeq|30.16|23.28|
|Meta-Diffu$B$ (exploiter=SeqDiffuSeq)|**32.41**| **26.14**|
|Dinoiser|31.61|30.30|
|Meta-Diffu$B$ (exploiter=Dinoiser)|**33.82**| **32.09**|

>Q4: How does the model scale with larger datasets or more complex text generation tasks? Are there any specific optimizations or modifications required to handle real-world data volumes?

**Response**: Meta-Diffu$B$ scales effectively with larger datasets, as demonstrated by our experiments on the Commonsense Conversation (CC) dataset with 3 million data points. We maintained consistent model parameters, architecture, and optimization methods across all datasets, including CC. Our framework's exceptional performance, even on large datasets, highlights its robustness and scalability. Additionally, as shown in the table from Q3, our Meta-Diffu$B$ performs well on machine translation tasks with the IWSLT14 dataset (160k data points) and the WMT14 dataset (4475k data points), without requiring any adjustments.
>Q1: For example, Figure 3, the noise level is quite stable for the proposed method but in the text below, it claims 'Meta-Diffu$B$ applies adaptive noise at varying training epchos.

**Response**: 
In Figure 3, our method differs from approaches like SeqDiffuSeq or Dinoiser, which rely on predefined rules. Instead, our approach is data-driven, with the model learning the underlying patterns. As a result, the degree of variation—whether large or small—is not the primary focus. Additionally, the learned noise can be flexibly integrated into other systems in a plug-and-play manner.

>Q2: Another example is in Figure 4 at least for QQP dataset - Meta-Diffu$B$ applies less noise than the other methods as its curves lay below the curves of the other methods. thus, the claim 'This noise-scheduling approach—applying more noise to the harder sentences (H) and less to the easier sentences' looks conflict with such observation from the figure.

**Response**: We appreciate the opportunity to correct our previous statement. We will correct this error in the final version.

>Q3: Besides, as algorithm 1 alternates the training of scheduler and exploiter, It would be guideful if some insights about the convergence rate are provided.

**Response**: We supplement the [Training Curve Image](https://github.com/metabeta-diffusion/metabeta-diffusion/blob/main/img/training%20curve.jpg?raw=true). Our Meta-Diffu$B$ shows Our model converges significantly faster than DiffuSeq, achieving convergence within just 10k epochs.
We sincerely thank all the reviewers for their valuable suggestions. We have addressed all the comments as below. 

Our contributions include:
1. **General Applicability:** Meta-Diffu$B$ can be integrated with various S2S-Diffusion models, enhancing their performance through our noise scheduling method. Section 4.5 demonstrates integration with DiffuSeq, while Appendix F shows experiments with SeqDiffuSeq and Dinoiser, confirming our method's broad applicability and significant performance improvement.
2. **Efficiency:** We used the simple LSTM-Seq2Seq model as our scheduler for all tasks. As shown in Table 1 (Section 4.5), even with this small parameter scheduler combined with DiffuSeq, we achieved the best performance across four datasets.
3. **Task Performance:** Our model consistently outperforms previous models in tasks such as Commonsense Conversation (CC), Quasar-T (QT), Wiki-Auto (WA), and Quora Question Pairs (QQP) with our innovative noise scheduling approach.
4. **Versatility:** 1. Training from Scratch: As shown in Table 1 (Section 4.5) and Table 4 (Appendix F), Meta-Diffu$B$ achieves superior results when training our scheduler with DiffuSeq and other S2S-Diffusion models from scratch. 2. Plug-and-Play: In Tables 3 and 8 (Sections 4.8 and Appendix I), we demonstrate that our scheduler, trained on different datasets, can be seamlessly integrated with various S2S-Diffusion models without any fine-tuning.

In this rebuttal, We supplement our work with training curve, machine translation experiments, plug-and-play tests with other S2S-Diffusion models, and the inclusion of more recent baseline models. These experiments will be incorporated in the latest version.

**Training curve**: [Training Curve Image](https://github.com/metabeta-diffusion/metabeta-diffusion/blob/main/img/training%20curve.jpg?raw=true)

**Meta-Diffu$B$'s computational complexity compared to DiffuSeq**
|Method|increased parameters (%)|increased training time (%)|increased inference time (%)|
|-|-|-|-|
|Meta-Diffu$B$|2.2%|5%|0.5%|

**Experiment of Meta-Diffu$B$ on Machine Translation dataset** 
|Methods|SacreBLEU ↑ (IWSLT14 DE-EN)|SacreBLEU ↑ (WMT14 DE-EN)|
|-|-|-|
|DiffuSeq|29.43|22.72|
|Meta-Diffu$B$ (exploiter=DiffuSeq)|**31.71**| **26.17**|
|SeqDiffuSeq|30.16|23.28|
|Meta-Diffu$B$ (exploiter=SeqDiffuSeq)|**32.41**| **26.14**|
|Dinoiser|31.61|30.30|
|Meta-Diffu$B$ (exploiter=Dinoiser)|**33.82**|**32.09**|

**We compared more recent baselines with our Meta-Diffu$B$ on the QG and QQP datasets. These baselines include both discrete and continuous S2S-Diffusion models.**
|Methods|BLEU ↑ (QQP)|BERTScore ↑ (QQP)|BLEU↑ (QG)| BERTScore ↑ (QG)|
|-|-|-|-|-|
|DiffuSeq|0.2413|0.8365|0.1731|0.6123|
|Diffusion (Easy-First Schedule)|0.2503|0.8692|0.1812|0.6253|
|Meta-Diffu$B$ (exploiter=DiffuSeq)|**0.2552**| **0.8821**|**0.1826**|**0.6357**|
|DiffuSeq-v2|0.2411|0.8393|-|-|
|Meta-Diffu$B$ (exploiter=DiffuSeq-v2)|**0.2556**| **0.8829**|-|-|
|BG-DiffuSeq|0.2619|0.8427|0.1744|0.6280|
|Meta-Diffu$B$ (exploiter=BG-DiffuSeq)|**0.2790**| **0.8757**|**0.1838**|**0.6571**|
|TESS|0.3020|0.8570|0.1950|0.6580|
|Meta-Diffu$B$ (exploiter=TESS)|**0.3142**|**0.8975**| **0.2055**|**0.6761**|
|RDM (Discrete Diffusion)|0.2510|0.8472|0.1802|0.6310|
|Meta-Diffu$B$ (exploiter=RDM)|**0.2684**|**0.8724**|**0.2271**|**0.6542**|


**Plug-and-play experiments on SeqDiffuSeq integrated with our scheduler**
|Scheduler|SeqDiffuSeq|BLEU ↑|BERTScore ↑|Dist-1 ↑|
|-|-|-|-|-|
|WA|QQP|**0.2627**|**0.8481**|**0.9814**|
|Null|QQP|0.2434|0.8400|0.9807|
|WA|QT|**0.1834**|**0.6226**|**0.9369**|
|Null|QT|0.1746|0.6174|0.9248|

**Plug-and-play experiments on Dinoiser integrated with our scheduler**
|Scheduler|Dinoiser|BLEU ↑|BERTScore ↑|Dist-1 ↑|
|-|-|-|-|-|
|WA|QQP|**0.2079**|**0.8121**|**0.9765**|
|Null|QQP|0.1949|0.8036|0.9723|
|WA|QT|**0.0495**|**0.4740**|**0.8289**|
|Null|QT|0.0477|0.4690|0.8191|