Paper: Self-Consuming Generative Models Go MAD (ICLR-2024)

Reviewer comment (Theory):
To enhance the rigor of the paper, a more comprehensive theoretical framework should be developed to formalize the concept of MAD and its implications. A deeper exploration of the mathematical foundations underlying MAD would contribute to a stronger theoretical basis for the research.

Author response:
**To enhance the rigor of the paper, a more comprehensive theoretical framework should be developed to formalize the concept of MAD and its implications.** 

We are also very interested in a unifying mathematical theory of MAD. Our limited analytical results in the Gaussian case for the fully synthetic loop relied on specific properties of Gaussian parameter estimation. Developing a general theory would require additional nontrivial innovations that are outside the scope of this paper. We believe that this will be a rich line of research in the future.

Reviewer comment (Experiments):
Could the authors provide more insights into potential strategies or mitigations for addressing MAD in generative model training?

Author response:
**This work would benefit from a dedicated section discussing potential solutions or mitigations for Model Autophagy Disorder (MAD) in the training of generative models.**

We have already incorporated a discussion on this matter within the existing Discussion section, although it is limited in length due to the page restriction. We state that effort should be made toward increasing the amount of real data compared to synthetic data. As a direction for future work, we suggest watermarking techniques for synthetic data, so that the system can detect and remove them if they are unintentionally in the dataset. The question of how to address this issue is an open area of research, which our work calls the attention of the research community to.

Reviewer comment (Experiments):
The paper discusses the impact of sampling bias (λ). Could the authors provide a more nuanced analysis of how varying levels of sampling bias affect MAD, particularly in terms of quality and diversity?

Author response:
**Could the authors provide a more nuanced analysis of how varying levels of sampling bias affect MAD, particularly in terms of quality and diversity?** 

We do provide experiments on both real and Gaussian data that detail the effect of varying $\lambda$ – see Figure 14 for the fully synthetic loop, Figures 8, 28 (new), 29 (new) for the synthetic augmentation loop, and Figures 10, 30, 31, 32, 33 for the fresh data loop. We have ensured that $\lambda$ captures the trend of having value between 0 and 1 with smaller values corresponding to more sampling bias, but due to the fact that we consider many different generative models, datasets, and biasing mechanisms, we cannot ensure that $\lambda$ is calibrated in such a way that we can give any meaning to specific values. However, the general conclusion is consistent among all models and methods, where the increasing sampling bias enhances the quality of the data, at the expense of losing diversity. We recommend the existing literature for further nuance on specific sampling bias methods in different generative models.

Reviewer comment (Theory):
Although the authors define the autophagous mechanisms such that "each generative model $\mathcal{G}^t$ is trained on data that includes samples from previous models" (p3), but to my understand, the theory (around eq 1) and the experiments (described in A.1 and A.2 diffusion model) only consider the case that each model $\mathcal{G}^t$ is trained on data generated by $\mathcal{G}^{t-1}$. The discrepancy between the definition of autophagous mechanisms and the theory and experiments is worth noting otherwise it diminishes the soundness.

Author response:
**To my understand, the theory (around eq 1) and the experiments (described in A.1 and A.2 diffusion model) only consider the case that each model $\mathcal{G}^t$ is trained on data generated by $\mathcal{G}^{t-1}$** 

It is correct that the theoretical results consider training only on data generated by the previous model $\mathcal{G}^{t-1}$, but this is **not** true for all of our experiments. For example, in the synthetic augmentation loop for StyleGAN (Figure 7 in Appendix A.2) and the newly added MNIST DDPM experiment (Figures 28 and 29 in Appendix H), we accumulated synthetic data from all previous generations. As another example, in Figure 33 in Appendix I, we experiment with fresh data loops for Gaussian data using synthetic samples from all previous generations.

Reviewer comment (Experiments):
Why the authors choose StyleGAN2 with FFHQ and DDPM with MNIST?

Author response:
**Why the authors choose StyleGAN2 with FFHQ and DDPM with MNIST?**

We selected these models because they achieve near-state-of-the-art performance at a reasonable computational expense. These models represent two widely used approaches to generating image data: diffusion models and GANs. Due to the substantial computational requirements for training and sampling from diffusion models, we opted for a less complex dataset (MNIST) for DDPM. In contrast, we used FFHQ for StyleGAN, as it is less computationally demanding.

Reviewer comment (Theory):
It is unclear what Claim in Section 2 aims for. Although it is proved for a toy model of gaussians, I assume that it is aimed for a hypothesis for general case but the authors seem not to explain such aspects. For example, is random walkness in Claim observed for practical models? If not, the claim would be overclaimed and somewhat doubtful for general case. Finally note that "Claim" in a paper is usually considerred as to be proven in the paper, not a hypothesis.

Author response:
**Is random walkness in Claim observed for practical models?** 

Thank you for pointing out this ambiguity in the “random walk” aspect of the Claim from Section 2. To clarify the claim and remove any possible connotation with a “pure” random walk (e.g., Brownian motion), we have removed “that follows a random walk” from the Claim.

Reviewer comment (Writing):
Eq. (2) is totally unclear. Is it a hypothesis or theorem? Where is $WD(n_r, n_s, \lambda)$ defined? If it is just a hypothesis, the author should make it clear in the text.

Author response:
**Eq. (2) is totally unclear… Where is $WD(n_r, n_s, \lambda)$ defined?**

$WD(n_r, n_s, \lambda)$ represents the asymptotic Wasserstein distance that models converge to in the fresh data loop, which we have observed in our experiments depends solely on $n_r, n_s,$ and $\lambda$ (the amount of new real data, new synthetic data, and sampling bias at each iteration). We have modified Eq. (2) slightly to denote that we define $WD$ as the limiting value to increase clarity.

Reviewer comment (Writing):
It is unclear why $n_{ini}$ is considered in the first part of Section 5. How does it relate to the main claims in Introduction?

Author response:
**How does [$n_{ini}$] relate to the main claims in Introduction?**,  **The observation of a phase transition is interesting…I recommend to briefly discuss the motivation and the results in Introduction.**

Based on this useful feedback, we have revised the fresh data loop takeaway in the Introduction to better reflect the observations in Section 5 regarding the independence of the limiting behavior on $n_{ini}$ and regarding the phase transition in improvement due to the inclusions of synthetic data.

Reviewer comment (Presentation):
The observation of a phase transition is interesting, but it suddenly appears at Section 5. I recommend to briefly discuss the motivation and the results in Introduction.

Author response:
**How does [$n_{ini}$] relate to the main claims in Introduction?**,  **The observation of a phase transition is interesting…I recommend to briefly discuss the motivation and the results in Introduction.**

Based on this useful feedback, we have revised the fresh data loop takeaway in the Introduction to better reflect the observations in Section 5 regarding the independence of the limiting behavior on $n_{ini}$ and regarding the phase transition in improvement due to the inclusions of synthetic data.