Paper: On the Stability of Iterative Retraining of Generative Models on their own Data (ICLR-2024)

Reviewer comment (Evaluation):
What is meant by precision and recall in the evaluation section? Is there a discriminator deployed that tries to differentiate between the real and fake samples? This needs to be clarified. I don’t currently see how generative models can have recall/precision.

Author response:
**Description of Precision and Recall** 

We answer this question in detail in our global response to all reviewers. In summary, the precision/recall we use here differs from those used for evaluating a classification model. Intuitively, at the distribution level, precision and recall measure how the training and the generated distributions overlap. Precision can be seen as a measure of what proportion of the generated samples are contained in the “support” of the training distribution. On the other hand, recall measures what proportion of the training samples are contained in the “support” of the generated distribution.  If the training distribution and the generated distribution are the same, then precision and recall should be perfect. If there is no overlap—i.e. disjoint between the training distribution and the generated distribution, then one should have zero precision and zero recall More details and references are provided in the joint answer to reviewers and in Appendix G.2. Feel free to ask any clarification questions if needed.

Reviewer comment (Theory):
Some proofs in the paper offer room for improvement of clarity and were inaccessible for verification in their current form. For instance, in appendix A.1., I can follow the proof of Lemma 1 and Lemma 2. However, it is not obvious how equation (13) implies the form of $\alpha(n)$ using the $\Gamma$-function. This should necessarily be clarified as there is not a word on how the form of $\alpha(n)$ comes into play.

Author response:
**Improving the clarity of the proofs.**
We agree with the reviewer that some of our proofs could benefit from additional clarity to ease the verification of their claims. To improve readability the proofs in contention as highlighted in the review have been encapsulated into simpler lemmas, with additional intermediate steps, which we believe are much easier to follow. The key changes with respect to the initial submission have been highlighted with blue text in the updated PDF.
More precisely, **Lemma A.1 and Lemma A.2 in Appendix A.1 and A.2 should now provide detailed guidance on the proofs** in the Gaussian case. We thank the reviewer again for helping us strengthen this aspect of our submission and are we more than happy to add anything else to these proofs should the reviewer deem it necessary. 
[1] Boyd, S.P. and Vandenberghe, L., 2004. Convex optimization. Cambridge University Press.

[2] M. S.-M. Sajjadi, O. Bachem, M. Lucic, O. Bousquet, and S. Gelly. Assessing generative models via precision and recall. Neurips 2018

[3] Kynkäänniemi, Tuomas, et al. "Improved precision and recall metric for assessing generative models." Advances in Neural Information Processing Systems 32 (2019).

Reviewer comment (Theory):
Some assumptions made in this work may not reflect reality well and may be oversimplifying. In particular, we basically assume convexity of the loss function through assumption 1 and 2. As far as I can see, due to the Lipschitz constant on the Hessian being L (assumption 1) and the Hessian's eigenvalues being smaller than $-\alpha$ (interpreting the operator that way, please correct me if this is not correct), in the ball of $\epsilon = \alpha/L$ around $\theta^*$, the Hessian will be negative definite, implying convexity. As the theorems only state the existence of a radius $\delta > 0$ in which the convergence properties hold, we basically consider the convex part of the problem. Modern loss landscapes are far from convex, making it highly unlikely that both the initial value and the optimum $\theta^*$ lie in a ball in which the loss function is convex.

Author response:
**Assumptions 2 and 3.** Thank you for your suggestions. More details on the intuitions and the implications of Assumptions 1 and 2 have been added in the manuscript, especially the discussion on local strong convexity and local Lipschitzness. 

- **Initialization in a Convex Region.** 
We believe that initializing in a convex region is possible because of:
1- For modern architectures (typically, ResNet with skip connections), local optima with good generalization properties are believed to be locally convex, see for instance Figure 1-6 of *Visualizing the loss landscape of neural nets* [1], that shows that modern architectures are believed to have flat local minima.
 2- Our proofs and experiments significantly rely on **finetuning**. In other words, we always used **pre-trained** models. For $\lambda=0$, it is realistic to assume that the parameters of the finetuned model $\theta_0$ are already in the local convex region. Combined with the Lipschitness of the neural networks [2], it seems reasonable to assume that for good enough models and a small enough proportion of synthetic data $\lambda$, the pre-trained model's initialization parameters are in a locally convex region.

We hope this convinces you that the initialization is possible in a locally convex region.

Reviewer comment (Theory):
Another impractical assumption may be the assumption of no approximation error. This is usually only shown for infinite-parameter models. I would be fine with these approximations if the empirical results confirmed the assumptions and the analysis that follows from them. However, the evaluation results seem to rather confirm the doubts.

Author response:
**Over Simplifying Assumptions**

Assumptions 1 and 2 (local Lipschitzness + strong convexity)  “We basically assume convexity of the loss function through assumptions 1 and 2”. We would like to highlight an important point: assumptions 1 and 2 imply local Lipschiptness and local strong convexity, which is much weaker than global Lipschitness and strong convexity, which would be indeed much stronger and unrealistic. To the best of our knowledge, local Lipschitness and strong convexity are among the mildest assumptions required for understanding the local landscape of neural networks in terms of optimization [1,3] or generalization [2]. We would be eager to consider other sets of assumptions if the reviewer can point us toward such works. 
The question on Assumptions 3 (generalization error) is answered as a joint response to reviewers


[1] Nesterov, Y., & Polyak, B. T. Cubic regularization of Newton method and its global performance. Mathematical Programming, 2006.

[2] Keskar, N.et al. On large-batch training for deep learning: Generalization gap and sharp minima. ICLR, 2017

[3] Jin, Chi, et al. "How to escape saddle points efficiently." ICML. PMLR, 2017.

Reviewer comment (Evaluation):
The evaluation does not confirm the claims. The key claim is that there is some value $\lambda > 0$ such that a certain amount of generated data can be injected and model training remains stable. Unfortunately, the experimental results are not convincing. The leftmost column of Figure 3 shows that the resulting FID curve is almost a linear interpolation between $\lambda=0$ (stable) and $\lambda=1$ (unstable), suggesting that for every $\lambda>0$ training will diverge in the end. Even for the smaller lambdas, the training FID gradually increases. For $\lambda=0.001$, more than 5 steps of training would likely be needed to observe a statistically significant effect, especially in the infinite retraining regime. Furthermore, there are no measures of disparity (e.g., standard deviations) displayed, which could solidify the empirical evaluation.

Author response:
**Experimental Evaluation**

We acknowledge the reviewer's concern about the empirical evaluation in Fig. 3 concerning our presented theory. We would like to politely push back against the reviewer's assertion that in the infinite regime, one could conceive of a scenario that is equivalent to adding two sequences—one divergent and one non-divergent—which may lead to instability. We politely disagree. Such a scenario is not realizable in our setup because every iterative retraining iteration uses the **same fixed $\lambda$**. Even in the infinite regime, this cannot be a superposition of two sequences of iterates with **different $\lambda$** values. Moreover, we have a complete understanding of what each iteration consists of—we simply maximize the likelihood of the (mixed) dataset—and the maximum likelihood framework is well understood for the generative models we consider in this paper (e.g., flows and diffusion). Thus, from a theoretical point of view, maximizing the likelihood corresponds to only $1$ sequence of iterates.

We believe our current experimental findings in Fig 3 clearly show stability for many $\lambda > 0 $ values. In some instances, we even see **subtle improvements** in precision and recall see for instance the EDM graphs for small values of $\lambda$, or experiments on synthetic data (Figure 6 in Appendix G.4). Furthermore, we would like to add the paper [3] mentioned by the reviewer considered only $10$ iterations. In contrast, our paper has double the amount with $20$ iterations, and we argue the trend (of stability in our case) is equally clear. We see the slope of the lines in Figure 3 very specifically. For the stable regimes to be flat and consistently flat. Could the reviewer please clarify which parts of the plots in Fig 3 they feel do not support well the theory in this paper?

Reviewer comment (Writing):
There is a missing parenthesis in the last line before the statement of the contributions: “(DDPM…”.

Author response:
**Minor points**

We are grateful to the reviewer for pointing out the small typos and mistakes in our manuscript. We have fixed this in our updated rebuttal PDF. The PDF size of the figures has been reduced, and the PDF should render smoothly now.

Reviewer comment (Writing):
In the last point of the contribution section, “Using two powerful diffusion models in DDPM and EDM”, the use of “in” seems unexpected at that place.

Author response:
**Minor points**

We are grateful to the reviewer for pointing out the small typos and mistakes in our manuscript. We have fixed this in our updated rebuttal PDF. The PDF size of the figures has been reduced, and the PDF should render smoothly now.

Reviewer comment (Writing):
There is a typo in the proofs: “Since most methods are local search method[s]” should use the plural form.

Author response:
**Minor points**

We are grateful to the reviewer for pointing out the small typos and mistakes in our manuscript. We have fixed this in our updated rebuttal PDF. The PDF size of the figures has been reduced, and the PDF should render smoothly now.

Reviewer comment (Presentation):
In the proof in Appendix C, there are some formatting errors below equation 33 (theta is not properly displayed).

Author response:
**Minor points**

We are grateful to the reviewer for pointing out the small typos and mistakes in our manuscript. We have fixed this in our updated rebuttal PDF. The PDF size of the figures has been reduced, and the PDF should render smoothly now.

Reviewer comment (Presentation):
The PDF seems to render very slowly. The authors could check some of the vector graphics again to increase overall accessibility.

Author response:
**Minor points**

We are grateful to the reviewer for pointing out the small typos and mistakes in our manuscript. We have fixed this in our updated rebuttal PDF. The PDF size of the figures has been reduced, and the PDF should render smoothly now.

Reviewer comment (Evaluation):
The empirical evaluation of two simpler models is limited to 2 visualisations without quantitative measurements.

Author response:
**Empirical evaluation of simpler models**
Quantitative metrics have been added for the synthetic experiments (see Figure 6 in Appendix G.4).  The Wasserstein-$1$ and $2$ distances between the true data distribution and the generated one is displayed as a function of the number of retraining. The distances are averaged over $50$ runs, the line corresponds to the mean, and the shaded area to the standard deviation. When the model is fully retrained only on its own data (Fully Synthetic, dashed red line with triangles), the distance to the true data distribution diverges. When the model is retrained on a mixture of real and synthetic data ($\lambda=0$, $\lambda=0.5$, $\lambda=1$), then distance between the generated samples and the true data distribution converges.

Reviewer comment (Evaluation):
For Diffusion models, the evaluation is performed on 3 datasets, which is sufficient. However, if I understand the setup correctly the whole analysis is performed on a single training seed. The differences presented in the plots are extremely small so it is unclear whether they are statistically significant, and therefore whether the main claims hold.

Author response:
**Evaluation over seeds**
We acknowledge the reviewer's comment regarding the number of seeds used in our iterative retraining experiments for natural image datasets. We first highlight that we have improved this experiment in the updated version of the manuscript but substantially increased the number of iterative retraining steps (5 vs 20) while adding a fully synthetic baseline ($\lambda = \infty$). In our plots, we see that the margin of difference between the fully synthetic case and the more stable regime of $\lambda > 0$ of iterative retraining is quite dramatic. While the reviewer has requested more random seeds, we wish to politely push back due to the high computational cost of running each experiment. For example, just for one curve of FFHQ-64, conducting iterative retraining using the SOTA EDM model amounts to a full week of experimentation using 4 RTW8000 GPUs. This yields to $40$ GPUs over one week for Figure 3 itself. Unfortunately, running additional random seeds for all models and datasets is beyond the computational budget within the period of this rebuttal, and even beyond our academic resources. However, to promote reproducible research we commit to releasing the full training code of all our experiments such that one can readily verify the stability and collapse property of iterative retraining.

Reviewer comment (Theory):
How likely is assumption 3? Can the authors provide example where such a bound stands?

Author response:
Assumption 3 implies sample complexity generalization bounds on the parameter for generative models. From a theoretical perspective, numerous works have proved sample complexity bound on the distribution, more precisely universal *density approximation* and *function approximation* capabilities of model classes like diffusion models, affine coupling and residual normalizing flows.

Reviewer comment (Evaluation):
How is precision and recall computed in the experimental section? What classifier is used here?

Author response:
Precision and Recall  Intuitively, at the distribution level, precision and recall measure how the training and the generated distributions overlap. Precision can be seen as a measure of what proportion of the generated samples are contained in the “support” of the training distribution. On the other hand, recall measures what proportion of the training samples are contained in the “support” of the generated distribution.  If the training distribution and the generated distribution are the same, then precision and recall should be perfect. If there is no overlap—i.e. disjoint between the training distribution and the generated distribution, then one should have zero precision and zero recall More details and references are provided in the joint answer to reviewers and in Appendix G.2. Feel free to ask any clarification questions if needed.