Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The authors propose a diffusion-based approach to stabilise the learning process of GANs in a instance noise fashion.
strength_and_weaknesses: Strength:
- The abstract and introduction give a nice overview and motivation for the problem.
- The method is supported by a theoretical analysis.
- The results are good and evaluation is reasonable.
- The objective is still differentiable wrt the generator using the classical rewritting of the perturbed samples.

Weaknesses:
- 25-gaussians example (Fig. 5): the purpose of the figure is to exhibit the fact that the propose method is not prone to mode collapse but I am curious to know if the proposed method has shown some mesurable improvement wrt SotA methods on this "toy example"? 
- The authors did an ablation study on the adaptive diffusion process. But, reading Section 3.3, it feels like the adaptative diffusion strategy is key to the method whereas the ablation study does not clearly support it.
clarity,_quality,_novelty_and_reproducibility: Overall the writting is good, the proposed method looks sound and in line with previously proposed methods.
summary_of_the_review: Overall I think this is a good paper tackling a well motivated task.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper introduces the idea of diffusion-GAN by incorporating diffusion process with GANs. In the framework, an adaptive diffusion process for sampling the noise and a diffusion timestep-dependent discriminator are added to a traditional GAN generator. The main contributions are:
1. Propose a diffusion-GAN framework which combines the technique of diffussion models and GAN;
2. Theoretically and empirically show the effectiveness of the method and the ability of stabling GAN training;
3. According to the results reported, the quality of the generative images are better than the state-of-the-art GANs.
strength_and_weaknesses: The main strength:
1. Proposed a new architecture which combines two state-of-the-art generative models and theorectially showed the ability of acheiving a better GAN training and also sampling;
2. The experiments are well-designed and the results are significant and interesting;
3. The paper is well-written and easy to follow

Concerns/questions:
1. While the FID and Recall are reported, is it possible to also include other metrics(IS, LPIPS, ..);

clarity,_quality,_novelty_and_reproducibility: The paper is novel by combining two model training process and is of good clarity. It should also be easy to reproduce.
summary_of_the_review: This paper studies a combination of two state-of-the-art generative models and theoretically shows its ability to stable the GAN training. According to the results, the images are of better quality than current GANs. I recommend accepting it.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper proposes an improved GAN training method that involves a diffusion chain that generates Gaussian mixture distributed instance noise. The diffusion level is dependent on timestep, and the discrimination task is made increasingly difficult for the discriminator over the course of training. The approach improves generation quality in terms of FID measurements and recall capacity, while allegedly retaining stability.
strength_and_weaknesses: Strengths:
- The proposed approach makes intuitive sense and it is explained with sufficient clarity.
- Generation performance is superior to relevant baselines.
- Computational costs are on a par with relevant baselines (StyleGAN-ADA).
- Some theoretical support is provided to ensure the injection of extra noise does not hamper the basic GAN learning properties (though it is difficult to assess the ultimate significance of the provided theorems)

Weaknesses:
- Claims of stable training were made, but I did not easily find where this was justified. Perhaps the authors could point me to where it is shown that the training is especially stable, in comparison to baselines?
- It was not clear which hyper-parameters need to be addressed while switching the datasets. Again, a simple summary clarification from the authors might help here.
clarity,_quality,_novelty_and_reproducibility: - The paper is in general easy to follow.
- As far as I am aware, the approach is original; however, given the ubiquity of diffusion-based approaches in recent years, I might be missing some recent similar efforts in this area.
- Source code is provided.
summary_of_the_review: The paper contributes a non-trivial new approach to GAN training. It could prove a useful addition to the GAN models and provides one possible bridge between diffusion and GAN methods.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes to stabalize GAN training by training Discriminator with adapitvely noised images. The method shows improved performance over GAN baselines across low-resolution and high-resolution image generation benchmarks as well as low and high data regime.
strength_and_weaknesses: Strength
1. The paper is clearly written with theoretical analysis to support proposed method. 
2. Experiments are well designed and demonstrated clear improvments over baseline methods, ranging from low-reoslution to high-resolution and from low-diversity to high-diversity

Weakness
The method improvment is somewhat marginal: introducing noise to discriminator training has been explored before, introducing different levels of noises as defined in diffusion process seems like a minor improvement. 
clarity,_quality,_novelty_and_reproducibility: Clarity: High.

Quality: High. Method and experiments are well written with high quality. 

Novelty: Fair.

Reproducibility: High.
summary_of_the_review: The paper is well structured with theoretical analysis and convincing experiments. Introducing adaptive noise as formulated in diffusion process to GAN Discriminator training is somewhat a natural extension of previous works but could be a go-to solution to stabilize GAN  training in the future.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This paper improves GAN training by leveraging the forward diffusion process in diffusion models. The basic idea is that instead of only comparing real and generated data distribution directly using a discriminator, samples from both distributions are perturbed following the forward diffusion and they are contrasted to each using a time-dependent discriminator. This way the discriminator can provide feedback (more precisely data likelihood ratios) to the generator on all the intermediate smoother real/fake distributions. Theoretical analysis shows that matching real and fake distributions in the intermediate perturbed steps can in fact result in matching the base non-perturbed setting as well. 

On the positive side, the paper is written clearly and motivations are stated nicely. The experimental results support the claims and the theoretical analysis provides insights into why the diffusion process can provide consistent training. Although prior works have considered Gaussian noise injection as an augmentation technique for training GANs, this paper provides new perspectives on this aspect. On minor issue with the proposed technique is that it does not fundamentally change the mode dropping behavior of GANs. 
justification_for_why_not_higher_score: Given the huge success of recent diffusion models in image generation, the significance of the proposed improved GAN training is questionable. Additionally, noise injection has been used in the past for GAN training.
justification_for_why_not_lower_score: The paper is strong enough for publication at ICLR.
Decision: Accept: poster