Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 1: Reject
review_content: The paper states that training generative models is challenging when there are noisy data points in your training set. To address this, the authors propose to training methodology (or curricula)  where "easier" or more "relevant" points are presented first followed to the model followed by the less relevant ones. The relevance is determined used by calculating centrality on a graph constructed out of the data points. 

I lean towards rejecting the paper, primarily because 1) The authors have not provided evidence to claim the noisy data points makes training challenging or characterized anything about under how much noise the training breaks down. 2) Experimental evidence is not convincing  3) There are not of imprecise statements in the paper. 

The authors claim (without citation) that training generative models in the presence of noisy data is challenging. How much noise are we talking about? If the entire training set is very noisy, maybe we don't have any hope to learn to generate clean samples, but if it's just a little bit of noise, maybe it's fine. I also understand that when the authors use the word "noise" they don't really mean noise,  but changes in view point etc. Some convincing demonstration that such characteristics of dataset adversely affects the training of generative models will be helpful (in addition to one passing sentence in the experiment section about it).

I am not convinced by any of the experiments. More importantly, it seems like the proposed training curricula is in general valid for a lot of generative models. The experimental evidence is really not convincing. Very limited experiments is done on two datasets only using one particular GAN. To convincingly demonstrate that your training methodology is doing something non-trivial, you should show that this works on multiple generative models on multiple datasets and compare your performance (and visualize some generated samples) against just training blindly on the entire dataset. In addition to this, I don't understand the relevance of some of the results in the paper. For ex, what does Corollary 3 signify?

Some other points:
1. First para. "non-trial' -> non-trivial
2. First para: "model collapse" -> mode collapse
3. 3.2 para 2. "base set that guarantees a proper converge of generative models". What do you mean by "proper convergence"?
4. 3.2 para 2. "moderate compared to m". Again, what is moderate? 
5. Why do you use ResNet features (5.1) for distance? Why is this a reasonable or the best metric while computing your graph?
6. "We determine the parameter \alpha of the edge weight by enforcing geometric mean of the weights to be 0.8". It seems very arbitrary to me. Can you justify this choice?




review_title: Official Blind Review #3

final_score: rating: 6: Weak Accept
review_content: Summary: This well written paper presents an effective way to remove outliers for deep generative models, provided examples are ranked along their centrality. One question I have is how exactly this centrality is measured

The word ‘curriculum’ has become a terminology used to describe a wide variety of totally different algorithms. While the authors provide an excellent introduction to this diversity and clearly differentiate their own flavor of ‘cluster curriculum’, I am wondering if they would not have been better off by describing the proposed approach in terms of outlier removal, especially has it has very little in common with the original idea of curriculum, which is a learning progression designed by the teacher.

Nevertheless, the paper is very clearly written and reads well in the present form. I am especially impressed about the clarity of the rather technical part on percolation.

In terms of outlier removal, this could be interpreted as the following constructive algorithm:
•	Examples are ranked along their centrality measure 
•	One identify the critical point in percolation for a starting point where too many examples are removed
•	More examples are added until a minimum is found on validation data
This process can be made faster as:
•	The optimum can happen very soon after the critical point
•	Otherwise, an active set algorithm can be used to control the size of the training set.

While my recent expertise has been more NLP than Vision, I think this algorithm is original, and could have a significant impact as it can be extended beyond GANs. The technical presentation is excellent.

One puzzling issue is the computation of the centrality measure: all I could find in Appendix A1. and A.3 is that it is directly measured on the raw image (RGB pixels?) by taking some distance, which I assume is Euclidean? My experience in image classification suggests such a distance is meaningless, so I may be missing something. I looked for a Github pointer, but none was provided.

The English is OK, but there are missing words and strange constructions:
•	Page 1:  “Deep generative models HAVE piqueD researchers’ interest in the past decade”
•	Page 4: “The training of many loops will lead to time-consuming (MISSING WORD)”

In particular, the usage of “the”, for instance in pages:
•	7 “Therefore, a fast learning strategy can be derived from THE percolation process.”
•	7 “Training may begin from the curriculum”
•	8 “Cluster curriculum is proposed for robust training of generative models.”

review_title: Official Blind Review #2

Meta-review: The paper proposes a curriculum learning approach to training generative models like GANs. The reviewers had a number of questions and concerns related to specific details in the paper and experimental results. While the authors were able to address some of these concerns, the reviewers believe that further refinement is necessary before the paper is ready for publication.
Decision: Reject