Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: *The fine-grained class clustering is more challenging than coarse-grained due to lower sample representation and large scale- and color- variations between the fine-grained classes. The main goal of the proposed approach is to learn stronger representations in an unsupervised fashion. To this end, the paper proposes C3-GAN which uses the ability of InfoGAN with contrastive learning to learn feature representations that maximize the mutual information between the latent code and its corresponding observation. The proposed approach is able to achieve best performance in comparison to the related works.* 

*The results demonstrated quantitatively and qualitatively on 4 different datasets along with ablation study validate the proposed approach. The method is promising since it is unsupervised way of learning cluster centroid for unlabeled data.*
main_review: *Strengths*
+ Information theory based regularization with contrastive loss learns the features of the cluster centroids. The proposed method is well adapted for this purpose using InfoGAN and PerturbGAN. 
+ The proposed method effectively avoids mode collapse while training generators, and are able to generate, with good control, various objects with varying background (while keeping foreground fixed), and varying foreground objects (while keeping background fixed) better than related methods.
+ The paper demonstrates that overclustering is a viable approach to hyper-parameter optimization that can achieve better fine-grained clustering due to learning dense features. 

*Weaknesses*
+ The approach is heavily inspired by FineGAN and PerturbGAN. 
+ In overclustering, it is not clear what would happen if the chosen cluster size is such that it is not a multiple of no. of classes? We assume that the data does not have labels in unsupervised feature learning, therefore it would be interesting to see if the cluster size is not a multiple.

*Grammatical/Typographical errors*
+ Page 1, last line: `...we formulate the auxiliary probability...`
+ Fig. 2 caption: `C-3GAN`.
+ Page 5, first line: `...In mathematical, ...`
summary_of_the_review: *The paper presents a method that is a hybrid of 2 major previous state-of-art methods. Although heavily inspired, the paper proposes solutions to reduce the drawbacks of the previous approaches - which is the highlight of the contribution. Additionally, the biggest take-away is that the method is unsupervised. Given the breadth of the experiments to validate each of the proposed solutions, and substantial ablation experiments to justify each proposal, the paper overall is a good contribution.*
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper studies the problem of fine-grained image clustering. Similar to recent work such as OneGAN and FineGAN, the paper proposes to use a GAN setup, called C-3GAN, where the fine-grained image synthesis and clustering are performed in the same end-to-end pipeline. The main contribution is to use a contrastive loss in maximizing the mutual information of the discriminator encoded features and class-centric features. There are also some modifications in the foreground and background synthesis mechanisms compared to prior work such as the FineGAN. Experiments are conducted on fine-grained datasets such as CUB-200-2011.
main_review: [Strengths]
1. The method is straightforward and the paper is easy to follow. 
2. The performance boost compared to SOTA for clustering seems to be very good (Table 1). 
3. The ablation study covers major components in the pipeline so it is easy to understand their contributions.

[Weaknesses]
While the paper is focusing on fine-grained image synthesis/clustering, it is not specifically designed for fine-grained classes: the whole pipeline is pretty generic. This is unlike for example the FineGAN paper where they design separate stages for parent and child classes. Other than showing in the experiment that the proposed method just *works* in the fine-grained setting, I'd be interested in seeing more analysis on the advantage of the proposed pipeline in fine-grained setting vs. coarse-grained setting. For example, the paper could show the performance of parent and child classes in those datasets and compare against SOTA. If the proposed method has a similar performance as others in the coarse setting but a clear advantage in the fine-grained setting, that will be a very strong signal.

In addition, there are some parts in the paper that are unclear or confusing.
1. Section 3.2, second paragraph. "The discriminator D aims to learn adversarial features r' --> what does 'adversarial features' mean here? I checked the architecture it seems to be just the discriminator score (a scalar). 
2. What is the effect of under-clustering where Y is much smaller? 
3. The loss defined in Eq. 5 does not use the notion Q or q_k defined in Eq. 4. It is not clear from the text directly what their connection is. It should be written clearly L = -log q_k for example.
4. Table 1 last sentence: I think OneGAN is an unsupervised method.

[Other suggestions for editing]
1. The first sentence in the Abstract has redundant words: 'Unsupervisedly' and 'without ground truth annotation' are the same thing.
2. Figure 4 (a): the same cluster c is very confusing here because those birds are from different classes. I'd suggest simply dropping the symbol c.

summary_of_the_review: Overall, this paper proposes a straightforward pipeline to synthesize and cluster images in fine-grained classes. A contrastive loss is used in place of the regular softmax loss in the InfoGAN framework. The performance seems to be very solid compared to SOTA methods on both synthesis and clustering tasks. There are some unclear or confusing parts in the paper, but I think given the simplicity and good performance of the method, it might be worth being seen by the community to inspire similar works.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The authors undertake the more difficult task of data clustering, based upon fine-grained features. They use contrastive learning for this, in conjunction with GAN losses. Their representation can be used in the downstream task of image generation, where they use their representations' strengths to show improved resilience to mode collapse, while displaying better intra-cluster variance.  
main_review: Strengths

	The experiments are well executed, and clearly communicated.
	The quantitative results for clustering, and the qualitative figures in the supplementary material are both impressive, and they support the authors' claims
	The method has novel components which are provably attributed to these results.

Weaknesses

	The paper is well rounded, with all aspects detailed to completion. There are no obvious weaknesses
summary_of_the_review: The paper combines several techniques to achieve unsupervised fine-grained clustering of semantic classes. Their representations' high quality is able to drive GAN generation without mode collapse, thereby achieving great two-fold contributions.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: -- The authors propose C3-GAN, a method that learns "clustering-friendly" feature representation for fine-grained clustering (main goal), by learning features of cluster centroids (latent codes) using contrastive loss on the mutual information of image-latent code pairs.

-- The method should improve the GAN's performance in terms of image diversity.

-- The method is unsupervised and applicable for single-object images only.

-- The method is built upon FineGAN (and InfoGAN idea), after adding significant improvements such as removing the dependency on bounding-box labels, applying the mutual information in the embedding space, and directly learning cluster centroids.

main_review: Strength:
	- The problem is known to be valid and challenging
	- The method idea, formulations, and figures are clear in general
	- The level of novelty is reasonable
	- The model is fully unsupervised
The authors provide an extensive evaluation and the results are impressive

Weaknesses:
	- The part of the contrastive loss is not totally clear. The authors should provide a better intuition of why the contrastive loss improves the feature representation. For example, how are image-latent pairs defined as positive?
	- The method focuses on learning cluster granularity for the object only, and not for the background.
	- It's unclear why the transformation matrix is used (other than the fact that it's part of PerturbGAN's pipeline)
	
A few comments on the text:
	- The phrase "coarse-grained images" is inaccurate, the "coarse-grained" adjective should refer to the clustering and not the images (in the intro).
	- The authors should share more details about the auxiliary distribution mentioned in the abstract and the intro.
	- Overall proofreading is required.
It would be great to add some of the model's notations to figure 2 (e.g. D_base, psi_r, psi_h)
summary_of_the_review: I tend to vote for accepting this paper as I think it proposes a great approach and presents a convincing comparative performance.
review_title: null

Meta-review: All the reviewers liked the paper. The proposed method contains novel ideas of learning feature representation to maixmize the mutral informatio nbetween the latent code and its corresponding observation for fine-grained class clustering. The model seems to successfully avoid mode collapse while training generators and able to generate various object (foregrounds) with varying backgrounds. The foreground and background control ability is an outstanding feature of the paper. Please incorporate the comments of the reviewers in the final version.

BTW, the real score of this paper should be 7.0 as Reviewer 5wFE commented that he/she would raise the score from 5 to 6 but at the time of this meta review, ths core was not raised. So the final score of the paper should be 8/8/6/6.
Decision: Accept (Spotlight)