Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper targets at relieving the massive labeled data consumption of deep learning through the framework of semi-supervised learning. In particular, it finds out that two training approaches, Positive-Unlabeled classification and the conditional generation, can benefit each other. Jointly conducting these two approaches can push better performance on both tasks, thus eventually achieving better performance with a limited amount of labeled data. The authors combined the two tasks with a new type of GAN network. They further gave the corresponding theoretical proof for this new GAN model and verified its performance on the benchmark datasets.
main_review: The pros and cons of the paper are described below.

## Pros ##
Bridging different generalization methods is a common technique to improve the performance of deep learning, as verified by much previous work. However, it seems to be the fact that utilizing PU and conditional generation is the unique contribution of this work. Such a new perspective does provide new insights to the community and can motivate future research in the field of semi-supervised learning. Meanwhile, combing PU and conditional generation is not an easy task. To this end, the authors designed the model of CNI-CGAN, which introduces several novel designs. The theoretical proof also seems reasonable.

## Cons ##
For the empirical evaluation part, the authors only verified their method on several relatively small datasets like MNIST and CIFAR. However, the main claim of this paper is that they can relive the massive labeled data consumption of training deep networks. Thus, it is important to verify the method on some relatively large datasets like ImageNet. The reviewer understands that there may not be a proper data source of unlabeled data or an out-of-distribution data source for ImageNet. However, this fact also suggests that the semi-supervised learning method this paper puts forward may not be easily implemented in practice.

summary_of_the_review: Considering the advantages and disadvantages of this paper, the reviewer recommends weak accept. If the authors can verify their method on ImageNet, the reviewer will raise the rating.

After reading the authors' rebuttal as well as the other reviews, I lower my rating because I don't see the experimental results on ImageNet as my expectation.
review_title: null

final_score: rating: 1: strong reject
review_content: summary_of_the_paper: This paper aims to make full use of agnostic unlabeled data to improve classification and generation performance by leveraging Positive-Unlabeled (PU) classification and the conditional generation with extra unlabeled data. They evaluate on some benchmarks in different positive label ratio. 
main_review: The problem is practical and interesting. The paper generally is easy to follow and read. 

However, my major concern is the novelty of the paper, which is a combination of PU and GAN. It is incremental with limited novelty. 
There are GAN model for semi-supervised learning. How about comparing with them? This paper only compares with two methods.
What is the benefit of confusion matrix C to get noisy label?






summary_of_the_review: Based on the novelty, I tend to reject the paper.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a unified framework to leverage Positive Unlabeled classification and conditional generation with extra unlabled data and benefits the two tasks simultaneously.
It also proved the optimal condition of the proposed Classifier-Noise-Invariant Conditional GAN theoretically.

main_review: ## Pros

- The combination of positive-unlabeled classification and conditional generation with extra unlabeled data is naturally and sounds reasonable. 
- The extensive experimental results indicate the effectiveness of the proposed framework on both classification and generation tasks.
- The writing is good, with a well-organized structure, clear illustration, rigorous equations.

## Cons

- Lack generation experiments on high-pixel datasets such as ImageNet.
- It should compare more SOTA PU and GAN methods.
- The proposed looks complex compared to the baseline methods, contains multiple components such as PU, G, C, D. 


summary_of_the_review: The proposed training framework which jointly targets PU classification and conditional generation is novel and sound.
But the novelty is limited and the lack some key experiments to show the improvements.
So I tend to reject the paper.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper proposes to leverage Positive-Unlabeled (PU) learning and the conditional generation with extra unlabeled data to simultaneously improve classification and generation performance. The authors show that their method is especially effective for handling out-of-distribution unlabeled data.
main_review: Pros:
1. The high-level idea that simultaneously conduct PU classification and conditional generation is reasonable.
2. Experimental results show the effectiveness of the proposed method.

Cons:
1. The motivation is not that clear, especially why adopting multi-class PU learning rather than semi-supervised learning with out-of-distribution data.
2. Some necessary baseline methods should be compared.
summary_of_the_review: This paper proposes CNI-CGAN to jointly exploit PU classification and conditional generation, so as to address the label sparsity problem. I feel that the high-level motivation is reasonable, but I still have some concerns:
1. I think the motivation of this paper need more explanations. For the label scarcity problem mentioned in this paper, I feel that some recent semi-supervised learning methods are also applicable. For example, Semi-supervised learning under class distribution mismatch (AAAI 20), Safe deep semisupervised learning for unseen-class unlabeled data (ICML 20) also tackle the out-of-distribution data in semi-supervised learning. Besides, they are naturally deep model which do not need any extensions. In this sense, my question is, why adopting Multi-PU learning (needs extension to deep model) rather than these semi-supervised methods?
2. The authors wrote "In classification, a common setting to utilize unlabeled data is semi-supervised learning (Miyato et al., 2018; Sun et al., 2019; Berthelot et al.,2019), which usually assumes that the unlabeled and labeled data come from the same distribution, ignoring their distributional mismatch. In contrast, Positive and Unlabeled (PU) Learning (Bekker & Davis, 2020; Kiryo et al., 2017) is an elegant way of handling this under-studied problem, where a model has the only access to positive samples and unlabeled data." Firstly, as mentioned above, there are some recent semi-supervised methods which can well handle the distribution mismatch. Moreover, the authors claim that PU learning can elegantly handle this problem, which I cannot  fully agree. Note that in the setting of classical PU learning (e.g., uPU and nnPU), they also assume P data and U data come from the same underlying distribution, therefore I feel that the nnPU method deployed in this paper cannot well handle the  distribution mismatch problem.
3. For estimating the transition matrix \tilde{C}, usually it is a not an easy task in label noise learning. Some assumptions are usually needed, such as anchor point assumption (see Are Anchor Points Really Indispensable in Label-Noise Learning? NeurIPS 19), etc. However, I do not see such pre-set assumptions in this paper, so I doubt about the estimation quality for this matrix.
4. For experiments, many recent PU learning works are not compared. Besides, another important work "On Positive-Unlabeled Classification in GAN" (CVPR 20) should also be discussed and compared. Therefore, I think the empirical study should be enhanced.
review_title: null

Meta-review: The paper combines discriminative and generative positive-unlabeled learning into a single framework. The reviewers argued the novelty and contributions are not enough for ICLR and unfortunately we cannot accept it for publication.
Decision: Reject