Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The work introduced a self-consistent learning framework, where a generator and a discriminator are trained cooperatively and alternatively. A selection mechanism is proposed to filter samples in the training process. The sentence semantic matching task is demonstrated as an example to support the effectiveness of the proposed framework. The results show the framework outperforms standalone discriminative models on four datasets in both zero-shot and full-data settings. 
strength_and_weaknesses: Strength: The paper is well-organized, and the main ideas are interesting, and easy to understand. The proposed method is evaluated on multiple datasets with good performance in sentence sematic matching. 

Weakness: 1. The work included both positive and negative samples in the training of discriminator, and claimed cooperative setting can lead to better training stability than adversarial setting. The idea makes sense but it is not clear to me how this point is supported either in theoretical or experimental aspects. 2. The definition of the output probability for generator $p_G$ cannot be found. 3. It is mentioned in Eq (1) that $p_D$ depends on the embedding representation H of generated samples. Since it is shown in the paper that the selective mechanism is important to the performance, I think it would be helpful to provide more details on this embedding process, and some evidence or explanation to justify the resulting output probability is somehow “good enough” for selective mechanism. 4. It is mentioned in page 4 that both positive and negative samples would be included in the training of discriminator, (which is a key difference between this method and GAN.) Due to the lack of information on the embedding representation, I fail to see why Eq (2) can make sure of this. 
clarity,_quality,_novelty_and_reproducibility: In my opinion, the work is written in good quality and clear in general. Some questions mentioned above may need some clarifications. Codes were provided in supplementary materials, so it should be reproducible. 
summary_of_the_review: The paper includes interesting ideas and shows the proposed approach can improve performance for sentence sematic matching task. It may be more influential if fundamental evidences or analysis are available to compare cooperative and adversarial setting.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: In this study, authors proposed a self-consistent learning algorithm to generate data samples to improve downstream performance. The self-consistent learning was designed on generator-discriminator framework which is similar as GAN, but different from that of GAN, the generator and discriminator were cooperatively learned to enhance each other. In the self-consistent learning, authors designed a data selection mechanism to select the generated data with a dynamic threshold, and the selected data were further applied to enhance the generator and discriminator (similar as data augmentation). Finally, authors examined their idea on several sentence semantic matching experiments, and confirmed the effectiveness of their method.
strength_and_weaknesses: Strength: different from GAN where generator and discriminator are in a competitive learning strategy which is difficult to train, and difficult to use the generator data in further training (it is difficult to judge the quality of the generated data samples ), authors' idea with self-consistency learning could learn both the generator and discriminator in a cooperative way to enhance each other. Therefore, the training is stable and guarantee to converge.

Weakness: The dynamic threshold design as given in Eq. 3 seems important in deciding the selection of the data, however, authors gave a simple incremental formulation in later part for designing the dynamic threshold. And there is no further explanation and ablation study on the design of the dynamic threshold. 
clarity,_quality,_novelty_and_reproducibility: The purpose and motivation were clearly explained, and is novel as providing a new way for training generator and discriminator with data selection. Authors claimed to open their code and data. Also with the explanation in the paper, the idea is not difficult to reproduce.  
summary_of_the_review: The paper proposed a novel cooperative training of both generator and discriminator to generate data for enhance model training. In the model, authors designed a dynamic threshold for data generated data selection in a self-consistent learning framework. Authors experiments showed effectiveness of their idea.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper proposes a a self-consistent learning framework to train a discriminator and a generator cooperatively in a closed-loop form. The discriminator and the generator enhance each other during multiple rounds of alternating training. Experimnetal results are given to validate that the proposed framework is easy to train and free from instabilities such as mode collapse and non-convergence. 
strength_and_weaknesses: Strength:
The paper proposes a self-consistent learning framework to enable cooperative training of the generator and the discriminator. The generator and the discriminator are trained alternatively until reaching a score consensus.  This framework makes plenty use of the limited labeled data and large-scale unlabeled domain-related corpus. 

Weaknesses:
The paper formulates the main theoretical result into theorem 1, but the proof of theorem in the Appendix is not obvious. In particular, how do you derive from the first line optimization problem to the second optimization problem. Could you provide further detailed proof for the theorem?
The choice of the dynamic threshold looks quite arbitrary, could you provide some criterion for the parameter selection?

clarity,_quality,_novelty_and_reproducibility: The writing is good, but the proof of the main result needs to be further validated.
summary_of_the_review: The paper proposes a self-consistent learning framework to enable cooperative training of the generator and the discriminator. The framework can utilize both limited labeled data and large-scale unlabeled domain-related corpus. But the proof of the main theorem still needs to be validated.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper proposes a co-operative technique for Generator and Discriminator to cooperate and aid each other instead of the standard adversarial game of of the GAN. The proposed technique uses an augmentation scheme to augment the learning data of the generator and data to aid each other's learning progress. 
strength_and_weaknesses: Strengths:

* Simple technique for assisting the learning of generators and discriminators using a co-operative technique rather than adversarial technique 

Weakness:

* Several cooperative GAN based augmentation systems have been proposed and the novelty is extremely low.
* The experiments only show some NLP based experiments and no Vision based experiments. 
* Ideally the model should also work with image generation but the conspicous absence indicates the non-generality of the technique
* The proofs are hand wavy and not concrete. 
clarity,_quality,_novelty_and_reproducibility: The paper is fairly clear. The writing can be significantly improved. The quality of the technical writing especially the proofs are significantly lower than ICLR standards. 

The novelty is quite low. Several other works such as https://www.nature.com/articles/s41598-019-52737-x have proposed Data Augmentation schemes with GANs. 

The zip file provided with the submission and the code looks that it can be used to reproduce the experiments easily. There are a bunch of differences that haven't been highlighted in the paper which could be improved in the text. 
summary_of_the_review: The paper significantly lacks novelty. The proposed technique should ideally also be beneficial to image generation. However, the lack of results raise the suspicion on the generalizability of the technique. The metrics reported on the text based datasets are moderate at best.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: 1. This paper proposes a self-consistent learning framework, in which a discriminator and a generator are cooperatively trained in a closed-loop form. Experimental results demonstrate the effectiveness of the proposed framework.
2. The proposed algorithm could learn both the generator and discriminator in a cooperative way to enhance each other. Therefore, the training is stable and guaranteed to converge. 
3. The experiments only show the improvements on NLP tasks but not other tasks, such as Vision. 
4. Several similar works (Cooperative Training for GAN) shave been proposed previously [1-4]. Thus, the novelty is relatively low. 
5. The paper has some degree of scientific novelty, but may not meet ICLR's very high standards. 

[1] https://arxiv.org/pdf/1804.03782.pdf

[2] https://hal.archives-ouvertes.fr/hal-03736116/file/lamprier22a.pdf 

[3] http://www.stat.ucla.edu/~ywu/PAMI.pdf 

[4] https://www.sciencedirect.com/science/article/abs/pii/S0031320321002855
justification_for_why_not_higher_score: 1. Reviewers agree that if the proposed algorithm is not completely new, then additional experiments on Vision should be conducted to further confirm the effectiveness of the proposed approach. 
2. One reviewer provided several previous works [1-4] that share similar ideas to this paper.

[1] https://arxiv.org/pdf/1804.03782.pdf 

[2] https://hal.archives-ouvertes.fr/hal-03736116/file/lamprier22a.pdf 

[3] http://www.stat.ucla.edu/~ywu/PAMI.pdf 

[4] https://www.sciencedirect.com/science/article/abs/pii/S0031320321002855
justification_for_why_not_lower_score: 1. The paper is well-written. The experimental results on text tasks are convincing. 
2. The proposed algorithm could learn both the generator and discriminator in a cooperative way to enhance each other. Therefore, the training is stable and guaranteed to converge.
Decision: Reject