Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper proposes an unsupervised adversarial pre-training framework called LogiGAN to improve the ability of logical reasoning in pre-trained models. The authors adopt heuristic rules to detect logic indicators and replace the vanilla masked language modeling with masked logical statement prediction. To explore more logically reasonable statements, the authors devise a GAN-like training process which iteratively trains the generator and verifier. Experimental results on 12 datasets show the superior performance of LogiGAN.
strengths_and_weaknesses: Strengths:

1) The motivation to improve the logical reasoning abilities of pre-trained models is intuitive. The proposed method directly supports the motivation.
2) Experimental results show that LogiGAN can outperform T5 on 12 datasets.
3) This paper is overall well-written and easy to follow.

Weaknesses:
1) The novelty of the proposed method is somewhat limited. In my view, the main contribution of this paper includes the pre-training task of masked logical statement prediction and the adversarial training framework. However, masked logical statement prediction seems like a directly improved version of masked language modeling based on logic indicators. And the adversarial training framework which avoids Monte Carlo search and adopts the sentence-level training loss has also been deeply studied in the community of text GANs, such as [1] and [2]. Thus, I feel that the contribution of this paper is mainly on the applicational side.
2) The sampling of pseudo-statements plays an important role in the training of verifier. I wonder why self-sampling and retrieving similar statements based on BM25 can provide high-quality negative samples to benefit the verifier. From my perspective, the negative samples should be logically inconsistent, which are not intuitively reached by these two methods. The authors say that they use a NLI model to assign labels in the appendix. But this only affects the label assignment without improving the quality of negative samples.
3) The authors mask the task-specific texts (which are related to logical reasoning) in the pre-training task, which may degrade the generalization ability of the pre-trained models to general NLP tasks. Other similar works which enhance the pre-trained models with specific abilities mostly conduct experiments on general benchmarks such as GLUE in addition to task-specific datasets, aiming to show that their method can maintain the generalization ability of pre-trained models. I suggest that the authors should also do this to make readers understand the ability of LogiGAN comprehensively.


[1] Maximum-Likelihood Augmented Discrete Generative Adversarial Networks.

[2] ARAML: A Stable Adversarial Training Framework for Text Generation. EMNLP 2019.

questions: I have included my questions in the weaknesses part.
limitations: The authors have not adequately addressed the limitations and potential negative societal impact of their work.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper proposed LogiGAN. It circumvents the non-differentiable challenge of sequential GAN via a novel Generator-Verifier scoring consensus mechanism, and enables large scale pre-training with arbitrary target length. Extensive experiments and ablation studies reveal the effectiveness and functional components of LogiGAN. Experiments on 12 datasets requiring general reasoning abilities, revealing the fundamental role of logic in broad reasoning, as well as the effectiveness of LogiGAN.
strengths_and_weaknesses: 1. The paper is well organized and easy to follow.

2. The motivation of this paper is quite attractive. It firstly hypothesizes that (i) logic ability plays a key role in a wide scope of tasks requiring general reasoning; and (ii) PLMs’ logic ability can be further improved beyond their original linguistic ability. Upon automatic identification of logical reasoning phenomena in massive text corpus via detection heuristics, they train language models to predict the masked-out logical statements.

3. The experiments are solid. One concern here is the numbers in table 1 are kind of close. Is that possible to make a significant test for the improvement? Also it will be better to have a column for avg performance.
questions: 1. Are the vanilla T5 base model's results from your implementation or they are reported by some other people?
limitations: None
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The paper presents a LogiGAN, a generative adversarial approach inducing logic reasoning language models. The core claims made in this paper are: (a) logic helps with reasoning, (b) adversarial pre-training improves performance on NLP tasks requiring reasoning, (c) Generator-Verifier mechanism enables large scale pre-training with arbitrary target length. The claims are supported by experiments over 12 datasets and an ablation study.
strengths_and_weaknesses: The paper has an original approach to GAN for language models that demonstrates improvements of results. The quality of the paper is in its evaluation on 12 downstream tasks, although it would be nice to see a comparison with SOTA on the given task for reference.

Strengths:
- The paper is well written and presents consistent improvements over all experiments.
- Novel way of circumventing the non-differentiability issue of SeqGAN

Weaknesses:
- The paper claims that their method allows for large scale pre-training with arbitrary target length. This is not clear to me since (a) the verifier is based on a ALBERT-large and therefore, as far as I understand, is still limited to 512 tokens, (b) I would expect a more detailed explanation as well as empirical results of how much more efficient the LogiGAN is compared to sequential methods to substantiate this claim as a contribution.
- The paper claims that linguistic and logic ability is orthogonal. However, the evidence for this comes from Ablation I, where the model is trained to predict random masked sentences. I do not understand how this demonstrates that linguistic ability does not play a role in improving the model performance. The random selection of masked sentences surely removes logical and linguistic merit equally. Furthermore, in Ablation II, it is showed that the model does considerably improve with MLE pre-training.

Formatting: 
- Figure 2 is hard to read in print
- Table 1 has a typo in the title (devleopment setsjudgement).
- paragraph names should end with a period
- 75- GAN posed (GAN put*)

questions: - 330-331: What is the difference between 'qualities of generated examples' and 'intrinsic logic ability'?
- What is 'x' in equation 1? Is it supposed to be 'c'?
limitations: The authors have not mentioned either. The work does not seem to pose any threat to society beyond those inherent to a large language model.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper aims at improving the logical reasoning ability for language models via a pre-training architecture LogiGAN. The adversarial framework adopts a Generator-Verifier style. The Generator learns to recover the masked statements instead of random tokens/spans in traditional pre-training settings. It also learns to score candidate statements to reach a scoring consensus with the Verifier. The verifier, on the other hand, learns to judge the logical consistency between the context and the predicted statement. Additionally, in this way, LogiGAN overcomes the challenge of the non-differentiability of sequential GAN. LogiGAN is verified on 12 datasets designed to test reasoning abilities and surpasses the baseline model.

The key contribution of this paper, as I understand, is to leverage sententious logical relations in pre-training to enhance the reasoning abilities for language models.

strengths_and_weaknesses: **Strengths**:
1. This paper is overall well-written with clear motivations and good structures.
2. The research topic is interesting, i.e., improving general reasoning ability for language models via taking Logic as an entry point. 
3. The proposed LogiGAN is technically sound and surpasses the baseline models in a broad range of reasoning datasets with different kinds of task settings.
4. The ablation studies are nice.

**Weaknesses**:
1. The major weakness of this paper is, in my eyes, the superficial modeling of logical relations. Deductive, Inductive, and Abductive reasoning types are only manifestations of logical reasoning, they do not touch down to the essence. Based on this classification, this paper only models the consequential relations in a sentence-level style, which ignores a broad range of other logical reasoning expressions such as negation, disjunction, and conjunction. 
2. The so-called automatic identification of logical reasoning phenomena is in fact requires many manual labors. The authors have to pre-define words/expressions that indicate conclusion or premises, which greatly limits the universality of the proposed approach.
3. The authors only report the experimental results on the development set, which made the results less convincing. Also, the performance improvement over generation datasets is less noticeable than in classification datasets. It's hard to know that the model is really behaving as advertised rather than just adding capacity on top of the baselines.
5. In the related work section, especially `Pre-training for Reasoning Ability Improvement`, the authors describe several papers of other relation types that LogiGAN is not compared with. This makes it difficult to place this work in comparison with prior literature and understand the novelty of the proposed framework. Also, the key contribution of this paper could be blurred with improvement over GAN-like architecture rather than pre-training tasks or something else. 
6. I do not see any open-source statements or code submissions in the supplementary materials, so I take issues with the reproducibility of this paper.

**Minor Issues**:
1. Line 19: “Learning without thought is labor lost” -> “learning without thinking is labor lost”
2. Caption of table 1: “development setsjudgement” -> “development set judgment”

Overall, I'm borderline with this paper. I think the paper tests three aspects: 1) GAN-style training in language models. 2) predict the masked-out logical statements as the pre-training task. 3) detection heuristics for logical reasoning phenomena. It certainly demonstrates great experimental results compared with vanilla T5 baselines. But none of them **alone** is novel enough. The most important part in my eyes, leveraging logical reasoning phenomena for pre-training, is after all a sententious logical relation extraction, whose effectiveness is already demonstrated in [1].

[1] Huang, Yinya, et al. "DAGN: Discourse-aware graph network for logical reasoning." NAACL 2021.


questions: 1. How to ensure that the masked-out statements “are declarative and have a complete subject and predicate structures and are capable of being factually true of false”?
2. How to ensure that the Detection Heuristics indeed reflect ‘conclusion’ or ‘premise’? For example, `so` in “so happy” could be just an adverb indicating degrees. 
3. What are the statistics of parameters for LogiGAN? This could be important for a fair comparison. 

limitations: The authors provide an insightful `Discussion` section.
review_title: null

Meta-review: metareview: This paper presents a pretraining approach for integrating logical reasoning into language models. The authors develop an adversarial training method where they minimize the difference between the verifier scores and generator scores for logically inconsistent statements, thereby making the model less likely to generate sequences the verifier identifies as logically inconsistent. All the reviewers were positive (to varying degrees) on the contributions of this paper, specifically highlighting the clarity of the method, as well as the empirical rigor and results. Weaknesses pointed out by reviewers included the underspecification of different type of logical relations and the weakness of the pseudo-statement sampling procedure, which doesn't seem to explicitly retrieve logically inconsistent statements as negative examples. Despite these questions, the method is well motivated and clear, and the empirical results speak for themselves. This paper represents a solid piece of work and I'm inclined to see it accepted.
confidence: Less certain
Decision: Accept