FGRCAT: A Fine-Grained Reasoning Framework through Causality and Adversarial Training
Authors: Hanghui Guo, Jia Zhu, Shimin Di, Changfan Pan, Zhangze Chen, Xiaofang Zhou


Reviewer paper summaries: This paper introduces FGRCAT, a framework for Natural Language reasoning.

FGRCAT employs causality and adversarial training to improve the NL reasoning capabilities of pretrained models such as BERT by following a two-pronged training approach.
On the one hand, GPT-2 and GNNs are employed to generate causal data for enhanced training, providing reasoning over causalities to the model at training time. On the other hand, synonym and antonym perturbation methods are used to generate augmented training data.

Experiments show that FGRCAT is able to improve the underlying model's performance on a variety of datasets. In some cases, a BERT trained with FGRCAT is able to outperform strong LLM baselines such as GPT-4.
Summary of strengths: Extensive experiments.
Summary of weaknesses: The paper is hard to follow.
Some claims seem weekly supported.
Comments, suggestions and typos: The paper is quite hard to read, with lots of information (especially in very dense figures) that could be presented more clearly.

In parts this reads like a report for an MSc project, with some superfluous things (e.g., lines 231--238, "This learned knowledge [...] is encoded within its internal parameters." -> yes, obviously, that's why we train it.), or missing bits (e.g., Eqn 2 is just introduced as "the loss", but it's never explicitly stated which part of the whole pipeline is trained with it. No loss for the GNN is introduced?)

What is "anti-inference"? The term is used a few times, but never explicitly defined.

Lines 305--307: "Compared to traditional methods, it better handles fine-grained information [...]" -> Did you verify this claim?

I am not sure what is adversarial about the "Adversarial Training Enhancement Model Reasoning" (4.2). Usually, adversarial methods need a target that they adverse however, neither the Synonym  nor the Antonym Perturbation methods specifically target model weaknesses (e.g., by specifically generating examples that are classified wrongly by the base model).
Instead, these look like general data augmentation methods which can be used to generate new "artificial" training data, but I cannot see any actual adversarial part.

In Figure 4, left, is the example correct? The Cause is given as "The man was craving a cigarette.", while the Effect is given as "He was addicted to nicotine." I would argue that Cause and Effect are swapped here, and the addiction should be the cause of the craving. Am I wrong?

Results seem a bit weak. For example on COPA Staliunaite et al. (2021) which is cited in the paper and also employs synonym replacements and adversarial training, report up to 90% and more on the COPA test set.

Some model selections seem questionable; e.g., why use BERT as the base model for the method, and not RoBERTa (which should be stronger in most scenarios)? BERT and BART are both 5 years old by now, and better encoder models exist.
Overall assessment scores: 2