MIRAGE: Evaluating and Explaining Inductive Reasoning Process in Language Models

Authors: Jiachun Li, Pengfei Cao, Zhuoran Jin, Yubo Chen, Kang Liu, Jun Zhao
Primary area: other topics in machine learning (i.e., none of the above)
Keywords: inductive reasoning, large language model, model explanation

Abstract:
Inductive reasoning is an essential capability for large language models (LLMs) to achieve higher intelligence, which requires the model to generalize rules from observed facts and then apply them to unseen examples. We present {\scshape Mirage}, a synthetic dataset that addresses the limitations of previous work, specifically the lack of comprehensive evaluation and flexible test data. In it, we evaluate LLMs' capabilities in both the inductive and deductive stages, allowing for flexible variation in input distribution, task scenario, and task difficulty to analyze the factors influencing LLMs' inductive reasoning. Based on these multi-faceted evaluations, we demonstrate that the LLM is a poor rule-based reasoner. In many cases, when conducting inductive reasoning, they do not rely on a correct rule to answer the unseen case. From the perspectives of different prompting methods, observation numbers, and task forms, models tend to consistently conduct correct deduction without correct inductive rules. Besides, we find that LLMs are good neighbor-based reasoners. In the inductive reasoning process, the model tends to focus on observed facts that are close to the current test example in feature space. By leveraging these similar examples, the model maintains strong inductive capabilities within a localized region, significantly improving its deductive performance.

Decision: accept (poster)
Reviewer scores: 5, 6, 8, 5

Reviews and discussion:
<thread>
creation date:
1730193348536

thread_type:
official_review

<thread_object>
creator: reviewer_ctbs

<content>
summary:
Authors created a synthetic dataset MIRAGE to evaluate the reasoning ability of LLMs, with two conclusions: (1) LLMs are not good at deductive reasoning, (2) LLMs are good at performing inference based on neighborhood information, thus, they are "neighbor-based inductive reasoner". 

Authors try to distinguish the ability of deductive reasoning from the ability of inductive reasoning. The missing experiment(s) is/are to distinguish the ability of reasoning from the ability of pattern matching. For example, it is necessary to show the so-called "neighbor-based inductive reasoner" is the ability of reasoning, instead of the ability of pattern-matching.

strengths:
A new synthetic dataset is created and contributed to the reasoning evaluation of LLMs. 
Authors conclude that LLMs are poor in rule-based reasoning, and good at neighbor-based reasoning.

weaknesses:
Authors coined the nice term "neighbor-based reasoning". But, is it reasoning or just pattern-matching? I am afraid there is no term of "neighbor-based inductive reasoning" in the literature of logic or psychology. 

It is not surprising that LLMs are good at "predicting" based on neighborhood context, as this is one of the important methods to train them.

questions:
1. why do you only mention inductive reasoning in the title? you also report experiments on deductive reasoning of LLMs. 

2. do you assume any causal relations between the input and the output of rules, or just associative relations? If only associative relations, how do you distinguish deductive from inductive reasoning?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730621836456

thread_type:
official_review

<thread_object>
creator: reviewer_cttk

<content>
summary:
This paper investigates the inductive reasoning abilities of LLMs, introducing Mirage, a synthetic dataset designed to comprehensively evaluate LLMs' inductive and deductive reasoning from various perspectives. Using Mirage, the authors demonstrate that LLMs are weak at rule-based reasoning but perform well with neighbor-based reasoning. These findings suggest that LLMs' deductive reasoning can be enhanced by leveraging similar examples to support a robust inductive reasoning process.

strengths:
Strengths:
- Present a synthetic data generation framework to evaluate inductive and deductive reasoning processes. The authors provide clear steps detailing the data generation process, specifically outlining the rule generation and question generation procedures.
- Various advanced prompting techniques, including SR and HR, along with the latest LLMs, are tested in the experiments.
- The effective scope analysis for neighbor reasoning supports the claim that LLMs are effective neighbor-based reasoners.

weaknesses:
Weaknesses:
- In the section on neighbor reasoning, the authors use the term "Feature Space" to denote the space of observed facts and test samples. However, it’s unclear where the features in this space originate.
- The data setting may lack sufficient complexity to effectively demonstrate inductive reasoning capabilities in real-life scenarios.

questions:
Questions:
- Could the authors offer additional insights or empirical evidence on the model's performance with real-world datasets or in less controlled environments?
- Additionally, could the authors also check if fine-tuning over the synthetic data helps the inductive reasoning of LLMs?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730633306297

thread_type:
official_review

<thread_object>
creator: reviewer_dd2x

<content>
summary:
This paper presents a new dataset for evaluating the induction and deduction ability of LLMs. The authors provide a very detailed explanation of the generation method of the dataset and make extensive experiments to evaluate current state-of-the-art models. The authors also propose two hypotheses about LLMs' reasoning behaviours, and the experiments are designed to verify these hypotheses. Detailed explanations and discussion are provided, and the supplementary material also gives good support to the conclusion.

strengths:
- The paper covered a broad range of related works and provided another valuable benchmark for evaluating LLMs' reasoning power. The tasks has been formulated in four representations to ensure the diversity.
- The authors make two interesting hypotheses on LLM's behaviour, and carefully designed several experiments to verify them.
- In order to evaluate LLMs' behaviours, the authors designed several metrics specifically, most of which make sense to me. Detailed discussions are provided, and all the charts and tables are explained well.

weaknesses:
- First of all, all the experiments are behavioural studies and empirical evaluations, and there is no mathematical evidence about LLM's induction/deduction power. IMHO, if there is no mathematical proof, then claims such as "we prove that LLMs are poor in inductive reasoning" would be too strong.
- It is fine to evaluate the models' induction ability by using the designed tasks. However, I wouldn't call the other task as "deduction" since the rules are not provided. From the definition from logic, a deductive inference needs both input facts and the logic rules for deducing the outputs.

questions:
Please see above.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730713460435

thread_type:
official_review

<thread_object>
creator: reviewer_6phz

<content>
summary:
This paper evaluates and analyzes LLM's ability for inductive reasoning. Compared to previous works in inductive reasoning, it adopts a new setting, which is to compare the performance of LLMs under two settings: (1) specific --> general --> specific, and (2) specific --> specific.
It is interesting to see that the performance of LLMs under the two settings is comparable. The result indicates that setting (2) will have a better performance, even though traditionally we think a general rule is important to make further inferences to unseen specific examples.

As for results, the main findings of this paper are that 
(1) setting (2) outperforms setting (1);
(2) many techniques such as self-consistency can't improve performance in the proposed task; and 
(3) in-context demonstrations that are closer to the input can help more in terms of the accuracy of output.

strengths:
1. The main finding is insightful, that setting (2) outperforms setting (1), which can bring in more discussions on how LLMs perform inductive reasoning.
2. The analysis in terms of neighbor-based reasoning, especially the investigation into the scope is also interesting. However, it might not be surprising, since the similarity of in-context demonstrations has been discussed extensively in previous papers.

weaknesses:
1. The writing is not clear. Specifically,
a). what does the "substitution" in line 414 and line "443" mean?
b). after finding IF, CF, and OF, how are they used?
c). what is the metric in Table 1?

2. The claim "We prove that LLM is a poor rule-based inductive reasoner" is too strong and unacceptable. What is the definition of "poor"? And have the authors proved it mathematically? 

3. "We prove that LLM is a neighbor-based inductive reasoner.": Have the authors proved it mathematically? When LLM is a neighbor-based inductive reasoner?

4. Although the performance that setting (2) outperforms setting (1) is provided. There might lack of a more in-depth analysis of when LLMs use neighbor-based reasoning, and when LLMs use the general pattern thinking? Or is it that LLMs can purely do neighbor-based reasoning, and never use the general pattern thinking at all? It seems that although each experiment is interesting, but I don't know what is the takeaway knowledge from this paper.

questions:
See above
</content>
</thread_object>

</thread>
<thread>
creation date:
1731914226055

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Response to Reviewers

comment:
Dear Reviewers

We sincerely appreciate the time and effort you've dedicated to reviewing our work. In response to your valuable feedback, we have provided detailed clarifications to the questions raised and updated the version of our manuscript, mainly including:

+ **More rigorous expression:** Based on Reviewer 6pHZ's questions 2 and 3, as well as Reviewer DD2X's question 1, we have **revised the main contributions of the paper** to: "We find that LLM is a poor rule-based inductive reasoner" (see line 105) and "We demonstrate that LLM is a neighbor-based inductive reasoner" (see line 106).
+ **Clearer task categorization:** To address the confusion regarding inductive and deductive reasoning raised in Reviewer DD2X and Reviewer cTBS, we have added a footnote in line 44 of the Introduction to **clarify the task definition and distinguish it from traditional logic**.
+ **Clearer presentation:** We have **emphasized some of the original but unclear expressions in our manuscript**. For example, Reviewer 6pHZ's question 1 (refer to line 414 and Table 1) and Reviewer cTBS's question 1 (refer to lines 455-456).
+ **More Experiments**: Following the suggestion of Reviewer ctTK's question 4, we have **added experiments in Appendix B.3 to explore the impact of fine-tuning methods** on the model's inductive reasoning capabilities.
 
We sincerely apologize once again for the confusion caused by these presentation issues. At the same time, **we eagerly await your thoughts on our response**, particularly whether it adequately addresses your concerns. We remain fully committed to incorporating all of your suggestions into our revision process to further improve the quality of our manuscript.
</content>
</thread_object>

</thread>


Metareview:
metareview:
The paper claims that LLMs perform better when asked to generalize from facts to neighbor facts (dubbed neighbor-based reasoning) than when asked to learn rules from the available facts, and thereafter use these rules (rule-based reasoning). 
A dataset MIRAGE is built to investigate and experimentally support this claim. The dataset generation proceeds by defining transformation rules, synthesizing facts based on these rules, designing inductive and deductive questions on the facts, and designing tasks (list and string transformations, code generations).
The importance of the filtering step should be assessed too; can fact or rule redundancy alter the neighbor-based reasoning (e.g. biasing the underlying similarity) ?