CofCA: A STEP-WISE Counterfactual Multi-hop QA benchmark

Authors: Jian Wu, Linyi Yang, Zhen Wang, Manabu Okumura, Yue Zhang
Primary area: datasets and benchmarks
Keywords: LLM evaluation, Multi-hop QA evaluation

Abstract:
While Large Language Models (LLMs) excel in question-answering (QA) tasks, their real reasoning abilities on multiple evidence retrieval and integration on Multi-hop QA tasks remain less explored. Firstly, LLMs sometimes generate answers that rely on internal memory rather than retrieving evidence and reasoning in the given context, which brings concerns about the evaluation quality of real reasoning abilities. Although previous counterfactual QA benchmarks can separate the internal memory of LLMs, they focus solely on final QA performance, which is insufficient for reporting LLMs' real reasoning abilities. Because LLMs are expected to engage in intricate reasoning processes that involve evidence retrieval and answering a series of sub-questions from given passages. Moreover, current factual Multi-hop QA (MHQA) benchmarks are annotated on open-source corpora such as Wikipedia, although useful for multi-step reasoning evaluation, they show limitations due to the potential data contamination in LLMs' pre-training stage. To address these issues, we introduce the Step-wise and Counterfactual benchmark (CofCA), a novel evaluation benchmark consisting of factual data and counterfactual data that reveals LLMs' real reasoning abilities on multi-step reasoning and reasoning chain evaluation. Our experimental results reveal a significant performance gap of several LLMs between Wikipedia-based factual data and counterfactual data, deeming data contamination issues in existing benchmarks. Moreover, we observe that LLMs usually bypass the correct reasoning chain, showing an inflated multi-step reasoning performance. We believe that our CofCA benchmark will enhance and facilitate the evaluations of trustworthy LLMs.

Decision: accept (poster)
Reviewer scores: 6, 5, 5, 8

Reviews and discussion:
<thread>
creation date:
1730264531443

thread_type:
official_review

<thread_object>
creator: reviewer_gch3

<content>
summary:
The paper is designed to evaluate large language models (LLMs) on multi-hop counterfactual reasoning abilities. CofCA generates counterfactual contexts that LLMs likely haven’t encountered during pre-training, and ask questions based on the counterfactual contexts.  It also emphasizes sub-question evaluation for each reasoning step. 

The evaluation in this paper reveals that models like GPT-4 show inflated scores in standard QA due to these contamination issues, while performance drops on CofCA due to its reasoning on counterfactual contexts, highlighting the gap between LLM memory and genuine reasoning.

strengths:
1. CofCA minimizes data contamination by creating a cleaner benchmark for evaluating LLM reasoning.

2. The diverse counterfactual and factual datasets allow researchers to observe performance discrepancies, adding depth to LLM evaluation.

3. The constructed dataset is clearly different from existing counterfactual QA benchmarks, including distentQA and IfQA. The multi-hop challenge lacks of enough study in counterfactual setting.

weaknesses:
1. More open-source LLMs should be evaluated on the benchmark, like llama-3 or 13B-level models.

2. OpenAI o1 was evaluated in the paper, it shows using long reasoning-chains could help a lot on this reasoning tasks, and more insights on comparing o1 reasoning with 4o reasoning could be insightful.

questions:
As given in the weakness.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730300876296

thread_type:
official_review

<thread_object>
creator: reviewer_csw3

<content>
summary:
This paper introduces a new dataset called CofCA, designed to effectively assess LLMs' capability in counterfactual multi-hop question answering. The authors generate the dataset by starting with Wikipedia passages and applying techniques such as entity replacement and back-translation to create question-answer pairs. The models are then evaluated at both the final-answer level and the sub-question level. To ensure the quality of the dataset, the authors perform thorough quality checks and include human-in-the-loop validation throughout the dataset construction process.

strengths:
1. The paper is well-written and easy to follow, presenting the content in a clear and accessible manner.

2. The authors demonstrate an awareness of the limitations of using Exact Match (EM) and F1 scores when evaluating the output of large language models (LLMs). To address this, they introduce an additional Partial Match metric, which helps mitigate the issue and provides a more nuanced evaluation.

weaknesses:
1. One major drawback of the paper is that, while it claims to assess LLMs' true reasoning abilities, it primarily reports the accuracy of different sub-questions without fully addressing whether the models are capable of reasoning holistically. For a more comprehensive evaluation of reasoning abilities, other factors should also be considered, such as whether the model is able to effectively decompose the problem into meaningful or correct sub-questions, rather than treating them as isolated tasks.

2. The quality of the dataset is also in question. Currently, the correctness and informativeness of the dataset are evaluated using GPT-4. However, as the paper itself shows, GPT-4 struggles to correctly answer these questions, which suggests that it may not be fully reliable for assessing the quality of the dataset. This raises concerns about the robustness of the dataset validation process.

3. There is further concern about ensuring that the generated questions are non-trivial. As shown in Table 9, the prompt generates the multi-hop questions and the sub-questions, along with their answers, simultaneously. This raises a natural concern that the generated multi-hop questions may either be independent of these sub-questions or follow a single pattern (e.g., q1->q2->q3), leading to a lack of diversity in question structure. The paper does not address or discuss this potential limitation.

4. When benchmarking models, the paper lacks evaluation on more recent and advanced models, such as GPT-4 Turbo, GPT-4O, and Claude series, as well as newer open-source models like LLaMA 3 or Deepseek. Including these models would provide a more up-to-date and comprehensive analysis of the current state-of-the-art in LLMs.

questions:
1. Could you provide a more intuitive explanation for the **JOINT COMPUTATION** metric mentioned in Appendix B? An example or simplified breakdown might help readers understand its significance and how it is calculated in practical terms. This would not only make technical details more accessible but also allow readers to better appreciate its role in evaluating model performance.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730543629301

thread_type:
official_review

<thread_object>
creator: reviewer_royr

<content>
summary:
The paper introduces CofCA, an evaluation benchmark for multi-hop reasoning abilities of LLMs over counterfactual data. CofCA is created by (1) LLMs alter Wikipedia passages by replacing keywords with counterfactual information, and (2) LLMs generate multi-hop questions over the modified passages. Both steps are manually verified by human annotators, and the benchmark includes 900 multi-hop questions from 300 passages. Experimental results show a gap between performance on the counterfactual CofCA and existing multi-hop QA datasets. Additionally, the paper shows that the low performance is due to reasoning failures.

strengths:
- A counterfactual multi-hop benchmark can be helpful to understand the ability of LLMs to perform multi-hop reasoning. The benchmark will be publicly available in addition to the annotation guidelines and implementation details.

- The paper features extensive quantitative analysis examining the effect of the annotation pipeline and the performance of sub-questions amongst others.

weaknesses:
- Given the examples presented in the paper, I was not fully convinced about several aspects of the collected data. Mainly, how do you ensure that the counterfactual information does not contradict real-world facts? Also, both examples in the main paper seem to have shortcuts, e.g., there is only one position in the example in Fig.1, and only one designer in the example in Fig. 2.

- Qualitative analysis - the analysis presented in Section 5 did not fully address the concerns above, and is limited to 20 correct/incorrect questions with GPT-4. I think the paper could benefit from a thorough analysis regarding the quality of the generated data examining contradiction and shortcuts and verifying that the main claims of the paper (multi-hop QA is more challenging on counterfactual information) is supported. For example, it could have been interesting to have an error analysis that compares different error categories between factual and counterfactual questions.

- Presentation - the presentation in several parts of the paper was a bit hard to follow. For example, in Fig.1, the text is very small and there is an error (the bottom-right box the question is about the coach and the answer is Point Guard). In Tab.2, the ‘Whole data property’ is not the sum of the other columns (I assume this is due to the factual questions sampled from other datasets?). The main results in Section 4.2 require comparing between information presented in different tables (Tables 3 and 4). Table 8 is referenced before Table 7 but presented afterwards.

questions:
See weaknesses, mainly:
- Does the counterfactual information contradicts real-world facts? If so, isn't some degradation in performance expected? 
- How often are shortcuts in the collected examples? 
- What qualitative analysis did you consider that might strengthen the main claims of the paper?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730706904738

thread_type:
official_review

<thread_object>
creator: reviewer_6xmp

<content>
summary:
This paper proposed a new dataset, CofCA for evaluating LLM’s counterfactual multi-hop reasoning ability, which focuses on both final answer evaluation and reasoning step evaluation. The dataset construction involves 2 steps, 1) the sampled Wikipedia paragraphs are sent to GPT-4 for entity replacement and paraphrase, and then back-translation is done to further modify the paragraphsm and then these paragraphs are manually checked 2) GPT-4 is then used to generate QA pairs based on the newly generate paragraphs. Here GPT-4 is asked to generate both overall multi-hop QA pair and subquestion answer pairs, and the resulting data is also manually verified. 
The authors then conducted experiments on both newly generated data and HotpotQA, 2WikiHop and Musique, using both closed-sourced models and open-source models. The results show that all LLMs suffer significant performance drop on counterfactual questions and the joint performance which considers both final answer and reasoning process correctness can be even worse, exposing the weakness of current LLMs.

strengths:
1. CofCA could be a valueable resource for studying LLM’s true reasoning ability. Due to its counterfactual nature, the LLMs can no longer use their internal knowledge for finding reasoning shortcut, hence the model have to reason over provided context. Also, the annotation of intermediate sub question answer pairs can also enable examination of the reasoning process. 

2. The experiments and analysis are very thorough. The results show a clear trend of performance vs question complexity and the analysis on the reasoning process further exposes the weaknesses of the SOTA LLMs.

weaknesses:
My main concern is the scale of the passages set when constructing the data. For previous datasets such as. HotpotQA and 2WikiHop, they usually construct questions using multiple passages to ensure that the answer have to be deducted from multiple pieces of evidence. However, it seems that for CofCA, the questions are generate using only a single passage? If this is the case, how can you make sure the generated multi-hop QA pairs are fully grounded to the passage?

questions:
See weaknesses
</content>
</thread_object>

</thread>


Metareview:
metareview:
There is a discrepancy among the review scores. Yet it is worthy to note that the two reviewers who gave 5 did not participant in the after-rebuttal discussion.

I have personally read this paper and find it to be a valuable contribution. It tackles an important challenge in our field: how to disentangle the evaluation of LLMs’ reasoning abilities from their reliance on internal memory. The dataset construction process proposed in the paper is well-conceived, and the experimental results support its validity. Despite the evaluation being limited to specific scenarios and some unresolved issues raised by the reviewers, I believe the paper makes meaningful progress toward the long-standing goal of evaluating pre-trained models effectively.

That said, I concur with the reviewers on several notable weaknesses that remain unresolved:

- The paper lacks additional insights, particularly regarding the decomposition of questions.
- There is insufficient clarity on how the counterfactual information is ensured to not contradict real-world facts.

Furthermore, the absence of human performance benchmarks in the results table is a notable omission. Including a comparison involving human participants unfamiliar with the dataset construction would provide a more comprehensive evaluation.