$\texttt{CLR-Bench}$: Evaluating Large Language Models in College-Level Reasoning

Authors: Junnan Dong, Zijin Hong, Yuanchen Bei, Feiran Huang, Xinrun Wang, Xiao Huang
Primary area: datasets and benchmarks
Keywords: Large Language Models Evaluation, Benchark and dataset, College-level Reasoning

Abstract:
Large language models (LLMs) have demonstrated their remarkable performance across various language understanding tasks. While emerging benchmarks have been proposed to evaluate LLMs in various domains such as mathematics and computer science, they merely measure the accuracy in terms of the final prediction on multi-choice questions. However, it remains insufficient to verify the essential understanding of LLMs given a chosen choice. To fill this gap, we present $\texttt{CLR-Bench}$ to comprehensively evaluate the LLMs in complex college-level reasoning. Specifically, $(i)$ we prioritize 16 challenging college disciplines in computer science and artificial intelligence. The dataset contains 5 types of questions, while each question is associated with detailed explanations from experts. $(ii)$ To quantify a fair evaluation of LLMs' reasoning ability, we formalize the criteria with two novel metrics. Q$\rightarrow$A is utilized to measure the performance of direct **a**nswer prediction, and Q$\rightarrow$AR effectively considers the joint ability to **a**nswer the question and provide **r**ationale simultaneously. Extensive experiments are conducted with 40 LLMs over 1,018 discipline-specific questions. The results demonstrate the key insights that LLMs, even the best closed-source LLM, i.e., GPT-4 turbo, tends to '***guess***' the college-level answers. It shows a dramatic decrease in accuracy from 63.31\% Q$\rightarrow$A to 39.00\% Q$\rightarrow$AR, indicating an unsatisfactory reasoning ability.

Decision: reject
Reviewer scores: 5, 3, 5

Reviews and discussion:
<thread>
creation date:
1729521355264

thread_type:
official_review

<thread_object>
creator: reviewer_mxsi

<content>
summary:
This work presents CLR-BE, a dataset consisting of questions from 16 challenging college disciplines in computer science and artificial intelligence. The dataset contains 5 types of questions, while each question is associated with detailed explanations from experts. The paper introduces a method to judge the models’ reasoning abilities based on the correctness of their reasoning paths.

strengths:
1.The dataset provides a new way to evaluate the reasoning abilities of LLMs, focusing on the correctness of the reasoning process instead of only on the final answer

2.The paper is well presented, with clear figures and illustrations

weaknesses:
1.The domain of the problems seems limited, as it seems to only contain computer science related questions. Computer science seems not to be a very representative field for the evaluation of reasoning abilities. The dataset can benefit from inclusion of more challenging and reasoning intensive tasks from subjects such as math and physics.

2.The evaluation of the correctness of reasoning processes involves semantic similarity and GPT-4-assisted expert evaluation. I am not so sure about the accuracy of this evaluation process as reasoning paths can be varied. Also, the errors in reasoning paths can be subtle, and this method of evaluation seems a little coarse to me.

questions:
The dataset contains a lot of choice questions. Perhaps a more efficient way is to transform choice questions into open-ended ones with more possible final answers to ensure that a correct final answer indicates a correct reasoning process?

It would be good to have more case studies about various model generated solutions, including false positives, false negatives, ect.
</content>
</thread_object>

</thread>
<thread>
creation date:
1729914208745

thread_type:
official_review

<thread_object>
creator: reviewer_wite

<content>
summary:
This paper proposes a novel benchmark CLR-Bench to evaluate the reasoning ability of LLM in college-level **computer science and artificial intelligence** tasks.
Based on the experimental results, the authors found that current LLMs tend to ‘guess’ answers for college-level questions, i.e., LLMs can reach the correct answer but contains incorrect reasoning steps.

strengths:
- a novel benchmark on computer science and artificial intelligence tasks is proposed
- the writing is easy to follow

weaknesses:
- many wrong statements in this paper try to mislead readers:
  - In Table 1, GSM8K and MATH are not multiple-choice questions. Moreover, their samples contain complex rationale to evaluate LLMs. 
  - In L105, the authors claim that $Q\to A$ and $Q \to AR$ are two novel metric proposed by them. however, $Q\to A$ (i.e., outcome accuracy) has been widely used in many reasoning tasks to evaluate the performance. also, $Q\to R$ is equivalent to the process accuracy proposed in the paper Let's verify step by step (https://arxiv.org/abs/2305.20050). The only minor novelty in this paper is the $Q\to AR$ metric, which assigns 0.5 to the prediction if the answer is wrong but its process is correct. 
- evaluating the correctness of the reasoning path is a popular topic in LLM (some related works are list below), however, none of them are discussed in this paper.
  - Evaluating Mathematical Reasoning Beyond Accuracy
  - SELF-[IN]CORRECT: LLMs Struggle with Discriminating Self-Generated Responses
  - SELFCHECK: USING LLMS TO ZERO-SHOT CHECK THEIR OWN STEP-BY-STEP REASONING
  - Let's Verify Step by Step
  - The Generative AI Paradox: “What It Can Create, It May Not Understand”
- the proposed benchmark is very small (only about 1k samples) and is very narrow (computer science). Hence, the observations are also limited to the computer science domain and cannot be generalized to other reasoning tasks. 
- Previous works usually study the process correctness of mathematical problems. compared with existing math benchmarks, what are the essential advantages of using computer science discipline?

questions:
see above Weaknesses
</content>
</thread_object>

</thread>
<thread>
creation date:
1730346218881

thread_type:
official_review

<thread_object>
creator: reviewer_nzlv

<content>
summary:
This paper evaluates the reasoning capabilities of current LLMs, introducing a new evaluation method and corresponding dataset. The author argues that assessing only the correctness of answers is insufficient; it is also necessary to evaluate how the model arrives at these answers. To address this, the author constructed a dataset containing different types of questions and answers, along with the reasoning processes behind the answers. The study evaluated both the accuracy of the answers provided by the LLMs and the correctness of their reasoning processes. Testing on mainstream LLMs revealed that these models tend to guess the answers, as the accuracy of their reasoning processes is generally much lower than that of their answers.

strengths:
1. Currently, considering only answer correctness is indeed insufficient to comprehensively evaluate the reasoning ability of LLMs. The author’s proposal to assess the correctness of the reasoning process is a valuable direction.

2. The author evaluated mainstream LLMs, covering a wide range of models.

3. Using this newly proposed evaluation method, the author arrived at different conclusions, finding that LLMs tend to guess answers, as their reasoning correctness is significantly lower.

weaknesses:
1. This paper discusses a lot about the logic and process of building datasets, but it does not include any steps for ensuring or validating data quality within this process. Although the paper emphasizes the high quality of the newly proposed evaluation dataset, there is no guarantee of this quality in the outlined process. Simply assuming that human-provided annotations are of high quality is unconvincing.

2. Some descriptions in this paper lack precision. For example, in line 321, how exactly is "partially correct" determined? Does it mean that as long as the first sentence is correct, it counts as partially correct, or is it based on the subjective judgment of experts?

3. The author provides an assessment method to validate Q->R accuracy. However, this method still requires a substantial amount of manual involvement. So, how can the evaluation data proposed in this paper be better spread?

4. The author only provided the 1-shot results. Although some explanation was provided, it was not convincing to me. In my view, conducting 0-shot and k-shot validations remains an important metric for evaluating the performance of different models.

questions:
Although this work provided a good point to evaluate LLMs, there are still many drawbacks as shown in weakness. Please address them accordingly.
</content>
</thread_object>

</thread>
<thread>
creation date:
1733211860624

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
General Responses to All reviewers and AC

comment:
Dear Reviewers and AC,

We would like to sincerely thank all the reviewers for your valuable comments and efforts. We are also happy to be acknowledged for our contributions and significance to the community. To address the concerns raised by reviewers, we have correspondingly added new experiments and case studies, which we believe have made our approach much more comprehensive and convincing.

We wish to invite all reviewers to check our new results and observations, as well as our rebuttal to support our contributions:  
* **Dataset Size**: As a concentrated dataset focusing on one valuable domain, the size of our CLR-Bench, i.e., 1,018, has already satisfied the requirements to comprehensively evaluate the models' performance and make convincing and interesting observations. We have listed 6 of the former papers that are published on top-tier conferences with small number of questions/samples for your kind reference.
* **Value of CS/AI Domain**: We highly value this domain more than all the existing benchmarks for four reasons.
**First**, CS and AI naturally contain a variety of types of challenging tasks compared with all the existing 'single-type' benchmarks. In CS and AI domain, we have conceptual understanding tasks (algorithms, networks, databases, paradigms, machine learning, NLP, CV), Math problems (Discrete mathematics, linear algebra, matrix computation, calculus), Code generation problems (programming, optimization, code completion, error detection), etc. **Second**, the performance on LLMs on CS and AI is still very unsatisfactory or even bad It is valuable to benchmark and research. **Third**, it remains a blank in the benchmark of LLMs considering CS and AI. This also brings two opportunities. (i) LLMs will have few chances to be fine-tuned based on our questions. (ii) Following research on RAG or LLMs themselves could benefit a lot from our benchmark. **Fourth**, we are pioneering in a more comprehensive one in terms of multiple questions types and the difficulty in college-level. The real scenarios of applying LLMs could be much more complicated while single-type questions (in existing benchmarks) could hardly satisfy.
* **Dataset Quality**: We have multiple mechanisms to ensure the quality of dataset construction, i.e., ``Authoritative textbooks``, ``Expert-enhanced Rationale Generation`` and ``Majority Vote-based Expert Review``.
* **Evaluation Quality**: Following the widely adopted way of ``LLM-as-a-judge``[1], e.g., OlympicArena[2] and Infimm-Bench[3], we employ GPT-4 as the base referees scoring the rationale among {0,0.5,1}. **What's more**, in this paper, we propose a paradigm for more reliable expert-enhanced auto-evaluation for different types of questions. We carefully introduce the expertise to constrain the marking criteria for different types of questions through elaborate prompts and instruct GPT-4 to score the rationale. This could effectively **align** the auto-evaluation with human experts.

Thank you again for all the reviewers' suggestions that have greatly improved our paper, and we are still looking forward to receive the feedback before the deadline.
</content>
</thread_object>

</thread>


Metareview:
metareview:
This paper has several notable shortcomings. The dataset is relatively small, with only 1,018 questions, and narrowly focused on computer science and artificial intelligence, limiting its generalizability to other domains. The evaluation methodology heavily relies on GPT-4 for scoring, which introduces potential biases and lacks robustness in capturing diverse reasoning paths. Furthermore, the claimed novelty of the metrics (Q→A and Q→AR) is overstated, as similar approaches have been explored in prior works. Finally, the paper fails to provide meaningful comparisons with existing benchmarks, such as scalability issues and dataset quality assurance.