Paper: SciBench: Evaluating College-Level Scientific Problem-Solving Abilities of Large Language Models (ICLR-2024)

Reviewer comment (Experiments):
The experiments report a combination of external tools and few-shot learning. On the other hand, Table 4 shows that the accuracy of few-shot learning is sometimes lower than that of zero-shot learning. The reviewer would like to know why the authors did not try the combination of external tools and zero-shot learning.

Author response:
Thank you for bringing this question up. We had concerns about the ability of LLM to adapt programming languages, especially Wolfram Language, in the zero-shot setting. However, we conducted the experiment on the zero-shot python setting using gpt-3.5-turbo and show the result below. 
| atkins | cal   | chemmc | class | diff  | fund  | matter | quan  | stat  | thermo | avg |
|--------|-------|--------|-------|-------|-------|--------|-------|-------|--------|---------|
 | 11.21  | 19.04 | 30.77  | 0.0   | 14.00 | 9.59  | 2.04   | 11.76 | 42.67 | 4.48   | 14.75   |

The result is lower than the 19.91% achieved in the few-shot setting. We will further conduct experiments with a combination of zero-shot setting and external tools.

Reviewer comment (Evaluation):
In the experiment, if the LLM responses were within a relative error of 0.05 as a numerical value, they were considered to be correct. On the other hand, as shown in the center of Figure 1, there are cases where the numerical calculation is wrong even though the idea or formula itself is correct, and cases where the numerical calculation is correct by chance but the idea or formula is false. In this evaluation setup, the former case is simply considered wrong, while the latter case is correct. It is questionable whether this is a suitable way to evaluate scientific problem-solving ability.

Author response:
Problems with wrong numerical calculation are considered as wrong since calculation ability is considered one of the problem-solving ability. Models should have the calculation ability to answer problems correctly.  Considering the overall difficulty of our tested problems, it has a very low probability that the model derives the answer correctly but with wrong formulas. 
Unlike multiple-choice questions, where random guessing has a high chance of yielding the correct answer, our evaluation method is more stringent as it involves a wide range of rational numbers, making it highly improbable to guess the correct answer randomly. Our confidence in the evaluation metric is based on the belief that if the model arrives at the correct answer, it is very likely due to a correct solution process, given the complexity involved in reaching these answers. Our answers, which can be as specific as -0.0987 or as large as 1036592783, are difficult to guess accurately. 
Implementing a process verifier could enhance the quality of evaluation. However, employing human annotators for this task could significantly increase project costs. Our evaluation criteria are consistent with most existing benchmarks, like GSM8K[1], where the assessment is based solely on the numerical equivalence of the final answer. We also include further error analysis for inaccurately answered questions in Section 5.

Reviewer comment (Experiments):
Section 3 says "All problems are carefully verified by human annotators to ensure that LaTeX documents can be compiled without any syntax errors." The reviewer wonders if the authors do not check for parsing errors other than syntax errors.

Author response:
A2: Thanks for your questions. Every type of error, including parse errors, is thoroughly checked, and we employ human annotators for final verification in the last phase of the process.

Reviewer comment (Evaluation):
In section 5, it seems that LLM is only applied to numerically incorrect examples for error analysis. What would be the result if the same error analysis was applied to answers that are numerically correct?

Author response:
A3: Thanks for your question. We have manually checked the correct answer of LLM and didn’t detect any wrongly classified examples.

Reviewer comment (Evaluation):
It is possible to find numerically correct answers that contain some errors listed in section 5.

Author response:
A3: Thanks for your question. We have manually checked the correct answer of LLM and didn’t detect any wrongly classified examples.

Reviewer comment (Evaluation):
It would be even better if there was also an evaluation of error analysis for numerically correct responses, while it has been reported that about 20% of error analysis by LLM for numerically incorrect responses are discarded.

Author response:
A3: Thanks for your question. We have manually checked the correct answer of LLM and didn’t detect any wrongly classified examples.

Reviewer comment (Experiments):
The work evaluates five LLMs; however is critically missing GPT-4 with code interpreter (ADA). The results are now superseded by GPT-4 turbo.

Author response:
Thanks for your information. Currently the API of code interpreters for GPT models is not accessible. Since as the reviewer mentioned, OPENAI has spent much effort on adding code support for their model, we conduct a new experiment with their newly released API, GPT4-Turbo, and results are shown below:
|           | atkins | cal  | chemmc | class | diff | fund | matter | quan | stat | thermo | avg |
|-----------|--------|------|--------|-------|------|------|--------|------|------|--------|---------|
| gpt4-turbo| 28.0   | 23.8 | 20.5   | 21.3  | 36.0 | 31.5 | 16.3   | 35.3 | 53.3 | 9.0    | 28.3    |
| gpt4      | 21.1   | 42.9 | 41.0   | 17.0  | 34.0 | 38.4 | 28.6   | 38.2 | 69.3 | 29.9   | 35.8    |

As shown in the table, the newly released GPT4-Turbo performs slightly lower than the original GPT4 version.

Reviewer comment (Experiments):
The work lacks a comparison with state-of-the-art methods such as hypothesis search and refinement and the use of a code interpreter.

Author response:
Thanks for your information. For the use of code interpreters, our experiment results under ‘CoT+Py’ and ‘CoT+Wol’  settings are the results that integrate LLM with external programmatic tools. Regarding some extensive search methods, many of them cannot be easily extended to general scientific problem-solving tasks. Among these methods we choose the more advanced self-consistency[1] method using gpt-3.5-turbo and show the results as below: 
 | atkins | chemmc | quan | matter | fund | class | thermo | diff | stat | calculus | avg   |
|--------|--------|------|--------|------|-------|--------|------|------|----------|-------|
 | 15.6   | 23.8   | 11.4 | 11.8   | 15.8 | 2.0   | 4.5    | 8.0  | 31.2 | 26.0     | 15.6  |

We are exploring the other techniques as well and will further update our manuscript with these findings.

Reviewer comment (Experiments):
The selection criteria of the college textbooks and exams for the new datasets lack detailed explanation and justification.

Author response:
We choose the most representative and widely used textbooks as our dataset. Most of them are assigned course material in college courses. Furthermore, those problem sets are suitable for evaluation purposes since they contain many practice problems and their answers are in numerical value.

Reviewer comment (Presentation):
The abilities and errors may be extended and refined by dividing into sub-categories.

Author response:
Thank you for your information. The current abilities have general coverage for chemistry, physics and math domains. It is hard to further differentiate them. However, we show the lacking ability results for different sub-categories in Appendix D. By checking the solution steps made by LLM, code conversion ability can be further divided into sub-reasons like syntax errors, translation errors. We will do the analysis in the future to further investigate this area.

Reviewer comment (Evaluation):
How many of the questions in the new datasets are available online by search? or in the latest LLMs?

Author response:
A1: We conducted a random search in our dataset and found that none of the problems together with solutions can be directly searched from search engines .

Reviewer comment (Novelty):
the name of the paper, SciBench, is similar to another dataset, called *BIG-bench* (https://arxiv.org/abs/2206.046159), which also overlaps partly with SciBench. Nonetheless BIG-bench is not cited.

Author response:
Thank you for your information. We added BIG-bench as a reference in Section 2. BIG-bench is different from SciBench because it mainly targets specific tasks like predicting element names or deducing  how physical mechanisms work, with some of its tasks as high-school level with multiple-choice questions. On the other hand, SciBench uses harder college-level free-response problems.

Reviewer comment (Novelty):
not quite novel impact of the dataset: Some important reasoning datasets on college-level (and beyond) were omitted from the literature review, e.g., *Mathematical Capabilities of ChatGPT* (https://arxiv.org/abs/2301.13867) and *NaturalProofs* (https://arxiv.org/abs/2104.01112) for math, which both already do some of the things SciBench proposed to achieve (see page 4, "Enabling of assessing advanced problem solving ability" which both mentioned papers achieve; and "Inclusion of college-level problems" and "Inaccessibility in text formats" which the first reference paper achieves.) The *Mathematical Capabilities of ChatGPT* paper also essentially collects error profiles, as the authors do in Figure 3, as does the *NaturalProver* paper (https://arxiv.org/pdf/2205.12910.pdf), not to be confused with *NaturalProofs*.

Author response:
Thank you for mentioning this question. The mentioned datasets focus solely on mathematics.  Our dataset includes problems from different disciplines such as chemistry, physics, and computer science, which require specific in-domain background knowledge for their solution process, rather than purely mathematical questions that rely solely on reasoning ability. As far as our knowledge, there are significantly fewer corpus and evaluation benchmarks about other scientific subjects compared with mathematics.

Reviewer comment (Novelty):
There should be a more detailed, in-text comparison with these datasets mentioned above to highlight similarities and differences of SciBench with these other, pertinent datasets. I also have some doubts (see *Questions* section), if Table 1 is really 100% accurate.

Author response:
Thank you for your question. The key difference of our approach from others lies in the use of a dataset comprising college-level questions that require complex reasoning processes and computational steps such as differentiation. We choose free-response answers over multiple-choice formats. Furthermore, we employ various configurations of Language Learning Models (LLM) like Chain of Thought, or external tools, to evaluate our dataset. Additionally, we utilize an automated error analysis pipeline for categorizing errors.   For Table1, the column “Level”, “computation”, “solution”, “type” denote the Dataset referring to Section 3. Column “zero-shot”, “few-shot”, “CoT”, “tool” denote the experiment referring to Section 4. Column “Human” and “auto” denote the analysis referring to Section 5. Evaluation part represents if other benchmark work has been tested using those configurations. Analysis represents if other benchmark works use manual or automatic analysis for their dataset.  We further include an in-text comparison in Appendix D, Figure S15.

Reviewer comment (Experiments):
695 examples is a rather small dataset.

Author response:
Thanks for bringing up this question. Our dataset goes through human extraction for each problem, which is highly costly. Our goal for the dataset is to serve as an evaluation dataset for problem solving ability. We're currently using the same annotation pipeline to scale up the dataset.

Reviewer comment (Reproducibility):
Section 5: "From 112 such error annotations and with the assistance of GPT-4, we distill these errors into ten essential skills that GPT-3.5 might lack". Using a LLM to make a decision is a source of errors. It would be best if no LLM were used - and if it is used, a very detailed explanation should be given of how exactly it provided assistance.

Author response:
Thank you for your question. Two human annotators participate in the process. Decisions on the final abilities are determined by annotators, aided by assistants. By going through errors, these two annotators develop ten abilities and then employ a Language Learning Model (LLM) as a third evaluator to suggest additional abilities. They then compare and refine their findings based on this input. Ultimately, the final outcomes are determined by the annotators. After LLM annotate the error reasons, we conduct human-check by sampling 151 examples across all settings to make sure the annotations make sense. We make this human-AI cooperated analysis pipeline to reduce the cost of human post-analysis, while incorporate human checking to make sure the correctness of LLM decision and try to reduce the risk that reviewer mentioned. Though not perfect, we believe it can be another type of analyzing framework for future study of LLM problem-solving.

Reviewer comment (Experiments):
The analysis section could be further enhanced. For instance, regarding the observation "The zero-shot learning setting exhibits comparable performance to the few-shot learning setting," did the author conduct comprehensive testing on different in-context examples to ascertain consistent findings?

Author response:
Thanks to the reviewer for bringing the question. For the in-context examples, we randomly selected them from the problems with solutions. We provide all prompt and in-context examples in Github Repo. Additionally, we conducted four experimental runs under the few-shot CoT setting using gpt-3.5-turbo. Each experiment used different randomly selected prompt examples, achieving an average result of 13.17% across all experiments. This 13.17% result, being higher than 11.99%, reflects the continuous updates to OpenAI's API. However, this figure is still comparable to the 12.17% seen in the zero-shot CoT setting, which supports our claim.

Reviewer comment (Experiments):
Different qualities of few-shot examples may yield different results. High-quality prompts and in-context examples should help stimulate the model's capabilities.

Author response:
Thanks to the reviewer for bringing the question. For the in-context examples, we randomly selected them from the problems with solutions. We provide all prompt and in-context examples in Github Repo. Additionally, we conducted four experimental runs under the few-shot CoT setting using gpt-3.5-turbo. Each experiment used different randomly selected prompt examples, achieving an average result of 13.17% across all experiments. This 13.17% result, being higher than 11.99%, reflects the continuous updates to OpenAI's API. However, this figure is still comparable to the 12.17% seen in the zero-shot CoT setting, which supports our claim.

Reviewer comment (Experiments):
Why does utilizing prompts from Wolfram hinder the capabilities of the model? What specific attempts have been made, and could it be due to inadequate adjustments to the prompts? It would be more appropriate to refrain from drawing such a statement without further investigation.

Author response:
Thank you for highlighting this crucial question. As far as we know, Wolfram language is not appearing frequently in pre-training corpus compared to Python. Secondly, the grammar of Wolfram Language is complex and hard for LLM to learn from a few examples. Those two could be the reasons that using Wolfram Language hinders performance. For our project, we provided a detailed description of the Wolfram Language prompt in the Appendix C and Github Repo (https://anonymous.4open.science/r/anonymous-4FFB/dataset/original/wolfram). This prompt was crafted to align with the Python setup. We maintained consistency in the wording for both external tools, simply substituting "Python" with "Wolfram". While there may be potential for further accuracy enhancements, this approach was chosen to maintain fairness in comparing the two tools. To further improve the performance, we need to fine-tune LLM to learn how to use integrated tools, like ToRA[1], and we leave this for future work.

Reviewer comment (Writing):
On page 8, third line from the bottom, is it written incorrectly? Is "15.2% of casual ability" referring to the zero-shot setting?

Author response:
Thank you for bringing this up. This sentence means that the error rates of zero-shot setting are 15.2% for both causality ability and logical decomposition ability. We have rephrased this in the main paper for clarification.

Reviewer comment (Experiments):
Few-shot prompting leads to a trade-off in skills. Is this due to the content of the example prompts? Can it be resolved by replacing few-shot examples?

Author response:
Thanks to the reviewer for bringing the question. As shown in R1, we rigorously conduct multiple rounds of random selection of prompt examples and the result shows that LLM under the few-shot setting performs comparable to the zero-shot setting, which is consistent with our claim.  The reason behind this could be that the number of examples are not large enough to cover all possible abilities needed, and thus providing limited examples distracts the model.