Have LLM s Advanced Enough? A Challenging Problem Solving Benchmark For Large Language Models (2023)

Abstract: The performance of large language models (LLMs) on existing reasoning benchmarks has significantly improved over the past years. In response, we present JEEBench, a considerably more challenging benchmark dataset for evaluating the problem solving abilities of LLMs. We curate 515 challenging pre-engineering mathematics, physics and chemistry problems from the highly competitive IIT JEE-Advanced exam. Long-horizon reasoning on top of deep in-domain knowledge is essential for solving problems in this benchmark. Our evaluation on various open-source and proprietary models reveals that the highest performance, even after using techniques like self-consistency, self-refinement and chain-of-thought prompting, is less than 40%. The typical failure modes of GPT-4, the best model, are errors in algebraic manipulation, difficulty in grounding abstract concepts into mathematical equations accurately and failure in retrieving relevant domain-specific concepts. We also observe that by mere prompting, GPT-4 is unable to assess risk introduced by negative marking for incorrect answers. For this, we develop a post-hoc confidence-thresholding method over self-consistency, which enables effective response selection. We hope that our challenging benchmark will guide future re-search in problem-solving using LLMs.

## introduction the capabilities of large language models (llms) have been improving since the last decade on a plethora of tasks including reasoning. most recently, gpt-4 demonstrates significant improvements over gpt-3 on tasks such as code-generation, arithmetic and commonsense reasoning @xcite , exhibiting impressive performance on standard reasoning and stem benchmarks such as gsm-8k @xcite , math (hendrycks * equal contribution, † work done while at iit delhi figure 1 : an example problem from jeebench et al., 2021b), mmlu @xcite and scienceqa @xcite rising capabilities of llms call for harder benchmarks. we introduce jeebench, a benchmark consisting of 515 problems that require complex logical and mathematical reasoning on top of deep in-domain knowledge of pre-engineering level physics, chemistry and mathematics. problems have been curated from the past 8 editions of the joint entrance examination (jee)-advanced exam, held annually in india as an entrance test to india's premier engineering institutes: the iits. the exam is designed to be time-consuming, diffi-cult, and has a low selection rate (approx. 5%). the problems in the dataset require a complex interplay of employing multiple high-level domain specific concepts, grounding them into mathematical equations/constraints, followed by algebraic manipulation and arithmetic operations. figure 1 is a problem from the dataset along with an expert's solution. in this problem, the ideal solution involves the retrieval of the appropriate concepts: the rules of static equilibrium, grounding the concepts into mathematical equations for the specific problem instance, followed by solving the equations in order to find the final answer. other instances of domain-specific concepts can be balancing of redox reactions (chemistry), current into a junction equals current out of the junction (physics) and integration by parts (mathematics). more such examples can be found in the appendix a.2. we conduct a qualitative and quantitative study of contemporary open-source and proprietary llms on these problems and also highlight avenues for further research. our analysis indicates that gpt-4 is unparalleled in performance compared to other models. it demonstrates long horizon reasoning and the ability to manipulate complex algebraic equations in quite a few problems. we observe that chain-of-thought prompting @xcite and self-consistency @xcite , which are recent proposals to improve llm performance, are indeed effective on our dataset. we also explore self-critique @xcite , where an llm (verifier) is instructed to improve the outputs of the same llm @xcite . we find that this approach is not helpful on jeebench. the verifier is weak in spotting conceptual errors, and like the generator, is itself prone to hallucinations. it would be interesting to explore the class of problems where this approach of self-refinement is (not) helpful. we further conduct a critical analysis of the limits of gpt-4's reasoning abilities, and highlight major areas that require considerable improvement. a detailed error analysis suggests that it frequently struggles in retrieving relevant concepts required to solve problems, and performing algebraic manipulation & arithmetic. inability to perform even simple algebra highlights an important question: can we build llms faithful to mathematical logic? another important question is how to estimate gpt-4's performance in comparison to humans. the jee advanced exam comes with the bane of negative marking for incorrectly answered questions. this makes the exam even more challenging, because in addition to advanced problem solving skills, it requires an accurate risk assessment and computing a good policy based on it. our experiments demonstrate that when prompted with the marking scheme, gpt-4's performance actually drops. to mitigate this, we employ a simple method -thresholding over self consistency. self consistency generates multiple responses for each question. relative frequency in the set of responses can be considered as a proxy for confidence score of each option. threshold on the confidence score can be tuned using a validation set. we find that gpt-4's score, after augmenting it this way, lies in the top 10-20 percentile of human scores in the 2023 edition of the exam. overall, we hope that this benchmark serves as a strong and reliable test-bed and fosters future research on problem solving with llms. our code and dataset are available at https://github.com/ dair-iitd/jeebench . ## related work reasoning has been studied under various contexts such as logical reasoning, commonsense reasoning, mathematical reasoning, and theorem proving. we summarize some key works in two sub-areas, most closely related to our work: mathematical reasoning and science qa. mathematical problem solving: gsm8k @xcite , dolphin18k @xcite , aqua-rat @xcite , math @xcite and ape210k @xcite are datasets that contain mathematical reasoning questions. dolphin18k, gsm8k, and aqua-rat consist of elementary problems, requiring only basic arithmetic and problem comprehension. thus, there is a general lack of variety in the underlying reasoning steps across problems. in terms of difficulty, math, containing problems from amc, aime and olympiads, comes close to jeebench in terms of complexity. however, compared to math, the mathematics questions in our dataset span many additional topics such as differential and integral calculus, differential equations, 3d geometry, and conic sections. also, the problems in jeebench are harder, as we discuss later in the paper. minif2f @xcite consists of mathematics problems from math dataset and other sources in a formal language. in contrast, problems in our dataset are in natural language. general science: in the context of physics and chemistry, scienceqa @xcite , sciq @xcite and mmlu @xcite are prominent available datasets. sci-enceqa and sciq, built from elementary and high school science curricula, mainly test factual knowledge of the subject. the skills required to solve such problems are primarily information extraction, reading comprehension and commonsense reasoning. in contrast, questions in our dataset require long-horizon reasoning and grounding of complex scientific concepts into equations and arithmetic. concurrently, c-eval @xcite and scibench @xcite are datasets along similar lines. c-eval consists of a variety of disciplines such as engineering, medicine and humanities has been created. scibench creates a dataset from college-level mathematics, physics and chemistry questions. problems present in jeebench are significantly harder than those in other contemporary datasets. to verify this, we sample 50 questions each from jeebench and the test sets of math and the high-school physics, chemistry and mathematics sections from mmlu and conduct zeroshot evaluations on gpt-4. the results can be seen in figure 2 . as we can see, gpt-4 can easily solve more than 80% problems from mmlu. the math dataset is harder, where the performance is approximately 60%. however, gpt-4 struggles in jeebench-math, solving close to a mere 20% problems. ## the jeebench dataset the dataset consists of 515 problems extracted from the past 8 editions of the jee-advanced from the year 2016 to 2023. the problems are harvested from publicly available sources. 1 the exam consists of 2 papers held every year, each containing 50-60 questions equally distributed among physics, chemistry, and mathematics. we use online tools to extract problems from pdf-format exam papers into l a t e x. we remove all problems containing diagrams in their description (approximately 40%). manual quality checks are performed to fix/eliminate possible errors in preprocessing. figure 3 shows representative problems from the final dataset. the problems are categorised by subject: physics, chemistry and mathematics, and the format of expected response: multiple choice questions (mcq) with single option correct, mcqs with multiple options correct, integertype and numeric-type. in integer-type questions, the answer is an unbounded non-negative integer, whereas for numeric-type, the answer is a floating point number upto 2 digits after the decimal point. the breakdown of the problems based on answer-type and subject is shown in ## experimental setup and results we wish to investigate the following research problems: ## discussion the general performance trend demonstrates the efficacy of high-quality data, instruction fine-tuning, rlhf and parameter scaling in improving the reasoning capabilities of llms. for many problems, gpt-4 is able to give a sketch of the correct, humanlike solution that is impressive given the extent of reasoning involved in the problems. however, our analysis also reveals major areas where progress is needed. although gpt-4 performs flawless logical and mathematical reasoning in some instances, sometimes it commits grave errors in trivial steps. errors in retrieval and application of concepts suggest an interesting research question: can we augment an llm such that it's generation is constrained by faithfulness to a set of facts? such a system will demonstrate robustness in reasoning, critical for long-horizon tasks. physics problems in the benchmark often require an understanding of spatial reasoning. we found that while gpt-4's spatial reasoning is far from perfect. appendix a.7 provides an example where gpt-4 commits errors which might be attributed to its inability to reason spatially. with the release of the multi-modal version of gpt-4, evaluating this aspect of physics problems might be easier. finally, an llm that understands its own confidence in an answer is a key missing piece, as highlighted by our experiments in the exam setting. our simple post-hoc wrapper does slightly improve performance in this regard. ## conclusion we present a challenging problem solving benchmark to evaluate large language models. we perform a detailed analysis of the performance of various llms on the benchmark, and identify areas of improvement in the best current llms. our work raises interesting research directions such as mathematical logic-augmented gpt, multi-modal evaluations on gpt-4 and the decision-making capabilites of gpt in an exam setting. we hope that jeebench guides future research in reasoning using llms. our code and dataset are available at https://github.com/dair-iitd/jeebench .