Give me a hint: Can LLMs take a hint to solve math problems?

Authors: vansh  agrawal
pratham  singla
amitoj  singh miglani
shivank  garg
ayush  mangal
Venue: NeurIPS_cc_2024_Workshop_MATH-AI
Published: 2024-09-19 19:49:31

Abstract: While state-of-the-art LLMs have shown poor logical and basic mathematical reasoning, recent works try to improve their problem-solving abilities using prompting techniques. We propose giving "hints" to improve the language model's performance on advanced mathematical problems, taking inspiration from how humans approach math pedagogically. We also test robustness to adversarial hints and demonstrate their sensitivity to them. We demonstrate the effectiveness of our approach by evaluating various diverse LLMs, presenting them with a broad set of problems of different difficulties and topics from the MATH dataset and comparing against techniques such as one-shot, few-shot, and chain of thought prompting. Our code is available at https://github.com/vlgiitr/LLM-Math

Main paper: 
 introduction: 
 The ability to reason and logically solve complex mathematical problems is essential for progress in nearly every field, whether it be modeling complex environments, developing new algorithms, or engineering new devices. Recent works have explored Large Language models' mathematical capabilities and found them lacking in logic and basic mathematical reasoning [1] [2] . Improving these reasoning capabilities is important in a future where LLMs can be used to assist humans in increasingly complex mathematical tasks or act like AI math teachers. In this work, taking inspiration from how humans are taught math pedagogically, we propose "hinting" LLMs by giving subtle guidance or clues as a method to improve problem-solving capabilities on mathematical tasks and then compare its effectiveness against other prompting methods, such as one-shot [3] , few-shot [3] , and chain of thought prompting [4] . We evaluate our approach using the MATH dataset [5] , consisting of math problems categorized into distinct classes based on subtopics such as algebra, probability, geometry, etc., with different levels of complexity (1) (2) (3) (4) (5) . We use a diverse set of LLMs, including base language models, instruction fine-tuned models, models specifically tuned for mathematical tasks, and closed source models such as GPT-4o-mini [6] and Gemini Flash [7] for our evaluation.We further examine the robustness of these models to adversarial prompts, misleading hints, and clues of varying levels. We investigate how sensitive the models are to incorporating these incorrect hints as context, which may degrade performance, versus their ability to reject such misleading information. Through these experiments, we seek to contribute to the ongoing research on improving the current state-of-the-art language model's reasoning capabilities and their practical applications in solving mathematical tasks [8] .Various prompting methods have been shown to increase the accuracy of LLMs in solving complex problems that require understanding and reasoning [9] . A few popular methods being -1. One shot prompting [3] : Giving a single example problem and its final answer to the model in-context to learn from.2. Few shot prompting [3] : Providing multiple in-context example problems and their final answers instead of one.3. Chain of thought prompting (CoT) [4] : Providing a detailed step-by-step solution to the in-context example to provide intermediate reasoning steps to solve a problem.However, these methods have not been extensively explored in the context of solving more complicated mathematical problems. Further, their generalization capabilities, to apply learned knowledge to a broader domain of questions (e.g., algebraic equations, geometry problems) rather than specific problems (e.g., solving a specific algebraic equation, finding roots for a quadratic polynomial), have not been sufficiently researched [10] . Previous math benchmarks [10] [11] show that math is a valuable ability to test an LLM's reasoning and problem-solving capabilities. While there has been work on hinting [12] [13] as a prompting technique, they have not been robust and diverse enough to see if these techniques are generalizable to various types of models and what the effect and sensitivity of these models to adversarial hinting can be. 
method: 
 We first prompt the Gemini 1.5 flash model to generate hints for our dataset by passing it the question and final answer. These hints are then provided in context to the model and the target problem to help the model reason about the task. We believe this aligns more with how humans solve math problems by getting hints instead of the complete solution for an example problem, as in Chain of Thought, or just the final answer, as in one/few-shot approaches.To test the adversarial robustness of these models to hints, we provided either an adversarial misleading hint or a hint from a random question to observe how sensitive the models are to our hints. Similar versions for one/few shot and CoT prompting are also generated as shown in Figure 1 . More details about the prompting process can be found in Appendix A.We evaluate a diverse ensemble of LLMs ranging from base models to instruct fine-tuned models and math-finetuned LLMs, including nine open-source models and two closed-source models (More details in Appendix C). We use the MATH dataset [5] , which contains problems of seven different classes (e.g., algebra, geometry, etc.) of varying difficulty levels from 1 (easy) to 5 (hard), more details about the difficulty levels are given in Appendix B.3 Experiments and Results 
setup: 
 We evaluate a set of 11 models for our experiments, using open-source implementations where possible and public API offerings for closed-source models. We evaluate 17 different types of prompting techniques, ranging from baseline to hinting to adversarial, along with one/few shot and CoT as given in Appendix A. The problem set consisted of all seven topics from the MATH dataset [5] , with 100 problems for each topic. Exact details about the data split are given in Appendix B. We report the comparison by checking the fraction of questions that the model got correct.Model OutputThe answer is 5Model OutputThe answer is 4A: ?Model OutputThe answer is 7 Q1: ... A1: ... Q2: ... A2: ... ...Model OutputThe answer is 3Model OutputThe answer is 4 Q: When you divide a binary number ... Step 1 :... We observe that hinting improves the performance of models, as shown in Figure 2 . We attribute the poorer performance of other approaches to a lack of generalization. The improvement of hinting over chain of thought, can be explained as the latter restricts the model's generalization by forcing it to follow the entire reasoning steps of a solution, which might not be the same for all the problems. On the other hand, hints only provide certain helpful directions, giving the model more freedom to generalize better to different problems.Approaches like the chain of thought [4] give step-by-step solutions, which might not generalize to other problems restricting the search space of these models. This can also be due to a "snowball" effect [5] , in which intermediate-generated steps with mistakes can derail the model from the logical direction to the right answer. One-shot and few-shot perform relatively better as they do not restrict the steps, however, the model still gets confined to a narrower subdomain, with few-shot showing more generalisation due to more examples and hence, better performance. Hinting leads to better performance as it helps the model reason about question-specific knowledge more than other prompting techniques and gives initial steps to ensure the right direction without over-fitting to an exact solution.We find that giving Adversarial hints drastically reduces the model's performance, dropping it below CoT, which performed the worst in our non-adversarial approaches, as shown in Fig 2  and Table 1 . We also apply this to few-shot [3] and one-shot [3] settings and find that adversarial hinting affects the performance in those cases as well. The inferior performance of CoT and its prompting variants can be attributed to over-fitting various factors within irrelevant information that influence the model's sensitivity to distractions based on lexical similarity. [14] . Model OutputThe answer is 448Model OutputThe answer is 56Model Input Model Input Q:Find r if 3(r -7) = 4(2 -2r) +4Step 1: ... We find that hinting performs better for all models except the base models, which in general are worse at instruction following. We also notice a drastic drop in performance for all model types when given adversarial hintsWhile comparing the performance of different models, we observe that closed-source models exhibit the highest performance, followed by models fine-tuned for mathematical tasks, instruct-tuned models, and finally, base models, as shown in table 2 . Additionally, variations in model size impact performance, with smaller models generally performing worse. However, performance also depends on the extent of fine-tuning; for example, the Qwen-2-Math-Instruct model [15] achieves comparable results to GPT-4o-mini [6] as shown in Appendix E. Among the 7B and 8B models, Qwen-2-Math-Instruct [15] performs best, while Mistral-Instruct [16] ranks the lowest. Our observations further reveal that base models struggle to incorporate and utilize hints effectively, performing worse than the baseline, likely due to their limited ability to follow instructions [17] . We list our exhaustive results in Appendix D.Further, we also evaluated hinting multimodal models on visual and mathematical tasks and found a similar improvement due to hinting and deterioration due to adversarial hinting. We provide more details in Appendix C.1. 
conclusion: 
 In this work, we have evaluated the mathematical reasoning abilities of various models and approaches.Our results indicate that providing hints is more effective than giving direct answers because it guides the model to the correct solution instead of restricting its search space like few-shot [3] , one-shot [3] , and chain of thought [4] which lack generalization. However, CoT may outperform hinting in some cases where the example and target problem are very similar. Hence, for math problems with unknown solutions, it is better to provide the user's knowledge about the problem as hints to improve its reasoning and problem-solving capabilities than a full solution for a similar problem, and hinting is the natural way in which humans solve a reasoning task as well, as only intermediate directions to approach a problem are required, the rest can be reasoned by humans themselves. Although extracting hints requires and extra inference, it is compensated by the increase in performance over other prompting techniques, which are only useful when dealing with similar problems. Finally, we see the sensitivity to random and adversarial prompting [18] techniques demonstrated performance loss due to adversarial/random hints in few-shot [3] , one-shot [3] , and chain-of-thought [4] settings. 
limitations and future work: 
 Our work mainly focuses on prompting LLMs with problems as textual input. Testing these techniques in multi-modal models like VLMs, where geometrical problems can be passed as image inputs, is a possible future direction. Further, due to computational limitations, our experiments involved a small subset of problems to test the models. These techniques can also be evaluated on larger sample sizes to increase generalizability. Additionally, It is yet to be tested whether hinting can increase performance in other general tasks as well. 
 
 Appendix: We compare 17 different prompting techniques, including base-prompting techniques such as fewshot, one-shot, and chain-of-thought, and their variants involving hinting and adversarial prompting techniques for rigorous and exhaustive evaluation. We group the approaches as follows:A.1 Base Prompting techniquesIn our baseline approach, we provide the models with the target questions and then prompt them to solve them. For One-Shot [3] and Few-shot prompting [3] , we provide the model with example problems from the same class as the target problem. Since the dataset is divided into difficulty levels, we provide one Level-3 problem for one-shot and 5 examples, one from each level for few-shot prompting. We only provide the final numerical answer without any explanation for these approaches.More details about the difficulty levels are given in B. For Chain-of-Thought prompting [4] , the model is given the complete step-by-step solution of the example problem in context instead of the final numerical answer only as shown in (Figure 4 )Our final baseline prompting techniques involve:• Baseline: Only target question given • One-shot: (Table 6 ) One example question and final answer pair (Difficulty level: 3) and then the target question given • Few-shot: (Table 5 ) Giving five example questions and final answer pairs, each corresponding to one of the 5 difficulty levels, and then the target question • Chain Of Thought: (Table 6 ) Giving one example QnA pair with step-by-step-solution (Difficulty level: 3) and then the target question A: ?? For our baseline approach, we only provide hints for the target problem. In the one-shot [3] and few-shot hinting cases [3] , example problems of the same category and its hint (without the final answer) are provided to the model. (Figure 5 )• Baseline + Hinting: Providing each question with its corresponding correct hint • One-shot-Hinting: Providing one example Question and Hint pair (Difficulty level: 3) and then the target question • Few-shot-Hinting: Providing five example Question and Hint pairs, each corresponding to one of the 5 difficulty levels, and then the target questionTo test the robustness of our approach, for one and few-shot, we adversarially prompt the model with an example problem and the incorrect final answer to test how much the models make use of the problem-answer pairs provided. We further extend the idea of adversarial prompting to chain of Hint: Express the sums of the geometric series using their formulas ... We experiment with two different types of adversarial prompting:• Case-I -Random : The model is provided with an example problem of the same category as the target problem, but the hint being provided for the example problem is of a different category.• Case-II -Adversarial: We deliberately make errors in the correct hints of the example problems and feed them to the model, such as changing the signs, reordering the steps, etc.We experiment with both random and adversarial prompting for our baseline, one-shot, and few-shot approaches.We further extend the concept of adversarial prompts to hinting and make adversarial hints to test the sensitivity of models to these hints. We add the adversarial hints case for one-shot and few-shot prompting cases similarly by replacing the right hints with the adversarial hints.• Baseline + Random-hint: Giving a question with a random hint (hint of some unrelated question)• Baseline + Adversarial-hint: Giving each question with its adversarial hint (wrong hint of the same question)• One-shot-Adversarial: Giving one-example QnA pair but with the adversarial answer (Difficulty level: 3) and then the target question• One-shot-Random-Hinting: Giving one example question and random hint (Case-I) pair (Difficulty level: 3) and then the target question• One-shot-Adversarial-Hinting: Giving one example question and adversarial hint (Case-II) pair (Difficulty level: 3) and then the target question• Few-shot-Adversarial: Giving five example QnA pairs with adversarial answers, each corresponding to one of the 5 difficulty levels, and then the target question• Few-shot-Random-Hinting: Giving five example questions and random hints (Case-I) pair, each corresponding to one of the 5 difficulty levels, and then the target question• Few-shot-Adversarial-Hinting: Giving five example questions and adversarial hints (Case-II) pair, each corresponding to one of the 5 difficulty levels, and then the target question• Chain-Of-Thought-Adversarial: Giving one example QnA pair with the adversarial (Case-I) step-by-step-solution (Difficulty level: 3) and then the target question• Chain-Of-Thought-Random: Giving one example QnA pair with random (Case-II) stepby-step-solution (Difficulty level: 3) and then the target question Hint: Equate the volumes of the two cylinders using the formula for the volume of a cylinder ...A: ??A: ??Q: A sphere is inscribed inside a hemisphere ...A: ??Q: Find the distance between the points (-5, -2) and (7, 3) .Hint: Use the distance formula, which calculates the cube root of the difference of the squared differences in the xcoordinates and y-coordinates We evaluate on a subset of the MATH dataset [5] . The MATH dataset consists of problems from various popular math competitions, including the AMC 12, AIME, and more. Owing to the structure of mathematics, these problems have a particular method of solving them with multiple related steps.Humans generally use problem-solving techniques and "heuristics" to solve such problems, thus making them a good metric for assessing a model's problem-solving and reasoning skills [5] . The dataset categorizes the problems into various categories and difficulties. The seven categories are Pre-algebra, Algebra, Number Theory, Counting and Probability, Geometry, Intermediate Algebra, and Pre-calculus. The problems are divided into five levels: level 1, including more straightforward questions; and level 5, including advanced questions.We enhanced the dataset with hints and adversarial hints for our experiments. To generate hints for each problem in the dataset, we used the Gemini-1.5-flash model, prompting it to generate hints and adversarial hints related to the question. Due to computational limitations, we used a subset of the dataset, 100 problems from each topic (20 questions of each difficulty), leading to a total sample set of 700 problems.All the prompting techniques were tested on a set of models, chosen in a manner to ensure diversity for our experimentation and testing. Nine open-sourced models and two closed-source have been used models as listed below:• Gemma-2-2b-it 1 • Qwen2-Math-7B-Instruct 2• Meta-Llama-3.1-8B-Instruct 3• Meta-Llama-3.1-8B 4• Mistral-7B-Instruct-v0.3 5• Qwen2-7B-Instruct 6• Qwen2-7B 7• Mathstral-7B-v0.1 8• Deepseek-math-7b-instruct 9 • Gemini• GPT-4o miniThe ground truth numerical answers and the generated final numerical answers were compared using DeepSeek-7B-Math-Compare-Answer [25] 10 , fine-tuned to compare mathematical answers with high accuracy and extract answers from the generated step-by-step solution by the model.We evaluated the mathematical visual question answering ability of multi-modal models by providing them with the text for a question and an image containing the diagram. We tested the visual problem-solving with four prompting techniques: baseline, hinting, adversarial hinting, and random hinting. We evaluated these approaches on the multimodal Gemini and GPT-4o mini models. For our multimodal experiments, we created another dataset using a subset of 700 non-MCQ questions from the Math Vision Dataset [26] , which contains problems of various levels from 1-5, in increasing difficulty. We sampled equally from each difficulty level. Hints were generated using the Gemini-1.5-Flash model using the textual content of the question. We also generated their adversarial hints in a similar manner as our other evaluations. For random hinting, we provided the hint from another question. As seen in Table 3 , hinting improves the overall model performance, and the performance is degraded due to adversarial and random hinting, similar to our other evaluations. We note that we could not exhaustively evaluate a broader set of multimodal models on a larger dataset due to computational limitations and see it as an interesting future direction to explore. In this section, all the results from our experiments have been compiled to maintain transparency and clarity. Table 4 contains all the results for all the prompting techniques, individually mentioning the accuracy of each question type as well as model-wise performance. Figure 8 shows the average of all the models on a given technique, giving a better comparison of each type's performance.h i n t f (x) = ax + 3, if x > 2, x -5 if -2 ≤ x ≤ 2, 2x -b if x < -2.Find a + b if the piecewise function is continuous (which means that its graph can be drawn without lifting your pencil from the paper)." Answer: 0 .There exist constants p, q, and r such thatA 3 + pA 2 + qA + rI = 0,where I and 0 are the 3 × 3 identity matrix and zero matrix, respectively. Enter the ordered triple (p, q, r)." Answer: (0,-6,-4) Example Problem: "What is the coefficient of x 8 in the expansion of (x -1) 9 ?" Solution: * Step 1: Understanding the Problem: Find the coefficient of a specific term in the expansion of (x -1) 9 using the Binomial Theorem. https://huggingface.co/google/gemma-2-2b-it8 = 6435." Answer: 24310 



Reviews: review: title: title: Review
main_review: main_review: Strengths:
* Potential for a more comprehensive analysis of different prompting techniques for mathematical reasoning, with a focus on hinting and effects of adversarial hinting

Weaknesses:
* The idea of hinting is not novel
* Limited evaluations (also see questions below)

Questions:
* What is the effect of various techniques in prompt engineering? The examples in figure 6 seem to suggest that little effort was put into prompt engineering.
* What are the results with zero-shot CoT? They usually improve over the baseline.
* How were the in-context learning examples chosen? What do the results look like with smarter retrieval for in-context learning examples?

Suggestions:
* Appendix D, with the full evaluation results, should be referred to in the main text.
title: title: A Promising Approach to Enhancing LLM Mathematical Reasoning
main_review: main_review: This paper investigates the use of "hinting" to enhance LLMs' performance on mathematical tasks, comparing it with other prompting strategies and assessing robustness to adversarial hints. Using a variety of models and problems from the MATH dataset, the study shows that hinting generally leads to better performance than baseline, one-shot, few-shot, and chain-of-thought prompting. While providing valuable insights into LLM math capabilities and prompting techniques, the work has some limitations in depth and scope.

The paper follows a clear structure but could be improved in several areas:
1. The abstract should be more concise and highlight the key findings.
2. The methodology section is dense and could be organized better with subheadings.
3. Figure captions need to be more descriptive for better data interpretation.
4. A more thorough literature review would help contextualize the research.

Methodologically, the paper is strong, though certain aspects could be refined:
1. The experimental setup is comprehensive, covering multiple models and problem types.
2. The comparison between hinting and other prompting techniques is well-executed.
3. The use of the MATH dataset provides a standardized benchmark.
4. Adversarial robustness testing is a valuable addition.
5. However, the statistical analysis could be improved with clearer reporting of significance and effect sizes.

Strengths of the paper :
1. Innovative approach to LLM math performance through hinting.
2. Extensive evaluation across models and problem types.
3. Inclusion of adversarial testing for robustness.
4. Clear visualizations of results.
5. Practical applications for AI in math problem-solving.

Weaknesses of the paper:
1. Limited sample size may reduce generalizability.
2. Lacks in-depth analysis of why hinting outperforms other techniques.
3. No comparison to human performance or advanced math-solving systems.
4. Insufficient exploration of optimal hint generation processes.
5. Potential biases or shortcuts introduced by hinting are not fully addressed.

Comments and Questions : 
1. How does hint quality affect results? A deeper analysis would be helpful.
2. What are the implications for AI-driven math education tools or tutoring systems?
3. Could hinting be combined with chain-of-thought reasoning for even better performance?
4. A more detailed error analysis would help understand persistent model failures.
5. How does performance vary across different difficulty levels within each math category?
6. What are the computational costs of hinting compared to other methods, especially for larger models?
7. Can this approach generalize beyond mathematics?

This paper offers contributions to understanding how LLMs handle mathematical reasoning. The hinting technique shows promise, although the study has some limitations. Addressing these gaps and further exploring the practical implications would strengthen future work.
title: title: Prompting method seems effective but lacks novelty
main_review: main_review: This work demonstrates that providing hints before generating solutions can improve model performance. The method proves effective across multiple models when compared to baseline approaches like Chain of Thought (COT) and few-shot learning. While the experiments are thorough and the results seem promising, the approach of introducing hints is neither novel nor particularly impressive, and can be regarded as a special case of COT. Therefore, I would rate this work a 5.
title: title: The authors propose giving hints to improve mathematical problem-solving in large language models (LLMs). Hinting is compared to existing prompting methods on a range of math problems from the MATH dataset, with experiments on various open-source and closed-source LLMs. The results show hinting generally outperforms baselines like chain-of-thought prompting.
main_review: main_review: The paper proposes giving "hints" to large language models (LLMs) as a technique to improve their mathematical problem-solving performance. The hints are intended to guide the models to the correct solution in a manner analogous to how hints are used pedagogically with humans. The authors compare the hinting approach to other prompting techniques like one-shot, few-shot, and chain-of-thought prompting on a diverse set of math problems from the MATH dataset. They also test the models' robustness to misleading adversarial hints. The experiments are conducted on a range of open-source and closed-source LLMs.


Pros:
- The paper addresses an important and timely topic of improving the mathematical reasoning capabilities of LLMs, which is critical for many potential applications.
- The hinting technique proposed is intuitive and well-motivated by drawing an analogy to human math pedagogy. Providing hints to guide the model's reasoning process is a natural extension of existing prompting techniques.
- The experimental evaluation is thorough, covering a diverse set of math problems of varying types and difficulties from the MATH dataset. This helps establish the generality of the findings.
- Comparing hinting to strong baseline techniques like one-shot, few-shot, and chain-of-thought prompting helps contextualize the results and demonstrate its advantages.
- Testing robustness to adversarial hints is a valuable addition that explores the hinting approach's limitations and failure modes.


Cons:
- The hint generation process could be described in more detail. How exactly were the hints extracted from the Gemini model? Were they manually curated or filtered in any way?
- The authors mention that hinting outperforms CoT prompting and attribute this to CoT overfitting to the example solution and getting derailed by mistakes. However, this could be discussed and analyzed further. Are there some problems where CoT still performs better than hinting?
- The adversarial hinting experiments are useful but could be expanded. How exactly were the adversarial hints generated? Are they mathematically invalid or just suboptimal? Varying the types of adversarial hints could yield further insights.
title: title: A good paper but have some questions
main_review: main_review: ### Summary

The paper explores the use of "hints" as a new prompting technique to improve the problem-solving capabilities of LLMs mathematical tasks. The study is inspired by pedagogical approaches in human math learning, where hints are used to guide learners towards a solution without giving away the full answer. The authors compare this hinting method against other prompting methods such as one-shot, few-shot, and chain of thought (CoT) prompting using various LLMs across the MATH dataset. 

### Strengths

1. Introducing hinting as a way to guide models through the reasoning process seems like a new approach. It mimics human pedagogy, where learners are given clues to nudge them in the right direction. I am slightly curious to know how this hinting method fundamentally changes the way in which the LLMs give outputs. Some insights from the authors would be good to have.
2. The paper evaluates multiple LLMs, including fine-tuned, instruction-tuned, and base models. This diversity helps demonstrate the general applicability of the hinting technique across different types of models, and provides a well-rounded comparison of model performances.
3. Testing the robustness of models against adversarial hints is a good addition. It provides insights into how sensitive LLMs are to misleading information.

### Weaknesses

1. **Limited Dataset Scope**: The experiments are conducted on a subset of the MATH dataset. While the results are promising, testing on a more extensive range of datasets like GSM-8k or broader math domains would provide a more thorough validation of the hinting technique.
2. **Adversarial and Random Hinting Unexplored in Depth**: While the paper mentions the drop in performance due to adversarial and random hinting, it lacks a deep exploration of the failure modes and reasons why certain models are more sensitive to these hints. A more granular error analysis could strengthen the paper’s conclusions.

### Questions

1. How would the hinting technique perform on larger-scale models, such as GPT-4 or Llama 3.1, which might already exhibit stronger mathematical reasoning capabilities?
2. Could the hinting method be extended to non-mathematical domains (e.g., language understanding or scientific reasoning) where guidance is also essential for problem-solving?
3. What specific types of problems in the MATH dataset benefited most from hinting versus other techniques like CoT or few-shot prompting? A deeper dive into problem categories might help identify where hinting is most advantageous.
4. Would the performance drop due to adversarial hints be more pronounced in models fine-tuned for specific domains? Can future work offer solutions to mitigate these adversarial effects?