Hint-before-Solving: A Framework to Effectively Utilizing Inherent Knowledge of Large Language Model
Authors: Jinlan Fu, Shenzhen Huangfu, Hang Yan, See-Kiong Ng, Xipeng Qiu


Reviewer paper summaries: This paper proposes a new way of prompting and fine-tuning the LLMs to improve the mathematic and logic reasoning abilities. In particular, the model is guided to generate "hints" before generating (the steps of) the solution to the problem. The paper compares different ways of adding the hints (generated by GPT-4), including fine-tuning different pre-trained models. The constructed dataset and the code are also released.
Summary of strengths: * Proposed the approach improves the performance on mathematic and logic reasoning tasks, including some complex problems
* The experiments are quite extensive and the analyses are detailed
* Release the code and the dataset
Summary of weaknesses: * The performance improvement isn't so large (in some cases, it decreases)
* The hints (and the dataset) are generated by GPT-4
Comments, suggestions and typos: I think one baseline could be missing for a fair comparison, which is to ask GPT-4 to generate CoT data. Otherwise, the current improvement could be due to the mixture of two factors: 1) the hint format and HinSo framework; and 2) GPT-4's input. Maybe the authors could clarify this.
Overall assessment scores: 3