Kati8/ICLR_2023_2025 huggingface dataset:
authors: Longhui Yu
authors: Han Shi
authors: Jincheng YU
authors: Zhengying Liu
authors: Zhenguo Li
authors: Adrian Weller
authors: Weiyang Liu
reviews: 
Comments: The authors aim to bridge the noticeable performance gap of open-access LLMs in solving complex mathematical problems. The paper introduces a framework that includes (i) a diverse dataset of math problems generated through transformations such as forward-backward reasoning and self-verification (MetaMathQA) and (ii) open-access LLMs (llama series) fine-tuned on MetaMathQA. Experiments on benchmark datasets demonstrate clear and impressive gains with MetaMath over other open LLMs. Additionally, the authors conduct insightful analyses, highlighting the role of question diversity in enhancing LLM performance.
Strengths: - **Novel Approach:** The paper introduces a unique data augmentation strategy for mathematical reasoning. The MetaMath framework is generic and can be easily extended to other numerical reasoning datasets.

- **Rich and Comprehensive Analysis:** The analysis is rich and comprehensive, offering numerous insights into data augmentation and the fine-tuning of LLMs for reasoning tasks.
Weaknesses: - **Potential for Benchmark Hacking:** Given the experimental setup, there is a slight risk that the proposed approach could lead to benchmark hacking.

- **Dependence on High-Quality Initial Questions:** Given that both datasets used have extensive training data available, the performance of the proposed method in the absence of high-quality initial questions available for mutation remains uncertain.

To some extent, both the weaknesses can be addressed by doing 0-shot evaluation on some other datasets like DROP (https://allenai.org/data/drop)
Recommendation: 8: accept, good paper
Reviewer confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Soundness/correctness: 4 excellent
Clarity: 4 excellent
reviews: 
Comments: This paper proposes to fine-tune smaller open-source LLMs (LIama) based on data augmentation from large closed-source LLMs (GPT-3.5). A set of data augmentation techniques are employed: answer augmentation, question bootstrapping by rephrasing, and backward reasoning, including self-verification and FOBAR. The data augmentation is applied to the GSM8K and MATH datasets. The augmented MetaMathQA dataset is then used to fine-tune the LIama model series. 

Experiments on the fine-tuned 7B, 13B, and 70B LIama models demonstrate significant improvements over various baselines. The authors also made insightful analyses regarding how the perplexity and diversity of the training data affect performance, the reversal mathematical ability, reasoning paths with incorrect answers, as well as data quantity.
Strengths: 1. The proposed MetaMathQA dataset will be a very valuable contribution to the community.
2. The proposed data augmentation techniques achieve good performances compared to various baselines.
3. The authors made insightful analyses regarding different factors affecting the performance of such small LM fine-tuning. This analysis will not only contribute to the specific topic of mathematical reasoning but also will help the general direction of small LM fine-tuning as well.
Weaknesses: 1. Some baseline approaches to compare are missing, e.g., [1, 2] and code-based LLMs like [3]
2. The ablation study is not comprehensive enough. Only the 7B model is tested. Table 3 is confusing - should add a line breaker between SFT and MetaMath. 

[1] MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuning, Xiang Yue, Xingwei Qu, Ge Zhang, Yao Fu, Wenhao Huang, Huan Sun, Yu Su, Wenhu Chen, 2023

[2] Platypus: Quick, Cheap, and Powerful Refinement of LLMs, Ariel N. Lee, Cole J. Hunter, Nataniel Ruiz, 2023

[3] Code Llama: Open Foundation Models for Code, Rozière et al., 2023
Recommendation: 8: accept, good paper
Reviewer confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Soundness/correctness: 4 excellent
Clarity: 4 excellent
reviews: 
Comments: The authors propose a method for data augmentation to train LLMs for improving mathematical reasoning.
The authors combine several existing techniques such as question re-writing, self-verification, forward-backward reasoning, and answer augmentation to create a larger dataset called MetaMathQA.
The paper shows that this dataset can be distilled back into the model resulting in a fine-tuned model that outperforms several baselines on two benchmarks of mathematical reasoning.
Strengths: - The proposed approach for bootstrapping seems sound and also results in better mathematical reasoning performance through thorough experimentation
- The authors also perform ablations that show that all of the bootstrapping techniques help improve performance
- The paper is well presented and easy to follow
Weaknesses: - The major weakness I see is the lack of novelty. The paper in essence combines existing methods for bootsrapping. 

Nevertheless, I feel that the empirical findings of the paper would be interesting to the community and therefore vote for acceptance
Recommendation: 8: accept, good paper
Reviewer confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Soundness/correctness: 3 good
Clarity: 4 excellent