Let GPT be a Math Tutor: Teaching Math Word Problem Solvers with Customized Exercise Generation (2023)

Abstract: In this paper, we present a novel approach for distilling math word problem solving capabilities from large language models (LLMs) into smaller, more efficient student models. Our approach is designed to consider the student model’s weaknesses and foster a tailored learning experience by generating targeted exercises aligned with educational science principles, such as knowledge tracing and personalized learning. Concretely, we let GPT-3 be a math tutor and run two steps iteratively: 1) assessing the student model’s current learning status on a GPT-generated exercise book, and 2) improving the student model by training it with tailored exercise samples generated by GPT-3. Experimental results reveal that our approach outperforms LLMs (e.g., GPT-3 and PaLM) in accuracy across three distinct benchmarks while employing significantly fewer parameters. Furthermore, we provide a comprehensive analysis of the various components within our methodology to substantiate their efficacy.

## conclusion in this work, we present a novel approach cemal to use large language models to facilitate knowledge distillation in math word problem solving. our method first generates an exercise book to evaluate student models and then provides additional training exercises that are customized to the learning needs of the student model, thereby improving the student solver's ability to solve mwp. our extensive experimental results demonstrate that our proposed approach outperforms all fine-tuned and knowledge distillation baselines on all datasets, while achieving competitive performance against llms with significantly fewer parameters. additionally, we explore different selection generation strategies, revealing that our proposed customized generation strategy is the most effective method, especially in the in-distribution setting. in our future work, we aim to extend this approach to other nlp tasks to evaluate its generalization capability.