Paper: Improving Large Language Model Fine-tuning for Solving Math Problems (ICLR-2024)

Reviewer comment (Experiments):
The experiments are insufficient and unclear, specifically in the reranking approach, where it is not explained why K is set to 8. More experiments are needed to justify this choice.

Author response:
Thank you for the suggestion. In the left subfigure of Figure 1, we illustrate the effect of K. We observed that the improvement in solution-cluster re-ranking remains consistent across a wide range of K values.

Reviewer comment (Theory):
Are equations (10) and (11) used in the experiments? It seems that only equations (12) and (13) are used, making equations (10) and (11) appear unnecessary.

Author response:
We did not incorporate them in the experiments; they are introduced solely for discussion purposes.

Reviewer comment (Writing):
In sec 5.1, I am not sure what's the dataset GSM800K? maybe it is PRM800K. The authors should carefully check the paper writing.

Author response:
Thank you for spotting this typo. Yes it should be PRM800K.

Reviewer comment (Experiments):
The authors train the model by a sequential manner, I wonder what's the result if the three tasks are trained together. Or what's the result if only two tasks are used.

Author response:
There are two tasks involved: (1) solution re-ranking and (2) solution generation. We observed that training the models together leads to easy overfitting on the second task.

Reviewer comment (Experiments):
I am not sure the authors combine the reranking in the Section 4.4 or not. If it is not, what about the performance of reranking + majority-voting + multi-task tuning?

Author response:
In Section 4.4 we train the model in a sequential manner, i.e., (1) training the model as a solution generator, (2) training the model as a solution evaluator (re-ranker), (3) training the model again as a solution generator. We report the performance of the model as a generator.