Exploring the Mystery of Influential Data for Mathematical Reasoning
Authors: Xinzhe Ni
Yeyun Gong
Zhibin Gou
yelong shen
Yujiu Yang
Nan Duan
Weizhu Chen
Venue: COLM (2024)
Keywords: Influential Data Composition, Mathematical Reasoning
TL;DR: For mathematical reasoning, we first propose a Quality-aware Diverse Selection (QaDS) strategy to select influential data, and then construct OpenMathMix, an influential data mixture with open-source data selected by QaDS.

Abstract: Selecting influential data for fine-tuning on downstream tasks is a key factor for both performance and computation efficiency. Recent works have shown that training with only limited data can show a superior performance on general tasks. However, the feasibility on mathematical reasoning tasks has not been validated. To go further, there exist two open questions for mathematical reasoning: how to select influential data and what is an influential data composition. For the former one, we propose a Quality-aware Diverse Selection (QaDS) strategy adaptable for mathematical reasoning. A comparison with other selection strategies validates the superiority of QaDS. For the latter one, we first enlarge our setting and explore the influential data composition. We conduct a series of experiments and highlight: scaling up reasoning data, and training with general data selected by QaDS is helpful. Then, we define our optimal mixture as OpenMathMix, an influential data mixture with open-source data selected by QaDS. With OpenMathMix, we achieve a state-of-the-art 48.8% accuracy on MATH with 7B base model. Additionally, we showcase the use of QaDS in creating efficient fine-tuning mixtures with various selection ratios, and analyze the quality of a wide range of open-source datasets, which can perform as a reference for future works on mathematical reasoning tasks.

Published as a conference paper at COLM 2024
Exploring the Mystery of Influential Data for Mathematical Reasoning
Xinzhe
Yeyun Gong
Institute of Data and Information
Microsoft
Tsinghua University
Beijing, China
Shenzhen, China
eggongemicroooft.coo
Microsoft Beijing, China
nxz22@mails. tsinghua. edu. cn
Zhibin Gou
Yelong Shen
Institute of Data and Information
Microsoft Beijing, China yeshe@microsoft.co
Tsinghua University
Shenzhen, China
Microsoft
Beijing, China
gzb22@mails. fr022emails.tssinnuua.ccom edu.
Yujiu Yang* + Institute of Data and Information Tsinghua University
Nan Duan & Weizhu Chen
Microsoft Beijing, China {nanduan, wzchen @mi emicrrosoft.omm
Shenzhen, China
yang. bong.yujjussz tsinghu. edu. cn
Abstract
Selecting influential data for fine-tuning downstream tasks is key factor for both performance and computation efficiency. Recent works have shown that training with only limited data show superior perfor- mance on general tasks. However, the feasibility on mathematical reasoning tasks has not been validated. To go further, there exist two open questions for mathematical reasoning: how to select influential data and what an in- fluential data composition. For the former one, we propose a Quality-aware Diverse Selection (QaDS) strategy adaptable for mathematical reasoning. A comparison with other selection strategies validates the superiority QaDS. For the latter one, we first enlarge our setting and explore the influ- ential data composition. We conduct a series of experiments and highlight scaling up reasoning data, and training with general data selected by QaDS is helpful. Then, we define our optimal mixture as Math an influential data mixture with open-source data selected by QaDS. With OpeMMathMix, we achieve a state-of-the-ar 48.8% accuracy on MATH with 7B base model. Additionally, we showcase the of QaDS in creating efficient fine-tuning mixtures with various selection ratios, and analyze the quality of wide range of open-source datasets, which can perform a reference for future works on mathematical reasoning tasks.
1 Introduction
Mathematical reasoning tasks necessitate models with stronger reasoning ability beyond basic understanding ability than general tasks, e.g. alignment with instructions, asking for
"This work was done during the internship of Xinzhe Ni at Microsoft.
I *Corresponding authors
1

---

Published as a conference paper at COLM 2024
both strong models and influential data. For model aspect, large language models (LLMs), e.g. GPT-4 (OpenAI, 2023) and Gemini (Team et al., 2023), have displayed remarkable capabilities on wide range of natural language tasks recently (Wei et al., 2022a; Schaeffer et al., 2023; Liu et al., 2023), including the challenging mathematical reasoning tasks (Luo et al., 2023; Ying et al., 2024; Shao et al., 2024; Zhu et al., 2022). When turning to data aspect, several works argue that fine-tuning with only limited data can lead to superior performance on general tasks (Taori et al., 2023; Wang et al., 2023; Zhou et al., 2024), and sparked series of works (Wu et al., 2023; Chen et al., 2024; Li et al., 2023b;c; Du et al., 2023; Lu al., 2023; Liu et al., 2024; Wang et al., 2024) focusing on the fine-tuning data. However, there exist no works exploring an influential data composition on mathematical reasoning tasks. To be specific, we explore such mystery by answering two questions for mathematical reasoning: (1) How to select influential data? (2) What is an influential data composition?
Since the feasibility of prior strategies on mathematical reasoning tasks has not been vali- dated, it is of urgency to propose an effective selection strategy for mathematical reasoning. To this end, we propose a Quality-aware Diverse Selection (QaDS) strategy adaptable for mathematical reasoning, taking into account both diversity aspect and quality aspect of data. For diversity aspect, we adopt a simple but effective K-center Greedy strategy to involve diverse data distributions. While for quality aspect, we ask: what does "high-quality" mean on mathematical reasoning tasks? Although the quality of data on general tasks can be indicated by many direct aspects such as input length, fluency, naturalness and so on, they are hardly associated with the reasoning ability. Prior works (Dai et al., 2022; Irie et al., 2022) have shown the inherent duality between attention and gradient descent. Inspired by that, we define abstract "quality" with specific "quality score" based on the positive influence of data on each other, simulating whether sample is influential for other samples in the training process. Specifically, we measure the positive influence in "one-shot inference" process. Nuggets (Li et al., 2023c) shares similar insights on the quality of data, while only performing within single instructionffolloingg dataset, and ignoring the importance of diversity. We validate the effectiveness of QaDS on a wide range of mathematical reasoning tasks, which performs as an answer to the first question above.
When coming to the second question, we first enlarge our training datasets and construct 4 sub-settings to explore the influential data composition. We highlight 2 observations: (1) Scaling up reasoning data is helpful. (2) Together with general data, especially with those selected by QaDS, the performance can be further improved. We define our optimal mixture as OpenMath Mix, an influential data mixture with open-source data selected by QaDS. We also apply QaDS construct subsets of OpenMathMix with selection ratios of 10%, 40% and 70% for computation efficiency, and validate the generalization ability of OpenMath.Mxx Additionally, we analyze the quality of corresponding datasets as a reference for future works on mathematical reasoning tasks.
Our contributions are below: (1) We are the first to propose a Quality-aware Diverse Selection (QaDS) strategy for selecting influential data focusing on mathematical reasoning tasks, and QaDS shows superiority over other selection strategies.
(2) We explore an influential data composition for mathematical reasoning tasks, and construct Mpen Mix, an influential data mixture with open-source data selected by QaDS. With Open we achieve state-of-the-ard 48.8% accuracy on MATH. (3) We showcase the use of QaDS in creating efficient fine-tuning mixtures with various selection ratios, and analyze the quality of a wide range of open-source mathematical reasoning and general datasets.
2 Related Work
The data selection strategies have been widely discussed on general tasks. Although they focus on various aspects of data, we summarize them into only two aspects for clarity: diversity and quality. (1) Diversity. SSL Prototype is proposed in Sorscher et al. (2022), and applied to LLM in D4 (Tirumala et al., 2023), which discards the nearest data points to clusters after K-means. DiverseEvol (Wu et al., 2023) iteratively selects diverse data with K-center Greedy (Sener & Savarese, 2018), and updates the text embeddings as well. (2) Quality. AlpaGasus (Chen et al., 2024) leverages ChatGPT (OpenAI, 2022) to filter
2

---

Published as a conference paper at COLM 2024
Diversity
Quality
Diversity & Quality
D
PP
thnn'ss onyy are enpty, 2 galtone per minue many minuaes wil it take bannls?
Sample in samples
Quality Score
placing the
many crars were her attgather?
Distance
X;
Aau
Xiaa
Selectt ty samples
Eusikdenn
inter One-shot Score
O
Agul
Play has 20 cent in nickets it Pay gives as cens to Fater and tuton many cento Fandi give to Pear,, how reduaa doca lent?
Sample to samples
Viler
LIM
Zero shot Soore
App
De
(5(+%)
(SG
On furms on average enity 6 cows produce 108 has milk per week. in the wesks the cowe prockced P150 law of ria Hew cowo are on lane?
acceer
Figure 1: An overview our proposed QaDS, including three parts from left right. Left: A pipeline with diversity aspect. Middle: A pipeline with quality aspect. Right: Combining diversity aspect and quality aspect, QaDS selects influential data for fine-tuning.
high-quality data, and shows a remarkable performance on instruction-following tasks. IFD (Li et al., 2023b) is also a quality-related metric, while focusing on how to estimate the difficulty of a given instruction. Nuggets (Li et al., 2023c) applies one-shot inference as quality measure, thus exploring the inherent relation between pair-wise data. (3) Diversity and Quality. MoDS (Du et al., 2023) uses reward model to obtain quality scores, and selects high-quality data. Then, filters the high-quality data with K-center Greedy (Sener & Savarese, 2018) to obtain seed data. Finally, second quality evaluation is performed on the high-quality data with fine-tuned model with seed data. InsTag (Lu et al., 2023) first employs ChatGPT to generate open-set and fine grained labels, then selects diverse and complex data with a complexity-firs diverse sampling strategy. Deita (Liu al., 2024) labels small part of data with ChatGPT (OpenAI, 2022), and trains scorer to obtain quality scores. With these scores, Deita proposes a score-first and diversity-aware selection method.
We aim to explore influential data composition for mathematical reasoning tasks, while above strategies are only validated on general tasks. To this end, we first propose Quality- aware Diverse Selection (QaDS) strategy, and then construct an influential data mixture OpenMathMix with open-source data selected by QaDS.
Data Selection Strategy
Taking into account both diversity aspect and quality aspect, we propose a Quality-aware Diverse Selection (QaDS) strategy for mathematical reasoning. As shown in Figure 1, QaDS includes three parts from left to right. We will introduce the three parts respectively below.
3.1 Diversity Aspect
As data diversity is key factor for training powerful LLM, we pay attention to diversity aspect in our data selection as well. According to the practice prior works (Wu et al., 2023; Du et al., 2023), use simple but effective method, K-center Greedy (Sener & Savarese,
3

---

Published as a conference paper at COLM 2024
Algorithm 1 K-center Greedy
Input: data X;, existing pool 50 and budget b Initialize 5 repeat
U [c]] minjes A (x),
= uuu
Until |s| b + +|
Return s/,50
2018), to select data with high diversity. As shown in Algorithm 1, the objective of K-center Greedy is to select subset of all data in manner that maximizes the minimum distance between each new data x; and the current data pool s. By iteratively adding the farthest data to the existing pool so, the diversity is guaranteed.
We initialize |so| as 100 in our experiments. For embeddings, we first randomly select small subset of data and train brief pre-experienced model. Then, we obtain the output hidden states after average pooling of our pre-experienced model rather than the original model as the required embeddings. An ablation is shown in Table 6.
3.2 Quality Aspect
Quality Score. In addition to data diversity, data quality is another key factor to be taken into account. We define abstract "quality" with specific "quality score" based on the positive influence of data on each other. Specifically, measure the positive influence in "one-shot inference" " process. For clarity, we illustrate the process on a single dataset below. In our experiments, the average score across multiple datasets is used.
Given a dataset D = {x X, h, each sample is composed of a question and an answer, and is denoted X; { }. We divide D into prompt set D, {x Xn, and
test set D; = { (,,, For each sample x;
in Dr, we compute the zero-shot score s
S/2 = Σ log p (wf)) xxi, w ...., - MiM)
(1)
where L denotes the length of tokens in x"; p(-) denotes the probability of certain token, w denotes the jth token x and M denotes LLM. Therefore, we obtain a zero-shot score set S.s ( 2 Then, the one-shot score of each sample xę Dp on x; in D; is:
I Σ
log (w x" , x0" w
/ Mi
(2)
When prompt sample improves the probability of correct answers in the test sample, it is reasonable to regard the prompt sample as a high-quality one. According to the premise, compute the final quality score of Xx:
11
QS (xx)
ki 50s
(3)
112
where 1(.) is the indicator function. More implementation details are demonstrated in Appendix Section A.
Quality Scorer. To reduce the computational costs of above quality scores, we first compute quality scores with small ratio of data, then discretize the quality scores to a range of to 5 as labels and train quality scorer. The rationality of scorers is discussed in Section
4

---

Published as a conference paper at COLM 2024
Dataset
Task
Source
Obtained Size
Soose
AQuA-RAT (Ling al., 2017)
Human
69K
Camel-Math (Li al,, 2023a)
GPT
48K
College-Math (Yue et al, 2024)
GPT
1.8K
GSM8K (Cobbe et al., 2021)
Human
7.4K
Mathematical
GSM8K-RFT (Yuan et al., 2023)
LLaMA
16K
MATH (Hendrycks et al., 2021b)
Human
7.4K
Number- Comparisn (Yue et al., 2024)
300
TheoremQA (Chen al., 2023)
GPT
577
CoT (Wei et al., 2022b)
Human
132K
General
Bvol-instruct-22 (Xu al., 2023a)
GPT
142K
Total
428K
Starge
DMath (Kim et al., 2023)
Human
7.9K
CSM8K-ScRel (Yuan et al, 2023)
LLaMA
7.4K
Lila-OOD (Mishra et al., 2022)
Human
19K
MathInstruct (Yue et al., 2024)
Human, GPT
262K
MetaMath (Yu al., 2024)
GPT
395K
MMIQC (Liu Yao, 2024)
Mathematical
GPT
2.2M
KPMath- (Huang al., 2024)
GPT
1.0M
Open DennMathnstrrut-11 (Toshniwal al., 2024)
Mixtral
2.5M
Open-Platypus (1 al., 2023)
Human, GPT
24K
TAL-SCQÁK-CN (Math-eval, 2023)
Human
3.0K
TAL-SSQ5K-EEN (Math-eval, 2023)
Human
3.0K
Alpaca (Taori et al., 2023)
Davinci-003
52K
Baize (Xu et al., 2023b)
GPT
210K
CoT (Wei et al., 2022b)
Human
149K
Fvol-Instruct-v2 (Xu et al., 2023a)
General
GPT
143K
Flan-v2 (Longpre al., 2023)
Human
50K
Open-Assistant-1 (Köpf al., 2023)
Human
10K
Self-Instruct (Wang al., 2023)
GPT
82K
Total
5.0M
Table 1: An overview of our training datasets. Stose denotes our base setting, composed of 8 mathematical reasoning datasets and 2 general datasets. Starge denotes our large setting, composed of 11 mathematical reasoning datasets and 7 general datasets. For OpenMathInstruuct, we only use the CoT part.
5.3. Finally, we obtain 4 scorers: Scorer-LLaMA-2 and Scorer-Mistral in Stuser and Scorer- DeepSeek-Reasoning and Scorer-DeepSeek-Geneer in Starge-
3.3 Quality-aware Diverse Selection (QaDS)
After obtaining both embeddings and quality scores, we perform QaDS to select the data which is most influential for mathematical reasoning. The process can be implemented by changing the selection metric in Algorithm
=1 argmax (QS(x)) min A (x)),;)
jes
where 2x) denotes the distance metric between x; and xj, and QS(x;) denotes the quality score of x;. QaDS maximizes product of diversity metric and quality metric and thus leading to effective data selection strategy.
Experimental Setup
Training Datasets. We conduct two settings in our work as shown in Table 1: In Spaser aim to validate effectiveness over other selection strategies. We first collect the

---

Published as a conference paper at COLM 2024
CoT subset of MathInstruct (Yue et al., 2024), which include relatively diverse range of 8 mathematical reasoning datasets. Some researches (Shi et al., 2023; Dong et al., 2023) have pointed out that data in general domain can also be helpful for mathematical reasoning ability to some extent, thus we then include two general datasets of CoT (Wei et al., 2022b) and Evol-Instruct-v2 (Xu et al., 2023a). As result, we construct base setting with 153K reasoning data and 275K general data, resulting in a total size of 428K. After validating the effectiveness of QaDS, we aim to explore the influential data composition for mathematical reasoning tasks on a larger setting of Starge. We enlarge the range of both reasoning and general datasets, and construct base setting with 4.3M reasoning data and 697K general data, resulting in a total size of 5.0M.
Selection Scheme. In Stsse we notice that the gap between datasets is particularly outstand- ing (e.g. 300 142K). With a standard uniform selection scheme, the influence of small datasets may be submerged. To validate our QaDS on a relatively balanced setting, we prefer to performing selection only on those large datasets, and reducing to a balanced size. A formal definition of selection ratios is demonstrated in Appendix Section B. Specifically, we perform selection on 3 sub-settings:
(1) Shose 69K involving selection on AQuA-RAT, Camel-Math, CoT and Evol-Instruct-v2. (2) Shose 130K involving selection on AQuA-RAT, CoT and Fvol-Instruct-v2. (3) Sbase-203K involving selection on CoT and Evol-Instruct-v2. A detailed composition is shown in Appendix Table 7. Note that we aim to validate our QaDS in reasonable settings, and the specific composition is not a key point in this part. In targer since we aim to explore an optimal composition for mathematical reasoning tasks, the above scheme is not appropriate enough. Instead, we set the selection size by the average quality score we computed. Formally, let QSanng and Q5max denote the average quality score of ith dataset and the upper bound of quality score relatively, the selection ratio r; satisfies: = Q m Q5 As quality scores by our scorer range from 1 to 5, QS is 5. As a result, we perform comparison between 5 sub-settings in Table 2. According to our observation in Section 5.2, we define our optimal sub-setting Starge 4.7M as OpenMathMix, A detailed composition is shown in Appendix Table 8
Sturge- (OpenMathMixx
Task
Strgge 4.3M
Starge-5.0M 5.0M
Starge-26M 226
Starge-3.0M
Mathematical Reasoning General
All
All All
Selected
Selected Selected
All Selected
Table 2: Sub-settings for comparison in Starge- "All" indicates using all data, and "Selected" indicates using selected data by QaDS.
Evaluation Datasets. Our evaluation datasets in the two settings consistent with the computation process of quality scores. In Spase, we compute quality scores across training datasets to fully exploit the reasoning ability. To validate the generalization ability, we adopt 9 datasets including AQuA-RAT (Ling et al., 2017), GSM8K (Cobbe et al., 2021), MATH (Hendrycks et al., 2021b), NumGLUE (Mishra et al., 2022), Mathe- matics (Davies et al., 2021), MMLU-Math (Hendrycks et al., 2021a), SAT-Math (Zhong et al., 2023), SimulEq Kkoncel-Kedziorsk et al., 2016) and SVAMP (Patel et al., 2021). In Starge we adopt subset of MATH as D,, and thus choose the corresponding test set of MATH for evaluation. We adopt CoT evaluation consistent with the type of our most training datasets.
Baselines. In Spaser we involve 6 data selection strategies to validate the effectiveness of QaDS. In addition to random selection, we consider 2 strategies with diversity aspect including SSL. Prototype (Tirumala et al., 2023) and K-center Greedy (Sener & Savarese, 2018), 2 strategies with quality aspect including IFD (Li et al., 2023b) and Quality Score computed by us, and 1 strategy with both diversity aspect and quality aspect including Deita (Liu et al., 2024). In Starger we involve 5 base models including LLaMA-2 (Touvron et al., 2023), Mistral (Jiang et al., 2023), Llemma (Azerbayev et al., 2024), IntemLMM-Mathn Base (Ying et al., 2024) and DeepSeekMath-Bee including WizardMath-7B--i (Luo et al., 2023), MAmmoTH-Coder (Yue et al., 2024), ToRA-
(Shao et al., 2024), and 6 fine-tuned models

---

Published as a conference paper at COLM 2024
Coder (Gou et al., 2024), IntemLMM--Mthh (Ying et al., 2024), DeepSeekMath-Instruct (Shao et al., 2024) and MathScale-Mistral (Tang et al., 2024).
Training Details. In Susse we adopt LLaMA-2-7B (Touvron et al., 2023) and Mistral-7B (Jiang et al., 2023) as base models. We set the global batch size to 128 and train for 3 epochs. We use learning rate of 2e-5 and cosine scheduler with 3% warm-up. We apply DeepSpeed ZeRO Stage3 (Rajbhandari et al., 2021) and Flash-Attention 2 (Dao, 2023) for efficient training. All experiments are conducted with 4 NVIDIA A100 GPUs. In Starger we adopt DeepSeekMath-Ban (Shao et al., 2024), state-of-the-art model, base model. The training schedule is as same as that in Stase We train for epoch with NVIDIA H100 GPUs.
Experimental Results
5.1 Main Results on Sbase
To validate the effectiveness of QaDS, we conduct experiments on Stuse- 69Kr Shase-30KK and 203K- As shown in Fig- ure 2, with gradually enlarged data sizes, the performance of all strategies improves as well. Among them, QaDS achieves the best performance in all three sub-settings, especially with scarce data (i.e. Spase 69k). detailed analysis of Spase 69k discussed below. Furthermore, QaDS outperforms the performance with all 428K data in 203K On the contrary, some well- performed strategies on general tasks are far below random selection mathemati- cal reasoning tasks, indicating the necessity of our QaDS. Besides, we observe that when fine-tuning with a combination of reason- ing data and general data (the dark gray dashed line), LLM achieves performance gains over only fine-tuning with reasoning data (the light gray dashed line). It turns out that general data can also be influen- tial on mathematical reasoning tasks. To explore deeply, we enlarge our setting to Starge and discuss in Section 5.2.
12
- QuDS
-0- K-esster Greedy
Random
28
Quality
SSL Prototpp
27
69K
170K
203K
Data Size
Figure 2: Average accuracy in S bbake699K Sasse-130K and S Shase-203K 203K with LLaMA-2. The dark gray dashed line: performance with all 428K mathematical reasoning and general data. The light gray dashed line: performance with all 153K mathematical reasoning data.
Table 3 presents comparison of QaDS with other selection stategies in our sub-setting Shose-69K, highlighting the following observations: (1) Some strategies are worse than random selection mathematical reasoning tasks. Specifically, IFD and Deita only achieve a lower average accuracy of 27.4% and 28.9% than 29.5% by random selection with LLaMA-2. With Mistral, SSL Prototype and IFD only achieve that of 31.6%, and 32.0% than 32.2%. (2) Two components of QaDS perform well individually. K-center Greedy performs the best (30.4% 29.8%, and 32.4% 31.6% with different base models) considering diversity aspect, and so does Quality Score (30.2% as 27.4%, and 33.0% as 32.0% with different base models) considering quality aspect. (3) Combining the above two aspects, QaDS achieves the best performance on 4 mathematical reasoning tasks, and finally, the best overall performance of 31.3% and 33.9% with LLaMA-2 and Mistral respectively.
5.2 Main Results on Starge
Table 4 presents a comparison of fine-tuning with DeepSeekMath-BBrt in 5 sub-settings, as discussed in Table 2, by QaDS and other LLMs in Starge- We highlight the following ob- servations: (1) Comparison between DaDSS-Saagee-2 and QaDDS-Iinge-43M validates that
7

---

Published as a conference paper at COLM 2024
Strategy
Aspect
All
GSM.
MATH
AQuA
Num.
SVA.
Math.
Sim.
SAT.
MMLU.
LLaMA-2 base model
Random
29.5
45.6
8.4
34.3
36.7
49.9
8.8
15.6
32.3
33,9
SSL Prototype
29.8
46.4
8.9
31.9
35.7
48.6
8.0
12.8
38.2
38.1
Diversity
K-center Greedy
30.4
46.8
8.9
37.4
37.9
48.8
9.0
11.9
37.3
35.6
Quality
27.4
43.6
8.9
29.5
34.7
48.0
8.0
11.3
31.8
30.6
Quality Score
30.2
45.7
9.4
29.9
37.4
47.2
15.0
40.9
36.3
Deita
Diversity
28.9
45.0
8.5
29.9
36.8
45.4
7.8
15.0
36.4
35.4
QaDS
Quality
31.3
47.2
8.6
33,9
38.0
50.0
9.2
14.8
42.7
36.9
Mistrill base model
Random
32.2
50.6
12.5
32.3
36.5
51.1
12,4
16.5
38.6
38,9
SSL Prototype K-center Greedy
31.6
49.4
12.3
32.3
40.0
46.8
11.2
15.6
36.8
39.7
Diversity
32.4
49.5
13.1
39.0
35.3
48.2
10.7
20.4
32.7
43.0
IFD Quality Score
Quality
32.0
52.4
12.6
33.1
39.9
48.0
11.6
17.9
35.0
37.1
33.0
51.1
12.8
38.6
42.4
44.8
12.3
18.5
39.5
37.3
Diversity Quality
32.9
51.8
12.8
48.8
12.0
18.1
38.2
38.8
QaIsS
33.9
52.5
13.4
43.7
38.2
48.7
12.3
20.4
35.0
41.3
Table 3: CoT accuracy comparison with 7B models mathematical reasoning tasks in Shsse- 69K- All: overall accuracy, GSM.: GSM8K, Num NumGLUE, SVA.: SVAMP, Math.: Mathematics, Sim.: SimulEq, SAT.: SAT-MAth, MMLU.: MMLU-Math. The bold font indicates the best results. QaDS outperforms all other data selection strategies overall.
50
Model
SFT
MATH
Random
LLaMA-2 (Touvron et al., 2023)
X
4.1
QaDS
Mistral (fiang et al., 2023)
X
10.5
Llemma (Azerbayev et al., 2024)
X
18.0
MiternLM2-MMthh-Bal (Ying et al., 2024)
X
21.5
46
DeepSeekMath--Bar (Shao et al., 2024)
X
30.6
WizardMath-7B-rL.11 (Luo et al., 2023)
33.0
Accuracy
44
MAnmoeH-Coder (Yue et al., 2024)
33.4
ToRA-Coder (Gou et al., 2024)
44.6
InternLMM2Matth (Ying et al., 2024)
34.6
DeepSeekMath-Instrue (Shao et al., 2024)
46.8
MathScale-Mistral (Tang et al., 2024)
35.2
DaDS-Sjange-330
44.2
45.6
38
QaDS-Starge Starge-26M
42.6
0.5M
1.8M
3.3M
4.7M
QaDS- 1.00--S
45.2
Data Size
QaDS- Starge 47M (QaDS-OpenMathin
48.8
Figure 3: Accuracy of 0.5M, 1.8M,
3.3M and 4.7M data of Open Mix with selection ratios of 10%, 40%, 70% and 100% by QaDS.
Table 4: CoT accuracy comparison with 7B models on
MATH in Starge The bold font indicates the best result.
general data can also be influential for mathematical reasoning tasks (45.6% us 44.2%). And comparison between DaDS-Starge-3nn0 and QaDS-S MaDDS-Starge-26.. 226 also comes to the same con- clusion (45.2% us 42.6%). (2) Comparison between QaDSS-Stage-266 and QaDS- DaDD-Siinge-4.MM indicates that the reasoning ability improves the scale of reasoning data increases (42.6% vs 44.2%), which consistent with Dong et al. (2023). (3) Hypothesizing there is more abun- dance in general data than reasoning data, it is feasible to perform selection on general data, leading to QaDS-Starge 4.7M- With all reasoning data and general data selected by QaDS, we achieve the best performance, and validate the hypothesis in turn. (4) Compared to base models and 6 fine-tuned models,
QaDS- QaDS-S Starge-4.7M achieves state-of-the-art perfor- mance of 48.8% on MATH with 7B base model, and we define Starge-4.M as OpenMathMix, an influential data mixture with open-source data selected by QaDS.
8

---

Published as a conference paper at COLM 2024
P 0.91
Apaca
MMICC
8
Boizo
KPMaassPaaa
232
OoT
30
2.8
Mathinstruct
28
Open-Asostamn
2.8
TAL-SCOSK CN
2.
Snftunt
2.4
TAL-SCOSKEEN
24
EvoHnstrust-22
2.2
22
2.0
20
1.8
18-
1,8
16
14
144
12
12
1.0
(a) Pearson analysis of Senner-DeeSSee-RReesone (b) Pearson analysis of Scorer- Scorer-DeepSeck-Gemte
Figure 4: Pearson analysis between real quality scores and scorer quality scores.
We also construct 3 subsets of OpenMathMix with selection ratios of 10%, 40% and 70% by QaDS. In Figure 3, the accuracy improves as data sizes increase, and outperforms random selection by a large margin. Considering the computation efficiency, these 3 subsets can be used for achieving strong performance with limited data.
5.3 Analysis
OpenmathMix is generalizably influential data. Since construct our OpenMathMix with two scorers trained on Mathh Base, is of necessity to validate the feasibility of Open with other base model, egg. Mistral. An underlying hypothesis that in- fluential data is supposed to be largely model- agnostic. In Table 5, fine-tuning with Open- MathMix compared to randomly selected data, we achieve significant performance gains (6.4% and 3.2%) with both two models. It turns out that OpenmathMix is generalizably influential data, and can bring improvement to various base models. Also, the results demonstrate that our scorers can capture the common pattern of influential data, which will be discussed further below.
Base Model
Random
OpenMathMix
DeepSeekMath-Bas
42.4
48.8
Mistral
35.2
38.4
Table 5: Validation of the generalization ability of OpenMathMix.
Scorers are accurate indicators of real quality scores. The computational costs real quality scores in Equation highly depend the product of πj and f2, which can not be ignored in large-scale datasets. With scorer, however, we approximately reduce the computational complexity from O O(n) n22) (MM) O(n)) ((M), where (MM) is the original complexity with model M. To validate the rationality, we sample 300 samples from 7 datasets, which are non-overlapping with our training data of scorers, and perform Pearson analysis. As shown in Figure 4, there is a strong correlation between quality scores by Seerer-DeeSSee-RReaaa Reasoning and real quality scores (Pearson r 0.91). Although scoring general data is more difficult, Exooer-DeepSeek--Gee General can still achieve high Pearson r of 0.74. Besides, we observe that CoT dataset achieves high scores. It mainly because CoT dataset can improve the sfain-of-thought ability in reasoning process. Such observation explains the positive influence of general data to some extent.
To further explore the common pattern of influential data, we compute the average quality scores of each dataset with our scorers in Figure 5. For mathematical reasoning datasets, complicated reasoning process more likely to be scored high. (2) Samples with no reasoning process or code-like answers are scored low. Also for general datasets: (1) Samples with complete reasoning process math-related
conclude: (1) Samples with complete

---

Published as a conference paper at COLM 2024
5.00
5.00
4.54
4.17
4.0K
4.00
4.00
3.29
Quality Score
3.00
Quality Score
3.18
2.56
3.00
2,38
2.00
1.98
2.00
1.67
1,33
1.00
1.00
0.00
OpaMMahtannet
KPM Pluc
GSMRK ScRee
ManMath
TAL- ALL-scOsK-BB ENV
TAL PLL-sC29K-CM
MMIOC
Mablistnst
Open- Open-Playynn Play-ylayyuu
DMafft
Lit-OO0
0.00
CaT
Alpaca
Open Lsistett-
Flan-
Self-hminntt
Enol-noiiud--2
Batze
(a) Quality scores by Scorer-DeepSeek Seeeee-DeeSSeeeasssoeneee Reaneonin.. (b) Quality scores by Scorer- DeeeDeee-eSSeeeeeeeeeee
Figure 5: Average quality scores of mathematical reasoning and general datasets in Starge
answers are scored high. (2) Samples with little reasoning-related process are scored low. Cases can be found in Appendix Table 9 and 10.
Pre-experienced embeddings are appropri- ate for selection. According to IFD (Li et al., 2023b), it is useful to force the model to first experience subset of data. Inspired by that, we train a pre-experienced model for comput- ing embeddings. As shown in Appendix Table 6, selection with these two embeddings leads to similar performance with Mistral. However, pre-experienced embeddings can achieve 1% performance gains with LLaMA-2. A possible reason that LLaMA-2 weaker base model, and thus needs to experience in-domain data all the more.
Original Embeddings
Pre-experienced Embeddings
Base Model
LLaMA-2
30.3
31.3
Mistral
34.1
33.9
Table 6: Ablation of embeddings in QaDS.
Conclusion
In this work, we shed light on two questions for mathematical reasoning: (1) How to select influential data? (2) What is an influential data composition? For the former one, propose a Quality-aware Diverse Selection (QaDS) strategy for selection, outperforming other strategies three sub-settings. For the latter one, conduct series of experiments and observe that: (1) Scaling up reasoning data is helpful. (2) General data can also be influential on mathematical reasoning tasks. We construct OpenMathMix, an influential data mixture with open-source data selected by QaDS, and achieve a state-of-the-art 48.8% accuracy on Math. Additionally, we showcase the use of QaDS in creating efficient fine- tuning mixtures with various selection ratios, and analyze the quality of wide range of open-source datasets. The quality scores and associated analysis can perform as a reference for future works on mathematical reasoning tasks.
Acknowledgments
This work was partly supported by the National Natural Science Foundation of China (Grant No. U1903213) and the Shenzhen Science and Technology Program (SGCC2020830093040000)
10

---

Published as a conference paper at COLM 2024
References
Zhangir Azerbayev, Hailey Schoelkopf, Keiran Paster, Marco Dos Santos, Stephen McAleer, Albert Q Jiang, Jia Deng, Stella Biderman, and Sean Welleck, Llemma: An open language model for mathematics. In ICLR, 2024.
Lichang Chen, Shiyang Li, Jun Yan, Hai Wang, Kalpa Gunaratna, Vikas Yadav, Zheng Tang, Vijay Srinivasan, lianyi Zhou, Heng Huang, et al. Alpagasus: Training a better alpaca with fewer data. In ICLR, 2024.
Wenhu Chen, Ming Yin, Max Ku, Pan Lu, Yixin Wan, Xueguang Ma, Jianyu Xu, Xinyi Wang, and Tony Xia. Theoremqa: A theorem-driven question answering dataset. 2023.
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems. arxiv preprint ar Xit:22010. 14168, 2021.
Damai Dai, Yutao Sun, Li Dong, Yaru Hao, Zhifang Sui, and Furu Wei. Why can gpt learn in-context? language models secretly perform gradient descent as meta optimizers. arv preprint arXiv:2212.00553 2022.
Tri Dao. Flashattention-2: Faster attention with better parallelism and work partitioning. 2023.
Alex Davies, Petar Veličković, Lars Buesing, Sam Blackwell, Daniel Zheng, Nenad Tomašev, Richard Tanburn, Peter Battaglia, Charles Blundell, András Juhász, et al. Advancing mathematics by guiding human intuition with ai. Nature, 600(7887):70-74, 2021.
Guanting Dong, Hongyi Yuan, Keming Lu, Chengpeng Li, Mingfeng Xue, Dayiheng Liu, Wei Wang, Zheng Yuan, Chang Zhou, and Jingren Zhou. How abilities in large lan- guage models are affected by supervised fine-tuning data composition. arXiv preprint arXiv:2310.05499 2023.
Qianlong Du, Chengqing Zong, and Jiajun Zhang. Mods: Model-oriented data selection for instruction tuning. ar preprint ar arXio:2311.15653, 2023.
Zhibin Gou, Zhihong Shao, Yeyun Gong, Yujiu Yang, Minlie Huang, Nan Duan, Weizhu Chen, et al. Tora: A tool-integrated reasoning agent for mathematical problem solving. In ICLR, 2024.
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. In ICLR, 2021a.
Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. In NIPS, 2021b.
Yiming Huang, Xiao Liu, Yeyun Gong, Zhibin Gou, Yelong Shen, Nan Duan, and Weizhu Chen. Key-point-driven data synthesis with its enhancement on mathematical reasoning. arXiv preprint ar XrXo:22403.0333, 2024.
Kazuki Irie, Róbert Csordás, and Jürgen Schmidhuber. The dual form of neural networks revisited: Connecting test time predictions to training patterns via spotlights of attention. In ICML, pp. 9639-9659, 2022.
Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. Mistral 7b. arv preprint ar xrXio:2310006825, 2023.
Jiwoo Kim, Youngbin Kim, Ilwoong Baek, Jin Bak, and Jongwuk Lee. It ain't over: A multi-aspect diverse math word problem dataset. In EMNLP, 2023.
Rik Kedziorke,,i,,,ki, Subhro Roy, Aida Amini, Nate Kushman, and Hannaneh Hajishirzi. Mawps: A math word problem repository. In NAACI, pp. 1152-1157, 2016.
11

---

Published as a conference paper at COLM 2024
Andreas Köpf, Yannic Kilcher, Dimitri von Rütte, Sotiris Anagnostidis, Zhi Rui Tam, Keith Stevens, Abdullah Barhoum, Duc Nguyen, Oliver Stanley, Richard Nagyfi, et al. Openas- sistant converrstttooseeeerrrrrzzii large language model alignment. In NIPS, volume 36, 2023.
Ariel N Lee, Cole J Hunter, and Nataniel Ruiz. Platypus: Quick, cheap, and powerful refinement of Ilms. arXiv preprint ar xrXto:2308.07317, 2023.
Guohao Li, Hasan Abed Al Kader Hammoud, Hani Itani, Dmitri Khizbullin, and Bernard Ghanem. Camel: Communicative agents for" mind" exploration of large scale language model society, 2023a.
Ming Li, Yong Zhang, Zhitao Li, Jiuhai Chen, Lichang Chen, Ning Cheng, Jianzong Wang, Tianyi Zhou, and Jing Xiao. From quantity to quality: Boosting 11m performance with self-guided data selection for instruction tuning. arXiv preprint arXiv:2308..100000 2023b.
Yunshui Li, Binyuan Hui, Xiaobo Xia, Jiaxi Yang, Min Yang, Lei Zhang, Shuzheng Si, Junhao Liu, Tongliang Liu, Fei Huang, et al. One shot learning as instruction data prospector for large language models. Xrviv preprint ar 1xXx:2312.10300 2023c.
Wang Ling, Dani Yogatama, Chris Dyer, and Phil Blunsom. Program induction by rationale generation: Learning solve and explain algebraic word problems. In ACL, pp. 158-167, 2017.
Haoxiong Liu and Andrew Chi-Chih Yao. Augmenting math word problems via iterative question composing. arviv preprint arX:2:4401 00000000,,,0003,, 2024.
Peiyu Liu, Zikang Liu, Ze-Feng Gao, Dawei Gao, Wayne Xin Zhao, Yaliang Li, Bolin Ding, and Ji-Rong Wen. Do emergent abilities exist in quantized large language models: An empirical study. Xrviv preprint wxXi::2377080077 2023.
Wei Liu, Weihao Zeng, Keqing He, Yong Jiang, and Junxian He. What makes good data for alignment? a comprehensive study of automatic data selection in instruction tuning. In ICLR, 2024.
Shayne Longpre, Le Hou, Tu Vu, Albert Webson, Hyung Won Chung, Yi Tay, Denny Zhou, Quoc V Le, Barret Zoph, Jason Wei, et al. The flan collection: Designing data and methods for effective instruction tuning. In ICML, pp. 22631-22648, 2023.
Keming Lu, Hongyi Yuan, Zheng Yuan, Runji Lin, Junyang Lin, Chuanqi Tan, Chang Zhou, and Jingren Zhou. # instag: Instruction tagging for analyzing supervised fine-tuning of large language models. arXiv preprint ar Xii:2308.00074 2023.
Haipeng Luo, Qingfeng Sun, Can Xu, Pu Zhao, Jianguang Lou, Chongyang Tao, Xiubo Geng, Qingwei Lin, Shifeng Chen, and Dongmei Zhang. Wizardmath: Empowering mathematical reasoning for large language models via reinforced evol-instruct. arXiv preprint ar Xiv: 095883, 2023.
Math-eval. Tal-scq5k. https://github.. https://gihubb.om/mmatt///////// http:////wt/ttaat/////aaa/CCCACA 2023.
Swaroop Mishra, Matthew Finlayson, Pan Lu, Leonard Tang, Sean Welleck, Chitta Baral, Tanmay Rajpurohit, Oyvind Tafjord, Ashish Sabharwal, Peter Clark, et al. Lila: A unified benchmark for mathematical reasoning. In EMNLP, pp. 5807-5832, 2022.
OPenAI. Introducing chatgpt. https://opendd.cmm//hhh//aaggg/ 2022.
2022.
OpenAl. Gpt-4 technical report. 2023.
Arkil Patel, Satwik Bhattamishra, and Navin Goyal. Are nlp models really able to solve simple math word problems? In NAACL, pp. 2080-2094, 2021.
Samyam Rajbhandari, Olatunji Ruwase, Jeff Rasley, Shaden Smith, and Yuxiong He. Zero- infinity: Breaking the gpu memory wall for extreme scale deep learning. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, pp. 1-14, 2021.
12

---

Published as a conference paper at COLM 2024
Rylan Schaeffer, Brando Miranda, and Sanmi Koyejo. Are emergent abilities of large language models a mirage? NIPS, volume 36, 2023.
Ozan Sener and Silvio Savarese. Active learning for convolutional neural networks: A core-set approach. In ICLR, 2018.
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Mingchuan Zhang, YK Li, Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint ar drXIv:2402.03300, 2024.
Chufan Shi, Yixuan Su, Cheng Yang, Yujuu Yang, and Deng Cai. Specialist or generalist? instruction tuning for specific nlp tasks. In EMNLP, pp. 15336-15348, 2023.
Ben Sorscher, Robert Geirhos, Shashank Shekhar, Surya Ganguli, and Ari Morcos. Beyond neural scaling laws: beating power law scaling via data pruning. In NIPS, volume 35, pp. 19523-19536, 2022.
Zhengyang Tang, Xingxing Zhang, Benyou Wan, and Furu Wei. Mathscale: Scaling instruc- tion tuning for mathematical reasoning. arviv preprint arXit:2403.02884 2024.
Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B Hashimoto. Stanford alpaca: an instruction-following llama model (2023). http:s///ttubb.omm//ttssss/sssss alpuca, 2023.
Gemini Team, Rohan Anil, Sebastian Borgeaud, Yonghui Wu, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, et al. Gemini: family of highly capable multimodal models. ar arviv preprint ar XrXi::2312.11805, 2023.
Kushal Tirumala, Daniel Simig, Armen Aghajanyan, and Ari Morcos. D4: Improving Ilm pretraining via document de-duplication and diversification. In NIPS, volume 36, 2023.
and
Shubham Toshniwal, Ivan Moshkov, Sean Narenthiran, Daria Gitman, Fei Jia, and Igor Gitman. Openmathinssruct--l A 1.8 million math instruction tuning dataset. Xiv preprint 1rX1v:2402.10176, 2024.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, al. Llama Open foundation and fine-tuned chat models. arv preprint arXiv:2307.092. 092888, 2023.
Yifan Wang, Yafei Liu, Chufan Shi, Haoling Li, Chen Chen, Haonan Lu, and Yujiu Yang. Inscl: A data-efficient continual learning paradigm for fine-tuning large language models with instructions. In NAACL, 2024.
Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A Smith, Daniel Khashabi, and Hannaneh Hajishirzi. Self-instruct: Aligning language models with self-generated instructions. In ACL, pp. 13484-13508, 2023.
Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, et al. Emergent abilities of large language models. arXiv preprint ar drXio:2206.07682, 2022a.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoe V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. In NIPS, volume 35, pp. 14824-24833 2022b.
Shengguang Wu, Keming Lu, Benfeng Xu, Junyang Lin, Qi Su, and Chang Zhou. Self-evolved diverse data sampling for efficient instruction tuning. arXiv preprint arXiv:2311008182, 2023.
Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, and Daxin Jiang. Wizardlm: Empowering large language models to follow complex instructions. ar preprint 12224440..,224,, 2023a.
13

---

Published as a conference paper at COLM 2024
Canwen Xu, Daya Guo, Nan Duan, and Julian McAuley, Baize: An open-source chat model with parameter-effcciet tuning on self-chat data. In EMNLP, pp. 6268-6278, 2023b.
Huaiyuan Ying, Shuo Zhang, Linyang Li, Zhejian Zhou, Yunfan Shao, Zhaoye Fei, Yichuan Ma, Jiawei Hong, Kuikun Liu, Ziyi Wang, et al. Internim-math: Open math large language models toward verifiable reasoning. ar preprint ar Xri::2402.06332, 2024.
Longhui Yu, Weisen Jiang, Han Shi, YU Jincheng, Zhengying Liu, Yu Zhang, James Kwok, Zhenguo Li, Adrian Weller, and Weiyang Liu. Metamath: Bootstrap your own mathemat- ical questions for large language models. In ICLR, 2024.
Zheng Yuan, Hongyi Yuan, Chengpeng Li, Guanting Dong, Chuanqi Tan, and Chang Zhou. Scaling relationship on learning mathematical reasoning with large language models. arxiv preprint ar Xii::33088.01825 2023.
Xiang Yue, Xingwei Qu, Ge Zhang, Yao Fu, Wenhao Huang, Huan Sun, Yu Su, and Wenhu Chen. Mammoth: Building math generalist models through hybrid instruction tuning, In ICLR, 2024.
Wanjun Zhong, Ruixiang Cui, Yiduo Guo, Yaobo Liang, Shuai Lu, Yanlin Wang, Amin Saied, Weizhu Chen, and Nan Duan, Agieval: A human-centric benchmark for evaluating foundation models. Xiv preprint arXiv:2304.06364, 2023.
Chunting Zhou, Pengfei Liu, Puxin Xu, Srinivasan Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, et al. Lima: Less is more for alignment. In NIPS, volume 36, 2024.
Xinyu Zhu, Junjie Wang, Lin Zhang, Yuxiang Zhang, Ruyi Gan, Jiaxing Zhang, and Yujiu Yang. Solving math word problem via cooperative reasoning induced language models. ar Xiv preprint arXim:2210.16257, 2022.
14

---

Published as a conference paper at COLM 2024
A Implementation Details with Quality Aspect
In Stuser we adopt AQuA-RAT (Ling et al., 2017), Camel-Math (Li et al., 2023a), CoT (Wei et al., 2022b) and Evol-Instruct--2 (Xu et al., 2023a) as Dp. For DI, we adopt all datasets except CoT and Evol-Instruct-v2 for two reasons: (1) CoT and Evol-Instruct-v2 mainly focus on general tasks, and are not appropriate for mathematical reasoning test. (2) We aim to fully exploit the knowledge within training datasets to keep the generalization ability, and thus adopt all other datasets as test sets. In Starger we adopt all datasets on mathematical reasoning tasks and general tasks relatively as Dp for two independent scorers. For D, we adopt the CoT part of MATH (Hendrycks et al., 2021b) in MathInstruct to achieve better performance on MATH. In our experiments, we sample n1 samples from the original dataset, while sample n2 samples after K-center Greedy to guarantee the diversity of the test set D,. We set 11 as 2,000 and n2 as 100. For our scorers, we train models with the same architecture and training strategy as the fine-tuned models for 2 epochs.
B Selection Scheme of Sasse
Formally, the selection ratio r; of the ith dataset satisfies:
Ee ((low<n<<upp) me
E llownn cupp)
ni upp
ni
N:<upp
where 1(-) is the indicator function and n, denotes the size of the ith dataset. low and upp denote the lower bound and the upper bound, which controls the involved datasets in the numerator. We set low 1K. By adjusting upp, decide the datasets to perform selection. The final datasets are composed of the combination of selected data and all data of the rest unselected datasets.
C Composition of Data in Sbase
Dataset
Spase 69K
S busee 130K
Stuse 203K
AQuA-RAT (Ling et al., 2017)
8.3K
16K
69K
Camel-Math (Li et al., 2023a)
9.6K
48K
48K
College-Math (Yue et al., 2024)
1.8K
1.8K
1.8K
GSMSK (Cobbe et al., 2021)
7.4K
7.4K
7.4K
GSMSK-RFT (Yuan et al., 2023)
16K
16K
16K
MATH (Hendrycks al., 2021b)
7.4K
7.4K
7.4K
Jumber-Comparee
(Yue et al., 2024)
300
300
300
TheoremQA (Chen et al., 2023)
577
577
577
CoT (Wei et al., 2022b)
7.9K
15K
25K
Evol-Instruct-v2 (Xu et al., 2023a)
8.5K
15K
25K
Total
69K
130K
203K
Table 7: An overview of sub-settings Spase 69K, Stase 130K and Spase 203K in Spase. indicates we perform selection on these datasets.
15

---

Published as a conference paper at COLM 2024
D Composition of Data in Starge
S1prge-1.7M
Dataset
Sarge 4.3M
Surge- 5.0M
Stagg 26M
Slarge-26M
Starge-3.01 3.0M
(OpenMathMMix)
DMath (Kim al., 2023)
7.9K
7.9K
2.1K
2.1K
7.9K
GSMSK-ScRel (Yuan et 2023)
7.4K
7.4K
6.0K
6.0K
7.4K
Lila-OOD (Mishra al, 2022)
19K
19K
4.0K
4,0K
19K
Mathinstruct (Yue et al., 2024)
262K
262K
107K
107K
262K
MetaMath (Yu al. 2024)
395K
259K
MMIQ (Liu & Yao, 2024)
2.2M
2.2M
L.0M
10M
2.2M
KPMathh Platt (Huang et al,, 2024)
1.0M
1.0M
896K
896K
1.0M
OpenMathInstrrutt- (Toshniwal et al., 2024)
255K
255K
232K
232K
255K
Open-Platyyus
2023)
24K
24K
9.8K
24K
TAL-SCQŠK-CH (Math-eval, 2023)
3.0K
3.0K
1.5K
1.5K
3.0K
TAL-SCQQ5KEN (Math-eval, 2023)
3.0K
3.0K
1.6K
1.6K
3.0K
Alpaca (Taori al, 2023)
52K
38K
38K
Baize al., 2023b)
210K
70K
CoT (Wei et al., 2022b)
1 149K
132K
132K
Evol-Instruct-v2 (Xu et al., 2023a)
143K
70K
70K
Flan-v2 (Longpre et al. 2023)
50K
31K
Open-Assistant-1 (Köpf
2023)
10K
6.6K
6.6K
Self- Instruct (Wang et al,, 2023)
82K
46K
46K
Total
4.3M
5.0M
2.6M
3.0M
4.7M
Table 8: An overview of sub-settings Starge-43M, Strge-5.OM. 26Mgge-26M. Starge-3.0M and
SarMge4..M in Sbase- indicates we perform selection on these datasets. According to our observation, we define our optimal mixture Starge-4.7M as Optn
E Pearson Analysis of Scorers in Shase
Pearson
3.4
AQUARAT
Cand-VWt
5.0
O CaT
Camer
Evci Insrudt
2.4
222
20
2.5
1.-
1.2
1.5
14
14-
12
Rex Qutl Stow
Real Quaiity Booro
(a) Pearson analysis of Scorer-LLaMAN
(b) Pearson analysis of Sécorer-Mister
Figure 6: Pearson analysis between real quality scores and scorer quality scores. With strong correlation, our scorers are reliable to indicate the quality scores.
16

---

Published as a conference paper at COLM 2024
F
Cases by Scorers
Dataset
Instruction
Output
Score
OpenMathInstruct-11
What is the sum of all val- ues of for which the ex- y+6 is unde- pression y²-3y44
In order for the expression be defined, y² 5y 4 (y 4) 1) must equal zero. So y must be either or 1. Thus, the sum of all values of is 411= boxxd (5).
4.99
fined?
KPMath-Plus
A baker is planning to make rectangular cake using right triangle with one leg measuring 15 inches and the hy- potenuse measuring 25 inches
First, we need to find the length of the other leg of the right triangle. By the Pythagorean Theorem, we can find that the length v 15a 4000 inches. Then, we know that this leg forms the shorter side of a rectangle, and the perimeter of this rectangle is 70 inches. Therefore, the longer side of the rectangle is 70 70-2+20 = 15 inches. A rectangle with fixed perimeter has minimal area when as long as possible and the other as short as possible. As the shorter side is 20 inches and the longer side is 15 inches, the rectangle's area is 20 15 |boxed (300} square units.
4.99
guide. The other leg the triangle is used as the shorter side of the cake, If the total length of the cake's edge is 70 inches, what is the smallest possible area of the cake?
dimension is
Lila-OOD
Com- 2 pute the median of
Problem:
1.00
{ (-4/5,2,
64 17
8, 3
7 2
12
7,
75
DMath
Find the area of rect- angular wall whose sides are meters (m) long and 17/20 meters (m) wide.
= var 0.85
1.02
var var var.b htte:///ffffommtttuuddyyrrrr 10,2)))
Table 9; An overview of two samples with high scores and two samples with low scores by Soore--DepSSee--RRsooninn
Samples with complete or complicated reasoning process are scored high, and those with with little reasoning-relle related process or code-like answers are scored low.
17

---

Published as a conference paper at COLM 2024
Dataset
Instruction
Output
Score
CoT
Is it normal to red panda in Shanghai out- side of a zoo? Please an- swer step by step:
The red panda is endemic to the temper- ate forests of the Himalayas, and ranges from the foothills of western Nepal to China in the east. The red panda lives between 2,200 and 4,800 (7,200 and 15,700 ft) altitude, inhabiting areas of moderate temperature between 10 and 25 "C (50 and 77 PF) with little annual change. Shanghai is located on an al- luvial plain, as such the vast majority of its land area is flat, with average elevation of 4 m (13 ft). So, The answer is no.
5.00
Alpaca
Find the number that is non-prime and non- composite between 10.
The number that is non-prime and non- composite between 1 and 10 is 6.
4.48
Baize
how define friend func- tion and operator over-
The conversation between human and AI assistant.
1.02
loading in
Human-] Can you tell how define a friend function C++?
[-AI]] Sure, friend function is function that declared
friend of class and can access the private and pro- tected members of the class. To define friend function, you need declare it inside the class definition using the friend keyword and then define it out- side the class definition like a normal function.
[-l That makes sense. And how about operator overloading?
[
Evol-Instruct-v2
Please provide a [Java code] implementation that demonstrates the efficiency capitalism in resource allocation by comparing it with other economic systems, such as socialism and commu- nism. Additionally, show how the principles of sup- ply and demand can be modeled mathematically to maximize profits and promote economic growth in capitalist system.
I'm sorry, but as an Al assistant, can- not provide a Java code implementa- tion that demonstrates the efficiency capitalism in resource allocation by comparing it with other economic sys- tems, such socialism and commu- nism. This topic highly debated and complex, and it involves various eco- nomic, political, and social factors that cannot fully captured plementation. However, I can provide some general information how the principles of supply and demand can be modeled mathematically to maximize profits and promote economic growth in a capitalist system. In a capitalist sys- tem,
1.01
code im-
Table 10:
An overview of two samples with high scores and two samples with low scores by -enerall. Samples with complete reasoning process or math-related answers are scored high, and those with little reasoning-relater process are scored low.
18


Reviews:
[{"review_id": "Qc1DdoGJms", "reviewer": "Reviewer_rr87", "summary": "This paper aims to propose the data curation method which identifies highly influential data within a large-scale mathematical reasoning dataset.\n\n### Quality\n\nThe paper is generally clear and effectively communicates the problem it addresses. However, it lacks a crucial discussion on the computational costs involved in curating the dataset using the proposed method.\n\n### Clarity\n\nThe paper is well-written, but some sections may be challenging to grasp on first reading.\n\n### Originality\n\nThe application of the K-center Greedy method is not particularly novel, as it adapts an existing approach to a new problem. Nonetheless, the introduction of a quality scoring system using a zero-shot and one-shot inference appears to be a novel contribution.\n\n### Significance\n\nThe approach to curating the training dataset is significant. This paper addresses the important issue of fine-tuning language models for reasoning datasets.", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer rr87,\n\nThank you for your comprehensive and meticulous review! We are pleased to address your points below.\n\n> It is not clear which dataset is referred to in Table 3. \n\nWe use 69K selected data from *S_{base}* in this table. A detailed composition can be found in paper \"Appendix Table 6\". We will add a clearer summary in future updates.\n\n> The explanation of the scoring method in Figure 1 is confusing.\n\nThe bi-directional arrows indicate the comparison between one-shot scores and corresponding zero-shot scores, showing the process of paper \"Equation (3)\". Also, we will polish the figure and make it clearer.\n\n> The paper does not adequately address the computational costs.\n\nWe take our *S_{base}*-69K setting as an example (from 428K all data to select 69K data). We will show our computational costs on LLaMA-2 (for there is no obvious gap between LLaMA-2 and Mistral) from two aspects:\n\n* Diversity costs: It takes about **4.43** gpu hours to compute 428K data embeddings. The K-center Greedy process is implemented on CPU and there are no GPU costs.\n\n* Quality costs: we only use a part of representative samples to perform one-shot inference, i.e. n1=2,000 and n2=100 in our paper \"Appendix Section A\". In this case, the computation of zero-shot and one-shot inference takes about **0.02** and **8.42** gpu hours per dataset. After that, we train a scorer and infer the 428K scores taking **2.00** and **3.32** gpu hours respectively.\n\nWe also compute the costs of another popular strategy Deita from our implement and their paper. Deita contains the process of obtaining embeddings, training quality and complexity scorers, and inferring scores. We  show the gpu hours and overall accuracy in the table below.\n\n| Strategy | Diversity Costs | One-shot Costs | Scorer Training Costs | Scoring Costs | Total  Costs | Overall Accuracy |\n| :------: | :-------------: | :------------: | :-------------------: | :-----------: | :----------: | :--------------: |\n|  Deita   |      4.43       |       -        |        > 4.00         |     6.62      |   > 15.05    |       28.9       |\n| **QaDS** |      4.43       |      8.44      |         2.00          |     3.32      |    18.19     |       31.3       |\n\nIt turns out that our costs are close to Deita. Moreover, once we finish the training of our scorer, the most expensive process, i.e. the one-shot inference, is not used. We can perform fast scoring on different reasoning tasks fully based on our scorer.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- **Thorough analysis:** This paper introduces a strategy for selecting data for a mathematical reasoning dataset. It applies this method to two different dataset sizes, S_{base} and S_{large}, demonstrating that the proposed method outperforms existing baselines. Additionally, it provides insight into the relationship between estimated and actual quality scores, showing a strong correlation with the actual scores.", "reasons_to_reject": "- **Needs clearer writing**: Some parts of the paper are difficult to follow. For example, it is not clear which dataset is referred to in Table 3. The explanation of the scoring method in Figure 1, particularly the bi-directional arrows between the one-shot and zero-shot score blocks, is also confusing.\n\n- **Insufficient discussion on computational costs**: The paper does not adequately address the computational costs required to embed data for diversity measurement and computing zero-shot and one-shot scores using LLMs like LLaMA-2 and Mistral. The cost implications of these computational demands, especially when seeking a modest accuracy improvement of 2% as shown in Table 4, need further discussion.", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2N2ylQTgOP", "reviewer": "Reviewer_oJ7q", "summary": "__Summary__\n\n1. Propose a quality-aware diverse selection approach for mathematical reasoning. This approach aims to integrate both quality and diversity in selecting influential data.\n2. Diversity: sampling by maximizing the minimum distance between the data points, which are encoding by embeddings. \n3. Quality: the average times that one-shot performance is better than the zero-shot performamce.\n     1. They further train a quality scorer for that to reduce the intractable computational cost.\n4. Then they can select the instance by the QS score.\n5. Construct two settings $S_{base}$ and $S_{large}$  and different selection scheme to verify the effectiveness of QaDS, try to better demonstrate the effectiveness of QaDS", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer oJ7q,\n\nThank you for your thoughtful review! We are pleased to provide more information to your confusion.\n\n> Only the \"General\" is selected by QaDS.\n\nFirstly, we do perform selection on **both reasoning and general** data in *S_{base}*, and we achieve the best results as shown in paper \"Figure 2\".\n\nWhen coming to *S_{large}*, we aim to **give some insights on a wide range of data sources**:\n\n* **Scaling up reasoning data is helpful**: In mathematical reasoning, a common law is that mathematical reasoning consistently improves with increasing data amount [1]. And our comparison between QaDS-*S_{large}*-4.3M and 2.6M in paper \"Table 4\" also validates it.\n* **General data can also be influential for reasoning**: According to [1] and [2], other relevant tasks can also be helpful for reasoning. So we contain general data in our setting, and successfully selects the helpful general data by QaDS.\n\nTherefore, we contain **all reasoning data and a part of influential general data selected by QaDS** as our OpenMathMix.\n\n> Concerning about what's really improving.\n\nOur improvement comes from two parts:\n\n* The direct improving mathematical ability by more mathematical reasoning data.\n* The indirect improving reasoning-related ability by influential general data. We show cases in paper \"Appendix Table 9 and 10\".\n\n> Table 4 only shows the performance of one dataset.\n\nWe provide our reason in paper \"Section Evaluation Datasets\": we adopt a subset of **MATH as *D_{t}* when computing quality scores** in *S_{large}* , and thus choose the **corresponding test set of** **MATH for evaluation**. Also, taking into account your careful comment, we evaluate on other 6 tasks and achieve best results:\n\n|         Model         | Overall  |   MATH   |   Num.   |  Math.   |   Sim.   |   SAT.   |  MMLU.   |\n| :-------------------: | :------: | :------: | :------: | :------: | :------: | :------: | :------: |\n|    InternLM2-Math     |   35.5   |   34.6   |   43.2   |   19.9   |   25.5   |   49.5   |   40.5   |\n| DeepSeekMath-Instruct |   53.1   |   46.8   |   57.7   |   42.0   |   43.6   |   67.7   |   60.5   |\n| **QaDS-OpenMathMix**  | **55.6** | **48.8** | **67.3** | **42.2** | **46.3** | **68.2** | **60.7** |\n\n[1] How abilities in large language models are affected by supervised fine-tuning data composition. *arXiv preprint arXiv:2310.05492* (2023).\n\n[2] Specialist or generalist? Instruction tuning for specific NLP tasks. *arXiv preprint arXiv:2310.15326* (2023).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. Motivation is clear and reasonable to incorporate both diversity and quality. The underlying approach is also solid.\n2. Experiments demonstrate improvements with QaDS selection, also better performance compared to DeepSeek, which is the base model.", "reasons_to_reject": "1. In Table 4, althouh QaDS-$S_{large}-4.7M$ achieves the best performance $\\mathbf{48.8}$, but according to Table 2, this OpenMathMix set contains all data for mathematical reasoning, only the \"General\" is selected by QaDS. Thus, it is confusing that the motivation is to improve mathematical reasoning, but we actually use all the corpus there.\n      1. And Table 4 only shows the performance of one dataset.\n      2. Although QaDS-$S_{large}-5M$ is worse, but again, we only perform QaDS on \"General\" according to Table 2, readers will be concerning about what's really improving here.", "questions_to_authors": "Refer to \"Reasons to Reject\".", "extra_scores": {}}, {"review_id": "rbIgzO56ri", "reviewer": "Reviewer_YKSQ", "summary": "This paper focuses on selecting influential training data for improving LLMs' mathematical reasoning ability. The authors propose a quality-aware diverse selection strategy (QaDS) over other selection strategies and also define an OpenMathMix which is a data mixture selected by QaDS. The experiments show advantages of the proposed strategy and the paper is comprehensive.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer YKSQ,\n\nThanks for your valuable comments of our paper! We are pleased to share our ideas on your questions.\n\n> Lack of further analysis of the selection strategy, e.g., more than one-shot.\n\nWe provide analysis from two aspects to your concern:\n\n* Existing strategies mainly consider the data quality from some direct aspects such as input length, fluency, naturalness and so on, while **not fit for reasoning**. Therefore, we found that they do not perform well on reasoning tasks as shown in paper \"Figure 2\".\n* The quality of resoning data is not easy to measure, and our solution is to measure **the positive influence on each other**: if there exists a right and comprehensive reasoning process in a certain sample, it should be positive to another sample, leading to the \"one-shot inference\". The theoretical support is the inherent duality between attention and gradient descent, and can be found in [1].\n\nWe are the first to explore data selection for reasoning, and we hope we can provide insights to the community. We will explore more strategies in our future work.\n\n> Could this strategy be applied to other tasks than the mathematical reasoning?\n\nYes! Although we focus on the challenging mathematical reasoning task, we also conduct experiments on two general tasks taking into account your question.\n\nWe first adopt 2000 samples of Alpaca and Dolly datasets as *D_{p}*, and 500 mixing samples of Alpaca and Dolly as *D_{t}* to perform one-shot inference. For the gap between reasoning and general tasks, we then train a new scorer for general tasks with above data. The scoring and selection process is the same as we mentioned in our paper. Finally, we fine-tune LLaMA-2 with our selected data (6.6K from 66K) by QaDS. Also, we implement IFD and Deita to select the same size 6.6K of data. We show results below:\n\n| Strategy | MT-Bench | AlpacaEval (%) |\n| :------: | :------: | :------------: |\n|   IFD    |   5.39   |      56.6      |\n|  Deita   |   5.23   |      49.8      |\n| **QaDS** | **5.49** |    **72.6**    |\n\nIt turns out that our QaDS also achieves the best results, showing the effectiveness of our strategy.\n\n[1] Why can gpt learn in-context? language models implicitly perform gradient descent as meta-optimizers. *arXiv preprint arXiv:2212.10559* (2022).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "* Proposed a reasonable way of selecting high-quality data (one-shot better than zero-shot)\n* Define an influential data mixture with open-source data selected by QaDS", "reasons_to_reject": "* Lack of further analysis of the selection strategy, e.g., more than one-shot", "questions_to_authors": "Could this strategy be applied to other tasks than the mathematical reasoning?", "extra_scores": {}}, {"review_id": "RDiDKVGROv", "reviewer": "Reviewer_gRx8", "summary": "This paper mainly studies data selection for supervised instruction tuning of LLMs, targeted at math reasoning. Specifically, it follows existing data selection work that selects supervised tuning data based on the combination of its diversity and quality. Experiments on large-scale open-source data show that the proposed data selection approaches outperform (not by a big margin) both random selection and other baseline data selection approaches.", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer gRx8,\n\nThank you for your thoughtful feedback! We are pleased to share our ideas to your concerns.\n\n> No clear motivation for math.\n\nWe emphasize our motivation below:\n\n* Math reasoning task is a **more challenging** task compared to general tasks. We found that **existing strategies do not perform well on reasoning tasks** in paper \"Figure 2\", and thus focus on math reasoning.\n* If there exists a right and comprehensive **reasoning process** in a certain sample, it should be positive to another sample. We'd like to use the positive influence of reasoning data, leading to the \"one-shot inference\". The theoretical support is the inherent duality between attention and gradient descent [1].\n\nAlso, we conduct experiments on two **general tasks** taking into account your question. Please see the table in our rebuttal for **Reviewer YKSQ**.\n\n> Whether different combinations could lead to improvements.\n\nWe perform ablation of the two aspects of QaDS in paper \"Table 3\", and we order it more clearly below:\n\n|             Strategy             | LLaMA-2 Overall | Mistral Overall |\n| :------------------------------: | :-------------: | :-------------: |\n|              Random              |      29.5       |      32.2       |\n| Only Diversity (K-center Greedy) |      30.4       |      32.4       |\n|   Only Quality (Quality Score)   |      30.2       |      33.0       |\n| Diversity and Quality (**QaDS**) |    **31.3**     |    **33.9**     |\n\n> Gains might be overshadowed by LLAMA 70B.\n\nFirstly, we conduct experiments on LLaMA-2 70B considering your insightful comment: \n\n| Strategy | Overall  | GSM.     |   MATH   | AQuA     |   Num.   | SVA.     |  Math.   |   Sim.   |   SAT.   |  MMLU.   |\n| :------: | :------: | -------- | :------: | -------- | :------: | -------- | :------: | :------: | :------: | :------: |\n|   IFD    |   43.3   | 69.4     |   18.0   | 42.7     |   47.9   | 75.8     |   19.7   |   23.5   |   44.1   |   48.6   |\n|  Deita   |   42.7   | 68.2     |   18.0   | 41.3     | **52.1** | 75.7     | **21.1** |   22.4   | **45.1** |   40.5   |\n| **QaDS** | **44.7** | **71.0** | **18.2** | **46.9** |   48.0   | **77.9** |   20.1   | **29.0** |   42.3   | **48.9** |\n\nThen, we also emphasize that we reach an improvement of 6.6% in paper \"Figure 3\" (48.8 vs 42.2) in *S_{large}*-4.7M. \n\n[1] Why can gpt learn in-context? language models implicitly perform gradient descent as meta-optimizers. *arXiv preprint arXiv:2212.10559* (2022).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The experiment on math reasoning is solid, as it evaluates the proposed data selection from multiple perspectives.\n2. The paper is relatively easy to understand.", "reasons_to_reject": "1. My primary concern is the motivation behind the paper, specifically the research question addressing the math reasoning task. While the paper focuses on math reasoning, the proposed method appears universally applicable to various tasks. After a thorough review, I found no clear motivation or adaptation of the method specifically for the challenges in math reasoning. An interesting idea might involve a compositional aspect, such as whether different combinations of formulas could lead to improvements.\n\n2. The reported improvements in results are quite marginal. It is unclear what a ~1.5-2% improvement in math reasoning signifies in this context. Such modest gains might be overshadowed by the capabilities of a base model, such as LLAMA 70B. Consequently, the effectiveness of the proposed approach remains ambiguous.", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper explores the selection of influential data for mathematical reasoning tasks, proposing a Quality-aware Diverse Selection (QaDS) strategy and constructing OpenMathMix, a mixture of influential data selected by QaDS.

Strengths:
1. The quality scoring is novel and could contribute to the field of data curation for mathematical reasoning (Reviewer rr87, Reviewer oJ7q).
2. The paper is well motivated -- incorporating both diversity and quality in data selection is important for reasoning tasks (Reviewer rr87, Reviewer oJ7q).

Weaknesses:
1. Generalizability (or task uniqueness): It is unclear how the method addresses any specific challenges in math reasoning, as the challenge and the approach seems quite sharable with various more general tasks (Reviewer gRx8). Concurrently, Reviewer YKSQ asked whether the approach could be "applied to other tasks than the mathematical reasoning"
2. Marginal Improvements: The reported improvements in results does not seem too significant (Reviewer gRx8).
3. Reviewers also asked for more discussions on computational costs in curating the dataset using the proposed method (Reviewer rr87).

The authors provided additional experiments on general tasks, and provided more discussions on the method efficiency and writing clarity. Unfortunately, in a reviewer discussion thread, gRx8 mentioned that they still felt that "there is no strong motivation and findings for math reasoning". That said, given that the method can generalize to more non-math tasks, I think the paper could still be beneficial if the authors could explicitly discuss the generalizability potential in a revised paper.
Decision: Accept