Transferring Reasoning Capabilities between LLMs operating via Curriculum Learning Policy
Authors: Leonardo Ranaldi
Giulia Pucci
Fabio Massimo Zanzotto
Venue: Accepted by TMLR ()

Abstract: In-context reasoning methods, exemplified by Chain-of-Thought (CoT) (et alia.,) empower the reasoning abilities of large language models (LLMs), eliciting them to solve complex reasoning tasks step-by-step. Nevertheless, the capacities to deliver robust CoT explanations arise only in models with billions of parameters, representing a barrier to entry for many users forced to operate on a smaller model scale, i.e., Small Language Models (SLMs). Even though many companies are releasing LLMs of the same family with a reduced number of parameters, these models sometimes produce misleading answers and are unable to deliver accurate step-wise reasoned answers. This paper proposes a method to transfer step-wise reasoning over SLMs by operating via Instruction-tuning (IT) on synthetic demonstrations delivered in a pedagogically motivated manner. In particular, firstly, we propose aligning step-wise reasoning capabilities via IT using Demonstrations "taught" by LLMs teacher to SLMs students. Then, we operate via Curriculum Learning, a pedagogically motivated learning method that improves the IT phase. We analyse the impact on the downstream performances of four question-answering benchmarks. The results show that SMLs can be instructed to reason via Demonstrations delivered by LLMs. We move a step further
in research: conceiving SLMs as human learners, we expose them to a CL teaching-based approach, obtaining better results on downstream performances.

Published in Transactions on Machine Learning Research (09/2025)
Transferring Reasoning Capabilities between LLMs operating via Curriculum Learning Policy
Leonardo Ranaldi
name.suraaeeBedd.ccc
University of Edinburgh
University of Rome Tor Vergata
Giulia Pucci
namee.surammeeBadnnnccc
University of Aberdeen
Fabio Massimo Zanzotto Universily of Rome Tor Vergata
name.surname@@uniroe...
Reviewed on OpenReview: https: oppnrrveepnn net/ form?? id= jmyay
Abstract
In-context reasoning methods, exemplified by Chain-of-Thoughh (CoT) (et alia.,) empower the reasoning abilities of large language models (LLMs), eliciting them to solve complex reasoning tasks step-by-step. Nevertheless, the capacitics to deliver robust CoT explanations arise only in models with billions of parameters, representing a barrier to entry for many users forced to operate on a smaller model scale, i.e. Small Language Models (SLMs). Even though many companics are releasing LLMs of the same family with a reduced number of parameters, these models sometimes produce misleading answers and are unable deliver accurate step-wise reasoned answers. This paper proposes method to transfer step-wise reasoning over SLMs by operating via Instruction-tuning (IT) synthetic demonstrations delivered in a pedagogically motivated manner. In particular, firstly, we propose aligning step-wise reasoning capabilities via IT using Demonstrations "taught" by LLMs teacher to SLMs students. Then, we operate via Curriculum Learning, a pedagogically motivated learning method that improves the IT phase. We analyse the impact on the reanstream performances of four question-answering benchmarks. The results show that SMLs can be instructed to reason via Demonstrations delivered by LLMs. We move step further in research: conceiving SLMs as human learners, expose them to CL teaching-besee approach, obtaining better results on downstreem performances.
Introduction
In-context reasoning methods, exemplified by Chain-ofFThought (CoT) (et alia..) empower the reasoning abilitics of large language models (LLMs), eliciting them to solve complex reasoning tasks step-by-step. These methods enable large language models (LLMs) deliver multi-step, controlled reasoning Kojima ct al. 2023); Wei et al. (2022), achieving outstanding results in commonsense, symbolic, and mathematical reasoning lasks. LLMs achieve all these results with least several billions of parameters, such as GPTs
OpenAl 2024), Llamas Touvron et al. (2023); Grrataafori et al. (2024) and Mistral MistralA] 2023
In contrast, Small Language Models (SLMs) break down the problems and deliver step-wise answers less effectively. Although these models are highly functional across different lasks, the CoT mechanism consis- tently benefitted only models at a certain threshold scale or through costly and tine-consuming post-training SLMs are crucial fostering research since these are smaller versions of LLMs that are often open-source and accessible most researchers, e.g., Llama-3-11 and Llama-3-8b Grattafiori al. (2024). However, these SLMs produce illogical answers when prompted under the CoT framework.

---

Published in Transactions on Machine Learning Research (09/2025)
We propose approach to align the reasoning abilities of the SMLs (students) with the LLMs (teachers) via Instruction-tuning-Co (TT-CoT), that is, an instruction tuning over synthetic CoT Demonstrations
delivered from larger models (Figure I following teacher-student pipeline Ranaldi & Freitas (2024a).
Concerning the foundation teacher-student approach Magister et al. (0233] we move a step further by introducing the IT via CoT, and, concerning Ranaldi Freitas (2024a): we improve the strategies to expose
order Ranaldi al. (2023), adapting Curriculum Learning Bengio et al.] (2009). Starting from the idea that humans acquire first elemental concepts and then, gradually, more complex ones, Bengio et al. 2009)) proposed Curriculum Learning (CL) and demonstrated its benefits in several tasks Ranaldi et al. (2024). We adopt this idea to reorder the IT Demonstrations in meaningful way. Hence, we evaluate the reasoning chains that are answers delivered by teachers via CoT prompting to elicit student learning.
the student to examples in reasonable, pedlagogically
This leads the target research questions, which are the focus of this paper: RQ1 How does IT-CoT via synthelic Demonstrations impact the reasoning abilities of students' models? And what is the effect of step- wise Demonstrations delivered with the CoT process? RQ2 How important is reasoning choin valuation to facilitate the presentation of demonstrations during IT?
Step1
construction
Step2 instruction-tuning
QA task
demonstrations
LLM
LLM
Curriculum Learning
Teacher model
Student model
LLM
LLM
Instructed Student
In Instruction-tuningg the smaller models instruct themselves using the synthetic demonstrations generated by the larger models. Hence, we clicit a larger model to deliver step-wise reasoned answer solutions. Moreover, we evaluate the reasoning chain using Curriculum Learning metrics to facilitate the instruction phase and expose the Demonstrations a mcaningful way.
Figure 1:
We answer these questions by selecting smaller LLMs (Llama-3-11) 1b, -8b as students and larger LLMs (Llama- 3-70b, and GPT-4) as teachers. We perform comprehensive analysis using four marks. We use Llama-3-70 and GPT-4 to deliver Answers at the core the Demonstrations (sce Figure 1 to instruct Llama-3-1 and -13. Furthermore, we evaluate the complexity of the reasoning chains in generated answers to expose the students to Demonstrations delivered by teachers. Hence, we propose metric based informativeness comprchensibility used a pivot in the IT phasc.
bench-
Behind wide analysis, we show that the IT and IT-CoT approaches Demonstrations instruct students, and they outperform baseline SLMs in all proposed benchmarks. In addition, the students exposed to the Demonstrations via the CL approach outperformed students instructed via non-CL.
Our findings can be summarized as follows:
1) The IT' of SLM students via Demonstrations delivered by an LLM teacher outperformed the baselines in terms of downstream performance. The SLMs instructed via Demonstrations consistently outperformed the baselines defined by non-tuned SLMs on the four proposed question benchmarks.
ii) The CL-based /T' approach outperforms standard IT. Llama-3-1 and Llama-3-8, instructed via the CL method, outperform the instructed models without CL.
iii) Finally, the CL method favours the alignment of CoT abilitics within the family. Llama-3-1 and Llama- 3-8 were exposed CL Demonstrations produced by Llama-3-70 which outperformed students instructed by GPT-4 teachers in other SMLs well.
2

---

Published in Transactions on Machine Learning Research (09/2025)
2
Method
We propose three steps to align the reasoning abilities of smaller Language Models using further knowledge generated by larger Language Models, as shown in Figure In the first part, there is an annotation phase where the large language models (LLMs) systematically prompl generale outputs (Section 2.1j. The outputs will be the core of Demonstrations used during the Instruction-tuning IT phase from the smaller Language Models, presented in Section 2.2 However, the Curriculum Learning approach is behind the IT phase, where the Demonsttratons are reorganized following our measure introduced in Section 2.3
2.1 Teacher Model
As leacher model, we selected the largest Llama version Graitafiori el al. 2022)) that is, Llama-3-70b, and in terms of comparison, SPT--0ppentl 2024h. We selected GPTT4 because it generates high-quality data with and without the CoT prompting approach. Mcanwhile, Llama-3-70t because it has smaller versions that can be used as students of the same family (presented in Section 2.2 and these smaller versions obtain remarkable results despite the reduced number of parameters.
Hence, we proposed the following prompt in zero-shot scenario:
Choose the answer to the question only from options A, B, C,
Question: <Question>
Choices:
A) <0ptioni>
B) <0ption2>
C) <0ption3>
D) <0ption4>
Answer:
Let's think atep by step
Input prompts have a generic structure, but behind 'Answer:" " we insert the formula 'Let's 's think step by step' that is shown in Table 7
Following the annotation process performed by LLMs, the answers generated by teachers models that are the annotations have been used to construct the Demonstrations (sec Table 1
2.2 Student Model
Several SLMs have been fine-tuned for instruction-follovingg Taori et al. (2023 and reinforcement learning with human feedback. However, whatever the techniques, the smaller Language Models do not seem able to break down a given task and deliver step-wise solution.
Recent work proposes techniques of knowledge distillation and skill-refinmment to teach SLMs to produce reasoned answers Ranaldi & Freitas (2024b). Following this idea, we analysed the impact of synthetic from Demonstrations provided by different teachers. Hence, we chose two SLMs students, Llama-3-1b and Llama-3-8b. In both cases, used "instruct' versions, i.c., Tlama-3-1-chat and -lama-3---instruet (versions detailed in Table 8 which will refer to in the paper Llama-3-1 and -8.
Student models are evaluated before and after the IT phases. In particular, we propose the standard approach where the students are instructed via Demonstrations delivered arbitrarily and in the Curriculum Learning manner (introduced Section 2.3 The structure IT arises from Alpaca Taori et al. 2023)) which operates via Demonstrattins consisting an instruction that, our case, fixed, i.e., Choose the answer the question only from options A, B, C, D., an input which is the question, and an expected output which, in our case, are the output generated by the LLMss teachers. Table I shows example of input. Additional details about the IT steps provided in Section 3.2.1
We GPT GPT-b,turb,, however in the rest work will use only GPT-4
3

---

Published in Transactions on Machine Learning Research (09/2025)
Instruction
Choose the answer to the question only from options A, B, C, D.
Input
Question:
Which form can be made into triangle?
Choices:
A) ice
B) steam
C) water
D) salt water
Output (that is the Anawer produced by teacher)
Answer:
A) ice: Ice is a solid form of water. In solid form, substances can molded shaped into various forms, including triangles. If you had block of ice, you could carve or mold it into a triangular shape. B) steam: Steam is the gaseous form of water. Being gas, it doesn't have a fixed shape. You can't shape steam into triangle without then, the
containing it within
form of solid boundary, and steam itself isn't triangular the container is.
C) water: Water is liquid that takes the shape of container. You can pour water into a triangular container, and while the volume inside would triangular, the water itself isn't inherently triangular.
D) salt water: This is essentially the same as option C, but with salt dissolved in it. Being liquid, it takes the shape of its container.
Given these considerations, the only substance among the options that be ahaped into triangle by itself is A) Ice.
Table 1: Example of Demonstration. The structure is composed by: Instruction, Input and Dutput. In this case, the last part is the Answer produced by GPT-4 CoT prompted.
2.3 Curriculum Learning
Aligning the teacher-student reasoning abilities via synthetic Demonstrations delivered by step-wise answers provided by teachers, CoT prompted is promising tochnique. However, some aspects need clarification: what constitutes an answer containing a good reasoning chain and how evaluate it to optimize the IT phase. Following the Curriculum Learning (CI.), where training algorithms training data presented according to the model's current skills, [Bnggio et al. (2009). We propose method for evaluating the reasoning chain present in the CoT Answers (that represent the outputs of CoT Demonstration) using two fundamental properties: (1) comprehensibiiityy that is, the comprehensibility of a text according to metrics proposed by Talburt (1986), and (2) informativencss, that is, every step of the chain provides new information that is useful and informative for deriving the generated answer. We apply this metric the CoT Answers provided by the teachers; then, we reorder the demonstrations according our measure.
achieve better results when
Informativeness
To quantify the effectiveness of cach step contributing novel information beneficial for deriving the final Answer A, we propose an assessment based on the Entropy and Information Gain (IG). The Entropy, represented by (S), evaluates the unexpected within given sequence S, where S; E A. The
4

---

Published in Transactions on Machine Learning Research (09/2025)
entropy is given by:
H(S)= - Σp(w) p(w) log, p(w)
(1)
WES
where p(w) denotes the probability of token occurring in the sequence. Hence, we compute the IG between a previous Spree and current sequence S; as:
[G S.) (S + St) H (Spree)
(2)
This metric quantifics how much new information the current stop adds relative to the cumulative content previously considered. To obtain a comprehensive measure, we calculate the average IG across the different steps follows:
1 IG(Sprev S:)
dr(A.)
Σ N 1=1
(3)
where N represents the total number of steps in the Answer the sequences S.. We calculate this value for each answer A; and obtain the maximun dimar and the minimum dimn scores, Finally, normalize these values:
(A,) = d((A.) dlmen E [0,
(4)
dimcs dmmi
where [D] are all answers to specific benchmark.
Comprehensibility Typical factors for measuring comprehensibility are Speed of perception, Perceivabil- ity in peripheral vision, Reflex blink technique, Eye movements, Cognitively motivated features, and Word difficulty. However, it is not always possible to capture all these features.
Hence, we used the Flesch- Kincaid metric Talburt (1886). This metric is used to assess the comprehensibility of text. Tt is based the length of sentences and words within text and provides score that indicates the text's difficulty level. The lower the score, the easier it is to read and comprehend the text. The formula for calculating the Flesch-Kinens Grade Level score is as follows:
(A,) = 0.39 AA)(dr(A.)) + 11.8
Aug(dr(w:)) 15.59
100
100
where Avg(dr(A.) average answer length is the number of words in sentence divided by the mumber of sentences, and Aug(dz (w:) is the average word length, i.e. is the number of syllables per word divided by the number of words. The value 0.39 is used to scale the effect of the average sentence length to compare it to the effect the average word length, weighted by 11.8. The final score is then adjusted by subtracting the value of 15.59, which adjusts the score scale match the grading levels used in education more closely. We calculate this value for each Answer A, and obtain the maximum dcmos and the minimum dem.. scores. Finally, we normalize these values:
(A do(Ai) dCmin Vi E [0,
(6)
demur dCm.n
Constructing the CL-Demonstration We gather the annotations (answers) delivered by the CoT- prompted teachers (as explained in Section 2.1,, and we estimate the informativeness d, (A,) and complexity (A,) for each answer A,, Vi E ID]. Then we merge the two values in:
- d((A)) do(,))
We use (Ac(A:) as a pivol value to reorder the Answers provided by the teachers. The Answers (which form the output the Demonstrations) will be delivered in the Instruction-tuning phase to the students in ascending order with respect the value dAc(A,). These heuristics are very lightweight: using only 16GB of memory, we can process up to 20k Responses per second to produce the informativeness and comprehensibility metrics.
5

---

Published in Transactions on Machine Learning Research (09/2025)
3 Experimental Setup
To make the experiments comparable with state-of-thh-art models, use four benchmarks (introduced in Section 3.1) that are generally used to assess the abilities of Large Language Models (LLMs). Moreover, to conduct the Instruction-tuningg phase on the Small Language Models (SMTs), we use two approaches: the first one is presented in Section 3.2 which we call Instruction-tuning on Demonstrations the second is based on the Curriculum Learning (CL) approach where the students are exposed CL-Demonstrations that are Demoustrations reordered in a CL way, as exemplified in Section 2.3 All code is available in the supplementary material, to be released if accepted.
3.1 Data
General Commonsense Reasoning We evaluate the models' ability perform general reasoning on the CommonSenseQA Talmor al. (2019) (CSQA) and OpenBookQA Mihaylov et al. (2018) (OBQA). CommonSenseQA of the best-known datasets of answers to multiple-choice questions dealing with different types of general commonsense knowledge. OpenBookQA is resource that contains questions requiring multi-step reasoning, common knowledge, and rich text comprehension. It is inspired by high school-level open-book exams in physics and biology, aiming to assess human comprehension and application of foundational concepts.
Physical Interaction Reasoning We evaluate the models' ability to perform physical reasoning on the Interaction Question Answering (PIQA) Bisk al. (2019). It a resource consisting of everyday situations with typical and atypical solutions.
Social Interaction Reasoning We evaluate the models' ability to perform social reasoning the Social Interaction Question Answering (SIQA) Sap et al. (2019) It is benchmark focusing reasoning about people's actions and social implications. The actions in Social TQa. cover various social situations and candidates for plausible and not plausible answers.
Splitting Details Since a test split. for all benchmarks not always available open-source, we adopt the following strategy: we use 4000 examples with equally distributed target classes as training data and the validation versions found on hugginglace as lest. data. We performed this split to observe the impact of the responses provided by the teacher models on different benchmarks. The same is true for validation, since needed open-source and reproducible data to conduct detailed evaluation of the student models. In Table g report the quantitative information, global, and splitting ratios. The data are fully accessible and open-source, as in Table 10.
3.2 Teaching to Reason
We selected Llama-3-70 and GPT-4 as the teachers (introduced in Section 2.1 Consequently, the LLMs are prompted in the zero-shot scenarios, shown in Table 6 and Table 7
We selected Llama-3-1 and Llama-3-8 Grattafiori et al. (2024) as student models (as described in Section
2.2 Therefore, the students models are Instruction-tumed via Demonstrations, as introduced in Section
3.2 and via CL-Demonstrations, as explained in Section 2.3 Table [ shows a Demonstration containing
the Instruction, Input, and, as Output, the Answer-delivering CoT, an output generated by GPT-4 CoT- prompted.
3.2.1 Models Setup
We conduct the Instruction-tuning phases using QLoRA proposed by Dettmers el al. 20233 This approach allows tuning to be conducted while reducing memory usage and preserving the performance. We follow the training approach proposed in Alpaca Taori et al. (2023) and trainoed the models for epochs and set the learning rate to 0.00002 with 0.001 weight decay. We conducted our experiments on a workstation equipped with two Nvidia RTX A6000 with 48GB of VRAM.

---

Published in Transactions on Machine Learning Research (09/2025)
OpenBook
CommonSense QA
GMM
0664
Intrctionsa NO
Foncter Ravin
Baseline
Ixtt Adio TuRin
Intrucday Buring Deviriottionss
Physical Interaction QA
Social Interaction QA
Docel
berudientttig deremutationn
betrudlav-uuing Demoviralaai
Lleme-3-1o (Ileme-3-Tob)
Liame-3-1h (GPT-1)
3-10
(caf]
Liom 3 8b [Uema- 3 706)
Lama 3-8b (SPT-4)
Llema-1-Bo
Lama 3-70 (CoT)
Figure 2: Accuracios (%) on benchmarks (Section 3.1 before Instruction-tuiing (i.c., Baselines) and after on Demonstrations and CI-Demonstrations. Moreover, there are the teachers' performances also shown Table
3.3
Evaluation
The most commonly used evaluation methods for question-answering tasks are language-mode probing, in which the option with the highest probability is chosen Brown et al. (2020). and multiple-choice probing, in which the models are asked to answer. The evaluation is performed with a function taking the maximum value and, the second case, with string matching. The second method is widely used recent evaluations because it applies to models such as GPT-4 and GPT-4 OpenAI (2024) where probability values cannot be accessed. In experiments, chose the latter to have comparable and scalable pipeline. Hence, performed a string matching between the generated outputs and the targets.
4 Results
Language Models that do not get it can be elicited to do it through the knowledge of teacher models. These conclusions can be observed in Figure p where we reported the downstream performances without the Instruction-tuninn tuningg phase (see the Baseline) and the Instruction-tuning on Demonstrations. As dis- cussed Section 4.1 Small Language Models (SLMs) CoT prompted obtained weak results. In contrast, Chain-of-Thougee (CoT) Demonstrations, i.c., Demonstrations produced by
models that are instructed CoT-prompted Large Language Models (LLMs), oulperform non-instructed models (Section 1.2). However, although Demonstrations produced better students, the complete alignments between students and teachers are realized with the Curriculum Learning procedure, as discussed in Section 4.3 In particular, the stu- dents instructed via the CL (Instruction-ttuiing CL-Demonstration in Figure outperformed the students instructed via standard Instruction-tuning.
Finally, the CL approach delivers the teacher-student family-alignment. In Figure (horizontal lines), is possible to observe the phenomenon of family-alignment between Llama-3-70 3-70 and Llama-3-1 and -13 in more detail in Section 4.4

---

Published in Transactions on Machine Learning Research (09/2025)
Opentious ON
Lecenonseens ON
Plysica Inseractioo On
SCCIM inderacciion
Jama-3-1b (L 3-70m
Mistral-7b (Uama-33770) Mistral-7b
Mistral-7b Liama
Jama-3-1b
Figure 3: Accuracies of Llama-3-1 1 and Mistral-7 Instruction-tuned using sctup proposed Section
4.1 CoT-abilities of Small Language Models
Chain-of-Thought (CoT) prompts do not consistently deliver downstream performance improvements. SLMs, i.., with fewer parameters, have not benefitted the prompting with the CoT mechanism. In particular, we evaluated performance on four question-answering benchmarks, described Section 3.1 using two versions of Llama. Using a baseline demonstration produced via zero-shot prompt (which call "Baseline") and CoT prompt (Table 6 and Table 7 obtained the performances in Table
The results confirm what Wei et al. 2022) have claimed about the limitations of the emergent CoT prompt- ing abilities that are not observable in SLMs. Moreover, using CoT prompting leads to model confusion, resulting in the subsequent degradation of downstream results. is possible to observe these phenomena in OpenBookQA (OBQA) and CommonSenseQA (CSQA) (down arrows in Table 2). particular, there is a marked deterioration in Llama-3-1 (sce 4), which has half the parameters of Llama-3-8 (sce 4).
The same behaviour was not observed for Physical- and (SIQA). Not considering the nature of benchmarks, unlike the others, they always question-answering multiple-choice-uusste
Question Answering (PIQA) and
but have fewer possible choices, as shown in Table g In this regard, we hypothesise that the most controllable scenarios, where chain reasoning is limited to fewer options, are reasonable by SLMs elicited with CoT prompts.
Benchmarks
Llama-3-1
Llama-3-8
Baseline
CoT
Baseline
CoT
OBQA
53.6
51.04
58.7
57.01
CSQA
58.9
51.24
63.8
61.21
SIQA
46.8
45.9
48.7
47.8
PIQA
62.0
63.9
67.2
69.0
Table 2: Accuracies of Llama-3-1 and Llama-3-8, both without further tuning, on testing data with the standard prompt (Baseline) (see Table 6 and CoT prompt (CoT) (see Table 7)
4.2 The Enttucttiontuning Method
Instruction-tuning led by Large Language Models (teachers models), able to reason elicit the Smaller Lan- guage Models (students models) to do the same. This is shown in Figure 2 The student models after Instruction-tuning on synthetic Demonstrations delivered by teacher models outperformed the basclines the four proposed benchmarks. While performances are conspicuous improvements overall, they have sen- sible variations. The teacher models have different characteristics. They consequently achieve different performances in the proposed benchmarks. Table shows the performances in the zero-shot scenario (CoT prompting and not) on the data used to conduct the Instruction-tuning phase and on the same test set used to evaluate the proposed models.
Although the performances on the 'training set" are different (performances of GPT-4 and the same for Llama-3-70 in Table 5) this bias does not affect the students. The Llama-3-1 and -8 with GPT-4 teacher outperform the 1Jama-3-1 and -8 with Llama-3-70 teacher only on OBQA. As far CSQA and PIQA
8

---

Published in Transactions on Machine Learning Research (09/2025)
are concerned, there is balance that is not present in SIQA, where the students of Llama-3-70 outperform the others.
However, in the dustruction-tuuing method, instruction is conducted using Demonstrations (composed of Answers provided by teachers) delivered arbitrarily. Therefore, we propose to study both the intrinsic complexity of the answers and their impact on the students' exposure. To this end, we introduce a CT.-based instruction approach where demonstrations are delivered to students in a meaningful order (Section 4.3)
4.3 The Impact of Curriculum Learning
Instruction- Instruction-tuning tuning via Curriculum Learning Demonstrations elicits the reasoning abilities of students. The students gradually exposed incrcasingly mcaningful Demonstrations (CI-Demonstrations) learn better than those exposed to arbitrary Demonstrations. This is shown in Figure ? (bars Instruction-tuned CL-
Demonstrations), where Tlama-3-1 and -13 consistently outperformed the other models.
The benchmarks where the most significant effects can be observed are CSQA and OBQA, with an increase in average accuracy scores of 6 and 5 points, respectively (also in additional evaluations in Appendix G The same effects are less evident in PIQA and SIQA. One possible reason for this phenomenon might be tied again to the nature the benchmarks, as hypothesised in Section 4.1 To analyse this phenomenon, we studied the components of the complexity measure proposed in Section 4.5
4.4 The role of CL in aaiily-alignment
Instruction-tuiing via CL-Demonstrations still aligns students' reasoning abilitics with family teachers, even as instruction decreases. In fact, from Figure 2 we can observe that the performances of students instructed via CI-Demonstrations delivered by teachers from the same family outperform the others.
Moreover, to validate our hypothesis of family alignment, we introduced Mistral-7b MistralAI 2023 new SLMs with 7 billion parameters that outperforms the Llama-3-8 version several benchmarks, shown by MistralAT (202)). Specifically, we reproduced the experiments introduced in Section 4.2 In Figure 3 it can be seen that Llama-3-1 instructed on different types of Demonstrations delivered by Llama-3-70 almost consistently outperforms Mistral-7b. In order to confirm this speculation, we scaled the experiments on additional models chosen by family and number of parameters, presenting the results Appendix H The results discussod this section, as well as those presented in the appendices, confirm that Demonstrations derived from in-family teachers have more significant. impact on student models than those from other sources.
4.5 Ablation Study
The informativeness and complexity exposed students in a meaningful order in- structs better students. We conducted Ablation study estimate the impact of our evaluation measures proposed in Section 2.3 Hence, reproduced the same configurations proposed in Section 4.2 but removed one of the components (informativeness and complexity presented in Section 2.3 The results in Table 4 show that students instructed on D--Demonstraaions ordered by comprehensibility and informativeness con- sistently outperform students instructed via Demonstrations. The results show that students trained on the Demonstrations sorted by informativeness are more productive in QA tasks with more choices. In compar- ison, complexity proved helpful in cases where the number of options is minor. Phenomenon manifested in CSQA and OBQA with 5 and 4 choices, and PIQA and SIQA with and choices.
The role of measures
The Impact of Demonstration Quality The quality of the demonstrations plays a crucial role. By filtering them using a heuristic based on string matching and GPT-4o annotations (as described in Ap-
pendix C we analysed the effects of different types of training examples. As displayed in Table 3 models instructed with demonstrations ordered via our CL metrics consistently outperform others, even when the
demonstrations are incorrect or misleading. This suggests that the metric introduced in Section 2.3 has a beneficial effect regardless of the correctness of individual training examples.
9

---

Published in Transactions on Machine Learning Research (09/2025)
Method
OBQA
CSQA
SIQA
PIQA
Instraction-tuniig CL
71.2
80.2
84.3
66.4
Instruction-turing CL (gold)
72.7
80.2
85.0
66.8
Instrackoon-tuiing (gold)
70.3
78.5
82.6
65.4
Instruction-tuning CL (misleading)
60.0 1
63.0
48.9 f
68.4 f
Instruction-tuniie (misleading) baseline
59.2
62.6 4
46.5 J
67.0
58.3
63.1
47.9
67.3
Table 3: Accuracies of Llama-3-8, instructed using Instruction-tuning (NB without CL) and Instruction- tuning CL demonstrations as in Section 3. using mislcading (incorrect) and gold (correct) demonstrations.
5 Related & Future Work
5.1 Learning from Natural Language Explanation
Current methods for conditioning models on task instructions and provided explanations for individual data points replace the ancient intermediate structures Hase & Bansal (2022) that used rationales Zhang et al.) (2016) or inputs Narang al. (2020 Talmor et al. (2020) learn the models. Reasoning via the CoT builds upon prior efforts wherein explanations are viewed as intermediary constructs produced during inference Rajani et al. (2019). Our research stems from the studies of Ranaldi & Freitas (2024a b). Particularly, we adopt the idea of an LLM teacher and second, smaller LLM that takes student's 's position Magister
let al.
2023
Learning uses leacher-generated explanations, demonstrating prompt CoT rean
ee al.
2023 Ho et al. (2023). et al. (2023) claims that massive demonstrations significantly improve
performance over the singlo-sample approach Shridhar et al. (2023)
5.2 Large Language Models as Teacher
Several papers have been published simultaneously, including those by Ranaldi & Freitas
2024a Paul
let al. 2024), and [Saha et al. (2023) that prove the effect of transferring ability to produce CoT reasoning from larger to smaller models. Table resumes all main points of these contributions.
Our work goes beyond the following ways: i) We propose method for aligning CoT abilities via Instruction- tuning through Demonstrations produced by answers generated by larger models. ä) We study how provide Demonstrations students by proposing measure for evaluating the answers provided and analyse the alignment performance between in- and out-family models. tti We propose approach to improve the teachers and students' alignment. by employing our evaluations to expose the students meaningfully.
5.3 Future Work
We plan to continue investigating ways to transfer models' capabilities by introducing parallel work heuristics improve the structure of demonstrations by maximising abstraction abilities Ranaldi et al. (2025b), tuning models beyond single language Ranaldi & Pucci (2025) and modalities Ranaldi et al. (2025a).
6 Conclusion
We propose method to enable step-wise reasoning SLMs by introducing two mechanisms. First, we propose aligning CoT abilities via Instruction-tuning with the support of CoT Demonstrations delivered by LLMs leacher. Second, we use the Curriculum Learning to empower the tuning phase. We analyse the impact on the downstream abilities of four benchmarks. Results show that SLMs can be instructed to deliver robust, reasoned answers via Demonstration produced by LLMs. We move a step further rescarch: conceiving SLMs human learners, we expose them a CL teaching- based approach, obtaining better results downstream performances.
10

---

Published in Transactions on Machine Learning Research (09/2025)
References
Yoshua Bengio, Jérôme Louradour, Ronan Collobert, and Jason Weston. Curriculum learning. In Proceedings of the 26th annual international conference on machine learning, pp. 41 48, 2009.
Yonatan Bisk, Rowan Zellers, Ronan Le Bras, Jianfeng Gao, and Yejin Choi. Piga: Reasoning about physical commonsense in natural language, 2019.
Tom B Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in Neural Information Processing Systems (NeurIPS), 2020.
Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. Qlora: Efficient finetuning of quantized llms, 2023.
Aaron Grattafiori, Zhenyu Yang, Zhiwei Zhao, and Zhiyu Ma. The llama 3 herd of models, 2024. URL http::://axxvv..r/aaaa/ 2407 21783
Peter Hase and Mohit Bansal. When can models learn from explanations? a formal framework for under- standing the roles of explanation data. In Proceedings of the First Workshop on Learning with Natural Language Supervision, pp. 29-39, 39, Dublin, Treland, May 2022. Association for Computational Linguistics. doi: 10.18653/v1/2022.0
1.4. URL https://aclantiology.oom www.oog...rrg/22...ooo 1n1s-114
Namgyu Ho, Laura Schmid, and Ye-Young Yun. Large language models are reasoning teachers. Tn Proccodings of the 61st Annual Mceting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 14852 14882, Toronto, Canada, July 2023. Association for Computational Linguistics. doi: (1188853/112000 aol-.....--oo...... 830. URL https://aclantholle wwrplllggooogoooo.ooooo.ooo org/2023 acl-long 8800
Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large language models are zero-shot reasoners, 2023.
Liunian Harold Li, Jack Hessel, Youngiae Yu, Xiang Ren, Kai-Wei Chang, and Yejin Choi. Symbolic chain-of- thought distillation: Small models can also "think" step-by-step. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 2665-2679, Toronto, Canada, July 2023. Association for Computational Linguistics. doi: 10.18653/v1 222/222222clllll 150. URL https://aclanthol http://cclanttoloo..o/////////// http:/222..cccl1110 acl-long.11 150.
Lucic Charlotte Magister, Jonathan Mallinson, Jakub Adamck, Eric Malmi, and Aliaksci Severyn. Teach- ing small language models to reason. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pp. 1773-1781 1781, Toronto, Canada, July 2023, Associ- URL httxp::/aclanthhologgy
ation for Computational Linguistics. doi: 10. org/2023. ach-sort.. 151
Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. Can a suit of armor conduct electricity? a new dataset for open book question answering, 2018.
MistralAI.
-istrall-b-instruet, Mistral-7B- Instruct-otv
2023.
URL https://lugcingfaaec.ooooosssstl
Sharan Narang, Colin Raffel, Katherine Lee, Adam Roberts, Noah Fiedel, and Karishma Malkan. Wt5?! training text-to-text models to explain their predictions, 2020.
OpenAL. Gpt-4 technical report, 2024.
Debjit Paul, Mete Ismayilzada, Maxime Peyrard, Beatriz Borges, Antoine Bosselut, Robert West, and Boi Faltings. Refiner: Reasoning foodback on intermediate reprosontations, 2024.
Nazneen Fatema Rajani, Bryan McCann, Caiming Xiong, and Richard Socher. Explain yourself! leveraging language models for commonscnse reasoning. In Procoedings of the 57th Annual Moeting of the Association for Computational Linguistics, pp.
Florence, Italy, July 2019. Association for Computational Linguistics. doi: 10.18653/41/P19-145 URL http:////cccaaahooooooooog//////
11

---

Published in Transactions on Machine Learning Research (09/2025)
Leonardo Ranaldi and Andre Freitas. Aligning large and small language models via chain-of-thought reasoning. In Yvette Graham and Matthew Purver (eds.), Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1812-1827, St. Julian's, Malta, March 2024a. Association for Computational Linguistics. URL org w00024.eaallloooo 109)
Sttpss//aclantheen
Leonardo Ranaldi and Andre Freitas. Self-refine instruction-tuning for aligning reasoning in language models. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.), Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 1325-2341 Miami, Florida, USA, November 2024b. Association for Computational Linguistics. doi: 10. http://1112224.mmm-mmm 139. URL https: //aclanthology org/2024 emnle-maaae 139
Leonardo Ranaldi and Giulia Pucci. Multilingual reasoning via self-training. In Luis Chiruzzo, Alan Rit- ter, and Lu Wang (eds.), Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguisties: Human Language Technologies (Volume 1: Long Papers), pp. 11566-11582, Albuquerque, New Mexico, April 2025. Association for Computational Lin- guisties. ISBN 979-8-89176-18944 doi: 10. 10.18653/v /t0.//2222..2/0onn...... URL https://aclaatthol.gg oxg/2023 naacl-long.577) 577/
Leonardo Ranaldi, Giulia Pucci, and Fabio Massimo Zanzolio. Modeling easiness for training transform- ers with curriculum learning. Tn Ruslan Mitkov and Galia Angelova (eds.), Proceedings of the 14th International Conference on Recent Advances in Natural Language Processing, pp. 937 948, Varna, Bul- garia, September 2023. INCOMA Ltd., Shoumen, Bulgaria. URL httpp://aclanthoology www.ooloooyyr.g/2222//// ran1p-1.101/
Leonardo Ranaldi, Giulia Pucci, and Andrè Freitas. Does the Order matter? Curriculum learning over languages. Tn Nicoletta Calzolari, Min-Yen Kan, Veronique Hoste, Alessandro Lenci, Sakriani Sakti, and Nianwen Xue (eds.), Procoodings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pp. 5212 5220, Torino, Italia, May 2024.
ELRA and ICCL, URL https://aclanthhlogg org/2024 lrec- mine-main.444. 464/
Leonardo Ranaldi, Federico Ranaldi, and Giulia Pueci. R2-MultiOmnia:
Leading multilingual multimodal reasoning via self-training. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar (eds.), Proceedings of the 63rd Annual Meeting of the Association for Computational Linguisties (Volume 1: Long Papers), pp. 8220-822 8234, Vienna, Austria, July 2025a. Association for Computational Linguistics. ISBN 979-8-891 8-888888877-8 76-21-00- doi: 10. 10.18653/v v1/2222..clll acl-oogg.002.0. URL https:///aclatthooogyy
aww.020.acl-ooo.4444 402/
Leonardo Ranaldi, Marco Valentino, and Andre Freitas. Improving chain-of-thought. reasoning via quasi- symbolic abstractions. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar (eds.), Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 17222-1724B Vienna, Austria, July 2025b. Association for Computational Linguistics. ISBN 979-8-89176-251-4 doi: 10. 10.18653/vv acl-ooolllooog........ URL https:://cclaatthl.l.o
org/2025.acl2 acl acl-1ong..43
Swarnadeep Saha, Peter Hase, and Mohit Bansal. Can language models teach weaker agents? explanations improve students via personalization, 2023.
teacher
Maarten Sap, Hannah Rashkin, Derek Chen, Ronan Te Bras, and Yejin Choi. Social TQa: Commonsense reasoning about social interactions. In Proccedings of the 2019 Conference on Empirical Mothods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP- LJCNLP), pp. 4463 4473, Hong Kong, China, November 2019. Association for Computational Linguistics. doi: 10. 10.18653/v1 P0/553/11/D991111 119-1454. URL httpp:://cclatthologg http://ccaaatttoooooooooo/////- org/D19- 1454
Kumar Shridhar, Alessandro Stolfo, and Mrinmaya Sachan. Distilling reasoning capabilities into smaller language models. In Findings of the Association for Computational Linguisties: ACL 2023, pp. 7059- 7073, Toronto, Canada, July 2023. Association for Computational Linguistics. doi: 10.18653/v1/200 findings-acl.444 URL
12

---

Published in Transactions on Machine Learning Research (09/2025)
John Talburt. The flesch index: An easily programmable readability analysis algorithm. New York, NY, USA, 1986. Association for Computing Machinery. ISBN 089791 1865. URL https://doi....... org/10..//... 10. 1145/ 10563. 10588
Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. CommonsenseQA:: A question swering challenge targeting commonsense knowledge. In Procoedings of the 2019 Conference of the North American Chapler of the Association for Compulat ional Linguisties: Human Language Technologies, Volume (Long and Short Papers), pp. 4149 4158, Minneapolis, Minnesota, June 2019. Association for 1421
Computational Linguistics. doi: 10.18653/ve v1 200653/v1/N11--/ 1N19-1421. URL
Alon Talmor, Oyvind Tafjord, Peter Clark, Yoav Goldberg, and Jonathan Berant. Leap-of-thought: Teaching pre-trained models to systematically reason over implicit knowledge, 2020.
Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xucchen Li, Carlos Guestrin, Percy Liang, and Tatsuori B. Hashimoto. Stanford alpaca: An instruction-folloinng llama model. https://github. com/ 2023.
batsu-labbstaaffrd_adll
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023.
Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, Ed H. Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, and William Fedus. Emergent abilities of large language models, 2022.
Ye Zhang, lain Marshall, and Byron C. Wallace. Rationale-angmented convolutional neural networks for text classification. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pp. 795-804, Austin, Texas, November 2016. Association for Computational Linguistics. doi: 10. (0.18633/v1/DI6-100 1076. URL
13

---

Published in Transactions on Machine Learning Research (09/2025)
A
Ablation study on CL-based Instruction-tuning
Benchmarks
Students
OBQA
CSQA
PIQA
SIQA
Llama-3-1 (Lloma-3-70)
Arbitrary Teaching
64.7
71.6
80,2
62.8
Teaching via IC
70.5
76.5
65.3
Teaching via I
70.2ft
77.21
81.2
61.8.
Teaching via C
66.4
69.74
84.31
66.2
3-1 (GPT-4)
Arbitrary Teaching
65.3
70.8
80.5
62.2
Teaching 1C
69.2
74.2
83.3
64.8
Teaching via I
68.54)
73.7f
79.64
63.8
Teaching via
66.3
69.8
83.9f
65.71
Llama-3-8 (Llama-3-70)
Arbitrary Teaching
66.5
76.5
81.9
64.5
Teaching via 1C
72.3
82.2
86.2
67.7
Teaching via 1
73.4f
81.9f
80.74
63.8
Teaching via C
67.9
76.6
84.31
70.3
Llama-3- 3-1 (GPT-4)
Arbitrory Teaching
68.5
77.3
82.6
63.3
Teaching IC
71.6
80.5
84.9
66.1
Teaching via I
70.81
81.7
81.9
62.7
Teaching via C
68.2
78.5
82.3
65.94
Table 4: Ablation study on our Instruction-Luming CL-Demonnstaatinss approach.
Benchmarks
Llama-3-70
GPT-4-0
Baseline
CoT
Baseline
CoT
Training
OBQA
65.8
71.3
67.1
78.8
CSQA
75.0
79.6
80.4
85.6
SIQA
66.2
67.5
72.6
PIQA
82.9
86.0
83.5
85.8
Testing
OBQA
66.0
72.0
68.3
75.6
CSQA
73.4
81.8
81.4
84.0
SIQA
65.2
66.9
67.3
71.8
PIQA
83.8
85.6
85.2
86.5
Table 5: Accuracy (%) of Tlama-3-70 and GPT-4 (teachers) on training and lesting data with CoT prompt (CoT) and with the standard prompt (Baseline).
14

---

Published in Transactions on Machine Learning Research (09/2025)
B Instruction used
Zero-Shot
Choose the
the question only from options A, C, Question: Which animal gives birth to live young?
A) Shark
B) Turtle
C) Giraffe
D) Spider
Answer:
Table 6: Example of Zero-Shot prompting.
Zero-Shot Chain-of-Thowe
Choose the answer to the question only from options A, B, c, Question: Which animal gives birth to live young?
A) Shark
B) Turtle
C) Giralle
D) Spider
Answer: Let's think step by step
Table 7: Example of Zero-Shot Chain-of- Thought prompting.
C Evaluation Metrics
We used a double-check to assess the accuracy of the responses delivered in the different experiments. In the first step, we used an exact-match heuristic (this was used for most of the evaluations, especially in cases of multiple-choice QA). However, since some experiments required a more accurate response check, we used GPT-40 as a judge.
GPT-40 Evaluation Prompt
Given the following #Senteces', you are decider that decides whether the 'Generated Answer' the same as the "Target Answer'. If the output doesn't align with the correct answer, respond with '0', whereas it's correct, then respond with '1'. Please, do not provide any other answer beyond '0' or '1! #Senteces:
Generated Answer: [moodel_result]
Target Answer: ).orrect_ansseer..
D Models
Model
Version
Llama-3-1
peeta-llama/llamm-3.-1111
Instrrct
Llama-3-8
meta- meta-Ilama/ LlteeettaaaLaaaa--------tttttttt
Llama-3-70
meta-llama/Mettan Malllammm/MMtta-llmm---77771
Mistral-7
mistralai Mistsaara-----IInsttttt--111
GPT-4o
OpenAI API (gpt-44-20044-08-066
Table 8: In this table, we list the versions of the models proposed in this work, which can be found on huggingface.co. We used all the default configurations proposed in the repositories for each model.
15

---

Published in Transactions on Machine Learning Research (09/2025)
E
Training and Testing Data
OBQA
CSQA
PIQA
SIQA
classes
5
2
3
Training
# examples for each class
1000
800
2000
1330
Test
# examples for
125 (= 8)
235*
924"
640
each class
(# 11)
(+ 18)
(#: 19)
Table 9: Characteristies Training and Test set of benchmarks proposed in Section 3.1
The indicates that
the number of examples are not perfect balanced, but the difference from the average is marginal.
Name
Repository
CSQA
Talmor et a.. (2019)
OBQA
Mihaylov et al. (2018)
PIQA
Bisk et al. 2019
auggingfaace.co//tttteess/////
SIQA
Sap et al.
2019
Sugginggface.co/datasssaa
i_qa
Table 10: In this table, we list the versions of the benchmark proposed in this work, which can be found on huggingface.co.
F
Comparison with related works
Work
Method
Teachers
Students
Magister et al. (2023)
SFT
PaLM GPT-4
T5-small, -medium T5-large, -xxl
al.
2023
SFT
GPT-3 175B
OPT-1.3b
Shridhar et al.
2023
SFT
GPT-3 175B
GPT-2
Ho et al.
2023
SFT
InstructGPT
GPT-3 (ada,babbage,curie)
(text-davinci-002)
Ranaldi & Freitas (2024a)
IT
GPT-3.5
Llama-2-7,
Llama-2-13
Llama-2-70
Ours
IT
Llama-3-70b
3lama-3-1b, -13b
GPT-4
Mistral-7b
Table 11: Summary of methods, teacher and student models of previous work, we indicate Supervised Fine- tuning as (SFT) employed in most previous work.
16

---

Published in Transactions on Machine Learning Research (09/202))
G
Evaluation on additional benchmark
Benchmarks
Method
GSM8K
MATH
AIME24
MMLU-Redux
Instructionn-aante CL
78.6
38.5
26.0
72.0
Instruclion-taanee CL (gold)
80.2
37.0
22.8
70.7
Instruction- tuning (gold)
75.9
35.8
18.4
67.2
Instruction-tunne tuning CL (misteading)
70.4
25.0
12.6
59.0
cnrrrcctoo-tuuiiie (misleading) Boseline
67.6
24.8
10.0
57.3
73.8
30.0
16.0
60.2
Table 12: Results of different instruction-tuming strategics GSM8K, MATH, AIME24, and MMLU-Rodux using Llama-3-8h base model.
H Evaluation on different
Settings
Benchmarks
Additional QA Benchmarks
Method
GSM8K
MATH
AIME24
MMLU-Redux
OBQA
CSQA
PIQA
SIQA
IT-CL (GPT-4o)
78.6
38.5
26.0
72.0
71.2
80.2
66.4
84.3
IT-CL QQee2-726)
79.2
39.0
25.0
73.1
72.8
82.6
68.5
83.2
IT-CL (Mixstrll8776
77.0
36.4
24.2
72.0
71.2
81.0
66.0
80.4
Baseline
73.0
28.0
10.0
58.6
58.0
62.9
66.5
48.2
Table 13: Instruction-tuning CL (IT-CL) with different teachers (GPT-40, Qwen2-72b, Mixtral8x7b) com- pared on GSM8K, MATH, AIME24, MML U-Rdux, and QA benchmarks.
Benchmarks
Additional QA Benchmarks
Method
GSM8K
MATH
AIME24
MMLU-Redux
OBQA
CSQA
PIQA
SIQA
IT-CL (GPT-40)
76.9
38.0
22.8
70.2
71.4
81.0
66.1
83.6
IT-CL (Qwen2-1eh 12b)
78.1
37.2
21.8
70.4
71.0
80.5
67.5
82.1
IT-CL (Miztral8276)
79.6
39.8
25.0
72.8
73.1
82.4
69.0
84.6
Buseline
72.8
28.2
10.0
58.3
59.2
63.5
67.2
48.6
Table 14: Instructiiontuninn CT. (IT-CL) with different teachers (GPT-40, Qwen2-72b, Mixtral8x7b) com- pared on GSMSK, MATH, AIME24, MMLU- MMLU-Redux, and QA benchmarks (second experimental setting).
17


Reviews:
[{"review_id": "mn4bWA4rUm", "reviewer": "Reviewer_H517", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. The curriculum learning design of this work is interesting.\n\n2. The discussion of using a teacher model from the same or different family is useful.\n\nWeaknesses:\n\n1. The novelty of this work is limited, and the current draft does not properly highlight the new design. The idea of using a larger language model to generate synthetic CoT demonstrations for training a smaller language model is not new, and has been studied in a bunch of prior works. Even if this work performs experiments using different models from prior works, the experimental results are expected and do not bring new insights. The paper should have instead focused more on curriculum learning and model family alignment between the student and the teacher model.\n\n2. To validate the model family alignment hypothesis, the authors should evaluate on more model families other than Llama. For example, the authors can take different models from Gemma, Qwen or Mistral families, and see if the same conclusion holds there.\n\n3. For curriculum learning, the authors should compare their approach to some simpler heuristics, e.g., reordering demonstrations according to their reasoning token counts. Does the proposed metric improve the performance over these baselines?\n\n4. The authors should add experiments on standard reasoning benchmarks, such as GSM8K, MATH or harder tasks like AIME.\n\n5. The writing of this paper should be improved. For example, the second paragraph of Section 4.5 include problematic pointers to sections.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper presents the idea of instruction-tuning a smaller language model with synthetic CoT demonstrations generated by a larger language model. The authors propose to perform curriculum learning via reordering demonstrations based on their designed metrics. Specifically, their metric quantifies the informativeness and comprehensibility of each demonstration. They evaluate their approach on 4 QA datasets, and perform training on Llama and Mistral models. The teacher models in their evaluation include models from the Llama family and GPT-4. Their evaluation shows that curriculum learning brings some performance gain, and picking a teacher model from the same family as the student model achieves better performance.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "See the weakness section.", "broader_impact_concerns": "No concern.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "kx8rpvGJFf", "reviewer": "Reviewer_hrCx", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**\n1. Introducing curriculum learning during the distillation process is quite intuitive. \n2. The authors also provide comparisons of distillation performance across different teacher models, along with some analysis, such as the observation that models from the same family are more suitable for distillation.\n\n**Weaknesses**\n1. One major concern is that the authors' description of the motivation for solving the problem is incomplete. The reasoning capabilities of small models are not weak, especially in this era, where we see that R1-distill-Qwen-7B models significantly outperform the teacher models proposed in the paper (e.g., Llama3-70B, GPT-4) on the majority of reasoning datasets. Additionally, the authors mention that models smaller than 60B parameters benefit less from Chain-of-Thought (CoT) [1], but this is an outdated view from over two years ago. Due to the insufficient and outdated motivation, the study's use of Llama3-70B / GPT-4 models to distill Llama3-7B seems to lack significant purpose.\n2. The second major concern is that the method is overly simple and lacks innovation. Teacher-student distillation and curriculum learning are well-established concepts that have been extensively studied in many works. The only difference in this paper is that the authors propose two metrics for measuring curriculum learning: Informativeness and Comprehensibility. However, these two metrics are also not entirely reasonable. For instance, the formula for Comprehensibility is overly simplistic, and a potentially better evaluation method might be to have a LLM assign scores. As for Informativeness, while it may measure changes in information gain, this does not necessarily correlate with the quality of CoT, such as whether it directly determines if the answer is correct.\n3. The third major concern is that the experimental validation is not comprehensive. First, the dataset is limited to multiple-choice question-answering, which covers a very narrow scope. Second, the most commonly used reasoning tasks, such as mathematics and coding datasets, are not addressed in this paper. Furthermore, at the model level, the paper does not include mainstream model families such as the Qwen series. Finally, regarding the selection of teacher models, it is clear that neither GPT-4 nor Llama3-70B represents the strongest available teacher models. The insufficient experimental validation undermines the credibility of the conclusions drawn in this paper.\n\n[1] Chain-of-thought prompting elicits reasoning in large language models", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes a method to enhance the reasoning capabilities of small language models by incorporating curriculum learning into the teacher-student distillation approach. The method has been experimentally validated on four multiple-choice question-answering benchmarks and the Llama series of models.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Please see the major concerns in the Weaknesses.\n\nMinor Concerns:\n1. There is an issue with the subscript i in Formula 3.\n2. The paragraph containing the description of Comprehensibility seems unrelated to this paper.", "broader_impact_concerns": "Not involved.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "apyKUuM9kl", "reviewer": "Reviewer_8eHQ", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths:**\n1. The paper addresses a relevant problem in the field of Large Language Models, which is improving the reasoning capabilities of Small Language Models.    \n2. The proposed approach of using Curriculum Learning to enhance Instruction-tuning is novel and shows promising results.    \n3. The paper provides a comprehensive analysis using four different question-answering benchmarks.    \n4. The ablation study effectively demonstrates the impact of the proposed evaluation measures. \n\n**Weaknesses:**\n1. The paper has minor typographical errors throughout such as \"Llama-3-1 and -13\" in the introduction, Missing brackets in \"Avg(dL(wi)\" .\n2. The paper does not include an evaluation of mathematics-specific datasets, which is a critical aspect of assessing model reasoning and CoT capabilities capabilities.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper addresses the challenge of enabling SLMs to perform complex reasoning tasks.    \n\nThe authors propose a method to transfer step-wise reasoning capabilities to SLMs through IT using synthetic demonstrations generated by LLMs.  They further enhance this process using Curriculum Learning to order the training demonstrations in a meaningful way.  The authors evaluate their approach on four question-answering benchmarks.    \n\nTheir results indicate that SLMs can be effectively instructed to reason using demonstrations provided by LLMs.  Furthermore, the Curriculum Learning-based Instruction-tuning outperforms standard Instruction-tuning.  The study also finds that CL facilitates the alignment of Chain-of-Thought (CoT) abilities within the same model family.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "I would request the authors to :\n1. Fix the typographical errors in the paper.\n2. Add some results for Mathematics specific datasets", "broader_impact_concerns": "None", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)