Efficient Construction of Model Family through Progressive Training Using Model Expansion
Authors: Kazuki Yano
Sho Takase
Sosuke Kobayashi
Shun Kiyono
Jun Suzuki
Venue: COLM 2025 (2025)
Keywords: pre-training
model familly
compute efficiency
TL;DR: We propose a progressive training approach that efficiently builds a family of LLMs, reducing total computational requirements while achieving comparable or even better performance.

Abstract: As Large Language Models (LLMs) gain widespread practical applica-
tion, offering model families with varying parameter sizes has become
standard practice to accommodate diverse computational requirements.
Traditionally, each model in the family is trained independently, incurring
computational costs that scale additively with the number of models. In
this work, we propose an efficient method for constructing model families
via progressive training, where smaller models are incrementally expanded
to larger sizes to create a complete model family. Through extensive ex-
periments on a model family ranging from 1B to 8B parameters, we show
that our approach reduces total computational cost by approximately 25%
while maintaining comparable performance to independently trained mod-
els. Moreover, by strategically adjusting the maximum learning rate based
on model size, our method outperforms the independent training across
various metrics. Beyond these improvements, our approach also fosters
greater consistency in behavior across model sizes.

Published as a conference paper at COLM 2025
Efficient Construction of Model Family through Progressive Training Using Model Expansion
Kazuki Yano", Sho T Sosuke Kobayashi', Shun Kiyonož,Jum Suzuki". TTohoku University
4SB Intuitions yano. kazuki@dc. tchoku.ac.
Abstract
As Large Language Models (LLMs) gain widespread practical applica- tion, offering model families with varying parameter sizes has become standard practice to accommodate diverse computational requirements. Traditionally, each model in the family is trained independently, incurring computational costs that scale additively with the number of models. In this work, we propose an efficient method for constructing model families via progressive training, where smaller models are incrementally expanded to larger sizes to create complete model family. Through extensive ex- periments on model family ranging from 1B to 8B parameters, we show that our approach reduces total computational cost by approximately 25% while maintaining comparable performance to independently trained mod- els. Moreover, by strategically adjusting the maximum learning rate based on model size, our method outperforms the independent training across various metrics. Beyond these improvements, our approach also fosters greater consistency in behavior across model sizes.
1 Introduction
As As Large Language Models (LLMs) gain widespread practical application, providing models with (i) consistent architecture and (ii) varying parameter sizes (hereafter referred to model family) has become standard practice in the NLP community. For instance, Llama 3.1 includes models with 8B, 70B, and 405B parameters (AI@Meta, 2024), while Gemma provides 1B, 4B, 12B, and 27B parameter variants (GemmaTeam, 2025). Similarly, Qwen2.5 offers model family with 0.5B, 1.5B, 3B, 7B, 14B, 32B, and 72B parameters (Yang et al., 2024b).
Such model families are designed address wide range of computational constraints and application scenarios, Smaller models offer faster inference and lower resource consumption, making them suitable for daily tasks and deployment in resource-constraiied environments such as smartphones and edge devices (Abdin et al., 2024). In contrast, larger models are deployed for scenarios requiring advanced reasoning capabilities and complex task processing, typically on large-scale servers (Wei et al., 2022).
The standard approach to constructing model family involves training each model inde- pendently from scratch. However, training large-scale models demands extensive resources, e.g., thousands of GPU days (Touvron et al., 2023). The total computational cost of con- structing a model family poses significant burden on its builders. (Figure (Top)). This motivates us to explore more efficient methods for model family construction.
We identify model expansion as potential approach to more efficiently constructing model families, including large-scale models. Model expansion leverages the parameters of pre- trained smaller models initialization for training larger models (Chen et al., 2022; Du et al., 2024). However, prior work on model expansion has primarily aimed at producing a single final model, with limited focus on the potential utility of intermediate models.

---

Published as a conference paper at COLM 2025
Total Computational Cost of Independent Training
4B
Trainung
Training
Training
Total Computational Cost of Progressive Training
Model Expansion
Model Expansion
Training
Training
Training
Figure 1: Comparison of approaches for constructing a model family. (Top): Conventional approach, where each model in the family (2B, 4B, 8B) is trained independently from scratch. The total computational cost is the sum of the training costs for all individual models. (Bottom): Proposed progressive training utilizes model expansion, where smaller models are expanded to initialize larger ones. The total cost equals only that of the largest model (8B).
In this work, we empirically explore model expansion as means to efficiently construct model family. Specifically, we propose method that repeatedly applies model expansion to construct models from smaller to larger sizes, namely progressive training, thereby reducing the total training cost of constructing a model family (Figure (Boftom)).
Through our experiments with model family ranging from 1B to 8B parameters, we demonstrate that the proposed method reduces the total computational cost of constructing complete model family by approximately 25% compared to training each model inde- pendently
Moreover, by adjusting the maximum learning rate based model size, our method consistently achieves superior performance across a range of benchmark tasks rela- tive to independently trained models. We also found that progressive training yields greater behavioral consistency across the model family, as indicated by lower Kullback-Leibh (KI) divergence between models' output distributions.
2 Task Definition and Notation Rules
We consider a model family consisting of models with monotonically increasing sizes and consistent architecture. Formally, define this model family sequence of models of sizes [X, X,,..., Xz] and corresponding sequence of model parameters [0, .,. Onl, where f; E R denotes the parameters of the i-th model. For each model of size X;, let Ti represent the number of training tokens used to train the corresponding model. The computational cost (FLOPs) required to train given model is determined by the model size X; and the number of tokens T₁. Following Brown et al. (2020), we define FLOPs as: FLOPs(X;, T;) 6X,T, Note that analysis does not depend on this specific formulation and can be generalized to other reasonable cost approximations (Hoffmann et al., 2022).
Let Tscratcch denote the number of tokens for training a randomly initialized model of size X; from scratch. The total computational cost of constructing the entire model family then given by Σ FLOPs(X, T,scratch). The task of this work reduce the overall computational cost while preserving the performance of each model. Formally, we aimt to
IThis 25% reduction corresponds to savings of approximately 3.2K GPU hours in our experimental setup.
2

---

Published as a conference paper at COLM 2025
construct each model under the following constraint:
Σ FLOPs(X,, PL T
Σ II,
(1)
where T7 denotes the number of tokens used to train the model of size X; under the progressive training approach (Section 3).
Method: Progressive Training
We propose a method for constructing language models of varying sizes more efficiently than the common practice, i.e, training each model independently from scratch. In the common approach, the cost of training model of size X; from scratch using T,cratch tokens? is FLOPs(X,, T leading to total cost of Σ FLOPs(X, (X,, T.aratch) for the entire model family. A simple strategy satisfy Equation (1) is reduce the number of training tokens for each model by selecting T. such that Teratchh > 7. for > 1. However, naively reducing the number of training tokens typically results in degraded performance if other factors remain unchanged. Instead, we initialize the i-th model of size X; for by using better initialization than random, which is obtained by applying model expansion to the previously trained model sizesize X;- We refer to this process as progressive training.
Specifically, our progressive training begins by training initial model of size X1 using 7pcraich tokens. At each subsequent stage, we initialize the next model using a model expansion method f,
ginit = f(0;; Xi+1) (i2 1),
(2)
where (1:1 RX - Rx an off-the-shelf model expansion method. The expanded parameters Oinit serve as an effective initialization for training the subsequent model.
There exists a wide range of possible token allocation patterns [rprog ".. (.ro] that satisfy the constraint in Equation (1)'s condition regarding the total computational cost. In this work, we determine each T such that the overall computational cost matches that of training the largest model X₃ from scratch:
FLOPs(X,r T Tscratch)
Σ (X, T.TUB).
(3)
Specifically, we set T. to satisfy the following FLOPs constraint:
FLOPs(X T = FLOPs (OOPs(XH11.. X1+1 Tscratch)
FLOPs(X,, 7,P00).
(4)
Intuitively, we allocate the computational cost of training a model of size X from scratch, subtracting the computational cost already used in previous stages. Through this procedure, we can obtain an entire model family from size X, to X, with total computational cost equal FLOPs(X,, ]seratch).
How we choose f. In this work, we adopt bert2BERT (Chen et al., 2022) as our model expansion method for the following reason. Unlike approaches that focus solely on depth expansion (such as stacking (Du et al., 2024)), bert2BERT allows us to increase both the width and depth dimensions of Transformer models (Vaswani al., 2017), offering greater
Note that the choice /erratthh is arbitrary depending on available resources, though most re- searchers follow the Chinchilla law (Hoffmann et al., 2022) as principled approach to determine the compute-optimal number of training tokens. Our experimental setup also adopts the Chinchilla law (Section 4.2).
3

---

Published as a conference paper at COLM 2025
flexibility?. Specifically, for width expansion, bert2BERT increases the hidden dimensions by duplicating the weights of linear layers; for depth expansion, it stacks additional layers by duplicating pre-trained onest. This flexibility to expand along both dimensions is particularly valuable as we repeatedly apply model expansion in progressive training.
4
Experiment
To evaluate the effectiveness of progressive training, we pre-train and compare two model families: one with independently training each model from scratch (Independent) and the other with progressive training (Progressive). Specifically, we demonstrate that progressive training improves computational efficiency over independent training, and the performance of the two model families is comparable.
4.1 Experimental Setup
We used FineWeb-Edu (Penedo et al., 2024) as the training data for pre-training and adopted the GPT-2 (Radford et al., 2019) tokenizer for tokenization. While progressive training can accommodate any parameter size increase, this work, we adopted a configuration where the parameter count doubles at each stage. Specifically, we construct a model family with sizes [X1 1B, X2 2B, X3 4B, X4 8B]5. All models follow the Llama architecture (Touvron et al., 2023) with a maximum input sequence of 1024 tokens. Each model was trained using cosine learning rate scheduler, with maximum learning rate of 3.0 10
To evaluate each pre-trained model in the family, we measured perplexity on the validation data (Valid) from FineWeb-Edu and WikiText (Merity et al., 2017). To more comprehensively assess the performance of pre-trained models, we also evaluated zero-shot performance across multiple downstream tasks. Specifically, we include tasks including language model- ing (LAMBADA (Paperno et al., 2016)), commonsense reasoning (WinoGrande (Sakaguchi et al., 2021), PIQA (Bisk et al., 2020), HellaSwag (Zellers al., 2019),, and question answering (ARC-e, ARC-c (Clark et al., 2018), OBQA (Mihaylov et al., 2018)).
Training Data Size and Computational Cost
We prepare the following two data sizes, (i) Chinchilla law (Hoffmann et al., 2022) and (ii) 2x Chinchilla law, determine the number of training tokens Tyeratch for each model in model family. Chinchilla law provides the optimal FLOPs required to achieve a specific loss. The guideline is use 20 tokens per model parameter for training (Hoffmann 2x Chinchilla law is used to simulate the case such that the amount of training data greatly exceeds the optimal values indicated by the Chinchilla law. In fact, exceeding the Chinchilla law has become standard practice in recent LLM literature (Sardana et al., 2024).
al., 2022).
In the progressive training approach, determine the number of training tokens T.Prog for each model to satisfy the constraint defined in Equation (4), ensuring it matches the total computational cost equals that of training the largest model from scratch. Under the 2x Chinchilla law setting, this constraint yields the following token allocations: T.Poog 40B, 1 60B, gorog = 120B, gorog = 240B6
One of the key advantages of progressive training its ability to obtain models of multiple sizes efficiently. As defined in Section 2, the computational cost is calculated as FLOPs
3 Allhhough recent methods such as LEMON (Wang et al., 2024b) have emerged, LEMON builds upon the foundation of bert2BERT with only minor modifications to its core mechanisms. We choose bert2BERT for its simplicity and ease of implementation.
"The original bert2BERT paper (Chen et al., 2022) introduces two expansion variants: AKI and FPI. In this work, exclusively adopt the AKI
9The specific width and depth settings for each model are detailed in Appendix A.
6or the Chinchilla law setting, T.Pro shown in Appendix B.
4

---

Published as a conference paper at COLM 2025
Perplexity
Accuracy 1
Valid Wikitext LAMBADA ARC-e ARC-c Winogrande PIQA OBQA HellaSwag
1B Independent|133
113.14
22.81
39.3
59.3
31.7
54.0
70.5
35.6
46.9
Independent
11.30
18.57
45.5
65.2
37.7
55.3
72.1
38.8
54.3
2B
Progressive 11.29
18.74
45.1
63.6
36.6
57.4
72.7
39.2
54.7
4B Independent 9.91 15.46
49.5
68.8
41.1
57.2
75.1
43,8
60.5
Progressive 9.87
15.12
51.0
69.4
40.0
58.6
75.1
40.6
61.2
Independent 8.65
12.24
53,9
71.8
43.1
62.4
76.2
42,6
65.8
Progressive 8.61
11.98
55.4
73.5
45.1
62.1
76.6
45.6
67.5
Table 1: Evaluation results of pre-trained models under the Chinchilla law setting. For each model size, we compare the performance of models trained independently from scratch (Independent) versus those built using progressive training (Progressive).
Perplexity
Accuracy f
Valid Wikitext LAMBADA ARC-e ARC-c Winogrande PIQA OBQA HellaSwag
1B
Independent
12.04
20.56
42.0
62.3
34.6
55.5
72.0
35.0
51.6
Independent
10.44
17.11
48.5
66.9
38.4
56.6
75.3
41.6
58.0
2B
Progressive 10.43
16.94
50.3
70.0
38.1
58.5
73.8
41.4
59.0
Independent
9.18
13.99
51.7
71.7
43.2
59.2
76.7
40.8
63.3
4B
Progressive
9.00
13.63
51.9
72.6
43.9
61.0
76.1
43.0
65.6
Independent
7.85
10.23
55.2
74.5
47.4
62.4
77.0
46.4
67.9
8B Progressive 7.74
10.21
57.8
75.3
46.7
64.6
77.3
44.6
70.2
+Fixed Data 7.73
10.22
56.2
74.8
47.8
63.1
77.8
45.2
70.4
Table 2: Evaluation results of pre-trained models under the 2x Chinchilla law setting, Perfor- mance compared between models trained independently from scratch (Independent) and those built using progressive training (Progressive). Progressive+Fixer Data indicates training on a fixed dataset of 320B tokens.
6XT, where X denotes the number of model parameters and T is the number of training tokens. For instance, independently training each model in the family [1B, 4B, 8B) under the 2x Chinchilla law would require FLOPs of [0.24Z, 0.9,2, 3.84Z, 15.42) respectively. Thus, constructing the entire model family would require 20.4ZFLOPs in total.
In contrast, with progressive training, the total computational cost required to construct the entire model family is equivalent to the computational cost required train the largest-size, ie., 8B, model. Therefore, the computational cost is 15.4ZFLOPs. Compared to training each model independently, progressive training can reduce computational cost by approximately 25%. This demonstrates that progressive training improves the computational efficiency of constructing a model family.
4.3 Results
Model Performance. Tables 1 and 2 present the evaluation results of pre-trained models in the Chinchilla law and 2x Chinchilla law settings, respectively. Our progressive training approach (Progressive) achieves performance comparable to or better than models that are independently trained from scratch (Independent) across different parameter sizes in both settings. Notably, for the largest model (8B), Progressive shows improvements in perplexity and most downstream tasks.
Computational Cost. In Figure (Left), for given computational budget, i.e., ZFLOPs, Progressive consistently outperforms Independent. In addition, Progressive reaches the same training loss Independent while reducing the total FLOPS by 26%.

---

Published as a conference paper at COLM 2025
1B Independent
Independent
2B Independent
Independent
Independent
Independent
Independent
independent
Train loss
2B Progressive
Train loss
Progressive
48 Progressive
4B Prog esivessivee
8B Progresse essive
8B Progressive
FLOPs reduction
reduction
7.5
10.6
12.5
15.
17.5
28.0
0.0
10.0
12.3
15.5
17.5
20.0
ZFLOPs
ZFLOPs
Figure 2: Train loss curves comparing models trained by Independent versus Progressive approach with the 2x Chinchilla law setting. (Left): Models trained with fixed maximum learning rate, achieving 26% FLOPs reduction. (Right): Models trained with maximum learning rate adjustment, from 1.5 x 10 (1B) to 3.0 x 10 4 (8B), achieving 31% FLOPs reduction.
Independent Training
40RT
60RT
120BT
100BT
Progressive Training
10013
140B3T (New Tokens)
Progressive Training
100B1
140BT (Reused Tokens)
+ Fixed Data
Number of Training Tokens for 8B Model
320RT
Figure Comparison of token allocation for the 8B model training. (Top): Independent uses standard approach with 320B tokens. (Middle): Progressive uses 240B tokens (100B + new 140B tokens). (Bottom): Progressive-Fixe Data maintains the same 240B total tokens as Progressive by reusing 140B tokens from previous stages.
Does Progressive Training Exploit More Unique Data? No. For each model size X,(i 1) in our model family, we use fewer tokens in progressive training than independent training (T,Poog < TTeratch). However, when considering the entire model family construction process, progressive training consumes more total tokens than training only the largest model from scratch. As detailed in Section 4.2, the total number of tokens processed in progressive training is _ 7. 460B tokens, which exceeds the 75cratch 320B tokens used when training the 8B model independently, even though the total computational cost is the same (Figure 3, (Top) and (Middle)). Thus, possible that progressive training benefits unfairly from exposure to larger volume of unique training data. To isolate this factor, introduced controlled setting denoted as Progressiveds ixed Data, in which total amount of unique data is capped at 320B tokens the same amount used when independently training the 8B model. Once the 320B tokens were consumed during the training of the 8B model, the remaining 460 320 140B tokens are reused, i.e., the second epoch begins, as illustrated in Figure 3 (Bottom).
The result is shown in Table in which 8B Progressive ixed Data achieves comparable performance. This confirms that the observed improvements from progressive training are not attributable to access to a greater quantity of unique data.
4.4 Effectiveness Maximum Learning Rate Adjustment
In LLM pre-training, selecting an appropriate learning rate requires a careful balance between optimization efficiency and training stability. Higher learning rates can accelerate convergence and potentially lead to better final model performance (Takase et al., 2023), but they must be tuned carefully, especially with respect to model size. As models scale up,

---

Published as a conference paper at COLM 2025
Perplexity
Accuracy 1
Valid Wikitext LAMBADA ARC-e ARC-c Winogrande PIQA OBQA HellaSwag
1B
Independent
12.04
20.56
42.0
62.3
34.6
55.5
72.0
35.0
51.6
Independent
10.44
17.11
48.5
66.9
38.4
56.6
75.3
41.6
58.0
2B
Progressive
10.07
16.17
52.6
71.8
42.2
61.9
75.0
40.8
62.5
Independent
9.18
13.99
51.7
71.7
43.2
59.2
76.7
40.8
63.3
4B
Progressive
8.72
13.13
55.7
74.5
47.8
65.2
77.4
45.8
68.1
Independent
7.85
10.23
55.2
74.5
47.4
62,4
77.0
46.4
67.5
8B
Progressive 7.64
10.10
59.0
76.7
48.3
65.8
78.3
46.6
71.0
+Fixed Data 7.64
9.77
58.0
76.1
50.6
64.7
78.2
47.2
71.2
Table 3: Evaluation results of pre-trained models with the maximum learning rate ad- justment strategy described in Section 4.4. Progressive models consistently outperform Independent models. +Fixed Data indicates training on fixed dataset of 320B tokens.
they become increasingly sensitive the learning rate, typically requiring smaller learning rates to preserve training stability (Touvron et al., 2023; Wortsman et al., 2024). For example, in our preliminary experiments, we observed that learning rate of 1.5 x 10-3 enabled effective and stable training for 1B model. However, applying the same learning rate to an 8B model resulted in the loss value spikes that ultimately led to training collapse'.
Progressive training method is characterized by starting training from smaller models. Therefore, is possible to employ high learning rates to enhance performance during small- model training while lowering the learning rate for larger models to stabilize training. In this experiment, we apply this maximum learning rate adjustment under the 2x Chinchilla law setting, linearly decreasing the maximum learning rate from 1.5 x 10-3 for the 1B model to 3.0 x 10 for the 8B model, in accordance with the increase in model size, while maintaining cosine decay schedule during training for all models
Table 3 presents the results of our maximum learning rate adjustment strategy. Progressive consistently outperforms the Independent counterparts, with more substantial improve- ments than those observed with no learning rate adjustments (Table 2). As illustrated in Figure (Right), our maximum learning rate adjustment facilitated more efficient con- vergence during training, achieving 31% FLOPs reduction compared to 26% reduction observed without learning rate adjustment. These results demonstrate the effectiveness of our progressive training approach and show that appropriate learning rate adjustments can further enhance performance".
The Consistency across Model Family
In this section, we investigate the consistency of model behaviors across different sizes in our progressively trained model family. We first discuss the advantages of maintaining consistent behaviors across the model family (Section 5.1). We then present empirical analyses of consistency through probability distribution and speculative decoding, Our findings show that models trained using the progressive training approach exhibit higher consistency compared to independently trained models.
5.1 The Potential Effectiveness of Consistency across Model Family
Ensuring consistent behavior across model family offers several practical benefits. First, simplifies deployment flexibility by ensuring that switching between different parame-
A visualization of this training instability with the 8B model is provided in the Appendix C.
8The learning rates for each model size are described in Appendix D.
Moreover, shown in Appendix F. the benefits of our approach persist under post-training setups DPO), confirming its robustness for practical applications.
7

---

Published as a conference paper at COLM 2025
Model Pair
Training Approach
DxL [P
Independent
0.2821
1B-2B
Progressive
0.2162
Progressive+LR adjus tment
0.2538
Independent
0.3087
2B-+4B
Progressive
0.2265
Progressive+LR adjustment
0.2542
Independent
0,4584
4B-BBB
Progressive
0.3378
Progressive+LR adjustment
0.3518
Table 4: KL divergence between adjacent model sizes in the different model families. Lower values indicate greater consistency. Results are shown for models trained from scratch (Independent), models built through progressive training (Progressive), and models built with maximum learning rate adjustment (Progressive+LR adjustment).
ter sizes does not lead to large, sudden shifts in model outputs or user experience. This is particularly valuable when developers need to adapt model size to varying compu- tational budgets or runtime constraints, as they can easily scale up or down (e.g,, from smaller to larger model) without retraining users or extensively revalidating system performance (Srivastava et al., 2020; Echterhoff et al., 2024).
Second, such consistency enables more efficient incremental improvements or patches across the entire model family. For instance, if developers collect preference data or build reward model based on the outputs of one model (e.g., by annotating the specific texts that this member tends to generate), they typically face a distribution mismatch when applying those artifacts to other models. a consistent model family, however, differences in generation patterns are small enough examples or learned preference remain effective for other mod- els (Guo et al., 2024; Zhou et al., 2024a; Tajwar et al., 2024). Likewise, implementations such as safety content filters calibrated for one model will likely transfer to others with only minimal adjustments (Inan et al., 2023).
Third, consistency across models of different sizes benefits speculative decoding (Leviathan et al., 2023), a technique that accelerates inference by allowing a smaller model to generate "draft" outputs, which the larger model then either accepts refines. When the smaller and larger models produce similar probability distributions, the larger model is more likely to accept the drafts, reducing the frequency of rejections and subsequent regenerations (Zhou et al., 2024b). Consequently, consistent behavior across range of model sizes further contributes more efficient and user-friendly deployments real-world applications.
5.2 Probability Distribution Consistency across Model Family
The consistency of the underlying probability distributions across the model family offers deeper insights into how knowledge and behaviors are propagated. Such consistency indicates that models rely similar internal mechanisms when generating text.
Experimental Setup. To evaluate the probability distribution consistency across our model family, we measured the KL divergence between adjacent model sizes. KL divergence quantifies how one probability distribution differs from another reference distribution, with lower values indicating greater similarity. For this analysis, we used 10,000 examples from the FineWeb-Edu validation dataset. We calculated the KL divergence DKL (Px, || Px,,)) between adjacent models by examining their next-token prediction distributions, where Px; represents the probability distribution over the vocabulary given by model of size X, 10, We conducted this analysis for three different model families: (i) Models trained independently from scratch, (ii) Models built through progressive training, (iii) Models built through progressive training with maximum learning rate adjustment.
10ehe detailed calculation method for KT. divergence described in Appendix F.
8

---

Published as a conference paper at COLM 2025
Draft Model (4B)
Generator Model
DKL (Paaft
Pgenerator)
Acceptance Rate (%)
Generation Time (s)
(8B)
Progressive Independent
Progressive Progressive
0.3378
93.20
7.02
0.4281
87.20
8.24
Table 5: Speculative decoding performance with different draft-generator configurations. Progressive draft models demonstrate lower KL divergence with the generator, resulting in higher acceptance rates and faster generation times.
Results. Table 4 presents the KL divergence between adjacent model sizes for our three different model families. We found that, across all model pairs, models trained via progres- sive training demonstrate substantially lower KL divergence compared to independently trained models, indicating higher consistency their probability distributions. This trend is consistent across all adjacent model pairs (1B- 2B +4B, 4B-+BB.
This finding provides evidence that our progressive training produces a more coherent model family with consistent internal behavior. The lower KI. divergence suggests that progressively trained models share similar probability distributions and decision-making processes, offering practical advantages for techniques like speculative decoding, where alignment between smaller and larger models is crucial.
5.3 Validating Consistency Benefits through Speculative Decoding
We empirically assess whether this consistency translates into practical benefits for specula- tive decoding, as discussed in Section 5.1.
Experimental Setup. In speculative decoding, smaller "draft" model proposes multiple tokens that a large "generator" model subsequently accepts or rejects. For our experiments, we fixed the 8B model by Progressive training the generator and paired it with 4B draft models trained using either Progressive or Independent training. Since the effectiveness of speculative decoding depends on how well the draft model's predictions align with the generator's distribution, this setup validates whether the distributional consistency yields practical benefits. Each draft model was configured to propose 8 speculative tokens. To evaluate speculative decoding performance, we measured three key metrics on 1,000 prompts sampled from the FineWeb-Edu validation dataset. First, we computed the KL divergence DKL D (Pxamn P between the draft and generator models' output distri- butions. Second, we tracked the acceptance rate, which is the percentage of draft tokens accepted by the generator model. This metric directly reflects the efficiency of speculative decoding. Finally, we recorded the average generation time across all prompts to quantify the practical speedup.
Results. Table presents the speculative decoding performance with different draft- generator configurations. The results demonstrate the practical advantages of employing draft model constructed through Progressive Training. The pairing of a 4B Progressive draft model with an 8B Progressive generator model (Prog-Prog) exhibited superior performance, underpinned by greater consistency between the two models. This enhanced consistency is quantified by a lower KL divergence for the Prog-Prog pair, compared to using an independently trained model as drafter (Inde-Prog). This improved alignment in output distributions directly translated to more effective speculative decoding: the Prog-Prog configuration achieved higher acceptance rate of 93.20%, as opposed to 87.2% for the Inde-Prog setup. Consequently, this led to a reduction in inference time, with the Prog-Prog pair completing generation approximately 14.8% faster. These findings experimentally demonstrate that the consistency fostered by Progressive Training offers practical benefits. Soreennoorrgsseee
The lower KL divergence between progressively trained models leads to more effective draft proposals in speculative decoding, thereby improving acceptance rates and reducing inference latency.

---

Published as a conference paper at COLM 2025
6
Related Work
Model Family. Offering models in multiple sizes has become a standard practice in lan- guage model development address diverse computational requirements. Major research labs have released model families with multiple parameter sizes: Llama (Touvron et al., 2023; Al@Meta, 2024), Qwen (Yang et al., 2024a), Gemma (GemmaTeann 2024), and others. Scaling laws (Kaplan et al., 2020; Hoffmann et al., 2022) provide the theoretical foundation for these families, establishing relationships between model size, data, compute, and performance. Recent research highlights the complementary roles of models of different sizes (Wang et al., 2024a); while large models excel in zero/ /fooowewhttt generalization, smaller models offer advantages for satency-sensitive applications, edge deployments, domain-specific tasks, and privacy-sensitive contexts. This functional differentiation underscores the importance of efficient methods for constructing coherent model family that maintains consistent ca- pabilities across different parameter sizes. However, the conventional approach of training each model size independently incurs substantial computational costs that scale additively with the number of models in the family. Our work addresses this inefficiency by proposing progressive training that significantly reduces the total computation required to construct complete and coherent model family.
Model Expansion. Model expansion has emerged as an approach to reduce computational costs when training large-scale models. These methods leverage parameters of smaller, pre-trained models to initialize larger ones, accelerating training. The bert2BERT (Chen et al., 2022) adapted preeeeinpressrving methods to Transformers, reducing pre-training costs by approximately 45% for BERT models, LEMON (Wang et al., 2024b) refined this approach with an optimized learning rate scheduler for larger models. For scaling LLMs, Du et al. (2024) identified depthwise stacking as particularly effective, achieving 54.6% speedup for 7B-parameter models. Alternative approaches combine knowledge distillation with model expansion. Qin et al. (2022) introduced Knowledge Inheritance (KI), using knowledge distillation during pre-training to achieve approximately 27% computational cost reduction. However, these methods typically focus on the efficient training of single, final model rather than the training of an entire model family. For example, many methods demonstrate the effectiveness of the expansion in single-shot manner, i.e., applying model expansion only once. In addition, these methods do not take the performance of intermediate model(s) into account. In contrast, we propose progressive training and demonstrate that the off- the-shelf model expansion methods can be used to efficiently train a model family. The progressive training can incorporate emerging novel model expansion techniques, further enhancing efficiency.
7 Conclusion
In this paper, we proposed an efficient approach for constructing model family via progressive training, wherein smaller models are incrementally expanded to larger sizes. Through comprehensive experiments we demonstrated that our method significantly reduces the total computational cost required to construct complete model family by approximately 25% compared to training each model independently. Our method, particularly when combined with maximum learning rate adjustments tailored to each model size, not only matches but exceeds the performance of independently trained models. Furthermore, models built through our progressive approach demonstrate greater consistency across different sizes, as evidenced by lower KL divergence between probability distributions.
model family ranging from 1B to 8B parameters,
This work offers researchers and practitioners with a computationally efficient approach to constructing high-perf,,,ing, coherent model family, offering particular value in resource-oonstrained environments where training multiple models at different scales is desirable.
10

---

Published as a conference paper at COLM 2025
Acknowledgements
This work was supported by the JST Moonshot R&D Grant Number JPMJ]M2011-3 (fun- damental research).
References
Marah Abdin, Jyoti Aneja, Hany Awadalla, Ahmed Awadallah, Ammar Ahmad Awan, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Jianmin Bao, Harkirat Behl, et al. Phi-3 technical report A highly capable language model locally on your phone. arviv preprint MwXi::244411229, 2024.
AI@Meta. The llama 3 herd of models, 2024. URL httpp:://akkiei htttss///axxi..orr////////...
Yonatan Bisk, Rowan Zellers, Ronan bras, Jianfeng Gao, and Choi Yejin. Piga: Reasoning about physical commonsense in natural language. Proceedings of the AAAÍ Conference on Artificial Intelligence, 34:7432-7439, 04 2020. doi: 10.1609/aaai.vv2 /wap://aiivv..... vaii..5340056229
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877-1901, 2020.
Cheng Chen, Yichun Yin, Lifeng Shang, Xin Jiang, Yujia Qin, Fengyu Wang, Zhi Wang, Xiao Chen, Zhiyuan Liu, and Qun Liu. bert2BERT: Towards reusable pretrained language mod- els. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (eds.), Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 2134-21483 Dublin, Ireland, May 2022. Association for Computational Linguistics. doi: 10.18653 /0003/vv1/220/ 2022.acl-ooo 151. URL htttss://aclatteee
org/oog//22 acl-long. 151/.
Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try arc, the ai2 reasoning challenge, 2018
Ganqu Cui, Lifan Yuan, Ning Ding, Guanming Yao, Bingxiang He, Wei Zhu, Yuan Ni, Guo- tong Xie, Ruobing Xie, Yankai Lin, Zhiyuan Liu, and Maosong Sun. ULTRAFEEDBACK. Boosting language models with scaled AI feedback. In Forty-first International Conference on Machine Learning, 2024. URL httpss://pperrvview.ncc net/ wtmm:///fo
Ning Ding, Yulin Chen, Bokai Xu, Yujia Qin, Shengding Hu, Zhiyuan Liu, Maosong Sun, and Bowen Zhou. Enhancing chat language models by scaling high-quality instructional conversations. Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Proceedings of the 2023 Conference Empirical Methods in Natural Language Processing, pp. 3029-3051, Singapore, December 2023. Association for Computational Linguistics. doi: 10. (0.1866/vll 2023.emmlp-main 183. URL https: httttp://aclanthholggyy wrgg//2/3333 emnlp-mainn main. 183/.
Wenyu Du, Tongxu Luo, Zihan Qiu, Zeyu Huang, Yikang Shen, Reynold Cheng, Yike Guo, and Jie Fu. Stacking your transformers: A closer look at model growth for efficient LLM pre-training. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URL https: http:////pppeeeie../mm wttp://ww.ee//ooomn.idd..o...o.
Jessica Maria Echterhoff, Fartash Faghri, Raviteja Vemulapalli, Ting- Yao Hu, Chun-Liang Li, Oncel Tuzel, and Hadi Pouransari. MUSCLE: model update strategy for compatible LLM evolution. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.), Findings of the Association for Computational Linguistics: EMNLP 2024, pp. 7320-7332, Miami, Florida, USA, November 2024. Association for Computational Linguistics. doi: 10.18653 06653/v1/202. findings-emnlp.455 430. URL https:://// httttt:///cclleeee hatps:///ccla org/ 2024. Findings-emnlp..r 4301..430/.
Tio
Gemma Team. Gemma: Open models based on gemini research and technology. arXiv preprint ar arXio:2403.08295, 2024.
GemmaTean Gemma 3 technical report. ar preprint ar arXii:2503.19788 2025.
11

---

Published as a conference paper at COLM 2025
Shangmin Guo, Biao Zhang, Tianlin Liu, Tianqi Liu, Misha Khalman, Felipe Llinares, Alexandre Rame, Thomas Mesnard, Yao Zhao, Bilal Piot, et al. Direct language model alignment from online ai feedback. aro preprint ar arXio:2402.0477 047922 2024.
Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de las Casas, Lisa Anne Hendricks, Johannes Welbi, Aidan Clark, Tom Hennigan, Eric Noland, Katherine Millican, George van den Driessche, Bogdan Damoc, Aurelia Guy, Simon Osindero, Karen Simonyan, Erich Elsen, Oriol Vinyals, Jack William Rae, and Laurent Sifre. An empirical analysis of compute-optimal large language model training. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Advances in Neural Information Processing Systems, 2022. URL https: https:// //pper////prr.view... net/ forum21d=1BBRRUIOAP.
Hakan Inan, Kartikeya Upasani, Jianfeng Chi, Rashi Rungta, Krithika Iyer, Yuning Mao, Michael Tontchev, Qing Hu, Brian Fuller, Davide Testuggine, et al. Llama guard: LLM-based input-output safeguard for Human-Al conversations. arXiv preprint arXiv:2312.06674, 2023.
Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. Xiv preprint arXi::2001.0836 2020.
Yaniv Leviathan, Matan Kalman, and Yossi Matias. Fast inference from transformers via speculative decoding. In International Conference on Machine Learning, pp. 19274-19286. PMLR, 2023.
Yu Meng, Mengzhou Xia, and Danqi Chen. SimPO: Simple preference optimization with a reference-free reward. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URL httpps://pperreview.. het.e/e.nnttooooomnmdd3toooo
Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. Pointer sentinel mixture models. In International Conference on Learning Representations, 2017. net http://wwnee//ooomidddiiir..e
2017. URL https:
Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. Can a suit of ar- mor conduct electricity? a new dataset for open book question answering. In Ellen Riloff, David Chiang, Julia Hockenmaier, and Jun'ichi Tsujii (eds.), Proceedings of the 2018 Conference Empirical Methods in Natural Language Processing, pp. 2381-2391, Brus- sels, Belgium, October-Novemble 2018. Association for Computational Linguistics. doi: (1116653/11/118-111 1260. URL https: / wwtpetthoooogyooo/1ereoee
Denis Papemn,, Germán Kruszewski, Angeliki Lazaridou, Ngoc Quan Pham, Raffaella Bernardi, Sandro Pezzelle, Marco Baroni, Gemma Boleda, and Raquel Fernández. The LAMBADA dataset: Word prediction requiring a broad discourse context. In Katrin Erk and Noah A. Smith (eds.), Proceedings of the 54th Anmual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1523-1534, Berlin, Germany, August 2016. Association for Computational Linguistics. doi: 10.18653 20.18633/v1/PP0 /1/33/111P6111/ URL /aapp//oo//..aaahollooo..../ wrwp:/eyyooo///1-/11111111
Guilherme Penedo, Hynek Kydlíček, Loubna Ben allal, Anton Lozhkov, Margaret Mitchell, Colin Raffel, Leandro Von Werra, and Thomas Wolf. The FineWeb datasets: Decanting the Web for the finest text data at scale. In The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2024. URL /tttss///pperrweee net/
Yujia Qin, Yankai Lin, Jing YI, Jiajie Zhang, Xu Han, Zhengyan Zhang, Yusheng Su, Zhiyuan Liu, Peng Li, Maosong Sun, and Jie Zhou. Knowledge inheritance for pre-trained language models. In Marine Carpuat, Marie-Cathering de Marneffe, and Ivan Vladimir Meza Ruiz (eds.), Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 3921-3937, Seattle, United States, July 2022. Association for Computational Linguistics. doi: 10.18653/v1/20 naacl-main 288. URL hwtpt:t//tclaaeae //w./t/hollan/ooyyooooo.. org/2022. aaac-mmain 288/.
12

---

Published as a conference paper at COLM 2025
Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and llya Sutskever. Language models are unsupervised multitask learners, 2019.
Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly reward model. In seirth-seventhh Conference on Neural Information Processing Systems, 2023. URL https://openrevieww hett/forum?id=HPPSIXX.aa
Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. Winogrande: An adversarial winograd schema challenge at scale. Communications of the ACM, 64(9):99-106, 2021.
Nikhil Sardana, Jacob Portes, Sasha Doubov, and Jonathan Frankle. Beyond chinchilla- optimal: Accounting for inference in language model scaling laws. In Forty-first Infer- national Conference on Machine Learning, 2024. URL https::/openrewe wwwpeerreeeeeeeeeeorrrieee net/orrm?dd @bmXrtTDUu.
Megha Srivastava, Besmira Nushi, Ece Kamar, Shital Shah, and Eric Horvitz. An empirical analysis of backward compatibility in machine learning systems. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD '20, pp. 3272-3280 New York, NY, USA, 2020. Association for Computing Machinery. ISBN 9781450379984. doi:
URL
org.0...rrg/.... 145/3394485
3403379.
Fahim Tajwar, Anikait Singh, Archit Sharma, Rafael Rafailov, Jeff Schneider, Tengyang Xie, Stefano Ermon, Chelsea Finn, and Aviral Kumar. Preference fine-tuning of LLMs should leverage suboptimal, on-policy data. In Proceedings of the 41st International Conference on Machine Learning, 2024.
Sho Takase, Shun Kiyono, Sosuke Kobayashi, and Jun Suzuki. Spike no more: Stabilizing the pre-training of large language models. arxio preprint ar XrXiv:2312.16903, 2023.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXio:2302.13971, 2023.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural informa- fion processing systems, 30, 2017.
Fali Wang, Zhiwei Zhang, Xianren Zhang, Zongyu Wu, Tzuhao Mo, Qiuhao Lu, Wanjing Wang, Rui Li, Junjie Xu, Xianfeng Tang, et al. A comprehensive survey of small language models in the era of large language models: Techniques, enhancements, applications, collaboration with LLMs, and trustworthiness. arXiv preprint arXiv:2411.03350 2024a.
Yite Wang, Jiahao Su, Hanlin Lu, Cong Xie, Tianyi Liu, Jianbo Yuan, Haibin Lin, Ruoyu Sun, and Hongxia Yang. LEMON: Lossless model expansion. In The Twelfth International Conference on Learning Representations, 2024b. URL https:: http://ppeeeeeeeeeeeeeeeoreeeeee 3VW7DQqq7U.
Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yo- gatama, Maarten Bosma, Denny Zhou, Donald Metzler, Ed H. Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, and William Fedus. Emergent abilities of large language models. Transactions on Machine Learning Research, 2022. ISSN 2835-8856. URL htte//oormm?idyyykssszaee
https://openrevier
Survey Certification.
Mitchell Wortsman, Peter J Liu, Lechao Xiao, Katie Everett, Alexander A Alemi, Ben Adlam, John D Co-Reyes, Izzeddin Gur, Abhishek Kumar, Roman Novak, Jef- frey Pennington, Jascha Sohl-Dickstein, Kelvin Xu, Jaehoon Lee, Justin Gilmer, and Simon Korblithh. Small-scale proxies for large-scale transformer training instabili- X ties. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.c wttp://www..eeeorrrra/dddeeeaaaa
13

---

Published as a conference paper at COLM 2025
Configuration
1B
2B
4B
8B
Hidden dimension
2048
2560
3200
4096
FFN dimension
7168
8960
11200
14336
Layers
22
27
33
Heads
16
20
25
32
Batch size
960
1920
3840
7680
The number of updates
40700
40700
40700
40700
learning rate warmup fraction
0.01
0.01
0.01
0.01
Learning rate decay style
cosine
cosine
cosine
cosine
Adam B1
0.9
0.9
0.9
0.9
Adam B2
0.95
0.95
0.95
0.95
Gradient clipping
1.0
1.0
Weight decay
0.1
0.1
0.1
0.1
Precision
bfloat16
bfloat16
bfloat16
1float16
Table Experimental configuration for each model size
An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Ma, Jianxin Yang, Jin Xu, Jingren Zhou, Jinze Bai, Jinzheng He, Junyang Lin, Kai Dang, Keming Lu, Keqin Chen, Kexin Yang. Mei Li, Mingfeng Xue, Na Ni, Pei Zhang, Peng Wang, Ru Peng, Rui Men, Ruize Gao, Runji Lin, Shijie Wang, Shuai Bai, Sinan Tan, Tianhang Zhu, Tianhao Li, Tianyu Liu, Wenbin Ge, Xiaodong Deng, Xiaohuan Zhou, Xingzhang Ren, Xinyu Zhang, Xipin Wei, Xuancheng Ren, Xuejing Liu, Yang Fan, Yang Yao, Yichang Zhang, Yu Wan, Yunfei Chu, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, Zhifang Guo, and Zhihao Fan. Qwen2 technical report, 2024a. URL https://arxtt or wtggbbs///a 2400 10671.
An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. technical report. arXiv preprint ar arXiv:2412.15115 2024b.
Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. HellaSwag: Can machine really finish your sentence? In Anna Korhonen, David Traum, and Lluís Mårquez (eds.), Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 4791-4800, Florence, Italy, July 2019. Association for Computational Linguistics. doi: 10.18653 00.18653/v1/P19-1 1v/211--11. URL htttps://aclattiii http::///oeoaaaooooooooo//////// Pr9/
Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, Hao Zhang, Joseph E. Gonzalez, and lon Stoica. Judging LLM-as-a-judge with MT-bench and chatbot arena. In Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2023.
URL https::/tooooo /otpnrvii/peeevvieww httt::///oeeeeevvww...//////nont net//oormmidduuccccGl...
Wenxuan Zhou, Ravi Agrawal, Shujian Zhang, Sathish Reddy Indurthi, Sanqiang Zhao, Kaiqiang Song, Silei Xu, and Chenguang Zhu. WPO: Enhancing RLHF with weighted preference optimization. In Proceedings of the 2024 Conference Empirical Methods in Natural Language Processing, November 2024a. doi: 10.18653 10.18653/v1/2024 http////22224..emmmm// URL BRRLhttts://aalannholee. org/ 2024. emnlp-main 475/.
Yongchao Zhou, Kaifeng Lyu, Ankit Singh Rawat, Aditya Krishna Menon, Afshin Ros- tamizadeh, Sanjiv Kumar, Jean-François Kagy, and Rishabh Agarwal. Distillspec: Improv- ing speculative decoding via knowledge distillation. In The Twelffh International Conference on Learning Representations, 2024b. URL http://ppnrreiieeeeeee/oorrrnn//
A Details of Experimental Configurations
In our experiments (Section 4), we constructed models with 1B, 2B, 4B, and 8B parameters based on the Llama architecture. Table 6 shows the specific experimental configurations for
14

---

Published as a conference paper at COLM 2025
Model Size
Tpcratch (Independent)
7, (Progressive)
1B
20B
20B
2B
40B
30B
4B
80B
60B
8B
160B
120B
Table 7: Training token allocation for Independent and Progressive the Chinchilla law setting.
Model Size
Thcratch (Independent)
T (Progressive)
1B
40B
40B
2B
80B
60B
4B
160B
120B
8B
320B
240B
Table 8: Training token allocation for Independent and Progressive the 2x Chinchilla law setting.
each model size. As the model expands, we gradually increase the hidden dimension, FFN dimension, number of layers, and number of heads.
B Data Size for (2x) Chinchilla Law Setting
As described in Section 4.2, we determined the number of training tokens for each model size according to Hoffmann et al. (2022). We report token allocations for each model in Table (Chinchilla law setting) and Table (2x Chinchilla law setting).
C 8B Model Loss Visualization
Figure 4 illustrates the training loss curves for 8B models trained from scratch with different
learning rates: 3.0 x 10 and 1.5 10 While the lower learning rate of 3.0 x 10 4 results
in stable training, the higher learning rate of 1.5 x 10-3 causes significant loss spikes during training.
D Specific Configurations in the Learning Rate Adjustment Strategy
In our learning rate adjustment strategy (Section 4.4), decrease the maximum learning rate as the model size increases. We apply maximum learning rates of 1.5 10 3.1. 10 7.0 10-4 and 3.0 x 10-4 for the 1B, 2B, 4B, and 8B models, respectively. This gradual reduction accommodates the increasing sensitivity of larger models to high learning rates.
E Experiment: Model Family Post-Training
This experiment verifies whether the trends observed in pre-training experiments in Sec- tion 4 are also seen in post-training. We examine whether models built using progressive training with fixed learning rates can achieve comparable performance to models trained from scratch. In addition, we confirm that performance improvements observed with maximum learning rate adjustments in pre-training also extend to post-training.
E.1 Experimental Setup
We evaluate the effectiveness of progressive training by conducting post-training on the model family constructed in Section 4. We adopted the settings from Meng et al. (2024) for
15

---

Published as a conference paper at COLM 2025
14
Ir 3.0e-4
Ir 1.5e-3
12
10
Train loss
8-
6
2
2
6
8
10 12
14
16
ZFLOPs
Figure 4: Training loss curves comparing 8B models trained from scratch with different learning rates. The model trained with learning rate of 3.0 x 10-4 (blue line) shows stable training, while the model trained with higher learning rate of 1.5 10 (orange line) exhibits severe loss spikes indicating training instability.
Adjusted LR 2x Chinchilla MT-Bench
Size
Training Approach
Chinchilla MT-Bench f
2x Chinchilla MT-Bench
Independent Progressive
1.61
2.04
2.04
2B
2.03
2.46
3.22
Independent
2.78
3.02
3.02
4B
Progressive
2.87
3.01
3.63
Independent
3.42
3.45
3.45
8B
Progressive
3.28
3.21
3.96
ProgressiverFixed Data
3.32
3.41
3.98
Table 9: MT-Bench evaluation results after post-training (SFT+DPO) for models trained with different approaches. We compare models trained from scratch (Independent) versus those built using our progressive training approach (Progressive) under both fixed maximum learning rate and learning rate adjustment (Adjusted LR) strategies. For the fixed learning rate, results are shown for both the Chinchilla law and 2x Chinchilla law settings. For the models with learning rate adjustments, the maximum learning rates are decreased from 1.5 x 10 3 for 1B to 3.0 10 for 8B as the model size increases.
post-training. Specifically, we performed Supervised Fine-Tuning (SFT) using the UltraChat- 200k dataset (Ding et al., 2023), followed by DPO (Rafailov et al., 2023) using the Ultrachat Feedback dataset (Cui et al., 2024).
The maximum learning rate was set to 3.0 x 10-5 for SFT and 5.0 x 10 for DPO, stan- dardized across all models. We evaluated the response quality of each model using MT- Bench (Zheng et al., 2023) using gpt-4-0613 models (2B, 4B, 8B) independently trained from scratch and conducted the same post- training.
evaluator. For comparison,
prepared
16

---

Published as a conference paper at COLM 2025
E.2
Results
Table 9 presents the MT-Bench evaluation results after post-training across different training approaches. With fixed learning rates, models trained with progressive training perform comparably to models trained from scratch, with some variation across model sizes.
When applying our maximum learning rate adjustment, progressive training consistently outperforms from-scratch training across all model sizes, with substantial improvements observed. The models with maximum learning rate adjustment significantly outperform those with fixed maximum learning rates for progressive training.
These results indicate that our progressive training approach, particularly when combined with maximum learning rate adjustment, can deliver both computational efficiency and enhanced model performance after post-training.
KL Divergence Computation
To simplify the computation, at each token position, we computed DKL (Px, Px++1) = Leev Px, (v) (log Px, (v) - log ( where V is the vocabulary and Px, (v) is the prob- ability assigned to vocabulary token by model X,. We first averaged these KL divergence values across all token positions within each example, then computed the final KL. diver- gence by averaging across all 10,000 examples from the FineWeb-Edu validation dataset, as described in Section 5.2.
17


Reviews:
[{"review_id": "Z1gdy1x1s4", "reviewer": "Reviewer_hRL8", "summary": "This paper presents a method for constructing a family of language models using progressive training through model expansion. The authors claim a 25% reduction in computational cost compared to training each model independently, while maintaining comparable performance. Additional benefits include consistent behavior across different model sizes.", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The problem of efficiently training a model family is relevant and timely, particularly with increasing deployment of LLMs across diverse platforms and constraints.\n- The proposed progressive training pipeline is sensible and leverages model expansion effectively.\n- The method shows empirical improvements over independent training baseline in terms of both computational cost and performance on downstream tasks.", "reasons_to_reject": "- The novelty of paper is limited. The method employs existing off-the-shelf model expansion tools. The idea of progressive training is also not new. The paper essentially repackages them for constructing a model family without proposing a fundamentally new algorithm or mechanism.\n- The most interesting aspect is the problem definition and the constraints on the FLOP computations and the token budgets. However, this is not algorithmically addressed. In the experiments, instead of searching or optimizing for configurations under constraints, the authors use a predefined token budget schedule. This makes the paper pure empirical without any new proposed methods or surprising findings.\n- Section 5.1 provides an extended discussion on the advantages of consistency across model sizes, such as in speculative decoding. However, these claims are mostly speculative. While KL divergence analysis (Section 5.2) does quantify similarity across adjacent models, the claimed practical benefits are not experimentally verified.", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "XoJlmcJETr", "reviewer": "Reviewer_D8FB", "summary": "This paper proposed a new method for training a LLM series containing models of varying sizes. Instead of training models of different sizes independently (from scratch), the authors propose to only train the smallest model from scratch, while larger models are instantiated from the immediately smaller model using the model expansion method, bert2BERT (AKI). Empirical results show that this method, named progressive training, can reduce the total training FLOPs of the model series by roughly 25%, while getting models with more consistent behavior (output distributions).", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "* The paper is very well-written. The motivation is clear, the presentation is very easy to follow.\n\n* The method is effective and reasonable. It also has great practical value and is interesting to many LLM practitioners.\n\n* The method is validated on up to models up to 8B parameters with more hundreds of billions of training tokens, which verifies the scalability of the contributions.", "reasons_to_reject": "* The 25% reduction in training compute seems rather small in the large picture. Many existing research (data engineering techniques, architectural changes, etc.) works have shown much greater compute savings. Moreover, this only reduces the training cost, but not the inference cost. So the total compute saving throughout the life of a model family is even lower when accounting for inference costs.\n\n* It is more common to use different learning rates (LR) for different model sizes. Thus, the choice of using the same LR of 3e-4 for all models (mentioned in Line 117) is unreasonable. This also makes Section 4.4 unreasonable, since the proposed maximum LR is the more common practice in the community. This also means that the “independent” baseline reported in Table 3 and 8 is unreasonable, because it should have adjusted its maximum LR, which likely results in better performance.\n\n* While the arguments regarding the potential effectiveness of model consistency (in Section 5.1) are convincing, the authors should have performed some experiments to show the actual quantitative benefits we can gain from employing some of the techniques mentioned (e.g., speculative decoding). It is very difficult to understand the significance of the reduction in KL divergence between different model sizes just from the numbers in Table 4.", "questions_to_authors": "* Why does using repeated data outperform the model that uses unique data (shown in Table 2)?\n\n* Can we perform progressive training in a more fine-grained manner? For instance, start from 1M parameters and progressively scale up to 8B?\n\n* Is the expansion method compatible with muP [1], which has become a popular technique in LLM training?\n\n* I think it is more common to have a space between the number before “ZFLOPs” in, for instance, “20.4ZFLOPs”\n\nReferences:\n\n[1] Yang et al. 2021. “Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer”", "extra_scores": {}}, {"review_id": "b5mNQy7POg", "reviewer": "Reviewer_oPhZ", "summary": "This paper proposes progressive model expansion. Compared with single‑shot bert2BERT or LEMON, the novelty is treating every intermediate checkpoint as a usable model and tuning the LR per stage. On a FineWeb‑Edu pre‑train + SFT + DPO pipeline, the 8 B progressive family matches or slightly beats scratch baselines while saving around 25 – 31 % compute and yielding smoother KL between sizes.", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- Simple & reproducible: re‑uses public bert2BERT and standard optimisers.  \n- Thorough evaluation: 2 compute budgets, 8 downstream tasks, etc.\n- Smoother speculative‑decoding pipeline thanks to lower intra‑family KL.\n- Practical impact: ‑25 % compute at 8 B saves thousands of GPU‑hours.", "reasons_to_reject": "- Using LLM up to 8 B -- unclear at ≥70 B.  \n- Incremental novelty: cascaded expansion is a straightforward extension.  \n- Compute vs. data: progressive run still sees 2× epochs (possible confound).\n- No robustness or safety analysis: bias and toxicity are not reported.\n- There might be more related work relevant to be mentioned (e.g., https://www.arxiv.org/abs/2502.07832, https://arxiv.org/abs/2402.11700).", "questions_to_authors": "1. Does the method remain stable at even larger models?\n2. How would depth‑only or LEMON expansion change results?  \n3. What is the val‑ppl change on reused tokens in the Fixed‑Data ablation?", "extra_scores": {"ethics_concerns_details": "n/a"}}, {"review_id": "OHOFgnyBm4", "reviewer": "Reviewer_Az33", "summary": "This paper aims to train a family of LLMs efficiently. Specifically, the authors propose to progressively train the model family from small to large ones. When training a larger model, bert2BERT method is utilized to initialize the larger model from smaller ones. Experiments show good results, with comparable performance while reducing computation overhead.", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. This paper tackles an important and interesting problem. Training a family of models is important.\n2. The ideas are well validated by good experimental results in convincing settings.", "reasons_to_reject": "1. The idea is simply to apply the bert2BERT method to LLMs for model expansion. Although this new experimental setting is good and interesting, the novelty is relatively limited.\n2. The intuition and insight are not well illustrated. Why can progressive training lead to better results? How does this intuition guide the design choices of each component? This can be made more clear.\n3. Missing reference. There are many model expansion works in the literature. This paper cites only very few of them. The missing reference can be supplemented. For example, Knowledge Inheritance for Pre-trained Language Models. NAACL 2022.", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This work presents a method for progressive training a family of language models—that is, gradually expand the model as it trains to obtain multiple complete language models in a signle training run. The paper experiments with a model family of 1B to 8B parameters; results show that the proposed model achieves same results of independently trained models while achieving a 25% reduction in compute budget.

All reviews agree that this work is very relevant and timely; they also remark the soundness of the proposed method, as well as the empirical evidence provided. Some also praise the ease of reproducibility, asn well as the writing style.

Reviewers raise three main concerns with this work: lack of novelty, some experimental configurations (namely, LR tuning), as well as claims in sections 5.2 not being supported by experimental evidence. During rebuttal, authors address the last two with additional experimental evidence. With respect of lack of novelty, I think the ample experimental provided by authors more than make up for it, as it makes this method considerably easier to adopt by practicioners.

Authors should make sure to incorporate ALL aditional experiments and reference provided by authors in their final camera ready. 

### Pros

- topic of efficient pretraining of a family of LLM is timely and important (hRL8, D8FB, Az33)
- the proposed pipeline is sound (hRL8, D8FB)
- empirical evidence of proposed method (hRL8, D8FB, Az33)
- paper well written (D8FB)
- work is easy to reproduce (oPhZ)
- -25% is significant reduction in compute hours (oPhZ)
- more effective speculative decoding  pipeline (oPhZ)

### Cons

- limited novelty: off-the-shelf methods, and progressive training is not new (hRL8, oPhZ, Az33)
- training configurations are arbitrarily chosen, rather justified by experiments or theoretical framework (hRL8)
- authors do not provide empirical evidence that progressively constructed model family helps with speculative decoding (hRL8, D8FB)
  - **addressed in rebuttal**
- -25% is limited reduction in compute costs (D8FB)
- LR is not tuned per model configuration (D8FB)
  - **addressed in rebuttal**
- speculative decoding run sees 2x the epochs (oPhZ)
- no robustness or safety benchmakrs (oPhZ)
- some related work missing (oPhZ)
- intuition behind the proposed method is unclear (Az33)
- analysis limited to 8B models; unclear how much the approach would scale past that (oPhZ,Az33)
Decision: Accept