Bohdi: Heterogeneous LLM Fusion with Automatic Data Exploration
Authors: Junqi Gao
Zhichang Guo
Dazhi Zhang
Dong Li
Runze Liu
Pengfei Li
Kai Tian
Biqing Qi
Venue: NeurIPS 2025 poster ()
Keywords: Heterogeneous Model Fusion
Large Language Models
Primary area: applications

Abstract: Heterogeneous Large Language Model (LLM) fusion integrates the strengths of multiple source LLMs with different architectures into a target LLM with low computational overhead. While promising, existing methods suffer from two major limitations: 1) **reliance on real data from limited domain** for knowledge fusion, preventing the target LLM from fully acquiring knowledge across diverse domains, and 2) **fixed data allocation proportions** across domains, failing to dynamically adjust according to the target LLM's varying capabilities across domains, leading to a capability imbalance. To overcome these limitations, we propose Bohdi, a synthetic-data-only heterogeneous LLM fusion framework. Through the organization of knowledge domains into a hierarchical tree structure, Bohdi enables automatic domain exploration and multi-domain data generation through multi-model collaboration, thereby comprehensively extracting knowledge from source LLMs. By formalizing domain expansion and data sampling proportion allocation on the knowledge tree as a Hierarchical Multi-Armed Bandit problem, Bohdi leverages the designed DynaBranches mechanism to adaptively adjust sampling proportions based on the target LLM's performance feedback across domains. Integrated with our proposed Introspection-Rebirth (IR) mechanism, DynaBranches dynamically tracks capability shifts during target LLM's updates via Sliding Window Binomial Likelihood Ratio Testing (SWBLRT), further enhancing its online adaptation capability. Comparative experimental results on a comprehensive suite of benchmarks demonstrate that Bohdi significantly outperforms existing baselines on multiple target LLMs, exhibits higher data efficiency, and virtually eliminates the imbalance in the target LLM's capabilities.

Bohdi: Heterogeneous LLM Fusion with Automatic Data Exploration
Junqi Gao Zhichang Guo', Dazhi Zhang', Dong Li'
Runze Liu", Pengfei Li Kai Tian', Biqing Qi
School of Mathematics, Harbin Institute of Technology
Shanghai Artificial Intelligence Laboratory
Tsinghua Shenzhen International Graduate School, Tsinghua University
Department of Electronic Engineering, Tsinghua University
5 Shanghai Innovation Institute
g gjunq1970gmail.commmmm com, www.giiggommmmm...ooo
Abstract
Heterogeneous Large Language Model (LLM) fusion integrates the strengths of multiple source LLMs with different architectures into a target LLM with low computational overhead. While promising, existing methods suffer from major limitations: 1) reliance on real data from limited domain for knowledge fusion, preventing the target L.LM from fully acquiring knowledge across diverse domains, and 2) fixed data allocation proportions across domains, failing to dynamically adjust according the target LLM's varying capabilities across domains, leading to a capability imbalance. To overcome these limitations, we propose Bohdi, a synthetic-data-only heterogeneous LLM fusion framework. Through the orga- nization of knowledge domains into a hierarchical tree structure, Bohdi enables automatic domain exploration and multi-domain data generation through multi- model collaboration, thereby comprehensively extracting knowledge from source LLMs. By formalizing domain expansion and data sampling proportion alloca- tion on the knowledge tree as a Hicrarchical Multi-Armed Bandit problem, Bohdi leverages the designed DynaBranches mechanism adaptively adjust sampling proportions based on the target LLM's performance feedback across domains. Inte- grated with our proposed Introspection-Rett dynamically tracks capability shifts during target LLM's ' updates via Sliding Win- dow Binomial Likelihood Ratio Testing (SWBLRT), further enhancing its online adaptation capability. Comparative experimental results on comprehensive suite of benchmarks demonstrate that Bohdi significantly outperforms existing baselines multiple target LLMs, exhibits higher data efficiency, and virtually eliminates the imbalance in the target LLM's capabilities. Our code available ? Bohdi
(IR) mechanism, DynaBranches
1 Introduction
With the rapid advancement of Large Language Model (LLM) [ 2 capabilities and the thriving
development of open-source communities, both enterprises and individual users are actively engaged
in building their own LLMs. As an efficient solution, LLM fusion B 4
5 o technology consolidates
the advantages multiple source LLMs into a more compact target LLM without the need for
"This work was done during his internship Shanghai Artificial Intelligence Laboratory. Cooresponning author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

L.ama3.2388-inteee
Llama3. Hamma.2238--mae 1-a.aa.2------ntttt-oaoo (Ours)
Gemma2-9B-IT
Gemmaa-98-1T-Bhhhl (Ours)
L6
22.
ML 23.9
2
10 37.8
38.8
O
815
Bobdi
Bohdi
23.
9
SUS
8'
Figure 1: Results of Bohdi across various benchmarks, using Gamma-.2-3B-Insstie (left) and Gemma2-9B-IT (right) as target LLMs.
pretraining from scratch or post-training based on large amounts of data. This significantly shortens the development cycle of private LLMs and reduces the required computing resources.
Due to the differences in architecture and size among open soures LLMs, there is always a heterogene- ity between source and target LLMs. This limits knowledge fusion through direct parameter merging approaches [3 and has driven research into heterogeneous LLM fusion, with methods mainly divided into two types: Explicit Fusion (EF) and Implicit Fusion (IF). EF methods 6 an initial vocabulary alignment, followed by supervised training of the target LLM using output probability distributions collected from the source LLMs on given dataset. However, the vocabulary alignment process introduces errors and noise 77 compromising the target LLM's learning stability, In contrast, IF methods 7 8 g collect responses from the source LLMs and uses a external reward LLM to select high-quality response for end-to-end training of the target LLM, thereby bypassing the vocabulary alignment process and enabling more stable knowledge injection.
involve
While demonstrating promising performance, current heterogeneous LLM fusion methods still face two key limitations: 1) Limited Domain Coverage: Due to the scarcity of real data in systematically finely-grained domains, current approaches are limited to relying on open-source data from a few fixed domains (e.g., mathematics, coding). This prevents the target LLM from fully acquiring knowledge from source LLMs across diverse domains, thereby restricting comprehensive performance improvement. 2) Rigid Data Allocation: Current methods pre-specify the data proportions allocated to each domain. According the "Buckets effect", ideal data allocation should emphasize domains where the target L.LM performs poorly. However, fixed data proportion assignments cannot adapt to the varying data proportion needs of difffrrnnt target LLMs, leading to capability imbalance where improvements in some capabilities come at the expense of others.
Addressing the aforementioned limitations, we propose BOHDI, a heterogeneous LLM fusion framework. By constructing a hierarchical knowledge tree, Bohdi systematically manages multi- domain knowledge. Equipped with the designed Sprout and Harvest operations, Bohdi enables dynamic exploration of new domains through multi-model collaboration and automatic generation of data from each domain, thus achieving more comprehensive domain expansion without relying on real data. To achieve adaptive adjustment of domain data proportions, we formalize dynamic domain expansion and sampling proportion allocation on the knowledge tree as a Hierarchical Multi-Armes Bandit (HMAB) problem. We then propose DynaBranches, which employs Thompson Sampling (TS) 110 to efficiently sample multi-domain data and adaptively update sampling proportions for each domain through the Sprout and Harvest operations. Combined with our proposed Introspection- Rebirth (IR) mechanism, DynaBranches uses the designed Sliding Window Binomial Likelihood Ratio Test (SWBLRT) to dynamically detect changes in domain capabilities during the target LLM's updates, thereby avoiding over-reliance outdated observations in the dynamic sampling process.
Integrating the aforementioned components and mechanisms, Bohdi's ss fusion process is modeled as an iterative two-phase optimization: MEDITATION and ENLIGHTENMENT The Meditation phase focuses on domain exploration, data collection, and adjustment of domain data proportions, while the Enlightenment phase trains the target LLM based on the current optimal sampling proportions.

---

Through the alternating iterative execution of these two phases, Bohdi achieves dynamic knowledge domain expansion and multi-doman source LLM knowledge acquisition without relying on any real data. It also enables online adaptive adjustment of domain data proportions, effectively injecting multi-domain knowledge from multiple source L.LMs. Comprehensive comparative experiments across range of benchmarks demonstrate that Bohdi achieves significant performance improvements over existing baselines on multiple target LLMs without relying on real data, while also exhibiting higher data efficiency and virtually eliminates the imbalance in the target LLM's capabilities. The contributions our work can be summarized as follows:
We propose Bohdi, the first synthetic-data-one heterogencous LLM fusion framework that man- ages multi-domain knowledge through hierarchical knowledge tree, while enabling automated domain expansion and data collection via multi-modee collaboration.
By formalizing the problem of domain expansion and sampling proportion allocation as HMAB problem, we propose DynaBranches with integrated IR mechanism for dynamic capability moni- toring and efficient online multi-domain proportion adaptation.
Through the constructed Meditation-Enlightenee Enlightenment iteration, Bohdi automatically performs multi- domain source LLM knowledge acquisition while adaptively adjusting cross-domain data propor- tions online, efficiently injecting source L.L knowledge and effectively preventing the imbalance in target L.MM's capabilities.
2 Methodology
2.1 Overall Objective of The Bohdi Framework
We first define the overall framework of Bohdi. Let the set of source LLMs be denoted as {M [M]]- each M (-, 0 : Q - A (k denotes the subscript index) is an LLM parameterized by 0 which accepts a question q from the set of queries Q and returns an answer a E A, where AA is the set of answers. To fully integrate the strengths of the K source LLMs into the target LLM MM parameterized by O, aim ensure that for any domain D, the target LLM performs worse than the best source LIM in S for that domain. This objective can be formally expressed as:
0 = arg min EpE,Eg= (v (v (2. (q) < max V (g. (g. 0002 ISRCK
(1)
where V(g, a) measures the quality of the answer a in response to the question g. QD denotes the question space belonging to domain DD Given the broad range of problem domains, it is not feasible to list them all at once. Moreover, obtaining real data for each specialized domains is challenging. Additionally, directly assigning static fixed proportions data from each domain can easily lead to a severe capability imbalance. Therefore, we aim for the domains in Problem (T) to dynamically expand and automatically generate corresponding domain data, while also enabling online adjustment of data proportion across different domains. To this end, we transform the problem in Eq. | into the following two-phase optimization problem:
VEQ 1 (v (1.M (x)0(t)) D(++))
((++)) arg max E
V (q. (g. (q.0.)))
(2)
ISkSK
((+1) arg min E vEOD g00+1)
(v (1. (2, (1,044)))
max V (g. (q. (1.
(3)
ISRSK
where D and ( denote the problem domain and target LLM parameters in the t-th round of dynamic iteration, respectively. The fusion process of Bohdi consists of the Meditation phase formal- ized by (2) and the Enlightenment phase formalized by (3), carried out in sequence. The Meditation phase performs dynamic domain expansion and online adjustment sampling proportion for each domain, while collecting question-answar data for the corresponding domain. The Enlightenment process trains M based on the collected multi-domain data to inject knowledge from multiple source models into the target LLM.
3

---

Sprout and Horvest Operations for Domain Expansion & Generative Data Sampling
Y Sprout Operation D Sprout (D. v)
Harvest Operation Harvest (P. (SM"
Chosen P
Elect by Majority S2
Randomly Selected Lenddr
Collect Responses
Expond, Parent (D)
pper Elecs (17m hr
M)
Sompled Path P
0
S1 Domain Proposal
Ma (p Inquiry;
0
0g
Return
H2 Select the Best Answer
Return a g..e
DDew
gnftet Mi (plodorns
glaste
Figure 2: Flowchart of the proposed Sprout (left) and Harvest (right) operations.
2.2 Dynamic Domain Exploration
To enable dynamic knowledge domain exploration, structured organization of knowledge that permits further exploration and data sampling is necessary. Therefore, we introduce the following components:
Structural Knowledge Tree T To provide more structured organization of problem domains, we employ tree-like structure in the format of Main--Secondaryy-SSae originate from the root domain [Root], and are constructed the form [Di,: + D2., - Da,xl (1 S N1. 1 S N2, S k < Ns), where N1. 22, and Ng denote the number of optional domains at levels 1, 2, and 3 respectively. Then we define a knowledge tree T(V,E), where
All question domains
hierarchical domain set satisfying V1 S 3, 3, V { [Diunk with Di, representing the unexplored unknown (denoted as "unk") domain, and Vo {Do as the sole root domain.
V (Vo, V1, 122 Va}
E = {(Dij, Di+1.)) Di,j E Vi, DILLA E Vi+1 is child domain of Dij, E (0, 20} repre- sents the of edges.
Sprout Operation for Domain Expansion On the defined knowledge tree, we can sample a path from the root level-3 domain to determine the question domain. For domain D.,j satisfying 0 S i < 2, we introduce the Sprout operation, as illustrated on the left Fig. [ to propose new domain Denw Sprout (D, S, V)), with the steps as follows (for detailed descriptions of these two operations, please refer to Appendix [.I]::
S1 Collect the proposed domains DIS" DD Dnew from each source model M E S.
S2 Remove the proposed domains D.eew that belong to the existing domain set L ==1 most frequently proposed domain Diew from the remaining candidates to be added to Dr
Vj. and elect the
Harvest Operation for Generative Data Sampling For any path == [Do.0 + D1,5 + D2.j + Da,a] sampled on T, we need to obtain answer pairs (g, a) belonging to the sub-domain D3,k on this path to simulate the question space Pa and provide corresponding evaluations to assess the quality of each L.LM's response V(g, a) the question q. However, due to the highly granular professional division of paths and sub-domains in the knowledge tree, existing open-source datasets generally lack a matching fine-grained annotation system, making it difficult to directly support sampling for these specific domains. To this end, we introduce the Harvest operation, as shown on the right Fig. 2 to generate the question-answer pair (g, a) = Harvest (P,,eesttP,S,..... S,M") and add it to the question-answer pair set Ss corresponding to domain D3, for training purposes:
H1 Collect the responses a1, ....... ak+1 from all K source models and the target model.
H2 Randomly select Leader model Mlaaler e S from the set of source models to choose the best answer Best among all the responses {a:)
K+1
With clearly defined problem domains and empirical observations of each LLM's performance across these domains, the problem the Meditation phase now can transforms into a decision-mas -aking process for selecting problem domains based on the observed performance of both source and target LLMs as feedback. Consequently, we need to integrate these empirical observations into our decision- making considerations. In the subsequent discussion, we will demonstrate that, given the component designs established earlier, the domain selection process in the Meditation phase can naturally reformulated as a HMAB problem.

---

Firstly, based on the defined answer quality score (gg, the objective in the Meditation phase Eq. can be transformed into
D(++1) arg max E
(E (g.M1 (
(4)
D(++1)
where
r. (I.M {M R= )
1, (1, K} s.t. a9.Best= = ME (g. 0%) 0, if at = M (g. O)
10,
(5)
serves as the reward function that evaluates the quality of the sampled question denoted as y pitt)) When ,
1, indicates that under the current sampling, the target LLM's performance in domain (+++)) is inferior to that of the source LLMs, meaning the corresponding question-answer pair (q, Be)Be is worth learning for the target LLM. With the objective above, the path selection process knowledge tree T can be formulated as a HMAB problem:
Global-Level: HMAB (T) (MAB (D) (R D', D', Dr T),
(6)
Local-Level: MAB (D) = (C,,T) (R
D', "D)) VD' E U
that is, for any domain Di.j on T, we define a MAB instance where the arm set is C(D..). j. 7)- = {D++1,k Di+1,6 Vi+1 and (Di,. D++1*) }) representing the set of child domains under Da.j- R(r )xr) denotes the reward distribution for selecting domain D'. With the reward defined in
Eq. (5). it is a Bernoulli distribution with parameter ÀD i.e., Pp = 1 D') AD. Thus Eq.
can be further simplified to D)++1) arg paxpu++) Apu++)
Each data sampling process in HMAB(T) begins at the root domain Do,y and sequentially selects arms through three MAB instances: MAB( MAB(DD,,), and MAB(D2.j) thereby sampling pairs are sampled for the target
path P Do.o -> Di, + D2.j - D3,a), then the question-ansve LLM training, while obtaining the rewards for updating the parameters of each MAB along the path. Thus the problem of domain exploration and selection in the Meditation phase is now formalized as a sampling problem on HMAB(T). By adjusting the reward distribution parameters each arm (we use the term "arm" to refer to the corresponding optional child domains in the following sections), we can achieve the adjustment of sampling proportions across different domains.
2.3 Online Adaptive Sampling
Next, we introduce the DynaBranches and IR mechanisms in sequence to achieve efficient sampling and online adaptive sampling proportion updates on HMAB (T).
DynaBranches for Efficient Adaptive Sampling To ensure the sampling efficiency and the adaptive- ness of sampling proportions, batch sampling must be conducted while ensuring sampling diversity, and reward distribution must be updated based on batched reward feedback. To this end, we propose DynaBranches, which specifies prior distributions for the reward distribution parameters of each domain. Leveraging the TS algorithm [10), samples paths on the HMAB and adjusts the posterior based on feedback to achieve adaptive tuning of the sampling proportions. This process models the reward distribution parameters as random variables to introduce randomness into the sampling process, thereby achieving parallel sampling while ensuring the diversity of sampled domains.
Specifically, for arm D that has been sampled times, the cumulative reward distribution observa- tions Σmn r.P follow a binomial distribution, where r? is the observed reward for the i-th sampling in that domain. This makes the Beta distribution conjugate prior for Ap [Л]. Therefore, we assign the prior distribution of the reward distribution parameter An for each domain D as Beta(1, 1, 1), which equivalent a uniform distribution on [0, 1]. Consequently, the posterior of AD remains in the form of a Beta distribution Beta (orp 3D), where op 1 + Σi= r3 and Bp + n Σ P.P
Then the execution process of DynaBranches can be summarized in the following two steps:
Stepl: Batchwise Path Sampling. In this step, we sample B paths {ph] B
path pb, we initialize it from the root domain as ph = [Dt]. where D 6=1 on HMAB(T). For each = Do,0, and then sample its child domains for each D (0 < i 2) according to the following two sub-steps until obtaining the complete path pd Di D4, D2. D3]:
5

---

Meditation Phase
Bodhi
Enlightenment Phase
DynaBranches with
Meeditation-Enlighteemee Iteration For R Rounds
Sample from HMAB (T)
Sample Paths pl
pa
on Non-unk Arms
MAB (22,)
Vo
Root
Enlightenment
Collect Sampled domains
D
D
V1
Bollouu
Mothematics
MAB (P))
-MAB 1
Sprout
Randomly Sample Data From Question-Answer Pair Sets
unkk
1/2
Bloloyy OOLLD
Stontstica
Reward
MAR
(q
RondomScloct (?"
1/3
Membronn
Bayeskon Startstics
Target Model Training with Sampled Data
Get Dato Update Postarior
Harvest based p PB
Training ((g") aa )
For Non-unk De LiDi)
Meditation
Conduct SWBLRT
Figure 3: Schematic Diagram of the Iterative Process of Bohdi.
For each arm D in the arm set C(D), T) of the MAB instantiated by the current domain D) sample the reward distribution parameter AD Beta(arD, (p) from its posterior Beta (D)), and then select the arm with the maximum observed reward distribution parameter 'D = arg maxpec(P) \p as the candidate arm.
@ For the candidate arm D, if is unk arm, then directly add it as the new domain D D to the path pà and Vi+1- Otherwise, perform the Sprout operation to obtain Dexp = Sprout ( S, V). If the returned Desp is non-empty, then add as the new domain D then update the posterior parameter of the unk arm D with = + 1. If D Dexp is empty, then repeat step O to resample on MAB (D!) and update the posterior parameter of the unk arm D with 3D = 3D +
D Deep to pb and V,+1.
Step2: Parallel Posterior Update. In this step, we perform the Harvest operation on the sampled B
paths {pb) ] to generate question-answer pairs and add them the corresponding question-answer pair set 2 Meanwhile, calculate the reward based on the model's 's response and update the parameters of the sampled arms in ph.
For detailed descriptions of the two steps mentioned above, please refer to Appendix B.2
IR Mechanism for Online Sampling To further enable the sampling process adapt online to changes in the reward distribution caused by target LLM updates and to avoid over-reliance on old observed rewards, we propose the IR mechanism. dynamically checks the consistency between the observed reward distribution within sliding window and the overall empirical reward distribution via the designed SWBLRT. When change distribution is detected, the reward distribution parameters of the corresponding arms are reset to reduce empirical dependence on old observations.
Specifically, for each non-unk most recent w reward observations. Assuming that there are currently n observed rewards (ri)) ) on D, if < w. sampling and updating proceed according to DynaBranches. If n > W, construct an SWBLRT to test the samples within the sliding window {ri [n u= with the
DD initialize sliding window of width to collect the
null hypothesis and the alternative hypothesis are Ho Ei=n-w+h DD Binomial (w, Al:a) and
Hi En 1 D & Binomial (w, X!!r) respectively, where X An1in2 = カ--n: +1 denotes the probability parameter of the binomial distribution for the observed rewards from the n1-th to the nh. Let L(H() and L(H,) denote the likelihood functions of the observed rewards {r "/Di=--w1
Di,,, rb
under Ho and H41 respectively. Then the test statistic for the SWBLRT is A -2 2og L(H1)
L(Ha)
[2]
with its specific form and derivation under our settings detailed in Appendix B.3

---

Table 1: Results of the comparative experiments (accuracy in %). The best results and the second-best results are highlighted bold and underlined, respectively. The numbers after the arrows indicate the absolute improvement (shown in red) decline (shown in green) compared with the original model.
Mullidisciplnnry
Mathematic
Programming
Reasoning
Model
Data Volume
MMLU (5-Shot)
GSM6K (0-Stex CaT)
HumanEEvl (0-Shor)
ThorenQA (5-Shor)
BBH (3-Shor CaT)
AVG
(0-Shot CoT)
(B-Shat CoT)
(3-Shor)
Souree Models
Q-14B
78.54
45,45
90.22
75.04
78,66
70:60
22,85
$0.45
67.73
M-24B
79
91.26
52.53
84.15
71.20
37.38
$2.09
71.27
Tarret Model: L. Llama 2-3B-Instroct
Base
59,77
27.78
68.92 92
34.82
56.10
48.20
23.88
53.86
46.67
FuseChea
95K
59.88 m
28.28-0.0
54.27,
24 24.25-w.a
46 87
SFT
90X
62.32
25.76,a.0
48.92 48.92-161
55.49 p.01
50,50-2.00
46.08
47.5 77.57
PoseChurt3.D
90X
62 y2:00
22.22.2.00
738 22.
52.42.15.
37.93-1.0 1.00
51,00-2.50
18.0 28.6
44.97%..00
48.45-L.7
Condor
90X
60 10.00 na
27.76:0.00
71.57 a2
48 96
33.66
45 80
22. 18 18.3.3
44.01 45
46 83
Boldi (Ours)
1733
61. 61.33
30,30
7X062
47.92 47.
58.54
49 60
25.12
54 $4.90
50.79
Target
Gemma2-9B-II
Base
71.59
28.79
$1.50
45.96
65.55
57.60
15.62
70.97
54,74
FuseChal
95K
71.76
34,34rs.00
S0 14
46.88
66.46
58.50;
16.62
71.42
55 55.90
SFT
98K
60.50
30,81
68
50.00-1.0
64,02-1
56,40
15. 15,38.0
71 6..60...
52. (2.14:p.4
PoseChat
90K
66.12
30.81
70.35
47,54
62.20
57.20.0.
20.00
71.08
53.16,1.ts
Condor
70.03 15
36.87-x.05
68.61.22.
51.08 -
07.50
21.74
74.46 1.49
54.95
Bobdi (Ouns
1756
71.77 14
37,88 m0.0
83.62-2.12
54.20
64.02, 18
60.40;2-0
26.88
1.000
72:40 45
58.90 24.0
According to Wilks' theorem [33. the test statistic A asymptotically follows a chi-squared distribution with degree of freedom, X2. Therefore, we construct the rejection region {A xi.1-u}. where xi.1- is the (1 - u--th quantile of x². When falls into the rejection region, we reject the null hypothesis Hg. indicating significant difference between the reward distribution within the sliding window and the historical empirical distribution, and the current posterior of arm D no longer accurately reflects its truc reward characteristics. To address this, reset the posterior of arm D to Beta (Ein-wel (Z==n-441 r.D w - Σx=n w-1- rD) based on the observed rewards within the sliding window, in order to eliminate the reward distribution bias brought by outdated observed rewards. By integrating the proposed IR mechanism, DynaBranches can adapt online to changes in the reward distribution and maintain sensitivity to the most recent observational data.
Combining the two mechanisms described above, Bohdi samples paths on HMAB(T) and updates the MAB parameters using DynaBranches incorporating the IR mechanism in the Meditation phase. During the Enlightenment phase, we conduct sampling solely on HMAB(T), while disabling the Sprout operation, i.e., sampling is restricted to non-unk arms, then randomly sample from all existing question-answe pairs under each sampled path to obtain A data for training M In each round, the Meditation and Enlightenment phases are executed sequentially until the R-th round is completed, as shown in Fig. 3] The corresponding overall pseudocode is provided in Appendix B.4
3 Experiments
To validate the effectiveness of Bohdi, we select wide range of models with different architectures and sizes as source and target models. The source models include Qwen2.55-4BB 5-144B-nnstruc (denoted as Q-14B) (1.
(denoted as M-24B) 14, and phi-4 (denoted as P-14B) [15.. We choose two L.MMs with different architectures and sizes as target models: Llama3.2 2- 3B-Instruct [16] and Gemma2-9B-IT (17). Additional experimental results for more target models can be found in Appendix D
3.1 Experimental Setup
We provide a general introduction to the experimental setup. For detailed experimental settings, including training and evaluation configurations, please refer to Appendix
Benchmarks for Evaluation To more comprehensively evaluate the performance improvements of the fused target model across different dimensions, we select two benchmarks for each of the four main capability dimensions: multidisciplinary knowledge, mathematics, programming, and reasoning. For multidisciplinary knowledge, we select the multidisciplinary question-answerigg benchmark MMLU [8] and the natural science subject question-answering benchmark GPQA [19].
For mathematics, we choose the mathematical problem-solving benchmarks GSM8K 20 and MATH
7

---

Bohdl
Bohdl-n-update
Bohdi-Global
Bohdi-w mm
MMLU
GPQA
GSMRK
MATH
62.9
01.3
61.0
Acturacy
60.5
00.0
50.3
29.0
20
20
30
70
20
59
59
R
R
IumannEval
MBPP
TheoremQA
BB1
13
Figure 4: Comparison of Bohdi under various component setting across different fusion rounds R.
21I For programming, we select the programming benchmarks HumanEval 22 and MBPP (3] For reasoning ability, choose the benchmark BBH [24] for measuring logical reasoning ability and the theorem-drre -riiennn- reasoning benchmark TheoremQA [25]. To ensure a unified and fair evaluation,
use Opencompass [26] the evaluation suite.
Baselines We compare the target models fused using Bohdi with target models trained using several other approaches. These include fused models obtained through the most representative EF method FuseChat (6] and the state-of-the-art IF method FuseChat-3,
control, we also consider target models that are Supervised Fine-Tuned (SFT) directly on given dataset by collecting responses from each source model, as well as those that use Condor 27 to generate questions and answers for multiple domains based on each source model and then perform SFT on the target model for knowledge fusion.
9
Dataset and Training Settings For FuseChat, we use the MuseChat-Mixture dataset, which consists of 95,000 dialogue data as used in its original paper [6. For SFT and FuseChat3.0 we collect 10,000 dialogues each in mathematics, programming, and general domains, totaling 30,000 dialogues, and gather responses from the three source models, resulting in a dataset of 90,000 data for training the target model. For FuseChat3.0 O0, following the original paper 77. we use ArmoRM-Llamas, v0.1 (88 as the reward model. To ensure a similar data volume, for Condor, we use the provided instructions (27 to construct 30,000 dialogues based on each source model, totaling 90,000 data. For Bohdi, in the comparative experiments, we perform R = 50 iterations, sampling B = 90 paths the Meditation phase and M 180 data in the Enlightenment phase for training the target model in each round, with the quantile parameter u = 0.2 and window width w = 20 for SWBLRT.
3.2
Main Results
The comparative results are reported in Tab. [ For each target model, the average performance after fusion with Bohdi surpasses that of all other baseline methods, achieving average performance and 4. 1.6 for Gemma2-9B-IT. Benefiting from
improvements of 4. 1.12% for 1-.ma3.2-3B-Instrree Bohdi's comprehensive knowledge domain expansion and automatic domain allocation capabilities, the target models fused with Bohdi demonstrate improved average performance without significant degradation in any specific capabilities, achieving either optimal or sub-optimal performance on most benchmarks. Notably. Bohdi provides consistent performance improvements across all benchmarks for Llama3.22 2----B-BB-sstuue
Moreover, Bohdi enhances the performance of Gemma2-9B-IT on Theo- remQA from 15.62 to 26.88, a performance improvement of over 72%. surpassing the larger Q-14B. In contrast, although FuseChat-3.0 and SET can achieve the highest performance improvements on MMLU and MATH for the original Lama3.2-3B--B Instructt they also cause severe performance drops (up to over 30%) on TheoremQA and BBH, leading to a significant capability imbalance. Additionally, in Tab. D we also report the amount of data used for training by each method. Bohdi's training process uses only 1.7K data, which is approximately 1.8% of the data used by other methods, further demonstrating Bohdi's advantage in data efficiency.
8

---

31.0
U-03
50.25
49.57
50.92
50.5
4-02
50.0
All
213
U-033
49.5
50.68
49.46
49.37
8/-30 10
10- 20
u- 50
-49.0
(a)
Thorswos
(c)
Figure 5: Results of various ablation studies. (a): The average performance of Bohdi across benchmarks under different settings of the quantile parameter u and the window width parameter w. (b): Ablation results under different Meditation-Enlie Enlightenment sampling quantity ratios. (c): Ablation results under different number of source models K.
3.3 Ablation Studies
We conduct extensive ablation studies to verify the impact of various components and parameter settings on Bohdi. Unless otherwise stated, the settings the experiments below are the same as those in the main experiments. Additional ablations and results provided in Appendix D
Component Ablation To verify the impact DynaBranches and the IR mechanism on Bohdi's performance, we compare Bohdi without DynaBranches (performing only domain exploration and using uniform probability sampling for all known domains, denoted as Bohdi-no-update) Bohdi using DynaBranches but without the IR mechanism (with SWBLRT window size = oo to disable IR, denoted as Bohdi-Global), and Bohdi with both mechanisms combined. We evaluate these variants on 2aama..2-3B I-ntuunnttuuct across different number of fusion rounds / = {10, 20, 30, 40, 50). The results shown in Fig. f indicate that Bohdi with both mechanisms exhibits the most stable growth trend in capabilities across all benchmarks. In contrast, Bohdi-Global shows more fluctuation in its growth curves across benchmarks, demonstrating that the IR mechanism enables Bohdi to more stably track the target model's capability changes. no-di---- noticeable performance degradation on HumanEval and BBH, further highlighting that the design of DynaBranches can better avoid the capability imbalance through data allocation adjustments. Additionally, we are curious whether evaluating responses using a Reward Model is more effective than using Leader Model. To explore this, we use the same reward model as FuseChat-33 AmmoRM-Llama3- 8B- to replace the Leader Model for response evaluation (denoted as Bohdi w/ rm). The results are also documented in Fig- Due the limited cross -ommain generalization ability of the reward model (29), Bohdi w/ rm exhibits more pronounced performance fluctuations compared to Bohdi evaluated with the Leader Model, especially in GPQA and MBPP.
the other hand, experiences
Parameter Ablation for SWBLRT With Llama3.2-31 2-3B-Inntruct as the target model, we conduct ablation studies on the quantile parameter
10.1, 0.2, 0.3) and the window width parameter w = {10, 20, 30} in the IR mechanism, reporting the average performance across each benchmarks. The results shown in Fig. (a) indicate that Bohdi achieves relatively higher performance at u = 0.1, w = 30, u = 0.2, w 20, and u = 0.3, w = 10. This suggests that stricter detection criteria (lower u values) require a longer window enhance stability, while more lenient criteria benefit from a shorter window to speed up the update of reward distribution parameters.
Ablation of Sampling Quantities for Each Phase We evaluate the performance of the target model Llama3.2-3 2.-.3...2----ttttt --I--nntt on various benchmarks before and after fusion with Bohdi under different sampling ratios between the Meditation (B) and Enlightenment (M) phases: B 1M 1 2. B : M- =1:1. 1, and B A = 2 1. The results are shown in Fig. (b). The fused model achieves the best average performance across dimensions when B : M = 1 2. obtaining the best performance in four dimensions. In contrast, when B : M = 2 1, the overall capability improvement is minimal. This is because larger B M ratio generates excessive during the Enlightenment phase, limiting effective knowledge injection into the target model.
data that cannot be sufficiently learned
Ablation on the Number of Source Models We compare the performance of the target model Llamaa.2-3B-Instruue fused using Bohdi on each benchmark when using K source model (only Q- Q4B), K 2 source models (QB14B and M-24B), and all three source models for fusion. The

---

Table 2: Experimental results of fusing the target model Gemma2-9B-IT with three smaller source models, including L-ama3.2-3B-Ina. Instrucct,,, Qwen2.5-3B-Inmg Instrucc,,, and Gemma2-2B-IT using Bohdi. The numbers after the arrows indicate the absolute improvement (shown in red) or decline (shown in green) compared with the original model.
Muliidiscphinary
Mathematic
Programming
Reasoring
Model
MMLU (5-Shon)
GPQA (0-Shot CoT)
GSMSK (0-Shve CaT)
MATH (D-Shot CCT)
HumanEval (0-Shoe)
MBPP
TheoremQA (5-Shot)
AVG
(3-Shot)
(3-Shve CoT)
1 2-3--nnmret
59.77
27. 78
68.92
34,82
56. 10
48.20
23.88
$1.85
46.67
Qwen2 3B Instruct
65
37.88
82.34
62.96
73.17
$6.80
15.00
46.18
$4.95
Gemma2-22B 2B-IT
$6.69
20.20
59,21
16.68
45.73
41,00
7.25
42.52
36.41
Gemma2-0B-IT
71.59
24.79
81.50
45.96
65.85
$7.60
15.02
70.97
54,74
Behdi-Gemma2-9B-I
71.46.0.15
35.86
$1.50
48.80 04
62.20,,.45
59.40g1 o0
19.25 3.66
71.22
46.24
results are shown in Fig. (c). Additionally, in Fig. 3.3 we display the number of domains at each level proposed by Bohdi during the entire 50 rounds of fusion when using K 1, 2,3 source models. As K increases, the number of domains of each level proposed during fusion almost consistently rises, and the model tends to exhibit more pronounced growth in individual capabilities. This also indicates that involving more source models can introduce more comprehensive knowledge and bring about further improvements in the target model's capabilities.
Attempts at Weak-To-Strone To-Strong Supervision To test Bohdi's s potential as a weak-to-strong supervision paradigm, we use the largest of several target models, Gemma2-9B. target model and employ three models with significantly smaller
as the
parameter sizes, including Llama3.2- 3.B---BB--ssstuc Qwen2.5- 3B-Instruct M. and Gemma2-2B-IT [17), as source models for fusion using Bohdi (the fused model is denoted Bohdi- Gemma2-9B I--IT.. The corresponding results are documented in Tab. 2 The results indicate that Bohdi still holds promise for enhancing the target model's 's capabilities, even when the source models much smaller in size compared to the target model. Specifically, even when the source models are slightly smaller and overall weaker than the target model, the target model can still absorb the strengths of the source models in areas where it was originally weaker. Notably, the target model Gemma2-9B-IT outperformed by the source model (wen2.5--BBIInstte
359
42
Number Domains
244
Figure 6: The number of proposed domains at each level during Bo- hdi's fusion process for different numbers of source models K.
on GPQA, MATH, and TheoremQA. However, the fused model Bohdi- Gemma2-9B-IT shows improved performance on these three benchmarks, while maintaining stable performance in other domains, resulting in a absolute average performance improvement of 1.47%. This demonstrates Bohdi's 's potential as a weak-to-stroot soak-to-strong supervision paradigm.
4 Conclusion
In this paper, we introduce Bohdi, a heterogeneous LLM fusion framework that organizes knowledge domains into a hierarchical tree structure, and formalizes both novel domain exploration and cross- domain data proportion allocation on this knowledge tree as a HMAB problem. We then propose the DynaBranches mechanism for dynamic sampling on the HMAB This mechanism integrates the designed Sprout and Harvest operations to enable automated dynamic domain exploration, data acquisition, and adaptive adjustment of data proportions tailored to the target LLM's capabilities through multi-model collaboration. Furthermore, we develop the IR mechanism, empowering DynaBranches to dynamically track the target L.M'' capability evolution and perform online adjustment of domain data proportions via the proposed SWBLRT. By orchestrating these components into two-phase iterative process of Meditation and Enlightenment, Bohdi achieves automated knowledge boundary expansion and source LLM knowledge acquisition without relying on real data, while effectively mitigating the imbalance in target LLM'' s capabilities during fusion. The Bohdi's design philosophy also holds promise for providing new research ideas for mudti-model capability synergy enhancement and the development of weak-to-stre to-strong supervision paradigms.
Acknowledgements
This work is supported by the Shanghai Municipal Science and Technology Major Project.
10

---

References
[1] An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li, Tingyu Xia, Xingzhang Ren, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yu Wan, Yuqiong Liu. Zeyu Cui, Zhenru Zhang, and Zihan Qiu. Qwen2.5 technical report. CoRR. 00s/2412.15115, 2024.
[2] DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma. Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu. Zhibin Gou, Zhihong Shao. Zhuoshu Li. Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei Feng, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, Fuli Luo, Guangbo Hao, Guanting Chen, Guowei Li, H. Zhang. Han Bao, Hanwei Xu, Haocheng Wang, Honghui Ding, Huajian Xin, Huazuo Gao, Hui Qu, Hui Li, Jianzhong Guo, Jíashi Li, Jiawei Wang, Jingchang Chen, Jingyang Yuan, Junjie Qiu, Junlong Li, J. L. Cai, Jiaqi Ni, Jian Liang, Jin Chen, Kai Dong. Kai Hu, Kaige Gao, Kang Guan, Kexin Huang, Kuai Yu, Lean Wang, Lecong Zhang, Liang Zhao, Litong Wang. Liyue Zhang, Lei Xu, Leyi Xia, Mingchuan Zhang, Minghua Zhang. Minghui Tang, Meng Li, Miaojun Wang, Mingming Li, Ning Tian, Panpan Huang, Peng Zhang, Qiancheng Wang. Qinyu Chen, Qiushi Du, Ruiqi Ge. Ruisong Zhang, Ruizhe Pan, Runji Wang. R. Chen, R. L. Jin, Ruyi Chen, Shanghao Lu, Shangyan Zhou, Shanhuang Chen, Shengfeng Ye, Shiyu Wang, Shuiping Yu, Shunfeng Zhou, Shuting Pan, and S. S. Li. Deepseek-rl: Incentivizing reasoning capability llms via reinforcement learning. CoRR, 0ttp22511.12994.5 2025.
[3] Le Yu, Bowen Yu, Haiyang Yu, Fei Huang, and Yongbin Li. Language models are super mario: Absorbing abilities from homologous models as a free lunch. In International Conference on Machine Learning, 2024.
[4] Prateek Yadav, Derek Tam, Leshem Choshen, Colin A. Raffel, and Mohit Bansal. Ties-merging: Resolving interference when merging models. In Advances in Neural Information Processing Systems, 2023.
[5] Fanqi Wan, Xinting Huang, Deng Cai, Xiaojun Quan, Wei Bi, and Shuming Shi. Knowledge fusion of large language models. In Twelfth International Conference Learning Representa- tions.
[6] Fanqi Wan, Ziyi Yang, Longguang Zhong, Xiaojun Quan, Xinting Huang, and Wei Bi. Fusechat: Knowledge fusion of chat models. CoRR, abs/2402.16107, 2024.
[7] Ziyi Yang, Fanqi Wan, Longguang Zhong, Canbin Huang, Guosheng Liang, and Xiaojun Quan. Fusechat-3.0: Preference optimization meets heterogeneous model fusion. CoRR, has/2503.04223. 2025.
[8] Ziyi Yang, Fanqi Wan, Longguang Zhong, Tianyuan Shi, and Xiaojun Quan. Weighted-reward preference optimization for implicit model fusion. CoRR, 0bss/442..031 187, 2024.
[9] Longguang Zhong, Fanqi Wan, Ziyi Yang, Guosheng Liang, Tianyuan Shi, and Xiaojun Quan. Fuserl: Dense preference optimization for heterogeneous model fusion. arXiv preprint arXiv:2504.06566 2025.
[10] Olivier Chapelle and Lihong Li. An empirical evaluation of thompson sampling. In Advances in Neural Information Processing Systems, pages 2249-2257, 201
[11] William Thompson. On the likelihood that one unknown probability exceeds another in view of the evidence of two samples. Biometrika, 25(3/4):285-294 1933.
[12] Jerzy Neyman and Egon Sharpe Pearson. Ix. on the problem of the most efficient tests of statistical hypotheses. Philosophical Transactions of the Royal Society of London, Series A, Containing Papers of Mathematical or Physical Character, 231(694-706))29- 1933.
11

---

[13] Samuel S Wilks. The large- samgle distribution of the likelihood ratio for testing composite hypotheses. The annals of mathematical statistics, 9(1):60 62, 1938.
[14] Mistral AI. Mistral small 3, 2024.
[15]
Marah I Abdin, Jyoti Aneja, Harkirat S. Behl, Sébastien Bubeck, Ronen Eldan, Suriya Gu- nasekar, Michael Harrison, Russell J. Hewett, Mojan Javaheripi, Piero Kauffmann, James R. Lee, Yin Tat Lee, Yuanzhi Li, Weishung Liu, Caio C. T. Mendes, Anh Nguyen, Eric Price, Gustavo de Rosa, Olli Saarikivi, Adil Salim, Shital Shah, Xin Wang, Rachel Ward, Yue Wu, Dingli Yu, Cyril Zhang, and Yi Zhang. Phi-4 technical report. CoRR, atw//411.0099// 2024.
[16] Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan. Anirudh Goyal, Anthony Hartshomn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao, Aston Zhang, Aurélien Rodriguez, Austen Gregerson, Ava Spataru, Baptiste Rozière, Bethany Biron, Binh Tang, Bobbie Chern, Charlotte Caucheteux, Chaya Nayak, Chloe Bi, Chris Marra, Chris McConnell, Christian Keller, Christophe Touret, Chunyang Wu, Corinne Wong. Cristian Canton Ferrer, Cyrus Nikolaidis, Damien Allonsius, Daniel Song, Danielle Pintz, Danny Livshits, David Esiobu, Dhruv Choudhary, Dhruv Mahajan, Diego Garcia-Olano, Diego Perino, Dieuwke Hupkes, Egor Lakomkin, Ehab AlBadawy, Elina Lobanova, Emily Dinan, Eric Michael Smith, Filip Radenovic. Frank Zhang, Gabriel Synnaeve, Gabrielle Lee, Georgia Lewis Anderson, Graeme Nail, Grégoire Mialon, Guan Pang. Guillem Cucurell, Hailey Nguyen, Hannah Korevaar, Hu Xu, Hugo Touvron, Iliyan Zarov, Imanol Arrieta Ibarra, Isabel M. Kloumann, Ishan Misra, Ivan Evtimov, Jade Copet, Jaewon Lee, Jan Geffert, Jana Vranes, Jason Park, Jay Mahadeokar, Jeet Shah, Jelmer van der Linde, Jennifer Billock, Jenny Hong, Jenya Lee, Jeremy Fu, Jianfeng Chi, Jianyu Huang, Jiawen Liu, Jie Wang. Jiecao Yu, Joanna Bitton, Joe Spisak, Jongsoo Park, Joseph Rocca, Joshua Johnstun, Joshua Saxe, Junteng Jia, Kalyan Vasuden Alwala, Kartikeya Upasani, Kate Plawiak, Ke Li, Kenneth Heafield, Kevin Stone, and et al. The Ilama 3 herd of models. CoRR, abs/2407.22 21783, 2024.
[17] Morgane Rivière Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju. Léonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ramé, Johan Ferret, Peter Liu, Pouya Tafti, Abe Friesen, Michelle Casbon, Sabela Ramos, Ravin Kumar, Charline Le Lan, Sammy Jerome, Anton Tsitsulin, Nino Vieillard, Piotr Stanczyk, Sertan Girgin, Nikola Momchev, Matt Hoffman, Shantanu Thakoor, Jean-Bastien Grill, Behnam Neyshabur, Olivier Bachem, Alanna Walton, Aliaksei Severyn, Alicia Parrish, Aliya Ahmad, Allen Hutchison, Alvin Abdagic, Amanda Carl, Amy Shen, Andy Brock, Andy Coenen, Anthony Laforge, Antonia Paterson, Ben Bastian, Bilal Piot, Bo Wu, Brandon Royal, Charlie Chen, Chintu Kumar, Chris Perry, Chris Welty, Christopher A. Choquette-Choo, Danila Sinopalnikov, David Weinberger, Dimple Vijaykumar, Dominika Rogozinska, Dustin Herbison, Elisa Bandy, Emma Wang, Eric Noland, Erica Moreira, Evan Senter, Evgenii Eltyshev, Francesco Visin, Gabriel Rasskin, Gary Wei, Glenn Cameron, Gus Martins, Hadi Hashemi, Hanna Pluciikkk Harleen Batra, Harsh Dhand, Ivan Nardini, Jacinda Mein, Jack Zhou, James Svensson, Jeff Stanway, Jetha Chan, Jin Peng Zhou, Joana Carrasqueira, Joana Iljazi, Jocelyn Becker, Joe Fernandez, Joost van Amersfoort, Josh Gordon, Josh Lipschultz, Josh Newlan, Ju-yeong Ji, Kareem Mohamed, Kartikeya Badola, Kat Black, Katie Millican, Keelin McDonell, Kelvin Nguyen, Kiranbir Sodhia, Kish Greene, Lars Lowe Sjösund, Lauren Usui, Laurent Sifre, Lena Heuermann, Leticia Lago, and Lilly McNealus. Gemma 2: Improving open language models at practical size. CoRR, abs/2408.00 18, 2024.
[18] Dan Hendrycks. Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. In International Conference on Learning Representations, 2021.
[19] David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang. Julien Dirani, Julian Michael, and Samuel R. Bowman. GPQA: A graduate-level google-proof q&a benchmark. CoRR, abs/231 1.12022, 2023.
[20] Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers solve math word problems. CoRR, 10./21100.11666 2021.
12

---

[21] Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the MATH dataset. In Proceedings of the Neural Information Processing Systems Track on Datasets and Bench- marks, 2021.
[22]
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Pondé de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray. Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian, Clemens Winter, Philippe Tillet, Felipe Petroski Such, Dave Cummings, Matthias Plappert. Fotios Chantzis, Elizabeth Barnes, Ariel Herbert- Voss, William Hebgen Guss, Alex Nichol, Alex Paino, Nikolas Tezak, Jie Tang, Igor Babuschkin, Suchir Balaji. Shantanu Jain, William Saunders, Christopher Hesse, Andrew N. Carr, Jan Leike, Joshua Achiam, Vedant Misra, Evan Morikawa, Alec Radford, Matthew Knight, Miles Brundage, Mira Murati, Katie Mayer, Peter Welinder, Bob McGrew, Dario Amodei, Sam McCandlish, Ilya Sutskever, and Wojeiech Zaremba. Evaluating large language models trained on code. CoRR, abs/2107.03377 2021.
[23] Jacob Austin, Augustus Odena, Maxwell I. Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie J. Cai, Michael Terry, Quoe V. Le, and Charles Sutton. Program synthesis with large language models. CoRR, abs/2108.07732. 2021.
[24] Mirac Suzgun, Nathan Scales, Nathanael Schärli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoe V. Le, Ed H. Chi, Denny Zhou, and Jason Wei. Challeng- ing big-bench tasks and whether chain-of-thought can solve them. In Findings of the Association for Computational Linguistics, 2023.
[25] Wenhu Chen, Ming Yin, Max Ku, Pan Lu, Yixin Wan, Xueguang Ma, Jianyu Xu, Xinyi Wang, and Tony Xia. Theoremqa: A theorem-driven question answering dataset. In Conference on Empirical Methods in Natural Language Processing, 2023.
[26] OpenCompass Contributors. Opencompass: A universal evaluation platform for foundation models. https:///ithuu:: http://ttttuuuccooooeeeeeeeeeeee hwtppeen-ccomppsss/pppccpppppppa 2023.
[27] Maosong Cao, Taolin Zhang, Mo Li, Chuyu Zhang, Yunxin Liu, Haodong Duan, Songyang Zhang, and Kai Chen. Condor: Enhance LLM alignment with knoviedge-driven data synthesis and refinement. CoRR, abs/2501. 2025.
[28] Haoxiang Wang. Wei Xiong, Tengyang Xie, Han Zhao, and Tong Zhang. Interpretable prefer- ences via multi-objective reward modeling and mixture-of-experts. In Findings of the Associa- tion for Computational Linguistics: EMNLP, 2024.
[29] Rui Yang. Ruomeng Ding, Yong Lin, Huan Zhang, and Tong Zhang. Regularizing hidden states enables learning generalizable reward model for 1lms. In Advances in Neural Information Processing Systems, 2024.
[30] Zhaoyi Yan, Zhijie Sang, Yiming Zhang, Yuhao Fu, Baoyi He, Qi Zhou, Yining Di, Chunlin Ji, Shengyu Zhang, Fei Wu, and Hongxia Yang. Infifusion: A unified framework for enhanced cross cross-model reasoning via LLM fusion. CoRR, abs/2501.02795, 2025.
[31] Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. CoRR, 0ass22011088361 2020.
[32] Jordan Hoffmann. Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbi, Aidan Clark, Tom Hennigan, Eric Noland, Katie Millican, George van den Driessche, Bogdan Damoc, Aurelia Guy, Simon Osindero, Karen Simonyan, Erich Elsen, Jack W. Rae, Oriol Vinyals, and Laurent Sifre. Training compute-optimal large language models. CoRR, abs/2203. 15556, 2022.
[33] Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, and Hannaneh Hajishirzi. Self-instruct. Aligning language models with self-generated instruc- tions. In Annual Meeting of the Association for Computational Linguistics, pages 13484-13508. 2023.
13

---

[34] Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, Qingwei Lin, and Daxin Jiang. WizardIm: Empowering large pre-trained language models to follow complex instructions. In International Conference on Learning Representutions, 2024.
[35] Haoxiong Liu, Yifan Zhang, Yifan Luo, and Andrew C. Yao. Augmenting math word problems via iterative question composing. In Association for the Advancement of Artificial Intelligence, pages 24605-2461 24613, 2025.
[36] Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah D. Goodman. Star: Bootstrapping reasoning with reasoning. In Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems 35; Annual Conference on Neural Information Processing Systems 2022, NeurlPS 2022, New Orleans, LA, USA, November December 9, 2022, 2022.
28
[37] Ning Ding, Yulin Chen, Bokai Xu, Yujia Qin, Shengding Hu, Zhiyuan Liu, Maosong Sun, and Bowen Zhou. Enhancing chat language models by scaling high-quality instructional conversations. Conference on Empirical Methods in Natural Language Processing, pages 3029-3051, 2023.
[38] Jiuhai Chen, Rifaa Qadri, Yuxin Wen, Neel Jain, John Kirchenbauer, Tianyi Zhou, and Tom Goldstein. Genqa: Generating millions of instructions from a handful of prompts. CoRR, abs/2406.10323, 2024.
[39] Zhangchen Xu, Fengqing Jiang, Luyao Niu, Yuntian Deng, Radha Poovendran, Yejin Choi, and Bill Yuchen Lin. Magpie: Alignment data synthesis from scratch by prompting aligned Ilms with nothing. CoRR, abs/2406.08466 2024.
[40] Shuo Tang, Xianghe Pang, Zexi Liu, Bohan Tang, Rui Ye, Xiaowen Dong, Yanfeng Wang, and Siheng Chen. Synthesizing post-training data for llms through multi-agent simulation. CoRR, abs/2410. 14251, 2024.
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
We clarify the contributions and scope in the Introduction section.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
The limitations of our work are discussed in the "Limitaions" part in the Appendix.
Justification:
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
15

---

Answer:
[NA]
Justification:
This is not theoretical work, nor does it involve theoretical conclusions.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
We provide detailed descriptions of the experimental settings, including the models used, the libraries employed, the training and testing configurations, and the hardware, in the "Experiment" section of the paper and the "Experimental Details" section in the appendix.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
16

---

reproducible, they 5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Answer: [No]
Justification: We will include the code link in the official publication, and our method does not require open-sourcing data.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.co
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https for more details.
/nips. wwtpcc/pcccccpcccceooooooop httpc//gguu.eeeeeeeuuuubiiiiiiia wwweiisssonPoPiiiiooooo
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
We provide these details in the "Experimental Details" section in the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: Due to the high experimental costs, we did not include an error bar. But as reported in the "Experimental Details" in the appendix, we endeavored to minimize the influence of randomness during the testing process.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
17

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We provide these details in the Experimental Details section in the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics https: http///euurrp..pee Sttarisssccppubic/eBeec cccppbllc//thhiicsuieelllll
Answer: [Yes]
Justification:
This research fully complies with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- cration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: Our work is focused on methodological aspects, and there are no specific application scenarios that would directly lead to socictal issues at this stage.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
18

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), faimess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: This work focuses on methodological research without involving models or data with high misuse risk. Therefore, no specific safeguards are required.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging. and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
The code, data, and models used in this paper are properly cited and referenced.
Guidelines:
The answer NA means that the paper does
existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include a URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
19

---

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com/dataseta
latasets.paperssit
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Question:
Answer: [NA]
Justification: This paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/cooeemmodll as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve any experiments with human subjects or crowd- sourcing research.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/teview based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve any experiments with human subjects or crowd- sourcing research.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
20

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [Yes]
Justification: We provide detailed introductions to these aspects in the Methodology section, the Experiments section, and the Experimental Details section in the appendix.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy (https://neurd) httprrippp..c/mmemeeeeee////2222 for what should or should not be described.
21

---

A Related Works
A.1 Heterogeneous LLM Fusion
Heterogeneous LLM fusion aims to inject the knowledge from multiple source LLMs into a target LLM that serves pivot, thereby integrating the strengths of multiple powerful LLMs into single LLM. Given that heterogeneous LLMs typically exhibit differences in architectures, parameter scales, and tokenizer vocabularies, current research primarily employs two dominant strategies address these constraints: EF [5] 6 30 and IF
7 g 9
EF methods perform supervised distillation of the target LLM by collecting output probability distributions from source LLMs on a given dataset to explicitly transfer knowledge from multiple source LLMSs to the target L.MM. Due to misaligned output probability distributions caused by different vocabularies, EF methods employ various approaches to align these distributions across vocabularies. For example, aligning vocabularies by minimizing the edit distance between the output sequences of the source and target LLMs [5] [). while others directly align the tokens with the highest prediction probabilities by minimizing the 1-Wasserstein distance [3). In terms of fusion approaches. EF methods either uniformly align the target LLM with all source LLMs' probability distributions [3] or first perform pairwise alignment, followed by parameter merging of the aligned target LLMs to obtain a final fused target LLM, thereby more stably integrating knowledge from multiple source LLMs. However, the vocabulary alignment process inevitably introduces alignment errors, which adds noise to the fusion process of EF.
In contrast, IF methods train the target LLM directly using responses from source LLMs, bypassing the vocabulary aligmment process and thus enabling more stable knowledge injection. Their fusion approaches include selecting the best response from multiple source LLMs based on scores from an external reward LLM to guide the target LLM's preference optimization [8], or a two-phase process involving weighted SFT and DPO based source LLM responses and reward LLM scores to more comprehensively utilize all responses from multiple source LLMs 9
Although current heterogeneous fusion methods have shown promising performance, they rely on the given data for the fusion process. Since comprehensive and systematic datasets across diverse domains are difficult to obtain, these methods can only rely on open-source data from few domains for LLM alignment, which prevents the full injection of source LLMs' knowledge into the target LLM. Moreover, these methods use fixed data proportions, which cannot adaptively adjust the data proportions for each domain based on the target LLM's capabilities, leading to a imbalance in the target LLM's capabilities across different domains.
A.2 LLM Data Synthesis
With the rapid increase in the parameter size of LLMs and the discovery of Scaling Laws B1 32),
the importance of data for the development of LLM capabilities has gradually become a consensus. Considering the high costs associated with data collection, organization, and annotation, using LLMs to generate synthetic data has emerged as a more cost-effective and efficient approach. The primary synthetic data generation schemes be divided into seed-based synthesis and seedless synthesis.
Seed-baser based data synthesis involves augmenting or iteratively refining instructions and responses based on partially real, given instance data. For example, guiding the LLM to generate instruction and response data for various tasks based on carefully crafted task seed sets 33 synthesizing deeper and more diverse instructions by performing deep and broad evolution based on an initial set of instructions [34], iteratively enhancing initial questions and adding additional reasoning steps [35). and generating the logic to answer a large number of questions from few examples, then detecting and iteratively improving responses to unresolved problems (66
In contrast, seedless synthesis does not rely on external real data examples to generate data from scratch, thereby further avoiding the collection and annotation costs of pre-given real data. Examples include using LLMs to generate concepts, objects, and entities present in the real world, and creating corresponding questions and dialogues for each sub-topic (37]; ; using hand-written meta-prompts to have L.LSs automatically generate multiple instructions and randomly select them to construct diverse instruction sets (38] guiding LLMMs to generate instructions and their corresponding responses using the response templates of Chat LLMs [3); and generating dialogue data in multiple scenarios through
22

---

large-scale agent systems playing different roles 440. However, since the aforementioned approaches either rely on closed-source models [37], or fail to cover extensive and diverse domains [8] 39 or
require massive agent collaboration with intricate framework design thus necessitating cumbersome preparatory steps [40], Condor (7 organizes world knowledge through a tree structure, employs LLMs to generate comprehensive hierarchical domain labels, and further produces domain-specific QA data based on these labels.
Yet during each domain expansion, Condor generates all child domain labels at once, lacking autonomous exploration capability for new domains at various levels. Condor does not truly assign node information to each domain, failing to manage the diverse generated data through genuinely tree-structured organizational scheme, and consequently cannot design feedback based data ratio adjustment mechanisms across different domain levels.
B Details of Bohdi Framework
B.1 Details of Operations Sprout and Harvest
Details of Operation Sprout For domain at level (0 < i S 2, when i 3, fur- ther domain expansion is not necessary), if the selected child domain Di+i,j is an unk node Di+1,unk. all source LLMs collaboratively propose a new domain Dnew = Sprout SDDS, V) for domain expansion. Specifically, for the already sampled set of parent domains Par. construct the expansion prompt Expand, Parent (Di,)] (see Appendix [F) for the specific for- mat), which is answered by all source LLMs M to obtain proposed new domains Die
M (p [Expand; Parent (Dig)] Og). For the proposed domains (DD Jk=II ] we first exclude any existing domains contained within them, and then perform a majority vote among them elect the
new domain:
DNow
Elect (DDgew Jk=1 - V)) then add it to the node Vi of the corre- )
K
=== - - V2 = 0, then this node expansion will be discarded. If all proposed domains are different, the election operation degenerates to randomly selecting one from the candidate domains as Deew
sponding level and update N₂ = N; + 1. If (D
Details of Operation Harvest Given the sampled path P= [Du + D1. D2. Da,kl, Harvest operation leverages the given LLMs to generate question- pairs (g. Harvest (P,s S.M") belonging to the subdomain D3,6 on P to obtain the sampled data. Specifically, we first ran- domly assign a Leader LLM Mlealer
K
{M from the set of source LLMs to gen- k=1
erate a question q = MTenndr (p [Inquiry; Pl) E Pa.. based on the sampled path, where p lIqquiry: P| is the prompt that guides the LLM to generate question corresponding to the given path P. To further obtain answers and calculate V(g, a), we collect set of answers K
from all LLMs for question y, and use the Leader LLM to judge and select the best answer: a.. = MM (p|Select; q. A"), el ( eader)). where p|Select; q, A9] represents the prompt that guides the Leader LLM in selecting the best answer. The specific formats of related prompts can be found in Appendix [F For any A4, we define Vgg, a) = 1 if and only if a = ag,Bess and V (q, a) = 0 otherwise. We then add the question-answw pair (4, Best) to the sampled question-answer pair set S2 of the corresponding subdomain Ds,k for target LLM training in the Enlightenment phase.
A = {{as
04 a; = -M; (g, ,,), M, E {M]]
k== U [MT}}
B.2 Details of the DynaBranches
Here we provide the detailed descriptions of the two steps of DynaBranches.
Stepl: Batchwise Path Sampling. In this phase, we sample B paths { B
6=1 on HMMB(T).
where B is the batch size. For each path pb, we first initialize pb [Do.o] and sample the reward distribution parameters Xp= for each arm Do E C(D 'T) the first MAB MAB(Do.o): App ~ Beta(Opp {p=)). We then select the arm with the highest sampled reward distribution parameter: D = arg max .T) Ap. If the chosen arm D V is not unk arm, directly add it as D/11 to the path pb_ If D is an unk arm Dunk: we perform the Sprout operation to expand an unknown arm: Dexp= SSprout(ooor S, V). If Dexp is known arm, i.c., Dexp E V, the Sprout
23

---

operation is discarded, and reward = is fed back to the arm Dunk, then update PTm. PD... + and resample an arm on (D,o) If the Sprout operation finds a new arm Doxp & V, we add it as D 1 to the path Pb, then feed back reward r
the unk arm Dunk. and update ar = aDD.. + After sampling D++1 we continue sampling subsequent arms on instantiated child-level MAB MAB(D)) ++11 1) until the bottom-level MAB is sampled. To avoid too many invalid Sprout operations, if there are 10 invalid Sprout operations during the sampling process on the path Pt, we mark the path pa as "Invalid" and update the posterior distribution of the most recently sampled arm DNow in the current po to Beta oo) to prohibit sampling DNow in subsequent sampling processes.
b==1" we perform the Harvest operation in parallel on all valid ph to obtain question-answer pairs (q", a²) = Harvest(P), S.M"). Then calculate the reward y = 1 [aa f MM 0²"),, and update the posterior of the reward distribution parameters for each arm D in ph except for Do,o: app apt + rd, Bp 3px + (1 -*).
Step2: Parallel Posterior Update. For the sampled paths {P=)
B.3
The formal construction of the SWBLRT test statistic
Since the sum of reward observations within the window S > -+1 rb follows binomial distribution, its likelihood function L (A) can be expressed as
L (A) =
w Sw
S (1 11)-)(0-50 ())
(7)
where À is the probability parameter of the binomial distribution Binomial A) that S follows. When the null hypothesis Ho Σ Limn-w41 r/p Binomial(w, Ai"²) holds, the likelihood function (Ho) can be rewritten as:
L (Mo)
W 5t
(AIm))
5 (1- A (w
(8)
When the alternative hypothesis Mi : Ez=n-+t D o Binomial ww Aba)) holds, we consider its maximum likelihood. First, examine the derivative of the logeliiel in Eq. (7)
dL (A) S yx x 1
1 - (w
S
let dL(A)
dx O, we obtain the maximum likelihood estimate of x as Ā= = _ Substituting this into Eq.
yields the likelihood function corresponding to the alternative hypothesis:
S
S
S
L (Ht)
=
S
1
(10)
Then the test statistic A for SWBLRT can be derived as:
A -2 L(Ho)
(11)
L(Hi)
Sie
9
2. log
wr X
+ 2w
S"))
L:n
n
Xm X2-w+1n
n1
Σ
D
log
Σ
r
log
(12)
1im
1
11+1
D
Therefore, the test statistic used in the SWBLRT is calculated in the form shown in Eq.
12
B.4 Pseudocode Bohdi
We present the pscudocode for Bohdi's 's actual execution process in Algorithm I
24

---

Algorithm The execution process of Bohdi
Input :Source LLMs (M target LLM M number paths sampled in the Meditation phase and the Enlightenment phase B and M, the window width parameter and the quantile parameter u of the IR mechanism, initialized knowledge tree T and HMAB (7)
Output Fused target model M Fuse
for / <- to R do
MEDITATION PHASE:
DynaBranches
-aatnchhee--tcciisese Batchwise Sampling: for b t B do
i <- 0, Dj t- Do.0. pb f- [Do.o]. MAB; <- MAB (Do,o). Fail + 0 while i and Fail 10 do
parallel for D C (D.) do
ADs ~ Beta (a.De; 3D*)
D arg maxDDE(D))) \D
// Sample arm MAB;
if D is unk then
D + Sprout SD).S
while Dexp is None and Fail < 10 do
Dexp + Sprout (D). S, V). Fail < Fail + 1, 3D +- BD +
if Fail 10 then
Initialize an unk domain Dunk as child domain of Dexp with XD Beta(1, 1)
Initialize the reward observation pool WD for D exp
ap < a& + 1, D, +1
Dexp. V41 + V4+1 (D) 1 + EU((D), D(+1)
i 2 then
Initialize the question-answer pair set Ω for D
else
D D pd append Di+1 if Fail 10 then
1/ Append D4g to the sampled path pb
Bpp) + (1, 00), Mark ph as Invalid
else
Initialize the question-answer pair set ΩP for D2
Batchwise Posterior Update: for all valid pà {p") B do ( a²) +- Harvest (Ph, (.S' t (a + M (q',
Banarrnches-Batchwiss
for E pb (Do do aD + ap + r. BD + BD (1- WD append n
// Update Posterior
D = D3 then
n SP
U ((d)aa)))
IR Mechanism with SWBLRT: Dn f- Do. for +- to do
for all non-unk D:+1 C do the number of reward observations RrD:.1 in Wi satisfies IDD441 > w then Calculate the SWBLRT statistic Ap for D;+1
if AD > Xi.1-u then
Reset the reward distribution parameter posterior of arm D;41 based on the latest W observations in WD
ENLIGHTENMENT PHASE:
// Target Model Training
Sample M paths (pm )M from all non-unk arms in HMAB(T) without updating the posteriors
Initialize B =
parallel for m +- to M do
Randomly select a question-anssee pair (q"",""))
a²m) E 22" from the question-ansse answer pair set
? corresponding to the sub-domain D in pm
= B U {(g") )
Train the target model M using ((qT),00 aa"))
25

---

C Experimental Details
Training Details We introduce the specific experimental settings in the main experiments. For FuseChat, FuseChat-3.0, and Condor, we refer to the settings in their original papers [ 7 27]. including the learning rate and training epochs. For SFT, we use training schedule of 3 epochs and consistent learning rate of 5e 6 across all target models. Since Condor does not provide learning rate adjustment scheme in its original paper; we uniformly set a cosine learning rate decay strategy for all baseline methods except FuseChat and FuseChat-3.0, with warmup rate of 0.03. For Bohdi, we also consistently use constant learning rate of 5e 6 across all target models. Given the limited number of samples per training round, we train the target model for 1 epoch based on the sampled M question-answer pairs in each round. Additionally, apart from FuseChat providing training code, since FuseChat-3.0 and Condor do not provide training code, we uniformly use tr as the training framework and train with 6 precision. All of our training is conducted 8xNvidia A100 GPUs.
[E] Testing Details To ensure unified and fair evaluation. we use opencompass aa the evaluation suite. In the generation settings for inference, we uniformly set a max input token length of 2048 for all methods. For MATH, GSM8K, HumanEval, and MBPP, which typically require longer reasoning outputs to solve, we uniformly use a max new token length of 4096. For MMLU, GPQA, TheoremQA, and BBH, which require shorter responses, uniformly use a max new token length of 1024. To further mitigate the impact of randomness, we use greedy sampling for generation in all tests.
Generation Settings Regarding the temperature parameter settings during Bohdi's generation process, for models whose generation configs explicitly specify a temperature value includinn Llama3.2. IB-Instruct and Llama3, 11mmaa,,1-88 1-B-- --8-ttrtutttttt set to 0.6, Swenn.5-7B-Instruue and 1rren2.5-14B-Imsse to 0.7, and Mistral-Small-2244
-501 set 0.15 we we follow the temperature parameters specified in their configs. For other models whose generation configs do not specify a temperature, including phi-4 and Gemma2-9B- we uniformly apply temperature of 0.7. During the domain proposal process in the Sprout operation and the question generation process in the Harvest operation, we use a top-p value of 0.95 across all models to encourage diversity in the generated domains and questions. In the answer generation process of the Harvest operation, we use top-p value of 0.8 for all models to ensure response stability. Additionally, during the answer selection stage in Harvest, we apply a top-p value of 0.95 for the Leader model.
D
Additional Experiments
Fusion experiments on more target models To further verify the universality of Bohdi, we conduct fusion experiments on two additional target models from different model classes and with different parameter sizes, Llama3. 115.1-88-Innttuu [16] and the more powerful Qwen2.5-7BB-nntmmnn
[.. We then compare them with other baselines. As shown in Tab. 3 the experimental results demonstrate that Bohdi continues to deliver outstanding performance on both target models. Notably for Llama3. Instruct, Bohdi delivers uniform performance gains across all benchmarks. This advantage becomes particularly pronounced on GSM8K and TheoremQA, where all other baseline methods degrade model performance, Bohdi alone achieves positive enhancements. The most striking improvement occurs on TheoremQA, where Bohdi achieves a 3.76% absolute performance gain over the base model. When Qwen2.5-7B- Instrrctt serves the target model. due to its already sufficiently strong foundational capabilities, almost all methods lead to performance degradation in most benchmarks after fusion. Bohdi, however, still manages to bring about significant improvements in the target model's performance on MBPP and TheoremQA, without causing any noticeable decline on individual benchmarks. Furthermore, while other baselines exhibit average performance decay across each benchmarks, Bohdi continues to deliver an overall performance boost for Qwen2.5-7B- 5- highlighting its stability.
The impact of the number of source models on domain allocation preferences In Fig. (a), (b). and (c), we show the domain hierarchy generated and corresponding sampling proportions during the Bohdi fusion process when using K = source model (Qwen2.5-144) I1st-uust K = and
source models (0wen2.5-144B-11
and all three source
26

---

Table 3: Comparative results for 1amma..--8BInnssie models (accuracy in %). The best results and the second-bexs results are highlighted bold and underlined, respectively. The numbers after the arrows indicate the absolute improvement (shown in red) or decline (shown green) compared with the original model.
[16] and Qwen2.5-7B-Inns as target
Multidisippinrry
Programming
Ressening
Model
Data Volume
MMLU
DPQA
GSMRK
MATH Sbot CoT)
Hamanisal Shot)
Mhuy Shoo)
TheorenIAA (5 Sbot
JVG
15 Sho0
(9 Shot Co CoT)
@ Shot CoT)
@ Shot CdT)
Models
Q-148
78
7% 66
70:60
67.71
Mi 2413
$0.95
46.46
91.89
50.84
79.88
68.20
37.12
82.71
09.51
71.20
Target 1almsh.1-80--mm
Hawn
6701
29.19
79.61
45.88
65.85 85
55,00
31.75
66.47
55.13
PuseChat
95K
-7.31
28.28
79.23
46.98
67.68:
67.94
55.53
908
67.72
33.84.100
1:0
74,30,
50.00 100
67.07
$6.40
11:12,
52.09 p..0
FuseChaz 3.0
908
68.73
53.33
18.17.144
$3.46
67.07
55. (5.00
14.75
59. 49.98
53.89
Condor
90K
68.65-10
75.68,
$1.76-134
14.37
p1.00
Boldi (Qun)
1473
16-
30.30-11
84.23 cm
70.73
38,00...
69.31
58.99
Target Model: Qwen2.5-7H6 Tow-n2.--B-
73.34
37,88
82.32
95K
73.52-a x
17.58-11m
um
17.4
61.20
19.58
73.05
62.50
SPT
90K
64.31 64.31.
34.34.104
K3.85
66 66.46,11
99.27uxee
65 40
18.12
69.2 69.21
60.00
FuseChat 3,0
90K
34.85,110
85.24.cm
68.96
02.20
21,00,
61.12
Condor
90K
61.86
35.36,1.0
83.15
64.12.106
21.95
63 10.00
12.38,7.0
63.72,xe1
57.01
Bobdi (Ouns)
1802
13.29.
35.35_150
88.17:
70.28
79.58.244
66.80
24.12
71,83
63.72
0.5
Proportion
0.9
0.2
0.1
(a)
(b)
(c)
Figure 7: Impact of increasing the number of source models on domain allocation preferences. (a), (b), and (c) depict the domain hierarchy and corresponding sampling proportions generated by Bohdi during the fusion process for Llama3.2-3B Instructt as the target model using K source model (Qwen2.5-14B I.,tsttcc,,,,, K 2 source models (Qwen2.5-144) 1nan..---4B-Instruet and Mistral-Small-22 Instruct-2501), and all three source models, respectively.
as the target model. As the results shown in Fig. 7 due the inherent preferences of each model, fewer source models struggle to comprehensively evaluate the target model's capabilities across various dimensions. Consequently, with fewer source models, the domain sampling proportions exhibit more pronounced bias. Especially when K 1, the domain proportions are almost entirely concentrated on natural sciences and quantum mechanics, while other domains are consistently neglected. As the number of source models K increases, the number of proposed domains also grows, and the proportioning of domain data becomes relatively more uniform, preventing any single domain from being excessively preferred.
models, with Llama3.2-
Bohdi's Fusion Efficiency To further clarify Bohdi's 's fusion efficiency, we tally the runtime (in minutes) for each method across various models in the comparative experiments. All times are normalized the runtime on device setup of NNNiiia A100. For FuseChat, we include the preparatory time for fusion, encompassing the time consumed in representation extraction and vocabulary alignment stages, as these are integral parts of the fusion process. For SFT and FuseChat- 3.0, we account for the time taken to collect responses from source models on the given dataset and the training time. For Condor, we consider the data generation time, including label synthesis, question and answer synthesis, and the answer refinement phase. For Bohdi, we include the time for all steps its entire process, including domain proposal, question and answer generation, and answer selection by the leader model in the Meditation phase, as well as the training in the Enlightenment
27

---

Table 4: Runtime Comparison of Different Fusion Methods (in minutes)
Method
FuseChat
SFT
FuseChat-3., 3.0
Condor
Bohdi (Ours)
1-ama3.2-3B-Instee
2107
576
734
1433
515
Llama3. 10mma3.1-8B.. 1-8B-Instruct
2581
674
927
1595
457
Qwen2.5-7B -n--ttruccttt
2392
642
856
1629
567
Bemmaa-9B-Ii
2917
773
1109
1651
547
phase. As shown in Tab. 4 Bohdi's execution time on each target model is significantly less than that of other methods, further highlighting its advantage in runtime efficiency. This indicates that Bohdi more swiftly and conveniently bring about more pronounced improvements compared to other methods.
Table 5: Performance Comparison of Bohdi between Fixed Leader and Random Leader Selection (Target Model: Llama3.2- 2-3B-Instrruct)
Strategy
MMLU
GPQA
GSMSK
MATH
HumanEval
MBPP
TheoremQA
Fix Seader-Mistral
60.43
35.35
75.51
45.08
56.10
45.0
24.38
51.3
49.14
Fix. Leadee phi
61.25
32.32
74.68
45.42
56,10
46.6
25,50
46.0
48.48
Fix-1 Leader-Qeen
61.01
23.23
75.06
46.52
$7.93
48.8
25.87
54.9
49.17
Random Leader
61.33
78.62
47.92
58.54
25.12
50.79
The Impact of Random Leader Selection on the Stability of Model Fusion To evaluate the impact of randomly selecting the leader model fusion quality, conduct comparative experiments using 1ermaa..-3B-Insttrre as the target model. Specifically, we successively fix each of the three source models Miistrl--Small--a 244B-Instruet-2500 and then compare these configurations against a strategy that randomly chooses the leader for every fusion step. The results shown in Tab indicates that the random-leader strategy achieves the highest average score across the eight benchmarks (50.79%) and exhibits noticeably more stable performance. Because any single leader tends to carry intrinsic judgment biases that can skew the fusion outcome, introducing randomness at the leader selection process helps cancel out these negative effects.
phi-4, and Qwen2.5-14B. 1-nnn--tt---nsstuutt--- as the sole leader
Table 6: Performance comparison (mean = standard deviation) across three independent runs under aligned data budgets (1,782 samples).
Method
Data
MMLLU
GPQA
GSMBK
MATH
HumanEval
MRPP
TheoremQ4
BBH
AYG
Base
59.77
27.78
08.92
34,82
56,10
48.20
23.88
$3.86
46.67
PuseChat
1782
39,84-0.0 0.06
21.44::005
69.22400115
35.52-00.22
54.68:1227
47.40=0.40
24.21=0.39
54.17:0.199
46.56-60.16
61.28
75.18::1.25
42.644--177
57.12::0335
23.37=0.78
PusoChan--3.D
1782
61.10=0.13
25.08:1::60
75.31.:1.99
19:31
58.74:1141
48.73
20,64
23.54
-1.56
35.99
=1.57
47.10
$:050
Condoe
3782
00.70
-0,07
27.79
4:216
71.45
+087
46.58
1-0388
57.32
48.33
-038
25.17
=0.50
30.60
=0.99
45.99 4-0.44
Bohdi
1781.66
1:45.33
61.19
0.13
1166
76.67
1-227
47.08
1.07
58.95
1:0.5
49.87
0.64
25.41
0.73
55.46
0.76
50.53 1:0.6
Controlled Data Efficiency Comparison under Aligned Data Volumes To ensure a fair comparison of different approaches under identical data budgets and to assess their stability, we conduct three independent runs of our proposed method, Bohdi, and unify the data usage of all baselines to 1782 samples-the the rounded average of the data generated across Bohdi's three runs (1781.66). This guarantees that all methods are evaluated under approximately the same data budget. For FuseChat, randomly sample 1782 instances from its generated dataset in each run; for the other baselines, we randomly select 594 samples from each of the three mamains--mthematiiccs programming, and gensral-resulting in total training set of 1782 samples. Under this aligned setting, we report the average performance and standard deviation across the three runs. Experimental results shown in Tab T that Bohdi achieves an average absolute performance gain of 3.86% while maintaining low performance variance, significantly outperforming all baseline methods and demonstrating excellent data efficiency and stability. Moreover, thanks to its automated data ratio allocation mechanism, Bohdi exhibits smaller performance variance compared to other implicit fusion methods, indicating stronger robustness. worth noting that although the explicit fusion method FuseChat shows relatively small inter-run variance, its conservative fusion strategy fails to deliver noticeable performance improvements. These results fully validate the advantages of Bohdi in efficiently utilizing generated data and stably improving model performance.
28

---

E
Limitations
While the Bohdi framework has already demonstrated advantages multiple aspects, including per- formance and efficiency, brand-new prototype framework, it still has the following two areas that could be further explored and improved: 1) It uses fixed instructions for data generation throughout the fusion process, which the introduction of adaptíve prompt design schemes could further promote more comprehensive and diverse data generation and knowledge injection. 2) Further improvement strategies such as self-refinement or multi-agent debate could be considered to help enhance data quality and more deeply extract knowledge from the source models. We will further explore and attempt the above two points in our subsequent research work.
extent limits the diversity of the generated data. Considering
29

---

Related Instruction Formats
Note: In the formats below, the bold text enclosed "()" represents input variables.
F.1 p Expand, Parent (Di,)) in The Sprout Operation
For Main Domain Generation
I need generate hicrarchical systematic knowledge tree. First, I need to determine set of main subject domains, please use your world knowledge to propose a **Main Domain** that systematically taught in primary/sccondaryliihhrr education (e.g., in exact sciences, computer engineering, or other natural sciences and humanities), which should be as broad as possible to cover wide range of child domains.
**STRICT REQUIREMENTS**
Must propose * EXACTLY ONE** new domain name
2. The proposed domain must be clearly defined academic field related to naturaa sciences** (such physics, chemistry), **social sciences** (such law, philosophy), **humanities** (linguistics, art), **formal sciences** (such as mathematics, computer science), or * *interdisciplinaary fields (such as medicine, social psychology, etc.).
**STRICT RESPONSE FORMAT**; The proposed domain must be enclosed between [Proposition Start] and [Proposition End], following the format below: [Proposition Start|propose domain (ornaiin(ooooiiooon End]
Now, please provide your proposed domain according to the requirements mentioned above.
For Secondary Domain Generation
This is path of a hierarchical systematic knowledge tree: (Sampled Main Domain), and you need to propose subject domain that logically and structurally follows this path, i... the domain you propose must be secondary domain of (Sampled Main Domain). **STRICT REQUIREMENTS
Must propose * EXACTLY ONE** new domain name
2. The proposed domain must be clearly defined academic field related * **natural sciences** (such as physics, chemistry), # *osccial sciences** (such as law, philosophy), **humanitie**** (linguistics, art), **formal sciences** (such mathematies, computer science), or innterdiscipplinar"" fields (such as medicine, social psychology, etc.).
**STRICT RESPONSE FORMAT**; The proposed domain must be enclosed between [Proposition Start] and [Proposition End], following the format below: [Proposition propossd domain CamainnPrrpositiinn End]
Now, please provide your proposed domain according to the requirements mentioned above.
For Sub-Domale Generation
This is a path of hierarchical systematic knowledge tree: (Sampled Main Domain) (Sampled Secondary Domain), ). and now you need to propose subject domain that logically and structurally follows this path, i.e., the domain you propose must specific sub-domain of (Sampled Secondary Domain).
**STRICT REQUIREMENTS ke:
1. Must propose #XAATLYY ONE** new domain name
The proposed domain must be clearly defined academic field related **natural *ciences** (such physics, chemistry), **social sciences** (such as law, philosophy), **humanitics*** (linguistics, art), **formal sciences** (such as mathematics, computer science), or "iiterdisccipliaary"" fields (such medicine, social psychology, etc.).
**STRICT RESPONSE FORMAT**, The proposed domain must be enclosed between [Proposition Start] and [Proposition End], following the format below: [Proposition proposproposed domain (aaaannnProoiiioonn End]
Now, please provide your proposed domain according to the requirements mentioned above.
30

---

F.2
[Inquiry; P] and p p|Select: A.) in The Harvest Operation
p IInquiry P) for Question Generation
Now I need to create high-quality SFT data for LLM training, so I need you to generate such data.
For now, y* only need to create one question ** will provide you with a specified main domain, its secondary domain, and further refined sub-domain in the format [Main [ormain--[Secomee
(ammainj-(SubDDmmaain
The corresponding topic is:
(Sampled Main Domain - (Sampled Secondary Domain) + (Sampled Sub-Domain]
The question must meet these requirements:
Strictly fall within the scope of Sub--Domin]] neither too broad nor too narrow, and the stem of the question should first contain sufficient background information or relevant conditions
2. The question you provide should be relatively challenging, but it must be solvable, and the answer should be definitive
3. [Selected Style)
Must be as original and concise as possible
5. The expression style of the question should be **as diverse possible**
Enclose your response strictly between [Question Start] and [Question End] as shown below:
[Question Start]Question[ End]
Now provide **EXACTYY ONE** question for the sub-domain ** (Sampled Sub-
Domain ** within secondary domain ** (Sampleed Secondary Domain ** of main domain ** (Sampled Main Domain **
The aforementioned (Selected Style} is randomly drawn from the following question type style prompts in each question generation process:
Note:
Candidate Question Style Prompts
The question should be a high-difficulty one that requires step-by step-by-step solution, with the answer numbered accordingly.
The question should be open-ended and require the answer include least different perspectives.
The question should require coding to solve, with the answer presented in Markdown code block format.
The question should require comparative analysis, with the answer displayed in table format show pros and cons.
The question should require association with knowledge from other fields (e.g., math music).
The question should be styled as casual conversation and Q&AA daily life, with the tone and speaking style of the reply specified (e.g., using metaphors, rhyming).
31

---

p SSelect: y, A for Response Selection
Please compare and evaluate the quality of the multiple answers to the following question, and return the index of the best one using ** Arabic numerals**;
**Question**
(Generated Quesition)
** **Answers to be Evaluated**
(Candidate Answers}
* **Evaluttoon Criteria**
1. Accuracy (40%): Whether the answer correctly solves the problem and match the required
response style.
2. Clarity (30%): The expression is structured clearly and smoothly, and the content is concise without being verbose.
3. Completeness (20%): Whether all necessary details are covered.
Relevance (10%): Whether the answer closely relates the question.
** **Additional Constraints*
Incoherent and repetitive answers should be directly assigned low score, regardless of whether they contain the correct answer.
**Output Requirements**
1. You only need to reply with which answer you consider to be the best answer, and return the index (1 (1-digit) of the best answer enclosed between [Best Answer Start] and [Best Answer End], following the format below:
The best answer is [Best Answer StartJindex(Best Answer End].
2. Only one index for the best answer can be returned.
32

---

G Examples of Bohdi's Synthetic Data
Here we present the synthesized data generated during the Bohdi fusion process from our main experiments. We label the title of each colorbox containing an example with its corresponding branch (Main Domain - Secondary Domain - Sub Domain).
Mathematics - Algebra + Linear Algebra
Question:
Given two matrices \V( A (begin((patrrx] 1 & 1 \V) and W Wbbegin(paatrix) -1 & \\ 0 & 2 Vend(ppatrrix) \V, compare their eigenvalues and eigenvectors. Discuss the implications for diagonalizability and invertibility in table format.
\MI 3 & (eed(ppaarii)
Answer:
To compare the eigenvalues and eigenvectors of the given matrices VV W and AV AV, to discuss their implications for diagonalizability and invertibility, we first need to calculate the eigenvalues and eigenvectors of each matrix. \mln#s Matrix \V A WV'nGiven \V( A W/begin(pnatrix) \I 3 & 4 (maa(iiaa W. Ал\л#а# Eigenvalues AVC AnThe characteristic equation for matrix \V( \) given by AV( (Adet \ WWtaWW/ailldet (begee//bogiinmpmaarrr// 1-\llambda
lambda I) lambda Vlend(pmatrix) (1-111ambda) (4-11lambda) 6 \llambda"2 51llambda Vet:Naa//nn/aammaaa = \\
\11 3 & 4-11
- = 0'm 0\n\\]\nSolving the quadratic equation,
frací5 Vpm \lagrt(25 9ソ)2] Vifrac(5 Vppm (1agr
the
eigenvalues are \1( \llambda_1 \frrac(5 (1sgrt(33))<12) W) \V( \\ lambda_2 - (5frac (3sgrt(33)) AnAnRRи Eigenvectors of A A W) 1-\llambda__ & 2 \11 3
nFor \VC \\lambda_1
4- 4-1\1ambda_1 (emdtrmatrix)) ()begin(pmaarrix> \111 y Vend(pmatrix)
begin(pmatrix) 0 1111 <eend(pmatrix//l11 \nSolving this system, get an Vi( \llambda_ \V. \n\nFor \V( \\lambda_2 A\)
eigenvector corresponding \//la\\begin(ppaariix 1-1llambda_2
Alll 3 & 4- 4-1llambda_2 Vlend(pmatrix) Alll \lend{ pmatriz/anWWnnSSorriiaaa this system, get eigenvector correaponding \V( \\lambda_2 (V. AnlnSince \V W has two distinct eigenvalues, it diagonalizable. ininsse Invertibility of AY \nSinc \VC \V) is a W 2 \\times \1) matrix with two distinct eigenvalues, neither of which is zero, \VC 11) is invertible. \n\n#e# Matrix AV B W\nGiven WV {pmatrix} -1 & \IIl
\bbein((mattrix) x \M y Vend(pmatrix) = (bbggi((matrri)
Wbeeii Wend(pmatriz) W) \ Eigenvalues of \V \llambda I) = o equation for matrix \VC \D is given by AVC Videt (ew.aaaeeeeeeeeeiifammaarriiii - 11-1llambda & MM Oln\VAnnSo the IV. Am\n##*# 11) and \V( \\lambda_2 (pegin Eigenvectors \\( \nro \VC \Vlambda_1 /o// :1:
\nThe characteristic
2- 2-1laaadaa (lend(pmatrix) - (-m-daaambda) (2-\aaa \V( \1lambda_1
eigenvalues
-1-(-11) & 1111 0 & (-1)) Venndppattiix) (Mbogi<pmmatrix> 1111 y Mend( pmatrix) = \ibegin(pmatrix) 0 1111
(lend(pmatrix) \lbegin(pmatrix) <\M (lend(pmatrix) = Wbeginn(mattris 0 111l (lend(mattriz////W
nSolving this system, get an eigenvector corresponding to \VC \|lambda_1 \V AnnoFor \VC Wllamdda_2 0 & 2-2 Wend(pmatrix) \begin(pmatriz> x 1111 \end((pattrix) Albegin( pmatrix] -3 pmatrix) - \1begin(pmatrix) 0 (Ill get an eigenvector corresponding to \V( \[lambda_2 2 An).Annnimm AVC В \V has two distinct eigenvalues, it is diagonalizable. \m\n####
-1-2 WII
(1ll 0 & o (lend(pmatrix) \/begin(pmatrix/ x AII Wlnnd
this system,
Invertibility of \Y( B \nince \ B \V) is a \1 A\times 2 \V) matrix with two distinct eigenvalues, neither of which is zero, \VC B \V is invertible. \n\n### Summary TableAn/ Matrix Eigenvalues Diagonalizable Invertible /mnl-.......I---.
/n Anl WC
WD I A Vifrac(5 ((sqrt(33))(2,, Vfrrac(5 Vlsqrt(33)) \V) Yes Yes Anl WC A W -1, 2 \V) Yes Yes An
33

---

Biology Geneties Molecular Geneties
Question:
Given DNA sequence represented as string, write Python function to find the positions of all occurrences of specific palindromic DNA sequence (e.g \"CGGCG\") within tuples, where each tuple contains the start and end indices of each occurrence. For instance, if the input DMA sequence is \"ACCGGOCTACCCeeCTTY and the target palindromic sequence is \"CGGGG)", the function should return [(2, 6), (9, 13)]. Note: In DMA sequence, the indices 0-based, and the sequence read from to 3',
sequence.
function should return
Answer:
(sequence, pattern): : \n \l\"\"ln Finds all occurrences of a specific palindromic DNA sequence within given DNA sequence. An'n Args: in sequence (str): The DÑA sequence to search in, In pattern (str): The palindromic sequence to search for. An'n Returns: in list[ tuple[int, intll: list of tuples, where each tuple contains the end indices of each occurrence. \n'n ""\"(""n Initialize an empty list to store the positions of the occurrences positions [nn Iterate over the sequence with sliding window of the pattern's length'n for i in range len (en(seqenee) len (pattern) 1): Check if the current window matches pattern:
and
the pattern'n sequence(i:i [en(pateenn]] add the start and end indices to the positions list'n positions. positions..ppend((C (1, i len(pattern)) # Return the list of positions'n return positions An Example usage: 'nsequence ("ACOOGCCTACCOGOOT) " "'apaateen \"CGGCG\ " (mprint( hhallppllnaaooicccseeeeeeeeeeees
it matches,
pattern)) Output: [(2, 6), (9, 13)]
34

---

Formal Sciences Theoretical Computer Science Computational Complexity The- ory
Question:
Consider a computational problem where you are given a Boolean formula in Conjunctive Normal Form (CNF) with variables and m clauses. From the perspective of computational complexity theory, discuss how the difficulty of determining whether there exists assignment of truth values these variables that makes the formula true, varies as function both Additionally, explore how the problem's complexity might differ the clauses are restricted to having exactly three literals each, forming a 3-SAT problem.
m.
Answer:
The problem of determining whether there exists an assignment of truth values variables that makes Boolean formula Conjunctive Normal Form (CNF) true known as Satisfiability problem (SAT). problem in computational complexity theory and is the first problem to be proven NP co-cleeee in General CNF SAT'nThe complexity of the general SAT problem
fundamental
function of the number of variables \V( V\) and the number of MP-complete,
clauses AV \V is straightforvard The problem meaning that any instance relative to the size of the input if and only if P = NP, which is major unsolved problem in computer science. In practice, the difficulty of solving SAT instances can vary greatly depending Generally, as \Y n \V) and \\( \V) increase, the problem becomes harder to solve in the worst case. However, the exact relationship between the difficulty and the parameters AY \l) and A m W) is not well understood, and there \V and large \\( \D), and vice versa. An\nss# 3-SAT\nWhen the clauses the CNF formula are restricted to have exactly three literals each, the problem is known as 3-SAT. This is special case of SAT and is also NP- complete. The restriction to three literals per clause does not make the problem easier from complexity-theoreiic perspective; 3-SAT remains NP- complete. However, practical algorithms for solving 3-SAT can sometimes more efficient than for general SAT because of the restriction on clause size. The problem's structure can sometimes be exploited by specific algorithms to reduce the search space. An\a#re Complexity as Function of \V n W)) and \V( Wn **General SAT**: The complexity of SAT increases with both WV WD and VI m WV, but the exact relationship problem is in NP, and no polynomial-tine algorithm is known for solving it in the general case, An- **3-SAT**: Similarly, the complexity of 3-SAT increases clause with both \V( n \V and \V( W. However, due the restriction size, certain algorithms may perform better on average or for certain types of instances. \n'ntts Practical Implications
the problem can be solved in polynomial time
the structure of the CNF formula.
instances of SAT problems that are hard even for small AVC
not well-defined. The
practice, the difficulty of solving SAT and 3-SAT instances can be highly dependent on the specific structure of the problem instance. For example, instances with many clauses compared to variables might be easier to solve than those with fewer clauses. Similarly, instances with certain types of variable interactions might easier or harder to solve. Advanced algorithms like Conflict-Driven Clause Learning (CDCL) SAT solvers exploit such structures to solve large SAT instances efficiently in practice. \n\n
35

---

Environmental Science Environmental Law Environmental Litigation
Question:
Consider hypothetical scenario where company is facing lawsuit for violating environmental regulations. The lawsuit alleges that the company emitted
200 tons pollutants over period of 5 years, exceeding the permissible limit by 50 tons. The company claims that due to software error, the emissions data was incorrectly recorded, and the actual emissions were only 150 tons. Assuming the permissible limit is 175 tons per 5 years, and the company needs prove its claim using statistical evidence. Write Python script to simulate the emissions data for years, considering normal distribution of daily emissions with a mean of 10 tons and a standard deviation of 2 tons, and calculate the total emissions to validate the company's claim.
Answer:
This script should also include function calculate the probability that the total emissions exceed the permissible limit.
"'python\nirport numpy as np Function to simulate emissions data\ndef 365 botal_days
simulate simulate_emissienn myissions(years, mean, std_dev) : An days_per_yees years days_per_yearlv emissions = random. normal (mean, std_dev, total_days) in return omissions\mnnf Function calculate total emissions) ndef calculate_too wow.tolemeeeiinnseeeeeeeee (emissions): :'n return np. ap.sum (emissions) \n\n# Function to calculate the probability of exceeding the permissible 1imit Andef calculate_ calculate_exceedant www.eeeacceeececeeeeeeeeeeeeeeee total_emission: acceedance_probbbillie (tt. mp..eanttotaa return exceedance hrt.:ddccee_ooobbbitttllllaae Paramtttr Parameters nyeare 5'nmean, emissions 5\mmean__missions
potpelaacce_pobbblllyyyo (emissions. permissible_limis
(emmissible_limitt)
desttddder_mmissins 10 1o'mstd_dev.. emissions Sumpermissible_limit = 175\m\n Simulate emissions
data\nemissa data simulate_emissiine (years, rean_emissions
std_dev_ [ei_dde__missions) \n\n# Calculate total emissions intotal_emissionn
\n Calculate probability of exceeding the permissible limit\unexceedance, prctiiiipoobbbbttteeeeeee =
www.ceeeaacc...rbbbbbillllaaaaaa _ataa, permissible limestibleel_mitittee \nAnf
Print results\uprints (f "Total Emissions:
tons) ") (fprint(())
Probability Exceeding Permissible Limit: (exceedance_ prterduuc_ppobbbbiiitt..... An
\n
36

---

Natural Sciences
Physies Quantum Mechanics
Question:
Imagine you are tasked with simulating two-level quantum system, often referred
qubit, in magnetic field. The Hamiltonian of such system can described as \\H WVrrac(\\bbar (2oeeaa) \lsigma_z\V) where W\C\/bbr\\ is the reduced Planck constant, WV(\ommega\) is the angular frequency, and \1(\lsigma_2\) is the Pauli z-matrix. Given an initial state (otett////ppi)))///pggti
+ \lleft time \(tt\)
(1\lright>\\V, find the state \CClleff Write a Python function that calculates the state any given time \(((\ using the provided parameters. Assume \\(\\hbar 11)) for simplicity.
Answer:
To solve the problem of finding the state of a qubit in a magnetic field at any given time \((t\)) can use the time-evolution operator \((((t) = e- iltt/\hbar]1 Given the Hamiltonian (\( \ifrac() \hbar (ooneaa \\ sigma_z\V and setting W\AVbbar 11V, get \V(H V\frac([lomegaaae sigma_z\V). \n\nThe Pauli z-matrix AN(Vleigma_z\) is: Vn>>m//// \eigma__ W begin(pmatrix>)ul & 0 \1llln & - w1wp: - \aThus, the Hamiltomian becomes: \n\\[\nH (wfrar/lllllnnooco (emgga)(( \begin(pmatrix//ll & \llllno -- can be written as: (g:Un))(( e- e<<=[(Mfraac(\ommeeaa2
time-evolution operator (1(())
Wasigaaz )t) @"CiN\fracc(\omme t){2) \/eigma_//Ln/ \nlnooor a diagonal matrix like \((\ssima_z21), the exponential of the matrix is simply the exponential of each diagonal (11
()) \/sigma_z) Abbeiing matzzzz)nne<<1115rrraa\\ooor t}{2}} Illllnn & e D-fillfrac(Veemee tH²)) n allendfpaastrrii//ul//hhhaiienne the initial state (NClieet/NNpaaaNNNrrrpggp
Mtfacl(((lver(000(111110100111 + + we can write it begin(pmatrix//ll \ = (Nfrac((HNNegr((2)) W find the state time W(tVV, we apply the time-evolution operator \I(U(t) to the initial state /lal/haallet////eeset)) (rigtt/ = U(t) \1left/\/psi (ogeet//\ MaWWggggtaaV/Waanllettt write the Python function to calculate this:\nin"" 'python \n'ndef calculate_stare state(t, omega): '"("("ln Calculates the state of the qubit at time given float): Time at which to calculate the state. omega (float): : Angular frequency of the magnetic field. An \n Returns: in np.ndarray: The state vector time t. An \" # Define the time-evolution operator\n U_t array ([[np.exp(-1j omega
column vector: hatp
operatt
"yyron "python\nimport wnimortt numpy as np
angular frequency omega. An Parameters:
2), 0] An [0, exj omega
2) An
Initial state vector psi_0 np. a[r 1)) (2) \n Calculate the state at time tln psi_t up. do. psi_0)\n \n return psi_t\ n'n# Example maage:Ant = 1.0 Time\\omeee calculate_stateel omega) imprint "State at time t:\", psi_t))nn
1.0 # Angular frequency\apsi.
37


Reviews:
[{"review_id": "l7KPEfqX5O", "reviewer": "Reviewer_YY3V", "summary": "This paper presents a novel framework for fusing knowledge from heterogeneous LLMs into a compact target model without real data. It addresses key limitations of existing methods, including limited domain coverage and capability imbalance,  via organizing knowledge into a hierarchical tree and automating domain exploration/data generation using multi-model collaboration. The key innovations include dynamic domain expansion and data allocation as a Hierarchical Multi-Armed Bandit problem, solved by the DynaBranches mechanism with online capability tracking. Experiments demonstrate significant empirical improvements. Overall, this is a very good work. But without the available code, it is not easy to fully justify this framework.", "questions": "Check the weakness part mentioned above.", "limitations": "Check the weakness part mentioned above.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 4, "originality": 4, "strengths_and_weaknesses": "Strength:\n1. Removes dependence on any real or in-domain dataset, allowing the target LLM to inherit knowledge that is both broader and privacy-safe.\n2. Adaptive allocation virtually eliminates capability imbalance and achieves markedly higher data efficiency than static-mix baselines.\n3. Online adaptation yields consistent performance gains across diverse benchmarks, outperforming previous fusion methods on every evaluated target LLM.\n\nWeakness:\n1. The major concern about this work is that the framework is quite complex and without the available codes, it makes the justification and  and replication challenging.\n2. The magnitudes of improvement and decline reported in Table 1 are unclear, as the reference baseline against which they are calculated is not specified. \n3. The designations “H1” and “H2” in lines 140–142 do not align with the labels shown in the corresponding figure, which may cause confusion.\n4. The influence of choosing the auxiliary model H2 at random has not been explored; a dedicated ablation study is needed to assess its impact.", "was_revised": true, "final_justification": "I am satified with the author's response. My final rating is accept.", "rebuttal": "# Response To Reviewer YY3V\n\nWe sincerely appreciate the reviewer’s thorough and exacting review. Your comments and suggestions are pivotal to improving our manuscript. We provide a point-by-point response below.\n\n### Response To Weakness #1\nWe appreciate the reviewer's insightful feedback.\n\nFor comprehensive design, Bohdi incorporates multiple components and mechanisms. To ensure readers fully grasp its functionality, we've utilized various presentation methods. Beyond the main methodology in the body of the paper and operational details in the appendix of the supplementary materials, we include **schematic diagrams** for each component and the overall framework (Figures 2 and 3 in the main text). Crucially, Appendix B.4 provides a **highly detailed pseodocode of the entire algorithm's workflow**, and Appendix F contains **all associated LLM instructions**. We believe this offers readers a thorough understanding of our algorithm's operational logic and specific procedures.\n\nRegarding code availability, **we commit to including a link to the open-source code in the final version of the paper**. This will clearly outline all running, evaluation, and environment setup instructions, along with links to the **Bohdi-fused model weights**, as promised in our checklist. We hope you understand our decision not to directly include the full code at this stage, as it serves to protect our original work. We again appreciate your valuable comments.\n\n### Response To Weakness #2\nWe appreciate the reviewer's valuable feedback.\n\nTo clarify, all our reported improvements and declines are determined **relative to the base model's performance on its corresponding task**. For instance, when the target model is Llama3.2-3B-Instruct, Bohdi improves its BBH accuracy to 54.90% from the vanilla Llama3.2-3B-Instruct's 53.86%, marking a 1.93% relative increase over the original model. For overall average performance, the Bohdi-fused Llama3.2-3B-Instruct achieves an average accuracy of 50.79%, which is an 8.83% relative improvement compared to the vanilla Llama3.2-3B-Instruct's 46.67%. We've explicitly noted this clarification in the revised manuscript to enhance reader understanding. We again appreciate the reviewer's valuable suggestions, which have been highly helpful in enhancing the clarity of our paper.\n\n### Response To Weakness #3\nWe appreciate the reviewer's meticulous examination and insightful suggestions. We have taken note of this point and made revisions accordingly in the updated version. Additionally, we have conducted a thorough review and polishing of the entire manuscript. We again appreciate your insightful comments, which play an important role in refining the details of our work.\n\n### Response To Weakness #4\nWe appreciate the reviewer's constructive suggestions. Following your advice, we conducted experiments where each of the three source models was fixed as the Leader Model, and compared these with the current scheme of randomly selecting the Leader Model, using Llama3.2-3B-Instruct as the target model. The results are presented in the table below. In comparison, the scheme of randomly selecting the Leader Model demonstrates more balanced performance gains and achieves the best average (AVG) performance. While fixing a single model as the Leader may yield better results in certain individual tasks than random selection, it often leads to unbalanced performance due to the inherent biases of each model. Randomly selecting the Leader can better counteract the negative impacts of such biases and alleviate the issue of performance imbalance.\n\n| Strategy              | MMLU  | GPQA | GSM8K | MATH | HumanEval | MBPP | TheoremQA | BBH  | AVG       |\n|-----------------------|-------|------|-------|------|-----------|------|-----------|------|-----------|\n| Fix-Leader-Mistral    | 60.43 | 35.35| 75.51 | 45.08| 56.1      | 45   | 24.38     | 51.3 | 49.14  |\n| Fix-Leader-phi        | 61.25 | 32.32| 74.68 | 45.42| 56.1      | 46.6 | 25.5      | 46   | 48.48  |\n| Fix-Leader-Qwen       | 61.01 | 23.23| 75.06 | 46.52| 57.93     | 48.8 | 25.87     | 54.9 | 49.17    |\n| Random-Leader         | 61.33 | 30.3 | 78.62 | 47.92| 58.54     | 49.6 | 25.12     | 54.9 | 50.79  |\n\nFurthermore, in the current version of the manuscript, we have also verified the approach of using a Reward Model for Answer evaluation in the ablation experiment shown in Figure 4. The results indicate that randomly selecting a Leader Model from the source models for evaluation remains a more optimal strategy. We again appreciate the reviewer's valuable suggestions, which are of great significance for enhancing the quality of our work."}, {"review_id": "NbXmEmL0vd", "reviewer": "Reviewer_ShRp", "summary": "This paper proposes Bohdi, a heterogeneous large language model fusion framework based only on synthetic data. Bohdi systematically organizes domain knowledge into a three-level structured knowledge tree. It formulates domain expansion and data ratio allocation as a hierarchical multi-armed bandit (HMAB) problem and solves it using the proposed DynaBranches mechanism. Bohdi contains an introspection-rebirth (IR) module that adaptively adjusts domain sampling using a sliding window binomial likelihood ratio test (SWBLRT) based on performance feedback.", "questions": "- Can you rovide detailed output and evaluation of intermediate steps? I would like to know the numerical details and detailed examples of the new data created in the paper. Rather than just seeing mathematical proofs, I want to see actual proof that this method can be implemented.\n\n- The paper mentions 5 different models, but only evaluates two of them as target models, and there is randomness in the selection of the method, which raises questions about the stability and generalizability of the method.\n- What are the memory, computation, and latency costs as the number of domains and source LLMs increase? Is there a practical limit to the granularity or number of domains that Bohdi can handle?", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 4, "strengths_and_weaknesses": "**Strengths:**\n\n- It is claimed that Bohdi is the first framework exclusively using synthetic data for heterogeneous LLM fusion and data can be allocated dynamically among different LLMs.\n- The method in this article is novel and clearly explained, as well as using clear mathematical symbols.\n- The amount of experimental data is large, especially the ablation study, the angles are diverse, and there are experimental verifications in different fields and different models.\n\n**Weaknesses:**\n- One of the innovations of the entire framework is to synthesize data without using existing data, but the article rarely discusses or conducts experimental analysis on the quality and correctness of the synthesized data, as well as comprehensiveness and reproducibility of newly generated data. In addition, this method of synthesizing data has a lot of randomness, which is mainly reflected in the selection of the Leader model in Harvest and the selection of the sub-field with the highest sampling value according to the Beta distribution in DynaBranches as the next direction to be explored. The article does not provide a detailed analysis of these intermediate results and randomness.\n- The source models used in the experiments vary significantly in parameter scale, and the target models come from different companies and also differ in size. This raises concerns about the consistency and fairness of the experimental setup. If the goal is to demonstrate that your method is adaptable across different models, the experiments should first be conducted on models of the same size. Only one variable should be changed at a time in order to ensure a valid controlled comparison. And then you can report the results containing variable settings.\n- The construction and management of a hierarchical tree, the need to run multiple source models for every “Sprout” and “Harvest” operation, and the use of adaptive sampling across potentially hundreds of domains (as illustrated in Figure 6) could become significant bottlenecks when scaling to very large source LLM pools or fine-grained domains. However, potential challenges related to compute, memory, or latency are not addressed in the main text.", "was_revised": true, "final_justification": "After carefully checking out the authors' response, I think my initial score still reflects my evaluation after checking out the author's rebuttal, so I am keeping it unchanged.", "rebuttal": "# Response To Reviewer ShRp\nWe appreciate the reviewer's meticulous examination and valuable feedback, which are of great significance to our manuscript. Below, we will provide point-by-point responses to each of your comments and suggestions.\n\n### Response To Weakness #1\nWe appreciate the reviewer's valuable comments. In line with your feedback, we have added a dedicated section in the appendix of the revised manuscript to present and discuss the quality of synthetic data. Here, we briefly showcase a piece of data from the path Mathematics - Algebra - Linear Algebra. Due to space constraints, we have omitted part of the response here:\n\nExample:\nQ: Consider two matrices, $A = \\\\begin{pmatrix} 2 & 4 \\\\\\\\ 3 & 6 \\\\end{pmatrix}$ and $B = \\\\begin{pmatrix} 1 & 2 \\\\\\\\ 0.5 & 1 \\\\end{pmatrix}$. Compare the eigenvalues and eigenvectors of $A$ and $B$. Discuss the properties of these matrices based on their eigenvalues and eigenvectors, and summarize your findings in a table format.\n\nA: ...... Matrix A: ...... **Eigenvalues**: \\n   The characteristic equation for matrix $A$ is given by ...... Thus, the eigenvalues are $\\\\lambda_1 = 0$ and $\\\\lambda_2 = 8$.\\n\\n2. **Eigenvectors**:\\n   - For $\\\\lambda_1 = 0$:\\n     ...... we can choose $x = 2$ and $y = -1$ as a solution ...... ### Matrix B: ......\\n\\n### Summary Table\\n\\n| Property/Matrix | A                     | B                     |\\n|---|---|---|\\n| Eigenvalues     | 0, 8                  | 0, 2                  |\\n| Eigenvectors    | (2, -1), (2, 3)       | (2, -1), (2, 1)       |\\n| Properties      | $A$ has one zero eigenvalue and one non-zero eigenvalue. It is singular and non-invertible.<br> $A$'s eigenvectors are not orthogonal. ...... |\\n\\n\n\nThe question in the above example data involves comparing the eigenvalues of a matrix and requires an answer in table format. The generated response correctly solves this problem and strictly follows the requirement to present the answer in a table, thus demonstrating full correctness and completeness, which indicates that the synthetic data is of high quality.\n\nFurthermore, the randomness introduced by the Beta distribution sampling serves as the foundation for ensuring diversified and parallel domain exploration, as discussed in Section 2.3. However, it does not unduly affect the stability of the algorithm. Existing research has also provided solid guarantees for the convergence of such sampling algorithms [1][2]. Meanwhile, in the ablation experiment illustrated in Figure 7 of the supplementary materials, we visualized the domain structure generated when Llama3.2-3B-Instruct is used as the target model, with statistics showing that as the number of source models increases, the generated domains still encompass most of the domains observed when there are fewer source models. This strongly confirms the stability of the overall domain expansion.\n\nRegarding the impact of randomly selecting the Leader Model, we conducted experiments where each of the three source models was fixed as the Leader Model, and compared these with the current scheme of randomly selecting the Leader Model, using Llama3.2-3B-Instruct as the target model. The results presented in the table below show that randomly selecting the Leader Model yields more stable performance. Since each model has inherent judgment biases, which can lead to unbalanced performance, random selection of the Leader can better counteract the negative effects of such biases. Therefore, the randomness introduced by the random selection of the Leader Model and the Beta distribution random sampling does not have a negative impact on the stability of the algorithm.\n\n| Strategy              | MMLU  | GPQA | GSM8K | MATH | HumanEval | MBPP | TheoremQA | BBH  | AVG       |\n|----|--|--|--|------|---|-|---|------|---|\n| Fix-Leader-Mistral    | 60.43 | 35.35| 75.51 | 45.08| 56.1      | 45   | 24.38     | 51.3 | 49.14  |\n| Fix-Leader-phi        | 61.25 | 32.32| 74.68 | 45.42| 56.1      | 46.6 | 25.5      | 46   | 48.48  |\n| Fix-Leader-Qwen       | 61.01 | 23.23| 75.06 | 46.52| 57.93     | 48.8 | 25.87     | 54.9 | 49.17    |\n| Random-Leader         | 61.33 | 30.3 | 78.62 | 47.92| 58.54     | 49.6 | 25.12     | 54.9 | 50.79  |\n\n[1]Analysis of Thompson Sampling for the Multi-armed Bandit Problem, in JMLR.\n\n[2]An Information-Theoretic Analysis for Thompson Sampling with Many Actions, in NeurIPS.\n\n### Response To Weakness #2\nWe appreciate the reviewer's constructive suggestions, which are crucial for enhancing the quality of our work.\n\nWe appreciate the reviewer's constructive suggestions. The primary purpose of the structural differences between the source models and target model used is to ensure heterogeneity between models, which aligns with the focus of our research.\n\nRegarding the **selection logic for target models**, we aimed to cover the most widely accessed and used mainstream model types, thus choosing Qwen2.5-7B-Instruct, Gemma2-9B-IT, and Llama3.1-8B-Instruct. These three models, with comparable parameter sizes, form the main body of our experiments. To further verify the effectiveness and generalization ability of our method, we also included Llama3.2-3B-Instruct, a model with a smaller parameter size, as a target model. Due to space constraints, these results are reported in Table 1 of the main text and Table 2 of the appendix, fully demonstrating the universality of our method across target models of different structures and scales.\n\nIn summary, we have **fully considered controlled comparisons of parameter sizes in model selection**, including both model combinations with similar parameter sizes and models of different scales. Meanwhile, selecting models from **different companies** is intended to **ensure the premise of heterogeneity** and strengthen the persuasiveness of our experimental results by using the most mainstream open-source models available. Furthermore, in **each set of experiments, all compared methods were conducted under the same source and target model settings**, thus ensuring the fairness and consistency of the experiments. We have clearly elaborated on the above content in the revised version. We again appreciate your valuable suggestions, which are vital for improving the clarity of our work.\n\n### Response To Weakness #3 & Question #3\nWe appreciate the reviewer's insightful comments. Regarding the cost of constructing and managing the hierarchical tree, we will address this from two dimensions: the number of models and the number of domains.\n\n1) **Concerning the number of models**: When the number of involved LLMs increases, the cost rises linearly with the number of LLMs. However, any scheme built on multiple LLMs will inevitably face this objective cost increase due to a larger number of LLMs, which is also true for all the methods we compared with. New domains generated through the Sprout operation are fixed as new components of the tree after generation, so Sprout is typically executed a limited number of times. The response collection in the Harvest operation is only related to the amount of generated data—for each piece of data generated, responses from each model need to be collected once. Similar operations are equally necessary for other baselines, whether explicitly or implicitly fused. Thus, while an increase in model parameters leads to higher costs, this is an unavoidable objective cost increase for any current method.\n\n2) **Concerning the cost from an increase in the number of domains**: The computational cost brought by more domains is mainly reflected in the sampling process of the TS algorithm. However, this process does not require direct participation of any LLM; it only involves variable sampling based on the Beta distribution parameters. The computational effort required here is merely O(1) floating-point operations per data point. Even when sampling across 1000 domains, this can be completed in less than 0.001 seconds on a regular consumer-grade CPU. Moreover, our method involves sampling across at most several hundred domains (see Figure 6 in the main text). Therefore, the efficiency cost from an increase in the number of domains and their management is almost negligible. Additionally, in Table 3 of our Appendix, we have provided statistics on the actual total running time (in minutes) of each baseline method and Bohdi on each target model when using 8 A100 GPUs, which practically demonstrates Bohdi's efficiency advantage over other baselines.\n\nWe again appreciate your valuable comments.\n\n### Response To Question #1\nWe appreciate your valuable question. In response to this, all instructions involved in Bohdi can be found in Appendix F. For examples of data and analysis of data quality, please refer to our response to weakness #1.\n\n### Response To Question #2\nWe appreciate your insightful comments.\n\nRegarding the comprehensiveness of target models, we aimed to cover the most widely accessed and used mainstream model types, thus selecting four target models: Qwen2.5-7B-Instruct, Gemma2-9B-IT, Llama3.1-8B-Instruct, and Llama3.2-3B-Instruct. Due to space constraints, the results with Llama3.1-8B-Instruct and Qwen2.5-7B-Instruct as target models are recorded in Table 2 of the appendix, where Bohdi still shows significant advantages over other baselines. In the final version of the paper, we will also move this part of the results to the main text.\n\nAs for the logic behind selecting baselines, we chose FuseChat3.0, the current state-of-the-art implicit heterogeneous model fusion baseline, and FuseChat, the most representative explicit fusion baseline. Additionally, to validate the effectiveness of Bohdi as an implicit model fusion scheme, we included a baseline that directly collects data for SFT and Condor, a baseline that performs data synthesis based on a static label tree. We have further emphasized this baseline selection logic in the experimental section of the revised version. We again appreciate your valuable suggestions."}, {"review_id": "XObQ8I3S3N", "reviewer": "Reviewer_3FuQ", "summary": "The paper introduces Bohdi, a framework that fuses knowledge from several heterogeneous source LLMs into a smaller target LLM without using any real data. Bohdi (1) arranges knowledge domains in a three-level tree, (2) explores new sub-domains through a Sprout operation and generates synthetic Q-A pairs through a Harvest operation, and (3) frames domain selection and data-allocation as a Hierarchical Multi-Armed Bandit solved by a DynaBranches variant of Thompson Sampling, equipped with an Introspection-Rebirth mechanism that resets posteriors via a sliding-window BLRT when the target model’s ability shifts.", "questions": "How many total tokens were generated by Sprout + Harvest over 50 rounds? Please provide an approximate compute cost (GPU-hours).\n\nDid you check for overlap between harvested Q-A pairs and any of the evaluation test sets (particularly GSM8K and MATH)?", "limitations": "Please refer to the Weaknesses and Questions.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Pros:\n\n1. Casting domain expansion + data-ratio selection as an HMAB and augmenting Thompson Sampling with IR is interesting and new in the LLM-fusion context.\n\n2. The authors evaluate their method on diverse benchmarks across knowledge, coding, math and reasoning; includes ablations on IR window/quantile, data split ratios.\n\n3. The authors use two orders of magnitude fewer training samples than competing fusion or SFT baselines yet often matches or beats them.\n\nCons:\n\n1. The improvements over SOTA methods are incremental. Also since only single runs are reported and the authors explicitly acknowledge the absence of error bars due to cost. Therefore the variance in evaluations can be large.\n\n2. Condor and FuseChat-3.0 rely on explicit reward models or preference optimization. The authors disable reward models in Bohdi’s main results, yet the strongest baselines in recent literature (e.g., InfiFusion, SuperMario) are not included or are lightly tuned.\n\n3. While Bohdi uses only 1.7K target-training samples, it repeatedly queries three ≥ 14B source models for Sprout/Harvest. The total token budget and wall-clock cost are not reported.", "was_revised": true, "final_justification": "While I'm willing to raise my score to 4, I will not actively defend the paper if other reviewers or the AC express concerns.", "rebuttal": "# Response To Reviewer 3FuQ\nWe appreciate the reviewer's valuable comments and suggestions. We will provide point-by-point responses to each of your questions below.\n\n### Response To Cons #1\nWe appreciate the reviewer's valuable suggestions.\n\nRegarding **the performance improvements of our method**, it is evident from the experiments on all four target models (results in Table 1 of the main text and Table 2 of the appendix) that Bohdi achieves the optimal performance improvement on average. Moreover, it accomplishes this with significantly less data than other baselines and without relying on any real external data, which is no easy feat under such conditions.\n\nEven setting these prerequisites aside, the performance improvements brought by Bohdi are still prominent. For instance, compared to explicit fusion baselines like FuseChat, Bohdi delivers remarkably significant performance gains in single domains. For example, on TheoremQA, Bohdi brings a 72.09% relative improvement to the target model Gemma2-9B-IT compared to the original model, while FuseChat only achieves a 6.40% relative improvement. When compared to other implicit fusion baselines such as FuseChat3.0, Bohdi's advantages remain evident, mainly reflected in more balanced performance improvements across tasks and better average performance. For example, when the target model is Llama3.2-3B-Instruct, the model fused with Bohdi shows performance gains in every domain, and in most cases, achieves the optimal improvement—particularly a 9.07% relative improvement on GPQA. In contrast, FuseChat3.0 exhibits significant performance declines on GPQA, TheoremQA, and BBH, with a 20.01% decline specifically on GPQA.\n\nThus, **the performance improvements brought by our method are relatively significant, especially considering the prerequisites that it does not rely on any real data and uses far more data-efficiently than baseline methods**.\n\nRegarding **the stability of the evaluation**, we have strongly demonstrated the consistency and stability of Bohdi's performance through multiple comprehensive experiments. For example, in the comparative experiment dimension, we compared all methods on four target models with different structures and parameter sizes: Llama3.2-3B-Instruct, Qwen2.5-7B-Instruct, Gemma2-9B-IT, and Llama3.1-8B-Instruct. The results reported in Table 1 of the main text and Table 2 of the appendix show that Bohdi stably outperforms other baselines across different target models. In the ablation experiments, we also tried using several more compact source models (Llama3.2-3B-Instruct, Qwen2.5-3B-Instruct, and Gemma2-2B-IT) and conducted fusion experiments with the larger Gemma2-9B-IT as the target model. The results indicate that the target model fused with Bohdi can still effectively absorb the advantages of the source models, particularly evident in tasks where Gemma2-9B-IT itself performs weakly, such as GPQA and TheoremQA. Additionally, in Figure 7 of the appendix, we visualized the generated domain structure organization when Llama3.2-3B is used as the target model. As the number of source models increases, the generated domains still include most of the domains observed when there are fewer source models, which strongly illustrates the stability of Bohdi in domain expansion. **The consistent improvements across these diverse evaluations and experimental results fully demonstrate Bohdi's stability under different conditions**. Furthermore, during the evaluation, we used greedy sampling to minimize the impact of randomness in autoregressive generation, as described in Appendix C. Following your suggestion, in the final version, **we will incorporate these analyses and results into the main text as much as possible to help readers understand these improvements more intuitively**.\n\nWe again appreciate your valuable suggestions, which have been very helpful in enhancing the clarity of our paper.\n\n### Response To Cons #2\nWe appreciate the reviewer’s insightful comments.  \n\nRegarding Condor and FuseChat3.0, only FuseChat3.0 relies on a reward model, whereas Condor does not [1]. In our comparison, **we retained FuseChat3.0’s reward model exactly as prescribed in the original paper, as stated in our experimental setup**. Bohdi itself is designed to operate without any additional reward model.\n\nConcerning **the two baselines you mentioned**, we are unsure if the SuperMario you referred to refers to the DARE method in [2]. **This is a homogeneous model fusion scheme, which only applies to scenarios where the models involved in fusion have identical structures and parameter sizes**. It is **not applicable to the heterogeneous model fusion setting we are researching**.\n\nAs for InfiFusion [3], no code has been made publicly available in its published paper, making it impossible for us to reproduce it. Additionally, as an explicit LLM fusion scheme, the performance reported in the InfiFusion paper does not show significant improvements compared to FuseChat, the baseline we selected, and its design is similar to that of FuseChat. Therefore, the results of FuseChat can, to some extent, roughly represent the performance of InfiFusion. We have further clarified this point in the revised version of the paper. We again appreciate your valuable suggestions.\n\n[1]Condor: Enhance LLM Alignment with Knowledge-Driven Data Synthesis and Refinement, in ACL 25.\n\n[2]Language models are super mario: Absorbing abilities from homologous models as a free lunch, in ICML 24.\n\n[3]InfiFusion: A Unified Framework for Enhanced Cross-Model Reasoning via LLM Fusion\n\n### Response To Cons #3 & Question #1\nWe appreciate the reviewer's insightful feedback. Among the comparative baselines, explicit model fusion baselines do not actually generate text during operation, making it impossible to directly compare the number of generated tokens across methods. However, in Table 3 of the appendix, we have provided statistics on the actual total running time (in minutes) of each baseline method and Bohdi on each target model when using 8 A100 GPUs, allowing for a fair comparison of the practical efficiency of each method. **Across all target models, Bohdi's actual running time is consistently shorter than that of other baselines**. This result practically demonstrates Bohdi's efficiency advantage over other baselines. In the final version of the paper, we will move this result to the main text to present Bohdi's efficiency advantage more clearly to readers. We again appreciate the reviewer's valuable suggestions, which are very important for enhancing the quality of our work.\n\n### Response To Question #2\nWe appreciate the reviewer's valuable question. In response to your inquiry, we compared the data generated by Bohdi with the test task data across all four target model settings, and found that the number of overlaps between the generated data and the test data is zero. This further confirms that **there is no overlap between the collected QA pairs and the test data**. We have further elaborated on this result in the experimental section of the revised manuscript. We again appreciate the reviewer's insightful question."}, {"review_id": "hDPL6yJ1Uw", "reviewer": "Reviewer_wt9t", "summary": "The paper introduces the Bohdi framework for heterogeneous LLM fusion, which at its core is a HMAB framework for domain exploration and adaptive data collection. To further improve on the original HMAB framework, it adds the Introspection-Rebirth mechanism, which is essentially to re-estimate the target's model's performance utilizing a binomial likelihood ratio test. The overall framework is an iterative two-stage process: (1). HMAB for parameter update; (2). sampling and target model training. Experiments show Bohdi significantly outperforms baselines (different EF, IF fusion methods), demonstrating much better data efficiency (~1.x%)  and reduced capability imbalance among various tasks.", "questions": "- Most of the questions are in the Weakness Section.\n\n- One question re the baseline methods, for each question, instead of collecting \"golden responses\" from all three source models, can we use a leader / reward model to select one response and train on it? What's the performance there? It could give a clearer picture on the benefit of using HMAB.\n\n- The paper mentions two main motivations, i.e., drawback from existing LLM fusion methods, one is fixed data selection, this is resolved by HMAB; the other is limited domain coverage, could you make it clearer how the Bohdi framework address this in an effective way?", "limitations": "N/A", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strength:\n\n- The paper is well-motivated, outlined the two main drawbacks of the popular LLM fusion methods: (1). limited domain coverage; and (2). fixed data allocation; this makes bandit framework a naturally choice to update the posterior of the target model's performance and perform adaptive sampling.\n\n- The results are pretty strong, it shows better performance compared with existing data fusion baselines (i.e., FuseChat, FuseChat-3.0, Condor), while at the same time has much better sample efficiency, with only utilizing ~1.x% of the data in training of the target LLM.\n\n- The paper is well-written, the method is explained in a very clear manner, the HMAB algorithm, sprout and harvest operations, IR mechanism with sliding window, etc, the figure also helps the illustration. \n\n\nWeakness:\n\n- Some of the details are missing, for example, how to construct the knowledge graph tree in detail, what's the granularity there? More details regarding the reproducibility can be added, such as the number of nodes in each level, how are they determined, etc. Also, how do we determine if a question falls into which category? Could you give more details on the \"unk\" arm?\n\n- While the improvement in sample efficiency is impressive, the performance in each benchmark seems lack statistical significance. It would be great to add error bars on some of the tasks, at least, if experimental costs is a concern.\n\n- The results on Bohdi with an external reward model seems perform worse, and it somehow shows that Bohdi is pretty sensitive to the quality of the reward model, it might be a concern of the method. It would be great if the authors could either add more ablations regards to the robustness respect to the reward model, or arguing why choosing the leader model is always good here.\n\n- It would be great if the paper could mention the computational / inference cost by running the HMAB algorithm, by making a fair comparison with other baseline methods.", "was_revised": true, "final_justification": "I will keep my score unchanged, the score is already positive.", "rebuttal": "# Response To Reviewer wt9t\nWe greatly appreciate the insightful comments and suggestions from the reviewer. Below, we will provide point-by-point responses to each of your questions and recommendations.\n\n### Response To Weakness #1\n\nWe appreciate the reviewer's insightful feedback.\n\n**Knowledge Tree Granularity and Node Organization:** The knowledge tree is constructed with three layers. It originates from a `[Root]` node, branching into distinct Main, Secondary, and Sub-level domains. You can visualize its specific structure in Figure 7 of our appendix within the supplementary materials.\n\n**New Node Creation:** During each meditation phase, starting from the root, our Thompson Sampling (TS) algorithm samples child nodes from existing ones. When a sampled child node is `unk` (e.g., sampling `unk` from the `Mathematics` node), it signifies that the corresponding parent node (`Mathematics`) requires new sub-node expansion. At this point, a **Sprout operation** is executed, prompting the source model to propose new domains to be added as sub-domains under `Mathematics`. For more specific operational details, please refer to the algorithm pseudocode in Appendix B.4.\n\n**Node Category Determination:** The resulting node tree is stored as a dictionary, which clearly defines the hierarchical relationships. During the meditation phase, after a path of nodes (Main → Secondary → Sub) is sampled, we perform a Harvest operation in parallel for each path to generate data corresponding to that path. For instance, if data A is sampled along the path `Mathematics` (Root) → `Algebra` (Secondary) → `Linear Algebra` (Sub), it will be stored in the list corresponding to the innermost key (`Linear Algebra`) of that path in the dictionary. This allows for clear querying of its associated Main, Secondary, and and Sub domains.\n\n**The `unk` Arm:** Simply put, the `unk` arm represents the need for \"current parent node expansion.\" Each node, upon creation, is automatically assigned an initialized `unk` arm, and these `unk` arms are independent across nodes. When the `unk` arm is sampled from node 'a', a Sprout operation is executed to generate new child nodes for node 'a'. You can find clear details regarding the specific execution of this operation in the algorithm pseudocode in Appendix B.4.\n\n### Response To Weakness #2\nWe deeply value the reviewer's insightful comments. Through a series of comprehensive experiments, we have compellingly demonstrated the consistency and stability of Bohdi's performance.\n\n**In the comparative experiments**, we conducted a thorough evaluation of all methods across four target models with different architectures and parameter sizes: Llama3.2-3B-Instruct, Qwen2.5-7B-Instruct, Gemma2-9B-IT, and Llama3.1-8B-Instruct. As shown in Table 1 of the main text and Table 2 in the appendix, Bohdi consistently outperforms other baseline methods across different target models.\n\n**In the ablation experiments**, we further explored Bohdi's performance under different source model configurations. When using more parameter-efficient source models (Llama3.2-3B-Instruct, Qwen2.5-3B-Instruct, and Gemma2-2B-IT) and fusing them with the relatively larger Gemma2-9B-IT as the target model, the results still indicate that the target model fused by Bohdi can effectively absorb the advantages of the source models. This is particularly evident in tasks where Gemma2-9B-IT originally performed weaker than the source models, such as GPQA and TheoremQA.\n\n**Regarding the organization of generated domains**, as visually illustrated in Appendix Figure 7, when Llama3.2-3B-Instruct serves as the target model, the domains generated by Bohdi consistently include the vast majority of domains from scenarios with fewer source models as the number of source models increases. This indicates that Bohdi maintains stability in terms of domain expansion.\n\nFurthermore, throughout the evaluation process, we uniformly adopted greedy sampling to minimize the impact of autoregressive generation randomness on the evaluation results, ensuring the rigor of the assessment, as elaborated in Appendix C.\n\nThe **consistent improvements observed across these diverse evaluations and experimental results fully validate Bohdi's remarkable stability under various conditions**. In light of your valuable feedback, we will incorporate these analyses and findings into the main text as much as possible in the final version, enabling readers to more intuitively grasp these significant improvements. We deeply appreciate your valuable input.\n\n### Response To Weakness #3\nWe deeply value the reviewer's valuable feedback. **Bohdi is designed to evaluate feedback by randomly selecting a Leader Model from among the source models, rather than relying on an external reward model**. The primary motivation for including an ablation study with an external reward model is to validate the effectiveness of the random Leader Model selection scheme. Following the reviewer's suggestion, we conducted an additional set of experiments comparing the current approach of randomly selecting a Leader Model with fixing each of the three source models as the Leader Model, using Llama3.2-3B-Instruct as the target model. **The results, presented in the table below, demonstrate that random selection of the Leader Model yields more consistent average performance improvements**. This is because each model inherently possesses judgment biases that could lead to performance imbalances, and random Leader selection better mitigates the negative impacts of such biases. We have incorporated this experiment into our revised version. Your valuable input is crucial to enhancing the quality of our work.\n\n| Strategy              | MMLU  | GPQA | GSM8K | MATH | HumanEval | MBPP | TheoremQA | BBH  | AVG       |\n|-----------------------|-------|------|-------|------|-----------|------|-----------|------|-----------|\n| Fix-Leader-Mixtral    | 60.43 | 35.35| 75.51 | 45.08| 56.1      | 45   | 24.38     | 51.3 | 49.14  |\n| Fix-Leader-phi        | 61.25 | 32.32| 74.68 | 45.42| 56.1      | 46.6 | 25.5      | 46   | 48.48  |\n| Fix-Leader-Qwen       | 61.01 | 23.23| 75.06 | 46.52| 57.93     | 48.8 | 25.87     | 54.9 | 49.17    |\n| Random-Leader         | 61.33 | 30.3 | 78.62 | 47.92| 58.54     | 49.6 | 25.12     | 54.9 | 50.79  |\n\n\n### Response To Weakness #4\nWe deeply value the reviewer's valuable suggestions. In Table 3 of the appendix to our paper, we present statistics on the actual total runtime (in minutes) of each baseline method and Bohdi on each target model when using 8 A100 GPUs, allowing for a fair comparison of the practical costs of each method. Across all target models, Bohdi's actual runtime consistently remains shorter than that of other baselines. This result effectively demonstrates Bohdi's practical efficiency advantages over other baselines. In the final version of the paper, we will also move this result into the main text to more clearly present Bohdi's efficiency advantages to readers. Your valuable suggestions are greatly appreciated.\n\n\n### Response To Question #1\nWe deeply value the reviewer‘s insightful questions. \n\n**Bohdi's response generation takes place within the Harvest operation**. In fact, **we adopt the approach you described: the Leader Model (including the reward model in the ablation study) is used to select the superior answer from the responses of the three source models, rather than leveraging all source model responses collectively for training**. \n\n**The role of HMAB here is not targeted at the problem itself, but constructed at the granularity of problem domains**. When sampling training data, a batch of paths is sampled via the TS algorithm, and data generated during the meditation phase under each corresponding path is drawn to construct the training set. Further details of this process can be found in the description of the Sprout and Harvest operations in Appendix B, as well as the pseudocode in Appendix B4.\n\n### Response To Question #2\nWe deeply value the reviewer's valuable suggestions.\n\nRegarding **the issue of limited domain coverage**: This constraint is addressed through the self-construction of the knowledge tree in HMAB. By enabling autonomous construction of a structured knowledge tree among models, comprehensive and systematic organization of problem domains is achieved automatically, ensuring the breadth of domain knowledge. Meanwhile, the Harvest operation allows for data synthesis tailored to each domain path, thereby resolving the issue of missing open-source data in fine-grained domains.\n\nRegarding **the problem of fixed data allocation**: This limitation is overcome through dynamic adjustment of the posterior distribution of reward distribution parameters for each arm in HMAB. Based on the performance evaluation of the target model and source models across different domains, the alpha and beta parameters of these arms are dynamically adjusted. This facilitates a shift in data allocation toward domains where the target model performs weaker relative to the source models, enabling more rational distribution.\n\nWe will elaborate on these two points in greater detail in the main text of the final version. Your valuable suggestions are crucial for enhancing the clarity of our work."}]

Author rebuttal:

Meta-review: This paper proposes a heterogeneous large language model fusion framework using synthetic data only. It organizes domain knowledge into a three-level tree, formulates domain expansion and data ratio allocation as a hierarchical multi-armed bandit (HMAB) problem. The   framework also uses DynaBranches mechanism with an introspection-rebirth (IR) module that adaptively adjusts domain sampling using a sliding window binomial likelihood ratio test (SWBLRT).


The proposed framework seems very complicated and the reviewers raised concerns about the implementation details. Also, the experiment setup (the characteristics of the set of source LLMs) seem narrow. Additional information and experiment results can make it paper better.

Overall, this is a solid work but I am conservative for its real impact.
Decision: Accept (poster)