Title: Bohdi: Heterogeneous LLM Fusion with Automatic Data Exploration
Venue: NeurIPS 2025
Accepted: yes
Keywords: Heterogeneous Model Fusion

Large Language Models
Abstract: Heterogeneous Large Language Model (LLM) fusion integrates the strengths of multiple source LLMs with different architectures into a target LLM with low computational overhead. While promising, existing methods suffer from two major limitations: 1) **reliance on real data from limited domain** for knowledge fusion, preventing the target LLM from fully acquiring knowledge across diverse domains, and 2) **fixed data allocation proportions** across domains, failing to dynamically adjust according to the target LLM's varying capabilities across domains, leading to a capability imbalance. To overcome these limitations, we propose Bohdi, a synthetic-data-only heterogeneous LLM fusion framework. Through the organization of knowledge domains into a hierarchical tree structure, Bohdi enables automatic domain exploration and multi-domain data generation through multi-model collaboration, thereby comprehensively extracting knowledge from source LLMs. By formalizing domain expansion and data sampling proportion allocation on the knowledge tree as a Hierarchical Multi-Armed Bandit problem, Bohdi leverages the designed DynaBranches mechanism to adaptively adjust sampling proportions based on the target LLM's performance feedback across domains. Integrated with our proposed Introspection-Rebirth (IR) mechanism, DynaBranches dynamically tracks capability shifts during target LLM's updates via Sliding Window Binomial Likelihood Ratio Testing (SWBLRT), further enhancing its online adaptation capability. Comparative experimental results on a comprehensive suite of benchmarks demonstrate that Bohdi significantly outperforms existing baselines on multiple target LLMs, exhibits higher data efficiency, and virtually eliminates the imbalance in the target LLM's capabilities.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces the Bohdi framework for heterogeneous LLM fusion, which at its core is a HMAB framework for domain exploration and adaptive data collection. To further improve on the original HMAB framework, it adds the Introspection-Rebirth mechanism, which is essentially to re-estimate the target's model's performance utilizing a binomial likelihood ratio test. The overall framework is an iterative two-stage process: (1). HMAB for parameter update; (2). sampling and target model training. Experiments show Bohdi significantly outperforms baselines (different EF, IF fusion methods), demonstrating much better data efficiency (~1.x%)  and reduced capability imbalance among various tasks.\n\nSTRENGTHS AND WEAKNESSES:\nStrength:\n\n- The paper is well-motivated, outlined the two main drawbacks of the popular LLM fusion methods: (1). limited domain coverage; and (2). fixed data allocation; this makes bandit framework a naturally choice to update the posterior of the target model's performance and perform adaptive sampling.\n\n- The results are pretty strong, it shows better performance compared with existing data fusion baselines (i.e., FuseChat, FuseChat-3.0, Condor), while at the same time has much better sample efficiency, with only utilizing ~1.x% of the data in training of the target LLM.\n\n- The paper is well-written, the method is explained in a very clear manner, the HMAB algorithm, sprout and harvest operations, IR mechanism with sliding window, etc, the figure also helps the illustration. \n\n\nWeakness:\n\n- Some of the details are missing, for example, how to construct the knowledge graph tree in detail, what's the granularity there? More details regarding the reproducibility can be added, such as the number of nodes in each level, how are they determined, etc. Also, how do we determine if a question falls into which category? Could you give more details on the \"unk\" arm?\n\n- While the improvement in sample efficiency is impressive, the performance in each benchmark seems lack statistical significance. It would be great to add error bars on some of the tasks, at least, if experimental costs is a concern.\n\n- The results on Bohdi with an external reward model seems perform worse, and it somehow shows that Bohdi is pretty sensitive to the quality of the reward model, it might be a concern of the method. It would be great if the authors could either add more ablations regards to the robustness respect to the reward model, or arguing why choosing the leader model is always good here.\n\n- It would be great if the paper could mention the computational / inference cost by running the HMAB algorithm, by making a fair comparison with other baseline methods.\n\nQUESTIONS:\n- Most of the questions are in the Weakness Section.\n\n- One question re the baseline methods, for each question, instead of collecting \"golden responses\" from all three source models, can we use a leader / reward model to select one response and train on it? What's the performance there? It could give a clearer picture on the benefit of using HMAB.\n\n- The paper mentions two main motivations, i.e., drawback from existing LLM fusion methods, one is fixed data selection, this is resolved by HMAB; the other is limited domain coverage, could you make it clearer how the Bohdi framework address this in an effective way?\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces Bohdi, a framework that fuses knowledge from several heterogeneous source LLMs into a smaller target LLM without using any real data. Bohdi (1) arranges knowledge domains in a three-level tree, (2) explores new sub-domains through a Sprout operation and generates synthetic Q-A pairs through a Harvest operation, and (3) frames domain selection and data-allocation as a Hierarchical Multi-Armed Bandit solved by a DynaBranches variant of Thompson Sampling, equipped with an Introspection-Rebirth mechanism that resets posteriors via a sliding-window BLRT when the target model’s ability shifts.\n\nSTRENGTHS AND WEAKNESSES:\nPros:\n\n1. Casting domain expansion + data-ratio selection as an HMAB and augmenting Thompson Sampling with IR is interesting and new in the LLM-fusion context.\n\n2. The authors evaluate their method on diverse benchmarks across knowledge, coding, math and reasoning; includes ablations on IR window/quantile, data split ratios.\n\n3. The authors use two orders of magnitude fewer training samples than competing fusion or SFT baselines yet often matches or beats them.\n\nCons:\n\n1. The improvements over SOTA methods are incremental. Also since only single runs are reported and the authors explicitly acknowledge the absence of error bars due to cost. Therefore the variance in evaluations can be large.\n\n2. Condor and FuseChat-3.0 rely on explicit reward models or preference optimization. The authors disable reward models in Bohdi’s main results, yet the strongest baselines in recent literature (e.g., InfiFusion, SuperMario) are not included or are lightly tuned.\n\n3. While Bohdi uses only 1.7K target-training samples, it repeatedly queries three ≥ 14B source models for Sprout/Harvest. The total token budget and wall-clock cost are not reported.\n\nQUESTIONS:\nHow many total tokens were generated by Sprout + Harvest over 50 rounds? Please provide an approximate compute cost (GPU-hours).\n\nDid you check for overlap between harvested Q-A pairs and any of the evaluation test sets (particularly GSM8K and MATH)?\n\nLIMITATIONS:\nPlease refer to the Weaknesses and Questions.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes Bohdi, a heterogeneous large language model fusion framework based only on synthetic data. Bohdi systematically organizes domain knowledge into a three-level structured knowledge tree. It formulates domain expansion and data ratio allocation as a hierarchical multi-armed bandit (HMAB) problem and solves it using the proposed DynaBranches mechanism. Bohdi contains an introspection-rebirth (IR) module that adaptively adjusts domain sampling using a sliding window binomial likelihood ratio test (SWBLRT) based on performance feedback.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n\n- It is claimed that Bohdi is the first framework exclusively using synthetic data for heterogeneous LLM fusion and data can be allocated dynamically among different LLMs.\n- The method in this article is novel and clearly explained, as well as using clear mathematical symbols.\n- The amount of experimental data is large, especially the ablation study, the angles are diverse, and there are experimental verifications in different fields and different models.\n\n**Weaknesses:**\n- One of the innovations of the entire framework is to synthesize data without using existing data, but the article rarely discusses or conducts experimental analysis on the quality and correctness of the synthesized data, as well as comprehensiveness and reproducibility of newly generated data. In addition, this method of synthesizing data has a lot of randomness, which is mainly reflected in the selection of the Leader model in Harvest and the selection of the sub-field with the highest sampling value according to the Beta distribution in DynaBranches as the next direction to be explored. The article does not provide a detailed analysis of these intermediate results and randomness.\n- The source models used in the experiments vary significantly in parameter scale, and the target models come from different companies and also differ in size. This raises concerns about the consistency and fairness of the experimental setup. If the goal is to demonstrate that your method is adaptable across different models, the experiments should first be conducted on models of the same size. Only one variable should be changed at a time in order to ensure a valid controlled comparison. And then you can report the results containing variable settings.\n- The construction and management of a hierarchical tree, the need to run multiple source models for every “Sprout” and “Harvest” operation, and the use of adaptive sampling across potentially hundreds of domains (as illustrated in Figure 6) could become significant bottlenecks when scaling to very large source LLM pools or fine-grained domains. However, potential challenges related to compute, memory, or latency are not addressed in the main text.\n\nQUESTIONS:\n- Can you rovide detailed output and evaluation of intermediate steps? I would like to know the numerical details and detailed examples of the new data created in the paper. Rather than just seeing mathematical proofs, I want to see actual proof that this method can be implemented.\n\n- The paper mentions 5 different models, but only evaluates two of them as target models, and there is randomness in the selection of the method, which raises questions about the stability and generalizability of the method.\n- What are the memory, computation, and latency costs as the number of domains and source LLMs increase? Is there a practical limit to the granularity or number of domains that Bohdi can handle?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 4, "DATE": "2025-07-04"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a novel framework for fusing knowledge from heterogeneous LLMs into a compact target model without real data. It addresses key limitations of existing methods, including limited domain coverage and capability imbalance,  via organizing knowledge into a hierarchical tree and automating domain exploration/data generation using multi-model collaboration. The key innovations include dynamic domain expansion and data allocation as a Hierarchical Multi-Armed Bandit problem, solved by the DynaBranches mechanism with online capability tracking. Experiments demonstrate significant empirical improvements. Overall, this is a very good work. But without the available code, it is not easy to fully justify this framework.\n\nSTRENGTHS AND WEAKNESSES:\nStrength:\n1. Removes dependence on any real or in-domain dataset, allowing the target LLM to inherit knowledge that is both broader and privacy-safe.\n2. Adaptive allocation virtually eliminates capability imbalance and achieves markedly higher data efficiency than static-mix baselines.\n3. Online adaptation yields consistent performance gains across diverse benchmarks, outperforming previous fusion methods on every evaluated target LLM.\n\nWeakness:\n1. The major concern about this work is that the framework is quite complex and without the available codes, it makes the justification and  and replication challenging.\n2. The magnitudes of improvement and decline reported in Table 1 are unclear, as the reference baseline against which they are calculated is not specified. \n3. The designations “H1” and “H2” in lines 140–142 do not align with the labels shown in the corresponding figure, which may cause confusion.\n4. The influence of choosing the auxiliary model H2 at random has not been explored; a dedicated ablation study is needed to assess its impact.\n\nQUESTIONS:\nCheck the weakness part mentioned above.\n\nLIMITATIONS:\nCheck the weakness part mentioned above.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-07-05"}